T&E to Meet the Pace of Need... Will Require a Disruptive Transformation! Dr Ed Kraft. Associate Executive Director for Research.

Transcription

1 T&E to Meet the Pace of Need... Time to IOC for Fixed Wing Aircraft Will Require a Disruptive Transformation! Dr Ed Kraft Associate Executive Director for Research May 16,2018 NDIA 33 rd Annual National T&E Conference Solomon Island, Maryland Source: Dan Patt, DARPA

2 The Global Digital Revolution a Disruptive Transformation of Industry.. History of Industrial Revolutions Always Driven By New Enabling Technologies Most T&E Infrastructure and Tools Developed Here Acquisition Policies, Processes, and Tools Developed Here Our Weapon Systems Are Going Here Cyber Warfare Autonomy Third Pivot Data as a Weapon The Digital Revolution is Already Here... And T&E is Falling Behind

3 Our T&E Baby Is Ugly!! Late Delivery 1 st Flight Vehicle Late Defect Discoveries $ Peak Burn Rate Occurs Around FF First Flight RDT&E Overrun Standing Army Effect CDR Nunn-McCurdy Breach Should We Look at the Head and Not the Tail? Wind Tunnel Campaign Wind Tunnel + Flight Test Campaigns Overlaps ~ 85% Cycle Time Flight Test Campaign OT&E Pause Test Rate Non- Suitability Fewer Systems Delivered MS B ~48 mos ~96 mos ~24 mos Nominally 14 Years IOC Delayed Or Are We Holding It Upside Down and Backwards? IOC

4 Cumulative Total Ownership Costs Decision Framework for Defense Systems Lifecycle Cost Committed MDD A B C DMIL Decision System Requirements Defined in RFP for MS B Contract CDD Validation CPD Validation Impact on O&S, Optimum Maintenance Forecast FRP FOC Post IOC Reviews Expenditure Profile Operations & Sustainment ICD Validation Selection of Preferred Concept AoA Study Plan Draft CDD RFP Release Decision AoA Validation Need 90% Confidence System Meets Requirements DT&E PDR CDR FF LRIP IOT&E Development IOC Critical Factors Impacting T&E Occur Back Here Production & Deployment Resulting in the Ugly Baby Syndrome Here Requirements Decision Points Acquisition Decision Points Operations & Sustainment Reviews Most Significant Cost/Capability Trade Opportunities 4

5 Digital Transformation and T&E Shifting from linear, document-centric system lifecycle process towards a dynamic, connected, digital model-centric ecosystem enables disruptive transformations including Digital TEMP dynamically coupled to requirements, reduce preparation time, employ model based-engineering and uncertainty quantification to develop an optimum T&E campaign, minimize defects set stage for the digital revolution Validated Digital Surrogates Expanding STAT to elevate T&E as key to the validation of an Authoritative Digital Surrogate Truth Source vice just calibrating a model Uncertainty Quantification Employ digitally captured quantified epistemic and aleatory uncertainties for MRTFB test capabilities to allocate uncertainty reduction budgets to particular tests learning when you have to iterate the design or move on to the next level of modeling or testing Test Digital Twin by test asset, use a real-time digital surrogate to assess the system state, early indication of potential aberrant behavior, and ability to forecast the next best data point even during the sortie Digital Critical Decision Points Use Authoritative Digital Surrogate Truth Sources to quantify margins and risks, project optimum course of action The Digital Revolution Physical Science + Data Science + Learning System = Velocity

6 Integrated Test Team - Stuck in a Document Centric Mode Current TEMP as required by the DODI Represents a linear, document-centric, positional approach Not connected to changing Requirements Does not use all of the available knowledge / information Doesn t readily support optimization of T&E campaign Does not address generating the digital authoritative truth source through an integrated application of UQ Test and Evaluation Master Plan (TEMP) requires 12 months to prepare and 10 months to review with up to 56 organizations at 8 levels involved in the review. GAO , February, and the next day someone changes a requirement and it is obsolete!!

7 Moving to a Digitally Connected ITT, Digital TEMP The Digital TEMP as an enabler for all of the digital transformations to reduce time/costs and assure Requirements are met A Digital TEMP will Provide a model-centric approach dynamically coupled to digital Requirements Apply digitally captured system performance parametric sensitivities from trade studies to quantify T&E requirements Use digitally captured MRTFB capabilities / performance maps / schedules to identify gaps for investments to meet requirements Use digitally managed uncertainty measures for MRTFB capabilities with Requirements to develop optimum test campaigns. Think Digitally, Not Digitized

8 Decision Analytics A Digital Critical Design Review See bring all authoritative digital surrogate truth sources to understand and quantify margins and uncertainties in the performance of the system do we have 90% confidence the system meets requirements, can be produced, and can be integrated? Think use data analytics/probabilistic analyses to assess risk, impact on military utility, and total ownership cost of any requirements gaps Do apply value-based decision analysis, Bayesian Belief Networks to select the best value course of action including data-driven risk mitigation strategies Risk = Uncertainty with Consequences See Think Do Use All Available Information Use Probabilistic Analysis to Inform Select Best Value COA Assuring a 90% Confidence Level at CDR That the System Under Development Meets Requirements and Can Be Produced Will Alleviate Most of the Post-CDR T&E Ugly Baby Syndromes 8