Matthew Clark V&V of Autonomous Systems Autonomous Control Branch (AFRL/RQQA) Integrity Service Excellence

Transcription

1 Test and Evaluation, Verification and Validation of Autonomous Systems From AFRL/RQ to DoD 10 June 2014 Matthew Clark V&V of Autonomous Systems Autonomous Control Branch (AFRL/RQQA) Integrity Service Excellence Air Force Research Laboratory

2 Air Force Research Lab Aerospace Systems Directorate Power and Control Division Trust and Certification Strategy

3 From Automation to Autonomy Shifting the V&V Paradigm WHAT MAKES AUTONOMY HARD TO CERTIFY? Insufficient engineering methods to V&V highly complex, software intensive systems Near infinite state / non-deterministic responses to unknown / untrained environments Composition of systems resulting in potentially catastrophic emergent behavior It is possible to develop systems having high levels of autonomy, but it is the lack of V&V methods that prevents all but relatively low levels of autonomy from being certified for use Air Force s 2010 Technology Horizons Report

4 Limitations on Current Certification Methodology Increased system complexity makes testing alone impossible Currently systems are fully tested Certification is at system level Sub-systems certified as part of a system Verification after system integration Errors found are costly to fix Validation of requirements happens as the last developmental step A problem in requirements can invalidate a whole system developed

5 Civil Aircraft Design Guidelines An Argument for Safety Source: ARP 4754 Process Starts with Requirements Refines the Req and Arch With safety in mind DO-178C is primarily a Quality document Demonstrate Software Assurance after criticality is determined

6 DO-178B Verification Objectives Safety Critical Systems must meet all objectives Accurate / Consistent traceable Coverage traceable Accurate Consistent Verifiable Coverage Assurance Coverage - Do the high-level requirements capture system intent? - Are the requirements traceable to the System Requirements? - Are they accurate, consistent, and verifiable? - Is the architecture compatible with the requirements? - Are they accurate, compatible, and verifiable? As complexity increases how do you achieve full coverage? How do you achieve appropriate assurance?

7 Trust and Certification Process Compositionally Verified Systems of Systems Safety Case Reuse New Autonomy Certification Need Requirement Formalization & Analysis Architecture Formalization & Analysis FORMALIZED SAFETY ASSESSMENT HAZARD MITIGATION REQUIREMENTS Multiple V&V Technology Paths Analytical Proof Synthesis Modeling, Simulation, Test & Evaluation Run Time Assurance Assurance Validator Certified Safety Case System Design and Safety Requirements (ARP 4761, ARP 4754/A, MIL-HDBK-882E) Testable Requirements & Verification Plans (DO-178C/254, MIL-HDBK-516)

8 Air Force Research Lab Test and Evaluation, Verification and Validation (TEVV) of Autonomy Strategic Planning

9 Overview Goal: to define TEV&V of autonomy and articulate current gaps in traditional TEV&V processes Approach: Hold three workshops by Q3 FY14 Industry (DoD / Non-DoD) Academic Government Focused on three questions What does certified mean? What is an autonomous system? What new innovations in software / hardware certification can be applied to Autonomous Systems? Distill challenges, create strategy, identify current AFRL programs, identify gaps / investment opportunities

10 Industry Workshop Challenge Near Mid Far Modeling and Simulation Requirements Defacto Standards Licensing vs Certification Test 11 6 Uncertainty Trust Security Tool Verification System of Systems V&V Human/Machine Interaction Runtime Verification Synthesis Emegent Behavior Priority and Score for Each Technical Challenge (Green = Near Term, Yellow = Mid Term, Red = Far Term)

11 Licensure vs. Test & Certification De-facto Standards emerged as a potential solution to the brick wall associated with applying current certification standards to Autonomous software systems Highly complex, safety critical systems are require exhaustive test and evaluation to ensure adequate coverage Method to establish trust in Pilot / Human Operator is drastically different than certifying Systems Aircraft, Ground Vehicles, RPVs Operators are trained and tested then issued a license

12 Academic Workshop Challenge Near Mid Far Requirement generation Standards & architecture Human automation interaction System assurance methodology Learning & memory Security Teaming of multiple entities Capabilities & limitations Complexity Emergence Priority and Score for Each Technical Challenge (Green = Near Term, Yellow = Mid Term, Red = Far Term)

13 Government Workshop Challenge Near Mid Far Certification of Unpredictable Systems Formal Methods Efficient Testing Runtime Assurance & Monitoring System Compose-ability & Recertification Human-Autonomy Interaction Unknowable Environment Policy & Standards Cultural Acceptance Priority and Score for Each Technical Challenge (Green = Near Term, Yellow = Mid Term, Red = Far Term)

14 Notional Groupings Industry Challenge Near Mid Far Challenge Near Mid Far Challenge Near Mid Far Modeling and Simulation Efficient Testing Test 11 6 Defacto Standards Licensing vs Certification Standards & architecture Policy & Standards Trust Cultural Acceptance Tool Verification Formal Methods Requirements Requirement generation 13 3 Synthesis Learning & memory 5 7 Certification of 7 Unpredictable Systems Uncertainty Complexity Unknowable Environment Emegent Behavior Emergence Human/Machine Interaction Human automation interaction Runtime Verification Capabilities & limitations System of Systems V&V System assurance methodology Academic Security Security Teaming of multiple entities Government Human-Autonomy Interaction Runtime Assurance & Monitoring System Compose-ability & Recertification

15 Today Verification and Validation of Autonomous Systems Through Argument Based Assurance System level test for small changes Decision making burden on Humans Difficult to objectively measure risk V&V is late in design process Test for all known conditions Testing

16 Today Verification and Validation of Autonomous Systems Through Argument Based Assurance System level test for small changes Decision making burden on Humans Difficult to objectively measure risk V&V is late in design process Future Test for all known conditions Alternate Evidence Modularity/Composability Precedent Setting/Reusability Iterative and Continuous Testing

17 Today Verification and Validation of Autonomous Systems Through Argument Based Assurance System level test for small changes Decision making burden on Humans Difficult to objectively measure risk V&V is late in design process Future Test for all known conditions Alternate Evidence Modularity/Composability Precedent Setting/Reusability Iterative and Continuous Testing Cumulative Evidence Through RDT&E, DT & OT Progressive sequential modeling, simulation, test and evaluation

18 Today Verification and Validation of Autonomous Systems Through Argument Based Assurance System level test for small changes Decision making burden on Humans Difficult to objectively measure risk V&V is late in design process Future Test for all known conditions Evidence Generation During Design Guarantee appropriate decisions with traceable evidence Alternate Evidence Modularity/Composability Precedent Setting/Reusability Iterative and Continuous Testing Cumulative Evidence Through RDT&E, DT & OT Progressive sequential modeling, simulation, test and evaluation

19 Today Verification and Validation of Autonomous Systems Future Through Argument Based Assurance System level test for small changes Decision making burden on Humans Difficult to objectively measure risk V&V is late in design process Test for all known conditions Requirements Dev. and Analysis Precise, structured standards to automate requirement evaluation for testability, tractability, and deconfliction Evidence Generation During Design Guarantee appropriate decisions with traceable evidence Alternate Evidence Modularity/Composability Precedent Setting/Reusability Iterative and Continuous Cumulative Evidence Through RDT&E, DT & OT Testing Progressive sequential modeling, simulation, test and evaluation

20 Today Verification and Validation of Autonomous Systems Future Through Argument Based Assurance System level test for small changes Decision making burden on Humans Difficult to objectively measure risk V&V is late in design process Test for all known conditions Decision Assurance Real time monitoring and migration of undesired decisions and behaviors Requirements Dev. and Analysis Precise, structured standards to automate requirement evaluation for testability, tractability, and deconfliction Evidence Generation During Design Guarantee appropriate decisions with traceable evidence Alternate Evidence Modularity/Composability Precedent Setting/Reusability Iterative and Continuous Cumulative Evidence Through RDT&E, DT & OT Testing Progressive sequential modeling, simulation, test and evaluation

21 Today Verification and Validation of Autonomous Systems Future Through Argument Based Assurance System level test for small changes Decision making burden on Humans Difficult to objectively measure risk V&V is late in design process Test for all known conditions Compositional Case Generation Enable reusable evidence building blocks Decision Assurance Real time monitoring and migration of undesired decisions and behaviors Requirements Dev. and Analysis Precise, structured standards to automate requirement evaluation for testability, tractability, and deconfliction Evidence Generation During Design Guarantee appropriate decisions with traceable evidence Alternate Evidence Modularity/Composability Precedent Setting/Reusability Iterative and Continuous Cumulative Evidence Through RDT&E, DT & OT Testing Progressive sequential modeling, simulation, test and evaluation

22 Department of Defense s (DoD) Science and Technology (S&T) Autonomy Community of Interest (COI) Test and Evaluation, Verification and Validation (TEVV) Working Group

23 ASD/R&E framework of technical coordination groups Communities of Interest (COIs) Missions Capabilities enabled by advanced technologies and systems Cyber Security Counter IED Counter WMD Systems Electronic Warfare Air Platforms Human Systems Info Systems Integrating multiple technologies into complex systems Sensors & Processing Weapons Technologies Space Engineered Resilient Systems Enabling Science & Technology S&T with multiple applications Advanced Electronics Materials & Processes Energy & Power Technologies Autonomy ASBREM 23

24 Autonomy COI Structure COI Senior Steering Group (Currently Chaired by Morley Stone, USAF) Members: Working Group Leads and Senior Reps from USAF, USA, USN, DTRA, DARPA, and OSD Human and Agent Teaming Working Group (Co-Led by USN, USAF) ScalableTeaming Working Group (Co-Led by USN, USA) Current Members: Jon Bornstein USA Jim Overholt USAF Brian Sadler USA Tom Wettergren USN Matthew Clark USAF Jeff Depriest DTRA Alan Schultz USN Will Curtis USAF Marc Steinberg USN Stephen Dowling DTRA Greg Hudas USA Tim Broderick DARPA DH Kim OSD AO: Laura Barnes and Kris Kearns USAF Machine Reasoning and Intelligence Working Group (Co-Led by USA, TBA) Test, Eval and V&V Working Group (Co-Led by DTRA, USAF) 24

25 Overview of Investment Technical Challenge % Investment Per year by Autonomy COI Working Group 5.3% 18.9% 13.1% Test and Evaluation Verification and Validation (TEVV) Scalable Teaming of Autonomous Systems (STAS) Machine Perseption Reasoning and Interaction (MPRI) 62.8% Human/Autonomous System Interaction and Collaboration (HASIC) NOTE: Average investment over three years Does not include DARPA funding Change in TEVV funding is due to missing OSD funding beyond FY14 25

26 Test, Evaluation Validation and Verification (TEVV) Central Tech Challenge: From algorithms up to scalable teams of multiple agents--new V&V technologies needed to enable complete system evaluation Fundamental science of V&V for Autonomy Reduce time & cost to field emerging tech. Must improve safety and reliability for landscape of autonomous systems RDT&E New methods to augment gap in exhaustive testing Run Time (Operational) Assurances Formal and Enhanced Analysis Techniques New Design of Experiments Methods for Autonomous Systems Guaranteed trust in Human-Agent Teaming Assessing risk Operation in Complex, Contested, Uncertain Environments Early verification of coordinated Actions by Multiple Agents DOD Investment by Fiscal Year R&D Investment in $M V&V of Design TBD TBD New T&E Methods Adv Component Dev (BA4) Advanced Development Applied Research Basic Research FY14 FY15 FY16 FY17 FY18 FY19 NOTE: NOT COMPREHENSIVE, ADDITIONAL INVESTMENT LIKELY 26

27 Gap & Recommendations GAP: Lack of focused S&T investment in T&E, V&V for complex, autonomous systems PROBLEM: DoD lacks consistent strategy that enables the development of technology, tools, and culture to V&V autonomous systems T&E of critical software costs SEVEN TIMES software development costs.* Only low throughput, low capability, high cost Autonomous systems fielded Errors found in operational test increases system development cost and time PROPOSED WAY FORWARD: COI TEVV Working Group is aligning around focused technology challenges RECOMMEND: Focused OSD investment directed to TEVV Working Group emphasizing: Run Time Decision Assurance Design of Experiments methods for Non-deterministic / learning algorithms Formal requirements and architecture analysis tools RECOMMEND: COI develop OSD SBIR topics for Fall 2014 focused on same topics as above. * "Synthesizing cyber-physical architectural models with real-time constraints." Computer Aided Verification. Springer Berlin Heidelberg,

28 Gap & Recommendations GAP: Lack of focused S&T investment in T&E, V&V for complex, autonomous systems PROBLEM: DoD lacks consistent strategy that enables the development of technology, tools, and culture to V&V autonomous systems The cultural paradigm is based on primarily M&S & T&E and does not include emerging software V&V methods PROPOSED WAY FORWARD: Develop and provide T&E workforce training on formal methods based software verification tied into DAU T&E Certification training RECOMMEND: Develop training materials for non-deterministic software systems to include in DAU T&E Certification Investigate licensure vs certification RECOMMEND: Execute a study, consisting of Academic and Industry experts with COI serving as technical advisors 28

29 Conclusions V&V of Autonomy is a DoD recognized challenge Common technical challenges across the landscape Current research is disjointed with an opportunity for additional investment There is a need to coordinate research in V&V Technologies, Tools, and Training across DoD to enable effective transition of Autonomous Systems 29