Virtual Integration for Model Based Safety Assessment of Complex Systems

Transcription

1 Aerospace Vehicle Systems Institute Virtual Integration for Model Based Safety Assessment of Complex Systems System Architecture t Virtual Integration Program David Redman, AVSI Director Presentation to the 16thAnnual NDIA Systems Engineering Conference 29 October 2013

2 Outline Overview of SAVI The AADL Error-Model lannex Support of Safety Evaluation with AADL Case-Study Future Work October 29,

3 The Aerospace Vehicle Systems Institute Full Members Liaison Members Airbus FAA Boeing NASA DoD Aerospace EADS Valley Embraer GE Aviation Goodrich (now UTC) Honeywell Rockwell Collins Rolls Royce Saab United Technologies Associate Members BAE Systems Bombardier Gulfstream Lockheed Martin October 29,

4 The Aerospace Vehicle Systems Institute Full Members Liaison Members Airbus FAA Boeing NASA DoD Aerospace EADS Valley Embraer GE Aviation Goodrich (now UTC) Honeywell Rockwell Collins Rolls Royce Saab United Technologies Mission AVSI addresses issues that impact the aerospace community through international cooperative research and collaboration conducted d by industry, government and academia. Associate Members BAE Systems Bombardier Gulfstream Lockheed Martin October 29,

5 The AVSI SAVI Program Ln(O Onboard SLOC) One Measure of Complexity Estimated Onboard SLOC Growth M $160 B Slope = M Intercept = M Curve implies SLOC doubles Assumed 27M $7.8 B about every 4 years Affordability 16 8M Limit B777: 4M A330/340: 2M 14 $290 M B737: 470K A320: 800K B747: 370K A310: 400K $81 M 12 B757, B767: 190K $38 M A300FF: 40K Line Fit 10 Boeing Airbus A300B: 4..6K Unaffordable 8 The line fit is pegged at 27M INS: 0.8K SLOC because the projected 6 SLOC sizes for 2010 through 2020 are unaffordable. The COCOMO II estimated costs to develop that much software Year are in excess of $10B. Airbus data source: J.P. Potocki De Montalk, Computer Software in Civil Aircraft, Sixth Annual Conference on Computer Assurance (COMPASS 91), Gaithersburg, MD, June 24-27, Boeing data source: John J. Chilenski Private . Late Discovery of System-Level Problems 9/24/12 SAVI Presentation to 2012 INCOSE SE Conference 5 Requirements Engineering 70% requirements and 80% late error system 70% interaction errors discovery at high repair cost System Design Software Architectural Design Component Software Design Sources: NIST Planning report 02-3, The Economic Impacts of Inadequate Infrastructure for Software Testing, May D. Galin, Software Quality Assurance: From Theory to Implementation, Pearson/Addison-Wesley (2004) B.W. Boehm, Software Engineering Economics, Prentice Hall (1981) 70%, 3.5% 1x 10%, 50.5% 16x x 20.5% 110x 0%, 9% 40x x Integration Test Acceptance Test System Test 20%, 16% Unit 5x Test 3-6x Where faults are introduced INCOSE 2010 Where faults are found The estimated nominal cost for fault removal Code 9/25/12 SAVI Presentation Development to 2012 NDIA SE Conference 14 Launched in 2008 to address the problem of growth in complexity of systems leading to cost and schedule overruns The objective is to develop a standards-based Virtual Integration Process (VIP) that allows multiple parties to virtually integrate and analyze systems throughout development life cycle The result is earlier detection and correction of errors leading to cost savings Highly focused on integration defining the state of the art in system integration consistency checking October 29,

6 SAVI Engages g Stakeholders The SAVI Program has continually sought any and all stakeholders to contribute to the definitions of the standards-based solution SAVI has also sought out partners with best-inclass technology that supports the VIP to avoid duplication of effort Current and past participants include: Adventium Labs Airbus BAE Systems Boeing US DoD Embraer Esterel Eurostep US FAA GE Aviation Honeywell Lockheed Martin NASA Rockwell Collins SEI at CMU Texas A&M October 29,

7 Past Results Several proof of concept phases have researched the feasibility of the SAVI VIP, exploring topics including: Model-based vs. document based systems acquisition SAVI return on investment (RoI) Architectural description language capabilities and extensions Inter-domain tool integration Model repository attributes for virtual integration Model data exchange protocols and technologies IP protection in an integrated, multi- participant i t modeling environment Assurance methods October 29,

8 Past Results Several proof of concept phases have researched the feasibility of the SAVI VIP, exploring topics including: Model-based vs. document based systems acquisition SAVI return on investment (RoI) Architectural description language capabilities and extensions Inter-domain tool integration Model repository attributes for virtual integration Model data exchange protocols and technologies Scalability IP protection in an integrated, multiparticipant modeling environment Assurance methods SAVI Members have concluded that there is compelling evidence to justify development of the SAVI VIP October 29,

9 Current Project Focus Focus of SAVI V. 1.0A Focus of SAVI V. 1.0C Focus of SAVI V. 1.0B Focus of SAVI V. 1.0D October 29,

10 Safety Demo Focus Application on a standardized example (AIR6110) Automated generation of certification documents Compliance with standards requirements Highlight the iterative design process First safety evaluation Refinement through system development Use of commercial and open-source tools Reproducible at no-cost Adaptation with state-of-the-art analysis tools October 29,

11 THE AADL AND THE ERROR MODEL ANNEX

12 The Architectural Analysis and Design Language (AADL) An SAE standard (AS5506B) maintained by the SAE Aerospace AS-2C Committee Semantically precise language suitable for quantitative analyses Originally developed for analysis of embedded systems, but language is extensible standard consists of a core language g definition and annexes Application of AADL is growing both in the the US and internationally Supported by open-source and commercially available tools More information at October 29,

13 Overview of Error-Model Annex Extension of AADL for fault description: error events, propagations, etc. Integration with current models by extending existing components Draft document to be proposed as a standard annex Support for Safety Evaluation and Analysis October 29,

14 Error Types and Propagations Error types: error classification ValueError Extensions and renaming OutOfRange Inconsistent Error propagations across components Associate errors with system connections Define error sources, sinks and containment Error Source Sink for ValueError & Error Sink of ValueError source for NoData for NoData ValueError NoData Sensor Processing Actuator October 29,

15 States machines SAVI Error Model Evaluation Case Study Future Error-related transitions Propagation rules Use of error types Composite behavior Error behavior Failure (BadData) Normal Failed Failed (NoValue) Recover Define system states according to its parts ex: I am failing if one of my component is failing Subsystem 1 Subsystem 2 (Normal) (Normal) Subsystem 1 Subsystem 2 (Normal) (Failing) October 29,

16 Specific Error-Model Properties Severity, likelihood, error description Support for generating validation documentation ti Tailoring for safety standards (ARP4761, MIL- STD-882) October 29,

17 SUPPORT OF SAFETY EVALUATION WITH AADL

18 AADL & Safety Evaluation Tool Overview Architecture centricity enables generative technologies to support analyses FHA FTA Markov Chain SPN/SANs FMEA Spreadsheet Use error propagations CAFTA OpenFTA Use composite behavior Error flows PRISM Use error flow Error behavior Stochastic Petri Nets and Activity Nets Use error flow Error behavior traditional methodologies (a la ARP 4754/4761) Spreadsheet Error behavior Propagations October 29,

19 Safety Analysis & AADL Preliminary System Safety Assessment (PSSA) support High-level component, interfaces from the OEM Automatic generation of validation materials (FHA, FTA) System Safety Assessment (SSA) support Use refined models from suppliers Enhancement of error specifications Support of quantitative safety analysis (FTA, FMEA, MA) Sy ystem Dev velopment Cycle October 29,

20 Evolution of Safety Analysis process with AADL Preliminary System Safety Assessment Component types (system interfaces) Component implementation System Safety Assessment Validation Materials (FHA, FTA) Check PSSA and SSA consistencies Validation with quantitative fault rates (FMEA, FTA, DD, MA) Refinem ment & dev velopment evolution October 29,

21 Functional Hazard Analysis Support Use of component error behavior Error propagations p rules Internal error events Specify initial failure mode FHA Define error description and related information Create spreadsheet containing FHA elements To be reused by commercial or open-source tools October 29,

22 Fault-Tree Analysis Support Use of composite error behavior FTA nodes Use of component error behavior Incoming error events FTA Walk through the components hierarchy Generate the complete fault-tree Focus on specific AADL subcomponents Export to several tools Commercial: CAFTA Open-Source: OpenFTA October 29,

23 Markov-Chain Support Use of component error behavior Error propagations rules Error transitions Map states and error types into specific values Tool-specific approach Ability to evaluate system state t over time Markov Chain What is the probability my system is failing within 30 days? Export to open-source tools PRISM Transient failure of a component October 29,

24 Failure Mode and Effects Support Use of component error behavior Error propagations rules (source, sink, etc.) Internal error events Traverse all error paths Record impact over the components hierarchy FMEA Use error description and related information Create spreadsheet containing FHA elements To be reused by commercial or open-source tools October 29,

25 CASE STUDY

26 Safety Analysis Overview and Demo Sequence Demonstrate a select set of PSSA analyses in the context of the Wheel Braking System (WBS) example Potential ti scenarios Baseline design (pre-rfp) Competing Architectures (RFP responses) Architecture t refinement (iteration ti on RFP selection) Safety property specification refinement Preconditions Aircraft and higher-level safety artifacts provided to PSSA following progression of AIR 6110 (be specific) WBS model(s) and supporting environment models configured with ARP property sets Consistency check scenarios confirm model consistency Reminder: Per ARP 4761 the PSSA is the method for determining how failures can lead to the functional hazards identified by the FHA, and how the FHA requirements can be met. October 29,

27 Wheel Brake System Development of a public model to complement the models developed in the SAVI Program Use of Error-Model and ARINC annexes Relevance for the avionics community Apply the technology/toolset on a known example Generation of FHA, FTA, MA & FMEA October 29,

28 AADL model Parent System SAVI Error Model Evaluation Case Study Future NoService NoPower NoPressure InvalidReport Software and/or RuntimeError October 29,

29 AADL model, BSCU variations October 29,

30 FHA of the Parent System October 29,

31 FTA of the Parent System Focus on a specific AADL subcomponent October 29,

32 FTA of the BSCU subcomponent October 29,

33 FMEA of the Parent System Current State Out propagation Propagation path Out propagation or error containment Component 1 Component 2 October 29,

34 FUTURE WORK

35 Safety Analysis Consistency Checks Consistency at integration time Consistency between models from different suppliers Strengthen the Virtual Integration promoted by SAVI Consistency of the internal model ex: Can I propagate this error according to my actual state? Consistency across error models specifications Component Error Behavior with Composite Error Behavior Correctness of a state according to subcomponents Error information with Behavior information October 29,

36 SAVI Consistency Checks Solid models BSCU AADL-SysML models Hyd power supply Accumulator October 29,

37 Current Project Focus Focus of SAVI V. 1.0A Focus of SAVI V. 1.0C Focus of SAVI V. 1.0B Focus of SAVI V. 1.0D October 29,

38 Questions? Contacts: Dr. Don Ward Phone: (254) Mobile: (903) Dr. Dave Redman Office: (979) Mobile: (979) Dr. Julien Delange Office: (412) October 29,