Supporting Safety Evaluation Process using AADL

Transcription

1 Supporting Safety Evaluation Process using AADL Software Engineering Institute Carnegie Mellon University Pittsburgh, PA Julien Delange and Peter Feiler 12/09/2013

2 Safety Analysis issues (aka the problem) Manual process, inaccurate with system implementation Some errors are not captured/caught during analysis Long and error-prone evaluation process Implementation + tests System Specs. Manual Process Certification documents Safety Evaluation Report 2

3 Automate/Improve Evaluation (aka the solution) Derives materials from existing artifacts (i.e. architecture models) Avoid manual process traps and pitfalls Automate evaluation, reduce analysis time System Specs. (ideally a model) Automatic Process Implementation + tests Certification documents Safety Evaluation Report Improve Safety/Reliability Assessment 3

4 Combine existing methods (aka the approach) Add safety-information to existing models Automate the evaluation process, avoid manual efforts Generate materials required by safety standards (ARP4761) 4

5 Agenda Overview of AADL & Error-Model Annex Overview of ARP4761 and Safety Evaluation Support of Safety Evaluation with AADL Case-Study On-Going Work Discussion 5

6 Agenda Overview of AADL/Error-Model Annex Overview of ARP4761 & Safety Evaluation Support of Safety Evaluation with AADL Case-Study On-Going Work Discussion 6

7 Architecture Analysis Design Language Modeling language standardized by SAE Inherit MetaH concepts Initiated in 2003; revised in 2008 Design of hardware and software Analysis of different criteria (performance, safety, security, etc.) Tool support: OSATE, Ocarina, MASIW Evaluation during research projects SAVI: avionics community ASSERT: aerospace community 7

8 AADL ecosystem Reliability Performance Evaluation Code Generation System Validation System Configuration Security Safety ARINC653 Requirements description 8

9 Overview of Error-Model Annex Extension of AADL for fault description: error events, propagations, etc. Integration with current models by extending existing components Draft document to be proposed as a standard annex Support for Safety Evaluation and Analysis 9

10 Error Types and propagations Error types: error classification ValueError Extensions and renaming OutOfRange Inconsistent Error propagations across components Associate errors with system connections Define error sources, sinks and containment Error Source Sink for ValueError & Error Sink of ValueError source for NoData for NoData Sensor ValueError Processing NoData Actuator 10

11 Error behavior States machines Error-related transitions Propagation rules Use of error types Failure (BadData) Normal Failed Recover Failed (NoValue) Composite behavior Define system states according to its parts ex: I am failing if one of my component is failing Subsystem 1 (Normal) Subsystem 2 (Normal) Subsystem 1 (Normal) Subsystem 2 (Failing) 11

12 Support of AADL textual syntax Error types mechanism w/ extensions Error propagations Sensor ValueError Processing Composite error state Machine Sensor (Operational) Processing (Operational) Actuator (Operational) Operational 12

13 Specific Error-Model Properties Severity, likelihood, error description Support for generating validation documentation Tailoring for safety standards (ARP4761, MIL-STD-882) 13

14 Agenda Overview of AADL & Error-Model Annex Overview of ARP4761 & Safety Evaluation Analysis of System Safety with AADL Case-Study On-Going Work Discussion 14

15 ARP4761 Safety Standard SAE standard for Safety Assessment Avionics Community mostly (relation with DO178B) Assurance of System Safety Define Safety Evaluation Process Materials & Methodology Iterative process, follow development workflow Inter-connection between documents (cross checks) Use in the SAE AIR6110 standard Example of safety evaluation process Wheel-Brake System Example 15

16 Safety Analysis Workflow Aircraft-level (functions) Define failure conditions Allocate failure to system functions Preliminary System Safety Assessment System Functional Hazard Analysis (FHA) System Fault-Tree Analysis (FTA) System Safety Assessment Failure Mode and Effect Analysis Refined FTA with Quantitative Failures Rates System Development Cycle 16

17 Functional Hazard Analysis ARP4761, section 3 Identify and classify functions failure conditions Aircraft or System Level Aircraft, High-Level View Refinement at System Level Input for safety requirements specification Description and specification in FTA, DD or MA Reference of Aircraft Low-Level to System FHA Spreadsheet with reference to functions failures description 17

18 Fault-Tree Analysis ARP4761, section 4.1 Relationship of failure effects and failure modes Initial Failure Mode Reference to system hierarchy Support with Open-Source and Commercial Tools Failure Mode Fault Occurrence 18

19 Markov Chain ARP4761, section 4.1 Evaluation of system behavior over time Probability of being in particular states Analysis and evaluation of fault states Support with Commercial and Open-Source Tools 19

20 Failure Mode and Effect Analysis ARP4761, section 4.2 Impact of Fault at a Higher Levels Start from Function Level to System/Aircraft Level Spreadsheet/textual document 20

21 Agenda Overview of AADL & Error-Model Annex Overview of ARP4761 & Safety Evaluation Support of Safety Evaluation with AADL Case-Study On-Going Work Discussion 21

22 AADL & Safety Evaluation Tool Overview FHA Spreadsheet FTA CAFTA OpenFTA Markov Chain PRISM FMEA Spreadsheet Use error propagations Use composite behavior Error flows Use error flow Error behavior Error behavior Propagations 22

23 Safety Analysis & AADL Preliminary System Safety Assessment (PSSA) support High-level component, interfaces from the OEM Automatic generation of validation materials (FHA, FTA) System Safety Assessment (SSA) support Use refined models from suppliers Enhancement of error specifications Support of quantitative safety analysis (FTA, FMEA, MA) System Development Cycle 23

24 Evolution of Safety Analysis process with AADL Preliminary System Safety Assessment Component types (system interfaces) Component implementation Validation Materials (FHA, FTA) Check PSSA and SSA consistencies Validation with quantitative fault rates (FMEA, FTA, DD, MA) Refinement & development evolution System Safety Assessment 24

25 Safety Analyses on Refined Architecture Aircraft-Level Safety Analysis Define aircraft failure conditions Allocate failure to system functions Perform PSSA and SSA Avionics Subsystem Level Safety Analysis Perform PSSA and SSA at subsystem level Ensure consistency with aircraft level analysis Navigation Sub-Subsystem Level Safety Analysis Perform PSSA and SSA at sub-subsystem level Ensure consistency with aircraft level analysis System Architecture Refinement System System Subsystem Subsystem System Subsystem Subsystem Subsystem Subsystem 25

26 Evolution of the AADL model Component extension, refinement & implementation AADL model Version n AADL model Version n + 1 Development Process 26

27 Evolution of Safety Assessment with AADL AADL model version n AADL model version n + 1 Automatic Fault-Tree Generation Automatic Fault-Tree Generation FTA refinement & improvement FTA Version n FTA Version n + 1 Development Process 27

28 Functional Hazard Analysis Support Use of component error behavior Error propagations rules Internal error events FHA Specify initial failure mode Define error description and related information Create spreadsheet containing FHA elements To be reused by commercial or open-source tools 28

29 Fault-Tree Analysis Support Use of composite error behavior FTA nodes FTA Use of component error behavior Incoming error events Walk through the components hierarchy Generate the complete fault-tree Focus on specific AADL subcomponents Export to several tools Commercial: CAFTA Open-Source: OpenFTA 29

30 Markov-Chain Support Use of component error behavior Error propagations rules Error transitions Markov Chain Map states and error types into specific values Tool-specific approach Ability to evaluate system state over time What is the probability my system is failing within 30 days? Export to open-source tools, PRISM 30

31 Failure Mode and Effects Support Use of component error behavior Error propagations rules (source, sink, etc.) Internal error events FMEA Traverse all error paths Record impact over the components hierarchy Use error description and related information Create spreadsheet containing FHA elements To be reused by commercial or open-source tools 31

32 Reliability Block Diagram aka ARP4761 Dependence Diagram (DD) Use of composite error behavior Error propagations rules (source, sink, etc.) Internal error events RDB Compute reliability of the Dependence Diagram Use of recover and failure events Overall probability of system failure Support in OSATE (built-in) 32

33 Agenda Overview of AADL & Error-Model Annex Approach for Safety Evaluation Support of Safety Evaluation with AADL Case-Study On-Going Work Discussion 33

34 Wheel Brake System Development of a public model Available on AADL public wiki Use of core and additions of AADL Error-Model (safety) + ARINC653 annexes (specific architecture) Demonstration for the System Architecture Virtual Integration consortium Relevance for the avionics domain Apply the technology/toolset on a known example Generation of FHA, FTA, MA & FMEA 34

35 AADL model root system NoService NoPower NoPressure InvalidReport Software and/or RuntimeError 35

36 AADL model, BSCU variations 36

37 FHA of the root system 37

38 FTA of the root system Focus on a specific AADL subcomponent 38

39 FTA of the BSCU subcomponent 39

40 FMEA of the root system Current State Out propagation Propagation path Out propagation or error containment Component 1 Component 2 40

41 Agenda Overview of AADL & Error-Model Annex Overview of ARP4761 & Safety Evaluation Support of Safety Evaluation with AADL Case-Study Conclusion Discussion 41

42 Conclusion Facilitate Safety Evaluation Derives safety materials from existing assets Automate evaluation & check architecture consistency Improve evaluation reliability & robustness Support for incremental evaluation Investigate interaction with other system characteristics Behavior specification 42

43 Agenda Overview of AADL & Error-Model Annex Approach for Safety Evaluation Support of Safety Evaluation with AADL Case-Study Case-Study Discussion 43

44 Contact Presenter / Point of Contact Dr. Julien Delange Telephone: jdelange@sei.cmu.edu U.S. Mail Software Engineering Institute Customer Relations 4500 Fifth Avenue Pittsburgh, PA USA Web Customer Relations info@sei.cmu.edu Telephone: SEI Phone: SEI Fax:

45 Copyright 2013 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under Contract No. FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN AS-IS BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at permission@sei.cmu.edu. Carnegie Mellon is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. DM