Software Aspects of Aviation Systems Certification

Transcription

1 Software Aspects of Aviation Systems Certification Andrew J. Kornecki Embry Riddle Aeronautical University Daytona Beach, FL Heavily plagiarized from the works of RTCA DO-205 / EUROCAE WG-71 committee members Copyright A.J. Kornecki, 2011 page 1 Where I have been stuck for a quarter of century? Embry Riddle Aeronautical University teaching and research in engineering, manufacturing, marketing, management, piloting, and maintenance for aerospace ~500 full-time faculty members, 81% with a doctoral degree Bachelor, Master, and PhD programs ~4,900 students The annual operating budget of ~$240M Copyright A.J. Kornecki, 2011 page 2 1

2 ERAU Daytona Beach Campus College of Engineering College of Aviation: College of Business Instructional Center Copyright A.J. Kornecki, 2011 page 3 What so special about software in aviation? August 2005: a Malaysian Airlines Boeing ER suffered an inflight upset en-route from Perth to Kuala Lumpur The Australian TSB concluded A contributing safety factor was that an anomaly existed in the component software hierarchy Inputs from a known faulty accelerometer were allowed to be processed by the air data inertial reference unit (ADIRU) and used by the PFC, SAADR and other aircraft systems Example of a systems requirement error where the ADIRU would reinstate known failed accelerometers Copyright A.J. Kornecki, 2011 page 4 2

3 Equipment Rules (JAR/FAR ) Essential equipment must be designed to perform its intended functions The airplane systems and associated components, considered separately and in relation to other systems, must be designed so that: o The occurrence of any failure condition which would prevent the continued safe flight and landing of the airplane is extremely improbable, and o The occurrence of any other failure conditions which would reduce the capability of the airplane or the ability of the crew to cope with adverse operating conditions is improbable Copyright A.J. Kornecki, 2011 page 5 How aviation systems are developed? ARP4754A/ED79A Guidelines for Development of Civil Aircraft and Systems SAE International guideline to support certification of aircraft systems FAA Advisory Circular AC A System Design and Analysis, explains certification methodology ARP4761 Safety Assessment ARP4754A System Requirements correct? complete? System people Allocated to SW Allocated to HW W A L L SW and HW people RTCA DO-178C D&V processes RTCA DO-254 D&V processes Copyright A.J. Kornecki, 2011 page 6 3

4 Typical Avionics LRU Line Replaceable Unit include hardware in form of PLD, ASIC, and FPGA components The LRU processor contains software: BSP, RTOS, DRIVERS, LIBRARIES, and the APPLICATIONS PLD ASIC FPGA CPU BSP RTOS APP SW LIB DRIVERS SW: DO-178C HW: DO-254 Copyright A.J. Kornecki, 2011 page 7 Relations: System Software Hardware Aerospace Recommended Practice ARP 4761 address safety assessment as the base for development defining assurance levels Assurance level establishes the level of process rigor commensurate with the functional failure condition ARP 4754A, DO-254 and DO-178C highlight the need for systems to assess the potential system safety and system requirements impacts of derived requirements They establishes confidence that the development has been accomplished in a sufficiently disciplined manner to limit the likelihood of development errors that could impact aircraft safety They are all dependent on each other and use objectives-based tables Copyright A.J. Kornecki, 2011 page 8 4

5 So, what is this certification about? Certification guidelines are designed and agreed upon by the RTCA special committee Subsequent approval and regulatory AC by the FAA Airborne software (DO-178), airborne hardware (DO- 254), and CNS/ATM ground systems (DO-278) DO-178 guidelines describe software aspect of system lifecycle, planning, development, verification, configuration management, quality assurance, certification, and additional considerations The standard defines five assurance categories: from A catastrophic 10-9, B hazardous 10-7, C major 10-5, D minor 10-3, and E no effect) called software levels The guidelines address the issue of lack of software visibility at the system level Copyright A.J. Kornecki, 2011 page 9 We cannot have... Executable Object Code Applicant PROCESS Copyright A.J. Kornecki, 2011 page 10 5

6 So what is the secret? Standards/guidance must be written and evidence of compliance with these rules should be provided rules in form of documents defining precisely how to perform an activity (methods, means, constraints, expected outputs, etc.) consistent precise traceable developed Each requirement or design item should be verifiable thus we need to pay attention to CM and QA Copyright A.J. Kornecki, 2011 page 11 Configuration Management (CM) CM assists in satisfying general objectives: o Control configuration of the software throughout the software life cycle o Be able to replicate the executable object code o Control process inputs and outputs during the software life cycle o Provide baselines for review, assessment and change control o Ensure problems management and change control o Ensure archiving and recovery Copyright A.J. Kornecki, 2011 page 12 6

7 QA is to provide evidence that: Quality Assurance (QA) o What s done is what s described in plans o Transition criteria are reached o A conformity review of the software is conducted Main characteristics of QA o Active role during the life cycle process o Independence? Does this program ever halt? NO YES end Copyright A.J. Kornecki, 2011 page 13 Safety Assessment & Requirements Start QA PLANNING PHASE System Requirements Develop Plans, Standards, Checklists Develop Traceability Lifecycle Implement CM Verification & Validation High Level Requirements Certification Conformity Review Integration Logic & Code Architecture & Design Low Level Requirements DEVELOPMENT AND VERIFICATION PHASE Copyright A.J. Kornecki, 2011 page 14 7

8 40 years of DO-178 History Progress: o DO-178 (1982) artifacts, document, traceability, testing o DO-178A (1985) process, testing, components, criticality levels, reviews, waterfall methodology o DO-178B (1992) integration, transition criteria, diverse development methods, data, tools In 1992, software engineering was 24 years old... Two revisions in ten years, then twenty years nothing? Striving for a better consensus, taking into account : o legacy from the clarification groups o lessons learnt in applying DO-178B / ED-12B o newly available industrial means and technologies Copyright A.J. Kornecki, 2011 page 15 Long Awaited Transition to DO-178C SC-205/WG-71 joint committee comprised of seven sub-groups sponsored by RTCA and EUROCAE Terms of Reference: Correct Known Errors, Make Guidance More Usable, Allow Current and Future Technologies/Methods to Be Utilized Designed to reduce subjectivity and address modern software technologies Six years of committee work, 22 plenary meetings and countless sub-group meetings, concluding with the final plenary at ERAU November 15-18, 2011 Approved by RTCA/EUROCAE (Nov 2011/Jan 2012) FAA to publish an Advisory Circulars (AC) recognizing DO-178C and related work products (supplements) Copyright A.J. Kornecki, 2011 page 16 8

9 SC-205/WG-71 Committee Organization USA: SC-205, RTCA, Inc. o Jim Krodel (Pratt & Whitney) Chair o Leslie Alford (Boeing) Secretary o Barbara Lingberg, FAA Representative o Hal Moses, RTCA Representative Europe: WG-71, EUROCAE o Gerard Ladier (Airbus) Chair o Ross Hannan, (Sigma Associates Aerospace) Secretary o Jean-Luc Delamaide, EASA Representative o Roland Mallwitz, EUROCAE Representative SG1: Document Integration: Gasiorowski (WCS)/Ashpole (Silver Aten) SG2: Issues & Rationale: Moyer (Rockwell Collins)/Hannan (Sigma) SG3: Tool Qualification: Rierson (Digital Safety)/Pothon (ACG) SG4: Model Based Design: Lillis (Goodrich)/Lionne (EADS) SG5: Object Oriented Technologies: Chelini (Verocel)/Hunt (AICAS) SG6: Formal Methods: Hayhurst (NASA)/Brown (Rolls Royce) SG7: Special Considerations & CNS/ATM: Heck (Boeing)/Stewart (NATS) Copyright A.J. Kornecki, 2011 page 17 DO-178C/ED-12C: Content clarification and improvement in consistency and accuracy separation of why? from how? evolution of the why? DO-178C ED-12C AIRBORNE SYSTEMS Interface Spec Supplement Object Oriented Supplement Formal Methods Supplement Model Based Supplement Tools DO-278A / ED-109A CNS/ATM DO-248C/ED-94C FAQ/DP/RATIONALE Copyright A.J. Kornecki, 2011 page 18 9

10 DO178C Objectives The main element of DO178C are tables of objectives related to: planning, development, verification (requirements, design, coding), testing, verification of verification, configuration management, quality assurance, and certification liaison process Each table includes entries for objective (identifier, description, reference), applicability by software level (A-D), output (description, reference), and control category (CC1,CC2) by software level There is distinction of objectives to be satisfied either with or without independence The number of objectives differs for different levels The focus is on the lifecycle transition criteria Each supplement provides domain specific guidance and additional entries to the tables of objectives (or an additional table) Copyright A.J. Kornecki, 2011 page 19 Example: DO-178C Table of Objectives (one of many) Copyright A.J. Kornecki, 2011 page 20 10

11 DO178C Software Life-cycle Data plan for software aspect of certification (PSAC) software development plan (SWDP) software verification plan (SWVP) software configuration management plan (SCMP) software quality assurance plan (SQAP) requirements, design and code standards software requirements specification data (SWRS) software design specification data (SWDS) source and executable code verification procedures and test cases (SWTC) verification results life-cycle software configuration index (SWCI) problem reports records: o configuration management (CM) o quality assurance (QA) software accomplishment summary (SWAS) Copyright A.J. Kornecki, 2011 page 21 Software Means of Conformance It is not feasible to assess the number or even the types of software errors, if any, that may remain after completion of a complex system design, development, and test RTCA DO-178 / EUROCAE ED-12 provides guidance and description of acceptable means for assessing and controlling the software installed in airborne systems The concept is based on five principles: o Process o Verification o Independence o Consensus o Flexibility Copyright A.J. Kornecki, 2011 page 22 11

12 DO-178/ED-12: PROCESS DO-178/ED-12 is primarily a process-oriented document => Set of requirements on the processes (planning, development, verification, etc.) and their outputs The occurrence of any failure condition which would prevent the continued safe flight and landing of the airplane is extremely improbable => Evidences on the fulfilment of these requirements vary depending on the software criticality level PRINCIPLE 1: PROCESS We can t get clean water from a dirty pipe Copyright A.J. Kornecki, 2011 page 23 DO-178/ED-12: Life Cycle and Processes Definition of three separate processes that will be combined for a given project to describe its life cycle: o Planning (organization/plans) o Development (specification, design, coding, integration) o Integral (verification, configuration management, quality assurance, certification liaison) Define for each process: o assurance objectives o means of achieving those objectives o process input data o process activities o process products o transition criteria Copyright A.J. Kornecki, 2011 page 24 12

13 DO-178/ED-12: VERIFICATION The most important section of DO-178/ED-12 o Many more pages in the document o Workload incurred (e.g. for A380: four lines of test for one line of embedded code ) Integral process => applies to all processes used to detect and identify errors introduced during development Approaches: Analysis (quantitative examination), Review (qualitative inspection), Test (functional and structural coverage) PRINCIPLE 2: VERIFICATION A clean pipe may not deliver clean water thus filters are used to detect and eliminate impurities Copyright A.J. Kornecki, 2011 page 25 DO-178/ED-12 Example: Verification Process A-3.2 Accuracy & Consistency A-3.3 HW Compatibility A-3.4 Verifiability A-3.5 Conformance A-3.7 Algorithm Accuracy A-4. 8 Architecture Compatibility A-4.9 Consistency A-4.10 HW Compatibility A-4.11 Verifiability A-4.12 Conformance A-4.13 Partition Integrity Software Architecture System Requirements (A-2: 1, 2) High-Level Requirements (A-2: 3, 4, 5) A-3.1 Compliance A-3.6 Traceability A-4.1 Compliance A-4.6 Traceability Low-Level Requirements Compliance: with requirements Conformance: with standards A-4.2 Accuracy & Consistency A-4.3 HW Compatibility A-4.4 Verifiability A-4.5 Conformance A-4.7 Algorithm Accuracy A7 Verification of verification (Functional & Structural coverage) A-5.2 Compliance A-5.3 Verifiability A-5.4 Conformance A-5.6 Accuracy & Consistency (A-2: 6) Source Code A-5.1 Compliance A-5.5 Traceability A-6.3 Compliance A-6.4 Robustness A-5. 7 Complete & Correct (A-2: 7) Executable Object Code A-6.5 Compatible With Target A-6.1 Compliance A-6.2 Robustness Copyright A.J. Kornecki, 2011 page 26 13

14 DO-178/ED-12: INDEPENDENCE Certification Liaison the objective is to ensure effective communication and understanding between the applicant and the certification authorities Means: o The Plan for Software Aspects of Certification (PSAC), given to the Authorities as early as possible o Reviews carried out by the certification authorities software specialists as much as they want o Software Accomplishment Summary(SAS) o Software Configuration Index (SCI) PRINCIPLE 3: INDEPENDENCE Potentially opposite interests are at stake thus independent authorities assess the process and product Copyright A.J. Kornecki, 2011 page 27 DO-178/ED-12: CONSENSUS Joint meetings RTCA SC-205 and EUROCAE WG-71 with openness and consensus principle o more than 1,200 people registered on the WEB site o ~120 attendees in each meeting: aircraft/engine manufacturers, equipment suppliers, authorities, scientists and consultants o the final text agreed by each of the attendees PRINCIPLE 4: CONSENSUS All the interests must be taken into account to build a guidance recognized by all concerned specialists and actors Copyright A.J. Kornecki, 2011 page 28 14

15 DO-178/ED-12: FLEXIBILITY DO-178B/ED-12B takes into account the inputs, constraints, and requirements from all stakeholders forging a consensus between airframe manufacturers, equipment suppliers, certification authorities DO-178B/ED-12B attempts to stay not prescriptive on the means thus less sensitive to technology evolution PRINCIPLE 5: FLEXIBILITY Only requirements are mandatory, not the means of achieving them and thus building the pipe is to be dealt with by the suppliers Copyright A.J. Kornecki, 2011 page 29 SC-205/WG-71 Committee Work Products 1. DO-178C/ED-12C Software Considerations in Airborne Systems and Equipment Certification 2. DO-248C/ED-94C Supplemental Information for DO-178C and DO-278A 3. DO-278A/ED-109A Software Integrity Assurance Considerations for Communication, Navigation, Surveillance, and Air Traffic Management (CNS/ATM) 4. DO-330/ED-215 Software Tool Qualification Considerations 5. DO-331/ED-216 Model-Based Development and Verification Supplement to DO-178C and DO-278A 6. DO-332/ED-217 Object Oriented Technology & Related Techniques Supplement to DO-178C and DO-278A 7. DO-333/ED-218 Formal Methods Supplement to DO-178C & DO-278A Copyright A.J. Kornecki, 2011 page 30 15

16 Tools Supplement (1) Necessary when processes required by the rest of the document are eliminated, reduced or automated by the use of a software tool whose outputs are not verified Three qualification criteria depending on the risk associated to the tool usage o Criteria 1: Development Tool o Criteria 2: Verification Tool which could have an impact on the resulting software and is used to justify the elimination or reduction of: a verification process other than that automated by the tool, or a development process which could have an impact on the resulting software o Criteria 3: Verification Tool Copyright A.J. Kornecki, 2011 page 31 Two distinct roles are defined : Tools Supplement (2) o The user defines needs (Tool Operational Requirements - TOR) and validate the tool in its usage context o The developer defines development specification (Tool Requirements -TR), develops the tool, and provides the tool life cycle data Sharing activities between these two roles are defined for COTS tools Combination of the qualification criteria with the software level to give the TQL - Tool Qualification Level Developer oriented new table of objectives Copyright A.J. Kornecki, 2011 page 32 16

17 Tools Supplement (3) TQL 5 needs only contracting authority requirements, mainly concentrating on the TOR defining all the user requirements including the validation of the tool versus the need expressed in the avionics PSAC + Impact of known problems on the TOR TQL 4 adds project management requirements: o TOR reviews (complete, accurate, and consistent) o Definition of processes in plans o Definition of the TR and verification of TOR o Verification of the tool and TR coverage o Configuration and change managements TQL 1/2/3 add multiple implementation requirements which depend on the level of the final product software which is developed with the tool Copyright A.J. Kornecki, 2011 page 33 Model Based Technology Supplement Provides guidance on how model-based development and verification (MBDV) can produce software that meets the DO-178C objectives Use of models supports: o Unambiguous documentation of requirements and architecture o Use of automated code generation o Use of automated test generation Models can be used for system requirements Analysis tools for verification of the requirements and architecture Testing is still required! Copyright A.J. Kornecki, 2011 page 34 17

18 Object Oriented Technology Supplement Provides guidance on how object-oriented and related techniques can produce software that meets the DO- 178C objectives OOT Basic Concepts o Classes and Objects o Types and Type Safety o Hierarchical Encapsulation o Polymorphism o Function Passing and Closures o Method Dispatch OOT Key Features and Related Techniques include: inheritance, parametric polymorphism, overloading, type conversion, software exceptions handling, dynamic memory management, virtualization Copyright A.J. Kornecki, 2011 page 35 Formal Methods Technology Supplement (1) Formal methods - mathematically based techniques for the specification, development, and verification of SW Formal Models - unambiguous syntax and semantics o Syntax rules for the notation used for the model so that models can only be constructed in a certain way o A model constructed using the correct syntax can only be interpreted one way Formal Analysis - deductive methods (e.g. theorem proving), model checking, and abstract interpretation Copyright A.J. Kornecki, 2011 page 36 18

19 Formal Methods Technology Supplement (2) The Formal Analysis (FA) may replace: o reviews & analysis o conformance test versus /HLR et /LLR o software integration tests o robustness tests FA may help for the verification of compatibility with the hardware FA cannot replace HW/SW integration test Structural Coverage demonstrated by: o each requirement is completely covered o requirements are complete in regards of the attended function o no non expected dependencies between output and input data o there is no dead code Copyright A.J. Kornecki, 2011 page 37 Summary : differences between DO-178B and C Fixed Known Errors Updated text to ensure use of specific terms is consistent throughout DO-178 Identified "hidden" objectives Coordinated flow between software (DO-178C) and system (ARP 4754A) processes Expanded/Clarified definitions for dead code and deactivated code Clarified that structural coverage analysis of data and control coupling between code components should be based on the results of the requirements-based tests Improved definition for Modified Condition/Decision Coverage (MCDC) to allow "Masking MCDC" and "Short Circuits" Clarified tool qualification guidance and moved it to its own supplement Removed some objectives for Level D software Created supplements for technology specific guidance (for example, modeling, object oriented technology (OOT) and formal methods) Formalized requirement for traceability data; previously implied but expected Introduced Parameter Data Items (configuration files) as a software life cycle data item A parameter data item (PDI) is a set of data that influence the behavior of the software without modifying the Executable Object Code and that is managed as a separate configuration item. Examples include databases and configuration tables. Added "Activities" to the Annex tables Synchronized Annex tables with DO-278A Copyright A.J. Kornecki, 2011 page 38 19