Airbus A350 CERTIFICATION REVIEW ITEM

Transcription

1 European Aviation Safety Agency Airbus A350 CERTIFICATION REVIEW ITEM Ref.: F-32 Status: Closed Date: Page: 1 of 9 Next Action: Subject: Management of Open Problem Reports (Software and Complex Electronic Hardware) Category: Interpretative Material Requirements: CS , CS Advisory Material: ED-12B/DO-178B, ED80/DO254 Primary Panel: Software Secondary Panels: Statement of Issue Problems related to software and Complex Electronic Hardware (CEH) may surface relatively late in the industrial development process. When these problems do not affect the safety of the aircraft (compliance to CS /1309 is demonstrated), the applicant may decide to propose for certification, items with known software and CEH problems. This situation is assumed by DO-178B/ED-12B, section 11.20(j), copied hereafter: Software status: this section [of the Software Accomplishment Summary - SAS - document produced for certification] contains a summary of problem reports unresolved at the time of certification, including a statement of functional limitations. For Complex Electronic Hardware this situation is assumed by DO-254/ED-80, section , copied hereafter: Hardware status: this section [of the Hardware Accomplishment Summary- document produced for certification] contains a summary of problem reports unresolved at the time of certification, including a statement of functional limitations. The intent of this CRI is to discuss the issues related to Problem Management for Software and Complex Electronic Hardware. EASA Position (dated 3 April 2008): Objectives The objective of any airborne software and CEH development and qualification should be to minimise the number and severity of Open Problem Reports (OPRs) in any software and CEH release proposed for certification. The OPR management principles and assessment guidelines detailed in this CRI should not, in any case, be understood as a justification to deviate from this prevailing objective.

2 Airbus A350 CRI: F-32 Page: 2 This CRI has two purposes: 1. To clarify the role of the aircraft manufacturer and the supplier in the assessment of limitations of a piece of equipment embedding software or CEH because of known problems at the time of certification. It should be noted that even if the equipment supplier/system supplier has sufficient knowledge to explain the functional effect on the item of equipment/system for an OPR, only the aircraft manufacturer can assess or confirm the potential effect at aircraft level. 2. To help the assessment of the acceptability of a baseline released with Open Problems reports, by defining a harmonized classification of OPRs with an adequate recording. Scope This CRI is applicable to all systems with digital equipment containing Software with Development Assurance Level (DAL) A, B, C, D or containing Complex Electronic Hardware with a DAL A, B, and C. Terminology Text in italic in the definitions below is extracted from ED-12B/DO-178B and ED-80/DO-254 glossary. Problem: any of the following features: error, fault, failure, deviation to the rules Error: with respect to software, a mistake in requirements, design or code. A mistake in requirements or design generally causes an error in the code. The term defect is also used for mistake in the code. For CEH, a mistake in requirements, design or implementation. Fault: A manifestation of an error in software (at execution, an anomalous variable or state of the software). An error in the code may cause or contribute to a fault if executed under certain conditions. A fault, if it occurs, may cause a failure. For CEH, (1) A manifestation of a flaw in hardware due to an error or random event. A fault, if it occurs, may cause a failure. (2) An undesired anomaly in an item. Failure: The inability of a system or system component to perform a required function within specified limits. A failure may be produced when a fault is encountered. A failure is a manifestation of a fault at system level. But a fault may also remain hidden at system level and have no operational consequence. Failure condition: The effect on the aircraft and its occupants both direct and consequential caused or contributed to by one or more failures, considering relevant adverse operational and environmental conditions. A failure condition is classified according to the severity of its effect as defined in EASA AMC Baseline - An identified and approved configuration that thereafter serves as the basis for further design, and that is changed only through change control procedures. Deviation to the rules: non-conformity of the development process to the plans, development standards, applicable CRIs. In the particular case where a non conformity to plans or development standards is intentional and plans or development standards planned to be modified accordingly, such a non conformity may not be recorded as a problem but identified and justified in the compliance status of the SAS or HAS. Open Problem Report (OPR): problem which has not been corrected at the time the software or CEH is presented for certification.

3 Airbus A350 CRI: F-32 Page: 3 Typology of Open Problems Reports An OPR logged should be classified depending on the nature and effect of the OPR as: Type 0: a problem whose consequence is a failure under certain conditions - of the system with a safety impact. Type 1: a problem whose consequence is a failure under certain conditions - of the system having no safety impact on the aircraft (this needs to be confirmed by the aircraft manufacturer). If agreed between aircraft manufacturer and supplier, this type should be divided into two sub-types: - Type 1A: failure with a significant functional consequence; the meaning of significant should be defined in the context of the related system in agreement between aircraft manufacturer and supplier (for instance cockpit effect ). - Type1B: failure with no significant functional consequence. Type 2: a fault which does not result in a failure (i.e.: no system functional consequence, fault not detectable by the crew in foreseeable operating conditions). Type 3: Any problem which is not type 0, 1 or 2, but a deviation to the rules (plans or development standards, applicable CRI s). If agreed between applicant and supplier, this type should be divided into two sub-types: - Type 3A: a significant deviation whose effects could be to lower the assurance that the Software or the CEH behaves as intended and has no unintended behaviour. - Type 3B: a non significant" deviation to the methodology (plans) that does not affect the assurance obtained. Guidelines on OPR management: EASA considers that as far as possible a root cause analysis should be performed for all OPRs, except in exceptional cases when the root cause analysis is not feasible. This nonfeasibility should be justified. In the case of Type 0, 1 or 2, this root cause analysis should lead to the identification of the error in the code for software or in the hardware implementation for CEH and, if any, associated methodological deviations. For type 3, the root cause analysis consists of the identification of the methodological deviation associated to the problem. For CEH Commercial Off-The-Shelf Components (COTS), on a case-by-case basis, when root cause analysis is not feasible, EASA could accept that Electronic Component Management (DO 254/ED ) and Product Service Experience (DO254/ED ) in support of the list of existing OPR from suppliers could be provided as acceptable compliance demonstration to this IM (as recalled by IM of A350 CRI F-08 on Electronic Hardware Development Assurance). All OPRs should be classified according to the typology of problems defined in this CRI, or equivalent typology. If an equivalent typology is proposed, any new type(s) should correspond to only one type (0, 1, 2 or 3) as defined in this CRI. All OPRs should be described in order to substantiate their classification into adequate types; this description should be recorded. When previously developed software or CEH is used, previously existing OPRs shall be reassessed in the A350 operational environment. Operational environment being the installation and the application aspects In order to avoid decreasing the assurance of the software or CEH quality due to an increasing number of OPRs, the following objectives should be taken into account:

4 Airbus A350 CRI: F-32 Page: 4 Remove limitations at the earliest opportunity. Restore conformity to the specifications at the earliest opportunity. Fix any OPR within a delay compatible with the assessed consequences. Per ED-79/ARP4754 section 9.2.2, problem reporting should be managed at system level The following type based objectives should be taken into account: Type 0: such OPRs basically should be fixed before certification, or adequate mitigation means (for instance operating limitations) should be proposed, such that there is no adverse safety effect at the aircraft level. Type 0 and 1: Potential effects should be assessed at the system level and, if necessary, at the aircraft level. If necessary appropriate limitations should be defined in order to ensure safety. Type 1: No safety impact on the aircraft should be justified; this justification should be recorded. Type 2: Justification that the error cannot cause a failure should be recorded. For simple cases, this justification may be a simple statement based on engineering judgement. In some specific cases, this justification may imply specific additional validation and/or verification activities Content of Software/Hardware Accomplishment Summaries (SAS/HAS): All type 0 to 3 OPRs should be recorded in the SAS/HAS or equivalent certification document with the following information: Supplier identification of the OPR (configuration management number) Type Function affected (description if Yes, or Statement No ) Short description (including a brief summary of the root cause, when available) Date when the OPR was opened Timely objective for closure of this OPR Short justification why it can be left open Mitigating means to ensure no adverse safety effect - if applicable OPR interrelationship - if applicable. Although a limited number of type 2 or 3 OPRs should normally not prevent certification, a high number of type 2 or 3 OPRs, or lack of action plans for closure of type 2 and 3 OPR s are general indicators of a lack of software or hardware assurance. The EASA team may reject a request for certification if the number of remaining open OPR s is too high, or if there is no evidence of an adequate action plan to close open OPRs. Content of System Certification Summary or equivalent document: The System Certification Summary or equivalent certification document should mention: The identification of type 0 and 1 OPRs and the description of their impact at the system level or, if necessary, at the aircraft level (including, any associated operational limitations and procedures).

5 Airbus A350 CRI: F-32 Page: 5 Airbus Position (dated 20 February 2009): With the letter ref. V02M dated 20 February 2009 Airbus concurs to CRI F-32 Management of open problem reports (Software and Complex Electronic Hardware) issue 1. Conclusion (dated 8 December 2009): Agreement has been reached. The agreed interpretative material to be used for the A350 Type Certification is defined in the appendix to this CRI. The CRI is closed at Issue 2. Heiko Honert A350 Project Certification Manager

6 Appendix to A350 CRI F-32 Page: 1 A350 Interpretative Material F-32 Management of Open Problem Reports (Software and Complex Electronic Hardware) Objectives The objective of any airborne software and CEH development and qualification should be to minimise the number and severity of Open Problem Reports (OPRs) in any software and CEH release proposed for certification. The OPR management principles and assessment guidelines detailed in this Interpretative Material should not, in any case, be understood as a justification to deviate from this prevailing objective. This Interpretative Material has two purposes: 1. To clarify the role of the aircraft manufacturer and the supplier in the assessment of limitations of a piece of equipment embedding software or CEH because of known problems at the time of certification. It should be noted that even if the equipment supplier/system supplier has sufficient knowledge to explain the functional effect on the item of equipment/system for an OPR, only the aircraft manufacturer can assess or confirm the potential effect at aircraft level. 2. To help the assessment of the acceptability of a baseline released with Open Problems reports, by defining a harmonized classification of OPRs with an adequate recording. Scope This Interpretative Material is applicable to all systems with digital equipment containing Software with Development Assurance Level (DAL) A, B, C, D or containing Complex Electronic Hardware with a DAL A, B, and C. Terminology Text in italic in the definitions below is extracted from ED-12B/DO-178B and ED-80/DO-254 glossary. Problem: any of the following features: error, fault, failure, deviation to the rules Error: with respect to software, a mistake in requirements, design or code. A mistake in requirements or design generally causes an error in the code. The term defect is also used for mistake in the code. For CEH, a mistake in requirements, design or implementation. Fault: A manifestation of an error in software (at execution, an anomalous variable or state of the software). An error in the code may cause or contribute to a fault if executed under certain conditions. A fault, if it occurs, may cause a failure. For CEH, (1) A manifestation of a flaw in hardware due to an error or random event. A fault, if it occurs, may cause a failure. (2) An undesired anomaly in an item. Failure: The inability of a system or system component to perform a required function within specified limits. A failure may be produced when a fault is encountered. A failure is a manifestation of a fault at system level. But a fault may also remain hidden at system level and have no operational consequence. Failure condition: The effect on the aircraft and its occupants both direct and consequential caused or contributed to by one or more failures, considering relevant adverse operational and environmental conditions. A failure condition is classified according to the severity of its effect as defined in EASA AMC Baseline - An identified and approved configuration that thereafter serves as the basis for further design, and that is changed only through change control procedures.

7 Appendix to A350 CRI F-32 Page: 2 Deviation to the rules: non-conformity of the development process to the plans, development standards, applicable CRIs. In the particular case where a non conformity to plans or development standards is intentional and plans or development standards planned to be modified accordingly, such a non conformity may not be recorded as a problem but identified and justified in the compliance status of the SAS or HAS. Open Problem Report (OPR): problem which has not been corrected at the time the software or CEH is presented for certification. Typology of Open Problems Reports An OPR logged should be classified depending on the nature and effect of the OPR as: Type 0: a problem whose consequence is a failure under certain conditions - of the system with a safety impact. Type 1: a problem whose consequence is a failure under certain conditions - of the system having no safety impact on the aircraft (this needs to be confirmed by the aircraft manufacturer). If agreed between aircraft manufacturer and supplier, this type should be divided into two sub-types: - Type 1A: failure with a significant functional consequence; the meaning of significant should be defined in the context of the related system in agreement between aircraft manufacturer and supplier (for instance cockpit effect ). - Type1B: failure with no significant functional consequence. Type 2: a fault which does not result in a failure (i.e.: no system functional consequence, fault not detectable by the crew in foreseeable operating conditions). Type 3: Any problem which is not type 0, 1 or 2, but a deviation to the rules (plans or development standards, applicable CRI s). If agreed between applicant and supplier, this type should be divided into two sub-types: - Type 3A: a significant deviation whose effects could be to lower the assurance that the Software or the CEH behaves as intended and has no unintended behaviour. - Type 3B: a non significant" deviation to the methodology (plans) that does not affect the assurance obtained. Guidelines on OPR management: EASA considers that as far as possible a root cause analysis should be performed for all OPRs, except in exceptional cases when the root cause analysis is not feasible. This nonfeasibility should be justified. In the case of Type 0, 1 or 2, this root cause analysis should lead to the identification of the error in the code for software or in the hardware implementation for CEH and, if any, associated methodological deviations. For type 3, the root cause analysis consists of the identification of the methodological deviation associated to the problem. For CEH Commercial Off-The-Shelf Components (COTS), on a case-by-case basis, when root cause analysis is not feasible, EASA could accept that Electronic Component Management (DO 254/ED ) and Product Service Experience (DO254/ED ) in support of the list of existing OPR from suppliers could be provided as acceptable compliance demonstration to this IM (as recalled by IM of A350 CRI F-08 on Electronic Hardware Development Assurance). All OPRs should be classified according to the typology of problems defined in this Interpretative Material, or equivalent typology. If an equivalent typology is proposed, any new type(s) should correspond to only one type (0, 1, 2 or 3) as defined in this Interpretative Material.

8 Appendix to A350 CRI F-32 Page: 3 All OPRs should be described in order to substantiate their classification into adequate types; this description should be recorded. When previously developed software or CEH is used, previously existing OPRs shall be reassessed in the A350 operational environment. Operational environment being the installation and the application aspects In order to avoid decreasing the assurance of the software or CEH quality due to an increasing number of OPRs, the following objectives should be taken into account: Remove limitations at the earliest opportunity. Restore conformity to the specifications at the earliest opportunity. Fix any OPR within a delay compatible with the assessed consequences. Per ED-79/ARP4754 section 9.2.2, problem reporting should be managed at system level The following type based objectives should be taken into account: Type 0: such OPRs basically should be fixed before certification, or adequate mitigation means (for instance operating limitations) should be proposed, such that there is no adverse safety effect at the aircraft level. Type 0 and 1: Potential effects should be assessed at the system level and, if necessary, at the aircraft level. If necessary appropriate limitations should be defined in order to ensure safety. Type 1: No safety impact on the aircraft should be justified; this justification should be recorded. Type 2: Justification that the error cannot cause a failure should be recorded. For simple cases, this justification may be a simple statement based on engineering judgement. In some specific cases, this justification may imply specific additional validation and/or verification activities Content of Software/Hardware Accomplishment Summaries (SAS/HAS): All type 0 to 3 OPRs should be recorded in the SAS/HAS or equivalent certification document with the following information: Supplier identification of the OPR (configuration management number) Type Function affected (description if Yes, or Statement No ) Short description (including a brief summary of the root cause, when available) Date when the OPR was opened Timely objective for closure of this OPR Short justification why it can be left open Mitigating means to ensure no adverse safety effect - if applicable OPR interrelationship - if applicable. Although a limited number of type 2 or 3 OPRs should normally not prevent certification, a high number of type 2 or 3 OPRs, or lack of action plans for closure of type 2 and 3 OPR s are general indicators of a lack of software or hardware assurance. The EASA team may reject a request for certification if the number of remaining open OPR s is too high, or if there is no evidence of an adequate action plan to close open OPRs. Content of System Certification Summary or equivalent document:

9 Appendix to A350 CRI F-32 Page: 4 The System Certification Summary or equivalent certification document should mention: The identification of type 0 and 1 OPRs and the description of their impact at the system level or, if necessary, at the aircraft level (including, any associated operational limitations and procedures).