CARE/ASAS Activity 3: Airborne Separation Minima: Extension Study

Transcription

1 CARE/ASAS Action CARE/ASAS Activity 3: Airborne Separation Minima: Extension Study Lessons learnt from the OSED/OHA activities and hazard brainstorm CARE/ASAS/Sofréavia/ Version September 17, 2002

2 DOCUMENT REVIEW Version Date Description of evolution Modifications /03/02 Initial report structure and objectives for Kick-Off meeting All /03/02 Integration of comments from Kick-off meeting All /04/02 Integration of comments from NLR dated 29/03/02, Analysis the OSA methodology applied to ASAS applications /04/02 Analysis of the OSED/OHA steps performed within the ASM study, by Sofréavia and EEC following audio-conference held on 03/04/ /04/02 Analysis of hazard brainstorm results by NLR, Update following comments from partners during the audio-conference held on the 23/04/ /05/02 Updates following internal review and discussions during the audio-conference held on the 14/05/ /05/02 Updates following comments from Progress Report Meeting held on the 24/05/02, Relationship between hazard brainstorm and OSA methodology /05/02 Integration of updated Analysis of hazard brainstorm results by NLR, Executive summary and conclusions /06/02 Editorial changes following partners review, Proposed issue of the final report /09/02 Minor changes following CARE-ASAS Management Board comments Sections 1 and 7, Section 2 and Annex A Section 3 Section 4 and Annex F, Sections 2 and 3 and Annex B Sections 3 and 4, Annex D deleted All, Annexes C, D and E deleted Sections 4, 6 and 7 All Sections 1, 2,3, and 4 CARE/ASAS/Sofréavia/ Version September 17, 2002 page i

3 CONSORTIUM Authors Authors Authors Representatives Anne Cloerec, Béatrice Raynaud Eric Hoffman Franck-Olivier Ripoll Henk Blom, Mariken Everdij Organisations SOFREAVIA EEC NLR Reviewer Colin Goodchild University Of Glasgow Reviewers Jean-Marc Loscos, Thierry Miquel CENA CARE/ASAS/Sofréavia/ Version September 17, 2002 page ii

4 Executive summary Background and context The CARE/ASAS Activity 3: ASM projects consisted of the investigation of safety influencing factors that may affect airborne separation minima. In that perspective, the use of the RTCA/EUROCAE OSA guidelines, at least the initial steps of OSED and qualitative OHA, were investigated on two selected ASAS applications. As part of the TOPAZ methodology, a hazard brainstorm was also organised to identify additional hazards that were due to the mitigation means identified in the autonomous operations OHA. Based on the experience gained through the ASM study, this extension study aims at developing lessons learnt from the OSED/OHA activities and the hazard identification brainstorm. Analysis of the OSA methodology for ASAS applications Fisrt, the characteristics of ASAS applications; the type of safety assessment required for ASAS applications; and the relevance of the OSA guidelines for ASAS applications are discussed, and lessons learnt from the use of the OSA guidelines in the ASM study are proposed. In the perspective of performing an end-to-end safety assessment, the RTCA/EUROCAE OSED/OSA guidelines were found particularly relevant for ASAS applications. However, some amendments would be required to the better address the characteristics and safety issues of these ASAS applications. In particular, the OSED guidance material should be amended to allow the adequate description of the airborne CNS and ASAS characteristics, as well as the operational use of these ASAS applications. The Hazard Classification Matrix (HCM) developed by the RTCA/EUROCAE should be amended with proper definition of the safety margins associated with ASAS operations. And, appropriate ASOR guidance material should be developed to help people taking into account the essential role of airborne separation criteria when determining the safety performance requirements. Analysis of the OSED/OHA steps for identifying safety-influencing factors A critical analysis of the OSED/OHA approach performed during the ASM study in the perspective of identifying safety-influencing factors is performed, and lessons learnt through the OSED/OHA performed in the ASM study are developed. The main OSED items discussed are the operating environment characteristics, the operating method with and without ASAS, the operational service description, and the functional characteristics description. Lessons learnt from the OSED development performed in the ASM study would be: to support an iterative approach for the OSED development taking into account the development and validation stage of the ASAS application; to better adapt the OSED information depending on the ASAS application category; CARE/ASAS/Sofréavia/ Version September 17, 2002 page i

5 to better adapt the type of information and level of detail in the OSED to the objectives of the safety assessment. The main OHA items analysed are the failure scenarios either related to technical or human errors, the relationship between failure scenarios and operational hazards (OHs), the relationship between OHs and phases of operations, the development of the risk mitigation strategy (on qualitative basis), and the implicit relationship between OHs. Lessons learnt from of both OHA performed in the ASM study would be: to combine both top-down approach (deductive) and bottom-up approach (inductive) within an iterative OHA process including both qualitative and quantitative assessment; to better determine and validate, through the OHA, the additional mitigation means that actually need to be put in place; to better assess the impact of human factors in ASAS operations; to allow effective clustering of adverse scenarios, and identification of relevant safety influencing factors, at different stages of development of the ASAS applications. Analysis of hazard identification brainstorm results The hazards from the different sources (i.e. OHA and hazard brainstorm) have been compared on two aspects: types of individual hazards and overlap between group of hazards. The example results of the ASM hazard type and overlap analysis indicate that the TOPAZ-based brainstorms and the OHA s have identified hazards that were complementary to each other. In particular, hazards that will not easily be identified with an OHA approach, i.e. OHA-unimaginable hazards, are typically the hazards that gobeyond the functional level of human tasks. However, the results of such a brainstorm is a list of hazards that are not analysed and clustered. A further analysis and clustering could be necessary after the brainstorm is finished. On the other side, hazard identification through brainstorming can be done at a very early stage of the operational concept development process. Relationship between the OSA methodology and brainstorm based hazard identification Based on the lessons learnt from the CARE-ASAS ASM study, an integrated and iterative approach for the safety assessment of ASAS applications, based on the RTCA/EUROCAE OSA methodology and the use of hazard brainstorm, is proposed. This approach supports the iterative assessment of operational hazards related to the ASAS application, with early identification through brainstorm and stepwise analysis of hazards at different level of detail. Successive OHA steps would consist in the identification and severity assessment of operational hazards; the preliminary frequency assessment of operational hazards, and finally the assessment of failure conditions leading to operational hazards. Along this iterative process, any refinement of the OSED, related or not to some previous safety assessment, would imply another iteration in the safety assessment of the ASAS application. CARE/ASAS/Sofréavia/ Version September 17, 2002 page ii

6 Recommendations On the basis of the overall work performed during the CARE-ASAS ASM study, the following recommendations need to be considered to support future work related to Operational Safety Assessment of ASAS applications: RTCA/EUROCAE Operational Safety Assessment (OSA) guidelines should be adapted to better address the characteristics of ASAS applications; Guidelines should be developed to support an iterative development of the OSED of an ASAS application along its development life cycle and safety assessment; Guidelines should be developed to support an iterative OHA process combining both top-down (or deductive) and bottom-up (or inductive) approach, as well as severity and frequency assessment. CARE/ASAS/Sofréavia/ Version September 17, 2002 page iii

7 Table of Content 1. INTRODUCTION DOCUMENT OBJECTIVES BACKGROUND AND CONTEXT CARE/ASAS ACTIVITY CARE-ASAS ACTIVITY 3: THE ASM STUDY OTHER RELATED WORK IN EUROPE FAA/EUROCONTROL ACTION PLAN 1 COORDINATION DOCUMENT OVERVIEW ANALYSIS OF OSA METHODOLOGY FOR ASAS APPLICATIONS THE OSA METHODOLOGY APPLICABILITY OF THE OSA METHODOLOGY TO ASAS APPLICATIONS CHARACTERISTICS OF ASAS APPLICATIONS WHICH SAFETY ASSESSMENT FOR ASAS APPLICATIONS RTCA/EUROCAE OSA GUIDELINES APPLIED TO ASAS APPLICATIONS LESSONS LEARNT ABOUT RTCA/EUROCAE OSA APPLIED TO ASAS APPLICATIONS ANALYSIS OF OSED/OHA FOR IDENTIFYING SAFETY INFLUENCING FACTORS WHAT HAS BEEN DONE IN THE ASM STUDY (AND WHY) CRITICAL ANALYSIS OF THE WORK DONE IN THE ASM STUDY ANALYSIS OF THE OSED DEVELOPMENT WITHIN THE ASM STUDY ANALYSIS OF BOTH OHA PERFORMED WITHIN THE ASM STUDY LESSONS LEARNT FROM THE OSED/OSA ACTIVITIES ANALYSIS OF HAZARD IDENTIFICATION BRAINSTORM RESULTS WHAT HAS BEEN DONE ON HAZARD IDENTIFICATION IN THE ASM STUDY (AND WHY) COMPARATIVE ANALYSIS OF THE HAZARDS IDENTIFIED DURING BRAINSTORM, OHA AND TOPAZ HAZARD DATABASE...31 CARE/ASAS/Sofréavia/ Version September 17, 2002 page iv

8 4.3. LESSONS LEARNT FROM TOPAZ-BASED HAZARD BRAINSTORMS RELATIONSHIP BETWEEN THE OSA METHODOLOGY AND BRAINSTORM BASED HAZARD IDENTIFICATION CONCLUSIONS RECOMMENDATIONS REFERENCES ANNEX A: RTCA/EUROCAE OPERATIONAL SAFETY ASSESSMENT METHODOLOGY OVERVIEW...1 ANNEX B: SPECIFIC USE OF THE OSA GUIDELINES IN THE CARE-ASAS ACTIVITY 3: ASM STUDY...6 CARE/ASAS/Sofréavia/ Version September 17, 2002 page v

9 List of figures Figure 1: RTCA/EUROCAE OSA methodology overview...6 Figure 2: High Level Conceptual Safety Model [15]...8 Figure 3: Various shared risk mitigation strategies of ASAS applications...8 Figure 4: High Level Conceptual Safety Model applied to the OHA step Figure 5: Interaction between airborne separation criteria and safety objectives and requirements in the ASOR step Figure 6: OSED step conducted in the ASM study Figure 7: OHA step conducted in the ASM study Figure 8: Operating environment characteristics Figure 9: Various failure modes analysed during the OHA Figure 10: Illustration of the relationship between failure scenarios and OHs Figure 11: Illustration of relationship between OHs and phases of operations Figure 12: Illustration of alternative risk mitigation strategies based on the OHA Figure 13: Illustration of implicit relationship between different OHs (depending on environment conditions) Figure 14: Illustration of implicit relationship between different OHs (depending on mitigation effectiveness) Figure 15: Input-output scheme for identification of the AA hazards within ASM project Figure 16: Analysis of overlap between identified AA-hazards within the ASM project Figure 17: Co-ordinated Requirements Determination process in RTCA/EUROCAE methodology [1]...1 Figure 18: Relationship between hazard classification and greatest likelihood of occurrences [1]...3 Figure 19: Concept Model of Operational Environment, the Relationships Among Faults, Failures, Procedural Errors, and Airspace Characteristics, and the Risk Mitigation Strategy [1]...4 CARE/ASAS/Sofréavia/ Version September 17, 2002 page vi

10 List of Tables Table 1: The four groups of hazards identified during the ASM project Table 2: Summary of analysis of hazard types identified during the ASM project Table 3: Operational Safety Assessment hazard classification matrix [1]...3 Table 4: Sequence in ASAS operations...7 Table 5: Air/ground operations (and communications) during sequence in ASAS operations...8 Table 6: ASAS and other CNS functions...8 Table 7: Activated functions during the sequence in ASAS operations...8 Table 8: Operational failure scenario template Table 9: Operational hazard description table CARE/ASAS/Sofréavia/ Version September 17, 2002 page vii

11 Acronym list AA ADS ADS-B ASAS ASM ATC ATM ATS CARE CD&R CDTI CENA CNS EEC EMERTA EUROCAE FAA FD FFAS FMS ICAO MAS MASPS NLR OH OHA OSED PO-ASAS R&D Autonomous Aircraft Automatic Dependent Surveillance Automatic Dependent Surveillance-Broadcast Airborne Separation Assurance System Airborne Separation Minima Air Traffic Control Air Traffic management Air Traffic Service Co-operative Actions of Research and development in EUROCONTROL Conflict Detection and Resolution Cockpit Display of Traffic Information Centre d Etude de la Navigation Aérienne Communication, Navigation, Surveillance EUROCONTROL Experimental Centre EMERging Technologies opportunities, issues and impact on ATM EURopean Organisation for Civil Aviation Equipment Federal Aviation Administration Flight Deck Free-Flight AirSpace Flight Management System International Civil Aviation Organisation Managed Airspace Minimum Aviation System Performance Standards National Aerospace Laboratory Operational Hazard Operational Hazard Analysis Operational Service and Environment Description Principles of Operations in the use of ASAS Research and Development CARE/ASAS/Sofréavia/ Version September 17, 2002 page viii

12 RCP RNP RSP TBS TIS-B TOPAZ Required Communication Performances Required Navigation Performances Required Surveillance Performances Time Based Sequencing Traffic Information Service Broadcast Traffic Organization and Perturbation AnalyZer CARE/ASAS/Sofréavia/ Version September 17, 2002 page ix

13 1. INTRODUCTION 1.1. Document objectives The document aims at developing lessons learnt from the OSED/OHA activities and the hazard brainstorm performed during the CARE-ASAS Activity 3: Airborne Separation Minima (ASM) study. More precisely, based on the experience gained through the CARE-ASAS ASM study of two ASAS applications (i.e., Autonomous Aircraft in FFAS, and Time-Based Sequencing in MAS), the document discusses the following items: Adequacy of the RTCA/EUROCAE Operational Safety Assessment (OSA) methodology (that include the OSED/OHA steps), initially developed for D/L applications, for safety assessment of Airborne Surveillance and Separation applications, in general; Adequacy of the OSED/OHA steps for the identification of safety influencing factors that may impact the airborne separation minima, and critical analysis of the tailored OSED/OHA methods used in the CARE-ASAS ASM study; Contribution of hazard identification brainstorm towards identifying safety critical hazards of ASAS applications; and Relationship between hazard identification brainstorm and the OSA methodology, in the perspective of an overall framework allowing for the assessment of operational hazards and allocation of safety requirements. The first and second items consolidate the lessons learnt through the work packages 1 (WP1: Identification of ASAS operational scenarios) and 2 (WP2: Identification of safety influencing factors) of the ASM study; the third item does the same for Task 3.1 (T3.1: Hazard brainstorm) of the WP3. The last item benefits from the relationship experienced between the work packages 2 and Task 3.1 of the ASM study. Based on the TOPAZ methodology, the ASM study (WP3: First estimation of a safe separation minima) also performed a risk assessment and evaluation of the Autonomous Aircraft application. However, the lessons learned from this work are not considered in this document. Finally, conclusions and recommendations relative to operational safety assessment of ASAS applications are provided Background and context CARE/ASAS Activity 3 In the prospect of giving flight crews some responsibilities related to aircraft separation, as envisaged with some ASAS applications, it is essential to identify all the equipment and operational elements required for the provision of safe separation by flight crews, defined as airborne separation. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 1

14 Therefore the major issue is the establishment of the airborne separation minima so as to achieve safe flight operations. Optimistic views are that they could be much smaller than radar separation and contributed to the publicity of autonomous separation. Other views are much more reserved and warn that minima might even be much larger. The CARE/ASAS Activity 3 deals with the assessment of the magnitude of the airborne separation minima as it directly affects the usefulness of ASAS applications. An investigation of experience in modelling and determining separation minima has been conducted initially, followed by the CARE/ASAS Activity 3: Airborne Separation Minima (ASM) study, that was carried out from November 2000 to December CARE-ASAS Activity 3: the ASM study As part of this activity, the CARE-ASAS Activity 3: Airborne Separation Minima (ASM) project consists of the investigation of safety influencing factors that may affect airborne separation minima, and of a first attempt of quantification [4], [5], [6], [7]. Nevertheless, since the work has been performed for R&D purposes and not with any implementation objectives, the results shall not be taken as definitive ones. This project was carried out by a consortium of organisations, composed of Sofréavia, NLR, and the EUROCONTROL Experimental Centre (EEC) and with the participation of CENA and the University of Glasgow. Methodologies and scope In the prospect of investigating the airborne separation minima, the project adopted a stepwise approach using recognised methodologies, tailored to meet the objectives of the CARE-ASAS Activity 3: ASM project. The first two steps, i.e. Operational Service and Environment Description (OSED) and Operational Hazards Analysis (OHA) are derived from the RTCA SC-189/EUROCAE WG-53 guidance. For the other steps, i.e. the likelihood of occurrence for hazards and consequence modelling, the risk estimation and evaluation and an ICAO Target Level of Safety, the TOPAZ methodology was used. Two ASAS applications, i.e. the Time-Based Sequencing (TBS) application and the Autonomous Aircraft (AA) application, were selected as case studies in the project. These ASAS applications, both challenging in terms of safety, were considered as complementary since they allowed to identify safety influencing factors that could more generally apply to two groups of ASAS applications: Airborne Self-Separation in FFAS and Co-operative Separation in MAS. OSED/OHA of the two selected ASAS application During the project, a customised OSED template and a tailor-made OHA method have been defined to meet the CARE/ASAS Activity 3: ASM objectives. These methods have evolved during the project based on the experience gained through the assessment of each ASAS application. The OSED objective was to obtain the relevant information for the safety assessment. The OHA allowed for the identification of operational hazards, their severity assignment and determination of safety influencing factors from a qualitative perspective. At the same time, a list of safety assumptions, requirements and recommendations used to mitigate the operational consequences of these hazards was identified. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 2

15 Although based on different operational and technical characteristics, some commonalties have emerged from the safety influencing factors identified through the OHA of the two selected ASAS applications. Among the common operational safety influencing factors, both ASAS applications rely on clear and unambiguous procedures, and the protagonists have to be qualified and well trained. Regarding the system safety influencing factors, the correct and safe execution of both ASAS applications is supported by on-board assistance that computes and provides the relevant information to the flight crew. This on-board assistance and the CNS capabilities (e.g. ADS-B, TIS-B) will have to respect minimum performances in order to reduce the likelihood of occurrence of system-related hazards to an acceptable level. TOPAZ based accident risk assessment for autonomous aircraft The next step was to exploit the TOPAZ risk assessment methodology to provide insight into estimating safe separation criteria for ASAS-based operational concepts, and in particular for the Autonomous Aircraft application. The influencing factors and operational hazards identified in the associated OHA have been taken into account. First, a hazard brainstorming has been organised to identify additional hazards that are due to the mitigation means identified in the autonomous operations OHA. The subsequent task was to develop a mathematical model for the ASAS operation considered (by re-using a model previously developed within TOPAZ for a similar ASAS-based operation) and perform accident risk assessment for this mathematical model. The accident risk-spacing curve that was based on this accident risk model was used as a reference curve. The next stage was to identify and assess the differences between the mathematical accident risk model and the Autonomous Aircraft application considered in the ASM study. These differences were identified and assessed on risk bias and uncertainty, and the modelbased reference curve was compensated for this risk bias and uncertainty. These assessments resulted in a first estimation of safe separation criteria for the Autonomous Aircraft operational concept. Suggestions for future work In the prospect of determining airborne separation minima, and on the basis of the overall work performed during the ASM study, some items appeared to particularly deserve further investigations. These include: a more in-depth analysis of both operational and technical factors that may influence the airborne separation minima; an improvement of the mathematical models for risk assessment of ASAS applications; a more integrated and iterative approach allowing for the assessment of operational hazards, the allocation of safety operational requirements and the determination of safe airborne separation minima and finally, to support the previous approach, the selection of the most appropriate methods and tools. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 3

16 Other related work in Europe Some Operational Service and Environment Descriptions and Operational Hazard Assessments of ASAS applications, also based on the RTCA/EUROCAE guidelines, have already been conducted within the scope of the NUP (NEAN Update Programme) of the European Commission [10], [11], [12], [13]. More recently, a preliminary OHA of ASAS Case Studies inspired from NUP have also been developed by the EUROCONTROL ADS Programme [14]. Operational Hazard Assessment of other ASAS applications should be performed within the European MFF (Mediterranean Free-Flight) program. Although based on the same methodology, the level of detail of the associated OSEDs, and consequently the OHAs, vary depending on the project scope and objectives. Related TOPAZ work on ASAS includes the projects EMERTA [18], Initial Free Flight [19] and Extended Free Flight [16], in which different ASAS-based operational concepts have been studied on safety-separation. In addition, reference [17] gives an overview of NLR free flight projects FAA/Eurocontrol Action Plan 1 coordination The FAA/EUROCONTROL R&D committee Action Plan 1 has been tasked to review the safety assessment methodologies under development in USA, Europe and ICAO, and find and agree on a common approach to safety assessment methodology for ASAS applications. In that context, the CARE-ASAS Activity 3: ASM study, as well as the lessons learnt from that work, are expected to provide relevant information on the use of the RTCA/EUROCAE OSA methodology in general, the OSED/OHA steps in particular, and on the TOPAZ methodology Document overview This first section describes the purpose of the document and presents the scope of the CARE-ASAS Activity 3: ASM Extension study. Section 2 analyses the OSA methodology (that include the OHA step), initially developed for D/L applications, for Airborne Surveillance and Separation applications, in general. Section 3 more specifically and thoroughly analyses the OSED/OHA steps for the identification of safety influencing factors that may impact the airborne separation minima. Based on the two OSED/OHA conducted during the CARE-ASAS ASM study, this analysis of the work done highlights the characteristics of the tailored OSED/OHA methods used and discusses their benefits and drawbacks in performing qualitative operational hazard assessment of ASAS applications. Section 4 analyses the contribution of the hazard identification brainstorm performed within of the CARE-ASAS Activity 3: ASM study of the Autonomous Aircraft application. This analysis is particularly focused on the overlap and differences with the hazards identified during the OHA, those identified through the hazard brainstorm and those already available in the TOPAZ database. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 4

17 Section 5 evaluates the possible complementary contribution of hazard identification brainstorm to the OSA methodology, and more generally discusses an overall framework relying on these two methodologies, allowing for the assessment of operational hazards and allocation of safety requirements. Finally, sections 6 and 7 provide conclusions and recommendations relative to safety assessment of ASAS applications based on the experience gained within the CARE-ASAS Activity 3: ASM study. The annexes provide additional information about the background of the study: ANNEX A: provides an overview of the RTCA/EUROCAE Operational Safety Assessment methodology; ANNEX B: describe the specific use of the OSA guidelines in the CARE-ASAS Activity 3: ASM study; CARE/ASAS/Sofréavia/ Version September 17, 2002 page 5

18 2. ANALYSIS OF OSA METHODOLOGY FOR ASAS APPLICATIONS 2.1. The OSA methodology The initial purpose of the RTCA SC189/EUROCAE WG53 Operational Safety Assessment (OSA) methodology is to identify safety requirements and allocate them between the various ATM segments of Data-Link applications (Cf. ANNEX A:). In the context of the OSA methodology, the OSED objective is to obtain the relevant information for the safety assessment of the considered CNS/ATM system.. In a wider scope, as considered in the RTCA/EUROCAE guidance, the OSED is also used as a basis for assessing and establishing the performances and interoperability requirements. The purpose of the OHA step is to develop an end-to-end qualitative assessment of potential operational hazards. The next step is the establishment and Allocation of Safety Objectives and Requirements (ASOR) to stakeholders and air/ats segments. The OHA and ASOR constitute two interrelated processes, which development may be iterative and dependent on the degree to which the operational concept has been developed. Operational Service Environment Definition OSED Operational Safety Assessment OSA Operational Hazard Analysis OHA Allocation of Safety Objectives and Requirements ASOR ATS Segment Airborne Segment Figure 1: RTCA/EUROCAE OSA methodology overview The OHA is a qualitative assessment of the operational hazards associated with the OSED. For the OHA, operational functions are examined to identify and classify hazards that could adversely affect those functions. Hazards are classified according to a standardised classification scheme based on hazard severity and taking into account human factors. Overall safety objectives are assigned to the identified hazards according to a risk classification matrix: the most severe the hazards are, the less frequently they are tolerated. Based on the OHA results, the ASOR allocates safety objectives to organisations, develops and validates risk mitigation strategies that are shared by multiple organisations, and allocates safety requirements to those organisations. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 6

19 2.2. Applicability of the OSA methodology to ASAS applications Based on the ASM study experience, this section discusses the use of the OSA methodology for ASAS applications. The main items developed are the followings: Characteristics of ASAS applications; Type of safety assessment required for ASAS applications; and Relevance of the OSA guidelines for ASAS applications Characteristics of ASAS applications Even if already being used for the assessment of ASAS applications in various context (e.g. European and US programs like NUP II, MFF or Safe-flight 21), one should remind that the OSA methodology was initially developed for assessing ATS services based on data communications. Airborne surveillance and separation requirements In general terms, ASAS applications can be considered as more challenging than most of D/L applications since the scope of the safety assessment has to address not only the air/ground communications requirements, but also the airborne surveillance and separation capabilities requirements. From that perspective, some similarity can be found between the assessment of ASAS applications and ADS services supported by data communications. However, ASAS applications are raising rather new safety issues, since they relate to the procedural use by flight crew of airborne surveillance data to support separation between aircraft (which is a new paradigm for ATM). Since the separation between aircraft plays a major role in the safety of the ATM system, both the human and technical components of ASAS applications would probably have to meet minimum levels of performance to achieve an agreed target level of safety. These safety requirements should support the definition of Minimum Aviation System Performance Standards (MASPS) for ASAS systems, including minimum performances for the separation functions on-board. More basically, these safety requirements may have an impact of the Required Surveillance Performances (RSP) expected from the ASAS system, as well as (typically in case of an ASAS based on ADS-B), on the Required Navigation Performances (RNP) of the aircraft involved and the Required Communication Performances (RCP) between the various segments involved. Air/ground shared risk mitigation strategy The different levels of flight deck implication in the risk mitigation strategy of ASAS applications is illustrated here after using the following high level safety model for assessing safety of ATM operational improvements developed by EUROCONTROL SPF (Strategic Performance Framework) Safety Group [15]: CARE/ASAS/Sofréavia/ Version September 17, 2002 page 7

20 RESOLUTION INCIDENT RECOVERY CRITICAL EVENT Air/ground comms, Detection/ correction means INCIDENT Safety nets See& Avoid ACCIDENT ATC/flight deck procedures Chance Figure 2: High Level Conceptual Safety Model [15] Typically, different sharing of risk mitigations with respect to aircraft separation can be identified depending on the ASAS application category [8]. Essential elements of that risk mitigation strategy for ASAS applications are the requirements on the separation criteria applied by flight crew, and the capabilities of either the ATS or the airborne segments to detect and solve critical events related to aircraft separation. CRITICAL EVENT FOUR ASAS APPLICATION CATEGORIES Enhanced Visual separation Hazards Airborne Spacing Hazards Airborne Separation Hazards Airborne Selfseparation Hazards RESOLUTION (Primarily) By flight deck (Primarily) By ATC (Primarily) By flight deck (Primarily) By flight deck Loss of Visual separation Loss of ATC separation Loss of Airborne separation Loss of Airborne separation INCIDENTS Figure 3: Various shared risk mitigation strategies of ASAS applications This figure is only intended to illustrate the main differences between the various ASAS application categories, and not to be exhaustive. In particular, the resolution of critical events occurring during an ASAS application, even if mainly under the responsibility of either ATC or the flight deck, may involve both air and ground mitigation measures. Incidents related to airborne separation standards Another major characteristic of ASAS applications is the possible change in the evaluation of critical event consequences. Indeed, for ASAS applications that will require the flight crew to comply with airborne separation standards, infringements of these new standards will constitute new operational incidents related to the loss of separation. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 8

21 Which safety assessment for ASAS applications The safety of an ASAS application should be assessed from the early stages of development. And, this should be an iterative process in which the granularity of the assessment is increased as the operational, technical and environmental characteristics of the ASAS application are being defined. When assessing the safety of an ASAS application, it is essential to first define the objectives of the safety assessment to be performed. In particular, depending on the development stage of the ASAS application, focus may be put in one of the following objectives (although not exclusive one from each other): Development of safe operational procedures for the use of ASAS, including safe airborne separation criteria 1 applicable during these procedures; Allocation of safety objectives and requirements to all segments involved in the ASAS application (as advocated by the OSA methodology); Assessment of the impact on the level of safety (i.e. no adverse effect on safety or increase in safety) of the ASAS application. Depending on the objectives of the safety assessment, different methods of safety assessment may be used. For instance, different methods for assessing the safety of complex systems, such as Hazard Identification and Fault/Event Trees, are already widely used in the assessment of changes to ATM systems. In other respects, separation modelling and risk collision assessment methods are being used within ICAO to establish separation standards in ATM. In the perspective of developing safe ASAS applications, the OSA methodology should provide an adequate framework allowing for critical events (i.e. operational hazards according to the RTCA/EUROCAE terminology) identification and assessment through the OHA step, and a more in-depth analysis of these critical events generation through the ASOR step. Nevertheless, for those ASAS applications that require the establishment of airborne separation minima, relationship between the use of the OSA methodology (to allocate safety objectives and requirements) and of collision risk modelling and assessment methods will have to be put in place. Indeed, a combination of both methodologies would provide an overall framework for the establishment of airborne separation standards and associated air/ground safety requirements. In addition, for ASAS applications claiming for additional safety benefits, specific methods may have to be used allowing for relative assessment of safety compared to an existing one. 1 Examples of such airborne separation criteria in the Time-Based Sequencing application studied in the ASM study include: the airborne separation minima the flight crew have to comply with during the procedure (and which depend on their CNS/ASAS performances), as well as, the time separation value requested by ATC (which is not necessarily compatible with the ATC radar separation minima, but which has to be compatible with the airborne separation minima). CARE/ASAS/Sofréavia/ Version September 17, 2002 page 9

22 RTCA/EUROCAE OSA guidelines applied to ASAS applications As an R&D contribution in early stages of ASAS applications development, the purpose of the CARE-ASAS Activity 3: ASM study was to identify safety-influencing factors that may have an impact on the airborne separation minima. In that perspective, it was planned to investigate the use of the OSA guidelines, at least the initial steps of OSED and qualitative OHA on two selected ASAS applications. These OHA consisted in the identification of operational hazards, their severity assignment and determination of safety influencing factors through the analysis of operational and functional failure scenarios. Although this specific approach were outside the defined scope of the RTCA/EUROCAE methodology [1], some general conclusions can be stretched out with regard to the applicability of the OSED and OSA guidelines to ASAS applications. Operational Service and Environment Definition In the perspective of performing an end-to-end safety assessment of ASAS applications, the OSED guidelines (Cf. Annex C of reference [1]) were found particularly relevant to define the scope and main characteristics of the applications to be assessed. Nevertheless, some amendments would be required to the better address the characteristics and safety issues of ASAS applications. In particular, the guidance material should be amended to allow the description of not only D/L communication characteristics, but also those related to airborne surveillance and separation capabilities, and to the operational use of these ASAS functions. The approach used in the ASM study to describe ASAS applications and their operational environment is further discussed in section 3. The OSA guidance material (Cf. Annex E of reference [1]) on how to define safety requirements was also found particularly relevant in the context of ASAS applications development. Operational Hazard Analysis In a first stage, the OHA should allow to enumerate operational hazard events that could pertain to an ASAS application, to identify the mitigation measures that support safety in case of these events and to assess the resulting operational consequences. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 10

23 RISK MITIGATION STRATEGY HAZARD PREVENTION Airspace & procedures design Air/ground system perfos. Operational Hazard HAZARD RESOLUTION Air/ground comms, Detection/ correction means ATC/flight deck procedures INCIDENT Hazardous Major Minor INCIDENT RECOVERY Safety nets See& Avoid Chance ACCIDENT Catastrophic Avoidance and mitigation factors Undesired Operational Consequences Figure 4: High Level Conceptual Safety Model applied to the OHA step The Hazard Classification Matrix (HCM) developed by RTCA/EUROCAE already provides a basis for such assessment of operational hazards. Nevertheless, proper definition of the safety margins (as referred in the HCM) associated with ASAS operations should be developed to allow the adequate evaluation of the effects of system failures and procedural errors on ASAS operations. Indeed, the HCM was found particularly oriented towards the assessment of D/L applications in which the separation is provided by ATS. Since the operational use of ASAS introduces a new paradigm for separation provision in ATM, appropriate and agreed classification of operational incidents related to ASAS operations will have to be defined. The approach followed in the ASM study to identify and qualitatively assess operational hazards related to the selected ASAS applications is further discussed in section 3. In addition, the relationship with the results of the hazard brainstorm conducted as part of the TOPAZ study of airborne separation criteria is discussed in section 4. Allocation of Safety Objectives and Requirements Even if the ASOR step was not performed as such during the ASM study, the associated RTCA/EUROCAE guidelines were also considered of particular interest in the perspective of identifying, through quite in-depth OHA (cf. section 3), both technical and operational safety influencing factors relevant to ASAS applications. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 11

24 Nevertheless, it appears that appropriate ASOR guidance material should be developed to help people taking into account the essential role of the airborne separation criteria in the safety of ASAS applications. Indeed, the major issue when allocating the safety objectives and requirements for ASAS applications would probably consist in finding an acceptable combination of the safety performance requirements put on the air/ground segments and the airborne separation criteria requirements. In that perspective, different approaches can be envisaged: The airborne separation criteria are first established taking into account operational considerations (for instance, airspace capacity constraints). And, safety performance requirements for the ASAS procedures and systems are derived so as to achieve the safety objectives derived from the OHA. The safety performances that can be reasonably expected from the air/ground segments during ASAS applications are established first taking into account existing limitations from technologies or procedures. Then, the airborne separation criteria are established so as to achieve the safety objectives derived from the OHA, while taking into account the level of performances required from ASAS procedures and systems. Actually, both approaches would probably have to be investigated depending on the ASAS applications, and potentially a combination of these approaches in an iterative process for some ASAS applications. SAFETY OBJECTIVES (in terms of accidents, hazardous, major and minor incidents) AIRBORNE SEPARATION CRITERIA (used either by ATC or the Flight Deck) SAFETY & PERFORMANCE REQUIREMENTS (in terms of RNP, RCP, RSP, ) Figure 5: Interaction between airborne separation criteria and safety objectives and requirements in the ASOR step 2.3. Lessons learnt about RTCA/EUROCAE OSA applied to ASAS applications This sub-section presents the main lessons learnt from the use of the OSA guidelines within the CARE-ASAS Activity 3: ASM study, and briefly discusses possible amendments of the RTCA/EUROCAE guidelines to support the future Operational Safety Assessment of ASAS applications. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 12

25 In the perspective of performing an end-to-end safety assessment, the RTCA/EUROCAE OSA guidelines were found particularly relevant for ASAS applications, which claim for a greater involvement of the flight deck in air traffic services and separation provision. Nevertheless, some amendments would be required to the better address the characteristics and safety issues of these ASAS applications. Operational Service and Environment Definition First, the scope of the safety assessment of ASAS applications has to address not only the air/ground communications requirements, but also the airborne surveillance and separation capabilities requirements. Therefore, the OSED guidance material should be amended to allow the adequate description of the airborne CNS and ASAS characteristics, the operational use of these ASAS functions, as well as the expected performances of both the human and technical components of ASAS applications. Such description of the operating method with ASAS and their functional characteristics should support the analysis of operational hazards related to ASAS-based aircraft separation, as well as the definition of safety requirements related to surveillance and separation functions on-board. Operational Hazard Analysis Since the operational use of ASAS introduces a new paradigm for separation provision in ATM, appropriate and agreed classification of operational incidents related to ASAS operations will have to be defined. As a consequence, the Hazard Classification Matrix (HCM) developed by RTCA/EUROCAE should be amended with proper definition of the safety margins associated with ASAS operations. Indeed, the HCM was found particularly oriented towards the assessment of D/L applications in which the separation is provided by ATS. Allocation of Safety Objectives and Requirements Since the separation between aircraft plays a major role in the safety of the ATM system, both the human and technical components of ASAS applications would probably have to meet minimum levels of performance to achieve an agreed target level of safety. Another major issue when allocating the safety objectives and requirements for ASAS applications would probably consist in finding the best compromise between the safety performance requirements put on the air/ground segments and the airborne separation criteria requirements. Therefore, appropriate ASOR guidance material should be developed to support taking into account the essential role of the airborne separation criteria in the safety of ASAS applications. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 13

26 3. ANALYSIS OF OSED/OHA FOR IDENTIFYING SAFETY INFLUENCING FACTORS This section aims developing lessons learnt from the OSED/OHA activities performed during the ASM study, in the perspective of identifying safety influencing factors. As such, the section is organised as follows: Section 3.1 briefly describes the OSED/OHA activities performed in the ASM project and their rational. More detailed description of the OSED template and customised OHA method used is given is ANNEX B:. Section 3.2 provides a critical analysis of the work done, and develops lessons learnt from the ASM study about these OSED/OHA steps, though examples derived from either of the two ASAS applications studied within the ASM study; Section 3.3 summarises the lessons learned from the OSED/OHA activities, and briefly discusses possible improvements of the approach, in the prospect of further investigating safety influencing factors related to ASAS applications using the OSED/OHA framework What has been done in the ASM study (and why) Within the scope of the CARE-ASAS Activity 3: ASM project, Operational Service and Environment Definition and preliminary Operational Hazard Assessment of the two selected ASAS applications (i.e., Autonomous Aircraft in FFAS, and Time-Based Sequencing in MAS) have been performed. Rationale for the OSED/OHA steps Within the ASM study, the rationale for conducting OHA of the two selected ASAS applications, using the RTAC/EUROCAE methodology, was to identify safetyinfluencing factors that may have an impact on the airborne separation minima. Although the ASOR step was not intended to be developed in the ASM study, the need for identification of both operational and technical safety influencing factors was clearly identified during the study. Indeed, the elements potentially affecting the separation minima applicable during ASAS operations include the operational procedures themselves, the Communication, Navigation, Surveillance and ASAS separation capabilities on board, as well as the human capabilities to conduct such operations. For that purpose, customised OSED template and tailored-made OHA method have been defined, which have evolved during the project (Cf. ANNEX B:). This framework also permitted to get common understanding of OHA process to be performed between the various participants of the ASM project. OSED approach overview Originally oriented in a data link and communication aspect, the RTCA/EUROCAE OSED template used was adapted to cope with the specificity of ASAS applications studied within the CARE-ASAS ASM study. The objective of the OSED was to provide an operational perspective of these ASAS applications in a defined anticipated environment of use: CARE/ASAS/Sofréavia/ Version September 17, 2002 page 14

27 Autonomous Aircraft operations in a Free Flight Airspace, where the operations give to flight crew the complete responsibility of the flight, and Time Based Sequencing application, in a Managed Airspace, where the operations could be defined as an application based on the extended visual clearance procedure in current control practice, but with specific procedures and using suitable instruments to comply with airborne separation minima in IMC. One of the challenges of the Time Based Sequencing OSED was to describe the operations with a level of details allowing the extrapolation of the results to a general context of co-operative ASAS operations. For both ASAS applications, operational environment and procedures were described, together with the different functional characteristics of the system to properly perform the procedures (see Figure 6). The environment description aimed at being generic as far as possible to avoid the specificity of an airspace. However to be representative, it was derived from a real airspace. Once the broad characteristics of the environment and ASAS operations were jointly defined, the capture of the detailed information required to fulfil the OSED template [2] was performed in parallel by two members of the ASM project. Then, the functional characteristics of both CNS and ASAS systems supporting the operations were described. Operational Service Environment Definition OSED Environment characteristics description Operations with (and without) ASAS Functional characteristics description Operational Hazard Assessment OHA Figure 6: OSED step conducted in the ASM study Whereas the Autonomous Aircraft OSED provided a quite detailed description of the functional characteristics of the ASAS system, the description of the Time-Based Sequencing OSED identified more precisely the operations (including communications) between controllers and pilots. This more operational-oriented approach advocated limited airborne and ground system/support tool assumptions in the OSED. The purpose was to facilitate the identification of operational safety-influencing factors, in addition to environmental and technical factors. CARE/ASAS/Sofréavia/ Version September 17, 2002 page 15