Measuring En Route Separation Assurance Performance. Ella Pinska, Brian Hickling

Transcription

1 EUROCONTROL Safety R&D Seminar 2009, Munich, Germany Measuring En Route Separation Assurance Performance Ella Pinska, Brian Hickling Eurocontrol Experimental Centre Bretigny sur Orge, France 1 Introduction According to SESAR Performance Target, in 2020 the traffic capacity is expected to accommodate a 1.7 fold increase in demand compared to In order to maintain the current safety target level of 1.55 x10-8 accidents per flying hour set by ESARR 4 in predicted traffic growth, safety has to be improved substantially. The current high level of safety observed in European ATM is a result of various elements in the air traffic management process such as strategic conflict management, separation provision performed by tactical controllers and collision avoidance. The current work focuses on the investigation of separation provision performed by the tactical controller. To maintain the safety target level in predicted traffic growth the upstream safety barriers will have to be reinforced. Also their usability requires further investigation in order to define if there is any missing defence (barrier), or an improvement needed at any stage of planning or executing the control. The analysis involves a closer look at the risk bearing events in terms of time to potential losses of separation as well as severity of losses of separation in en-route air space. 1.1 Separation assurance The ICAO Document Procedures for Air Traffic Management [6] provides the separation guidelines considering vertical, lateral and longitudinal separation. The documents states: the radar separation minimum shall be prescribed by the appropriate ATS authority according to the capability of the particular system to accurately identify the aircraft position in relation to the centre of the radar position symbol (ICAO Doc 4444 Part VI, par ). The minimum separation is therefore to be related to the accuracy to which the radar system is capable of representing the aircraft positions. EUROCONTROL produced a guidelines for the application a 5NM separation minima for en-route airspace in ECAC. However, neither the procedures nor guidelines provide a strategy as how to obtain a desirable traffic configuration. The strategy obviously depends on formal aspects such as: ATC restrictions, aircraft type, aircraft equipment, requested cruise level and distance to destination (see Figure 1). However, there are additional factors taken into account such as intentions of flight crew, or the probability that separation could be violated due to unexpected events. Such factors are difficult to characterise, since they are based on controller s trust in crew performance or controllers expectations. Other considerations are the location of the conflict in the sector and the geometry of conflicts, as these define the spatial geometry of a conflict. Lamoureux (1999) [11] 1

2 found that controllers workload is closely dependant on the geometry of the conflict. In this study, the following situations were found to be associated with high workload: where one aircraft is level and the other is climbing or descending where the aircraft are on converging tracks, but not head on when the aircraft are climbing or descending and turning at the same time The combination of influences of these various factors is determinant in the strategies a controller employs to resolve conflicts. Seriousness: Time left before LoS Point of closest approach Conflict Geometry: Horizontal Vertical Formal aspects: Procedures, ATC restrictions Adjacent sector procedures SEPARATION ASSURANCE Unmeasurable factors: Certainty, Intention Trust Position in a sector: Distance to sector boundaries Aircraft related: aircraft type, aircraft performance and equipment Interim conditions: Weather Sector load Figure 1 Factors affecting controllers' conflict resolution strategies (based on Kirwan, B. 2002) An additional factor affecting conflict resolution performance is that Tactical controllers will apply their own additional safety margins to the standard separation these are variable and are intended to provide space beyond the separation minima to allow some spare time to react in case of unexpected events. Reynolds & Hansman, (2001) isolated four components of separation assurance budget (see figure 2): protection zone, surveillance uncertainty procedural safety buffer personal safety buffer. The Protection zone indicates the sphere around aircraft, which should never be penetrated by another object if aircraft safety is to be assured. This is a physical area occupied by the aircraft itself. The Surveillance uncertainty describes the zone of uncertainty in the aircraft state information due to limited accuracy of surveillance systems (radar sensors, refresh rates, latency of data processing and the result of integration of multiple sources of data into a single state vector multiradar tracking). The surveillance uncertainty is also affected by external factors such as weather. The zone this represents is that where the aircraft currently can be located. Typical radar accuracy is around 1/16NM 2

3 range but more importantly (since it is an increasing size with distance from the radar) about 0.25 degrees of measured bearing from each radar. The Procedural Safety Buffer represents the formal, proceduralized buffer, required for separation criteria within operational system. This is also has to incorporate a time buffer to carry out the recovery actions in potentially hazardous situations. This has been determined for different airspace by historical means (aerial visibility, radar performance, aircraft performance, air-ground communications and controller acceptability). Finally the personal safety buffer describes the time or space over the separation minima applied by controllers to assure that separation standard is not violated. In the study of Ballin and Erzberger (1996), who analysed the aircraft path retracted from radar tracks, the safety buffer for final approach in Dallas/Fort Worth International airport was greater than 1 nautical mile. Less personal buffer space leaves less time, which might have a negative impact on safety. Although safety buffers are a significant component of separation, no adequate and easily applicable methodology for quantitative measurement of the buffers has been identified. The evaluations of new tools that might have an impact on safety buffer, is traditionally performed via real time simulation. Therefore, there is a need to create a means for measurement the safety buffer possible to apply in real time simulations, or real data analysis. Personal safety Protection Zone MINIMUM SEPARATION REQUIREMNT Procedural safety Surveillance Uncertainty Figure 2 Separation Assurance Budget Components (adopted from Reynolds & Hansman,

4 2 Research approach Previous judgements of the separation performance were derived from the failures in the maintaining the separation such as STCA alarms or incident data. These fragmentary and selective data provide a limited view of controllers behaviour. We propose a new, success approach based on both successful separation and the failures providing a more balanced, holistic understanding of controllers performance. The success approach is the measurement of how separation assurance is being achieved during successful operations i.e. the majority of operations. We try to extract from current working methods the best working practices that assure a successful operation. In a simulation environment which has limited traffic samples and relatively small durations of experimentation failure events such as STCA are rare and can only be produced in useful quantities by creating an operational environment that is so far from the nominal one as to force higher levels of error. These may be far from the real operating environments for which the tool, procedure or airspace configurations are intended and so are not reliable measures. Therefore, a Separation Performance Tool (SPT) is proposed, which predicts the aircraft trajectory and dynamically defines potential losses of separation. The indicators of separation performance are: Interventions of tactical controllers to resolve predicted conflict (Heading instruction, Speed change, Altitude change) Time of interventions before potential loss of separation Distance before potential loss of separation that the tactical controller intervenes Localisation of interventions points Localisation of predicted losses of separation The following table lists the SPT metrics and the areas of investigation for separation assurance. The indicators refer to detected potential losses of separation based on prediction of flight trajectories. Thus the main area of the investigation is before an infringement of separation minima, e.g. beyond five NM in case of an en-route sector. This method takes a closer look at working methods and does not rely on data from the few erroneous events. Separation assurance focuses on pairs of aircraft, but the analysis of the separation assurance performance in a sector aims to identify trends and extremes in the management of the sector. It should be noted that analysis of the intervention times helps to identify the size of the safety buffer being applied. There is no formal requirement of such buffers, so certain assumptions have been made, informed by operational expertise. The association of controller actions with a potential loss of separation has been limited as follows. An action is related to a future loss of separation if it occurs 10 minutes or less before an anticipated loss of separation. Ten nautical miles (Nm) horizontally and Ft vertically were judged as meaningful cut-off points for associating intervention actions. 4

5 Outside of the above parameters controller actions are categorised as traffic management i.e. interventions needed to ensure an efficient flow of traffic through a sector. In the following demonstration case, we have applied the values presented above, however SPT parameters can be set up to any separation value, e.g. if desired to test tools used by Planner Controller, for example, which might require a longer timescale. The measurements taken into analysis by SPT are derived from system data. The data are measured continuously with a 15-second update. The tool detects potential losses of separation using the aircraft performance models from BADA 1. The SPT tool can be applied both to live operational data and to data coming from real time simulation. Controller s interventions, in case of operational data, are retrieved from aircraft manoeuvres and verified when present with any operational controller inputs such as CFL data. For the real-time simulation data, controllers actions are additionally verified with controllers actions recorded from mouse inputs. The data obtained in this manner are therefore objective and user-independent. The data collection is completely transparent and non-intrusive for controllers assuring that the performance is free from observer effect (changing the performance of an act due to awareness of being observed). 1 BADA (Base of Aircraft Data) the data base of aircraft performance and operation models, which are suitable for trajectory prediction and calculation within ATC simulations and on-line applications. 5

6 AIRCRAFT PAIR SECTOR Enquiry Conflict resolution strategy : Which instruction was given to solve a conflict? How long in time and how far in advance in distance to potential LoS s an instruction was given? How are different instructions related to time to potential LoS? What type of instruction is given with different types of geometry of a conflict? How do changes in a sector design affect the distribution of traffic? Safety buffers, When is the conflict solved? How close are the aircraft allowed to fly? What is the severity of the potential LoS? Separation assurance What percentage of total traffic risks LoS? What are the total flight times under various risk categories? Traffic distribution What is the distribution of the demand to process the traffic in the sector (e.g. in the centre of the sector or sector s boundaries)? What is the variation in the density in various regions of the sector? Hotspot Where are the hotspots? What is an impact of modification of flight routes on hotspots? Severity of action How urgently does the controller have to intervene (in time and distance) to potential LoS? Where is the demand for the interventions? Traffic complexity What is the geometry of the traffic? What is the most common conflict geometry? Which conflict geometry contributes to actions that are more urgent? SPT Metrics Type of instruction Time of intervention Separation Interventions, Separation interventions, Location of Intervention, Location of LoS Location of LoS Controller Intervention grouped by risk categories LoS grouped by risk categories Horizontal geometry of conflict Vertical geometry of conflict 6

7 3 Demonstration case The Separation Performance Tool was applied to en-route simulation that was run in EUROCONTROL Experimental Centre in The simulation investigated the functionality of a prototype of new conflict detection tool (CDT) as a support for tactical controllers. The data presented below presents the analysis of controllers performance in baseline organisation (reflecting current working conditions) and CDT organisation, when a conflict detection tool was applied. Figure 3 Type and time of controllers intervention for baseline (a) and CDT (b) organisations. 7

8 3.1 Type of instructions Traditionally during real time simulations, an investigation on controller performance is focused on measuring the number of heading and altitude changes. The total number of interventions however, does not tell much about the strategy applied to solve a conflict e.g. when a controller is close to losing separation, what strategy (s)he uses, etc. In contrast, confronting the type of interventions and the time to potential loss of separation allows us to interpret the conflict solving strategy applied by controllers.in the example presented in Figure 3 we see that for the categories more than 7.5 min or less then 0.5 minutes before potential LoS, there were more altitude changes given in CDT (Fig 3b) than in baseline (Fig 3a). In addition, in CDT organization there was less heading instructions (Turn) than in baseline for the same category less then 0.5 min. Thus, the controllers when using CDT tool, preferred to give altitude changes either much in advance or in contrast, very close to the potential loss of separation. In the case of en-route sectors, the altitude changes are generally more efficient than headings (turns). Re-routing extends the flight time, and demands more attention of a controller who has to monitor an aircraft and bring it back to the flight route. Altitude changes solve the conflict in one intervention. Application of altitude changes instead of headings contains the controller workload, and consequently maintains capacity in the sector. 3.2 Intervention Separation Intervention Separation indicator represents the severity of the traffic situation based on plotting each tactical intervention in terms of time to loss of separation and predicted closest approach distance. This indicator compares the number of interventions taken in different time categories in both organisations. According to Figure 4, the controllers intervened in advance (between 5 and 10 min) in the baseline organisation whereas in CDT there were more interventions closer to potential loss of separation (less then 1 min). The controllers having the automated support for conflict detection reacted later to solve potential losses of separation. 8

9 Intervention separation (predicted LoS less then 5 NM) 25 Number of Interventions BASELINE CDT 0 <= Time before predicted LoS Figure 4 Intervention separation to solve predicted LoS less then 5 nm for baseline and CDT organisations These results can be further analysed with the additional factor of distance to potential loss of separation. Tables 1 and 2 present the controllers interventions grouped in risk matrices depending on time and distance to predicted loss of separation. Table 1 Time of intervention for baseline organisation Baseline <=0.5min min min min min min min min <=2,5Nm ,5-4Nm Nm Nm ,5-10Nm >10 Nm Table 2 Time of interventions CDT organisations CDT <=0.5min min min min min min min min <=2,5Nm ,5-4Nm Nm Nm ,5-10Nm >10 Nm

10 Considering the action taken to solve the most severe situation (less then 2.5 NM and less then 3 minutes before potential LoS) there were only 2 interventions in the baseline against 5 interventions in CDT. The above results suggest the following conclusion about the safety buffers applied by controllers. In the baseline organisation, wherever the controllers reacted to predicted loss of separations, then in the CDT organisation the actions were taken later. In CDT, the controllers were acting closer to the limit of the separation thus there were more aircraft flying closer to each other than in baseline conditions. Although the controllers kept separation minima in both organisations, in CDT, the controllers intervene later. These results cannot be captured by any other measurements. In a traditional simulation, only the STCA alarms are measured. However, during real time simulations, the STCA alarms occur rarely, not permitting a statistically significant comparison between the organisations. Using the SPT tool, we can fully understand the impact of new tools on the safety buffers. 3.3 Traffic distribution in sectors The SPT offers the possibility to represent the geographical distribution of controller interventions and predicted locations for losses of separation. Both interventions and losses of separation (LoS) can be grouped by each instruction type, distance and time to potential LoS.This kind of representation in an immediate way visualises the main traffic flows and hotspots occurring in a sector. The representation of geographical locations of the interventions could be very useful for interpreting the sector complexity and sectors hotspots. This indicator serves to test the impact of route changes on risk export (i.g. exporting risk from one sector o airspace to another) Intervention Locations Figure 5 represents the geographical distribution of controllers interventions in simulated sectors. The representation takes into account the type of instruction (Turn, Speed, and Altitude) as well as the time to potential loss of the separation. The representation of geographical locations of the interventions could be very useful for interpreting the sector complexity. The actions are taken to prevent a potential conflict occurring in another part of the sector. The visualisation can be used to relate the interventions with problems on adjacent sectors. The geographical distribution of controller actions also provides information of sectors hotspots. This indicator again serves to test the impact of route changes on risk export. The graphs can show the geographical area where there is a high concentration of actions. Traditionally during the real time simulation, the measurement of controllers workload is interpreted as radio time and number of given instructions. However, these data indicate the total numbers and do not provide any information about situations where the workload is the highest. The visualisation of controller s interventions shows precisely where in geographical terms controllers have to react to solve potential conflicts. It can help to understand if the main problems are coordination between sectors, or within sectors. Representation of interventions according to the risk categories (the time against the distance) shows how urgent the actions were, and what was the severity of a situation requiring an intervention to solve it. 10

11 Figure 5 Location of interventions according to type and time of instructions for baseline (top) and CDT (bottom). 11

12 3.3.2 Location of predicted losses of separation The Separation Performance Tool offers the possibility to represent the geographical distribution of predicted locations for losses of separation. Interventions to resolve these losses of separation can be grouped by each instruction type, distance and time to potential LoS (Figure 6). This kind of representation gives a rapid way of visualising the main traffic flows and hotspots occurring in a sector. The data represented according to risk categories shows as well the severity of potential losses of separation. The tool provides a possibility to extract the number of LoS per sector according to type of intervention of severity of predicted LoS. In real time simulations, the traffic complexity is judged using STCA alarms and controller feedback about the traffic structure. SPT visualisation demonstrates precisely an area with high risk. Such a visualisation can be also useful in experimental validation to verify the complexity of a traffic sample in a preparation phase of real time simulations. The visualisation of severity of losses of separation can be used for training purposes for air traffic controllers. The various strategies lead to results of different criticality. The SPT can be used to display how the strategy affects the criticality of losses of separation. The SPT also provides the possibility to analyse the actions taken in order to solve the different types of conflicts (in horizontal or vertical space). The following section presents the possible conflict configuration and the analysis of interventions taken to solve them. 12

13 Figure 6 Location of predicted losses of separation or baseline (a) and CDT (b) Traffic Complexity Evaluations Separation Performance Tool enables visualisation of the horizontal geometry of the two aircraft at the time of predicted loss of separation. The possible situations for horizontal losses are: - Acute - Crossing - Overtaking - Head-on - Obtuse Using the SPT, we can compare the number and what kind of conflicts we would create with new traffic routes. The vertical geometry of potential loss of separation can also be presented in the following categories: - Climbing /Climbing - Climbing /Descending - Climbing /Levelled - Levelled /Descending - Levelled /Levelled - Descending /Descending The tool offers the possibility to filter out all listed conflicts. In addition, there is a possibility to filter in or out specific flights related with the occurrence of Medium Term Conflict Detection (MTCD) or STCA events. Flynn, Leleu and Zerrouki [4] investigated the traffic complexity of European and US air traffic centres. The investigation was based on the following quantitative indicators: volume of the sectors, number of IFR flights, controlled kilometres (sum of routes flown by each aircraft), total flight hours, average route length, and average 13

14 transit time. The vertical movements were classified as departing, landing or overflights. In addition, the average number of altitude changes was calculated. The assessment of workload due to separation effort was calculated on the basis of proximate pairs (along tracks, opposite direction and crossing tracks). The traffic density was calculated as a ratio between the flight hours, kilometres controlled and the volume of the sectors. Considering that the traffic is not equally distributed over the sectors, the density has to be adjusted by a concentration index calculated for a specific sector. The indicators used in the presented study had to be simplified in order to allow a global statistical comparison between various air traffic centres. However, the vertical geometry of traffic was found to be a contributor to workload of controllers (see reference [11]). Separation Performance Tool builds on this structure and permits deeper analysis of the various conflict types, showing precisely exact vertical and horizontal geometry of conflicts, located in the geographical area. Each type of conflict can be filtered out for further analysis of the strategy to solve it. Choice of strategy has an impact on controller workload and consequently determines the sector capacity. In addition, the geographical distribution of the controllers actions can identify potential hotspots and high workload areas with safety and capacity limits. The visualisation of potential conflicting situations can also serve as a support material for trainees in air traffic control field in order to identify what are the possible consequences of applying specific strategies as well as training resources for sector designers. 4 Conclusions The main advantage of using the SPT tool is the ability to identify controller actions and relate them to successful traffic management strategies including how, when and where successful resolutions were executed. These data are captured automatically and are not dependent on controller opinions or the occurrence of exceptions. The resolution strategy is characterised as the relationship between type of controller s intervention, time and distance left to potential loss of separation, geographical situation (boundaries or centre of a sector) and the geometry of conflicting traffic. The quality (usually a subjective term requiring input from subject matter expertise) of applied resolution can be expressed on a general level, as the total number of conflicts that occurred, when the conflicts were solved and how critical they were. These results allow us to conclude on the robustness of separation provision in a sector. The SPT gives an indication of the safety buffers the controllers apply around the aircraft and how these safety buffers can be affected by the usability of automated support tools or increased traffic load. In addition, using SPT offers the possibility to localise the conflicts, and to identify what type of intervention the controller performed to solve them. The tool provides a means to focus on a specific pair of aircraft, or a specific type of conflict e.g. associated with a STCA alarm, which can be useful for analysis in particular situation. The originality of the Separation Performance Tool relies on the fact that it provides a more in-depth picture of controller separation performance. The results show how controllers act continuously, not only based on partial data of losses of separation or STCA alarms. Therefore it can be use to understand how a new tool changes controllers behaviour and how they use a new tool. It can also identify the area where training of new working practices may be required. SPT may also highlight interaction between controllers display functions and their resolution strategies. 14

15 The SPT can also serve in the re-sectorisation process, to evaluate the distribution of traffic, but also to judge the flight plan complexity. In addition, quantitative metrics such as intervention separation or separation performance allow us the comparison between various air traffic control centres. The SPT provides the means for the analysis of an impact of new tools, useful for system designers and developers in air traffic control field. The output of SPT can be compared with safety monitoring the systems such as ATM Safety Monitoring Tool (ASMT). The SPT allows us to provide evidence of the impact of new tools/procedures in the success case (where separation is assured) which is complementary to the traditional study of failure cases (ASMT/incident reporting). Since incidents and accidents generally occur when a set of rare circumstances occur together then the study of failure will always be a study of when the limits are exceeded. With tools like the SPT it should be possible to recognise subtle changes in controller behaviour before it results in higher STCA/incidents thereby increasing the opportunity to improve safety proactively rather than in a reactive fashion. SPT, in further perspective, could enable the benchmarking process of controllers performance in human-in-the-loop simulations. The characterisation of baseline performance extracted from the real live data could serve as a model for further analysis of simulation effects. The model could be applied in validation processes to identify the simulation effects and its impact on controllers behaviours. Reference 1. Ballin, M. G. and Erzberger, H., "An Analysis of Landing Rates and Separations at the Dallas/Fort Worth International Airport," NASA TM , July EUROCONTROL, (2001). ESARR 4 - Risk Assessment and Mitigation in ATM, Ed Eurocontrol 1998 GUIDELINES FOR THE APPLICATION OF THE ECAC RADAR SEPARATION MINIMA ASM.ET1.ST REP Flynn, G., Leleu, C., and Zerrouki, L., (2003). Traffic Complexity Indicators and Sector Typology Analysis of US and European Centres, EEC Note 2003/20, Bretigny sur Orge, France. 5. Fucke, L., and Porras, J.F., (2008), From Tacit to Explicit: Understanding Foundations of Safe Separation Minima, in proceedings of EUROCONTROL Annual Safety R&D Seminar, Southampton, UK 6. ICAO4444. (2007). Air traffic management - procedures for air navigation services (Fifteenth Edition No. Document 4444 ATM/501). International Civil Aviation Organization. 7. Kirwan, B., (2002). Towards a controller-based conflict resolution tool a literature review. Report Nr. ASA.01.CORA.2.DEL04-A.LIT, EUROCONTROL 8. Lamoureux, T. (1999). The influence of Aircraft Proximity Data on Subjective Mental Workload of Controllers in the Air Traffic Control Task. Ergonomics, Volume 42, No. 11, (pp ). 9. Reynolds, T., and Hansman, R.J., (2001). "Analysis of Separation Minima Using a Surveillance State Vector Approach," Chapter 34 in Air Transportation Systems Engineering, Donohue, G., Zellweger, A., Rediess, H., and Pusch, C., Eds., Vol. 193 of Progress in Astronautics and Aeronautics, AIAA, Reston, VA, , SESAR Consortium, (2006). Air Transport Framework the Performance Target D2, DLM a, December