Episode3 D Simulation Report on Collaborative Airport Planning EPISODE 3. Single European Sky Implementation support through Validation

Transcription

1 EPISODE 3 Single European Sky Implementation support through Validation Document information Programme Sixth framework programme Priority 1.4 Aeronautics and Space Project title Episode3 Project N Project Coordinator EUROCONTROL Experimental Centre Deliverable Name Simulation Report on Collaborative Deliverable ID D Version 1.00 Owner Reiner Suikat DLR Contributing partners NLR, EUROCONTROL Page 1 of 105

2 - This page is intentionally blank - Page 2 of 105

3 Approval DOCUMENT CONTROL Role Organisation Name Document owner DLR Reiner Suikat Technical approver AENA Pablo Sánchez-Escalonilla Quality approver EUROCONTROL Catherine Palazo Project coordinator EUROCONTROL Philippe Leplae Version history Version Date Status Author(s) Justification - Could be a reference to a review form or a comment sheet /12/2009 Approved Reiner Suikat (DLR) Bernhard Weber (DLR) Hugo de Jonge (NLR) Approved by the Episode 3 consortium Page 3 of 105

4 TABLE OF CONTENTS EXECUTIVE SUMMARY INTRODUCTION PURPOSE OF THE DOCUMENT INTENDED AUDIENCE SCOPE AND STRUCTURE OF DOCUMENT EXPERIMENT BACKGROUND AND CONTEXT CONCEPT OVERVIEW Detailed outline of the Operational Concept of Interest OVERVIEW OF EXERCISE: GLOSSARY OF TERMS SUMMARY OF EXPERIMENT AND STRATEGY PLANNING EXPECTED EXPERIMENT OUTCOMES, OBJECTIVES AND HYPOTHESES Description of Expected Experiment Outcomes Description of Experiment Objectives and Assumptions Description of Experiment Hypotheses CHOICE OF METRICS AND MEASUREMENTS CHOICE OF METHODS AND TECHNIQUES VALIDATION SCENARIO SPECIFICATIONS Scenario Details Scenario events Tasks given to agents EXPERIMENTAL VARIABLES AND DESIGN CONDUCT OF VALIDATION EXERCISE RUNS EXPERIMENT PREPARATION EXECUTED EXPERIMENT SCHEDULE DEVIATIONS FROM THE PLANNING EXPERIMENT RESULTS MEASURED EXPERIMENT RESULTS Negotiation Duration Negotiation Process Negotiation Results Communication content Questionnaires CONFIDENCE IN EXPERIMENT RESULTS Quality of Results of Experiment Significance of Results of Experiment ANALYSIS OF EXPERIMENT OUTCOMES ANALYSIS OF OUTCOMES ON THE BASIS OF DETERMINED HYPOTHESES ANALYSIS OF CONSEQUENCES OF OUTCOMES FOR EXPERIMENT OBJECTIVES AND ASSUMPTIONS CONCLUSIONS AND RECOMMENDATIONS KEY FINDINGS ISSUES RECOMMENDATIONS CONCLUSIONS REFERENCES AND APPLICABLE DOCUMENTS ANNEX: NETWORK MANAGEMENT MODELLING IN SUPPORT OF GAMING INTRODUCTION Page 4 of 105

5 8.1.1 Purpose of the ANNEX Background Intended Audience Structure of the Annex EXERCISE SCOPE AND EXECUTION Stakeholders and their expectations Description of ATM concept being addressed Exercise objectives Choice of indicators and metrics Experimental Design for Collaborative Validation scenario Experimental Set-up to Conduct the Exercise CONDUCT OF EXPERIMENT NETWORK MANAGEMENT ANALYSIS Step-wise experiment Scenario selection Network Aggregation Assessment of DCB conditions of the Kernel Network Emulation of feed-back from hub-airport CDM to CNM RESULTS AND DISCUSSION Scenario Selection Aggregation to a simplified Kernel Network Assessment by Demand and Capacity balancing (DCB) Feeding the Hamburg Gaming exercise with feedback from CNM Impact from CDM processes at airport level ANALYSIS OF EXPERIMENT OUTCOMES Analysis of Outcomes on the Basis of Determined Hypotheses Analysis of Consequences of Outcomes for Experiment Objectives and Assumptions CONCLUSIONS AND RECOMMENDATIONS Key findings Issues Recommendations OPERATIONAL CONCEPT OPERATIONAL IMPROVEMENTS Page 5 of 105

6 LIST OF TABLES Table 1: Operational Improvement Steps Table 2: Exercise Overview Table 3. Glossary of terms Table 4: Operational Improvement Steps Table 5: The coding scheme Table 6: Metrics and data sources Table 7: Experiment Variables Table 8: Individual Negotiation Goals Table 9: Sessions Table 10: Negotiation Times Table 11: The Negotiation Processes in both Experimental Teams Table 12: Negotiation Results Table 13: Total Number and Negotiation Related Communicative Units Table 14: Coding Frequencies Table 15: Items and Means (Post-Trial Ratings) Table 16: Items and Means (Post-Experimental Ratings) Table 17: Stakeholder expectations Table 18: KPIs relevant to Network Modelling Table 19: Overview on experimental runs by CNM modelling part of experiment Table 20: The list of nodes of airports and airspace sectors of the selected aggregated network, representing the NOP Table 21: Overview of NAM-constrained waiting periods of non-aggregated Kernel Network Table 22: Overview of NAM-constrained waiting periods of the aggregated Kernel Network (52 nodes) Table 23: Results of ATFCM applied on an aggregated European Kernel Network with 439 nodes in 24 hours Table 24: Operational Improvement Steps Page 6 of 105

7 LIST OF FIGURES Figure 1: Agents working in the Airport Operations Centre Figure 2: Hamburg airport layout Figure 3: Arrivals and Departures (Cumulative Diagram) Figure 4: Powerwall design Figure 5: Experiment setup in ACCES Figure 6: Experimental Setting Figure 7: Scope and Context of operations of APOC and CNM Figure 8: General overview of the Exercise Set-up Figure 9: Overview of European traffic density in a 24-hours 2005 scenario Figure 10: Aggregated route network Figure 11: Main airports, aggregated airport nodes and out-nodes in the route network of Europe Figure 12: Overview of departure/arrival operations from/to airports in the Kernel Network of Europe Figure 13: The selected network of airspace and airports, representing the major part of the European ATM network Figure 14: Distribution of Demand/Capacity ratios for different levels of aggregation Figure 15: The number of observed overload periods for each level of aggregation 89 Figure 16: The number of "waiting" hours counting for overloads occurring at airspace sectors for each level of aggregation Figure 17: Distribution of Demand/Capacity ratio through the selected network Figure 18: Accumulated waiting time through the kernel Network, represented by pink circles Figure 19: Average imposed delays by ATFCM on aggregated Kernel Network of Europe Figure 20: Hourly distribution of imposed delays at Hamburg and at other airports of the network Page 7 of 105

8 EXECUTIVE SUMMARY This document describes the work carried out conducting a gaming exercise and network management modelling to evaluate collaborative airport planning, which shall lead to a more efficient utilization of airport resources. The goal of the gaming exercise was to analyse and evaluate the collaborative planning process and to get some insights into possible elements of the Airport Operations Centre (APOC). The exercise also looked at the usefulness of gaming supported by simulation as a validation tool for advanced operational concepts related to pre-tactical planning. A simplified Network Operations Plan (NOP) of the kernel network of ECAC-wide operations has been modelled and processed through 24 hours. This NOP should provide information for the agents in the APOC to use in their planning process, and also should provide indications on the effects of proposed local planning solutions on the network. Due to the limitations of the platform at the time of the experiment the gaming exercise had a very explorative character. It opened up the investigation of the collaborative decision process at the airport using gaming techniques and produced some initial findings of a more general nature. It was found that a common situation overview (powerwall) is beneficial for the negotiation process. Moreover a centralized APOC enabling face-to-face communication has a positive effect. It was not possible at this stage to fully evaluate operational scenarios as described in the DoDs and therefore no feedback to the DoDs or Operational Scenario documents was produced. Gaming itself proved to be a highly useful tool to investigate future concepts, as it allows experts to get a much better feel for the concept than is possible otherwise. The network management activity succeeded in providing a model sufficiently accurate to generate data that may be used in the airport collaborative planning process. The actual use of these network data in the planning process could however not be demonstrated within the scope of the exercise. It is seen as a highly interesting topic for future research to investigate the coupling of a real-time capable network analysis model with the collaborative planning at the airport level in a combined exercise. Page 8 of 105

9 1 INTRODUCTION 1.1 PURPOSE OF THE DOCUMENT This document provides the Validation Exercise Report for the gaming exercise on Collaborative, which will contribute to the elaboration of the Integrated Report of work package Episode3 WP3. The experiment is conducted by DLR in conjunction with NLR in work package WP3.3.4 of the Episode3 project, and the experiment is based on the Experimental Plan of this work package, Ref. [29]. The report includes copies of relevant material of this plan, adapted as required, and the report concludes with conduct, results, analysis and key findings of the experiment. The experiment is split in two parts, the gaming exercise run by DLR, and a Network Analysis Experiment run by NLR. This report describes: The concept addressed; Objectives, metrics and indicators of the validation experiment; Scenarios, experimental set-up and conduct of the experiment; Experimental results and analysis. The second part of the experiment on the Network Analysis model is added to this report as an annex, because it plaid not a direct role in the Gaming exercise and had as objective to develop a Gaming-supportive model. This model was proved to work appropriately to provide input to Gaming, and will be used in the future; however, its results were actually not used in the present experiment due to other priorities in organising the Gaming sessions and scenarios. 1.2 INTENDED AUDIENCE The intended audience includes: EP3 WP3 Leader; EP3 WP3.2 Validation strategy, support and operational concept refinement; EP3 WP3.3.1 Expert Group; EP3 WP3.3.4 Collaborative ; EP3 WP3.3.5 Global Performance at Network-wide level Simulation and Macromodelling. In addition the document is intended to inform the European Commission and the SESAR Joint Undertaking about the results of the gaming exercise and its applicability to specific parts of the SESAR Work Programme. 1.3 SCOPE AND STRUCTURE OF DOCUMENT The scope of the document is to describe the experiment and to explain background, objectives, approach and conduct of the experiment as well as its results, analysis and Page 9 of 105

10 findings. For clarity and to make the document easier to read the parts related to the Network Analysis Experiment are put into an Annex. Section 1 gives an overview of the document and provides an overview of the concept of interest; The gaming exercise run by DLR is described in Sections 2-4. Section 0 introduces the scope and justification of the gaming exercise. This is followed by an overview of experimental set-up, applicable validation scenarios and metrics; o o Section 3 describes the conduct of the experiment, the scope of validation and scope of results obtained for further analysis; Section 4 describes the experimental results and its analysis. The relevance of results is discussed. Section 5 summarises experimental outcomes, followed by Section 6 presenting key findings and recommendations. These two sections focus primarily on the gaming experiment run by DLR, but will also summarize the corresponding sections from Annex describing the network analysis experiment run by NLR; Section 7 lists the references and applicable documents; Section 8 (Annex) describes the network analysis processing central network management functions to provide information on constraints on airport planning due to network limitations. Some annexes are added providing tables and relevant data. This annex is structured in the same way as the main document. 1.4 EXPERIMENT BACKGROUND AND CONTEXT Based on the corresponding exercise plan, validation exercises are performed to provide evidence (preferably measured) about the ability (of some aspect) of the concept to deliver on (some aspect) of the performance targets. According to step 4 of the E-OCVM [2], an exercise report should be produced to lay down the evidence of qualities and shortcomings together with issues and recommendations. The simulation report in this document describes the validation exercise EP3 WP3.3.4 Collaborative which is done within EP3 WP3: Collaborative Planning Processes. During the preparation of the ACCES platform it became obvious that the platform development process went slower than anticipated and that not all objectives stated in the Experimental Plan could be reached in the given timeframe. The focus of the gaming exercise had to be shifted more towards analysing the negotiation process itself. In addition the connection between the collaborative planning process (Gaming exercise) and the DCB process (Network Analysis Model) could not be investigated in the scope of the exercise. The deviations from the Experimental Plan are stated clearly in body of this document. 1.5 CONCEPT OVERVIEW The concept of interest is adopted from SESAR, the ATM target concept, D3, Ref. [13], and its Concept of Operations, Ref. [15]. The experiment aimed to contribute to the validation of parts of this concept as part of the early validation process of Episode3. Page 10 of 105

11 The experiment is linked to the Episode3 Detailed Operational Descriptions (DODs) and the Operational Concept of SESAR. In this section, the following topics are addressed: The scope of the operational concept of interest, providing links to Lines of Changes (LoCs) and Operational Improvements (OIs); A detailed outline of the operational concept, derived from the DODs; The level of maturity of the concept of interest; and The Key Performance Areas (KPAs) related to the concept of interest, giving indications of relevant areas of potential benefits and performance assessment. According to the EP3 DOW [1], the experiment should explore the following elements of the SESAR Concept of Operations: Collaborative ; The implementation of an Airport Operations Centre (APOC). One concept for collaborative airport planning which is fully in line with the SESAR Concept of Operations is Total Airport Management (TAM), a concept that has been jointly defined by DLR and EUROCONTROL [24]. TAM is a future integrated method of airport management. Within the TAM concept, the Airport Operations Centre (APOC) is seen as the heart of the operation. In the APOC, agents of the airport stakeholders will constantly communicate and co-ordinate, develop and maintain dynamically joint plans and execute those in their respective area of responsibility. The core information basis of TAM is the Airport Operations Plan (AOP). The Airport Operations Plan is an en-route-to-en-route-conversion of the Network Operations Plan (NOP), enriched by airport specific data. Crucial to the AOP is that all constraints of all stakeholders are brought together: both the airport level and the network level planning constraints. For this reason, the experiment not only addresses the airport-centred operations, but also its repercussions on the network of which it forms part. Airports are always part of a network. This is what Total Airport Management entails: managing airport operations as a part of the totality of connected airports and airspace sectors. In this way, TAM is expected to lead to a better use of available capacity at an airport, since the external constraining factors are also taken into account. In the experiment, decisions taken at airport level are therefore analysed in terms of their consequences at network level. After all, solutions that may seem good for the airport may lead to large imbalances in capacity and demand in the airspace network as a whole. Apart from the above elements of the SESAR ConOps to be validated, the experiment addressed the following elements, putting airport planning processes within the context of the NOP: Improving the interoperability between Network Capacity Management Processes and planning processes at airport level (through Network Management); Monitoring ATM Performance (through Network Performance Assessment) and providing alerting information on bunching and/or congestion to planning processes at airport level. The table below shows the list of OI steps that have been addressed by the exercise. Page 11 of 105

12 OI Id OI Title OI Step Id OI Step Title How addressed? L10-03 Improving Airport Collaboration in the Pre-Departure Phase AO Improved Operations in Adverse Conditions through Airport Collaborative Decision Making In the scenario used in the experiment, an adverse condition is presented to all actors in the APOC. Next, a CDM process is started to find a commonly agreed solution to the problem of reduced capacity due to the adverse condition. L10-03 Improving Airport Collaboration in the Pre-Departure Phase AO Improved Turn-Round Process through Collaborative Decision Making Turn-Round milestones are used to set TOBT in the simulation and as constraints in the planning process. L03-02 User driven Prioritization Process AUO User driven Prioritization Process While not directly addressed the stakeholders inputs during the collaborative planning process reflect their prioritizations. L03-01 Collaborative Layered Planning Supported by Network Operations Plan DCB Interactive Rolling NOP The network analysis part of the exercise (see Annex) provides the airport planning actors with information that represents a network with a number of hub airports experiencing planning constraints due to bunching conditions. This reflects overload conditions that are rare in principle but that may occur due to incidental demand capacity balancing problems. L04-01 Improving Network Capacity Management Processes DCB Coordinated Network Management Operations Extended Within Day of Operation The improved working relationship is established in the APOC whilst reacting to capacity shortfall situations. Crucial to this coordinated network management operation is the integration of DCB results with airport capacity and demand figures. The network is expected to receive APOC planning and to provide APOC with information concerning constraining conditions. L04-02 Monitoring ATM Performance SDM Network Performance Assessment The exercise addresses part of this OI. A KPI is developed to establish the network performance in terms of throughput and incurred delay. The modelling experiment building a kernel network is appropriate to generate DCB monitoring information that can be sent to each airport. In addition, planning preference information can be processed and evaluated on acceptable and conflicting constraining conditions. Table 1: Operational Improvement Steps Detailed outline of the Operational Concept of Interest Below, an outline is given of the Operational Concept on which the experiment was based. This outline summarises applicable concept elements from the Episode3 DODs, in particular DOD M1 (Collaborative ) and M2 (Medium/Short Term Planning) [7], [8]. Collaborative Page 12 of 105

13 Today (2009) in many airports, operational decisions are often made with a limited knowledge of the most pertinent data. In addition, decisions by a given actor are often taken in isolation without reference to other actors who may be impacted by such decisions. Addressing these shortcomings individually brings small improvements but in order to improve the whole complex set of issues, it is necessary to follow the principles of Airport Collaborative Decision Making (CDM). Airport CDM is embedded in the ATM operational concept as an important enabler that will improve efficiency and punctuality. The CDM elements have been developed through airport trials and are now being widely implemented at many major European Airports. The basic foundation of Airport CDM is to have improved information sharing and data quality. It is important that the right airport partners get accurate data at the right time in the right place in order for them to make decisions while working together. This will lead to a better use of resources, partners being able to make preferences, improved punctuality and predictability. The accurate and accessible data is also used for post analysis, which is an increasingly important factor in order to measure success and learn from situations. In order to reinforce the decision making process and to provide the basis for performance based airport management in 2006 EUROCONTROL and DLR wrote the first ideas to a Total Airport Management (TAM) in a concept document, comprising a CDM approach in an APOC (Airport Operation Centre). It was then envisaged that the initial concept shall be further developed and validated, eventually by performing human-in-the-loop simulations, e.g. in SESAR or in activities like Episode3 related to SESAR. This concept document provided significant input to the M1-DoD, which served as the basis to define the gaming exercise. The approach for TAM is the development of a hierarchical structure for an optimized reaction e.g. on predicted or ad hoc capacity shortfalls, an over-demand or lack of punctuality. TAM can also result in an optimized traffic flow during normal conditions and increase the punctuality or throughput (e.g. runway system, taxiways, stands). In this way TAM includes an overall macroscopic view with necessary filtered airport information concerning the overall flow, demand and capacity. The main objectives and benefits of TAM are: Improved predictability of the behaviour of the system airport within the air transport network, i.e. increased prediction look-ahead-time and reduced variability of schedules compared to today, in order to give the network more time to pro-actively manage the air transport and to become more stable and robust; More equal performance of different airports with respect to each other, measured by one common set of performance indicators, the airport shall agree with other stakeholders and the ATFCM on a guaranteed QoS with respect to these indicators a QoS Contract (QoSC); TAM shall provide ways to handle degraded situations in the most appropriate way to ensure that the QoS is fulfilled as well as possible. In terms of the development of a Total Airport Management (TAM), the function allocation between human actors and future assistance systems (for example planning systems) could be realised by the design and use of scenarios. Scenarios describe the behaviour of users and the future system, interaction between the two, and the wider context of use. Scenarios also aid the analysis of multiple aspects of a complex problem more or less simultaneously on a qualitative level. Page 13 of 105

14 Use cases describe the system s behaviour under various conditions as the system responds to a request from one of the stakeholders (the primary actor). Therefore we have a more detailed representation of the work flow. The primary actor initiates an interaction with the system to accomplish a goal. In the TAM concept the main focus can be located in the design of the common decision making processes of actors from different stakeholders. Based on the stakeholder interests potential conflicts should be identified and concepts for conflict solution should be found. The initiation of a task (every task can be described through a Use Case) starts with the request of an agent in the APOC. Every task corresponds to a definite time window and comprises a deadline, to which the task has to be processed. Every task will be sorted into a task list according to the deadline. At the same time only one task can be processed. It is the task sorted at the first position in the task list. The definition of the human role in TAM and the resultant interaction of actors is a theme for the collaborative decision making process. The creation of TAM will permit the generation of a global information space derived from a number of local information sources. Aspects are the integration of local information into a global picture or the kind of intermediation between human actors of different organizations with respect to their different goals and intents. Expected benefits of a cooperative working and decision process in TAM could be: possibility of direct verbal communication and discussion; representation of information by means of common used displays; common computer aided simulations; transmission of planning orders, action proposals or action instructions; better negotiation and solution of conflicts and communication of interests. Figure 1: Agents working in the Airport Operations Centre The central tool of TAM is the APOC where representatives of all involved stakeholders work (Figure 1). Good communication between aircraft operators, ATC, airport and other Page 14 of 105

15 stakeholders is decisive for the success of cooperation in the management process of an airport. A main idea of APOC is attaining the best possible cooperation through direct communication between the different stakeholders through their APOC representatives. Future advantages of this central collaborative planning approach are located in possibilities for a faster reaction to arising critical traffic situations and consideration of customer wishes. Furthermore each representative stakeholder will be aware of the objectives and interests of the other stakeholders, due to the individual contacts amongst them. Following the TAM approach all APOC operators use shared information and plan base (AOP). The improvement of the general situational awareness supports a better quality for the collaborative decision making process. All parties know the constraints and are able to react to these constraints. Especially the local contiguity of operators offers the possibility to take priorities of ones neighbour into account. The specific information and knowledge necessary for each operator to perform his tasks depending on his specific role could be identified. The off-line modelling part of the experiment, performed by NLR, addressed the interactions and connectivity between the airport under investigation and its surrounding network. This network represents the NOP or part of the NOP and consists of routings, airspace sectors and airports. To determine the mutual interactions and consequences of collaborative decision making at airport level and the surrounding network, the operations of the following conceptual elements are important: Demand and Capacity Balancing In SESAR, the Network Operations Plan (NOP) is central to the concept of operations for large airports and their environment. In the NOP, a planning of operations is available that is converging in level of confidence, level of detail and quality of planning towards the executive phase. From the NOP, the Airport Operations Plan (AOP) is derived: an en-route-to-en-routeconversion of the NOP, enriched by airport specific data. Both departure and arrival operations are expected to follow the AOP, being consistent and in agreement with the NOP, and to behave in compliance with this constantly updated planning, making these operations reliable and predictable. The NOP comprises information concerning demand, capacity and the proposed and agreed measures to balance demand and capacity. The details on Demand and Capacity balancing are described in the concept of operation of the CNM model in Annex, section Network Management and Design The network is considered in the experiment from the point of view of management and control on DCB and throughput analysis only. Network requirements are derived from optimised routings through sectors and airports. As described under the DCB paragraphs, the network consists of airports, i.e. more than 500 ECAC-wide, and sectors, i.e. more than 2000 ECAC-wide. The experience is that this network determined by capacity of executive controller workload, is super-critical; see e.g. SESAR Performance Assessment, [19]. To come to a Network that is manageable from the point of view of DCB and throughput analysis there is a need to identify main- and sub-structures in the network, i.e. the Kernel Network, and to find a strategy to select a less overload-critical structure. Moreover, structuring of the Network can be helpful for better response and enhanced transparency on information provision. Such a Kernel Network is constructed, derived from the ECAC-wide network, that is assessed to provide network constraining information and that is capable to process departure preferences agreed at airport level. The Page 15 of 105

16 details on Network Management and Design are described in the concept of operation of the CNM model in Annex, section OVERVIEW OF EXERCISE: Validation Exercise ID and Title EP3 WP3.4.4 Gaming Exercise on Collaborative Leading organization Validation objectives Rationale Expected results per KPA DLR Study and assess the process of collaborative decision making in the APOC. Analyse interaction between planning at the local airport and the surrounding network. The idea of the gaming exercise is to put stakeholder agents into a simulated situation, where a problem is foreseen with enough time ahead to engage in a collaborative planning process. The planning itself is be supported by a tool allowing what-if analyses and supporting the negotiation of among the stakeholder agents. The effects of the collaboratively reached decision can then be analysed by simulation. The exercise was expected to demonstrate that the collaborative planning process leads to an AOP that in critical situations defines an agreed compromise between achievable punctuality and throughput (Efficiency); a network model can generate data that can be used in a gaming exercise to evaluate the effects of the local planning on the network. OI steps addressed Validation Technique Supporting DOD / Operational Scenario Geographical area performance framework level AO-501, AO-601, AUIO-0102, DCB-0102, DCB-0206, SDM-0101 Gaming, Modelling DOD M1: Collaborative Scenarios: Severe capacity shortfall in the Short Term DOD M2: Medium/Short-term Planning The gaming exercise focused on a single airport; Hamburg is taken as example airport for these gaming runs. The network modelling was done for the European core area. Table 2: Exercise Overview 1.7 GLOSSARY OF TERMS 4D A-CDM ACCESS Term Definition Four-dimensional Airport Collaborative Decision Making Airport Control CEnter Simulator Page 16 of 105

17 AENA AMAN ANSP AOC AOP APOC ASGARD ASM ATC ATFCM ATM CDM CFMU CMC CNM ConOps DCB DLR DMAN DOD DOW ECAC E-OCVM EP3 ERC ETMA F2F FA FCFS FMS ICAO JU KPA KPI LoC NAM Term Definition Aeropuertos Españoles y Navegación Aérea Arrival MANager Air Navigation Service Provider Airline Operations Centre Airport Operations Plan Airport Operations Centre Autonomous Simulation of Ground movements, Arrival and Departure Air Space Management Air Traffic Control Air Traffic Flow and Capacity Management Air Traffic Management Collaborative Decision Making Central Flow Management Unit Computer mediated communication Central Network Management Concept of Operations Demand and Capacity Balancing German Aerospace Centre Departure MANager Detailed Operational Description Description of Work European Civil Aviation Conference European Operational Concept Validation Methodology Episode3 EUROCONTROL Extended Terminal Control Area Face to face Focus Area First-Come First-Served Flight Management System International Civil Aviation Authorithy Joint Undertaking Key Performance Area Key Performance Indicator Line of Change Network Analysis Model Page 17 of 105

18 NATS NLR NOP OI OS OSD QoS QoSC RBT SBT SES SESAR SJU SMAN SNA SWIM TAM TMA TMAN TOBT TOP TRAFSIM TWR VOIP WP Term Definition National Air Transport Service National Aerospace Laboratory Network Operations Plan Operational Improvement Operational Scenario Operation Sequence Diagram Quality of Service Quality of Service Contract Reference Business Trajectory Shared Business Trajectory Single European Sky initiative Single European Sky ATM Research and Development Programme SESAR Joint Undertaking Surface MANager Social Network Analysis System Wide Information Management Total Airport Management Terminal Control Area Turn-round MANager Target Off-Block Time Total Operations Planner TRAFfic SIMulator Tower Voice over IP Work package Table 3. Glossary of terms Page 18 of 105

19 2 SUMMARY OF EXPERIMENT AND STRATEGY PLANNING 2.1 EXPECTED EXPERIMENT OUTCOMES, OBJECTIVES AND HYPOTHESES Description of Expected Experiment Outcomes The table below shows the list of OI steps that have been addressed by the exercise. OI Id OI Title OI Step Id OI Step Title How addressed? L10-03 Improving Airport Collaboration in the Pre-Departure Phase AO Improved Operations in Adverse Conditions through Airport Collaborative Decision Making In the scenario used in the experiment, an adverse condition is presented to all actors in the APOC. Next, a CDM process is started to find a commonly agreed solution to the problem of reduced capacity due to the adverse condition. L10-03 Improving Airport Collaboration in the Pre-Departure Phase AO Improved Turn-Round Process through Collaborative Decision Making Turn-Round milestones are used to set TOBT in the simulation and as constraints in the planning process. L03-02 User driven Prioritization Process AUO User driven Prioritization Process While not directly addressed the stakeholders inputs during the collaborative planning process reflect their prioritizations. L03-01 Collaborative Layered Planning Supported by Network Operations Plan DCB Interactive Rolling NOP The network analysis part of the exercise (see Annex) provides the airport planning actors with information that represents a network with a number of hub airports experiencing planning constraints due to bunching conditions. This reflects overload conditions that are rare in principle but that may occur due to incidental demand capacity balancing problems. L04-01 Improving Network Capacity Management Processes DCB Coordinated Network Management Operations Extended Within Day of Operation The improved working relationship is established in the APOC whilst reacting to capacity shortfall situations. Crucial to this coordinated network management operation is the integration of DCB results with airport capacity and demand figures. The network is expected to receive APOC planning and to provide APOC with information concerning constraining conditions. L04-02 Monitoring ATM Performance SDM Network Performance Assessment The exercise addresses part of this OI. A KPI is developed to establish the network performance in terms of throughput and incurred delay. The modelling experiment building a kernel network is appropriate to generate DCB monitoring information that can be sent to each airport. In addition, planning preference information can be processed and evaluated on acceptable and conflicting constraining conditions. Table 4: Operational Improvement Steps Page 19 of 105

20 2.1.2 Description of Experiment Objectives and Assumptions As mentioned in the Introduction it was not possible to fully address all the objectives stated in the Experimental Plan [29], since the gaming platform and negotiation tool development had progressed slower than anticipated. As a consequence the objectives had to be modified accordingly. The main objective still remains to study the collaborative decision making process. The restated objectives are as follows: 1. The main objective of the exercise is to study the collaborative planning process in an Airport Operations Centre. This high level objective can be further subdivided into the following specific objectives: a. Evaluate a workflow model of the decision process; b. Examine the role of a moderator to support the decision making process; c. Given the APOC and given a realistic scenario demanding a commonly reached pre-tactical decision: i. Determine how actors reach a decision in the APOC (following the chosen negotiation protocol); ii. Demonstrating and investigating a prototype planning support tool. 2. Assess the effects of this decision making on the airspace network surrounding the example airport. This objective is addressed by the network modelling part of the exercise (see Section 8). 3. Finally experience in the use of gaming techniques shall be gained from the exercise. This experience will be very valuable in the further validation process of the SESAR operational concept. Compared to the original objectives described in the Experimental Plan, D [29] the main change in the objectives it is no longer an objective to do any performance assessment on the output of the collaborative planning process. This is owed to the fact that the experiment is run without the simulation suite and so any assessment would have been based on planning results solely, while originally the plan should have been simulated and the output of the simulation should have been used for performance assessment. On the other side this allowed to perform more gaming runs, as no time was needed in between the runs to perform the simulations. Hence the main objective, to analyse the decision making process itself, could be expanded significantly and more conditions (e.g. centralised vs. decentralised APOC) could be investigated. The other objectives, assessing the effect of the decision making on the network and evaluating gaming as an experimental technique remained unchanged. The following statements were not to be tested in the scope of this exercise and therefore set as assumptions: A1: The agents in the APOC behave in a cooperative way. (Effects of uncooperative behaviour and how to get them to behave cooperatively are to be studied in separate experiments outside EP3); Page 20 of 105

21 A2: The negotiation in the APOC is sequentially, i.e. tasks are not negotiated in parallel. (Limitation of the support tool prototype); A3: A moderator or supervisor agent is present in the APOC (currently necessary to technically control the negotiation process and the powerwall display); A4: The absolute level of traffic is irrelevant for this exercise. The important parameter is the demand/capacity imbalance; hence real traffic data from 2004 can be used Description of Experiment Hypotheses Again some changes have been made with respect to the Experimental Plan [29] due to the shift in focus of the exercise. The above listed hypotheses H3 relating to performance assessment has been dropped completely, and the other two hypotheses have been refined: H1: Sharing all relevant data within the AOP ensures a high level of situational awareness of the involved stakeholders (high level objective 1); H2: Collaborative airport planning allows stakeholders to agree on a set of performance parameters for the airport to deal with a forecast problem situation (high level objective 1); H3a: A common information platform (powerwall) improves the situation awareness of the involved stakeholders; H3b: A common information platform (powerwall) has a positive effect on the negotiation process; H4: Direct face-to-face communication has a positive effect on the negotiation process. These hypotheses reflect the change in focus to a more thorough analysis of the negotiation process itself rather than potential performance gains associated with it. For the Network Management Modelling part of the exercise the following hypotheses are stated (see also Section ): H1: The Kernel Network of Europe represents the behaviour of network constraining decision making in a sufficiently realistic way to provide a realistic context of network-wide operations for the APOC and its decision making at airport level; H2: The constraining conditions of the Kernel Network are providing appropriate guidance to the Gaming exercise, which can be used in an effective way to keep delays to an acceptable minimum throughout the Network; H3: The departure preferences, agreed at airport level, can be accommodated in an appropriate way by applying prioritisation within the Kernel Network. 2.2 CHOICE OF METRICS AND MEASUREMENTS Measurements for Gaming at airport level In contrast to the statements given in the Experimental Plan no performance assessment has been carried out and no analysis of the achieved plans relating to KPIs has been done. After each experimental run, questionnaires were administered, measuring task clarity (own task, others tasks), perceived negotiation style (e.g., fairness, fact orientation), and mental Page 21 of 105

22 work load. Finally, participants filled out a post-experimental questionnaire including a usability assessment of the negotiation support tool (provision of information, coordinative support). Additionally, all experimental runs were video taped and the communication content was transcribed, coded (e.g., task-related communication, coordination, system-related communication) and analysed. 2.3 CHOICE OF METHODS AND TECHNIQUES Usability of the negotiation support tool, team situation awareness and negotiation style were measured with a modified version of the SHAPE Teamwork Questionnaire (STQ, EUROCONTROL, 2008, [32]). With a modified version of the SHAPE questionnaire Assessing the Impact of Automation on Mental Workload (AIM, EUROCONTROL, 2008, [32]), the mental workload was measured. For the communication analyses a categorisation scheme was developed that is based on Bales Interaction Process Analysis (IPA; 1950, [31]) and Stempfle & Badke-Schaub s (2002, [33]) analysis approach for design teams. Four main communication categories were defined: Task-related communication, coordinative communication, and task- or system-related questions (see Table 5). Moreover, the main categories were subdivided in several subcategories for a more fine-grained analysis of the communication contents. Category TASK-RELATED COMMUNICATION Sub-Category Strategic discussion Proposal Evaluation of a proposal Request of opinion Individual Goal/ Information Question COORDINATIVE COMMUNICATION Next steps Information about actions QUESTIONS System-related question Task-related question Page 22 of 105

23 Category Sub-Category MISCELLANEOUS Table 5: The coding scheme During the whole experimental run, an observer documented all relevant negotiation steps, events, and any comments on the concept or system. After each trial and at the end of the experimental sessions, the participants evaluated the TAM concept, the experimental scenario, and the usability of the support tool in semi-structured interviews. Supported Metric / Measurement Platform / Tool Method or Technique Team situation awareness Negotiation style Mental work load Usability Questionnaire Questionnaire Questionnaire Questionnaire Debriefing SHAPE Questionnaire (EUROCONTROL, 2008); SHAPE Questionnaire (EUROCONTROL, 2008); Semi-structured interview Communication Content Video taping Content analysis Negotiation Process Observation Event-sampling Table 6: Metrics and data sources 2.4 VALIDATION SCENARIO SPECIFICATIONS Taking into account the objectives of this exercise, a set of gaming exercises has been executed to investigate the decision making process in the APOC. The validation scenario was based on the following operational scenarios defined within EP3 WP2: OS-18: Airport Operational Plan Lifecycle for Medium-Short-Execution Phases; OS-19: Severe Capacity Shortfalls impacting Departures in the Short-Term. These operational scenarios have set the baseline for the detailed scenarios set up for the exercise. The scenario covered a full day of traffic for one airport, and only the local planning at the airport planning was investigated in the gaming exercise. The airport under consideration was Hamburg airport, which is also used in other TAM projects at DLR. This airport has been chosen for the current experiment, since it had already been modelled for the ACCES platform in another project, and so the experiment was possible within the given budget. Hamburg does have an interesting topology with its crossing runway system and the resulting dependence between arrival and departure traffic and thus makes an interesting airport to study collaborative planning processes. Page 23 of 105

24 2.4.1 Scenario Details Airport modelling The airport under consideration was Hamburg airport. Hamburg has a system of two crossing runways (see Figure 2). For the validation scenario only one runway configuration was active, using 23 for arrivals and 33 for departures. The restriction to only one configuration is owed to the limitations of the platform, which at the moment does not allow changing runway configurations within a simulation run Traffic modelling The traffic used in the gaming exercise was based on one day of real traffic data from Hamburg airport dating from 25 th May Traffic data for this kind of simulation needs a number of data that are not present in the reference traffic produced in EP3 WP2. In particular the simulation requires gate and stand allocation as well as references between in- and outgoing flights to be able to model the turn around process, which is a major factor in the airport process model. It was not possible within the given time to add this data to the reference traffics, and hence only the available traffic data could be used in the exercise. This restriction is however fully in line with the stated objectives of the exercise, which is primarily to study the decision making process itself and the usefulness of this type of exercises for concept validation work. Figure 2: Hamburg airport layout For the decision making process the most relevant factors are the available data and the presence of a capacity/demand imbalance. For the gaming exercise, a scenario from Hamburg Airport (05/25/2004) was used. Figure 3 shows the distribution of arrivals and departures over the day. For the exercise a time window containing the morning peak was used, and a predicted capacity reduction was introduced during the time of maximum traffic to create a demand/capacity imbalance to be solved by the actors in the exercise. Page 24 of 105

25 DEP ARR bis 1 1 bis 2 2 bis 3 3 bis 4 4 bis 5 5 bis 6 6 bis 7 7 bis 8 8 bis 9 9 bis bis bis bis bis bis bis bis bis bis bis bis bis bis bis 24 Figure 3: Arrivals and Departures (Cumulative Diagram) Scenario events Two operational scenarios based on the same flight plan data were used in the experiment. These are stated below. In both cases the gaming starts at 7:00 am and the expected capacity shortfall is made known then. Case 1: Reduced capacity This scenario featured a predicted reduction in available runway capacity due to bad visibility (fog) to a value of 20 movements/hour total (arr + dep) between 08:30 and 10:00. Case 2: Closed runways This scenario featured a predicted closure of the runway system due to a heavy thunderstorm moving directly over the airfield between 09:00 and 09: Tasks given to agents In each run the stakeholder agents were given specific tasks to accomplish. These tasks were selected such that each agent had to look at a different set of flights and also that it optimal Page 25 of 105

26 solutions for the two agents were different, reflecting the different goals of the stakeholders. The goals were set such that in case 1 (Reduced capacity) no solution satisfying both targets was possible, and hence a compromise had to be found. In case 2 (Runway closure) there was a solution that satisfied both targets. The tasks were given to the agents in printed form immediately before the exercise in question. The tasks given to them are listed below: Please note that for each exercise the agent only got the part relevant to him (he did not know the other stakeholder s goal) and only for the current case. AIRLINE AGENT The basic scenario takes place at an airport looking similar to Hamburg. The traffic data is taken from real flight data from a normal day of operations in 2004 and should pose no capacity problem. The session begins at a simulated time of 7:30 am. Case 1: Reduced capacity This scenario features a predicted reduction in available runway capacity due to bad visibility (fog) to a value of 20 movements/hour total (arr + dep) between 08:30 and 10:00. The agents in the APOC were to negotiate a setting for the Arrival/Departure ratio that fits the needs of both the airline and ATC or airport agent. To do this they were using the Use-Case Set ADRatio defined in the TOP-client. This use case allows setting the upper and lower bounds for the ADRatio. The ratio was to be modified in the time from 08:00 to 10:00. The airline agent s task was to make sure as many aircraft as possible are departing during this time to avoid delays later during the day from roundtrips. Specifically the number of flights departing in the time from 8:00 to 10:00 should not be less than 21 aircraft. Case 2: Closed runways This scenario featured a predicted closure of the runway system due to a heavy thunderstorm moving directly over the airfield between 09:00 and 09:45. The agents in the APOC were to negotiate a setting for the Arrival/Departure ratio that fits the needs of both the airline and ATC or airport agent. To do this they were using the Use-Case Set ADRatio defined in the TOP-client. This use case allows setting the upper and lower bounds for the ADRatio. The ratio was to be modified in the time from 09:00 to 11:00. The airline agent s task was to make sure as many aircraft as possible are departing during this time to avoid delays later during the day from roundtrips. Specifically the number of flights departing in the time from 8:00 to 10:00 should not be less than 21 aircraft. ATC/AIRPORT AGENT The basic scenario takes place at an airport looking similar to Hamburg. The traffic data is taken from real flight data from a normal day of operations in 2004 and should pose no capacity problem. The session begins at a simulated time of 7:30 am. Case 1: Reduced capacity This scenario featured a predicted reduction in available runway capacity due to bad visibility (fog) to a value of 20 movements/hour total (arr + dep) between 08:30 and 10:00. Page 26 of 105

27 The agents in the APOC were to negotiate a setting for the Arrival/Departure ratio that fits the needs of both the airline and ATC or airport agent. To do this they were using the Use-Case Set ADRatio defined in the TOP-client. This use case allows setting the upper and lower bounds for the ADRatio. The ratio was to be modified in the time from 08:00 to 10:00. The ATC/airport agent s task was to make sure as few aircraft as possible are held airborne during this time. Specifically the number of flights arriving in the time from 8:00 to 10:00 should not be less than 26 aircraft and the number of arrivals delayed by more than one hour ( very late ) during the time from 08:00 to 13:00 should not exceed 7 aircraft. Case 2: Closed runways This scenario featured a predicted closure of the runway system due to a heavy thunderstorm moving directly over the airfield between 09:00 and 09:45. The agents in the APOC were to negotiate a setting for the Arrival/Departure ratio that fits the needs of both the airline and ATC or airport agent. To do this they were using the Use-Case Set ADRatio defined in the TOP-client. This use case allows setting the upper and lower bounds for the ADRatio. The ratio was to be modified in the time from 09:00 to 11:00. The ATC/airport agent s task was to make sure as few aircraft as possible are held airborne during this time. Specifically the number of flights arriving in the time from 9:00 to 11:00 should not be less than 29 aircraft and the number of arrivals delayed by more than one hour ( very late ) during the time from 08:00 to 13:00 should not exceed 8 aircraft. 2.5 EXPERIMENTAL VARIABLES AND DESIGN A 2 (Communication mode: direct vs. computer-mediated) x 2 (Powerwall: with vs. without) design was utilised for the gaming exercise. Due to the small number of participants, a within subjects design was used, i.e., all participants completed all experimental conditions. ID Description IV 1 IV 2 Communication mode (direct communication vs. computer-mediated communication) Powerwall (with vs. without powerwall) A 2 x 2 factorial within subject design was utilized Table 7: Experiment Variables Page 27 of 105

28 Scenario ATC/ Airport Agent Individual Negotiation Goals Airline Agent Reduced Capacity (8:30-10:00) Closed Runways (9:00-9:45) Number of ARR between 8:00-10:00: 26 Number of delayed ( very late ) a/c between 8:00-13:00 7 Number of ARR between 9:00-11:00: 29 Number of delayed ( very late ) a/c between 8:00-13:00 8 Number of DEP between 8:00-10:00: 21 Number of DEP between 8:00-10:00: 21 Table 8: Individual Negotiation Goals Page 28 of 105

29 3 CONDUCT OF VALIDATION EXERCISE RUNS 3.1 EXPERIMENT PREPARATION Experimental Setting The experiment was conducted in the ACCES facility, which provides up to ten operator working positions as well as a large projection screen (2.2 by 5.5 metres), the so-called Powerwall. In the experiment the powerwall displayed general information about the current task, the negotiation deadline, and the current negotiation status. Furthermore, the agents individual proposals and their consequences on the flight plan were shown. Figure 4: Powerwall design For the gaming exercise, five operator positions, one observer and one experimenter position were used. The moderator position, the Airport/ ATC, and Airline positions were placed in the first row to have a good view on the powerwall. The observer position was in the second row, to monitor the negotiation between the agents. A movable wall was positioned between the both participants (Airport/ ATC and Airline) in conditions in which face-to-face communication was not allowed. Page 29 of 105

30 Figure 5: Experiment setup in ACCES The negotiation between the agents was recorded for subsequent content analysis. As shown in Figure 6, two digital cameras were placed besides the working positions of the Airport/ ATC and Airline. Page 30 of 105

31 Technical Supervisor Observer Movable wall Moderators Airport/ ATC Airline Experimenter Cam Cam Powerwall- Projection Figure 6: Experimental Setting Instructions and Training Prior to the experimental runs, the participants were instructed about the TAM concept, the technical system (negotiation support tool, powerwall), and the experimental task. Specifically, they were informed about their role, their individual and the common goals in the experimental task. In a next step, participants completed a test trial to become acquainted with the tools and the negotiation logic. In the test trial, participants had to complete at least one complete negotiation task, including the initialisation of a negotiation task, the modification of the lower and upper bound of the A/D ratio, sending the proposal to the moderator, and responding on alternative proposals of the other agent. The whole negotiation procedure was repeated unless all participants indicated that the felt sufficiently prepared for the experimental trials. 3.2 EXECUTED EXPERIMENT SCHEDULE Two teams of two experts each participated in the exercise. While it certainly would be highly desirable to have a larger number of participants (more stakeholders per run as well as more runs with different teams), this was not possible in the scope of the exercise. The small Page 31 of 105

32 number of participants however is sufficient given the explorative character of the gaming exercise. The results of a pilot study indicated strong learning effects in the first experimental trials. This might be explained by the rather complex technical system. Although participants were thoroughly instructed and completed several test trials, the experimenter had to support participants in initial phase of the experimental sessions. Thus, we chose an experimental order which allowed direct communication in the first experimental sessions. Moreover, participants were supported in the negotiation process by the powerwall in the initial session. Afterwards, participants had to negotiate without the powerwall (Session 2). Similarly, participants communicated via computer in session 3 and 4 with powerwall support in session 3 and without powerwall support in the last session 4. Experimental Session 1 Direct communication (face-to-face) 2 Direct communication (face-to-face) Communication Powerwall Scenario 3 Computer-mediated communication 4 Computer-mediated communication With Powerwall Without Powerwall With Powerwall Table 9: Sessions Without Powerwall Reduced capacity Closed runway Reduced capacity Closed runway 3.3 DEVIATIONS FROM THE PLANNING The development state of the supporting simulation suite and the negotiation support tool demanded a shift in focus of the exercise, which now only analysed the negotiation process itself. This is reflected in changes in objectives and hypotheses stated in Section 2.1. Some additional changes to the planned experiment setup have been made as stated in the following paragraph. Originally, a variation of the experimental condition order was planned, with the second team completing the conditions in reverse order. Yet, as noted above, a pilot study indicated that participants became more and more acquainted with the system during the experimental sessions. Therefore, we decided not to start with the more demanding condition (no direct communication, no common display), but to choose the same order for both two-person teams. Consequently, order effects could not be controlled in the present gaming exercise. However, with the chosen condition order, we have a more conservative test for our assumptions, since the positive effect of learning should have a positive impact on the negotiation process and result in the latter sessions (without direct communication). During the experimental runs with computer-mediated communication, a technical problem with the implemented messaging tool occurred, i.e., the moderator could not send or receive any messages. Thus, an alternative messaging tool was used for the second team. Page 32 of 105

33 4 EXPERIMENT RESULTS 4.1 MEASURED EXPERIMENT RESULTS Several aspects of the negotiation process have been analyzed and the results are stated below Negotiation Duration The average time the both teams needed to find consensus solutions was 32.6 minutes (standard deviation = 18.3). However, the mean times for the both teams varied significantly (see 1 Face to Face 2 Computer mediated communication Table 10), what might be explained by the fact that Team 2 completed all trials on one single day. For Team 1 a strong learning or memory effects occurred, which is reflected in decreasing negotiation times. Team 1 completed Trial 1-2 on the first and the remaining trials on the second day. Thus, the longer negotiation times of team 1 on the second day might be a consequence of diminished learning effects. Furthermore, the whole experimental run had to be restarted on the second day due to a maloperation. Experimental Condition TEAM 1 TEAM2 1. F2F 1 / Powerwall 45 min 43 min 2. F2F/ No Powerwall 19 min 21 min 3. CMC 2 / Powerwall 56 min 16 min 4. CMC/ No Powerwall 52 min 9 min 1 Face to Face 2 Computer mediated communication Table 10: Negotiation Times Negotiation Process In a second step, the negotiation process was analysed. Again, results indicate learning effects. In the first trials, both teams were searching individual solutions first and then discussed their individual goals afterwards. In the later trials, they changed their negotiation strategy and first clarified their individual goals and searched for a common solution afterwards (with only one exception; cf. Table 11). Accordingly, the number of negotiation cycles decreased. Thus, the shorter negotiation times might also be due to the change of negotiation strategy. Page 33 of 105

34 Experimental Condition TEAM 1 TEAM2 1. F2F/ Powerwall Airline proposal Goal clarification ATC proposal Airline accepts Airport proposal Airline proposal Goal clarification Airport proposal AL accepts 2 negotiation cycles 2. F2F/ No Powerwall Goal clarification ATC proposal Airline accepts 3 negotiation cycles Goal clarification Airline proposal Airport accepts 1 negotiation cycle 3. CMC/ Powerwall AL proposal ATC proposal Airline accepts occasionally DEADLINE expired - 2 negotiation cycles 4. CMC/ No Powerwall Goal clarification ATC proposal Airline proposal ATC accepts 1 negotiation cycle Goal clarification Airline.proposal Airport accepts 1 negotiation cycle Goal clarification Airline.proposal Airport accepts 2 negotiation cycles 1 negotiation cycle Table 11: The Negotiation Processes in both Experimental Teams Negotiation Results As mentioned earlier, both teams had to find a common solution in two different scenarios (Reduced capacity; closed runway). However, in the reduced capacity scenario, participants had to concede for a common solution, i.e., a distributive conflict situation had to be solved. In the second closed runway scenario, an integrative conflict situation was given, i.e., there was a solution that allowed both agents to meet their individual goals. Both teams found solutions for both scenarios with one exception (due to maloperation and consequential deadline expiration). Thus, there was no clear effect of experimental conditions on negotiation results. These results are shown in Table 12. Page 34 of 105

35 Experimental Condition TEAM 1 TEAM2 1. F2F/ Powerwall Compromise found: ATC: 22 of 26 ARR, 7 of 7 Delays Airline: 22 of 21 DEP Compromise found: ATC: 25 of 26 ARR, 9 of 7 Delays Airline: 19 of 21 DEP 2. F2F/ No Powerwall Integrative solution found Integrative solution found 3. CMC/ Powerwall Deadline expired (due to maloperation) Compromise found (results not recorded) 4. CMC/ No Powerwall Integrative solution found Integrative solution found Table 12: Negotiation Results Communication content In a next step of analysis, the communication video taped during the negotiations was transcribed. All communication units (a message, spoken or typed, from one participant to one or more others, containing one piece of information) was listed and grouped by content type. The following Table 13 shows the number of communicative units in for each participant (Moderator (Mod), ATC agent (ATC), Airline agent (Airl.) and Experiment supervisor (Exp.) )and condition. Experimental Condition TEAM 1 TEAM 2 Powerwall/ F2F Mod ATC Airl. Exp. Mod APT Airl. Exp. Total Negotiation Content No Powerwall/ F2F Total Negotiation Content Powerwall/ CMC Total Not recorde d Page 35 of 105

36 Experimental Condition TEAM 1 TEAM 2 Negotiation Content No Powerwall/ CMC Total Not recorde d Negotiation Content Table 13: Total Number and Negotiation Related Communicative Units Obviously, a strong learning effect occurred, i.e., the number of communication units that were task or tool related decreased across the trials (indicated by a higher proportion of negotiation content throughout the experimental sessions). Besides, there was a statistically significant main effect of communication mode, i.e., the team members negotiated significantly less when communicating via computer (M = 4.6) compared to face-to-face negotiation (M = 27.8). After having analyzed number of communication units, we explored the communication content in a next step. All communication units were categorized according to the coding scheme described above (Section 2.3) by two independent raters. The inter-rater reliability of the codings was determined by computing the Holsti coefficient [30]. The average reliability for the main categories was high (C R = 0.89) with a range from 0.81 to Yet, the corresponding values for the sub-categories were rather moderate (C R = 0.52). Therefore, our main analyses were based on the main categories. Table 14 shows the category entries for both teams. Experimental Condition Team 1 Team 2 ATC Airl Airp Airl Powerwall/F2F Goal clarification TG Proposal TP Evaluation of proposal TE Asks for opinion TA Ind. Goal/ Information TI Question TQ TASK-RELATED COMMUNICATION (T) Coordination: Next steps CN Coordination: Information CI COORDINATION (C) System-related question Page 36 of 105

37 Experimental Condition Team 1 Team 2 Task-related question QUESTIONS Miscellaneous No Power./F2F Goal clarification TG Proposal TP Evaluation of proposal TE Asks for opinion TA Ind. Goal/ Information TI Question TQ TASK-RELATED COMMUNICATION (T) Coordination: Next steps CN Coordination: Information CI COORDINATION (C) System-related question Task-related question QUESTIONS Miscellaneous Powerwall/CMC Goal clarification TG Proposal TP Evaluation of proposal TE Asks for opinion TA Ind. Goal/ Information TI Question TQ TASK-RELATED COMMUNICATION (T) Coordination: Next steps CN Coordination: Information CI COORDINATION (C) System-related question Task-related question QUESTIONS Miscellaneous Page 37 of 105

38 Experimental Condition Team 1 Team 2 No Power./CMC Goal clarification TG Proposal TP Evaluation of proposal TE Asks for opinion TA Ind. Goal/ Information TI Question TQ TASK-RELATED COMMUNICATION (T) Coordination: Next steps CN Coordination: Information CI COORDINATION (C) System-related question Task-related question QUESTIONS Miscellaneous Table 14: Coding Frequencies Analyzing the entries for the main categories revealed that participants exchanged more taskrelated information when working face-to-face (M = 35,25) compared to conditions with computer-mediated communication (M = 3,75). Yet, the number of coordinative communication units was similar in all experimental conditions (M = 4). Thus, the positive overall effect of face-to-face communication can be attributed to intensified task-related communication. Finally, the number of task- or system-related questions declined across the experimental runs, i.e., a learning effect occurred Questionnaires After each experimental run, participants rated their situation awareness, negotiation style, workload, and support tool usability in a post-trial questionnaire. Additionally, the overall usability and workload was measured in a post-experimental questionnaire. In Tables 15 and 16 the mean ratings for both questionnaires are shown. All scales ranged from 1-7. Page 38 of 105

39 SITUATION AWARENESS Experimental Condition Items (1-7 scales) In the previous working session it was clear to me which tasks were my responsibility F2F/ PW F2F/ no PW Means CMC/ PW 6,5 5,8 6,8 6,5 it was clear to me which tasks I shared with the team members 5,8 5,5 5,3 5,0 the team agreed on how to evaluate information 5,8 3,7 5,8 2,5 NEGOTIATION STYLE CMC/ no PW the final team solution considered the interests of both agents appropriately 6,0 6,3 4,8 5,5 the communication in the team was well coordinated 5,5 5,7 4,5 4,5 every team member was able to appropriately communicate his interests 6,5 6,3 6,0 5,5 every team member participated in the communication equally 6,0 6,3 5,5 4,8 the negotiation was fair 6,5 6,7 6,3 5,8 I had the impression that I could communicate my interests appropriately 6,5 5,7 6,3 6,0 the negotiation style was constructive and fact oriented 5,5 6,0 6,3 4,5 WORKLOAD In the previous work session, how much effort did it take to identify potential conflicts? 4,0 4,5 3,0 3,3 to communicate and negotiate with other agents 3,5 4,5 3,5 4,3 USABILITY In the previous work session, the system helped the team to synchronize their actions the system provided the team with sufficient information about the current status of negotiation the system provided the team with sufficient information about the different solutions (AOPs) the system provided the team with sufficient information which steps are required next 5,5 3,8 4,3 4,8 5,8 4,0 5,5 4,5 5,0 4,0 4,3 4,8 5,3 3,3 3,3 3,3 Table 15: Items and Means (Post-Trial Ratings) Situation Awareness: The overall values for the situation awareness were above the scale mean (= 4), providing evidence that participants were sufficiently conscious of their individual and shared goals. Yet, in face-to-face conditions, individuals indicated slightly higher team situation awareness (M = 5.6) as compared to conditions with computer-mediated communication (M = 5.2). For both communication modes, the ratings were higher when working with powerwall support (M PW = 5.5 vs. M NoPW = 5.3). Similarly, a positive effect of the powerwall was evident as regards the common evaluation of information. While participants working without powerwall reported rather low values (M = 3.1), these values were significantly higher when having powerwall support (M = 5.8; p <.01). Page 39 of 105

40 Negotiation style: Altogether, the negotiation style was rated better in face-to-face settings, although differences did not reach the conventional level of significance. Participants reported that individual interests were considered to a higher degree (M F2F = 6.2 vs. M CMC = 5.1), the communication was better coordinated (M F2F = 5.6 vs. M CMC = 4.5), the participants felt that each agent and had a better chance to communicated his interests (M F2F = 6.4 vs. M CMC = 5.8), that each agent participated more equally in the communication (M F2F = 6.2 vs. M CMC = 5.2), and that negotiation was fairer (M F2F = 6.6 vs. M CMC = 6.0). Furthermore, a positive effect of the powerwall was found for the participants impression that they had the chance to appropriately communicate their own interests (M PW = 6.4 vs. M NoPW = 5.9). Finally, in working conditions without powerwall and no direct communication (CMC) constructiveness and fact orientation were rated worst. Workload: Participants indicated that is was less strenuous to identify potential conflicts when working computer-mediated (M F2F = 4.3 vs. M CMC = 3.2). However, this effect might be due to learning, since participants worked face-to-face in the first two trials and computer-mediated in trial 3 and 4. Moreover, it was reported that it was easier to communicate and negotiate with each other when having support by the powerwall (M PW = 3.5 vs. M NoPW = 4.4). Usability: Analyzing the usability items of the post-trial questionnaire revealed a consistent result pattern: The ratings were highest when working with a powerwall in face-to-face settings and worst when working without a powerwall in face-to-face setting. No such effect was evident for the conditions with computer-mediated communication. This effect might be explained by the fact that when working with the chat system, all information exchanged is well documented for each agent. Moreover, agents might be fixated on their individual clients when exchanging information via the chat tool. However, when having the opportunity to discuss the information in face-to-face settings, information about the negotiation displayed on a common screen seems to be helpful to synchronize actions (M PW = 5.5 vs. M NoPW = 3.8): Furthermore, the system rated as providing sufficient information about the negotiation status (M PW = 5.8 vs. M NoPW = 4.0) as well as process (M PW = 5.3 vs. M NoPW = 3.3) and the individual solutions (M PW = 5.0 vs. M NoPW = 4.0). OVERALL USABILITY Items (1-7 scales) I felt that the system was useful 5,5 Mean The system was understandable 4,5 The system worked accurately 5,7 I was confident when working with the system 3,5 The system provided a good support of my work 4,3 The system drew my attention from the actual task 4,3 OVERALL WORKLOAD How much effort did it take to access relevant information? 3,5 How much effort did it take to follow the information on the displays? 5,0 How much effort did it take to manage the output devices? 3,3 How much effort did it take to understand all information displayed by the system? 4,5 How much effort did it take to communicate and negotiate with the other agents? 3,5 Page 40 of 105

41 Items (1-7 scales) How much effort did it take to align verbal communication and system information? 3,3 Mean Table 16: Items and Means (Post-Experimental Ratings) In the final questionnaire, participants rated the overall usability and experienced workload. In sum, the ratings indicated that participants felt that the system was useful and worked accurately. However, the rather moderate values for comprehensibility of the system, confidence when working the system, and perceived system support can be attributed to the high degree of system complexity. The data for workload are also consistent with these findings. With the current system it was too demanding to monitor and understand the displayed information. Thus, system usability should be improved in the next design cycles, because it potentially draws too much attention from the negotiation task. 4.2 CONFIDENCE IN EXPERIMENT RESULTS Quality of Results of Experiment Two two-person teams participated in the gaming exercise. Consequently, the study had an explorative character and cannot easily be generalized. Yet, in additional laboratory studies (Weber & Papenfuß, 2009 [34]), the main results for the different communication modes could be replicated with a larger sample (N = 40) and under controlled experimental conditions. Nevertheless, it is mandatory to investigate TAM-negotiations with larger teams, for longer periods, and more complex negotiation tasks. As noted above, preliminary studies yielded significant learning effects, although participants were thoroughly instructed and well trained. These training effects have to be considered when testing the hypotheses and interpreting the objective and subjective results. We did not control for order effects, because it would have been to demanding for the participants to start the gaming exercise without the possibility to communicate with the moderator and the other agents directly. Thus, the results may be biased by order and time effects (like learning). Finally, there was not one best solution for all scenarios, i.e., there is no objective criterion to assess the teams negotiation result in these cases Significance of Results of Experiment Statistical Relevance The number of participants was N = 4. Therefore, most results were analysed on descriptive level. Besides, the operational realism is limited, because participants negotiated only one specific use case (Adjustment of A/D ratio) in two situations (closed runway and reduced capacity). A larger set of more realistic negotiation tasks would be necessary to evaluate the usability of the technical system and the structural conditions on the negotiation process and results. The results obtained from the gaming exercise using the ACCES facility match well with results obtained from the laboratory study mentioned above ([34]). This indicates that the experimental setup is basically sound. Operational Relevance Page 41 of 105

42 The gaming exercise provided the actors with a sufficiently realistic feel so that the negotiation was taken seriously, despite the reduced scope of the scenario and only two actors being involved. Hence the results may be viewed as operationally relevant with respect to the main objective (analysing the negotiation process taking place in the APOC). The evidence provided regarding certain aspects of the negotiation process (use of powerwall, face-to-face vs. computer mediated communication) is viewed as relevant for the design of the future APOC. Page 42 of 105

43 5 ANALYSIS OF EXPERIMENT OUTCOMES This section provides an analysis of the experiment results with respect to the hypotheses and high level objectives stated earlier. It is mainly focussed on the gaming part of the exercise, but for completeness the findings of the Network Management Modelling part are summarized here also. These are given in more detail in Annex. 5.1 ANALYSIS OF OUTCOMES ON THE BASIS OF DETERMINED HYPOTHESES In this section the results given in Section 4 are analysed with respect to the rephrased hypotheses stated in Section Each hypothesis is stated followed by the results providing evidence to accept or reject it. As stated above in Section the statistical relevance of the results is very limited, and hence the evidence provided can only be viewed as an indicator towards accepting or rejecting the hypotheses. H1: Sharing all relevant data within the AOP ensures a high level of situational awareness of the involved stakeholders. The answers provided to the questionnaires after each run indicate a good situation awareness (M=5.44 on a scale from 1-7). This may be attributed to sharing the same data using the APOC database and hence indicates that a common information platform provides a good situation awareness of the involved stakeholders. H2: Collaborative airport planning allows stakeholders to agree on a set of performance parameters for the airport to deal with a forecast problem situation. Table 12 shows that the gaming participants found a consensus solution in all experimental conditions, with only one exception resulting from a user error operating the negotiation support tool. Hence evidence has been provided to accept the hypotheses. H3a: A common information platform (powerwall) improves the situation awareness of the involved stakeholders In addition the stakeholders reported higher situation awareness when powerwall support was provided. This may be attributed to the fact that the powerwall design allowed them to view the planning inputs of the other stakeholder and also provided an overview of the negotiation process. H3b: A common information platform has a positive effect on the negotiation process Participants indicated that they could better communicate their interests and that negotiation was more constructive and fact oriented when working with the powerwall. H4: Direct face-to-face communication has a positive effect on the negotiation process Agents exchanged more task-related information when working in a conventional face-to-face setting; According to participants ratings, face-to-face communication had a positive influence on situation awareness and negotiation style; Participants reported that they were less content with the common solutions. Page 43 of 105

44 Hence it can be concluded that the modified hypotheses stated in Section are supported by the evidence produced by the gaming exercise. It should be noted again that the hypothesis H3 as defined in the Experimental Plan (Airport stakeholders are able to produce a solution that is better than a do-nothing solution by applying collaborative planning (high level objective 2) could not be addressed. The findings from the Network Management Modelling experiment are described in detail in Section In summary the experiment demonstrated that the network modelling of a European Core Network was successful. The interfacing of the network modelling with the gaming could however not be analysed within the scope of the experiment. 5.2 ANALYSIS OF CONSEQUENCES OF OUTCOMES FOR EXPERIMENT OBJECTIVES AND ASSUMPTIONS This section relates the accepted or rejected hypotheses with the corresponding high-level and low-level objectives and will therefore report on the consequences of the evidence for reaching the experiment objectives, i.e. it must be determined which parts of the experiments were successful and which parts might need further study. The first and main objective was to study and assess the process of collaborative decision making in the APOC. It was to be studied how the actors in the APOC reach an agreed solution to a problematic situation, what information they require for their decision and how this decision making can be supported by specific tools. The gaming exercise succeeded in providing results related to the decision making progress and also use of tool support. Evidence is provided that a common situation overview display (powerwall) can increase the situation awareness of the actors and that face to face communication (centralized APOC) is preferred over computer mediated communication (decentralized APOC). It has also been demonstrated by using a prototype developed by DLR how a negotiation support tool can be used to facilitate and structure the negotiation process. It was however not possible to analyse the information needs of the stakeholder due to the reduced capabilities of the gaming platform compared to the original planning. The second objective, to provide an initial performance assessment of the Collaborative Planning process in the APOC, could also not be realized with this experiment. This is again due to the platform development being slower than anticipated and therefore the simulation and scenario setup was not realistic enough to allow a performance evaluation. The third objective, relating to the Network Management Modelling part of the exercise, could also be achieved only partly (see Section 8.5.2). A network model representing the core area of Europe has been created successfully. It was however not possible to interface this with the gaming exercise within the scope of the current project and further research is needed here. It has been demonstrated that the Network Management Model is capable of producing data that can help the actors in the gaming exercise understand the consequences of their planning on the network, and it is anticipated that future projects will further analyse this. Finally the gaming exercise was aimed at gaining more understanding into the use of gaming and the use of negotiation support tools for future research in the Total Airport Management area. This objective has been reached even with the reduced platform capabilities at the time of the experiment. It is anticipated that the knowledge gained here will be highly useful to future projects looking into collaborative planning processes. Page 44 of 105

45 6 CONCLUSIONS AND RECOMMENDATIONS This section provides conclusions and recommendations pertaining to the overall experiment, consisting of the gaming and the network modelling part. It is mainly focussed on the gaming part of the exercise, but for completeness the findings of the Network Management Modelling part are summarized here also. These are given in more detail in the Annex (Section 8). 6.1 KEY FINDINGS The gaming experiment succeeded in showing that a collaborative planning process supported by negotiation support tools can provide viable action plans in adverse conditions such as capacity shortfalls. Involving all actors in the decision increases their situation awareness, especially regarding the view of their counterparts, and also increases their commitment to achieve the planned results. Some aspects of a future APOC have been analysed. It was shown that a common overview display (e.g. powerwall) provides a noticeable benefit and also that it helps to have the actors working together in a single room allowing direct face to face communication. It also became very obvious, especially from the discussions in the debriefing sessions, that gaming itself provides an excellent means to study future concepts. It is very difficult for domain experts, who are normally experts in today s operations rather than in a future concept, to envision, discuss and find the weak spots in future concepts based on documents. Gaming provides a means for them to put themselves into a situation where the concept becomes meaningful and it makes it much easier for them to understand and evaluate the concept. For this to work however a high degree of realism is essential and the combination with planning tools and simulations can provide this realism. At the network level it could be demonstrated that an aggregated and simplified network model for the core area can provide useful and realistic data that can be used in future gaming experiments. 6.2 ISSUES The network modelling part of the experiment demonstrated that constraints coming from the network may have a significant influence on the local planning at the airport level. At the same time the planning results from local airports may again have a significant impact on the network. It becomes apparent that the links between planning at the network and airport level need to be strengthened to achieve the full benefits of collaborative planning. This needs to be developed further at the concept level and supported by corresponding validation exercises. 6.3 RECOMMENDATIONS The gaming experiment provided some very useful and interesting data regarding the negotiation process, even though the scope of the experiment had to be reduced from the original planning. This shows that gaming itself and the ACCES platform, once the full simulation suite and tool support is implemented, will provide a highly valuable validation platform for collaborative planning concepts. It will be very important, however, to develop proper and realistic simulation scenarios based on several of the scenarios defined by the EPISODE-3 project, to study the decision making in a much broader scope. Exercises should be set up to cover most of the process steps defined in the Collaborative (M1) DoD [7] for validation. There may also be concept elements within the Collaborative DoD that cannot be validated by gaming. Page 45 of 105

46 It is recommended to build on the experience gained in this exercise and to make use of it in the definition and validation of the APOC during the SESAR JU work. At the network level it became apparent that the findings and unresolved issues are strongly suggesting that the interest is much wider than supporting an experiment. It is recommended therefore to address the main topics of the study in a more application focused way. Due to the limitations of the platform at the time of the experiment the gaming exercise had a very explorative character. It opened up the investigation of the collaborative decision process at the airport using gaming techniques and produced some initial findings of a more general nature (e.g. centralised/decentralised APOC, effect of large information display, experience with gaming). It was not possible at this stage to fully evaluate operational scenarios as described in the DoDs and therefore no feedback to the DoDs or Operational Scenario documents was produced. 6.4 CONCLUSIONS This section summarizes the findings and issues per objective and thus the major results of the experiment. O1: The gaming experiment succeeded in showing that a collaborative planning process supported by negotiation support tools can provide viable action plans in adverse conditions. Involving all actors in the decision increases their situation awareness, especially regarding the view of their counterparts, and also increases their commitment to achieve the planned results. The results of the gaming indicate that a central APOC (all agents in one room) is preferable over a decentralized APOC (agents in stakeholders offices and connected via computer mediated communication). The agents participating in the exercise reported better situational awareness in the centralized APOC case and analysis of their communication showed more task oriented communication than in the decentralized case. Further experiments should be performed when the ACCES platform is fully ready to also analyse the information needs of the actors in a broader range of situations. O2: The network management modelling activity succeeded in providing a useful model of the core network and to show that such a model can provide constraint data that can be used by the actors at a local airport in their decision making process. Further studies will be needed to investigate how such a tool may be used to predict network effects of local decisions and how to integrate such a tool into the what-if probe of the negotiation support tool. For initial validation activities both at the local airport level as well as the network level both sides can be investigated independently without looking very closely at the coupling. This means that for example collaborative planning at the airport planning can be studied with the response from the network to local planning requests replaced by simple scripts rather than a complex model. In the long term however, as realism of the exercises increases, it will become important to perform coupled experiments with local airport planning as well as network level planning, and the two parts of this exercise can form a good basis for this. The experiment opened up the discussion on how local collaborative planning processes at a single airport may be connected with the planning processes at the network level. While this connection could not be established in the experiment, it provided some ideas how the two parts may be linked together. The main concept is depicted in Figure 7 (section ). The planning at the network level provides bunching and constraint information describing the situation at the network level. This information can be made available to the stakeholder agents in the APOC and also Page 46 of 105

47 used by the planning support tool, so that the local planning will take network constraints into account. On the other hand, during a simulation run, the changes in SBTs produced by local decisions at the airport may be fed back into the network model to determine the effects of the local decisions on the surrounding network. The mechanisms how this feedback may be accomplished will still have to be developed. O4: Gaming has been demonstrated to be very useful both in concept clarification as well as validation. It provides experts a very intuitive way to grasp the consequences of a new concept and to detect weak and strong points. The experience from the exercise also shows the importance of training for the execution of the experiments. The investigated concept is very complex, and support tool used in the exercise as well as gaming itself is new to the participants. In the current experiment the participants have been briefed and trained for about half a day before the actual exercise, and this was barely enough. It also became obvious that a high degree of realism, both in the environment (gaming facility) and also in the scenario, is required to get optimal results from the studies. Page 47 of 105

48 7 REFERENCES AND APPLICABLE DOCUMENTS [1] Episode3 DoW, version 3.1, July [2] E-OCVM European Operational Concept Validation Methodology - Approved Version 2, 17/03/2007 [3] Episode 3 Guidelines for E-OCVM steps , D [4] Episode 3 Guidelines for E-OCVM steps , D [5] Episode 3 Performance Framework, D [6] Episode 3 Collaborative Planning WP3 Validation Strategy, D [7] Episode 3 SESAR DOD M1 - Collaborative Detailed Operational Description, D [8] Episode 3 SESAR DOD M2 - Medium/Short Term Planning Detailed Operational Description, D [9] Episode 3 Validation Themes, Andy Barff, EEC - EP3 WP2 RMC2.5 - Version 0.1, [10] SESAR The SESAR Performance Booklet, RPT [11] SESAR D1: The Current Situation [12] SESAR D2: Air Transport Framework, The Performance Target [13] SESAR D3: The ATM Target Concept [14] SESAR D4: ATM Deployment Sequence [15] SESAR Concept of Operations [16] SESAR Scenarios illustrating the SESAR CONOPS DLT [17] SESAR Analysis of the air transport value chain, Task Deliverable: DLT _T111_D1 - Draft [18] SESAR Identification of limits/blocking points for airport environment, Task Deliverable: DLT _T322_D1 - Draft [19] SESAR Performance Assessment Task Report Capacity and Quality of Service - Version 00.04, 04 June 2007 [20] SESAR Performance Objectives and Targets RPT Draft [21] EU 6th Framework Programme Cooperative Air Traffic Management Phase 1: D3.1.3 Validation Plan, Jan Approved Version 1.2, [22] EUROCONTROL Study Report Challenges to Growth, EUROCONTROL, Approved Version 1.0, [23] EUROCONTROL Long-Term Forecast Flight Movements Version 1.0, [24] EUROCONTROL Total Airport Management (Operational Concept and Logical Architecture), Version 1.0, DLR, EUROCONTROL, 2006 Page 48 of 105

49 [25] Kirwan, B. & Ainsworth, L. K. (Eds.) (1992). A guide to task analysis. London, UK: Taylor & Francis. [26] Driskell, J.E. and Mullen, B. (2005), 'Social Network Analysis', in N.A. Stanton et. Al (eds), Handbook of Human Factors and Ergonomics Methods, pp , London: CRC [27] SESAR Compute and Map operational Concepts & Airspace KPIs Based on Identified Available Tools and methodologies, SESAR Definition Phase, Task 2.3.1, DLT , Status: Validated, Confidential, Version 0.09, EUROCONTROL, Brétigny, September [28] NLR, H.W.G. de Jonge, R.R. Seljee, and J.N.P. Beers, Demand and Capacity Balancing and Collaborative Flow Management in Europe, Performance Assessment Study by fast-time simulation, NLR-CR , Amsterdam, October [29] Episode 3 Experiment Plan for Gaming Exercise on Collaborative Airport Planning, D , Version V1.00 [30] Holsti (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison-Wesley. [31] Bales, R. F. (1950). A set of categories for the analysis of small group interaction. American Sociological Review, 15, [32] EUROCONTROL (2008). The SHAPE Questionnaires, Retrieved from s.html [33] Stempfle, J. & Badke-Schaub, P. (2002). Thinking in design teams An analysis of team communication. Design Studies, 23, [34] Weber, B. & Papenfuß, A. (2009, October). Cooperative decision processes in air traffic management: The influence of situational and dispositional factors. 51th Committee Meeting Anthropotechnik, Brunswick, Germany. [35] SESAR WP2.2.4 Baseline Operational description for the Mid-term Annex with Operational Improvements - DLT v0.06 [36] SESAR Integration of European ATM Initiatives & Programmes Volume 1. WP3.1 DLT X Draft Version [37] EU 6 th Framework Programme Cooperative Air Traffic Management Phase 1: D3.1.3 Validation Plan,, Jan Approved Version 1.2, [38] EUROCONTROL Note No. 17/06 CDG Real-time simulation results - Published November 2006 [39] Episode 3 Episode3 Exercise Report Template D2.5-03, Version V1.00, 8/04/2009 [40] EUROCONTROL: Study Report Challenges to Growth, EUROCONTROL, Approved Version 1.0, [41] Episode 3 EP3 WP2 EUROCONTROL, SESAR DOD Collaborative Airport Planning M1, E3-D2.2-M2, Version 0.32, Brétigny, December [42] Episode 3 EP3 WP2, EUROCONTROL, SESAR DOD Medium/Short Term Network Planning M2, E3-D2.2-M2, Version 0.33, Brétigny, December Page 49 of 105

50 8 ANNEX: NETWORK MANAGEMENT MODELLING IN SUPPORT OF GAMING 8.1 INTRODUCTION Purpose of the ANNEX This ANNEX provides results of the NLR part 2 of the Validation Exercise of EP3 WP3.3.4 Collaborative. DLR performed a Gaming Exercise and NLR supported this exercise by emulating a Kernel Network of ATM in Europe. This off-line process emulated a Central Network Management process by managing the ATM performance of this network by Demand & Capacity Balancing (DCB). A simplified Network Operations Plan (NOP) of the Kernel Network of the planning of ECACwide flight operations was modelled and processed. The model can be used to provide data that may be used in the collaborative planning process at the airport level: To provide bunching information to APOC for those flights that have a risk to be delayed and that will need high attention from network operations perspective; To provide proposed departure constraints for those flights that are constrained by network restrictions; and To process prioritisation preferences generated by the Gaming exercise and applicable to the airport of interest, i.e. Hamburg. Also, other airports of the Kernel Network maybe emulated to provide similar proposed constraining preferences to the network and to evaluate their effect. The objective of processing this model was to create the network management environment for the exercised hub airport that acts realistically regarding the external constraints imposed on collaborative planning at airport level. Originally it was planned to investigate the use of the network modelling data in the gaming exercise and to also investigate the effects of the local planning decisions at the airport on the surrounding network. However, this analysis was not possible in the scope of this exercise. Nevertheless, it is foreseen and deemed possible to run coupled exercises in the future, for which the NLR Kernel Network model may serve as a background service and feeder process to the Gaming exercise performed by DLR. This report on simulation results describes the concept and context of operations, the set-up and conduct of the experiment, as well as results obtained, their analysis, and some conclusions and recommendations Background With the increase of air traffic in Europe, airports are becoming a major bottleneck in Air Traffic Control (ATC) operations. In EUROCONTROL s Challenge to Growth study [40], EUROCONTROL suggests that, under the most optimistic of circumstances, existing airport capacity in Europe is capable of absorbing a maximum of twice the traffic demand of Other studies by Episode3 and SESAR, RMC 2.5 Validation Themes [9], and SESAR Performance Assessment Task Report [19], suggest a (maximum) traffic growth rate of between 4% & 5% of air traffic demand per annum can be expected through the years up to At these rates, a total capacity barrier would be reached around Noting that this includes capacity filling at regional airports as well as current major hub airports, it is Page 50 of 105

51 reasonable to assume that the practical capacity barrier will be reached well before the theoretical barrier, on the condition of persistent unconstrained growth. The Single European Sky (SES) launched by the European Commission was drafted with the following objectives, according to their website: to restructure European airspace as a function of air traffic flows, rather than according to national borders; to create additional capacity; and to increase the overall efficiency of the air traffic management system. Within the SES context, the Single European Sky ATM Research (SESAR) programme will deliver the technology and research necessary to re-engineer the fragmented European ATM network and achieve these objectives. The purpose of the Episode3 project is to undertake a first step in validation and performance assessment of the concept of operations expressed by SESAR Task [15] and consolidated in SESAR D3 [13]. Thus, Episode3 performs a key role in the SES Implementation Support by validating the concept s ability to deliver the defined performance benefits in the 2020 time horizon. The validation process as applied in Episode3 is based on the European Operational Concept Validation Methodology (E-OCVM) [3], which describes an approach to ATM Concept validation. However, to date the E-OCVM has not been applied to validation of a concept on the scale and complexity of SESAR. Such a validation assessment at system-wide level must be constructed from data derived from a wide range of different validation activities, integrating different levels of operational, procedural, functional and system descriptions. Also, validation has to address different operational segments and contexts as well as several planning horizons. The data will be collected through a variety of methods and tools and will vary in quality, validity and reliability. The process of performing systematic validation and integration of results must be actively planned and managed from the beginning. Management of the validation process is coordinated in the Episode3 project by EP3 WP2.3, which workpackage is responsible for ensuring the effective application of E-OCVM, consolidation of the Episode3 Validation Strategy, and the establishment of a Validation Framework. This will allow the integration of validation results and the construction of the necessary system-level view. Validation exercises should produce evidence, preferably measured, about the ability of some elements of the concept to deliver the expected performance targets for specifically identified Key Performance Areas (KPAs). In order to be able to perform Validation Exercises, there is a need for concept clarification, requirements development and further concepts elaboration activities in preparation of a progressive completion of the concept s validation process. The experimental plan of the experiment on Collaborative is based on an E- OCVM validation scheme in-line with the Validation strategy, collaboratively produced by EP3 WP2.3 and the supportive Tasks within EP3 WP 3, 4 & 5. Complementary guidance material has been provided by EP3 WP2.3.4 to allow execution of E-OCVM Step 2 [3]. The experiment has been performed in two parts: The first part of the experiment is an interoperable Gaming Exercise, performed by DLR, to experience the interactive process of negotiating favourable departure sequencing and the planning of constrained flights under departure-conflicting conditions. The result was assumed to be an optimised compromise from the perspective of the Airline s User operational plan, and this result was obtained by full participation of Airline Operators in the planning process. The scope of this exercise was to accomplish flight departure planning during 1 to 2 hours at one congested airport. Page 51 of 105

52 The second part of the experiment was a modelling exercise to exchange information with the Gaming exercise by off-line processing of network managed information, being relevant for collaborative planning at airport level. The information provided to the Gaming exercise is information that concerns departure flights, prepared, planned and monitored for departure operations at the airport of interest during the Gaming exercise period. The Network Management modelling process took care to provide timely information for those flights that are departing from the airport of interest and that are involved in bunching traffic conditions of any node of the Network. On the other hand, the model was prepared to receive departure planning preferences and was reacting by re-processing the throughput through the network and by adapting proposed departure constraints. The scope of this exercise was to process management constraints through the selected Kernel Network during a 24-hours period and to support the Gaming exercise with realistically emulated planning information derived from the processing of network management functions. This part of the report addresses the second part, i.e. the modelled process to emulate Central Network Management (CNM) Intended Audience The intended audience is primarily to inform the Commission, the SJU and the Episode3 project members on the Gaming exercise on airport planning, as well as the experimental background experiment on building a Central Network Management/Demand and Capacity Balancing (CNM/DCB) model. The document is public to make information available to the outside world as well and to foster the dissemination objectives of the project Structure of the Annex The annex is structured in four main parts. The following parts are described: The Introduction (Section 8.1); The Exercise Scope and Execution (Section 8.2); Results and Discussion (Section 8.4); Analysis, Conclusions and Recommendations (Section 8.5 and 8.6). The document is completed by a list of References (Section 8.7) and some Annexes with details on experimental data and processing results. 8.2 EXERCISE SCOPE AND EXECUTION Stakeholders and their expectations The most important stakeholders concerning airport operations are the airport and airspace users and their requirements as expressed in SESAR D2. Representatives of these stakeholders are involved in the experiment. The following stakeholders are involved in the decision making process by the Airport Operations Centre (APOC): Representatives of the major airlines at the airport, The Airport Operator; The local Air Navigation Service Provide (ANSP); Airport ATC; and Page 52 of 105

53 The Central Network Management representative performing Central Network Management (CNM) functions. This Episode3 WP3 Experiment concerned a Gaming exercise on planning of departure traffic. This planning process is accomplished by actors, representing the stakeholders as much as possible as an operation planning process that takes place at present, 2009, during the timeframe and within the scope of activities of the Gaming exercise. The following actors are representing the stakeholders: Airlines are represented by flight dispatchers and a Central Network Management coordinator, acting in real-life at a Flight Operations Centre; The Airport Operator is represented by actors involved in flight planning in the APOC. The actors are responsible for gates and stands management and planning, as well as the planning and coordination of service provision to each flight; The ANSP is represented by a planner on departure operations; Airport ATC is represented by a planner of surface movement operations. At some airports this service provider is the same as the ANSP, responsible for departure planning. Also, the scope and area of responsibility can be different for different airports; The Central Network Management representative is today, 2009, the Central Flow Management Unit (CFMU). In the future, this is foreseen to become a role of Network Operations Planners. This actor coordinates with ANSPs on sufficient capacity for the required level of service provision, and manages the throughput through the network by balancing demand and capacity, i.e. by Demand and Capacity Balancing (DCB). This actor is represented by this process, possibly assisted by human interoperable support, to manage the DCB process throughout the ECAC-wide network. All these actors are participating in the Gaming exercise to simulate the process at APOC level. This (part of) the results report describes the role and participation of the Central Network Management (CNM) actor. His actions are represented by a modelled process to simulate CNM functions and their interoperability with APOC. In the table below, the role of each stakeholder is described in more detail. SESAR-external stakeholders such as passengers and the community around the airport are not taken into account during the exercise, and therefore their interest is not addressed here. Because this part of the report deals with CNM functionality, the Stakeholder expectation table addresses the interests of stakeholders in the Central Network Management process, experienced in the network modelling part of the experiment. Stakeholder Involvement Why it matters to stakeholder Performance expectations Airport Operator Airport Operators have participated in the EP3 WP3.3.1 Expert Group on Collaborative Airport Planning. - Airport capacity is the key challenge in the SESAR timeframe. The airport operator is highly interested in making best use of available resources (even to reduce overcapacities) and to ensure smooth and predictable operations of the airport as a whole. Due to improved predictability available resources might be used in an optimized way, resulting in a probably higher throughput in reduced capacity situations and an earlier Objectives (amongst others): - Improved collaboration in the decision making process: more stakeholders involved in deciding how to respond to these conditions. - Improved situational awareness and better use of available resources. - improved average punctuality Page 53 of 105

54 Stakeholder Involvement Why it matters to stakeholder Performance expectations Airlines An airline agent was involved in the exercise. recovery to normal situations. With CDM the airport operator gets the opportunity to understand needs and attitudes of other stakeholder in a better way and is able to participate in negotiation of different parameters, also in parameters he is not responsible for. CNM is one of the actors providing information beyond control of the Airport Operator, but directly impacting the operations at airport level. Airlines are customer of the Airport and the ANSPs. Due to that fact they are highly interested in transparency in all information having an influence on their flights. The exercise investigates how airlines as a stakeholder can play a direct role in the decision making process in the APOC. Airlines can thus directly influence decisions to mitigate unforeseen conditions. Due to improved predictability available resources might be used in an optimized way, resulting in a probably higher throughput in reduced capacity situations and an earlier recovery to normal situations. Network Management: Accommodating more traffic and flying more scheduled traffic as planned is made possible by performing DCB at network level. Planning at airport level has to be considered therefore as an integrated part of network management. - improved stability of operations - less impact on environment (i.e. decreased number of holdings) Network Management: Reduced queuing times for both arrivals and departures as a result of smoothing traffic flows through a collaborative and optimised network management process. Critical role: The critical role of APOC is to facilitate operations as planned. The airport has an enabling role in adherence to the departure planning and is therefore a critical actor in predictable and undisrupted performance of ATM. The NOP makes constraining information available by CNM, that is critical for APOC planning operations and that has to be taken into account during collaborative planning. Objectives: - Through an APOC the Airline gets the opportunity to plan own flights more accurate. They can benefit from a higher stability of own operations and of the whole airport system and enhanced efficiency of fleet operations. - With less fuel burn to improve cost effectiveness and reduce environmental impact. - Improvement of predictability (punctuality). Network Management: More cost-effective operation through accommodation of user preferences by prioritisation. The concern is if priorities can be acknowledged without a negative impact on network performance. Critical role: Increased throughput and efficient utilisation of available runway capacity (especially in adverse conditions). The concern is if local airport decision making fits with network management constraints. There is a potential risk to create Page 54 of 105

55 Stakeholder Involvement Why it matters to stakeholder Performance expectations instability. ATC/ANSPs Research and development centres An ATC agent was involved in the exercise. Represented by EUROCONTR OL/DLR as exercise leader ANSPs are service provides and fulfil the function of guidance and control of airplanes. Due to that role they are interested in satisfying wishes of their customers (according to their possibilities). ANSPs are responsible amongst others for the runway utilization and the ARR/DEP ratio at the airport. It will be in the interest of the customers and ANSP to collaboratively decide about the right setting of these parameters. ANSPs play also an important role in the decision making process in the APOC by focussing on the effects of proposed decisions on the demand and capacity balancing of the airspace. Network Management: By balancing demand and capacity bottlenecks are decreased, throughput increased and workload is reduced. - Deeper knowledge of strategies to collaboratively reach decisions in the APOC. - Deeper knowledge on the effects of these airport-centred decisions on the surrounding airspace and airport network. Objectives: Less fuel burn and to improve reduce environmental impact. Improved predictability might lead to an optimized usage of available resources. Network Management: Increased knowledge of the impact and interdependency between airport and network capacity. A DCB process that limits variability of demand reduces the controller load and allows to perform traffic synchronisation in a most effective way. A DCB process allows to ensure safety by accommodating a maximum acceptable amount of traffic demand. - Knowledge of the areas where more research is needed to increase both airport and network capacity. Table 17: Stakeholder expectations Description of ATM concept being addressed In this section, the experiment is linked to the Episode3 Detailed Operational Descriptions (DODs), [42], and the Operational Concept of SESAR, [15]. The following topics are addressed: The scope of the operational concept of interest, providing links to Lines of Changes (LoCs) and Operational Improvements (OIs); A detailed outline of the operational concept, derived from the DODs; The level of maturity of the concept of interest; and The Key Performance Areas (KPAs) related to the concept of interest, giving indications of relevant areas of potential benefits and performance assessment. Airports are always part of an ATM network. This is what Total Airport Management entails: managing airport operations as part of the totality of connected airports and airspace sectors. In this way, TAM is expected to lead to better use of available capacity at an airport, since the external constraining factors are taken into account as well. In the experiment, decisions Page 55 of 105

56 taken at airport level are therefore also to be analysed in terms of their consequences at network level. Solutions that may seem good for the airport may lead to large imbalances in capacity and demand in the airspace network as a whole. Therefore, planning at airport level has to take into account the management and planning at network level. The CNM modelling experiment addressed the following elements, putting airport planning processes within the context of the NOP: Improving the interoperability between Network Capacity Management Processes and planning processes through Network Management at airport level: o o By issuing proposed departure constraints applicable to flights departing from the airport of interest; and By processing preferences evaluated at airport level, and accepting such preferences as prioritisation constraints. Monitoring ATM Performance through Network Performance Assessment and providing alerting information on bunching and/or congestion to planning processes at airport level Scope of the Operational Concept of Interest The list of OI steps that have been addressed determines the operational scope of interest of the Experiment. Annex 8.7 lists and justifies the Operational Improvements related to this scope of interest. The applicable OIs are: L10-03 AO-0601: Improving Airport Collaboration in the Pre-Departure Phase; L03-01 DCB-0102: Collaborative Layered Planning Supported by Network Operations Plan; L04-01 DCB-0206: Improving Network Capacity Management Processes; L04-02 SDM-0101: Monitoring ATM Performance. The following figure, Figure 7, shows the interoperability between Actors at Airport level and their interoperability with actors at Central Network Management level. Page 56 of 105

57 Figure 7: Scope and Context of operations of APOC and CNM Detailed outline of the Operational Concept of Interest Below, an outline is given of that part of the Operational Concept on which the experiment is based and that addresses the Central Network Management functionality. This outline summarises applicable concept elements from the Episode3 DODs, in particular DOD M1 and M2, [7][41] and [8]. The conceptual overview is aligned with SESAR, and presents an overview with the following purpose: The experiment cannot be understood without understanding the Operational Concept, this overview provides a minimal summary of the concepts addressed; A systematic justification of conceptual improvements helps to support an understanding of the validation process; The present description is focused on direct applicability and performance assessment. Sections and describe how the experiment addresses this operational concept. A detailed experiment description is given in the sections and 8.3. Management and planning of flight operations is derived from the Shared Business Trajectories (SBTs), the 4D planned flight plans submitted by Airline Operators. Constraints management on these SBTs is based on the monitoring of constraining conditions in the central network of operations as well as constraints on flight preparation and departure planning at airport level. The results are filed as Reference Business Trajectories (RBTs) in the NOP. The Gaming exercise has been dealing with decision making due to CDM at Airport level, and took place in the Airport Operations Centre (APOC) by an interoperable process of the participating stakeholders. The constraints at central network level are provided as input to this CDM process, and the output of the CDM process should feed and influence the management process at the level of central network management. Page 57 of 105

58 The off-line modelling part of the experiment, performed by NLR, addressed the interactions and connectivity between the airport under investigation and its surrounding network. This network represents the NOP or part of the NOP and consists of a representation of routings, airspace sectors and airports. In addition, with other information of the NOP, representing the planning of air traffic by RBTs, this specifies the ATM network and how to deploy this network. To determine the mutual interactions and consequences of collaborative decision making at airport level and the surrounding network, the operations of the following conceptual elements are important: Demand and Capacity Balancing In SESAR, the Network Operations Plan (NOP) is central to the concept of operations for large airports and their environment. In the NOP, a planning of operations is available that is converging in level of confidence, level of detail and quality of planning towards the executive phase. From the NOP, the Airport Operations Plan (AOP) is derived: an en-route-to-en-routeconversion of the NOP, enriched by airport specific data. Both departure and arrival operations are expected to follow the AOP, being consistent and in agreement with the NOP, and to behave in compliance with this constantly updated planning, making these operations reliable and predictable. The NOP will comprise information concerning demand, capacity and the proposed and agreed measures to balance demand and capacity. Air Traffic Demand: Air traffic demand is roughly known half a year in advance and traffic demand is known in detail one day in advance. Air traffic demand is submitted as Shared Business Trajectories (SBTs) and agreed the day before operation, to be filed as Reference Business Trajectories (RBTs). Nevertheless, RBTs are submitted at the day of operation as well. Air traffic demand sets the scene for the capacity and throughput requirements of the NOP. In the experiment, the scenarios represented the air traffic demand of one day of operations. Capacity and the Network: Available capacity and throughput limitations are determined firstly by available infrastructure, i.e. by the capacity of Airports. Possible restrictions in airport capacity are found mainly in the major and hub airports. 60% to 70% of flight-executive delays are allocated at the largest 20 airports of Europe, Ref. [27] and [28]. These delays are forthcoming from runway and runway throughput limitations but also from airspace limitations. The large flows of departing and arriving traffic to and from the hub airports are creating a major part of the congestion problems at airport level and in the immediate vicinity of the airport, i.e. the TMA. Therefore, the throughput capacity of airports sets the scene for airspace throughput requirements as well, not only in the TMA, but thereafter also in ETMA and En-route airspace. This completes the capacity requirements of the NOP. Due to the critical role of air traffic to and from the 20 largest airports of Europe, the network connecting these airports can be considered to function as a kernel network. However, such a kernel network has to function also as part of the total network and has to provide connectivity therefore to all relevant airports of Europe. The significance of this kernel network is that it can provide a focused view on critical flows and critical nodes in the network. Page 58 of 105

59 The network through airspace is determined by the routings, which form a fixed route network structure at present (2009) and which will evolve towards a free route network within the SESAR timeframe. Nevertheless, even then there is a de facto network determined by routings, although probably more refined than today. The control on the network by ATC is achieved by sectorisation. This sectorisation is mainly determined by the control capability of one executive controller, and a declared capacity figure is associated with each identified sector. This determines the airspace capacity of the network that serves the routing network, and this determines together with the airports the throughput capacity of the NOP. Such a demand/capacity scenario is selected for the network modelling experiment from an ECAC-wide scenario. Balancing the Network: RBTs, planned by routings through the network, are passing sectors, whilst departing and arriving at an airport. Assuming the Airline to accomplish an optimal planned routing, and allowing the Airline, if necessary, to adapt the routing to his discretion and to his judgement of optimal deployment of operations, the RBTs are accepted for DCB processing, and not to be subject of re-routing at that stage. If business trajectories are planned and re-planned by Airlines, the planned routings and re-routings shall be accommodated. Further requirements are that the DCB processing functions in a predictive and flight-efficient way. A cost-efficient network will not exist without any overload condition and it is the task of DCB to ensure sufficient capacity available but avoiding incidental overloads of the network under these conditions. To provide sufficient capacity to the network depends on the capacity of the individual nodes of the Network, i.e. the capacity of airports and sectors. There are problems concerning the capacity of the Network: o o o Airport nodes are characterised by an hourly capacity figure, but in reality airport capacity is a more variable and operations-dependent quantity. For example, departure or arrival peaks may cause throughput variations; also the mix of weight categories of traffic and incidental weather conditions may cause variations of capacity figures in time. In addition, airports are critical for overload due to inflexible throughput conditions over the runways, and Airports can be characterised by significant differences between sustainable and peak load capacity. Finally, the capacity can be restricted due to local policies, defining a capacity less than the physically achievable capacity. The network modelling process operates (as applied in the experiment) with at most two capacity figures: sustainable and peak load capacity. Sectors are mostly dimensioned by the workload capacity of one executive controller. This is not necessarily the ideal dimensioning of a Network. If there are too many nodes, there is a risk that the Network becomes super-critical for sector overload conditions. The ECAC-wide European Network consists of more than 2000 sectors, which is likely to be beyond the optimum. The optimum can be defined as being a condition where most of the sectors, or at least the sectors of busy parts of the network during busy hours of air traffic operations, are characterised by a low and balanced traffic-load/nodecapacity ratio for all sectors. If not, some form of sector aggregation may help the Network to become more robust. The sectorisation of the Network is not designed together with the infrastructure that is supported by the Network and the varying traffic flows through the Network, determined by the half yearly scheduling. At any times, Page 59 of 105

60 o it will be necessary to evaluate capacity balancing along the applicable routings and through the sectors, used by these routings. Throughput analysis through the Network may lead to local sector capacity adjustments which will become beneficial to overall throughput through the Network. The ideal dimensioned network is as simple as possible, providing just the required connectivity via shortest routings between all nodes of the network. The Network consists of nodes and the ANSPs together with the Central Network Management actors will ensure sufficient capacity available. However, the required demand as well as the available capacity will vary over the day. In addition, there is no reason to assume that demand through congested nodes is balanced and equally distributed over the network and in time by nature. Whatever the capacity adjustments, there will be hotspots and periods of bunches in the network, albeit for reasons of cost-efficiency. The way to equalise traffic demand during bunchy periods is to balance demand as part of the DCB process which may include the application of Air Traffic Flow & Capacity Management (ATFCM) algorithms that determine proposed departure constraints on not yet departed traffic. The way to deal with proposed departure constraints is left to the user, but in order to be able to manage these constraints, the user must be informed about bunching periods and flights involved in these bunches. In addition, the user must be informed about the status of the network in a transparent way that will allow him to adapt proposed constraints on this flight planning. The ATFCM process has to suppress bunches in order to protect sectors and airports against overloads and to justify the declared capacity of sectors and to save airports for queuing problems due to runway access limitations. An ATFCM process can be designed such that proposed constraints are least penalising regarding the proposed departure constraints, and least penalising can be defined then as reaching a proposed planning with a minimum average delay by imposed constraints and a minimum spread of delaying constraints. It should be noted that without the second condition there is a risk that solutions are identified with an optimum that extremely delays some flights for the sake of punctuality of the others, and this is deemed to be judged as a sub-optimal solution. Analysing this ATFCM process, it could be feasible and favourable to select penalising strategies that are not equal towards all users. For example, it can be beneficial for a least penalising strategy to select departure constraints with more priority for highly congested flows and to promulgate expeditious throughput of these flows. The airspace users are allowed to propose flight plan changes on their SBTs/RBTs negotiated and agreed in a collaborative way at airport level. They are allowed also to reject or adapt flight plan changes accomplished at airport level by local agreement. This requires the ATFCM process to be able to deal with these preferences and to adapt constraining conditions in such a way that bunches are still suppressed, whilst user preferences are respected. However, this may yield a degradation of network performance regarding average and spread of proposed departure delays. Network balancing shall be applied during the processing of planned traffic through the network model. This part of the experiment consisted of: o o Defining and selecting an appropriate ATM Network of operations; Analysing and balancing the capacity of this Network; Page 60 of 105

61 o o o Identifying the traffic flows through the Network, and informing the airspace users about hotspots and bunches; Balancing the demand through the selected Network and informing the airspace users at airport level about proposed departure constraints; and Receiving and processing user preferences and accommodating these preferences by prioritising accommodation of the planned target departure times of these flights. Network Management and Design The network is considered here from the point of view of management and control on DCB and throughput analysis only. Network requirements are derived from optimised routings through sectors and airports. This network is based on: City-pair connectivity and to a large extent unconstrained routings; Ideal vertical and lateral profiles to reach the destination in the most fuel-efficient way; Constraints at departure and destination to follow flight profiles which ensure the required capacity around the airport of interest; Constraints that are optimised towards fuel efficiency, but that respect environmental regulations, in particular noise load regulations; A routing structure required to build up manageable traffic flows to and from the airport of interest; All constraints that meet the requirements of other traffic flows and other flight operations and that establish a best compromise for conflicts of interest. As described under the DCB paragraphs, the network consists of airports, i.e. more than 500 ECAC-wide, and sectors, i.e. more than 2000 ECAC-wide. The experience is that this network determined by capacity of executive controller workload, is super-critical; see e.g. SESAR Performance Assessment, Ref. [27] and [28]. To come to a Network that is manageable from the point of view of DCB and throughput analysis there is a need to identify main- and substructures in the network, i.e. the Kernel Network, and to find a strategy to select a less overload-critical structure. Moreover, structuring of the Network can be helpful for better response and enhanced transparency on information provision. The proposed structuring of the network yields: The ECAC-wide area has natural boundary conditions by relatively thin flows of air traffic that leaves and enters the area 1. The experiment selects for reasons of scale a sub-area with less ideal boundary conditions. This area covers part of Western Europe, servicing airport connectivity between 17 large airports of Europe. Airports in Europe are split in groups of hub airports (~20), major airports (~110), and other remaining airports. In the experiment, a network is identified with a subgroup of 17 of these 20 large hub airports and the other airports within this area, 1 A thin flow can be understood here as a flow of air traffic with relative modest impact on traffic pattern compared to other flows, moving through the different nodes of the network under consideration. Page 61 of 105

62 covered by the connectivity between those 17 airports. The remaining other airports are aggregated to a set of cluster airports servicing the geographical area of interest. Experience from the sectorised network learned that there are too many thin flows of air traffic and there is too much unbalance in an effective deployment of the capacity of the network. To improve the performance of the Network, aggregation is applied. This means that the least significant parts of the Network are aggregated with their nearest neighbours. Each thin flow of air traffic over the day is aggregated with its nearest neighbour by: o o o o o The lowest loaded sector/node of the Network is aggregated with its nearest neighbour, simplifying the Network by removing a node. This is repeated in an iterative way until an optimal, transparent and/or useful Network is obtained. The traffic flows through the aggregated node are added to the more significant flows of its nearest neighbour sector/node. The capacity of the aggregated sector is added to the capacity of its nearest neighbour. Summation is acceptable because aggregation is only applicable to the DCB process, and it can be assumed that flows are re-distributed as much as possible over the applicable sectors later on during flight execution in agreement with their RBT, i.e. by Dynamic Flow Management, see DOD M2, ref. [42]. The aggregation of sectors takes place by creation of a super-sector for DCB purposes only, which is done by creation of a synthesised node in the Network on the centre of gravity of the aggregated sectors. The planning of air traffic remains unchanged because all RBTs, i.e. 4D flight plans, are still applicable, not changing any planned waypoint. The only change is the sequence of sectors related to the 4D-planned routing. It is expected that aggregation not only simplifies the Network but also improves the trafficload/capacity ratio. The expected result of selection and aggregation for the experiment is: The experiment addressed a Kernel Network that represents the most relevant areas of hot spots and congestion in Europe in a representative way. Most areas of congestion are part of this DCB process. The selection, together with aggregation, provided a manageable Network which could be experienced with prototyping DCB models without serious and/or prohibitive software performance restrictions. The selected and aggregated Network was expected to be more robust and to require less capacity than the super-critical originally sectorised network. Either the sector capacity can be reduced maintaining equal performance levels, or the departure constraining delays can be reduced operating with unchanged capacity figures. An optimal level of aggregation regarding the DCB process, may serve as an indicator to identify an optimal distribution between centralised and local flow management activities. The experiment constitutes an aggregated Network with feasible and acceptable throughput performance characteristics, which is representative for ECAC-wide Network performance at the same time. Model processing is applied on this Network to feed the airport Gaming exercise with network constraints information, and the network is used thereafter to evaluate the impact of user preferences, agreed at airport level, on the Network. However, it should be Page 62 of 105

63 noted that very likely the impact of user preferences can be evaluated only if all hub airports involved in the Kernel Network are allowed to propose user preferences regarding their departure planning in a similar way. If not, an unbalanced and only partial impact of preferences on the Network is processed. Collaborative Decision Making (CDM) during planning Different CDM options are envisaged during traffic management and planning: In principle, the planning of departure and arrival operations takes place in an equity respecting way, aiming to reach overall optimised punctuality with minimal average delay and minimal spread in delay. Exchange of 4D trajectory planning between AOC, ANSP and APOC may play a role in increasing the level of confidence of reliability, consistency and predictability of the accomplished planning. The interest is to establish a planning: o o o That provides planning consistency in all flight planning and flight preparative offices; That defines a highly efficient planning, servicing punctuality and efficiency of operations; and That is a best compromise between conflicting interests. Airlines preferences and solutions are part of the outcome of the departure planning process, created with monitoring support by centrally managed departure constraints management. Optimisation can be achieved by airspace users and airport operators by selecting alternative routings and by adapting the prioritisation of specific flights Level of Maturity of Concept of Interest Work on performance assessment, undertaken in SESAR Definition Phase, Ref. [27] and [28], made clear that the Network could not be used for performance assessment in an appropriate way without clarifying the notion of capacity and without being more confirmative in assessment of criticality of the capacities available in the network. Lack of control on throughput analysis was the major problem in network performance assessment. Therefore, the emphasis is laid here on analysing the availability of capacity in the network. Concerning the level of maturity of the concept of interest for CNM, the following can be stated: Demand and Capacity Balancing: DCB is a pre-departure process. The process doesn t make use of advanced technology and therefore there are no major transition issues from a technical point of view. Advanced algorithms have to be prototyped, evaluated and validated. However, the most complicated issue might be agreement and operational implementation may be mainly dependent therefore on acceptance by the ATM community, i.e. the stakeholders. From the point of view of institutional and regulatory perspective the concept is therefore immature because detailed and comprehensible outlines of concepts of interest are still missing. The concept is in a V2 to V3 status and there is a need for explorative investigations showing alternatives to address optimisation towards minimal delays and maximum user accessibility. Network Management and Design: Network Management operates today, i.e. 2009, by elaborate operations on ASM and DCB, coordinated, executed and managed by CFMU. All proposed concepts are refinements to Page 63 of 105

64 improve performance, interoperability and user access. There are no principal limitations to implement advanced DCB algorithms and evaluation of robust and optimised structured Networks are achievable as well. Again, stakeholder acceptance is the major issue. The dependency on 4D planning puts the concept in a V2 status, whilst also acceptance is V2 rather than V3. Collaborative Decision Making (CDM) during planning: CDM during departure and arrival management is relatively mature. Parts of this concept are in operation and passed the V5 status KPAs related to the Concept of Interest With reference to ICAO, SESAR has defined a set of 11 Key Performance Areas (KPAs), and within each area a set of Focus Areas (FA) focussing on well defined understandable subjects. This exercise is primarily focused on the KPA Capacity, and more specifically on the Focus Area Airport Capacity. The operations at airports form the ground segment of the business trajectory. The airport throughput is one of the main processes that determine the on-time performance of the Reference Business Trajectory (RBT). In the second part of the exercise, the planning of the airspace segment of the business trajectory was central. In this part, the impact of a collaboratively reached decision at airport level on the overall network surrounding the airport has been studied. Thus, the effect of the airport-centred operational improvements, the focal point of part one, is investigated and quantified in terms of network throughput and punctuality. Furthermore, the following Key Performance Areas are directly relevant to the exercise: KPA Capacity (throughput): This KPA addresses the ability of the ATM system to cope with air traffic demand and to accommodate a maximum number of flights and an optimal distribution through time and space, as tightly as possible in adherence to planning. Related to Network Management, the focus in the gaming exercise is to make effective use of existing capacity. For given traffic demand, the delay can be minimised by collaborative decision making in the APOC, taking into account possible repercussions on the performance of the network. This aim leads to an efficient use of existing capacity, maximizing throughput in the global ATM system. Network Management ensures providing the capacity required by the network. KPA Efficiency (punctuality): The objective of Network Management is to accommodate air traffic as much as possible as planned. The DCB process applied to accomplish this goal, aims to make use of network capacity as efficiently as possible, minimising delays. The collaborative planning process at airport level aims to reach agreement on departure planning amongst stakeholders involved and to express the agreement by planned departure preferences. These preferences are input to the Network Management process to be treated as flight prioritisation indicators. Again the aim of the network management process is to accommodate the extra planning prioritisation preferences as efficiently as possible, i.e. with minimised average delay and minimised spread in delays. KPA Predictability (enhanced quality of planning, i.e. 4D): A consistent planning in 4D, starting from SBTs and converging to planned and agreed RBTs aims to make effective use of available capacity. Adherence to the planning is the objective of executive services and a feasible and reliable 4D planning by agreed RBTs is the way to support executive services in meeting this goal. Simulation of executive processing of planned flights through the Kernel Network is required to assess the Page 64 of 105

65 performance regarding predictability, but this is outside the scope of the Gaming/modelling experiment. KPA Access and Equity/Interoperability: The objective of the gaming exercise is to demonstrate that the Airlines, as the owner of the flights, are benefitting from enhanced access to ATM planning information and may profit of interoperable and collaborative flight planning and preparation processing. The modelling of Network Management is not followed by any quantitative assessment or performance validation, being outside the scope of the experiment. Therefore, results are indicative only. Improved predictability is assumed to be part of the concept; enhanced efficiency and throughput are measured in an indicative way Exercise objectives High level objectives The high level objective of the exercise described in this exercise plan was twofold. First, the process and effects of collaborative decision making in the APOC are assessed at airport level. It has been studied how the actors in the APOC reach an agreed solution to a problematic situation, what information they require for this decision and how this decision making can be supported by specific tools. Secondly, the effects of this decision making on the airspace network surrounding the example airport were assessed. To this end, the throughput of air traffic through the busiest parts of the European airspace, possibly prioritised by a preferential departure schedule, has been investigated in detail by a modelling experiment. The objective of this experiment was to build a representative kernel European ATM network and to provide an ATM background environment with response similar to real-life operations and to be used to exercise hub airport operations. This network represents the constraining conditions that can be experienced in terms of delays and throughput problems. The emulated behaviour of the network aimed to provide constraints information regarding the planning of air traffic through flows imposed on airport operations and to receive and process flight planning information, agreed at airport level Specific objectives 1. The first objective of the exercise was to study and give some indication of the airport level impact of collaboratively reached decisions in an Airport Operations Centre. This objective was specifically related to the Gaming part of the exercise. 2. Another important specific objective was that this exercise should investigate the benefits of the specific kind of gaming exercise combined with simulation as an innovative tool for validating advanced operational concepts. This objective benefits the exploration of innovative validation tools and methods. 3. The next main objective of the exercise was to establish the effects of commonly reached decisions at the network level. These effects were needed to provide the Gaming exercise with a realistic operational environment regarding network management behaviour under constraining conditions. The objective to generate network effects was complemented by specific objectives required to realise the functioning of the network management environment by use of functions to feed the Gaming exercise. This objective can be further subdivided therefore in the following specific objectives: Page 65 of 105

66 a. Determine the bottlenecks of the network caused by an unbalance of demand and capacity (given the scenario and the commonly reached solution to counter this unbalance) and provide information on bunches related to flights departing from the applicable airport, impacting collaborative departure planning processes. b. Determine the best strategy to balance the (required) capacity at critical bottlenecks with the demand. This strategy should accomplish a balance sufficient to accommodate the demand with least penalising proposed departure constraints. c. Determine the impact of demand balancing as part of DCB by applying an optimising ATFCM algorithm on the bottlenecks in the network, by network throughput analysis. 2 d. Determine an effective prioritisation mechanism to accept collaborative decided flight planning preferences and to evaluate adapted constraining departure planning conditions. e. Determine the impact of prioritisation by departure preferencing on the throughput of flights in the network. The modelling experiment emulated the behaviour of Network Management. The output was determined by the quality of applicable optimisation algorithms. There was no requirement to evaluate direct benefits of applying the algorithm on the NOP, or part of the NOP; however, the outcome should represent future NOP behaviour under SESAR. The quality of emulation of the NOP has been determined by overall network throughput behaviour, measured in terms of waiting periods to access network nodes. These values are measured by network throughput analysis and not by fast-time simulation! Moreover, only so-called waiting periods have been measured and no delays! The specific objectives described above were addressed by advanced planning and coordination procedures, achieved in a collaborative and interoperable way. The expectation was to improve planning in a cost- and flight-efficient way against lowest possible costs. The benefits have been assessed by comparing the results of Gaming and DCB processing and their impact on planned air traffic with the original operational conditions provided by the scenario before intervention Choice of indicators and metrics Metrics for Network Management The model of the kernel network has been assessed for minimal delayed throughput. Throughput was assessed under varying conditions, amongst others impacted by proposed departure constraints, determined by the Optimising ATFCM model. Applicable KPIs are summarised in Table 18. KPI description SESAR framework Identifier EP3 framework identifier + description 2 It should be noted that the usual impact of ATFCM measures on flight performance could not be validated in this modelling experiment due to the scope of the project. Validation of flight performance along usual flight performance indicators like queuing delays and workload characteristics requires a network-wide fast-time simulation process. Page 66 of 105

67 Total daily throughput CAP.LOCAL.ER.PI 1 Total number of aircraft controlled in the en route airspace volume during the day Maximum hourly throughput Departure Delays (IMC) (proposed by Central Network Management) CAP.LOCAL.ER.PI 2 CAP.LOCAL.APT.PI 12 Maximum number of controlled aircraft per hour in the airspace volume ATFCM proposed departure delays are calculated and proposed. Throughput waiting periods are measured. When accumulated, this indicator will be provided as the total departure delay along the day, % of flights with arrival delay more than 1 minute, 2 minutes, 3 minutes,. and average delays for delayed departures The accumulation relates to ATFCM delays caused by bunches in nodes of the ATM network. In the experiment only network waiting periods are measured, being modelled representations of delay causing events. Table 18: KPIs relevant to Network Modelling Experimental Design for Collaborative The Collaborative exercise consisted of two parts. First, a gaming exercise was held to solve pre-tactical planning events at an airport level in the APOC. Second, a network management experiment was conducted to diagnose the consequences of collaboratively reached decisions in the APOC at a network level. These consequences could be used to feed back to the decision makers in the APOC, supporting them in making better decisions in response to the simulated planning events. Figure 8 below depicts the general set-up of the Collaborative exercise. Figure 8: General overview of the Exercise Set-up Page 67 of 105

68 The Network Management exercise is discussed below Network Management Exercise This exercise started with the establishment of an aggregated Kernel Network, and proceeded thereafter with the proposed RBT updates resulting from the collaboratively reached decision in the gaming exercise: Create an aggregated Kernel Network representing CNM functionality on an ECAC-wide network in an acceptable and representative way. Given these updates, a network analysis process was executed to calculate the consequences of these decisions with respect to the previous planning. Network management took place in next four steps: 1. Input of commonly agreed RBT updates from the gaming exercise: This input was considered as trajectory planning updates to be added to the kernel network scenario, representing part of the ECAC-wide network including the airport of Hamburg. 2. Calculation of the throughput in the network: Using NLR s Network Analysis Model (NAM), the resulting incurred delay ( network waiting time ) in the European core network of airports and airspace sectors (including Hamburg) has been calculated. The incurred delay was compared with the cumulative delay in the baseline scenario, in which no trajectory planning updates were implemented (the do-nothing option). The incurred delay is a reliable criterion for the network throughput. 3. Optimisation of the throughput: This step consisted of optimised Air Traffic Flow & Capacity Management (ATFCM), balancing demand as part of DCB. By optimising the traffic flow in the network, a better throughput can be accomplished. Using NLR s Optimised ATFCM tool, certain flights are given a pre-departure proposed departure constraint to ensure a better flow through the network. To determine which flights should be given this delay, the tool offers a look-ahead feature that allows the user to determine future demand and capacity imbalances. Given these future imbalances, it is possible to trace back which flights are to be given a proposed pre-departure delay to ensure optimal throughput through the ATM network. The process above represents an isolated update process from agreed planning updates impacting the departure planning at the airport of Hamburg. This can be considered as a sufficient step to feed a gaming exercise. However, for a network representative process, the inputs at Hamburg would have to be complemented by a similar feed-back process at all airports of the Kernel Network. Given project constraints, it was optional to feed the Kernel Network with similar emulated trajectory planning updates for all other airports, and allowing the network to react with proposed departure constraints. Only in that case, it would have been possible to get an indicative impression on how much the local collaborative planning process may affect the Central Network Management process. Representative emulation of agreed trajectory planning updates can be generated for example by running a DMAN process on RBTs at airport level for each airport of the Kernel Network. The DMAN process represents the optimal achievable results regarding minimal deviation from planning for each individual airport in this case, and regarding the airport of Hamburg, this result can be compared with results achieved by gaming. However, it turned out to be overambitious to close the iterative loop at this stage of platform development, and no feed-back by gaming was processed yet. Page 68 of 105

69 8.2.6 Validation scenario The Network Management scenario was derived from performance analysis of ECAC-wide fast-time simulation experiments of the SESAR Definition Phase; see the performance assessment report, Ref. [27] and [28]. The 2005 baseline scenario of this experiment is in balance, and represents today s, 2005 to 2009, operations quite well, This scenario was compressed by aggregation to be used as a feasible and manageable scenario to provide the required interoperability with the Gaming exercise. The scenario was analysed for appropriate balance in demand and capacity before being made applicable to gaming. To align with the selected scenario of the Gaming exercise allocated at the airport of Hamburg, the scenarios were processed in two steps: Process the SESAR Definition Phase scenario for all steps of the emulated CNM process. This output represents real-life operations and can be used as a reference case. Replace all Hamburg departure and arrival traffic over the day by the Hamburg departure and arrival traffic of the selected scenario for the Gaming exercise and perform the emulated CNM process again. The outcome of these experimental runs can be compared with Gaming exercise results Hypotheses Four hypotheses were defined for the Gaming exercise. In addition, three hypotheses were defined to describe the expected outcome of the Kernel Network modelling experiment, and to relate the role of the CNM model to the TAM concept. Network Management Analysis emulates the behaviour of Central Network Management on a Kernel Network of Europe. The hypothesis is: H1: The Kernel Network of Europe represents the behaviour of network constraining decision making in a sufficiently realistic way to provide a realistic context of network-wide operations for the APOC and its decision making at airport level. H2: The constraining conditions of the Kernel Network are providing appropriate guidance to the Gaming exercise, which can be used in an effective way to keep delays to an acceptable minimum throughout the Network. H3: The departure preferences, agreed at airport level, can be accommodated in an appropriate way by applying prioritisation within the Kernel Network Assumptions The scenario applicable for Central Network Management is derived from a Baseline scenario of the SESAR Definition Phase (Ref. [27] and [28]). It is assumed that this scenario can be used to emulate the effects of CNM functionality for Gaming exercises. In addition, it is assumed that the DCB process shall operate with better performance than the today s, 2009, DCB process, and that more interoperability is permitted. In addition, preferences can be honoured by accepting certain demand balancing prioritisation proposals. The ATFCM algorithm is based therefore on optimisation, taking into account different priority levels. Finally, it is assumed that a Kernel Network, covering the network management processing of the most significant part of the ECAC-wide ATM Network, represents sufficiently well this network to be applied as an ATM network environment for Gaming. Page 69 of 105

70 8.2.7 Experimental Set-up to Conduct the Exercise As explained above, the exercise consisted of a gaming exercise part and a network management exercise part. The set-up of the Network Management part is described below in more detail Network Management Analysis In addition to the ACCES simulation facility used in the gaming exercise, three tools were required as equipment to perform the network management exercise: Firstly, a Network Aggregation Model was used to model the Kernel Network, representing a network around the most significant airports in Europe. This network represents the planning environment of Hamburg airport, and includes this airport. Secondly a throughput analysis model was used to analyse the bottlenecks of the network. This tool was the Network Analysis Model (NAM). Thirdly, an Optimised ATFCM tool was used to process optimal throughput through this network with minimal delays Network Aggregation For the exercise, a network model was built of the core area of Europe. This network, encompassing all sectors and main airports in the south-west and central region of airspace in Europe, has been produced by aggregating flights per sector. The reason for aggregation was to simplify the network as much as possible. The figure below illustrates what an average day of traffic over Europe (based on a 24-hours scenario from 2005) may look like (Figure 9). Figure 9: Overview of European traffic density in a 24-hours 2005 scenario Page 70 of 105

71 This network is too detailed for the actual modest research purposes in the current experiment; therefore, an abstraction from the route network was mandatory. NLR has developed a methodology to abstract from the complexity of a real network situation by aggregating lowest loaded nodes. Lowest loaded nodes are aggregated by applying objective geographic criteria to identify the most appropriate nearest neighbour as aggregation candidate, and aggregation is applied in an iterative way until the properties of the network show a satisfying level of aggregation. The criterion for an optimal level of aggregation can be identified by sufficient simplification to apply emulation, but also optimisation of throughput characteristics can be assessed. The following figure, Figure 10, gives the result of applying this methodology to a high level of aggregation, on the traffic sample presented in Figure 9. Figure 10: Aggregated route network Aggregation in this network took place at three levels: Upper airspace sectors are aggregated above the somewhat arbitrary flightlevel 245. Each sector with a part above FL 245 is counted as upper airspace sector. Aggregation takes place when the lowest loaded sector node is added to its nearest neighbour. Lower airspace sectors are aggregated between FL 0 and FL 245 when sectors are fully allocated below FL 245. Again the lowest loaded sector is aggregated with its nearest neighbour. Page 71 of 105

72 Primary airports of the network are not aggregated. These are pre-selected hubairports and they are identified by name. Secondary airports are aggregated by land-code and applying nearest neighbour criteria. In this way, each country will end up with one or more aggregated ground airports. For example, Germany, end up with at least one ED** and one ET** ground node because military airbases have their own country-code identification. All flights in this network kept their unmodified 4D flight plan, i.e. SBTs submitted by airspace users, following unmodified routes through the sectorised network. The flights were passing along the routes through sectors that may have reached a higher or lower level of aggregation. The aggregated sector is represented by its geographic weight point. The above network of sector nodes and airport nodes corresponds to the following figure, Figure 11, visualising those nodes in the network that serve as flight feeding and flight absorbing nodes of the network, i.e. hub airports, aggregated airports and aggregated feeder sector nodes. Figure 11: Main airports, aggregated airport nodes and out-nodes in the route network of Europe In this network, the main hub-airports are marked by dark-green nodes. Furthermore, the boundaries of the selected part of the core area are marked by a purple polygon. All flights entering and leaving this area are lead through so-called out-nodes: these are the light-yellow Page 72 of 105

73 nodes through which all air traffic entering and leaving the selected part of the core area are bundled in order to control the in- and out-going flows of the Kernel Network. The smaller airports in the selected part of the core area are modelled as aggregated-airport ground nodes of this network. All flights not entering or leaving the area via the out-nodes are departing or landing at an airport, and this might be either a pre-selected hub-airport or one of these aggregated ground nodes, gathering departures and arrivals for all remaining airports in the network. The ground nodes are represented by the light-green nodes in the figure The Network Analysis Model The Network Analysis Model (NAM) developed by NLR can be used to perform validation studies on use of airspace and on the regulation of traffic scheduling. This monitoring tool is implemented by means of a Petri-net, which is composed of nodes (either airports or airspace sectors) and transitions between nodes. In the Petri-net model, flights move from node to node following their established 4-D flight planning possibly revealing critical bottlenecks in the capacity of certain sectors or airports. The model allows the user to measure throughput performance of an airspace network. The Network Analysis Model is used to visualize the flows of flights through the network and to get insight into potential bottlenecks, i.e. those nodes for which demand is greater than the declared capacity. The tool takes the RBTs of the network over 24 hours as the planned load of each node. The updated flight plans (RBTs/SBTs) resulting from the gaming exercise are taken as updates on this input. Further, the declared capacities for each node in the network are input to network analysis, and this relates to declared capacities of sectors as well as airports. Output of the model is a list of delayed flights per node of the aggregated network, sectors as well as airport nodes. In addition, accumulated statistics are recorded, comprising information on throughput per node as well as delays per node. NAM operates in two modes: Unconstrained, measuring the required throughput per node through the network, just by pushing the demand through the network according to its scheduling; and Constrained, measuring throughput including waiting periods accumulated whenever the node was unable to accept the traffic. This mode measures the capacity of the network as a whole to accommodate demand, respecting the declared capacity of each node Optimised ATFCM, balancing demand as part of DCB To optimise the throughput in the network, optimised Air Traffic Flow & Capacity Management (ATFCM) will be employed as part of the concept of DCB by balancing demand. This concept takes analysis one step further. Not only is the traffic throughput and saturation of the core area network aggregated and measured and/or visualized (NAM), but applying the model for optimised ATFCM, the effects of smoothing traffic flows can be studied to improve throughput. To this end, the concept offers the possibility to determine possible demand and capacity unbalances a few hours ahead. The idea behind optimised ATFCM, performing part of the DCB process, is simple. Typically, traffic is unevenly distributed in the network. This variability leads to overloaded sectors and departure and arrival delays at airports. Variability can be reduced, however, by optimising the distribution of traffic flows through the network. Optimising ATFCM leads to a more balanced distribution of traffic demand and a smoother flow of flights through the network. The result of ATFCM is a time-table (scheduled demand, specified by RBTs) with adapted (delayed) planned departure times, leading to a decrease of queuing delays (departure and Page 73 of 105

74 arrival queuing), a decrease of workload in overloaded sectors and a decrease of flight duration due to a more flight-efficient arrival management process. Optimised ATFCM is done as follows. A Flow Management process on the full network proposes a pre-departure delay for every flight of a submitted flight trajectory planning, the SBT/RBT, that encounters a capacity overload in the airport of origin, any of the sectors along its route or the airport of destination. This calculation is possible since an algorithm has been implemented to offer the possibility to look ahead at future bottlenecks in the network. Only flights with sufficient time before departure to allow for trajectory planning updates are considered, i.e. taking into account the minimum time needed to accommodate a planned departure change under nominal operational conditions; flights that have gone off-blocks or are in-flight cannot be given pre-departure delay. RBTs are processed in the order of their scheduled departure times (off-blocks) and arrival times at a node of the network. Network Management using optimised ATFCM is based on applying prioritisation, if applicable. The prioritisation rules used by optimised ATFCM are partly based on general network operational principles and partly on individual flight priority assignments. Flight with a high priority will have a small chance of receiving pre-departure delays, since lower prioritised flights will be chosen first to resolve overloaded sector conflicts. Network Management based on applying optimised ATFCM algorithms is specifically effective when the flight preferences resulting from collaborative decision making, are input to ATFCM, forcing this process to honour departure preferences as much as possible. For this reason, apart from trajectory planning updates, also other output from the gaming exercise is required 3, specifying its departure preference status. Given a scenario in which, due to sudden capacity limitations, not all flights can be executed according to plan, decisions will have to be made about which flights should have priority over others. These decisions can be fed to the emulated model of Central Network Management, guaranteeing that flights with high priority will be given small or no pre-departure delays when optimising the flows in the network. Optimisation is in this way most effective when a limited part of air traffic is prioritised. To make the outcome of the optimising ATFCM algorithm more realistic, it is optimal to feed the network model with emulated trajectory planning updates (RBTs) for all other, non- Hamburg airports, and to allow the network to react to these updates as well with proposed departure constraints. In that case, it will be possible to get a more realistic impression on how the local collaborative planning process may be able to cope with constraints, fed by the Central Network Management process. These emulated flight plan updates can be generated by running a DMAN process for each airport of the Kernel Network, but other emulation alternatives might be considered. After network analysis and applying optimised ATFCM, two results are returned to the actors in the APOC. The first result, stemming from network analysis, gives a first indication on the effects of commonly agreed flight plan updates on the network. An insight in the potential bottlenecks is provided by returning a list of delayed flights per node, i.e. per sector or airport, in the network. The second result, derived from the network management process and applying optimised ATFCM, will yield proposed departure constraints to guarantee an optimal flow through the network by minimizing the bottlenecks. This result constitutes a list of flight plans, updated with proposed departure planning constraints. 3 Note that it cannot be simply assumed that the flight updates suggested by the gaming exercise should receive a high priority. After all, flights may be given a later departure slot (resulting in a flightplan update) to make room for flights having a higher priority. Page 74 of 105

75 Finally, this departure constraints list can be extended with less enforcing warnings to provide planning alert indicators for those flights going through bunchy conditions. The participants to the Gaming exercise are free to take notice and to use these weak planning constraints as input to collaborative agreements on planning problems. 8.3 CONDUCT OF EXPERIMENT NETWORK MANAGEMENT ANALYSIS Step-wise experiment The Gaming exercise is supported by Network Management Analysis, providing the context of operations of a Central Network Management (CNM) function and feeding the Gaming exercise with inputs that can be expected from a CNM function operating under real-world conditions. The required execution steps to provide such a context of operations are: Select a scenario that satisfies real-world conditions. Simplify this scenario to dimensions that are compliant with the scale and needs of the Gaming exercise, i.e. apply aggregation to simplify the network to an aggregated Kernel Network. Validate appropriate DCB properties to make this Kernel Network applicable to constraints management in support of hub airport operations at Hamburg, i.e. the Gaming exercise. Generate input data for hub airports of the Kernel Network, feeding the Hamburg Gaming exercise as one of the hub-airports with alerting information on bottlenecks and proposed departure delays to mitigate bunching conditions in the Kernel Network. As far as time and effort permitting, perform assessment of Gaming exercise inputs by considering the impact of hub-airport inputs, i.e. of Hamburg in this case, on the functioning of the Kernel Network. CDM determined departure preferences may have an impact on DCB processing of the whole network. And as far as possible, emulate the feed-back of CDM departure planning at hubairport level for all hub-airports of the Kernel Network and for the whole day, i.e. for 24 hours, and determine the impact of these departure preferences on the effectiveness of the DCB process on the whole Kernel Network Scenario selection The first step is to select a scenario that satisfies real-world conditions and that represents operations in ECAC airspace in a sufficient complete way to be used to simulate constraints management in the NOP. Appropriate and representative ECAC-wide scenarios are the scenarios from the SESAR Definition Phase, provided by EUROCONTROL, and used in the performance assessment studies in Task of this programme. There were three applicable scenarios and they were analysed during these studies, see SESAR deliverable on performance assessment by available tools and methodologies, ref. [27], and its annex on performance assessment of ATFCM functionality, ref. [28]. The two future scenarios, 2012 and 2020, created large delays due to unbalance in air traffic demand, the applicable routing and the available capacity. The reference scenario, 2005, was in balance and was realistic in performance behaviour. Some Page 75 of 105

76 reference information was available on imposed flow management delays, and these delays could be compared with simulated flight performance results. Volumes of air traffic demand were not significantly different from 2008/2009. Therefore, the reference scenario, 2005, was judged to be appropriate for creating an operational CNM environment for the Gaming exercise. There was no need to use more up-to-date or future scenarios because the scenario had to be representative for realistic behaviour of the planning of air traffic operations, not to be identical to the anticipated future operational conditions of SESAR. The selected scenario comprises a full ECAC-wide network of one day of operations, 24 hours, in July The selected day was a day without specific weather conditions, e.g. wind, and/or any kind of disruption. The scenario consists of: Scheduled ICAO flight plans: These plans are characterised by departure, destination, a list of waypoints and a planned cruise altitude mainly, as well as a scheduled departure and arrival time. The traditional flight plans were converted to 4D flight plans, the SBTs/RBTs, following their planned routes and flying the most efficient trajectories along these routes by fast-time simulation. Optimal 4Dtrajectory plans were established by avoiding any inefficiency as result of conflict detection, conflict resolution and separation. These 4D trajectories were stored as RBTs. The total number of 4D planned RBTs through Europe is: flights in 24 hours. Airports and airport capacity: The total number of airports in Europe is more than 500, however, 133 airports can be characterised as significant major airports. Airports capacity figures were specified as sustainable declared capacity and sometimes also as ceiling peak-capacity. For demand regulation purposes the peak-capacity figures were applied, and if not available, the sustainable capacity. The airport capacity was increased in addition with 5% hourly capacity to take into account uncertainty in departure/arrival demand and other uncertainties such as unbalanced demand distribution due to runway usage procedures and the applicable schedules. From the performance assessment study of SESAR Definition Phase, it was concluded that ~50% of flight-executive delays per day were allocated at 20 airports and most significant delays at the 10 largest hub airports, Ref. [27], [28]. Sectors and sector capacity: RBTs are following routes through airspace sectors. These sectors were associated with declared capacity figures. These figures were applied unmodified in the experiment. All sectors were considered to be open. Page 76 of 105

77 Figure 12: Overview of departure/arrival operations from/to airports in the Kernel Network of Europe The modelling experiment was performed using the 4D planned RBTs without any modification except the planned or proposed departure take-off time. The Network, and use of the Network, was defined by the routings from airport to airport and through sectors. The capacity figures of airports and sectors were defining the permissible throughput through the Network, but only the sector definitions and possibly also the sector capacity figures were subject of change during the modelling experiment feeding the gaming exercise. To limit the extent of the experiment a Kernel Network was selected. The selection of this Network was such that it included the 17 largest airports of Europe and all air traffic to, from and within this area, see Figure 11. The selected Kernel Network covered 94 airports with more 25 operations per airport per day, and 4884 airport movement operations were scheduled during 24 hours. (See an overview of airport operations in Figure 12.) This Network is deemed to be sufficiently representative to experience and evaluate Demand and Capacity Balancing (DCB) on incidental and/or local air traffic overload problems in central parts of Europe Network Aggregation The selected Kernel Network is still unnecessary complex in terms of number of airports and number of sectors regarding the requirements of network management. Therefore, network aggregation was applied as described in section The complexity of the network was judged on the one hand regarding the objective to use it for an airport gaming exercise and on the other hand the objective to be representative for a future DCB process in the core area of Europe. The selected non-aggregated Kernel Network consists of: 17 major hub and large airports: These airports were excluded from any form of aggregation, these are: o EBBR Brussels Page 77 of 105

78 o o o o o o o o o o o o o o o o EDDF Frankfurt EDDH Hamburg (specially added as the subject of the Gaming exercise) EDDK - Köln EDDL Düsseldorf EDDM Munich EGKK London Gatwick EGLL London Heathrow EGSS London Stanstead EHAM Amsterdam Schiphol LEBL Barcelona LEMD Madrid Baragas LFPG Paris, Roissy - Charles de Gaulle LFPO Paris, Orly LIMC Milan Malpensa LIML Milan, Linate LIRF Roma, Fiumicino Other airports: These airports were aggregated on country code to a number of aggregated Ground nodes. Upper airspace sectors: These sectors, fully or partly above FL 245, were aggregated on geographical criteria only. Lower airspace sectors: These sectors, fully below FL 245, were aggregated on geographical criteria only. Out sectors: These pseudo sectors were bundling all flights going in and out the selected Kernel Network. 9 Out-sectors were selected and positioned as appropriate. These sectors have unlimited capacity and are excluded from any aggregation process. The aggregation process is without restrictions. In principle, the end is defined as where no aggregation takes place anymore, and aggregation takes place as long as there is a node with the lowest traffic load during 24 hours that is allowed to be added to any other node of the network with the same properties, i.e. the type of the node is airport, lower or upper airspace sector. The optimal aggregation result can be decided on at least three possible criteria: Based on the effectiveness of the DCB process regarding its control on the load of ATM operations during the executive flight phases; Based on the achieved robustness of the network, i.e. considering demand/capacity ratio, or other bottleneck sensitivity criteria; By a simplicity criterion in terms of number of airports and sectors to represent the selected geographical area in an appropriate way regarding its objectives, i.e. to support the Gaming exercise. For the experiment, the last criterion was decisive, together with feasibility to offer high performance processing capabilities by the model. Page 78 of 105

79 In first instance a network was selected with 27 aggregated upper airspace sectors and 25 lower airspace sectors. To increase realism, later on a network was used with 68 upper and 50 lower airspace sectors Assessment of DCB conditions of the Kernel Network The aggregated Kernel Network had to be assessed on realistic and appropriate throughput properties, and, thereafter, the Network had to be balanced against demand overload, i.e. proposals had to be evaluated to suppress bunches through bottlenecks. Three evaluation steps were applicable, using the available toolset: Capacity assessment and bottleneck analysis applying NAM in unconstrained mode; Throughput analysis applying NAM in constrained mode; Demand balancing by de-bunching and throughput optimisation using an ATFCM tool. Capacity assessment and bottleneck analysis with NAM, unconstrained The Kernel Network was analysed with the Network Analysis Model (NAM). The interest was to analyse the load of 2005 air traffic on airports and sectors over the day and to compare the performance of the non-aggregated and aggregated network. The required result of this analysis step was to identify a specification of capacities throughout the network that was necessary and sufficient to run the flight operations with minimal delay against lowest (capacity) costs. Some bunching behaviour would always remain because the demand can never be completely in balance by nature and provision of capacity to accommodate every possible fluctuation in demand would be cost-inefficient. Therefore, some bunches at some sectors during some periods had to be present in order to present natural constraints behaviour towards the CDM process at airport level, i.e. the Gaming exercise at the simulated airport of Hamburg. For this analysis step, NAM was applied by accumulating waiting periods at overloaded nodes, but without respecting capacities and using delays to determine and measure the effective throughput through the Network. The bottleneck analysis was performed over a busiest period of the day of 15 hours from 7:00 until 22:00 hours. The result was an overview of Demand/ Capacity ratios for a number of aggregation steps accomplished in an iterative and step-wise manner, and these results of Demand/Capacity ratios per aggregation step justified how to continue the next step with constrained throughput analysis. The 2005 July scenario was a perfect result on a perfect day with moderate and traditional demand. The observed delays after aggregation were very low and it was decided therefore to continue the analysis on throughput applying a general decrease of capacity of all sector nodes with respectively 5% and 10% of the specified hourly capacity. Throughput analysis with NAM, constrained Throughput analysis was applied using NAM to produce delay figures, i.e. waiting period information. This time, NAM was processed each time over 24 hours including the activated option to measure effective throughput through the network by waiting at each overloaded node. This means that throughput performance problems could create also networkdependent bunching problems in bottleneck-sensitive parts of the network. Applying throughput analysis on the selected aggregated network and using the 2005 demand and 2005 capacity figures left effectively no sector delays anymore. The only delays Page 79 of 105

80 left, where delays at the major airport nodes, but these nodes were excluded from aggregation a priori, and their performance could therefore not be improved by aggregation. The observed decrease in measured waiting time could be interpreted as too much simplification or could be justified to apply some decrease of capacity of all sector nodes. Throughput analysis with NAM on a scenario with decreased capacities resulted in potentially delayed flights that could be candidate for delay management. The potentially delayed flights could be flights departing and arriving at any airport of the network. However, only those flights departing and/or arriving at Hamburg airport could be candidate for local management by AOP collaborative planning. These flights were filtered out and information concerning their expected delays (waiting periods) was collected and recorded as input to the Hamburg Gaming exercise. It was concluded that the applied level of aggregation was suppressing almost all delays, therefore the final experimental runs were conducted on a less aggregated Kernel Network with 433 nodes including 68 upper and 50 lower airspace sectors. Demand balancing, de-bunching and throughput optimisation The traffic demand through the Kernel Network is processed now in order to determine proposed pre-departure delays that may solve the bunching problems through the network. The options to determine pre-departure delays are: 1. First-Come First-Served (FCFS); 2. Optimised towards network performance; and 3. Prioritised, taking into account priorities by users. The second option was applicable, giving mainly priority to flows between congested hub airports. This option was processed by looking ahead about one hour and selecting those flights that were least penalised by receiving delays. ATFCM was applied on the traffic through the Kernel Network with 433 nodes. The resulting delays were allocated for 60% of these proposed departure delays at the 17 hub airports, including the airport of Hamburg. The demand/capacity ratio for Hamburg was adapted to be compliant with the scenario of the Gaming exercise. In total 439 departing and arriving flights were processed in 24 hours with a ceiling capacity of 63 movements per hour. This is not equal but very similar to the reference scenario of the SESAR Definition Phase. The output derived from this exercise run was, after filtering on planned Hamburg departure and arrival flights, the output applicable as input to the Gaming experiment. Two output files were produced: A file with alerting information of traffic involved in a bunch at one or more nodes of the network. This file was produced by NAM running in constrained mode. A file with proposed pre-departure ATFCM delays that could be applied to suppress the bunch or that could be used as input to the AOP planning process, simulated by the Gaming exercise. In that case, the solution of Gaming is expected to suppress the bunch in an interoperable way. Both files were submitted off-line as pre-processed input files for the Gaming exercise. These files are examples that may serve a future implementation by real-time coupling. Page 80 of 105

81 8.3.5 Emulation of feed-back from hub-airport CDM to CNM The output from the Gaming exercise was impacting a few flights only. The best way to process feed-back in a realistic way is to emulate impact by departure CDM for all airports of the network over the whole day, but this was outside the scope of the present experiment, and no feed-back was processed for that reason. Page 81 of 105

82 8.4 RESULTS AND DISCUSSION The experiment was performed step-wise and also results will be present step-wise. The experimental conduct steps performed in the CNM modelling part of the exercise are summarised in Table 19. Scenario SESAR Def. Phase 2005 scenario Scenario selection V, section Network aggregation V, section Throughput analysis (NAM, constrained) Demand balancing (Optimising ATFCM) Impact from CDM at airport level on CNM SESAR Def. Phase 2005 scenario, and Hamburg traffic replaced by air traffic of Gaming scenario - V, section and V, section and Not performed, see section Interfacing Gaming Exercise, offline interaction Alerts on bunching conditions to Gaming Proposed constraints on RBTs to Gaming RBT changes from Gaming Table 19: Overview on experimental runs by CNM modelling part of experiment The required execution steps to provide such a context of operations are: Scenario selection, section 8.4.1; Aggregation to a simplified kernel network, section 8.4.2; Assessment by Demand and Capacity Balancing (DCB), section 8.4.3; o Capacity assessment and bottleneck analysis on network aggregation (NAM, unconstrained), section ; o Throughput analysis (NAM, constrained), section ; o Demand balancing: de-bunching and throughput optimisation, section ; Feeding the Hamburg Gaming exercise with feedback from CNM, section ; and Impact from CDM processes at airport level, section Scenario Selection A scenario was selected to represent real-world CNM conditions in view of Airport collaborative planning. The 2005 scenario of SESAR Definition Phase represents air traffic as experienced today in The scenario shows properties that are appropriate in most respects: Traffic density varies in intensity over the day and over the geography of the network; Traffic demand is reasonably in balance with the capacity of the airports and the volumes of airspace, which is also required for operations under nominal conditions Page 82 of 105

83 in the future, and which means that there are certain numbers of periods with overloads; Traffic is sufficiently structured to be manageable in high density areas, which is required in order to maintain manageability under disruptive conditions; The network is super-critical, which is the reason to undertake aggregation and to make the network more robust than today. Today s DCB practise is to designate certain sectors as overloaded and then to start regulations for those sectors. Future operational conditions require to manage the total network as a planned network, i.e. the NOP. This implies monitoring congestion and overload conditions for all elements of the NOP and to inform users systematically on flight constraining conditions in order to allow them to anticipate these constraints during collaborative airport planning. More accurate planning put high requirements on the network, but this network is not yet appropriate to address these requirements. For example: Airports are part of the network, but their capacity figures are not unambiguous. The capacity is represented by a nominal- and a peak-load capacity figure, however, there are variations due to different runway configurations, departure/arrival traffic mixes, weather-determined variations in runway separations and capacity variations due to noise abatement regulations. In a monitored network all volumes of airspace have to be monitored against traffic overload conditions, whilst today only the open sectors, designated for regulations, are managed. In a dynamic network, also the airspace related capacity figures are more variable than today. This requires transparency towards the user to clarify the underlying causes for capacity deficiencies and the means to mitigate congestion. The non-aggregated network was supercritical. The criticality of the network was suggested by the number of detected hourly periods of the day with sector overload conditions, whilst the total demand/capacity ration was not excessive, i.e. around 60% Demand/Capacity over the total network over the 15 busy hours of the day. Aggregation could make the network more robust, reducing the counts of overloaded hourly periods of sectors Aggregation to a simplified Kernel Network Aggregation was applied selectively, and this impacted obtained results: Main airports were not aggregated because these were considered to be basic nodes of the network. The result was that delays due to main airport throughput restrictions could not be solved by aggregation. The bottlenecks at the level of major airports are present even in the most aggregated modus of the network. Other airports could be aggregated to country-related airport nodes. The delays due to these airports were not very high, but solutions by aggregation are purely artificial and for reasons of modelling. The objective is to reduce complexity of the network, not to solve congestion, or to increase effective use of available capacity. It can be assumed that none or very few flights will change their operations and in this case aggregation will not prevent these flights from being delayed. Lower airspace sectors were aggregated assuming that there were too many sectors due to controller workload limitations and that redistribution of traffic may take place according to actual Demand/Capacity ratios within the sectors. This last condition might strongly depend on local conditions. Aggregation is deemed to be unsuccessful if overload conditions cannot be solved and traffic can not be Page 83 of 105

84 redistributed in real-life due to a too far-reaching level of aggregation. Traffic cannot always be redistributed as needed, and that may limit therefore the acceptable level of aggregation. Upper airspace sectors were aggregated with the least problems. Sector capacity figures were added, assuming to be able to re-distribute traffic as needed within an aggregated node. The objective of aggregation was to simplify the selected scenario to dimensions that are compliant with the scale and needs of the Gaming exercise, i.e. by applying aggregation to a simplified network where network and airport management are impacting each other in a traceable but yet realistic way. The selection of the network was based on a representative geographic area including a relevant number of major airports of Europe, and an acceptable number of nodes, comprising a Network with 81 nodes to represent the selected part of European airspace. Figure 13: The selected network of airspace and airports, representing the major part of the European ATM network The selected network was appropriate for the needs of the experiment, but aggregation was carried through somewhat over the top, considering the simplification by consolidated network nodes: Aggregated upper airspace sectors were achieved by aggregation of 5 to 10 sectors, servicing mostly between 4000 and 8000 flights a day. Page 84 of 105

85 Lower airspace sectors were composed sometimes of more than 20 sectors. A lower level of aggregation would probably provide more manageable levels of regulation. The smaller airports were aggregated, however, without physical significance. Aggregation was only relevant for model processing and emulation. The main airports were not aggregated and played a role as kernel nodes of the network. The aggregated Network could be used for throughput analysis and demand management by the persistent delays of the main airports and due to the increased delays by reduction of sector capacity with 10%. After aggregation, the selected network consists of 81 nodes, listed in Table 20. Network Node Airp./Sector Benelux Number of aggregated nodes Nr. of Passages (24 h.) All aggregated nodes Centre of Gravity (Sum of) Capacities Long. Latt. Ident. Aggregated Node EBBR Main airport EBBR EHAM Main airport EHAM EBLG EB_Ground EHRD EH_Ground ELLX EL_Ground EBBUHUS Lower Airspace EBBUWLS Lower Airspace TMAEBAW Lower Airspace EHACOD Lower Airspace EBMAKOH Upper Airspace EBMALNL Upper Airspace EHDELMD Upper Airspace Germany EDDF Main airport EDDF EDDH (Hamburg) (Main airport) EDDH EDDK Main airport EDDK EDDL Main airport EDDL EDDM Main airport EDDM EDDT ED_Ground ETOU ET_Ground EDFFNR Lower Airspace EDFFOR Lower Airspace EDLARN Lower Airspace EDLARS Lower Airspace EDMMNR Lower Airspace EDWWHAN Lower Airspace EDMMSR Upper Airspace Page 85 of 105

86 Nr. of Centre of Network Number of Passages (24 h.) Gravity Ident. Node Airp./Sector aggregated nodes All aggregated nodes (Sum of) Capacities Long. Latt. Aggregated Node EDUUFFMB Upper Airspace EDUUNTMML Upper Airspace EDUUSLNH Upper Airspace EDUUWURB Upper Airspace EDYSOHI Upper Airspace United Kingdom EGKK Main airport EGKK EGLL Main airport EGLL EGSS Main airport EGSS EGBB EG_Ground EGTTBIG Lower Airspace EGTTLAM Lower Airspace EGTTWIL Lower Airspace TMAEGKB Lower Airspace EG01LUW Upper Airspace EG02LUE Upper Airspace Spain LEBL Main airport LEBL LEMD Main airport LEMD LEPA LE_Ground LECBLRDNL Lower Airspace LECBCE1H Upper Airspace LECBLRDNH Upper Airspace LECMPAL Upper Airspace France LFPG Main airport LFPG LFPO Main airport LFPO LFLL LF_Ground LFBTA Lower Airspace LFFTS Lower Airspace TMALFPB Lower Airspace TMALFLL Lower Airspace TMALFMD Lower Airspace LFBH Upper Airspace LFBL Upper Airspace LFBR Upper Airspace LFEKR Upper Airspace LFMB Upper Airspace LFMY Upper Airspace LFRZI Upper Airspace Italy Page 86 of 105

87 Nr. of Centre of Network Number of Passages (24 h.) Gravity Ident. Node Airp./Sector aggregated nodes All aggregated nodes (Sum of) Capacities Long. Latt. Aggregated Node LIMC Main airport LIMC LIML Main airport LIML LIRF Main airport LIRF LIRA LI_Ground LIMMES Lower Airspace LIPPMC Lower Airspace TMALIMN Lower Airspace TMALIRE Lower Airspace LIPPME Upper Airspace LIPPN Upper Airspace LIRRMI1X Upper Airspace LIRRNE2X Upper Airspace LIRRNW Upper Airspace Czech Republic LKPR LK_Ground Austria LOWW LO_Ground Switzerland LSZH LS_Ground TMALSZF Lower Airspace LSAZMP Upper Airspace Table 20: The list of nodes of airports and airspace sectors of the selected aggregated network, representing the NOP Assessment by Demand and Capacity balancing (DCB) The aggregated Kernel Network had to be assessed on realistic and appropriate throughput properties, and thereafter, the Network had to be balanced against demand overload, i.e. bunches through bottlenecks had to be suppressed. Three evaluation steps were applicable: Capacity assessment and bottleneck analysis on network aggregation (NAM, unconstrained); Throughput analysis (NAM, constrained); Demand balancing: de-bunching and throughput optimisation Capacity assessment and bottleneck analysis on Network aggregation (NAM, unconstrained) Network aggregation was applied to achieve a robust and simple network. In order to select the most appropriate network to support gaming, the performance of several levels of aggregation had to be compared. The most decisive criterion for aggregation was the number Page 87 of 105

88 of aggregated airspace sectors, and the traffic load through these sectors. The characteristics of airport loads were less relevant due to the nature of the aggregation process. The next figure, Figure 14, shows several levels of aggregation, identified by the number of airspace sector nodes and characterised by the distribution of Demand/Capacity ratios. 70% 60% 50% 40% D/C percentage distribution of sector nodes for each level of aggregation Aggregation level 607 sector nodes 522 sector nodes 394 sector nodes 138 sector nodes 52 sector nodes 25 sector nodes 14 sector nodes 30% 20% 10% 0% 0-10% 10-20% 20-30% 30-40% 40-50% 50-60% 60-70% 70-80% 80-90% % Figure 14: Distribution of Demand/Capacity ratios for different levels of aggregation Figure 14 shows first that aggregation leads to a decrease of contributions of high Demand/Capacity ratios, in other words there is a decrease of overload conditions and bunching as expected. Secondly, a far-reaching level of aggregation, i.e. 14 nodes, leads to a typical highest Demand/Capacity ratio of roughly 60% use of available capacity per hourly period per sector node. This means that the total load of the applicable network is not higher than 60% during the busiest hourly period, and because this relates to a wide core area of Europe this implies that the total load factor for the whole ECAC area is likely to be even much lower. Although a fully equalised load over the day over all sectors is quite evidently not possible, a load of 60% per hourly period per network node could be seen as an indication of inefficient deployment of capacity. The throughput through the network was measured by counting the total amount of overload hours as well as the number of hourly overload periods in the network, and the throughput was measured for the full set of sector aggregation results. Of course, decreasing the number of network nodes is expected to cause also a decrease of overload effects which stands for the aimed robustness of the network. Nevertheless, the decrease is very significant and more than linear; observed throughput delays are dropping rapidly. Page 88 of 105

89 600 Total number of overloads at sectors for each level of aggregation sector nodes 522 sector nodes 394 sector nodes 138 sector nodes 52 sector nodes 25 sector nodes 14 sector nodes Figure 15: The number of observed overload periods for each level of aggregation Total number of hours in which overload occurs at sectors for each level of aggregation sector nodes 522 sector nodes 394 sector nodes 138 sector nodes 52 sector nodes 25 sector nodes 14 sector nodes Figure 16: The number of "waiting" hours counting for overloads occurring at airspace sectors for each level of aggregation The figures, Figure 15 and Figure 16, show both a steep dropping of delays by aggregation of 394 to 138 airspace sectors. The functionally interesting domain, preserving effectiveness of demand management is expected to be above 394 airspace sectors for this scenario. Due to tools performance limitations, an aggregation of 52 airspace sectors was selected for the experiment initially. However, observed delays were below natural levels, but this could be compensated by decreasing airspace capacity figures. This can be justified only for pure experimental reasons. The real applicability of the level of aggregation of the network can be assessed only by applying first ATFCM (demand balancing by flow management) and then to evaluate by fast-time simulation if flight performance and ATC is still manageable under the Page 89 of 105

90 applicable process of demand balancing. However, this was outside the scope of this experiment. The unconstrained throughput analysis on airspace sectors provides an overview of measured levels of congestion at airspace sector nodes for different levels of aggregation. The next step was to assess levels of congestion through the whole network, including airports for the selected level of aggregation, i.e. the network with 52 (airspace) nodes. The next figure, Figure 17, shows the distribution of Demand/Capacity ratios for the selected network. The distribution is split in main airports, other airports and airspace sectors. As can be observed, from the figure as well as by comparison, the aggregation process has resulted to: A very low loading of small airports, i.e. always less than 30% per hourly period for that airport. This has not much significance, it is indicative only for the relative low load of small airports, given the available capacity. No changes in loading of main airports. This is logical because main airports were not aggregated. A very low loading of airspace sectors, as seen before. After aggregation, the loading stays below an unnatural level of 70% highest occupation rate per node per hour. 30 D/C Percentage airport Number of Occurrences sector main airport % 20-30% 30-40% 40-50% 50-60% 60-70% 70-80% 80-90% % Category Figure 17: Distribution of Demand/Capacity ratio through the selected network It can be concluded that the remaining overloads in the selected network consists almost completely of main airport overloads. This is not necessarily a problem, although it does not reflect the characteristics of the ATM network today, It was chosen therefore to investigate the impact of some variations of airspace sector capacity on the induced delays Throughput analysis (NAM, constrained) The load on the network was investigated by reduction of sector capacities with respectively 5% and 10% hourly capacity per node. The throughput through the network was compared with the original non-aggregated network, also reducing the capacity with respectively 5% and 10% hourly capacity per node. Page 90 of 105

91 The reference case is the original non-aggregated network. Running NAM for constrained throughput analysis with 3 airspace capacity levels, caused between 22 and 26 airports to accumulate waiting periods. More airports are congested now than only the 17 main airports of interest. The airports waiting time reduces due to throughput limitations of the airspace part of the network when decreased capacity is applied for airspace sectors. The airspace sector part receives 2447 hours waiting period in 24 hours (N.B. It should be noted that waiting hours have an indicative notion only, it is not the same as delay!) The airspace sector waiting time is strongly increasing by decrease of airspace sector capacity, as could be expected. The outcome is summarised in Table 21. Aggr-1 (original sectorisation) Capacity -10% Capacity -5% Capacity -0% Node Capacity Total Total Total Airports Hrs.waiting Hrs.waiting Hrs.waiting Sub-total: Airspace sectors Sub-total: Total: Table 21: Overview of NAM-constrained waiting periods of non-aggregated Kernel Network The next table, Table 22, shows in which ways the network has become more robust for constraining conditions than without aggregation, and what happened by systematically decreasing the capacity of airspace sectors: The airport waiting periods are allocated now at 9 main airports only. These airports were not subject of aggregation and similar measured waiting time could be expected as for the non-aggregated case, i.e. compare with the previous table. The airport waiting periods were insensitive for airspace capacity reduction. This stems from the robustness of the network. Waiting periods due to lack of airspace capacity is limited now to just 4 airspace sectors, which is unrealistically low. The reason is the rather extreme level of aggregation applied for the model that had only to feed the gaming exercise. Decrease of airspace sector capacity figures did increase the number of overloaded sectors to just 6. This is still very low. The decreased capacity offers 746 hours waiting time which is still much lower than the original non-aggregated Kernel Network, i.e hours, and in addition this waiting time is caused mainly by main airports. Aggr-2500 (Aggregation in experiment) Capacity -10% Capacity -5% Capacity -o% Node Capacity Total Total Total Airports Hrs.waiting Hrs.waiting Hrs.waiting Page 91 of 105

92 Aggr-2500 (Aggregation in experiment) Capacity -10% Capacity -5% Capacity -o% Node Capacity Total Total Total Hrs.waiting Hrs.waiting Hrs.waiting LEMD EDDF EGKK EDDM EDDL EGLL LIML LFPG EHAM Sub-total: Lower airspace sectors EHACOD Upper airspace sectors LECBLRDNH EG01LUW EHDELMD EDMMSR EG02LUE Sub-total: Total: Table 22: Overview of NAM-constrained waiting periods of the aggregated Kernel Network (52 nodes) The conclusions from decreasing the capacities of airspace sectors are: Reducing the capacity of airspace sectors was limitedly successful. The preferred level of aggregation is too extreme from the point of view of generating waiting time due to throughput limitations. This preference was selected however for pragmatic reasons. The best Kernel Network scenario is the 52-nodes Kernel Network with an airspace capacity reduction of 10% per node per hour, but this network provides still less constraining conditions to the Hamburg Gaming exercise than real-life today s, 2009, operational conditions. The measured waiting time is an indicative value for performance of this network. The real performance can only be measured by doing a fast-time simulation on the original ECAC-wide scenario by measuring if the performance of the network is improved in terms of efficiency and capacity. The only valid performance assessment is to measure runway throughput sector control load under simulated real-life operational conditions. Page 92 of 105

93 Because tools performance could be improved, it was decided to select a less aggregated Kernel Network, to perform demand balancing as part of Demand and Capacity Balancing (DCB) Demand balancing: de-bunching and throughput optimisation The selected aggregated Kernel Network (433 nodes, with 68 upper and 50 lower airspace sectors) was used to perform ATFCM as part of Demand and Capacity Balancing (DCB). ATFCM was applied by processing an optimising ATFCM algorithm on the traffic through the network. The result of processing ATFCM was to propose imposed departure delays, assumed always to be accepted. Delays were imposed on flights departing from countries of the kernel network only. The total amount of imposed delays was 2820 hours delay. It turned out that most imposed delays were allocated at the 17 main airports, including Hamburg (1744 hrs.). The impact on network performance was not assessed by fast-time simulation but by network throughput analysis only. The result is measured as waiting periods per constrained node. As stated before, it should be noted that waiting time is similar but not identical to delay! Throughput improvement shows significant gains at airport level as well as at sector level. The total amount of benefits shows that only 19% of measured throughput delays remained (809 in-flight delayed hours after imposed departure delays instead of 4273 in-flight delayed hours without ATFCM). A summary table of results is presented in Table 23. Airports FM imposed Throughput waiting periods Before FM After FM Dep.delay LEMD EDDM EDDF EGKK EHAM EGLL LFPG LIML EBBR EDDH EDDK EDDL EGSS LEBL LFPO LIMC LIRF Main Airports Total (in hrs.) Page 93 of 105

94 Minor Airports Total (in hrs.) Airspace FM imposed Throughput waiting periods Before FM After FM Dep.delay EB-low ED-low EG-low EH-low EL-low 6 0 ET-low LE-low LF-low LI-low LS-low 16 2 Lower AirSp Total (in hrs.) EB-upper 9 2 ED-upper EG-upper EH-upper LE-upper LF-upper LI-upper LS-upper 7 3 Upper AirSp Total Total Total Throughput waiting periods, in Total (hrs.): Imposed delay: Table 23: Results of ATFCM applied on an aggregated European Kernel Network with 439 nodes in 24 hours A graphical representation of results is presented below in Figure 18. This map shows measured waiting periods through nodes after applying ATFCM on the aggregated Kernel Network. Waiting time is represented as pink circles, proportional to the accumulated waiting time. Page 94 of 105

95 Figure 18: Accumulated waiting time through the kernel Network, represented by pink circles The following figure, Figure 19, presents the average imposed delay per flight during a 24 hours period in seconds per flight (4D-planning). These delays were calculated for the Kernel Network and are applicable only to countries being part of the network. In addition, no threshold values are applicable as long as no total delays per flight are known. The real balance between imposed delays and reduced flight duration can be assessed by fast-time simulation only. The result shows highest imposed delays for Germany, including Hamburg, but this can be biased by choice of the network as well. The imposed delay figures together with the outcome of Figure 18 suggest sensitivity of the network by bottlenecks caused by Frankfurt and Munich. Looking at the data for this scenario, it turns out that these bottlenecks are caused by two overloaded ED upper airspace sectors. Page 95 of 105

96 Figure 19: Average imposed delays by ATFCM on aggregated Kernel Network of Europe Because Hamburg was subject of the Gaming exercise, the last figure presents data for the airport of Hamburg. The imposed delays are quite large, presumably due to the experimental conditions, lack of calibration and lack of tuning of capacity figures. The figure presents the distribution of imposed delays at the airport of Hamburg (yellow) compared to total imposed delays (red-brown). It should be noted that Hamburg imposed delays and other airport delays are not proportionally scaled, see Figure 20. Figure 20: Hourly distribution of imposed delays at Hamburg and at other airports of the network The ATFCM module generated imposed constraints for the selected Kernel Network. This is the busiest core area part of the network and areas outside could not take part in the ATFCM Page 96 of 105