EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL EUROCONTROL EXPERIMENTAL CENTRE

Transcription

1 EUROPEAN ORGANISATION FOR THE SAFETY OF AIR NAVIGATION EUROCONTROL EUROCONTROL EXPERIMENTAL CENTRE NETWORK EFFECT: A POSSIBLE MODEL TO HIGHLIGHT THE INTERDEPENDENCIES BETWEEN FLOW MANAGEMENT REGULATIONS EEC Note No.22/2006 Project: NCD-1-CD-FLOW Issued: December 2006 The information contained in this document is the property of the EUROCONTROL Agency and no part should be reproduced in any form without the Agency s permission. The views expressed herein do not necessarily reflect the official views or policy of the Agency.

2 This document has been collated by mechanical means. Should there be missing pages, please report to: EUROCONTROL Experimental Centre Publications Office B.P BRETIGNY-SUR-ORGE Cedex France

3 REPORT DOCUMENTATION PAGE Reference: EEC Note No. 22/2006 Originator: EEC - NCD (Network Capacity and Demand management) Research Area Sponsor: EUROCONTROL Experimental Centre Security Classification: Unclassified Originator (Corporate Author) Name/Location: EUROCONTROL Experimental Centre Centre de Bois des Bordes B.P.15 F Brétigny-sur-Orge CEDEX FRANCE Telephone: +33 (0) Sponsor (Contract Authority) Name/Location: EUROCONTROL Agency 96, Rue de la Fusée B Brussels BELGIUM Telephone: TITLE: Network Effect : A Possible Model to Highlight Interdependencies between Flow Management Regulations Author Brankica PEŠIĆ LE FOLL (EGIDE) Contributors Date 10/2005 Pages 106 Figures 44 Tables 12 Annexes 7 References 14 Serge MANCHON (EEC) Vojin TOŠIĆ (University of Belgrade) Claus GWIGGNER (EGIDE) Project Task No. Period NCD-1-CD-FLOW Sponsor 2005 Distribution Statement: (a) Controlled by: Head of NCD (Network Capacity and Demand management) (b) Special Limitations: None (c) Copy to NTIS: YES / NO Descriptors (keywords): Regulations, Interdependences, COSAAC Simulations, Tree-Based Method. Abstract: This master thesis addresses the problem of interdependencies between air traffic flow management measures called. A regulation aims at avoiding congestion at the level of a piece of airspace i.e. aims at avoiding too many aircraft getting through a piece of airspace during a certain time period (generally, one hour). Traffic flows link pieces of airspace. Thus, interaction between is likely to happen. Propagation of this impact is known as network effect. In this thesis, network effect is analysed as a system behaviour problem for which it is difficult to give a mathematical formulation. For this reason, network effect has been modelled by applying a tree-based method. Tree-based methods belong to the class of data-mining algorithms, one of the supervised learning methods. As a result of this application, a classification tree that presents the prediction model has been generated. Since the objective of this research study was to create a model that would detect interaction in the general case, the tree should detect interactions without taking into account a specific day. Interactions which are significant could be analysed. The accuracy of prediction depends on available data as well as on the tree size. The choice of a relevant indicator is important for interaction detection. Therefore, in this report several indicators were tested such as an indicator delta, or difference in delay in comparison to the baseline delay. The conclusion presents directions for future work and possible operational use. i

4 Intentionally blank ii

5 EVOLUTION SHEET Date Change status Changes Version 23/08/05 CREATION Draft /10/05 UPDATE All chapters /01/06 UPDATE All chapters /11/06 UPDATE Chapter 1 named annexe a 1.1 iii

6 EUROCONTROL EXECUTIVE SUMMARY This master thesis is the result of an internship realised in EUROCONTROL Experimental Centre (EEC) in the Network Capacity and Demand Management (NCD) Research Area. It was sponsored by EUROCONTROL. This internship permitted Mrs Brankica PEŠIĆ LE FOLL to accomplish her graduated studies at the Faculty of Transport and Traffic Engineering in Belgrade. This thesis addresses the problem of interdependencies between air traffic flow management measures called. A regulation aims at avoiding congestion at the level of a piece of airspace i.e. aims at avoiding too many aircraft getting through a piece of airspace during a certain time period (generally, one hour). Traffic flows link pieces of airspace. Thus, interaction between is likely to happen. Propagation of this impact is known as network effect. In this thesis, network effect is analysed as a system behaviour problem for which it is difficult to give a mathematical formulation. For this reason, network effect has been modelled by applying a tree-based method. Tree-based methods belong to the class of data-mining algorithms, one of the supervised learning methods. As a result of this application, a classification tree that presents the prediction model has been generated. Since the objective of this research study was to create a model that would detect interaction in the general case, the tree should detect interactions without taking into account a specific day. Interactions which are significant could be analysed. The accuracy of prediction depends on available data as well as on the tree size. The choice of a relevant indicator is important for interaction detection. Therefore, in this report several indicators were tested such as an indicator delta, or difference in delay in comparison to the baseline delay. The conclusion presents directions for future work and possible operational use. EEC Note No.: 22/2006 iv

7 EUROCONTROL ACKNOWLEDGEMENTS This master thesis is result of an internship realised in EUROCONTROL Experimental Centre (EEC) in the Department for Network Capacity and Demand Management (NCD). It is sponsored by the EUROCONTROL. This internship permits me to accomplish my graduated studies at the Faculty of Transport and Traffic Engineering in Belgrade. I would like to thank Serge MANCHON (EEC) and Dc Vojin TOŠIĆ, my mentor, Professor from the Faculty of Transport and Traffic Engineering in Belgrade for their support in this work. Also I would like to thank Claus GWIGGNER (EGIDE PhD student) for his enthusiast participation and fruitful comments. I would like to thank the CFMU for a three day visit of their Operational Room and to all the people in NCD. I am very happy to have had the opportunity to visit the CFMU and to see the pre-tactical and tactical phases of flow and capacity management. During this visit, I spent time with the Network Management team and during the last day of my visit I spent with the Tactical Network Coordinator in order to see how he manages the network effect. This visit was of great importance for me, a human experience that helped me better understand the ATFCM and visualise the activity and importance of the CFMU in air traffic flow and capacity management. And last but not least many thanks to my family for their moral support. EEC Note No.: 22/2006 v

8 CONTENTS INTRODUCTION... 1 SOME DEFINITIONS PROBLEM FORMULATION PROBLEM DEFINITION DIFFERENT MODELLING APPROACHES FOR NETWORK EFFECT PROBLEM RELATED WORKS METHODOLOGY TO MODEL NETWORK EFFECT DESCRIPTION OF TREE-BASED METHOD INTRODUCTION TO DATA MINING INTRODUCTION TO SUPERVISED LEARNING TREE-BASED CLASSIFICATION AND REGRESSION CLASSIFICATION TREE IN R - LIBRARY (RPART) DATA COLLECTION AND ANALYSIS COSAAC - COMMON SIMULATOR TO ASSESS ATFM CONCEPTS METHODOLOGY FIGURE 4. SIMULATION METHODOLOGY Batch mode DATA COLLECTION Simulations in COSAAC ANALYSIS OF SIMULATION RESULTS Analysis of grouping LFEESE: traffic demand and delay Analysis of sector LFEUN: traffic demand and delay The Analysis of sector LFEUEXE: traffic demand and delay APPLICATION OF CART AND CLASSIFICATION TREE ANALYSIS ANALYSIS OF A CLASSIFICATION TREE, ESTIMATION OF DELAY ANALYSIS OF CLASSIFICATION TREE, ESTIMATION OF RELATIVE DELAY ANALYSIS OF CLASSIFICATION TREE, OUTPUT -1, 0 OR VALIDATION USING DELAY CALCULATION IN NEVAC INTRODUCTION TO NEVAC VALIDATION OF CLASSIFICATION TREE RESULTS BY USING NEVAC TOOL Validation of the detected interactions and estimations classification tree gives if regulation rate for LFEESE change Validation of detected interactions and estimations that a classification tree gives if regulation rate for LFEUEXE change Validation of detected interactions and estimations that the classification tree gives if regulation rate for LFEUN change Validation of detected interactions and estimations that the classification tree based on output (-1, 0, 1) gives EEC Note No.: 22/2006 vi

9 8. CONCLUSION ANNEXES ANNEX A ANNEX B ANNEX C ANNEX D ANNEX E ANNEX F ANNEX G ANNEX H EEC Note No.: 22/2006 vii

10 LIST OF FIGURES Figure 1. First (to the left) and second (to the right) points to split Figure 2. Third splitting point and corresponding tree for those three splits Figure 3. Full partitions where partitions are pure and the full tree Figure 4. Simulation methodology Figure 5. On the left, the traffic demand for EDUUFF5Y traffic volume before and after slot allocation is represented. The green line shows the maximal throughput. On the right, the individual delay generated by each capacity constraint, which has contributed to at least 20% (display threshold) of the total CFMU delay, and distribution delay (fairness graph, inequality rate of 74,4%), is given Figure 6. Traffic demand in Reims ACC compared to traffic demand in the ECAC area Figure 7. Traffic demand in Reims ACC Figure 8. Daily distribution of traffic demand in Reims ACC Figure 9. Regulation referent locations in the ACC Reims Figure 10. Traffic orientations in LFEEACC for 25th June Figure 11. Correlation between total delay and number of delayed flights in grouping LFEESE Figure 12. Delay in sector LFEESE in function of regulation rate Figure 13. Distribution of traffic demand in LFEESE for 25 June Figure 14. Distribution of traffic demand in LFEUEXE for 25 June Figure 15. Distribution of traffic demand in LFEUN for 25 June Figure 16. Delay in LFEESE depending on regulation rate in LFEUEXE Figure 17. Delay in LFEESE when regulation rate for LFEUEXE is Figure 18. Delay in LFEESE when regulation rate for LFEUEXE is Figure 19. Delay in LFEESE depending on regulation rate in LFEUN Figure 20. Delay in LFEUN for regulation rate LFEUEXE 35, LFEESE Figure 21. Delay in LFEUN depending on regulation rate in LFEUEXE Figure 22. Delay in LFEUN depending on regulation rate in LFEESE Figure 23. Delay in LFEUN depending on regulation rate in LFEUN Figure 24. Correlation between total delay in sector LFEUN and delayed flights Figure 25. Delay in LFEUEXE for every simulation day Figure 26. LFEUEXE delay for different scenarios, LFEUN Figure 27. Delay in LFEUEXE depending on regulation rate in LFEESE Figure 28. Correlation between total delay and number of flights in LFEUEXE Figure 29. Classification tree when regulation rates and day are variables Figure 30. Graphical presentation of relative cross-validation error in the function of tree size Figure 31. New classification tree obtain by pruning with cp= Figure 32. Classification tree for LFEUN (only regulation rates on the left and with day and regulation rates on the right) Figure 33. Classification tree for LFEUEXE EEC Note No.: 22/2006 viii

11 Figure 34. Classification tree for LFEESE according to relative delay (on the left, full tree and on the right pruned one) Figure 35. Classification trees for LFEUN when relative delay output (on the left, only regulation rates are independent variables and on the right a categorical variable day is introduced) Figure 36. Classification trees for LFEUEXE when relative delay output (on the left, only regulation rates are independent variables and on the right, categorical variable day have been used) Figure 37. Classification tree for LFEESE when coded output -1, 0 and Figure 38. Classification tree for LFEESE when output coded -1, 0, 1 for variables regulation rate and day (left window full tree, right window pruned tree for cp =0.013) Figure 39. Histogram for difference in delay in LFEESE Figure 40. Classification tree: detection of interaction for a delta between -200 min and 500 min (code 0) Figure 41. Classification tree when variable day is included in tree construction for coding, with a code 0 when delta is between -200 min and 500 min Figure 42. Classification tree for LFEUN Window on the left: Tree when regulation rates are independent variables. Window on the right: Variable day is introduced Figure 43. Analyse of two cases in the classification tree for LFEUN Figure 44. Classification tree for LFEUEXE when output -1, 0 and EEC Note No.: 22/2006 ix

12 LIST OF TABLES Table 1. Indices of Table 2. Dates and traffic for simulations Table 3. Description of sectors Table 4. Labels for a specific day Table 5. Results for Scenario Table 6. Results for Scenario Table 7. Results for scenario Table 8. Results for scenario Table 9. Results for scenario Table 10. Results for scenario Table 11. Results for scenario Table 12. Results for scenario EEC Note No.: 22/2006 x

13 REFERENCES [1]. V. Tošić, O. Babić, Air Route Flow Management Problems and Research Efforts, Transportation Planning and Technology, Vol. 19, pp (1995) [2]. Eurocontrol Experimental Centre/NCD Research Area, NEVAC project, Description of delay calculation (2005) [3]. V. Kapp, Modélisation de flux de trafic par des réseaux de neurones, mémoire de fin d étude (Juin 1995) [4]. L. Negrete, A. Urech, F. Saez, ATM System Status Analysis Methodology, 5th USA/Europe ATM 2003 R&D Seminar, Budapest (2003). [5]. J.E. Sherry, C.G. Ball, S.M. Zobell, Traffic Flow Management (TFM) Weather Rerouting, MITRE (2001) [6]. C. Wanke, N. Taber, S. Miller, C. Ball, L. Fellman, Human-in-the-Loop Evaluation of a Multi-Strategy Traffic Management Decision Support Capability, MITRE (2003) [7]. M. Berge, C. Hopperstand and A. Haraldsdottir, Airline Schedule Recovery in Collaboration Flow Management with Airport and Airspace Capacity Constraints, Boeing Company, 5th USA/Europe ATM 2003 R&D Seminar, Budapest (2003) [8]. C. Verlhac, S. Manchon, Optimization of opening schemes, 4th ATM R&D Seminar (2003) [9]. D. Hand, H. Mannila, P. Smyth, Principles of Data Mining, Massachusetts Institute of Technology (2001) [10]. J. Maindonald, J. Braun, Data Analysis and Graphics Using R - An Example-Based Approach, Cambridge University Press, ISBN (2003) [11]. MiningSpring2003/338F02AD-0DD FCF5A15B57/0/L3ClassTrees.pdf [12]. W. N. Venables, D. M. Smith and the R Development Core Team, An Introduction to R, Version (2002) [13]. NCD EUROCONTROL, COSAAC User Manual, Version with batch mode (May 2005) [14]. R Development Core Team, The rpart Package (2005) EEC Note No.: 22/2006 xi

14 Annex Reference [a]. [b]. EUROCONTROL Performance Review Commission, Performance Report Review 8: An Assessment of Air Traffic Management in Europe during the Calendar Year 2004 (April 2005) [c]. EUROCONTROL, Eurocontrol Medium-Term Forecast, Flight Movements , Volume 1, Edition 2.0 (May 2005) [d]. EUROCONTROL, Air Traffic Flow and Capacity Management, Evolution Plan for the ECAC States, Edition1.0 (2004) [e]. EUROCONTROL, ATFCM operating procedure for flow management position, supplement to the CFMU handbook, Version 1.0 (2004) [f]. EUROCONTROL, Assessing Future ATC Capacity Requirements, A User Guide, Version 1.1, Capacity Enhancement Function (July 2002) [g]. T. Hastir, R. Tibshirani, J. Friedman, The Elements of Statistical Learning, Data Mining, Inference and Prediction, Springer (2003) EEC Note No.: 22/2006 xii

15 BIBLIOGRAPHY A. Bayen, P. Grieder, G. Mayer, C. Tomlin, Lagrangian Delay Predictive Model for Sector-Based Air Traffic Flow, AIAA Journal of Guidance, Control, and Dynamics, D. Bertsimas and S.S. Patterson, 2000, The Traffic Flow Management Rerouting Problem in Air Traffic Control: A Dynamic Network Flow Approach. M. Carey, A. Srinivasan, 1994, Solving a class of network models for dynamic flow control, European Journal of Operational Research 75, G. Davidson, J. Krozel, S. Green, C. Mueller, 2004, Strategic Traffic Flow Management Concept of Operations, AIAA Aircraft Technology, Integration, and Operations Conference, Chicago, IL, September. Hall, H. Schilling, 2005, Flows over Time: Towards a more Realistic and Computationally Tractable Model, Proceedings of the 7th Workshop on Algorithm Engineering and Experiments (ALENEX 05), January. M. Klopfenstein et al., 2003, NAS Genomics: New techniques and initial results for system-level understanding of NAS behaviour, 5 th USA/Europe ATM 2003 R&D Seminar, Budapest. V. Tošić, O. Babić, M. Čangalović, D. Hohlacov, 1995, Some models and algorithms for en route air traffic flow management, Transportation Planning and Technology, Vol.19, pp V. Tošić, O. Babić, M. Čangalović, D. Hohlacov, 1997, A model to solve en route air traffic flow management problem: A Temporal and Spatial Case, 1 st ATM R&D Seminar, Saclay. L. Wojcik, J. Pepper, K. Mills, 2003, Predictability and Uncertainty in Air Traffic Flow Management, CAASD, MITRE, June. N. Xu, G. Donohue, K. B. Laskey, C. Chen, 2004, Estimation of Delay Propagation in the National Aviation System Using Bayesian Networks, George Mason University. EUROCONTROL Experimental Centre, 2005, Improved Configuration Optimiser ICO, Methodology to use a decision support tool, EEC Note No.10/05, Project NCD-F-FM, June. EEC Note No.: 22/2006 xiii

16 ACRONYMS AND ABBREVIATIONS Abbreviation De-Code ACC Area Control Centre AMC Air Space Management Cell ANM ATFM Notification Message AO Aircraft Operators ATC Air Traffic Control ATFM Air Traffic Flow Management ATFCM Air Traffic Flow and Capacity Management ATM EUROCONTROL ATM Strategy for the Years ASM Airspace Management CARAT Computer Aided Route Allocation Tool CART Classification And Regression Tree CASA CFMU Computer Assisted Slot Allocation CDM Collaborative Decision Making CFMU Central Flow Management Unit COSAAC Common Simulator to Assess ATFM Concepts ECAC European Civil Aviation Conference EATMP European Air Traffic Management Programme EEC EUROCONTROL Experimental Centre FAP Future ATM Profile FMP Flow Management Position FMD Flow Management Division (CFMU) HMI Human Machine Interface NAT North Atlantic Traffic NCD Network Capacity and Demand management NEVAC Network Evaluation Validation and Analysis of Capacities NMC Network Management Cell NOP Network Operations Plan PACT Portable ACC Capacity Tool RNAV area NAVigation RVSM Reduced Vertical Separation Minimum TNC Tactical Network Coordinator EEC Note No.: 22/2006 xiv

17 INTRODUCTION Following the forecasts, air traffic is expected to grow, creating more and more congestions in the already congested airspace of Europe. In order to face the increasing traffic, the air traffic community is searching for a solution in increasing the capacity of system elements for growth: new organisation of airspace and of traffic flows, new methodologies in control, new technology with tools that can increase productivity of controllers, etc. Identification of capacity shortfalls at the level of airports and airspace volumes called sectors, according to a forecast demand, will require the possible solutions to be considered: utilisation of available capacity opportunities by applying traffic reassignment; regulation of demand by applying the ground delay programme. The aim of air traffic flow and capacity management ATFCM is to manage airspace capacity whilst minimising restrictions. It will limit the impact on airspace users while ensuring greater efficiency in both traffic and capacity management. Improving ATFCM can be achieved through: a better alignment of the ATM capacity towards the traffic demand, the improvement of the efficiency of the ATFCM process in the balancing of demand and capacity, the development of a network management which will consider the overall impact of restrictions and measures to utilise available capacity. Network management is one of the major concepts of ATFCM. Thus, the need to predict the impact of those solutions to resolve capacity shortfalls could all have upon elements of the system (airports and sectors) is obvious. There are several tools which use simulation in order to quantify that impact but the modelling of the network effect is still an open question. The subject of this master thesis is the presentation of a post-event analysis. Using the results of the arithmetical simulation tool COSAAC (Common Simulator to Assess ATFM Concepts developed by the Eurocontrol Experimental Centre EEC) we shall try to achieve the following: Fist, to find correlations between and traffic demand in sectors, Second, to predict the impact that modification in regulation can have on the network. The work can be introduced through the following steps: In the first part, the problem and the most challenging questions are formulated. Related work is introduced in the second part. In the third part, the methodology for modelling a network effect is given. An approach to resolving problem is presented in part 4. In part 5, a brief description of the COSAAC simulator is given, as well as the collection and analyses of input data. Application of the tree-based method and discussion of results are given in part 6. Part 7 contains validation results of tree-based method compared to results of delay calculation with NEVAC tool (Network Evaluation Validation & Analysis of Capacities), a tool aiming at evaluating the network effect between capacity constraints through macroscopic simulations. This tool is developed by the EEC. EEC Note No.: 22/2006 1

18 Finally, results and directions for future work are suggested in eight and last part. Annex is dedicated to the ATFCM, to a presentation of working practices of Central Flow Management Unit applied to the network effect assessment and possible improvements. Besides, the annex contains the supplement information. SOME DEFINITIONS In the context of Air Traffic Flow and Capacity Management ATFCM, it will be useful to clarify the meaning of certain terms: Area Control Centres (ACC): The airspace is divided in ACCs and each ACC is subdivided into individual airspace blocks called sectors. Traffic demand: All flights flying through an analysed airspace. Collapsed sector: According to traffic demand and to manpower, some sectors can be grouped from time to time. One group of sectors is called a collapsed sector. Possible collapsed sectors are defined long time in advance during the ATFCM strategic phase i.e. no dynamic partitioning mechanism can be envisaged due to operational constraints (for example, it is obvious that two sectors that can be collapsed have to be adjacent. Moreover, the radio frequency of one of the two sectors shall be available on the second one). Airspace network: It consists of airspace entities (sectors and collapsed sectors). Capacity: The capacity of a sector is a maximum sector entry rate (number of flights per hour) for the sector to be handled safely. Congestion: Congestion in one airspace entity arises when traffic demand is higher than its capacity (maximal hourly throughput). Flow and capacity management: The purpose of this activity is to optimise utilisation of existing network and to resolve congestion problems. Traffic volume: It is an element allowing the selection of a specific volume of air traffic, in order to compare the traffic load and the available capacity during the activation period. Regulation or restriction: A regulation is a flow management measure that is defined in order to regulate traffic demand in case of congestion. It is defined by a number of consecutive time periods, each of them being characterised by a flow rate. A flow rate is the maximal number of aircraft that can enter a protected traffic volume. Generally, a traffic volume represents an entire sector or a collapsed sector. When capacity of the sector or collapsed sector has to be spread over several flows or when some flows have to be excluded from the sector or collapsed sector for operational reasons, then explicitly included, excluded, or exempted flow are defined. Restriction can be applied on a given airspace where traffic demand exceeds the airspace declared capacity or regarding impact to the network, to find dedicated areas where restrictions could be implemented in order to protect traffic in a given airspace. Regulation plan: A regulation plan is a set of that aims at protecting the entire European airspace during one day. During the day of operation, have to be continuously adapted according to changes in available capacity, to deviate between the predicted traffic demand and the actual one. Regulation plan also includes traffic assignment measures (re-routing, FL capping ) in order to optimise traffic/demand balancing. Network effect: Regulations might have overlapping effects being linked by traffic flows. Thus, if a regulation needs being modified, this might have upstream and downstream impact on other but also on non-protected sectors or/and on collapsed sectors. That impact is called network effect. EEC Note No.: 22/2006 2

19 1. PROBLEM FORMULATION 1.1. PROBLEM DEFINITION From the brief description of CFM given in ANNEX a, one can see that there is a direction which can be taken to improve ATFCM tactical and pre tactical phases. One possible improvement is to evaluate effect of change in regulation plans to overall capacity/demand balance. Therefore this report presents one study which has as its goal to propose one way to model network effect, to find and to quantify inter-dependences between CFMU-like. Interdependencies between mean that the modification of one can impact upon others. Regulations are linked by flows; intuitively, one can assume the hypothesis that the inter-dependence between is a function of the traffic demand and regulation rates. What are existing relations between? What metrics are to be used in order to quantify that dependence, or degree of dependence? What is the overall impact of a restriction? How that impact can be quantified? Answering those questions will clarify this network effect phenomenon DIFFERENT MODELLING APPROACHES FOR NETWORK EFFECT PROBLEM Behaviour of an ATM system is affected by many factors that make it hard to predict what will happen. Therefore modelling of system behaviour is a difficult problem. It can be said that the network effect problem is a problem of this kind. Several methods have been developed in order to model behaviour of the system. The problem of network effect cannot be solved by simply applying existing maximum flow algorithms because of the dynamic character of demand. Traffic demand on the network consists of superimposing on that network, traffic flows between all origin-destination pairs. These flows are strongly dynamic with characteristics that change over time and are stochastic. Therefore, it is not possible to resolve dynamic and stochastic problems using classical maximum flow algorithms as Dc Tošić et al. [1] showed. Specially adapted algorithms have to be applied. Multi criteria models might be explored for network effect problem. Regarding related works in recent research, a problem of network effect is relatively new and the only approach which I met, at present, is based on simulations like in the NEVAC tool (Network Evaluation Validation & Analysis of Capacities) [2]. Metrics which are used to quantify interdependences are overloaded by time periods, delays and numbers of flights in sectors. There is also a possibility to quantify network effect by assessing the number of flights that propagate delays in the network. This allows identification of flights that penalise sectors and protect others. Modelling traffic flow using neural networks, one of models for multiple interacting system elements, has been studied by Vincent Kapp in [3]. The objectives of this study were to model the consequences of modification in one regulation on traffic in a sector. He concluded that it is possible to model important traffic flows using a linear model and that small traffic flows are difficult to model. EEC Note No.: 22/2006 3

20 What can be done in this study in order to model network effect? As the ultimate goal is to identify interaction between, a method that automatically detects interaction has been chosen to be used in this study. A tree-based method will be applied to the input data set, to model interaction between. It is expected that results of analysis could help in decision making when a change in regulation plan is needed, to avoid combination of that could generate greater delay and to avoid critical regulation rate. EEC Note No.: 22/2006 4

21 2. RELATED WORKS Different Air Traffic Management problems, like resolution of conflicts, re-routing problems etc. are subjects of many research studies. Early research efforts in the area of en route flow management problems were reported in 1981 and they still going on. Predicting the impact of flow management measures on the capacity/demand balance ECAC wide is a relatively new direction in research. Moreover, the problem of modelling this network effect is still open. In fact, there is no relevant paper concerning network effect assessment. However, it is interesting to review research efforts that could have common points with my approach to the problem i.e. finding inter-dependences between and modelling of network effect. In this section some posters of those works are given. In paper [4], L. Negrete, A.Urech, and F. Saez, present analytical methodology designed to evaluate the current status of a given ATM system. This methodology has for it s objective, to ensure that the capacity of the system is adapted to match the air traffic demand well in advance. To develop this methodology, an analytical or inductive method has been followed. Used data and the assumed hypothesis, played an essential role in obtaining results. Specific evaluations were avoided, ones that could bias the result towards subjective estimations. Using diagnostic and analysis methodology the relationship between the system elements (sector groupings) have been identified. Network effect analysis has been used to evaluate proposed grouping of sectors. The relationships between offered performances of the system components (ATC sectors capacity) were determined through the identification and quantification of main traffic flows and volume of traffic shared between them. With the goal of evaluating network effect i.e. the impact that the status and actions on a sector may have on the rest of the system, a quantitative indicator, the traffic shared between sectors, was defined as the number of aircraft per day that cross any pair of ATC sectors in order to identify the relations among the different components of the system and to quantify their intensity. Sectors were ordered by nominal criticality (the sum of saturation factor and occupation factor) and presented in two entry tables in order to visualise the degree of interdependence amongst sectors. The criterion to group the sectors was exchange of movements greater than a given value. Three categories of groupings have been found: critical (not able to cope with the air traffic demand), non critical (can efficiently cope with air traffic demand), and critical and support sector grouping (mix of the two others). This paper was useful in definitions of methodology to model network effect. In the following, some papers that deal with weather problems in air traffic management in the USA and the impact of proposed strategies to the rest of the system are presented. To avoid a bad weather situation, flights are re-routed, delayed or cancelled. Those actions have impact not only on flights which are planned to fly over a restricted area but also to the traffic around. Several tools, simulators, were developed to help in decision making. EEC Note No.: 22/2006 5

22 The Collaborative Routing Coordination Tool (CRCT), designed by MITRE/CAASD is presented by J. Sherry et al. in [5]. The CRCT is developed in order to deal with weather uncertainty and the influence that it produces on the National Airspace System. The CRTC is a simulator which integrates a set of decision support concepts and capabilities like rerouting functionality, identification of aircraft which enter a flow controlled area (FCA) and assessment of the impact of proposed reroutes on sector traffic volume. Automation-assisted Weather Problem Resolution (AWPR) is a new area of TFM research which builds upon the baseline CRCT platform by providing increased automation and initial integration of multiple strategies for addressing traffic congestion. To effectively manage the flow of traffic through sectors that are in close proximity to areas of severe convective weather, the AWPR concept simplifies the reroute traffic patterns by defining reroute corridors. Although the rerouted traffic will add increased traffic volume to those sectors, it will not overly add to their traffic complexity. Assessment metrics for planned reroutes include sector loading, extra air miles flown, total aircraft delay, and ground delay. The results of simulations using this sophisticated tool are visualised. NAC Monitor Display depicts impact of proposed action to the sectors: the maximum number of flights predicted to be in a given sector during a one minute period, and predicted change to sector loading. Further development of the CRCT tool is presented by C. Wanke et al in paper [6]. Actions to manage demand are themselves complex (presence of severe weather or unusual high demand) and interact in difficult-to-predict ways. Therefore traffic managers, whose goal is to choose a set of actions to maximize throughput while maintaining safe traffic volume, need a tool which has the capability to evaluate the impact of the multiple, different flow management strategies (the sector load impact and the imposed delays due to the combined strategy). The new Traffic Flow Management Integrated Impact Assessment (IIA) decisionsupport concept, designed to improve planning and execution of complex traffic management actions is discussed in this paper. The CRTC tool has been used to evaluate the individual influence of each strategy re-routing and Mile In Tail and with IIA the impact of combination of both. The impact is presented by sector load impact and delay. The tool is found to be very useful and permits better control of sectors volume, with less imposed delay on airspace users. From those two papers it can be concluded that network effect is quantified using simulation. Inter-dependences are not analysed. It seems that is very difficult to analyse all interactions which result in the behaviour of the system. The flow management problem is also subject of research in airline companies in the United States. The focus of flow management in the U.S. is primarily an airport arrival capacity limitation but the U.S. system is now experiencing a growing need to protect airspace sectors from overload, particularly due to convective weather system. Thus, the U.S. is taking on an increasing role in coordinating combined re-routing and delay strategies to avoid sector overloads. Flow management service needs extending to include a more complete set of constraints, combined airport and airspace resources, a complete set of delay, re-routing and cancellation options and real time collaboration between service providers and airspace users. EEC Note No.: 22/2006 6

23 M. Berger & al., in paper [7], present an initial implementation of an analysis capability to explore a range of collaborative flow management operational concepts for such an extended flow management service. The Boeing National Flow Model is an event-based simulation which incorporates aircraft and ATC operations (airports with landing and take-off rates, sectors with transit capacities, flight plans with airways and waypoints, and actual weather with associated forecasts derived from historical weather). This tool is a major component of the Boeing ATM preliminary design toolset, aimed at supporting trade studies on ATM operational concepts and architectures in early phases of a major modernization step. The purpose of an airline Schedule Recovery Model is to represent a variety of current and future adaptive airline behaviours in which an airline reacts to forecasts of reduced airport and airspace capacities by re-planning the flight schedule. It is also interesting to have a look to the work of C. Verlhac and S. Manchon [8] in the optimisation of opening schemes (descriptions of different ACC s configurations during a day). They developed two models for local resolution of an opening scheme and the third models, global resolution, where degree of dependences between airspace entities was introduced in order to take into account propagation of delay from predecessors proportionally to their degree of dependences. This model is a kind of macroscopic slot allocation process based on the traffic flow and not on individual aircraft. This model is interesting to mention because a time and space were integrated in the optimisation. Finally, the NEVAC (Network Evaluation Validation & Analysis of Capacities) tool is developed in the context of the NEVAC project [2], which consists in the development of tools and methodologies for the Enhancement Capacity Function of EATMP (European Air Traffic Management Programme). The NEVAC tool is developed by the FAP (Future ATM Profile) development team. NEVAC permits simulation of flight delays, extraction of analysis periods, and calculation of ACC baseline capacities and evaluation of protection effects between two ACCs. New algorithms are intended to be developed as detailed analysis of the ACC demand, improved simulation of the demand increase, configuration optimisation (opening schema-what sectors are open for traffic) and calculation of profiles at the sector level with target delay, hold. The function of a delay calculation in NEVAC is of particular interest in the simulation of network effect. Network effect is measured by performances of the system, number of protecting and penalising flights for a given time period. A protecting flight is one which will be delayed for the next integration period and which leaves a place for other traffic (it decrease saturation of the system). In comparison, a penalizing flight is one which has been delayed in one of the previous time periods and which will be realised in an actual busy time period. Evaluation of the impact of creating/modifying/cancelling a regulation has on the performance of the system is possible to do in a short time. In the next part, the methodology of the proposed approach is presented. EEC Note No.: 22/2006 7

24 3. METHODOLOGY TO MODEL NETWORK EFFECT The objective of the methodology shall aim at highlighting the inter-dependences between, which permits identification of combinations of that are critical, and to predict delays. The methodology to follow, defines general criteria, which are supported in the assumptions that are used to create scenarios, to collect data. The proposed method is a statistical one, which predicts and finds interactions between elements. Network effect can be detected through simple results analysis. The steps which have been followed in order to model and implement the network effect, and, ultimately to compare it to NEVAC simulation results based on the same input data set, are briefly described below: Definition of the principal assumption: regulation applied to sector; Definition of the metrics that would allow modelling the system s behaviour; Tree-based method: description of the method with its advantages and drawbacks; Scenario characterization: process to obtain all the necessary data corresponding to the scenario, introduction to the COSAAC simulation tool used to collect data and to analyse results of simulation; Application of the tree-based method: Identification and quantification of interactions among using the performance of the system, the delay, as the indicator; Comparing results with results of the NEVAC tool: introduction to NEVAC, results for characteristics cases. Conclusions: formulation of conclusions after application of tree-based method, validation, and question of possible use in an operational environment. Regulations are applied to traffic volumes. As a simplification, the focus is on traffic volumes that would protect one entire sector or group of sectors. This does not have impact either on the quality of the analysis or on the model itself, but it reduces time in treating complex traffic volumes based on multiple traffic flow definition. In conclusion, the choice to apply regulation over sector or collapsed sector, is aiming at creating a simple understandable example which will be used to apply the tree-based classification method and carry out analysis. In order to model behaviour of the system, it was interesting to collect data about generated delay for a given regulation plan as well as the number of flights by 5-minute time periods. Change in delay will be used as an indicator of interaction between when rate of applied regulation is modified. Increase or decrease of delay in any other sector where regulation rate did not change is a sign of interaction. In the following, this methodology is developed step by step. EEC Note No.: 22/2006 8

25 4. DESCRIPTION OF TREE-BASED METHOD In order to present the tree-based method, the literature about data mining, supervised learning and about data analysis in [9], [10] and [11] have been used. The tree based method is one of data-mining algorithms, supervised learning method which will be applied to the input data set in the part seven. The choice of this method is a function of the nature of the network effect problem. It is a complex behaviour problem where many elements interact. Supervised learning permits learning about a system when the relation between elements is not known and to predict, with certain accuracy, behaviour of the system. The objective of the predictive method, in this particular case, is to estimate delay taking into account the regulation plan with defined rates and to identify dependences between. Between different classification algorithms, it was interesting to use the classification tree, automatic classification, the CART (Classification And Regression Tree) algorithm which is developed by Breiman & al. in Introduction to data-mining, to supervised learning, and detailed description of the tree-based method / classification tree, are given after INTRODUCTION TO DATA MINING Data mining is the analysis of observational data sets to find unsuspected relationship and to summarize data in a novel way that is both understandable and useful to the data owner. It can be applied to large set of data. A data mining algorithm is a well-defined procedure that takes data as input and produces output [9]. It has the following characteristics: task: the algorithm is used to address prediction and interaction detection in our case, the structure of the model or pattern: the decision tree is fitting to the data, the score function: measure of impurity, the search or optimisation method: to search over parameters and structures, to find maximal or minimal scoring function (for example greedy search over structure). Depending on the task, one possible classification of data mining activities is given: exploratory data analysis: to explore data without clear ideas of what we are looking for, descriptive modelling: the goal is to describe variation of the data using for example cluster analysis, predictive modelling: classification and regression, aim to predict etc. The different kinds of representation may be used to present the output of the data-mining algorithm. A model structure is a global description of the data set. Even when some of the measurements are missing, a model can make some statements about the incomplete object. Pattern structure is a local feature of the data. It gives statements about restricted regions of the space. The relationship and summaries derived through a data mining exercise are often referred to models and patterns [9]. Score function quantifies how well a model of parameter structure fits a given data set. The choice of score function reflects the utility of a particular predictive model. Generic score functions, like a sum of squared errors and a misclassification rate are commonly used. EEC Note No.: 22/2006 9

26 The goal of optimisation and search is to determine the structure and the parameter value that archive a minimum or maximum, depending on the context and value of the score function. Between different methods of data-mining, a method of statistical learning will be highlighted. Statistical learning can be applied in many areas of science, finance and industry. A typical scenario in statistical learning consists of measurement where one wishes to make predictions based on a set of data. There is a training set of data to build a prediction model, which will enable to predict the output for new input. There are two types of learning: supervised and unsupervised INTRODUCTION TO SUPERVISED LEARNING In supervised learning, there is a set of variables that might denote inputs, which are measured and have some influence on one or more outputs. The goal in supervised learning is to use inputs to predict the value of the output. The prediction is based on the training sample of previously solved cases [9]. Error in prediction is characterized by score function and the most common and convenient is a squared error. In supervised learning the success of prediction can be measured. Output that one wants to estimate can be quantitative (continuous) or qualitative (categorical). This distinction in output type has led to a naming convention for the prediction task: regression when quantitative outputs are predicted and classification when qualitative outputs are predicted. Regression models play an important role in many data analysis, providing prediction and classification rules, and data analytical tools for understanding the importance of different inputs. For example, in linear regression the most common prediction method, a prediction rule is usually found by minimising a least squares score function. It has been enhanced and new methods have been developed like the tree-based one. In the ANNEX c, introduction to statistical decision theory is given with some details about estimation of prediction error in order to better understand prediction in the tree-based method. In unsupervised learning, the input is observed and there is no measurement of outcome. The goal is to do estimations without the help of a supervisor; providing correct answers or degree of error for each case. It is rather used to describe how data are organised or clustered TREE-BASED CLASSIFICATION AND REGRESSION Tree-based methods are one type of supervised learning method. These methods derive from earlier methodology called automatic interaction detection [10]. They are appropriate when there are extensive data, and there is uncertainty about the form in which input variables will enter into the model. They may be useful for initial data exploration. Exploration of a new data set with tree-based regression or classification may be a relevant way to gain a quick handle on which variables have major effects on the outcome (most significant regulation in our case) [10]. A popular method for tree-based regression and classification is called CART. The basic principle of a tree-based method is to use recursive binary partitioning because it is the simplest one and results are relatively good. It partitions the feature space into a set of rectangles, and then fits a simple model in each one. The process is binary because parent nodes are always split into exactly two child nodes and recursive because the process can be repeated by treating each child node as a parent. EEC Note No.: 22/

27 A key advantage of the recursive binary tree is its interpretability. Comparing to linear models it can adapt to cases where some values are missing; it can create classification and regression trees, and results are immediately in a useful form. An advantage of this method is that it makes it possible to find non-linear and complex interaction data (what can be seen in our application). A weakness of this method is that the tree might not be optimal but only each split is optimal. A major problem with trees is their high variance. A small change in the data can result in a very different series of splits. The reason for this instability is the hierarchical nature of the process: the effect of an error in the top split is propagated down to all of the splits below it. One can alleviate this to some degree by training to use a more stable split criterion, but the instability is not removed. A tree-based method is not suitable for small data sets (in order to learn about system, extensive data is needed). It is not a precision tool and it is not the right tool for every problem. The tree-based methodology is relatively easy to use [10]. It may be applied for two broad types of problem, classification and regression, depending on the output variable. A regression tree is constructed if the output is a continuous variable, and if the output is a categorical variable then a classification tree is built. Categorical data are present by factors, a numerical vector that has integer indices for unique field. The number of indices present the number of categories in which the data will be classed. Here, the tree-based method, a classification tree technique is discussed. The objective is to construct a tree that will present a model to estimate an output (dependent variable, here categorical) for a given input (independent variable which can be mix of categorical and continuous ones). The following example is detailed in [11] and here is used to show simplicity of this method. Example: A ride-on mower manufacturer would like to find a way of classifying families in a city into those that are likely to purchase a ride-on mower and those who are not likely to buy one (table with data and plot are given in the following). Income Lot Size Owners=1, ($ 000's) (000's sq. ft.) Nonowners= EEC Note No.: 22/

28 The Income and Lot size are the input variables. The output is a categorical variable with two classes: owners and non-owners and the goal is to find a way of classifying. The idea is to divide data in such a way that each class is as homogeneous or pure as possible. In order to do that one has to define splitting criterion and what to use as a measure of how splitting is good. As the classification tree method is with hierarchy, the right point where the split will be applied has to be chosen. In CART, each variable and all possible split values are examined. If the input value is continuous as in this example, then the point of split is the mid-point between pairs of consecutive values for the variable. In case of a categorical variable, then split choice is always the one on which the set of categorical values can be divided into two. Then, splitting points are ranked, using impurity of partition (heterogeneity of partitions). The split should reduce impurity of partition before the split. There are a number of ways that impurity could be measured. Here, the points that are misclassified will be counted. In this example, for the proposed impurity measure, the best point for the first split is variable Lot Size with its value 19; this point is the first on the ranked list. On the Figure 1 left, one can see partitions and that in each partition there are points that are misclassified (three blue in the partition where the majority is rose and three rose in the majority blue partition). Figure 1. First (to the left) and second (to the right) points to split The next split in the part where the majority is rose, which will reduce impurity, is for variable Income with value (Figure 1 on the right). One new part is pure and other has only one misclassified point. In the part where majority is blue, variable Income is used, with the value to split. Two new parts occur, each having one misclassified point (see Figure 2 left). The discussed splitting is presented by tree on Figure 2 right. Figure 2. Third splitting point and corresponding tree for those three splits EEC Note No.: 22/

29 The tree growth ends when all partitions are pure (homogeneous). The full tree is shown on Figure 3. Each terminal node (leaf) presented in the tree as a rectangle-shaped node corresponds to one of the final partitions. Circles in the tree are nodes that present a classification (decision) rule to obtain pure class. These nodes are used to predict output in the case where one knows only the values of the input variables. One descends the tree (with input variables) in such a way that at each decision node the appropriate branch is taken until one gets to a node that has no successors. The class on the terminal node is a predicted class. It is hard to obtain pure partitions when there are many classes and variables. In such a case, the predicted class is one with the majority (meaning if 10 owners and 2 non-owners, prediction will be owners). If terminal nodes are analysed, it can be seen that some terminal nodes present noise in the tree construction, meaning that some terminal nodes are not accurate for prediction (very few points in partitions). Those terminal nodes are not useful and they should be cut. To do so a pruning procedure will be used. Figure 3. Full partitions where partitions are pure and the full tree In order to determine the tree that has the greatest predictive power, the validation data are used to prune the tree that has been overgrown using the training data. Pruning procedure consists of successively selecting a decision node (circles) and re-designing it as a leaf node (the sub-tree of decision node is looped off). By this way the size of the tree is reduced. The pruning process trades off misclassification error in the validation data, set against the number of decision nodes in the pruned tree in order to choose the tree that captures the patterns but not the noise. It uses a criterion called the cost-complexity of a tree to generate a sequence of trees which are successively smaller to the point of having a tree with just the root node. For example, the cost complexity criterion that CART uses is simply the misclassification error of a tree plus a penalty factor for the size of the tree. The best tree is one in the sequence that gives the smallest misclassification error in the validation data. This tree is called the minimum error tree. Besides this tree the pruning process highlights another tree, the best pruned tree. It is the smallest tree in the pruning sequence that has an error that is within one standard error of the minimum error tree. By this way a new tree of optimal size is selected for prediction. EEC Note No.: 22/

30 More information about regression trees and classification trees, as well as details about CART algorithms, are in ANNEX c. The tree construction is explained together with growing, cross-validation and scoring (prediction with missing values) in the same Annex. A free software R-project (A Programming Environment for Data Analysis and Graphics) which has a package rpart (Classification and Regression Tree Package) will be used in order to apply the tree-based method, CART, to input data set; classification trees in this study CLASSIFICATION TREE IN R - LIBRARY (RPART) R is an integrated suite of software facilities for data manipulation, calculation and graphical display. R is used for newly developed methods of interactive data analysis. Many classical and modern statistical techniques have been implemented in R [12]. Construction of classification and regression trees is based on a CART algorithm. Before each figure in the text, which is the result of rpart, a command in R will be given to favour its utilisation. The Rpart library offers different information about the method applied. Using reports one can obtain, for example, information about predictive accuracy of the tree, command printcp(). Predictive accuracy gives information about estimated error of the tree. In this study, cross-validation has been used as an approach for unbiased assessments of predictive accuracy. The crucial feature of cross-validation is that each prediction is independent of the data to which it is applied. printcp(lfeese.rpart) Classification tree: rpart(formula = Bin ~ R1 + R2 + R3 + Day, data = lfeese, method = "class") Variables actually used in tree construction: [1] Day R1 R3 Root node error: 1115/2738 = n= 2738 CP nsplit rel error xerror xstd One can distinguish three types of error: an error in a tree construction (relative error), an error in prediction with validation data (cross-validation error) and an error of prediction in a new sample. Relative error gives error in prediction for the data that generated the tree: 21% (red value in column rel error). From the following report, an absolute error as well as a cross-validation error could be calculated: Absolute error is the product of the relative error and root node error. This error is often called the resubstitution error rate for prediction of the error rate in a new sample. Xerror, in report relative cross-validation error, is an error rate used to calculate crossvalidation error as a product of root node error and xerror: * =0.09. Crossvalidation error estimates expected error rate with new data in prune process [10]. This kind of report is interesting to analyse in order to clarify the explanation of the previous part concerning the minimum error tree and the best pruned tree. EEC Note No.: 22/

31 The minimum error tree is the tree for xerror, ; the tree with 8 splits. The best pruned tree is one for the error less than the sum of blue values ( = ). This rule used to find the best pruned tree is also called the one standard deviation rule, where the error in validation is less than the sum of the minimum relative cross-validation error and standard deviation for that error. In the application of CART to an input data set, the one standard deviation rule will be used to prune the full tree in order to find optimal tree size. My interest in this method is that it makes it possible to classify input data, to highlight interdependences and to predict with certain probability the output value. It would be useful to do analysis of data in order to detect interdependences which may exist between the elements of the system. In this way, it would be possible to predict, with probability, the impact of some ATFCM measures applied. In order to implement the proposed method, an input data set is collected using the COSAAC tool presented in the following section. EEC Note No.: 22/

32 5. DATA COLLECTION AND ANALYSIS 5.1. COSAAC - COMMON SIMULATOR TO ASSESS ATFM CONCEPTS METHODOLOGY This part gives a brief description of the COSAAC tool and its application to collect data for analysis. The COSAAC User Manuel [13] has been used for the COSAAC description. Air Traffic Flow Management is a complex process and it is useful to have a tool that will help in management. It is very difficult to define flow management measures that are not necessarily of the same type (local traffic re-assignment, flow regulation) and which are often overlapping. COSAAC is one of the tools that can help experts in evaluating precisely and quickly the effects of ATFM measures. Those tools help also in providing objective feedback thus go towards enhancing a collaborative process between the ATFM actors from simple data sharing up to e-conferencing. Nevertheless, a simulation is an approximation of reality based on a model of the real world. With arithmetical ATFM simulations, late flight plan filing, late updating, and generally last minute events (related to traffic demand and regulation) are therefore not considered and thus discarded. COSAAC is an extension to the management of the ECAC area of SHAMAN (a prototype initially developed by the Centre d Études de la Navigation Aérienne). Since 1999, Network Capacity and Demand Management (NCD) has been improving COSAAC. COSAAC is now a tool fully adapted for the management of the ECAC area. It is used by the NCD in support of the CFMU for operational studies. It is used operationally at the CFMU in the Network Management Cell (NMC) of the Flow Management Division (FMD) for post-ops analysis and during the pre-tactical phase. COSAAC is adapted for strategic and pre-tactical studies as well as simulations in the fields of the flexible use of airspace and air traffic flow management. Its human interface (HMI) is based on levels of representations and data granularity: The ATFM problem can be diagnosed at the level of several ACCs, of one ACC, of a group of sectors, of one sector, of a flow and finally down to the individual aircraft. COSAAC is used in a what-if mode to: Analyse the traffic demand and compare it to the available capacity of a sector, of any traffic flow, of a CFMU traffic volume Change capacity according to military activity. Perform a slot allocation following the CASA algorithm (CFMU Computer Assisted Slot Allocation) or any other slot allocation strategy. This allows the evaluation of the cost of a set of capacity constraints based on ACC configurations or/and CFMU in terms of: total ground delay generated by all capacity constraints, total number of flights that have been delayed, individual delay generated by each capacity constraint, periods of delay generation (for improving ACCs configurations), delay distribution (fairness indicator of delay distribution). EEC Note No.: 22/

33 Reroute traffic flows manually or automatically with CARAT 1 or on the basis of a route catalogue. Every flight of a traffic flow can be rerouted individually or according to the representative flight of the group of flights it belongs to. In/de-crease traffic demand applying a traffic growth rate by flow Apply a standard noise to departure hours to simulate operational disturbances and evaluate potential capacity overloads after slot allocation, for example. Figure 4 gives a schema of simulation methodology of the ATFM simulator. The main activity that is performed using an ATFM simulator consists of evaluating the impact of a set of capacity constraints upon the traffic demand, in terms of delay. CASA slot allocation strategy is always used for operational simulations. Figure 4. Simulation methodology The ATFM expert analyses all parameters dealing with delay to evaluate the efficiency of the ATFM scenario he tested, usually by comparing scenarios but also with a scenario of reference based on CFMU archived data. After the analysis of the simulation results, the ATFM specialist can fine tune capacity constraints and start a new session. Simulation in COSAAC can be realised in two modes: using an interface or in batch mode (faster and adapted to many successive simulations). But to use COSAAC properly it is necessary that the data in question must first be prepared and selected. For this purpose, COSAAC contains a reference database for archiving data from the operational environment. 1 CARAT stands for Computer Aided Route Allocation Tool. This project has been developed by NCD. EEC Note No.: 22/

34 COSAAC input data is: 1. Environment ENV data: includes all Air Traffic data which is updated every 4 weeks (AIRAC cycle). The ENV data contains a structural description of the airspace for the country or region concerned: the description of the control centres, with sector boundaries and flight levels; the description of the boundaries and flight levels of the military areas; the beacons or listed cross-over points, and possibly groupings of beacons (in order to facilitate traffic analysis); the airports and airport groupings used for the definition of traffic flows. 2. Flight plans (traffic samples): the flight plans contain the traffic handled for any specific day. Each flight (in each set of traffic data) is characterised by the following data: the general flight description (identifying CAUTRA, origin, destination, aircraft type, departure and arrival times, etc.); the various flow control measures imposed on the flight; the beacon route (with cross-over times and levels); sector route; more general information such as 8.33 equipment, RNAV, RVSM. 3. Opening schemas: describe the development of a centre's configurations in the course of the day. Any arrangement of a sector's basic sectors is called a configuration (groups of sectors and open groupings). The opening scheme of a centre describes, for each successive period in the day, the centre's configuration and the capacity of each of its sectors or groupings in this period. The opening schemes make it possible to quantify the capacity provided by the control system in the course of the day. They are therefore indispensable for any evaluation of load/capacity ratio. They also make it possible to estimate whether flow control delays are required in order to avoid exceeding capacity. 4. Military activation schema: sets out the activation or otherwise of its military areas for each military centre and for each period of the day. It allows the computerised management of capacities which vary as a result of military activation. 5. Regulation plan: COSAAC s interface allows us to create, to edit and to use (or validate) regulation plans. In this context: a regulation is defined by a period during which an element of space (of the opening scheme) needs protections to avoid an overflow of capacity ; a regulation plan can be defined as all the protections held (retained) on the scale of a country. There are several possibilities to edit regulation. To watch or to edit the current regulation plan, the ISACASA s allocation s window has been used in this study. It is often very useful to check the consistency of data. To this end, many consultations and representations (and possibly corrections of data) can be carried out using COSAAC. It is then possible to measure system performance indicators for selected traffic. COSAAC can provide various statistics: counts of traffic distribution observed or estimated delays, assessments of load/capacity ratio, detection of system blockages, etc. EEC Note No.: 22/

35 Figure 5 gives an example of some of the graphical representations that help experts in analysing the traffic demand, for air traffic via EDUUFF5Y in this particular case. The background of the Figure 5 is screenshot COSAAC which gives the traffic orientation. On the left, two load graphs are given. The first load graph presents the traffic demand for flow of traffic via EDUUFF5Y, without capacity constraint. The second graph shows how traffic demand is changed after the application of capacity constraint (59 flights per hour). On the top right corner on Figure 5 the individual cost distribution is presented. It shows what delay is generated by what regulation (capacity constraints), here presented is the option of the most penalising capacity constraints. It can be seen that four capacity constraints are producing 20% of total CFMU delay. This kind of graphical presentation could be useful in detecting if any network effect exists meaning delay produced by other constraint different than one analysed. I have been using this kind of graph in order to choose a hypothetical example where the network effect could be detected. The graph of delay distribution is to present fairness of delay distribution, what % of the flights had what % of the total delay. Delay generated by every capacity constraint Hourly traffic demand/capacity (green line) before and after slot allocation Delay distribution (full equity would be reached if x% of the flights had x% of the total delay ` Figure 5. On the left, the traffic demand for EDUUFF5Y traffic volume before and after slot allocation is represented. The green line shows the maximal throughput. On the right, the individual delay generated by each capacity constraint, which has contributed to at least 20% (display threshold) of the total CFMU delay, and distribution delay (fairness graph, inequality rate of 74,4%), is given. EEC Note No.: 22/

36 The performances of the COSAAC tool are widely used and I have had the opportunity to launch simulations in COSAAC for defined scenarios to collect data for further analysis. COSAAC has been used in batch mode, the fastest way to launch numerous simulations. It offers less presentation of results for analysis but information about delays and sector loads necessary for further application has been collected in the previous step. It is worth giving a brief description of the COSAAC batch mode while further analysis is also required Batch mode When used in batch mode, COSAAC waits for commands on the standard input. It is possible to use the COSAAC batch mode in 3 different ways: interactive mode: all commands are typed in the console where the process is running; file mode: a text file containing all the commands is used as the standard input; pipe mode: the standard output of a process can be connected to the COSAAC batch mode standard input. The available functions of COSAAC batch that we used are the following ones: quit: exit from COSAAC batch mode exec command : runs an external process set [param value]: update a configuration parameter load_ca [-perso] [country] YYYYMMDD [extension]: loads COSAAC environment data load_traf [-perso] YYYYMMDD [extension]: loads a COSAAC traffic file load_flux [-flux_analyse -flux_aug -flux_rrt -flux_rrt_sect -flux_ppc -flux_perso] filename: loads a flow definition file alsecs [-initial -final -realise] [-h_geo -h_strip] [-sch_dep - sch_real] centre [extension]: compute countings on sectors on a given ACC isacasa nom_alloc [-flux [nom_fic_flux]] [-regulation_scheme -regulated_flows -opening_scheme] [-initial -final -realise] [-h_geo -h_strip] [-sch_dep -sch_real]: run the ISA slot allocation module annule_act [-initial -final -realise]: cancels a performed action flight_list ficresult [ flux [nom_fic_flux]] format_sortie type_traf -type_heure: save a flight list reg_graph ficresult flux nom_fic_flux [ type_traf] [-type_heure]: saves a regulation graph The file mode has been used for application in this study. The text file has been created in Perl (Practical Extraction and Report Language) language in order to launch simulations in this mode. Part of this text file is given in ANNEX d. The results of simulations are further adapted, using Perl, to be input in to the tree-based method. But, before the implementation of the tree-based method it would be interesting to take a look at the data obtained by COSAAC simulations. EEC Note No.: 22/

37 5.2. DATA COLLECTION Regarding the real size of a problem, one should generate a regulation plan with in ECAC, but for this report only one simple example for the regulation plan was used. The first idea was to create a hypothetical example with a regulation plan of 10 where regulation rates would vary by ±30% of initial rate value. Two problems appeared; first, the number of combinations for the regulation plan rapidly increased, with an N number of and with a k number of values for regulation rate. If all have the same number of regulation rates then the number of all combinations is k N. For example, if N=10 and k= 10 then there is combinations, and this means launching a huge number of simulations. This process would last too long so the number of was reduced. The second problem was what percentage to choose for the regulation rate variation. Thirty percent is too high for traffic demand and in that case many flights would have to be delayed until the next day, not a realistic case. For these two reasons, the regulation plan was reduced: three were finally used and the variation of the initial capacity rates varied by ±10 percent (seven values for regulation rate for each regulation) which makes 7 3 simulations to launch for a specific day. The COSAAC tool was used for several experiments in order to collect data for the application of the new approach using the tree-based method Simulations in COSAAC The major significance of the COSAAC tool for this paper is that it can give data about traffic demand in sectors before and after the application of the regulation plan. The slot allocation process that aims at satisfying a set of capacity constraints (a regulation plan), matches demand and resource and, as a result, gives counts of regulated traffic in sectors and generated delays at total but also by regulation. The COSAAC tool was used to collect the input data set. The tree-based method was applied to this set. The classification tree aimed at coming to conclusions about relations between. Numerous simulations were launched for several days. The choice of days was made so to allow the comparison of our results with those of NEVAC simulations. During this work, only those days were available for NEVAC delay calculations. The following scenario for simulations and for further analysis was used: traffic for eight Fridays in the summer (peak traffic) of 2004 for all ECAC states, i.e. 11 th June, 18 th June, 25 th June, 2 nd July, 9 th, 16 th, 23 rd and 30 th July. Traffic over collapsed sector LFEUEXE (with R1 as a label in the rest of the report), over sector LFEUN (R2 in the rest of the report) and over collapsed sector LFEESE (R3 in the rest of the report) managed by Reims FMP in France (LFEEACC) was regulated by three flow (Table 1). The CFMU data obtained for the environment and the final traffic demand FTFM, was used for those simulations. The FTFM is composed of the actual flight plans as they were treated by the Computer Aided Slot Allocation tool CASA of the CFMU's Traffic Flow Management System called ETFMS. Table 1. Indices of Indice R1 R2 R3 Regulation LFEUEXE LFEUN LFEESE EEC Note No.: 22/

38 Initial capacities of sectors/collapsed sectors were 35, 33 and 30 flights per hour respectively. Regulation rates will vary by ±10%. For example for the regulation applied on LFEUEXE, regulation rates will be 32, 33, 34, 35, 36, 37 and 38 flights per hour. We collected the results of the following simulations: counts of traffic per sector, the delays per regulation, the number of delayed flights and the list of flights over regulated sectors with delays. Regarding the chosen days, the total traffic demand for the eight Fridays is presented in Table 2. The total number of flights during each day is between 29,521 and 30,514. The traffic sample which is used in simulations is the first number in the traffic sample column. The graphical presentation of the total traffic demand is given in Figure 6. Compared to the total traffic demand in ECAC, the traffic demand for ACC Reims presents about 1/10 of the traffic. Table 2. Dates and traffic for simulations Date Traffic sample ECAC (number of flights) Traffic LFEEACC (number of flights) 11 June June June July July July July July If the regulation rate is not adapted to the distribution of the traffic demand, delays can be heavy. Traffic demand for the eight Fridays in Reims ACC is given in Figure 7and Figure 8. Number of flights Traffic demand of LFEEACC in total traffic demand in ECAC Flights in the rest of ECAC Flights in LFEEACC 11 June June June July July July July July 2004 Day Figure 6. Traffic demand in Reims ACC compared to traffic demand in the ECAC area. EEC Note No.: 22/

39 Figure 7 shows a segment of the traffic demand in the LFEEACC. The y-axe covers the interval from 2250 to 2550 in order to better present the variation of the total traffic demand in LFEEACC, number of flights Traffic demand in LFEEACC Number of flights June June June July July July July July 2004 Figure 7. Traffic demand in Reims ACC Collapsed sectors LFEUEXE, LFEESE and elementary sector LFEUN will be the points of interest. Designed regulation plans will be based on these three reference locations. Daily distribution of traffic demand Number of flights Hour 11 June June June July July July July July 2004 Figure 8. Daily distribution of traffic demand in Reims ACC Why only three? Why LFEUN, LFEESE and LFEUEXE? There is no specific reason to focus this study on LFEACC itself except that SE-NW traffic flows across several sectors in Reims which results in the interactions between distant. The selected sector and collapsed sectors as locations on which regulation would be applied is a result of an assessment with the COSAAC tool in which the network effect analysis is not integrated by nature but which offers information about delays generated in one sector and its distribution among the different that generated the delay (like in Figure 5, in the top right graph). Three have been identified that could be representative of interaction among. The example consists of the applied to the elementary sector LFEUN, to the collapsed sector LFEESE made of sectors LFEE and LFESE, and to the collapsed sector LFEUEXE composed of sectors LFEUE and LFEXE (Table 3). EEC Note No.: 22/

40 Table 3. Description of sectors Elementary or collapsed sector Flight Level (FL) Nb of sectors Declared Capacity (flights per hour) LFEESE (LFEE, LFESE) FL 0 - FL LFEUN FL265 - Fl LFEUEXE (LFEUE, LFEXE) FL FL Collapsed sectors LFEESE and LFEUEXE are neighbours and connected by traffic flows. In fact Figure 9 shows the location of these groupings is LFEEACC in the horizontal plane at FL 270 (27000 ft). LFEESE is between flight level ground and FL 190. Collapsed sector LFEUEXE is between FL 190 and FL 600. LFEUEXE is composed of two elementary sectors: LFEUE which is between FL 190 and FL 310 and LFEXE which is between FL 310 and FL 600. Sector LFEUN is not geographically connected to those collapsed sectors, neither significantly by traffic flows (Figure 10). It spreads between FL 250 and FL 350. This example demonstrates that dependences between that share traffic flows are obviously stronger than those between that do not have any traffic in common, but also that those interdependences may exist even if there is no influence at the first sight due to physical distance between them. Figure 9. Regulation referent locations in the ACC Reims Now, it is interesting to see what delay is generated in the chosen ACC for the three reference locations. Since the change in delay is chosen as the indicator of regulation interaction, delay analysis is presented here after. EEC Note No.: 22/

41 Figure 10. Traffic orientations in LFEEACC for 25th June ANALYSIS OF SIMULATION RESULTS Analysis of grouping LFEESE: traffic demand and delay Results of the COSAAC simulation give some information about relations between delay and the number of flights. Correlation that exists between the total delay and number of delayed flights in LFEESE, according to input data, is presented in Figure 11. This is useful information when searching dependences between, to avoid the data repetition in further application. Correlation between total delay and number of delayed flights in LFEESE Number of delayed flights Delay in min Figure 11. Correlation between total delay and number of delayed flights in grouping LFEESE EEC Note No.: 22/

42 To start delay analysis, a baseline scenario is defined where the following regulation rates: LFEUEXE=35 aircraft can enter LFEUEXE every 60 min, LFEUN=33 and LFEESE=30 flights. The influence that the modification of LFEESE regulation rate produces on delay is presented in Figure 12: clearly, the delay increases if the regulation rate decreases. The difference in delay for different Fridays with the same flight number could be explained by the fact that the distribution of traffic may vary from one day to another: possibly a high peak period followed by a slow traffic period does not create as much delay as a smaller but longer peak that might overlap with another one. Delay in min June June June July July July July July Regulation rate (a/c per hour) Figure 12. Delay in sector LFEESE in function of regulation rate A more detailed analysis can be done within one day. On the 25 June 2004 the regulation rate was not well adapted to the traffic demand in sector LFEUEXE. In Figure 14, where traffic distribution in LFEUEXE is presented, during several hours the traffic load was over regulation rate 35. This produced huge delay. The following Figure 13, Figure 14 and Figure 15 show why there were heavy delays on that day with those regulation rates. Figure 13. Distribution of traffic demand in LFEESE for 25 June 2004 EEC Note No.: 22/

43 Figure 14. Distribution of traffic demand in LFEUEXE for 25 June 2004 Figure 15. Distribution of traffic demand in LFEUN for 25 June 2004 How does that influence the delay in the LFEESE? An interaction between the LFEESE and LFEUEXE is clearly expected. A hypothetical example is chosen to demonstrate the interaction. To show that some other could have influence on delay in sector LFEESE, a network effect illustration, LFEUEXE regulation rate will be changed to show that LFEUEXE will protect LFEESE. Figure 16 shows that LFEESE generates less delay when LFEUEXE regulation rate is 32 (scenario 1) than when LFEUEXE regulation rate is 38 (scenario 2). This means that there is a network effect between LFEESE and LFEUEXE. This is not surprising bearing in mind the orientation of the traffic flows that connect these two and their physic position, as close neighbours. Delay in LFEESE for three scenarios Delay in min Scenario 2 (LFEUEXE 38) Baseline (LFEUEXE 35, LFEUN 33, LFEESE 30) Scenario 1 (LFEUEXE 32) 11 June June June July July July July July 2004 Figure 16. Delay in LFEESE depending on regulation rate in LFEUEXE EEC Note No.: 22/

44 Another interesting thing is to see how delays vary for each of the eight Fridays in LFEESE. Figure 16, shows different delays for different Fridays. For more details, Figure 17 presents how delay is distributed when regulation rate in LFEUEXE is 32 (the lowest rate), in LFEUN 33 (baseline regulation rate) and for LFEESE regulation rate change from 27 to 33 flights per hour. This figure is useful to highlight the interaction that exists when regulation rate changes. Delay in LFEESE for regulation rate LFEUEXE 32 Delay in min June June June July July July July July Regulation rate LFEESE (a/c per hour) Figure 17. Delay in LFEESE when regulation rate for LFEUEXE is 32 From the above said and according to the available data, a possible conclusion is that there is a relation between regulation in LFEESE and LFEUEXE. Depending on the regulation rate over LFEUEXE different delay is generated in LFEESE. If the capacity rate for LFEUEXE increases, the protection of LFEESE decreases. Figure 18 shows how delay in LFEESE changed and increased with regulation rate 38 flights per hour in LFEUEXE. For example, on the 11 th June, when the regulation rate in LFEESE was 27 and the regulation rate in LFEUEXE was 32 (Figure 17), the delay was about 2200 minutes. When the regulation rate in LFEUEXE went up to 38 it increased the delay to about 5500 minutes (Figure 18). Delay in LFEESE for regularion rate LFEUEXE 38 Delay in min June June June July July July July July Regulation rate LFEESE (a/c per hour) Figure 18. Delay in LFEESE when regulation rate for LFEUEXE is 38 EEC Note No.: 22/

45 What is the influence of changes in the regulation rate for sector LFEUN? What is the conclusion from the result of COSAAC simulations? If the regulation rate for LFEUN decreases to 32 or increases to 36, the delay does not significantly change in LFEESE. That means that there is no network effect between these two regulation locations, Figure 19. Delay in LFEESE for tree scenarios (LFEUN) Delay in min Baseline (LFEUEXE 35, LFEUN 33, LFEESE 30) Scenario 1 (LFEUN 30) Scenario 2 (LFEUN 36) June June June July July July July July 2004 Figure 19. Delay in LFEESE depending on regulation rate in LFEUN To summarize the analysis, LFEESE interacts with LFEUEXE and does not with LFEUN Analysis of sector LFEUN: traffic demand and delay From the previous analysis, what could be the conclusion of data analysis for sector LFEUN? No interaction between LFEESE and LFEUN? To analyse delay in LFEUN, Figure 20 gives a variation of delay in LFEUN when regulation rate for LFEUN increases or decreases for each of the eight selected Fridays. For this sector, the initial value for regulation rate was not adapted to traffic demand. The delays were very heavy. Delay in LFEUN depanding on day and regulation rate Delay in min Regulation rate LFEUN (a/c per hour) 11 June June June July July July July July 2004 Figure 20. Delay in LFEUN for regulation rate LFEUEXE 35, LFEESE 30 EEC Note No.: 22/

46 The interaction between of LFEUN and of LFEUEXE is presented on Figure 21. It shows that there is no change in delay in LFEUN if regulation rate in LFEUEXE changes. Delay in LFEUN for three scenarios Delay in min Baseline (LFEUEXE 35, LFEUN 33, LFEESE 30) Scenario 1 (LFEUEXE 32) Scenario 2 (LFEUEXE 36) 11 June June June July July July July July 2004 Figure 21. Delay in LFEUN depending on regulation rate in LFEUEXE Furthermore, the interaction among of LFEUN and of LFEESE is presented on Figure 22. No change in delay in LFEUN when regulation rate in LFEESE changes. Delay in LFEUN depending on LFEESE Delay in min Baseline (LFEUEXE 35, LFEUN 33, LFEESE 30) Scenario 1 (LFEESE 27) Scenario 2 (LFEESE 33) June June June July July July July July 2004 Figure 22. Delay in LFEUN depending on regulation rate in LFEESE Figure 21, 22 and 23, demonstrated that the only influence that could be detected comes from the inherent regulation rate. As for the other two which do not have common traffic and are not direct neighbours, no influence was detected. It is of no interest to do further delay analysis for different regulation rates in LFEUN and changes in other regulation rates because of the absence of dependence. Delay in min Delay in LFEUN when regulation rate change in LFEUN Baseline (LFEUEXE 35, LFEUN 33, LFEESE 30) Scenario 1 (LFEUN 30) Scenario 2 (LFEUN 36) 11 June June June July July July July July 2004 Figure 23. Delay in LFEUN depending on regulation rate in LFEUN EEC Note No.: 22/

47 To end analysis for LFEUN, Figure 24 gives some information about the number of flights that were delayed. When the total delay in LFEUN increased, the number of delayed flights also increased. As was said before, the regulation rate in LFEUN was too low for the traffic demand therefore all flights were delayed. Correlation between total delay and number of delayed flights in LFEUN Number of delayed flights Delay in min Figure 24. Correlation between total delay in sector LFEUN and delayed flights The Analysis of sector LFEUEXE: traffic demand and delay The delay analysis in LFEUEXE will be given here in order to establish dependence with other. The previous analysis revealed a strong dependence between LFEESE and LFEUEXE. By analysing the results, a possible conclusion is that LFEESE does not have much influence on LFEUEXE. First, Figure 25 shows the total delay in LFEUEXE depending on the selected day and the variation of regulation rate in LFEUEXE. For collapsed sectors LFEUN and LFEESE, the regulation rates are 33 and 30 respectively. Delay in LFEUEXE depending on day and regulation rate Delay in min Regulation rate for LFEUEXE (a/c per hour) 11 June June June July July July July July 2004 Figure 25. Delay in LFEUEXE for every simulation day EEC Note No.: 22/

48 On Fridays 23 rd and 30 th of July 2004 there was heavier delay compared to other selected days, probably because regulation rate LFEUEXE did not correspond to the distribution of traffic demand on those days. In order to find an indication of interactions between, an analysis of simulation result was done. The results from several scenarios (different regulation rates) were compared. Figure 26 shows that on the basis of the data, no influence of LFEUN on LFEUEXE was detected. Delay in LFEUEXE depending on LFEUN Delay in min Baseline (LFEUEXE 35, LFEUN 33, LFEESE 30) Scenario 1 (LFEUN 30) Scenario 2 (LFEUN 36) June June June July July July July July 2004 Figure 26. LFEUEXE delay for different scenarios, LFEUN Figure 27 shows that, for available input data, no influence of LFEESE on LFEUEXE was detected. Delay in LFEUEXE depending on LFEESE regulation rate Delay in min Baseline (LFEUEXE 35, LFEUN 33, LFEESE 30) Scenario 1 (LFEESE 27) Scenario 2 (LFEESE 33) 0 11 June June June July July July July July 2004 Figure 27. Delay in LFEUEXE depending on regulation rate in LFEESE EEC Note No.: 22/

49 Figure 28 shows that there is a correlation between the total delay and number of delayed flights in collapsed sector LFEUEXE. Correlation between total delay and number of delayed flights in LFEUEXE Number of delayed flights Delay in min Figure 28. Correlation between total delay and number of flights in LFEUEXE Judging on the basis of simulation results, after the application of the tree-based method to input data set, the detection of interactions between LFEUEXE and LFEESE can be expected. It is likely that regulation in LFEUEXE can influence delay in LFEESE, while the influence in the opposite direction is less probable. It is also possible that no influence of regulation LFEUN to LFEUEXE and LFEESE will be detected. After a quick analysis of simulation results, the tree-based method will be applied to detect and confirm interactions between using delay as a possible indicator of the network effect. EEC Note No.: 22/

50 6. APPLICATION OF CART AND CLASSIFICATION TREE ANALYSIS Results of the CART algorithm application will be presented in this part. Free software R- project with its library rpart in which CART algorithm is implemented has been used for this study. In order to apply a tree-cased method, different literature like: Part 10 of the book Data Analysis and Graphics Using R [10], User manual for R [12] and documentation about package Rpart [14] have been used. The utility of this prediction method is in the detection of influence that one combination of, each being characterised by a flow rate (also called throughput), has on an analysed airspace element (sector, ACC ). Analysing the classification tree, one should detect if one regulation protects another. This will be detected through a change in delay generated by the regulation that is impacted. The constructed tree should have power to predict, with certain accuracy, delay that one regulation combination rate will generate in an analysed sector and highlight the influence of other. The proposed approach will be tested on one hypothetical example. The input data set has 2744 cases per reference location on which one regulation is applied (for example, 2744 cases available for the tree construction in LFEESE). Data contains information about regulation rate, delay, number of delayed flights, and a difference in delay resulting from a comparison to a baseline scenario. This data set contains results of simulations for regulation plans with three and for eight Fridays. For each day, 343 (7 3 ) simulations have been launched. Then following analysis has been done: analysis of interdependencies between sectors where were applied, estimation of delay for a given regulation plan, the class where the delay belongs to, analysis of a classification tree when output relative delay (delay/total delay), analysis of a classification tree when output -1, 0 or 1, estimation of delta. Results of COSAAC simulations were collected, analysed and used for construction of the classification tree. The idea is to model interactions between using available data. Results of the CART algorithm application to results of COSAAC simulation are presented in the following. Some general conclusions are expected to be made and evaluation of the hypothesis that interdependences are the function of traffic demand (depending on the day of traffic: variable day) and regulation rate (R1 for LFEUEXE, R2 for LFEUN and R3 for LFEESE) carried out ANALYSIS OF A CLASSIFICATION TREE, ESTIMATION OF DELAY Conclusion based on data analysis, given in the previous part, was that the delay varies depending on the day of traffic. The histograms in 5.3 indicate that delay analysis of the collapsed sector LFEESE, where interactions between are expected, could be interesting to be carried out. Then, to start, a prediction model for absolute delay will be build for input variables, the regulation rates. The output variable, the estimation of delay, is absolute delay generated for a given plan. Following classification trees will present a prediction model for each sector in which the interaction can be detected. EEC Note No.: 22/

51 LFEESE: The first tree was constructed for the collapsed sector LFEESE. From Figure 18 where a histogram of delay for different regulation rates is given (case when delay is the most important) and from the input file for CART application (see ANNEX e), the output variable delay ranges between 0 and minutes. The CART algorithm determines the number of classes itself thus the number of classes of categorical output variable. In this particular case, CART would generate about classes. With such a number of classes, the problem is more likely to be resolved with a regression tree. As the objective was to detect interaction and not to estimate exact value for delay, intervals for delay were defined in order to reduce number of classes. Each interval presents one class where the expected delay for an applied regulation plan belongs. For example, one class can be characterised by a delay in [0 min, 200 min). There is no precise method to find out the right number of classes. So, the first tree was constructed with 10 classes, each class containing 2000 min. Using available input data, a tree could not be created. Other numbers of classes (50, 75, 100, 150, and 200) were tested but there was no sense in increasing the number of classes. There are not enough data to give a good prediction, using a classification tree, for each class when they are numerous. The conclusion is that variation in number of classes for tree construction does not help much; the prediction accuracy is really weak. Also, it is important to say that use of regulation rates as variables to build a classification tree is not enough in this particular case. That is the reason for introduction of a new variable in the tree construction; the day of traffic. What is the contribution of a variable day in the construction of a classification tree and in the increase of the prediction accuracy? Analysis of a classification tree should give answers to these questions. There is a possibility that the days of traffic have different traffic complexity thus it was not possible to make general conclusions or construct the tree without variable day. Introduction of the categorical variable day when there are 10 classes, gives a new tree presented on Figure 29. > lfeese$delaycut10<-cut(lfeese$delay, breaks=10, labels=false) >lfeese.rpart<rpart(delaycut10~r1+r2+r3+day,data=lfeese,method="class",control=list(minsplit=100)) > plot(lfeese.rpart) > text(lfeese.rpart, cex=0.5) EEC Note No.: 22/

52 Figure 29. Classification tree when regulation rates and day are variables Variable day has eight labels that correspond to each day. It describes traffic for a specific day. Label a is for 11 th June, b for 18 th June and so on (see Table 4). Table 4. Labels for a specific day Label a b c d e f g h Day 11 June June June July July July July July 2004 From the classification tree and report that rpart library offers, one can see that delay in LFEESE is influenced by another regulation. Variables R1 (LFEUEXE), R3 (LFEESE) and day were used in construction of a classification tree and prediction model for delay in the collapsed sector LFEESE. Each terminal node gives an estimation of delay, more precisely, an interval for delay value by index of class, from 1 to 10. Greater class index indicates larger amount of delay. Note that in defining parameters for the tree constructions, control parameter minsplit is set to 100, which means that estimation for each class is based at least on 10 cases. >printcp (lfeese.rpart) Classification tree: rpart(formula = DelayCut10 ~ R1 + R2 + R3 + Day, data = lfeese, method = "class", control = list(minsplit = 100)) Variables actually used in tree construction: [1] Day R1 R3 Root node error: 590/2738 = n= 2738 CP nsplit rel error xerror xstd EEC Note No.: 22/

53 Discussion about error in prediction can be based on the output report that rpart offers and on its graphical presentation, Figure 30. Both the report and the meaning of each column have been explained in 5.4. In order to choose the optimal tree size, on Figure 30 one can see that choosing the complexity parameter (the first value of the curve that is below dotted line) to prune initial full tree, the new tree with 9 splits can be obtained. This tree would present the best pruned tree. A prune in rpart determines a nested sequence of subtrees by recursively 'snipping' off the least important splits, based on the complexity parameter cp. By pruning, some branches were cut and a new better tree is obtained. > plotcp (lfeese.rpart) size of tree X-val Relative Error Inf cp Figure 30. Graphical presentation of relative cross-validation error in the function of tree size The initial classification tree has the absolute error of 10 %. Pruned tree is presented on Figure 31. It has 12% for absolute error. How one can read from the classification tree? A classification rule like IF(x1 a) AND (x2 b) THEN CLASS = 2 is leading to the terminal node and to a prediction of output. It is important to note that the left branch in the tree presents a way to follow if constraints are satisfied and in the opposite case to go on the right one. Following this rule, one is able to make classification rule that introduces all constraints that have to be satisfied to reach each terminal node in the tree. To demonstrate the rule to follow in the constructed classification tree, Figure 31 will be used. From the root node, the first split, there are two principal branches no matter what day is selected for analysis: regulation rate LFEESE if greater or equal to 29.5 flights per hour, condition satisfied, follow left branch, and if it is less than 29.5 the right branch has to be followed. >lfeese.rpart<-prune(lfeese.rpart, cp=0.013) > plot(lfeese.rpart) > text(lfeese.rpart, cex=0.5) EEC Note No.: 22/

54 Figure 31. New classification tree obtain by pruning with cp=0.013 In this way, it is possible to analyse the entire tree and at the same time using a new input data to descend in the tree to the terminal node which gives prediction for the new input variables. Where are the smallest delays in the tree? The tree analysis should start with terminal nodes. The left branch is the same on Figure 29 and Figure 31 meaning that the pruning process did not change the estimation of delay for this case. Regarding the pruned classification tree on Figure 31, the classification rule like: if regulation applied in LFEESE (R3) has a rate that is greater or equal to 29.5 flights per hour then the classification tree predicts a small delay, can be made. In a terminal node, this is presented by class 1. Small delay could be expected if regulation rate in LFEESE is less than 29.5 and if day is the 2 nd, 16 th, 23 rd or 30 th July The same prediction, class 1 is if LFEESE is between 28.5 and 29.5 and if day 11 th June or 9 th July Increase in delay could be expected if regulation rate for LFEESE is between 28.5 and 29.5 and if day 18 th or 25 th June 2004 then the predicted delay belongs to the class 2. Detailed information about the classification tree is given using command print (lfeese.rpart). This output report gives, for each node in the tree, estimated delay for the set of. This report is given in ANNEX e EEC Note No.: 22/

55 Where do we have important delay? In the tree, Figure 31. New classification tree obtain by pruning with cp=0.013, there is one terminal node where the highest value is 7. This means that one can expect the highest delay in LFEESE (delay in class 7) when constraints presented in the following are satisfied (taken from report in ANNEX e, node #63). >path.rpart (lfeese.rpart, node=63) node number: 63 root R3< 29.5 Day=1,2,3,5 Day=2,3 R3< 28.5 R1>=35.5 How one can detect interactions? Interaction in the tree is presented by input variables that have been used in the tree construction. For the analysed case of important delay, one can see that one interaction is detected, between R1 (LFEUEXE) and R3 (LFEESE). Analysing the rpart report for path.rpart(), the highest delay is reached for Friday 18 th or 25 th June (Day=2,3, label b,c) and if the regulation rate for LFEESE is less than 28.5 and if LFEUEXE is greater then For this case (presented with red line in the tree) one can expect an important delay in collapsed sector LFEESE. This also means that regulation rate for LFEUEXE should be considered when defining LFEESE regulation rate. How one regulation can interact with another? To show an example of interaction, the important delay for Friday 25 th June (day with label c) will be discussed. This case is presented with a combination of red and blue lines in the tree on Figure 31. If 25 th June and if regulation rate for LFEESE is less than 28.5 and if LFEUEXE is less than 35.5 then estimation is class 6, represented with a blue circle in the tree. To show effect of protection that LFEUEXE has on LFEESE, one can vary the regulation rate for LFEUEXE. If it is regulated by 35.5 or more (red line), the LFEESE will be less protected: the delay will increase in LFEESE, class 7. What conclusion can be made from the classification tree for LFEESE? From the previous study, one could conclude that the classification tree permits detection of interactions with LFEUEXE, but they are highly dependent on the day. Regulation over LFEUN has not significant influence on delays in LFEESE; it is not used as a variable in the tree construction. By choosing eight Fridays, regularity has been expected, but it seems that is not likely to exist for available data and for this collapsed sector LFEESE. Also, the classification tree for LFEESE is not sufficiently accurate in the general case in finding all dependences: delay range is large; also the intervals for classes. For two other, classification trees are presented in the following. Each classification tree given on the figure is the best pruned tree. EEC Note No.: 22/

56 LFEUN: For sector LFEUN, classification trees for delay prediction have been constructed and presented on Figure 32. Delay is described with 10 classes. The tree-based method has used only R2 (LFEUN) as a variable in the tree construction. R2>=32.5 R2>=32.5 R2>= R2>=31.5 R2>= R2>=35.5 Day=abce R2>=31.5 Day=abce 2 R2>=34.5 Day=bc R2>=30.5 R2>=30.5 Day=cf R2>= Day=c Day=c Day=dgh 1 2 Day=abce Day=abce R2>=33.5 R2>=33.5 Day=c 4 Day=c Day=c Day=dgh Day=dgh Figure 32. Classification tree for LFEUN (only regulation rates on the left and with day and regulation rates on the right) It has to be said that the classification tree in the left window on Figure 32 is not dependant on the day, meaning that it has a power to estimate delay in general case. Also, it means that in a general case the delay in this sector is not influenced by two other used in this hypothetical example. Introducing categorical variable day, the classification tree becomes very large and only day and the LFEUN variable construct the tree meaning that it does not detect influence of any other regulation. Only change in delay is connected to the change in regulation rate for LFEUN and it depends on the day. EEC Note No.: 22/

57 LFEUEXE: Classification trees for LFEUEXE have been constructed and presented on Figure 33. On the tree in the left window on Figure 33, one can see that in classification tree construction, only variable R1 (LFEUEXE) has been used. The output variable has 10 classes for delay. This tree could be used to make a general conclusion about regulation interactions and prediction of delay concerning hypothetical example. No interaction with two other regulation is detected. Introducing a day as a categorical variable, a new tree could be constructed, a bigger one. For its construction, the rpart uses again only variables R1 (LFEUEXE) and day, meaning that there is no interaction with other two. From this point of view regulation in LFEUEXE is not influenced by other, ones analysed in this example. It can be said that interaction with LFEESE is not a symmetric one because in the classification tree for LFEESE it was found that delay will depend on regulation rate in LFEUEXE. In order to finish the first classification tree analysis, several conclusions can be made. From the previous trees, it can be concluded that obtained results are confirming expectation of interactions detected by input data analysis. The expected interaction between LFEESE and LFEUEXE is not symmetric, meaning that LFEESE does not have the same impact to LFEUEXE as LFEUEXE has on LFEESE. There was no detection of interaction between LFEUEXE and LFEUN and no detected interaction between LFEESE and LFEUN. R1>=36.5 Day=abcdef R1>=35.5 Day=acef R1>=37.5 R1>= Day=abcdef R1>= R1>=34.5 R1>=32.5 Day=abcdef Day=adf Day=abcde Day=acde Day=bce 6 Day=f Day=bf 6 10 Figure 33. Classification tree for LFEUEXE 5 8 EEC Note No.: 22/

58 To avoid present dependence on a specific day for LFEESE it could be interesting to analyse traffic in general and to choose days with the same quality of traffic. Then, new variables might be determined which could be used as indicators of network effect. Since daily delay can vary a lot from one day to another, it can be thought that switching to relative delay as the indicator of interactions could be relevant. If the classification trees were constructed for this new output, the relative delay defined as a ration between delay and total delay, one is still addressing classification trees. Relative delay is characterised by 10 classes ANALYSIS OF CLASSIFICATION TREE, ESTIMATION OF RELATIVE DELAY This part will address tree construction and analysis for a hypothetical example: three in LFEESE, LFEUN and LFEUEXE. The output variable is relative delay, participation of delay in analysed location in total delay, described by 10 classes. The input variables are regulation rates and day. As in the previous, the first analysis will be done for regulation in collapsed sector LFEESE which will be followed by analysis for other two. LFEESE: Introducing a new indicator of interaction detection, the relative delay, did not allow construction of the classification tree for LFEESE, a tree that is not dependant on a specific day. If the day categorical variable is introduced, classification trees for collapsed sector LFEESE are constructed and given on Figure 34. From those two trees it can be seen which one input variables were used in the tree constructions: day, R1 (LFEUEXE) and R3 (LFEESE). The first one, in the left window on Figure 34, is a full tree which is created using available data. To demonstrate one more time the pruning process, the best pruned tree is presented in the right window on the same figure. This second tree will be used for further analysis. EEC Note No.: 22/

59 From the tree it can be seen that estimation of relative delay is dependent on regulation rates in LFEUEXE, LFEESE and on the day. It has to be said that the pruned tree offers a possibility to analyse only the cases for 18 th and 25 th June (day with labels b, c). For other days, there are no sub-trees, the classification tree estimates small relative delay, the class 1 and no interactions. Day=adefgh 1 R3>=27.5 Day=fgh R1< R3>=29.5 R1< R3>=28.5 Day=b Figure 34. Classification tree for LFEESE according to relative delay (on the left, full tree and on the right pruned one) The interaction between two LFEESE and LFEUEXE can be detected and discussed if one follows a red line on Figure 34. From the right branch in the pruned tree it can be concluded following: if regulation rate for LFEESE (R3) is less then 29.5 and if regulation rate for LFEUEXE (R1) is less then 34.5 and if the days are 18 th or 25 th June then there is less delay than for the case where LFEUEXE is greater or equal to 34.5 for the same days. Talking about predictive accuracy of the tree, it is possible to calculate error using the rpart reports. The absolute error is 10% for the LFEESE pruned tree. Interaction that was detected in the classification tree is the expected one, between LFEESE and LFEUEXE, one that had been detected in the analysis of simulation results. EEC Note No.: 22/

60 LFEUN: For LFEUN, it is possible to construct a classification tree for input variables and regulation rates. Relative delay for this collapsed sector is between and The classification tree presented in the following is a pruned one in order to do the analyses on the best tree. This tree is on the left window, Figure 35. Some new interactions could be detected. Variables R1 (LFEUEXE) and R2 (LFEUN) have been used in the tree construction. That means that interaction between LFEUN and LFEUEXE is detected. To demonstrate this interaction, a red line on Figure 35 has to be followed. The highlighted case shows that there is less relative delay if regulation rate in (R1) LFEUEXE is less than 35.5 and if LFEUN (R2) greater or equal to 33.5, relative delay is in 2 nd, 3 rd or 4 th class. In the case when LFEUEXE is greater or equal to 35.5, relative delays belong to class 5. On Figure 35, in the right window, a pruned classification tree where a categorical variable day has been used in the tree construction together with R1 (LFEUEXE) and R2 (LFEUN) is presented. This tree gives an estimation of relative delay for a specific day of traffic. This tree is not interesting to analyse because introducing a variable day is reducing utility of the prediction model. One has to be careful in exploitation of predictions from the LFEUN classification tree because the calculated absolute error in prediction is 50 %, meaning that this model is not really accurate in prediction. Where does it come from this unexpected interaction? As total delay is taken into account in the output variable, it could reflect when regulation rates are not well adapted to the traffic demand. This could influence the relative delay and it may be a reason for the unexpected interaction. R1< 35.5 R2>=33.5 R2>=33.5 Day=bcgh Day=cgh Day=bcegh R2>=35.5 R1< R1< Day=bcgh Day=bcgh Day=abde R1< R2>=31.5 R1< R2>= Figure 35. Classification trees for LFEUN when relative delay output (on the left, only regulation rates are independent variables and on the right a categorical variable day is introduced) EEC Note No.: 22/

61 LFEUEXE: For relative delay in the collapsed sector LFEUEXE, the following classification trees were constructed and presented on Figure 36. Relative delay for LFEUEXE is between and In the left window, the classification tree constructed using only input variables regulation rates is given. This tree shows detected interactions between LFEUN (R2) and LFEUEXE (R1). Analysing this classification tree, one can see that there are similarities with the classification tree for LFEUN in the previous figure s window on the left. On the tree, Figure 36, the situation is opposite than on the tree Figure 35: relative delay would decrease with a smaller regulation rate for LFEUEXE meaning that delay induced by LFEUN (R2) will take part in proportion to relative delay. The classification tree for the relative delay when a variable day is used is presented in the right window, Figure 36. This tree, when a day is introduced, is similar to those used for the classification tree in the window on the left. Calculated absolute error for this tree is 52%, an important error in prediction. As in the case for LFEUN, the prediction model is not really accurate. Detected interaction with regulation in LFEUN has to be interpreted with care. R1>=35.5 R1>=35.5 R2< 33.5 R2< 33.5 Day=abcdef R2< 33.5 R1>=36.5 R1>= R2< 32.5 R2< R1>=33.5 R1>=33.5 R2< 35.5 R1>=36.5 R1>= Day=abcdef Day=abdefh 4 R2< R1>= R1>=33.5 R2< R2< 35.5 R2< Figure 36. Classification trees for LFEUEXE when relative delay output (on the left, only regulation rates are independent variables and on the right, categorical variable day have been used) To conclude the analysis of classification trees for absolute and relative delay, it has to be said that from previous application of the tree-based method, it was not possible to build a classification tree for LFEESE which is not dependant on the day, using available data. Classification trees are especially accurate with binary output in other words, with a small number of classes. Therefore it could be interesting to use another indicator to detect interactions, with fewer classes, a new way to present output. In this study, a new indicator will be proposed, a change of delay, either an increase or decrease comparing to the baseline delay. This change in delay will be classified in three classes; the new idea is developed in the following. EEC Note No.: 22/

62 6.3. ANALYSIS OF CLASSIFICATION TREE, OUTPUT -1, 0 OR 1 Previous results make it obvious that it is important to find a good indicator of interaction detection and to increase prediction accuracy if operational use is assumed. A new indicator is proposed, the difference in delay. The difference in delay between delay generated by the regulation plan and the delay for the baseline scenario, delta, might be useful in evaluating network effect. This delta also can be considered as an indicator of interaction intensity. It could be said that the more important the difference in delay is, the stronger the interaction. The following coding has been used: -1 if delta is negative: The delay for a given regulation plan is less than the baseline delay. 0 if delta 0: The delay is equal to the delay generated in baseline scenario simulation; 1 if delta is positive: The delay is greater than the baseline delay. As in two previous analyses, a prediction model will be made for each of three. Baseline regulation rates are: LFEUEXE 35 flight per hour, LFEUN 33 and LFEESE 30 flights per hour. The first classification tree will be constructed for input variables: regulation rates for three and the second classification tree will take day as an input variable besides regulation rates. What about the new classification tree? LFEESE: The new classification tree constructed for LFEESE is presented on Figure 37. One can see that the tree is relatively small. This tree shows that if regulation rate for LFEESE is less then 29.5 the delay will increase, comparing to the baseline delay. On the left branch, one can see that there is an influence of regulation LFEUEXE on delay in LFEESE, the variable R1 (LFEUEXE) was used in the tree construction. If regulation rate in LFEESE is between 30.5 and 29.5, and if regulation rate in LFEUEXE is less than 34.5, then there will be a reduction in the delay for LFEESE. This means that regulation LFEUEXE might protect LFEESE. It is important to say that this tree was constructed using only three variables i.e. the three regulation rates. This is the first general tree for LFEESE in this study. Predictions that make this classification tree were generalised for any day. The classification tree on Figure 37 presents a model to estimate reduction or increase in delay compared to the baseline. The -1, 0 and 1 were created according to a comparison to the baseline delay. The question to discuss is; what else could be used as a baseline? For the moment, the baseline delay is related to the baseline plan, with regulation applied in the LFEESE (with a regulation rate of 30 aircraft per hour), in LFEUN (33 aircraft per hour) and in LFEUEXE (35 aircraft per hour). EEC Note No.: 22/

63 > lfeese.rpart<-rpart(bin~r1+r2+r3, data=lfeese, method="class") Figure 37. Classification tree for LFEESE when coded output -1, 0 and 1 Once more, the main objective was to detect interactions. LFEUEXE interacts (protects) with LFEESE and delay in LFEESE will change. According to the rpart report, the estimation of prediction error can be calculated: 14%. Also, the information of which variables are used in the tree construction is given in the report. It gives interactions between R1 (LFEUEXE) and R3 (LFEESE). > printcp (lfeese.rpart) Classification tree: rpart(formula = Bin ~ R1 + R2 + R3, data = lfeese, method = "class") Variables actually used in tree construction: [1] R1 R3 Root node error: 1115/2738 = n= 2738 CP nsplit rel error xerror xstd If a new variable is introduced, the day, it was interesting to take a look to a new classification tree. The best pruned tree is presented on Figure 38, window on the right. The left branch of the tree is more developed: the sub-tree shows that network effect exists, in particular if regulation rate for LFEESE (R3) is greater or equal to The LFEUEXE has influence on LFEESE. EEC Note No.: 22/

64 What does it mean -1, 1 in the tree for LFEESE? R3>=29.5 R3>=30.5 R1< Day=de Day=abcde R3>=31.5 R1< 36.5 R1>= Figure 38. Classification tree for LFEESE when output coded -1, 0, 1 for variables regulation rate and day (left window full tree, right window pruned tree for cp =0.013) To highlight detected influence, the red line on Figure 38 can be followed. It will show that regulation rates play a role in increasing or decreasing delay compared to the baseline delay. The red line focuses on the following three days: 16 th, 23 rd and 30 th July Descending along the tree one arrives to two terminal nodes; -1 and 1. The first terminal node -1, indicates that the delay will decrease compared to the baseline delay, if the LFEESE regulation rate is greater or equal to 31.5 and if LFEUEXE regulation rate is less than The second terminal node, 1, shows a greater delay than with the baseline scenario, if the LFEESE regulation rate is greater or equal to 31.5 and if the LFEUEXE regulation rate is greater or equal to This example makes evident the existence of an interaction between LFEESE and LFEUEXE: the reduction of delay generated by LFEESE is influenced by the choice of LFEUEXE regulation rate. For this classification tree, the calculated error of prediction in a new sample, using a standard report, is 8.8%. By analysing the classification tree, it is obvious that there is not real influence of LFEUN on LFEESE. On the contrary, one could see that there is interaction between LFEESE and LFEUEXE, where LFEUEXE has a protecting effect upon LFEESE. It has to be said that LFEESE has to have at least 30 flights per hour in order to generate a decrease in the delay. Previous analysis was based on the delay difference, delta, with delta=0 when the delay equals the delay of the baseline scenario. What will be the impact on results if following coding is? -1 for a delta less than -200 min, 0 for a delta between -200 min and +500 min, 1 if delta is greater than 500 min A histogram of variable delta for LFEESE is presented on Figure 39. With this new classification, three classes of delta with approximately the same number of observations each can be obtained. EEC Note No.: 22/

65 Histogram of difference in delay Frequency Delta delay in minutes Figure 39. Histogram for difference in delay in LFEESE A classification tree for the new coding is presented on Figure 40. From the new classification tree, one can detect a protecting effect of LFEUEXE (R1) if its regulation rate is less than 35.5 aircraft per hour. This effect is presented by the red line on Figure 40. > lfeese$bin2<-rep(1,times=length(lfeese$bin)) > lfeese$bin2[lfeese$delta < -200]<- -1 > lfeese$bin2[lfeese$delta >= -200 & lfeese$delta < 500] <- 0 > lfeese.rpart<-rpart(bin2~r1+r2+r3, data=lfeese, method ="class") Figure 40. Classification tree: detection of interaction for a delta between -200 min and 500 min (code 0) In such a classification tree construction the stability is worth being checked. One test consists of changing the coded intervals. Results show high variance of the tree. The difference in classification trees on Figure 40 and Figure 37 is one demonstration. EEC Note No.: 22/

66 Introducing the day as a new variable, a new classification tree was constructed, Figure 41. Figure 41. Classification tree when variable day is included in tree construction for coding, with a code 0 when delta is between -200 min and 500 min. What about two other? For the two other, the classification trees were constructed for difference in delay coded by, -1 when delay is less than the delay of baseline scenario, 0 when it equals to, and 1 if it is greater than the baseline delay. EEC Note No.: 22/

67 LFEUN: For the elementary sector LFEUN, a classification tree when regulation rates are input variables, is presented in the left window on Figure 42. This tree was constructed using R1 (LFEUEXE) and R2 (LFEUN) meaning interaction occurred between LFEUN and LFEUEXE. As in the previous case this interaction can be a candidate to use in order to compare with a result from the NEVAC tool. Introducing the day as a new variable, a new classification tree was constructed, Figure 42, the window on the right. It is obvious that the classification tree when the variable day is included is bigger. There are also different variables that were used in the tree construction. In this particular tree, it can be seen that in estimation of output, the classification tree will use four variables, depending on the case. That also means that all might interact depending on a specific day. R2>=32.5 R2>= R2>=33.5 R1< R2>=33.5 Day=ad 0-1 R3< 29.5 Day=fgh R1< Figure 42. Classification tree for LFEUN Window on the left: Tree when regulation rates are independent variables. Window on the right: Variable day is introduced The blue line in the classification tree on Figure 43 indicates two cases: First, if regulation rate for LFEUN is less than 32.5 then the tree estimates an increase in delay no matter what the regulation rate for the other. If regulation rate in LFEUN is greater or equal to 33.5 flights per hour then the tree estimates a reduction in delay for LFEUN. EEC Note No.: 22/

68 Figure 43. Analyse of two cases in the classification tree for LFEUN The case is more complex when regulation rate for LFEUN is between 32.5 and This situation is presented by the red line on Figure 43. The tree estimates increase in delay for days other than 11 th June or 2 nd July where regulation rate for LFEESE is greater than 29.5 flights per hour. For 16 th, 23 rd or 30 th July, the delay in LFEUN is influenced by regulation in LFEUEXE, meaning that if LFEUEXE regulation rate is less than 34.5, the delay in LFEUN will decrease compared to the baseline. If LFEUEXE is greater than 34.5, this indicates increase in LFEUN delay. It seems that there is an indirect influence of this regulation. LFEUEXE: A constructed classification tree for regulation in LFEUEXE is presented on Figure 44. Variables R1 (LFEUEXE) and R2 (LFEUN) have been used in the tree construction. R1>= R2< R1>= Figure 44. Classification tree for LFEUEXE when output -1, 0 and 1 For the classification tree when variable day is introduced, only R1 and R2 were used in the tree construction. Thus, the tree is the same as the one presented on Figure 44. By analysing this tree, it appears that whatever the day, the regulation in LFEUEXE generates less delay if its regulation rate is greater or equal to This seems logical. This classification tree shows also that LFEUEXE can be protected by LFEUN. The tree estimates less delay in a collapsed sector LFEUEXE when the LFEUEXE regulation rate is between 35.5 and 34.5 flights per hour and if regulation rate in LFEUN is less than EEC Note No.: 22/

69 To conclude classification tree analysis when output is (-1, 0, 1), it can be said that results obtained for the collapsed sector LFEUEXE and the sector LFEUN are surprising because some detected interactions were not expected. It seems that variable delta chosen as the indicator of network effect has to be refined. Nevertheless, this study shows that the classification tree that uses delta as an indicator of network effect is the one that gives prediction with relatively high accuracy. The success of this indicator makes it possible to construct classification trees for all three without taking day as a variable. What is the first step in making a general conclusion? The detected interaction shows that the choice of presentation is quite decisive in construction of classification tree. Estimations and detected interactions in the highlighted trees will be used to compare with simulation results of the NEVAC tool. In this way it would be interesting to see where results obtained by the tree-based method are placed. Additional cases for validation by using another tool to confirm interactions have to be performed. Computation time to work out such a tree is negligible. Of course, a more realistic scenario would surely increase computation time quite significantly. In the following part, the NEVAC tool will be used to compare results in some cases. EEC Note No.: 22/

70 7. VALIDATION USING DELAY CALCULATION IN NEVAC 7.1. INTRODUCTION TO NEVAC The NEVAC tool is a part of the NEVAC project that assists in the development of tools and methodologies for the Enhancement Capacity Function of EATMP. The simulation tool NEVAC is based on PACT (Portable ACC Capacity Tool) hypotheses, where sector level analysis is used to estimate the ACC capacity. NEVAC permits simulation of flights delay, to extract analysing periods and to evaluate effect of protection between two ACCs, taking into account network effect. It stays portable and has intuitive user interface. NEVAC gives information about overloads, number of aircraft and delay as indicators to detect change in network, called the network effect. Changes are recorded every 10 minutes and integrated over three time periods. A delay calculation is needed to obtain a good vision of the capacity constraints that appear in the network of ACCs and sectors. It is also useful for detecting interactions between two ACCs. Following each flight and the propagation of delay, it is possible to evaluate these interactions. A test which will be presented in the next part, focuses on the already introduced cases, in order to compare results of classification trees and NEVAC results. It is also important to highlight that classification trees generated in the context of this research study are for the total delay, the relative delay or the difference in the delay. Those system performance indicators have been used to detect interactions. It is evident that more detailed analysis has to be done in order to have real (ECAC wide) validation of results. Daily delay calculation that the NEVAC tool offers have been used for the purpose of this work VALIDATION OF CLASSIFICATION TREE RESULTS BY USING NEVAC TOOL Basic interest in this validation is to clarify interactions that the classification tree detected. Some of them are expected. The following eight scenarios will be used to test model and prediction interactions detection and to compare them with results that NEVAC tool offers with its daily delay calculation. The first two scenarios will be used to test detected interactions and estimations that the classification tree finds if regulation rate for LFEESE change. Scenario 1: the most important delay in LFEESE (regulation rates: LFEUEXE 36, LFEUN 33 and LFEESE 27 flights per hour, 25 th June); Scenario 2: what interactions can be detected if the regulation rate for LFEESE change from 27 to 29 flights per hour (regulation rates: LFEUEXE 36, LFEUN 33 and LFEESE 29 flights per hour, 25 th June). Next two scenarios will be used to test detected interactions and estimations that the classification tree finds if the regulation rate for LFEUEXE changes. Scenario 3: interactions and delay if the regulation rate in LFEUEXE is 36 (regulation rates: LFEUEXE 36, LFEUN 33 and LFEESE 28 flights per hour, 18 th June); EEC Note No.: 22/

71 Scenario 4: what interactions can be detected if the regulation rate for LFEUEXE changes from 36 (scenario 3) to 34 flights per hour? (regulation rates: LFEUEXE 34, LFEUN 33 and LFEESE 28 flights per hour, 18 th June). Scenario 5 and 6 will be used to test detected interactions and estimations that classification tree finds if regulation rate for LFEUN change. Scenario 5: interactions and delay if regulation rate in LFEUN is 30 (regulation rates: LFEUEXE 34, LFEUN 30 and LFEESE 28 flights per hour, 18 th June); Scenario 6: what interactions can be detected if the regulation rate for LFEUN changes from 36 (scenario 5) to 34 flights per hour (regulation rates: LFEUEXE 34, LFEUN 34 and LFEESE 28 flights per hour, 18 th June). The next two scenarios will be used to test detected interactions and estimations obtained by the classification tree based on output (-1, 0, 1). Scenario 7: demonstration of detected interactions using indicator delta (regulation rates: LFEUEXE 37, LFEUN 33 and LFEESE 28 flights per hour, 18 th June); Scenario 8: what is the intensity of interaction between LFEUEXE and LFEESE (regulation rates: LFEUEXE 37, LFEUN 33 and LFEESE 30 flights per hour, 18 th June)? Validation of the detected interactions and estimations classification tree gives if regulation rate for LFEESE change The first scenario has for it s objective to show if the classification tree gives a good estimation of absolute and relative delay knowing that the delay intervals are very large. This case corresponds to the situation when a regulation plan is not well adapted to traffic demand. For this scenario, the classification tree for LFEESE on Figure 31 has one interesting case where the delay is the most important, index class 7. This delay is for 25 th June 2004, where regulation rates are respectively greater than 35.5 for LFEUEXE and 27 flights per hour for LFEESE. In order to calculate delay by NEVAC, two cases for comparison were defined. The delay calculations are launch for 25 th June 2004 (label c in the classification tree). Estimation of the prediction model, classification tree, is presented in the Table 5 below, in the column Classification tree. Classification trees on Figure 31, 32 and 33 have been used for the absolute delay predictions for rates defined in scenario 1. Classification trees on Figure 34, 35 and 36 have been used for the relative delay predictions. Table 5. Results for Scenario 1 Scenario 1 for 25 th June Classification tree NEVAC Absolute delay Relative delay Absolute delay Relative delay LFEUEXE (10400,17000] class 2 same class when day introduced (0.23,0.315] class 3 when day (0.315,0.401] class LFEUN (23200,28000] class 5 when day (18400,23200] class 4 (0.637,0.718] class 8 when day (0.555,0.637] class LFEESE No tree (11500,13400] class 7 No tree (0.141,0.212] class EEC Note No.: 22/

72 From Table 5 one can see that the absolute delay predictions using a prediction model for LFEUEXE are relatively the same as calculated by NEVAC. The tree estimation was presented by interval value. The classification tree predicts the absolute delay between and minutes (the delay belongs to class 2, same with day and means that the absolute delay estimation is in the same class as in the classification tree where a categorical variable day has been used in the tree construction) in LFEUEXE and the NEVAC tool calculates minutes. On the contrary one can see that estimated delays for the two other are different than the NEVAC ones. This could be explained by the imprecision caused by large intervals for delay. Concerning the relative delay estimation, the relative delay prediction in LFEUEXE is similar with the one the NEVAC tool calculates. Classification trees estimate less relative delay for LFEESE than NEVAC gives. NEVAC calculates for relative delay in LFEESE. For a regulation applied in LFEUN, the classification tree estimates more delay than NEVAC does. The second scenario has as it s objective to show influence that change in regulation rate in LFEESE has to other two. Delay calculations in NEVAC will be done for 25 th June 2004 and for the following regulation plan: LFEUEXE 36, LFEUN 33 and LFEESE 29 flights per hour. Results obtained by tree-based method and NEVAC are presented in Table 6 below. Table 6. Results for Scenario 2 Scenario 2 for 25 th June Classification tree NEVAC LFEUEXE LFEUN LFEESE (10400,17000] class 2 (23200,28000] class 5 No tree Absolute delay Relative delay Absolute delay Relative delay same class when day introduced same class when day introduced (1910,3830] class 2 (0.230,0.315] class 3 (0.637,0.718] class 8 No tree when day (0.315,0.401] class 4 same class when day introduced (0.141,0.212] class As in Table 5, the estimated value for relative delay is closer to ones the NEVAC tool calculates compared to the previous comparison that addressed the absolute delay. Those two scenarios were introduced in order to compare interactions that a tree-based method and NEVAC detect. In the classification tree for LFEESE, the absolute delay changes when regulation rate changes in LFEESE. For two other, the classification tree could not detect interactions; the index of class stays the same. If one compares delay in the two tables, Table 5 and 6 for the NEVAC calculation, it can be seen that the regulation in LFEESE is interacting with the two other, in certain scenarios, stronger and in others, weaker. For this particular case, if the regulation rate increases from 27 to 29, the delay in LFEESE will reduce, but the delay for the collapsed sector LFEUEXE and sector LFEUN will change too. To see if the classification trees detect this influence for LFEUN and LFEUEXE one has to refer to Figure 31 and Figure 33. The classification trees did not detect that influence. EEC Note No.: 22/

73 Validation of detected interactions and estimations that a classification tree gives if regulation rate for LFEUEXE change For scenario three, the goal is to compare classification tree results and NEVAC results: to show the influence that regulation in LFEUEXE has on other two. NEVAC calculated interactions and delay for 18 th June 2004 and following regulation plan: LFEUEXE 36 flights per hour, 33 for LFEUN and 28 for LFEESE. Results for this scenario are presented in Table 7. Table 7. Results for scenario 3 Scenario 3 for 18 th June Classification tree NEVAC LFEUEXE LFEUN LFEESE (10400,17000 ] class 2 (23200,28000 ] class 5 No tree Absolute delay Relative delay Absolute delay Relative delay same class when day introduced same class when day introduced (11500,13400] class 7 (0.230,0.315] class 3 (0.637,0.718] class 8 No tree when day (0.315,0.401] class 4 same class when day introduced (0.070,0.141] class In order to detect influence of a change in regulation rate for LFEUEXE, the scenario 4 was defined in which regulation rate for LFEUEXE was 34, 33 for LFEUN and 28 for LFEESE. Results for this scenario are in Table 8. Table 8. Results for scenario 4 Scenario 4 for 18 th June Classification tree NEVAC Absolute delay Relative delay Absolute delay Relative delay LFEUEXE (23700,30400] class 4 when day (30400,37100] class 5 (0.315,0.401] class 4 same class when day introduced LFEUN 23200,28000 (class 5) same class when day introduced (0.555,0.637] class 7 when day (0.392,0.473] class LFEESE No tree (3830,5750] class 3 No tree (0.070,0.141] class In Table 7and 8, the results of classification trees are quite close to ones NEVAC gives. The classification tree for LFEESE estimates that the absolute delay will reduce in LFEESE from class 7 to class 3 if the regulation rate in LFEUEXE decreased from 36 to 34. This interaction between LFEESE and LFEUEXE has been detected. The NEVAC delay calculation gives the result that the delay decrease is from 5785 to 5610 minutes in LFEESE. Here, one can see a big difference in results for delay in LFEESE. EEC Note No.: 22/

74 Concerning regulation itself, applied on collapsed sector LFEUEXE, the absolute delay will amplify together with an increase of restriction as expected from class 2 to class 4/5. NEVAC finds the same influence. Besides NEVAC and the classification tree did not find interactions for absolute delay of regulation in LFEUN. By analysing results based on relative delay in Table 7, it can be seen that results of the classification tree and NEVAC, for scenario 3, are close. In Table 8, the results for scenario 4 based on relative delays were presented; estimations and detected interactions of the classification tree compared to the NEVAC delay calculation model. Predictions were close. It can be seen classification tree and NEVAC find interaction between LFEUEXE and LFEUN. So, one can conclude that estimation of relative delay in LFEUN is influenced by regulation rate in LFEUEXE Validation of detected interactions and estimations that the classification tree gives if regulation rate for LFEUN change Scenario five together with scenario six will be used to validate impact of regulation in LFEUN on the other. Estimation of classification trees and calculations by NEVAC have been done for the 18 th June 2004 and regulation plan based on LFEUEXE, LFEUN, and LFEESE with flow rates respectively of 34, 30, and 28 aircraft per hour in scenario 5. Results are presented in Table 9. In the scenario 6, the same regulation rates for LFEUEXE and LFEESE as in scenario 5, will be used and the regulation rate for LFEUN will be set to 34 flights per hour. Results for scenario 6 are presented in Table 10. Table 9. Results for scenario 5 Scenario 5 for 18 th June Classification tree NEVAC Absolute delay Relative delay Absolute delay Relative delay LFEUEXE (23700,30400] class 4 with day (30400,37100] class 5 (0.315,0.401] class 4 same class when day introduced LFEUN (47000,51800] class 10 with day (42300,47000] class 9 (0.555,0.637] class 7 with day (0.473,0.555] class LFEESE No tree 3830, 5750 (class 3) No tree (0.070,0.141] class Regarding results for the absolute delay calculated by NEVAC for those two scenarios, 5 and 6, the major impact of a change in LFEUN is evident. There is some difference in delay for LFEUEXE and LFEESE but it is really weak in this particular case meaning that interaction is detected but is weak. The estimated interval for absolute delay of classification trees were the ones where results of NEVAC belonged to, and comparing results in two tables it can be seen that classification tree did not find an interaction between LFEUN and LFEESE, and between LFEUEXE and LFEUN. EEC Note No.: 22/

75 Table 10. Results for scenario 6 Scenario 6 for 18 th June Classification tree NEVAC Absolute delay Relative delay Absolute delay Relative delay LFEUEXE (23700,30400] class 4 with day (30400,37100] class 5 (0.571,0.657] class 7 with day (0.657,0.742] class LFEUN (23200,28000] class 5 with day (18400,23200] class 4 (0.31,0.392] class 4 with day (0.229,0.31] class LFEESE No tree (3830,5750] class 3 No tree (0.070,0.141] class A significant variation of total delay between scenarios 5 and 6 results in a big change in relative delay figures. Nevertheless, relative delay intervals in scenario 6 obtained with classification trees are compatible with relative delays obtained by NEVAC. Interactions detected in those two scenarios are the same for the two methods used. Relative delay as an indicator seems not to be adequate in this example. In the following validation step, the focus will be on results based on delay variation indicator (-1, 0, 1) Validation of detected interactions and estimations that the classification tree based on output (-1, 0, 1) gives In order to do the following test it was necessary to calculate the baseline delay for the 18 th June 2004 for the NEVAC tool. In scenario 7 the test was for a following regulation plan: LFEUEXE 37 flights per hour, LFEUN 33, and LFEESE 28 flights per hour. Results are presented in Table 11. Two columns are used to show results for the classification tree: The green one, when the difference in the delay, delta, is coded by 0 (when delay generated is the same as the baseline delay). The blue column, when delta is coded by 0 if the difference of delay is between -200 and 500 minutes. Table 11. Results for scenario 7 Scenario 7 for 18 th June 2004 Classification tree NEVAC 0 when delay same as baseline delay -200<Delta=0<500 Baseline delay Delta delay LFEUEXE -1 LFEUN -1-1 when day introduced -1 when day introduced Not constructed Not constructed LFEESE 1 1 when day introduced 1 1 when day introduced EEC Note No.: 22/

76 In the column NEVAC Baseline delay, the delay for the baseline scenario was presented and Delta delay is the difference in delay for scenario 7 compared to the baseline. Minus - and indicates that for a new scenario, there is less delay than in the baseline. In scenario 7 there is a change in regulation rate in LFEUEXE from 35 (baseline regulation rate) to 37 and in LFEESE regulation rate from 30 (baseline regulation rate) to 28. The delay in LFEUEXE will decrease and increase in LFEESE. On the contrary, NEVAC finds a small influence on LFEUN compared to the baseline. So, it can be concluded that NEVAC detects each significant interaction (in this case superior than 10 minutes). Concerning the classification tree, green column in Table 11, this shows that there is an indication of interaction, but it is hard to say to which extent. Therefore, it could be interesting to see what the shape of the classification tree is when delta is presented by intervals. For the moment only this kind of tree is constructed for LFEESE. The estimation of this classification tree is given in the Table 11 in the blue column. For the tested case, the estimation of the classification tree is the same in two ways of coding. And finally, following scenario 8 has for its objective to show the intensity of interaction that LFEUEXE has on LFEESE. For this purpose, a regulation rate for LFEESE will be set on 30 flights per hour and for LFEUN 33 as in baseline scenario. Regulation rates in LFEUEXE will be the 37. Classification tree estimations and results of delay calculations by NEVAC are presented in Table 12. The classification tree when delta is defined by interval, did not detect interaction between LFEESE and LFEUEXE, it estimates 0 contrary to the estimation of the classification tree in green column, 1. Defining delta by intervals aims in defining the intensity of interactions. This example shows that this indicator could be more developed in order to give also the intensity of interactions. NEVAC has an important contribution in detecting significant interactions. Table 12. Results for scenario 8 Scenario 8 for 18 th June 2004 Classification tree NEVAC 0 when same as baseline delay -200<Delta=0<500 Baseline delay Delta delay LFEUEXE -1 LFEUN 1-1 in the tree with day 1 in the tree with day Not constructed Not constructed LFEESE 1 1 when day introduced 0 0 when day introduced It can be concluded that the estimation of the classification tree is going in the right direction but that the tree has to be more sophisticated in order to allow detailed analysis of results and validation. After this validation it is too early to give concrete statements. It was interesting to test three different indicators to detect interactions: With the absolute delay, analysis is limited to a one day case. With the relative delay, the estimation of relative delay has to be carefully analysed in order to detect all side influences. Finally, with the difference in delay, it was interesting to analyse and to try to use that information in detecting and in quantifying interactions. It seems that this indicator could be more developed in order to construct new classification trees that would estimate the strength of interactions. EEC Note No.: 22/

77 8. CONCLUSION The objective of this work was to model interaction between. In the first part of this report, we presented CFMU and possible improvements in the ATFCM process, like developing network management - taking into account network effect during regulation plan construction and modification. In order to evaluate changes in a regulation plan, it is interesting to model interdependences between. This problem is relatively new and there is no real review of literature. In this report, the network effect problem is presented as a system behaviour problem. Mathematical formulation for one complex problem where many elements interact is very difficult to give. Even if numerous simplifications were introduced, the problem stays complex. One possible model which highlights interactions between flow management could be built using a tree-based method. In order to apply a proposed method and to develop a predictive model, the input data have been collected using the systematic traffic simulator. Using this data analysis, the initial interactions among have been identified. Application of a tree-based method showed that most of the interactions were identified as expected. Classification trees that use the regulation rates and the day as variables to predict the absolute delay and relative delay were constructed. Interactions which have been detected in many cases depend on a specific day, thus generalisation is not yet possible. Having a more homogeneous input data set where the opening scheme matched traffic demand and reducing the difference in delay (depending on the day), could provide a solution to construct a classification tree where detected interactions are independent of the day. Also, a new indicator has been proposed. The difference in delay comparing to one baseline delay-delta have been used to evaluate the importance of detected interactions. In classification trees with delta, interactions not dependent on a specific day were identified and are interesting to analyse. Classification trees with delta were easier to read than trees with absolute and relative delay. It has to be said that unexpected interaction was detected too. This fact poses a question for future work where a refinement of the indicator delta has to be done. The interest of this particular method is in the detection of interactions. In order to make more precise statements, these interactions should be analysed with classical regression and classification methods like logistic regression or a linear model. Also, the stability tests would have to be done if further operational use were envisaged. To reduce instability of trees, several methods have already been developed. During this work, possible directions for future work were traced. A main objective of this modelling is of course to build a tree for an ECAC wide scenario. A future work should consist in improving a simulation scenario. This means to create trees for different periods of the day where opening schemes and regulation plans with are taken into account in order to present a real situation. EEC Note No.: 22/

78 In this work I could not exploit all the other possibilities that CART algorithm propose. For example, CART offers a possibility to predict when some values in the input data are missing. This could be interesting. In any case the objective is to predict interaction between for a new regulation plan. This also could be one direction for future work in order to develop a classification tree that will take into account various operational situations. One important question remains open: Which other indicators might be relevant in interaction detection? Maybe, the number of aircraft, or the change in sector load could be tested. In addition, the possibility of developing the proposed approach in combination with other classical methods should be examined. Finally, the interactions detected by tree-based method application, once a classification tree would have been generated for the global case i.e. ECAC wide, might result in use during all ATFCM phases, from the strategic and pre-tactical ATFCM phases in order to avoid situations that generate important delay (demand/capacity imbalance). When changes in regulation rate have to be decided, the useful information about interactions that a classification tree could give can be taken into account during the tactical ATFCM phase. The information could be given with short notice. EEC Note No.: 22/

79 ANNEXES EEC Note No.: 22/

80 Page intentionally left blank EEC Note No.: 22/

81 ANNEX A AIR TRAFFIC FLOW AND CAPACITY MANAGEMENT Introduction The idea of organising the Central Flow Management Unit was conceived in 1988 in response to growing air traffic in Europe as well as increasingly frequent delays and congestions. The purpose of the CFMU, which became fully operational in 1996, was to cooperate with all regional flow management centres and to make the best possible use of the airspace made available by ATC centres throughout Europe for civil aviation, by balancing demand and capacity and centralising air traffic flow management. This would be a more efficient system than the previous national flow management units, which had been operated by their own national administrations. Also in terms of a quick reaction to reduce traffic to an airspace or airport due to weather or other occurrences, the CFMU would raise the efficiency of air traffic flow management. The CFMU would be able to take data from the flow management positions at every air traffic control centre in Europe in order to draw up strategic demand forecasts based on the number of flights being planned by aircraft operators. The flow of traffic could then be managed accordingly so as to create as few delays as possible Ref [a]. The ATM Strategy identifies that flow management, together with capacity management is a core operational process of Air Traffic Management and that it will be evolutionary step forward in managing the dynamic balance between capacity and demand. The role of the ATFCM evolution plan is to ensure seamless Flow and Capacity Management operations through Strategic, Pre-tactical and Tactical ATFCM phases that are continuously iterative and interactive. The new ATFM process should not be restricted to slot allocation mechanisms but should also be extended to the optimisation of traffic flow patterns and capacity management. Some figures will be given below which will show the importance of CFMU work in reducing delays, hence the positive effects of flow and capacity management measures. Fig. annex 1, shows the air traffic growth since 1990 and forecasts up to the year The source of data is Performance Report Review 8, Ref [b]. The number of flights almost doubled from 1990 to the year 2000 reaching the number of about 8.2 million flights in Europe. In 2001, the traffic demand decreased, but in 2004, the traffic demand returned to the level of the year 2000 and the growth continues. Average annual medium term forecasts for traffic growth in Europe is given in Fig. annex 2a. It shows that more than 5 percent of average annual traffic growth can be expected in Eastern Europe. However, the traffic forecast is very sensitive to re-routings; Fig. annex 2b illustrates the additional growth of total flight movements gained by States if Kosovo airspace was open. It is assumed that every flight takes the shortest routes in a likely future route network. EEC Note No.: 22/

82 Fig. annex 3 shows how delay time per flight has changed since Delay time continued to increase till 1999 and then, there was an important reduction in minutes of delays per flight as well as a decrease in en route delay. As the CFMU started operations in 1996 their effort to improve management of flow and capacity resulted in an important decrease in delays in 2000, as shown in this figure. Nowadays, the en route ATFM delay is below the agreed target delay (1.7 min/flight) and is not far from reaching the medium term optimum delay target (1 min/flight). Fig. annex 4 gives repartition of delayed flights by reasons for their delays. The percentage of ATFM delays in total delay has decreased since 2000, and nowadays reactionary delay makes up the majority in repartition of total delay. Reactionary delay (or rotation delay) is the delay resulting from the primary delay impacting on a first leg that is propagated on the next legs (knock-on effect or snowball effect). 12 million flights per year % 6% 6% 7% 4% 5% 6% 5% 5% 2% 3% 2% 3% 5% 4%4% 4%3%3% 0% 1990 Traffic Year on year % Forecast ( before 1997, estimation based on Euro 88 traffic variation) source : EUROCONTROL Fig. annex 1. Forecast of traffic demand in Europe a) b) Fig. annex 2. a) Summary of average annual growth from , by State or region; b) Potential net change in the growth of the total flight movements from shortest-routing EEC Note No.: 22/

83 minutes per flight Target: 1.7min/flight 4.1 En route ATFM delay Actual: 1.2min/flight En route target En route optimum Airport ATFM delay Summer source : EUROCONTROL Fig. annex 3. En route ATFM delay in Europe % of flights with arrival delay > 15 min 30% 25% 20% 15% 10% 5% 0% AEA ecoda ATFM Security Airline Airport Weather Reactionary data source: AEA (->2001), ecoda Fig. annex 4. Reasons for delay of delayed flights greater than 15min CENTRAL FLOW MANAGEMENT UNIT Today CFMU is a dynamic and indispensable part of the European air traffic management system, providing both an operational and strategic service 24 hours per day. To meet the objectives of balancing demand and capacity, keeping the delays on minimum and avoiding congestion, bottlenecks and overloading, the CFMU undertakes flow management in three phases. Each flight will usually have been subjected to these phases, prior to being handled operationally by ATC. The main differences between the ATFCM phases are the number and role of partners involved in the Collaborative Decision Making CDM process, the time given to do it and the need for advance notice of the decision taken. The output of a phase aims at preparing the next phase (stated in Ref [a]). Evolution Plan for the ECAC States [d] and web site Ref [a] have been cited for further description as well as ATFCM operating procedure for flow management position [e]. EEC Note No.: 22/

84 Strategic phase: Strategic ATFCM activity takes place during the period from several months until a few days before a flight. During this phase, comparison is made between the expected air traffic demand and the potential ATC capacity. Objectives are set for each ATC unit in order for them to provide as much as possible the required capacity. These objectives are monthly reviewed in order to minimise the impact of the missing capacity on the airspace users. In parallel, an assessment of the number and routings of flights, which aircraft operators are planning, enables the CFMU to prepare a routing scheme, which is a structure of mandatory European air routes balancing the air traffic flows in order to ensure maximum use of the airspace and minimise delays. Main strategic inefficiency in current implementation of traffic orientation policies which is extremely complex can be observed. Hence, there is a need to simplify and reconcile the two processes of building and using the route network in an efficient and continuous way with the definition of a consistent route and traffic policy. Pre-tactical phase: Pre-tactical ATFCM starts a few days before the day of operation. It is handled by the CFMU s Network Management Cell NMC. ATFCM Notification Message ANM for the next day is based on traffic forecast, on the information received from the Flow Management Positions FMP, by ATC centre of the ECAC, and the CFMU statistical data. The ANM defines the tactical plan for the next (operational) day and informs Aircraft Operators AOs and ATC units about the ATFCM measures that will be in force in European airspace on the following day. The purpose of these measures is not to restrict but to manage the flow of traffic in a way that minimises delay and maximises the use of the entire airspace. With a declared objective to maintain collaborative optimisation of the network, the pretactical activity is based on a close relationship between CFMU, FMPs and ACCs. The success of this organisation depends to a large extent on the quality of human relations between individuals and confidence in each other, as well as accuracy, reliability and timeliness of information shared. Main pre-tactical inefficiencies are a consequence of enlarged use of mechanically-driven processes, based on electronic exchange of messages and these have inherently reduced the communication and negotiation process between individuals Ref [a]. The Network Management Cell (NMC) The NMC conducts the operational ATFCM planning in pre-tactical phase. NMC coordinates and collaborates with the ACCs and AOs via the telephone, through daily teleconferences and various publications, and creates the Daily ATFCM Plan taking into account factors such as: the traffic patterns of scheduled and leisure operators for that day, the anticipated flows of North Atlantic Traffic (NAT), both Westbound and Eastbound, the expected capacities and sectorisation provided by the ACC, airport capacities, special events. This Plan focuses on the optimisation of capacity across the European Network of ACCs. NMC works actively to minimise the number of ATFCM, consistent with protection of ATC sectors. NMC performs analysis of traffic patterns and re-routing of traffic flows Ref [f]. They use a COSAAC (Common Simulator to Assess ATFM Concepts) tool together with a PREDICT tool (used to support the flow management division in pre-tactical planning) to evaluate solutions to match traffic demand and capacity. New regulation plans and re-routings can be tested using a simulation tool before its operational implementation. EEC Note No.: 22/

85 Tactical phase: Tactical ATFM is the work carried out on the current operational day. Some real-time events will impact on the Network Operations Plan in generating instability. The Tactical phase will consist of continuously and pro-actively monitoring the real-time situation in order to identify these real-time events. This phase will assess their impact on the ATFCM situation and manage them accordingly through a dynamic reaction with the implementation of adequate co-ordinated ATFCM solutions that will lead to retrieve network stability and capacity optimisation. Flights taking place on that day receive the benefit of ATFCM, which includes the allocation of individual aircraft departure times, re-routings to avoid bottlenecks and alternative flight profiles to maximise efficiency. Tactical phase is carried out by Tactical Network Coordinator Ref [1]. The Tactical Network Coordinator (TNC) The TNC position has been established to meet the changes of the ATFCM environment, and in particular to ensure the most effective network management in tactical operations. Its purpose is to make best use of the available capacity and increase network efficiency by use of ATFCM solutions such as Scenarios, Advisory and Mandatory Re-routings, Regulation and Individual coordination with AOs. This is to be effected by close cooperation between all members of the tactical team, Network Management Cell and outside partners to produce a coherent plan for the tactical period. The main responsibility of the TNC is to ensure that problems arising in the tactical area are solved in a coordinated manner compatible with the overall network situation. In addition, TNC responsibilities include to: Obtain and maintain an accurate overview of the ATFCM situation throughout the Flow Management Division area of operations. Adapt and enhance the D-1 plan from NMC to fit the tactical situation and to devise and implement new tactical solutions to solve unforeseen problems. Ensure that all solutions are compatible with the overall network situation by using Collaborative Decision Making techniques and that all relevant partners are informed of the situation and the solutions by the appropriate messaging and teleconferencing. Feedback to all relevant parties appropriate information to enhance future planning, to participate at teleconferences as well as leading the daily tactical conference. Ensure that as the situation develops during the day of operation, plans are assessed and revised, and are compatible with the overall traffic situation Ref [e]. ATFCM EVOLUTION PLAN In 1999, an Independent Study, Ref [d], was initiated to optimise the use of existing capacity and to improve ATFM strategy, processes and operations, in order to reduce delay. The study has identified the urgent need to enhance ASM/ATFM/ATC processes at strategic, pretactical and tactical levels, (see Fig. annex 5). EEC Note No.: 22/

86 Fig. annex 5. Scope for improvements (Source : EUROCONTROL) The final report stated that the current focus was primarily on avoiding the saturation of the control systems rather than on the optimisation of the efficiency of the global ATC system. In order to achieve the ATFCM seamless process, the following directions for change and associated lines of actions are identified: 1. Improving Traffic Flow and Capacity Management which aim to ensure a better alignment of the ATM capacity towards the traffic demand, and a better efficiency of the ATFCM process in the balancing of demand and capacity by: Developing Capacity Management which consists of pro actively optimising the use of ATM/airport capacity to comply with the demand profile and of identifying and utilising other appropriate available capacity. Improving Traffic Flow Management aims at developing the range of flow measures and procedures with ATC to best manage the expected traffic with the latest known resources. Developing Network Management aims by considering the network effect from a central perspective. A chain of collaborative actions and decision-making activities involving all partners concerned, will allow a smooth transition from the identification of expected capacity shortfalls to implementation of the adequate ATFCM solutions. As the decision of one partner may have a consequential effect on others, the CDM process will consider the network effect, allowing each of the participants to take their decision within the global framework. Ensuring Quality of Service through the assessment of performance indicators, aims at guaranteeing an efficient ATFCM Process in order to respond to the AOs needs and this will be assessed through the monitoring of performance indicators. EEC Note No.: 22/

87 The improvement of Traffic Flow and Capacity Management is conducted at National and International Level through a chain of collaborative actions that compares the traffic forecast with the planned ATM environment in order to identify the expected capacity shortfalls (Strategic activities). ATFCM solutions aiming at maximising the network capacity are assessed and the resulting change requests are considered within the ATM environment to define the most compromise between all parties. Pre-defined scenarios and associated modus operandi related to the use of routes, areas and sectors are established and validated before being published. The new environment database, the scenarios and any additional data (strategic forecast) required to conduct pre-tactical activities are provided to all partners. As more accurate traffic demand and user capability data become available, they are compared with the strategic forecast data in order to identify the remaining capacity shortfalls. Through continuously iterative and interactive pre-tactical activities, the ATC and AOs behaviour are organised in order to optimise the capacity and to use other available capacity. Where required, the demand is regulated through the application of restrictions. The result is the promulgation of an optimised and detailed operational plan (Network Operations Plan - NOP). Due to real-time events, instability in NOP may require the application of some refinements. Several performance indicators aiming at ensuring a certain level of quality of the ATFCM Process are monitored in order to respond with efficiency to the AOs needs (e.g. reduction of costs, maximum freedom of movements, etc). ATFCM Solutions for Capacity Shortfalls Resolution Acting as a co-ordinator between all parties concerned, the CFMU will identify the lack of capacity in regard to the expected demand and, taking into account the constraints of the clients (ACCs, AOs, etc), will explore all possible solutions in collaboration with the partners concerned (ATS Providers, AOs, Airports, FMPs, AMCs). Additionally, the CFMU will analyse the benefits of flow change, the overall impact on ATM/AO and will disseminate the information to all partners. This will be supported by the use of improved historical data and simulation tools, to provide a better picture of short-term events. The identification of capacity shortfalls compared to the forecast demand will require possible solutions depicted in Fig. annex 6 to be considered. The aim is to manage the ECAC Airspace Capacity whilst minimising restrictions. It will limit the impact on airspace users while ensuring greater efficiency in both traffic and capacity management. For this study, the addressed cases are ones where demand is regulated. CFMU is using this solution where required. The demand is regulated through the application of restrictions. When implementing restrictions that decrease the demand according to the sector capacity, the CFMU is required to achieve a very fine balance between the contradictory objectives of protection, the safety of operations in ATC sectors and the minimisation of penalties incurred by AOs. Therefore, the restrictions shall be minimised and consistent with ATC protection. EEC Note No.: 22/

88 Fig. annex 6. Possible ATFCM solutions for capacity shortfalls resolution (Source EUROCONTROL) Two different methods for the implementation of restrictions can be considered: implementation on a given airspace requiring ATC protection; implementation considering the network effect. The first method consists of implementing a restriction on a given airspace where traffic demand exceeds the airspace capacity declared by the ATS Provider in order to comply with the ATC protection. The second method considers the network to find dedicated areas where restrictions could be implemented in order to protect a given airspace. The restriction could be applied: to one sector in order to protect other sectors; to a flow of traffic; to the destination as a means of capturing a flow; on a flow generating a high complexity of traffic in order to reduce the controller workload that will result in capacity increase. When implementing a restriction, delays generated are attributed to the ACC originator, without any other consideration. A potential result is that a particular ACC may have excellent results due to upstream ACC protections. Therefore, more and more decisions take place at the last minute for flexibility and business purposes. For example, neighbouring ACCs wait for others decisions to implement restriction measures to see what extent upstream restrictions could protect them. An increased emphasis on individual performances of ACCs tends to diminish common awareness of the global performance of the system. Use of Scenarios An important enabler for an efficient optimisation of the capacity is the use of predefined scenarios established at Strategic and/or Pre-tactical Level. The scenarios will consider as assumptions the ATM environment together with the traffic forecast in order to define an associated modus operandi leading to the achievement of the expected results. The modus operandi will describe the recommended links and working methods in terms of network use, required ATFCM measures and sectorisation. The scenarios will be fine-tuned and improved, if required, through simulations. These simulations will aim at evaluating their efficiency to provide CFMU and ATS Providers with the necessary flexibility to respond to the EEC Note No.: 22/

89 traffic demand, and to provide AOs with multiple options. The adequate scenarios will be stored, made accessible to the external users and used when and where required. Fig. annex 7. General working process for the establishment of scenarios (Source: EUROCONTROL) During my visit in CFMU, I checked utility of scenarios in the ATFCM pre-tactical phase. Those solutions utilise other available capacity in order to avoid overloads and to reduce total delay. In ANNEX b, one scenario for Prag is presented and how it is used when overload is detected. 2. Improving Collaboration with the ATM Partners addresses the relationship between ATFCM and other ATM activities and focuses mainly on the exchange of accurate ATM data (FPL, Airspace, crisis decision, etc.) within a regulatory framework between all stakeholders. Ensuring Flight Plan Data Consistency and Dissemination Optimising Interface with Airspace Management in order to involve all airspace users, in particular the Military Authorities, and to take into account their needs in the decision making process. Collaborating with Airport Operations. The airport must be seen as just one part of the whole ATM system in a gate-to-gate environment. To facilitate effective collaborative decision making, it is imperative that all data required to make an airport function smoothly and to enable full ATM system integration is made available where it is needed at the moment it is required. Managing Critical Events. Implementing a Regulatory Process to ensure equity between all partners and to assess compliance to the rules Ref [d]. EEC Note No.: 22/

90 ANNEX B A part of table from CFMU with defined scenarios to apply in regulation of traffic over ACC Prag: LKAA scenarios, associated flows, reroutings.xis LKAA Grouped RR TFVs AS, Associated flow in ATFMS Combined scenarios Re-route: RR2LKA EDF4>EPW EDBBACC 6 RR3LKA EPW>EDF4 EDBBACC 8 RR30LKA EDBB EDUU airspaces RR5LKA1 EB>EPWA EDBBACC 1 RR6LKA1 EPWA>EB EDBBACC 9 RR14LKA1 LFP>EPWA EDBBACC 10 RR16LKA1 EDDF>U7 EDBBACC 3 + EDUUUAC4 RR31LKA EDUU, EDYY, EDW DER UN858 VIA EDMM, LOVV, LZBB UL6056EGG6UM749 RR4LKA1 EDDF ><LOW LOWW >EDF EDUUUAC 2 10 RR7LKA1 SET1 > EG (LTMA) EDUUUAC 14 RR8LKA1 RR9LKA1 EG (LTMA) > SET1 (LHBP, LOWW,LZIB) EGT > SET1 (EG.. >LHBP, LOWW,LZIB) EDUUUAC 6 EDUUUAC 7 RR32LKA/ RR32LKA1 same capture VIA LOVV, EDMM, EDUU, EDYY AIRSPACES EEC Note No.: 22/

91 Analyse of traffic over EDUUUAC between 14:00-15:00 time period where overload expected and choice of scenario to apply. EEC Note No.: 22/

92 EUROCONTROL This is one example for application of scenarios when overloads detected. The RR16LKA scenario recommends alternative routes in the case to avoid LKAA for departure from EDDF to Russia. EEC Note No.: 22/

93 ANNEX C Statistical Decision Theory In order to penalize errors in prediction methods, for quantitive output, the most common method is the squared error loss that presents squared difference between prediction f(x) and target value (Y) for the input X. This error leads to choosing a criterion for the prediction function, the expected squared prediction error (EPE): EPE ( f ) = E ( Y f ( X ))² Using a conditioning estimation of prediction error on X, using the conditional probability rule P(X,Y)=P(Y X)P(X), we obtain [ Y f ( X )] ² ) EPE( f ) = E E ( X X Y X The sample mean has the property that it s value is central in the sense that it minimises the sum of squared differences between it and the data value. Therefore the solution is the conditional expectation where the best prediction of Y at any point X=x is the conditional mean measured by the average squared error, known as the regression function: f ( x) = E( Y X = x) When output is a categorical variable we need also a loss function to penalise the prediction errors. An estimate Ĝ will assume values in G, the set of possible classes. Most often it is a zero-one loss function. That means that if estimation is equal to the target value we have one, else it is zero. As presented before EPE it has to be minimised and in the case of classification we have estimate Ĝ for maximal probability of class for given input X: Gˆ ( X ) = Gk if P( Gk X = x) = max P( g X = In this way we classify to the most probable class, using conditional distribution P(G X). g G x) EEC Note No.: 22/

94 Regression tree We demonstrate tree based methodology, called CART, using a simple regression example from [a] with continuous response Y and inputs X1 and X2, each taking values in the unit interval (see Figure 1). The space is split into two regions, and in each partition element we can model the response by the mean, Y. We choose the variable and split the point to archive the best fit. Then one or both of those regions are split into two more regions, and this process is continued, until some stopping rule is applied. For example, we first split at X 1 = t 1. Then the region X 1 t 1 is split at X 2 = t 2 and the region X 1 > t 1 is split at X 1 = t 3. Finally, the region X 1 > t 3 is split at X 2 = t 4. The panel of Figure 1a shows a recursive binary splitting for example. The result of splitting is a partition into five regions R 1, R 2 R 5. Using the regression function f(x) we predict Y in each region R m and we model the response as a constant c m. Presented in the binary tree (Figure 1b), the full data set sits at the top of the tree. Observations satisfying the condition at each junction are assigned to the left branch, and the other to the right branch. The terminal nodes or leaves of the tree correspond to the regions R 1, R 2 R 5. Figure 1b shows the tree corresponding to the partition presented on Figure 1a panel and a perspective plot of prediction surface appears on Figure 1c. a) b) Figure 1 c). Partitions and CART EEC Note No.: 22/