Available online at ScienceDirect. Transportation Research Procedia 14 (2016 )

Transcription

1 Avalable onlne at ScenceDrect Transportaton Research Proceda 14 (2016 ) th Transport Research Arena Aprl 18-21, 2016 Safety assessment of mplemented NextGen operatonal mprovements Sherry S. Borener a, *, Vtaly S. Guzhva b,c, Ian Crook d, Rafael Fraga e a Federal Avaton Admnstraton, Washngton, DC, USA b MCR, LLC., McLean, VA, USA c Embry-Rddle Aeronautcal Unversty, Daytona Beach, FL, USA d ISA Software, Pars, France e ISA Software, Bethesda, MD, USA Abstract Snce most NextGen Operatonal Improvements (OIs) have been mplemented recently, there s no tradtonal safety statstcs, such as the number of accdents and/or ncdents avalable to perform ther formal safety evaluatons. In ths paper we present a process of safety assessment usng precursors that may lead to unsafe outcomes. We use the radar track data to estmate traffc separatons before and after NextGen OI mplementaton at Memphs Internatonal Arport (MEM). The FAA mplemented Phase 1of wake re-categorzaton (RECAT) separaton standards at MEM n November We examne the counts of wake, proxmty, and anomalous trajectory events for two 30-day samples of pre- and post-recat perods. Whle the paper focuses on the methodology and s not ntended to provde a detaled safety evaluaton of RECAT, the fndngs suggest that RECAT mplementaton dd not adversely affect the ar traffc safety. 2016The Publshed Authors. by Elsever Publshed B.V. Ths by Elsever s an open B.V.. access artcle under the CC BY-NC-ND lcense ( Peer-revew under responsblty of Road and Brdge Research Insttute (IBDM). Peer-revew under responsblty of Road and Brdge Research Insttute (IBDM) Keywords: NextGen; operatonal mprovements; safety; wake vortex re-categorzaton; traffc separatons * Correspondng author. Tel.: E-mal address: Sherry.Borener@faa.gov Publshed by Elsever B.V. Ths s an open access artcle under the CC BY-NC-ND lcense ( Peer-revew under responsblty of Road and Brdge Research Insttute (IBDM) do: /j.trpro

2 3732 Sherry S. Borener et al. / Transportaton Research Proceda 14 ( 2016 ) Introducton The Federal Avaton Admnstraton (FAA) s workng towards the Next Generaton Ar Transportaton System (NextGen) that wll transform how the Natonal Ar Transportaton System (NAS) operates to meet future traffc demand and enable effcences for users and servce provders. Advances n navgaton, communcatons, and traffc management provde emergng concepts that wll solve exstng and antcpated operatonal problems n the NAS. In managng the development of NextGen, the FAA s assessng ntatves and potental components of the future system. For proposed Operatonal Improvements (OIs) the assessments typcally are conducted through modelng, smulaton, and/or Subject Matter Expert (SME) surveys. For the OIs that have already been mplemented there s also an opportunty to carry out assessments whch are done by estmatng safety metrcs before and after the mplementatons. Whle operatonal beneft assessment s somewhat straghtforward snce the throughput, delays, fuel consumpton, and other operatonal data are readly avalable, measurng the safety mpact s more complcated. Measurng the number of accdents as an outcome varable does not adequately capture the true safety state of the system snce t measures somethng that does not happen frequently over tme. Snce most of NextGen OIs, have been mplemented farly recently, adequate accdent statstcs may not exst. Also, relance on SME opnon and voluntary reportng ntroduces subjectvty and self-selecton bas. Proactve safety management approach s based on the early dentfcaton and mtgaton of hazards before negatve events occur. In other words, proactve approach dentfes precursors that may lead to unsafe outcomes. For example, reductons n ar traffc separatons below requred mnmums can be consdered as precursors to mdar collsons, therefore, the analyss of proxmty events before and after NextGen OI mplementatons can be used n safety assessment. In ths paper we use the radar track data from the Offload Extractor of the Sector Desgn and Analyss Tool (SDAT) to estmate traffc separatons and anomalous trajectory events, such as mssed approaches, S-turns/360 degree turns close to outer marker, and holdng patterns before and after NextGen OI mplementaton at Memphs Internatonal Arport (MEM). The FAA mplemented Phase 1 of wake re-categorzaton (RECAT) separaton standards at MEM n November The new standard establshes sx new wake categores based on an analyss of the sxty-one most common Internatonal Cvl Avaton Organzaton (ICAO) arcraft types operatng n the busest arports of the EU and US. Several studes have documented operatonal benefts of RECAT mplementaton at MEM, however, the safety mpact was not yet addressed wth the exceptons of voluntary reports by FedEx plots. In ths study we examne traffc separatons and counts of anomalous events before and after the OI mplementaton. Our fndngs suggest that RECAT mplementaton at MEM dd not adversely affect safety. However, ths paper s focus s more on the process of NextGen OI safety assessment than on the RECAT analyss results. The paper s organzed as follows: Secton 2 provdes an overvew of the System Safety Management Transformaton Program (SSMT); Secton 3 descrbes the FAA RECAT ntatve; The process of rsk assessment usng safety precursors s presented n Secton 4; Secton 5 provdes the results of the analyss; and Secton 6 concludes the paper. 2. System Safety Management Transformaton Program In the evolvng Ar Transportaton System (ATS), safety management transtons from a forensc approach that provdes an ndcaton that the system s not safe when accdents happen, to a proactve approach that s based on the early dentfcaton, assessment, and mtgaton of any credble hazards before undesrable events occur (Pasqun and Pozz, 2005). As the ATS s becomng more technologcally and operatonally complex, emergng rsks are more dffcult to understand. Also, quanttatve assessment of rsk s especally challengng when undesred events are extremely rare and the causal factors are numerous and dffcult to quantfy (Hadjmchael, 2009). To be able to meet future safety goals, the safety management process has to evolve wth the ATS. Cvl avaton authortes n the Unted States and Europe have launched several ntatves that attempt to address NextGen and SESAR safety needs and focus on the proactve approach to safety management. One of such ntatves wthn the FAA s the System Safety Management Transformaton (SSMT) program that provdes an ntegrated safety management approach and supports a proactve strategy for buldng ncreased safety nto the ATS. SSMT s developng nnovatve safety montorng and rsk assessment methods, whch can apply to both current and future

3 Sherry S. Borener et al. / Transportaton Research Proceda 14 ( 2016 ) ATM operatons ncludng emergng OIs durng the transformaton to NextGen. SSMT covers the entre spectrum of the ATM processes from ground management at the arport, to traffc management whle the arcraft s n flght, to the network/system level processes relatng to the entre set of flght operatons. The SSMT team has developed an Integrated Safety Assessment Model (ISAM) for analyzng the mpact of NextGen OIs and emergng technologes on NAS safety. ISAM combnes exstng ar transport safety models n a unfed and scalable framework enablng the collecton and analyss of precursor event data to mprove the qualty of safety modelng avalable to the avaton communty. To compute and understand rsk, ISAM models accdent and ncdent scenaros through Fault Tree Analyss (FTA) and Event Sequence Dagrams (ESDs) (NASA, 2002). FTA s a top-down deductve falure analyss of an undesrable top-event (for example, an ntatng event for Mdar Collson s that arcraft are postoned on collson course). FTA represents the falure of the chosen top-event usng Boolean logc to combne the varous causes and faults that lead to the event. The lowest-level causes are Base Events (e.g., loss of communcaton) whch may not be resolved further. The FTA Boolean logc descrbes the flow of precursor events (e.g., loss of separaton) and ther correspondng controls or barrers (e.g., ATC warnng) from Base Events to the top ntatng event. ESDs track the propagaton of one or more undesrable states (ntatng/pvotal events, or top-events from the fault trees) to ether successful or unsuccessful End Events (e.g., arcraft contnues flght or mdar collson). ISAM can be used to assess how system rsk s affected by mplemented or proposed OIs. However, the correspondng Fault Tree probabltes should be quantfed n order to measure the changes n the system safety. For proposed OIs, that are not yet mplemented, modelng and smulaton or SME opnon s typcally used to quantfy these probabltes. For already mplemented OIs, the most obvous method s to compare safety metrcs before and after the mplementaton. However, snce most of the NextGen OIs have only been ntroduced recently, there s often nsuffcent accdent or ncdent statstcs to support ths. Alternatvely, nstead of accdents and ncdents, ther precursors can be used to assess the changes n system safety. These precursors that have not yet become safety ncdents or statstcs can be derved from avalable operatonal data, such as radar tracks or ATC-plot communcaton. In ths paper we use traffc separatons and anomalous trajectores estmated from the radar track data for MEM traffc before and after RECAT mplementaton as safety precursors. 3. Wake Re-categorzaton As part of the NextGen Multple Runway Operatons focus area, t has been dentfed that for the effcency of parallel runway operatons, and n partcular for cases where those runways are closely spaced, one of the potentally constranng ssues s related to the potental mpact of arcraft wake vortex on other arcraft whch are operatng nearby. Arcraft operatng n the termnal area are subject to specfc wake separaton standards whch are often larger than the Mnmum Radar Separaton (MRS) standards of 3 or 2.5 nm and are defned by the FAA and ICAO on a par-wse bass 1. The objectve of these addtonal separaton standards s to provde safe separatons to arcraft that are approachng or departng the arport so that they do not rsk encounterng the wake of a precedng arcraft, whch s potentally hazardous. Table 1 presents FAA wake separaton standards, for dfferent arcraft weght classes: Super (A380 and AN225 arcraft); Heavy (maxmum certfcated takeoff weght of 300,000 pounds or greater); Large (41, ,999 pounds maxmum certfcated takeoff weght); B757 (classfed as a large, but wth a specal wake turbulence category); and Small (41,000 or less maxmum certfcated takeoff weght). 1 The MRS s 3 NM when radar capabltes at a gven locaton permt. A reduced separaton of 2.5 NM may be appled when the average runway occupancy tme of landng arcraft s statstcally proven, by means such as data collecton and statstcal analyss, not to exceed 50 seconds, brakng acton s reported as good, and the runway turnoff ponts are vsble from the control tower.

4 3734 Sherry S. Borener et al. / Transportaton Research Proceda 14 ( 2016 ) Table 1. FAA Wake Separaton Standards (nautcal mles at the threshold). Leader/Follower Super Heavy B757 Large Small Super MRS Heavy MRS B757 MRS Large MRS MRS MRS MRS 4 Small MRS MRS MRS MRS MRS As the understandng of wake vortces has mproved over the last decade, many organzatons have concluded that the weght-based wake separaton mnma reman somewhat conservatve and that revsed separaton standards could be defned based more on the wake characterstcs of the arcraft, rather than smply beng aggregated by weght categores. FAA experts n wake turbulence and safety/rsk analyss workng n collaboraton wth partners DOT/Volpe Natonal Transportaton System Center, EUROCONTROL and the avaton ndustry determned that reduced separatons between smlar type arcraft s as safe, or safer than current standards and f appled can also lead to consderable ncreases effcency and capacty though optmzed runway usage. The wake re-categorzaton program, Phase 1 was approved by the FAA n 2012 and Memphs became the frst US arport to utlze the new RECAT Phase 1 separaton standards n Nov RECAT Phase 1 replaces the prevous FAA/ICAO wake- -separaton categores, whch were based on arcraft weght, wth an enhanced set of separatons based on the wake turbulence characterstcs of dfferent groups of arcraft to provde more optmal separatons durng approach and departure operatons. Table 2 presents RECAT Phase 1 separaton standards. Table 2. RECAT Phase 1 Separaton Standards (nautcal mles at the threshold). Leader/Follower A B C D E F A MRS B MRS C MRS MRS MRS D MRS MRS MRS MRS MRS 5 E MRS MRS MRS MRS MRS 4 F MRS MRS MRS MRS MRS MRS The new RECAT Phase 1 categores are labeled A to F, wth Category A ncludng very large arcraft such as the Arbus A380 and Category F ncludng smaller planes such as the Embraer 120. Under RECAT Phase 1 several arcraft category parngs have been allocated reduced separatons, but there are also some arcraft pars wth unchanged or ncreased separaton mnma. Snce the ntroducton of RECAT phase 1 n Memphs, FAA has subsequently ssued a new order allowng 6 addtonal arports to adopt RECAT and NextGen plan to adopt RECAT across all of the man NAS arports by RECAT Phase 2 wll be an extenson of RECAT Phase 1 wth a statc par-wse regme, where each arcraft par has ts approprate wake turbulence separaton mnma. In the longer term, NextGen are also consderng RECAT Phase 3, whch may support dynamc par-wse wake separaton mnma takng nto consderaton actual arcraft weght and atmospherc/meteorologcal condtons. MEM was consdered a sutable canddate for the ntal mplementaton of RECAT as t s the hub for FedEx, whch s the largest carrer at the arport. FedEx were keen to mprove the capacty at the arport and dentfed RECAT as an opportunty to acheve that goal. FedEx has around 500 operatons a day at Memphs and the majorty of ts fleet fall nto category C whch wth RECAT s able to operate at reduced separatons (from 4 nm prevously to 2.5 nm MRS). As a result of RECAT, the FAA reports that many FedEx arcraft can now proceed drectly from the gate to the runway, reducng tax tme on average by around three mnutes per departure. The FAA also estmates that due to the new separaton standards MEM s capacty has ncreased by over 15% resultng n nne addtonal flghts per hour. Also, lower fuel consumpton and fewer emssons are added benefts of ths newly ganed effcency.

5 Sherry S. Borener et al. / Transportaton Research Proceda 14 ( 2016 ) Wth all of the benefts of RECAT, snce mplementaton n 2012 no wake-related accdents were reported. However, to our knowledge, there were no formal safety studes conducted to assess the RECAT safety mplcatons. 4. The process of rsk assessment To study the safety mpact of the RECAT mplementaton at MEM we use the SSMT Safety Investgaton Toolkt for Analyss and Reportng (SITAR) whch processes the radar track data, analyzes flght trajectores and estmates traffc separatons. Fgure 1 presents SITAR processng dagram. Snce RECAT operatons were ntroduced at MEM n November 2012, we use 30 days of radar-track data from pre-recat perod (Aprl 2012) and 30 days of data from the post-recat perod (Aprl 2013). The source of the radar-track data s the Natonal Offload Program (NOP) obtaned from the Offload Extractor of the Sector Desgn and Analyss Tool (SDAT). The selecton of tme perods ensured that the weather patterns are smlar n pre- and post- -RECAT samples. Also, the post-recat analyss perod s 5 months after RECAT mplementaton, whch suggests that FedEx plots and MEM ATC had enough tme to get used to the new wake separaton standards. RAMS Plus Analyss Metrcs Sector Load / Throughput Flght Effcency / Delay Conflcts ATC Controller Workload CHILL Dynamc Densty Proxmty Events PE Severty PE frequency SMS Rsk Factor Complexty Radar data Santy Check Data adapters SITAR Analyss & Reportng Component KML Model data Proxmty Analyss Component Google Earth CAISS Fg. 1. SITAR Processng Dagram. To support ths analyss SITAR was enhanced to nclude Wake Proxmty Event Indcators that recognze f the arcraft are n a leader-follower confguraton and the separatons are below the specfed wake-separaton mnma. FAA/ICAO wake separaton standards were used for Aprl 2012 sample and RECAT wake separaton standards were used for Aprl 2013 sample. In addton, followng metrcs collected before and after RECAT mplementaton were analyzed: Counts of Category A, B, C, and PE Loss of Separaton (LoS) Events Counts of mssed approaches Counts of S-turns and 360 degree turns Counts of holdngs Counts of departures and arrvals Our defntons of Category A, B, C, and PE LoS events are consstent wth the FAA Order C that s presented n Table 3. Category A events are the most dangerous among four categores wth 34% or less retaned vertcal or horzontal separaton. Category PE events are the least dangerous among four categores.

6 3736 Sherry S. Borener et al. / Transportaton Research Proceda 14 ( 2016 ) Table 3. Separaton Conformance Categorzaton. Category A Category B Vertcal Separaton Retaned 34% >34% 75% Horzontal Separaton Retaned 34% >34% 75% Category C >75% 90% >75% 90% Category PE >90% <100% >90% <100% Fg. 2 llustrates how the horzontal and vertcal trajectory data were used to dentfy the anomalous events. Thus, the SITAR Anomaly Assessment Component consumes 4D flght trajectory data and analyzes the profle n order to automatcally dentfy anomales: Arcraft holdng an arcraft s nstructed to hold at a fxed alttude and performs a crcut n order to absorb a requred amount of delay for arrval sequence. S-turn or 360-turn an arcraft performs an unexpected S-turn or 360 degree turn close to outer marker n order to mantan requred separaton. Mssed approach an arcraft s on fnal approach descendng for landng and then clmbs agan to execute another approach. All these events have dstnctve flght profles and can be dentfed by analyzng the combnaton of lateral and vertcal trajectory data. Fg. 2. Anomaly dentfcatons from Trajectory Data. In addton to the radar-track data we use the FAA Avaton System Performance Metrcs (ASPM) data to examne operatonal demand and capacty, actual takeoff and landng tmes, and weather condtons for each 15-mnute perod. Snce traffc load and weather condtons may have contrbuted to metrcs of nterest, we parttoned the pre- and post-recat sample perods nto four subsamples: Lght traffc load VFR; Heavy traffc load VFR; Lght traffc load IFR; and Heavy traffc load IFR.

7 Sherry S. Borener et al. / Transportaton Research Proceda 14 ( 2016 ) Analyss results Fg. 3 presents the average hourly ntraday traffc counts across the 30 sample days for MEM for Aprl 2012 (pre- -RECAT) and Aprl 2013 (post-recat). The traffc counts are very smlar for both perods suggestng that the results should not be nfluenced by dfferent levels of traffc between perods Apr-12 Apr-13 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 0: Aggregate analyss of proxmty and wake events Fg. 3. Memphs Daly Traffc Loads for pre- and post-recat perods. As presented n Fg. 4, the total number of LoS events n the post-recat perod s margnally lower than n the pre-recat perod for all category. Also, daly counts of proxmty events presented n Fg. 5 suggest that RECAT mplementaton dd not lead to reduced traffc separatons. The counts of LoS events n the post-recat perod do not exceed the counts of the pre-recat perod for all categores and tme perods. Fg. 4. Total number of proxmty events by category for pre- and post-recat perods.

8 3738 Sherry S. Borener et al. / Transportaton Research Proceda 14 ( 2016 ) Fg. 5. Daly proxmty events by category for pre- and post-recat perods Proxmty and wake events by traffc load and weather condtons As dscussed n Secton 4, to examne f the traffc load and weather condtons affect number of wake and proxmty events, we parttoned the samples nto four subsamples based on the level of traffc and VFR/IFR meteorologcal condtons. Table 4 summarzes the results: Wth the excepton of Lght traffc load IFR, the number of LoS events s lower n the post-recat sample. In Lght traffc IFR subsample Category C, B, and PE event counts are margnally hgher n the post-recat perod. Smlarly, wake event counts are lower for all subsamples except Heavy traffc load IFR, where t seems that there were 5 more potental wake events n the post-recat perod. Table 4. Proxmty events by traffc load and weather condtons. Events W PE C B A Traffc Load/Weather Lght Traffc Load VFR Heavy Traffc Load VFR Lght Traffc Load IFR Heavy Traffc Load IFR Statstcal analyss Whle the aggregate analyss of wake and proxmty events suggests that RECAT mplementaton dd not produce safety concerns, aggregaton may obscure the results. To avod potental bas we conduct the Negatve Bnomal (NB) regresson analyss wth wake and proxmty event, as well as anomaly counts as dependent varables. NB regresson

9 Sherry S. Borener et al. / Transportaton Research Proceda 14 ( 2016 ) models are wdely used n safety research for accdent count analyses when the data suggest the presence of over- -dsperson (.e., when the varance exceeds the mean) (Hall, 2000; Mauo, 1994). The NB regresson model used n ths study can be presented as follows: Assume Y s a dscrete random varable descrbng the count of separaton events. X s a vector of covarates that potentally affect the occurrence of such an event and β s a vector of regresson parameters. 1 y p Y y p y ( ) ( ) y ( 1) ( 1 1/ 1 ) 1 1 y, 0,1,2,3,... (1) y where E( Y ) e x e k x j1 j j 1,2,3,..., n., (2) Var 2 ( Y ) and the varance s (3) and α 0 (dsperson parameter). Specfcally, the NB regresson employed n ths study contans the followng varables: SE a b1 Log( TraffcCount) b2 RECAT e where: SE s a separaton event (W, Cat A, B, C, or PE), Log(TraffcCount) s a Log of the count of daly of operatons at MEM (arrval and departures), (4) RECAT s a dummy varable that ndcates that an observaton belongs to a perod after RECAT mplementaton ( 1 for after RECAT perod and 0 for before RECAT perod), and b2 s the coeffcent of nterest; b1 s the coeffcent of varable that controls for traffc densty. The NB regresson model parameter estmates are presented n Table 5. Table 5. Negatve Bnomal regresson parameter estmates. W PE Cat C Cat B Cat A Holds Mssed Approaches S-turns/360 turns Constant -4.39** -3.73** -7.59** ** ** Traffc Count 0.92** 0.82** 1.30** ** RECAT ** RR* N/A N/A N/A N/A N/A N/A N/A 2.09 (*) Relatve Rsk (RR) change n the dependent varable n response to unt change n an ndependent varable;, where b s a acorrespondng coeffcent. (**) Indcates statstcal sgnfcance (5% or better). The coeffcent of RECAT provdes the dfference of event count Logs. For example, for S-turns/360 turns (S/360), Log(S/360 wth RECAT) Log(S/360 wthout RECAT) = Hence, Log(S/360 wth RECAT/S/360 wthout RECAT) = 2.09, whch means that n the post RECAT mplementaton envronment S-turns and 360 turns close to outer marker are about twce more lkely than n the pre RECAT mplementaton perod. However, as ndcated by

10 3740 Sherry S. Borener et al. / Transportaton Research Proceda 14 ( 2016 ) the non-sgnfcant coeffcents of RECAT for Wake, Category A, B, C and PE events, the mplementaton of RECAT at MEM dd not affect the lkelhood of wake or proxmty events. Also, the counts of anomales wth the excepton of S-turns and 360 degree turns are not sgnfcantly dfferent n the post RECAT mplementaton envronment. 6. Conclusons and recommendatons The analyss presented n ths paper focuses on the methodology of the safety assessment usng safety precursors n a before and after approach. Ths methodology allows the evaluaton of mplemented operatonal changes n the NAS from an ar traffc safety perspectve. Whle the presented analyss was not ntended to provde a detaled safety evaluaton of RECAT, the fndngs are encouragng: there were no sgnfcant changes n the number of wake and proxmty events and anomales between the pre and post RECAT perods. The only anomaly that seems to be more lkely n the post RECAT envronment s S and 360 degree turns. Future analyses for other US arports where RECAT wll be ntroduced and whch have dfferent traffc mxes and operatng condtons may provde more nsghts on ths result and clarfy f ths result s arport and/or perod-specfc. References Hadjmchael Mchael, 2009, A Fuzzy Expert System for Avaton Rsk Assessment. Expert Systems wth Applcatons 36, pp Hall DB. Zero-nflated posson and bnomal regresson wth random effects: A case study. Bometrcs 2000; 56: Maou S. The relatonshp between truck accdents and geometrcal desgn of road sectons: Posson versus negatve bnomal regressons. Accdent Analyss and Preventon 1994; 26(4): NASA Offce of Safety, 2002, Fault Tree Handbook wth Aerospace Applcatons Verson 1.1. Pasqun Alberto, Smone Pozz S, 2005, Evaluaton of Ar Traffc Management Procedures Safety Assessment n an Expermental Envronment. Relablty Engneerng and System Safety 89, pp