Enhancing the role of Human Factors in Safety Investigation and Analysis Katie Berry Ph.D. Michael Sawyer Ph.D. March 14-18, 2016
Human Performance and Operations Procedures & Policies Human Performance Maintenance Operational Performance Aircraft Fleet Dispatch / Flight Management Software and Avionics 2
Human Performance in Aviation Fatigue The human element is the most flexible, adaptable and valuable part in the aviation system but it is also the most vulnerable to influences which can adversely affect its performance. ICAO circular 216-AN/131, 1989 Workload & Stressors Procedures & Tasks System Design Human Performance Training Health & Wellbeing Capabilities & Limitations... 3
Assessing Human Performance & Safety Responsive Identify & Mitigate Risks After an Event Collect and Analyze The factors impacting human performance and recovery during an adverse safety event Goal Prevent future events by managing the contextual and systemic causes Emergent Identify & Mitigate Risks Before an Event Collect and Analyze Human performance risks across operations to identify key performance indicators Goal Identify and reduce the risk factors impacting daily operations Proactive Identify & Mitigate Risks Prior to Implementation Collect and Analyze Human performance risks associated with new procedures, systems, or capabilities Goal Reduce the time, cost, and risk associated with implementing changes 4
AirTracs Air Traffic Analysis and Classification System Based on Human Factors Analysis and Classification System (HFACS) Agency Influences Facility Influences Resource Management Agency Climate Operational Process Supervisory Planning Supervisory Operations Traffic Management Controller Workspace Operating Context Airspace Interactions Controller Readiness Physical Technological Airport Conditions Airspace Conditions Communication Aircraft Actions Cognitive / Physiological Knowledge / Experience Operator Actions Acts Sensory Decision Execution Violations Willful Violations 5
Factor Classification Classification Causal Observed Factor Definition An immediate/direct factor that identifies an active error or failure of critical components of equipment, systems, or human error. Causative: If A occurs, then B will occur. An underlying/root factor that identifies latent errors or failures related to human performance, operating environment, task procedures, training, supervision, or policy that influence the presence of causal factors. Probabilistic: If A occurs, then the probability of B occurring increases. A factor that is present but the associated impact of the factor on the safety event has not been proven. It is recorded to note its potential influence on the event or actors involved and to be incorporated into trend analysis. Positive A factor that positively contributed to the safety of an event. This can include factors or actions that contributed to the detection of or recovery from an adverse outcome. 6
Human Factors in Investigations
High-Risk Loss of Separation Example Parallel arrivals into major airport Multiple controllers were actively monitoring the situation Trailing aircraft traveling significantly faster XYZ123 ABC123 8
Investigating the Causes CPF05 Expectation Bias E. Duplicate callsigns typically result from aircraft that have landed but are still in the system TE01 Automation Procedures Causal D. CARTS allows drop of data block for active aircraft on approach EX05 Procedural / Technique Error F. Local 2 misinterpreted data displayed on CARTS CC03 - Coordination G. Local 2 failed to coordinate data block drop with Local 1 DE04 Rule-Based Error Causal H. Local 2 forced drop of data block Duplicate call signs occur frequently at Airport Aircraft will remain in system after landing Tower controllers force the system to drop the data block Lack of standard procedure for handling duplicate call signs or data blocks Automation allows controllers at a facility to drop a data block not under their control 9
Investigating the Causes TE04 Software / CPF05 Expectation SP02 - Staffing Automation Bias Observed I. Ghosting issues where false J. Supervisor and controller N. Initially flight data & D-side targets are displayed on associate limited data blocks positions were unstaffed CARTS 1-2 times per day with ghost targets KE03-Unfamiliar DE04 Rule-Based Task / Procedure Error K. Controller was unaware of L. Controller did not use flight the possibility of automation progress strips dropping the datablock Recent update to automation resulted in many false targets Occurred approximately 1 to 2 times a day Frequency of false target resulted in expectation bias Misdiagnosed ABC123 limited data block as ghost target CPF02 High Workload Q. High controller workload due to traffic & conditions Outcome Factors R&T 10
Airspace Conditions Aircraft TRACON Tower Airline Dispatch OI01 Organizational Influences Observed A. Dispatchers unaware of consequences of duplicate callsigns OI03 Operational Influences B. Company did not have adequate policy to cross-check callsigns for duplication OI03 Operational Influences Causal C. Dispatchers assign duplicate callsigns to two aircraft CPF05 Expectation Bias E. Duplicate callsigns typically result from aircraft that have landed but are still in the system EX05 Procedural / Technique Error F. Local 2 misinterpreted data displayed on CARTS DE04 Rule-Based Error Causal H. Local 2 forced drop of data block TE01 Automation Procedures Causal D. CARTS allows drop of data block for active aircraft on approach CC03 - Coordination G. Local 2 failed to coordinate data block drop with Local 1 TE04 Software / Automation I. Ghosting issues where false targets are displayed on CARTS 1-2 times per day KE03-Unfamiliar Task / Procedure K. Controller was unaware of the possibility of automation dropping the datablock CPF05 Expectation Bias J. Supervisor and controller associate limited data blocks with ghost targets DE04 Rule-Based Error L. Controller did not use flight progress strips SP02 - Staffing Observed N. Initially flight data & Radar Associate positions were unstaffed CPF02 High Workload Q. High controller workload due to traffic & conditions EX01 Attention Error Observed R. R-Side fails to respond to ABC check-in CPF03 Complacency/ Vigilance S. Controller vigilance reduced with end of workload in sight SO06 Supervisory Coordination M. Supervisor drawn into tactical debate rather than strategic management EX01 Attention Error T. R-side fails to respond to ABC traffic query TE05 Warnings / Alerts Observed U. CARTS conflict alert did not activate Outcome Significant Loss of Separation TMU04 Traffic Flow P. Unbalanced arrival distribution OI03 Operational Influences V. AirFly V. Flight crew failed to to notify ATC ATC of of observed overtake AA03 Aircraft Automation Observed W. TCAS did not issue Resolution Advisory ASC02 Sector Weather O. Significant differences in winds aloft at high vs. low altitude Latent Factors Day of Event Event 11
Beyond Individual Event Investigations
Identification of Safety Benefits Unusual Airport Design/Layout Controller Prevented a Runway Incursion The hold short line is unusually far from the runway. Unfamiliar pilots commonly taxi over the line despite a warning on ATIS. Observe aircraft non-conformance Issue control instructions to ensure separation Issue control instructions to correct non-conformance Pilot Deviation ATC Instructions N123 taxied past the hold short lines onto RWY Ground controller mentions that N123 was past the hold short line. Local controller calls aircraft on approach to prepare for a possible goaround. Local controller calls N123 instructing him to taxi back across the hold short lines. 13
Human Performance during RNAV/RNP Procedures Examining execution of RNAV/RNP procedures in current operations Identify factors impacting pilot and controller performance Provide recommendations to RNAV/RNP procedure design process Narrative safety report analysis (April 2011 July 2012) Air Traffic Safety Action Program Voluntary, non-punitive Implemented October 2010 Over 41,00 reports from controllers in year 1 100 ATC RNAV reports Aviation Safety Reporting System Voluntary reporting for aviation community Public database Queried for flight deck narratives 68 flight deck RNAV reports 14
Agency Influences Facility Influences Resource Management 0% Agency Climate 3% Operational Process 39% Supervisory Planning 15% Supervisory Operations 6% Traffic Management 0% Operator Context Physical Environment 1% Technological Environment 22% Airport Conditions 4% Airspace Conditions 23% Aircraft Actions 61% Coordination & Comm. 42% Cognitive & Physiological 8% Knowledge / Experience 5% Sensory 2% Operator Acts Decision 19% Execution 13% Willful Violations 0% 15
Agency Influences Facility Influences Resource Management 0% Agency Climate 3% Operational Process 39% Supervisory Planning 15% Supervisory Operations 6% Traffic Management 0% Operator Context Physical Environment 1% Technological Environment 22% Airport Conditions 4% Airspace Conditions 23% Aircraft Actions 61% Coordination & Comm. 42% Cognitive & Physiological 8% Knowledge / Experience 5% Operator Acts Sensory 2% Decision 19% Execution 13% Willful Violations 0% 0%-10% 11%-20% 21%-40% 41-60% 16
Analysis of Causal Factor Occurrence ATC & Flight Deck Automation ATC-Flight Deck Communication RNAV Procedures Track Deviations 34% of ATSAP Reports 46% of ASRS Reports 30% of ATSAP Reports 21% of ASRS Reports 54% of ATSAP Reports 24% of ASRS Reports 71% of ATSAP Reports 79% of ASRS Reports ATC Systems Aircraft FMS Hearback/Readback Descend Via Route Interactions Charting Features NOTAMs Pilot Deviation Lateral Deviations and Turns 17
OR=3.1362 OR=3.083 RNAV/RNP Risk Pathway RNAV Procedures ATC-Flight Deck Communication Route Interactions Charting Features NOTAMs 54% ATSAP 24% ASRS Hearback/Readback Descend Via OR=2.8162 30% ATSAP 21% ASRS Track Deviations ATC & Flight Deck Automation ATC Systems Aircraft FMS Pilot Deviation Lateral Deviations and Turns 71% ATSAP 79% ASRS 34% ATSAP 46% ASRS Pathways presented resulted in significant associations with the Fisher s Exact Test & odds ratios (p<0.05) 18
RNAV Procedure NOTAMs RNAV Procedures NOTAMs ATC & Flight Deck Automation Aircraft FMS Track Deviation Cited as an issue by both controllers (10% - ATSAP) and pilots (41% - ASRS) May be published and modified on a daily basis Response to temporary events (e.g. airport construction, special use airspace) Necessary and vital to safety 19
RNAV Procedure NOTAMs RNAV Procedures NOTAMs ATC & Flight Deck Automation Aircraft FMS Track Deviation Response to non-temporary events Could be indicative of inadequate procedure design (e.g., route interactions, crossing restrictions) Examples: altitude and speed restrictions, ATC assign only, types of aircrafts authorized Not necessarily being incorporated into FMS navigational database causing unintended flight deviations 20
A Unified Approach to Human Performance & Safety Responsive Identify & Mitigate Risks After an Event Prevent future events by managing the contextual and systemic causes Emergent Identify & Mitigate Risks Before an Event Identify and reduce the risk factors impacting daily operations Proactive Identify & Mitigate Risks Prior to Implementation Reduce the time, cost, and risk associated with planned changes Shifts focus away from Human Error Provides a common risk assessment language Promotes scalable analyses and investigations Identifies developing risk trends Develops targeted mitigation strategies Improves implementation time, cost, and safety 21
Enhancing the role of Human Factors in Safety Investigation and Analysis Michael Sawyer Michael.Sawyer@FortHillGroup.com Katie Berry Katie.Berry@FortHillGroup.com www.forthillgroup.com