HUMAN FACTORS. FASTI Human Factors and Managing the Transition Good Practice Guidelines. Making change in En-Route Air Traffic control DELIVERABLE

Transcription

1 HUMAN FACTORS FASTI Human Factors and Managing the Transition Good Practice Guidelines Making change in En-Route Air Traffic control DELIVERABLE

2 EC_FASTI02_A5brochure :04 Page 1 For more information contact : EUROCONTROL FASTI Programme DAP/ATS Division Telephone: Please contact Una MELLETT for further information Una.mellett@eurocontrol.int Published by : EUROCONTROL General Secretariat Directorate 96, rue de la Fusée B Brussels, Belgium First ATC Support Tools Implementation (FASTI) Human Factors and Managing the Transition Good Practice Guidelines European Organisation for the Safety of Air Navigation (EUROCONTROL) Published in June 2007

3 EC_FASTI02_A5brochure :04 Page 3 CONTENTS EXECUTIVE SUMMARY 4 1. FASTI CONCEPT 6 2. CONTEXT 8 3. HF FRAMEWORK HUMAN IN SYSTEM TEAMS AND COMMUNICATIONS PROCEDURES, ROLES AND RESPONSIBILITIES TRAINING ORGANISATION AND STAFFING THE TRANSITION LIFECYCLE AND HUMAN FACTORS 25 3

4 EC_FASTI02_A5brochure :04 Page 5 EXECUTIVE SUMMARY June 2007 The aim of EUROCONTROL s First ATC Support Tools Implementation (FASTI) Programme is to ensure the rapid and coordinated deployment of an initial set of controller support tools. As part of the FASTI Programme a set of good practice guidelines focusing on Human Factors (HF) have been developed to help with the planning and management of the transition to the FASTI tools. Data was collected from ten Air Navigation Service Providers (ANSPs) within Europe to capture the lessons learned from their experiences of implementing new tools to support controllers. The purpose of thee guidelines are to highlight HF issues that need to be addressed during the transition to the FASTI tools. It should be noted, however, that the list of guidelines provided is not exhaustive or prescriptive. The value of the guidelines are as follows: Based on real lessons learned: The guidelines are developed from practical lessons learned from ANSPs in implementing new systems over the last 5 years. This booklet provides a summary of Section 3 to 7 from the document Good Practice Guidelines for First ATC Support Tools Implementation (FASTI) with a focus on Human Factors and Managing the Transition. In 90% of the cases the FASTI tools will be implemented as part of a new ATM system, so the scope of the data collected was wider than the FASTI tools themselves. Practices and trends were reviewed for managing the transition to a new system/tool which focused on the following HF categories: Human in System, Procedures, Roles and Responsibilities, Teams and Communication, Training, Organisation and Staffing. Practical Check list: The document provides information to derive practical material, e.g. checklists, guidance for training, staffing, procedure development and HMI. Adapted to local needs: The topics and guidelines are generic in nature and can be modified to local needs. The contents of a guidelines document can be used, where appropriate, as a checklist for project managers and their team members in managing the transaction to new systems/tools and can be adapted to local contexts. Comprehensive but not exhaustive: The guidelines are comprehensive in nature but not exhaustive in that they can be further added to as experience is gained by ANSPs in implementing the FASTI tools over the coming years. 4 5

5 EC_FASTI02_A5brochure :04 Page 7 1. FASTI PROGRAMME The aim of EUROCONTROL s First ATC Support Tools Implementation (FASTI) Programme is to ensure the rapid and coordinated deployment of an initial set of controller support tools. The FASTI Programme includes the following three tools: Medium Term Conflict Detection (MTCD), Monitoring Aids (MONA), System Supported Coordination (SYSCO). MONA MTCD The phases of the FASTI Programme is as follows: Initial Implementation and Operational Validation Phase ( ). Implementation and Operations Planning Phase (2009) Implementation and Coordination Phase ( ). The FASTI toolkit will involve a change in Air Traffic Control (ATC) which will require new operational concepts and new working methods. The FASTI Programme wants to ensure that the HF elements are addressed fully to enable acceptance of the FASTI tools and that full benefits of the FASTI tools can be realised. SYSCO MTCD MTCD is a collection of software data processing functions designed to assist controllers in the search for conflicts in a particular time frame. The specific time frame will depend on the local implementation; however, the look-ahead horizon may be up to 20 minutes. MTCD achieves conflict detection through continuous monitoring and prediction of aircraft trajectories. MONA MONA functions inform the controller of the progress of flights through the sector. The system can remind the controller regarding actions that need to be carried out (a turn or descent instruction or transfer of an aircraft, for example). Warnings provide alerts related to deviations of a flight from a clearance or a plan (lateral deviations, FL deviations, and FL busts). SYSCO SYSCO offers screen-to-screen dialogue in the coordination and transfer of control of a flight to/from a neighbouring unit or sector. 6 7

6 EC_FASTI02_A5brochure :04 Page 9 2. CONTEXT The objective was to produce a set of good practice guidelines for HF based on lessons learned from Air Traffic Service Providers (ANSPs) in managing the transition to new systems/tools with FASTI-like capabilities for ATCOs over the last five years. Table 1: Transition life-cycle PROJECT PHASE PROJECT STAGE Data was collected from the following ten ANSPs: DFS (Germany), DSNA (Toulouse, France), ENAV (Rome, Italy), Finavia (Finland), IAA (Shannon, Ireland), LFV (Sweden), MUAC (Maastricht, The Netherlands), NATS (Hurn, United Kingdom), NAVIAIR (Copenhagen, Denmark), Skyguide (Geneva, Switzerland). In total 43 interviews were conducted and 47 people were interviewed. In 90% of the cases the FASTI tools will be implemented as part of a new ATM system, so the scope of this data collection needed to be wider than the FASTI tools themselves. DEFINITION/FEASIBILITY Operational concept Preliminary design Requirements capture and specification PROCUREMENT Call for tender Select supplier/contract DEVELOPMENT Detailed design Construction COMMISSIONING FAT/System-Acceptance Main focus of good Testing (SAT) practice guidelines Hardware/software installation Integration on-site testing Shadow-mode testing Operational validation Training OPERATIONS Shadow-mode operations O-Date POST O-DATE Monitoring Dismantle old system The focus was on the latter part of the transition cycle as shown in Table 1 i.e. the period of time after the project phase Installation (system-acceptance testing). However, when HF issues arose related to the earlier phases of transition this information was also captured. 8 9

7 EC_FASTI02_A5brochure :04 Page HF FRAMEWORK HF is a broad discipline, which considers all the issues that influence human and system performance, such as job or role, procedures and task design, team issues and Human-Machine Interface (HMI) design. In addition, also incorporated is the impact on Human Resources (HR) practices such as selection, training, planning, staffing, competency checking and licensing, and so on. Figure 1 broadly illustrates the HF considered within the HF Pie which is used to categorise HF s in the HF Case methodology. The aim of HF for the FASTI Programme is to gain an understanding of HF impacts upon human performance in implementing the FASTI tools. The following are examples of HF issues impacting human performance. Figure 2. HF Impacts Wheel Figure 1. HF Pie Categories Implementing a new system/tool for controllers raises a number of HF questions such as: Will the controllers situational awareness be maintained or improved? Will there be an increase in monitoring demands? Will the controllers trust and use the new system/tools? Will the automation of certain tasks change the nature of the controllers tasks and required skills? Will there be a need for new selection/training requirements? How to effectively design the training required? The FASTI guidelines are provided for issues related to the following categories Human in System Procedures Roles and Responsibilities Teams and Communication Training Organisation and Staffing 10 11

8 EC_FASTI02_A5brochure :04 Page HUMAN IN SYSTEM 4.1 INTRODUCTION Human in System is concerned with issues related to human-machine interaction such as input devices, output devices, information requirements, alert signals, Human-Machine Interface (HMI) usability, allocation of function between human and machine and new technology and automation, and recovery from failure. 4.2 HF GUIDELINES HMI Requirements and Preferences Managing different views and preferences on HMI H1 A clear streamlined process should be in place to capture users requirements/preferences and to give feedback on what is feasible. Involve HMI design specialists, prototype the HMI and test it with the users. Recovery from Failure Awareness and building up trust Understanding why the system is changing Over-trust/distrust Individual differences H9 H10 H11 H12 Controllers should be given exposure to the new system/tool at appropriate stages during transition. The rationale for new system/tool should be explained clearly and explicitly. s related to over-trusting/distrusting the system should be assessed and monitored and action should be taken to mitigate any adverse effects. An assessment should be made of how different users may adapt to the new system/tool and action should be taken to mitigate any adverse effects. Understanding the HMI and clarifying ideas H2 Rapid prototyping should be used to explain the HMI, the different design options and clarify ideas. Risk of human error H13 Effort and time should be allocated to the analysis of possible failures (system failure or human errors) and to the development of contingency training. Learning from others H3 Learn from past research and current systems. HMI Usability - Look and Feel User-friendliness H4 HMI should be carefully designed to be user-friendly and intuitive to use. Minimizing clutter H5 Only essential information should be permanently displayed on screen while all other information should be accessible on request. Minimizing inputs H6 Interaction should be fast and simple. Using colour H7 Colour-coding should be applied carefully. Impact of New Technology and Users Acceptance Tasks Automation of tasks H8 An assessment of the impact of automation on the tasks of the user should be made and understood

9 EC_FASTI02_A5brochure :04 Page TEAMS AND COMMUNICATION 5.1 INTRODUCTION Teams and Communication is concerned with issues related to team structure, team dynamics/relations, task sharing, information requirement and communication methods. mance using the changed communication methods (silent coordinator). 5.2 HF GUIDELINES Communication workload TC9 An assessment should be carried out on the impact of the new system/tool on the efficiency of the users performance due to changes in communication workload. Team Interaction Team structure Team dynamics and relations Task sharing Team workload Position handover/ takeover TC1 TC2 TC3 TC4 TC5 An assessment should be made on the impact of the new system/tool on the team structure in terms of supervision and team formation and its impact on working method/procedures/roles should be clarified. An assessment should be made on the impact of the new system/tool on the team dynamics/relations and the potential adverse effects minimised. An assessment should be made on the impact of the new system/tool on the allocation of tasks between the team members and any risks associated with the diffusion of responsibility between the users minimised. An assessment should be made on the impact of the new system/tool on the team workload and any adverse effects minimised. An assessment should be made on the impact of the new system/tool on the position handover/takeover and any adverse effects minimised.. Team interaction with the system TC6 TRM module on automation should be revisited. Team Communication Responsibility Communication methods TC7 TC8 An assessment should be made on the impact of a change in team communication on legal responsibilities. An assessment should be made on the impact of the new system/tool on the efficiency of the users perfor 14 15

10 EC_FASTI02_A5brochure :04 Page PROCEDURES, ROLES AND RESPONSIBILITIES 6.1 INTRODUCTION Procedures, Roles and Responsibilities category is concerned with issues related to prescribed working methods and individual working practices, procedure format and positioning, procedure structure content and realism, and working methods in relation to task demand and complexity. The following terms have been defined as they are often confused: Procedures: Represent the organisation s accepted working methods. Procedures are particular courses of action intended to achieve a result.they are a set of activities that are performed by each person intervening in the process according to pre-established rules to enable a successful operation (e.g. abnormal and emergency procedures). Working Method: The way in which individuals perform their tasks. Roles: The position(s) or purpose(s) that someone has in an organisation. The typical or characteristic function performed by someone relating to the tasks that have been assigned to them. Responsibilities: Things that are a person s job or duty to deal with. Having responsibilities means to have a duty to make certain that particular things are done. Developing New Procedures Roles and Responsibilities Resources needed Who should be involved Learning from others Access to simulator System functionality Impact on roles and responsibilities P6 P7 P8 P9 P10 P11 There should be adequate planning for resources to develop new procedures. A dedicated Procedures team should be established. Learn from others in the design of new procedures The Procedures team should have access to the simulator The system should be fully functional when assessing procedures. An assessment should be made of the impact of the new system/tool on the role and responsibilities of the users and new roles and responsibilities defined (if necessary). 6.2 HF GUIDELINES Change in role and interaction with the system P12 An assessment should be made of how the allocation of function between the user(s) and between the user and the system can be optimised. Assessing the Need for New Procedures Relevance of existing procedures P1 Existing procedures should be evaluated and modified (where necessary) to exploit to the maximum the new potential arising from the new system/tool. Users performance and new system/tools P13 Assess the impact of the new system/tool on users effectiveness to use the system/tool to its full potential. Assessing New Procedures Impact on LOA Legal implications Realistic scenarios Format and content P2 P3 P4 P5 Impact on LOA should be assessed. The impact on legal implications should be assessed. Procedures should be tested using realistic scenarios. The documentation of procedures should be presented in an appropriate format and the content should be clear and comprehensible. Working Methods Design of working methods Prescribed working methods and working practices P14 P15 New working methods should be considered as essential in using the system/tool to its full potential. The working methods should be realistic and reflect the way tasks are actually carried out. After O-Date users should be observed to assess how the prescribed working method are being applied in practice and necessary action taken to mitigate any negative effects on the users performance

11 EC_FASTI02_A5brochure :04 Page TRAINING 7.1 INTRODUCTION Training is concerned with such issues as training needs, training course development and roll-out, training content, training methods, and transfer of training. Training methods T9 Training methods and media chosen should be appropriate to the training content and training objectives HF GUIDELINES Training Requirements and Needs Deciding who needs to be trained and when T1 An assessment of who to train and when should be planned as early as possible. When to start developing the training course Amount of effort Type of teaching approach T10 T11 T12 The training course should be developed as soon as the system/tool is mature and relates as closely to what will be implemented on O-Date. The amount of effort to develop a training course should not be underestimated. The teaching approach for a new system/tool should be a step-by-step approach - think building blocks. Training Instructors Identifying the training requirements Number of instructors required Continuity of instructors Instructors role/responsibilities and competencies T2 T3 T4 T5 A Training Needs Analysis (TNA) should be carried out. The requirement for training instructors should be identified. Provision should be made to ensure continuity of instructors. Clear criteria should be defined for the profile and expectations of training instructors used in introducing a new system/tool. Training Course Delivery Understanding the system/tool CBT design Testing the training course When to start training Amount of time needed for training T13 T14 T15 T16 T17 The training course should incorporate adequate level of information on how the system works. CBT should be used to support controllers learning and be available to support classroom lesson. The training course developed should be piloted on a small group of users before full training roll-out. The timing of the start (or roll-out) of the training programme should be planned in advance. The time required for training should be carefully assessed and enough time should be allocated for training. Training Course Development Amount of time required for training instructors Ensuring performance standards Training course structure T6 T7 T8 Enough time should be allocated for the training of instructors taking into account the volume of information needed to be assimilated by the instructors. Performance/competence standards should be specified to assess that trainees have achieved the required performance standards. The formal training course should be well-structured, objective based and divided into several parts. Level of completeness of the system Gap between training and O- Date Individual differences T18 T19 T20 The training roll-out should not start with critical system functionality still subject to change and working methods not finalised. The training roll-out should be planned as close as possible to O-Date. The training course should be designed to accommodate that people learn at different paces

12 EC_FASTI02_A5brochure :04 Page ORGANISATION AND STAFFING 8.1 INTRODUCTION Documentation updates and readiness T21 The relevant documentation, such as procedures and working methods, should be updated and ready before roll-out of the training. Organisation and Staffing is concerned with issues related to change management, staffing levels, rating and recency, job profile, and personal factors such as age, motivation and stress. Dealing with resistance T22 Supervisors/trainers should have a dialogue with controllers who may be negative about the new system and give them the possibility to have extra training. 8.2 HF GUIDELINES Robustness and fidelity of the training simulator T23 The simulator should be robust and not have lots of software bugs and be of high fidelity (reflect the real environment). Planning and Organisation Planning for staff availability O1 The transition and contingency plan should be drawn up as early as possible to ensure staff availability for simulations, training and shadow mode. Resources for simulator T24 An assessment should be made to estimate the resources required to develop and run the training simulator. Determining the staff requirement O2 The staff requirement to successfully manage the transition to the new system/tool should be determined as early as possible. Training Effectiveness Feedback on training T25 Supervisors/trainers should have a dialogue with controllers after the training to get feedback about their confidence in the new system/tool. Establish a transition team O3 An integrated transition team should be set up to develop the working methods, procedures, and training course design, and participate in the system/tooltesting / -validation / -optimisation processes. Interference between old and new working methods Measuring the effectiveness of the training T26 T27 The potential for negative transfer of training should be assessed, monitored and measures taken to minimise these impacts. The training effectiveness should be evaluated immediately after the training and a mechanism should be in place that ensures that feedback is formally recorded and fed back to training, and requested changes are considered. Management of the Transition Team Widening the team Transition approach Visibility and accessibility O4 O5 O6 At a certain phase of the transition a bigger integrated team should be set up to bring the system on line. The big bang approach should be avoided, little change often is better. Members of the transition team should be visible and accessible. On-the-Job Training (OJT) Ab intio Training Contingency Training OJTI and situational awareness Ab initio training System failure and recovery T28 T29 T30 The impact of the new system/tool on the OJTI and OJT should be reviewed and actions taken to mitigate any risks/hazards. The impact upon ab initio controllers training should be assessed and revised/updated where necessary. The training for non-optimum situations and system failures should be planned and developed in good time. User involvement Wider systems view Finding a balance O7 O8 O9 Users should be part of the transition team. Team members should develop a holistic and broad view of the system/ tool change. The project team should be managed so that the users fully participate and have ownership whilst at the same time not giving too much autonomy so 20 21

13 EC_FASTI02_A5brochure :04 Page 23 that complex and never-ending requirements are produced. Communication with Unions O20 Regular communication with Unions about the system/tool should take place. Role of users in project team O10 The role and expectation of users within the project team should be defined and explained. People Management ATCO profiles O21 An assessment should be made of the potential impact on the ATCO profile/selection criteria. Managing the Transition Communicating about the project Deciding the O-Date Traffic levels at O-Date Communicating project progress O11 O12 O13 O14 How progress and issues are communicated outside the project team should be defined, clarified and respected by the project team members. Careful thought should be given to determining the O- Date, and once it is announced - not too early - and decided, stick to it. Preparation and planning to restrict traffic for the initial weeks of transition to the new system/tool should be undertaken. Communication efforts should be targeted at a wide audience and often. Personal Factors Staffing levels of users affected by the new system/tool Job satisfaction/ motivation Age O22 O23 O24 An assessment should be made of the impact of the new system/tool on the staffing levels for the various users impacted by the change over the long term. The impact of the new system/tools on job satisfaction and motivation should be monitored and adverse effects minimised. An assessment should be made of how different users may adapt to the new system/tool due to age and adverse effects should be minimised and monitored. Coaching during the early days of transition Freezing the design Documentation management Relationship with supplier Relationship with regulator O15 O16 O17 O18 O19 Extra support (coaches/specialists) should be planned and available to users on and after O-Date to support transition to the new system/tool. A cutoff date for changes to the system should be agreed. Key documentation should be managed effectively, updated and changes should be effectively tracked between documents. Criteria should be established early to support the development of trust and the effectiveness of the relationship with the system/tool supplier. The communication and involvement with the regulator about the new system/tool should be established early and regularly

14 EC_FASTI02_A5brochure :04 Page THE TRANSITION LIFE CYCLE AND HUMAN FACTORS 9.1 INTRODUCTION Figure 3. Transition lifecycle When considering the transition life cycle (see Figure 3 which illustrates phases of transition life cycle) phases and integrating HF, what is often unclear is: What HF questions should be addressed? What the objective of HF at this phase is? What HF tasks should be undertaken? Table 2 provides examples of what HF aspects should be considered during the various phases of the transition lifecycle (adapted from the EUROCONTROL deliverable Human Factors in the Development of Air Traffic Management Systems (1998))

15 EC_FASTI02_A5brochure :04 Page 27 Table 2. Transition Life cycle phases and HF issues. Transition Phase Key Stages HF Questions to be addressed HF Objectives Key HF Integration Tasks Definition/ Feasibility Concept Development: development of initial ideas and philosophies What should the human do? What should the machine do? How should they work together and share tasks? To guide the system development to consider and define the allocation of tasks between humans and machine. Define the allocation of tasks between human and machine Preliminary Design: sufficient design to prove the feasibility of developing the system Which working position do we want, what should they look like and what is expected from them? How many staff are needed and what should their profile be? Is the system workable and is there a fair correspondence between working procedures and working positions? The design should address: the analysis of tasks to be performed, the basic layout of the workplace, HMI and equipment, considerations on how many people are needed to operate the system, and the type of displays and consoles to be used. Definition of user profiles and analysis of tasks Preliminary HMI design Performance measurements in simulators Requirements: requirements capture and specification What HF requirements to integrate into requirements specification? To review and input into requirements capture and specification. Approved requirement specifications and prototypes including the allocation of tasks between human and system Procurement Call for tender Selection of supplier/ contractor Development Detailed Design: Complete design of the system, all of its components, controls and displays down to the precise location and including ergonomic design. Construction - Building the system - hardware/software engineering Do the working procedures and HMI correspond? Does the test plan cover all the requirements? The detailed design should cover: items ranging from working procedures to the workplace, the training of operators. This means that the following need to be defined: the final working procedures, the contents of the transition training, the dimensions and ergonomics of the workplace, user-friendly HMIs and other aspects of the working environment. Design of working positions Development of working procedures Development of HMIs Development of transition training plan Usability testing - Aspects related with human performance should be an integrated component of the test plan. The usability of the system in regard to the working procedures needs to be carefully analysed and the correspondence with the HMIs has to be studied

16 EC_FASTI02_A5brochure :04 Page 29 Transition Phase Key Stages HF Questions to be addressed HF Objectives Key HF Integration Tasks Commissioning FAT and SAT Hardware/software installation On-site testing Operational validation Training Does the implementation comply with the HF requirements and expectations? To ensure that the intentions from the first phases of the system development are carried out and that the system implementation will conform to the requirements. In parallel with the implementation of the system, to train users of the future system Final approval Training of users and monitoring and evaluation of training effectiveness Operations Shadow-mode operations O-Date Post O date Monitoring Dismantling the old system Are the operators satisfied with the system? Are there any inadequacies in the system? Have all operators received adequate training and are they ready to operate the new system? To devise mechanisms to take feedback from the operators into account. While the operators on the one hand have to accustom themselves with the new system and its working procedures, they can on the other hand provide significant feedback on inadequacies after O date. Mechanism to get feedback from operators. Final training of operators for new system if required

17 EC_FASTI02_A5brochure :04 Page 31 ABBREVIATIONS AND ACRONYMS ANSPs ATC ATM FASTI FAT HF HMI LOA MONA MTCD SAT SYSCO TP Air Navigation Service Providers Air Traffic Control Air Traffic Management First ATC Support Tools Implementation Factory Acceptance Testing Human Factors Human Machine Interface Letter of Agreement Monitoring Aids Medium Term Conflict Detection Site Acceptance Testing System Supported Coordination Trajectory Prediction 30 31