Which data provide the best insight? A field trial for validating a remote tower operation concept.

Transcription

1 Which data provide the best insight? A field trial for validating a remote tower operation concept. Maik Friedrich Christoph Möhlenbrink

2 What was the main validation aspect?

3 What was the main validation aspect?

4 Field trial schematic for Remote Tower Validations 1. Providing Air Traffic Service 2. Functional requirements 3. Validation Plan: Experimental Design 4. Controllability within Field Trials: Controlled traffic patterns 5. Remote Tower System (CWPremote) 6. Participants: Air Traffic Control Officers (ATCO) 7. Analysis of the system: objective data and subjective data 8. System Matter Experts

5 Key Performance Areas, Objectives & Success Criteria Human Performance Task 7 Success Criteria Safety during shadow-mode trials 3 Success Criteria Validation objectives Workload with respect to shadow-mode ATCO Tasks, Roles and Responsibilities Accessibility Perception 1 Success Criterion 5 Success Criteria 2 Success Criteria 1 Success Criterion ATC Procedures 1 Success Criterion Usefulness 1 Success Criterion

6 Key Performance Areas, Objectives & Success Criteria Human Performance Task 7 Success Criteria Safety during shadow-mode trials 3 Success Criteria Validation objectives Workload with respect to shadow-mode ATCO Tasks, Roles and Responsibilities Accessibility Perception 1 Success Criterion 5 Success Criteria 2 Success Criteria 1 Success Criterion ATC Procedures 1 Success Criterion Usefulness 1 Success Criterion

7 Objectives and success criteria that are covered by performance and subjective data Validation Objectives Human performance task Safety during shadowmode trials Subjective Success Criteria Perceived visibility at the apron Perceived safety during shadow mode trial Performance Success Criteria Detection of static objects Detection of safety related manoeuvers

8 Technical Setup of the Controller Working Position remote (CWP-remote) Video panorama Weather information system Radar Navigation lights Flight plan data PTZ camera

9 Panoramic View > Which data provide the best insight? > Maik Friedrich > July 2013 G B G B g 1 f 1000m 0.008m g f p= m 0.6m 5.5µ 1Pixel G Object distance B Image size g object size f focal length

10 Panoramic View and PTZ

11 ZOOM-KAMERA > Which data provide the best insight? > Maik Friedrich > July 2013

12 Validation Exercise Parameters Participants: 8 DFS ATCO Duration: 4 days, between until runs (two per day) Repeated measures design Each ATCO worked on both workplaces Presentation of information: both workplaces had the same sources of information Experimental objects: 1 research vehicle with 1.5 hours / day 1 research aircraft with 5 hours / day Required personal per run: 3 Validation experimenter 2 Validation assistances 1 Driver 2 Pilots + 1 Technicians + 1 Radio operator 1 active ATCO 2 Participants

13 Research Aircraft Flight Pattern

14 Example Research Aircraft Flight Pattern

15 Questionnaires (mid-run and debriefing) Mid-run: Conducted throughout the trial Approx. 190 online mid-run with an average distance of 45 seconds Aspects of the questions: Research Aircraft Manoeuvre Static objects Weather information Workload / Situational awareness Usability Additional scales: Certainty and used source of information

16 Example for the Mid-run Questioning Procedure Is the gear of the aircraft down? Validation experimenter Participant looks at the panorama and the PTZ camera Yes Which system did you use? Video panorama and PTZ How certain are you? 6 Participant (CWP-remote)

17 Demographic Data 8 Participants (male) Age: Mean = 35.3 years; std = years Work experience: Mean = years; std = years Airport experience International and national Airports

18 Objectives and success criteria that are covered by performance and subjective data Validation Objectives Human performance task Safety during shadowmode trials Subjective Success Criteria Perceived visibility at the apron Perceived safety during shadow mode trial Performance Success Criteria Detection of static objects Detection of safety related manoeuvers

19 Items for perceived visibility at the apron Item Key Items for Consideration Runway I was able to see all runway markers sufficiently good. Holding I was able to see all holding points sufficiently good. Airfield traffic Apron area The view on the airfield was sufficient to control the traffic safely and efficiently. I was able to see the apron area sufficiently good. Taxiways I was able to see the traffic on all taxiways sufficiently good. Success Criterion reached

20 Performance Data for the detection of static objects

21 Performance Data for the detection of static objects 8 survey points per distance and workplace No significant differences between the response times Effect Tower: 14.6s ( 8.3s ) Remote: 15.1s ( 9.9s ) t(21) = -0,208, n.s. Success Criterion reached

22 Performance Data for the detection of static objects

23 Performance Data for the detection of static objects

24 Objectives and success criteria that are covered by performance and subjective data Validation Objectives Human performance task Safety during shadowmode trials Subjective Success Criteria Perceived visibility at the apron Perceived safety during shadow mode trial Performance Success Criteria Detection of static objects Detection of safety related manoeuvers

25 Perceived safety during shadow mode trial The ATCO's perceived safety by working passive shadow-mode at the CWP-remote is comparable to the CWP-tower. Item Key Traffic Panorama Items for Consideration The CWP-remote would allow me to handle traffic at least as safe as the conventional CWP-tower. The quality of the 200 panoramic view, supported by the PTZ camera is sufficient to control traffic safely and efficiently conforming to current ATC procedures. PTZ CWP-remote Setup The PTZ camera enabled me to control the VFR traffic as safe as on the CWP-tower. The experimental setup of the RTC-WP with all its necessary functionalities is comparable to a ATC-WP regarding air-traffic control regularities. Success Criterion not reached

26 Detection of safety related manoeuvers

27 Detection of safety related manoeuvers The ATCO s performance to identify safety related manoeuvres of the research aircraft is not significant degraded in comparison to the CWP-tower. Manoeuvre Mean correct answers (SD) CWP-tower Mean correct answers (SD) CWP-remote Significance test (F-test) Decline 86.1% (34.9) 82.4% (38.3) F(1, 7) = 1.62, n.s. Landing lights 83.33% (37.0) 44.3% (49.37) F(1, 7) = 40.45, p <.05* Gear Down 94.32% (23.2) 94.52% (22.2) F(1,7) = 0.96, n.s.

28 Detection of safety related manoeuvers 8 survey points per distance and workplace No significant differences between the correct answers Effect Position: F(1,7) = 1.53, n.s. Bedingung: F(2,14) < 1, n.s. Position x Bedingung: F(2,14) < 1, n.s. Metric reaches goal

29 Detection of safety related manoeuvers

30 Detection of safety related manoeuvers

31 Detection of safety related manoeuvers The ATCO s performance to identify safety related manoeuvres of the D-CODE is not significant degraded in comparison to the CWP-tower. Manoeuvre Mean correct answers (SD) CWP-tower Mean correct answers (SD) CWP-remote Significance test (F-test) Decline 86.1% (34.9) 82.4% (38.3) F(1, 7) = 1.62, n.s. Landing lights 83.33% (37.0) 44.3% (49.37) F(1, 7) = 40.45, p <.05* Gear Down 94.32% (23.2) 94.52% (22.2) F(1,7) = 0.96, n.s. Success Criterion reached

32 Objectives and success criteria that are covered by performance and subjective data Validation Objectives Subjective Success Criteria Performance Success Criteria Human performance task Perceived visibility at the apron Detection of static objects Both support each other Safety during shadow-mode trials Perceived safety during shadow mode trial Detection of safety related manoeuvers Both are contradictory

33 Summary Validation Objectives and Success Criteria Human Performance task: The data support that CWPremote satisfied the need for information within the apron area. Safety during shadow mode trials: Sufficient information is provided, but in a different way than at the CWP-tower. Or the performance criteria do not cover the functional requirement exhaustively.

34 Implications for the next Iterations The results indicate that system and technical requirements must define in more detail which subsystem of a future remote tower system will provide the information to fulfill the functional requirements. For sharpening the functional requirements subjective and objective data sources are of relevance.

35 Conclusion and Outlook The results of this validation exercise provide an additional step for the remote tower concept validation, based on a live-video panorama. The validation focused on the functional requirements rather than the conventional engineering approach for system verification. Performance data provides an essential input to support the subjective results.

36 Thank you. > Which data provide the best insight? > Maik Friedrich > July 2013

37 Future Improvement Connection between Radar and PTZ control for direct aircraft selection Automatic tracking of the PTZ camera Increased discriminability of objects in the distance More than 200 horizontal viewport of the video panorama More ergonomically designed CWP-remote (e.g. bigger screens,...) More than 66 vertical viewport of the video panorama

38 Summarized Comments - Safety while Shadow- Mode Trials Main comments concerning the panorama : The southern traffic circuit could not be monitored. 360 horizontal viewport was deemed necessary. The quality of the video panorama was deficient. Main comments concerning the PTZ camera: The image stability and sharpness of the PTZ camera should be increased. The final approach should be further subdivided into different zoom levels. The liability to wind was problematic

39 Not Reached Detection of an Aircraft by Using only the Panoramic View

40 Not Reached ATCO s Perceived Usability & Acceptance Ratings Results Mean (SD) 3.05 (1.24) Positive value (>=4) 13 Negative value (<=3) 27

41 Not Reached - ATCOs Perceived Impact on ATCO Procedures Results Mean (SD) 3.73 (1.52) Positive value (>=4) 12 Negative value (<=3) 28

42 Not Reached - Tracking the D-CODE in the South Aerodrome Traffic Circuit

43 Summary, Objectives & Success Criteria SAFETY > Which data provide the best insight? > Maik Friedrich > July 2013 Safety while shadow- mode trials Human performance Human performance task Workload with respect to shadow-mode Accessibility ATCO Tasks, Roles and Responsibilities ATC Procedures Perceived safety in relation to CWP-tower Perceived safety Safety related manoeuvres Recognise test aircraft Detect an aircraft Detect static objects Status of the runway lights Windsocks indication Runway holding points Identify the manoeuvres Working passive-shadow Workload Perceived Usability / Acceptance Trust by working at the CWP-remote Impact on tasks, roles and responsibilities Track an aircraft Landing gear status States of weather South aerodrome traffic circuit Impact on ATCO procedures Usefulness Usability and utility of prototype features

44 Summary, Objectives & Success Criteria SAFETY > Which data provide the best insight? > Maik Friedrich > July 2013 Safety while shadowmode trials Perceived safety in relation to CWP-tower Perceived safety Safety related manoeuvres Not reached Reached Reached Human performance Recognise test aircraft Reached Human performance task Workload with respect to shadow-mode Accessibility ATCO Tasks, Roles and Responsibilities Detect an aircraft Detect static objects Status of the runway lights Windsocks indication Runway holding points Identify the manoeuvres Working passive-shadow Workload Perceived Usability / Acceptance Trust by working at the CWP-remote Impact on tasks, roles and responsibilities Track an aircraft Landing gear status States of weather South aerodrome traffic circuit Not reached Reached Unclear Reached Reached Reached Reached Reached Not reached Reached Reached Reached Reached Reached Not reached ATC Procedures Impact on ATCO procedures Not reached Usefulness Usability and utility of prototype features Reached