Presentation Outline

Transcription

1 UAV Operator Interface Research at AFRL/HEC May 24, 2004 Mark H. Draper, Ph.D. AFRL/HECI

2 Presentation Outline Problem/Context AFRL/HEC Overview Advanced Controls and Displays for Predator Interface Research: Multi-UAV Supervisory Control Control/display concepts & software framework Real-time UCAV operator state assessments & adaptive interfaces Situation assessment & decision support tools Summary 2

3 UAVs: Promises and Realities It shall be a goal of the Armed Forces to achieve the fielding of unmanned, remotely controlled technology such that by 2010, one-third of the aircraft in the operational deep strike force aircraft fleet are unmanned Defense Authorization Conference Bill - H.R It s an extremely fragile aircraft extremely pilotunfriendly. Unmanned aerospace vehicles (UAVs) are the hallmark of our future Dr. James Roche, Oct 2001 ng ns Increasing UAV Focus UAV Investment You have a two dimensional image and you re flying without all of your senses. I hope I ve demonstrated personally my commitment to UAVs, I am committed Gen. John Jumper,Sept 2001 A good number of them were lost - operator error. It s hard to land this thing. Fundi (Billio ) It s like driving your car with paper towel tubes over your eyes Danger is that you get too focused on what you can see, and neglect what you can t see 80-'89 90-'99 00-'10 Piloting is an intensely visual task. Gone are the large field of regard, the subtle seat-of-the-pants inputs and numerous Decade clues which allow better SA 3

4 Vision: Multi-UAV Supervisory Control by Single Operator Requires Increasing: UAV autonomy UAV intelligence : : :3? Operator s role must change Introducing autonomous agents changes the cooperative structure, creating new roles, new knowledge requirements, new judgments, new demands for attention, and new coordinative activities. Failing to address these leads to patterns of breakdowns and automation surprises. (Patterson et al., 1998) 4

5 Human-Automation Systems Lessons Added Learned UAV from Design Automated Constraints: Cockpits Added UAV Design Constraints: Most difficulties related Remote to lack of operation mode awareness and to gaps in pilots mental models of Remote the functional operation structure of the automation. of of Surprised pilots when time Multiple was critical. vehicles Multiple vehicles with Pilot anxiety about what the system with was/will be doing Added vehicle intelligence, Added vehicle intelligence, Limited comm bandwidth, Limited comm bandwidth, Automation heightens need for feedback. and Automation does not have uniform and effect on workload Lengthy system time delays Lengthy system time delays How is it done today and what are the limitations of the current practice? Automation should be viewed as operator aid, not simply replacement Advances in human-system integration technologies are required SA is everything & includes automation awareness: Is the aircraft to conduct increasingly complex UAV missions doing what was planned? AF SAB Summer Study on UAVs,

6 AFRL/HEC Goal Warfighter Interface Division Increase warfighter lethality and enhance combat survivability while reducing information processing demands through revolutionary human interface technology 6

7 Predator-focused UAV Operator Interface Research

8 Research Objective What are we trying to do? Design and evaluate multi-sensory interface technologies and visualization techniques to improve UAV operator performance, increase operator situational awareness (SA) and manage workload. 8

9 Why Multi-Sensory Interfaces? Multi-sensory Interfaces Facilitate simultaneous tasks (Wickens, 91) Offload the visual channel (Tannen et al., 00) Increase telepresence (NASA X-36 operator, 99) Expand display real estate (Nelson, et al., 98) Enhance SA of automation (Sklar & Sarter, 99) Improve control procedures (Williamson, 00) Research Issues Information display Display dimensionality Simultaneous global/local SA Inattentional blindness Intuitive control 9

10 Research Approach Baseline System: Predator Gather Interface Issues User/Issue User/Issue Centered Centered DEMO Milestones Partial Immersive Interface Reduced Crew Interface Incremental Evaluations Potential Solutions/Value Analysis Develop Simulation Facilities 10

11 Research Approach Interface Technologies HMD Tactile Displays Synthetic overlays Symbology Improvements Speech Interfaces Haptic Feedback 11

12 Speech Recognition for UAV Control Tasks Objective Compare utility of speech input to manual input in UAV control data entry tasks. Implementation Nuance speaker-independent system 160 word/phrase vocabulary Push-to-talk button (on joystick) HUD visual feedback of spoken commands Results SPEECH INPUT EFFECTIVE Checklist completion: 40% faster Errors reduced by approximately 70% Speech Recognition> 95% with no training required 12

13 Tactile Displays for Directing Attention Objective Evaluate as Cue high priority alerts Parameter states F1 and F2 F3 and F4 Implementation (ONGOING) Electromechanical tactors Vibration swept sec Tactors versus visual & auditory cues Left Both Right Results Faults quickly & accurately detected No interference with concurrent tasks Benefit strongest when sole alert Equal to audio as redundant cue 13

14 Synthetic Vision Overlay for Improved Operations Objective Enhance sensor imagery with database & real-time C2 & intel Battlespace SA to UAV crew Networked collaborative comm Implementation (ONGOING) INDUSTRY CRADA (RIS Inc.) RIS: UAV flight-test software HE: designs/tailoring ACC/DR-UAV SMO, ASC/RAB, 53rd TEG, GA-ASI, UAV Battlelab Interface development: usability Results: TBD Warfighter/industry evaluations Upcoming flight demonstrations 14

15 Interface Research: Multi-UAV Supervisory Control

16 Multi-UAV Supervisory Control DTO HS.48 What are we trying to do? Design and evaluate advanced, adaptable decision support interface technologies to maximize flexible, fault-tolerant supervision of multiple intelligent semi-autonomous UAVs by a single operator 16

17 Multi-UAV Supervisory Control Research Three related research thrusts to enhance the cognitive capabilities of UAV operator Thrust 3 Situation assessment & decision support tools Thrust 2 Real-time UCAV operator state assessments & adaptive interfaces Thrust 1 Advanced control/display concepts for supervisory control 17

18 Multi-UAV Supervisory Control Research Thrust #1: Enabled Multi-UAV Supervisory Control by Single Operator Flexible control station simulation Advanced design, prototype, test capability Boeing Collab Agreement (UCAV/J-UCAS) Common Simulation Software Framework Standard UAV operator interfaces Reduce stovepiped designs Automation, aerial re-fuel studies 18

19 Multi-UAV Supervisory Control Research UAV/UCAV Operator Vehicle Interface (OVI) Simulation Software 19

20 20

21 Multi-UAV Supervisory Control Research Thrust #2: Adapt Interface to Operator Functional State Develop/apply adaptive interfaces for improved system performance UCAV operator studies Boeing Augmented Cognition team Real-time measurement & inferencing of UCAV operator functional state Psycho-physiological/performance metrics Identify workload extremes Adaptive Interface Lab 21

22 Implementing Real-Time Human Engineering Operator Workload Operator SA Operator State Assessment Operator Performance Determine Adaptation Required System Momentary Mission Requirements 22

23 Multi-UAV Supervisory Control Research Thrust #3: Situation Assessment & Decision Support Tools FY04 Start Enhance UAV operator s ability to acquire, assess, and act on information Visualization methods supporting decision-making and trend analyses Conveyance of UAV behavior rationale & intent Maintenance of shared situation awareness 23

24 Decision Support Tools Intelligent Cueing Information Fusion Playbook Delegator Expectation Manager Intuitive Queries Dynamic Automation 24

25 Multi-UAV Supervisory Control Research Thrust 2 Adapt Interfaces to Operator State Thrust 3 Situation Awareness & Decision Support Tools Thrust 1 Multi-UAV Supervisory Control by Single Operator 25

26 Other HEC UAV Activities Operator interfaces for small UAVs Mission Management Display fusion Architectures for dynamic levels of automation Playbook task delegation method Gaming techniques Mission management enhancements NATO TG Uninhabited Military Vehicles- Human Factors of Augmenting the Force Global Hawk support Predator MAC support 26

27 Multi-UAV Supervisory Control Research Assuming success, what difference will it make to the user or in a mission area context? End-User Benefits Reduced operator-to-vehicle ratio Lifecycle cost savings/force multiplier Reduced deployed footprint Increased mission effectiveness Decisions from reacting to anticipating Increased mission flexibility Better awareness of system state/intent Timely response to contingencies Reduced mishaps Potential upgrades to existing fleet Reduction in stove-piped interfaces 27

28 Summary Thrust 2 Adapt Interfaces to Operator State Thrust 3 Situation Awareness & Decision Support Tools Thrust 1 Multi-UAV Supervisory Control by Single Operator Tele-operated UAV Research Advanced controls and displays Multi-UAV Supervisory Control GUI/software framework Operator state/adaptive interfaces Situation assessment/decision support 28