A UAV Pilotʼs Associate

09ATC-0287 A UAV Pilotʼs Associate Brian Cox Applied Systems Intelligence, Inc. Copyright 2009 SAE International ABSTRACT From 1986 through 1992, the Defense Advanced Research Projects Agency (DARPA) and the US Air Force collaborated to create a program called the Pilotʼs Associate. The Pilotʼs Associate was designed to analyze an aircraftʼs systems and environment resulting in real-time mission and tactical recommendations to the pilot. Technology developed for the Pilotʼs Associate Program is more relevant today than ever before. Pilots of unmanned vehicles are faced with making critical decisions during high-stress missions while located thousands of miles from the vehicle. Associate systems can aid unmanned vehicle pilots in making critical decisions, maintaining situational awareness and reducing task overload. INTRODUCTION the pilotʼs workload. The workload of a fighter pilot can be significant in combat situations due to the complex missions, multitude of threats and sophistication and speed of the aircraft. During high-stress situations, a pilot can be overwhelmed with data and actions to perform (Figure 2). Associate systems continue to benefit todayʼs pilots but are more relevant for pilots of unmanned vehicles. Pilots of manned vehicles have the benefit of sitting in the aircraft and experiencing everything first hand. Pilots of unmanned vehicles do not have the luxury of sensory feedback provided by the experience of sitting in the aircraft. They are fully dependent upon the vehicleʼs sensors to understand the state of the aircraft. This results in unmanned vehicle pilots having reduced situational awareness. Associate systems can aid pilots of unmanned vehicles in making critical decisions, maintaining their situational awareness and helping to reduce their task overload. In 1986, the Defense Advanced Research Projects Agency collaborated with the US Air Forceʼs Wright Laboratory to develop the Pilotʼs Associate Program. The Pilotʼs Associate was a $42 million dollar program spanning six years of effort, over half a dozen companies, and hundreds of engineers and scientist. By the end of the program, the Pilotʼs Associate was the most advanced, real-time intelligent system of its day. The overall goal of the Pilotʼs Associate was to analyze the tasks and the environment of a single seat, advanced tactical fighter and recommend plans or courses of action that would enhance the survivability of the pilot and the mission. The Pilotʼs Associate was essentially an intelligent, automated co-pilot designed to aid the pilot in making decisions (Figure 1) and to reduce Figure 1. The Virtual Co-Pilot The Engineering Meetings Board has approved this paper for publication. It has successfully completed SAEʼs peer review process under the supervision of the session organizer. This process requires a minimum of three (3) reviews by industry experts. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of SAE. ISSN 0148-7191 Positions and opinions advanced in this paper are those of the author(s) and not necessarily those of SAE. The author is solely responsible for the content of the paper. SAE Customer Service: Tel: 877-606-7323 (inside USA and Canada) Tel: 724-776-4970 (outside USA) Fax: 724-776-0790 Email: CustomerService@sae.org SAE Web Address: http://www.sae.org Printed in USA

operator [7]. An aviator operator is responsible for controlling the aircraft and navigation while a mission payload operator is responsible for searching for targets and monitoring system parameters. The Department of Defense would like to merge the responsibilities of these roles under the control of a single operator without increasing the operatorʼs workload [8]. Figure 2. Information Overload [1] A UAV PILOTʼS ASSOCIATE BACKGROUND Unmanned vehicles are far from unmanned. The pilot of an unmanned vehicle has simply been relocated from the cockpit to a remote command and control center that may be thousands of miles from the area of operations [2]. The removal of the human presence from the aircraft brings obvious advantages for the vehicle design and the hazardous missions that are enabled. However, the pilots of unmanned vehicles face more problems than pilots of manned vehicles due to the man-vehicle separation. One of the major problems with remotely piloting an aircraft is the sensory isolation. Unmanned vehicle pilots are deprived of the sights, sounds and smells of the cockpit. Their only interface to the vehicle is through the instrumentation and telemetry that is transmitted back to the control station. An example is the identification of turbulence. A pilot of a manned vehicle can feel the forces that turbulence places on the vehicle while pilots of unmanned vehicles cannot. The pilot of an unmanned vehicle determines the level of turbulence through instrumentation and by how much the video from the onboard camera is shaking [3]. The sensory isolation and limited camera angles result in impaired situational awareness and can degrade the pilotʼs performance [4]. Another problem faced by pilots of unmanned aircraft is monotony. Reconnaissance missions tend to be very similar and not very eventful. The cruise phases between take-off and the area of operation can be quite long. A term used by unmanned vehicle pilots to describe the duty is Groundhog Day in reference to the movie were every day is the same [5]. As a result of the long shifts, pilots can become disengaged and again lose situational awareness [6]. One of the most significant problems facing unmanned vehicle pilots is an increasing workload. Many unmanned vehicle platforms require more than one One method to reduce the operatorʼs workload is the use of variable autonomy. Multiple levels of automation allow an operator to delegate tasks and responsibilities to intelligent agents while continuing to be engaged at several abstract levels of control. Sheridan outlined a scale (Table 1) of automation from 1 to 10 where each level represents more automation [9]: Table 1 1. Manual Control. No Automation 2. Computer offers a full set of possible actions 3. Computer offers a subset of actions 4. Computer recommends a single action 5. Computer executes action if approved by operator 6. Computer allows limited time to veto execution 7. Computer autonomously executes action and notifies operator 8. Computer autonomously executes action and informs operator if requested 9. Computer autonomously executes action and informs operator at its own discretion 10. Computer executes all actions autonomously and does not inform operator Variable autonomy has been shown to improve task performance [10], situational awareness [11] and lower the operatorʼs workload [12]. Associate systems can provide many of the capabilities required to mitigate the problems that pilots of unmanned vehicles face. Associate systems support multiple levels of autonomy and allow the operator to interact with the system at different levels of abstraction. By providing information management, associate systems can reduce information overload by displaying relevant information in a context sensitive manner. Associate systems work along side a human operator, helping to maintain the operatorʼs situational awareness and assisting in the execution of complex jobs. ASSOCIATE SYSTEMS OVERVIEW Associate systems elevate the operator to the role of the decision maker or mission manager. Rather than have the operator manually control every servo and actuator of the vehicle, associates enable the operator to focus upon higher-level tasks that are critical to the success of the mission. Associate systems support active information collection, situation assessment, planning, plan execution and coordination with multiple human operators and other associates. They can be designed to assist with all command and control tasks of a vehicle. As a result,

tasks do not need to be allocated exclusively to the human operator or to the associate, but can be allocate to both the human operator and the associate. An associate architecture enables collaboration of multiple agents regardless of whether the agent is a human operator or an associate system [13,14]. An associate system supports multiple levels of automation but there is a distinct difference between conventional automation and the autonomous behaviors of an associate system. Conventional automation attempts to replace human control and decision-making. Intelligent associate systems do not attempt to replace human control but rather augment human control. Associate systems operate as intelligent aiding systems that attempt to enhance the operatorʼs judgment and responsibility. Every task that an associate executes is under the supervision of a human operator. The operator can assign authorization levels to each plan or action. Example authorization levels include manual, permission, veto and autonomous. In manual mode, the operator has full control of the task execution. The associate system may propose a plan in permission mode but may not execute it without explicit authorization from the operator. In veto mode, the associate is authorized to execute a proposed action if the operator does not veto the action within a set timeframe. In autonomous mode, the associate system is authorized to select and execute proposed actions. Knowledge Representation While there is no widely accepted definition of intelligent behavior in its detail, there is broad agreement about several of its features. Intelligence is a dynamic system that takes in information about the world, abstracts regularities from that information, stores it in memories, and uses a priori knowledge about the world to form goals, make plans and execute plans. The core knowledge of associate systems is represented in two graph data structures referred to as concept-node graphs (CNG) and plan-goal graphs (PGG). Figure 3. Plan Goal Graph Plan-Goal Graphs - PGGs store the actions that either a vehicle or pilot can perform. An example of a generic PGG can be seen in Figure 3. The PGG contains two types of nodes: plans and goals. Plan nodes are always children of a goal node. The plan nodes represent operations and their parent goals represent the intended effect. At the top level of the PGG are abstract goals. These represent desired behaviors that are highly aggregated and abstract. As the levels of the PGG are traversed downwards, each layer becomes more concrete and specific. The lowest level of the PGG consists of primitive actions that can be directly performed by a pilot or vehicle. Concept Node Graph - The CNG is a hierarchical description of the world state. It represents the associateʼs beliefs and trends about the environment. It is designed to increase in abstraction and aggregation when traversing up the graph, as well as show dependencies between concepts in the form of links (Figure 4). In a concept graph, sensor data is inserted into the concept graph in lower level nodes. Value is added to the sensor data by aggregating and abstracting the raw data to form higher level conclusions. An example of aggregation would be determining a vehicleʼs range based upon its fuel level, speed and payload weight. An example of abstraction would be calculating the time a vehicle has at a target location based on the vehicleʼs current range capability. Figure 4. Concept Node Graph Vehicle and environmental constraints are implemented in the concept-node graph. As sensors on the vehicle detect objects in the environment, the objects would be inserted into low-level nodes of the graph. As the vehicle moves through the environment, the location of these objects would be updated. If the range to an object falls below a specific threshold, the object can be abstracted and be classified as an obstacle. The existence of an obstacle concept in the CNG can trigger a new goal to become activated in the plan-goal graph. In this example, the new goal would be to avoid obstacles and a plan would be selected to achieve this goal. Knowledge Processes Several knowledge processes operate on the plan-goal and concept-node graph structures.

Planning Process A dynamic planning process operates on the knowledge to select the best plan to achieve a goal. At design time, the plan-goal graph is organized from high-level plans and goals down to primitive actions. At run time, the dynamic planner uses a skeletal planning approach to compose courses of action that can be executed in parallel by the operator or one or more vehicle associate systems. This algorithm is in the family of Partial Order Planners, and while not a provably optimal planner, it is known for its speed and heuristic correctness. It is capable of simultaneous plan generation and plan execution. The planning algorithm is specifically organized to permit continuous human interaction during plan generation and plan execution. This enables an associate system to continue to recommend plans to an operator even though the operator may have rejected the associateʼs current course of action. Intent Interpretation Process While planning is done through PGG decomposition from high-level plans and goals to primitive actions, intent interpretation works up the PGG by observing actions and then attempting to identify the high-level plan associated with that action. This enables an operator to communicate his intent using abstract plans. For example, a vehicle operator can send a mid-level plan or goal to an intelligent associate on a vehicle without explicitly transmitting the higher-level plans and goals. Once the node is received, the vehicle associate can infer the higher level plans and goals the operator is attempting to accomplish. The dynamic planner on the vehicle will complete those portions of the overall plan that were omitted. A benefit of this capability allows associate systems to operate with poor or intermittent communications [15]. Information Management Since the associate system has knowledge of the operatorʼs current plans and goals, this allows the system to manage information that is relevant to the operatorʼs current task. By monitoring the vehicleʼs state along with environmental conditions, the associate knows the context in which the operatorʼs plans are operating. If the operator attempts to execute a plan that conflicts with an environmental or vehicle constraint, the associate system can display this conflict to the operator. Situation Assessment Process An associate system independently analyzes the situation based on data collected from the vehicle sensors and communicated to the associate remotely. Situation Assessment provides a knowledge-based approach to the interpretation of realtime data by dynamically maintaining abstract concepts that describe the situation as understood by the associate. Error Handling Unlike conventional error handling approaches that make only limited use of the semantics of a user input to detect errors, the associateʼs Error Management process provides robust error detection for single and multi-user applications. The Error Management component works with the intent interpreter to detect operator errors (incorrect execution), user intent errors (mistakes, correct execution of the wrong task) and breakdowns in coordination across multiple users. Exceptions raised by the intent interpreter are analyzed by the Error Management component to determine the type of error and its best remediation. MULTI-VEHICLE COLLABORATION The Office of the Secretary of Defense (OSD) has created an Unmanned Systems Integrated Roadmap that details the type of missions that unmanned vehicles will be required to perform in the next 25 years [8]. These missions will place heavy requirements on the performance of unmanned systems. As a result, the performance attributes associated with unmanned systems must evolve significantly to keep pace with this demand. One of the highest goals is the ability to allow a single pilot to control multiple vehicles. Associate systems support multi-vehicle collaboration with several key capabilities [16]: Distributed Intelligent Architecture Mixed Initiative Framework Decision Aiding Situation Assessment Communications Management Distributed Intelligent Architecture When a person manages a team of people, the manager does not micromanage each individual by controlling every little required task. The manager delegates high-level tasks and allows the team to bring their intelligence and resourcefulness into the situation to implement the current assignment. An operator controlling a team of unmanned vehicles is no different. The operator must be able to send high-level, symbolic commands to the team of vehicles in order to collaborate. By having an associate onboard each vehicle as well as the operatorʼs ground control station, intelligent collaboration is enabled. In order for the unmanned vehicles and the operator to collaborate, they must share a common view of the mission goals and a common view of the world [17]. The plan-goal graph in each associate models the goals and actions that can be performed during the course of a mission. The concept-graph stores the state of the world. Since the plan-goal graph is a hierarchical abstraction of the goals and plans, the operator can send high-level goals to each vehicle. The associate onboard each vehicle can then decompose the abstract goal into lower-level sub-goals and actions. If the situation changes and the vehicle cannot achieve the desired goals set by the operator, the associate on-board the

vehicle can dynamically re-plan using its local plan-goal graph. By having the intelligent control architecture distributed across each vehicle and ground control system, the operator and the vehicle can support mixed initiative behaviors. Mixed Initiative - Mixed initiative behavior is described as the smooth blending of actions between the operator and the remote, autonomous vehicles. This enables each collaborating agent to contribute what it does best at the appropriate moment. The human operator brings common sense, intuition, creativity and value systems to problem solving and decision-making. The associate systems excel in speed of mathematical computations, storage and retrieval of large quantities of information, unaffected by stress and can use subject matter expertise to assist with problem solving. The hierarchical and abstract representations in the plan-goal graph and the concept-node graph allow the operator to interact with the system at very low levels such as setting flap angles or through high level commands such as a single command to return to base. However, when the vehicles determine the need to re-plan as in the case of an inbound missile, the vehicles must perform their actions within boundaries set by the operator. Adaptive Autonomy - A collaborative control architecture must support autonomous behaviors on-board each of the remote vehicles in order for the operator to efficiently control more than one vehicle [18,19]. The architecture must preserve the operatorʼs freedom, authority and accountability. Associate systems support adaptive automation that is not fixed at design time but varies appropriately with changes in the operational environment. In past associate implementations multiple authorization levels have been implemented [16]. Each task or action that an associate system can execute can be assigned an authorization level. An example of four authorization levels is described below: Manual - The operator is in full control of the vehicle and the onboard associate system cannot propose or execute any tasks. Permission - The onboard associate system proposes a course of action and waits for the operator to give approval. If the operator does not respond in a specified amount of time, the plan is assumed to be rejected. Veto - The onboard associate system proposes a course of action and waits for the operator to give approval. If the operator does not respond in a specified amount of time, the plan is assumed to be accepted. Autonomous - The associate system has full control of plan proposal and execution. By supporting adaptive autonomy and flexible authorization levels, an operator is able to maintain close supervision of the vehicles. As unexpected events occur or the communications become unreliable, the operator is able to authorize more autonomous operations onboard the vehicles. Conversely, if the operator wishes to exert more control over the vehicle activities, the associate architecture allows the operator to regain additional control at any time. The combination of the hierarchical plan-goal graph data-structure and the flexible authorization levels provides a powerful mixed initiative framework for multi-vehicle collaboration. Decision Aiding In a complex area of operations, there may be an overwhelming amount of information and tasks for an operator to manage. The amount of information and attention required of the operator is only compounded when controlling multiple vehicles. In order for the operator to manage the workload, tasks need to be delegated and executed by the vehicle associate system. The operator can remain in the control loop and supervise the vehicleʼs actions by setting the plan authorization levels to either permission or veto. Before an associate can execute the plan, the associate system must submit the plan to the operator for approval. The operatorʼs authority is always preserved since he can accept or reject the recommended actions. Associate systems can provide an effective decision aiding system to aid an operator in controlling multiple vehicles in complex situations. Situation Assessment In a system of collaborating vehicles, the concept-node graph allows multiple vehicles to cooperatively build and maintain a model of the state of the world. The associate system on each vehicle will process its own sensor data and maintain an instance of the concept graph that corresponds to the current situation of that vehicle. The vehicles can exchange situational information in the form of high-level concept nodes as opposed to raw sensor data. As a result, the communication bandwidth requirements between vehicles and the ground control station are reduced. 14 Other vehicles as well as the operatorʼs ground control station may incorporate the transmitted nodes into their own concept graphs. The intent interpretation process will fill in the missing concept nodes and allow the vehicles to collaborate by only sharing abstract goals. Communications Management Associate systems provide a framework for sharing a vehicleʼs intentions and plans to support collaboration. The plan-goal graph provides the common language between vehicles to support communications. The operator commands the vehicles by sending goals or activating goals in a vehicleʼs plan-goal graph. In return, each vehicle shares its world state information from its concept-node graph and active nodes from its plan-goal graph. As a result, the plan-goal graph and situation assessment provide a

context for vehicles to communicate enabling them to collaborate jointly on mission objectives. In addition to sharing a vehicleʼs intention and plans, vehicles must support a mechanism for resolving conflicts. The operator may not always have reliable communications to allow him to coordinate the vehicles tasks. For example, if the vehicles encounter a new target during a communication dropout, the vehicles must be capable of collaborating amongst themselves to determine which vehicle will attack or reconnoiter the target. The associate framework can support multiple methods of task allocation and conflict resolution. One notable example is the development of a market-based approach to task allocation [20]. In this approach, the associate systems on each vehicle conducted an auction to determine which vehicle could minimize a cost function and have the best chance of performing the task successfully. The cost function consisted of multiple constraints such as vehicle fuel, commitment to another task, available weapons, range to target, vehicle damage, etc. By supporting conflict resolution and autonomous task assignments, the associate framework eliminates the central communication fault point by distributing control of mission tasks across all unmanned vehicles instead of through a central control point. CONCLUSION Associate systems were first designed in the early 90ʼs for single seat fighter aircraft but are more relevant today for pilots of unmanned vehicles. Associate systems can aid pilots of unmanned vehicles in making critical decisions, assist in maintaining their situational awareness and help to reduce their workload. By combining the features of a distributed intelligent architecture, mixed initiative framework, decision aiding system, situation assessment and communications management system, associate systems can not only aid operators in controlling single vehicles but aid in the collaboration of multiple vehicles as well. REFERENCES 1. Yannone, R.M., The Role of Expert Systems in the Advanced Tactical Fighter of the 1990ʼs, Proceedings of the 1985 National Aerospace and Electronics Conference, Dayton OH, IEEE, New York, 1985. 2. Drew, J. et al, Unmanned Aerial Vehicle End-to-End Support Considerations, Santa Monica, Rand Corporation, 2005. 3. McCarley, J, Wickens, C., Human Factors Concerns in UAV Flight, University of Illinois, 2004. 4. Endsley, M., Situation Awareness Global Assessment Technique (SAGAT), Proceedings of the National Aerospace and Electronics Conference, Ohio, 1988. 5. Tyabji, A., Unique Problems Associated with UAV Employment,<http://findarticles.com/p/articles/mi_m 0IBT/is_5_63/ai_n19396165/>, May 2007. 6. Colebank, J., Miller, N.L., Platte, W., Swigart, C., Tvaryanas, A.P. A Resurvey of Shift Work-Related Fatigue in MQ-1 Predator Unmanned Aircraft System Crewmembers, 311th Performance Enhancement Directorate, March 2008. 7. Nas, M., The Changing Face of the Interface: An Overview of UAS Control Issues & Controller Certification, UATAR Working Group, Murdoch University, WA, 2008. 8. FY2009-2034 Unmanned Systems Integrated Roadmap, Office of the Secretary of Defense, 2009. 9. Sheridan, T.B. and Verplank, W.L., Human and Computer Control of Undersea Teleoperators, MIT, Cambridge, 1978. 10. Hilburn, B., Jorna, P.G., Byrne, E.A., and Parasuraman, R., The Effect of Adaptive Air Traffic Control Decision Aiding on Controller Mental Workload, Human-automation Interaction: Research and Practice, Lawrence Erlbaum, Mahwah, NJ, 1997. 11. Kaber, D.B., Endsley, M.R., The Effects of Level of Automation and Adaptive Automation on Human Performance, Situation Awareness and Workload in a Dynamic Control Task, Theoretical Issues in Ergonomics Science, 5(2), 2004. 12. Prinzel, L.J., III, Freeman, F.G., Scerbo, M.W., Mikulka, P.J. and Pope A.T., Effects of a Psychophysiological System for Adaptive Automation on Performance, Workload and the Event-Related Potential P300 Component, Human Factors, 45(4), 2003. 13. Elmore, W.K., et al, Intelligent Control of Automated Vehicles: A Decision Aiding Method for Coordination of Multiple Uninhabited Tactical Aircraft, DARPA No. MDA972-98-C-0011, 2000. 14. Lizza, C.S., Banks S.B., Pilotʼs Associate: A Cooperative Knowledge-Based System Application, DARPA Strategic Computing Initiative, IEEE Expert, June 1991. 15. Dunlap, R.D., Elmore, W.K. & Campbell, R.H., A Paradigm for Command and Control of Intelligent Vehicles with Restricted Communications, Proceedings of the AUVSI Unmanned Vehicles 2001 Symposium, Baltimore, MD. 2001. 16. Elmore, W.K., Dunlap, R.D. and Campbell, R.H., Features of a Distributed Intelligent Architecture for Unmanned Air Vehicle Operations, Proceedings of the AUVSI Unmanned Vehicles 2001 Symposium., Baltimore, MD, 2001. 17. Geddes, N., Intelligent Control for Automated Vehicles: A Decision-Aiding Method for Coordination

of Multiple Uninhabited Tactical Aircraft, Proceedings of the AUVSI Unmanned Vehicles 1998 Symposium, Huntsville, AL, 1998. 18. Parasuraman, R., Barnes, M., Cosenzo, K., Adaptive Automation for Human-Robot Teaming in Future Command and Control Systems, The International C2 Journal, Volume 1, Number 2, 2007. 19. Cummings, M.L., Bruni, S., Mercier, S., Mitchell, P.J., Automation Architecture for Single Operator, Multiple UAV Command and Control, The International C2 Journal, Volume 1, Number 2, 2007. 20. Atkinson, M., Results of Using Free Market Auctions to Distribute Control of UAVs, AIAA 3 rd Unmanned Unlimited Technical Conference, Chicago, Illinois, September 2004. CONTACT For more information please contact: Brian Cox bcox@asinc.com Applied Systems Intelligence, Inc. 3650 Brookside Parkway, Suite 500 Alpharetta, GA 30022 770-518-4228