Delegation Approaches to Multiple Unmanned Vehicle Control

Transcription

1 CERI Human Factors of UAVs Workshop May 24-25, 2004 Chandler, AZ Delegation Approaches to Multiple Unmanned Vehicle Control Christopher A. Miller, Robert P. Goldman, Harry B. Funk

2 Core Challenges of Multi-UAV Control Overhead surveillance FUTURE UAV DARPA s HURT Concept Rapid fly-by shooting of concealed targets LOS comms relay chain Persistent mobile sensors Ground and subterranean recce Multiple Vehicles per operator ¾ ¾ Increased Workload Situation Awareness Challenges Heterogeneous Vehicles ¾ ¾ Breadth of Training Workstation compatibilities Multilateral geolocation Side-looking RSTA Cueing sensors Agile urban navigation to see in portals Lower Echelon, Small Units ¾ ¾ ¾ Training burden Workstation portability Concurrent Workload Urban Terrain ¾ ¾ ¾ Concurrent Workload Burden Situation Awareness Demands Need for Mobility 28 May

3 How do you manage complex Agent(s)? Where can we look for a model of how to interact with complex, intelligent, automated systems? Supervisory control: how humans delegate tasks to other humans Delegation 28 May

4 Plays and Service Requests Plays are quick Delegation labels assigned to a defined range of behavior, Interfaces perhaps for multiple agents A parent task with, potentially, multiple alternate performance methods (= subtasks) Play Calling uses the label to activate the range of behaviors leaving decision making about subtasks to the actor(s) Plays can be refined, constrained and even created by reference to prior plays and their underlying structure ( Calling Signals ) E.g., constrain or stipulate alternate methods Service Requests are just like plays (use the same vocabulary) Playbooks in all but authority of the caller 28 May

5 Playbooks as an Instance of Delegation Delegation Interfaces Playbooks Playbooks are a type of Delegation interface Emphasis on Plans and Stipulations Other modes of delegation include goals, constraints, and policies Require a shared knowledge of domain actions We use a Hierarchical Task Network Agents are granted authority within the a space defined by the play that is called Plays are compiled plans But shared task model enables interactions at lower levels 28 May

6 Delegation Methods to be Supported 1. Goal (partial state) to be achieved 2. Plan (action sequence) to be performed 3. Constraints (states or actions) to be avoided 4. Stipulations (actions or states to be achieved) 5. Value statement on states and actions Supervisor Method Subordinate Responsibility Goal Plan Constraint Stipulation Value Statement Achieve goal if possible; report if incapable Follow plan if possible; report if incapable Avoid actions/states if possible; report if not Achieve actions/states if possible; report if not Work to optimize value 28 May

7 Sample Delegation Architectures Playbook 1. Goals 2. Plans 3. Constraints 4. Stipulations Policy 5. Values 1. (Goals) 2. (Plans) 3. (Constraints) 4. (Stipulations) 28 May

8 Policy for Value-based Delegation Logistics info woulda been mighty nice about now Incoming request.9.8 Match.6.1 Request with importance.5 Cmdr N s Policy A priori specification of goodness or value rules to cover predictive or hypothetical situations Can be linked to tasks, contexts, chain of command, etc. Evaluation of situation in terms of those rules Optimization of Score via Control or Allocation Function Summary Situation Visualizations based on Score Status: Prototype systems for Military Communication Resources and Flight Operations Planning 28 May

9 Combine Provides Playbook-enhanced Variable Autonomy Control System Phase II SBIR sponsored by DARPA/IXO ( ) Concept Playbook (analogy from sports) with Variable Autonomy Control System (VACS) end-to-end control of platforms from simple requests Technical Approach Playbook provides user selectable plays; SHOP2 planner turns play into abstract VACS commands VACS translates commands into platform specific behaviors Playbook monitors execution; does fallback re-planning Use multi-platform (OAV, GTMax, Dakota) 6-DOF aero /McKenna MOUT sim for realism Benefits Shared understanding of play between warfighter and automation unambiguous communication Rapid, flexible tasking Platform independence Multi-platform coordination in sim 28 May

10 Plan/Task Ingress Playbook Illustration Ground Attack Target Attack Defense Suppression ARTY Support Decoy Aux. Attack User Controls J Grnd Attack Actor: Target: Airfield Method: Cnsts: Egress Aux Attack UAV 1 SAM 4 Actor: Target: Method: Cnsts: Sabotage Stipulation P l a y L i b r a r y 28 May 2004

11 Playbook Reference Architecture Execution Environment Geneva s VACS Playbook UI Instructions Shared Task Model Special Purpose Planners (e.g., Route, Sensor) Feedback Analysis & Planning Component Event Handling Control Algorithms SIFT s Playbook Playbook Vehicle(s) Playbook + VACS = PVACS 28 May

12 Planning in/via a Playbook Approach: Hierarchical Task Network Planning Uses same task model as operator Planning as Constraint Propagation obeys human inputs critiques human when requests are impossible Can integrate with specialty planners (e.g., route planner) for more detailed projection/ analysis Currently integrating with Geneva s VACS Provably correct (achieves goals, honors constraints) expansion of plans to meet goals. Tradeoff between projection and execution 28 May

13 One Architecture, Many Interfaces The underlying task architecture of the Playbook permits many different kinds of interfaces More and less detailed by moving up and down in the hierarchy Broader or narrower domain by breadth of hierarchy included Plays are precompiled chunks of the hierarchy with options specified to gain efficiency (at the cost of flexibility) High Level Planning and Control Novel Play Creation Low Level Planning and Control In-Flight Control 28 May

14 Extended Illustration of PVACS Control of Heterogeneous UAVs in Urban Operations

15 PVACS Simulation System Network Playbook Execution Environment User Interface (Play Calling) Playbook Server (planner & executive) GTMax And Dakota Simulations VACS Ground Control Station (Execution) User Interface VACS Control Systems 28 May

16 Background Operator is a commander (or maybe UAV operator) of a recon platoon. Platoon in hot pursuit of saboteurs who have fled into an apartment building. Aware that they may be stumbling into a trap, Operator knows his team is too small to leave men outside to guard. Calls for back up may be as long as 30 min. before arrival. To protect themselves from unexpected enemies entering the area, the Operator wants to rapidly call for monitoring of the intersection outside the building where enemy reinforcements may come. 28 May

17 MOUT Scenario MOUT target area Target Intersection GTMax Starting Position Dakota Starting Position 28 May

18 Structure of Demonstration Operator has initial, prior-to-the-scenario training with PVACS about what plays are available and what they mean Brief because plays are drawn from standard doctrine and vocabulary Focus on one play: Overwatch illustrated on the next slide. Basically, Give me sustained surveillance of an area Examples of other plays for this type of user: Track Target Area Recon Route Recon Watch Perimeter Encircle Patrol Might be appropriate to configure available plays in a pre-mission briefing based on what s likely to be necessary and what available resources will support. 28 May

19 Overwatch Overwatch means to provide continuous relayed imagery of a fixed or moving point or area. Overwatch Overwatch Sortie X Aircraft: Target Area: Imagery Start: Imagery End: Obtain X Ingress Scan Egress Aircraft X X Destage X Maneuver Fly-to X Other likely plays: Track Target, Area Recon, Route Recon, Watch Perimeter, etc. 28 May

20 Interacting with a Desktop Playbook Playbook UI 28 May

21 Interacting with a Desktop Playbook 28 May

22 The User s Perspective Making a Request Video show how the user interacts with the PB UI to request Overwatch service At a minimum, this includes: Selecting Overwatch from a short list of alternatives Illustrate parameters available to be specified Illustrate those parameters which are filled in by (smart?) defaults Illustrate parameters the user is required to fill in Illustrate the user filling in one or more parameters (probably target area at a minimum) Certainly designating the Overwatch area and maybe stipulating a time) (Not Available for future version) Illustrate the difference between hard and soft constraints (in the interface) and the user selecting one of them for a parameter input (possibly the time input) Potential Additions Illustrate the same behaviors in the Falcon View UI (definitely though shortening would be good)\ Talk up how PVACS knows which vehicles are available to allow user to select between? 28 May

23 What Playbook does with the request Illustrate PB-APC reasoning to fulfill the requests What vehicles are available How to prune the list Basic Overwatch structure (illustrating Single and Multiple-sortie branches) Attempt to satisfy single failure Success with multiple sorties Query route planner for feasibility Stopping conditions when have we planned enough for VACS? (Talk) talk about how PB would respond to an over constrained request Any illustration of the Plan (as the user/ui would see it)? 28 May

24 Interaction with a PDA UI User designates overwatch target on map Feedback on image start time Abort/reject capability Overwatch play called via pulldown menu Selected and resources presented for review and editing Play monitoring and updating interactive control Video image monitoring eventually, alerting 28 May

25 Scenario/Play Selection User is presented with drop down of scenario and play to choose. (there will be a map here, but it isn't in place right now as I'm knee deep in changing code for it) User selects from drop downs and presses 'call play' 28 May

26 Play Parameter Defintion User is presented with parameters either in drop downs, or in text fields (drop downs if server provides list of values) User selects 'plan' when done, and it brings up the plan display (just xml right now) 28 May

27 Plan Display Currently in very rudimentary form. Basically takes the xml plan output from the planner and displays it on the scren. User can Abort the Plan (edit functionality not working yet) 28 May

28 What the APC does with the Plan The PB-APC route the resulting plan, as a series of waypoints to the various UAV(s) involved, mediated through the VACS GCS Illustrate Playbook (and VACS) begins execution of the plan immediately Illustrate: User gets an indication of this in the visualization of aircraft moving toward the target area (on Big display). Count down clock to surveillance initiation (on PDA). GCS relays ongoing state information to the PB to enable it to act as an outer loop controller Detect need for replans Detect need to issue context-based commands to vehicle (e.g., begin sensor operations) Illustrate plan execution Synthetic images of aircraft flying route UI images of clock counting down UI images of streaming video surveillance Synthetic images of aircraft being replaced by second aircraft Any potential to illustrate a plan upset? Probably not enough time 28 May

29 Planning Trace Depth 6, trying task (:TASK OVERWATCH ALPHA :UNBOUND :UNBOUND) Depth 6, trying method SINGLE-UAV-OVERWATCH Level 2, couldn't match goal (UNAVAILABLE 201) Level 3, state satisfies goal (TARGET-LOCATION ALPHA #:?TLOC369412) Level 2, state satisfies goal (GROUND-TARGET ALPHA) Level 0, couldn't match goal (:UNBOUND 200) Level 2, couldn't match goal (AIR-TARGET ALPHA) Depth 6, inapplicable method SINGLE-UAV-OVERWATCH Depth 6, trying method MULTIPLE-UAV-OVERWATCH Level 2, couldn't match goal (UNAVAILABLE 201) Depth 8, trying method #:OVERWATCH-SORTIE Level 0, state satisfies goal (UAV 201) Level 9, state satisfies goal (LOCATION 201 #:?UAV-LOCATION369596) Level 4, state satisfies goal (TARGET-LOCATION ALPHA #:?TARGET-LOCATION369592) Level 3, state satisfies goal (GROUND-TARGET ALPHA) Level 0, state satisfies goal (UAV 201) Depth 8, applicable method #:OVERWATCH-SORTIE Depth 11, applicable method SINGLE-UAV-OVERWATCH Depth 12, trying method #:OVERWATCH-SORTIE Level 0, state satisfies goal (UAV 101) 28 May

30 Conceptual Planning Process Overwatch Target Latest Start Earliest End Fails for all available UAVs Single-UAV- Overwatch Multiple-UAV- Overwatch Target Latest Start Overwatch- Sortie Single-UAV- Overwatch Sortie-End Earliest End OAV GTMax Dakota OAV Dakota Unavailable in Window Inadequate endurance Can t make start Still unavailable 28 May

31 Resulting Plan Tree 28 May

32 Execution and Monitoring VACS GCS Execution UI 28 May

33 28 May

34 Payoffs and Benefits Heterogeneous UAV commanding in urban terrain with minimal training and workload Complex, multiple UAV commanding via a PDA in under 15 sec. Flexible commanding not just static modes. YIELDING Human in charge of workload vs. control precision tradeoff Platform Independence Reduced training times Faster AND more accurate UAV performance 28 May

35 Conclusions and Future Work Increase number of plays and vehicles Enhance UI interactions Increased interaction with FalconView Pointing and sketching Inspection/Approval/Revision of plans Demonstrate plan upsets and planner- and human-initiated revisions Improve value tradeoff reasoning in planning SBIR Follow on Flight test? 28 May