Jamison Heard Vanderbilt University PhD Candidate May 11 th Adaptive Robotics for High-Stress Environments

Transcription

1 Jamison Heard Vanderbilt University PhD Candidate May 11 th 2018 Adaptive Robotics for High-Stress Environments

2 Human-Robotic Interaction 2 Mentor Roles Peer Information Consumer

3 High Stress Environments 3 Supervisory Peer

4 Performance Poor Good Workload and Performance 4 Acceptable Level Underload Normal Load Overall Workload Overload

5 Workload Components 5 Overall Workload Cognitive Physical Auditory Visual Speech

6 Objective Workload Metrics 6 Cognitive Brain-Activity Measures Cardiovascular Measures Physical Postural Measures Heart-Rate Skin Temperature Visual Eye-Tracking Measures Speech Speech-Based Measures Respiration-Rate Noise Level Auditory Noise Level Speech-Response Time

7 Challenges with Physiological Signals 7 Age Individual Differences Physical Fitness Training Day-to-Day Variability Circadian Rhythms Stressful Events

8 Workload Assessment Algorithm Development and Validation 8

9 Workload Assessment Algorithm 9

10 Experimental Design 10 Peer-Based Evaluation Supervisory-Based Evaluation

11 Experimental Design 11 Independent Variables Workload Peer Low and High Supervisory Underload, Normal Load, and Overload Peer manipulated teaming partner Human or Robot Dependent Variables Subjective Workload Metrics NASA TLX In-Situ Workload Ratings Objective Workload Metrics Heart-Rate Heart-Rate Variability Skin-Temperature Noise Level Respiration-Rate Posture Magnitude

12 Peer-Based Evaluation Participants 4 Tasks: 1. Photo-Search Hallway Search Liquid and Solid Containment Sampling

13 Supervisory-Based Evaluation Participants 4 Concurrent Tasks: Tracking System Monitoring Resource Management Communications NASA MATB

14 Algorithm Evaluation 14 Four Analysis: Population Generalizability Train on 70% of the Participants, test on the other 30% Cross-Teaming Generalizability Train on Peer Evaluation Data, Test on Supervisory Vice-Versa Cross-Task Generalizability Train on 3 Peer-based Tasks, Test on the 4 th task 4-Fold Cross-Validation Cross-Interaction Generalizability Train on Peer Human-Human Teaming Data, Test on Human-Robot Teaming Vice-Versa

15 Results: Population & Cross-Teaming 15 Average Classification Accuracy (%) by Workload Component and Evaluation SUP PEER BOTH SUP PEER BOTH SUP PEER BOTH Cognitive Workload Physical Workload Overall Workload Peer Evaluation Supervisory Evaluation

16 Results: Cross-Task 16 Average Classification Accuracy (%) by Workload Component and Peer Task T1 T2 T3 T4 T1 T2 T3 T4 T1 T2 T3 T4 Cognitive Workload Physical Workload Overall Workload T# represents peer-task number Peer Evaluation

17 Results: Cross-Interaction 17 Average Classification Accuracy (%) by Workload Component and Interaction Paradigm H-H H-R HH-HR H-H H-R HH-HR H-H H-R HH-HR Cognitive Workload Physical Workload Overall Workload Peer Evaluation H-H: Human-Human, H-R: Human-Robot, HH-HR: Human-Human and Human-Robot

18 Envisioned System 18 Adaptive Workload System Architecture Activity Recognition Future Tasks Current Tasks Workload Models i-cifhar Task (Re-)Allocations Workload Metrics Workload Assessment Algorithm Performance Prediction Workload Component Estimates Predicted Performance Interaction Decision Framework Communication Modality Interaction Changes Assess and Predict Determine Adaptations Apply Adaptations

19 Activity Recognition 19

20 Activity Recognition Background 20 External Sensors Cameras Environmental Wearable Sensors Accelerometer Gyroscope Physiological

21 Current EMS Hand-Off Process 21

22 EMS Procedures 22 Intravenous Therapy Placing an IV Administer IV Medication IO Line Vital Checking Stethoscope Placing Monitoring Equipment High Trauma Airway Management Nasal Airway Oral Airway Intubation Wound and Fractures Tourniquet Combat Gauze Crike Kit Splinting Chest Decompression Cardiopulmonary Resuscitation (CPR)

23 Hierarchical Task Analysis 23 Preparation Check for Breathing Lift Patient s Chin CPR Give Breaths Use Bag-Valve Mask Use Mouth Compressions 30 Chest Compressions

24 Wearable Sensors 24 Apple Watch Accelerometer Gyroscope MYO Accelerometer Gyroscope EMG

25 CPR Accelerometer Data Example 25 Compressions Breath

26 Video: Open Pose 26

27 Open Pose Challenges 27

28 Envisioned System 28 Automatic EMS Procedure Detection Architecture Wearable Sensors OpenPose Activity Recognition Triage Score Generation Heat Map Generation Health Database

29 Recap 29 Workload Assessment Uses objective workload metrics to derive workload estimates for overall workload and its contributing components Analyzed algorithm across populations, human-robotic teaming paradigms, and cross-tasks Needs contextual information about the human s current task Automatic EMS Procedure Detection Uses wearable and visual sensor data Procedures are decomposed into sub-tasks for easier recognition

30 Questions?? 30