Getting to the Finals of the DARPA Urban Challenge

Transcription

1 Getting to the Finals of the DARPA Urban Challenge 19 May 2008 Team UCF University of Central Florida College of Engineering & Computer Science Dr. Yiannis Papelis VMASC Old Dominion University Benjamin Patz President/CEO Coleman Technologies, Inc.

2 Overall objective of the 2007 Challenge was 60 miles / 6 hrs of Urban Driving Robot (Bot) version of city driving Safely drive city streets Safely pass through intersections Obey precedence rules at stop signs Avoid in-road obstacles Follow other vehicles Park Perform u-turn Reroute due to blockages While obeying CA traffic laws At moderate speed Up to 30 mph In Traffic Multiple autonomous vehicles Multiple human operated vehicles 2

3 Our Interpretation and How We Prioritized High Level Goals Based on a-priori (RNDF) and real-time (sensor) information define an environment; decompose a- priori objectives (MDF) into a set of sub-missions to be executed by the planner within that environment in such a manner as they meet overall objectives Safety was of primary importance Protect our vehicle collision, kinematic limits Protect other vehicles Protect obstacles Completion was of secondary importance Meet objectives Legality was of tertiary importance Following DUC rules Speed was least important 3

4 Team UCF s Initial Plan for a Path to the NQE Team UCF built upon previous 2005 Grand Challenge participation Same car 1996 Subaru Outback Enhanced sensors Initially planned multiple laser scanners, multiple video short range acoustic Integrated GPS/INS from Oxford Enhanced processors Initially planned 7-8 dual core CPUs (ended up using 3) ICE for inter-process / processor communication Enhanced team Don Harper, Remo Pillat, Gary Stein (2005 participants) Faculty technical leads Faculty project management Industry Partner (CTI) in advisory role Keep costs < $150K, including student reimbursement and travel We were a Track B (unfunded) participant 4

5 Our Team was Small We Focused on Meeting Requirements Safely drive city streets Safely pass through intersections Obey precedence rules at stop signs Avoid in-road obstacles Follow other vehicles Park Perform u-turn Reroute due to blockages (ID2,X2,Y2,S2,H2) (ID2,X2,Y2,0,H1) <= Dmin (~15m) (ID1,X1,Y1,S1,H1) (ID1,X1,Y1,S1,H1) (ID1,X1,Y1,S1,H1) (ID1,X1,Y1,S1,H1) (OID,X,Y,S,H) Human Driver (ID1,X1,Y1,S1,H1) (ID3,X1,Y1,-S1,H1) (ID0,X0,Y0,S0,H0) (ID0,X0,Y0,S0,H0) (ID2,X2,Y2,S2,180+H0) (ID0,X0,Y0,S0,H0) (ID0,X0,Y0,S0,H0) (ID0,X0,Y0,S0,H0) (ID4,X4,Y4,S4,180+H0) Normal Driving Parking 3-pt Turn Avoiding Following Starting Bot (ID1,X1,Y1,S1,H1) (ID1,X1,Y1,S1,H1) (ID0,X0,Y0,S0,H0) (ID0,X0,Y0,S0,H0) Merging Stop Sign 5

6 6 Team UCF System Block Diagram V5 Monoc Camera Monoc Camera Laser Scanner Laser Scanner Path Planning GPS/ INS Object Detection Monoc Camera Autopilot Steering Servo Brake Servo Accel/ Gear Servo Vehicle Kinematics Laser Scanner Doppler Radar Motion Detection Lane Detection Road Network Parser Mission Parser Context Based Tactical Reasoning Virtual Environment Model Environmental Model Emergency Stop Turn Signal RNDF MDF Intelligence Vision Vehicle E-Stop Rcvr Strategic Planning Fusion Navigation E-Stop Planning Monitoring Pan/ Tilt

7 Primary Sensor Systems were Adjusted During Development SICK LMS 291 Laser Scanners 3 forward mounted 1 rear mounted 2 top mounted rotating 1 top mounted lane/curb Stalker Devantech Doppler Acoustic Radar Sensors Actuated (one-axis) Speed 8 side only mounted Sony HDR-HC3 Camcorder 1 top mounted 7

8 Resulting Sensor Systems on the Bot GPS Laser Scanners Camera Doppler Radar 8

9 Sensor System Laser Scanners The system uses inexpensive 2D laser scanners (~ $3K) A pulsed laser beam is emitted and reflected if it meets an object A fan-shaped scan is generated by an internal rotating mirror Distance is derived from time-of-flight z-axis rotation => 3-D scanning 3-D point cloud X Z 2D LADAR Shaft / Axis of Rotation for LADAR Roll 16-bit absolute encoder Two laser scanners are mounted on top of KnightRider Y Rotational angle is tracked by 16- bit optical encoder mapping into 3D coordinate frame facilitated Approach was unique to TeamUCF Adjustment for LADAR Pitch Adjustment for LADAR Yaw Alternative 3-D scanner $$$$ 9

10 Rotating Scanners Eventually Cover 3D Volume Absolute Perpendicular Deviation (in meters) maximum deviation of ca. 20 cm highest error rates in turns at high speeds (fast changing heading) 10

11 Rotating Scanners Eventually Cover 3D Volume Absolute Perpendicular Deviation (in meters) maximum deviation of ca. 20 cm highest error rates in turns at high speeds (fast changing heading) 11

12 Rotating Scanners Eventually Cover 3D Volume Absolute Perpendicular Deviation (in meters) maximum deviation of ca. 20 cm highest error rates in turns at high speeds (fast changing heading) 12

13 Rotating Scanners Eventually Cover 3D Volume Absolute Perpendicular Deviation (in meters) maximum deviation of ca. 20 cm highest error rates in turns at high speeds (fast changing heading) 13

14 Top Laser Scanner Sees Lanes and Road Markings r dot r = r r + r i+ 1 i 1 i+ 1 i 1 Discontinuities tracked via relatively simple gradient Lane centerline inferred from lane boundaries Based on observed/expected lane width Lane centerline tracked via Kalman filter Result points between RNDF datums 14

15 Sensor Fusion Combined All Data Into a Single World View Task: Intelligently fuse data from 7 laser scanners, Doppler Radar, GPS, and Camera; Make the least amount of mistakes. Method: Lane Tracking: Assign confidence values to extracted lanes from vision and lasers; disregard uncertain values Near-Field Obstacles: Use probabilistic Occupancy Grid to merge data from 7 scanners; extract polygonal obstacles; track their position and velocity Far-Field Obstacles: Use heuristic to fuse Doppler and laser scanner information if object approaching 15

16 AI and Path Planner Context-Based Reasoning Approach Road Network Mission Definition File Discovery Mission Planning Core AI Re-plan trigger Tactical Plan to PP N Enable Entry Y Hierarchical State Machine Exit Sensor Mode Feasibility Approach Park Backup Sensors Drive Central Controlling Mechanism Inputs: current plan, sensor data, path feasibility Output: tactical-level path, mode information Tactical plan: list of waypoints & velocity, obstacles & action Mode information: sensor mode, re-routing, new road data Execution Semantics Periodic execution pseudo real-time Autonomous Driver Model Upper level: prioritized context-based modes Each mode: specific state machine Lower level: hierarchical state machines Nested SM, each state containing sub-states Forw Stop Context Essence: { HSM, EF, Entry, Exit } A hierarchical state machine focused on a sub-set of the driving problem An enabling function if true, context is willing to solve the short term problem Entry called when context first takes over Exit called when context looses control Rev Wait 16

17 AI and Path Planner Context-Based Reasoning Approach Mission Definition File Mission Planning Backup Road Network Discovery Core AI Re-plan trigger Tactical Plan to PP Stop Sign Other Intersection Sensor Mode Sensors Feasibility Central Controlling Mechanism Inputs: current plan, sensor data, path feasibility Output: tactical-level path, mode information Tactical plan: list of waypoints & velocity, obstacles & action Mode information: sensor mode, re-routing, new road data Execution Semantics Periodic execution pseudo real-time Autonomous Driver Model Upper level: prioritized context-based modes Each mode: specific state machine Lower level: hierarchical state machines Nested SM, each state containing sub-states Higher Priority UTurn Zone Road Lane Change Stop at Waypt Default 17

18 Context Execution Mechanics N Enable Entry Y Hierarchical State Machine Exit For each context in priority order call enabling function if true if old active context different call Exit of old context call Entry of this context endif call HCM endif endfor N Enable Entry Y N Hierarchical State Machine Enable Exit Entry Y Hierarchical State Machine Exit 18

19 HSM Zone Driving Example Park Pause Approach Park Backup Done Stop Drive Enter Move Stop Done Back-off Exit Start Back-up Pt Back-up Pt Stop Pt 30 deg End End 19

20 Testing Included Simulation and Limited Field Testing Matlab for subsystem characterization and early modeling Custom Simulation (Vevis) for all functional testing Simulation interacted with operational code via ICE UCF Parking Deck A ~350m navigation concrete light poles random obstacles 4-way stop Passing stopped obstacles on straights and corners Angle (deg) Vehicle Heading Steering Command Steering Angle Time (s) 20

21 Testing Included Simulation and Limited Field Testing Matlab for subsystem characterization and early modeling Custom Simulation (Vevis) for all functional testing Simulation interacted with operational code via ICE UCF Parking Deck A ~350m navigation concrete light poles random obstacles 4-way stop Passing stopped obstacles on straights and corners 21

22 National Qualifying Event (NQE) Victorville, CA Merge and Crossing Traffic Test Driving, Parking, and Obstacle Avoidance Stop Sign and Route Replan 22

23 Test Area B Driving, Parking and Obstacle Avoidance Basic mission was to drive onto a moderately complex urban road network... and return in less than 30 min The first time we let the vehicle out of our sight ever with enough stop signs to make things interesting Overall course was about 6.5km in length and a liberal sprinkling of obstacles We were a little stressed with a parking maneuver Test B Video until we saw our bot return 23

24 The Final Event A series of 3 ~2hr missions demonstrating all of the elements previously tested plus Bot on Bot traffic Off-road performance High speed parallel lanes Our bot failed ~2hr into the mission GPS failure we had seen before on Course A and misdiagnosed GPS reported data valid data was in fact NaN passed our data validity tests 24

25 Some Specific Observations SICK laser scanners are highly reliable and capable Usable range <40m reliable range <30m Data rate and range are suitable for speeds up to 20 mph (10 m/s) Both curb (discontinuity) and lane (intensity) tracking work well Upside down mounting requirement is a myth Oxford GPS / INS is highly capable 100Hz data stream Don t believe the validity flag Vision is problematic Lighting conditions are all over the board Difficult to model in simulation Internet Communications Engine (ICE) works well for inter-process communication Easily supports hardware in the loop SVN is a wonderful software management tool versions External tools are required to measure and analyze real-time timing Factor data latency into simulation Test data at every interface NaN fails every test 25

26 Lessons Know your Bot Know what the bot will do in any situation Know your Team Leverage your strengths, hide your weaknesses Solve the Problem We viewed this NOT as a research effort, but as a demonstration Clearly articulate each problem to be solved Identify roles and responsibilities Control the Software 3 rd party software is inherently inflexible Test / Test / Test Verify each subsystem, independently Inject faults Match simulation to vehicle performance (and vice versa) Drive / Drive / Drive Don t ever give up 26

27 Q & A 27

28 Team Pit Areas not exactly NASCAR ready Everything brought from Orlando via trailer We supplied power via generator (24x7) Local wireless network High gain antenna allowed us to get Internet off DARPA trailers Weather was generally wonderful Except for 40 mph winds Microscopic dust Freezing nights, hot days, and, yes, even rain 28

29 Test Area A Merge and Follow Testing Basic mission was to perform as many loops of this simple course as possible in 30 minutes Our car is the 1996 Subaru way back here crossing traffic here By the way this concrete barrier is on the edge of the course without hitting any of these and merging here ~100 m Test A Video Other Contestants 29

30 Test Area C Stop Sign and Reroute Behavior Desired behavior was to detect blockage, perform 3-pt turn and reroute Of course the map does not quite align with the GPS waypoints waypoints are truth Basic mission was to perform multiple loops of the belt-buckle Complicated and a windshield by two 4- way high stops stop with sign various bar traffic conditions On last loop a blockage was injected here 4 barrels Objective is to successfully determine precedence at stop signs Test C Video 30

31 Some Highlights Alternator Failed 3AM repair, before 7AM run Laser Scanner Power Cable Disconnected During First Failed Site B Run Proof that fix to pick up blocked checkpoint was going to work DARPA staff can t get enough or sleep Team has a little fun on Halloween KnightRider can fly 31

32 A-Priori Information RNDF information is a collection of NS segments each with NL lanes each with M waypoints in lan,lon coordinates AND a collection of NZ zones Waypoint order implies direction on lanes Zones are basically free driving areas Certain waypoints are checkpoints Certain waypoints are defined entry or exit points Certain waypoints are defined stop signs MDF defines speed limits for segments and zones and checkpoints to be met Implied Lane Width (ID,Lat,Lon) For each Lane: -Width - Speed Limits - Boundaries Double_Yellow Solid_White Broken_White 32

33 Team UCF Refocused in December 2007 Industry Lead Ben Patz Lead Engineer Don Harper AI Dr. Yiannis Papelis Sensors Remo Pillat (Graduate Student) Lasers determined to be primary sensor source Vision David Lyle (Graduate Student), Andrew Miller (Undergraduate Student) Lane tracking only Other (and all) software Gary Stein (Graduate Student) At this point still had hope for acoustic system or enhanced power system 33