Vision Based Autonomous Control of a Quadcopter

Transcription

1 Department of Electrical and Computer Engineering Vision Based Autonomous Control of a Quadcopter By Zack Woods, Jeff Deeds, Zach Engstrom, and Caleb Gill Advised by Drs. Y. Lu, J. Wang, and I. S. Ahn May 2, 2017

2 Agenda Motivation Objectives Previous Work System Design and Implementation Results and Discussion Project Management Conclusion and Future Work 2

3 Motivation 3

4 Agenda Motivation Objectives Previous Work System Design and Implementation Results and Discussion Project Management Conclusion and Future Work 4

5 Objectives This project accomplishes a simple set of instructions to test initial functionality and form a basis for future endeavors. Autonomously control a quadcopter to execute a mission using visual feedback. 5

6 Step 1 - Take off 6

7 Step 2 - Navigate to GPS Coordinate 7

8 Step 3 - Search and Detect Target 8

9 Step 4 - Center Over Target 9

10 Step 5 - Land Near Target 10

11 Agenda Motivation Objectives Previous Work System Design and Implementation Results and Discussion Project Management Conclusion and Future Work 11

12 Previous Work - Colored Target Project Research project from University of Queensland, Brisbane (2016) Color detection vision control: Red target square OpenCV and MAVProxy 12

13 Previous Work - AprilTag Visual fiducial system developed at the University of Michigan Similar to ARtags Uses 9 step detection process Benefits Detectable at high skew Binary image for computation efficiency Low/high light performance Calculates distance and orientation based on the size of the AprilTags 13

14 Previous Work - AprilTag Step 1: Grayscale and Blur Step 2: Local Gradient Step 3: Edge Detection 14

15 Previous Work - AprilTag Step 4: Edge Thinning Step 5: Line Segments Step 6: Lines Connected 15

16 Previous Work - AprilTag Step 7: Quad Detection Step 8: Quads Decoded Step 9: Refine Detections 16

17 Agenda Motivation Objectives Previous Work System Design and Implementation Results and Discussion Project Management Conclusion and Future Work 17

18 System Specifications - Quadcopter Quadcopter: 3D Robotics Iris+ Max Payload: 0.8 lbs Payload used: 0.17 lbs Controller Range: 300 meters Battery Capacity: 5100mAh Flight Time (Current Load): 10 minutes 18

19 System Specifications - Peripherals Peripherals GPS Module: ublox with integrated magnetometer Camera: 8 Megapixels Pi camera V2 Telemetry/Radio Control Frequency: 915 MHz GPS RC/Telemetry Camera 19

20 System Specifications - Onboard Autopilot Onboard Autopilot: PX4 (Pixhawk) CPU: 168 MHz Cortex M4F RAM: 256 KB Flash Memory: 2 MB Sensors: 3D Accelerometer Gyroscope Magnetometer Barometer Ports: 5x UARTs CAN I2C SPI ADC 20

21 System Specifications - Embedded System Embedded System: Raspberry Pi 3 SoC: Broadcom BCM2837 CPU: Quad Core ARM Cortex-A53, 1.2GHz GPU: Broadcom VideoCore IV RAM: 1GB LPDDR2 (900 MHz) Networking: 10/100 Ethernet, 2.4GHz n wireless Bluetooth: Bluetooth 4.1 Classic, Bluetooth Low Energy Storage: microsd GPIO: 40-pin header, populated Ports: HDMI 3.5mm analogue audio-video jack 4 USB 2.0 Ethernet Camera Serial Interface (CSI) Display Serial Interface (DSI) 21

22 System Connections Use wireless ssh connection to program the Raspberry Pi 3 Camera is connected to the Raspberry Pi 3 through the Camera Serial Interface Raspberry Pi 3 GPIO pins 1,3,4,5 connected to Telem 2 port on Pixhawk CSI 2 22

23 System Block Diagram PixHawk: An industry-standard advanced autopilot MAVLink (Micro Air Vehicle Link): A protocol for communicating with a small unmanned vehicle 23

24 Subsystems - AprilTag Detector AprilTag Detection Request AprilTag Detector Video Feed AprilTag s Position and Orientation Receives detection request from Control Algorithm. Receives a video feed from a camera that is external to the quadcopter. Uses the video feed to detect the desired AprilTag and determine its position and orientation, with respect to the camera s position and orientation. Sends the AprilTag s position and orientation to the control algorithm subsystem. 24

25 Subsystems - Control Algorithm AprilTag s Position and Orientation Quadcopter s Position and Heading Mission Commands AprilTag Detection Request Control Algorithm Desired Movement Commands Receives mission commands from the ground station. Sends detection request to AprilTag detector subsystem. Receives position and orientation of a detected AprilTag from the AprilTag detector subsystem. Receives the quadcopter s position and heading from the Pixhawk. Uses mission commands, AprilTag position and orientation, and quadcopter s position and heading to determine commands needed for moving to a desired location. Sends desired movement commands to the MavLink command encoder subsystem. 25

26 Subsystems - Control Algorithm 26

27 Subsystems - Control Algorithm 27

28 Subsystems - Control Algorithm 28

29 Subsystems - Control Algorithm 29

30 Subsystems - Control Algorithm 30

31 Subsystems - Control Algorithm 31

32 Subsystems - Control Algorithm 32

33 Subsystems - MAVLink Command Encoder Desired Movement Commands MavLink Command Encoder Movement Commands Receives desired movement commands from the control algorithm subsystem. Converts the desired movement commands into MAVLink commands. Sends converted desired movement commands to the Pixhawk. Handles serial I/O to and from the airborne embedded system. 33

34 Subsystems - MAVLink Command Encoder MAVLink is a protocol for communicating with a small unmanned vehicle. Encoder used is DroneKit-Python which is a Python API Each MAVLink Command has bytes 6 bytes of header bytes of payload 2 bytes of checksum Transmitted at a baud rate of 57,600 bits/s 34

35 Interprocess Communication Vision Detection and Control Algorithm subsystems run in concurrent processes Project has library dependencies in both C++ and Python Client-Server Communication Model Vision Detection C++ application is Server Control Algorithm Python script is Client 35

36 Agenda Motivation Objectives Previous Work System Design and Implementation Results and Discussion Project Management Conclusion and Future Work 36

37 AprilTag Detector Utilize C++ AprilTag library Library Functionality Video4Linux Camera Setup and Initialization Captures and processes frames from video feed Report information on tag detections of target family ID Position Orientation Processing Time 37

38 AprilTag Detector - Testing Detection Delay vs Resolution Resolution = 320 x 240, Detection Delay = 0.62 sec Resolution = 640 x 480, Detection Delay = 2.04 sec 38

39 AprilTag Detector - Testing Detection Reliability vs. AprilTag Family AprilTag Family = 36h11 39

40 AprilTag Detector - Testing Detection Reliability vs. AprilTag Family AprilTag Family = 25h9 40

41 AprilTag Detector - Specifications Resolution (pixel per frame) : 640 x 480 Detection Delay : 2.04 s Max Detectable Distance : 8 m Hovering Distance : 5 m AprilTag Family : 25h9 Field of View : 62.2 x 48.8 ( 6 m x 4.5 m ) 41

42 Quadcopter - Simpler Mission 42

43 Quadcopter - Simpler Mission Take off to an altitude of 5 meters. Land at a location 8 meters north and 5 meters west of the launch site. 43

44 Interprocess Communication Python Client C++ Server Integration for AprilTag Detection and Control Algorithm subsystems Client-server communication model using Linux named pipes Two handled requests CHECK - Verify IPC functionality GET - Request tag detection 44

45 Quadcopter - Objective Mission 45

46 Quadcopter - Objective Mission 46

47 Agenda Motivation Objectives Previous Work System Design and Implementation Results and Discussion Project Management Conclusion and Future Work 47

48 Project Schedule and Division of Labor 48

49 Bill of Materials 49

50 Agenda Motivation Objectives Previous Work System Design and Implementation Results and Discussion Project Management Conclusion and Future Work 50

51 Conclusion Able to successfully execute defined objectives Work that is applicable to all PX4 Autopilot platforms AprilTag Detector DroneKit-Python Control Algorithm Recommendations for improvement: New quadcopter platform with a better thrust to weight ratio 51

52 Future Work Obstacle avoidance algorithms to operate in more complex environments (extra sensors would be required). Search algorithms to cover a larger viewing area. Multiagent configuration for swarm intelligence. 52

53 References H. Choi, M. Geeves, B. Alsalam, and L. F. Gonzalez, Open Source Computer-Vision Based Guidance System for UAVs On-Board Decision Making, in Australian Research Centre for Aerospace Automation; School of Electrical Engineering & Computer Science; Institute for Future Environments; Science & Engineering Faculty, Yellowstone Conference Center, Big Sky, Montana, E. Olson, AprilTag: A Robust and Flexible Visual Fiducial System, in 2011 IEEE International Conference on Robotics and Automation (ICRA), 2011, pp L. Meier, D. Honegger, and EagleUp, Home - Pixhawk Flight Controller Hardware Project. [Online]. Available: [Accessed: 02-May-2017]. Raspberry Pi 3 is out now! Specs, benchmarks & more, The MagPi Magazine, 29-Feb First Time Setup Copter Documentation. [Online]. Available: [Accessed: 30-Nov-2016]. MAVLink Micro Air Vehicle Communication Protocol - QGroundControl GCS. [Online]. Available: [Accessed: 30-Nov-2016]. Iris Operation Manual, V2. (2013). 3DR Robotics. 53

54 References - Photos Anthony, 3DR IRIS+ review : many good and only few bad sides, DronesGlobe.com, 19-Sep [Online]. Available: [Accessed: 02-May-2017]. D. Kriege, Obsession Telescopes / Accessories / ServoCAT. [Online]. Available: [Accessed: 02-May-2017]. Aditi, 15 Best Drones for Aerial Photography, Dronomag, 17-Jun Panoptes Systems Introduces ebumper4 Drone Obstacle Avoidance System, Unmanned Systems Technology, 19-Mar B. C. Life, Drones and Condo Associations: Welcome Your Tiny Overlords, Better Condo Life, 21-Jan China has a flamethrower drone, and an awesome use for it, Daily Trendsetter, 24-Feb [Online]. Available: [Accessed: 02-May-2017]. Pizza delivery in the drone, Shutterstock.com. [Online]. Available: [Accessed: 02-May-2017]. Search and Rescue, an Application of Drones, Trackimo, 28-May New Law Allows Weaponized Police Drones, Exposing The Truth, 28-Aug [Online]. Available: [Accessed: 02-May-2017]. 54

55 55