Hardware & Software Platform for Next Generation Industrial Drones. Chetak Kandaswamy Kai Yan Helmut Prendinger

Transcription

1 Hardware & Software Platform for Next Generation Industrial Drones Chetak Kandaswamy Kai Yan Helmut Prendinger

2 What s next in industry drones? Market: Technical topics: Infrastructure inspection Advanced controller (No GPS, complexed obstacles) Agriculture Disaster Observation Search & Rescue Long endurance and Self-Diagnosis Vision based sensing

3 Infrastructure inspection Under-bridge inspection, No GPS. Using external camera to automatically detect and maintain the position of copter. Small LiDARs on-board for secondary collision avoidance. Wired power supply Developed by enroute Co., Ltd. In-use (Feb., 2016)

4 Long range observation Achieved 67 minutes hovering, with a single 450Wh battery Aero-efficient frame made by TORAYCA T-800S (The same on Boeing-787 Dreamliner) Self-Diagnosis battery pack, warning ahead of failure. developed by Hitach Maxell, Ltd. Developed by enroute Co., Ltd. In-production (May., 2016)

5 Vision based sensing Depth sensing with a single Camera Surrounding sensing with four cameras for 360 degree collision avoidance. Enabled by Jetson TX1 (Implemented in CUDA) Developed by LabRomanec Inc., Developer s kit available soon (

6 Lightweight Drones Drones as service Hobbyist and scientific research - Aerial video capturing - Journalism event capture - Cricket/Football/wedding - Track property - Dangerous place - Extreme sports Surveillance - Aerial reconnaissance - Track endangered animals/poachers - Track solar panels - Agricultural farms - Railway lines - Defense against other drones Deep Transfer Learning Rescue Missions - High range, good cameras - Good samaritans taking care of elderly - Dengue epidemic Delivery Drones - Vaccines to remote locations - Courier in crowded area - Pizzas/dry cleaning - Delivery in dangerous places Security

7 Object Recognition: ImageNet

8 Pixel-wise label: PASCAL VOC

9 FCN-8 Transform Fully connected layers into Convolutional layers Instead of classes, get a heatmap at the output Learnable upsampling to bring output to initial size Refine the output using different layer s predictions Shallow layers : fine scale Deep layers : coarse scale

10 An example : FCN-8s

11 Deep Transfer Learning (DTL) DTL emerged as a new paradigm in machine learning in which, a machine is trained using deep models on a source problem, and then transfer learning to solve a target problem. DTL is an alternative to transfer learning with shallow architectures, in which one specifies a model to several hidden levels of non-linear operations and then estimates the parameters via the likelihood principle. Why DTL? Utilizes the high-level features using Deep Models. Utilizes Transfer Learning method for limited labeled data problems. Overcomes traditional Transfer Learning methods negative feature transfer causing optimization to fall into bad solution space.

12 DTL method 1: Layerwise Transfer Learning Application: Drug-discovery Task : Classification of chemical mechanisms of action (MOA) by identifying substances that alter the phenotype of a cell which prevent tumor growth and metastasis. Classify : Host cell or Tumor cell Challenge : Every day thousands of drugs are tested on millions of samples. Each sample has ~5000 cells leading to billion of cells to check. Capturing the images for analysis takes 6 months at a time. Costing 10,000 Euros for each trail. Cancerous cells of Breast Examples of different MOA captured after compound incubation of Breast Cancer cells. Result of DTL: Transference of weights of the source model obtained positive transference and we observe around 30% computation speed up and improvement in overall efficiency.

13 DTL Method 1 for Drones Unlabeled Images Crowd sourcing Drone Data Deep transfer learning Labeled Images Other Data Unlabeled Images Aerial images: Google map Satellite Non-aerial images: Imagenet Validating the model with the drone data Feature search space Fine-tuning Layerwise Source-Target-Source Ensemble Multi source Implementing existing deep transfer learning methods on Caffee

14 DTL method 2: Multi-Source STS: Cross-sensor Biometrics Recognition Example: In case of periocular images captured rom multiple devices may have different resolution, size, Illumination setting, etc. Practical problems of cross-sensor biometrics is that these data is collected from various devices and often we need to train machine separately for different machines. (Intro-compatibility issues)

15 Result of Multi-Source STS Result : DTL performed ~10 % better than the Deep Learning model.

16 DTL Method 1 and 2 for Drones Input data variations: (Multiple sources) Angles - 45 degree or 90 degree Altitude - High or Low Resolution - High or Low Unlabeled Images Crowd sourcing Drone Data Deep transfer learning Labeled Images Other Data Unlabeled Images Aerial images: Google map Satellite Non-aerial images: Imagenet Validating the model with the drone data Feature search space Fine-tuning Layerwise Source-Target-Source Ensemble Multi source Implementing existing deep transfer learning methods on Caffee

17 Training methods Object detection Training methods Deep Learning ImageNet + Finetuning on drone dataset Source-TargetSource Multi-source Ensemble (Multi-source ensemble) (New method) VGGNet GoogLeNet ResNet Segmentation Inception-v4 FCN-8 SegNet (New model)

18 Initial members: Prendinger Lab. Check out for updates on Deep learning for Drones Jetson TX1 based flight controller Silver/Bronze cloud based mission controller Deep Drone Dataset (D3) Online Annotation Tool