Bio-inspired Active Vision Martin Peniak, Ron Babich, John Tran and Davide Marocco
GPU Computing Lab
Traditional Computer Vision 3
Traditional Computer Vision Teaching a computer to classify objects has proved much harder than was originally anticipated Thomas Serre - Center for Biological and Computational Learning at MIT Specific template or computational representation is required to allow object recognition Must be flexible enough to account with all kinds of variations 4
Biological Vision Researchers have been interested for years in trying to copy biological vision systems, simply because they are so good ~ David Hogg - computer vision expert at Leeds University, UK Highly optimized over millions of years of evolution, developing complex neural structures to represent and process stimuli Superiority of biological vision systems is only partially understood Hardware architecture and the style of computation in nervous systems are fundamentally different 5
Biological Vision 6
Seeing is a way of acting 7
Active Vision Inspired by the vision systems of natural organisms that have been evolving for millions of years In contrast to standard computer vision systems, biological organisms actively interact with the world in order to make sense of it Humans and also other animals do not look at a scene in fixed steadiness. Instead, they actively explore interesting parts of the scene by rapid saccadic movements 8
Creating Active Vision Systems Evolutionary Robotics Approach 9
Evolutionary Robotics New technique for the automatic creation of autonomous robots Inspired by the Darwinian principle of selective reproduction of the fittest Views robots as autonomous artificial organisms that develop their own skills in close interaction with the environment and without human intervention Drawing heavily on biology and ethology, it uses the tools of neural networks, genetic algorithms, dynamic systems, and biomorphic engineering 10
... Genetic Algorithms (GAs) are adaptive heuristic search algorithm premised on the evolutionary ideas of natural selection and genetic. The basic concept of GAs is designed to simulate processes in natural system necessary for evolution. Population (Chromosomes)...... Genetic operators Evaluation (Fitness) Artificial neural networks (ANNs) are very powerful brain-inspired computational models, which have been used in many different areas such as engineering, medicine, finance, and many others. Selection (Mating Pool) 11
Related Research Mars Rover obstacle avoidance (Peniak et al.) 12
Related Research Koala robot obstacle avoidance (Marocco et al.) 13
Related Research Autonomous driving car (Floreano et al.) 14
Related Research Object recognition (Floreano et al.) 15
Going Further Designing active vision system for real-world object recognition 16
Task Design active vision system that can learn to recognize the following objects 17
Method Evolution of the active vision system for real-world object recognition training the system in a parallel manner on multiple objects viewed from many different angles and under different lighting conditions Amsterdam Library of Object Images (ALOI) provides a color image collection of one-thousand small objects recorded for scientific purposes systematically varied viewing angle, illumination angle, and illumination color Active Vision Training trained on a set of objects from the ALOI library each genotype is evaluated during multiple trials with different randomly rotated objects and under varying lighting conditions evolutionary pressure provided by a fitness function that evaluates overall success or failure of the object classification trained on increasingly larger number of objects Active Vision Testing robustness and resiliency of recognition of the dataset generalization to previously unseen instances of the learned objects 18
Experimental Setup Recurrent Neural Network Inputs: 8x8 neurons for retina, 2 neurons for proprioception (x,y pos) No hidden neurons Outputs: 5 object recognition neurons, 2 neurons to move retina (16px max) Genetic Algorithm Generations: 10000 Number of individuals: 100 Number of trials: 36+16 (object rotations + varying lighting conditions) Mutation probability: 10% Reproduction: best 20% of individuals create new population Elitism used (best individual is preserved) 19
Experimental Setup Each individual (neural network) could freely move the retina and read the input from the source image (128x128) for 20 steps At each step, neural network controlled the behavior of the system (retina position) and provide recognition output The recognition output neuron with the highest activation was considered the network s guess about what the object was Fitness function = number of correct answers / number of total steps 20
GPU Accelerating GA and ANN GPUs were used to accelerate: Evolutionary process parallel execution of trials Neural Network parallel calculation of neural activities 21
fitness Results Fitness can not reach 1.0 since it takes few time-steps to recognize an object All objects are correctly classified at the end of the each test 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 generations best fitness average fitness 22
Evolved Behavior 23
Future Work Extending the current system and applying it to the real-world problems requiring fast and energy-efficient pattern recognition Implementing island model GA on GPU AXA Research Fund Active Vision in search and rescue scenarios 24
"Imagination is the highest form of research" Albert Einstein Thank you! 25