Metaheuristics and Cognitive Models for Autonomous Robot Navigation

Metaheuristics and Cognitive Models for Autonomous Robot Navigation Raj Korpan Department of Computer Science The Graduate Center, CUNY Second Exam Presentation April 25, 2017 1 / 31

Autonomous robot navigation (ARN) Mobile robots move through an environment from one location to another without human intervention Two heuristic-based approaches Metaheuristics for path planning Cognitive models of human navigation These approaches Learn about the search space or the environment Balance the tradeoff between exploration and exploitation 2 / 31

Outline Background Challenges in ARN Heuristics and metaheuristics Cognitive science Conclusions 3 / 31

Outline Background Challenges in ARN Heuristics and metaheuristics Cognitive science Conclusions 4 / 31

Background Problem P = S, I, A, G S: set of states I S: set of initial states A: set of possible actions a A G s is a Boolean goal test Problem domain = set of related problems that share some characteristic Path = a finite ordered sequence of interleaved states and actions s 1, a 1, s 2, a 2,, s r 1, a r 1, s r Solution = a path where s 1 I and G s r = True Optimal solution has minimum path cost Satisfactory solution is good enough 5 / 31

Path planning Plan = path proved to be a solution before execution Path planning = search for a plan that minimizes travel time, travel distance, or resource consumption Path quality = value of a path based on domain-specific criteria Goal state Solution Satisfactory solutions Optimal solution? Initial state 6 / 31

Outline Background Challenges in ARN Heuristics and metaheuristics Cognitive science Conclusions 7 / 31

Challenges in ARN (1) Path planning = how do I get to where I need to go? Localization = where am I in the environment? How do I detect my pose x, y, θ despite sensor error? Mapping = what does the environment look like? Obstacle avoidance = how do I get around obstructions in the environment? Motion control = how do I account for error from my actuators? NP-Hard [Canny, 1988] Simultaneous localization and mapping (SLAM) [Durrant, 2006] Reactive and online Handled by hardware Current state-of-the-art for implemented ARN systems on commercial robots = SLAM with A* search Lack of standard testbed and performance metrics no conclusive evidence that any approach is best Since sub-optimal solutions must suffice many other approaches have been investigated 8 / 31

Challenges in ARN (2) Real world vs. simulation Environmental issues = observability, multiagency, dynamism, continuity, and modality [Bartel, 2007] [Zhou, 2014] 9 / 31

Outline Background Challenges in ARN Heuristics and metaheuristics Cognitive science Conclusions 10 / 31

Heuristics and metaheuristics Heuristic = efficient strategy that can often solve problems, typically specific to a problem domain [Pearl, 1984] Metaheuristic = a broadly applicable and non-domain specific technique that uses a heuristic strategy to obtain satisfactory solutions [Glover, 2006] Hybrid metaheuristic = 2 metaheuristics or metaheuristic + another method [Blum, 2011] No guarantee that an optimal solution will be found in finite time satisfactory solutions suffice in path planning for ARN Candidate = path found during search Neighbor of a candidate = path with one change to the candidate s action sequence changed added removed 11 / 31

Taxonomy of search methods Single solution Number of candidates considered Population Local Hill climbing (HC) Steepest-ascent HC Simulated annealing Tabu search Swarm algorithms Other Locality of search Both Heuristic-based methods Metaheuristics Evolutionary algorithms Ant colony optimization Artificial bee colony Firefly algorithm Global Random search HC with random restarts Genetic algorithm Genetic programming Evolutionary programming Differential evolution Biogeography-based optimization Particle swarm optimization Bat algorithm Cuckoo search algorithm [Talbi, 2009] Gravitational search Memetic algorithm metaheuristics Harmony search 12 / 31

Heuristic-based methods Random search Pure exploration Hill climbing Pure exploitation of learned knowledge Hill climbing with random restarts Random exploration with periods of exploitation Hill climbing with steepest ascent Exploitation of learned knowledge with limited exploration around candidate Fast but highly susceptible to premature convergence to local optima 14 / 31

Single-solution metaheuristics Simulated annealing improves hill climbing [Kirkpatrick, 1983; Černỳ, 1985] Occasionally selects candidates with lower quality Changes the balance of exploration and exploitation by reducing the probability of selecting a lower quality candidates over time Tabu search also selects lower quality candidates [Glover, 1989; 1990] Prevents return to recently visited candidates Intentionally limits exploitation to increase exploration Advantages: efficient and can control memory requirements Disadvantages: highly dependent on initialization and parameter settings, may be slow Applications to path planning are rare and simulated in simplistic, unrealistic environments 16 / 31

Evolutionary algorithms (1) Inspired by Darwinian principles Genetic algorithm [Manikas, 2007] Evolves a population of candidates with reproduction, crossover, mutation, and selection Fitness function evaluates quality of candidates Genetic programming = candidates are computer programs [Koza, 1992] Evolutionary programming [Fogel, 1999] Candidates are value assignments to parameters of a single program The next generation is composed of the best candidates selected from among the parents and children Differential evolution [Storn, 1997] Candidates are vectors of real numbers that are function parameters Recombination incorporates a third population member during reproduction 18 / 31

Evolutionary algorithms (2) Applications to path planning Single robot and unmanned aerial vehicle (UAV) Multi-robot and multi-uav Multi-objective Addition of domain knowledge Advantages: incorporates exploitation/hill-climbing through selection, and exploration/randomization through mutation Disadvantages: Parameters must be hand tuned Can be computationally and memory intensive 19 / 31

Swarm algorithms Simulates crowd behavior of organisms Ant colony optimization [Dorigo, 2006] Indirect communication when foraging Individual ants use local search Pheromones indicate candidate desirability Artificial bee colony [Karaboga, 2007] Three groups: employed bees, onlookers, and scouts Combines local search and global search Particle swarm optimization [Kennedy, 1995] Birds in search of food as particles that move through search space Direction of each particle s local search is influenced by other particles Other approaches: firefly algorithm, bat algorithm, cuckoo search 21 / 31

Swarm algorithms comparison Algorithm Individuals Type of search Communication mechanism Ant colony optimization Artificial bee colony Particle swarm optimization Firefly algorithm ants hill climbing pheromone strength bees hill climbing and random search dance intensity particles hill climbing velocity of global best position fireflies hill climbing flashing brightness and distance Bat algorithm bats hill climbing and random search Cuckoo search cuckoos random search none echolocation frequency, loudness, and rate of emissions 22 / 31

Path planning with swarm algorithms Applications Single robot, UAV, and underwater Multi-robot and multi-uav Multi-objective Addition of domain knowledge Dynamic obstacles Advantages: communication mechanisms allow local information to influence entire population Disadvantages: Parameters must be hand tuned Can be computationally and memory intensive 23 / 31

Hybrid metaheuristics Categorized by Type of hybrid: dual (2 metaheuristics) or metaheuristic + another method Approach to ARN: blend components into a new system or use them separately for different tasks Applications to path planning Mostly in static, simulated environments Dual hybrids typically evolutionary algorithm + swarm algorithm Other methods include probabilistic roadmap, chaotic search, fuzzy logic, and artificial potential field Advantages: tries to resolve shortcomings of individual methods by combining multiple methods Disadvantages: Difficult to compare approaches Parameters must be hand tuned Increased computational complexity 24 / 31

Outline Background Challenges in ARN Heuristics and metaheuristics Cognitive science Conclusions 25 / 31

Cognitive science Multidisciplinary field that studies the mind and intelligence [Friedenberg, 2011] Spatial cognition = subfield of cognitive science that studies navigation and wayfinding Decision making = agent selects among a set of alternatives Reasoning = agent draws a conclusion from information to solve a problem or make a decision People exhibit goal-directed behavior People use heuristics to make decisions, especially when faced with limited time, knowledge, or computational power [Gigerenzer, 1999] People use case-based reasoning People employ hierarchical organization of spatial memory [Wiener, 2003] 26 / 31

Human navigation Strategies for human navigation What? Spatial knowledge: visual cues (landmarks), frame of reference (egocentric vs allocentric), spatial orientation, path integration, route knowledge, survey knowledge Internal representation: cognitive map [Golledge, 1999] Other knowledge: external information, previous knowledge and experience Who? Demographics: age and gender Spatial abilities: sense of direction Where? Simulated: virtual environments Real world: indoor and outdoor environments When? Why? How? How much? Before navigation: path planning, search and selection Adapted to overall goal or current task [Holscher, 2011] Sense: proprioception, vision and hearing Cognitive economy During navigation: following a plan, or decision making without a plan Adapted to overall environment or current state Decision and action: brain activity and muscle activation 27 / 31

Cognitive models Cognitive models simulate observed human behavior with a computational system or algorithm Application of cognitive models to ARN Build a hierarchical representation of the environment similar to a cognitive map [Kuipers, 2000] Represent spatial knowledge with a graph Shift between navigation strategies using a heuristic Bayesian approach + artificial neural network to learn topological maps and landmarks [Thrun, 1998] Learn abstract representations of the environment and use multiple heuristics to make decisions [Epstein, 2015] Advantages: exploit human knowledge and strategies Disadvantages: No one model has simulated all observed human wayfinding behavior or employed all their strategies Difficult to account for all individual differences among people 28 / 31

Outline Background Challenges in ARN Heuristics and metaheuristics Cognitive science Conclusions 29 / 31

Conclusions Most approaches have been evaluated in simplified simulated environments that underestimate the hardware challenges of physical robots and reduce computational complexity But people successfully navigate in the real world already so there is room for improvement inspired by human behavior Ultimately, autonomous robot navigation systems must operate in the real world contend with observability, multiagency, dynamism, continuity, and modality issues Future work should address these challenges in a real-world environment Potential future work could create more robust ARN systems by a synergy between metaheuristics and cognitive models 30 / 31

Thank You 31 / 31