NAVAL POSTGRADUATE SCHOOL THESIS

Transcription

1 NAVAL POSTGRADUATE SCHOOL MONTEREY, CALIFORNIA THESIS SWARM INTELLIGENCE FOR AUTONOMOUS UAV CONTROL by Natalie R. Frantz June 2005 Thesis Advisor: Second Reader: Phillip E. Pace David C. Jenn Approved for public release; distribution is unlimited

2 THIS PAGE INTENTIONALLY LEFT BLANK

3 REPORT DOCUMENTATION PAGE Form Approved OMB No Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA , and to the Office of Management and Budget, Paperwork Reduction Project ( ) Washington DC AGENCY USE ONLY (Leave blank) 2. REPORT DATE June TITLE AND SUBTITLE: Swarm Intelligence for Autonomous UAV Control 6. AUTHOR(S) Natalie R. Frantz 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Naval Postgraduate School Monterey, CA SPONSORING /MONITORING AGENCY NAME(S) AND ADDRESS(ES) N/A 3. REPORT TYPE AND DATES COVERED Master s Thesis 5. FUNDING NUMBERS 8. PERFORMING ORGANIZATION REPORT NUMBER 10. SPONSORING/MONITORING AGENCY REPORT NUMBER 11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government. 12a. DISTRIBUTION / AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE Approved for public release; distribution is unlimited 13. ABSTRACT (maximum 200 words) Unmanned Aerial Vehicles (UAVs) are becoming vital warfare platforms because they significantly reduce the risk of human life while accomplishing important missions. A UAV can be used for example, as stand-in sensor for the detection of mobile, low-probability-of-intercept battlefield surveillance and fire control emitters. With many UAVs acting together as a swarm, the location and frequency characteristics of each emitter can be accurately determined to continuously provide complete battlefield awareness. The swarm should be able to act autonomously while searching for targets and relaying the information to all swarm members. In this thesis, two methods of autonomous control of a UAV swarm were investigated. The first method investigated was the Particle Swarm Optimization (PSO) algorithm. This technique uses a non-linear approach to minimize the error between the location of each particle and the target by accelerating particles through the search space until the target is found. When applied to a swarm of UAVs, the PSO algorithm did not produce the desired performance results. The second method used a linear algorithm to determine the correct heading and maneuver the swarm toward the target at a constant velocity. This thesis shows that the second approach is more practical to a UAV swarm. New results are shown to demonstrate the application of the algorithm to the swarm movement. 14. SUBJECT TERMS Autonomous Behaviors, Unmanned Aerial Vehicles (UAVs), Particle Swarm Optimization (PSO) 17. SECURITY CLASSIFICATION OF REPORT Unclassified 18. SECURITY CLASSIFICATION OF THIS PAGE Unclassified 19. SECURITY CLASSIFICATION OF ABSTRACT Unclassified 15. NUMBER OF PAGES PRICE CODE 20. LIMITATION OF ABSTRACT NSN Standard Form 298 (Rev. 2-89) Prescribed by ANSI Std UL i

4 THIS PAGE INTENTIONALLY LEFT BLANK ii

5 Approved for public release; distribution is unlimited SWARM INTELLIGENCE FOR AUTONOMOUS UAV CONTROL Natalie R. Frantz Ensign, United States Navy B.S., United States Naval Academy, 2004 Submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE IN ELECTRICAL ENGINEERING from the NAVAL POSTGRADUATE SCHOOL June 2005 Author: Natalie R. Frantz Approved by: Phillip E. Pace Thesis Advisor David C. Jenn Second Reader John P. Powers Chairman, Department of Electrical and Computer Engineering iii

6 THIS PAGE INTENTIONALLY LEFT BLANK iv

7 ABSTRACT Unmanned Aerial Vehicles (UAVs) are becoming vital warfare platforms because they significantly reduce the risk of human life while accomplishing important missions. A UAV can be used for example, as stand-in sensor for the detection of mobile, lowprobability-of-intercept battlefield surveillance and fire control emitters. With many UAVs acting together as a swarm, the location and frequency characteristics of each emitter can be accurately determined to continuously provide complete battlefield awareness. The swarm should be able to act autonomously while searching for targets and relaying the information to all swarm members. In this thesis, two methods of autonomous control of a UAV swarm were investigated. The first method investigated was the Particle Swarm Optimization (PSO) algorithm. This technique uses a non-linear approach to minimize the error between the location of each particle and the target by accelerating particles through the search space until the target is found. When applied to a swarm of UAVs, the PSO algorithm did not produce the desired performance results. The second method used a linear algorithm to determine the correct heading and maneuver the swarm toward the target at a constant velocity. This thesis shows that the second approach is more practical to a UAV swarm. New results are shown to demonstrate the application of the algorithm to the swarm movement. v

8 THIS PAGE INTENTIONALLY LEFT BLANK vi

9 TABLE OF CONTENTS I. INTRODUCTION...1 A. SWARM OF UAVS...1 B. PRINCIPAL CONTRIBUTIONS...2 C. THESIS OUTLINE...2 II. BACKGROUND...5 A. SWARMS Origin Self-Organization...5 a. Positive and Negative Feedback...5 b. Randomness Swarm Intelligence...6 B. SWARM ALGORITHMS Ant Colony Optimization Evolutionary Computation...8 a. Genetic Algorithm...9 b. Evolutionary Algorithm...10 C. NEURAL NETWORKS Basics...11 a. Single-Layer Perceptrons...13 b. Multi-layer Perceptrons...15 c. Exclusive OR (XOR) Problem Backpropagation...18 a. Forward Path...19 b. Backward Path...20 D. BACKGROUND CHAPTER SUMMARY...21 III. PARTICLE SWARM OPTIMIZATION ALGORITHM...23 A. THEORY Flocks Parameters...26 B. RECENT RESULTS Comparison to Backpropagation PSO Toolbox...28 C. CONCLUSIONS...34 IV. LINEAR ALGORITHM FOR AUTONOMOUS UAV CONTROL...35 A. THEORY Sensors Swarm Movement...36 B. RECENT RESULTS...38 C. NEW RESULTS...45 D. CONCLUSIONS...52 vii

10 V. SUMMARY...55 A. PSO VERSUS LINEAR ALGORITHM...55 B. FUTURE WORK...56 APPENDIX A. MATLAB PSOT TOOLBOX...59 APPENDIX B. EXAMPLE OF BACKPROPAGATION TRAINING...65 APPENDIX C. EXAMPLE OF PARTICLE SWARM OPTIMIZATION TRAINING...71 APPENDIX D. ORIGINAL SWARM.JAVA SIMULATION...83 APPENDIX E. MODIFIED SWARM.JAVA PROGRAM...97 LIST OF REFERENCES INITIAL DISTRIBUTION LIST viii

11 LIST OF FIGURES Figure 1. Bridge Experiment...8 Figure 2. Multilayer Perceptron Neural Network Architecture with Two Hidden Layers...11 Figure 3. Illustration of Weight Connections for Two Inputs...12 Figure 4. Single-Layer Perceptron as an AND Binary Logic Unit...13 Figure 5. Single-Layer Perceptron as an OR Binary Logic Unit...14 Figure 6. Single-Layer Perceptron as an NOT Binary Logic Unit...14 Figure 7. Graph of Sigmoid Nonlinearity Function...15 Figure 8. Architecture of XOR Problem...16 Figure 9. Graph of Linear Threshold Function...17 Figure 10. Signal Flow Diagram of XOR Problem...18 Figure 11. Error of the Network Output...19 Figure 12. Forward Path...20 Figure 13. Backpropagation...21 Figure 14. Plot of First 1000 Epochs of the Backpropagation Algorithm for an XOR Problem...29 Figure 15. Plot of Next 1000 Epochs of the Backpropagation Algorithm for an XOR Problem...30 Figure 16. Plot of Final 688 Epochs of the Backpropagation Algorithm for an XOR Problem...30 Figure 17. Plot of First 25 Epochs of the PSO Algorithm for an XOR Problem...31 Figure 18. Plot of First 600 Epochs of the PSO Algorithm for an XOR Problem...32 Figure 19. Plot of First 1000 Epochs of the PSO Algorithm for an XOR Problem...32 Figure 20. Plot of Next 25 Epochs of the PSO Algorithm for an XOR Problem...33 Figure 21. Plot of Next 340 Epochs of the PSO Algorithm for an XOR Problem...34 Figure 22. Control Architecture...37 Figure 23. Swarm.java Program: UAVs at Initialization...38 Figure 24. Swarm.java Program: UAVs Head South...39 Figure 25. Swarm.java Program: UAVs Travel Around Inner Circle...39 Figure 26. Swarm.java Program: First Orbit Circle...40 Figure 27. Swarm.java Program: 5 Orbit Circles...41 Figure 28. Swarm.java Program: 8 Orbit Circles...41 Figure 29. Swarm.java Program: UAVs Begin Attack Sequence...42 Figure 30. Swarm.java Program: 6 UAVs Are Taking Attack Positions...43 Figure 31. Swarm.java Program: All UAVs Are In Attack Positions...43 Figure 32. Swarm.java Program: Attack Sequence...44 Figure 33. Swarm.java Program: Last UAV Attacks...44 Figure 34. Swarm.java Program: Attack Completed...45 Figure 35. Modified Swarm.java Program: 8 Orbit Circles...47 Figure 36. Modified Swarm.java Program: First UAV Heads Towards Target...48 Figure 37. Modified Swarm.java Program: Second UAV Heads Toward Target...48 Figure 38. Modified Swarm.java Program: First UAV Passes Over Target...49 ix

12 Figure 39. Modified Swarm.java Program: UAVs Continue Attack...49 Figure 40. Modified Swarm.java Program: UAVs Continue Attack While Recycled UAVs Appear at the Top of the Screen...50 Figure 41. Modified Swarm.java Program: All UAVs Have Passed Over the Target and Head South Until Another Target is Found...51 Figure 42. Modified Swarm.java Program: New Target is Detected by the Swarm...51 Figure 43. Modified Swarm.java Program: UAV Finds First Orbit Circle...52 x

13 LIST OF TABLES Table 1. Binary XOR Table...16 Table 2. UAV Population Size and Average Time of Attack...46 xi

14 THIS PAGE INTENTIONALLY LEFT BLANK xii

15 ACKNOWLEDGMENTS This work was supported in part by the Tactical Electronic Warfare Division, Naval Research Laboratory, Code NRL 5700, and the Office of Naval Research, Code ONR 313. xiii

16 THIS PAGE INTENTIONALLY LEFT BLANK xiv

17 EXECUTIVE SUMMARY The Unmanned Air Vehicle (UAV) can be applied to more military missions than ever before. The UAV is perfect for military missions that are long and arduous on a crew such as surveillance and reconnaissance. By carrying a small selection of weapons, a UAV can perform the same functions as a piloted plane and destroy a target. The advantage is that the UAV eliminates the potential loss of human life since it can be sent into highly dangerous areas. Previously these areas were avoided because the risk of human life outweighed the potential gains from such a flight. The success of the UAV has encouraged more research and ideas to maximize the advantages of having an unmanned platform. The next generation of UAVs will be smaller and part of a collaborative group. This group will be able to autonomously control their own movements and react to the environment.[1] A group of UAVs is more capable than a single UAV. The UAVs can divide the workload among the group. The individual UAVs can be equipped for different functions in the mission whether it be surveillance and reconnaissance, strike, or battle damage assessment. Surveillance missions can be completed quickly by covering more search area when the group is spread out. They also offer redundancy to ensure that the mission is completed. If one UAV is destroyed by the enemy or drops out because of mechanical failure, the rest of the group will fill in and carry out the mission. By having more than one UAV assigned to a mission, the probability of success dramatically increases.[1] The UAV s airframe will be designed and fitted with the appropriate technology to carry out the given mission. The initial design aspect is the software required to program the UAVs to act autonomously. Responding to a central command structure with human control is relatively simple. When given a command, the UAV reacts according. Without human control, the group will need to gather data from the environment, interpret the data, and take appropriate actions to continue on the mission. The UAVs will need to communicate with each other to share information in order to decide when the xv

18 mission is completed. Behavior algorithms are necessary in order to act in a decentralized manner and self-organize to complete the mission. The concept of a group of UAVs under autonomous control closely resembles the ideas of swarms and swarm intelligence, which is similar to the concept used by insects and birds. [1] The swarm must have a realistic and practical method for completing the required mission of finding and attacking a target. Two autonomous UAV control methods were analyzed: the Particle Swarm Optimization (PSO) algorithm and a linear control method. The PSO algorithm was simulated using MATLAB. The actual program is training a neural network to solve a problem, but the concept is analogous to UAVs searching for a target. Through flock simulation and the derivation of PSO, scientists discovered that a synchronous flock is not essential.[2] The simulated synchronized flock limits the scope of the group because it does not allow of individual exploration of the area. The flock has to tightly travel together; so in order to search the area thoroughly, the entire flock would have to go over all possible locations. By allowing individuals to travel slightly outside the group, the group covers a larger search area at one time. As they identify individual best found positions thus far, the group is able to discover the target faster. For a group to cooperate and achieve goals such as finding a target, the group must communicate. Therefore, communication, rather than synchronization, is necessary for success.[2] The current PSO algorithm applies to weightless particles in multiple dimensions. The PSO algorithm can offer the advantage of finding the pattern in almost any problem space to reach a solution, but the current sequence can dead end and restart in a new position. It is a waste of computation time and resources to create an algorithm that would have a swarm of UAVs pursue a direction only to find it is the wrong path. If the target cannot be reached from the current path of the swarm, the PSO algorithm s solution is to start over. The swarm needs more guidance and a process to get out of a dead-end situation and back on track. With further research and improvements, the PSO algorithm can be applied to real objects limited to three dimensions. xvi

19 PSO is focused on minimizing error between the particles and the target. In addition to changing the particle s direction to head toward the target, the algorithm accelerates the particles. When applying PSO to real flying objects, the constant speed changes are the main drawback. Actual UAVs should maintain a constant velocity to operate in a stable and controlled manner to prevent chaos and collisions. The constant velocity will also increase fuel efficiency and decrease strain on the platform. Although the PSO method is not practical, the central idea of minimizing error is completely applicable to UAVs. When the target location is known, error minimization is a valuable tool. Since the first method investigated did not perform realistically, a second method using a linear model for UAV movement was investigated. The program used to model the linear method was initially designed by a student at North Dakota State University [3]. The simulation focuses on three simple maneuvers for UAV motion. Although the simulation has limited abilities, the concept is easily applicable to real UAVs missions. The program was modified in this thesis to incorporate more realistic mission scenarios. Compared to the PSO, the linear algorithm produces the most realistic results. The linear algorithm incorporates the ideas that have performed well in the PSO. The swarm does not have to move synchronously, and the UAVs move toward the target by minimizing the error in their position from the target. The error is minimized in a linear fashion since the velocity of the UAV remains constant. Linearity produces great results, and the simulated UAVs are able to find the target quickly and efficiently. The program also handles the UAVs as objects that occupy space. Each UAV has a threshold boundary distance, so they will avoid each other if they get too close. These movements allow the swarm to move toward a destination in space without collisions. Since the swarm does not travel in formation, the UAVs need to regroup once a target is found. The orbit stations around the target provide organization before the attack. While orbiting, the UAVs can communicate and coordinate when the attack will occur. The orbit circles are also far enough from possible dangerous areas surrounding the target. The simulation shows the distance to be small relative to the size of the target and UAVs, but the radius of the circle is adjustable. Overall the linear algorithm can be more easily simulated and applied to realistic missions on a larger scale. xvii

20 THIS PAGE INTENTIONALLY LEFT BLANK xviii

21 I. INTRODUCTION A. SWARM OF UAVS The Unmanned Aerial Vehicle (UAV) can be applied to more military missions than ever before. The UAV is perfect for military missions that are long and arduous on a crew such as surveillance and reconnaissance. By carrying a small selection of weapons, a UAV can perform the same functions as a piloted plane and destroy a target. The advantage is that the UAV eliminates the potential loss of human life since it can be sent into highly dangerous areas. Previously these areas were avoided because the risk of human life outweighed the potential gains from such a flight.[1] The success of the UAV has encouraged more research and ideas to maximize the advantages of having an unmanned platform. The next generation of UAVs will be smaller and part of a collaborative group. This group will be able to autonomously control their own movements and react to the environment. Currently two or more people are required to control a single UAV while it is in flight. The military is hoping to decrease the man-power required to operate the UAV. A UAV controlled by a group of people can complete the mission, but a large number of UAVs would require an even larger group of people. Such a large number of operators will be unpractical and inefficient compared to the idea of autonomous control.[1] A group of UAVs is more capable than a single UAV. The UAVs can divide the workload among the group. The individual UAVs can be equipped for different functions in the mission whether it be surveillance and reconnaissance, strike, or battle damage assessment. Surveillance missions can be completed quickly by covering more search area when the group is spread out. They also offer redundancy to ensure that the mission is completed. If one UAV is destroyed by the enemy or drops out because of mechanical failure, the rest of the group can fill in and carry out the mission. By having more than one UAV assigned to a mission, the probability of success dramatically increases.[1] The UAV s airframe is designed and fitted with the appropriate technology to carry out the given mission. The initial design aspect is the software required to program 1

22 the UAVs to act autonomously. Responding to a central command structure with human control is relatively simple. When given a command, the UAV reacts according. Without human control, the group will need to gather data from the environment, interpret the data, and take appropriate actions to continue on the mission. The UAVs will need to communicate with each other to share information in order to decide when the mission is completed. Behavior algorithms are necessary in order to act in a decentralized manner and still self-organize to complete the mission. The concept of a group of UAVs under autonomous control closely resembles the ideas of swarms and swarm intelligence, which is similar to the concept used by insects and birds.[1] B. PRINCIPAL CONTRIBUTIONS The goal of this thesis was to create a detailed simulation of a swarm of UAVs that has autonomous control. The swarm must have a realistic and practical method for completing the required mission of finding and attacking a target. The attack will be simulated by occupying the same point location as the target. Whether each UAV launches a missile or takes pictures while over the target is dependent upon the given mission. Two autonomous UAV control methods were analyzed: the Particle Swarm Optimization (PSO) algorithm and a linear control method. The PSO algorithm was simulated using MATLAB. The actual program is training a neural network to solve a problem, but the concept is analogous to UAVs searching for a target. Since the first method investigated did not perform realistically, a second method using a linear model for UAV movement is investigated. The program used to model the linear method was initially designed by a student at North Dakota State University [3]. The simulation focuses on three simple maneuvers for UAV motion. Although the simulation has limited abilities, the concept is easily applicable to real UAVs missions. The program is modified in this thesis to incorporate more realistic mission scenarios. C. THESIS OUTLINE Chapter II provides a background for swarms and swarm intelligence. The swarm algorithms that have emerged are briefly described. Since swarm algorithms are an efficient method for training neural networks, the concept of neural networks is briefly reviewed. 2

23 Chapter III describes the concept of PSO from theories of flocks of birds to training neural networks to applications for a swarm of UAVs. A MATLAB program simulates a particle swarm using the PSO concept, and the PSO algorithm is compared to the backpropagation algorithm for training a neural network to solve an XOR problem.[4] Chapter IV describes a linear control approach to organizing a swarm of UAVs. The linear concept is displayed in a program demonstrating a swarm of UAVs attacking a target. Chapter V summarizes the practicality of using PSO and a linear algorithm for UAV control along with future work with a UAV swarm. Appendix A describes the PSO function in MATLAB, created by Brian Birge in [4]. Birge also creates a demotrain file to use and illustrate the capabilities of the new PSO function while comparing PSO to backpropagation. Appendix B demonstrates a neural network backpropagation example through the demotrain file. file. Appendix C demonstrates a neural network PSO example through the demotrain Appendix D demonstrates the linear approach program created by Chin Lua from North Dakota State University.[3] Appendix E demonstrates a modified program from [3]. 3

24 THIS PAGE INTENTIONALLY LEFT BLANK 4

25 II. BACKGROUND A. SWARMS 1. Origin Gerado Beni began using the term swarm to describe his work with cellular robots. It was casually mentioned to him by Alex Meystel at a conference, and Beni found that swarm accurately described cellular automata.[5] The group of cells in robots exhibit similar characteristics found in biological swarms, like insects, such as decentralization, no synchronization, and simplicity in the members. Swarm correctly describes a robotic system, but while roboticists focus on performing tasks with the swarm, biologists take the swarm concept and analyze the social behavior of insects as they perform a function.[5] 2. Self-Organization Biologists investigate the idea that the swarm will identify a pattern in their system and self organize to find the optimal means of reaching the goal. Roboticists are trying to create patterns in the behavior so that the cells will self-organize. Communication is a key parameter in allowing the members to interact and self-organize their tasks. The characteristics of a swarm are extended from decentralization, no synchronization, and simplicity to include communication among the members.[5] In real swarms, such as bees and ants, members of a group can use direct communication with each other or indirect communication, referred to as stigmergy, to interact through the environment.[6] a. Positive and Negative Feedback Bees and ants are constantly pursuing food sources for their colonies to sustain their existence, therefore, location of a food source is a common example of social behavior and easy to demonstrate through experimentation. Their ability to establish paths to a food source illustrate one level of self organization.[6] Bees directly communicate the location of a food source through dancing. A bee s dance will show the distance and direction of the food source to the others. When one food source is superior in quality, the bees take advantage of the better source as more bees dance to indicate that location. Experiments indicate that differences in the rate of dancing and abandonment create positive feedback and cause more bees to follow 5

26 the best quality path. Not all bees will congregate to the superior food source. A small population will continue to go to the alternate food source or elsewhere as a response to negative feedback in the system. Negative feedback is generated through saturation, exhaustion, crowding, and competition at the food source. These realistic limits stabilize the system so all bees are not on one track. [6] Ants are most well known for communicating through the environment with pheromones to indicate the quality of a path. These pheromones provide a positive feedback method for the colony to reinforce the trail. The constant amplification of the trail persuades the other ants to continue along the same path, providing positive feedback. Negative feedback is introduced through the same general complications as bees: saturation, exhaustion, crowding, and competition at the food source.[6] b. Randomness Both bees and ants rely on the randomness of individuals in the group. It may seem counterintuitive to believe that self-organization is created among randomness, but randomness allows the introduction of new ideas into the group. It can provide simply new paths to a food source or more general new methods and solutions that allow for growth of the colony. Randomness is also a source of optimization. For example, two food sources that are identical in quality and equal distance from the bee hive should be utilized symmetrically. Experimentally, the deviation of a few bees will cause a swing to one source because those few bees recruit more bees and those continue to recruit even more. The same principle applies to ants. As more ants go along a path the pheromone strength becomes greater. One path is amplified and becomes the optimal path.[6] Multiple interactions occur throughout the group, causing more actions and reactions. System characteristics of positive feedback, negative feedback, and randomness provide the balance needed to keep the group responsive to an ever-changing environment.[6] 3. Swarm Intelligence The self organization of the group into ordered patterns is an intelligent characteristic. For a swarm to form ordered patterns, it needs to analyze patterns while finding the optimal method. This characteristic could allow the swarm to have intelligence. [5] 6

27 Beni struggled with the definition of swarm intelligence since the word intelligence has also been so loosely used. One preliminary definition of an intelligent swarm according to Beni is a group of machines capable of unpredictable material computation. He has to revisit the definition of a machine as an entity capable of mechanical behavior, i.e., of transferring and/or processing matter/energy. Unpredictability is also difficult to define, but Beni links it to the computational power of the system. He is looking for a system (the intelligent swarm) which cannot be predicted in the time it takes to form a new material pattern (of its own components). In his paper clarifying definitions related to swarms, he finally settles on Intelligent swarm: a group of non-intelligent robots ( machines ) capable of universal material computation. [5] Hundreds and even thousands of non-intelligent machines can comprise an intelligent swarm. There are advantages of having simple components in a group over having complex centralized components. Through self-organization and pattern identification, the individual machines working together as a swarm can accomplish tasks that could not have been possible by a single machine. Logistically, the individual members of the swarm are easier to design and build, so these simple components potentially can be cheaply replaced, interchanged, or disposed of.[5] The unpredictable function of a swarm comes from the method that it learns. The concept of universal material computation allows for the creativity of the designer. There are numerous algorithms in existence allow the swarm to compute a possible process to complete various tasks. B. SWARM ALGORITHMS 1. Ant Colony Optimization The behavior of ants within a colony inspired experiments and eventually algorithms to mimic the ants. The fundamental task of an ant colony is to find food sources. While performing this task, the ants are able to find the shortest path to that food source. This natural optimization is tested by Deneubourg through the bridge experiments.[6] The binary bridge experiment consists of two paths of equal length from their nest to the food source, so initially all ants choose a random path. As they continue to choose at random, ants travel on one path, the pheromone intensity increases on that path. As few more ants break the chance, they attract more ants, until the majority is on one 7

28 path, such as explained for randomness. Two equal length paths can be branched out to a longer path and a shorter path. Deneubourg also created a bridge from the nest to the food with two longer branches, shown in Figure 1. Nest Food Figure 1. Bridge Experiment Again, all paths are initially chosen at random. The ants that take the shortest path to the food source and back to the nest obviously make it back to the nest first. Their route is twice as intensified by pheromones than other options because the other ants on the longer paths have only passed over their route once. This initial difference is amplified until the majority of the ants are on the shortest path.[6] The ant colony optimization concept stimulated numerous more experiments and research. A few algorithms such as the Traveling Salesman Problem (TSP), Ant System (AS), Ant Colony System (ACS), and AntNet can be found in [6]. Overall the algorithms have limited success with performance for the problems originally intended. They are instead applied to combinatorial optimization, communications network routing, and packet-switching communications networks. Since they offer a more promising future in this area, other directions were taken for swarm theories.[6] 2. Evolutionary Computation Genetic and evolutionary algorithms are two examples of algorithms that use a population set to evolve to a solution. These algorithms are similar in concept but executed differently. The differences are becoming undecipherable in most cases, and ge- 8

29 netic and evolutionary algorithms are almost interchangeable in meaning. Both mimic natural evolution by using survival of the fittest and allowing manipulation of the population. Each individual in the population is assigned a fitness value based on the problem. By collecting the individual with higher fitness, the population will progress toward the solution to a given problem. The difference lies in the reproduction, crossover, and mutation process. The fitness values for the problems indicate where they are in a problem space. They represent how close the individual is to the goal, so the population is searching the problem space for the solution.[6] a. Genetic Algorithm The genetic algorithm follows the general pattern of initializing the population, calculating each individual s fitness in the population, reproducing the selected individual to create a new population, imposing crossovers and mutations on the populations, and repeating the process over again until the desired population is reached. The population size is typically between 20 and 200 since it directly affects the computation time. Larger populations can search more of the entire solution space, but the computational cost is too great. The initial population can be randomly chosen or contain a few seeded individuals with selected traits. The deserved initial population should cover a wide variety of traits to avoid limiting the algorithm from the start.[7] Calculating the fitness function of each individual can be a complex process, but the idea is simply to sort out the best individuals that satisfy the solution. Various processes for calculating the fitness function exist. Each individual is assigned a fraction of a roulette wheel to correspond with the fitness value. The fraction on the roulette wheel indicates the probability of an individual being selected. There are also numerous variations of the probability assignment procedure. The basic idea remains the same: a higher level of fitness has better chance of being chosen on the wheel. For example, individual A has a fitness value of 0.4 and individual B 1.2. Individual B will occupy three times more space on the roulette wheel and is three time more likely to be selected than individual A. Once all the individuals have been chosen, they proceed to the main step of genetic theory.[7] The population experiences crossover to model the results of sexual reproduction. The probability of crossover and type of crossover are specified for each prob- 9

30 lem. The probability typically ranges for 0.6 to 0.8, causing 60 to 80 percent of the population to experience crossover. The basic crossover type is one-point and easily described using binary bits. Take two individuals with the following characteristics: They experience crossover at a randomly chosen point indicated by the vertical line below: The bits to the right of the vertical line will be exchanged. The two resulting individuals after crossover are: Mutation is introduced after crossover. Mutation has a much lower probability of occurrence, generally down to Mutation simply involved the flipping of a random bit. Since the probability is extremely low, it may only occur to one individual in the entire population each generation.[7] b. Evolutionary Algorithm Evolutionary approaches focus on frequent mutations to change the population rather than genetic recombination. The general pattern for an evolutionary algorithm is initializing the population, exposing the population to the environment, calculating each individual s fitness in the population, mutating individuals at random, recombining to create a child population, reevaluating the entire population of parents and children, selecting individuals to create a new population, and repeating the process over again until the desired population is reached. Mutating the parent population before the reproduction phase is similar to individuals being altered by the environment growing up in life. This early mutation can be more successful than mutating the child population. The individuals also recombine in a specified manner in the algorithm rather than swapping randomly selected bits. The child then becomes a combination of both parents, even with their mutations. Overall, both genetic and evolutionary algorithms are similar and help inspire new ideas for modeling social behavior.[7] 10

31 C. NEURAL NETWORKS 1. Basics An Artificial Neural Network (ANN), or commonly referred to as simply a Neural Network (NN), is modeled after neurons and synapse connections in the brain. The design of a neural network will help simulate artificial behavior by establishing patterns when exposed to a situation. The multi-layer perceptron neural network has proven to be an excellent approximator of most non-linear functions. This neural network entwines three key components: an input layer, one or more hidden layers, and an output layer.[8] Figure 2 illustrates how the elements are connected. I 1 O 1 I 2 O 2 I 3 O 3 I i Input layer First hidden layer Second hidden layer Output layer Figure 2. Multilayer Perceptron Neural Network Architecture with Two Hidden Layers 11

32 The circles represent the neurons, or nodes, which are individual perceptions. The inputs range from 1 to i, the first hidden layer nodes from 1 to j, second hidden layer nodes from 1 to k, and so forth. The arrows represent the weighted connections between neurons. The weights in the first hidden layer will be referred to as w i, j, to represent the weight from the i -th input to the j -th node of the first hidden layer. The weight values determined the established pattern of the neural network. The output response is dependent upon the weighted connections.[8] w 1,1 I 1 w 2,1 w 1,2 w 2,2 I 2 w 1,3 w 2,3 Input layer w 1,4 w 2,4 First hidden layer Figure 3. Illustration of Weight Connections for Two Inputs 12

33 a. Single-Layer Perceptrons A single-layer perceptron has the ability to handle linear functions and to act as binary logic such as AND, OR, or NOT, demonstrated in Figures 4 through 6. These binary logic units are evaluated using a hard-limited non-linear activation function: f HL 1 v > 0 =. (1) 0 v 0 The connection from the input to the perceptron is given a weight, and the perceptron can also have a bias value to satisfy the desired pattern. The values inside the triangles represent the values of the weighted connections.[8] Single-Layer Perceptron X u 0.75 X v f HL u X v= 0.5X + 0.5X 0.75 u = X X X1 X2 v u Figure 4. Single-Layer Perceptron as an AND Binary Logic Unit 13

34 Single-Layer Perceptron 0.25 X y f HL u X y = 0.5X + 0.5X 0.25 u = X X X1 X2 v u Figure 5. Single-Layer Perceptron as an OR Binary Logic Unit Single-Layer Perceptron 0.25 X y f HL u y = 0.5X u = X 1 1 X1 v u Figure 6. Single-Layer Perceptron as an NOT Binary Logic Unit 14

35 b. Multi-layer Perceptrons There are three requirements for a multi-layer perceptron neural network. The neural network must have one or more hidden layers, meaning more than simply an input and output layer. The hidden layers are what allow the network to respond to more complex patterns. The second requirement is that the network must have a high degree of connectivity, so each node from one layer will be connected to all the nodes in the following layer.[8] The third requirement for the perceptrons in a multi-layer perceptron neural network is that they must use a differential nonlinear activation function. The sigmoid nonlinearity is a differentiable function and also provides values between 0 and 1. The advantage is that the values from 0 to 1 are analogous to probability distribution and provide easier pattern recognition. The sigmoid function is defined as follows: f sigmoid ( v) 1 = (2) v 1 + e β where β is the gain.[8] As shown in Figure 7, as β increase, the slope of the sigmoid function becomes steeper. f sigmoid (v) β =5 β =0.5 β = 1.0 Figure 7. Graph of Sigmoid Nonlinearity Function 15

36 c. Exclusive OR (XOR) Problem The XOR problem is an example of problem that must use a multi-layer perceptron because a nonlinear pattern is required to solve an XOR equation.[8] The two inputs to an XOR equation will produce an output according the values in Table 1. X1 X2 Output Table 1. Binary XOR Table When drawing the unit hypercube, no line exists that would separate correctly (0,0) and (1,1) from (0,1) and (1,0), therefore XOR requires a nonlinear solution. A single-layer perceptron has a linear decision boundary, so it cannot be used to solve this problem. Touretzky and Pomerleau designed the XOR solution in 1989 to have one hidden layer with two nodes.[8] Figure 8 represents the architecture of the design. X 1 w 1,1 w 1,2 w 1,1 b 1 Output w 2,1 b 2 w 2,1 b 1 w 2,2 X 2 w i,j w j,k Input layer First hidden layer Output layer Figure 8. Architecture of XOR Problem 16

37 The activation function for their model is the linear threshold function, where T = 0, as shown in Figure 9. It is similar to a sigmoid function with a large β value. f threshold (v) T = 0 v Figure 9. Graph of Linear Threshold Function The weights and biases can be any combination of values. The following values are commonly chosen because of their simplicity and are already published in [8]. The weights w i, jand biases b j are defined for the input layer as follows: w1,1 = w1,2 = 1 b1 = 1.5 w2,1 = w2,2 = 1 b = 0.5. The weights w jk, and biases b k are defined for the hidden layer as follows: 2 w1,1 = 2 w2,1 = 1 b =

38 The signal flow diagram in Figure 10 illustrates the network with the numeric weights and biases. The outputs at each stage are computed below. 1.5 X 1 1 v 1 f T u v 1 Output 1 f T X v 2 u 2 1 f T 1 1 X1 X2 v1 v2 u1 u2 v1,output Output Figure 10. Signal Flow Diagram of XOR Problem 2. Backpropagation The multi-layer perceptron neural network has the ability to learn or create a pattern to minimize the mean-squared error between the generated output and the desired output. There are other training algorithms, but the backpropagation algorithm is the most commonly demonstrated.[8] The backpropagation algorithm relies on the simple difference equation of the desired output ( y desired ) and the actual output ( y ) to find the error of the network output ( e y ), e = y y. (3) y 18 desired

39 As shown in Figure 11, the neural network produces the output values after each iteration, and each iteration is referred to as an epoch. After each epoch, the meansquared error of the network is evaluated to see if it has reached a minimum. The learning process continues on epoch by epoch until an arrangement of weights and biases produce the minimal error for a problem. By correlating the data and creating a pattern in the network through training, when the network is presented with input data outside the training set, the network will produce reasonable output values for the patterned function.[8] O 1 y desired O 2 y ± e y O 3 Last hidden layer Output layer Figure 11. Error of the Network Output a. Forward Path Since there can be multiple hidden layers, the output y is generated at the end of the forward path, as shown in Figure 11 and derived in [8]. One hidden layer is illustrated in Figure 12. The first term v j is the collection all inputs and weights on the node, where m is the total number of inputs (for simplicity, bias values are not included) v = w I. (4) j i, j i i= 1 The sigmoidal non-linearity function is solved for each value of v j : 19 m

40 f ( v ) 1 =. (5) + sigmoid j v j 1 e β Any value of β can be used. But once β is defined for a neural network, it will not be changed. The same process can be continued for all cascaded layers where fsigmoid ( v j ) will be multiplied by the next layer of weights until the final layer. The output of the network is y = fsigmoid, final ( vfinal ). (6) v 1 fsigmoid,1 ( v 1 ) v 1 I 1 I 2 v 2 fsigmoid,2 ( v 2 ) v 2 I 3 v 3 fsigmoid,3 ( v 3 ) v 3 I i v j ( j ) f v sigmoid, j v k Input layer w i,j First hidden layer w j,k Second hidden layer Figure 12. Forward Path b. Backward Path The backward path shown in Figure 13 is a result of the backpropagation algorithm. For each epoch, the error signal is propagated back through the network. By following the backward path, the weights of each connection will change according to The degree of changes in the weight value depend on a value referred to as the learning rate η.[8] The learning rate is directly related to the response time of the neural network in identifying a pattern. If η is too small, the network will require more iterations to con- 20 ( ) i, j η y sigmoid i j. w = e f v y (7)

41 verge on a pattern. If η is too large, the weights will adjust significantly, typically overshooting the desired pattern, causing the network to diverge rather than converge.[9] O 1 y desired O 2 y ± e y O 3 Hidden layer Output layer Forward Path Backward Path Figure 13. Backpropagation The backpropagation algorithm is effective when solving the non-linear neural networks problems. D. BACKGROUND CHAPTER SUMMARY A swarm is a group of simple individuals that display characteristics such as decentralization, no synchronization, and communication amongst the group. A swarm is able to self-organize to complete tasks as a group. For artificial swarms, behavioral algorithms have been created to model the real swarm characteristics of bees and ants. The genetic algorithm and evolutionary algorithm are two methods for finding a solution in a search space. Both algorithms perform well, so the algorithms continue to be improved upon. The focus of Chapter III is PSO, which is an emerging evolutionary algorithm. 21

42 Artificial neural networks are another method for simulating artificial behavior. Backpropagation is the most common method for finding the arrangement of weights and biases needed to solve a problem. Backpropagation is often compared to other evolutionary algorithms such as PSO. PSO is described in Chapter III, and its performance is compared to backpropagation. 22

43 III. PARTICLE SWARM OPTIMIZATION ALGORITHM The PSO algorithm is similar to evolutionary algorithms but altered to model flocks of birds. A flock of birds displays more desirable behavioral characteristics than the previous insect swarms. Therefore, the PSO algorithm incorporates more desirable movements than the previous evolutionary algorithms. The PSO equation determines the velocity of individual in the swarm. A simulated flock can locate a target while moving according to the PSO equation. The PSO algorithm also applies to neural networks, and it performs better than backpropagation. A. THEORY 1. Flocks Scientists are attempting to model the intriguing actions in bird flocking. The synchronous motions of the flock can be observed as they quickly change directions simultaneously, scatter, and regroup. The flock s social behavior is similar to swarms of insects, schools of fish, and herds of animals. Behaviors are stimulated through environmental factors. Whether finding food, avoiding predators, or pursuing better environmental conditions such as temperature, the groups typically move with a purpose. Their dynamic behavior in a flock appeared to be related to the distances between each bird in the flock. The initial simulations of flocks of birds were based on modeling these distances. The theory was that birds try to maintain an optimum distance between neighbors, which results in the synchronous movement.[2] The initial theories are lacking realism. They are lacking any avoidance measures, so the simulated birds will collide on the screen. Since two real objects cannot occupy the same space, another approach had to be taken. Another consideration is an evolutionary algorithm. In evolutionary algorithms, the positions of the population change because the current population creates another child population with similar positions. This position change is dependent upon the recombination of parent positions. The child does not occupy the same position as either parent, but the child s position is nearby.[7] The positions changes are abrupt, and there is not a smooth transition from one generation to the next. The evolutionary algorithm needs to be modified. 23

44 To provide a realistic algorithm, the population of a flock has to remain the same without reproducing. Real flocks of birds do not reproduce during mid-flight. To reach all the desired locations in the space, the population needs to physically move to the new position rather than simply relocate. The individual members of the population will move from position to position by an assigned velocity. To move together as a flock, the original concept of maintaining an optimum distance difference gave way to velocity matching. The velocity of an individual is changed to match the velocities of their neighbors. The agents, the common term for the individuals in the simulated flock, will begin to synchronize their movements by having the same velocity vectors. The initial simulations also lack stimulants to trigger changes in behavior. Once the simulated flock joins together in formation, it continues in one direction uninterrupted. To cause change, craziness is implemented, so a random variable is added to the matched velocities in order to create variations in the flock movement.[2] Frank H. Heppner, a zoologist at the University of Rhode Island, created simulations that involve a flock being attracted to a roost. [2] The roost is a selected position on the screen to give the flock a target. After each iteration, the agents can determine their distances from the roost by using a two-dimensional XY distance equation referred to as the cornfield vector, where ( 100) ( 100) 2 2 distance = presentx + presenty (8) for an agent at point ( presentx, presenty ) and a roost at the point ( 100,100 ). Each agent is allowed memory so that it can remember its best distance achieved. The best position distance for each agent i will be referred to as pbest[] i and separated into components pbestx[] i and pbesty[]. i Each agent can easily adjust the velocity components to move toward pbest. The degree of adjustment is limited by a predetermined value called p _ increment. Variation is still added to flock by randomizing the velocity adjustment from 0 to p _ increment. The new velocity vectors are: if presentx[] i > pbestx[],then i vx[] i = vx[] i rand()* p _ increment if presentx[] i < pbestx[],then i vx[] i = vx[] i + rand()* p _ increment if presentx[] i > pbesty[],then i vy[] i = vy[] i rand()* p _ increment if presentx[] i > pbesty[],th i en vy[] i = vy[] i + rand()* p _ increment. 24 (9)