Ant Colony Optimisation

Transcription

1 Ant Colony Optimisation Alexander Mathews, Angeline Honggowarsito & Perry Brown 1 Image Source:

2 Contents Introduction to Ant Colony Optimisation Natural Inspiration Ant System Advanced Ant Colony Optimisation Extensions to Ant System Max-Min Ant System Best-Worst Ant System Applications of Ant Colony Optimisation The Generalised Assignment Problem (Static) Packet-switched network Routing (Dynamic) 2

3 ACO:Introduction Originally proposed by Marco Dorigo in 1992 To solve the Travelling Salesman Problem (TSP) By simulation the movements of ants The original ACO was not competitive with specialised TSP algorithms Many improvements have been proposed ACO can be extended to solve generic optimisation problems Most applications are to routing and scheduling problems 3

4 Natural Inspiration Model the behaviour of ants searching for food Model based on experimental observations Ants mark a path to food using pheromone Other ants can detect the pheromone trail Ants tend to follow paths with more pheromone This is location based communication An ant trail. Source: Wikipedia, Ants, Retrieved 12/9/12 4

5 Natural Inspiration Example Ants leave the nest at regular intervals 50% take the top path 50% take the lower path Ants which find food return to the nest by a random path Ants taking the top path return first The top path starts to accumulate pheromone at A Ants leaving the nest tend to take the top path Has more pheromone Graphics adapted from: Dorigo et al

6 Towards an Algorithm Want to solve different problems those solved by real ants Intermediate nodes: are components of a solution Edges: are possible connections between components Food Sources: are a complete solutions (with a fitness) Virtual ants are allowed: Perfect memory: ants can exactly retrace their steps Variable pheromone strength: strength related to quality of solution found 6

7 Towards an Algorithm Typical example Travelling Salesman Problem (TSP) (Find the shortest cycle that visits every city once and only once) Intermediate nodes: are cities Edges: are paths between cities Food: A completed cycle of the graph Note: that nodes are cities not partially constructed tours. 7

8 Ant System (AS) We have: A population of ants (starting at random cities) Ants which construct one solution per iteration A simple heuristic giving the 'fitness' of an edge Only pheromone information kept to the next iteration 8

9 Ant System: Overview While not done: randomise ant positions For each ant: Construct a random tour For each edge: Evaporate pheromone For each ant: Lay pheromone on each edge in tour 9

10 Ant System: Overview Initialise the pheromone for every edge Construct a random tour Evaporate pheromone from every edge Increase the pheromone on edges in the random tour 10

11 Ant System: Overview After a number of iterations we get the optimal tour. Hopefully. 11

12 Ant System: Ant Movement Until a solution is completed: For each edge to a node not in the tour: Calculate heuristic for edge Check pheromone strength for edge Calculate attractiveness of edge Randomly select edge with probability based on attractiveness Attractiveness is calculated as: (Pheromone) α ( Heuristic) β (need normalise across outgoing edges to get a probability) 12

13 Ant System: Ant Movement An ant is at the green point trying to complete the tour Discard edges to nodes already in the tour Get the Heuristic and Pheromone values H(A) = = 200 H(B) = = 80 P(A) = 2, P(B) = 1 Calculate Attractiveness A(A) = 28.3 A(B) = 8.9 Convert to Probability P(A) = 0.76 P(B) = 0.24 An edge to (A) is likely 13

14 Parameters N : Number of ants α : Influence of the pheromone trail β : Influence of the heuristic ρ : Fraction of pheromone that evaporates Low evaporation leads to slower convergence & possibly better solutions Q : scales the strength of the pheromone laid Pheromone Strength= Q Length of Tour (Pheromone) α ( Heuristic) β 14

15 Termination Conditions Iteration/Time limit Solution is good enough Stagnation: Only one tour will be tried as the pheromone is too strong. Or: run forever to maintain a good solution to a dynamic problem. 15

16 Contents Introduction to Ant Colony Optimisation Natural Inspiration Ant System Advanced Ant Colony Optimisation Extensions to Ant System Max-Min Ant System Best-Worst Ant System Applications of Ant Colony Optimisation The Generalised Assignment Problem (Static) Packet-switched network Routing (Dynamic) 16

17 Elitist Ant System First improvement on Ant System Also called Elitist Strategy for Ant System Provide strong additional reinforcement to the arcs belonging to the best tour found Pheromone update rule 17

18 Elitist Ant System Appropriate value for parameter e allows EAS to: Find better tours Find them in a lower number of iterations Evaporation rule is implemented as in Ant System 18

19 Rank-Based Ant System Introduced by Bullnheimer, Hartl and Strauss in 1997 Ranking of pheromone update The m ants are ascending ranked based on solutions quality. (S 1, S 2,..., S m) Each ants deposit an amount of pheromone that decrease with its rank Experimental evaluation suggests that this system performs slightly better than EAS and significantly better than AS. 19

20 Ant Colony System Another extensions of Ant System Three major improvements Different transition rule, pseudo-random proportional rule Probability Distribution 20

21 ACS - Improvement Daemon triggers pheromone update Ant with global best solution, Sglobal-best Pheromone evaporation before update Online step-by-step pheromone trail update Rule: 21

22 Max-Min Ant System (MMAS) Developed by Stuzzle and Hoos in 1996 AS extension Similar to ACS, offline pheromone trail update Best Ant chosen to add pheromone Iteration-best ant or best-so-far ant Improve ant s solutions using local optimisers 22

23 MMAS - Extensions Pheromone trails range [ Tmin, Tmax ] Tmax : Tmin : constant factor lower than Tmax Pheromone trails are initialised to maximum allowed value Re-initialization of pheromone trails 23

24 Best-Worst Ant System Proposed by Cordon et al. in 1999 Incorporates Evolutionary Computation Transition rule and pheromone evaporation Similar with MMAS, use local optimisers to improve ant s solutions 24

25 Best-Worst Ant System Improvements: The best-worst pheromone trail update rule. Penalise all connection of the worst solution. Scurrent-worse Update rule: 25

26 BWAS Improvement Pheromone trail mutation for diversity in search process. Pheromone is mutated with probability Pm. Adding or substracting the same amount in each iteration mut(it,tthreshold) depend on the average of pheromone trails Re-initialization of the pheromone trail, setting every pheromone to T0 26

27 Contents Introduction to Ant Colony Optimisation Natural Inspiration Ant System Advanced Ant Colony Optimisation Extensions to Ant System Max-Min Ant System Best-Worst Ant System Applications of Ant Colony Optimisation The Generalised Assignment Problem (Static) Packet-switched network Routing (Dynamic) 27

28 Further applications of ACO Two major types of problem that ACO is particularly suited to (Static) NP-hard problems, including: Routing problems (e.g. TSP) Assignment problems (e.g. university timetabling) Scheduling problems (e.g. scheduling project tasks) Subset problems (e.g. multiple knapsack problem) Dynamic shortest path problems e.g. routing in packet-switched networks N.B.: The terms static and dynamic here refer to whether or not the problem instance changes at run-time 28

29 Defining the pheromone trails Key consideration for applications: decide what the pheromone trails should signify Relative positioning Trails represent the desirability of choosing a particular solution component j directly after some other component i Intuitively suited to problems for which the position of components in the overall sequence is inconsequential (e.g. in TSP, no preference as to which cities get visited sooner or later) Absolute positioning Trails represent the desirability of choosing a particular solution component j at the ith position in the sequence Intuitively suited to problems for which the position of components in the overall sequence is significant (e.g. in scheduling project tasks, different tasks are due by different dates) Either approach is possible for any problem, but the intuitive choices described above generally give much better performance 29

30 Example application: Generalised Assignment Problem (GAP) Example of a static NP-hard problem A number of tasks to be distributed among a number of agents, each with a limited capacity Assigning a particular task to a particular agent involves using a certain amount of that agent s capacity Each task-to-agent assignment also incurs a specific cost Goal is to minimise the total cost of assigning all tasks within the agents capacity constraints 30

31 Representing the problem graphically Both tasks and agents represented by nodes Ants must walk from task to agent to signify assignment, then back to next task until all tasks have been visited and assigned Tasks Agents 31

32 Representing the problem graphically Both tasks and agents represented by nodes Ants must walk from task to agent to signify assignment, then back to next task until all tasks have been visited and assigned Tasks Agents Assignment 32

33 Representing the problem graphically Both tasks and agents represented by nodes Ants must walk from task to agent to signify assignment, then back to next task until all tasks have been visited and assigned Tasks Agents Assignment Move on to next task 33

34 Representing the problem graphically Both tasks and agents represented by nodes Ants must walk from task to agent to signify assignment, then back to next task until all tasks have been visited and assigned Tasks Agents Assignment Move on to next task 34

35 Aspects of the problem representation Two types of pheromone deposits One to determine which agent to assign a particular task to (relative) Other to determine best order in which to assign tasks (absolute) Heuristic information (static) Our pre-existing knowledge of this particular problem suggests that assignments with low cost and/or capacity usage should be preferred Range of feasible solutions limited by capacity constraints Strategy: assign a task to an agent with sufficient remaining capacity whenever possible Ants that produce an infeasible solution (i.e. one that violates capacity constraints) may have their pheromone contribution reduced 35

36 Example application: packet-switched network routing (dynamic) Problem involves a network with links connecting pairs of nodes, reaching one node from another may involve traversing a number of links in sequence Each link has a cost associated with using it Goal to find minimum cost path between every pair of nodes Normally an easy problem, except that in this instance the link costs are subject to change over time ACO compared with several state-of-the-art algorithms for this application, performed superior to all of them 36

37 Aspects of the problem representation Two types of pheromone deposits One to determine which agent to assign a particular task to (relative) Other to determine best order in which to assign tasks (absolute) Heuristic information (static) Our pre-existing knowledge of this particular problem suggests that assignments with low cost and/or capacity usage should be preferred Range of feasible solutions limited by capacity constraints Strategy: assign a task to an agent with sufficient remaining capacity whenever possible Ants that produce an infeasible solution (i.e. one that violates capacity constraints) may have their pheromone contribution reduced 37

38 References M. Dorigo and T. Stutzle, "The ant colony optimization metaheuristic: algorithms, applications, and advances," in Handbook of Metaheuristics, Eds. F. Glover and G. Kochenberger, New York: Springer, 2003, pp M. Dorigo et al., "Ant colony optimization," IEEE Comput. Intell. Mag, vol. 1, (4), pp , Nov G. Di Caro and M. Dorigo, "AntNet: distributed stigmergetic control for communications networks," J. Artif. Int. Res., vol. 9, (1), pp , Aug

39 References B. Bullnheimer, R. F. Hartl and C. Strauss, A New Rank Based Version of the Ant System: A Computational Study. Central European Journal for Operations Research and Economics, 7(1):25-38, O. Cordon, F. Herrera, and T. Stützle, A Review on the Ant Colony Optimization Metaheuristic: Basis, Models and New Trends. Mathware and Soft Computing, 9(2-3), pp , 2002h 39

40 References M.Dorigo and L. M. Gambardella. Ant Colony System: A cooperative learning approach to the travelling salesman problem. IEEE Transaction on Evolutionary Computation, 1(1):53-66, T. Stützle and H. H. Hoos, MAX-MIN Ant System. Future Generation Computer Systems 40

41 References O.Cordon, I. Fernandez de Viana and F. Herrera and L. Moreno. A new ACO model integration evolutionary computation concepts: The best-worst Ant System. In M. Dorigo, M. Middendorf, and T.Stutzle, editors, Abstract proceedings of ANTS From Ant Colonies to Artificial Ants: A Series of International Workshops on Ant Algorithms, pages 22-29, IRIDIA, Universite Libre de Bruxelles, Belgium,