Carleton University. Honours Project in Computer Science. Applying Ant Colony Optimization using local best tours to the Quadratic. Assignment Problem

Transcription

1 Carleton University Honours Project in Computer Science Applying Ant Colony Optimization using local best tours to the Quadratic Assignment Problem Chris Kerr Student # Supervisor: Dr. Tony White, Carleton University School of Computer Science April 8, 2004

2 Abstract When they introduced the Ant System paradigm, Dorigo, Maniezzo, and Colorni also suggested an improvement to the original algorithm. By making use of global information, a set of elitist ants could focus the search, potentially accelerating convergence on an optimal solution. In a later paper, White, Kaegi, and Oda proposed a refinement of elitism in which local information is used instead of global information. By focusing the search on the local best tours (LBT) rather than the global best tour, the ants could search a wider area and avoid becoming trapped in local optima. Test results using the symmetric Travelling Salesman Problem showed considerable promise. We will examine the effectiveness of this refinement when applied against the assymetrical Quadratic Assignment Problem. 1

3 Acknowledgments To Dr. Irwin Reichstein, Dr. Mark Lanthier, Dr. Karl MacDorman, and most importantly Dr. Tony White go my profound grattitude for guiding me through the various phases of selecting and completing this project. 2

4 Table of Contents 1. Introduction Ant System Elitism Local Best Tour QAP Methodology Algorithm Test Cases Parameters Analysis Results Further Investigation Discussion Conclusions Future Work...41 References

5 List of Figures Figure 1. Cost Difference between LBT and Elitist Algorithms (nug15)...19 Figure 2. Cost Difference between best tour and optimum (nug15)...19 Figure 3. Cost Difference between LBT and Elitist Algorithms (nug20)...20 Figure 4. Cost Difference between best tour and optimum (nug20)...21 Figure 5. Cost Difference between LBT and Elitist Algorithms (nug30)...22 Figure 6. Cost Difference between best tour and optimum (nug30)...23 Figure 7. Cost Difference between LBT and Elitist Algorithms (kra30)...24 Figure 8. Cost Difference between best tour and optimum (kra30)...25 Figure 9. Cost Difference between LBT and Elitist Algorithms (els19)...26 Figure 10. Cost Difference between best tour and optimum (els19)...27 Figure 11. Cost Difference between LBT and Elitist Algorithms (tai60)...28 Figure 12. Cost Difference between best tour and optimum (tai60)...29 Figure 13. Cost Difference between LBT and Elitist Algorithms (tai100)...30 Figure 14. Cost Difference between LBT and Elitist Algorithms (tai60-a)...31 Figure 15. Cost Difference between best tour and optimum (tai60-a)...32 Figure 16. Cost Difference between LBT and Elitist Algorithms (tai60-b)...33 Figure 17. Cost Difference between best tour and optimum (tai60-b)...34 Figure 18. Cost Difference between best tour and optimum (tai100-a)...35 Figure 19. Cost Difference between best tour and optimum (tai100-b)...36 Figure 20. Cost Difference between best tour and optimum (tai100-c)...37 Figure 21. Cost Difference between best tour and optimum (tai100-d)...38 Figure 22. Cost Difference between best tour and optimum (tai100-e)

6 1 Introduction In nature, ant colonies display a remarkable ability to locate the shortest path to food, despite the lack of central leadership, the limited search radius of the individual ants, and the lack of direct communication between most ants. Individually each ant has relatively little chance of succes, however, using pheromones as an indirect means of communication an ant colony is able to efficiently search for and find food. As each ant travels it leaves pheromones behind, along its path. When an ant encounters a pheromone trail it has a likelihood of following that trail, just as a sheep might follow the sheep in front of it. By strengthening or weakening a pheromone trail, one can increase or decrease the probability of trailing ants to follow that trail. When an ant finds food, it immediately turns around and heads home, following its own trail. In doing so, it reinforces the trail, increasing the likelihood that other ants will follow it towards food. Similarly, those ants will find and return with food, again reinforcing the trail. In this way, an ant colony finds a path to food. Given paths to two food sources of differing length, the shorter path will be travelled faster. Ants are able to travel the shorter path more often than the longer path, thereby increasing the pheromone trail along that path at a greater rate than the longer path. Over time the shorter path will accumulate far more pheromones than the longer path, and the longer path will be abandoned. Over time, this process is repeated until the ant colony has found the shortest path to food. 5

7 It is this emergent behavior which inspired Dorigo, Maniezzo, and Colorni to propose Ant Colony Optimization (ACO). ACO mimics the behavior of ant colonies, and applies it to optimization problems. Simple agents (ants) travel the solution space, and communicate only through indirect means (pheromone trails). With its simple elegance, this hueristic has been succesful in tackling even the toughest of problems across many domains. 6

8 1.1 Ant System In their introductory paper, Dorigo, Maniezzo and Colorni not only proposed the idea of Ant Colony Optimization, but also provided an implementation called Ant System (AS). Three versions were suggested: 'ant-density' in which a node is reinforced by a constant amount when an ant moves onto it; 'ant-quantity' in which a trail is reinforced by an amount inversely proportional to the cost of the move, each time an ant moves onto it; and 'ant-cycle' in which each node along a path is reinforced by an amount inversely proportaional to the cost of the path, at the end of each iteration. Since 'ant-cycle' was clearly demonstrated to be the best, Ant System will be assumed to be 'ant-cycle' for the rest of this paper. The basic high level algorithm is: // Initialization Set pheromone trail for all nodes to the initial value. Place the ants on starting nodes. // Main Loop While (termination condition not met) For each ant, probabilistically construct a solution, according to the desirability of the nodes, and the levels of pheromone on them. Remember the best solution found. For each node, reduce the pheromone trail according to the decay rate. For each ant, reinforce the pheromone trail for all nodes it has visited. 7

9 For each node, enforce minimum and maximum pheromone trail. End While //End Return the best solution found. AS was applied against a number of problems including the Travelling Salesman Problem and the Quadratic Assignment Problem with excellent results. Further information about Ant System can be found in [Dorigo et al, 1996] and [Bonabeau et al, 1999]. 8

10 1.2 Elitism As an improvement to the standard Ant System, Dorigo, Maniezzo, and Colorni proposed that a certain number of ants be designated 'elitist.' These ants remember the best tour found by any ant, and every iteration, in addition to their normal actions, deposit pheromone along the best tour. In such a fashion the best tour is constantly reinforced, encouraging ants to search along it. This effectively narrows the search and potentially speeds up convergence towards an optimal solution. The results demonstrated in [Dorigo et al, 1996] showed a marked improvement over the 'classic' Ant System algorithm. Elitism does have problems, however. It is sensitive to the number of elitist ants. Too few results in little gain, while too many can cause premature convergence on a local optima. Additionally, the need for global knowledge makes it impractical for some situations. Further information about elitism can be found in [Dorigo et al, 1996] and [Bonabeau et al, 1999]. 9

11 1.3 Local Best Tour Proposed by White, Kaegi and Oda [White et al, 2003], the Ant System Local Best Tour (LBT) is a refinement of elitism. Rather than using global information to reinforce the global best tour found to date, an LBT ant remembers it own best tour and reinforces that instead. Otherwise, LBT is identical to elitism. The potential benefits of LBT are many. The lack of global knowledge make it practical for use in distributed environments. LBT suffers less risk of early convergence on local optima, since the ants are not all reinforcing one path. This also allows for more LBT ants to be used than elitist. By potentially reinforcing many paths, a wider beam is searched by the ants. Even if local optima are found, this wider search path allows for 'moat jumping' [Winston 1992]. Application of LBT to the symmetric Travelling Salesman Problem showed promising results by convincingly outperforming elitism [White et al, 2003]. 10

12 1.4 QAP The Quadratic Assignment Problem (QAP) is an NP-hard minimization problem which models many real world situations, including arrangement of departments in hospitals, and ordering of correlated data in magnetic tapes[burkard, 1984]. A QAP instance of order n consists of n locations, n activities, a distance matrix D, and a flow matrix F. Each activity must be assigned to one and only one location, such that total assigment cost Z is minimized. The distance matrix D=[d ih ] contains the distances between locations i and h. The flow matrix F=[f jk ] contains the flows between activities j and k. The total assignment cost Z is the sum, for all pairings of locations i and h, of the distance between i and h multiplied by the flow between the activities assigned to i and h ( π(i) and π(h) ) : Z = Σd ih f π(i)π(h) {i = 1,...,n} {h = 1,...,n} Steps must be taken to map the QAP data into an appropriate form for use in Ant System. 1. Create a vector DP for the distance potential, where each element in DP is the sum of the elements of a line in distance matrix D. dp i = Σ d ij {i = 1,...,n} {j = 1,...,n} 11

13 2. Create a vector FP for the flow potential, where each element in FP is the sum of the elements of a line in flow matrix F. dp i = Σ f ij {i = 1,...,n} {j = 1,...,n} 3. Create a coupling matrix B = DP FP. b ij = d i f j This coupling matrix becomes the hueristic matrix for Ant System [Bonabeau et al, 1999]. More information about the Quadratic Assignment Problem can be found in [Burkard et al 1997]. More information about the application of Ant System to the QAP can be found in [Maniezzo and Colorni, 1999]. 12

14 2 Methodology 2.1 Algorithm The algorithm used is based on[bonabeau et al, 1999 p60], and [White et al, 2003]. Initialization For every coupling (location, activity) do Set initial pheromone trail for (location, activity) End For For k = 1 to (number of ants) Determine type of ant k probabilistically based on weights. Set the best cost of ant k to infinity Set the best tour of ant k to null End For Compute the distance and flow potentials and the coupling matrix B. Set the global best tour to null, and the global best cost to infinity Main Loop For t = 1 to (maximum iterations) do For k = 1 to (number of ants) do Build solution A(k) by applying n times the following two steps: (1) Choose the location i with the lowest distance potential among those not already assigned. 13

15 (2) Choose the activity j to assign to location i, according to the probabilistic function. End For For k = 1 to (number of ants) do Compute the cost C(k) of the assignment A(k) produced by ant k. If C(k) is less than the global best cost do Set the global best tour to A(k) and the global best cost to C(k) End If If C(k) is less than the best cost for k do Set the best tour for k to A(k) and the best cost for k to C(k) End If End For For every coupling (location, activity) do Reduce the pheromone trail for (location, activity) by the pheromone decay rate End For For k = 1 to (number of ants) do If ant k is GBT then Update the pheromones for A(k). Update the pheromones for the global best tour. Else If ant k is LBT then Update the pheromones for A(k). Update the pheromones for the best tour for ant k. 14

16 Else If ant is Classic Update the pheromones for A(k). End If End For For each coupling (location, activity) Enforce pheromone minimum and maximum. End For End For Return the global best tour and cost. A significant departure in this algorithm from its origins is the use of weights for the different ant types (classic, elitist, lbt). Rather than introduce a static number of each different type of ants, the overall number is kept equal to the order of the QAP instances, while the type of each ant is determined probabilistically based on the different weights. This allows more flexible mixing of the different types, reduces the number of static parameters, and is more in keeping with the probabilistic nature of Ant Colony Optimization. This change should have little effect on the results, as over the course of many trials, any inconsistencies should average out. 15

17 2.2 Test cases For this paper, the following test cases were chosen: Nugent (15, 20, 30), Elshafei (19), Krarup (30a), and Taillard (60a, 100a). All test cases were taken from the online version of the QAPLIB library [Burkard et al., 1997]. The Nugent, Elshafei, and Krarup cases were chosen to match [Maniezzo and Colorni, 1999]. The Taillard cases were chosen to explore the effectiveness of elitism and LBT against larger instances of QAP. For Nugent (15, 20, 30) Elshafei (19) Krarup (30a) Taillard (60a) we ran 50 trials of 10,000 iterations for both Elitism and LBT. For Taillard (100a) we ran 10 trials of 5,000 iterations for both Elitism and LBT. 16

18 2.3 Parameters The following parameters were used (N is the order of the QAP instance): Number of ants: N Visibility linearity: 1 Initial pheromone: 1 Minimum pheromone: 1 Maximum pheromone: 3N Pheromone decay rate: 0.6 Pheromone linearity: 1 Elitist weight: 1/3N (Elitist experiments only) LBT weight: N (LBT experiments only) 17

19 3 Analysis 3.1 Results For each of the QAP problem instances except Taillard (100a) there are two graphs. The graphs entitled 'The cost difference between LBT and Elitist Algorithms' contain curves representing the per-iteration difference between the LBT and Elitist results for four values: the cost of the best tour found yet; the cost of the best tour found that iteration; the average cost of the tours found that iteration; and the standard deviation of the costs of tours found that iteration. All values are LBT less Elitist. For the first three, a lower value means a better solution is found, since QAP is a minimization problem. For the standard deviation, a higher value means that the LBT is searching a wider area. The graphs entitled 'The cost difference between best tour and optimum' contain curves representing the per-iteration percentage difference between the best tour found by LBT and the optimum tour listed in QAPLIB; and the best tour found by Elitism and the optimum tour listed in QAPLIB. The values are relative to the optimum value itself. As the curves approach zero, they approach the optimum value. In both cases, a lower value is better. There is no graph for Taillard (100a) since no optimum is listed for that problem instance in QAPLIB. 18

20 19

21 20

22 21

23 22

24 23

25 24

26 For each of Nugent (15, 20, 30) and Krarup (30), LBT clearly outperforms Elitism. Elitism finds better solutions during the early part, but LBT quickly races past, finding both better tours overall, as well as better tours every iteration, and on average better tours. The better average tour value combined with the higher standard deviation indicates that LBT is searching both a wider scope and and is directed towards better areas. Additionally, LBT seems to be gaining ground even towards the end, indicating that Elitism will not catch up. Both algorithms perform progressively worse as the problem instances grow larger. This is to be expected since a static number of iterations is used. 25

27 26

28 In Elshafei (19), LBT outperforms Elitism by a small margin. In fact, neither algorithm performs particularly well. This problem instance contains much higher distance and flow values than the earlier instances, as well as a higher deviation and a far higher proportion of zero values. While both algorithms are adversely affected, it bears investigation to see which factors caused LBT's more rapid deterioration. 27

29 28

30 29

31 In Taillard (60a and 100a), Elitism quickly finds a local optima and races ahead of LBT. From that point, the two algorithms perform almost identically. The standard deviations are the same, and the difference between best tours and average tours appears to remain fairly stable. Both algorithms appear to be searching almost randomly, clearly the pheromone trails are not being maintained at a high enough pace to keep up with this much larger problem instance. 30

32 3.2 Further Investigation The first experiments showed considerable promise for LBT with the smaller instances, but little success with the larger instances. To see if this was due to poor parameter choices, we ran a few more experiments. For Taillard (60a) we ran two more experiments of iteration trials. In the first (tai60-a), the decay rate was lower to 0.4 from 0.6, and in the second (tai60-b) the decay rate was lowered to 0.2. The theory behind these changes is that the pheromone paths were decaying too quickly to be effectively used by both algorithms. 31

33 32

34 33

35 As the decay rate is lowered, the Elitist algorithm approximately maintains its performance while LBT improves considerably, beating Elitism by a small margin in the first experiment (tai60-a) and by a large margin in the second (tai60-b). Clearly the lessening of the decay rate (and strengthening of the pheromone trails) benefits LBT. Three more experiments were run on Taillard (100a) to see if lessening the decay rate would have a similar impact. For each, 5 trials of 5000 iterations were run.decay rates of 0.3, 0.1, and 0.01 respectively were used in tai100-a, tai100-b, and tai100-c. 34

36 35

37 36

38 In tai100-a (Decay rate of 0.3), the same behavior is shown as before. Elitism jumps out to an early lead, but both algorithms perform similarly after that, with almost no difference in standard deviation. In tai100-b (Decay rate of 0.1), LBT outperforms Elitism. In tai100-c (Decay rate of 0.01), both algorithms perform equally. For Taillard (100a), a decay rate of 0.1 seems to be a sweet spot for LBT. To more clearly investigate the impact of the pheromone parameters on the performance of LBT and Elitism in Taillard (100a), we ran two more experiments of iteration trials. In tai100-d, the decay rate was set at 0.6, and the pheromone linearity 2.0. In 37

39 tai100-e, the values were 0.6 and 3.0. While the trails would evaporate faster, they no longer need to be as high to influence the ants. 38

40 In tai100-d, Elitism quickly finds a superior path, but LBT seems to be making some headway towards the end. The standard deviation for LBT is less than Elitism, so it is searching a far narrower area. In tai100-e, Elitism beats LBT hands down. From these results, it appears that the decay rate is a far more consistent tool for enhancing LBT performance than the pheromone linearity. 39

41 4 Discussion 4.1 Conclusions The experiments run show a clear superiority for Ant System with Local Best Tours over the standard Elitist Ant System, when applied to the Quadratic Assignment Problem. LBT finds better tours overall, by iteration, and on average. It searches a wider and better spaces, and overall approaches the optimum far faster than Elitism. When the other benefits of using local information instead of global information are added in, LBT wins decisively over Elitism. Some of the results do raise concerns, however. The relatively poor performance of LBT when applied to Elshafei (19), as well as the necessary decay rate changes for Taillard (60a and 100a) indicates that all is not perfect. Clearly some investigation must be done to discover the best relationship between the Ant System parameters and the problem instance. Overall, LBT appears to be worthy of further investigation. 40

42 4.2 Future work The performance of LBT appears to be affected not only by the size of the problem instances, but by the data. More work needs to be done to investigate the effects of the size and deviation of the problem data, as well as the occurance of zero values. While problem sizes continue to grow, the Ant System parameters must keep up. As clearly demonstrated, we cannot use static values with ever larger problems. There must be a relationship between the parameters and the problem size and data that is more effective than trial by error. Perhaps a formula based on the problem can be used to determine the parameters, or the parameters might adjust dynamically when performance is poor. Application of LBT towards other assymetric NP-hard problems may shed more light on these issues. Would it be possible to mix classic, elitist, and lbt ants to positive effect? Perhaps their strengths may offset each others weakness and produce a better overall result. Some investigation into this may prove beneficial. 41

43 References Bonabeau E., Dorigo M., and Theraulaz G. Swarm Intelligence From Natural to Artificial Systems. Oxford University Press, New York NY, Bukard R.E. Quadratic Assignment Problems, European Journal of Operational Research, 15, , 1984 Burkard R.E., Cela S.E., Karisch S.E. and Rendl F. QAPLIB - A Quadratic Assignment Problem Library. Journal of Global Optimization 10: , Dorigo M., Maniezzo V. and Colorni A. The Ant System: Optimization by a Colony of Cooperating Agents. IEEE Transactions on Systems, Man, and Cybernetics-Part B, 26(1):29-41, Dorigo M., and Gambardella L.M. Ant Colonies for the travelling salesman problem Maniezzo V. and Colorni A. The Ant System applied to the Quadratic Assignment Problem. IEEE Transactions on Knowledge and Data Engineering, September White T., Kaegi S. and Oda T. Revisiting Elitism in Ant Colony Optimization. GECCO

44 Winston P.H. Artificial Intelligence. Addison-Wesley Pub Co,