The Knapsack Problem

Transcription

1 The Knapsack Problem Main Project in Computer Technology Written by Helge Andersen, Ann Merete Børseth and Jenny Helen Nytun Teaching supervisor Noureddine Bouhmala Bakkenteigen, June 2003

2 Preface This project is the closure of our education for Bachelor of Science in Computer Technology at Høgskolen i Vestfold. The reason why we chose the Knapsack problem was that we took a course in optimisation and found this very interesting. We wanted to learn more about this, so we where very pleased to find out that we could choose this as our main project. The Knapsack Problem is a typical optimisation problem. The goal is to fill a knapsack with objects that together make out the highest possible value. Our aim is to see if the new methods we create are better than the ones that already exist. This assignment has been in progress since January 2003 and will end in June The assignment workload is equivalent to five weighting. We believe this paper will be worth reading for those interested in optimisation and the knapsack problem. We would like to thank Dr. Noureddine Bouhmala for the project and helpful guidance trough it. Jenny Helen Nytun Ann Merete Børseth Helge Andersen 2

3 Table of content Preface... 2 Table of content Introduction Introduction Optimisation The knapsack problem Applications Description of problem Structures of the report Summary Existing methods Introduction Random MinWeight MaxWeight MinValue MaxValue Improving Exchange IME Summary Testing of existing methods Introduction About the experiments Results Summary New methods Introduction Kick Multiexchange by Value MEV Multi-Exchange by Weight MEW Summary Testing of the improving methods Introduction About the experiments Results Summary Future work Introduction Ideas to make the methods better Summary Conclusion

4 1 Introduction 1.1 Introduction This chapter gives the required background and introduction needed for the following chapters. You will be able to read about the wide aspect of optimisation, the specific optimisation problem known as the knapsack problem [1], and a description of the problem we are looking deeper into. 1.2 Optimisation Optimisation is the process by which the optimal solution to a problem is produced. In an industrial setting, problems routinely arise that require making the best possible decision. Managers must plan and schedule operations efficiently to reduce the cost. Similarly, engineers design new, more efficient, but also more expensive systems. Problems arising in several areas become more complex when managers or engineers are faced with multiple alternatives. They must select the alternative most likely to produce the desired result. And how can such a choice be made? It is in this arena that optimisation techniques become invaluable. It is not always possible to find the perfect solution to a problem. We sometimes have to accept a solution that is not optimal. The Knapsack Problem is an example of this. It has not yet been found a perfect solution to this problem. The aim of this project is to find a better solution than the solutions that already exists. 1.3 The knapsack problem Description The knapsack problem arises whenever there is resource allocation with financial constraints. Given a fixed budget, how do you select what things you should buy? Everything has a cost and value, so we seek the most value for a given cost. The typical formulation in practice is the 0/1 knapsack problem, where each item must be put entirely in the knapsack or not included at all. Objects cannot be broken up arbitrarily, so it is not fair taking one can of coke from a six-pack or opening the can to take just a sip. It is this 0/1 property that makes the knapsack problem hard, for a simple greedy algorithm finds the optimal selection whenever we are allowed to subdivide objects arbitrarily. Another way to look at this problem is a Robin Hood tale. Robin Hood went to prince John late at night. He had a knapsack that could hold up to 20 kilos. In the prince treasure chamber he saw the following objects: 2 gold statues, each worth 15 gold coins and weighting 10 kilos, 1 sack with beads worth 7 gold coins and weighting 5 kg, 1 sword worth 5 gold coins weighting 3kg, and 4 silver bars worth 3 gold coins and weighting 1 kg each (figure 1.1). Robin Hood took 1 statue, 1 sack with beads and 4 silver bars (figure 1.3). When he came back to the camp, Little John got furious. He could not see why Robin had left one statue in the chamber, the statues were after all the most valuable items (figure 1.2). Robin then explained that if he just had taken the statues, he would only get goods worth 30 gold coins. Now, however, he had goods worth 34 gold coins. 4

5 2 statues Sack with beads Sword 4 silver bars 15 gold coins each 7 gold coins 5 gold coins 3 gold coins each 10 kilos each 5 kilos 3 kilos 1 kilo each Figure 1.1: available items in treasure chamber. Statue Statue Total Statue 1 sack with beads 4 silver bars Total 15 gold 15 gold 30 gold 15 gold 34 gold 7 gold coins 12 gold coins coins coins coins coins coins 10 kilos 10 kilos 20 kilos 10 kilo 5 kilos 4 kilos 19 kilos Figure 1.2: Little John s choice Figure 1.3: Robin Hood s choice The problem here is to fill the knapsack with goods to get the highest value, and not exceed the knapsack s weight limit. When there are as few objects as in Robin Hood s case, there really is not much a problem. But when there are hundreds of items, you must think quite long to get the best solution. The obvious solutions of taking as many as possible of the objects with the maximum value, or those with the maximum value per unit weight, do not necessarily give the optimal solution Formal Description Given a set of items, each with a cost and a value, determine the number of each item to include in a collection so that the total cost is less than some given cost and, the total value is as large as possible. Mathematically, the knapsack problem can be stated as: maximize v j (d j ) subject to d j w j W where d j = number of the j th object included; v j (d j ) = utility of d j objects of the j th type of object; w j = weight of the j th object; and W = capacity of the knapsack. 1.4 Applications Here are some of the real life uses of the knapsack problem. A shipping company wants an optimal load in a truck. The truck may load a certain amount of weight. Which goods must be loaded to obtain the highest value? The ongoing goal of any NHL team's management, particularly during the expansion draft, is to find the subset of players capable (at least!) of playing in Stanley Cup Finals. Each player has points to describe their skill level. The management wants the highest possible points in their team, but without exceeding the limited budget. These are just a few examples of what the 0/1 knapsack problem may be used in, it may also be used in financial and industrial management, computer and Internet security among others. 5

6 1.5 Description of problem Some knapsack problems allow us to use one item many times, and even half of that item (figure 1.4). But this does not seem to be a major problem to solve. So we have chosen to use the 0/1-problem (figure 1.5), meaning that we can only use one item once, and we must take the entire item, not just a part of it. Figure 1.4 Knapsack problem without the 0/1-constraints. Figure 1.5 0/1-knapsack problem. 6

7 1.6 Structures of the report Chapter 2 Existing methods. In this chapter you will be able to read about 6 methods, including one improvement method, which already exist. Chapter 3 Testing of existing methods. This chapter shows which of the methods from chapter 2 that is the best. Chapter 4 New methods. This chapter tells about the three new methods we have created. Chapter 5 Testing of the improvement methods This is the experiment that compares all methods used in this project. We will show you which method is the best, and how much better it is. Chapter 6 Future work. In this chapter we will think a little bit further. Can any of our new improve methods be better? Chapter 7 Conclusions This chapter tells our conclusion of this project. Chapter 8 References Appendix A This is the source code of our program, including all methods and a test program. Appendix B This is some graphs for the especially interested, and all the numbers that all graphs in this report are based on. 1.7 Summary Optimisation-problems are quite normal in the industrial world, and the knapsack problem is one of the problems. Whenever there is a need to select some items with a value and a cost, and you cannot choose all, it is a knapsack problem. This problem is common in sports and transportation-businesses. We have chosen to look at the 0/1-side of the problem, meaning that you cannot put parts of an item in a sack but only whole ones. We will try to get the best possible solution in these cases: 100 items with total weight of 500, 200 items with total weight of 1000 and 300 items with total weight of

8 2 Existing methods 2.1 Introduction This chapter explains the idea behind the existing methods, and how they work. The methods here are 5 initial methods, and one improve method. 2.2 Random Short description and pseudocode Begin with a random item and keep adding random items into the knapsack until the weight reach MAXWEIGHT. Start loop Pick random object Check if there is room for item in sack Put item in sack if there is room End loop (if not picked any new items in X attempts) Explanation The method starts by choosing a random item to add to the sack. The method continues to do this until there is no more room in the sack, or until it has tried to add items 100 times without success. If you are really lucky, this method may give you the optimal solution. This is, however, extremely unlikely to happen. When we use this method we will get a variety of weight and value, and this may be a good starting point for an improve method. That is why we wanted to include this method in this project. 2.3 MinWeight Short description and pseudocode Sort the items by increasing weight. Begin with the lightest item first and keep adding items into the knapsack until the weight reach MAXWEIGHT. This is not an optimal solution if there are heavier, more valuable items. Start loop Select lightest item not yet picked Check if there is room for item in sack Put item in sack if there is room End loop (no room for more items) 8

9 2.3.2 Explanation If Robin Hood wanted to have the most items with him, this is the approach for him. This method only considers the weight of the items it selects. The method simply starts by looking for the lightest item that is not in the sack, and put it in if there is room. If it is no room for this item, it is not necessary to look for any new items since they are equal or heavier than the one that could not fit in. This method would rather choose two items with a total weight of 2 and total value of 500, than one object with the weight of 2 and value of The method will give us many items in the sack, and may not be such a bad solution. In spite of the problem mentioned over. 2.4 MaxWeight Short description and pseudocode Sort the items by decreasing weight. Begin with the heaviest item first and keep adding items into the knapsack until the weight reach MAXWEIGHT. Start loop Select heaviest item not yet picked Check if there is room for item in sack Put item in sack if there is room End loop (no room left for the heaviest item left) Explanation This method is quite similar to the MinWeight method, but it would rather choose the heaviest items than the light ones. It will not consider the value of the item, it is just interested in the heaviest ones. This is not a good way to start the selection. 2.5 MinValue Short description and pseudocode This insertion method selects the items with lowest value, and puts them in the knapsack. Start loop Select least valuable item not yet picked Check if there is room for item in sack Put item in sack if there is room End loop (no room for more items) 9

10 2.5.2 Explanation The best thing with this method is that it leaves room for improvement. This method chooses the items with the lowest value, and put them into the sack. It does not consider the weight of the item, just makes sure that the value is low. This method will rather choose something with a value of 500, then a value of 2000, regardless of weight. Obviously, this is not a good method. But it gives us a variety of items, considering weight. This may come in handy when it is time to improve. 2.6 MaxValue Short description and pseudocode MaxValue selects the items with highest value, and puts them in the knapsack. Start loop Select most valuable item not yet picked Check if there is room for item in sack Put item in sack if there is room End loop (no room for more items) Explanation This is the method Little John would prefer. By only picking the items with a high value you will get quite good results, but it will not be optimal. This method does just that, picks the highest valued items. This method will rather choose one item with weight 8 kg and value 2000, than two items worth 1999 and weighting 4 kg each. Which of course is a better solution. 2.7 Improving Exchange IME Short description and pseudocode This is an improving method that takes the initial solution and tries to improve it. This is done by randomly exchanging one and one item, if the weight does not exceed the limit, until there not have been any changes for some iterations. Start loop Select random item1 from knapsack Select random item2 from items not picked Check if knapsack will get higher value if item1 and item2 are swapped If so: Check if knapsack gets to much weight if we swap objects Else: swap object1 and object2 End loop (not swapped any items for X attempts) 10

11 2.7.2 Explanation Once we have an initial solution, we may try to make it better. The best known method is to randomly exchange one and one item and look for higher total value. This method will give a better value in the knapsack, but maybe not the optimal solution. This method will also be known as the IME-method. 2.8 Summary This chapter reviewed 6 methods. 5 insertion methods, which only goal is to give a correct solution, and one improve method, which goal is to improve the solution given by an insertion method. The methods are: Random: Insertion method that randomly pick one item and add it to the sack. MinWeight: Insertion method that picks the item with the lowest weight, and put it into the sack. MaxWeight: Insertion method that picks the item with the highest weight, and put it into the sack. MinValue: Insertion method that picks the item with the lowest value, and put it into the sack MaxValue: Insertion method that picks the item with the highest value, and put it into the sack Improving Exchange: Improve method that randomly pick one item from the sack, and exchanges it with one other. This method is also called IME. 11

12 3 Testing of existing methods 3.1 Introduction This chapter shows the result of a comparing of the insertion methods and the Improving Exchange method that already existed. 3.2 About the experiments We have chosen to look closer into the 0/1 knapsack problem. We generate a given number of items, and choose which of the items that should be placed in the knapsack for the best solution (figure 3.1). Item Weight Value Figure 3.1 Example of how the items may appear The number of items will be 100, 200 and 300, and the maximum weight will accordingly be 500, 1000 and After creating the items, we must put some of them into the sack. We will make 5 different methods, insertion methods, to select which items to put into the sack. The approaches will be called Random, randomly chooses which items to insert, MaxValue, inserts items with highest value first, MinValue, inserts items with lowest value first, MaxWeight, inserts heaviest items first, and MinWeight, inserts lightest items first. After using one of these approaches, the tables of the knapsack and what is left outside may look like this (figure 3.2 and 3.3). Item Weight Value Figure 3.2: How the knapsack may look after an insertion method. Item Weight Value Figure 3.3: Items outside the sack after an insertion method. The value of the sack is in this case 3938, and the weight 32. Let us say that the maximum weight allowed in this example is 35. Is this a good solution? Considering that item 7 has a value of 4700, obviously not. In this experiment we will look at all insertion methods and the Improving Exchange method, and find which one that gives the best solution. The experiment is done as follows. We started the program with maxitems different for each experiment, and start all insertion methods. This is because we want to compare the methods when they are working on the same items. In the first experiment we sat maxitems to 100, meaning that there are 100 items we may choose from, and the total weight of all items to 500. Then we sat the capacity of the 12

13 sack to 10% of the total weight, meaning 50. When this was done, we tried all insertion methods with these numbers and the Improving Exchange method on them all. Next we increased the capacity of the sack to hold 20, 40, 60 and 80% of total weight and did the same routine as with 10%. Then we created 100 new items and went on it again, five times we did this. The same was done with 200 items with total weight of 1000 and 300 items with total weight of The results are presented in chapter 3.3, and you will find all numbers from the experiment in appendix B. 3.3 Results In these graphs you now will see, the y-axis represents the value of the sack, and the x-axis represents the weight capacity of the knapsack. TotalWeight 500, MaxItems Value Random MinWeight MaxWeight MinValue MaxValue % 20 % 40 % 60 % 80 % Capacity of knapsack in percentage of the total weight Figure 3.4 shows how the value increases for the different methods when we increase the capacity of the knapsack. In this figure we work with 100 items with a total weight of

14 TotalWeight 1000, MaxItems Value Random MinWeight MaxWeight MinValue MaxValue % 20 % 40 % 60 % 80 % Capacity of knapsack in percentage of the total weight Figure 3.5 shows how the value increases for the different methods when we increase the capacity of the knapsack. In this figure we work with 200 items with a total weight of TotalWeight 1500, MaxItems Value Random MinWeight MaxWeight MinValue MaxValue % 20 % 40 % 60 % 80 % Capacity of knapsack in percentage of the total weight Figure 3.6 shows how the value increases for the different methods when we increase the capacity of the knapsack. In this figure we work with 300 items with a total weight of

15 The 3 first graphs show us the value of the sack after the insertion methods. First graph (figure 3.4) has 100 items with a total weight of 500. Figure 3.5 has 200 items with a total weight of 1000, and figure 3.6 has 300 items with a total weight of These graphs are practically similar to each other, so we will talk about them as one. The best insertion method is MinWeight, at least when the knapsack has a capacity that is less than approximately 50%. After that, it is the MaxValue that becomes the best. This is not a surprise, since an item with weight 1 may be more valuable than an item with weight of 8. But why is not MaxValue the best? Because some of the light items may have a very high value, the result of this fraction: value/weight, is better for the light items. And since there is not much room in the sack, this solution is the best. But when the sack increases, there becomes room for more of the most valuable items. % better compared to MinValue % , ,60 241, ,17 332,18 802,41 70,90 629,75 121,72 246,87 23,89 242,46 59,01 102,95-2,12 114,94 25,04 39,57-12,12 47,93 Random MinWeight MaxWeight MaxValue % 20 % 40 % 60 % 80 % Capacity of the sack, in percentage of the total weigt Figure 3.7 shows how many percent each insertion method is better than the MinValue method. From figure 3.7 we see that when the capacity of the sack is 10% of maximum weight, MinWeight is 2048% better than Maxweight. MinWeight is better than MaxWeigth all the way, with a 40% better score as the worst result. At 10%, MinWeight is almost twice as good as MaxValue, which is 1203% better than MaxWeight. But the differences between these two, MaxValue and MinWeight, are almost nothing when the capacity is 40%. At 60 % there have been a change. MaxValue is 12 percentage points better than MinWeight, and at 80% it is 8 percentage points better. 15

16 Improving Exchange TotalWeight 500, MaxItems Value Random + IME MinWeight + IME MaxWeight + IME MinValue + IME MaxValue + IME % 20 % 40 % 60 % 80 % Capacity of knapsack in percentage of the total weight Figure 3.8 shows the increase of value for the Improving Exchange method, on all insertion methods, when we increase the capacity of the sack. In this figure we work with 100 items with a total weight of 500. Improving Exchange TotalWeight 1000, MaxItems Value Random + IME MinWeight + IME MaxWeight + IME MinValue + IME MaxValue + IME % 20 % 40 % 60 % 80 % Capacity of knapsack in percentage of the total weight Figure 3.9 shows the increase of value for the Improving Exchange method, on all insertion methods, when we increase the capacity of the sack. In this figure we work with 200 items with a total weight of

17 Improving Exchange TotalWeight 1500, MaxItems Value Random + IME MinWeight + IME MaxWeight + IME MinValue + IME MaxValue + IME % 20 % 40 % 60 % 80 % Capacity of knapsack in percentage of the total weight Figure 3.10 shows the increase of value for the Improving Exchange method, on all insertion methods, when we increase the capacity of the sack. In this figure we work with 300 items with a total weight of Even when we use the IME-method, the solution with MinWeight as insertion method is the best for small capacity sacks. The same as it was without the improvement. In the figure 3.8, 3.9 and 3.10 we can see that MinWeight is much better when the knapsack holds 10% and 20% of the total weight, and a little better at 40%. But after that, Random-, MinValue-, and MaxValue- with IME is better than MinWeight, and quite similar. 17

18 Improving Exchange % better compared to MaxWeight + IME , % ,23 88,23 84,12 83,14 131,88 80,15 82,99 59,36 65,80 58,68 59,31 40,59 37,26 40,90 42,43 22,06 18,36 21,79 22,42 Random + IME MinWeight + IME MinValue + IME MaxValue + IME 0 10 % 20 % 40 % 60 % 80 % Capacity of the sack, in percentage of the total weigt Figure 3.11 shows how many percent better Improving Exchange method is when it works on all insertion methods, compared to when it worked on the MinValue method. When the Improving Exchange method tries to improve a sack created with the MaxWeightmethod, the result is no good compared to the others. Figure 3.11 shows how many percent better the result is with the other insertion methods than when we used MaxWeight. When the capacity of the sack is at 10% of the total weight of all items, IME is more than twice as good when it improves MinWeight than any other. At 20% capacity it is almost twice as good. But after that there is not much difference if IME works on any of the four insertion methods Random, MinWeight, MinValue or MaxValue. At 80% capacity, there is less than 4 percentage points between them. Based on these statistics, we would say that we get the best results all over by using the Improving Exchange method on the MinWeight insertion method. That is this far of course, because in the next chapter we will introduce the new improve methods we have created. 3.4 Summary This chapter have shown the result of our first experiments. The MinWeight insertion method was by far the best, until the limit of the sack was approximately 50% of the total weight of the knapsack. After that the MaxValue method became the best, but only a few percent better than the MinWeight method. When we used the Improving Exchange method, there was the same development. When we used the IME on Minweight, it gave the best solution until about 50%. After that it was not much difference, but MaxValue gave the best result. 18

19 4 New methods. 4.1 Introduction This chapter is about the three new methods we have developed. They are all improvement methods. These new methods have two tables, one for the knapsack, and one for the outside. We have done this so we can use all improve method on the same results after an insertion method. 4.2 Kick Short description and pseudocode This improvement method does the improvement in two steps. First step is to change a random number of items from the knapsack with the same amount from what s not in the sack. This we call a kick [2]. The kick does not care if the exchange will give the knapsack a lesser value, or its limit of weight is exceeded. The next step is to randomly swap one item from the sack with one from outside. Improve initial solution by swapping one and one item Loop start Swap (kick) a random number of items regardless of weight and value Improve by swapping one and one item (make sure that weight doesn t get heavier than it was after the kick) ( this is done for example 500 times) End loop (number of iterations) If (weight of knapsack is more than MAXWEIGHT allowed) remove items with least value until it is not overloaded If (weight of knapsack is less than MAXWEIGHT) put items with best value into sack until it is full, or as full as it gets Explanation The idea of this improvement method is that it sometimes must get worse before it gets better. After having done an initial solution, one may only get to a certain value with the Improving Exchange method. This since there generally will be traded items with other items that have the same or less weight. It is first when you have traded in one item with one with lesser weight you may choose a heavier item. But with this kick you may get some brand new items in the sack, and then there will be possible to get new improvements that may be better or worse. Of course we only save the better solutions, and tries to improve them even more. 19

20 We start this method by using the Improving Exchange method. After doing that, we randomly pick a number of items from each place and make them change places, without concern of weight or value. This sort of exchange may give us worse solutions both concerning weight and value, but since there now are new items in the sack, the improvement done by exchanging one and one item may give a higher value than before. value after an IME Figure 4.1 value after a kick If each top of the wave in figure 4.1 is the value after the Improving Exchange method, and each valley is the value after the kick, this is how the variation of the value may be with the kick method. After doing the kick, the weight might be much higher than it is supposed to. This does not matter, since we remove excess weight at the end. However, we do not ignore the weight. When we do the IME-method after the kick, we make sure that the weight doesn t get any higher than it was after the kick. When we first finished this method, and began experimenting with it, we completely ignored the weight. The results were not good, quite like the results from the MaxValue method. Since we filled the sack with high valued items, with no concern of weight, the sack became way overloaded. We sometimes had to remove half of the items. That is why we introduced the weight control, making sure that the weight didn t get unreasonable high. Of course we have to remove excess weight at the end. This is done by removing the least valuable items in the sack until the weight is lower than maximum weight allowed. The weight of the sack may now be less than it could be, so we may have to fill the sack again by inserting the items with highest value and a weight that will not make the total weight exceed the allowed limit. 20

21 4.3 Multiexchange by Value MEV Short description and pseudocode This method picks out a random number of objects from both inside and outside the knapsack. It compares the objects by value and swaps them if the value of the items from outside the knapsack is higher than the value of the items from inside the knapsack. Choose a random number of nodes to be swapped. Choose randomly the numbers to swap from both tables: Loop start Choose the random numbers to swap from the in table and the out table. Find out which objects to swap: Loop start If the new value is larger than the old, mark the new value k. End loop Swaps the chosen objects: Swaps objects from 1 to k. End loop If (weight of knapsack is more than MAXWEIGHT) remove items with least value until it is not overloaded If (weight of knapsack is less than MAXWEIGHT) put items with best value into sack until it is full, or as full as it gets Explanation This improve method focuses on value first. We choose a random number of nodes from both inside and outside of the knapsack that we put into two tables. Then we start comparing the objects. We start with the first node in each table and calculate the difference. If the difference is not bigger than 0 we move on to the next node. If it is bigger, we mark this node and then move on to the next. We calculate the difference and add it to the difference of the first two nodes. If the difference is better than what we already have, we mark it and continue to the next nodes. If it is not better, we go on without marking it. We continue doing this until we have been through the whole table. When we have done that, we swap the items from the first node to the marked nodes in the table. 21

22 We will now take you thru this function step by step. The start difference = OUT IN Diff = = = = = = 0 Node 1 We have the value 20 outside the knapsack and 30 inside the knapsack. The difference is 10. This is not better than 0, so we move on to the next nodes in the table. Node 2 The difference here is 0. This is not better than 0 so we move on to the next nodes in the table. Node 3 The difference here is 5. This is better than 0. We mark this place so we can go back to it later and we move on to the next nodes in the table. Node 4 The difference here is 15. This is better than 5. We mark this place so we can go back to it later and we move on to the next node in the tables. Node 5 The difference here is 12. That is not better than 15, so we move on to the next nodes. Node 6 The difference here is 5. That is not better than 15, so we move on to the next nodes. Node 7 The difference here is 0. This is not better than 15. We have now been through all the nodes. We now choose node 1 4 in the out- table and swap them with the nodes 1 4 in the intable The knapsack has now a higher value than it had before, but the weight may be too high. If the weight is higher than maxweight, we have to remove some items from the knapsack. We start by removing the object with the lowest value in the knapsack. If the weight still is too high, we remove the object with the lowest value that is left in the knapsack. We continue doing this until the weight is equal to or lower than maxweight. 22

23 4.4 Multi-Exchange by Weight MEW Short description and pseudocode The idea of this improvement method is to exchange many items at once. When we only change one and one item, there is not much room for variations in the weight. But when we exchange many items at once, it does not matter if the first item we put in has a higher weight than the one we take out. As long as the total weight of all items we put in the knapsack don t exceed the maximum weight, we are free to try out more combinations of items. Loop Choose x items from sack and x items from outside Loop Check for increased value if we exchange items Deselect last items if no increase in value End loop (increase in value) If ( weight not exceeds its boundaries) do the exchange End loop (iterations) If (weight of knapsack is more than MAXWEIGHT) remove items with least value until it is not overloaded If (weight of knapsack is less than MAXWEIGHT) put items with best value into sack until it is full, or as full as it gets Explanation When we swap one and one item, there cannot be much variation in weight before the knapsack exceeds its boundaries. But if we try to swap many items at once, it does not matter if some items cause the knapsack to be overloaded. But in the end, the total weight off all new items that are put into the sack must not exceed the weight limit. An item with weight 3 and value 15 may now be swapped with an item with a weight 7 and a value 96, if the total weight with all other items going into the sack allows it. First the program decides how many items it wants to swap. This is done randomly, but it will not choose to swap more items than there is in the sack or on the outside. Once it has selected the items it wants to exchange, it is time to check the weight. If the weight, both those already inside the knapsack and those who want to get inside, exceeds the limit of the sack, the method deselects the last items chosen. It will continue to check the weight, until it is satisfied. If the weight is accepted, and there still are items left to exchange, it will look at the value. If the items will give the knapsack a higher value, it is accepted into the sack. Otherwise it will put all items back where they came from. 23

24 4.5 Summary This chapter introduced three new improve methods. Kick: This improve method uses the IME-method and a kick. IME exchanges one and one item, looking both for higher value and for the weight to not get unreasonable high. The kick randomly exchanges a random number of items regardless of weight and value. At the end, the weight of the sack is within the limit. Multiexchange by Weight MEW: This improve method exchanges a random number of items and are making sure there is an increase in value, and that the weight not exceeds its boundaries. Multiexchange by Value MEV: This improve method exchanges a random number of items regardless of weight. Its only concern is to increase the value. At the end, the weight of the sack is within the limit. 24

25 5 Testing of the improving methods 5.1 Introduction This chapter compares all improve methods. 5.2 About the experiments This experiment is quite similar to the experiment in chapter 3. In this experiment we have tried all improve methods on all insertion methods. This to see which method that gives the best result concerning value. We have tried three different settings for this experiment: 100 items with a total weight of 500, 200 items with a total weight of 1000, and 300 items with a total weight of For each setting, the knapsack is set to hold 10%, 20%, 40%, 60% and 80% of the total weight of all items. 5.3 Results These graphs compare all improvement methods on all insertion methods, and show how many percent better they are than the Improving Exchange method on the MaxWeight method. In appendix B there is more graphs for those interested. Those graphs tell more about each of the new methods, but do not compare them with each other. % ,23 220,10 88,23 84,12 107,76 220,36 189,62 % better compared to MaxWeight + IME 97,08 100,07 52,26 92,09 170,60 71,50 66,54 109,83 10% Capacity of the sack, in percentage of the total w eigt 231,70 97,58 100,24 98,86 Random + IME MinWeight + IME MinValue + IME MaxValue + IME Random + MEW MinWeight + MEW MaxWeight + MEW MinValue + MEW MaxValue + MEW Random + MEV MinWeight + MEV MaxWeight + MEV MinValue + MEV MaxValue + MEV Random + Kick MinWeight + Kick MaxWeight + Kick MinValue + Kick MaxValue + Kick Figure 5.1 shows how many percent better they are when the capacity of the sack is set to 10% of the total weight of all items. 25

26 % better compared to MaxWeight + IME ,88 132,77 123,18 116,41 136,75 Random + IME MinWeight + IME MinValue + IME MaxValue + IME Random + MEW MinWeight + MEW 100 % ,14 80,15 82,99 80,48 73,31 87,39 55,90 77,23 64,91 51,07 92,19 83,81 90,73 88,20 MaxWeight + MEW MinValue + MEW MaxValue + MEW Random + MEV MinWeight + MEV MaxWeight + MEV MinValue + MEV MaxValue + MEV 40 Random + Kick MinWeight + Kick 20 MaxWeight + Kick MinValue + Kick MaxValue + Kick 0 20% Capacity of the sack, in percentage of the total weigt Figure 5.2 shows how many percent better they are when the capacity of the sack is set to 20% of the total weight of all items. When the capacity of the sack is set to 10% and 20% of the total weight of all items, there is not much difference between the IME, MEW and the Kick when they improve the initial solution made by MinWeight. But Kick is better (11% on 10% capacity, and 4% on 20%). At 40% capacity (figure 5.3), there are many methods that are pretty much like each other, Kick on MinWeight is however still the best. Though it is only 3 percentage points better than the next one, the IME on MinWeight. 26

27 % better compared to MaxWeight + IME % ,36 65,80 58,68 59,31 45,67 65,86 65,43 46,19 62,42 31,10 52,47 59,63 31,78 35,80 65,20 40% Capacity of the sack, in percentage of the total weigt 68,51 60,36 64,50 61,44 Random + IME MinWeight + IME MinValue + IME MaxValue + IME Random + MEW MinWeight + MEW MaxWeight + MEW MinValue + MEW MaxValue + MEW Random + MEV MinWeight + MEV MaxWeight + MEV MinValue + MEV MaxValue + MEV Random + Kick MinWeight + Kick MaxWeight + Kick MinValue + Kick MaxValue + Kick Figure 5.3 shows how many percent better they are when the capacity of the sack is set to 40% of the total weight of all items % ,59 37,26 40,90 42,43 31,19 37,82 38,33 % better compared to MaxWeight + IME 32,32 43,55 24,22 37,16 35,23 24,01 23,89 44,06 38,97 60% Capacity of the sack, in percentage of the total w eigt 43,44 44,88 43,49 Random + IME MinWeight + IME MinValue + IME MaxValue + IME Random + MEW MinWeight + MEW MaxWeight + MEW MinValue + MEW MaxValue + MEW Random + MEV MinWeight + MEV MaxWeight + MEV MinValue + MEV MaxValue + MEV Random + Kick MinWeight + Kick MaxWeight + Kick MinValue + Kick MaxValue + Kick Figure 5.4 shows how many percent better they are when the capacity of the sack is set to 60% of the total weight of all items. At 60% capacity, there has been a little change. Kick is still the best, but now on the MinValue insertion method. However there are 5 other methods within 2,5 percentage points below the value of Kick on MinValue. Three of those are the Kick method on other insertion methods, one is MEW on MaxValue, and the last is IME on MaxValue. 27

28 % ,06 18,36 21,79 22,42 18,99 18,51 17,30 % better compared to MaxWeight + IME 17,86 22,79 12,54 17,91 16,86 13,23 12,79 22,63 19,38 80% Capacity of the sack, in percentage of the total w eigt 22,88 22,52 22,84 Random + IME MinWeight + IME MinValue + IME MaxValue + IME Random + MEW MinWeight + MEW MaxWeight + MEW MinValue + MEW MaxValue + MEW Random + MEV MinWeight + MEV MaxWeight + MEV MinValue + MEV MaxValue + MEV Random + Kick MinWeight + Kick MaxWeight + Kick MinValue + Kick MaxValue + Kick Figure 5.5 shows how many percent better they are when the capacity of the sack is set to 80% of the total weight of all items. At 80% capacity kick once again has the best score with almost 23%, but this time on the MaxWeight insertion method. Now there are seven methods within one percentage point, four of them are Kick on other insertions, IME has two and MEW has one. These experiments indicate that the Kick method is the best, at least when we only consider value. 5.4 Summary In this chapter we found out that Kick is the best improvement method, although sometimes the difference was not much. We also saw that the result depended on the initial solution. 28

29 6 Future work 6.1 Introduction In this chapter we look a bit beyond our project, and try to bring some thoughts about better algorithms for the knapsack problem. 6.2 Ideas to make the methods better Kick method When we do the kick, we could try to swap a different number of items from the sack and from outside the sack. There should be a limit on how many more/less items we should take from one part than we took from the first part. This should be done to prevent us from selecting one item from the knapsack and all of the items outside the sack MEW and MEV When we have chosen those to exchange, we must check that the weight doesn t exceed its limit if we perform the exchange. If it does, we should deselect one item from each stack, and check again. It would be worth checking if the result would have been better if we just removed one item from the stack that caused the weight to become to high. And of course the other way if the weight became much less than there was room for. 6.3 Summary Although we are satisfied with the results achieved with our new methods, we know they can be better. As we were working on this project we saw some possibilities for improvements and new ideas, both within the kick-, MEW- and MEV-methods. Mainly we think the results would be better if we exchanged a different number of items in the exchange-phases. 29

30 7 Conclusion This project, the Knapsack Problem, has been a very interesting project that we have learnt a great deal from. It has given us a deeper understanding of optimisation and the knapsack problem and, we have also learnt a lot about project work and working in a group. Our intentions when we started on this project where to make one new method that we hoped would be better than those that already exist. We ended up with three new methods, the Kick method, the Multiexchange by Weight method and the Multiexchange by Value method. As we can se from the experiments in chapter 5, one of the methods, the Kick method, proved to be the best. However it was not as good as we had hoped. We hope Dr. Bouhmala will be satisfied with our new methods, and may use them for further developments. 30