Optimizing Online Auction Bidding Strategies Using Genetic Programming

Transcription

1 Optimizing Online Auction Bidding Strategies Using Genetic Programming Ekaterina Smorodkina December 8, 2003 Abstract The research presented in this paper is concerned with creating optimal bidding strategies for online auctions At present time online auctions, that are very popular today, have not been given proper attention The strategies created using genetic programming were tested in an auction simulation environment in order to determine their strength This paper describes the experimental setup, the online auction simulation environment, and the genetic programming approach to developing bidding strategies It also gives a brief introduction to auction theory and genetic programming The last section of the paper discusses possible reasons for why the current setup failed to produce optimal bidding strategies Keywords Auction, English auction, online auction, ebay, genetic programming, evolutionary algorithms, agent, bidding strategy 1 Introduction 11 Overview In the past eight years online auctions have built a large person-to-person trading community on the Internet On any given day, there are millions of items sold through online auctions, such as ebay, Amazon, Yahoo, etc BusinessWeek online [6] reports that in 1999 ebay had 2 million items for sale with gross merchandise sales of $746 million, while Amazon had 16 million items for sale via online auction with gross merchandise sales of $610 million As stated in [1], in 2002 ebay members transacted $1487 billion in annualized gross merchandise sales The popularity of online auctions can be attributed to their ease of use, wide range of categories of items for sale, and availability 24 hours a day, seven days a week However, the biggest attraction of online auction is the possibility to purchase goods for a price significantly lower than the regular store prices Since online auctions play an important role in the trading world today, a better understanding of auction dynamics is necessary 1

2 As can be inferred from [3] and [5], evolutionary algorithms can be used to gain more knowledge about various types of auctions However researchers in computational economics have not given proper attention to online auctions, that are very popular today Online auctions resemble English auctions, also known as the open-outcry auctions or the ascending-price auctions As defined in [2] English auction is a type of sequential auction in which auctioneer directs participants to beat the current, standing bid New bids must increase the current bid by a predefined increment The auction ends when no participant is willing to outbid the current standing bid Then, the participant who placed the current bid is the winner and pays the amount bid In online auctions the current bid is also sequentially incremented However, online auctions differ from English auctions by an explicit time constraint Online auction participants have a limited time during which they can increment the current bid on an item Although English auctions have a time constraint imposed by the auctioneer, it is not as explicit and clear as in the case of online auctions Another major difference between online and English auctions is that the buyers and sellers can participate in online auctions from anywhere in the world, provided they have a computer that is connected to the Internet With English auctions the buyers are restricted to the physical location of where the auction takes place and the sellers typically have representatives that need to be present Finally, in online auctions the buyer is free to determine the amount by which to increment the current bid, as long as this amount is no less than the minimal bid increment imposed by the auction cite, while in English auctions the increment amount is typically fixed As mentioned in [3], the increase in the number of online auctions makes the consumers face the increase in the complexity of computing the right bid Another factor that increases the complexity of the consumer s decision making process is the increase in the number of items he desires to obtain Since most consumers have a restricted amount of money they are able to spend, bidding on multiple items requires a more careful bidding strategy This paper will focus on creating bidding strategies that are optimal for bidding on multiple items and explore the following questions: 1 Is it possible to come up with one all-purpose bidding strategy for various online auction scenarios? 2 How successful is genetic programming (GP) in evolving bidding strategies for online auctions? Various users may have different combinations of items on which they wish to bid, that is why the optimal bidding strategy must be adaptable to the various users requests However, since there are millions of auction listings for thousands of categories, it is safe to assume that the state of the auction, which is characterized by the types and the number of items listed, remains static Subsections 12, 13, and 14 give an overview of auction theory, computer agents, and genetic programming respectively Section 2 summarizes some previous work that has been done in creating autonomous bidding agents Section 3 2

3 describes the auction simulation used for this experiment Section 4 explains the methodology behind creating the bidding strategies Section 5 describes the experiment designed to test the strategies and presents the results of this research Finally, Section 6 discusses the conclusions derived from the results of the experiment and explores the possible extentions of this research 12 Auction Theory This section gives an overview of how auction mechanisms work It introduces various auction types and discusses bid formulation as a strategic decision Since online auctions have a strong resemlence with English auctions, bidding in English auctions is explored in more detail Simply stated, an auction can be thought of as a marketplace, where a seller tries to obtain as much money as possible and a buyer wants to pay as little as possible Typically an auction is used when a seller is unsure about the market values of the goods he wants to sell, while a buyer has a good idea of what he is willing to pay The amount a buyer is willing to pay may change depending on competitors reactions As mentioned in [4], the major difference between traditional markets and auction markets is that in the former the price is determined strictly by supply and demand, while in the later the trading rules influence the price and allocations Potential buyers compete with each other by submitting bids high enough to win an auction Sellers submit an ask price based on their cost of supplying the good The lack of complete information about buyers valuations and sellers costs results in a difference between ask and bid prices The bidding proces usually discovers the common bid and ask prices that equate supply and demand [4] Thus in an auction the price is set not by the seller but by the buyer However the seller sets the rules by choosing the type of auction to be used In [4] the authors define four basic auction types: English, Dutch, first-price sealed-bid, and Vickrey English auctions have the ascending outcry format The price is successively raised until one bidder remains The good is sold to the last remaining bidder at a price just above that which sees the second last bidder retire Dutch auctions are the reverse of English auctions, with bids announced in a descending order A bidder wins by being the first to accept an announced bid and pays that price First-price sealed-bid auctions require bidders to submit single confidential bids to the seller The bidder with the highest bid wins and pays that bid Vickrey auctions have a second-price sealed-bid format The bidder making the highest bid wins and pays the next highest bid There are two reasons for buyers to participate in an auction The first reason is a desire to purchase goods for personal consumption and the second reason 3

4 is to acquire goods for commercial use or resale In the first case a buyer has a private-valuation of goods and he is willing to pay up to a certain maximum, independent of valuations of other buyers Buying art for personal pleasure is an example of private-valuation In the second case the goods are worth the same amount to all buyers but this amount is unknown Each buyer estimates the value of such goods using the same measurements This is known as a common-value assumption With the common-value assumption bid formation largely depends on competitors reactions In an English auction the buyers continue to bid so long as the current bid price remains below their valuation of the good (either private-valuation or common-valuation) A buyer s private-valuation typically remains the same throughout the auction and a buyer withdraws from the auction if the current bid price exceeds this value A common-value may change during the auction as the buyers evaluate their competitors bids Lack of information about the actual worth of goods and lack of bidding experience may lead to what is known as winner s curse Winner s curse is the situation when the winner of an auction pays more than what the good is actually worth Thus the auction winner looses because his profits decrease 13 Agents Before going any further with the description of this research, a notion of agents needs to be introduced, since agents are a crucial part of the experiment A simple definition of an agent given in [9] is an agent is just something that acts However, simply creating just something that acts can hardly solve any problem This research is concerned with creating rational agents, the agents that produce the best outcome Computer agents have special attributes and qualities that distinguish them from other computer programs An agent operates within an environment and must be able to perceive the environment and act upon it Agent s actions depend on the sequence of inputs he receives from the environment The behavior of an agent can be described by the agent function that maps any given sequence of inputs to an action An agent is typically aware of his own actions, however he does not necesseraly know about the effect of his actions on the environment In this research the buyers that are participating in the auction are represented by agents The environment in which the agents operate is modeled using an auction simulation described in Section 3 An agent s bidding strategy is the agent function Throughout the experiment agents become more rational and learn to produce better outcomes through the evolution of their strategies 14 Genetic Programming Problem representation is a key issue in deciding which evolutionary algorithm to use The representation scheme chosen for the problem defines the window through which the system observes the world [8] For example with the fixed length strings of genetic algorithms only a certain set of parameters or other 4

5 items can be represented Genetic algorithms with variable length strings are capable of representing if-then rules Clissifier systems increase the complexity of string-based if-then rules Genetic programming paradigm is powerful enough to represent entire computer programs The individuals in genetic programming are compositions of functions and terminals appropriate to the particular problem domain [8] A set of functions can include mathematical functions, arithmetic operations, conditional and logical operations, and program specific functions Each function in the set must be defined for all values in the range of all other functions The search space of genetic programming is the hyperspace of all possible combinations of functions and terminals A convenient way to manipulate the compositions of functions and terminals is through the use of S-expressions represented in a tree data structure These expressions are executed and are assigned a fitness that depends on the outcomes of the execution The typical genetic operations of genetic programming are recombination and mutation A specific type of recombination, subtree crossover, gives birth to new individuals with a potentially better fitness This process is described in [7] As shown in Figure 1, to perform subtree crossover two parent trees are selected A subtree is chosen in Parent 1 and in Parent 2 The child tree is produced by replacing the subtree of Parent 1 with the subtree of Parent 2 As a result the child tree contains characteristics of the parent trees and has the same constraints on the number of inputs into each operator as the parent trees Mutation in genetic programming is performed by randomly selecting a tree node to mutate and changing its value If a function node is selected it is changed to a different function In the case of mutating a function node the number of function inputs must be verified against the number of child nodes If a terminal node is selected for mutation it is changed to a different terminal In this research genetic programming paradigm is applied to come up with successfull online auction bidding strategies 2 Previous Work Some work has already been done in developing bidding agents for online auctions Anthony and Jennings [3] developed autonomous agents that participate in multiple auctions These agents monitor and collect information from multiple ongoing auctions, and use this information to determine the amount of the next bid A genetic algorithm (GA) was used to codify the bidding strategies The authors took into consideration the consumer s private valuation for each item and the consumer s desire for bargain One drawback of this work is the lack of dynamic environment, since the evolution of agents is performed in an offline fashion Ashlock [5] used a more compex strategy representation, GP-Automata, to evolve bidding strategies for the electricity marketplace GP-Automata combie finite state automata with genetic programming The strategies developed 5

6 Parent 1 > Parent 2 > * + - % p1 p2 p3 p4 p1 * p3 p4 Child > p2 p3 * % p1 p2 p3 p4 Figure 1: Subtree Crossover in this research are able to adapt to the changing environment Although the results of the research are specific to the electricity marketplace, they are applicable for other markets The work previously done in this field shows that evolutionary algorithms are an appropriate approach to developing bidding strategies The approache involving genetic programming proved to be successful in dynamic environments, while GA s was more useful in an stable marketplace 3 Auction Simulation In this experiment we are concerned with the bidding strategies for online auctions Almost all online auctions have the same features Since ebay is one of the most popular online auctions, an auction simulator, ebaysim, was created based on the example of ebay ebaysim operates in a matter similar to an English auction where the price on each item is raised successively until the item listing closes In ebaysim each item is listed for a certain length of time When this time expires the listing closes and the agent who has the highest bid on the item is required to pay the amount of the winning bid Each item to be auctioned off starts with a minimum bid amount and contains information about the retail price on this item Table 1 demonstrates an example of four item listings at the start of the auction, and Table 2 shows the same item listings at the end of the auction Note that item 1 is listed twice If the same item is listed multiple times, each one of those listings must specify the the same retail price, while other parameters may differ At the start of an auction no agents have submitted any bids, so the Bid on each item is zero and the Agent ID is empty At the end of the auction in this example Agent 10 has the highest bid 6

7 on the first instance of item 1 and on item 2, Agent 5 has the highest bid on the second instance of item 1, and Agent 9 holds the highest bid on item 3 These agents buy the items by paying the amount of their highest bid ID Retail Price Time Bid Start Price Agent ID 1 $ units $0 $500 none 1 $ units $0 $700 none 2 $ units $0 $950 none 3 $ units $0 $400 none Table 1: Four Item Listings at the Start of an Auction ID Retail Price Time Bid Start Price Agent ID 1 $ units $1275 $ $ units $1300 $ $ units $1960 $ $ units $1525 $400 9 Table 2: Four Item Listings at the End of an Auction The agents participating in the auction have a shopping list of items that they need to purchase When the auction begins the agents start submitting their bids on the items on their shopping lists Bid calculations and submittions are performed concurrently and asynchronously This means that while one agent is placing a bid on an item another agent may be calculating his next bid on that same item When an agent looses the status of the highest bidder on an item, he receives a notification about it If the maximum time for auctioning off an item is reached, no more bids can be placed on this item Figure 2 demonstrates one possible scenario of concurrent bid submission process Although the agents know the retail price of each item they are still able to get a winner s curse - to bid higher than the retail price of the item However a desired strategy should not result in a winner s curse There may be multiple instances of the same item for sale during the simulation However, the agent needs to make sure that he only bids on one instance of the item at a time 4 Methodology 41 Agent Design An agent in this experiment is designed to be able to bid on multiple auction items at the same time These items are listed in the agent s shopping list An agent enters the auction house with a fixed amount of money that he can 7

8 Agent 1 Item 1 Agent 1 Item 1 Agent 2 Agent i Agent K Item j Item M Agent 2 Agent i Agent K Item j Item M (a) Solid arrows point to the agents that currently have the higheset bid on the item (b) Dashed arrows show which agents are submitting the bids Agent 1 Item 1 Agent 2 Agent i notify Item j Agent K item listing closed Item M (c) Dotted arrows show the message sent to the agents Now Agent 2 is the highest bidder on Item j Figure 2: Bid Submission Process 8

9 spend, this amount is referred to as account balance The sum of the agent s bids cannot exceed his account balance An agent knows on which items he currently has the highest bid If an agent looses the status of the highest bidder, he is immediately notified about this change An agent calculates his bids using a bidding strategy, the details of which are explained in Subsection 42 Figure 3 shows an example agent at the start of an auction Here the agent has not yet had a chance to place his bids, thus the bid on each item in his list is zero Agent 10: $40 Shopping List Item 1 Item 2 Item 5 Item 9 Bid List Item 1: $0 Item 2: $0 Item 5: $0 Item 9: $0 Figure 3: Agent 10 at the Start of an Auction Figure 4 shows the same agent at the end of the auction Now this agent holds the highest bids of items 1 and 2 As agents submit their bids, the bid amount is subtracted from their balances When an agent looses the status of the highest bidder on an item, his balance is incremented by the amount of his bid on that item, since he no longer will be required to buy the item Agent 10: $1775 Shopping List Item 1 Item 2 Item 5 Item 9 Bid List Item 1: $1275 Item 2: $ 1960 Item 5: $0 Item 9: $0 Figure 4: Agent 10 at the End of an Auction The agents proceed submitting their bids until all the item listings close The agents goal is to obtain all the items on their lists at the lowest price 9

10 possible 42 Strategy Representation An agent s bidding strategy is represented by an expression tree The expression takes in a fixed number of input parameters and produces the next bid on an item After a careful analysis of the factors that humans consider when bidding in an auction the following parameters were chosen as inputs to the expression tree: 1 Account balance - the amount of money an agent has 2 Item s retail price - the store price of the listed item 3 The total number of items in the agent s list - the size of the shopping list 4 The number of items on the list on which the agent does not have the highest bid - the number of items on the shopping list that the agent still needs to buy 5 The higest bid among all the instances of this item - if there are several listings with the same item id, this is the highest bid among all those listings 6 The lowest bid among all the instances of this item - if there are several listings with the same item id, this is the lowest bid among all those listings 7 The current bid on this item listing 8 A coefficient c in the range (0,1) - this is an additional parameter that may be necessary to adjust the bid It must be noted that this list may be incomplete - there may be other factors that need to be considered in the calculation of the next bid However this list gives a starting point for developing smart bidding strategies The factors above are the input parameters to the expression trees, or strategies, that act as equations for calculating a bid Some of the initial strategies are generated randomly while others are predefined as intuitive to the author Figure 5 shows an example of a predefined strategy In this example the current bid, represented by the right sub-tree, is incremented by a small amount, defined in the left sub-tree Note that the strategy in Figure 5 uses only four input parameters It is assumed that during the evolution process other parameters will be added to the strategy To simplify the task, the operators used by the expression trees are restricted to binary This simplifies the implementation of recombination and mutation by eliminating the need to check that the operators and the number of inputs into them match These operators are: addition, subtraction, division, multiplication, modulus, max, and min The list of operators can be expanded in the future 10

11 + p1 / * p4 p2 p3 Figure 5: Predefined Strategy Example: p1 - current bid, p2 - balance, p3 - coefficient, p4 - the number of items in the agent s list on which he does not have the highest bid 43 Fitness Evaluation Agents fitness is evaluated at the end of the auction round (ie, when all the item listings are closed) In an experiment of this sort fitness evaluation is a subjective matter and depends on what the human user values the most Some users may emphasize the importance of winning an item in the auction, others would consider it more valuable to receive a big discount on an item Since this experiment is concerned with a general bidding strategy, a combination of factors must be considered when evaluating an agent s fitness Let N be the number of items an agent obtains from the auction, M be the total number of items on the agent s list, and Max be the length of the longest shopping list among all agents Then below is a list of criteria that are used in computing an agent s fitness and that several users may find important 1 Relative discount received (Relative Discount = N i=1 (retail pricei highest bidi) N i=1 retail price i 2 The agent s ability to maximize the relative number of items obtained (Relative Number of Items Obtained = N M Max ) The fitness of each agent is calculated using the fitness function shown in Equation 1 The parameter Max in the denominator is necessary to prevent partiality towards the agents with short shopping lists This fitness function rewards the agents that obtain more items on their list at a smaller price Since agents are allowed to place bids that are higher than the retail price of the item on which they are bidding, it is possible to have a negative fitness Allowing bids higher than the item s retail price should help detect agents with undesired strategies, since their fitness will be negative ) N N M Max i=1 (retail price i highest bid i ) N i=1 retail price i (1) 11

12 For an example of fitness evaluation consider the agent shown in Figure 4 In this case N = 2, M = 4, N i=1 (retail price i highest bid i ) = 785, and N i=1 retail price i = 405 Let Max = 30, then the fitness assigned to this 2 agent is = Note that with Max = 30 no fitness can be greater than Agent Reproduction At the end of each auction round a new population of agents is produced The new agents are variations of the agents in the previous population These variations are produced by means of tree recombination and mutation as described in Section 14 The parents that are used to produce a new generation of agents are chosen using proportional selection Scaling of proportional selection is unnecessary in this case This can be justified by the fitness function: the fitness is a product of the relative discount received and the relative number of items obtained A 1 fitness can have a value in the range of (-X, Max ), where Max is the length of the longest shopping list among all agents, and X is an unknown positive real number X is unknown because the agents are allowed to place bids as high as they want Thus good and bad strategies will have significanly different fitnesses in this range Since negative fitness values are allowed it is possible that the total population fitness will be negative or will equal to zero Therefore the proportional selection must be adjusted as shown in Table 3 1 Total Population Fitness Agent s Fitness Selection Chance AgentF itness > 0 0 T otalf itness > 0 < > 0 AgentF itness AgentF itness < T otalf itness = 0 < 0 1 AgentF itness = 0 = 0 0 Table 3: Parent Selection Chance Mutation is an important genetic operator when it comes to producing new variations of strategies A random mutation may change a strategy that is close to an optimal solution into one that is optimal However, mutations may also be harmful and result in the decrease of the solution s optimality When too much mutation is introduced, the search for the optimal solution is no longer a directed search but rather a random one Therefore mutation chance must be carefully considered Since both genetic operators: crossover and mutation, result in drastic differences between the parents and the children, only one of them is used at each generation This approach avoids too much destruction of the parent strategies, which would lead the algorithm to a random search 1 The TotalFitness is defined as pop size i=1 fitness(agent i ) 12

13 45 Competing for Survival The strategies are passed on to the next generation using the Elitist strategy If a child strategy results in a better fitness than some strategy in the current population, it is passed on to the next generation, while a weaker strategy is removed from the original population Due to the dynamic nature of this experiment a strategy that produces a high fitness in one auction round may be a weak strategy under a different auction setup 5 Experiments and Results Agents trial strategies were tested in the simulation environment described in Section 3 It would be very inefficient and resource consuming to have millions of item listings in the simulation, the way real online auctions do Thus it was necessary to introduce modifications to the auction state (ie, the number of items listed and their characteristics) at the start of each auction round These modifications ensure that over time the simulation environment has significant resemblence with real online auctions The following environment characteristics were changed throughout the experiment: 1 Number and types of items in the auction 2 The starting bid amount on each item 3 Bidding time left for each item in the auction The agents shopping lists were also modified before the start of each auction round This ensured that if good strategies are to be produced throughout the experiment, they can be applied to any combination of items on the shopping list, given a reasonable account balance The success of the experiment was measured by the increase in the average population fitness After running the experiment through 67,000 generations the average fitness increased from to , which is an 11% increase However, the best strategy at generation 67,000 was one of the pre-defined strategies: current bid + c, where c = So after 67,000 generations the genetic programming approach failed to produce a new effective bidding strategy 6 Discussion and Conclusions Since the experiment failed to produce any significant results, it is not possible to conclude that genetic programming approach is appropriate for evolving bidding strategies for online auctions However, it is still too early to throw away this idea The failure to produce any good results may be attributed to several factors: 13

14 1 The agents used in the experiment lacked initial knowledge of auction theory The pre-defined strategies were unable to provide enough information about bidding in online auctions 2 Although it is necessary to have a dynamic environment in simulating an online auction, the simulation created for this experiment may have been more dynamic than what genetic programming could handle Making the environment more static towards the beginning of the experiment may have increased the overall population fitness 3 The agents designed for this experiment did not keep track of the bidding history The bidding history may be a crucial element in calculating the next bid Implementing bidding history in the future may result in a more successful experiment 4 The bidding strategies were not evolved until the end of the auction Modifying bad strategies during the auction may improve the overall performance Although the genetic programming approach to this problem did not prove to be successful this time, further experiments need to be performed in search for the optimal bidding strategy The future experiments will be modified to reflect the four failure factors listed above References [1] [2] htmlbr [3] Patricia Anthony and Nicholas R Jennings Evolving bidding strategies for multiple auctions [4] Chris Chan, Patrick Laplagne, and David Appels The role of auctions in allocating public resources php, 2003 [5] DAshlock, CW Richter Jr, and GBSheble Comprehensive bidding strategies with genetic programming/finite state automata In Power Systems, IEEE Transactions, pages , November 1999 [6] Robert D Hof and Linda Himelstein /99_22/b htm [7] Jr Kenneth E Kinnear Derivative methods in genetic programming In Evolutionary Computation 1, pages , Philadelphia, PA 19106, USA, 2000 Institute of Physics Publishing (IOP Publishing Ltd) 14

15 [8] John R Koza Genetically breeding populations of computer programs to solve problems in artificial intelligence In Proceedings of the Second International Conference on Tools for AI, Herndon, Virginia, USA, pages IEEE Computer Society Press, Los Alamitos, CA, USA, [9] Stuart Russell and Peter Norvig Artificial intelligence: A modern approach,