Learnng Curve: Analyss of an Agent Prcng Strategy Under Varyng Condtons Joan Morrs, Patte Maes MIT Meda Laboratory 20 Ames Street, E-0 Cambrdge, MA 02 USA {joane, patte}@meda.mt.edu Amy Greenwald Computer Scence Department Brown Unversty, Box 0 Provdence, RI 022 USA amygreen@cs.brown.edu Abstract - By employng dynamc prcng, the act of changng prces over tme wthn a marketplace, sellers have the potental to ncrease ther revenue by sellng goods to buyers at the rght tme, at the rght prce. As dynamc prcng systems become necessary as a compettve maneuver and as market mechansms become more complex, there s a greater need for prcng agents to be used, and also a greater challenge for sellers to understand what s the best agent prcng strategy for ther marketplaces. Ths paper addresses these ssues by presentng a market smulator desgned for analyzng agent prcng strateges for a market n whch a seller has a fnte amount of tme to sell a fnte number of goods. Through an n-depth analyss of our Goal-Drected prcng strategy, we demonstrate the use of our market smulator as a means for understandng the relevant factors n determnng an effectve dynamc prcng strategy. Keywords : Dynamc prcng, agent smulaton, electronc marketplaces.. Introducton Wth the ncreasng sophstcaton of market analyss and prcng tools avalable to sellers, dynamc prcng s becomng more common. Whle models and practces exst today for settng optmal prces, such as n the arlne ndustry [], there s a lmt to the potental of dynamc prcng f human sellers have to make ndvdual prcng decsons for each transacton. For a seller to compete n a rapdly changng, ncreasngly compettve marketplace, we suggest that sellers use software agents representng ther nterests to deploy ntellgent dynamc prcng strateges. If a seller s agent s aware of constrants on tme and nventory and can observe changes n buyers demand levels and purchasng behavors, then t can attempt to sell all of a seller s goods at the hghest possble prces over tme. One of the dffcultes n employng these real-tme agents s understandng the costs and benefts to dfferent agent prcng strateges. Our approach to studyng dynamc prcng and the applcaton of agents n a marketplace s to use a market smulator for testng and comparng dfferent agent strateges. Our market smulator, called the Learnng Curve Smulator, s desgned for testng agent prcng strateges under vared market condtons and realstc buyer behavors. Whle a theoretcal approach to agent prcng strateges could be taken, we beleve that a theory-based soluton s often dffcult to apply to a real-world marketplace because of the overly smplfyng assumptons that typcally need to be made n developng a theoretcal model. Smulated marketplaces are able to model more dverse and complex scenaros, rather than the general case. By producng tangble, numercal results, our Learnng Curve Smulator can be used as a tool for understandng real-world scenaros. Ths paper demonstrates how the Learnng Curve Smulator can be used to analyze dynamc prcng strateges. Our Goal- Drected strategy, whch we present and analyze here n detal, s an example of an adaptve prcng strategy that could be appled to a fnte market a market n whch there s a fnte amount of nventory and buyers under a fnte tme horzon. Through our analyss, we wll demonstrate the strength a smulator provdes n
producng tangble gudelnes for dynamc prcng strateges n fnte markets. 2. The Prcng Strategy In our prevous study of agent prcng strateges [2], we analyzed the effectveness of two dfferent prcng strateges wthn a specfc market scenaro of an arlne auctonng arlne tckets. Whle one of the strateges, the Goal- Drected strategy, was extremely successful n that arlne scenaro, t was not tested under enough condtons to extend t to a general prcng strategy concluson. In ths paper, we present an analyss of the Goal-Drected strategy usng the Learnng Curve Smulator. We call ths strategy goal-drected because by adjustng prce, the prcng agent attempts to reach a goal by the end of the market. In ths case, ts goal s to sell ts entre nventory by the last day of the market, and not before. The agent accomplshes ths by observng ts success n sellng goods each day and respondng wth ncremental changes n prce. For example, n the arlne scenaro we presented n [2], an arlne has thrty days to sell 00 arlne seats. On each day of the market, the arlne receves bds from buyers and accepts the hghest bds above ts reserve prce, and at the end of the day, the arlne s prcng agent calculates ts reserve prce for the next day usng the Goal-Drected strategy. To do ths, the agent compares the number of seats t has sold to the amount of seats t expected to sell. If too few seats have been sold, the agent responds by lowng ts orgnal offer prce by the percentage t s off-target. If too many seats have been sold, then the agent compensates by rasng the prce. goodssold expgoodssold n prce n = prce + prce á = + 0 0 ( expgoodssold ) expgoodssold ntalinventory = ( daysinmarket ) Fgure : Goal-Drected strategy calculaton In ths manner, the agent fne-tunes the prce of the good to the level of demand that enables the In Sardne: Dynamc Seller Strateges n an Aucton Marketplace, ths strategy was referred to as the Reserve Prcng Strategy. seller to sell all the goods by the last day of the market, and not before. Ths Goal-Drected strategy calculaton s presented n Fgure.. The Market Smulator The Learnng Curve Smulator s desgned to model real-world markets and buyer behavors, for the purpose of testng dynamc prcng strateges. To do ths, the smulator requres three categores of nputs: the Market Scenaro, the Buyer Behavor, and the Seller Strateges, enumerated n Table. Usng these nputs, the smulator constructs and runs a smulated market n whch buyer and sellers match on prce, perform transactons, and change ther behavor each day based on dfferent market condtons. At the end of the smulaton, the success of a prcng strategy s determned by the total revenue earned and the amount of nventory sold by each seller. The followng sectons explan the smulator nput categores n more detal.. Market Scenaro Our research focuses on fnte markets. In these markets, a seller has a certan amount of nventory t must sell by a certan date. There are many examples of ths type of market n today s economy; a few examples are arlne tckets, rental cars, theatre tckets, pershable tems, and seasonal retal goods. The fnte elements of the market are defned by the Market Scenaro nputs (see Table ). Unlke our prevous nvestgaton, we are analyzng a posted-prce market mechansm, not an aucton mechansm, n ths verson of the smulator..2 Buyer Demand over Tme An ntegral part of a fnte market s that the value of the good changes over tme, whether by a change n buyer percepton or the good s publcly known value. Thus, the Learnng Curve Smulator models ths change n buyers percepton of prce, otherwse known as valuaton, through a seres of valuaton/tme curves. To test the robustness of a prcng strategy, we test t under fve dfferent buyer valuaton/tme curves: flat, ncreasng, decreasng, md-peak, and md-dp.. Varaton Among Buyers Each Another mportant aspect of the buyers behavor s how the ndvdual buyers dffer from each other on a sngle day. We have modeled ths n several dfferent ways.
Frst, there s a varance among buyers n ther wllngness to pay for a good, so the smulator calculates a dstrbuton of buyer prces each day based on the nput values for varance and prvate vs. publc valuaton (see Table ). A prvate value good s one n whch the buyer s wllngness to pay s derved from hs/her personal utlty of the good. A publc good s one n whch the buyer s wllngness to pay s based on the publc s collectve assgned value for the good. Dependng on the smulator nput value of prvate or publc valuaton, the dstrbuton of buyer prces s a unform (uncorrelated) or normal (correlated) dstrbuton, wth range defned by the varance value. Second, for dfferent types of markets, buyers are wllng to search for the rght prce for dfferent lengths of tme. Ths s modeled n the smulator wth the buyer lfetme varable. For each smulaton, a lfetme value s selected that ndcates how many days a sngle buyer wll search for a seller offerng the good at an acceptable prce before leavng the market..4 The Smulator Cycle Gven these market nputs, the smulator sequentally runs through each day of the market. On a sngle day, each actve buyer, n random sequence, searches through the avalable sellers, n random sequence, and compares the seller s prce wth ts own reserve prce. If the seller s prce s less, a transacton occurs and the buyer leaves the market. If the seller s prce s more, the buyer contnues lookng. The day ends when each buyer has completed ts search through the sellers. At the end of the day, a new reserve prce for each buyer s calculated based on the valuaton/tme curve and varance values. Each seller updates ts prce based on ts strategy calculaton. If the seller s usng the Goal- Drected strategy, the seller examnes how many goods t has sold and what day t s, and makes a prce adjustment. If the seller s usng a Fxed- Prce strategy, there s no change to the prce. In ths manner, the market progresses untl the last day, stoppng only f there are no more buyers or no more goods n the market. More nformaton on the nner workngs of the smulator can be found n [] and screenshots of the smulator s Java Swng nterface can be found on-lne at [4]. Table : Smulator Inputs Smulator Inputs: Descrpton Sample Values Market Scenaro: Number of s Number of perods n whch the seller can mplement a 0 prce change Number of Buyers Number of buyers wthn entre market 20 Number of Sellers Number of sellers competng 2 Number of Goods Number of goods per seller 00 Market Mechansm Posted-Prce only. (In later analyses, auctons could be ncluded as a possble mechansm.) Posted-Prce Buyer Behavor: Lfetme Number of days each buyer s n market Prce Varance Per The buyers reservaton prces vary ± the varance n a 0 sngle day. Prvate or Publc Valuaton Unform or normal dstrbuton Prvate Mn/Max of Buyers prces The mnmum and maxmum average prce desred by / the buyers, over tme Valuaton over Tme Curve The buyers valuaton/tme curve can be ether flat, ncreasng, decreasng, md-peak, or md-dp All (flat, ncreasng, decreasng, mdpeak, and md-dp) Seller Strateges: Seller Strategy Ether Goal-Drected or Fxed-Prce Goal-Drected, Fxed-Prce Intal Prce The prce the seller offers the frst day of the market. In the case of a Fxed-Prce Seller, ths wll be the prce offered on all days of the market. Avalable Inventory per Amount of nventory a seller can sell n one day goods per day (* nventory/days)
Prcng under Flat Valuaton Prcng under Decreasng Valuaton Prce Prce 2 2 2 2 2 2 2 2 Avg. Buyer Prce Goal-Drected Fxed-Prce Avg. Buyer Prce Goal-Drected Fxed-Prce 2A: Flat Valuaton Curve 2B: Decreasng Valuaton Curve Prcng under Increasng Valuaton Prcng under Md-Peak Valuaton Prce Prce 2 2 2 2 2 2 2 2 Avg. Buyer Prce Goal-Drected Fxed-Prce 2C: Increasng Valuaton Curve Prcng under Md-Dp Valuaton 0000 Avg. Buyer Prce Goal-Drected Fxed-Prce 2D: Md-Peak Valuaton Curve Revenue under Each Valuaton Prce Revenue ($) 40000 0000 20000 0000 2 2 2 2 0 flat ncreasng decreasng md-peak md-dp Avg. Buyer Prce Goal-Drected Fxed-Prce Goal-Drected Fxed-Prce 2E: Md-Dp Valuaton Curve 2F: Revenue per Valuaton Curve 20 Buyers n Market per Sold Inventory under Each Valuaton 00 Number of Buyers 0 0 4 0 6 22 2 28 Inventory (max 00) 80 60 40 20 0 flat ncreasng decreasng md-peak md-dp Buyers Buyng Buyers Not Buyng Goal-Drected Fxed-Prce 2G: Buyers n Market Each 2H: Inventory Sold per Valuaton Curve Fgure 2: Sample Smulaton Results Fgures 2A-2E show the prcng behavor of a Goal-Drected (GD) seller and a Fxed-Prce (FP) seller under dfferent valuaton condtons. In each case, the FP seller offers whle the GD seller adjusts prce each day based on the amount of nventory t has sold at each pont n the market. The revenue each seller earns under each valuaton condton s shown n Fgure 2F. As shown, the GD seller captures more revenue under each valuaton curve, even under a flat valuaton curve. In the specfc case of flat valuaton, the GD strategy prevaled by adaptng to the hgh varance among the buyer populaton (on a sngle day, prce rangng between $00 and 0). In each tral, there were 20 buyers, each appearng n the market for one day, and 00 goods per seller. Fgure 2G shows when the buyers appeared n the market and when they made purchases, for the case of md-peak valuaton. Fgure 2H shows that the GD strategy consstently sells nearly ts entre nventory, whch results n hgher revenue despte the often lower sale prces.
4. Strategy Analyss We tested the Goal-Drected strategy under many dfferent market condtons n order to understand ts strengths, varyng the shape of the valuaton/tme curve, the amount and type of competton n the market, and the sze and behavor of the buyer populaton on a per day bass. Based on these numerous smulatons, we have made several conclusons about the effectveness of the Goal-Drected strategy. Before outlnng these results, we present here a sample set of smulaton results to llustrate our analyss process. Fgure 2 shows the results of fve smulaton trals generated from the nput values lsted n the rght column of Table. These smulatons each had two sellers, one usng the Goal-Drected strategy and the other usng a Fxed-Prce strategy. The ntal prce for both sellers was the average of the buyer prce range,. Fgures 2A-2E show the prcng behavor of the sellers under the fve dfferent valuaton/tme curves: flat, ncreasng, decreasng, md-dp, and md-peak. These charts llustrate the characterstc prcng pattern of the Goal-Drected strategy. The frst days of the market are characterzed by extreme over and undershootng of the buyers average prce as the strategy adjusts for over- and under-sellng. As the days of the market progress, the prce changes become less extreme as the strategy begns to track the buyers valuaton curve. Ths followng of the buyers valuaton curve s what makes the strategy so effectve: regardless of the type of buyer behavor presented, the Goal- Drected strategy s able to compensate for the change n buyer behavor and acheved ts goal of sellng goods by adjustng prce. As shown n Fgures 2F and 2H, the Goal-Drected strategy sells more nventory and earns more revenue, despte often offerng lower prces than the seller usng a Fxed-Prce strategy. Fgure 2G shows the number of buyers n the market for the specfc case of the Md-Peak valuaton curve, llustratng that most of the sales occurred durng the mddle day of the market when valuaton peaked, wth a few sgnfcant sales occurrng at the begnnng of the market when the Goal-Drected strategy captured crtcal sales.. Analyss Conclusons The results presented n Fgure 2 demonstrate the strengths of the Goal-Drected strategy. We analyzed the Goal-Drected strategy under a spectrum of market, buyer, and seller condtons, whch resulted n vared levels of success over a Fxed-Prce strategy. Here we present our analyss of when and why the Goal-Drected strategy experences success over a Fxed-Prce strategy.. Intal Prce. The further the seller s ntal prce s from the optmal sale prce, the better the Goal-Drected strategy performs. Each seller n the smulator offers an ntal prce for the good, as ndcated n the smulator nputs, and a seller usng a Fxed-Prce strategy wll offer ths same prce every day. In our trals, all the sellers began wth the same ntal prce, under the assumpton that the ntal prce s the sellers best guess at the optmal prce whch wll sell all the nventory at the hghest prce. In the example n Fgure 2, ths ntal prce was set to. If sellers do not make an optmal prce decson, due to lmted or ncorrect knowledge about the buyer populaton, a Fxed-Prce seller wll be unable to sell ts nventory and the Goal- Drected strategy wll preval by adjustng ts prce. If a seller can make an accurate predcton on how buyer demand wll change over tme, then the seller can confdently pck an optmal prce and acheve maxmum revenues, but n the more common stuaton where there s ncomplete nformaton, a Goal-Drected strategy allows a seller to adjust for mstakes. 2. Seller Inventory. If a seller s lmted n the amount of nventory t can sell each day, the Goal-Drected strategy wns. When the amount of nventory that can be sold each day s restrcted, whether because of a shelf-stockng fee or an mpractcalty of sellng the entre nventory n one day, then a Fxed- Prce strategy s unable to take advantage of a slm wndow of opportunty to sell all of ts goods at a hgh prce. The Goal-Drected strategy on the other hand, paces ts sales across the market. Under a condton where a seller can sell everythng quckly, then a hgh fxed-prce can work best, but f that s not possble, then the Goal-Drected strategy wll ensure that all or the
majorty of the nventory s sold at the hghest prce to be gotten on an ndvdual day.. Number and Lfetme of Buyers. In a market wth a lmted number of buyers wth a lmted lfetme, a Goal-Drected strategy s able to sell to more buyers than a Fxed- Prce strategy. When the number of buyers s close to the number of goods avalable n the marketplace, each seller needs take advantage of each day buyers are avalable n the market. Because the Goal-Drected strategy focuses on sellng goods every day, a Goal-Drected seller s able to sell more nventory than a Fxed-Prce seller. 4. Competton. The more Fxed-Prce sellers n the market, the better the Goal-Drected sellers perform, gven a lmted numbers of buyers n the market. Increasng the number of Goal-Drected sellers n the market does not sgnfcantly change the performance of the ndvdual Goal-Drected sellers. But when we ncreased the number of Fxed-Prce sellers n the market, the Goal- Drected sellers ncreased the amount of earned revenue and sold nventory n relaton to the Fxed-Prce sellers. The Goal-Drected strategy s adaptve prces allow a seller to sell to a hgher proporton of buyers than the Fxed-Prce seller, effectvely stealng sales from the Fxed-Prce seller. The more Fxed-Prce sellers n the market, the more these results are exaggerated.. Varance n Buyer Prce. The hgher the varance n prce among the buyers, the better the Goal-Drected strategy performs. If, on a sngle day, every sngle buyer wants to pay the same prce, then t would be possble for a Fxed-Prce seller to pck the rght prce and sell the entre nventory. A more realstc stuaton s that most buyers consder multple factors n calculatng ther reservaton prce and n ths stuaton the Goal-Drected strategy works effectvely. When there s a hgh varance among the buyers, the Goal-Drected strategy adjusts and fne-tunes ts prce wthn the spread of buyer prces to sell the hghest payng customers each day. 6. Buyer Valuaton/Tme Curve. A Fxed-Prce strategy performs best wth curves that are at a relatve hgh early n the market, but a Goal-Drected strategy performs consstently for all types of curves. When the demand s at a hgh pont n the early days of the market, a Fxed-Prce strategy can successfully sell all or most of ts nventory n the early days, whle a Goal-Drected seller s observng and make drastc adjustments to prce. Over longer market perods when buyers change ther behavor n unexpected ways, the Goal- Drected strategy has the tme to learn the valuaton/tme curve at the begnnng and then consstently outperforms a Fxed-Prce strategy for the duraton of the market. We consder one of the key strengths of the Goal-Drected strategy to be ths ablty to quckly learn a curve and then, as t follows the curve, decrease the amount of prce adjustment each day.. Number of s. The fewer days n the market, the less effectve the Goal-Drected strategy. The sample results n Fgure 2 contaned thrty days, whch provded enough tme for the Goal- Drected strategy to adapt to changes n buyer behavor. If the market has only seven days, the seller does not have enough tme to observe and adjust ts prce, yet when the market contans more than thrty days, the performance of the Goal-Drected strategy mproves further because t has more cycles n whch to learn the buyers behavor. So n a market wth a lmted number of days, or cycles n whch to change prces, a Fxed-Prce strategy could be the better strategy. 6. Related Work Theoretcal studes of prcng strateges n fnte markets have made conclusons about optmal prcng strateges, but the drawback of these theoretcal approaches s the dffculty n applyng the results to real-world markets. Gallego & van Ryzn [], for example, studed ths problem wth an assumed, flat valuaton/tme curve (.e. a statc demand curve). The beneft of usng the Learnng Curve Smulator s ts ablty to model dverse and complex scenaros, rather than only smplfed cases. Our nvestgaton of agent prcng strateges s unque n ts use of a smulator to model a fnte market. Several researchers have studed prcng strateges n smulated nformaton-good marketplaces [6-], n whch nventory and tme are not constrants. In these markets, the best strategy s the one that
competes best n a compettve market. The addtonal complextes of constrants on tme and nventory further llustrate the usefulness of studyng and developng strateges n a market smulator. Of current ndustres employng dynamc prcng, the arlne ndustry sets the standard for dynamc prcng by usng technques of revenue management to mplement automated prce changes over tme [, 0]. Commercal revenue management systems forecast demand, montor bookng actvtes and, n response, adjust the number of tckets avalable at each pre-defned prcng level, or fare class. Ths method s effectve and practced n other ndustres as well, but requres sellers to make assumptons and predctons about the behavor of the marketplace. Our Goal-Drected strategy makes no assumptons or predctons about future demand, but nstead learns to adapt to the current demand leve ls. The Learnng Curve Smulator allows multple types of strateges to be analyzed and compared aganst one another, provdng a method for comparng dynamc prcng approaches.. Concluson We found that the Goal-Drected strategy, desgned for a fnte market, works best n a market n whch a seller has restrctons on when and how much t can sell. When a lmted number of buyers, compettve factors, or restrctons on sellng practces constrct the amount of nventory sold each day, the Goal- Drected strategy prevals over a Fxed-Prce strategy because t focuses on consstently makng daly sales through basc prce adjustments. These conclusons demonstrate the strength of our smulaton-based analyss. The tangble results we found can now be used to nform a real-world seller s process of desgnng a Goal-Drected or smlar dynamc prcng strategy for ther fnte market. The complextes of a fnte market make theoretcal studes challengng, and our belef s that a smulator wth a rch set of market varable condtons allows for straghtforward strategy development and analyss. Through ths process sellers can gan an understandng of whch strateges are best for ther markets. In terms of future work, our analyss results lead us to more questons and nform the mmedate drecton of our research. In addton to developng more strateges, we plan to add more realstc behavor to the smulated buyers, eventually applyng adaptve buyng behavor that responds to the sellers dynamc prcng behavor. The Learnng Curve Smulator wll provde the platform for ths further market modelng and strategy analyss. References []. Boyd, E.A., Arlne Allance Revenue Management. ORMS Today, 8(October 8). [2]. Morrs, J., P. Ree, and P. Maes. Sardne: Dynamc Seller Strateges n an Aucton Marketplace. Proceedngs of the 2nd ACM Conference on Electronc Commerce (EC '00). 2000. Mnneapols, Mnnesota. []. Morrs, J., A Smulaton-based Approach to Dynamc Prcng, Master's Thess n Meda Arts & Scences. 200, MIT: Cambrdge, MA. [4]. Learnng Curve Smulator Screenshots, http://www.meda.mt.edu/~joane/learnngcurve/ screenshots/. 200. []. Gallego, G. and G. van Ryzn, Optmal Dynamc Prcng of Inventores wth Stochastc Demand Over Fnte Horzons. Management Scence, 4. 40 (8): p. -020. [6]. Deck, C.A. and B.J. Wlson. Interactons of Automated Prcng Algorthms: An Expermental Investgaton. Proceedngs of the 2nd ACM Conference on Electronc Commerce (EC '00). 2000. Mnneapols, Mnnesota. []. Brooks, C.H., S. Fay, R. Das, J. MacKe-Mason, J. Kephart, and E. Durfee, Automated Strategy Searches n an Electronc Goods Market: Learnng and Complex Prce Schedules. Proceedngs of the ACM Conference on Electronc Commerce (EC ').. Denver, CO. [8]. Greenwald, A., J.O. Kephart, and G.J. Tesauro. Strategc Prcebot Dynamcs. Proceedngs of the ACM Conference on Electronc Commerce (EC ').. Denver, CO. []. Kephart, J.O. and A. Greenwald. Shopbot Economcs. Proceedngs of the Ffth European Conference on Symbolc and Quanttatve Approaches to Reasonng wth Uncertanty.. [0]. Smth, B.C., D.P. Gunther, B.V. Rao, and R.M. Ratlff. e-commerce and Operatons Research n Arlne Plannng, Marketng, and Dstrbuton. Interfaces, Specal ssue on Operatons Research n the e -Busness Era, 200. (March/Aprl 200).