RELIABILITY AND ACCURACY OF BOOTSTRAP AND MONTE CARLO METHODS FOR DEMAND DISTRIBUTION MODELING

Transcription

1 Proceedgs of the 5th Aual ISC Research Symposum ISCRS 2011 Aprl 7, 2011, Rolla, ssour RELIABILITY AND ACCURACY OF BOOTSTRAP AND ONTE CARLO ETHODS FOR DEAND DISTRIBUTION ODELING Swth S. Razu Dept. of echacal ad Aerospace Egeerg ssour Uversty of Scece ad Techology Rolla, O, Shu Taka Dept. of echacal ad Aerospace Egeerg ssour Uversty of Scece ad Techology Rolla, O, ABSTRACT Estmato of demad s oe of the most mportat tasks ew product developmet. How customers come to apprecate ad decde to purchase a ew product mpacts demad ad hece proft of the product. Ufortuately, whe desgers select a ew product cocept early the product developmet process, the future demad of the ew product s ot kow. Cojot aalyss s a statstcal method that has bee used to estmate a demad of a ew product cocept from customer survey data. Although cojot aalyss has bee creasgly corporated desg egeerg as a method to estmate a demad of a ew product desg, t has ot bee fully employed to model demad ucertaty. Ths paper demostrates ad compares two approaches that use cojot aalyss data to model demad ucertaty: bootstrap of respodet choce data ad ote Carlo smulato of utlty estmato errors. Relablty of demad dstrbuto ad accuracy of demad estmato are compared for the two approaches a llustratve example. KEYWORDS: Demad Dstrbuto, Bootstrap, ote Carlo, ad Choce-based Cojot Aalyss. 1. INTRODUCTION Durg product developmet, a product cocept s selected early the product desg stage before detal desg [1]. At the tme of cocept selecto therefore, future customer demad ucerta. Realzg the eed to make ratoal cocept selecto decsos uder ucertaty or complete formato, the preset paper addresses customer demad ucertaty modelg. Ths research costructs demad dstrbutos by applyg bootstrap ad ote Carlo smulato (C) to demad data obtaed from choce-based cojot aalyss (CBC) ad also dscusses the relablty ad accuracy of these methods. Cojot aalyss (CA) ad dscrete choce aalyss (DCA) have creasgly bee used desg decso makg as the meas to estmate demad of a ew product desg. Customer survey data are used CA, whereas actual customer product purchasg data are used DCA. Compared to DCA, oe of the beefts of CA s ts ablty to estmate demad of product cocept. CA has bee corporated optmzato ad model market share estmato of a ew product [2, 3] ad the optmzato of the product desg corporatg retalers acceptace [4]. DCA has bee corporated the modelg of demad to optmze product desg [5, 6], product le desg [7], corporatg retaler s decso product desg [8], ad prce competto product desg [9]. Whe respodets choose ther preferred cocepts the cojot survey, the method s typcally called CBC. I CA, a logt model [10, 11] or a probt model [12] ca be used to estmate part worths of product attrbutes by aalyzg respodet choce data. Utltes of a product cocept ad competg products are calculated from part worths; a probablty of choosg the cocept s calculated from these utltes; ad product demad of the cocept ca be estmated from the choce probablty. To obta demad dstrbuto from the choce probabltes as a result of CA, the frst approach tegrates bootstrap [13], whch s a samplg wth a replacemet procedure that permts calculato of sample statstcs. The secod approach corporates the varato amog customers; ths radomess s corporated to the error term. The utltes alog wth the C of utlty estmato error terms are used to obta the demad dstrbuto. I the past customer demad ucertaty modelg has bee lmted to C of utlty estmato errors [14]. Ths paper compares bootstrap ad C applcato of utlty error term wth the true customer demad ad hece determes ther relablty ad the accuracy. Ths paper s orgazed as follows: Secto 2 brefly descrbes choce-based CA, bootstrap, applcato of bootstrap to CBC data, ad C of utlty estmato errors. Secto 3 demostrates the proposed approaches usg a llustratve example ad compares ther relablty ad accuracy. Secto 4 cocludes the paper wth a dscusso of future work. 1 Copyrght 2011 by ISC

2 2. ETHODOLOGY Demad dstrbuto s modeled by applyg bootstrap ad Cs methods to CA data. CA eables desgers to estmate customer utltes of a product cocept ad the product demad [15-18]. Data s collected by askg potetal customers to choose the most preferred cocept a cojot survey. As a result of CBC we obta a pot estmate for market share, to whch bootstrap ad C are appled to obta demad dstrbutos. Bootstrap s appled to the choce probabltes whereas C s appled to the error term of the dvdual part worth utlty. 2.1 Choce-based CA Whe a compay develops a ew product, the demad could be sgfcatly creased f formato were avalable regardg the reaso why cosumers choose oe brad over aother. CBC s oe marketg research method used to aalyze ths cosumer trade-off; CBC s a decomposto method that estmates the structure of the preferece of the cosumer, from whch demad ca be obtaed. To estmate a product utlty U, we sum up the partworths u of the j attrbutes of product : U u (1) j 1 Eq.(1) s used to calculate the total utlty of products ad product cocepts competg. The total utlty s coverted to choce probablty (Pr()) usg Eq.(2) for the chose cocept or product. Pr() 1 j exp( U ) exp( U ) For the th product ad j th attrbute, the total utlty u j cotas two compoets, the determstc coeffcet v j ad the partly uobservable error term e j as show Eq.(3).The e s a depedet ad detcally (2) uobservable error term j dstrbuted (IID) wth a extreme value dstrbuto. u v e (3) j j j The basc steps volved choce-based CA are as follows: Defe cocept: Idetfcato of product attrbutes ad ther levels that mpact the customers to make purchasg decsos. Oe such method s to tervew customers ad traslate ther eeds to product attrbutes. Idetfy attrbute levels: Bechmark exstg competg products or forecast cosumer eeds for a ew product to detfy attrbute levels cluded the CA survey. Desg cojot survey: Sets of cocepts to be show to the respodets to be evaluated are desged usg expermetal desgs such as orthogoal arrays or factoral desgs. Estmate market share: The results of the evaluated survey are aalyzed usg the logt model to estmate the probabltes for the cocepts, from whch market share of the varous cocepts ca be determed. 2.2 Bootstrap applcato to choce data The bootstrap algorthm uses the orgal sample data to geerate a umber of data sets usg samplg wth a replacemet techque, from whch fereces ca be draw about populato statstcs. Ths s llustrated by usg a smple example Fg. 1. Suppose we eed to estmate the populato average from a radomly sampled data set {1,2,3,4,5,6,7}, the result we obta s a sample average 4 ad we use ths as a estmate of the true populato average. The drawback wth ths method s that that a cofdece terval of ths estmate caot be obtaed because the sample average s a pot estmate. To overcome ths: bootstrap samples of the same data sze are geerated, the computer geerates dfferet sets of samples usg samplg wth replacemet techque to obta: {5,1,7,1,2,4,4} as the frst bootstrap sample, {6,4,2,5,7,3,6} as the secod bootstrap sample, ad so forth. Repetto of data may appear due to the samplg wth replacemet procedure. Samplg wth replacemets Data Sample statstcs (average) Ital sample : { 1, 2, 3, 4, 5, 6, 7 } 4 Bootstrap samples 1 st : { 5, 1, 7, 1, 2, 4, 4 } d : { 6, 4, 2, 5, 7, 3, 6 } rd : { 2, 3, 5, 2, 1, 1, 1 } th : { 4, 6, 4, 7, 3, 4, 2 } 4.3 Dstrbuto Fgure 1. Bootstrap procedure Iferece (95% cofdece terval) The sample statstc (.e., average) s calculated for each of the bootstrap samples. Hece, f there are 200 bootstrap samples, 200 sample averages are obtaed. Aalysts ca the costruct a cofdece terval or a dstrbuto of sample statstcs. I the above example we obta a 95% cofdece terval of the pot estmate 4, ad a hstogram of the bootstrapped sample averages, from whch a dstrbuto of sample statstcs ca be obtaed, as llustrated Fg.1. I CA, the respodets represet a very small fracto of the etre populato of customers. It would be deal to obta the true market share by applyg CA to the etre populato of customers, but ot feasble. The aalyss of the evaluated product cocepts ad compettors products yelds a pot estmate, as llustrated the mddle flow Fg. 2. Ths pot estmate s equvalet to assg a probablty of oe to that market share value. I cotrast, the applcato of bootstrap to CA dcates that a customer may appear multple tmes or oe at all because of the samplg-wth-replacemet procedure. Hece ths tegrated approach makes possble the costructo of market share dstrbutos from market share estmates obtaed as a result of bootstrap, as llustrated by the last row Fg Copyrght 2011 by ISC

3 Populato Customer {1, 2, 3,, N} Sample Radom samplg Customer {1, 2, 3,, } "True" market share, S True Probablty 1 Bootstrap Cojot aalyss Probablty { 1st sample} share, s 1 1 { 2d sample} share, s { B-th sample} share, s B 50% share Fgure 2. Applcato of bootstrap to CA 2.3 ote Carlo smulato of utlty estmato error term I choce-based CA the part-worths are the determstc coeffcets v j ad there exsts a uobservable error term e j that accouts for the varato across customers as show Eq.(3). The determstc coeffcets v j s estmated usg the β co-effcet as a result of maxmum lkelhood estmato; there exst varatos these β coeffcets represeted by the error term, E j, each of these β coeffcets are dstrbuted accordg to ormal dstrbuto or studet t dstrbuto. vj j E j (4) Hece, the total utlty for a cocept s obtaed from Eq. (1) ad the pot estmates are obtaed from Eq.(2). The fal dstrbuto s estmated usg 200 C teratos of the error terms whch assumed to IID wth ormal dstrbutos as show the bottom flow Fg.3. Populato Customer {1, 2, 3,, N} Sample Total Utlty Radom samplg Customer {1, 2, 3,, } Cojot aalyss "True" market share, STrue Cojot aalyss Pot estmate of market share, sestmate sestmate 50% Partworth utlty1 Partworth utlty2 Probablty + + Error term dst Error term dst µ=0, σ=std Error + µ=0, σ=std Error + Fgure 3. Applcato of C to CBC S True 50% share 3. ILLUSTRATIVE EXAPLE I ths secto, we llustrate ucertaty modelg for customer prefereces by the applcato of BS ad error term C to CBC. Ths example uses a o-bayesa CBC wth customers evaluatos based o preferred choce usg automoble cocept selecto. 3.1 Cocept ad compettors defto I ths llustrato, we assume that a maufacturer s developg a ew automoble (N) that wll compete wth two 0.5 Probablty STrue Probablty % 50% share share share compettors automobles (C1 ad C2), the maufacturer wshes to predct the future market share of N. The ew car cocept s defed by ts type ad fuel effcecy. The type s further defed by ts form ad the maxmum umber of passeger t ca accommodate, the fuel effcecy s defed by a ege type (a gasole ege or a hybrd ege). I addto the maufacturer chooses a basc warraty ad prce that flueces market share. Type SUV Cocept Fuel Effcecy Warraty Fgure 4. Selected combato of cocept, warraty, ad prce for N. The maufacturer selects the type to be a sports utlty vehcle (SUV), 25 mles per gallo as a fuel effcecy, 5/60,000 (years/mles) as a basc warraty, ad $35,000 as a prce as show Fg. 4. Fgure5. Competg cars C1 ad C2. Prce N 25 5 / 60,000 $ 35,000 8 passegers (mles/gallo) (years/mles) Type Fuel Effcecy Warraty Prce Covertble No-hybrd Compettor / 36,000 $20,000 2 passegers (mles per gallo) (years/mles) Seda Hybrd Compettor / 50,000 $50,000 5 passegers (mles per gallo) (years/mles) The compettors automobles are summarzed Fg. 5. The frst compettor car (C1) s a covertble that gets 10 mles per gallo as a fuel effcecy, 3/36,000 (years/mles) as a basc warraty, ad $20,000 as a prce. The secod compettor car (C2) s a seda that gets 40 mles per gallo as a fuel effcecy, 4/50,000 (years/mles) as a basc warraty, ad $50,000 as a prce. 3.2 Attrbute level detfcato To estmate market share usg CBC, possble levels of fuel effcecy, warraty, ad prce have to be detfed. Levels for the attrbutes are detfed by bechmarkg exstg automobles troduced to the market betwee 2003 ad 2009 (87 SUVs, 29 covertbles, ad 57 sedas) as summarzed Tables 1 through 3. Table 1 shows the mmum, average, meda, ad the maxmum fuel effcecy of the bechmarked vehcles. Based o these results, three levels of fuel effcecy are selected that are close to the mmum, meda ad maxmum for the choce-based CA study: 10, 25, ad 40 mles per gallo. 3 Copyrght 2011 by ISC

4 Table 1. Fuel Effcecy (les per Gallo) SUV Covertble Seda Average eda ax Table 2 dsplays the bechmarked results for the frequecy of the basc warrates offered. The results suggest that the frequecy of the three most wdely offered warrates selected for the CA study are 3/36,000, 4/50,000, ad 5/60,000 years/mles. Table 2. Basc Warraty Frequecy Warraty SUV Covertble Seda (Years/les) 3/36, /50, /60, /60, Smlarly, Table 3 dsplays the bechmarked results for the prces offered. The results suggest that the three prce levels that are selected close to the mmum, meda ad maxmum are $20,000, $35,000 ad $50,000. Table 3. Prce ($) SUV Covertble Seda 20,972 17,365 20,342 Average 36,900 41,579 36,798 eda 36,105 35,513 28,045 ax 67, ,855 90, share pot estmato A choce-based cojot survey [17] s coducted order to detfy utltes of cocepts ad compettor automobles at dfferet warrates ad prces. To estmate these utltes, L27 orthogoal array combatos of four attrbutes at three levels are created. Four attrbutes are type, fuel effcecy, warraty, ad prce. Three levels are covertble, seda, ad SUV for type; 10, 25, ad 40 mpg for fuel effcecy; 3 years/36,000 mles, 4 years/50,000 mles, ad 5 years/60,000 mles for warraty; ad $20,000, $35,000, ad $50,000 for prce. Each respodet s show oe of 27 sets of three automobles at a tme ad asked to choose oe from each set. Fgure 6 shows a set of three automobles the choce-based cojot survey. Each set are created accordg to the L27 orthogoal array combatos of attrbutes ad levels. At the ed of the survey, a addtoal 28 th choce set s show to the respodet. Ths set cossts of the cocept (N) Fg. 4 ad compettors (C1 ad C2) Fg. 5. The respodet choces ths 28 th set defe true demads of the cocept N as well as compettors C1 ad C2. CA data from the frst 27 choce questos are used to study relablty ad accuracy of bootstrap method ad C method for modelg.. Fgure 6. Choce-based cojot survey From the choce data of 50 respodets we use the logt model to obta ther dvdual part worths for all the attrbute levels for a sgle respodet. These dvdual part worths are summed to obta the total utlty of cocept N (Fg. 4) ad the two compettor s vehcles C1 ad C2 (Fg. 5). For example, the total utlty of C1 s calculated by addg the part worths of a covertble vehcle type, a fuel effcecy of 10 mles/gallo, ad a warraty of 3 years/36,000 mles ad, a prce of $20,000. The total utltes are the coverted to choce probabltes by applyg Eq.(5-7) to each respodet. Pr(N) Pr(C1) Pr(C2) u( N) exp exp exp exp u( C1) u( C2) u( N ) uc ( 1) exp exp exp exp u( C1) u( C2) u( N ) uc ( 2) exp exp exp exp Where, un ( ), uc ( 1) ad ( 2) u( C1) u( C2) u( N ) uc are the total utltes of N, C1 ad C2 respectvely for the chose respodet. Hece as a result of logt model, pot estmates terms of choce probabltes demad are obtaed for N, C1, ad C2 for each respodet. 3.4 Bootstrap applcato to CA data Dstrbutos of market share are obtaed by applyg bootstrap to the CA data. From the orgal 50 subjects (R1- R50) whch s assumed to be the populato sample, fve dfferet sample szes are chose to represet practcal scearos whe samplg of the etre populato s ot feasble. The 5 scearos chose are: 50choose5, 50choose10, 50choose20, 50choose30, ad 50choose40. 50choose5 represets fve subjects bootstrapped from the orgal sample (50), smlarly 50choose10 represets 10 subjects bootstrapped from the orgal sample, 50choose20 represets 20 subjects bootstrapped from the orgal sample, 50choose30 represets 30 subjects bootstrapped from the orgal sample, ad 50choose40 represets 40 subjects bootstrapped from the orgal sample. For each sample sze, 20 such smulatos are performed to obta dfferet bootstrapped samples each smulato (5) (6) (7) 4 Copyrght 2011 by ISC

5 % Relablty order to egate the effect of ay sgle bootstrapped sample. For example, for the frst smulato the case of 50choose5. From the data of fve respodets, 200 bootstrap samples are geerated by the sample-wth-replacemet procedure. Each bootstrap sample cossts of 5 respodets; however, because of the sample-wth-replacemet procedure, the same respodet may be cluded more tha oce or some respodets may ot be cluded at all. From these 200 bootstrap samples, 200 market share estmates ad a demad dstrbuto are obtaed for N, C1 ad C2. Hece 20 smulatos for 50choose5 we obta market shares for each N, C1 ad C2. The same procedure s appled to obta 50choose10, 50choose20, 50choose30 ad 50choose40 to obta a total of market shares for each N, C1 ad C Error term ote Carlo applcato to CA data As a result of choce-based CA we obta dvdual partworths usg the stadard logt model or the multomal logt model, where the part-worths are take as the determstc part ad the varato across cosumers s accouted for the error term. Hece to accout for varato across cosumers, error terms for each attrbute level for the chose cocept are sampled usg C smulato to produce 200 error terms whch are the added to obta the total utlty. The total utlty s the coverted to choce probabltes usg Eq.(5-7). C of the utlty error term s appled to each smulato to the exact smulatos as bootstrap applcato to CA data. Hece as a result of C, 200 market share probabltes are obtaed for cocept N ad compettor cocepts C1 ad C2 for every smulato. Hece for 50choose5, for 20 smulatos market share probabltes are obtaed for each cocept. The same procedure s appled to obta 50choose10, 50choose20, 50choose30 ad 50choose40 to obta a total of market shares for each N, C1 ad C Relablty ad accuracy The true market share s obtaed by calculatg the percetage of umber of respodets choosg N, C1, ad C2 from the 50 respodets obtaed from the 28 th questo of the survey as summarzed Table.4. The true market share for each smulato would be the percetage of umber of respodets that choose N, C1, ad C2 the bootstrapped sample. Table 4. True Share for N, C1 ad C2 N C1 C2 50% 18% 32% The relablty ad accuracy s measured from the probablty of market share obtaed as a result of bootstrap ad C of utlty error term compared to the true market share. The average market share for the frst smulato ( avg1 ) the case of 50choose5 s calculated by averagg the dvdual market share probabltes ( ) over the 200 bootstrap or C samples as show Eq.(7). Avg1 1 The 2.5% ad 97.5% percetles are calculated for each of these Avg1, Avg2 Avg20. To check for relablty, the umber of tmes that the true market share s captured by the 2.5% ad 97.5% percetles or 95% cofdece tervals of the average market share ( Avg ) were oted for each smulato the 5 cases as show Fg.7. Relablty for each sample sze s calculated by lookg at ths proporto. The closer ths proporto s to the 95% cofdece terval, the more the relablty. (7) Cocept N Share Smulato Percetles choose5 2.5% % choose10 2.5% % choose20 2.5% % choose30 2.5% % choose40 2.5% % Fgure 7. The true market share ad the percetles for each smulato for cocept N The results are summarzed Fg.8. whch suggest that the error term captures the true market share more relably for sample sze of 50choose5, based o the error term beg closer to the 95% cofdece terval le ad bootstrap s more relable for sample szes of 50choose10, 50choose20, ad 50choose30, sce bootstrap s closer to the 95% cofdece terval le ad both the methods are equally relable for sample sze of 50choose % 105% 95% 90% 85% 80% 75% 70% Error term Bootstrap 50choose5 50choose10 50choose20 50choose30 50choose40 Sample Sze 5 Copyrght 2011 by ISC

6 Accuracy Fgure 8. Relablty of bootstrap ad error term methods The accuracy s measured by calculatg the absolute dfferece betwee the average market share ( Avg ) ad true market share from earler ( True ) for all the 200 BS ad C samples for the 20 smulatos as show Eq.(9). Accuracy s Avg sc True The results suggest that the accuracy for C methods greater tha bootstrap the case of 50choose5, 50choose10, 50choose20 ad 50choose30 utl they coverge at 50choose40 as show Fg (9) Error term Bootstrap 50choose5 50choose10 50choose20 50choose30 50choose40 Sample Sze Fgure 9. Accuracy of bootstrap ad error term 4. CONCLUSION AND FUTURE WORK Cojot aalyss s a statstcal method that has bee creasgly corporated desg egeerg as a method to estmate the market share of a ew product desg; however, the use of choce-based CA data to model market share ucertaty has ot bee fully explored the past desg egeerg research. Ths paper has compared bootstrap ad C smulato of utlty estmato errors to model demad ucertaty usg cojot aalyss data obtaed from o- Bayesa choce-based CA. The results suggests that there are mxed advatages both the approaches terms of accuracy ad relablty. Ths paper used choce-based data obtaed from o- Bayesa CA wth customers choce dcates preferece. Comparso of relablty ad accuracy of demad modelg usg data obtaed from other cojot aalyss methodologes, partcular, Bayesa choce-based cojot aalyss, s a topc for future work. Future work to mprove ucertaty modelg for decsoaalytc cocept selecto cludes ucertaty of compettor reactos after a ew product s troduced to the market ad ucertaty of customer preferece due to chages exogeous varables (such as fuel prce) that mpact future customer preferece. ACKNOWLEDGEENTS We would lke to thak the Itellget Systems Ceter at ssour Uversty of Scece ad Techology for supportg ths research. REFERENCES [1] L, H., ad Azarm, S., 2000, Product Desg Selecto Uder Ucertaty ad wth Compettve Advatage, Joural of echacal Desg, 122, pp [2] L, H., ad Azarm, S., 2002, A Approach for Product Le Desg Selecto Uder Ucertaty ad Competto, Joural of echacal Desg, 124, pp [3] Wllams, N., Azarm, S., ad Kaa, P. K., 2008, Egeerg Product Desg Optmzato for Retal Chael Acceptace, Joural of echacal Desg, 130, pp [4] Wasseaar, H. J., ad Che, W., 2003, A Approach to Decso-Based Desg wth Dscrete Choce Aalyss for Demad odelg, Joural of echacal Desg, 125, pp [5] Wasseaar, H. J., Che, W., Cheg, J., ad Sudjato, A., 2005, Ehacg Dscrete Choce Demad odelg for Decso-based Desg, Joural of echacal Desg, 127, pp [6] chalek, J. J., Cerya, O., Papalambros, P. Y., ad Kore, Y., 2006, Balacg g ad aufacturg Objectves Product Le Desg, Joural of echacal Desg, 128, pp [7] Shau, C. N., ad chalek, J. J., 2009a, Should Desgers Worry About Systems? Joural of echacal Desg, 131, pp [8] Shau, C. N., ad chalek, J. J., 2009b, Optmal Product Desg Uder Prce Competto, Joural of echacal Desg, 131, pp [9] cfadde, D., 1974, Codtoal Logt Aalyss of Qualtatve Choce Behavor, Froters o Ecoometrcs. Academc Press, New York, pp [10] Louvere, J. J., ad George, W., 1983, Desg ad Aalyss of Smulated Cosumer Choce o Allocato Expermets: A Approach Based o Aggregate Data, Joural of g Research, pp [11] Rao, V. R., Wter, F. W., 1978, Applcato of the ultvarate Probt odel for Segmetato ad Product Desg, Joural of g Research, 15, pp [12] Gree, P. E., ad Wd, Y., 1975, New Way to easure Cosumers Judgmets, Harvard Busess Revew, pp [13] Gree, P.E. ad Srvasa, V., 1978, Cojot aalyss cosumer research: ssues ad outlook, Joural of Cosumer Research., 5, pp [14] Gree, P.E. ad Srvasa, V., 1990, Cojot aalyss marketg: ew developmets wth mplcatos for research ad practce, Joural of g., 54, Copyrght 2011 by ISC

7 [15] Louvere, J.J., Hesher, D.A., Swat, J. ad Adamowcz, W., 2000, Stated Choce ethods: Aalyss ad Applcatos, (Cambrdge Uversty Press, Cambrdge, UK). [16] Louvere, J. J., ad George, W., 1983, Desg ad Aalyss of Smulated Cosumer Choce o Allocato Expermets: A Approach Based o Aggregate Data, Joural of g Research, pp Copyrght 2011 by ISC