A CUSTOMER-PREFERENCE UNCERTAINTY MODEL FOR DECISION-ANALYTIC CONCEPT SELECTION

Transcription

1 Proceedings of the 4th Annual ISC Research Symposium ISCRS 2 April 2, 2, Rolla, Missouri A CUSTOMER-PREFERENCE UNCERTAINTY MODEL FOR DECISION-ANALYTIC CONCEPT SELECTION ABSTRACT Analysis of customer preferences is one of the most important tasks in new product development. How customers come to appreciate and decide to purchase a new product impacts market share and, therefore, the success of the new product. Unfortunately, when designers select a product concept early in the product development process, the market share of the new product is unknown. Conjoint analysis is a statistical methodology that has been used to estimate the market share of a product concept from customer survey data. Although conjoint analysis has been increasingly incorporated in design engineering as a tool to estimate market share of a new product design, it has not been fully employed to model market share uncertainty. This paper presents two approaches, which use conjoint analysis data to model market share uncertainty: bootstrap and binomial inference. Demonstration and comparison of the two approaches are presented using an illustrative example.. INTRODUCTION In product development, engineers select a product concept before they develop detailed designs and prototypes []. At the time of concept selection therefore, future market size, market share, competition, warranty cost, and product cost are uncertain. Uncertainties directly relevant to concept selection are modeled. These uncertainties include market share, warranty cost, and product cost. The present research addresses market share uncertainty modeling as a means to select a concept with the maximum expected utility of profit. Conjoint Analysis (CA), a method to measure consumer judgments, has been used in a variety of applications and has received considerable attention since the early 97s. This method has most often been applied in the fields of applied psychology, decision theory, and economics [2]. Robinson [] reports a multinational conjoint study of North Atlantic air travel involving airfare, discounts and travel restrictions. This study indicates that CA can accurately predict market shares. Benbenisty [4] published a conjoint study involving AT&T s entry into the data terminal market. His simulator forecasted an 8% share for AT&T four years after launch and obtained an actual share of just under 8%. These developments have been discussed by Paul E. Green and V. Srinivasan [5]. Hildebrand [6] has used CA for market definition, which is instrumental for the assessment of market power and central to competition policy. Currently however CA generates only one value for the percentage of market share for a particular concept or concepts. To overcome this limitation, the first approach, we propose uses bootstrap [7], which is a sampling with a replacement method that permits calculation of sample statistics. Researchers have applied bootstrap to make statistical inferences in clustering analysis and in phylogenetic trees [8- ]. Felsenstien [8] and Efron et al. [9] incorporated k-by-p data matrix consisting of k species and p sites. They generated bootstrapped samples of data matrices by sampling columns with replacement. By applying bootstrap to the results obtained from CA, a continuous distribution is obtained, which improves results. Furthermore, the continuous distribution can be discretized to obtain probabilities by using the Extended Pearson-Tukey method [4]. The second approach, which we propose as an approximation to bootstrap, is binomial inference. Binomial inference may be explained using a coin-flipping analogy [5]. In coin flipping, if we observe H heads and T tails in H+T flips, an uncertainty of probability of head is modeled by a beta distribution with parameter (H, T). Applying a similar analogy, this work proposes that the uncertainty of market share of a concept selected by M customers out of N total customers is modeled by a beta (M, N M) distribution. This paper is organized as follows: Section 2 briefly describes conjoint analysis, bootstrap, and a framework to model market share uncertainty by integrating the two. Section demonstrates and compares the proposed two approaches (bootstrap and binomial inference) in an illustrative example. Section 4 concludes the paper with a discussion of future work. 2. METHODOLOGY 2.. Conjoint analysis Conjoint analysis for estimating the market share of a new product concept involves the following steps: Copyright 2 by ISC

2 Concept definition: Identify product attributes and their levels that are important for customers to make purchasing decisions. Define new product concepts and competitors products as combinations of attributes and their levels. Product attributes may be identified by interviewing customers and translating the customer needs into product attributes. Attribute level identification: Benchmark existing products in the marketplace or forecast future customer needs for a new product to identify attribute levels included in conjoint analysis survey. Conjoint survey design: Determine which conjoint method will be used. That is, select the respondents concept evaluation method (rating/ranking or choice), completeness of profile to be used to describe concepts (full or partial profile), and decide whether or not to use a Bayesian approach (Hierarchical Bayes or non-hierarchical Bayes). Create concept profiles to be displayed to respondents according to the chosen method. Market share estimation: Estimate the market share of the concepts by analyzing respondents concept evaluation results Bootstrap Bootstrap is a computer-based sampling-with-replacement method that has been used to obtain a confidence interval of an estimate as illustrated using a simple example in Fig.. Suppose we wish to estimate a population average from a randomly sampled data set {, 2,, 4, 5, 6, 7}. We calculate a sample average 4 and use this as an estimate of population average. Because this is a point estimate of population average, no confidence interval of this estimate can be obtained from the initial data set alone; bootstrap samples of the same data size must be generated by sampling with replacements from the initial data set to obtain confidence intervals. For example, the first bootstrap sample may be {5,, 7,, 2, 4, 4}, the second Sampling with replacements Data Sample statistics (average) Initial sample : {, 2,, 4, 5, 6, 7 } 4 Bootstrap samples st : { 5,, 7,, 2, 4, 4 }.4 2 nd : { 6, 4, 2, 5, 7,, 6 } 4.7 rd : { 2,, 5, 2,,, } 2. 2 th : { 4, 6, 4, 7,, 4, 2 } 4. Distribution Inference (95% confidence interval) 4 7 bootstrap sample may be {6, 4, 2, 5, 7,, 6}, and so forth. Fig. Bootstrap procedure In the bootstrap samples, the same data may appear more than once or do not appear at all due to the sampling-withreplacement procedure. Each of these bootstrap samples provides a sample statistic (i.e., average). If 2 bootstrap samples are generated, 2 sample averages will be created. These sample averages permit a construction of a confidence interval or a distribution of sample statistics. For example, the 5th percentile and the 95th percentile of these bootstrap averages provide a range of 95% confidence interval of the point estimate 4, and a histogram of bootstrap sample averages provides distribution of sample statistics, as illustrated in Fig Bootstrap application to conjoint analysis In conjoint analysis, respondents are randomly selected from populations of customers as illustrated in Fig. 2. Although conjoint analysis yields a concept s true market share if it is applied to the entire population of customers, surveying the entire population is not feasible. By asking randomly selected customers to evaluate product concepts and competitors products, a point estimate of the market share of a concept is obtained, as illustrated by the middle flow from left to right in Fig. 2. Using a point estimate for the analysis is equivalent to assigning a probability of one to the point estimate. Population Customer {, 2,,, N} Sample Random sampling Customer {, 2,,, n} Conjoint analysis "True" market share, S True Conjoint analysis Point estimate of market share, s Estimate Probability S True Probability.5 s Estimate 5% Bootstrap Conjoint analysis Probability { st sample} Market share, s Market share % Market share % { 2nd sample} Market share, s 2.5 { B-th sample} Market share, s B 5% Market % share Figure 2 Application of bootstrap to conjoint analysis In contrast, if bootstrap is applied to conjoint analysis, bootstrap samples are generated from an initial set of randomly sampled customers, as illustrated in the bottom flow in Fig. 2. In the bootstrapped samples, a customer may appear more than once or may not appear at all because of the sampling-withreplacement procedure. By applying conjoint analysis to the bootstrap samples, market share estimates are obtained from which a distribution of market share can be constructed as illustrated in Fig. 2.. Illustrative example This section illustrates and compares two market share uncertainty modeling approaches: the application of bootstrap and binomial inference to conjoint analysis. This example presents non-bayesian full-profile conjoint analysis with customers evaluations in rating scales using automobile concept selection as an illustrative example; however, both approaches can be applied to any conjoint analysis approach...concepts and competition For an illustrative purpose, this example assumes that a firm wishes to estimate the future market share of a new automobile (N) that will compete with two competitor vehicles (C and C2). The concept of a new automobile is defined by its type and fuel efficiency. Type refers to its form and the maximum number of passengers that it can accommodate, and.5 5% 2 Copyright 2 by ISC

3 fuel efficiency is associated by an engine type (a gasoline engine or a hybrid engine). Furthermore, the firm selects a basic warranty and a price both of which influence market share. The firm selects a sport utility vehicle (SUV) as a type of the concept, 25 miles per gallon as a fuel efficiency, 5/6, (years/miles) as a basic warranty, and $5, as a price as summarized in Fig.. Type SUV Concept Fuel Efficiency Warranty Figure 4 summarizes the features of the two competitor vehicles. The first competitor car (C) is a convertible that gets miles per gallon (gasoline engine). It has a basic warranty of years/6, miles and a price of $2,. The second competitor car (C2) is a sedan that gets 4 miles per gallon (hybrid engine). It has a basic warranty of 4 years/5, miles and a price of $5,. Figure 4 Competitor cars C and C2.2.Market share point estimate To estimate market share by applying non-bayesian fullprofile conjoint analysis with customer evaluations in rating scales, the firm first identifies possible levels of fuel efficiency, warranty, and price to be analyzed in conjoint analysis. Based on the automobiles introduced to the market between 2 and 29 (87 SUVs, 29 convertibles, and 57 sedans), fuel efficiency, warranty, and price are benchmarked as summarized in Tables through. Table shows the minimum, average, median, and the maximum fuel efficiency of the benchmarked vehicles. Based on the benchmarking results, three levels of fuel efficiency are selected for the conjoint analysis study:, 25, and 4 miles per gallon. Table Fuel Efficiency (Miles per Gallon) Price N 25 5 / 6, $ 5, 8 passengers (miles/gallon) (years/miles) Figure Selected combination of concept, warranty, and price Type Fuel Efficiency Warranty Price Convertible Non-hybrid C / 6, $2, 2 passengers (miles per gallon) (years/miles) Sedan Hybrid C2 4 4 / 5, $5, 5 passengers (miles per gallon) (years/miles) SUV Convertible Sedan Min 7 7 Average Median Max Table 2 summarizes the frequency of basic warranties offered for the benchmark vehicles. Based on the benchmarking results, the three most widely offered basic warranties are selected for the conjoint analysis study: /6, 4/5, and 5/6 years/miles. Warranty (Years/Miles) SUV Convertible Sedan /6, /5, /6, /6, 2 Table 2 Basic Warranty Frequency Finally, Table shows the minimum, average, median, and maximum price of the benchmarked vehicles. Based on the benchmarking results, three price levels are selected for the conjoint analysis study: $2,, $5,, and $5,. SUV Convertible Sedan Min 2,972 7,65 2,42 Average 6,9 4,579 6,798 Median 6,5 5,5 28,45 Max 67,82 5,855 9,2 Table Price ($) To estimate market share, the firm must identify 8(= ) part worths of three levels of four attributes (type, fuel efficiency, warranty, and price) for each respondent. Based on the part worths, the firm predicts each respondent s utilities of the concept (as well as warranty and price) in Fig. and two competing vehicles in Fig. 4. By comparing the utility of the concept (and warranty and price) against utilities of competing vehicles, the firm can predict which of the concept and competitors vehicles each respondent will choose. From the predictions of respondents choices, the firm estimates the market share of a concept as well as that of each competitor vehicle. Part worths can be identified by asking respondents to rate a total of 8(= ) combinations of attribute levels and analyze the rating results by, for example, ordinary least square regression analysis. However, evaluations of 8 alternatives may impose a significant burden on respondents. To reduce that burden, the firm may use an L9 orthogonal array so that respondents evaluate only the nine combinations. From respondents evaluations of these nine combinations using a 9-point rating scale (with being the least preferred and 9 being the most preferred), the firm can obtain the respondents part worths. Figure 5 shows the part worth of a respondent. This individual prefers an SUV over a convertible or sedan, as indicated by its part worth (.), which is larger than that of the convertible ( 2.) or the sedan (.). Similarly, the highest fuel efficiency of 4 miles/gallon, the best warranty of 5 years/6, miles, and the lowest price of $2, are the preferred levels, with the highest part worth within each attribute. The part worths in Fig. 5 are added to obtain the utilities of concept N in Fig. and the two competitor vehicles C and C2 Copyright 2 by ISC

4 Relative Frequency Relative Frequency in Fig. 4. For example, the utility of C (a convertible with a fuel efficiency of miles/gallon, a warranty of years/6, miles, and a price of $2,) is calculated by adding the part worths of a convertible vehicle type, a fuel efficiency of miles/gallon, a years/6, miles warranty, a price of $2,, and the intercept: = Figure 5 Part worth example Figure 6 Utilities of N, C, and C2 for the ten respondents, R R After calculating utilities of N, C, and C2 for each respondent, the firm can estimate market share of the concept N. Figure 6 summarizes the utilities of concept N and the competitor vehicles C and C2 for all respondents; the largest utility of each respondent is highlighted. Respondents, 2, and 5 choose N because it has the highest utility of all three alternatives. On the same basis, respondent chooses C and respondents 4, 6, 7, 8, 9, and choose C2. Based on these results, the firm would estimate the market shares of these three alternatives to be % for N, % for C, and 6% for C2... Convertible SUV Sedan Type Fuel Efficiency (Miles per Gallon) /6, 4/5, 5/6, Warranty (Years/Miles) , 5, 5, Price ($) Attribute levels Utility Type Fuel efficiency Warranty (miles/gallon) (years/miles) Price R R2 R R4 R5 R6 R7 R8 R9 R N SUV 25 5/6, $5, C Convertible /6, $2, C2 Sedan 4 4/5, $5, Market Share Uncertainty Modeling: Bootstrap Bootstrap distributions of market share can be obtained by applying bootstrap to the conjoint analysis data. Using the original sample of respondents (R-R), the samplingwith-replacement procedure is applied to generate 2 bootstrap samples. Each bootstrap sample consists of data from respondents; however, each respondent may appear more than once or may not appear at all as shown in Fig. 7. st bootstrap sample = { R7, R, R8, R, R9, R6, R6, R9, R9, R6 } 2st bootstrap sample = { R, R8, R7, R6, R2, R7, R5, R6, R8, R } rd bootstrap sample = { R, R, R, R9, R4, R5, R2, R, R4, R2 } 4th bootstrap sample = { R8, R6, R, R, R8, R9, R8, R, R, R8 } 5th bootstrap sample = { R, R4, R5, R, R4, R, R9, R, R, R7 } 2th bootstrap sample = { R5, R, R9, R9, R2, R4, R, R, R4, R5 } Fig. 7 Bootstrap samples of respondents By mapping respondents in 2 bootstrap samples to their predicted choices, as illustrated in Fig. 8, the firm obtains 2 market share estimates for concept N and for the competitor vehicles C and C2. From these market share estimates, the firm constructs distributions of market share for its concept N and for the competitor vehicles; as illustrated in Fig. 9. Bootstrap samples Market share (%) N C C2 st = { C2, C2, C2, C2, C2, C2, C2, C2, C2, C2 } 2st = { N, C2, C2, C2, N, C2, N, C2, C2, N } 4 6 rd = { C, C2, C2, C2, C2, N, N, C, C2, N } 2 5 4th = { C2, C2, C2, N, C2, C2, C2, C2, C2, C2 } 9 5th = { C, C2, N, C2, C2, C, C2, C2, C, C2 } 6 2th = { N, N, C2, C2, N, C2, N, C2, C2, N } Fig. 8 Predicted choices Market Share (%) (a) N Market Share (%) (b) C 4 Copyright 2 by ISC

5 Probability Cumulative probability Cumulative probability Relative Frequency Market Share (%) (c) C2 Fig. 9 Market share distributions.4. Market share uncertainty modeling: binomial inference Based on the binomial inference, the market share distributions of N, C, and C2 may be approximated by Beta(,7), Beta(,9), and Beta(6,4) distributions; as illustrated in Fig.. The probability distribution shows in Fig. (b) was obtained by discretizing the cumulative distribution in Fig. (a) into brackets (i.e., [.5,.5], [.5,.5],, and [.95,.5]), calculating the probability of market share (m) in each bracket, and assigning the probability to the middle value of the bracket: For example, Pr(m=.)=Pr(m.5) Pr(m.5). N: Beta(,7) C: Beta(,9) C2: Beta(6,4).5. Comparison of bootstrap and binomial inference An important goal of decision analysis is the approximation (or discretizations) of continuous distributions by discrete probabilities. The extended Pearson-Tukey method is a simple yet accurate three-point approximation in which a continuous distribution is represented by the 5th percentile, 5th percentile (i.e., median), and 95th percentile of the distribution, with probabilities of.85,.6, and.85 respectively. Figure compares the distributions obtained from bootstrap with those from binomial inference (i.e., beta distributions). Table 4 compares statistics of the bootstrap and beta distributions: three points (5th percentile, 5th percentile, and 95th percentile), the means, and the standard deviations of the discretized distributions. The differences of statistics (the percentiles, means, and standard deviations) are small, and differ only by a maximum of.4 percentage points N: Beta(,7) C: Beta(,9) C2: Beta(6,4) N: Bootstrap C: Bootstrap C2: Bootstrap Market share (a) N: Beta(,7) C: Beta(,9) C2: Beta(6,4) Market share (b) Market share Fig. Cumulative distributions N C C2 Beta Bootstrap Diff. Beta Bootstrap Diff. Beta Bootstrap Diff. 5th percentile th percentile th percentile Mean St. dev Table 4 Statistics of discretized distributions Table 5 compares distributions in Figure using a chisquare goodness-of-fit test with a 5% confidence level. According to the results of this test, the null hypothesis that two distributions have a good fit cannot be rejected for all three cases (N, C, and C2); as shown by the p-values larger than.5 in Table 5. Chi-square test N C C2 p-value Test statistic Table 5 Comparison of distributions Fig. Binomial inference 5 Copyright 2 by ISC

6 7. CONCLUSION AND FUTURE WORK Conjoint analysis is a statistical methodology that has been increasingly incorporated in design engineering as a tool to estimate the market share of a new product design; however, the use of conjoint analysis data to model market share uncertainty has not been fully explored in the past design engineering research. This paper has presented two approaches (bootstrap and binomial inference as an approximation of bootstrap distribution) to model market share uncertainty using conjoint analysis data obtained from non-bayesian full-profile conjoint analysis with customers evaluations in rating scales. Once discretized using the extended Pearson-Tukey method, the beta distributions (obtained from binomial inference) were compared with the bootstrap distributions. The results indicated that there are small differences in the statistics (the 5th percentiles, 5th percentiles, 95th percentiles, means, and standard deviations) of discretized distributions. Furthermore, chi-square goodness-of-fit tests of beta distributions and bootstrap distributions indicated that these distributions have good fits. These results support the use of a beta distribution, which is obtained from a binomial inference of conjoint analysis data, as a means to approximate bootstrap distribution; thus, to model market share uncertainty. Continuation of this avenue of research, which is to support this preliminary finding with a larger number of respondents, is future work. This paper used conjoint analysis data obtained from non- Bayesian full-profile conjoint analysis with customers evaluations in rating scales. Market share uncertainty modeling using data obtained from other conjoint analysis methodologies, in particular, non-bayesian and Bayesian choice-based conjoint analysis, is a topic for future work. The accuracies of a market share forecast depend on competitors products simulated in the conjoint analysis. Furthermore, the future actions of competitors in response to firm s new product influence the accuracy of market share forecasts [6, 7]. Future work, therefore, should study the integration of competition uncertainty modeling into market share uncertainty modeling in conjoint analysis. 8. ACKNOWLEDGMENTS We would like to thank the Intelligent Systems Center at Missouri University of Science and Technology for supporting this research. 9. REFERENCES [] Ulrich, K. T., and Eppinger, S.D., 24, Product Design and Development, McGraw-Hill, New York. [2] Green, Paul. E., and Srinivasan., V., 978, Conjoint Analysis in Consumer Research: Issues and Outlook Journal of Consumer Research, Vol. 5, pp. -2. [] Robinson, P. J. 98, Application of Conjoint Analysis to Pricing Problems, in Proceedings of the First ORSA/TIMS Special Interest Conference on Market Measurement and Analysis, D. B. Montgomery and D. R. Wittink, eds. Cambridge, MA: Marketing Science Institute, pp [4] Benbenisty, R, L., 98, Attitude Research, Conjoint Analysis Guided Ma Bell's Entry Into Data Terminal Market, Marketing News (May ), 2. [5] Green, P. V., and Srinivasan, V., 99, Conjoint Analysis in Marketing: New Developments with Implications for Research and Practice, Journal of Marketing (October 99), pp. -2. [6] Doris, H., 26, Using Conjoint Analysis for Market Definition: Application of Modern Market Research Tools to Implement the Hypothetical Monopolist Test. World Competition 29(2), pp [7] Efron, B., and Tibashirani, R., 99, An Introduction to the Bootstrap, Chapman & Hall, London. [8] Felsenstien, J.,985, Confidence Limits on Phylogenies: An Approach Using the Bootstrap, Evolution, 9(4), July, pp [9] Efron, B., Holloran, E., and Holmes, S., 996, Bootstrap Confidence Levels for Phylogenetic Trees, Proceedings of the National Academy of Sciences of the United States of America, 9(2), pp [] Kerr, M. K., and Churchill, G. A., 2, Bootstrapping Cluster Analysis: Assessing the Reliability of Conclusions from Microarray Experiments, Proceedings of the National Academy of Sciences of the United States of America, 98(6), pp [] Holmes, S., 999, Phylogenies: An Overview, in Halloran, M. E., and Geisser, S., eds., Statistics and Genetics, IMA Volumes in Mathematics and its Applications, 2, Springer Verlag, New York, NY. [2] Holmes, S.P., 22, Statistics for Phylogenies, Theoretical Population Biology, 6, pp [] Holmes, S., 2, Bootstrapping Phylogenies, Statistical Science, 8(2), pp [4] Reilly, T., 22, Estimating Moments of Subjectively Assessed Distributions, Decision Sciences, (), pp. -8 [5] Howard AR (97) Decision analysis: Perspectives on inference, decision, and experimentation. In: Proceedings of the IEEE 58(5): [6] Choi SC, Desarbo WS (99) Game theoretic derivations of competitive strategies in conjoint analysis. Marketing Letters 4(4): 7-48 [7] Green PE, Krieger AM (997) Using conjoint analysis to view competitive interaction through the customer s eyes. In: Wharton on Dynamic Competitive Strategy, Day GS, Reibstein DJ, Gunther RE (eds) John Wiley, New York 6 Copyright 2 by ISC