NEIA/NEMRA 2018 Spring Conference May 22 nd, Market Replication Models A Case Study on Why Primary Research is Still Essential

Transcription

1 NEIA/NEMRA 2018 Spring Conference May 22 nd, 2018 Market Replication Models A Case Study on Why Primary Research is Still Essential

2 Two Takeaways One Thing to Remember Primary research on new product adoption is critical. Discrete Choice is significantly better than Conjoint. The key is to replicate the market as closely as possible. 2

3 "Had Strong Impact" 80% 70% 60% 50% 40% Taste Tests Tear-down Studies Sponsorships/Sports Marketing Product Adoption / Market Forecasting Sales Analysis Direct Marketing /Campaign Ideation Modeling/Multivariate Usability Testing Ethnographies Clear Winners (Average 66% Strong Impact) Pricing Profiling/Segmentation Advertising Branding In-Depth Interviews Tracking New Product Development Customer Satisfaction Focus Groups Propositions (Average 50% Strong Impact) 30% Laddering Employee Clear Losers (Average 31% Strong Impact) Big Data 20% 0% 10% 20% 30% 40% 50% 60% 70% 80% Source: Avidelo 2018 "Involved In" 3

4 Case Study: Overview This case study tests two primary research-based modeling methods: Conjoint Discrete Choice While many marketers recognize these techniques as similar, or identical, they differ critically in terms of Accuracy, Consistency, Approach and Capability By the end of this case study, you will have a clear view of how the two methods vary, and which method is clearly superior for the critical primary research needed for predicting new product adoption and pricing. These findings are based on a robust study of the U.S. home gaming console market (Wii, Xbox, PS3, PS2) with more than 1,600 respondents. 4

5 How are Conjoint and Discrete Choice different? Generic Products to Model: Questions: Example Attributes Conjoint is Disjunctive Random combinations generic attributes Which attributes drive interest? backward compatibility, system type, graphic capability, games included, color, hard drive size, price, DVD capability Specific Products to Model: Questions: Example Attributes Discrete Choice is Conjunctive Only relevant alternative specific variation of attributes What is the share at each price and each product? What is the impact of specific new products and features? Product specific variations of price, product availability, PS2 HDD, included games for Xbox 360 5

6 The Most Essential Difference: Conjoint Discrete Choice Share of Preference Market Share Actual Market Choices Accurate Market Share 6

7 Reference Point This study centered on the U.S. market for home video gaming consoles. Conjoint and discrete choice modeling techniques were used to estimate market shares. Actual market share data are reported below: System Units Percent Systems Nintendo Wii 675,100 56% Microsoft Xbox ,700 16% Sony PS3 186,600 17% Sony PS2 132,700 11% 1,203, % Methods should accurately predict these values 7

8 TO4 Screen Layouts Show Differences Conjoint Discrete Choice Focus in conjoint is on attributes Descriptions in DCM are true to the market except where experimental Products profiles are used to study attributes, not responses to specific products Products are used in discrete choice to predict their own share 8

9 Slide 8 TO4 There are animation in this slide to zoom into these examples. Tim O'Rourke, 4/16/2018

10 An Important Detail Data Quality The conjoint task prompted significantly higher fatigue, both in terms of completion time and abandonment rate Method Median Time to Abandonment Complete Rate Choice-Based Conjoint 9 min 18 sec 11.1% Market-Based Discrete Choice Modeling 3 min 26 sec 3.4% Abandonment rate for CBC is more than 3.3 times higher than for Discrete Choice Median time to complete for CBC is 2.7 times longer than for Discrete Choice Modeling Conclusion: A more realistic task is easier and faster for respondents 9

11 Discrete Choice Produced Consistently Accurate Results Share Estimates Wii Xbox 360 PS3 PS2 Actual 56% 16% 17% 11% Discrete Choice Aggregate Logit 57% 17% 16% 10% HB 56% 18% 18% 9% Latent Class 56% 18% 18% 8% Conjoint These estimates are way off! Latent Class 37% 30% 28% 5% HB w/o Interactions 31% 22% 40% 7% HB w/ Interactions 29% 29% 37% 5% Aggregate w/ Interactions 25% 40% 32% 4% Latent Class w/ Interactions 25% 41% 31% 3% Aggregate w/o Interactions 21% 44% 33% 3% 10

12 Key Insight: Wii Should Have Been Priced Higher (Remember? Wii Production was well short of demand.) 100% 90% 80% 70% 60% 50% DCM A B Options given demand and supply Endure shortage and lost profits until supply improves or demand ebbs Improve profits, consider production expansion, model expansion 40% 30% CBC Shortage 20% 10% 0% Loss per Unit (foregone profits) $ $ $ $ Only effective forecasts provide the ability to synchronize and optimize profitability throughout the company, from marketing to manufacturing, and from finance to sales. 11

13 Conclusions on Demand Predictions and Price Sensitivity Clearly possible to pick up demand and pricing signals with Discrete Choice Conjoint is exceptionally disappointing in its performance This stuff really matters!... Poor predictions for Wii were remarkably costly and unexpected 12

14 Did Nintendo Learn Its Lesson? 13

15 Achieve Better Outcomes Don't take chances with intuition when you can harness the power of advanced analytics in getting the right answers to your toughest business questions. Beau Martin CEO & Founder Tim O Rourke Managing Partner tim.orourke@avidelo.com 14