Prediction Markets for Pharmaceutical Concept Testing. John Barrett Infosurv

Transcription

1 Prediction Markets for Pharmaceutical Concept Testing John Barrett Infosurv

2 Key Learning's How to harness the power of Prediction Markets for Pharmaceutical Research How a more engaging survey design can drive speed to completion and responsiveness How a minor change in incentive can drive the thoughtfulness of responses How Prediction Markets can deliver strong quantifiable results for testing positioning statements and creative executions

3 What is a prediction market? Prediction markets are speculative markets created for the purpose of making predictions. The current market prices can then be interpreted as predictions of the probability of the event or the expected value of the parameter. Which creative concept will resonate most effectively with physicians who treat hypertension? Concepts Market Price Probability of Outcome Treatment A $ % Treatment B $8.64 9% Treatment C $ % Treatment D $ % Treatment E $ %

4 An Historical Perspective Organized Presidential prediction markets were active in the US from the 1860 s until WWII The Curb Exchange 92% accuracy rate from 1884 until 1940 Wall Street betting odds are never wrong DARPA Future Map - Policy Analysis Market A market in the future of the Middle East

5 Late 20 th Century Implementation Much wider adoption Iowa Electronic Market 25% average market error reduction vs. polls Hollywood Stock Exchange 92% accurate BetFair BetFair odds (PM) better than experts picks over 90% of the time

6 So how does it work? Which creative concept will resonate most effectively with physicians who treat hypertension?

7 Does it work? Inkling Blog: Judging a prediction market's accuracy, 6/25/08

8 Does it work? Results from a Dozen Years of Election Futures Markets Research, Berg, Forsythe, Nelson, & Rietz; March 2003

9 Does it work? Extracting Collective Probabilistic Forecasts from Web Games, Pennock, Nielsen, & Giles; 2001

10 Does it work? Putting crowd wisdom to work, Google Blog, Bo Cowgill; 9/21/2005

11 Market Design Prediction markets can be designed to predict the outcome of a specific business issue. Which advertisement will be most effective? Which treatment plan are PCP s most likely to adopt? Which positioning platform is most likely to be effective with specialists?

12 Who should participate? Trader selection should be determined by the business issue Longitudinal Designs For some business issues, on-going feedback is desirable. A longitudinal (or open) market allows traders to buy & sell shares as new information develops. Cross-Sectional Designs Some business issues only require a snapshot of current sentiment. Cross-Sectional markets provide you with a current forecast much like a survey.

13 Example of a Longitudinal Prediction Market Objective: Monitor project progress against deadline Market Question: Will product X be launched by December 31, 2015? Available Assets: Yes ($50), No ($50) Participants: All employees $ $ $80.00 $60.00 $40.00 $20.00 $- Yes No

14 Longitudinal Prediction Markets Pros: Highly accurate, platform for collaborative forecasting Cons: High cost of maintenance, rigid design choices

15 Example of a Cross-Sectional Prediction Market Objective: Determine the most effective advertising campaign for a new treatment Market Question: Which creative concept will resonate most effectively with physicians who treat hypertension? Available Assets: Ad A ($25), Ad B ($25), Ad C ($25), Ad D ($25) Participants: Sufferers and/or Individuals familiar with hypertension Concepts Average Price Ad A $14.84 Ad B $41.83 Ad C $22.15 Ad D $21.18 Field time: hours

16 The Trader Experience Introduction to Prediction Markets Self-selection for category/user knowledge Non-disclosure Agreement Background Exposure to concepts Trading Motivations & Optimization

17 Key Principles Prediction markets are effective tools for aggregating crowd wisdom Answers the question What would they do? Five Principles to follow Perspective Contextual understanding Diversity Diverse crowd of individuals Independence Immunity from coercion and manipulation Decentralization Use of local knowledge Aggregation Create a collective judgment

18 Cross-Sectional Prediction Markets Pros: Highly accurate, platform for screening and optimization Cons: Specialized recruiting

19 Illustrating Crowd Wisdom: Case #1 Objective: Identify concept with greatest revenue potential Tested four (4) men s hair care products Test Group A: Mall intercept, targeted user Test Group B: Online exercise, gen-pop screened for familiarity with category Concept Group A Index* Group B VWAP** Group B Index Concept L 102 $ Concept B 100 $ Concept O 85 $ Concept Q 75 $ *Strength Index determined by mean responses to six survey question ** Volume Weighted Average Price

20 Illustrating Crowd Wisdom: Case #2 Objective: Identify ad campaign with greatest impact Tested four (4) print ads Group A Group B Group C Group D Specialists Patients Target General healthcare providers General Population Method Survey to determine trust, believability, and intent to prescribe Prediction market to determine most effective ad (What will others like you do?) Prediction market to determine most effective ad (What will specialists do?) Prediction market to determine most effective ad (What will specialists do?)

21 Illustrating Crowd Wisdom: Case #2 Results Specialists (Survey)* Patients (PM)** HCP s (PM)** GenPop (PM)** Concept A 28% Concept B 26% Concept C 23% Concept D 23% *Percent of respondents indicating Likely or Very Likely to prescribe ** Net preference score for concept based on % Most Likely vs. % Least Likely to persuade

22 Case Study: Why Behind the Buy 7 versions of new creative print ads were tested N=350 traders recruited from a balanced, general population online panel Traders self-selected for familiarity with the category

23 K indexed 50 points ahead of next-best ad Rank Advertisement VWAP Index 1 Advertisement K $ Advertisement I $ Advertisement W $ Advertisement G $ Advertisement T $ Advertisement R $ Advertisement N $

24 Positive and negative response to ads illustrates resonance and polarity Net Preference % of traders Most Likely Least Likely

25 Advertisement K had a strong emotional appeal Count Percent Why it would be successful Feedback 23 19% Appeals to emotions / "Human" feel 19 16% 17 14% 17 14% Image of the child resonates / caring for the child Appeals to desire to help others & change lives / Displays humility Like the language (changing lives of others and your own) 11 9% Offers personal reward / helping yourself 11 9% 9 7% 7 6% 5 4% Gives a sense the profession has many rewards / You can make a difference See professionals making a difference / Accurately reflects what I think a professional really is and does The professional looks caring and concerned / Shows passion nurses have Expresses professional s confidence in situation 5 4% Most appealing 5 4% Speaks to dedication of nurses With whom would it resonate best? 88 72% Those seeking to start a career in xxx 25 20% I think it will resonate most effectively because of the tagline and picture. The picture shows a caring nurse tending to a child patient. That should affect students on a emotional level. I think all students want to change or help lives so the tagline works. Practicing professionals looking to further their education

26 Most thought K could be improved by more positive patient imaging Rating 34 What would make it more successful Suggestion Maybe show the parent's smiling face in the background, because she's pleased with the care her child is receiving. 18 Encouragement and knowledge 17 Show the parent's face. 13 ABC College makes great nurses. 11 Ad is superb "as is" 11 A more genuine smile on the face of the professional. Children smiling showing that they are feeling better with the professionals assistance in "changing their lives". They look uncomfortable. The parent should be shown in the shot with a pleased smile. 3 More participation of a professional helping 3 The professional almost looks more like an EMT than a professional with the clothing she is wearing 3 Show more of the warmth of the professional. 3 Have the professional smiling, her facial expression looks forced

27 Emotional Drivers Align with the Advertisement s Main Messaging Understanding emotional drivers helps marketers develop communications that connect with the emotional needs of their audience. 89% of respondents believed concept K would resonate most effectively because of the sense of connection (caretaking & bonding with others) - a primary theme of the advertisement. Half of the respondents also indicated a sense of dreaming (hope & inspiration) as primary drivers of their preference for concept K, while 41% were compelled by a feeling of mastery (acquiring & improving skills).

28 Study Take-away s Clear direction on which creative versions to consider for further development Advertisements with limited traction were discarded Specific direction for refinement of the messaging Parental involvement

29 What do Prediction Markets deliver? Clear differentiation between concepts Forced choice ranking Why behind the Buy Respondent Engagement Richer, candid feedback

30 How Do They Compare? Survey Prediction Market Targeted What would you do? Dozens of metrics Flat incentive 3-4 weeks General Population What would they do? Most important number Performance-based incentive 3-5 days

31 Why use Prediction Markets? Greater discrimination than traditional survey-based approach Speed: Results can be in-hand within weeks Increased Engagement = Richer Diagnostics

32 What do clients say? I am tired of seeing flat-lined top 2 box scores. Prediction Markets give me greater discrimination than traditional testing methods. Prediction Markets eliminate the over-stated purchase intent factor. My Prediction Market gave me the same rank order and relationship between the concepts as my traditional approach. The depth of insight from the traders in the Prediction Market allowed me to completely re-think my strategy and go to market with a vastly improved execution.

33 When to Use Prediction Markets Positioning statements Creative screening Physician and DTC Early stage prioritization Concept attribute strength analysis Concept Optimization

34 Q&A John Barrett VP, Client Solutions