Methods for Eliciting Preferences: Conjoint Analysis and Discrete Choice Experiments. Duane Blaauw

Transcription

1 Methods for Eliciting Preferences: Conjoint Analysis and Discrete Choice Experiments Duane Blaauw

2 Session Objectives Basic introduction to conjoint analysis and DCE Principles Potential applications Rigour in DCE Understand literature Unfortunately NOT: Detailed technical expertise Computer skills

3 Overview Basic principles Advantages / Use Practicalities: Step by step guide to doing a DCE Limitations Applications in health

4 Some Terminology Conjoint analysis (CA) Discrete choice experiment (DCE) Attributes Levels Profiles Utility

5 BASIC PRINCIPLES

6 Quantitative Method Quantitative approach to eliciting preferences (Table) Limitations of alternative methods (Table) 1. Rating No relative strength of preferences All factors important Generally want all attributes at low cost 2. Ranking No relative strength of preferences Unclear implementation of decision rule from results Ignores marginal context of decisionmaking (opportunity costs)

7 Economic Method Economic values Learn true value of things when force respondents to make economic trade-offs Random utility theory Decision makers are rational Choices aim to maximise the utility gained Alternative economic methods Contingent valuation (Traditional willingness to pay) Don t evaluate trade-offs between non-financial factors Lancaster theory of value Goods are made up of attributes Total utility gained from a product or service is the sum of the individual utilities provided by the attributes of that good Part-worths / Part utilities Choices determined by particular combinations of product attributes Stated preferences vs Revealed preferences Hypothetical choices rather than real market choices

8 Conjoint Analytical Method Trade-off analysis Evaluate conjoint characteristics Quantitative analysis of choices where options simultaneously vary across 2 or more attributes Trade-offs critical to choices in real world Able to measure the relative importance of different product attributes Decompositional method Respondents evaluate total products (combinations of attributes) from which we mathematically infer the utilities of the individual attributes comprising the products Identify underlying value system from the choices made about total products Derive interval scale data from ordinal/binary data

9 A Simple Conjoint Analysis Purchasing a laptop Power Rank Weight Rank 1.5 GHz 500g 2.0 GHz 1000g 3.0 GHz 1500g

10 A Simple Conjoint Analysis Purchasing a laptop Power Rank Weight Rank 1.5 GHz 3 500g 2.0 GHz g 3.0 GHz g

11 A Simple Conjoint Analysis Purchasing a laptop Power Rank Weight Rank 1.5 GHz 3 500g GHz g GHz g 3

12 A Simple Conjoint Analysis Weight 500g 1000g 1500g 1.5 GHz Power 2.0 GHz 3.0 GHz Trade-off analysis

13 A Simple Conjoint Analysis Weight 500g 1000g 1500g 1.5 GHz 9 Power 2.0 GHz 3.0 GHz 1

14 A Simple Conjoint Analysis Weight 500g 1000g 1500g 1.5 GHz Power 2.0 GHz GHz 1 3 7

15 A Simple Conjoint Analysis Weight 500g 1000g 1500g Ave 1.5 GHz Power 2.0 GHz GHz Ave

16 A Simple Conjoint Analysis Weight 500g 1000g 1500g Ave 1.5 GHz Power 2.0 GHz GHz Ave

17 A Simple Conjoint Analysis Weight 500g 1000g 1500g Ave 1.5 GHz Power 2.0 GHz GHz Ave

18 A Simple Conjoint Analysis Weight 500g 1000g 1500g Ave 1.5 GHz Power 2.0 GHz GHz Ave

19 Historical Development Origins Pschyometrics Econometrics Application Market research Transport economics Environmental economics Health (Late 1990s) Key milestones Luce & Tukey (1964) Simultaneous Conjoint Measurement: A New Type of Fundamental Measurement, Journal of Mathematical Psychology, 1, 1-27 Green & Rao (1971) Conjoint Measurement for Quantifying Judgmental Data, Journal of Marketing Research, 8, McFadden (1974) Conditional logit analysis of qualitative choice behaviour. In: Zarembka (Ed), Frontiers in Econometrics, Academic Press, New York Louviere & Woodworth (1983) Design and Analysis of Simulated Consumer Choice or Allocation Experiments, Journal of Marketing Research, 20,

20 Basic Approach Present different composite products Respondents evaluate total products Rate, rank, discrete choices Analyse choices to infer underlying values Provides estimation of: Relative importance of different attributes (part-worths / utilities) Willingness of respondents to trade-off between attributes Relative benefit/utility scores of different combinations Values of different subgroups

21 Travel Questionnaire

22 Basic Results Variable Airline Flight duration Type of seat Food Cost (/$500) Rho (ρ) Prob (LR χ 2 ) Coeff <0.001 P value <0.001

23 Trade-Offs Airline Flight duration Type of seat Food Cost Airline Flight duration Type of seat Food Cost (/$500) $ $ $ $

24 Utility of Different Products Option Airline Duration Seat Food Cost Utility Score 1 Virgin 5 Hrs + Legroom Meal $ Virgin 7 Hrs + Legroom Meal $ Virgin 5 Hrs + Legroom Meal $ Virgin 5 Hrs + Legroom Snack $ SAA 5 Hrs + Legroom Meal $ Virgin 7 Hrs + Legroom Meal $ Virgin 5 Hrs + Legroom Meal $ Virgin 5 Hrs Normal Meal $ Virgin 7 Hrs + Legroom Snack $ SAA 7 Hrs + Legroom Meal $

25 Advantages of CA Mimics real choice behaviour Types of decisions that consumers make every day Task easily understood by respondents Provide policy relevant information Relative importance of underlying values Tradeoffs between attributes Utility of all possible combinations WTP if include cost variable Basic results easy to interpret Well received by policy makers

26 Requirements for CA Products or service which are made up of bundles of attributes Most important attributes of products or services have been identified Respondents are familiar with the concepts and can rate the products or services Attributes and levels should be actionable

27 DOING A DCE

28 Main Steps 1. Clarify study objectives 2. Select attributes 3. Assign levels of attributes 4. Construct and select profiles (experimental design) 5. Decide on type of conjoint analysis 6. Design and pilot tool 7. Plan sampling strategy 8. Collect data 9. Analyse data 10. Interpret results

29 1. Clarify Study Objectives Small changes in study objectives affect DCE design Analytical objectives also influence design Main effects Interaction effects Sub-group analysis Alternative specific effects Cross-effects

30 2. Select Attributes Key attributes which define product or service (Table) Limited by experimental design consideration Usually 6 Therefore require careful selection Should be: Influential in real decision-making (determinant) A reasonably complete set Independent Uni-dimensional Based on: Literature review Expert opinion Key informant interviews Focus group discussions Surveys Policy relevance

31 3. Assign Levels Important variants of each attribute (Table) Cardinal / Ordinal / Nominal Usually 2-5 levels for each attribute Equal number of levels for each attribute produces more efficient designs Should: Be independent, mutually exclusive Cover appropriate range: current + future market Be unambiguous and objective Be plausible and actionable Be able to combine freely with each other in order to avoid producing impossible combinations

32 4. Construct and Select Profiles Profile: Potential product / service (Fig) Specific combination of attributes and levels Require sufficient number of profiles to test effects Increases with number of attributes and levels But respondents can only assess limited number (<15) Profile selection (Experimental design) Factorial designs rarely feasible Usually use fractional factorial designs (Table) Optimal designs are: Balanced: Each level is displayed an equal number of times Orthogonal: No correlation between any pairs of attributes Methods of profile selection Study design catalogues Software General: SPSS / SAS Purposive: Sawtooth / Speed / DCM

33 5. Decide on Type of CA Preference evaluation Ranking: Rank scenarios, card sort Rating: Rate scenarios Choice Binary: Discrete choice (Most widely used in health) Ordinal: Graded Likert (Fig) Other alternatives Choice-based CA Self-explicated analysis Adaptive CA / Hybrid CA Computer-based Can evaluate more attributes Full-profile CA Partial profile CA

34 6. Design and Pilot Tool Basic demographic information For subgroup and segmentation analysis Selected profiles Tables / Cards Not > 15 per person Otherwise must use: Blocked designs Partial profile CA Hybrid CA Include tests for internal consistency Dominant options Holdouts

35 7. Plan Sampling Strategy Standard criteria for probabilistic sampling Methods Random Systematic Cluster Multi-stage Sample size Formulas Empirical ~200 per sub-group of interest

36 8. Collect Data Methods Interview Face to face Telephonic Self-administered questionnaire Mailed survey Computer-based survey Computerised tool Adaptive CA Internet-based survey

37 9. Analyse Data Usually aggregated analysis for discrete choice Exclude: Inconsistent responses Dominant responses (?) Regression models: ( X X ) + β ( X X ) + + β ( X X ) + ε µ = β 0 + β1 1i 1j 2 2i 2j i j + Y 5 Analyse differences Model determined by type of response data Usually Probit / Logit Binary / Ordinal / Multinomial Adjust for panel data (multiple individual responses) Adjust for complex sampling Software General: Stata / Limdep / SPSS / SAS Specific: Sawtooth / Speed / DCM

38 10. Interpret Results Regression coefficients and p values Trade-offs Divide coefficients Produces willingness to pay (WTP) if one of variables is cost variable Utility of all possible combinations Relative to least valued combination Segmentation analysis Repeat regression for sub-groups Include interaction terms Cluster analysis

39 Some Limitations of CA Stated preferences Artificial / hypothetical constructs may not predict real choices Number of attributes and levels limited Significant design constraints Influence of excluded attributes Different results with different attributes Usually only aggregated analysis But choices rarely homogenous Complex conceptual task Respondents may resort to simplifying algorithms Complicated to design and analyse Require advanced software

40 APPLICATIONS IN HEALTH

41 Applications in Health Patient and community preferences for health care service delivery Patient preferences in doctor-patient relationship Preferences in priority setting Optimal treatment design Evaluate alternatives in RCTs Provider preferences

42 Patient Preferences for Provision of Orthodontic Services Ryan & Farrar (2000), BMJ, 320,

43 16 Combinations (4x2x2): 15 choice pairs Table 2. Results from regression analysis (1081 observations in 73 individuals) Waiting time * Variable Location of first appointment (0=local, 1=central) Location of second appointment (0=local, 1=central) Coefficient Log likelihood χ McFadden R P value <0.001 <0.001 <0.001 * Extra months willing to wait for a local first appointment=1.3; extra months willing to wait for a local second appointment=1.54.

44 Preferences for Hospital Quality in Zambia Hanson, McPake, Nakamba & Archard (2005), Health Economics, 14(7),

45 Design 216 Possible combinations 16 profiles, 15 choice pairs 2 Patients Adult with malaria + Child with pneumonia Sample of 300 for each

46 Tool

47 Results

48 Willingness to Pay for Low Cost Private Health Insurance LIMS Household Survey Blaauw & Schierhout (2006)

49 Design National household survey: 3593 Households 5 Attributes 1. Primary medical care 2. Dental care 3. Optometry 4. Hospital care 5. Cost No cover Cover for GP care only Cover for GP care + specialist care No cover Cover No cover Cover No cover Cover for emergency medical care Cover for maternal + neonatal care Cover for emergency, medical + neonatal care Comprehensive cover R50 R100 R150 R possible combinations 24 profiles Blocked into 3 sets of 8

50

51 What Components of Packages Do Respondents Value? - Basic Analysis Variable Coeff P value GP Care <0.001 GP + Specialist Care <0.001 Dental Care <0.001 Optometry Cover <0.001 Maternity Hospital Care Only Emergency Hospital Care Only <0.001 Emergency + Maternity Hospital Care <0.001 Comprehensive Hospital Care <0.001 Cost <0.001 MWTP (Relative to Cost) Combination Addition R R R R R R R R R R R R R R 6.98 R R Constant <0.001 Rho (ρ) <0.001 Prob (χ 2 ) <0.001 Combined Packages PHC-Spec (GP+Dental+Optom) R PHC+Spec (GP+Spec+Dental+Optom) R LIMS (GP+Dental+Optom+Emergency) R Full Cover R

52 What Components of Packages Do Respondents Value? - Segmentation Analysis PACKAGE COMPONENTS COMBINED PACKAGES n GP GP + Spec Dental Optom Mat Hosp Emerg Hosp Emerg+ Mat Hosp Comp Hosp PHC (-Spec) PHC (+Spec) LIMS Full Cover Total Male Respondent Female Respondent HH Income < R HH Income R2501-R HH None employed HH Informal employed only HH Some formal employment HH No MA HH Partial MA cover HH Full MA cover HH R2501-R6000, Some formal employed, No or partial MA

53 Health Worker Preferences for Job Characteristics Penn-Kekana, Blaauw & Hongoro (2004)

54 Methods DCE Design 5 Attributes Literature review 5 focus groups with nurses 1. Salary 2. Social Amenities 3. Equipment 4. Staffing 5. Facility Mx Same 15% More Double Under-developed Developed Fully equipped Poorly equipped Well staffed Under-staffed 48 Possible combinations Fractional factorial design 16 profiles, 15 choice pairs 147 Maternal health nurses in 3 provinces Poor Good

55 Example of DCE Choice Pair FACILITY A Pays 15% more than you are earning now Is not in cell phone range, has a dirt road, and no school or shopping centre nearby Is fully equipped Is well staffed Has poor, unsupportive and unfair management FACILITY B Pays the same salary you are earning now Is in cell phone range, has a tarred road, and a school and shopping centre nearby Is fully equipped Is under staffed Has good, supportive and fair management

56 Results Basic Model Variable Coeff P value Relative to Sal+15% Relative to Double Sal Salary + 15% < Double salary < Developed social amenities < Fully equipped < Well staffed < Good facility management < Rho (ρ) <0.001 Prob (LR χ 2 ) <0.0001

57 Results Subgroup Analysis CLINIC NURSES HOSPITAL NURSES Variable Coeff P value Coeff P value Salary + 15% < <0.001 Double salary < <0.001 Developed social amenities < <0.001 Fully equipped < <0.001 Well staffed <0.001 Good facility management < <0.001 Prob (LR χ 2 ) < <0.0001

58 Results Interaction Model Variable Coeff P value Salary + 15% <0.001 Interactions tested: Double salary Developed social amenities <0.001 <0.001 Facility type Hospital vs Clinic Location Fully equipped Well staffed <0.001 <0.001 Rural vs Urban Agegroup <40yrs vs > 40yrs Good facility management Salary 1 x Agegroup < Children under 18yrs Yes vs No Work overseas Salary 2 x Hospital Equipment x Hospital Yes vs No Feel motivated Yes vs No Equipment x Rural Management x Rural Management x Overseas Prob (LR χ 2 ) <0.0001

59 LINKS

60 Methods for Eliciting Preferences QUANTITATIVE TECHNIQUES Ranking Techniques Simple ranking Qualitative discriminant process Conjoint analysis Rating Techniques Likert scale Visual analogue scale Guttmann scales Conjoint analysis Semantic differential technique Satisfaction surveys SERVQUAL Choice-Based Techniques Simple choice exercises Conjoint analysis Analytic hierarchy process Standard gamble Time trade-off Person trade-off Willingness to pay Measure of value Allocation of points One-to-one interviews Dyadic interview Case study analysis Delphi technique Complaints procedures Individual Approaches QUALITATIVE TECHNIQUES Focus groups Concept mapping Citizen juries Consensus panels Public meetings Nominal group techniques Group Approaches Ryan, et al. (2001) Health Technology Assessment, 5(5)

61 Importance of factors Average Importance (95%CI) Price 9.04 Number of stops or connections en route 8.81 Duration of the flight 8.31 Friendliness of the cabin staff 7.31 Spaciousness of the seats 7.08 Airline carrier 6.92 Luggage allowance Membership of airline s loyalty programme for accumulation of air miles Possibility of getting upgraded to Business class Type of entertainment provided on the aeroplane 5.88 Type and quality of food provided during the flight 5.73 Type of aeroplane 5.52 Airports from which the airline operates 5.42

62 Ranking of factors Average Rank (95%CI) Price of the ticket 1.87 Total duration of the flight 2.38 The airline carrier 3.37 Getting a seat with extra legroom 3.52 Type of meal provided on the flight 3.87

63 ATTRIBUTES Airline Flight Duration Type of Seat Food Provided Price

64 ATTRIBUTES Airline Flight Duration Type of Seat Food Provided Price Virgin Atlantic South African Airways 4 Hours 7 Hours Standard economy seat Economy seat at emergency exit with extra legroom Snack Full meal $500 $750 $1000 $1500 LEVELS

65 Basic Profile Flight B Airline Duration of the flight Type of seat Food provided on flight Cost Virgin Atlantic 7 Hours Economy seat at emergency exit with extra legroom Full meal $ 750

66 Ranking and Rating Profiles Flight B Airline Duration of the flight Type of seat Food provided on flight Cost Virgin Atlantic 7 Hours Economy seat at emergency exit with extra legroom Full meal $ 750 Indicate how likely you would be to buy this ticket on a scale from 0 (Very Unlikely) to 100 (Very Likely)

67 Discrete Choice (Binary) Flight A Flight B Airline South African Airways Virgin Atlantic Duration of the flight Type of seat Food provided on flight 5 Hours Standard economy seat Snack 7 Hours Economy seat at emergency exit with extra legroom Full meal Cost $ 500 $ 750 Tick the flight you would choose:

68 Discrete Choice (Ordinal) Flight A Flight B Airline South African Airways Virgin Atlantic Duration of the flight Type of seat Food provided on flight 5 Hours Standard economy seat Snack 7 Hours Economy seat at emergency exit with extra legroom Full meal Cost $ 500 $ 750 Indicate your preference: Definitely choose A Probably choose A Unsure Probably choose B Definitely choose B

69 Multiple Choices Flight A Flight B Flight C South African Airways South African Airways Virgin Atlantic 5 Hours 5 Hours 7 Hours Economy seat at emergency exit with extra legroom Standard economy seat Economy seat at emergency exit with extra legroom Full meal Snack Full meal $ 1000 $ 500 $ 750 Which flight would you choose: Flight A Flight B Flight C None of these