Mixed-mode surveys a comparison of sequential and parallel designs

Transcription

1 Mixed-mode surveys a comparison of sequential and parallel designs Elena von der Lippe Jens Hoebel Stephan Müters Cornelia Lange Robert Koch Institute, Germany Ljubljana, July 2013 Elena von der Lippe

2 Background German Health Update (GEDA) study: observe health and risk related behavior of the German adult population living in private households conducted regularly by the Robert Koch Institute (RKI) on behalf of the German Federal Ministry of Health (BMG) as Part of the (national) health monitoring telephone sample

3 Background * Preliminary estimation

4 Background Mixed-mode surveys are conducted in social research with the intention to avoid non-response, sample attritions and low representativeness. There is no consent concerning the question what kind of mixed-mode design leads to higher response rates and better sample compositions. Research indicates that rather than enhancing response rates, offering multiple modes can actually depress overall response rates (Griffin et al. 2001). Sequential designs increased response rates (Klaush et al 2013)

5 Aims of the study Compare two different mixed mode designs (sequential and parallel) according to: response rates sample structures data quality costs Decide for a future design study for GEDA 2014

6 Method

7 Study design GEDA 2.0 Sample size N = Randomisation Participation cross section 50 % / 50 % Participation Access-Panel Sequential Mixed-Mode-Design Parallel Mixed-Mode-Design CATI Design Web Web, SAQ Web, SAQ, CATI Web, SAQ, CATI 1. Reminder 2. Reminder CATI 1. Reminder 2. Reminder

8 Method Sequential Mixed-Mode-Design Parallel Mixed-Mode-Design Web Web, SAQ Web, SAQ, CATI Web, SAQ, CATI 1. Reminder 2. Reminder 1 st contact: a password for the study is sent 2 nd contact: a printed version of the questionnaire + reminder for the web possibility 3 rd contact: reminder + an offer to be interviewed on telephone (the respondents have to send their telephone number) 1 st contact: a password for the study, a printed version of the questionnaire and an offer for a telephone survey 2 nd contact: a reminder 3 rd contact: a reminder

9 Results: Response rates N = 561 N = 617

10 Response rate after each contact

11 Response according to mode 1 % 2 % 52 %

12 Sample structure I

13 Sample structure II

14 Sample structure III

15 Sample structure IV

16 Data Quality Item nonresponse SAQ Web Education 0,8 0,0 Employment 5,4 0,0 Family status 1,7 0,0 Subjective health 1,1 0,0 Diabetes 6,8 0,0 Coronary heart disease 1,5 0,0 Smoking 3,2 0,0 Alcohol consumption 2,8 0,0 Physical activity 8,2 0,2 As the parallel design consists more SAQ, we assume that it also has a little bit lower data quality

17 Costs SAQ: Web: printing costs stationary costs data entry and verification server hosting license costs Web has the lowest costs CATI: Interviewer costs Interviewer software As the sequential design consists more web participants, we assume that the costs are also less

18 Conclusions Sequential a little bit lower response higher data quality (less missing values) lower costs Parallel a little bit higher response lower data quality (more missing values) higher costs Sample structures do not show great differences Except for the slightly worse response rate, the sequential design seems to us to be more efficient (with regard to cost and data quality)

19 Thank you! Questions or comments?