NBER WORKING PAPER SERIES DIFFICULTY TO REACH RESPONDENTS AND NONRESPONSE BIAS: EVIDENCE FROM LARGE GOVERNMENT SURVEYS. Ori Heffetz Daniel B.

Transcription

1 NBER WORKING PAPER SERIES DIFFICULTY TO REACH RESPONDENTS AND NONRESPONSE BIAS: EVIDENCE FROM LARGE GOVERNMENT SURVEYS Ori Heffetz Daniel B. Reeves Working Paper NATIONAL BUREAU OF ECONOMIC RESEARCH 1050 Massachusetts Avenue Cambridge, MA June 2016, Revised January 2018 We thank Adrian Chadi, Tatiana Homonoff, Bruce Meyer, Ted O Donoghue, Matthew Rabin, participants in seminars at Cornell, and two anonymous referees for useful early discussions and comments on previous drafts. We are grateful to John Dixon, Brian Meekins, and Polly Phipps for patiently responding to our questions regarding the CPS. The authors have no financial or other material interests related to this research to disclose. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research. NBER working papers are circulated for discussion and comment purposes. They have not been peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications by Ori Heffetz and Daniel B. Reeves. All rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full credit, including notice, is given to the source.

2 Difficulty to Reach Respondents and Nonresponse Bias: Evidence from Large Government Surveys Ori Heffetz and Daniel B. Reeves NBER Working Paper No June 2016, Revised January 2018 JEL No. C18,C83,I18,J60 ABSTRACT How high is unemployment? How low is labor force participation? Is obesity more prevalent among men? How large are household expenditures? We study the sources of the relevant official statistics the Current Population Survey (CPS), the Behavioral Risk Factor Surveillance System (BRFSS), and the Consumer Expenditure Survey (CEX) and find that the answers depend on whether we look at easy-or at difficult-to-reach respondents, measured by the number of call and visit attempts made by interviewers. A challenge to the (conditionally-)random-nonresponse assumption, these findings empirically substantiate the theoretical warning against making population-wide estimates from surveys with low response rates. Ori Heffetz S.C. Johnson Graduate School of Management Cornell University 324 Sage Hall Ithaca, NY and NBER oh33@cornell.edu Daniel B. Reeves Department of Economics Cornell University 474-B Uris Hall Ithaca, NY dbr88@cornell.edu A data appendix is available at

3

4 7+ 5+

5

6

7

8

9

10

11

12 3+

13 4 13

14 3+ N = 307, R 2 = 0.29 < < <

15 =

16 3+

17 3+ N = 200, R 2 = 0.05 < < <

18

19

20 ( ) 2 n = 22, 710 n = 2, 680 n = 85 30

21

22

23 7+ BMI 30

24 = 0.4 = 0.5

25 7+ N = 450, R 2 = 0.04 < < <

26 2.9 < 1

27

28 ln[1+] 65+

29 5+

30 5+ 5+

31 5+ N = 36, R 2 = 0.56 < < <

32

33

34

35 3+

36

37

38

39 3+ N = 20) 3+

40 Unemployment rate and difference Year Adj. means: Difference: 1 attempt 1 minus 3+ 2 attempts 2 minus attempts

41

42

43

44