A METHODOLOGICAL CLUSTER-RANDOMIZED CONTROLLED TRIAL IN BURKINA FASO TO ASSESS BIAS IN ESTIMATES OF AGRICULTURE-NUTRITION RELATIONSHIPS

Transcription

1 A METHODOLOGICAL CLUSTER-RANDOMIZED CONTROLLED TRIAL IN BURKINA FASO TO ASSESS BIAS IN ESTIMATES OF AGRICULTURE-NUTRITION RELATIONSHIPS Agriculture, Nutrition, Health Scientific Symposium Kathmandu, Nepal, July 13, 2017

2 BACKGROUND Analyses of agriculture and nutrition relationships often rely on data collected using heterogeneous survey design methods General direction of biases in surveys are known, though limited understanding of magnitude Assessing the magnitude of bias of estimates could: facilitate better survey designs inform choice of instruments improve programming decisions and priorities (e.g., production diversification interventions require accurate estimates of potential dietary impacts) reduce costs of survey development and implementation

3 OBJECTIVES Using a methodological cluster-randomized controlled trial varying survey respondents and agricultural units of analysis, determine the effect of survey design on: 1. estimates of agricultural inputs and crop production 2. associations of these characteristics with child diet diversity Photo source:

4 METHODS Cluster-randomized controlled trial design Villages serve as clusters Field work carried out January-February 2017 Stratified, staged sampling approach Random selection of administrative regions within each of the country s 3 agroecological zones: Sahelian (2), Sudan-Sahelian (4), and Sudanian (2) One province per region randomly selected

5 AGROECOLOGICAL ZONES OF BURKINA FASO Annual rainfall Sahelian: <600 mm Sudan-Sahelian: mm Sudanian: >900 mm Agro-pastoralism & subsistence millet, sorghum, cowpea production ubiquitous Production more diverse in Sudanian AEZ Sahelian agroecological zone Sudan-Sahelian agroecological zone Sudanian agroecological zone Source: FAO (2010) Cartographie Des Zones Socio-Rurales Burkina Faso

6 RANDOMLY SELECTED REGIONS & PROVINCES 8 of 12 regions selected (Centre region excluded) Soum Yatenga Terrorist attacks in Soum province weeks before survey precluded field work in that region Houet Bougouriba Sissili Oubritenga Zoundweogo Tapoa Base image source:

7 TRIAL PROFILE PROVINCE & COMMUNITY SELECTION 8 provinces planned 7 provinces included in study 1 excluded 10 villages per province

8 INCLUSION CRITERIA Household census conducted in each selected village Inclusion criteria at least one child aged months household s primary income/subsistence derived from agricultural production, keeping animals, or combination thereof 15 eligible households randomly selected within each village

9 TRIAL PROFILE PROVINCE & COMMUNITY SELECTION 8 provinces planned 7 provinces included in study 1 excluded 10 villages per province HOUSEHOLDS RANDOMLY ASSIGNED 5 households per village (treatment 1) n = households per village (treatment 2) n = households per village (control) n = 350 n = 14 n = 5 n = 3 HOUSEHOLDS RECRUITED n = 336 households n = 345 households n = 347 households

10 STUDY ARMS Treatment arm 1: Randomly selected household member (n = 350 HH) Aggregate data for entire household (across all plots and seasons) collected from randomly selected household member excluding household head Treatment arm 2: Household head (n = 350 HH) Aggregate data for entire household (across all plots and seasons) collected from household head Control arm: Plot manager (n = 350 HH) Data collected from household member that manages each plot, by plot and season

11 VARIABLES Data on agricultural production, land, labor, inputs, earnings, control of management decisions collected through in-person interview Crop diversity (CD): simple species count Diet diversity (DD) score: recent consumption 7 foods groups using quantitative 24-hour recall 1) grains, roots and tubers; 2) legumes and nuts; 3) dairy products; 4) flesh foods; 5) eggs; 6) VA-rich F&V; 7) other F&V Food groups for WHO Minimum Dietary Diversity (MDD) indicator

12 STATISTICAL ANALYSIS Regression analysis of treatment effects on agricultural outcomes controlling for unbalanced covariates Linear probability models for the binary outcomes (e.g., use of fertilizer; production of specific crops) Multiple linear models for continuous outcomes (e.g., land size) Tobit model of crop diversity (land and household size added covariates)

13 STATISTICAL ANALYSIS Two-stage least squares (2SLS) regression analysis of association between crop diversity and diet diversity Exploits exogeneity of treatment assignment to instrument crop diversity Instrument relevance strengthened by interacting treatment with land size in first stage All models cluster standard errors at village level

14 SELECT HOUSEHOLD CHARACTERISTICS, BY STUDY ARM Characteristics Treatment (1) Treatment (2) Control (3) P-values (n=336) (n=345) (n=347) (1)-(3) (2)-(3) (1)-(2) Female HH head, % Age of HH head, y HH head migrant, % < HH size HH asset index < # rooms in dwelling Any education HH, % HH head education, % None Primary Secondary Data show means and pairwise comparisons between arms; P-values obtained from regressions of each characteristic on treatment dummy variable; SEs clustered at village level

15 OUTCOME VARIABLES, BY STUDY ARM Characteristics Treatment (1) Treatment (2) Control (3) P-values (n=336) (n=345) (n=347) (1)-(3) (2)-(3) (1)-(2) Land size (ha) Fertilizer use, % Irrigation use, % Salaried labor use, % Crop diversity Cultivate cereals, % Cultivate legumes, % Cultivate vegetables, % Cultivate cash crops, % Diet diversity score Data show means and pairwise comparisons between arms; P-values obtained from regressions of each characteristic on treatment dummy variable; SEs clustered at village level

16 TREATMENT EFFECTS ON AGRICULTURAL INPUTS Land size (ha) Fertilizer use Salaried labor Irrigation use Treatment arm (1) (2) (3) (4) (5) (6) (7) (8) Random proxy -1.0** -0.64* -0.17*** -0.13*** -0.12** -0.10** HH head -1.1*** ** ** HH head migrant HH head education -0.63* Asset quartiles Second 1.3*** 0.12** Third 1.7*** 0.24*** Fourth 5.0*** 0.38*** 0.20*** 0.08** Observations 1,028 1,028 1,025 1,025 1,025 1,025 1,028 1,028 P-value Control means Data are coefficients from linear probability or multiple linear regression models; SEs clustered at village level in all models; *P<0.1; **P<0.05; ***P<0.01; reference for Treatment is control arm; reference for Asset quartiles is first quartile; control means are variable means for control HHs in the estimation sample

17 TREATMENT EFFECTS ON CROP DIVERSITY OLS Tobit Treatment arm Random proxy -0.58** -0.58* HH head HH head migrant HH head education Asset quartiles Second 0.47** 0.54** Third 0.56** 0.59** Fourth Observations R-squared/Pseudo R-squared P-value <0.001 <0.001 Data are coefficients from multiple linear regression and Tobit models; SEs clustered at village level in all models; *P<0.1; **P<0.05; ***P<0.01; reference for Treatment is control arm; reference for Asset quartiles is first quartile

18 TREATMENT EFFECTS ON CROP PRODUCTION HH cultivation of: Cereals Legumes Vegetables Cash crops Treatment arm (1) (2) (3) (4) (5) (6) (7) (8) Random proxy ** -0.11* HH head 0.02** 0.02** ** -0.11** HH head migrant * ** HH head education Asset quartiles Second * 0.07* Third -0.02* 0.09* *** Fourth *** Observations P-value Control means Data are coefficients from linear probability or multiple linear regression models; SEs clustered at village level in all models; *P<0.1; **P<0.05; ***P<0.01; reference for Treatment is control arm; reference for Asset quartiles is first quartile; control means are variable means for control HHs in the estimation sample

19 INSTRUMENTED EFFECTS OF CROP DIVERSITY ON DIET DIVERSITY SCORE OLS 1st-stage (1) (CD) 2SLS 2nd-stage (1) (DD) 1st-stage (2) (CD) 2nd-stage (2) (DD) Crop diversity 0.029* 0.240** 0.241*** Treatment arm Random proxy -0.79*** -0.82*** HH head Treatment arm*land size Random proxy*land size 0.03 HH head*land size 0.04 Land size 0.07* Constant 2.8*** 4.3*** 1.9*** 3.9*** 1.9*** Observations Sanderson-Windmeijer F statistic 5.87*** 9.81*** Hansen J statistic 5.3** 5.9 Hausman Chi-square * Data are coefficients from OLS and 2SLS regression models; SEs clustered at village level in all models; *P<0.1; **P<0.05; ***P<0.01; reference for Treatment is control arm

20 CONCLUSIONS Survey design choices have clear influence on estimates of agricultural production characteristics in Burkina Faso Relying on proxy household member (even HH head in some cases) to report agricultural production data results in downward bias in estimates of many characteristics >8-fold difference in magnitude of crop diversity estimate on diet diversity when instrumenting crop diversity by treatment and land size Ongoing work: Developing estimates of CIs for interpreting results across studies with different survey designs Analysis of data on anthropometry, diet quality, crop yields, value of output per hectare, commercialization, livestock, and income Jones AD (in press). Agricultural biodiversity, diet diversity, and nutritional status in low- and middle-income countries: a critical review of the emerging research evidence. Nutrition Reviews

21 ACKNOWLEDGEMENTS Investigator team Andrew Jones, School of Public Health, University of Michigan (UM) Andrew Dillon, Department of Agricultural, Food, and Resource Economics, Michigan State University (MSU) Jennifer Blesh, School of Natural Resources and Environment, UM Jérôme Somé, School of Public Health, UM (currently Tufts University) Edouard Mensah, Department of Agricultural, Food, and Resource Economics, MSU Implementing team Nicoló Tomaselli, Innovations for Poverty Action (IPA) Adrien Pawlik, IPA Issa Kiemtore, IPA

22 CROP CATEGORIES Cereal crops Rice, millet, sorghum, corn, fonio Legume crops Bean, peanut, bambaranut, sesame, soybean Vegetable crops Tomato, onion, okra, and sorrel Cash crops Cotton, sesame, corn, peanut Fruits; Roots and tubers Few observations (i.e., produced by <6 HHs in each study arm