The effect of program implementation quality on handwashing (and other) behavior

Transcription

1 The effect of program implementation quality on handwashing (and other) behavior Results from a national-scale water, sanitation, and hygiene intervention in rural Bangladesh Jade Benjamin-Chung, Sonia Sultana, Amal K. Halder, Benjamin F. Arnold, Alan Hubbard, Mohammed Ali Ahsan, Leanne Unicomb, Stephen P. Luby, John M. Colford, Jr. Water and Health Conference October 16, / 20

2 Background The Millennium Development Goals increased motivation to scale-up health interventions. (Sachs 2004) Many health interventions have been shown to be efficacious and cost effective (e.g. breastfeeding, insecticide treated materials, vaccines, vitamin A, water and sanitation). (Bryce et al. 2005, Jha et al. 2002, Victora et al. 2004) Many barriers to scale-up exist. (Mangham & Hanson, 2010, Subramanian et al. 2011, Tangcharoensathien & Patcharanarumol 2010, Gilson & Schneider 2010) There has been a call for empirical evidence to support scale-up strategies. (Subramanian et al. 2011) 2 / 20

3 Background A large body of evidence demonstrates the efficacy of water, sanitation, and hygiene (WASH) interventions. (Fewtrell et al. 2005) There have been few rigorous evaluations of large-scale WASH interventions in such settings. (Mangham & Kanson 2010, Bartram & Cairncross 2010) In the WASH sector, many questions remain about how best to deliver interventions at scale. (Cairncross et al. 2010) Population health impacts of scaled interventions depend on: Design of the intervention Implementation of the intervention 3 / 20

4 SHEWA-B The Sanitation Hygiene Education and Water Supply in Bangladesh Program Implemented by UNICEF and the Government of Bangladesh from Targeted 20.4 million beneficiaries in 19 of 64 districts Aimed to promote hygiene practices and reduce hygiene-related diseases among the poorest in rural Bangladesh Implemented by local Community Hygiene Promoters (CHPs) Figure: SHEWA-B intervention areas 4 / 20

5 Motivation for this study Interim evaluation of SHEWA-B in 2009 using matched control areas (Huda et al. 2012) Minimal improvement in health behaviors, such as handwashing No improvement in the prevalence of child diarrhea and respiratory illness Poor results could reflect either poor design or implementation of the intervention or both Our objective: estimate the effect of SHEWA-B implementation quality on health behaviors in a large sample of intervention villages across rural Bangladesh 5 / 20

6 Methods Study design I Cross-sectional survey of intervention households (June 2011-April 2012) I Rapid observation of health practices (e.g., observed hand cleanliness) I Questions about interactions with CHPs (e.g., recall of CHP house visits) I Survey of CHPs I In-depth interviews and focus groups with SHEWA-B recipients, CHPs, and communities 6 / 20

7 Methods Design of Index of Implementation Quality Metrics to measure implementation quality were not defined prior to the study. Used the Delphi Method to gather structured feedback from UNICEF (Linstone et al., 1975) Constructed a 100-point index using 12 variables from the cross-sectional survey and one variable from the CHP survey 7 / 20

8 Distribution of implementation quality index Most households implementation quality index values were below 42 (the 75th percentile) Number of households Implementation Quality Index 75th percentile of the index 8 / 20

9 Methods Observed and hypothetical percentages of respondents with selected outcomes at different index values Range of implementation quality index Respondents ever met a CHP 0% 74% 100% Respondents attended a community event 0% 44% 100% Respondent knew CHP s name 0% 48% 100% CHP knew all key messages of SHEWA-B 0% 53% 100% 9 / 20

10 Population intervention model parameter A technique arising from the causal inference literature The causal inference literature has proposed parameters that are interesting alternatives to traditional regression-based parameters, even when the goal is not causal inference. e.g., marginal structural models (Robins, 1998) Many of these parameters define counterfactuals under the potential outcomes framework defined by Neyman and Rubin. (Holland, 1986) The population intervention model is one such parameter which measures changes in health attributable to either the removal or enhancement of an exposure in a counterfactual scenario. (Hubbard & van der Laan 2008) e.g., the counterfactual of interest could be a scenario in which SHEWA-B was implemented perfectly 10 / 20

11 Methods Population intervention model parameter (Hubbard & van der Laan 2008) 1. Estimate probability of outcomes under observed level of implementation quality 2. Estimate probability of outcomes under a counterfactual scenario estimated from a model in which all clusters with implementation quality below the 75th percentile of the index were raised to that standard 3. Average the probabilities under both scenarios and subtract the means Observed distribution of implementation quality Number of households Implementation Quality Index Counterfactual distribution of implementation quality Number of households Implementation Quality Index 75th percentile of the index 11 / 20

12 Methods Statistical Model Models run using data aggregated to the cluster level (n=1,160) Used a generalized linear model to estimate the association between implementation quality and each outcome Adjusted for potential confounders: sub-district poverty level, geographic features, season of data collection Used non-parametric bootstrap with 1,000 replicates to estimate 95% confidence intervals and standard errors 12 / 20

13 Results I 33,027 households in 1,160 village clusters consented to participate (response rate=99%) I 1,164 CHPs consented to be interviewed (response rate=96%) I Focus groups and in-depth interviews with over 58 SHEWA-B recipients, 40 community members (e.g. teachers, local leaders), and 6 CHPs 13 / 20

14 Results Increasing implementation quality resulted in small increases in the probability of each outcome. Probability of having a dedicated handwashing location Implementation Quality Index Probability of outcome 95% CI 14 / 20

15 Results Difference in probability of outcomes under observed implementation quality level and under counterfactual scenario Outcome No open defecation Correct caregiver handwashing Dedicated handwashing location Correct caregiver handwashing* No open defecation* Drinking water container covered No feces on latrine slab or floor* Drinking water point is sanitary Dedicated handwashing location* Implementation scenario Observed Improved *Statistically significant difference Prevalence 15 / 20

16 Results Even when restricting the analysis to those who had been visited by a CHP, outcomes were only marginally better. 16 / 20

17 Qualitative findings Possible reasons for sub-optimal implementation and minimal effect of implementation quality Low CHP knowledge: under half (47%) of CHPs surveyed could recall all three general key messages of SHEWA-B. Low and irregularly disbursed stipends: one CHP said, We receive only 98 taka [approximately 1.27 USD] per day which is less than half of a daily laborer s payment. If I were not unemployed, I would not do this job at all. Lack of hardware provision: One CHP said, A big NGO is providing free latrines to its beneficiaries, but we do not have provisions to distribute any hardware support like them. Do you think people will listen to us if we do not give them anything except words? 17 / 20

18 Conclusions We found that, on average, this large-scale WASH intervention was delivered sub-optimally, highlighting the difficulty of implementing large-scale interventions. Improved CHP training and management may have led to better SHEWA-B implementation. Strong leadership and management are crucial for successful implementation at scale, and such factors could be explored further. (Mangham & Hanson 2010) Our findings contribute to the growing empirical evidence base describing best practices for and barriers to delivering interventions at scale. 18 / 20

19 References (1) Bartram J, Cairncross S (2010) Hygiene, Sanitation, and Water: Forgotten Foundations of Health. PLoS Med 7: e doi: /journal.pmed Bryce J, Black RE, Walker N, Bhutta ZA, Lawn JE, et al. Can the world afford to save the lives of 6 million children each year? The Lancet 365: doi: /s (05) Cairncross S, Hunt C, Boisson S, Bostoen K, Curtis V, et al. (2010) Water, sanitation and hygiene for the prevention of diarrhoea. International Journal of Epidemiology 39: i193i205. doi: /ije/dyq035. Fewtrell L, Kaufmann R, Kay D, Enanoria W, Haller L, et al. (2005) Water, sanitation, and hygiene interventions to reduce diarrhoea in less developed countries: a systematic review and meta-analysis. The Lancet Infectious Diseases 5: doi: /s (04) Gilson L, Schneider H (2010) Commentary: Managing scaling up: what are the key issues? Health Policy Plan 25: doi: /heapol/czp067. Holland P (1986) Statistics and Causal Inference. J Am Stat Assoc 81: Hubbard AE, Laan VD, J M (2008) Population Intervention Models in Causal Inference. Biometrika 95: doi: /biomet/asm097. Huda TMN, Unicomb L, Johnston RB, Halder AK, Yushuf Sharker MA, et al. (2012) Interim evaluation of a large scale sanitation, hygiene and water improvement programme on childhood diarrhea and respiratory disease in rural Bangladesh. Social Science & Medicine 75: doi: /j.socscimed Jha P, Mills A, Hanson K, Kumaranayake L, Conteh L, et al. (2002) Improving the Health of the Global Poor. Science 295: doi: /science / 20

20 References (2) Linstone HA, Turoff M (1975) Delphi Method: Techniques and Applications. Boston, MA: Addison-Wesley Publishing. 640 p. Mangham LJ, Hanson K (2010) Scaling up in international health: what are the key issues? Health Policy and Planning 25: doi: /heapol/czp066. Robins, JM (1998) Marginal Structural Models, 1997 Proceedings of the American Statistical Association, Section on Bayesian Statistical Science, pp Sachs JD (2004) Health in the developing world: achieving the Millennium Development Goals. Bulletin of the World Health Organization 82: doi: /s Subramanian S, Naimoli J, Matsubayashi T, Peters DH (2011) Do we have the right models for scaling up health services to achieve the Millennium Development Goals? BMC Health Services Research 11: 336. doi: / Tangcharoensathien V, Patcharanarumol W (2010) Commentary: Global health initiatives: opportunities or challenges? Health Policy Plan 25: doi: /heapol/czp069. Victora CG, Hanson K, Bryce J, Vaughan JP (2004) Achieving universal coverage with health interventions. The Lancet 364: doi: /s (04) / 20