WHO/UNICEF Toolkit on Monitoring and Evaluating Household Water Treatment and Safe Storage Maggie Montgomery, WHO HWTS Southern Africa Workshop, Mozambique 21 June 2011
Motivation Gap between needs and current HWTS coverage Effective policies (and programs) require knowledge of both "failures" and "successes" Data on HWTS use and benefits is limited and often biased Existing monitoring indicators and guidance is narrow in scope and fragmented Thus, there is the need for globally relevant indicators, monitoring and reporting HWTS M&E Recommendations.
Outline Document overview and context Proposed indicators Monitoring in the field Reporting, dissemination and feedback Next steps with HWTS M&E efforts
Why improved M&E for HWTS? Understand what works and what does not Technology Use, behaviour Implementation approaches Translate understanding into improved delivery, ongoing support and follow-up Compare and share Justify further investments
What's inside the document Overview of purpose of HWTS and proven solutions Developing a monitoring and evaluation framework Proposed indicators Conducting monitoring and evaluation Annexes Example surveys and sanitation safety plan forms Links and resources Overview of water quality monitoring field test options
Assessing safe water solutions-water safety plans Describe system Accessibility and types of water sources Reliability of sources Users Identify hazards Source water quality Contamination at collection, transport and in home WHO/ M Montgomery Assess risks Develop and implement mitigation measures HWTS!
Identifying what to monitor and/or evaluate Focus is on outputs and outcomes. Focus is NOT on health impact which requires careful selection of comparison group, rigorous data collection and review, complicated statistics.
Developing a M&E Framework Understand the context Engage stakeholders Determine aims, select questions and indicators Train M&E Team Conduct M&E Review and analyse data Disseminate results (internal and external) Use results to modify program, inform decisionmakers
What indicators do you use to measure HWTS?
Do they know how to use? Can you tell me how you use the product? How much time you wait? Not leading! Results: % of people who know correct use Tufts/ D Lantange
Reported use Have you ever used the product? Are you still using the product? NO - For how long did you use? NO - Why did you stop? Are you using the product today? Do you have the product right now? When do you not use? % of people with REPORTED use (note this is higher - survey/respondent bias) Tufts/ D Lantange
Reported Current Use Can you give me a cup of water right now as you would give to your child? Is this water treated in any way? If YES: With what? % of people with REPORTED CURRENT USE Tufts/ D Lantange
Confirmed use and safe storage Water quality testing Free chlorine residual (if chlorine) Microbiological testing Observation Is it present? Is it assembled/correctly? Is it wet? Is the storage container covered, clean and out of reach? % of people with CONFIRMED USE Tufts/ D Lantange
Monitoring Water Quality What water quality parameters can be used? Bacterial indicators (i.e. E. coli) Chlorine residual Turbidity Arsenic/flouride What types of testing can be done? Presence/Absence test (i.e. IDEXX) Semi-Quantitative (i.e. Aquatest, H2S) Quantitative (i.e. Oxfam Delaqua) Tufts/ D Lantange
Microbiological Testing - EFFECTIVE USE Test stored household UNTREATED water Test stored household TREATED water % of households with contamination in UNTREATED and no contamination in TREATED
Assessing Attitudes and Behaviour Belief that others treat drinking-water at home Who do you know that treats their water? How often do you think they treat their water? Confidence in improving quality of drinking-water I feel confident that I can correctly treat water to make it safer to drink. Community support in treating water My friends/community leaders /health care workers encourage me to treat water.
Conducting M&E in the field is not easy!
Analyzing Data Descriptive statistics i.e. % of households use HWT, % households safely store water Water quality results What does a positive result mean? Device is defective or not maintained properly Lack of correct use Possible re-contamination May still indicate improvement in water quality Risk Assessment i.e. % of households at "high risk" of diarrhoeal disease WHO/ M Montgomery
Dissemination/ translating results in action Gain initial feedback locally, then nationally, then internationally Willingness to learn and "do better" Agreement of stakeholders on interpretation of results Coordination and implementation Monitoring of gains
Acknowledgements and Links Daniele Lantagne (Tufts University) WHO/UNICEF Toolkit on Monitoring and Evaluating Household Water Treatment-Coming soon online in 2012 WHO (2011). Water quality assurance tool. Online resource. http://www.who.int/water_sanitation_health/publications/wsp_qa_tool/en/index1. html. UNICEF (2011). WaSH in Schools Monitoring Package. New York, USA. http://www.unicef.org/wash/schools/files/wash_in_schools_monitoringpackage_. pdf.
Kila jambo ni wakati wake. (Each thing has an opportune moment.) Obrigada. Questions?