GLOBAL RURAL WATER METRICS: TESTING AND VALIDATION

Transcription

1 GLOBAL RURAL WATER METRICS: TESTING AND VALIDATION Miguel Vargas-Ramirez, Libbet Loughnan, Sabrina Zimmermann Water and Health Conference October 30,

2 What we are here to share Our efforts to help build a Global Rural Water Metrics system, currently under testing and validation phase Objective / vision Background Origins it might draw on your work! Current status Preliminary validation and testing findings pilots, real world integration, exchange Looking forward what to anticipate 1

3 OBJECTIVE To make available an ambitious global good: a Universal Measure of What Works on Rural Water Supply That will help all of us stakeholders diagnose problems and successes, checking not just that services are expanding, but whether existing parts are being sustained. Ultimately, this means better water services for people, and more effectively reaching the SDGs. 2

4 3 BACKGROUND

5 Background How are we doing in rural water? We each ask this question. We all need to convey understand our achievements. baseline. Currently each country is struggling quite alone but on a issue that is common to all. 4

6 6 ORIGINS

7 The RWM foundations Systematic research: 40 Country Monitoring Systems Consultative process: JMP, WaterAid, SIASAR, etc This identified globally relevant and measurable factors of success in rural water metrics. 7

8 SIASAR Compatible now to compute 95% of RWM. Water and sanitation. It requires governmental endorsement to produce official data A, B, C, D Score easy to understand 22,000 communities and 11 countries 8

9 9 IBNET

10 Phase 1 findings: Global metrics for success in rural water 24 indicators that could be categorized into 3 dimensions and collected from 4 perspectives at one of 3 levels in each country studied. The next slides explain this. 10

11 Why 24 Indicators in 3 dimensions? 24 unique measurable elements were commonly held to be key to success. They can be summarized into 3 dimensions: Service level Characteristics of service received by users Functionality Physical condition and functioning of system 11 Sustainability factors Performance of service provider Extent of support to service providers

12 Why 4 distinct data sources? The responsibility and reality of the indicators is shared among 4 groups: household, service provider, service authority, spot test of infrastructure, which each: Provide unique insight Help diagnose areas of strength and weakness for swift address at the optimal level. Offer richer and more contextualized understanding than a single source could. 12

13 Why 4 distinct data sources? (continued) They also: - Allow triangulation, as part of quality assurance. - Link up to other monitoring efforts such as the JMP/SDG indicators. - The nature of the information requires different collection methods. 13

14 3 levels to for countries to launch from Acknowledging the diversity in current maturity of rural metrics in different countries, 3 levels were designed to meet the range of customer needs: 14

15 15 Rural Water Indicators Framework vs Country-Led Monitoring Systems Country X-led Monitoring System Country Y-led Monitoring System SIASAR Rural Water Metrics GLOBAL FRAMEWORK Mi Ba Ad Other Water Metrics relevant for Country X or Y

16 16 Three-by Four Indicator Matrix Minimum Basic Advanced Service Levels Functionability Sustainability: Service Provider performance Sustainability: Service authority or TA provider performace

17 17 CURRENT STATUS

18 Testing: pilots The RWM have just completed testing in 3 countries: Burkina Faso, Kenya, Sierra Leone Conducted by Akvo and the Global Water Challenge, using Akvo Flow Oriented around 2 guiding questions: How feasible (including validity and reliability) is it to collect the 24 indicators in the rural metrics frameworks? What is the value of the data generated by the rural metric indicators? 18

19 Testing: Consultation and exchange Learning loops between pilots, us, and stakeholders e.g. JMP, WHO, WaterAid, mwater Conferences e.g. IWA World Water Congress, UNC, etc. Online media - e.g. RWSN, blogs. 21

20 22 PRELIMINARY FINDINGS

21 Demand driven implementation Vietnam World Bank colleagues and government interest in TA to integrate elements of the RWM into their existing platforms Myanmar (& Cambodia?) WaterAid and government interest in TA to integrate elements of the RWM into their existing mwater-hosted platforms 23

22 Vietnam multiple responsible agencies Vietnam s NCERWASS rural water metrics look amenable to tweaks in achieving major harmonization with SDG/Global metrics norms, but national definitions did not quite match SDG (i.e. hygienic similar to improved but includes surface water) If data collection could record the components, rather than the aggregate, very SDG/Global metrics relevant calculations could be made 24

23 SIMILAR APPROACH, INDICATOR & QUESTIONS particular for Service Levels Moving towards harmonised indicators in the sector WaterAid indicators VS?? WB proposed Rural WS Metric Water assets status & age for budgeting & planning Service Levels - Quality, Quantity, Reliability, accessibility.. Water services sustainability proxy indicators - finance, management, technical support

24 Demand driven implementation (continued) 29 Very careful expansion until the pilot s results are ready, digested and disseminated. But the interest is encouraging, and we are using the opportunities to actively plan with the stakeholders involved, to collect additional information for our validation process. Thus the RWM are already now seeing some organic uptake among development partners, for generating real sector M&E data.

25 Tools development We have also prepared tools to support broader rollout, and are working on more. E.g. questionnaires ready in the open source free Survey Solutions software 30

26 Preliminary findings: pilot (Africa) The complexity of surveys grew, 24 metrics became nearly 80 individual questions (translation and localization can add additional complexity. Enumerators identified 10 top difficult questions. More than 12% of the questions could only be answered by 75% (or less) of the respondents. The overall survey durations were very reasonable, with three of the four surveys taking about 15 minutes or less 31

27 Preliminary findings: pilot (cont d) In Sierra Leone, enumerators found questions around financial management difficult because hardly any rural water is paid. In Burkina Faso, detailed questions from enumerators on terms such as operations, legal requirements, depreciation, liabilities, and others highlighted the need for additional documentation. Water quality testing was a success in both Kenya and Burkina Faso, faced initial delays for water testing but enumerators felt comfortable doing the test The logistical challenges of accessing remote and rural areas need to be fully considered 32

28 Preliminary findings: Consultation and exchange So far in consultations and the global sector conferences like today, stakeholders have been enthusiastic, collaborative, and keen to see the tested product ready for broader rollout. Convergence on the need for harmonization. 33

29 Preliminary findings: Demand driven implementation Agency coordination an integral part of RWMGF adoption Need to develop dictionary to move across regions Finding a proper window of opportunity is a factor of success 34

30 Optimal adoption approach Ideally, countries aim towards cohesion: adopt the core set of indicators as far as possible (at the level of ambition relevant to the country). Include other indicators where necessary. To fill any gaps in the global metrics To avoid sacrificing time-trends 35

31 Global approach, local tailoring Which launchpad should country X choose? The most relevant but ambitious one. 36

32 Benefits for countries from adoption Partnership, coaching, expertise, tools. Gain efficiencies of international technical standards. Benchmark potential with peers A shared concern not to sacrifice comparability with core previous data. 37

33 Benefits for Global practitioners from adoption Comparable data set will unveil evidence on effectiveness of policies and practices Rural Water systems benchmarking moves sector from aid-focus to professionalization. Analysis of different domains (SL, FC, SF) could help targeting support and methodologies for intervention Catalytic for efficiencies in data collection and adoption of innovative technologies 38

34 THANK YOU! For further questions: