MEMORANDUM. Please find attached the above-mentioned Report. * Questions on this document should be referred to:

Transcription

1 AFRICAN DEVELOPMENT BANK ADB/BD/WP/2016/167 Probable Date of Presentation to the Committee Operations/ Development Effectiveness (CODE) AFRICAN DEVELOPMENT FUND ADF/BD/WP/2016/115 6 October 2016 Prepared by: IDEV Original: English FOR CONSIDERATION 21 October 2016 MEMORANDUM TO : THE BOARDS OF DIRECTORS FROM : Vincent O. NMEHIELLE Secretary General SUBJECT : TANZANIA: IMPACT EVALUATION OF THE RURAL WATER SUPPLY AND SANITATION PROGRAMME* Please find attached the above-mentioned Report. Attach: Cc: The President * Questions on this document should be referred to: Mr. R. NANGIA Evaluator General IDEV Extension 2041 Mrs. R. AMIRA Division Manager IDEV.1 Extension 2696 Mr. F. TURAY Chief Evaluation Officer IDEV.1 Extension 3257 SCCD : K.M

2 AFRICAN DEVELOPMENT BANK GROUP TANZANIA IMPACT EVALUATION OF THE RURAL WATER SUPPLY AND SANITATION PROGRAMME IDEV DEPARTMENT October 2016

3 Impact Evaluation of the Rural Water Supply and Sanitation Programme, United Republic of Tanzania Main Report Cover: members of the COWSO at Kona, Kiteto district. Their scheme includes a stock watering trough.

4 Preface Safe domestic water supplies and appropriate sanitation and hygiene practices are vital for an adequate standard of living in rural Tanzania. With the support of their development partners, the country s Government and citizens have been working hard towards achieving these goals, through Phase I of the Water Sector Development Programme (WSDP). Through the Rural Water Supply and Sanitation Programme (RWSSP), the African Development Bank Group (AfDB) has been a strong supporter of these efforts. The Independent Development Evaluation of the AfDB as it did recently in Ethiopia has now commissioned an evaluation of the impact of the RWSSP on key aspects of rural livelihoods and the economy. We were honoured to be asked to undertake this evaluation, and have worked with colleagues in Government, the Bank and development partners to produce the data and analysis presented below. We hope it will be a useful input to the enhancement of rural water and sanitation efforts as the WSDP begins its second phase. The main evaluation report is presented below, together with Annexes 1 and 2 (terms of reference and a list of persons met). Volume II presents technical annexes on methodology and data, as well as the field survey instruments that were used. We thank the Resident Representative of the African Development Bank, Dr Tonia Kandiero, and the RWSSP Task Manager in the Bank s Country Office, Mr Sabas Marandu, for all their support during this evaluation. We are grateful, too, for all the logistical and administrative assistance provided by Mrs Halimah Kitandula of the Country Office. We acknowledge with sincere thanks the guidance and support of the Director of Rural Water Supplies in the Ministry of Water, Engineer Frida Rweyemamu, and of the Deputy Director, Engineer Jackson Mutazamba. At the University of Dar es Salaam, the field surveys commissioned to the Centre for Economic Research and Policy were ably co-ordinated by its research co-ordinator, Dr Stephen Kirama. We acknowledge with thanks the hard work of the Centre s survey supervisors and enumerators as they carried out the survey in 30 Local Government Authorities (LGAs) around the country. We are also grateful to one of the supervisors, Mr Shakir Msangi, for his able support to the institutional assessment mission that we undertook in October Most importantly of all, we respectfully thank the Council Water and Sanitation Teams, members of Community-Owned Water Supply Organisations and rural water users who gave so freely of their time during our investigations. Once more, we express the hope that this evaluation will lead to stronger and more sustainable rural water and sanitation systems in rural Tanzania. Kenneth Mdadila Stephen Turner Dar es Salaam and Manchester 30 March, 2016.

5 Contents: Volume I Abbreviations i Summary ii 1 Background Country context Programme funding and objectives Theory of change Summary of programme implementation Impact evaluation of the RWSSP Evaluation questions Scope of the impact evaluation Activities during the impact evaluation 9 Methodology Impact evaluation design Sample size and sampling procedures Survey instruments and implementation of field survey Water quality testing Post-survey data management and checks Existing empirical evidence Interviews with informants Limitations 14 Findings Introduction Policy Activities Outputs Capacity development Outcomes Equity Institutional outcomes Institutional assumptions Impact Impact on health Impact on girls education Impact on women s economic opportunities Impact on economic activity 38 Sustainability and replicability Technical sustainability Institutional sustainability 39

6 Impact Evaluation of the Rural Water Supply and Sanitation Programme, United Republic of Tanzania: Volume I 1.25 Links between technical and institutional sustainability Replicability 41 Conclusions, lessons and recommendations Conclusions Lessons Recommendations Policy recommendations Programming recommendations 43 Annex 1. Terms of reference Annex 2. Persons met References I VIII XI

7 Tables: Volume I Table 1. Distribution of the evaluation sample size 12 Table 2. Guide to responses to evaluation questions 15 Table 3. Primary schools' sources of drinking water 21 Table 4. Training received by 26 CWSTs, as reported to evaluation survey 24 Table 5. Subjects on which COWSOs and other user groups reported receiving training 25 Table 6. Impact of programme on incidence of diarrhoea 30 Table 7: Water quality at distribution points and at the point of use 31 Table 8: Household perceptions of changes in quality of water 31 Table 9. Water consumption per capita per day in programme and non-programme communities 32 Table 10. Availability of hand washing facility within 3m of toilet 34 Table 11. Impact estimation: availability of hand washing facility within 3m of toilet 34 Table 12. Households use of drinking water from improved sources 35 Table 13. Household perceptions of adequacy of water supply in dry season 35 Table 14. Households reported sources of water in dry and wet seasons 35 Table 15: Enrolment and dropouts at primary school 36 Table 16. Time spent by girls and women fetching water 37 Table 17: Impact on women's participation in income-generating activities 38 Figure 1. Water carts are an increasingly common means of transport 2 Figure 2. RWSSP Tanzania: theory of change 5 Figure 3. Water sellers' carts in a community with no supply: water drawn from a nearby dam 7 Figure 4. Water may be transported several kilometres to communities without a supply 18 Figure 5. A simple hand pump installed on a borehole 20 Figure 6. Community management structures at 114 water points in 'treatment' villages 22 Figure 7. Some schemes only supply water intermittently 31

8 Abbreviations AfDB BEST BRN CERP CLTS COWSO CWST DFID DHS DID DP DPs DWST ECCT EQ GOT IDEV IOB JMP LGA MDG ml MOHSW MOW nd np NSC O&M OC PE PIM PMO-RALG PSM RWSS RWSSP SE TOC UA UNICEF URT WASH WHO WLS WSDP WSP African Development Bank Group Basic Education Statistics Big Results Now Centre for Economic Research and Policy Community-Led Total Sanitation Community-Owned Water Supply Organisation Council Water and Sanitation Team Department for International Development Demographic and Health Surveys Difference-in-differences Distribution point Development Partners District Water and Sanitation Team Economics of Climate Change in Tanzania Evaluation question Government of Tanzania Independent Development Evaluation Department Policy and Operations Evaluation Department, Ministry of Foreign Affairs, The Netherlands Joint Monitoring Programme Local Government Authority Millennium Development Goal Millilitre Ministry of Health and Social Welfare Ministry of Water No date No page number National Sanitation Campaign Operation and maintenance Other Charges Personnel Emolument Programme implementation manual Prime Minister s Office Regional Administration and Local Government Propensity score matching Rural water supply and sanitation Rural Water Supply and Sanitation Programme Standard error Theory of change Units of Account United Nations Children s Fund United Republic of Tanzania Water, sanitation and hygiene World Health Organisation Weighted-least-squares Water Sector Development Programme Water and Sanitation Programme i

9 The Rural Water Supply and Sanitation Programme Summary 1. The Government of the United Republic of Tanzania (GOT), together with a number of development partners (DPs) including the African Development Bank Group (the AfDB), World Bank and United Kingdom Department for International Development (DFID) funded component 2 of the country s Water and Sanitation Development Programme (WSDP), Phase1 ( ). Component 2 of the WSDP was effectively the Rural Water Supply and Sanitation Programme (RWSSP). 2. The RWSSP (or component 2 of the WSDP, Phase1) was to improve the health and quality of life of the rural Tanzanians through provision of water and sanitation services on a sustainable basis. This goal was to be achieved by improving (i) district level capacity to implement demand-based RWSS programmes; (ii) access of rural communities to water and sanitation services operated and maintained by capable persons; and (iii) health and hygiene practices. The RWSSP was intended to cover all rural areas in mainland Tanzania, and to benefit the overwhelming majority of the rural population who lacked access to safe water and adequate sanitation. Its beneficiaries also include school children, local government authorities (LGAs) and community-based water organisations. The RWSSP comprised three key components (i) construction of rural water supply and school sanitation facilities, (ii) national sanitation campaign and school WASH, and (iii) management support for LGAs. 3. The RWSSP was implemented in , and the AfDB s contribution, through basket funding, was made in two phases; and The total planned commitment to the RWSSP by all funders for the period was USD (or UA4223) million (m) with the AfDB accounting for about 28 percent of it. As at the end of December 2015, the AfDB has disbursed a total of USD (UA ) m in loans and USD (UA 10) m in grant funding to the RWSSP. The other major contributors to the RWSSP funding include the GoT, World Bank (IDA) and DFID. The total commitment to the WSDP was USD 1.67bn, of which USD 1.48bn (88 percent) had been disbursed as at 31 December The GoT s contribution to this total commitment was USD 369m, all of which was disbursed. The implementation of the RWSSP was to end in 2014, but had to be extended to December With late and slow initial implementation, the RWSSP was restructured in Construction of new rural water schemes only began at scale in 2012, picking up speed in Implementation was accelerated by the inclusion of the water sector in the national Big Results Now (BRN) initiative (2013/ /16) and the adoption of a goal to implement ten rural water projects in each LGA area. The LGAs were responsible for the installation of water schemes, co-ordinated by the Ministry of Water (MOW). They contracted water engineering firms to install the water infrastructure and appointed consultant service providers to design and supervise construction of the schemes. In many cases, these consultants also provided institutional facilitation and training for the Community-Owned Water Supply Organisations (COWSOs) that were established to own and operate the schemes. The National Sanitation Campaign (NSC) resulted from the ii

10 Impact Evaluation of the Rural Water Supply and Sanitation Programme, United Republic of Tanzania: Volume I conversion of the sanitation subcomponent into a separate entity under the co-ordination of the Ministry of Health and Social Welfare (MOHSW). This restructuring was intended to enhance the performance of the sanitation subcomponent, which had been poor. Impact evaluation of the RWSSP 4. The Independent development Evaluation (IDEV) of the AfDB commissioned this impact evaluation of the RWSSP; effectively Component 2 of the WSDP, Phase I, with the purpose of: Providing credible information on the emerging impact of the RWSSP to relevant Ministries and agencies of the GOT, the Board of the Bank and DPs.; Providing relevant lessons and recommendations to the GOT, Bank management (including the Water and Sanitation Department) and DPs for sustaining the RWSSP s benefits, and for informing the design and implementation of Phase 2 of the WSDP; Helping IDEV build the evidence for its 2015 Tanzania Country Strategy Paper evaluation and water supply and sanitation thematic evaluation. 5. The evaluation questions mainly focus on the key RWSSP outputs (water, sanitation and hygiene facilities and services, and institutional capacity) and impacts (access to and use of improved water and sanitation; incidence of diarrhoeal disease among children under five years of age; enrolment of girls at school; women s participation in economic activities; self-employment in community-based WASH service delivery; sustainability), and also on the how and why concerning the extent of the RWSSP impact. The study used multiple lines of evidence to answer the evaluation questions. It responded to the impact questions by comparing the relevant RWSSP results at the household and community levels to those of appropriate and carefully matched non-rwssp community groups. This approach was taken because the RWSSP was designed and implemented without incorporating the data needs of an impact evaluation. Data were collected from RWSSP and non-rwssp communities (240) and households (2374) in 30 of the 79 qualified LGAs (an LGA was qualified if it had constructed at least four water schemes under the Ten Villages initiatives by July 2014). Data were also gathered from 168 water points, 128 primary schools, 28 Council Water and Sanitation Teams (CWSTs) and 114 COWSOs. Additional data were also obtained secondary data sources, and interviews with samples of officials of GOT, LGAs, health centres, DPs, NGOs, CWST and COWSOs, as well as water points. The data analysis included propensity score matching (PSM) and difference-in-difference (DID). iii

11 Impact Evaluation of the Rural Water Supply and Sanitation Programme, United Republic of Tanzania: Volume I Key findings 6. The RWSSP delivered structures and procedures at LGA and community level for the planning, development and operation of rural water and sanitation but the associated budgets were neither fully in place nor efficiently coordinated. The RWSSP structures and procedures were in place and widely understood and capable of implementation by CWSTs and COWSOs. However, fund management from central to local government was dysfunctional thereby greatly hindering programme implementation. The RWSSP not only ensured the provision of training to the LGAs especially the CWSTs, but also the establishment and strengthening of user groups or COWSOs. The COWSO, a key institutional output of the RWSSP, was meant to own, manage, and fund the operation and maintenance (O&M) of the rural water supply system. With the RWSSP support, these bodies have developed slowly though the many formal stages meant to lead to legal registration as corporate entities. Only 17 percent (460) of the planned COWSOs have formally been registered. Women s membership of these COWSOs was modest. It was between 21 percent and 50 percent in 84 percent of the user bodies, for which the information was reported compared to the MOW guidelines of at least 60 percent. 7. COWSOs, CWSTs and some community level operators received training from the RWSSP though the training were uneven. Among the 114 COWSOs surveyed, 111 (97 percent) of them reported to have been trained in at least one relevant subject area. Yet in the case of the 28 CWSTs surveyed, only 17 reported they had received at least one training. The levels of responsibility and confidence displayed by these groups also differ widely due in part to their exposure to variable training. Some of the longer-established groups have now been through one or more cycles of elections, resulting in substantial (sometimes complete) turnover of membership. Significantly, there do not appear to be systems in place for tracking the arrival of new members and arranging their training. So far, this element of the programme has been treated as a once-off exercise, whereas in fact institutional maintenance will be needed on a recurrent basis. The effectiveness of the RWSSP institutions was limited by three key factors: (i) the COWSOs lack of operating resources and recurrent budgets resulting from users limited ability to pay for safe water and improved sanitation services; (ii) irregular, untimely and uneven disbursement of the GOT contribution; and (iii) weak coordination of water, sanitation and hygiene initiatives at the LGA and community levels resulting from the transfer of the delivery of sanitation and hygiene outputs to the National Sanitation Campaign (NSC). 8. The RWSSP made significant progress in installing functional rural water supply systems, but less so in the delivery of sanitation and hygiene facilities and services, and of coordination between rural water supply and sanitation components. By October 2015, the programme had developed 43,255 water points, compared with a target of 31,747. These water points were estimated to have served about 10.8 million rural people relative to the target of 8 million. A survey of this impact evaluation study reported that 82 percent of the 168 primary water sources in programme communities were functioning, compared with 54 percent in non-programme iv

12 Impact Evaluation of the Rural Water Supply and Sanitation Programme, United Republic of Tanzania: Volume I communities. Defective design was by far the most commonly mentioned reason for water sources not functioning; over-use of the facility and improper use (nature of the water schemes technically sophisticated and costly; quality of the water) were other significant factors. 9. Regarding the construction of improved latrines and school water, sanitation and hygiene (WASH), only 25 percent of the improved latrines and 16 percent of the school programmes had been achieved by June Also apparent were coordination issues at LGA and field levels with respect to rural water and sanitation interventions, which resulted from the separate status of the NSC within the programme. In the 120 RWSSP communities surveyed, only five percent of households reported having participated in hygiene promotion or training events. Twenty-five percent of these households had a hand-washing facility within three metres of their latrine the same proportion as in the sample of non-rwssp communities. Nor was there a significant difference between the proportions of households with improved latrines in RWSSP communities (25 percent) and non-rwssp communities (20 percent). Some of the latter may have been separately served by the NSC. (Sanitation and hygiene work in communities where the RWSSP was installing water supplies was done by the water supply consultants, not the NSC.) 10. Almost all (66) of the 67 schools surveyed had toilet facilities for pupils, with 65 reporting that these facilities were in use. However, only 48 percent of these 67 schools had a water supply at the toilets for hand washing. Of the 67 surveyed, 76 percent of primary schools at or near communities served by the RWSSP had a drinking water facility, of which 51 percent were functional. 11. The RWSSP had a statistically significant positive impact on the incidence of diarrhoeal disease among children under five years of age and the rural population in general. This impact was higher for the population as a whole (almost 10 percent) than for children under five (three percent). Although this impact was significant, it was modest compared with the percent improvement recorded by rural water and sanitation impact evaluations in some other countries. This may be due to the fact that 29 percent of respondents in sampled RWSSP communities reported using rainwater (mostly collected from the ground, not collected safely from roofs) as well as, or instead of, the improved water supply during the rainy season. This significantly reduced their expenditure on water, as well as the revenues of COWSOs. It is also notable that, in communities where RWSSP has installed water schemes, the proportion of households using less than the Tanzanian national standard of 25 litres per person per day (71 percent) is almost as high as the proportion in communities where the programme has not intervened (75 percent); and that 62 percent of 285 sampled households in RWSSP communities were found to be using water contaminated with E. Coli. 12. The RWSSP had a significant and positive impact on women s participation in economic activities but not on the enrolment of girls at school. These impacts were expected to accrue from time savings enjoyed by girls and women who no longer have to walk long distances to collect water. The RWSSP generated significant time v

13 Impact Evaluation of the Rural Water Supply and Sanitation Programme, United Republic of Tanzania: Volume I savings for girls and women fetching water. These time savings appreciably increased women s participation in economic activities, in particular, with respect to productive paid labour and on income-generating activities. This improvement is estimated to amount to a rise of 9.1 percent. However it had no detectable impact on girls enrolment in primary schools, which was already very high. 13. The RWSSP failed to stimulate the expected significant economic growth in the local water engineering sector across the district towns of Tanzania. The construction of so many new water supply schemes by the programme, and the subsequent demand for maintenance services from the COWSOs operating them, were meant to serve as the stimuli for the expected economic growth. However this has not happened, although hardware shops and, in some cases, NGOs do make money from marketing spare parts and materials. Most construction work was contracted to larger firms based in Dar es Salaam or other major centres. Crucially, COWSOs maintenance budgets were too inadequate to inject much new money into the local economy. 14. The RWSSP made good progress in building institutions and generated positive impact but its benefits are unlikely to be sustainable mainly because of the COWSO s financial and technical viability, inefficient fund management and poor coordination of water and sanitation interventions. Other contributory factors include the failure to recognise the ongoing need for institutional maintenance at community level and the failure of public-private sector partnership to grow as intended in the national policy. 15. The RWSSP implementation arrangements led to consultants designing schemes that often spanned several villages and depended on comparatively high-cost pumping and reticulation approaches. These imposed management and maintenance demands on COWSOs that were not, in fact, adequately resourced or skilled to meet them. Furthermore, funding and fiscal management weaknesses greatly undermined the implementation and sustainability of the RWSSP interventions. 16. The continuing segregation of water and sanitation efforts without appropriate coordination mechanism was also an impediment to the RWSSP sustainability. The separate implementation of the NSC has impaired coordination of these efforts at community level. Sanitation and hygiene indicators measured by the study show hardly any difference between communities that have received new water supply systems and those that have not, even though consultants for the water schemes were supposed to implement the sanitation subcomponent in the former. 17. The technical sustainability of the WSDP s rural water and sanitation interventions is uncertain for several reasons. Climate change is likely to increase the challenge of obtaining enough water. The widespread use of diesel generators to power borehole pumps means frequent breakdowns in supply, and maintenance costs that many COWSOs cannot afford. Nor may they be able to afford the lower maintenance costs of alternative technologies. vi

14 Impact Evaluation of the Rural Water Supply and Sanitation Programme, United Republic of Tanzania: Volume I 18. The replicability of the RWSSP approaches to rural water and sanitation can be challenged by the institutional and technical constraints detailed above. Such constraints must be overcome if the RWSSP approaches are to become replicable. Above all, it is necessary to recognise that the development of user institutions in the sector is not a once-off achievement they impose ongoing support obligations for which systems, procedures and budgets must be in place. This goes beyond the existing, and widely recognised, challenge of inadequate recurrent budgets for LGAs, which greatly diminishes their ability to provide even technical support to COWSOs. Lessons 19. The efficiency and effectiveness of an otherwise sound and competent programme can be seriously compromised by inefficient management of funds at central government level. 20. The current levels of budget subventions to CWSTs gravely compromise the sustainability of the RWSSP achievements. 21. Without reformed financial management systems linking national and local government authorities, and without adequate recurrent funding to CWSTs, the progress and sustainability of Phase II of the WSDP will be as inadequate as the RWSSP. 22. For a rural water and sanitation programme to be adequately effective it is essential that the water and sanitation components be adequately coordinated. 23. The depth of poverty in rural Tanzania is such that many people are prepared to turn away from an improved water supply to an unimproved one when the latter is fairly readily available (in the rainy season). This has important implications for the achievement of the health outcomes that improved rural water supplies are expected to achieve. 24. Current conditions make it difficult for users to pay for private sector services during the operation and maintenance phase, and the private sector is consequently poorly developed in rural Tanzania with regard to such services. An ongoing public sector role in support of O&M must be anticipated. 25. Ongoing institutional maintenance is a prerequisite for sustainability in the sector. The public sector has a continuing responsibility to ensure that this maintenance takes place although elements of the required support services could, in theory, be provided by the private sector. vii

15 Impact Evaluation of the Rural Water Supply and Sanitation Programme, United Republic of Tanzania: Volume I Recommendations 26. On the basis of its findings, the evaluation makes the policy and programming recommendations below. As the RWSSP was funded by multiple partners including GOT, the GOT should lead the implementation of the recommendations although the AfDB and other DPs play a facilitating/supporting role. 27. The AfDB and other DPs in the water and sanitation sector should maintain continuous and effective dialogue with and/or support of the GOT and relevant Ministries and agencies in order to: Expedite Government s reforms to public finance management systems that ensure a predictable, smooth and timely transfer of domestic and donor funding to LGAs for the development, operation and maintenance of rural water and sanitation systems and programmes. Ensure that Government undertakes a careful review of affordability issues as they relate to the provision of the National Water Policy for users to pay for improved rural water supplies. While current poverty levels persist, the effectiveness of these improved supplies in enhancing health outcomes will remain compromised while users turn to unimproved water for financial reasons. Ensure that government replaces the current ad hoc approach to financial support from LGAs to COWSOs for these costs with revisions to the National Water Policy that specify what types of COWSO maintenance and repair cost will be subsidised, and how. It should adjust budgetary subventions to LGAs accordingly while government should recognise that, at least for the remainder of the WSDP period (to 2025), most COWSOs will be unable to pay in full for the maintenance and replacement costs of their water supply infrastructure. Ensure government increases, within feasible fiscal limits, recurrent funding to CWSTs. Ensure that the GOT s policy and procedures for rural water supply and sanitation recognise more explicitly that long-term institutional maintenance is vital for the sustainability of COWSOs and RWSSP benefits. With the support of DPs, government should adjust personnel, training and recurrent budgets to reflect this reality. Ensure the operational integration, in WSDP Phase II, of the National Sanitation Campaign with programmes for the development of rural water supplies, with appropriate arrangements for coordinated management and continuing provision for the ring fencing of sanitation budgets. Ensure that the MOW enhances its current efforts to review and develop the options of mains electricity connection and solar technologies for rural water supply systems using groundwater. Ensure that the MOW intensifies its efforts to upgrade field monitoring and data management for the rural water subsector in complementing the National Sanitation Management Information System that viii

16 Impact Evaluation of the Rural Water Supply and Sanitation Programme, United Republic of Tanzania: Volume I is being developed. In consultation with the President s Office Regional and Local Government, MOW should build capacity and systems for real time reporting of water supply status from COWSO level through LGAs to central Ministry databases. Ensure that the MOW and the MOHSW focus part of the upgraded field monitoring system on routine collection of a limited set of indicator information that can feed into impact studies covering, for example, water quality, amounts of safe water consumed, the extent to which non-scheme water sources continue to be used, the extent of adoption of sanitation and hygiene practices and time savings and alternative time uses by women and girls. Ensure that Government and its DPs in the rural water and sanitation sector undertake another impact evaluation of RWSSP in five years time long enough for the results and sustainability of these rural and water sanitation initiatives to be clearly visible. The current study has been undertaken quite soon after the implementation of the initiatives whose impact was to be tested. ix

17 1 Background 1.1 Country context The United Republic of Tanzania (URT) was formed in 1964 by Tanganyika and Zanzibar, both of which had recently gained independence from Britain. Over its first half century Tanzania has mostly enjoyed political stability. However, economic growth was slow and poverty remained entrenched during most of that period. Tanzania is still classified as a low-income country, ranking 159 out of 187 in the 2014 United Nations Development Programme Human Development Index with a score of (in the bottom quartile). Kenya ranked 147 with a score of 0.535; Mozambique, 178 with a score of (UNDP, 2015). The most recent Human Development Report states that with the exception of some notable progress in a few areas such as child survival (reduction of child mortality rates) and school enrolment, improvements in the overall status of human development in Tanzania are only marginal (UNDP and GOT, 2015a: viii). The report concludes that the Millennium Development Goal of reaching a poverty level of 18 percent by 2015 is unattainable and notes that poverty in Tanzania is a rural phenomenon (UNDP and GOT, 2015a: 3, 4). Over the past decade, however, economic growth has accelerated. Tanzania s average gross domestic product (GDP) growth rate was 6.9 percent over the period and reached 7.0 percent in Like Mozambique, Tanzania now has the prospect of major new revenues from large offshore gas reserves that it has started to exploit. New wealth and new equity challenges are in prospect. Meanwhile, however, while rapid economic development is evident in Dar es Salaam and some other large towns, poverty is still widespread among the estimated population of 47m (in 2014). Approximately 28.2 percent of the population lived below the poverty line in 2012, a reduction from 34 percent in During the 2007/2012 period, there were improvements in living conditions, access to basic education, health and nutrition and, labor force participation in nonagriculture employment. Nevertheless, these benefits were not distributed equitably. Inequality has increased between urban and rural population and approximately 12 million Tanzanians are still living in poverty. 1 World Bank, 2015a. In 2014, Tanzania ranked 47 th out of 142 countries in the Global Gender Gap Report (World Economic Forum, 2015). But women continue to be more likely than men to be poor and illiterate; 24.7 percent of female-headed households lived below the poverty line in 2012 (GOT, 2015a: 30). Climate change is a growing threat to livelihoods and prosperity in Tanzania (ECCT, 2015). While floods are an increasingly significant hazard, many parts of the country are currently suffering severe drought. In October 2015, the chronic national electricity shortages were exacerbated when all hydro-electric plants had to be closed due to a lack of water in their dams (BBC, 2015). The water and sanitation situation in Tanzania mirrors the growing urban-rural divide and underscores the increasing concern about equitable development in the country. Access to safe drinking water remains a major challenge for many Tanzanians, particularly those in rural communities. According to one recent study, it has declined from 55 percent to 53 percent for rural people over the last two decades despite high levels of development funding for the sector. Between 1995 and 2005, Tanzania received USD 57 per beneficiary in aid flows earmarked for water but coverage fell by one percent (Twaweza, 2014: 2, 3). According to recent official statistics, only 8.3 percent of the rural population were using an improved sanitation facility, while 34.2 percent of the urban population did so (GOT, 2015a: 82). In 2015, 40 percent of Tanzanian schools were reported to have no water supply; 84 percent had no functioning hand washing facilities and there was an average of one latrine per 56 pupils (SHARE et al., 2015: 10).

18 The most recent assessment of countries progress towards the Millennium Development Goals (MDGs) showed Tanzania having made limited or no progress towards the targets for use of improved drinking water sources and improved sanitation facilities. For the former it showed 56 percent of the national population using improved sources (compared with 54percent in 1990), and for the latter 16 percent using improved facilities (seven percent in 1990). In urban areas in 2015, it showed 77 percent of the population using improved water sources, compared with 46 percent in rural areas - up just one percentage point over the 1990 figure. For sanitation, urban use of improved facilities was 31 percent, and rural use eight percent of the population - again, one percentage point higher than in 1990 (UNICEF and WHO, 2015: 74-75). However, open defecation is comparatively rare. In 2015, 75 percent of the rural population were estimated to use shared or other unimproved sanitation facilities, while 17 percent practised open defecation - up from 10 percent in 1990 (UNICEF and WHO, 2015: 74). Figure 1. Water carts are an increasingly common means of transport 1.2 Programme funding and objectives In 2006, the African Development Bank (AfDB) approved funding of USD 84m for Phase I of the Rural Water Supply and Sanitation Programme (RWSSP) in Tanzania. Phase II of AfDB funding, contributing USD 65m, ran from 2011 to The RWSSP is a contribution to Component 2 (Rural Water Supply and Sanitation) of the National Water Sector Development Programme (WSDP), According to data provided by its Country Office, the AfDB has disbursed a total of UA m in loans and UA 10 m in grant funding (USD m and USD 13.72m at December 2015 exchange rates) to the RWSSP. The United Kingdom Department for International Development (DFID) is another major contributor to this component. As at 31 December 2015, total revised commitments to the WSDP were USD 1.67bn, of which USD 1.48bn (88 percent) had been disbursed. The government contribution to this total commitment was USD 369m; 100 percent of this commitment had been disbursed. Phase I of the WSDP was originally planned to end, from the GOT s perspective, in 2014, although from the donor perspective it continues until December 2015, in order to draw on the funds they provided for that phase. The Ministry of Water (MOW) subsequently extended its implementation of WSDP Phase I to December 2015 as well. 2

19 As stated for Phase II of RWSSP funding, the sector goal of AfDB support to the programme was to improve the health and quality of life of the rural Tanzanians through provision of water and sanitation services on a sustainable basis. The programme objectives were stated to be to improve district level capacity to implement demand-based RWSS programmes ; to improve access of rural communities to water and sanitation services operated and maintained by capable persons ; and to improve health and hygiene practices (AfDB, 2010: iv). 1.3 Theory of change In preparation for the evaluation, a theory of change (TOC) was drawn up, based on RWSSP design documentation. It should be noted that RWSSP design itself did not explicitly specify a TOC, although the results frameworks included in RWSSP appraisal reports included elements of one. The principal distinguishing feature of TOC approaches is that they identify the assumptions on which the design logic is based. This can guide a theory-based impact evaluation in its assessment of why intended final outcomes were, or were not, achieved. TOC analysis offers a critical analysis of the relationship between outputs and outcomes and can explain why, in some cases, positive outputs do not achieve the anticipated outcomes. The discussion below identifies the key assumptions inferred from the TOC, as shown in Figure 2 below and presented in the inception report for this evaluation (Mdadila and Turner, 2015: 5-7). The TOC focuses on the principal causal links and stages between inputs and final outcomes but adds a preliminary stage of targeting, awareness and planning that should lead to the required inputs for the RWSSP process being in place. A key assumption in this regard is that systems, procedures and budgets are indeed in place, co-ordinated and efficiently implemented. This will enable the targeting, planning and related preparatory processes that lead to the provision of inputs at scale in a manner that matches community awareness and expressed need to available resources. Crucially, the TOC assumes that the policy being implemented by the RWSSP is conducive to efficient achievement of the intended outputs and outcomes. Whether the inputs shown in Figure 2 achieve the specified outputs depends, in turn, on some important assumptions. One is that design and construction are technically sound. Another is that the new schemes are able to deliver water that is safe for human consumption. User responses are fundamental to achieving the intended outputs. The TOC assumes that communities and households will commit to developing the required user institutions and installing domestic sanitation and hygiene facilities. Another important assumption is that the awareness raising and institutional development carried out by service providers are effective in creating competent Community-Owned Water Supply Organisations (COWSOs) or similar structures. Additionally, the TOC assumes that these user management bodies are able to establish working relations with operation and maintenance (O&M) service providers. Finally, it assumes that the Council Water and Sanitation Team (CWST) 1 is competent and adequately resourced to co-ordinate planning and delivery of the intended outputs. Furthermore, important assumptions link outputs to outcomes in the RWSSP theory of change. For example, the TOC assumes that community members actually use the improved water supply at all times; that the quality of the water remains safe; and that the system continues to deliver this water, in adequate quantities, at all times. A second major assumption is that there is no relapse in sanitation and hygiene practice, and that people continue to use the facilities and practise the enhanced hygiene that have been introduced. Similar assumptions apply to the link between school WASH outputs and the intended outcomes in Tanzanian schools. Moreover, the TOC assumes that the institutional outputs are converted into sustained institutional outcomes: COWSOs or other structures that remain in place and continue to operate and maintain water services effectively with appropriate support from LGAs. This implies an ongoing, constructive relationship of communication and support between LGAs and user institutions. Outputs and outcomes are also linked by the significant assumption that the private sector provides an efficient, effective and affordable maintenance service to water user bodies. A final and vital assumption in the TOC is that the output of an improved water supply in the 1 More commonly described during WSDP Phase I as the District Water and Sanitation Team (DWST). 3

20 community reduces the amount of time that girls and women must spend collecting water and bringing it to their households. If the number of households per water point is too great, or the supply at the water point is limited, this assumption may not hold true and time savings may be limited or negligible. The final set of assumptions in the TOC links outcomes to the intended final outcomes: the results that this evaluation aims to test directly. Full health benefits are realized only when drinking water is safe at the point of use; enough safe water is available the whole year round and within a short distance from the household; there is large scale access to toilets that meet the requirement of hygienic separation of human excreta from human contact (through a slab that can be cleaned), and hygienic use of toilets; and when hands are washed with soap or a soap substitute at all critical times. Furthermore it is intended that the interventions will have economic benefits demonstrated, for example, through a positive effect on local (self) employment in WASH service delivery, and time savings that will improve women s employment opportunities and/or time spent on income generating activities. Both these sets of results depend on the economic context being conducive for example, local support enterprises must be able to source inputs and to operate profitably, and there must be economic opportunities for women to exploit with the time saved when water collection is made easier. This presentation of the RWSSP TOC is thus intended to help identify the key assumptions in the design logic of the Program. It also seeks to identify the factors and processes that the evaluation must explore to explain the extent to which the intended final outcomes (as represented by the proxy indicators shown in Figure 2) have been achieved. 4

21 Figure 2. RWSSP Tanzania: theory of change Targeting, awareness and planning Inputs Outputs Outcomes Final outcomes Communities aware, or made aware, of opportunities for - and benefits of - improved water and sanitation Communities apply for rural water supply and sanitation (RWSS) projects CWST appraises and prioritises village project proposals, in accordance with RWSSP Project Implementation Manual (PIM) CWST prepares Council Water and Sanitation Plan MOW and PMO-RALG review and approve CWSP Community Project Agreement signed MOW makes RWSSP funding (from AfDB and other sources) available Community make contribution in line with policy Consultants and contractors design and construct domestic water facilities and school WASH facilities Households build domestic sanitation facilities CWST service providers deliver institutional facilitation and development Private sector spares and maintenance services (made) available Adequate, functioning, safe domestic water supplies available to domestic users within appropriate distance Improved and functioning domestic latrines and hand washing facilities in place Adequate, functioning, safe school WASH systems in place Trained and capable COWSO or other management group in place Functioning maintenance services available All community members have adequate supplies of safe drinking water at all times All community members practise appropriate sanitation and hygiene Women and girls spend less time on water collection Girls, boys and teachers all have adequate safe drinking water supplies and appropriate sanitation and hygiene facilities (including menstrual hygiene) at schools COWSOs or other community RWSS institutions have resources, skills and commitment to operate and maintain RWSS facilities effectively CWSTs provide effective technical and institutional support to users Private sector provides affordable, effective maintenance services Reduced incidence of waterborne disease (proxy indicator: diarrhoea in children under five) Girls less constrained by water collection and sanitation/ hygiene concerns from advancing their education (proxy indicators: girls enrolment and attendance at primary school) Local employment created through WASH service delivery enterprises Women less constrained by water collection from pursuing income generating activities 5

22 1.4 Summary of programme implementation The WSDP and the RWSSP had a slow start. The WSDP - described as probably [the] largest national water programme operating in Africa today (GOT, 2013a: 1) - started one year late, and although a number of quick win (mostly rehabilitation) projects that had already been identified and designed could be implemented without too much delay, a 2010 review found that the main activities under WSDP will start only in its fourth year (GOT, 2010a: iv). A Programme Implementation Manual was developed, but proved to be too complex for practical application (GOT, 2013a: ii). A restructuring plan was approved in June 2011, including an amended results framework, procurement plan and disbursement indicators. The Programme Implementation Manual was revised, too (GOT, 2014a: 4). Construction of new rural water schemes only began at scale in 2012, picking up speed in This followed an extensive period of delay in developing and implementing procurement procedures for consultants to design new schemes which, once designed, often proved to be more technically sophisticated and expensive than had been envisaged. The 2013 evaluation of WSDP Phase I concluded that relatively few of the rural water schemes planned under basket funding have materialised so far and this component is unlikely to make a significant impact on national access figures by The reasons for the disappointing output appear to be a combination of (a) delays in procurement, (b) irregular and somewhat unpredictable fund releases from the centre (c) limited absorptive capacity at LGA level and (d) the time taken to establish an effective role for the Prime Minister s Office Regional and Local Government and achieve good MOW/PMO-RALG coordination. Expenditure in this component has fallen further behind scheduled allocations to a greater extent than the other programme components (GOT, 2013a: 28). The water sector was one of six sectors identified for focused development through the national Big Results Now (BRN) initiative (2013/ /16). This effort was mainstreamed within the WSDP and led to a renewed commitment to achieve 67percent rural water coverage by 2015 (GOT, 2014b: 22) through the implementation of 10 rural water projects in each LGA area. This ten villages campaign has helped to accelerate RWSSP implementation since It did not involve significant technical adjustments to the RWSSP approach, but did replace previous efforts at comprehensive district-wide RWSS planning with a focus on a few villages in each LGA a development deplored in the 2013 Water Sector Status Report (GOT, 2013b: 72). The National Sanitation Campaign (NSC) resulted from the conversion of the sanitation subcomponent of Component 2 of the WSDP into a separate entity under the coordination of the Ministry of Health and Social Welfare (MOHSW). This restructuring was intended to enhance the performance of the sanitation subcomponent, which had been poor. Since the 2011 restructuring, total investment in the NSC was USD 23.9m, of which USD 19.3m (81 percent) had been released by October 2015 (GOT, 2015c: 3). The four-year target was to construct 1.52m improved latrines and implement school water, sanitation and hygiene (WASH) programmes in 812 schools. By June 2014, only 25 percent of the improved latrines and 16 percent of the school programmes had been achieved (GOT, 2014a: 18). Through the RWSSP, the AfDB has been funding this sanitation component, along with DFID. The Water and Sanitation Programme (WSP) of the World Bank has provided technical assistance. The separate status of the NSC within the WSDP has led to coordination issues at LGA and field levels between rural water and sanitation interventions. Part of the restructuring agreement was an arrangement for the consultants already appointed to work on ten villages water schemes to carry out the sanitation subcomponent in those communities. Consequently, this sanitation initiative was separated from the new sanitation work undertaken elsewhere by the NSC. 6

23 Figure 3. Water sellers' carts in a community with no supply: water drawn from a nearby dam 1.5 Impact evaluation of the RWSSP The Independent Development Evaluation (IDEV) of the AfDB has commissioned this impact evaluation of the RWSSP. The evaluation was intended: to provide credible information on emerging development impact of the RWSSP to relevant Ministries and GOT agencies, the Board of the Bank and DPs; to provide relevant lessons and recommendations to the GOT, Bank management and DPs for sustaining the RWSSP s benefits, and for informing the design and implementation of Phase 2 of the WSDP; to help the Bank s IDEV build the evidence for its 2015 Tanzania Country Strategy Paper evaluation and water supply and sanitation thematic evaluation. The TOR for this evaluation (Annex 1) gives more detail on its purpose: The objective of the impact evaluation is to provide credible estimates of the impact of AfDB supported interventions at the level of the ultimate target groups, assess if Programme interventions work as expected, assess sustainability of results and derive lessons and recommendations that can be used to improve the effectiveness of AfDB assistance. The impact evaluation will also be used as an information source for a higher level AfDB Tanzania Country Strategy and Programme Evaluation and a Water Supply and Sanitation thematic evaluation. The AfDB support for this sector the RWSSP - was a basket funding contribution to Component 2 of the WSDP. There was no separate technical design for the AfDB input, and this evaluation is therefore concerned with the RWSSP as an input to Component 2 of the WSDP. It is not possible to offer a comparative attribution of impact achieved by different contributors to the basket, such as the AfDB or the World Bank. Effectively, therefore, this is an impact evaluation of Component 2 of Phase I of the WSDP. 7

24 1.6 Evaluation questions The TOR also set out a tentative list of evaluation questions (EQs) that the evaluators expanded, in consultation with IDEV, during the inception phase of the assignment. As set out and subsequently approved in the inception report, the 18 EQs that this study will address are as follows. These, rather than the EQs in the TOR, are the questions on which the evaluation is required to focus. 1. What types and quantities of water supply, sanitation and hygiene outputs has the RWSSP achieved at household and school levels? 2. What types, quantities and quality of capacity development has the RWSSP achieved at LGA and community levels with regard to the design, delivery, operation, maintenance and sustainability of WASH interventions? 3. What is the quality of drinking water in beneficiary communities, households and school, at point of supply and point of use? 4. Do the quantity and proximity of improved drinking water supplies 2 available to beneficiary households meet international standards? 5. What is the quality of latrines 3 in beneficiary households and schools? 6. Are beneficiary households and schools using drinking water as intended by RWSSP design? 7. What time savings are accruing to girls and women as a result of RWSSP interventions? 8. To what extent are the sanitation and hygiene practices promoted by the RWSSP implemented by beneficiary households and schools? 9. Are the changes in water, sanitation and hygiene use and practice achieved by RWSSP interventions uniformly distributed between and within beneficiary communities and households? 10. What targeting, awareness and planning activities precede RWSSP interventions? 11. What are the principal features of the policy that guides RWSSP interventions? 12. What institutional outputs has the RWSSP achieved at community level? 13. What institutional outcomes has the RWSSP achieved at community level? 14. Are the institutional assumptions embodied in the RWSSP TOC valid? 15. Can changes in the incidence of diarrhoea in children aged under five in beneficiary communities as a proxy for incidence of waterborne diseases be attributed to RWSSP interventions? 16. Can changes in the enrolment and attendance of girls from beneficiary communities at school a proxy for enhanced educational opportunity be attributed to RWSSP interventions? 17. Can changes in the income generating activities practised by women from beneficiary communities a proxy for enhanced economic opportunity be attributed to RWSSP interventions? 18. Can changes in local (self) employment in community-based WASH service delivery be attributed to the RWSSP? 19. Are the results of RWSSP interventions technically sustainable? 2 In the framework of the WHO/UNICEF Joint Monitoring Programme (JMP) of MDG improved water and sanitation facilities, an improved water source is defined as one, that by nature of its construction and when properly used, adequately protects from outside contamination, particularly from faecal matter. 3 In the framework of the WHO/UNICEF Joint Monitoring Programme (JMP) of MDG improved water and sanitation facilities, an improved sanitation facility is defined as one that hygienically separates human excreta from human contact. These include pit latrines with slabs. 8

25 20. Are the results of RWSSP interventions institutionally sustainable? 21. What are the links between technical and institutional sustainability, and are they adequately reflected in the RWSSP TOC? 22. Is the RWSSP TOC replicable at the intended scale? Table 2 below presents a summary guide to the study s responses to the evaluation questions, as discussed in chapter Scope of the impact evaluation Although commissioned by the AfDB as an impact evaluation of its RWSSP, the thematic scope of this study is Component 2 of the WSDP. The two programmatic titles are therefore used interchangeably in the report. It can be seen that the evaluation questions listed above comprise a mix of descriptive, causal and evaluative questions. Geographically, the evaluation spans mainland Tanzania. Sample communities (section 1.10 au-dessous) were drawn from the whole mainland area. The period covered is Phase I of the WSDP, from 2006 to mid-2015, six months before the extended close of the Phase in December This spans Phases I and II of AfDB funding for the WSDP through the RWSSP. However, as explained in section Erreur! Source du renvoi introuvable., physical implementation was limited in the early years. Implementation only began at scale in Effectively, therefore, the evaluation must assess the impact of rural water and sanitation efforts since 2012 with many of the schemes commissioned only in 2013 or As such, it complements the findings of the evaluation of WSDP Phase I carried out in 2013 when it was thought that that phase would end in 2014 (GOT, 2013a). 1.8 Activities during the impact evaluation The evaluation began with a preparatory mission by the evaluation manager and the consultants, between 26 February and 12 March During that mission, preliminary consultations were undertaken with government (in particular the MOW), the AfDB and DPs and with potential service providers who were invited to tender to evaluate the performance of the field survey. The mission also undertook familiarisation visits to water schemes, CWSTs and COWSOs in Mkuranga and Kisarawe districts and collected and reviewed relevant documentation. IDEV is committed to developing evaluation capacity in African governments. It has facilitated initial training on impact evaluation for Government staff, including staff members of the Tanzania MOW. The preparatory mission met two of them to arrange their participation in the evaluation. These officers subsequently took part in some of the fieldwork. The Centre for Economic Research and Policy (CERP) at the University of Dar es Salaam won the tender for field survey services and carried these out from 3 August to 7 September 2015 after an intensive period of survey instrument design, enumerator training and piloting. Questionnaires were administered in the field in Swahili. Although the main language used during enumerator training was English, the training process involved careful cross-language consolidation and checks to ensure that all staff understood and expressed the questions in the same way in Swahili. One of the consultants carried out a further mission in October November 2015 to investigate institutional issues arising from the WSDP in more detail. In addition to further meetings with the MOW, AfDB and other relevant agencies in Dar es Salaam, he visited CWSTs and COWSOs in seven of the 30 LGAs that had been sampled for the evaluation field survey to observe and discuss WSDP institutional arrangements in greater depth. For this fieldwork, he was assisted by one of the CERP survey supervisors. Seven LGAs were selected, 9

26 within logistical limits, to represent the range of more and less successful schemes as well as technologies, institutional progress, natural conditions and demography. Following analysis of the field survey and institutions mission data, this evaluation report has been prepared. Details of the methods used are presented in chapter 0 au-dessous. Methodology 1.9 Impact evaluation design EQs (section 1.6 ci-dessus) are direct impact evaluation questions. Ideally, the changes about which they ask would be measured over time with reference to a baseline. No baseline data are available. Differences between treatment and control communities must therefore be analysed as a proxy for the change that might be expected over time as a result of the programme intervention. Answering the other EQs will be necessary because this is a theory-based evaluation that seeks to assess the accuracy of the assumptions in the programme s implicit theory of change. Some of these other EQs could ideally be answered from routine performance monitoring data, but in reality not all such data are available. It was therefore necessary to answer them as fully as practicable by reference to interviews, available statistics and field observations. The key impact evaluation question this study seeks to answer is: what would have happened to the outcome variables of interest in communities supported by the RWSSP had the programme not been implemented? Answering this question requires identifying and estimating the counterfactual situation. More specifically, answering it requires estimating outcomes in two states - with and without programme intervention. Estimating the former is straightforward since it is directly observed. Estimating the latter however poses a challenge because it is not directly observed. This challenge is commonly referred to as the counterfactual problem or missing data problem (Heckman et al., 1998). The goal of an impact evaluation is to establish two groups, participants in the programme (treatment group) and non-participants in the programme (comparison group), that are statistically identical in all other aspects except programme participation/treatment. The challenge then is to identify a valid comparison group, a group with similar characteristics to the treatment group. The most robust way to measure impact would be to randomly assign some communities with programme treatment and keep the remainder as a comparison group. In a way, the average value of an outcome variable of a comparison group serves as a proxy for the counterfactual outcome of the treatment group had it not participated in the programme. In Tanzania, RWSSP interventions were not randomly assigned. The assignment used a demand-driven approach where, among other things, communities specifically expressed getting improved water supply as their priority. A direct comparison of mean outcomes between treatment and control communities in these circumstances is most likely to produce biased results and compromise the internal validity of the impact evaluation. Nevertheless, a before-and-after approach could also be misleading because it is most unlikely that the treatment communities experienced no other changes than the programme treatment. In the real world a number of factors that affect outcome variables of interest might be changing over time (Gertler et al., 2011). In particular, it is difficult to isolate general time trend effects from the actual programme impact (Winters et al., 2010). For these reasons the before-and-after approach was not ideal for this impact evaluation and was not applied in this study. Instead, the study used quasi-experimental methods, which are commonly used for estimating impacts due to programme implementation. In particular, the study utilised propensity score matching (PSM) methods; and matched-difference-in-differences, a combination of PSM and differencein-differences (DID) approaches. Estimating the impact of the programme using PSM and the matched-did methods is potentially subject to estimation bias emerging from observable heterogeneity across communities. To minimise such a bias, pre-intervention and time invariant community characteristics were taken into account in the estimations (Heckman et al., 1998; Lokshin and Yemtsov, 2005; Smith and Todd, 2005). It should be noted 10

27 that baseline information was not collected at the inception of the RWSSP. This constrained utilization of some methods such as DiD, which requires at least two data points including the baseline. For this study it was possible to reconstruct baseline information and use the DiD methods for only two variables, related to time taken to fetch water and the distance covered in fetching water. To increase the comparability of sample observations drawn from both treatment and non-treatment communities, the suggestions by Heckman et al. (1998) were followed: first, both treatment and non-treatment sample observations were drawn from the same geographic locations with similar socio-economic settings. Secondly, similar survey instruments were used to collect data from the respective two study groups. Thirdly, data from both treatment and non-treatment groups were collected around the same period using the same interviewers. To ensure consistency and quality of the collected data, training was provided to enumerators on the survey instruments; a pilot was conducted; and refinement of the instruments was undertaken. Swahili was the medium of communication during the interviews. As part of the training, enumerators and trainers went through the survey instrument and agreed on a common interpretation and translation into Swahili. This was important because the medium of exchange was Swahili and questionnaires were prepared using English. Furthermore, during the fieldwork there were close communication between supervisors and enumerators and between the teams across the zones on the survey challenges and successes. All these were aimed to ensure consistency in the data collected. Furthermore, apart from employing the PSM and DiD techniques, the study also estimated the impacts of the RWSSP using a weighted-least-squares (WLS) regression. Under the WLS, the programme and non-programme units of observation are weighted by the inverse of estimated propensity scores to estimate impact (Hirano et al., 2003). To check for vulnerability of the estimated PSM results due to the possibility of presence of unobservable characteristics that influence the participation in and the outcomes of the programme, Rosenbaum s sensitivity test was conducted (Rosenbaum, 2010). The study also examined the influence of the beneficiaries pre-existing and exogenous characteristics on the observed/estimated impact of RWSSP interventions. Table 1 in Annex 3 (Volume II) provides definitions and measurements of the outcome variables of the study Sample size and sampling procedures The empirical data to estimate the impact of the RWSSP in Tanzania were collected from 240 communities and 2,400 households living within them. The sampled observation units were equally divided between treatment (RWSSP) communities and non-treatment (non-rwssp) communities. A total of 30 LGAs (falling within ten regions of the country) were randomly sampled and covered in the study. These LGAs were among 79 LGAs that qualified on the required sampling criteria. A LGA qualified for sampling if it managed to complete the construction of at least four water schemes under the Ten Villages initiatives by July This approach, sampling only from LGAs with at least four schemes, enhanced the logistical feasibility of the survey. Next, an equal number of RWSSP and non-rwssp sample communities and households were randomly selected within each LGA. Table 1 below summarises the distribution of the sample observations across the Regions. Similarly, Table 3 (Annex 3, Volume II) details sample distribution by LGAs under the study. The descriptions of sample size determination and sampling procedures are also given in Annex 3. 4 A total of 138 LGAs had completed at least one water scheme by July

28 Table 1. Distribution of the evaluation sample size No. of Treatment Sample Control Sample Total Region LGAs Communities Households Communities Households Communities Households Geita Kagera Kigoma Kilimanjaro Lindi Manyara Mara Mbeya Mtwara Mwanza Njombe Pwani Rukwa Ruvuma Singida Tabora Tanga TOTAL , , ,400 During the field survey, four LGAs that were initially included in the sample were found not to have completed four water schemes by July A replacement was done and new LGAs captured in the sample Survey instruments and implementation of field survey Survey instruments were designed to reflect and explore the theory of change identified for the programme (see section 1.3 au-dessus). The main data used for this impact evaluation were captured using household, community and water point questionnaires. Additional information was collected from primary schools and health centres available within the study area. The household questionnaire, which is the main instrument of the study, captured information on socio-economic and demographic characteristics, education profile, health, usage of water and sanitation facilities, hygiene practices, ownership of assets and engagement in economic activities. Furthermore, the community questionnaire captured information on both past and present characteristics of the sampled communities. In particular, the community questionnaire was used to collect data on population density, water supply, sanitation and hygiene, public infrastructure, land use characteristics, agro-ecology features and major economic activities. The water point questionnaire was used to gather information from sampled RWSSP and non-rwssp communities on various aspects of primary water sources. The water point questionnaire also collected information about water point management issues and community participation. The school questionnaire recorded enrolment and attendance of boys and girls, availability and use of water and sanitation facilities, hygiene practices, and school participation among boys and girls. Similarly, the health facility questionnaire recorded information on access to water and sanitation facilities, hygiene practices and diarrhoeal outpatient records at the health facility. Lastly, the WASH team questionnaire provided information on how the RWSSP activities are conducted including provision of water supply and sanitation services as well as capacity building at various stages of service delivery. The survey questions were pilot-tested, improved and administered by trained supervisors and enumerators. The survey took place between August and September The survey instruments used during the data collection for this study are presented at Annexes 6 12 in Volume 12

29 II. Annex 12 presents semi-structured interview guides used during the October 2015 institutional assessment mission Water quality testing In communities where the programme had installed water supplies, the survey teams collected water samples at distribution points (DPs) and from storage containers at household level. These samples were stored in clean containers and transported quickly to regional laboratories of the Ministry of Water, where they were tested for E. coli content (section au-dessous). In every sampled treatment community, three out of ten randomly sampled households were randomly selected for the water quality test. In this way, 30 percent of the treatment households underwent water quality testing. If there were no time and financial constraints, it would have been ideal to undertake water quality tests for all the sampled households, both in treatment and non-treatment communities, and to compare the outcomes. Given the constraints that did exist, water quality testing focused on treatment communities only, and more specifically on 30 percent of the households in treated communities. This sample is statistically reasonable to give credible results given the constraints. Furthermore, testing unimproved water from nontreated communities would not provide very useful information, as the chance that such water is contaminated is already high Post-survey data management and checks During the field survey, supervisors carried out daily checks of the questionnaires submitted to them by survey staff, identified any gaps and inconsistencies and ensured that these were rectified. On completion of the field survey, CERP staff captured the datasets digitally. The data were subjected to rigorous checks by CERP during and after capture and subsequently by the consultants. In the course of the capture and iterative checking processes, some coding and related errors were identified. These were rectified by referring back to the raw survey data as collected on the questionnaires. The post-survey checking processes were protracted and thorough. Satisfactorily clean datasets were only available for analysis by the consultants at the start of November Two of the 30 LGA questionnaires (see section 1.11) are still missing at the time of writing. This is because the LGA questionnaire included some data that CWSTs were not expected to provide during the interview itself, but were asked to fill in over the following few days before returning the questionnaire to the survey team. In these two cases, the questionnaire was not returned. Although this meant that LGA questionnaire data are not available for two of the 30 LGAs, all other instruments were successfully used in these LGAs. Only 26 sample households were missing from the final dataset, of which four households were non-response cases. Consequently, after cleaning the data we were left with 2,374 responses, which is percent of the intended sample. Again, there were no issues with water point samples. The initial plan was to collect a 30 percent sample from both treatment and non-treatment communities, which translates to 72 sampled water points for each of these groups of communities. Consultations with AfDB suggested that we only focus on treatment water points for the collection of water samples. In other words, we randomly selected 142 water points in treatment communities and collect the samples there. We were ultimately able to sample and collect information from 168 water points, above the intended target by 18 percent Existing empirical evidence As noted in section 1.11, two sources of existing empirical evidence that were used in the sample scheme areas were the records of health facilities and primary schools. A further source was the files of CWSTs, which these teams were asked to consult in providing the information requested by the LGA questionnaire (Annex 9, Volume 13

30 II). At central level, reference has been made to all available documentation on the WSDP and the RWSSP, as well as the broader background of the rural water and sanitation sector in Tanzania. Documents consulted have included records of sector reviews and co-ordination meetings and WSDP/RWSSP planning, reporting and evaluation reports. However, the extent and quality of performance indicator monitoring data up to 2015 were limited, inter alia because the MOW must rely on CWSTs (which have limited or no resources for site visits) to send it data on the status of rural water and sanitation. This meant that the impact evaluation had to rely primarily on its own field data collection, rather than existing monitoring databases Interviews with informants From some perspectives, the most important informants for this evaluation are the beneficiaries and local/district personnel who provided information and opinions during the main field survey and the follow-up mission on institutional issues. Informants in Dar es Salaam, who were interviewed during the preparatory mission and the mission on institutional issues, also provided vital information. During the two missions just mentioned, qualitative data were collected during interviews with CWSTs, COWSOs and water users at distribution points. For this theory-based evaluation, the views of these informants in the rural water and sanitation sector were essential inputs to analysis of the assumptions in the theory of change and the explanation offered below for the findings of the study on the impact of the RWSSP. Annex 2 presents a list of persons interviewed Limitations The principal limitation on this impact evaluation is the fact that the schemes whose impact is reviewed only began construction in 2012 with some only completed in Schemes completed less than 12 months before the survey period were excluded from the sample (section 1.10). If a scheme is effective in achieving the intended outcomes and impacts, these should be measurable a year after it was installed. Conversely, of course, there is a possibility that any positive impact will have dwindled or disappeared some further years later for example, because water supply regularity has not been maintained, on account of water quality having deteriorated or since adherence to good hygiene practices has lapsed. The findings of this impact evaluation must therefore be assessed with full recognition of the limited period that elapsed between installation of the schemes and performance of the field survey. We believe that the (limited) findings on impact set out in section 1.22 below are meaningful; nonetheless, a more conclusive analysis will only be possible after some further years have elapsed. Lack of baseline information was a limitation for the study. By design, the RWSSP did not conduct baseline data collection. Lack of such data made it impossible to use experimental methods. Instead, the study had to rely on quasi-experimental methods, at the expense of getting less credible impact estimates. Similarly, the DiD approach could only be applied for a few questions where recall questions were possible. It should be noted that, in the programme or treatment communities that were sampled, sanitation work was carried out or should have been carried out by the consultants contracted to design and facilitate the development of the water scheme and its COWSO. Sanitation efforts by the NSC did not take place in these sampled communities but were undertaken elsewhere, at varying distances from the sampled places potentially also in some of the control communities, diluting the power of our comparative analysis. In terms of data availability, a minor limitation is the lack of two of the planned 30 LGA questionnaires (section 14

31 1.13 au-dessus). A broader limitation is the general low coverage and poor quality of available field monitoring data from the rural water and sanitation sector (section 1.14 above). Methodologically, it is important to note that some key information collected during the sample survey is based on respondent estimates, for example, distances to water supplies and time taken or saved in relation to water collection. Given the scope and budget of this study it was not feasible to attempt empirical measurement of these variables. Findings and conclusions on some issues would have been strengthened if the survey teams had been able to collect more data. For example, the number of households in which water quality samples were taken was limited. The intention to collect health data on treatment communities from local clinics proved impracticable because these facilities normally serve many villages and it is difficult to extract data from a particular place. School attendance data were not collected as planned. This study was ambitious in its data collection plans and not all these plans could be fulfilled. This somewhat narrowed the analytical scope of the evaluation but did not invalidate its main findings. Findings 1.17 Introduction The presentation of findings in this chapter aims to answer the evaluation questions developed and confirmed during the inception phase of the study (section 1.6 au-dessus). Table 2 below presents a summary guide to the responses to these 22 questions, as presented in chapter 0. Table 2. Guide to responses to evaluation questions No. Evaluation question Section Page 1 What types and quantities of water supply, sanitation and hygiene outputs has the RWSSP achieved at household and school levels? 2 What types, quantities and quality of capacity development has the RWSSP achieved at LGA and community levels with regard to the design, delivery, operation, maintenance and sustainability of WASH interventions? 3 What is the quality of drinking water in beneficiary communities, households and school, at point of supply and point of use? 4 Do the quantity and proximity of improved drinking water supplies available to beneficiary households meet international standards? 5 What is the quality of latrines in beneficiary households and schools? Are beneficiary households and schools using drinking water as intended by RWSSP design? 7 What time savings are accruing to girls and women as a result of RWSSP interventions? 8 To what extent are the sanitation and hygiene practices promoted by the RWSSP implemented by beneficiary households and schools? 9 Are the changes in water, sanitation and hygiene use and practice achieved by RWSSP interventions uniformly distributed between and within beneficiary communities and households? 10 What targeting, awareness and planning activities precede RWSSP interventions? 11 What are the principal features of the policy that guides RWSSP interventions? 12 What institutional outputs has the RWSSP achieved at community level? What institutional outcomes has the RWSSP achieved at community level? Are the institutional assumptions embodied in the RWSSP TOC valid? Can changes in the incidence of diarrhoea in children aged under five in

32 No. Evaluation question Section Page beneficiary communities as a proxy for incidence of waterborne diseases be attributed to RWSSP interventions? 16 Can changes in the enrolment and attendance of girls from beneficiary communities at school a proxy for enhanced educational opportunity be attributed to RWSSP interventions? 17 Can changes in the income generating activities practised by women from beneficiary communities a proxy for enhanced economic opportunity be attributed to RWSSP interventions? 18 Can changes in local (self) employment in community-based WASH service delivery be attributed to the RWSSP? 19 Are the results of RWSSP interventions technically sustainable? Are the results of RWSSP interventions institutionally sustainable? What are the links between technical and institutional sustainability, and are they adequately reflected in the RWSSP TOC? 22 Is the RWSSP TOC replicable at the intended scale? Policy Tanzania s Development Vision 2025 includes universal access to safe water as a goal that will help achieve a high quality livelihood for all Tanzanians (GOT, nd 5 : 13). Cluster II of the strategic interventions proposed by the National Strategy for Growth and Reduction of Poverty II (MKUKUTA II) concerns improvement of quality of life and social well-being. Goal 4 within this cluster is increasing access to affordable clean and safe water; sanitation and hygiene. Its targets include proportion of households in rural settlements provided with improved sources of water increased from 58.7 percent in 2009 to 65 percent by 2015 and proportion of population with access to improved sanitation facilities increased. No specific targets are stated for sanitation (GOT, 2010b: 76-78). Tanzania published its National Water Policy in With regard to rural water supply, it states its overall objectives is to improve health and alleviate poverty of the rural population through improved access to adequate and safe water. Among the stated specific objectives for this sector are the following goals: (i) to emphasise on communities paying for part of the capital costs, full cost recovery for operation and maintenance of services as opposed to the previous concept of cost sharing ; (ii) to depart from the traditional supply-driven to demand-responsive approach in service provision ; and (iii) to promote participation of the private sector in the provision of goods and services (GOT, 2002: 30). These represent three key points for this evaluation. The policy expects users to pay for operation and maintenance of their water supply systems in full and to be proactive in proposing and developing new schemes. One of the evaluation questions for this study is whether the policy s intention of stronger private sector participation is being realized. The rural water policy is based on a number of principles. These include water is a basic need and right ; use of water for human consumption shall receive first priority ; water is an economic good ; and improvement of health through sanitation and hygiene education. The policy emphasises subsidiarity as one of the principles for achieving sustainability: adopting the principle of managing water schemes at the lowest appropriate level with the beneficiaries themselves establishing, owning and managing their water schemes. Other principles for sustainability include ensuring full cost-recovery for operation and maintenance, and replacement ; and recognising women as being among the principal actors in the provision of rural water supply services (GOT, 2002: 31). A key strategy of the policy, on the basis of these objectives and principles, is therefore the legal registration of community structures that will own and operate rural water systems. The policy hinges on two areas of strong 5 nd: no date. 16

33 participation: firstly, by users and the communities of which they are part (the concept of community is itself loaded with assumptions); and, secondly, by the private sector, which has traditionally been weak in rural Tanzania. Another key element of the 2002 policy is its emphasis on integrating water supply and sanitation services and hygiene education (GOT, 2002: 35). However, this policy did not formally cover sanitation. A draft National Sanitation and Hygiene Policy has reportedly existed for some years, under the auspices of the MOHSW. Despite various calls for it to be finalised and implemented (for example, Thomas et al. 2013: np 6 ), this policy remains a draft only. Consequentially, despite the water policy s clear call for integrating water and sanitation efforts, there is a formal policy vacuum with regard to sanitation policy Activities A number of targeting, awareness and planning activities are meant to precede interventions under Component 2 of the WSDP, which has been undertaken throughout mainland Tanzania. The programme as a whole has been guided by a programme implementation manual (PIM). The first edition of this manual, as noted in section Erreur! Source du renvoi introuvable. was widely criticised as too complex. A second, simpler edition has recently come into use (GOT, 2015b). However, implementation leading to the interventions whose impact is evaluated here should mainly have been guided by the first edition of the manual, to which this evaluation therefore mainly refers. The concept of targeting is problematic in a community-driven approach, where users should theoretically target themselves and bring their proposals for water and sanitation schemes forward to the authorities. The first edition PIM identifies a number of early steps that need to be taken at LGA level, including orientation for staff and then promotion of demand at the community level (GOT, 2007: 27). It was obviously impossible for CWSTs to undertake this promotion and advocacy throughout their districts. An element of top-down targeting was therefore inevitable, as CWSTs identified the communities with the greatest need for improved water supplies and undertook their stimulation of demand there. At the same time, CWSTs selection of communities on which to focus was influenced by messages and appeals for support reaching them from local leaders and representatives. This promotion of demand by CWSTs (on behalf of their Councils) was meant to lead to community applications for water and sanitation schemes. Further steps for the CWST were therefore meant to be the appraisal of community project applications; the preparation of District Water and Sanitation Plans; and the annual budgeting procedures that were meant to lead to the disbursement of funds from the WSDP for scheme implementation. However, the 2013 evaluation of Phase I of the WSDP found that few (if any) LGAS have developed district-wide water and sanitation plans (GOT, 2013a: 29). Instead, after the period of quick win projects (often rehabilitation of old schemes) that characterised early implementation of Component 2 of the WSDP, CWSTs came to focus on a few specific community schemes sometimes large undertakings that developed a central water facility for several communities with lengthy reticulation to DPs scattered across comparatively large areas. This more limited focus was then reshaped as the ten villages initiative that has driven implementation of Component 2 since 2013 (section Erreur! Source du renvoi introuvable.). 6 np: no page number 17

34 As noted, the first edition of the PIM presents a lengthy exposition of the preliminary and planning steps that should be taken, notably at community level. Those steps include self-mobilisation and the election of what the manual called a Watsan Committee but was later converted to the concept of a COWSO that would be formally registered as a legal entity. Another important early task for the emerging community structure is collection of the user contribution for construction of the scheme. The PIM set this contribution as 2.5 percent of total cost for more complex schemes, including the common pumped and piped ones as well as gravity schemes; and five percent for hand pump installations. The PIM then required communities to develop facilities and management plans, outlining arrangements for design and construction and for O&M. The ultimate result of these processes, following CWST approval and MOW approval of the CWST s proposed budget, would be signature of a Community Project Agreement for the scheme. Figure 4. Water may be transported several kilometres to communities without a supply In line with the national policy effort on private sector engagement, consultants played a major role in the preparatory and planning processes (but not the selection of communities to be served, which was done by LGAs on the basis of need and community requests (GOT, 2013a: 29)). Consulting firms and consortia (the latter often combining international and Tanzanian companies) were contracted by LGAs to provide both the soft and the hard services needed for implementation of WSDP Component 2. Consultants would facilitate the emergence and appointment of local committees (later COWSOs), and train their members. At the same time, they would design the water scheme, prepare the tender documentation and support the CWST in the tender process. Following appointment of contractors, they would go on to supervise construction on behalf of the client. As part of their social services, consultants typically undertook sanitation and hygiene awareness programmes in communities where water schemes were to be constructed. However, these were not formally linked to the NSC, which as outlined in section Erreur! Source du renvoi introuvable. was split off from the WSDP. Formal NSC initiatives linked to the construction of improved latrines were not usually coordinated with support to community water projects by the WSDP Outputs Different sources of data on the outputs achieved by Phase I of Component 2 of the WSDP are not fully consistent and the information has not been fully verified. 18

35 The 2014 Water Sector Status report produced by the MOW at a stage when it was still thought that Phase I of the WSDP was about to end stated that by 30 June 2014, 32,846 water points had been completed, serving 8,211,500 people. According to this report, the impetus of the Big Results Now initiative in the financial year led to the installation of 16,784 water points in those 12 months (GOT, 2014c: 14). While BRN did accelerate performance in various government sectors, the increase in WSDP implementation rates was also due to the eventual achievement of broadly common understanding of systems and procedures that could feasibly and efficiently be implemented following years of slow development of those systems and the required individual and institutional capacities (section below). However, the 2014 report went on to warn that 82 projects are still not initiated because of lack of financing. This makes reaching the planned target of serving all 1,555 villages by December 2015 difficult (GOT, 2014c: 18). According to the draft Aide Memoire of the 15 th Joint Supervision Mission for the WSDP (October 2015), LGAs reported a total of 43,255 water points developed during Phase I, that is, since 2007, compared with a target of 31,747. The Aide Memoire pointed out that these were the figures reported by LGAs, which it had not been possible to verify. On the assumption that each water point served 250 people, it was calculated that 10,813,750 rural people had been served with improved water, compared with the target of 7,990,000 (GOT, 2015c: 14). These reported achievements included the early quick win projects, which mostly rehabilitated old water schemes rather than building completely new ones. LGAs are supervised by the Prime Minister s Office Regional and Local Government (PMO-RALG) 7, which produced an annual report on rural water supply and sanitation projects under the WSDP for the July 2014 June 2015 financial year. According to this report, 1,045 projects had been completed in 1,384 villages by the end of that financial year (so some projects serve more than one village) with 237 projects awaiting completion with respect to the revised MOW-imposed deadline for the end of WSDP Phase I, 31 December 2015 (GOT, 2015d: 3). Whatever the data source, it is clear that, over recent years, Component 2 of the WSDP made significant progress with its core output: the installation of rural water supply systems. Whether this output achieves the intended outcomes depends on a number of factors, as the discussion of assumptions in the theory of change makes clear in section 1.3 au-dessus. These include, of course, whether the water supply system actually works and whether the water it delivers is safe at the point of collection and at the point of use. Table 13 in Volume II shows that, in programme communities surveyed for this evaluation, 82 percent of the primary water sources were reported to be functioning, compared with 54 percent in non-programme communities. According to Table 14 in Volume II, defective design was by far the most commonly mentioned reason for water sources not functioning (accounting for 58 percent of the reason given), followed by over use of the facility (13 percent) and improper use (11 percent). Leaving aside the precise number of projects or water points achieved, it is important to consider the nature of the water schemes built by WSDP Component 2 during Phase I. Official reporting says little about this issue. The 2013 evaluation of WSDP Phase I noted what this evaluation observed during field visits: MOW had anticipated that a substantial proportion of the designs would be for relatively low-cost schemes such as boreholes with hand pumps, but in the event most were for higher cost technologies, particularly gravity flow schemes and boreholes with motorised pumps and local distribution networks. The total cost of these schemes exceeded many times over the amount of money that MOW had budgeted and led the ministry to revise their plans, so that the initial number of schemes per LGA was reduced to an average of three, though some LGAs were allowed more. GOT, 2013a: Following the October 2015 elections, this function was transferred to the President s Office. 19

36 The Programme s heavy dependence on consultants has been explained above. It is not surprising that these consultants tended to design more technically sophisticated schemes. They knew their theoretical operating advantages; they tended to overlook the local management challenges they posed; and the more complex the scheme, the higher the design and supervision revenues that would accrue to them. Very often, the technology introduced by the programme has been boreholes equipped with diesel generators running electric pumps. The water is pumped into an elevated storage tank from which water is reticulated to DPs, sometimes in more than one village. Influenced by Big Results Now, the aim of WSDP Component 2 in its final years was to install new water schemes in ten villages per LGA. Partly because of the size and complexity of schemes, few LGAs achieved this target. In an initial sample of 30 LGAs reviewed for this evaluation, six had completed ten or more schemes. Drawing largely on data from the 28 LGAs that returned their questionnaires (section 1.13 au-dessus), this evaluation yielded the following scheme output data. LGAs had on average started 9.6 new projects since January 2012 with a minimum of five and a maximum of 20. This does not mean that they completed the same number of projects. Persistent funding shortfalls and delays in transfer of funds from central government to LGAs have slowed and interrupted construction of many schemes. District Water Engineers have often had to coax reluctant contractors to continue despite incomplete payment. Some have co-operated; some have slowed or abandoned construction; and some have sued LGAs. CWSTs have been instructed to complete all Phase I projects by 31 December On completion of this evaluation s fieldwork in early November, CWSTs had received no funds since the start of the financial year on 1 July. Compliance with this instruction was therefore impossible. Figure 5. A simple hand pump installed on a borehole CWSTs were asked to give details of the first four projects they had started since A total number of 104 projects were reported by 28 LGAs. (Not all completed this section of the questionnaire.) The average number of DPs per project was 15 with a minimum of one and a maximum of 79. The average total scheme cost was TZS 20

37 370m (USD 167,972), with an average community contribution of 1.89 percent (not all CWSTs provided this information for all schemes). Delivery of sanitation and hygiene outputs under the WSDP has been dissipated by the transfer of overall responsibility to the NSC, which was poorly coordinated with the installation of new rural water supplies. In communities that were to be served with the latter, the arrangement (according to an NSC informant) was that hygiene and sanitation advocacy should be carried out by the contractors designing the water schemes and facilitating COWSO formation. By June 2015, the sanitation component of the WSDP had reached 58percent of its target of 1.5m households with improved sanitation: 4,383,535 people out of the target of 7,600,000, assuming an average household size of 5.07 (GOT, 2015c: 14; GOT, 2015d: 2). While the same number were meant to have installed hand washing points, only 2,702,965 people were reported to have access to these (GOT, 2015d: 2). These data were as reported by LGAs, and the Joint Supervision Mission that presented them had not been able to verify them. To date, 16,183 sub villages (3,236 villages) have signed commitments to construct improved sanitation facilities at household level. In the same period, 5,784 villages are covered by a sanitation service provider such as a hardware shop or mason against the target of 600 villages by On Sanitation and Hygiene in public institutions (schools, health facilities) and public areas - the progress indicates that 1,189 against 812 schools targeted have constructed/rehabilitated their toilets. Of these 715 have been achieved through NSC funds and 474 through other sources such as (the) council s own source, community initiatives and development partners. In addition, 1,443 schools have active Sanitation and Hygiene clubs to ensure all the latrine facilities are maintained clean. GOT, 2015d: 2. This evaluation s field survey covered a total of 128 primary schools, of which 67 were at or near the sampled communities where WSDP Component 2 had been implemented. With respect to the latter group of 67 schools, 76 percent had a drinking water facility, and 51 percent (34 schools) had a functioning drinking water facility. Table 3 below shows how all the schools reported obtained drinking water for their daily use. Asked about sanitation, 66 of the 67 schools said that they had toilet facilities for the pupils and 65 reported that these facilities were in use. Separate facilities for boys and girls were provided at 96 percent of these schools with an average of 66 boys and 66 girls per toilet. A water supply at the toilet for hand washing was reported by 48 percent of the 67 WSDP Component 2 schools, and 19 percent of those schools said that they supplied soap or a soap substitute for this purpose. Table 3. Primary schools' sources of drinking water Source of drinking water Frequency Percent Own improved source Own non-improved source Buying from tankers Buying from vendors Hand dug wells Buying from public pump Fetching at the river Buying from the hospital At home Shallow well Deep well From a near water point Hotel tank Everyone carry his own water School tans Pond Rainwater RWSSP water source

38 Source of drinking water Frequency Percent Water from public tape Dams No use of water services at all Protected spring Unprotected spring Not recorded Total Source: impact evaluation field survey. As shown above, the key institutional output of WSDP Component 2 was meant to be a local user institution latterly known as a COWSO that would own, manage and fund the O&M of the rural water supply system. Facilitated and trained in their early stages by scheme design consultants, these bodies have developed slowly through the many formal stages meant to lead to legal registration as corporate bodies. According to the 2014 Water Sector Status Report: Out of 2,728 planned BRN COWSO establishments by June 2014, only 460 were registered (17 percent). This slowness is partly due to the poorly coordinated flow of the necessary funds to the LGAs concerned, which was allocated to them for undertaking COWSO registration also beyond the WSDP project villages. Even where funds were sent, these arrived at the LGAs without clear instructions about their purpose and intended use, which resulted in these funds being used, instead of for COWSOs registration, for other LGA activities such as monitoring, supervision, and capacity building activities. Figure 6. Community management structures at 114 water points in 'treatment' villages 60 GOT, 2014c: Percent COSWO Water user group community water comittee None Source: impact evaluation field survey. The field survey for this evaluation covered a total of 114 water points in villages treated by the programme. Figure 6 above shows the frequency with which different types of community water and sanitation management structures were reported to exist at these water points. 22

39 Capacity development According to the most recent Joint Supervision Mission of the WSDP, Some of the challenges identified by the sector include the lack of effective management of O&M of the projects due to weak performance on setting up of COWSOs and wherever set up, building their capacity The involvement of the community and its representative organisation, the COWSO, should be ramped up, to start right from the beginning, and not as an after-thought. An initial period of community mobilisation, including setting up of the COWSO, will have to be mandatory; the community and the COWSO should be involved in the selection of the technical option, based on affordability (especially of the O&M) and its ability to manage the scheme, followed by its implementation and O&M. The 2013 evaluation of the WSDP Phase I remarked that GOT, 2015c: 15. The amount of external technical assistance available within the programme is surprisingly light, particularly for Component 2. If the right balance is to be struck between investment and capacity development, then implementing agencies should have access not only to occasional training courses but also to ongoing technical support and guidance. GOT, 2013a: ii. Figure 7. Members of a new COWSO, assembled for a training session The WSDP Phase I training manual set out proposed capacity development arrangements for LGAs and communities in some detail, identifying user groups, caretakers and operators as training targets at community level and referring to the availability of a number of training materials. It indicated that the MOW would hire training consultants who would train CWSTs and the consultants who would then undertake institutional and capacity development at community level (GOT, 2007: 65-69). Aggregate data on capacity development activities actually implemented at these levels are not available. There is no doubt that LGAs have received some training, and that consultants responsible for rural water schemes have facilitated the appointment of user groups or COWSOs, which they have then trained. Table 4 below shows 23

40 what training the 28 CWSTs told the evaluation survey they had received. It is remarkable that 11 of them said they had received no training at all. This may be an exaggeration, caused by short institutional memories linked to staff turnover and/or enumerator failure to probe. Just over one third of the 28 reported receiving one or more training events on the general management of the WSDP, and associated community level capacity development. Almost as many CWSTs received training on the National Sanitation Campaign and on sanitation and hygiene issues, with specific programmes on Community-Led Total Sanitation (CLTS) also taking place and just three CWSTs reporting training focused on school WASH approaches. Table 4. Training received by 26 CWSTs, as reported to evaluation survey Subject area (there may have been multiple training events on the same subject) No. of CWSTs mentioning one or more training events on this subject No training received 11 General rural water supply program management, community capacity development 10 National Sanitation Campaign, health, hygiene 8 Community-Led Total Sanitation 8 School WASH 3 Management information system 3 Water point mapping 2 Environmental and social safeguards 1 Report writing 1 Source: evaluation field survey. At community level, some operators received training on the generators and pumps for which they were to be responsible. Despite the detailed intentions of the PIM, the amount of training that COWSOs and previous user groups received was variable, which is one reason why the levels of responsibility and confidence displayed by these groups also differ widely. Some of the longer-established groups have now been through one or more cycles of elections, resulting in a substantial (sometimes complete) turnover of membership. There do not appear to be systems in place for tracking the arrival of new members and arranging their training. Among the 114 COWSOs and other user groups surveyed for this evaluation in communities served by the WSDP only three were reported to have received no training. The training subjects that were delivered are shown in Table 5 below. During the institutional assessment mission in October 2015, one COWSO reported that they had received no training at all. They said that they had written to the District Water Engineer in 2013 requesting it but had received no reply. Another, however, said that they had received three phases of training (from consultants) before the COWSO was formally inaugurated. They said that more training was needed to keep them updated. A third was new, having replaced a previous group that had been trained. The new body had received no training and did not know, for example, how to read the meters installed at DPs. Another community now had its third user group (a COWSO), which had had some training, including on sanitation and hygiene unlike the first user group, installed in 2009, which received no training on any subject. Another COWSO, only launched in the last 12 months, reported facilitation by the CWST s COWSO co-ordinator. The same sort of support was reported by another COWSO, although in this case the consultants also provided training and some members were taken on a sanitation training visit to another region of Tanzania. These random examples illustrate the variability of water and sanitation capacity development at community level. 24

41 1.21 Outcomes Equity Table 5. Subjects on which COWSOs and other user groups reported receiving training Training subject No. of COWSOs and other groups reporting training on this subject General management 31 Technical and maintenance training 27 Financial management 20 Committee procedures 16 None 3 Source: evaluation field survey: treatment villages. The National Water Policy makes general commitments to equity, stating for example that it is important that all members of the community including the disadvantaged groups efficiently and equitably use the water Communities will ensure the protection and conservation of water sources as well as equitable service provision to economically disadvantaged groups within the communities (GOT, 2002: 36). One of the principles for sustainability in rural water supply that the policy espouses is recognizing women as being among the principal actors in the provision of rural water supply services (GOT, 2002: 32). National policy does not give more specific guidance on these equity issues. Nor does the first edition of the PIM, which is, however, gender sensitive and states, for example, that women and men will have equal access and full participation in capacity building and training activities (GOT, 2007: 60). In practice, many (but not all) COWSOs and other user groups allow severely disadvantaged residents such as the elderly and disabled to obtain domestic water free of charge and the idea of women s participation management structures is increasingly but not universally - accepted. Members of one COWSO interviewed by the mission said that it was difficult to get women to take part because of their husbands attitudes. This evaluation asked what percentage of the general membership of the local water management structures in treated communities were women, and what percentage of the leadership positions in these structures were held by women? In 84 percent of user bodies for which the information was reported, women made up between 21 percent and 50 percent of the membership of these management structures. In 77 percent of management structures, they held between 21 percent and 50 percent of the leadership positions. The evaluation team were informed that MOW guidelines require women to make up at least 60percent of COWSO membership but no document stating this has been traced. The same questionnaire asked whether all water users were required to pay for their water, and if not, why not. The question was answered for 79 water points, at 53 percent of which all users were said to be required to pay. At water points where some users did not have to pay, the commonest category of exemption was old age (at 38 water points), followed by disability (31) and poverty (12). Although comprehensive data are not available, there is general consensus (based on interviews with central and local government and NGO officials, and with community informants) that the benefits of changes in water, sanitation and hygiene use and practice achieved by Component 2 of the WSDP are uniformly distributed between and within beneficiary households. Indeed, in a sense there is some bias towards benefits for women and girls, because they are somewhat more likely than men and boys to enjoy time savings because of the improved services although much of the time saved may be used for (less risky and physically arduous) domestic chores like cooking (sections , ). Little has been done about one aspect of equity, however, 25

42 which is provision of facilities for menstrual hygiene at school sanitation facilities. Again, no comprehensive monitoring data are available but the anecdotal evidence is that few school latrines offer these facilities. A recent study identified this as a knowledge gap: how to assist retention of girls at school through the provision of facilities for menstrual hygiene (Thomas et al., 2013: 60). The 2013 evaluation of the WSDP stated that: In Component 2, one positive feature is the use of a transparent formula for the Local Government Capital Development Grant water window, ensuring that every LGA receives some level of funding; without it, funds might be reserved for a few favoured locations. Set against this is the evident bias towards high cost schemes that has arisen in the ten village initiative, which limits the number of people who can benefit from WSDP support. On the selection of villages, it appears that pragmatic judgments are made about demand, need and opportunity in the selection of WSDP schemes, but there is no rational basis for equity considerations. GOT, 2013a: vi. This review in 2015 reached similar conclusions. There are anecdotal complaints and queries from District Water Engineers about alleged imbalances between districts in Component 2 funding allocations but this has been a genuinely nationwide initiative. Within districts, there are inevitably communities where natural conditions make assurance of a reliable water supply difficult. There is no early prospect of achieving full geographical equity Institutional outcomes The 2013 evaluation of the WSDP noted the inadequacies of the (repeatedly revised) results framework for the programme, and said that even in Component 2/RWSSP it is very difficult to find comprehensive and consistently-reported data on outputs and outcomes (GOT, 2013a: 20). There has been no marked improvement in this situation since then. In outlining institutional outcomes, in particular, it is necessary to give a qualitative overview based on observations during the fieldwork for this evaluation. One frequently offered argument about the early years of the WSDP is that, although they achieved very little physical progress in terms of enhanced coverage of improved rural water and sanitation, they made a major institutional contribution by developing systems, procedures and capacity for the implementation of the programme at scale. This evaluation endorses that argument. In rural water supply, the MOW, PMO-RALG and LGAs know what to do and how to do it. Although staff numbers remain inadequate and operating resources are severely lacking, CWSTs have most of the knowledge and skill that they need and include many able and committed individuals. They are partly able to provide effective technical and institutional support to users, as the institutional outcome is stated in the theory of change inferred for Component 2 of the WSDP (Figure 2 on page 5 above). But their effectiveness is compromised by their severe lack of operating resources and recurrent budgets. Furthermore, one vital, related institutional outcome has not been achieved. Although much of the WSDP system works, it is massively hindered by inefficient management of funding. A study commissioned by DFID in 2013 concluded that there exist significant shortfalls in the predictability, timeliness and completeness of intergovernmental transfers (these) are increasing and are probably the single greatest factor impeding improved LGA performance and service delivery (Tanscott Associates, 2013: 83). The study found that: The current practice of allocation and transfer of funds has resulted in unpredictability of both timing of receipt and amount both of which do not help LGAs in discharging their functions meaningfully. 26

43 The process of allocating available OC [Other Charges] resources, that is rationing, to LGAs is not transparent and responsibilities for deciding who receives how much and based on criteria, which are not defined. Whereas there is an accepted (and fairly reliable) basis for PEs [Personnel Emoluments] allocation, there is no comparable practice for OC. This means that under conditions of resource rationing revenues cannot meet budget requirements there is no guidance to MOF on what is critical and should therefore be funded in full and what can be scaled down without serious negative impact on LGAs operations. Tanscott Associates, 2013: vii. It is perhaps understandable that LGAs annual budget requests are not met in full by central government (although in some years, under the influence of BRN, districts received more than they had asked for). This evaluation asked the 30 sample CWSTs how much money they had applied for in each of the last four financial years ( ); how much they received; and how much they spent in each of those years? Not all the CWSTs provided this information, and some of those that did only gave it for some of the four financial years. Overall, however, the average amount received as a percentage of the amount requested was 64 percent in ; 81 percent in ; 123 percent in ; and 44 percent in These averages mask major variations with some CWSTs reportedly receiving more than twice as much as they asked for. There was a general wave of additional funding, associated with BRN, in The data that CWSTs provided on the proportion of funds received actually spent are less helpful because the variation is complicated by carry-overs, in some cases from one financial year to the next. Some LGAs apparently spent only 12 percent of what they received in a particular year, while others spent two or three times as much as they had received. Planning and management of water and sanitation at LGA level are also complicated by the separation of arrangements for the two subsectors. One respondent CWST wrote that Budget cycle for water supply and that of sanitation doesn t match. The NSC budget is prepared by a District Health Officer together with the MOHSW in December. In February NSC budget is incorporated in the Health budget, which is combined with budget from other Departments. Whereas the Water Supply budget is prepared in January by District Water Engineer; in February it is incorporated in the District budget. In April, the District Executive Director presents the District budget containing Health and Water Supply budgets to the Ministry of Finance. Only funds are integrated, plans are always separate! LGA Wash Team questionnaire survey response. In general, funds are usually received only several months after the start of the financial year. Financial flows from Dar es Salaam are irregular, uneven and unpredictable. While donor disbursements into the Component 2 basket are also irregular, there are often gaps of several months between those transfers and the arrival of any new money at district level. Particularly during BRN, LGAs were encouraged to commit to more projects than they have ultimately been able to pay for, causing the severe embarrassment and sometimes lawsuits referred to in section 1.20 above. The irregularities are compounded by late and uneven disbursements of government s own contributions to the basket, exacerbated over the last two years by general fiscal difficulties. Until this dysfunctional situation is comprehensively addressed, the overall institutional outcome of the WSDP Component 2 cannot be satisfactory. In the theory of change (Figure 2 above), the core institutional outcome is COWSOs or other community RWSS [rural water supply and sanitation] institutions have resources, skills and commitment to operate and maintain RWSS facilities effectively. This outcome has been only partially achieved. Some members of some COWSOs (and other user bodies) do have the required skills and commitment. But, as outlined above, training has been uneven and incomplete, and there is no comprehensive system in place for the ongoing (re-)training of existing or new cadres of COWSO members. So far, this element of the programme has been treated as a once-off exercise, whereas in fact it will be needed on a recurrent basis. 27

44 The biggest gap in this institutional outcome concerns the resources to operate and maintain RWSS facilities effectively. Few COWSOs have these resources. The amounts of money that COWSOs hold in their bank accounts vary. Some actually have nothing, and live hand to mouth, collecting funds from users as and when needed to buy fuel for their generators or to pay for minor repairs. According to informants, others have over TZS 10m (USD 4,550) in the bank. For 57 COWSOs and other management bodies for which data are available from treatment communities, the average amount held at the time of the survey was TZS 3.6m (USD 1,636), with a range from TZS 20,000 (USD 9) to TZS 6.2m (USD 2,818). While there is general acceptance among rural Tanzanians of the principle of users paying for domestic water, these users very commonly turn to unimproved sources during the rainy season when streams, dams and depressions fill up with water. As a result, some COWSOs collect little money for two or three months a year. Some of those interviewed report collecting nothing at all during the rainy season. Average monthly revenues reported for 72 management bodies in treatment villages were TZS 502,943 (USD 229), with a range from TZS 5,000 (USD 2) to TZS 4.1m (USD 1,863). Even if they were collecting user fees all year round, however, many COWSOs would still be challenged by the heavy cost of buying diesel. Those with pumps connected to mains electricity enjoy lower costs and the minority using gravitational schemes are of course in the best position of all. Furthermore, the consensus across CWSTs interviewed during this evaluation is that there is no prospect of COWSOs being able to pay for anything more than minor maintenance. Typically, COWSOs report problems to the CWST and the CWST sends a technician to help with the repairs sometimes using spares that the COWSO has been able to buy. Sometimes, also (according to CWST informants), the COWSO pays the field allowance of the visiting technician. Occasionally, they report, the CWST is able to provide the spares or other materials that are needed. If a major repair is required, the CWST includes the cost in its next annual budget which means that the COWSO must wait many months before there is a prospect of the repair being done. Consequentially, the institutional outcome at community level is that user bodies have some level of awareness and capacity. Yet, judged overall there is little prospect of the financial autonomy needed to carry O&M costs in the way that national policy expects. This remains the case even though most users are willing to pay for water during months when they cannot find it from unimproved sources. A further important institutional outcome should be that water, sanitation and hygiene initiatives are coordinated at national, LGA and community levels. This outcome has not been achieved. As reported above, significant numbers of CWSTs and communities report exposure to training on sanitation and hygiene. But fully co-ordinated introduction of enhanced water supplies and sanitation and hygiene campaigns has been the exception rather than the rule mirroring the poor co-ordination of the NSC with the rest of Component 2 of the WSDP at national level Institutional assumptions Evaluation question 14 for this study asks whether the institutional assumptions embodied in the RWSSP theory of change are valid. The assessment here refers back to the assumptions inferred in the discussion of the TOC in section 1.3 au-dessus. Systems, procedures and budgets are in place, co-ordinated and efficiently implemented so that the targeting, planning and related preparatory processes lead to the provision of inputs at scale in a manner that matches community awareness and expressed need to available resources. One part of this assumption is valid so far. Systems and procedures are in place. They are understood and operated with adequate understanding, coordination and efficiency. Budgets, however, are not fully in place and are not efficiently coordinated. As explained above, recurrent funding for CWSTs is wholly inadequate. WSDP funding flows to LGA level are so slow, erratic and inadequate that the overall effectiveness of the programme is gravely compromised. The policy being implemented by the RWSSP is conducive to efficient achievement of the intended 28

45 outputs and outcomes. Broadly speaking, Tanzanian authorities and citizens have understood and embraced the National Water Policy as it affects rural water and sanitation. But field research reveals an undercurrent of resentment about the perception that rural people are expected to pay proportionately more for water supplies, receiving in theory no subsidy for their O&M compared with their urban compatriots. Besides which, the current policy is not conducive to efficient achievement of the intended outputs and outcomes for two reasons. First, poverty drives many users to turn to unimproved sources when they are available, rather than pay for safer supplies. Secondly, there is little prospect of COWSOs, funded by users, being able to fund O&M in full. Communities and households will commit to developing the required user institutions. Field evidence shows that rural households and communities have supported the development of COWSOs. The concept of community responsibility for and management of local services is well established in rural Tanzanian life, and COWSOs have clear links to the formal structures of village government. Awareness raising and institutional development carried out by service providers are effective in creating competent COWSOs or similar structures. The institutional facilitation and training work carried out by consultant service providers at community level have, as outlined above, provided the foundations for competent COWSOs. But the quality of this training was uneven, and competent COWSOs will depend on recurrent refresher training and orientation of new members for which there is no formal provision at present. User management bodies are able to establish working relations with O&M service providers. This assumption is valid in the sense that COWSOs are quite ready and willing to work with private sector service providers. It is invalid in the sense that they often lack the funds to do so.. The consequent lack of market opportunities restricts the development of private sector capacity. The CWST is competent and adequately resourced to coordinate planning and delivery of the intended outputs. As argued above, CWSTs are generally competent but they are not adequately resourced. COWSOs or other structures that stay in place and continue to operate and maintain water services effectively with appropriate support from LGAs. This implies an ongoing, constructive relationship of communication and support between LGAs and user institutions. As argued further in section 1.24 audessous, there is as yet no guarantee that COWSOS will stay in place. Much depends on the consistency and quality of support they receive over the next five years. Field observations suggest that there are good, constructive relations between COWSOs and LGAs, although the latter are typically unable to respond to the former s problems as efficiently as they would wish Impact Impact on health Evaluation question 15 for this study (section 1.6 au-dessus) asks whether changes in the incidence of diarrhoea in children aged under five in beneficiary communities as a proxy for incidence of waterborne diseases can be attributed to RWSSP interventions. The impact of RWSSP on reduction of the incidence of diarrhoea is notable for children under five as well as for all persons in beneficiary communities. For children under five years of age, the program achieved a significant reduction in diarrhoeal incidence, by three percent. The impact for all persons was ten percent (see Table 6). One of the key outcomes intended from implementation of the RWSSP in Tanzania was to improve health outcomes in communities by reducing the incidence of waterborne diseases including diarrhoea. In this evaluation a comparison was done between programme and non-programme communities, using a question that asked about the occurrence of diarrhoea cases during the two weeks prior to the survey. Comparing the two categories of community, the impact of the programme on diarrhoea among the under-fives is statistically significant at the ten percent level. For the population as a whole, the results are statistically significant at the one percent level. Overall, the results of the analysis as indicated in Table 6 reveal that improving access to and 29

46 quality of water through the programme has exerted a positive impact on community health. Nonetheless, this impact has been considerably less than the 20 30percent reduction in diarrhoea revealed by other impact studies of rural water and sanitation (IOB, 2012: 31). However, Table 12 in Volume II suggests that the average number of outpatient visitors aged five years and under was higher in health facilities within or near programme communities than those in non-programme communities. The respective percentages are 89 (programme) and 76 (non-programme). This observation is consistent with the finding that the impact of the programme on the incidence of diarrhoea is relatively small and less statistically significant on children under the age of five as compared to the population as a whole. Moreover, it could be the case that programme communities have better health infrastructure that attracts and serves outpatient visitors from non-programme communities as well as local residents. It would have been useful to track which community each of the diarrhoea outpatient visitors at the health facilities came from. However, given the format of record keeping in health facilities rosters, it was not possible to do so.. Table 6. Impact of programme on incidence of diarrhoea Variable Estimated impact Standard error All persons *** Children under age of five * *** Statistically significant at the 1percent level. * Statistically significant at the 10percent level. The programme s modest impact on the selected health indicators is reflected by the fact that the more vulnerable group children aged under five appear to have enjoyed a smaller reduction in diarrhoea incidence than the rest of the population. The anticipated benefits of a safe water supply have probably been compromised, as explained below, by the fact that not all the water people in treatment communities consume is actually safe as discussed in sections and Quality of drinking water As explained in section 1.3 au-dessus, one of the key assumptions in the theory of change for Component 2 of the WSDP (or any similar programme) is that the safe water supply it delivers really is safe both when it is collected and when it is consumed. Based on water quality management requirements, water is supposed to be clean at the point of collection, during transport and when it is stored in the household. Although CWSTs acknowledged in interviews with the evaluation team that they are responsible for periodic testing of water quality at all water schemes, they stated that they do not have the resources to do this on more than an occasional basis. Not only is it a major logistical challenge to visit all the sites, collect samples and deliver them to the regional laboratories of the MOW but funds must be available to pay those laboratories for each test they carry out. In many cases, the only water quality test that does occur is that required before a contractor hands a new scheme over for operation. Without a satisfactory result from this test, the contract should not be certified as complete. In communities where the programme had installed water supplies, this evaluation tested water at 285 points of use (households) to detect E. Coli bacteria (section 1.12 above). Similarly, water was tested for E. Coli at 67 points of collection (DPs) in these communities. It would have been interesting to compare water quality in treatment and control communities but with limited resources it was decided to focus on the theory of change assumption with regard to the communities served by the programme. The results in those communities reveal that water was safe for drinking in only 38 percent of households and at 58 percent of the tested water points. This suggests that unsafe storage of water within households remains a significant challenge, as does the safety of water collected from DPs installed or rehabilitated by the programme. 30

47 The World Health Organisation (WHO) recommends that in order for water to be safe for human drinking, it should be free of contamination, including E. Coli (WHO, 1997: 6). In order to understand the quality of water used by communities, this study conducted a sample test for 285 households and 67 water points in communities served by WSDP Component 2. The former represents the point of use and the latter represents the source of the water. Figure 8. Some schemes only supply water intermittently Table 7 shows the distribution of the E. Coli results by the level of contamination at both the point of use and point of collection. Out of the 285 sampled households, 62 percent used contaminated water as evidenced by exposure to E. Coli of greater than 0 per 100 ml. Similarly, about 41 percent of tested water points revealed presence of contamination in water with 4.5 percent of the water points reporting the E. Coli level of over 50 per 100 ml. Generally, these results suggest that much of the water is contaminated during transportation and/or during storage at the household. Table 7: Water quality at distribution points and at the point of use E. Coli concentration per ml No. of households Percent No. of DPs Percent Less than Greater than 10 but less than Over Total Source: Computations based on household survey and water points data, 2015 Households were also asked to evaluate the quality of the water they obtain from various primary sources. About 70 percent of households in communities served by WSDP Component 2 (RWSSP) believed that the quality of water had improved over time, presumably since programme intervention, although the water quality tests quoted above suggest that this improvement is only relative. Table 8: Household perceptions of changes in quality of water Non-programme communities Programme communities Quality of water Frequency Percent Frequency Percent Do not know