Terms of Reference: Evaluation of UnitingWorld Project, Rural Water Supply and Sanitation Project. Introduction UnitingWorld is the overseas aid agency of the National Assembly of the Uniting Church in Australia and works with church-based partners throughout Asia, Africa and the Pacific to implement community development programs. UNITINGWORLD seeks to engage a consultant to evaluate a water, sanitation and hygiene (WASH) project in Milne Bay Province in Papua New Guinea. The Rural Water Supply and Sanitation Project (RWSSP) has been active in Milne Bay Province since 1998, and is nearing completion of a 3 year funding cycle supported by Australian Non-government organisation Cooperation Program (ANCP) funding. Purpose of the Evaluation This evaluation will inform future planning as the project moves from a pilot phase to a phase of rolling out hygiene behaviour change activities in conjunction with effective water supply and sanitation systems design. The evaluation will: Ask the following questions: o To what extent has the project been effective in achieving its stated objectives? o To what extent has impact been reported in local communities? o To what extent has the project met its monitoring goals? o To what extent has the project met its capacity building goals? o To what extent has the project successfully adapted to a change in focus from water supply and hygiene awareness to a hygiene behaviour change campaign? o To what extent have the recommendations of the last evaluation been implemented? o What are the stories of success, challenges and lessons learnt from implementation? o To what extent has the project enabled inclusion of marginalized groups in decision making, appropriate systems design, and implementation? o To what extent has the project incorporated environmental sustainability and efficient design principles into systems design? Make recommendations to guide future planning for the project as it enters its next phase Background The United Church in Papua New Guinea (UCPNG) implements the RWSSP through the UCPNG Development Unit based in Port Moresby and locally, the RWSSP Office in Alotau, Milne Bay. Rural communities in Papua New Guinea, are frequently isolated and lack access to many basic services. Lack of access to reliable, clean water supplies is a significant issue, with women and children highly affected by both the burden of carrying water and water borne diseases. For close to 20 years, this project has operated using ANCP and other donor funds to provide clean water, sanitation facilities and community health and hygiene awareness training. The introduction in 2015 of a dedicated behaviour change component to the program has shifted the focus from 1
health awareness and water supply to a community-led campaign to increase handwashing with soap. The project works with community enablers who are leaders in their community. These community enablers assist in the facilitation of a 4 week campaign to increase awareness about and change behaviour around handwashing at key times. Direct beneficiaries of this project are the men and women and children in the three trial communities who have participated in the campaign activities. Women and children particularly benefit through reduced workload freeing them up for other domestic and educational activities. Other beneficiaries include people from adjacent villages that access health and education facilities that are improved through water supplies. The provision of safe and adequate water supply and sanitation facilities also contributes to achieving the health outcomes of the PNG-Australia Partnership for Development. This project is aligned with the Government of PNG s National WASH Policy. The policy aims to see a significant, sustainable and equitable increase in access to safe water and sanitation and improved hygiene practices, particularly in the poorly served rural and peri-urban areas. Improvements in Water, Sanitation and Hygiene will have considerable impact in elevating the health the lifestyles of individuals and communities to a higher quality of life; improve school attendance for children and save time to collect water. The expected outcome of this policy will usher and contribute significantly to: - Reduce morbidity and mortality caused by water-related diseases; - Improve livelihood opportunities and economic growth through improved health and reduce economic and financial losses; and - increase equity of services between rural, peri-urban and urban areas, and to disadvantaged groups. An evaluation conducted in 2014 indicated that a shift to a greater emphasis on behaviour change would provide improved value for money. It was decided to focus on three pilot communities to trial a toolkit of campaign resources. The International Water Centre was engaged to prepare and trial participatory resources. This process resulted in the development of a handwashing with soap campaign called Rait Mama, Rait Famili, Rait Community. The design of water supply and sanitation systems continues alongside the behaviour change campaign. Project Objectives Intermediate Outcomes People in target communities have increased awareness of risky sanitation and hygiene practices and experience motivation to change People in target communities practice improved sanitation and hygiene. Outputs 1. Key risky hygiene and sanitation behaviours are identified 2. Hygiene Behaviour Change resources and tools are adapted or developed for the PNG context 3. Target communities participate in hygiene behaviour change program 2
People in target communities have increased access to clean water People in target communities have access to improved sanitation Women and girls and people with disabilities (PWD) are safer and gendered workload is reduced Women and girls and PWD experience improved well-being and increased opportunities for participation AND 4. Feasibility studies are conducted by trained staff and participatory context analysis conducted 5. Contextually appropriate and inclusive, user sensitive water systems are constructed 6. Existing or new committee equipped for ongoing management and maintenance 7. End users sensitised/trained 8. Feasibility studies are conducted by trained staff and participatory context analysis conducted 9. Contextually appropriate and inclusive, user sensitive sanitation facilities are constructed 10. Existing or new committee/households equipped for ongoing management and maintenance 11. End users sensitised/trained 12. Participatory and inclusive approaches are undertaken to ensure women and girls and PWD participate and are included in consultations, project design, implementation and evaluation 13. Water and sanitation facilities are designed to be inclusive and user sensitive including for PWD Enabling Outcome: Women, girls and PWD are included in the design, implementation and evaluation of the program Enabling Outcomes UCPNG s capacity to deliver WASH is strengthened Increased opportunities for regional cooperation within the WASH sector Outputs 14. Core RWSSP staff are trained in behaviour change, context analysis and use of resources and tools (ToT model) 15. Connections with CPP and other UCPNG Health and Education staff are established 16. RWSSP and other UCPNG staff network with other actors (local government, other NGOs, private sector) interested in or working in the WASH sector in Milne Bay Province 17. Opportunities for regional cooperation are explored Engagement with agencies such as the IWC leads to understanding of the behaviour change model and the development of tools and resources appropriate to PNG context 18. IWC engaged to facilitate Behaviour Change training and MoU signed between IWC and UnitingWorld/UCPNG 3
Methodology and scope The evaluation will be undertaken using a participatory, strengths based approach. The Consultant will lead and collaborate with an evaluation team of 3 other people, who are based in Alotau, PNG, and who are responsible for implementing the project. Tools suggested by the team in Milne Bay include transect walk, pocket chart, timeline, success stories, aid post data, focus group discussion (men and women separately). The Consultant will: Conduct a desk review of relevant literature including: o RWSSP Design document 2015 2017, including project objectives and Theory of Change o Rait Mama Toolkit, including posters o 2014 Project Evaluation o Project Monitoring and Evaluation Framework o 2015, 2016 and 2017 Annual Reports to ANCP o 2018 Monitoring report from DFAT at post, Port Moresby o Monitoring reports as requested o Lent event communications material 2017 Facilitate a field-based evaluation of WASH projects in 3 communities in Milne Bay Province, namely Lelehudi, Kwairwa, and Paneati involving: o 0.5 day in Alotau meeting with relevant stakeholders in Govt and civil society as well as briefing by UCPNG staff o 0.5 days in Alotau agreeing / trialing participatory evaluation tools with evaluation team including 3 RWSSP project staff o Visit 3 communities to collect evaluation data on research questions in Introduction: Meet with community leaders, community enablers, community health workers, community members particularly those often excluded such as women, people with disabilities, and those not in leadership positions Use the same evaluation tools consistently across the 3 communities 1 ; o Presentation to UCPNG including draft final report with a view to incorporating feedback from the evaluation team Use data collected in the evaluation to provide recommendations and facilitate discussion with the implementing agency on the planning, design and monitoring and evaluation for the next phase of project. Provide an evaluation report including at least the following sections: background, methodology, results (answering evaluation questions) and recommendations. Expected outputs: An evaluation report including: o Background / Introduction o Section on methodology and participatory tools used o Results Section addressing the evaluation objectives, possibly including stories of change, case studies and quantitative data relating to the above objectives o Recommendations for future project planning, 1 See 2014 Evaluation Report for participatory tools used previously 4
Facilitated discussion with the implementing agency and UnitingWorld to report findings with consideration given to informing future planning. Selection Criteria: Demonstrated program evaluation experience using a variety of participatory methodologies; particularly with programs working with/for marginalised communities including children and women in assessing impact Demonstrated experience in the community health, water, sanitation and hygiene (WASH) sectors or related fields. Able to undertake a strengths based approach, working collaboratively with a team of implementers Considerable knowledge and experience of working in Papua New Guinea and / or the Pacific Willing to sign and comply with UnitingWorld Code of Conduct and undergo criminal records and working with children checks Photography skills preferred but not essential Timing: Dates for field work will be determined by agreement between UnitingWorld, our partners in Papua New Guinea, and the consultant, with the work including final report being completed by 30 th June 2018. It is expected that the evaluation will involve: Preparation Desk review Field work Facilitated discussion with the implementing agency Production of final report 5