Terms of Reference for Final Evaluation of Oxfam s Syria Crisis Response programme in Syria

Transcription

1 Terms of Reference for Final Evaluation of Oxfam s Syria Crisis Response programme in Syria Programme title: Oxfam s Syria Crisis Response in Syria Sector of intervention: Water, Sanitation and Hygiene (WASH) Locations: Syria Damascus, Rural Damascus, Aleppo, Idleb, Dar a, Homs, Hama, Tartous- Quneitera Project duration: August 2013 September 2015 Duration: Estimated working days Start date of evaluation: Around 1 October INTRODUCTION: As of April 2015, UN OCHA estimated that 12.2 million people inside Syria are in need of humanitarian assistance, of which 4.8 million are in hard-to-reach areas. Approximately 7.6 million people have been displaced inside Syria. Severe damage has been done to the water supply and sewage network during over three years of conflict. Service provision is hampered by disruption to the equipment and power supply, lack of spare parts, and access restrictions among many others limitations. The substantial breakdown of infrastructure contributes to reduced quantities of drinking water available, and population movement has also contributed to acute shortages of water in areas with high IDP populations. An estimated 11.6 million people are in urgent need of access to clean water and sanitation (Humanitarian Needs Overview November 2014). 1 At the time of the design of the Oxfam WASH response in 2013, the per capita availability of water supply within Syria was reported to have decreased to one third of pre-crisis levels, as a result of power cuts, fuel shortages, infrastructure damage, and lack of maintenance of water supply systems. Reduced access to piped water and the high cost for trucked water, have resulted to vulnerable families resorting to unprotected water sources. The population in collective shelters majority of them women and children are exposed to unsanitary conditions due to inadequate WASH facilities, unsafe drinking water, lack of water for personal hygiene and disruption of solid waste disposal and collection. 2 Since mid-july 2013, Oxfam has been responding to the urgent need for safe water inside Syria, working with the Ministry of Water Resources (MoWR) and with local Water Establishments. Key priority geographical areas were identified based on the number of displaced people, the operating environment and assessment of the MoWR and other agency operations. Oxfam s objective is to enhance access to adequate safe water supply, sanitation and reduced risk of communicable disease for conflict-affected women, men, girls and boys. Areas of intervention include the following: - Rehabilitation and restoration of water supply and water distribution systems - Provision of emergency water supply through trucking services, materials, equipment and repair - Technical training of MoWR/sector staff on emergency WASH to support adaptation of existing water system and services to the emergency context 1 The Syria CALL portal presents a crisis timeline, situation analysis and other information relevant to this crisis. 2 Humanitarian Needs Overview /7

2 - Capacity development of MoWR/sector staff for improved emergency WASH coordination (including disease response, information management, and needs analysis) - Provision of WASH facilities and hygiene promotion in collective shelters, public buildings and schools 2. PURPOSE AND SCOPE OF THE EVALUATION: An external summative evaluation is planned to assess the achievement of Oxfam s programme against its objectives and expected results. The evaluation is also expected to enrich our learning process as an organization, particularly for continued WASH programming inside Syria. Equal weight is attached to the accountability and learning function of the evaluation. The evaluation aims to support further strengthening of Oxfam s performance in delivering WASH programmes inside Syria. In this regard, the objectives of the evaluation are the following: 1. To assess the extent to which the programme has delivered against its objective and expected results; 2. To assess the design, planning, delivery, management and monitoring of the programme by Oxfam and its partners in accordance with Humanitarian principles, Oxfam Gender in Emergencies Minimum Standards, and other relevant codes, principles and standards as outlined in Oxfam s Humanitarian Dossier. 3. To identify and assess key internal and external factors (positive and negative) that have contributed, affected, or impeded the programme s achievements, and how Oxfam and partners have managed these factors; 4. To draw key lessons and good practices from the programme and make recommendations that will help inform Oxfam s design and implementation of future interventions Particular focus is placed on activities funded by the UK Department of Foreign and International Development (DFID) and European Commission - Directorate General for Humanitarian Aid and Civil Protection (ECHO). The scope for examination is determined using OECD-DAC criteria for evaluating humanitarian action, and relevant criteria are associated with a number of indicative key questions (under section 3) that are to be addressed and explored. The main intended users of the evaluation are Oxfam management and staff in Syria Country Office and MECIS Regional Office, Senior managers and advisors in Oxfam Headquarters, representatives from DFID and ECHO. It is also envisaged that the evaluation should be of interest and use to the MoWR and other actors involved in the implementation of the programme; OCHA, UNICEF and other WASH actors operating in Syria, as well as other donors supporting Oxfam s work inside Syria. 3. EVALUATION QUESTIONS: The evaluation questions to be addressed by this evaluation are expected to be further modified or refined in consultation with the Evaluator in the inception phase of the evaluation process. Indicative questions are: Relevance/Appropriateness: How appropriate and effective was the methodology of identification of the needs? 2/7

3 To what degree did the interventions address the WASH needs of targeted women, men, boys and girls? To what extent did key contextual changes, threats and opportunities that arose during the programme influence and inform implementation? How appropriate and timely were the alternative solutions/changes implemented by the team to overcome the challenges faced during the programme? Were the risk mitigation and contingency measures implemented by relevant projects appropriate and effective, and were mitigation actions adapted to address any changes in the risks to programme delivery? Coverage Have Oxfam and its partner prioritised and targeted geographical areas based on humanitarian need, and in line with the selection criteria outlined in relevant project proposals? What were the main constraints in terms of prioritization and targeting? How these difficulties were overcome? Have Oxfam and its partners reached the most vulnerable women, men, boys and girls in areas on all sides of the conflict proportionate to need? How appropriate and relevant were the eligibility criteria for beneficiary selection (e.g. of capacity building activities) and how consistently have they been applied? What were the main constraints in terms of prioritization and targeting of beneficiaries? How these difficulties were overcome? Effectiveness and impact: To what extent have stated objectives and planned results (impact, outcome, outputs) been met? What factors inhibited or accelerated the meeting of objectives and results? How effective and appropriate were the strategies and modalities employed and implemented by the team? How well were the activities carried out in terms of the approaches adopted, the timeliness of the interventions and their quality? How timely and effective has been Oxfam s response to urgent emerging needs and new displacement? How and how well were gender and protection concerns and considerations integrated into the programme and how did this effect beneficiaries? If it did not happen, why not? How appropriate and effective has been Oxfam s approach to remote management of activities? How appropriate and effective has been Oxfam s approach to monitoring and remote monitoring of activities, including mechanisms to obtain feedback from beneficiaries? How could Oxfam strengthen its monitoring and remote monitoring mechanisms? What system and mechanism are in place to ensure accountability to the beneficiaries, and how well did it work? How could Oxfam improve its accountability to beneficiaries? To what degree have lessons from the first year of programme implementation been used to inform the second year of implementation and make modifications where appropriate? Were programme design and delivery consistent with the Do No Harm principle and safe programme approach? What if any unintended consequences, positive or negative, has the programme had on local populations and/or on the environment? Efficiency: Was the programme implemented based on best use of the existing internal and external resources and capacity? What could Oxfam s future projects do to increasingly develop and invest in existing resources? How effective was the programme s approach to deliver Value for Money (covering effectiveness, efficiency, economy)? 3/7

4 How and what could Oxfam improve on in its future interventions in terms of Value for Money? Coordination How effectively did Oxfam coordinate with other relevant actors in Syria? To what degree has coordination positively influenced the programme and contributed to achievement of planned results? How could this be strengthened? Connectedness How appropriate and effective were Oxfam s exit strategy and actions taken to enhance the sustainability of the programme? What are the recommendations for strengthening sustainability in future projects? What role has the quality of partnership played in the project, and how might this be built upon in the future? Have the ways of working with partners and the capacity development support provided been effective and did they contribute to the programme s achievements? Where possible, have affected populations been involved at different stages of planning, implementation and monitoring? 4. METHODOLOGY: It is expected that the evaluation will be carried out in conformity with the Evaluating Humanitarian Action using the OECD-DAC Criteria and evaluation best practices. The Evaluating Humanitarian Action using the OECD-DAC Criteria can be accessed through this link: It is expected that this evaluation will draw on and review existing sources of information, and complement this with key informant interviews and / or group interviews. The proposed evaluation methodology should include - but does not have to be limited to: - Desk review of relevant programme and project documents (i.e. assessment data, country strategy, relevant project proposals, progress reports, monitoring data) as well as relevant secondary sources (e.g. needs assessment, WASH response strategy, evaluations, WASH sector documents) - Key informant and/ or group interviews with Oxfam staff involved in the programme - Key informant interviews with staff of MoWR, Syrian Arab Red Crescent (SARC), contractors, and other actors involved in the programme implementation - Key informant interviews with representatives of DFID and ECHO - Key informant interviews with other WASH actors operating in Syria - Field visit to project locations, subject to approval In case it will not be possible for the evaluator/consultant to travel to Damascus due to access restrictions, primary data collection for Syria will be done remotely from Beirut (using skype, Oxfam/partner staff visiting Beirut). It is important to note that the evaluator will not have access to members of the affected population for conducting interviews, with the exception of those who benefitted from capacity building activities. The evaluator is expected to develop a detailed methodology and evaluation plan in close collaboration with Oxfam (to be endorsed by Oxfam). With regard to the approach taken by the evaluation, given the combined accountability and lesson-learning rationale of the evaluation, a balance will need to be struck between independent scrutiny and participatory approaches. With respect to the latter, a relatively high level of participation is expected in terms of feedback and discussion of preliminary findings and recommendations. One of the deliverables is therefore a participatory workshop with the Oxfam team to share, discuss and validate preliminary findings. 4/7

5 Time-frame: The evaluation is expected to start around 1 October 2015 [Exact dates are to be confirmed with the selected consultant] 5. DELIVERABLES AND TIMELINE: Total expected level of effort (including travel and report writing): maximum of 29 working days if data collection is carried out in Beirut and Syria, and permission for field visit is obtained. In case the evaluator/consultant will not receive approval to carry out field visit, then the workload is estimated at 27 days. In case it will not be possible for the evaluator to travel to Syria and primary data collection is done remotely from Beirut, then the total expected level of effort is a maximum of 25 working days. This includes report writing and travel from the home location of the consultant to Beirut and Damascus and back. Key activities: Have an inception meeting with Oxfam to clarify on the expectations from the evaluation and have a mutual understanding before the evaluation begins. Review the evaluation questions and if necessary propose adjustments Review key programme and project documentation and relevant secondary sources Develop an evaluation plan to be endorsed by Oxfam - outlining methodology and tools, targeted respondents, resources required, draft outline of report structure etc. Primary data collection Data analysis and interpretation Facilitate a participatory workshop with Oxfam team to share, discuss and validate the preliminary findings from the evaluation Prepare a draft evaluation report that reflects feedback provided by Oxfam during the abovementioned workshop Integration of Oxfam s feedback on the draft report, and finalization of the evaluation report in close discussion with Oxfam; Deliverables An agreed Evaluation plan and methodology (to be endorsed by Oxfam prior to the start of data collection) Participatory workshop with Oxfam team to share, discuss and validate the preliminary evaluation findings Final evaluation report: The evaluator is responsible for submitting a draft report in English (electronic copy) within 10 days after completion of the abovementioned workshop. Feedback from Oxfam will be provided within 10 days after the submission of the draft report. The consultant is expected to produce the final report within five working days of submission of the comments. It will include changes/modifications, agreed between Oxfam and the consultant. Minimum requirements of the reports include: - The report should systematically answer the key evaluation questions posed and cover the evaluation objectives - The report should present findings to evaluation questions separately for DFID and ECHO-funded interventions where relevant - It should fairly and clearly represent the views of the different actors/ stakeholders - It should give the conclusions of the evaluator, in a way that is clear and substantiated by the available evidence - The main report should have a table of contents, executive summary, introduction, methodology section, evaluation findings and analysis, conclusion and lessons learned, recommendations. In addition, the final report should contain the following annexes, as well as any other relevant material: Initial and final Terms of Reference for the evaluation, Itinerary and final timetable, List of persons interviewed, List of documents reviewed, 5/7

6 Workshop presentation of findings. The report should be approximately 30 pages (excluding annexes) with a stand-alone executive summary of no more than 4 pages. Details of the report outlines will be shared and discussed with the supervisor upon contracting. The evaluator will report directly to the Regional MEAL advisor for the Syria Response, with the MEAL Officer in Oxfam Syria acting as in-country focal person. A reference group will be established, composed of relevant advisors and managers. The reference group will be responsible for reviewing and approving the evaluation plan, draft and final reports. The consultant will closely work with other Oxfam staff in Syria Country Office. Time-frame: Proposed timeframe for this evaluation is starting data collection in country on/around 1 st October 2015, and to have the First Draft of the Evaluation Report by 30 th of October and the Final Evaluation Report- that includes all changes/modifications, agreed between Oxfam and the consultant - by 30 th of November [Exact dates are to be confirmed with the selected consultant] 6. EVALUATOR QUALIFICATIONS: Individuals that meet the following required qualifications and skills are invited to apply for this consultancy: Demonstrable experience of producing high-quality, credible evaluations (samples required) Practical experience in the design and implementation of evaluations of WASH programmes/projects, particularly in humanitarian contexts In-depth understanding of humanitarian principles and standards, and their application Excellent and proven quantitative and qualitative research skills, and familiarity with different methodologies for evaluation Excellent interviewing capacity and strong facilitation skills Excellent verbal/written communication skills and strong report writing skills Fluency in English. Arabic is an advantage Familiarity or direct experience with working in Syria is preferred Familiarity with Oxfam working modalities desirable 7. CODES OF BEHAVIOUR: The evaluation process will be directed by Oxfam s Code of Conduct and Oxfam s guidelines for the ethical conduct of evaluations and research, guiding the evaluator(s) through careful consideration of the key ethical implications at every stage of the evaluation. These guidelines are available at this link: 8. SHARING AND USING FINDINGS: The Oxfam International s Policy on Program Evaluation requires Confederation members to act on the commitment to transparency by making public the Executive Summary and a Management Response to all final evaluations. The Policy is available at this link: 6/7

7 9. BUDGET Payment will be done in two instalments, 25% upon contract signature, and 75% upon Oxfam s approval of the final evaluation report. Costs for international travel (home location of consultant to Beirut, Damascus and back), pickup/drop-off from airport, visa, accommodation in Beirut/Damascus, per diems (at standard Oxfam rate of $27 per day), in-country travel for data collection and meetings, will be covered in line with Oxfam policies. 10. DISCLOSURE: Although free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of Oxfam. All data collected as part of this consultancy belongs to Oxfam and public dissemination of the data and evaluation products can only be done with the written consent of the Oxfam. 11. EXPRESSIONS OF INTEREST (EOI): Oxfam invites EOI from individuals with the experience and skills described above. The EOI must include: 1. A cover letter of no more than 2 pages introducing the evaluator/consultant and how the skills and competencies described above are met, with concrete examples. Please also use this cover letter to confirm the consultant s availability for the proposed period and to indicate proposed daily rate; 2. A brief technical proposal (2 pages max) of the proposed process, including: a. Key considerations for this evaluation b. Proposed evaluation methodology c. Timeline for implementation specifying dates 3. A CV for the evaluator/consultant, including 3 referees (with phone number and address); 4. Two examples of reports from previous evaluations/reviews in the sectors of intervention of this project or other relevant areas. Please submit the EOI and other documents (as mentioned in 11 above) by 10 August 12:00 PM (Beirut time GMT+2 ) to job.syria@oxfam.org.uk copying Abaan@oxfam.org.uk with Final Evaluation of Oxfam s Syria Crisis Response programme in the subject line. All questions or clarifications are to be sent to Etakee@oxfam.org.uk and Abaan@oxfam.org.uk **END** 7/7