Page 1 of 9 External Evaluation - Terms of Reference An evaluation of the Rapid Response Mechanism from February 2015 to March 2017 UNICEF Iraq A. Background The fighting in Syria and in the west, north and east of Iraq has caused a rapid and ongoing displacement of at least 3.3 million internally displaced persons (IDPs) by January 2016 1 to over 3,000 locations across Iraq 2. Affected populations have experienced one or multiple displacements which has had negative consequences on their ability to cope and recover from shocks. Contributing factors to displacement include lack of access to services, social and-gender based violence and discrimination based on identity and ethnicity, along with more complex needs such as protection and resilience support. In a still highly centralized system, the disruption of supply chains between Baghdad and the Governorates has reduced the availability of essential commodities. The collapse of government services, the public sector, as well as the public distribution system in large parts of the country, has rendered a growing number of Iraqis vulnerable. Rapid Response Mechanism (RRM) The RRM was introduced in June 2014, based on the RRM model in the DRC to meet the immediate lifesaving needs of internally displaced persons in essential items (drinking water, emergency nutrition, and hygiene). The approach is tailored to the rapid, wide-scale and multiple displacements occurring within Iraq since late 2013 and serves as a first line response for recently displaced IDPs. The Rapid Response kits are provided to IDPs who are: on the move; stuck between checkpoints or do not have access to the commodities due to unavailability of markets and services within 72 hours of receiving the initial information on the displacement. The RRM is activated to rapidly assess needs and deliver the minimum package through implementing partners with previously prepositioned stocks. As the first line response, it is quickly followed-up by cluster-specific responses that are coordinated through the Inter-Cluster Coordination Group led by OCHA. The RRM methodology has evolved over time. Initially the RRM kits consisted of 6 litres of water, a water container, an adult hygiene kit and high energy biscuits, which covered a family of 6 for about a week; UNICEF prepositioned only 500 with one partner. As the diversified needs of beneficiaries became known, and displacement rapidly increased, the RRM continued to evolve. UNICEF joined forces with WFP in late 2014 to include WFP s Immediate Response Ration (IRR) and 12 litres of drinking water, while growing the partnership to 7 implementing NGO partners with contributions from UNFPA covering 16 governorates and up to 100,000 kits pre-positioned with partners. The RRM became a stand-alone 1 International Organization for Migration (IOM) Displacement Tracking Matrix (DTM) Round 36 January 2016 2 An estimated 10 million people across Iraq are in need of humanitarian assistance, including Internally Displaced People (IDP), Syrian refugees, returnees and host communities, as well as affected populations in Armed Opposition Group (AOG) held areas. As of 22 October 2015, 3,202,638 people (533,773 families) were displaced across Iraq (International Organization for Migration (IOM) Displacement Tracking Matrix (DTM) Data Sheet 22 October 2015.
Page 2 of 9 component in the 2015 and 2016 HRPs and actively participate in the Inter-Cluster Coordination Group (ICCG). As of February 2016, the RRM Consortium has settled with 7 implementing NGO partners shared by WFP and UNICEF as co-leads of the RRM. UNICEF is providing overall RRM coordination together with WFP, including strategic planning and technical leadership, capitalization of lessons learned, and scale up of best practices and innovations. Increased displacement across multiple locations required the RRM Consortium to adapt prepositioning guidelines and expand geographical coverage to ensure access and reach. This required additional resources from donors. To scale up and maintain the Rapid Response Mechanism as a first line humanitarian response for people displaced at the onset of an emergency, ECHO in February 2015 committed 4.85 million 3 for a period of February 2015 through 15 June 2016. Additional funding was provided by both OFDA and ECHO in 2016 to all RRM partners UNFPA, WFP and UNICEF. Since January 2016, the RRM Consortium distributed RRM kits benefitting 1.3 million people. RRM Results and Minimum Package Since July 2014, the RRM Consortium has provided rapid humanitarian assistance to more than 5 million internally displaced persons fleeing to safety in over 3,000 locations in 16 governorates across Iraq. The RRM assistance package is comprised of four components: For UNICEF: - one 12-pack of 1.5 litre bottled drinking water (equal to 12 litres); - one hygiene kit, including soap, sanitary napkins, hand sanitizer, a scarf, bleach and baby diapers; - one water container For WFP: - one 6.4kg immediate response food ration of culturally acceptable, ready-to-eat items including biscuits, canned chicken, canned beans, chickpeas, tea & dates For UNFPA 4 : - one female dignity kit, including sanitary pads, underwear, towel, comb, scissors, flipflops, scarf, Abaya, solar-powered flashlight & Clean Delivery/Newborn kit Presently, one package is provided to each family of seven persons and can last for about a week. Large families may receive two packages. Project Objectives and Results The objectives and results of the RRM are as follows: 3 Revised funding as per amendment dated Aug 21/08/2016 4 Where feasible, UNFPA also contributes Dignity Kits to families on the move as part of the RRM. In 2015, 20,000 Dignity Kits were provided. We are collectively seeking to integrate Dignity Kits as a standard RRM Kit component.
Page 3 of 9 Principal Objective: Assess humanitarian needs and provide the needed immediate support to displaced vulnerable populations in Iraq. Specific Objective: Provide immediate life-saving and dignity-raising supplies to displaced households in Iraq through RRM, WASH, Health, Nutrition and Winterization Services. Result: Iraqi displaced households in movement or temporary settlement receive a humanitarian basket of lifesaving and dignity-raising supplies B. Purpose of the Evaluation, Scope and Intended Use i) Evaluation Purpose The evaluation seeks to assess how the RRM is delivering on its stated objectives, its performance and outcomes between February 2015 and March 2017 through its RRM Consortium. The evaluation will analyse whether it is offers an appropriate and effective way forward for delivering aid, as well identifying lessons learned that can inform future programming in Iraq. ii) Scope and focus The evaluation should concentrate as far as possible on the programme in its entirety. It is not the intention to evaluate the performance of the individual agencies who are partners of RRM, although comparisons of these is expected in order to draw out lessons learned and help improve overall performance. The period of the evaluation is between February 2015 and March 2017. The evaluation framework will be based on the OECD-DAC criteria for the evaluation of humanitarian action (with the exception of the impact and coherence criterion, which has been replaced by a criterion related to coordination,). 5 The evaluation has the following objectives: 1) Assess UNICEF s co-leadership role of the RRM in terms of enabling the humanitarian organizations working in the RRM consortium to respond to the ongoing crisis by providing rapid and flexible funding to partners along with technical guidance and leadership to implement projects to meet humanitarian needs. 2) Assess the appropriateness, effectiveness, efficiency and coverage of the RRM in providing first line humanitarian response. Identify the sustainability of this mechanism. 3) Identify strengths and weaknesses in the coordination of the RRM in terms of enabling the humanitarian organizations working in the RRM consortium to respond to the ongoing crisis 4) Identify good practices and lessons learned of the RRM. 5 The evaluation should also reflect and make reference to the following norms and standards where appropriate: The Convention on the Rights of the Child, the Guiding Principles on Internal Displacement, and UNICEF s Core Commitments to Children in emergencies, the SPHERE standards for humanitarian action, and the Red Cross Code of Conduct for humanitarian organizations.
Page 4 of 9 Evaluation Questions: Appropriateness and relevance Were the priority needs of targeted populations accurately identified? Did the RRM correspond to priority needs of the targeted population? Was the design of the programme appropriate to achieve the objectives? Does the RRM succeed in identifying and targeting the most vulnerable people? Are their needs taken into account by the programme? Were protection risks considered in the design of the programme? Was the monitoring system appropriate and did the agencies take into account monitoring results and feedback during implementation? Were there appropriate control mechanisms in place to mitigate against diversion and corruption risks? How did the monitoring the quality and rapidity of the response provide feedback to the mechanism? To what extent was the RRM s intervention timeframe appropriate? To what extent the programme is gender sensitive or address gender issues? Why and why not? Effectiveness To what extent has the RRM served as part of key preparedness and response measures? To what extent has the RRM been able to attain its stated objectives and has produced the expected results? Was the assistance timely? Is the response sufficiently rapid, taking into account the objectives stated in the programme documents and the operational context? What type of assistance would beneficiaries have preferred and why (e.g. cash, voucher, inkind, combination, other types of programmes)? What were the main benefits of the intervention? What were the main downsides? Were there any missed opportunities? To what extent has the RRM contributed to local capacity building of NGOs and made linkages with other actors in ensuring follow up humanitarian response? Coordination Is the donor community responding to the ongoing crisis by providing rapid and flexible funding to the RRM along with technical guidance and leadership to implement projects to meet humanitarian needs? What is the quality of the relationship between RRM and other humanitarian structures (particularly clusters)? Did the RRM contribute to strengthening the coordination mechanisms and information sharing through Rapid assessments using Open data Kit (ODK)? Are there any operational or technical bottlenecks in effective utilization of rapid assessments through ODK? What were the factors inhibiting the effective utilization of ODK?
Page 5 of 9 Coverage Is the targeting strategy well-adapted to the context? How clear and appropriate are the intervention criteria which determine who (RRM or other humanitarian actor) should intervene once an alert is confirmed? Given the resources at its disposal, does the programme target and reach an appropriate proportion of the affected population? How effective is RRM s partnership strategy? Is the number of partners sufficient and do they have the necessary resources (human, material, financial) and access to deliver the programme s objectives? Efficiency Were the RRM funds appropriately targeted and prioritized? Is the RRM an efficient model for humanitarian aid delivery as a first line response How cost-efficient is the programme, in terms of the qualitative and quantitative outputs achieved as a result of the inputs? In what ways have UNICEF s and its partners financial and logistics arrangements contributed to the efficient use of resources and economies of scale? What improvements can be made? Is the programme structure and staffing efficient in terms of fundraising, management, and administrative arrangements? Were there any delays? If so, what caused the delays and what were the consequences? Coherence How RRM complements other Emergency Response mechanism in order to ensure that humanitarian needs are addressed? Has the design of the RRM served as complementarity to other aid delivery mechanism in Iraq in order to ensure that humanitarian needs are addressed? Connectedness How did the link between the first and second line response ensure that humanitarian needs are met? How could this link be improved? How do RRM interventions work alongside complementary interventions in other sectors or programmes? Is there any duplication? How could this be improved? iii) Intended use The evaluation results will inform and influence the future programming for various partners to address both immediate and longer-term strategic issues. The evaluation findings and conclusions will be shared with ECHO and OFDA (the donor), RRM consortium, OCHA, UNICEF Staff, other interested partners and relevant stakeholders. The results of the evaluation as well as the data collected through questionnaires and interviews are the property of ECHO and UNICEF. They can t be used by the evaluation consultants in any publication.
Page 6 of 9 The evaluation consultants have to hand over to UNICEF a clean dataset containing all the data collected through the questionnaires and focus group discussion. C. Methodology The evaluation will review the current programme of RRM carried out over a period of February 2015 to March 2017. The utilization-focused evaluation design should ensure that the evaluation purpose and focus questions are addressed through developing a robust mixed methodology which ensures that both qualitative and quantitative evidence are collated. The evaluation should adopt a contribution analysis approach which assess the delivering on its stated objectives, its performance and outcomes. A detailed methodology will be elaborated in the inception report. At minimum, the evaluation will use the following: Review of Literature and analysis of relevant document: The evaluation team shall examine independent reports, topical/thematic reports produced by the UNICEF implementing partners, UNICEF monitoring reports, donor and internal reports, and any other relevant materials. This will be made relevant in the inception report. Key Informant Interviews: The evaluation team will conduct interviews with key stakeholders, namely implementing partners, RRM steering committee lead, donors, WFP, OCHA, UNFPA and other relevant UN agencies, relevant local authorities, institutions and key government partner, interagency networks and other pertinent stakeholders. Focus group discussions: The evaluation team will conduct focus groups discussions. Due to the peculiar nature of the RRM, it will be very challenging to connect with the most recent beneficiaries given that they are difficult to identify as the population is on the move and there is no mechanism to keep track of individual beneficiaries; however, the Evaluator(s) should make an effort to meet with beneficiaries of the RRM. Field Visits: The evaluation team will make field visits to study the work of implementing partners, including direct observation of programme activities for any on-going activity and informal discussions with beneficiaries (where possible) and non-beneficiaries in the visited areas. Use of Surveys: A major country level household survey is not envisaged but the evaluation team will be expected to conduct household interviews with a clear sample design where possible. Ethical considerations The evaluation will be conducted in line with UNEG norms and standards for evaluation, as well as ethical considerations. All those engaged in designing, conducting and managing evaluation activities will aspire to conduct high quality and ethical work guided by professional standards and ethical and moral
Page 7 of 9 principles. The contractor must clearly identify any potential ethical issues and approaches, as well as the process for ethical review and oversight of the evaluation process in their proposal. Human rights and gender equality are understood as a cross-cutting aspects of the evaluation. D. Deliverables and payment All reports as listed below should comply with the standard reporting requirements and shall be written in good Standard English. i) Inception Report The inception report shall be submitted to the evaluation Management team. It shall contain the overall plan and evaluation framework to carry out the evaluation. The report should include a detailed methodological approach, work plans, and assessment tools. A full list of evaluation questions and methods relevant to answer these will be developed in the inception phase and incorporated into the inception report. The report could propose adjustment to the ToRs as well as any other relevant issue to successfully complete the assessment. The inception report is expected by April 15th, 2017 ii) Presentation of preliminary findings and initial recommendations The Evaluator(s) will present the initial findings and recommendations to stakeholders in Iraq. The in country workshop will be organised by UNICEF. The presentation will take place by April 30, 2017 iii) Draft Report The draft report of the evaluation must be submitted to UNICEF Iraq Country Office for their review and comments by May 10 th, 2017 20% of payment. iv) Final Report The Final Evaluation report will be in English and Arabic, structured as per UNICEF-adapted UNEG standards including: Executive Summary Object of the Evaluation (Background & context) Purpose, Objectives and Scope Methodology Findings Conclusions and lessons learnt Recommendations The final report should be between 40-50 pages excluding annexes and should be submitted by May 20 th, 2017 80% of payment. 6 6 The outline of the final evaluation report proposed by the evaluation team should be agreed by the Evaluation Management Team beforehand and should reflect UN Evaluation Report Standards (September 2004).
Page 8 of 9 F. Steering Committee A Steering Committee 7 will be established and will be supervised by the Chief of Planning, Monitoring and Evaluation. It will oversee administration and overall coordination of the evaluation, including monitoring progress. The evaluation team will report to the Steering Committee in terms of executing the evaluation. The main functions of the Steering Committee will be in: Selecting the external evaluation team; Reviewing and commenting on the inception report and approving the proposed evaluation strategy and budget; Reviewing and commenting on the draft report; Approve final report E. Evaluation Team composition The Evaluation will be conducted by an external consultant through either individual or institutional consultancy (a team of consultants) with the following qualifications: They lead consultant should have: Minimum of 5 years experience in conducting and leading humanitarian evaluations; Extensive work experience in humanitarian affairs and programs in conflict affected countries Demonstrated capacity for strategic thinking and ability to work collaboratively with multiple stakeholders; The team leader should be skilled in writing quality reports This contractor should have knowledge of/experience in the region. S/he needs to be fluent in spoken and written English and Arabic or Kurdish. If this is not possible, a member of the team should be fluent in one of the languages (Arabic or Kurdish). The evaluator or team will be responsible for organising and/or conducting all field and research activities, and for writing the inception and final evaluation reports. Support staff should be hired for the duration of the evaluation to assist with document collection, scheduling, research, interpretation, and interviews as necessary. The composition of the team and all evaluation team members should be gender aware. Where necessary, a staff member from one of the RRM organisations and/or UNICEF will help facilitate the in-country work of the evaluation team including specific meetings and field visits. 7 The Steering Committee will be made up of the Chief of Planning, Monitoring and Evaluation, the Chief of Field Operations, the Monitoring and Evaluation Specialist and Monitoring and Evaluation Officer. Technical support will be provided by the Regional Monitoring and Evaluation Advisor.
Page 9 of 9 Statements of interest and proposals from both groups of individual consultants, institutions and universities is encouraged. We also encourage joint proposals, e.g. from an Iraqis and an international organisation/university. However, all submitted proposals must cover the whole range of the evaluation. F. Expression of Interest and budget considerations Interested parties should submit a proposal including the following: A brief proposal (5-10 pages) for how they would carry out the evaluation. This should include: o a short presentation of the team and its experience in evaluations of humanitarian programmes, o the proposed evaluation methodology, o a work plan indicating milestones and time dedicated to field visits, and o the evaluation team setup. CVs for members of the evaluation team. Indication of availability Proposed budget and fees (to be submitted separately) with an adequate breakdown of the evaluation s cost structure. A selection process will be carried out by the Evaluation Management Group based on the following criteria: Prior experience of carrying out evaluations of large-scale, and in humanitarian programmes. Specialised knowledge of IDP / returnee programmes in contexts of complex crises. Knowledge of the Iraqi context. Pertinence of the proposed methodology and work plan. Proficiency in Arabic and English (mandatory). Cost and fees. International local partnerships are an asset. An evaluation team with a gender balance and experience of the suggested team members will be an asset. During the selection process, shortlisted candidates might be asked by the Steering Committee to provide additional information and documentation, such as reports of previous evaluations. H. Time Frame The evaluation is expected to take a maximum of 45 working days.