Haiti Earthquake 2010 Response and Recovery Programme Final Evaluation

Haiti Earthquake 2010 Response and Recovery Programme Final Evaluation Terms of Reference (TOR) Summary > Purpose: the British Red Cross seeks to evaluate its Haiti Earthquake 2010 Response and Recovery Programme. > Audience: members of the public, beneficiaries, British Red Cross management and staff, Red Cross Red Crescent, other humanitarian actors. > Commissioners: British Red Cross Performance and Accountability Department. > Reports to: Head of British Red Cross Performance and Accountability Department. > Timeframe: preferred start date 25 May 2015 (but no later than 8 June 2015) until end of August 2015. > Consultancy days: estimated 100 in total for a consortium of three consultants. > Methodology summary: desk reviews, interviews, focus group discussions, surveys. > Location: home-based with travel to London in June and August 2015. Field trip to Port-au- Prince, Haiti preferred 15 June 2015-4 July 2015 but no later than 29 June - 18 July 2015. Background Haiti s earthquake of 12 January 2010 killed more than 220,000 people and left 1.5 million homeless. The Red Cross Red Crescent launched their biggest single country response in the immediate aftermath, sending 21 emergency teams to provide food, water, shelter and health support. After the emergency response phase, the British Red Cross concentrated recovery efforts on two large-scale camps in the devastated capital. It also provided livelihoods support in Port-au-Prince and the South Department, where people who had lost everything in the earthquake had moved and were struggling to get back on their feet. These projects were completed by June 2012. Between 2010 and 2012, the British Red Cross also responded to the cholera outbreak that affected the population by setting up two cholera treatment units and three rehydration treatment points in camps in Port-au-Prince and the South Department. Overall, the British Red Cross provided support to around 400,000 people since the earthquake. 1

In June 2012, the British Red Cross began a third phase of recovery work, the community-driven Urban Regeneration and Reconstruction Programme (URRP). The programme adopted an integrated approach to promote the regeneration of the most affected and vulnerable communities within selected areas of Delmas 19 in Port-au-Prince. The programme consists of a series of multi-sectoral interventions covering areas of: > Livelihoods (microfinance and SME initiatives, literacy initiatives and health insurance support). > Public health infrastructures (canal drainage and grey water management, provision of sanitation and solid waste management, health promotion and first aid). > Urban planning (upgrading and regeneration of housing stocks through repairs, new build and rental support, spatial planning and providing urban and social infrastructures). The British Red Cross engagement in Haiti will have spanned over 5 years from the initial emergency response to the URRP which is due to end in June 2015. Purpose and scope of the evaluation The British Red Cross has committed to report, reflect and learn about its entire engagement in Haiti. This final evaluation forms part of an exit process that will include internal and external reviews, audits, programme evaluations and an impact assessment to understand the successes and challenges of BRC s engagement in Haiti. An internal consultation, identifying key areas and questions, was undertaken in early 2015 to inform the scope of this evaluation to ensure that BRC is better able to respond more effectively in the face of future emergencies. The main purpose of the evaluation is to be accountable internally and externally for the interventions delivered over five years in Haiti. The specific objectives of the evaluation are to: > Provide a summation of the extent to which the British Red Cross engagement in Haiti achieved its objectives across different interventions (January 2010 to June 2015). 1 > Assess the effectiveness and impact of the integrated approach adopted in the URRP in Portau-Prince s Delmas 19 area (June 2012- June 2015). > Identify lessons (positive and negative) for improved programming and to inform strategic policy and planning. > Inform management decision making for ongoing and future work. 1 See Annex for an overview of studies, audits and evaluations already carried out.we anticipate this objective to be achieved by existing data review for interventions prior to URRP. 2

Methodology The nature of the evaluation assessing different sector responses as an integrated programme approach, covering programme design, delivery and management, looking at community level outcomes, and surveying a five year intervention period calls for the use of a wide range of information sources, both primary and secondary, quantitative and qualitative. It also calls for a variety of data collection methods, drawing upon different evaluation approaches. Triangulation of data sources and methods of analysis will be essential. An experimental design approach is not intended but appropriate use of sampling, across the spectrum from random to purposive, will need to be explored especially when it comes to beneficiary and household level surveys. We invite consultants to outline a proposed methodology in their application. We expect this to include but not be limited to: > Desk reviews. > Surveys. > Focus group discussions. > Key informant interviews. > Beneficiary case studies. Evaluation questions The British Red Cross has identified key areas to be addressed by the evaluation. Questions will include but are not limited to the following: > Relevance & Appropriateness To what extent did the respective British Red Cross interventions support those most in need? Was the scale and nature of the British Red Cross response appropriate and proportional to the organisation s capacity, experience and mandate? How did the approach to the emergency phase inform subsequent interventions? What was the quality of the needs assessments and analysis over five years? Were the criteria for targeting of beneficiaries appropriate to the need and context? How was support for different groups prioritised? How much beneficiary participation was there in decision-making, changes to plans, and implementation in Delmas 19? Was British Red Cross model of community mobilisation in Delmas19 effective? Were beneficiaries complaints collected, analysed and acted upon efficiently and effectively in Delmas 19? 3

Were all key stakeholders fully aware of what the British Red Cross aimed to do and how? To what degree did the programme's objectives remain valid in each phase in relation, to what the wider humanitarian community was doing at the time and over time? > Coverage To what extent were the British Red Cross interventions appropriate to meet the needs at local, regional and national level? Has the British Red Cross approach of focusing on smaller areas with integrated programming had a greater impact on the improvement of people s lives? > Effectiveness Did the programme achieve its objectives both in terms of outputs and outcomes? Was support timely according to the perception of key stakeholders, both internally within British Red Cross and externally? Were all aspects of the project cycle (assessment, planning, monitoring and evaluation, closure and exit) effectively undertaken, documented and acted upon appropriately? To what extent did plans change over the programme life cycle and was this effective? How effective was the monitoring system? What was the monitoring and evaluation data used for and by whom? How effective was BRC decision making and what factors hindered or facilitated the programme decision-making process? > Efficiency Were inputs (financial, human and material resources) appropriate and relevant in relation to the desired and achieved results? How effective was the British Red Cross in providing appropriate human resources to support field operations? How effective were financial management systems in performing their function? How effective were logistics and procurement services? How effective was the support provided by the International Red Cross Red Crescent Movement in delivery of the British Red Cross interventions? > Impact What positive and negative unintended consequences has the programme in Delmas 19 brought to the beneficiaries and non-beneficiaries? To what extent did Participatory Approach to Safe Shelter Awareness (PASSA) and other participatory approaches employed in Delmas 19 result in community-led changes and action towards safer housing at community level? What was the impact of integrated programming at household level? Did multiple packages of support such as cash grants, house repairs, literacy lessons, health insurance support have a multiplier effect on the quality and speed of household recovery? 4

> Coherence To what extent has coordination happened with other actors within the Red Cross Red Crescent Movement and outside? Was the programme in line with the British Red Cross, the International Federation of Red Cross Red Crescent Societies and the Haiti Red Cross strategies? What was the performance and impact of British Red Cross programme in the context of the wider Red Cross Red Crescent response? > Sustainability & Connectedness Was the link between early response, recovery and resilience considered as part of the URRP programme design? How did changes in information and context impact exit strategies? Were exit strategies in place from the beginning? How effective was coordination with local NGOs and governmental partners in Delmas 19? To what extent has learning from past programmes such as, the 2004 Asian Tsunami, the 2005 Pakistan earthquake and 2010 floods and cyclones in Bangladesh been used to shape this programme? What lessons can be drawn from the livelihoods, urban planning and infrastructure components of the URRP programme? > Value for Money With reference to existing data and key British Red Cross Value for Money indicators, to what extent did the respective interventions demonstrate Value for Money? Timeframe and deliverables The evaluation will take place between end of May/beginning of June 2015 and end of August 2015. The preferred start date would be 25 May 2015 with some flexibility. The field trip needs to take place preferably between 15 June 2015-4 July 2015 but no later than 29 June-18 July 2015. Time Schedule Activities Week 1 Briefings. Desktop study: review interventions documentation, and related primary/secondary resources for the evaluation. Week 2 Desktop study continued. Preparation and pilot of data collection tools. Deliverables Following briefings in London and preliminary desk review, the consultants will submit a draft outline of the inception report. Inception report The full inception report will include a revised methodology with a data collection and analysis plan. 5

Time Schedule Activities Week 3 Desktop study continued. Week 4 in Haiti Week 5 in Haiti Week 6 in Haiti Week 7 Week 8 Development of data collection tools. Data collection in UK office. Data collection in target communities according to data collection schedule. Data collection in target communities according to data collection schedule Data collection and analysis in target communities according to data collection schedule. Data analysis. Draft evaluation report. Week 9 Present initial findings, conclusions, and map of lessons identified to main stakeholders in London. Week 10 Week 11 Revise report based on workshop feedback. Circulate revised report to key stakeholders and collect written feedback. Week 12 Revise and submit final evaluation report. Deliverables Data collection tools This will include piloted data collection instruments. Communities and individuals completed according to data collection plan. Communities and individuals completed according to data collection plan. Field data validation workshop Following data collection in the field, the consultants will present and validate data through a workshop for key stakeholders in the field. Draft version of evaluation report Following the field visit, and data collection in the field, the consultants will deliver a draft report for feedback and comments of key stakeholders. Evaluation presentation workshop The consultants will present their findings and conclusions to key stakeholders in London. Feedback received and incorporated as appropriate. Final evaluation report in English This will include a standalone Executive Summary (max five pages), background of the programme, a description of the evaluation methods and limitations, findings, conclusions, lessons learned, recommendations. External evaluation report for publication A summary (maximum 15 pages) of the full evaluation report ready for external publication. Data sets will also be delivered at the end of the consultancy. 6

Evaluation Team The nature of the evaluation calls for a team of three independent consultants: a team leader and two technical experts with strong evaluation skills. The team will be supported by internal British Red Cross Advisers and support staff, as well as in-country national and international staff. The evaluation team leader has overall responsibility for the management and delivery of the evaluation. She/he reports to the British Red Cross evaluation commissioner and is ultimately responsible for submission of all deliverables. The evaluation team leader also coordinates the work of the shelter and infrastructure evaluator and the livelihood evaluator in evaluating the shelter and infrastructure, and livelihoods component of the programme. We are looking for a team of three people who fit the following profiles: > Evaluation team leader Minimum 10 years of experience conducting and leading evaluations, including of large scale humanitarian programmes required. Proven track record of conducting quantitative and qualitative research including the development of interview schedules, household surveys, and data analysis required. Experience in recovery programming in response to large, complex emergencies, ideally in a management position required. Strong coordination and facilitation skills and proven ability to design and facilitate planning processes and workshops required. Excellent analytical, writing and presentation skills required. Experience of working in Haiti required. Excellent written and spoken English skills required. Minimum qualification of a PhD or Masters in International Development or related field required. Experience in community based programmes preferred. Working knowledge of French and/or Haitian Creole preferred. Knowledge and experience working with the Red Cross Red Crescent Movement highly preferred. We particularly welcome applications from teams who can provide technical experts with experience of working in Haiti and French and/or Haitian Creole language skills, and an understanding of the Red Cross Red Crescent Movement ways of working. We would prefer technical experts to have experience in community based programmes in humanitarian settings in addition to the following role specific requirements: > Shelter and infrastructure evaluator Minimum 7 years of international humanitarian experience in the fields of urban planning, shelter and infrastructures, including experience in conducting evaluations. Professional qualifications, such as RIBA, MICE, MCIOB, preferred. 7

> Livelihoods evaluator Minimum 7 years of experience in household economic security or related, including experience in implementing and evaluating livelihoods programmes. Proven track record of conducting quantitative and qualitative household surveys and interviews, and data analysis required. Evaluation Quality & Ethical Standards The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members, and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. The evaluation standards are: 1. Utility: Evaluations must be useful and used. 2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner. 3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation. 4. Impartiality & Independence: Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders. 5. Transparency: Evaluation activities should reflect an attitude of openness and transparency. 6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined. 7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate. 8. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation. It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality. Further information can be obtained about these principles at: http://www.redcross.org.uk/about-us/who-we-are/the-international-movement/fundamentalprinciples Application procedures We will consider applications from: > Consultancy Limited Companies who can provide the full team of requested consultants. > Self-formed groups of self-employed consultants who can cover each of the roles requested. > Self-employed consultants who can cover one of the roles requested [This is our least preferred option]. 8

Interested parties should submit their application material to: Annalisa Tidona, Performance and Accountability Adviser, atidona@redcross.org.uk by midnight (BST) on 4 May 2015. Interviews (in person or by Skype) of short-listed candidates will be held on 7 and 8 May 2015. Application materials should include: 1. Curricula Vitae (CV) for all members of the team applying for consideration. 2. Cover letter clearly summarizing the team experience as it pertains to this assignment, confirmation of availability in the timeframe indicated, and professional references. 3. Proposal not exceeding four pages expressing an understanding and interpretation of the TOR, a proposed methodology with time plan and an outline of the roles and responsibilities of each consultant. 4. Sample of evaluation reports most similar to that described in this TOR. Annex List of main reports to date 2010 Mass Sanitation Module evaluation Haiti emergency response evaluation of the British Red Cross mass sanitation module (MSM) emergency response unit (ERU). 2010 Six month review Haiti emergency response reflection of the British Red Cross response to the earthquake, in the sectors of logistics, relief, sanitation, health, economic security, and shelter. 2010 Cash transfer programme learning review Haiti emergency response review commissioned by the American Red Cross and British Red Cross to identify lessons learned from the approaches to cash transfer programming that were adopted in Haiti. 2011 Mid-term review Haiti recovery programme mid-programme review of sanitation and hygiene promotion in two camps in Port-au-Prince; cholera response and livelihood projects in the South Department, and the integrated (livelihoods, shelter, sanitation, hygiene) programme in Delmas 19 (later called URRP). 2011 South Department cholera response final report Haiti cholera response end of response final report of British Red Cross cholera interventions in the South Department. 2011 South Department livelihoods evaluation Haiti recovery programme-end of response final report of British Red Cross livelihoods programme in the South Department. 2012 Port au Prince camps sanitation evaluation Haiti recovery programme end of response endline survey of sanitation and hygiene promotion in two camps in Port-au-Prince. 2013 Mid-term review Haiti recovery programme URRP mid-term review. 2014 Urban livelihoods recovery case study Haiti recovery programme lessons learnt case study of the livelihoods component of URRP. 9