Terms of Reference Final Evaluation of Integrated Food Security, Livelihoods and Nutrition programme in Mwanza district, Malawi 1. Summary 1.1 Purpose: The main purpose is to assess the relevance of the Project 2013-2015 in the wider context of Red Cross food security programme in Malawi since 20115, see section 3 for details. 1.2 Audience: 1.3 Commissioners 1 : This external] evaluation is commissioned by the Finnish Red Cross in compliance with the FRC learning and evaluation framework. 1.4 Duration of evaluation: 27 days 1.5 Time frame: during May 2016, as agreed with MRCS 1.6 Location: Malawi, Lilongwe and Mwanza District 2. Background Malawi Red Cross Society (MRCS) has implemented Food Security Projects in Mwanza District since 2005. The programme came as a necessary step after the large scale emergency food assistance provided also by the Red Cross in Malawi in that time. There has been several phases; first the funding was channelled through the IFRC 2 under the its Africa wide frame of food security focusing on the nutritional needs of people living with HIV/AIDS through improved household level food production. Mwanza was one of four districts where MRCS operated (on top of Lilongwe, Nkhotakota and Nchinji). The programme was represented in the multi-country food security evaluation jointly undertaken by the Finnish Red Cross and four Southern African National Societies (Malawi, Mozambique, Swaziland and Zimbabwe). The evaluation for Malawi was conducted from, 23 November to 7 December 2009. In 2010 funding and programming became bilateral when MRCS and Finnish Red Cross (FRC) started direct co-operation, at the same with a bilateral community based health programme in Blantyre District. In Mwanza, there have since been two projects following seamlessly from one to another. 2010-2012 Integrated Food Security Project (FRC project 7942) 1 Commissioner organizes, finances, selects and contracts the evaluation team. 2 International Federation of Red Cross and Red Crescent Societies 1 P a g e
2013-2015 Integrated Food Security, Livelihoods, Water and Sanitation and Nutrition project in Mwanza district, Malawi (FRC project 7959) The project 2010 2012 was reviewed under Inter-Country Food Security Mid-term Review in Southern Africa: Mozambique, Malawi and Swaziland (2010-2011) in November, 2011. Final draft of the report is available. Further, the results have been described in an IFRC Case study A diversified approach to fighting food insecurity and rural poverty in Malawi (1250700-Malawi) but ut there was no final evaluation. The project was implemented in Traditional Authority (TA) Kanduku where the major hardware, mainly irrigation systems, were finalised in 2013. Project goal was to reduce food insecurity of vulnerable households and vulnerable communities to disasters and those affected by HIV in the Mwanza District by 2012 with project objectives: Improve and increase balanced and diversified agricultural production of food and cash crops by 25 per cent through small-scale irrigation to meet the food requirements of 1000 households by 2012. Build the capacity of 1,800 targeted households to meet their food security needs and improve their economic status by 2012. Increase household income of 400 vulnerable households by 25 per cent by 2012. Project 2013-2015 (2014-March 2016) While the phase 2010-2012 was approaching an end, an extension, a new project, was agreed and its design was based on the lessons learnt from the previous project. Lessons learned and recommendations being; the need to spread out the positive results more widely in the beneficiary communities, to promote the presence and uniqueness of Red Cross as a platform, to incorporate health and nutrition through Community Based Health and First Aid (CBHFA) approach and methods as per MRCS decision to use as a basis in its community work, to acknowledge improved livelihoods as a driving force in success and to ensure that cross-cutting issues of gender, social inclusion and climate change awareness were taken seriously. Mainly initiated by FRC the focus was changed from hardware components to more integrated approach. However, due to organisational reasons, the planning process was lengthy and only in early 2014 a compact project plan was concluded, two-year implementing period that was left was observed by reducing number of activities and restricting the target population to three communities. Project goal: Empowered and more resilient communities in sustainable livelihoods within changing climate context. Overall Objective: To contribute to well-being of targeted communities in Mwanza district through improved livelihood and food security by 2015 Outcome 1 Increased knowledge and capacity of beneficiaries on livelihood with the outputs: Households with new income generation activities 3 Village Savings and Loans groups established and functional Households with increased knowledge in balanced diet and nutritional practices Outcome 2 Improved availability and accessibility to food at household level, with the outputs: Households with increased and diversified food sources 2 P a g e
Households with improved post-harvest handling, storage and processing techniques Outcome 3 MRCS staff and Mwanza branch have improved capacity to implement and sustain Red Cross activities, with the outputs: 2010-2012 activities that required support are finalized and phased out Strengthened volunteer network Project staff has adequate skills to implement Red Cross community based project. Sound financial management and reporting of project funds in place The planned project implementation methods included: Direct support; supply of farming in-puts such as seeds, fertilizer and cassava cuttings, provision of treadle pumps for the farmers in small scale irrigation scheme, provision of livestock (goats), material for beekeeping and aquaculture (fish farming) Capacity building of volunteers and farmers through various training and technical advisory service in: diversified agricultural production at household level; preserving and processing of food using locally available materials, backyard gardens diversified nutritional practices and household level planning Village Savings and Loans groups The programme was implemented under the Disaster Management (DM) department, supported by other relevant departments (Finance, OD, HR, Health and Care) and implemented in the district by the FS team in Mwanza district in close collaboration with the MRCS Division and relevant district stakeholders. In 2015 a separate component was integrated to the project: Early warning systems and feedback cooperation between Malawi Red Cross Society (MRCS) and Department of Climate Change and Metrological Services- (DCCMS) The purpose was to create and provide a platform to the DCCMS to be able to disseminate localized weather forecast in selected areas in Salima and Kaliyeka districts and to use that later as an example for other areas. The specific objectives are: To build the capacity of MRCS staff, volunteers and stakeholders in the translation of weather forecast. To strengthen community structures for effective Early Warning Systems. To increase number of localized weather products produced by DCCMS 3. Purpose and Scope of the Evaluation FRC is committed to meet criteria for quality in the development projects and operations. Evaluations uphold FRC commitment to accountability and organizational learning and the results will be used when programming new initiatives, specifically the Consortium lead Resilience Programme in Malawi. The purpose of this evaluation is to look back on the period since 2005 and to search traceable signs of the impact of the MRCS Food Security programming. Specifically, the inputs between 2010 and 2012 will 3 P a g e
be reviewed. However, the main purpose is to assess the relevance of the Project 2013-2015. For this purpose the results of the baseline of 2014 will be compared to the situation in 2016. The evaluation will: - assess if there are signs of supporting an medium-long term impact most likely sourced to Red Cross programming since 2005 - asses the level of achievements in the phase 2014-2015 / today against the set targets and to reflect what have been the reasons for success and failure, gains and challenges - assess the relevance of the programme for MRCS and for the community in light of capacities, empowerment, resilience and sustainability - provide lessons learnt for future programming This final evaluation upholds FRC and MRCS commitment to accountability and organizational learning, and its findings and recommendations can be used for replication and roll-out of similar activities to new target areas conditional upon available funding. The donor, Ministry for Foreign Affairs of Finland records all the project evaluations and has a right to publish and share them. The report and findings will also be shared with the district and national level stakeholders in Malawi. 4. Evaluation Objectives and Criteria The specific objectives and key questions to be addressed in this evaluation, under the areas outlined above, are: Clear conclusions on the relevance, effectiveness, efficiency, impact and sustainability of the programme and the analysis of the reasons behind successes and challenges in each of these areas. Lessons learned and recommendations to be utilised in in the project planning, implementation and management The objectives are to be studied against the following evaluation criteria guided by the key question: Relevance (The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor. Retrospectively, the question of relevance often becomes a question as to whether the objectives of an operation or its design are still appropriate given changed circumstances.) Is the project/intervention consistent with the needs and priorities of the beneficiaries and the overall sector framework (food Security, community development) in the country and the National Society, MRCS? Has the situation changed since the approval of the project plan? Is the project strategy still relevant in the given context? To what extent the financial and technical support was adequate for the particular context and development stage of the partner? What worked well, what could have worked better? Has the programme and the outcomes been in line with the NS strategic plan? 4 P a g e
Effectiveness (A measure of the extent to which an aid activity attains its objectives.) To what extent the intervention/project objectives have been attained? Evaluate how the overall project plan and its subsequent annual plans were implemented. What worked, what did not work? Efficiency (Efficiency measures the outputs -- qualitative and quantitative -- in relation to the inputs. It is an economic term which signifies that the aid uses the least costly resources possible in order to achieve the desired results.) How well the resources have been used to produce achievements and results? Were activities cost-efficient? Was the project implemented in the most efficient way compared to alternatives? Were objectives achieved on time? Impact (The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended. Relates to the goal level of the logframe hierarchy. Comparison and analysis based on baseline and endline part of the evaluation of the impact of the intervention) What has happened/changed as a result of the programme or project? Did the project lead to improving the services delivered and access to the vulnerable, accountability and ownership? What impact did the project have on the service delivery to the vulnerable countrywide? Define how the project contributed to community development beyond the Red Cross activities. 5. Sustainability of the outcomes (Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially sustainable.) How sustainable are the project outcomes? In particular, to what extent are the [project outcomes/results] functioning thanks to the project? Determine whether the intervention demonstrates financial, institutional and social sustainability particularly in terms of ongoing costs and any required capacity. Identify the factors that may influence sustainability in the short, medium and long-term. Determine the appropriateness of an exit strategy, reorientation or planning for future interventions. 6. Inclusion of the cross-cutting issues in the entire project cycle (gender and diversity, climate sustainability, social inclusion and accountability to beneficiaries) in the project implementation. Is the project addressing the different needs of the beneficiaries in a consistent manner? I.e. is the project considering the different needs of women, men, girls and boys, different social and ethnic groups as well as disability issues? What measures is the project taking to contribute towards reduction of inequalities? Are there any ways to improve reduction of inequality and gender mainstreaming in the next project cycle? How is the project addressing climate sustainability? 5 P a g e
7. Coherence The extent to which stakeholders were involved throughout the project cycle (assessment, designing, implementation and monitoring) How transparent has the project been to communities, counterparts and the donor, e.g. information sharing, operating effective feedback mechanisms, participation, monitoring, evaluation and learning? Has the project been able to contribute to development of harmonised community health programming within MRCS? Was the coordination, both external and within MRCS, addressed and managed in a cohesive and effective manner throughout the programme keeping in mind sustainability? 5. Evaluation Methodology The evaluation will use the following data sources and reference documents: - Partner agreements; MoU, Project contract - Project Document and Amendments - Project related documentation provided by the FRC and MRCS (plans, budgets, financial and narrative reports, guidance documents, mission reports etc.) - Multi-country evaluation 2010, Mid-term review 2011 - Training reports, Minutes of Meetings - VCA, Feasibility study and baseline reports and results - MRCS strategic plan 2010-2014 - Relevant local government guidelines and policies - Federation strategy 2020 and relevant policies and guidelines incl. IFRC Framework for Evaluation - Ministry for Foreign Affairs of Finland development policy and programme (2012) - Measuring Community Resilience in the IFRC (2014) - District Social Economic Profile Proposed methods of data collection and analysis will include: - Document analysis/review; a critical review of the sources and reference documents - Interviews with project partners and key stakeholders; MRCS and FRC staff, MRCS volunteers and members (division), local authorities; key NGO partners - Beneficiary household survey to establish end line indicator values - Key informant / Focus Groups interviews with stakeholders including beneficiaries - Field observation and site visits All findings should be evidence based and methodology used explained in the Evaluation Report. 6. Deliverables The evaluation team will provide: 6 P a g e
1. An inception report by the Team Leader following the desk work prior to the mission to demonstrate a clear understanding and realistic plan of work for the evaluation. The inception report outlines final work plan and detailing the planned methodology. The report will be reviewed by the FRC Quality Group and MRCS focal person. 2. A debriefing to the MRCS management and possibly to FRC regional delegate and key project stakeholders at the end of the field mission to discuss the initial findings, conclusions and recommendations. The team will produce a memo on debriefing with the MRCS prior to departure. Relevant feedback from the debriefing will be integrated in the evaluation report. 3. A draft evaluation report within two weeks after return from the field mission. The draft will be shared with MRCS and other relevant stakeholders for comments. The comments from the FRC, MRCS and other stakeholders will be sent to the Team Leader within two weeks after receiving the draft. 4. A final evaluation report to be submitted to the FRC within 2 weeks of receiving the comments for its final approval. FRC will share the report with MRCS. The report will have an Executive Summary and will have a maximum length of 20 pages, excluding annexes. 5. A lessons learnt workshop in Malawi to the MRCS management and with the RC/RC Movement partners at the end of the mission to discuss the initial findings, conclusions and recommendations. The team will produce a memo on debriefing with the partner National Society prior to departure. Feedbacks from the debriefing will be integrated in the evaluation report. 6. A presentation of the evaluation findings at the FRC IOP by the Team Leader/Evaluation Manager. 7. Proposed Schedule A detailed timeline is to be presented in the inception report by the evaluator in a Gantt chart as part of the inception report. However, an initial estimate by the evaluation manager is outlined in this ToR. The evaluation team/consultant will be engaged for 27 working days between April and May 2016. The following work break down shows how the days will be distributed as 27 days of work and 3 days for travel. It summarizes tentatively the timing of key evaluation events [Note that the number of days allocated for each phase is only indicative]: Activities Duration Deliverables Briefing at the FRC or by skype (Helsinki and Harare) (team leader only) 1 day Briefing with the relevant FRC HQ staff. Background information Desk review 2 days Data collection and analysis 7 P a g e
Activities Duration Deliverables Development of inception report - and data 2 days Inception report collection/analysis plan and schedule, draft methodology, and data collection tools. Supervising of endline survey (MRCS implementation) or can be included in the in-country activities 1-2 Separate report, annex to main report analyzing the findings. In-country evaluation: Briefing on the context and background at the NS HQ. Evaluation team building and evaluation programme. Preparation of data collection tools. In-country evaluation: interviews, field trips. Debriefing workshop on initial findings, conclusions, and recommendations at the NS HQ. Data analysis and drafting evaluation report. [Agreement on the timeline to finalise the report within the evaluation team]. Submission of the draft final report to the FRC. [Debriefing at the FRC HQ to present the main findings, conclusions, and recommendations. A presentation of the evaluation report at the FRC IOP.] Revision and submission of the final evaluation report by the Team Leader/Consultant. Travel Total 2 days 7 days Completed questionnaires and other data collection tools. Endline survey Key informant interviews 1 day Debriefing presentation. Comments from the NS. 7 days Draft version of evaluation report. 1 day Final draft of evaluation report. First comments from the FRC. 1 day 3 days 27 days The scheduled timeline for the final evaluation report is: The draft final evaluation report presenting the main findings, conclusions and recommendations is to be ready within two weeks from return from the field for comments from FRC and MRCS. Comments for the draft report to the Team Leader/Consultant by Submission of the final evaluation report on 8. Evaluation Quality and Ethical Standards. The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members, and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards of the IFRC. The IFRC Evaluation Standards are: Utility: Evaluations must be useful and used. 8 P a g e
Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost effective manner. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders. Transparency: Evaluation activities should reflect an attitude of openness and transparency. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate. Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation. It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality. Further information can be obtained about these principles at: www.ifrc.org/what/values/principles/index.asp 9. Evaluation Team and Qualifications. The evaluation team will consist of a maximum of 3 participants/consultants with the following expertise: The external Team Leader (evaluator) shall have Solid evaluation expertise and experience in conducting evaluations. A Food Security specialist, with a minimum of 10 years of experience in the sector, preferably at least a part of it in rural Africa. Proven experience in evaluating development co-operation programmes or projects, incl. analyzing development impacts and using participatory approaches to evaluations. Preferably at least 2-3 reference projects, each reference being at least 20 days long. The evaluation team will be the following: 1. Team leader; 2. Team member 1; proposed MRCS PMER manager 3. Team member 2; a Red Cross Movement representative with a substantial knowledge in FS, HIV/AIDS, Health, Nutrition, a knowledge applicable for the Mwanza project (still in consideration depending on Team leader profile) In addition, the following person will contribute: MRCS focal person, not participating in the field work Interpreters (2), with fluency in English and Chichewa, experience in interpreting, recruited by the MRCS from outside of the project team and area. 9 P a g e
Drivers and facilitators, MRCS District/HQ staff members, assigned by the MRCS The evaluation team shall jointly have: Experience with technical knowledge of relevant programme delivery using community based and participatory methods in developing countries. Solid knowledge and experience of project monitoring and evaluation methods and approaches. Solid knowledge on gender and diversity aspects in development programming. Proven experience in evaluating development co-operation programmes or projects,incl. analyzing development impacts and using participatory approaches to evaluations. Excellent analytical, writing and presentation skills. Sound knowledge of the Red Cross and Red Crescent Movement and it works preferred. Good knowledge of written and spoken English The Team Leader will be responsible for the coherence of the evaluation report. The evaluation team will report to the Evaluation Manager at the FRC. 10. Application Procedures. The applicants are requested to send their applications (max. 2pages with CV) and to indicate the consultancy fee for the total length of mission of 27 days including the field trip of 12 days. The FRC will cover the costs (except per diems) during the field trip according to the Finnish government travel regulations (Valtion matkustussääntö 2016). The consultant is responsible of organising adequate travel insurance for the field trip by him-/herself. Applications invited with updated CV by 26.4.2016 to recruitment@redcross.fi, Annex 1 Responsibilities and tasks of team members and National Society focal persons Roles Team leader Responsibility Initial familiarisation with the project through briefing with FRC Further familiarisation through project and background documents Development of inception report, sharing with FRC for approval Communication with FRC and MRCS focal persons about practical arrangements and progress Division of roles and responsibilities with the other team member Leadership throughout the evaluation process Responsible of in country brief before departure Adherence to partner NS security guidelines Responsible of compilation of draft report and sharing with FRC and ERCS Responsible of compilation of final report If feasible and agreed a presentation of evaluation findings at the FRC HQs Familiarisation through project and background documents 10 P a g e
Team members (2) MRCS focal person: PMER Manager Participation to the evaluation process based on responsibilities agreed with the team leader Adherence to partner NS security guidelines Contribution to writing of draft report as agreed with the team leader Contribution to in country debrief as agreed with the team leader Represent MRCS during the evaluation in line with MRCS Evaluation Guidelines. Commenting to ToR of the evaluation Compilation of relevant project and background documents and sharing with the team All field arrangements, logistics, admin, meetings etc Communication with relevant stakeholders about the evaluation and timeframe Participation to the debrief by the evaluation team, including security brief Contribution to draft report by comments Approval of the report, with the core team Sharing of report with relevant stakeholders in country Ensuring the set budget and finance and procurement procedures are kept FRC HQ focal person: Regional focal person: Development of evaluation TOR and overall budget Identification and recruitment of the evaluation team members Briefing of team leader and if feasible team members Compilation of relevant project and background documents and sharing with the team leader and members Administrative arrangements for travel (flights, visa, insurance etc) Communication with MRCS about in country arrangements Contribution to the draft report by comments Debrief with the team leader Organisation of presentation of the report in Helsinki when possible Sharing of report with relevant stakeholders within and outside of the organisation 11 P a g e