Size: px
Start display at page:

Download ""

Transcription

1 Terms of Reference End of Programme Evaluation of CAFOD s UK Aid Match Round 2 Programme: Climate Resilient Agriculture: transforming the livelihoods of 450,000 poor women, men and children in marginalised, rural areas of Bangladesh, Kenya, Myanmar and Zimbabwe 20 March Background The Catholic Agency for Overseas Development (CAFOD) is the official development and humanitarian relief agency of the Catholic Church in England and Wales. CAFOD s mission is to promote human development and social justice in witness to Christian faith and gospel values by supporting long-term development, advocacy and humanitarian relief. CAFOD has been implementing a 36-month programme entitled Climate Resilient Agriculture: transforming the livelihoods of 450,000 poor women, men and children in marginalised, rural areas of Bangladesh, Kenya, Myanmar and Zimbabwe, funded by DFID through the UK Aid Match (Round 2). The UK Aid Match programme brings charities, the British public and the UK government together collectively to change the lives of some of the world s poorest people. Through Aid Match, the UK public can engage with international development issues and have a say in how UK aid is spent, by choosing to support an organisation s appeal. For every 1 donated to a UK Aid Match charity appeal, the government will also contribute 1 of UK aid towards the project 1. Following a successful UK Aid Match Appeal, in which CAFOD raised 5m from its supporter base, UK Aid awarded a grant of 5m to implement this climate resilient agriculture programme in Bangladesh, Kenya, Myanmar and Zimbabwe. The programme began in July 2015, and will be completed by 30 June 2018, with a total programme budget of 5,342, Programme purpose and objectives The programme aims to achieve improved livelihoods security for poor women, men and children from hazard-prone and/or marginalised rural communities in Bangladesh, Kenya, Myanmar and Zimbabwe through an integrated approach that: Increases the productivity and resilience of agricultural systems; Consolidates engagement in markets and diversified income generation; Strengthens adaptation to climate change; Increases the ability of local small-scale food producers to access services and influence development practices. The programme has been working towards the following impact and outcome statements: Impact: Poor women, men and children from hazard-prone or marginalised rural communities in Bangladesh, Kenya, Myanmar and Zimbabwe adapt their livelihoods strategies to better cope with climate-related shocks and stresses. Outcome: Improved resilience for approximately 80,000 households (450,000 men, women and children) in Bangladesh, Kenya, Myanmar and Zimbabwe. Target households are broken down as follows: Country Households Individuals Bangladesh 36, ,561 Kenya 16,151 96,908 Myanmar 4,479 20,022 Zimbabwe 23, ,

2 3. Purpose of the Evaluation In addition to CAFOD s own corporate commitment to capture learning, CAFOD is also contractually obliged to conduct a final evaluation on programme activities funded by the UK Government grant. To comply with our contract, CAFOD must commission an evaluation that will focus on the results achieved, efficiency, effectiveness of implementation and quality of administration 2. To this end, the specific purpose of the final evaluation is to: a. Identify the effectiveness and impact of the programme and recommend ways this can be improved and sustained; b. Assess the extent to which the impact and outcome statement was achieved; c. Assess the extent to which the programme targeted and met the needs of the poorest and most vulnerable; d. Record and share lessons learned; e. Verify whether the funds were used effectively and efficiently to deliver results; f. Identify, measure and record any distinctive or value-added contribution of CAFOD to the programme (beyond any CAFOD funding that was used), e.g. CAFOD s partnership approach with local civil society; g. Account to DFID (UK Aid Match team) and local stakeholders for the programme s performance. 4. Evaluation questions The following evaluation questions represent only an indicative list at this stage, but will broadly fall into the 5 key categories listed in the OECD DAC guidance as well as covering CAFOD priority areas, including replicability, lesson learning and recommendations Relevance: details of the programme s significance with respect to specific needs and its relevance to country poverty reduction priorities How effective was CAFOD s needs assessment of the target population? How well did the programme relate to the country s poverty reduction plans, disaster risk reduction plans, and other national and district plans? To what extent have livelihoods adapted and households flourished because of the training and inputs from this programme? To what extent are households using this knowledge to adapt their livelihoods strategies, and to what extent were the activities locally appropriate to foster a sustained improvement in their capacities to respond to climate shocks? 4.2. Effectiveness: Assessment of how far the intended outputs and results were achieved in relation to targets set in the original and revised logical framework How effective and appropriate was the programme approach? Were there processes which supported, or conversely, hindered effectiveness? 4.3. Efficiency: How far funding, personnel, regulatory, administrative, time, other resources and procedures contributed to or hindered the achievement of results How well did the partnership and management arrangements work and how did they develop over time? How well did the financial systems work? Were risks properly identified and well managed? How was value for money considered in programme decisions? What processes were in place to ensure good value for money? Were unit costs appropriate? How was the project managed, and how were decisions made? 4.4. Impact: Details of the broader economic, social, and political consequences of the programme. 2 Clause 28 in Revised Accountable Grant Agreement with DFID, May 2017.

3 What we the programme s overall impact and how does this compare with what was expected? Who were the direct and indirect/wider beneficiaries of the programme? What difference has been made to the lives of those involved in the programme? 4.5. Sustainability: Potential for the continuation of the impact achieved and of the delivery mechanisms, following the withdrawal of external support What as the prospects for the benefits of the programme being sustained after the funding stops? Did this match the intentions? How has/could collaboration, networking and influencing of opinion support sustainability? How was the exit strategy defined and how is this managed at the end of the funding period? 4.6. Replicability: How replicable is the process that introduced the changes/had impact? Please refer specifically to innovative aspects which are replicable What aspects of the programme are replicable elsewhere? Under what circumstances or contexts would the programme be replicable? What is the potential to disseminate and scale up innovative aspects of the programme, and who would the audience be? 4.7. Lessons: Identify key lessons that can be used to guide future strategies, projects or agencies working in development. These should be divided into project, sector and broader development lessons Were there any significant changes in the programme design or the programme context? What are the reasons for these and can any useful lessons be learned from this for application elsewhere? How did the programme engage with poor and marginalised groups and support their empowerment most effectively? For whom can these lessons have relevance? How do the lessons relate to any innovative aspects of the programme that were highlighted in the programme proposal? How has the design of the programme been amended as a result of lessons learned during implementation? How effectively was advocacy integrated into the programme? What were the challenges and advantages to the approach? What are the lessons for how this is done in the future? 4.8. Recommendations: Recommendations for improvements based on observations during the evaluation process (e.g. for sustainability, future programme design and management). 5. Methodology Interested parties will be asked to tender a short outline methodology of how they will approach this evaluation, both on a theoretical and practical basis. This should include: A desk review of programme information, including a list of key documents, to be provided; Interviews with key programme staff (CAFOD and implementing partners); Interviews with project beneficiaries; Interviews with key programme stakeholders; Presentation of preliminary findings to CAFOD programme team, to allow for their comments and feedback before the draft evaluation is completed; Submission of draft evaluation report to CAFOD UK Aid Match Co-ordinator for consultation and written comments before finalising the report; The selected evaluator will be expected to work collaboratively with CAFOD to refine the methodology and develop a detailed evaluation plan. The evaluator will report to Michelle Brady, UK Aid Match Co-ordinator, responsible for organising the evaluation for this programme.

4 The final evaluation schedule, including site visits will be organised by CAFOD s UK Aid Match Co-ordinator, in consultation with the evaluator and CAFOD programme staff in Bangladesh, Kenya, Myanmar and Zimbabwe. An indicative evaluation timescale is laid out in section Deliverables Required outputs include: An evaluation plan; A presentation detailing initial evaluation findings, for face-to-face discussion with a group of key CAFOD staff; A first draft report submitted to the CAFOD UK Aid Match Co-ordinator for consultation; A final evaluation report of publishable quality, in MS Word and PDF format. The report should be written in plain English, and in such a way that it is accessible to non-specialists, including UK taxpayers and CAFOD supporters; All figures, diagrams, tables, etc. to be provided separately in high quality jpeg or png files; Copies of all photographic material (please note these cannot be used outside of CAFOD without permission). The final evaluation report should be no more than 30 A4 pages long plus appendices (in Microsoft Word using Arial font, 12 point). The report should include the following sections: Title page; Contents page; Abbreviations and acronyms page; Executive summary (1 A4 page maximum); Completed Achievement Rating Scale (5 A4 pages maximum - see template in Annex 3). Please note that the overall achievement rating should have a score and a comment only; A short introduction to the programme; The evaluation methodology; Full evaluation of programme: Findings from the evaluation in relation to the final agreed evaluation questions. The report should be well-signposted to enable quick access to recommendations; and should include a brief description of methods; A summary of recommendations; A one-page summary of lessons indicating with whom and how lessons should be shared; The final terms of reference for the evaluation must be included as an annex, as well as names and contact details of the evaluators along with a signed declaration of their independence from CAFOD and its programme partners; Other annexes should include: o Methods used in greater detail, and a basic explanation of any theoretical underpinnings to understand why this method was selected, its limitations and caution around interpretation. o Copy of interview or synthesised trip notes. o Copy of the final ToR. o List of staff and people consulted. o List of documents reviewed and consulted. o Statistical data on baselines and end of programme surveys. Note that the original and revised logical framework should also be included. o List of staff or consultants performing the evaluation and what each contributed. While it is anticipated that the evaluation will begin in May 2018, some elements of the indicative timeline below will be refined by CAFOD s UK Aid Match Co-ordinator in collaboration with the selected consultant(s) and CAFOD programme staff in Bangladesh, Kenya, Myanmar and Zimbabwe.

5 6. Indicative evaluation timetable Activity Date Tender bids submitted to CAFOD Wednesday, 11 April 2018 (23:59) Consultant Interviews w/c 23 April 2018 Refine methodology, finalise planning Friday, 4 May 2018 Evaluation field visits undertaken 14 May 15 June 2018 Presentation of initial findings to CAFOD incountry 14 May 15 June 2018 staff Near final draft report due Friday, 29 June 2018 (23:59) CAFOD feedback on nearly final report Friday, 6 July 2018 (23:59) Final evaluation submitted to CAFOD Friday, 13 July 2018 (23:59) CAFOD management response Friday, 20 July 2018 Evaluation report published online, along Monday, 1 October 2018 with any agreed ancillary products 7. Skills and competencies of evaluator/s CAFOD is looking for a lead evaluator with a strong record in conducting evaluations as well as direct programme interventions. The lead evaluator will need to carry respect and credibility within the development field and have an excellent knowledge of evaluation and monitoring in theory and in practice. Successful consultants will be able to demonstrate the following skills and experience: Demonstrable experience of producing high-quality, credible evaluations (an example will be required with the tender); Demonstrable experience of working with/evaluating civil society work; Demonstrable understanding of participatory methodologies; Familiarity with the following programme areas: a) livelihoods in rural settings in Africa and Asia; and b) development agriculture; Experience of relevant evaluation/development interventions in Bangladesh, Kenya, Myanmar and/or Zimbabwe; Good understanding of NGO finance and audit particularly of DFID guidelines and requirements (specialist finance qualifications not essential); Experience of managing evaluation teams, and the capability to handle necessary logistics and, if required and agreed, sub-contracting; Ability to produce concise, readable and analytical reports; An understanding of public communications; Excellent English written and verbal communications skills; Experience and understanding of faith-based organisations (desirable); DBS check. As the evaluation will require exposure to communities, the evaluation team will need to have up-to-date DBS checks in place at the commencement of the evaluation. All contracted personnel must comply with CAFOD s Codes of Conduct, which will be provided as part of the contract. 8. Available budget CAFOD is anticipating 6 weeks to carry out this evaluation, this will include travel and other costs. CAFOD can support on providing interpreters for field visits and will also help to organise the logistics of the field visits. 9. Tender process CAFOD invites bids from organisations, individuals or a team of individuals with the experience and skills described above. By way of indication, initial tenders should include: A cover letter introducing the evaluator/s/organisation and how the skills and competencies described above are met, with concrete examples.

6 A two-page outline of the proposed evaluation process, including - proposed outline methodology - management arrangements A clear indication of availability / dates A one-page budget covering all major costs, and clearly identifying daily rates charged on any consultancies. A CV for each member of the evaluation team (if there is more than one consultant in the bid). One example of a relevant previous evaluation (one each in the case of joint bids) 9. Criteria for selection will be: Clear, credible, structured proposed methodology; Ability to meet the criteria described in Skills and Competencies, section 6 above; Ability to manage the totality of the evaluation, including logistics, recruitment and management of other team members (where relevant); Commitment to availability in the critical periods; Value for money. 10. Further information Tenders should be sent to Michelle Brady, UK Aid Match Co-ordinator, via (mbrady@cafod.org.uk) by Wednesday, 11 April 2018 (23:59). Please direct any questions to Michelle via .