Terms of Reference for Independent Progress Review (IPR) of Practical Action s Partnership Programme Agreement (PPA)

Transcription

1 Terms of Reference for Independent Progress Review (IPR) of Practical Action s Partnership Programme Agreement (PPA) Introduction to Practical Action; Practical Action believes that technology and innovation are a vital contributor to people s livelihoods. The capabilities of poor people to access, use and adapt appropriate technologies, knowledge and skills underpins their ability to escape poverty. To this end our work is founded on working alongside communities using approaches that support collaboration and shared learning, and using these experiences as the basis for wider knowledge services to development actors through our Practical Answers, Publishing and Consulting arms, and as the foundation of our efforts to influence the policies and practices of others. Practical Action was established in 1969 by E.F. Schumacher, the radical development economist and author of Small is Beautiful. We currently work in 11 countries with offices in Kenya, Sudan, Zimbabwe, Peru, Nepal, Bangladesh and Sri Lanka. Our Head Office is located near Rugby, Warwickshire, UK. We have more than 600 permanent staff and a turnover of US $ 30 million per annum. Practical Action has a range of implementing and strategic partners in each of our countries and major donors including the European Commission, the Department for International Development (DFID), multi- and bi- lateral agencies, trusts and supporters. Further details of Practical Action could be seen at its website; About our Partnership Programme Agreement (PPA); In mid-2011, the UK Department of International Development (DFID) entered into PPA arrangements with a number of Civil Society Organisations (CSOs), including Practical Action. A full list of PPA partners and further details are available at the DFID-PPA link below; Practical Action s PPA contributes across the organisation and supports our contributions to Millennium Development Goals 1, 3, 7 and 8. The expected outcomes are to improve access to technologies to enhance the resilience and adaptive capacity of poor women and men and their communities across four outputs; Increased representation of poor women and men in organisations and processes at the sub-national level that enhance their ability to access technologies, services, natural resources and markets and to mitigate risks. Poor women, men and children vulnerable to climate change, poverty and/or fragile states increase their access to material benefits and services as a direct result of Practical Action support 1

2 Increased use of Practical Action knowledge services by individuals, enterprises, development practitioners and decision makers in the North and South Increased capability of national governments, international institutions, private sector, practitioner organisations and/or civil society groups to deliver proven technologies and approaches at national and international levels as a direct consequence of Practical Action partnerships and support. Each of the above outputs have indicators, milestones, targets and assumptions set out in our organisational logframe. A copy of the Practical Action s logframe will be provided to the shortlisted organisations/ individuals. DFID Guidance on Independent Progress Review (IPR) Every year, all the PPA holders are expected to report to the DFID under a common Terms of Reference (Attached as Annexe 1). DFID appointed Evaluation Manager, currently Coffey and Associates, will be responsible for assessing the performance of individual grantees and of the funding mechanism as a whole. The common TOR outlines the purpose, approach and methodology of the IPR, as a guidance. In terms of grantees performance, the overall Evaluation is concerned with: the extent to which grantee organisations are performing against their objectives the extent to which grantee organisations and achievements align with DFID s theories of change the impact of DFID s funding in terms of the additional benefits realised because of funding and its attributable contribution to organisational effectiveness and the results set out in grantees logframes. The impact assessment will consider the value for money organisations derive from DFID funding. The IPR will be carried out under the overall guiding question; What has happened because of DFID funding that wouldn t have otherwise happened? and To what extent does the use of funding represent good value for money? Evaluation Purpose, Objective and Scope The purpose of this evaluation is to determine whether Practical Action s use of PPA funds has confirmed the common vision of the Agreement as outlined in the logframe, business case and the memorandum of understanding with DFID. The objective is to appraise the extent to which Practical Action have achieved the agreed Objectives and indicators agreed with DFID. The evaluation should also derive lessons about the relevance, efficiency, effectiveness and sustainability of Practical Action s delivery of organisational impact. In particular we seek learning that will help shape and inform operationalizing our new strategy, organisational set-up, partnerships at all the levels and programme focus. The evaluation will also use the recently completed annual impact reviews ( ), evaluation of the previous PPA and Annual Progress Review submitted to DFID. 2

3 The main audience and users of the evaluation findings will be Practical Action, the Civil Society Team at DFID, and a group of Practical Action s international stakeholders. To maximise learning, Practical Action would also like to actively engage with the BOND Effectiveness group to share some findings. Methodology The core methodology of the IPR should adopt the key steps of the Contribution Analysis as in Section of the DFID guidelines. This includes; 1) Establish a Theory of Change with Practical Action and the risks associated with it 2) Set out the attribution problem to be addressed i.e. cause and effects relationships 3) Gather existing evidence on the theory of change 4) Assemble and assess the contribution narrative and challenges to it 5) Gather additional evidence To achieve the above the consultant is suggested the following activities; Set up a series of meetings with senior Practical Action staff to understand the structure and range of initiatives included under the framework agreement and fund allocations; Examine Practical Action s stated impacts as outlined by its annual reviews and draft impact report Analysis of a sample to appraise the reliability and efficiency of the organisation s validation trail from projects, through programmes to the Regional and Organisational levels, including methods of data/ information collection and record keeping at different levels; Visit and assess a shortlist of project and programme evaluations to cross reference the PPA Self Assessments and gauge the integrity of Practical Action s systems and learning; To interview a selection of stakeholders including international programme staff and partners, staff in support services, project evaluators, government officials, and DFID s Policy and Civil Society teams to examine the effectiveness of our contribution in key Outcome areas of programmes, knowledge and influencing; Cross reference the stakeholder interviews against the results of an independent partner effectiveness analysis, to be undertaken by KeyStone, with a shortlist of local partners 1 ; 1 Dependent on the KeyStone timetable and currently under negotiation. If this is not possible, we will ask the evaluator to convene a similar process 3

4 Make recommendations to Directors and senior staff about issues and areas for the coming strategy round including Practical Action s comparative advantage, focus and structure; and approach to cost effectiveness and the performance monitoring of Outcomes. Initial negotiations with the IPR consultant or consulting firm will be for an estimated 35 person days contract with at least two regional visits. Skills and Qualifications The IPR will be carried out by a suitably-qualified and experienced consultant, small team of consultant or a consulting firm. The individuals/ team must have the following key skills and qualifications; Specialists/ team of specialists with at-least seven years of experience in programme/ project delivery in an international development context An understanding of the theory of change approach, contribution analysis, numerical and qualitative data management and analysis A comprehensive background of working with civil society organisations in the field of international development including significant hands-on overseas experience Up to date experience and insight into areas of value addition and value for money, ideally including training and use of Strategic Return on Investment and / or similar processes, and; A strategic, cross-sectorial capability for unpicking and simplifying complex systems. Some knowledge of Practical Action, our Mission and Values, organisational approach and international programmes. The selected consultants will be working and reporting to the Programmes team of Practical Action. In addition, s/he is also expected to work with the Evaluation Manager appointed by DFID, relevant contact points within DFID and representatives of other Civil Society Organisations. Outputs The evaluator will produce the following outputs; 1) The draft and final versions of the main report, not exceeding 30 pages (excluding the annexes) 2) All the annexes as agreed with Practical Action, including the Annexe with the criteria as outlined in Section 3 of the DFID IPR guidelines 3) A 2 page summary of the report to share with our country and regional offices and partners 4) A presentation made to Practical Action s PPA staff and senior management about the findings of the IPR 5) A 5 minutes video recording presenting the key findings 4

5 Copies of all the photos, audio-recordings and videos taken by the consultant will also be submitted to Practical Action. The report will follow Practical Action s Brand Guidelines, and be presented with multiple stakeholders in mind. The evaluator will be invited to present their approach and findings to senior staff and Trustees before, during and after undertaking process. The report will include the following sections: 1. Summary (1 page) will outline the key findings and state how far the evaluator feels Practical Action have achieved the Partnership Objectives. 2. Methodology (3 pages) will outline the approach undertaken by the evaluator, including details of how the contribution analysis was carried out and selection of the in-country programmes 3. Results (General 8 pages) will provide a progress review in terms of: Relevance; Effectiveness; Innovation; Partnership Approach; M&E and Efficiency This Section of the report must be arranged according to the criteria, cub-criteria and definitions provided in Annexe This has been adapted from the OECD guidelines and further details from the original source could be seen at the link below; 0.html 4. Results (Specific 8 pages) will provide a progress review in terms of: Performance against the LogFrame: To what extent is the organization achieving (or progressing towards) the intended outcomes? Changes in lives. Assess the information about what changes these outcomes are making in people s lives and how many people are affected. Changes in civil society. To what extent are citizens doing things for themselves (for example community organizations managing and delivering services)? To what extent is civil society enabled to hold government to account? Assess what conditions led to success and failure external, internal combination of interventions. To what extent does DFID funding achieve additionality, i.e. enable CSOs to achieve things they would have otherwise not been able to achieve? Assessment of additionality will be covered during the impact assessment as described below. 5

6 5. Organisational Development (2 pages) will outline the PPA s impact on the overall performance and integrity of Practical Action s cross cutting knowledge systems; 6. Value for Money (2 pages) will draw out specific evidence to show how cost effectively Practical Action used DFID s PPA inputs; 7. Governance (2 pages) will include details of Practical Action s procurement processes and procedures, and the measures in place to manage overall fiduciary risk; 8. Building Support for Development (2 pages) should outline the progress Practical Action has made against toward: (i) Increasing public knowledge and awareness of global poverty through Development Education; and (ii) Increasing international support and momentum for action to increase energy access for the poor while reducing global carbon emissions (both Outcome 4). 9. Issues and Recommendations (2 pages) will highlight areas arising during the evaluation which the evaluator feels should to be addressed by Practical Action and/or DFID (see also Annex 7) and to be reviewed by the IPR in the year Annexes In submitting the final report the following annexes will be added: Terms of Reference; Timeline of the evaluation process; People interviewed, their organisation and perspective (table); Reviewed documentation (list and references); Practical Action s theory of Change and working model Annexe with DFID criteria on the evaluation Practical Action Management Response. More Annexes may be included as necessary. Timetable Selection of the IPR consultant by 5 th August, 2012 Initial briefing, discussions in the UK and evaluation design early August 2012 Review of the available reports, literature and UK workshop August, 2012 Presentation and discussion of the full IPR methodology August, 2012 Visits to two countries programme early August and September, 2012 Submission of the first draft report September,

7 Submission of the final draft September, 2012 Presentation of the findings in the UK, October 2012 The final evaluation report agreed by Practical Action will be submitted to DFID together with the 4-page Management Response by no later than 10the October, 2012 Bidding Details; Interested consultants/ individuals/ groups are requested to send in an expression of interest (EOI) by the mid-night of 28 th July to Mansoor Ali, Monitoring and Evaluation Adviser. The EOI must include the following; 1) A 2 page note explaining why you consider yourself as the suitable candidate for this assignment 2) A one page each CV of the staff who will be available and actually working on the assignment, if possible divided as per different key tasks e.g. UK meetings, field visits etc. 3) An outline workplan, not exceeding 1 page. 4) An outline budget, including only the professional fees of various staff All the Direct Costs will be paid as per actuals, in line with DFID PPA guidelines The selected organisation/ individual will be informed by 5th August, Annexe 1; 7