The 2017-2019 Evaluation Programme of the Institute for Development Cooperation Thematic focuses, ongoing and scheduled evaluations April 2017 The evaluations conducted by the Institute for Development Evaluation are formalised by the multiannual and rolling evaluation programme of DEval, which is updated on a yearly basis. The information in the attached tables provides an overview of the ongoing evaluations and the evaluations scheduled to begin in the period 2017-2019. The overview is subdivided into three main parts. The first part shows all ongoing evaluations from the 2016-2018 evaluation programme; the second part outlines evaluations which begin in 2017 and the third part lists evaluation projects scheduled to begin in 2018 or 2019 according to current plans. The consultation and planning process under the DEval Evaluation Programme The multiannual Evaluation Programme serves as a medium-term strategic framework for DEval s core activity. In order to strengthen the relevance, legitimacy and process reliability of evaluations conducted by DEval, the institute engages in a systematic consultation process on programme content and the thematic focuses over two or three years. In this consultation exercise, which is conducted each year before submission of the annual programme update for ministerial approval, representatives of the Federal Ministry for Economic Cooperation and Development (BMZ) as well as members of the DEval Advisory Board (including members of parliament and representatives of civil society organisations) are asked for their proposals. All proposals, along with inhouse ideas, are then assessed on the basis of the established evaluation criteria and examined for consistency with the agreed thematic focuses. There must also be an appropriate balance of different evaluation types. The planning process results in a DEval programme proposal, which is presented to the institute's Advisory Board and requires final ministerial approval. The 2017-2019 Evaluation Programme embraces three thematic focuses with the aim of generating cross-cutting evaluation findings in highly relevant fields. The three thematic focuses are: 1) 2) Fragile states, peacebuilding and conflict prevention 3) Sustainability as a cross-sector issue in 1
Ongoing evaluations and evaluations scheduled for the period 2017-2019 PART I: Ongoing evaluations from the 2016 Evaluation Programme PART II: Evaluations from the 2017-2019 Evaluation Programme to begin in 2017 PART III: Evaluations from the 2017-2019 Evaluation Programme scheduled to begin in 2018 or 2019 PART I: Ongoing evaluations from the 2016-2018 Evaluation Programme Topic Background and relevance of the evaluation according to DEval criteria (annex) Subject of the evaluation and current status Thematic focus Philippines: Environment & Rural Development Background: interlinkage of topics relevant for policy (land use planning / decentralisation, adaptation to climate change, disaster risk protection); high potential for scaling-up and replication, great potential for learning and innovation; methodologically advanced impact evaluation. Subject: support of local land use plans through technical aiming at improved adaptation to climate change, disaster risk protection and rural. Status: reporting phase 1. & 2. Sustainability Criteria: no. 2, no. 3 Evaluation of sustainability in Background: high political relevance of sustainable policies and strategies, especially in the context of the 2030 agenda; Sustainable is one of the main challenges for implementation as well as for evaluation of projects and programmes. Subject: quality assessment of the measurement of sustainability in project evaluations; evaluation synthesis of existing evaluation results; methodological and enhancement of existing evaluation standards. Sustainability Status: report 1- meta evaluation: reporting phase Status report 2- evaluation synthesis: reporting phase 1
Action plan for the inclusion of persons with disabilities Background: inclusion of persons with disabilities as a major cross-cutting challenge for. Changes in the (national and international) normative framework; action plan to expire at the end of 2017; action plan clearly calls for an assessment of status of implementation and an evaluation. Subject: implementation and outcome evaluation of the action plan for the inclusion of persons with disabilities (adopted by BMZ in 2012) with regard to its three strategic goals. Status: reporting phase Stand-alone with reference to 1. 2. Sustainability Returnees of the volunteer service weltwärts and their civic engagement in y Effectiveness and sustainability of budget support - Synthesis and exit evaluation Background: politically and publicly highly visible instrument which is continually being further developed; the effectiveness of returned volunteers (in y) is to be evaluated in detail due to its high relevance against the backdrop of the SDGs and their focus on sustainable within y. Background: relevant instrument of ; increasing number of individual evaluations permits aggregated findings; methodologically demanding analysis. Uncertainties about implications of progressive withdrawal of bilateral funding from this instrument for impacts and established dialogue and coordination mechanisms at country levels. Subject: effects of the volunteer service on attitudes and behaviour of individual participants as well as effects of the -related civic engagement of former volunteers after their return to y. Status: reporting phase Subject: synthesis (aggregation) of existing evaluation results, implications of withdrawal of budget support for achieved impacts, coordination and dialogue mechanisms Status: report 1- synthesis evaluation: reporting phase Instruments structures 1. and 2. Sustainability and of Effectiveness of in dealing with Criteria: no. 2, no. 3, no. 4 Background: politically highly relevant topic for BMZ and the federal government as a whole. The linkage between managing short term refugee crises and long-term support for refugees (structural ) is one of the main challenges of. Status: report 2 - exit evaluation: data collection phase Subject: portfolio analysis of interventions dealing with conflict-driven migration crises, success factors Phase I: analysis of international evaluation literature on the linkage between humanitarian aid and long-term 1. Fragile states, peacebuilding and conflict prevention; 2. April 2017 2
conflict-driven migration crises Criteria: no. 1, no. 2, no. 3, no. 4 support in conflict-driven migration crises; BMZ / portfolio analysis and initial explorative field research Status: reporting phase (phase I) Phase II: in-depth analysis of the effectiveness of regarding conflict-driven migration crises in various case study countries / crisis regions Waste water sector Vietnam Background: complex challenges in planning, supervising and managing broad multiannual technical and financial programmes (esp. infrastructure sector); high relevance regarding instrumental and institutional aspects of Status: conception phase Subject: evaluation of a long-term technical and financial programme regarding the construction and operation of wastewater treatment plants in provincial towns in Vietnam Attitudes towards and sustainable in y Criteria: no. 1, no. 3, no. 4 Background: The current focus on sustainability in as well as the increasing number of issues at an international level led to an increasing demand for empirical findings on relevant attitudes towards policy issues in y. The study provides orientational knowledge to improve both the drafting and communication of policies while at the same time it strengthens the feedback loop between the population and policy makers in y which is very important for the legitimacy of this political area. Status: Synthesis phase/reporting phase Subject: attitudes, knowledge, and behaviour of the population in the field of policy and, sustainable as well as the sustainable goals (SDGs) of the Agenda 2030 Status: synthesis phase Sustainability April 2017 3
Programme evaluation in the SDG era Background: evaluation gap in the system of evaluation concerning country portfolios and programmes which leads to considerable evidence and management deficiencies. Subject: Through the pilot review on the thematic focus of sustainable economic in Nepal it has to be shown to what extent the evidence base for portfolio reviews can be improved by evaluations on thematic focuses; elaboration of a respective standard process based on these findings and other pilot evaluations at a country portfolio level; Sustainability Cooperation with the private sector in agriculture Criteria: no. 4 Background: high relevance to BMZ s thematic focus of rural, food security and integration of private sector; politically visible and controversial approach of ; Criteria: no. 1, no. 3, no. 4 Status: pilot review data collection phase Subject: evaluation of programmes and measures of to promote with the private sector in agriculture implemented between 2006 and 2016 Status: conception phase Stand-alone with reference to instruments and PART II: Evaluations from the 2017-2019 Evaluation Programme scheduled to begin in 2017 Topic Background and relevance of the evaluation according to DEval criteria (annex) Subject of the evaluation and current status Thematic focus Contribution of to gender equality in fragile contexts Background: high political relevance due to interlinkage between two particularly important cross-sectoral topics of ; low coverage in terms of existing evaluations; of political and strategical interest, particularly in the context of the forthcoming revision of the Gender Action Plan. Subject: contribution of to gender equality in fragile contexts Fragile states, peacebuilding and conflict prevention April 2017 4
Topic Background and relevance of the evaluation according to DEval criteria (annex) Subject of the evaluation and current status Thematic focus Evaluation synthesis of in countries affected by conflict and/or fragility Background: high political relevance of in countries affected by conflict and/or fragility; substantial challenges for the implementation and evaluation of Criteria: no. 2; no. 4 Subject: quality assessment of evaluation practice in fragile states; meta study/analysis of evaluation results; methodological of evaluation standards Fragile states, peacebuilding and conflict prevention Trilateral in Background: high political relevance to assess the strategic direction, efficiency, potentials and risks of this specific form of with emerging countries for the benefit of third beneficiary countries Subject: Increasing participation of emerging countries in facilitates a new form of by which the comparative advantages of emerging and industrial nations are combined to achieve better results. Apart from the objective to improve the efficiency of programme activities, is also aiming at creating a common understanding of policy with emerging countries and to open the dialogue about standard processes, criteria and values. Institutional evaluation of Engagement Global Background: high political relevance of civic and municipal commitment. Assessment of the set-up and positioning of Engagement Global as a service provider for civic and municipal commitment; high innovation and learning potential Subject: implementation of the BMZ strategy regarding focus and activities of Engagement Global ggmbh as a service provider for civic and municipal commitment. April 2017 5
Topic Background and relevance of the evaluation according to DEval criteria (annex) Subject of the evaluation and current status Thematic focus Criteria: no. 1, no. 2, no. 3, no. 4 from a partner perspective (Phase II) Background: high political and strategic relevance in the context of current and future performance of the system; by using new data and a mixed-methods approach, perceptions by decision makers in developing and emerging countries of the performance of actors will be assessed systematically. Subject: This joint study based on an international survey on experience-based perceptions of decision-makers in partner countries regarding the performance of official compared with other countries, creates systematic evidence, which shall be complemented by case studies in selected countries Criteria: no.1, no. 2, no.4 Risk sharing instruments for mobilising private capital to finance the SDGs (Structured funds) Background: high political relevance to assess instruments for mobilising additional private capital to finance the Agenda 2030. The high potential of mobilising funds has to be compared with the benefits for objectives. So far, only few evaluations exist on the impact of this instrument. Subject: effectiveness of structured funds as an instrument of financial. Analysis of the function of risk sharing regarding the financial and al benefit and respective risks. Criteria: no. 1, no. 2, no. 3, no. 4 TEIL III: Potential evaluation projects scheduled to begin in 2018 or 2019 (To be re-examined in the 2018 consultation process) Thematic focus report on sustainability Follow-up study on the attitudes towards and sustainable in y Promotion of municipal commitment April 2017 6
Evaluation of the Caribbean Bio Diversity Fund Sustainable economic Myanmar Country programme evaluation Afghanistan (at a cross-departmental level) Annex: Explanation of the evaluation criteria There is a two-step assessment process to identify an evaluation project for inclusion in the multiannual Evaluation Programme. First, project content is considered against the following inclusion criteria to determine acceptance for the Evaluation Programme: 1. Coverage: evaluation density in relation to the overall scope of the portfolio 2. Innovation and learning potential: availability of empirically sound knowledge of effectiveness, especially with a view to future engagement 3. Risk: probability or existence of concrete risks of any sort 4. Strategic importance: advisory or decision-making requirements (on the part) of BMZ and/or political importance: high policy visibility of the field to be evaluated Further selection criteria are evaluability, efficiency and coherence of the potential evaluation projects. Evaluability refers to the logistical, methodological and financial feasibility of the evaluation project (mainly with DEval's own human resources). Efficiency, i.e. whether an evaluation is the best and most cost-effective way of meeting the respective information and knowledge needs Coherence refers to the coherence of the evaluation programme by taking into account the current thematic focus April 2017 7