Independent Evaluation Department Work Program,

Transcription

1 FOR OFFICIAL USE ONLY R November 2017 Independent Evaluation Department Work Program, Attached for the consideration of the Board is a paper on the above subject. 2. In the absence of any request for discussion and in the absence of a sufficient number of abstentions or oppositions (which should be communicated to The Secretary by the close of business on 8 December 2017), the Independent Evaluation Department Work Program, will be deemed to have been approved, to be so recorded in the minutes of a subsequent Board meeting. Any notified abstentions or oppositions will also be recorded in the minutes. For Inquiries: Marvin Taylor-Dormond, Independent Evaluation Department (Ext. 5953)

2 Project Number: October 2017 Independent Evaluation Department Work Program, Distribution of this document is restricted until it has been approved by the Board of Directors. Following such approval, ADB will disclose the document to the public in accordance with ADB s Public Communications Policy 2011.

3 ABBREVIATIONS ADB DEC IED TA Asian Development Bank Development Effectiveness Committee Independent Evaluation Department technical assistance Director General Marvin Taylor-Dormond, Independent Evaluation Department (IED) Deputy Director General Veronique Salze-Lozac h, IED Team leader Team members Jose Antonio Tan III, Advisor, IED Renato Lumain, Senior Evaluation Officer, IED Ma. Catherine Malilay, Evaluation Assistant, IED In preparing any country program or strategy, financing any project, or by making any designation of or reference to a particular territory or geographic area in this document, the Asian Development Bank does not intend to make any judgments as to the legal or other status of any territory or area.

4 1. As specified in the 2008 policy paper Review of the Independence and Effectiveness of the Operations Evaluation Department, 1 the Board of Directors of the Asian Development Bank (ADB) is responsible for approving the work program of the Independent Evaluation Department (IED). 2. IED developed the rolling work program for in consultation with the Board s Development Effectiveness Committee (DEC), ADB Management, and selected departments. The work program takes into account IED s role in (i) providing timely input on the formulation of ADB strategies, policies, and programs; (ii) providing real-time feedback during loan processing and implementation; (iii) building evaluation capacity of regional departments and developing member countries; and (iv) producing strategically relevant knowledge on emerging regional development issues to inform ADB s operations and development policies in its developing member countries. The 3-year rolling work program aims to ensure that IED has stronger impact on ADB and its priorities and operations through high-quality evaluations; better alignment of work; and more effective communication and dissemination of evidence-based findings, lessons, and results. IED seeks to achieve this by (i) improving the balance of its product mix between macro and micro evaluations, (ii) strengthening the utilization of evaluation, (iii) addressing evaluation gaps and key risks to development outcomes of ADB s work, and (iv) strengthening the quality and credibility of evaluation work. 3. Building on the Board-approved work program for , and to increase the effectiveness and achieve better quality of evaluations, IED s work program includes new initiatives such as (i) reintroduction of sector evaluation work across and within countries, (ii) systematic assessments of key ADB corporate operational processes to improve development effectiveness, (iii) enhancement of country and project validations, and (iv) development of a validation system for technical assistance (TA) self-assessments. From 2018 to 2020, IED will also intensify its outreach, knowledge sharing, and evaluation capacity development activities within and outside ADB. 4. To strengthen its operational relevance, IED will be delivering four sector-wide evaluations in the next 3 years agriculture, transport, energy, and education and training. IED will reintroduce country sector work by preparing one sector assistance program evaluation per year. 5. IED will continue to scale up and undertake relevant private sector evaluations. It will deliver at least one private sector evaluation annually starting with private sector equity investment in 2018 and public private partnerships in An evaluation of the One ADB approach in 2020 will include ADB s engagement with private sector clients. With ADB s enhanced financial capacity as well as the upcoming Strategy 2030, 2 IED will provide evaluations on important business lines on state-owned enterprises and small and medium-sized enterprises. IED will also consider an evaluation on the extent to which ADB operations have helped to create jobs. 6. To improve ADB s institutional effectiveness, IED will undertake several evaluations addressing key ADB corporate operational processes during : (i) the effectiveness of the Safeguard Policy Statement, (ii) the delivery of knowledge, (iii) procurement, and (iv) the One 1 ADB Review of the Independence and Effectiveness of the Operations Evaluation Department. Manila (DEC R297-08). 2 ADB s enhanced financial capacity results from the merger of the Asian Development Fund and ordinary capital resources.

5 2 ADB approach. Complementing these evaluations will be special chapters in upcoming editions of the Annual Evaluation Review focusing on the quality of operations design and preparation for impact, and ADB s self-evaluation system. In response to policy requirements and institutional commitments, IED will deliver evaluations on policy-based lending, multitranche financing facility, and Asian Development Fund 12 operations. 7. IED s country, project, and TA work will be strengthened. Country program validations will be streamlined. All Board-circulated project and/or program completion reports and extended annual review reports will be validated. Recognizing the significant gap in systematic validation of TA self-assessments, IED will undertake a special initiative to develop a validation system for them, starting with a feasibility assessment in To be more responsive to development effectiveness issues affecting ADB, IED will set aside resources to meet requests from the Board and DEC for unanticipated and quick turnaround evaluations during the year. 9. On 17 October 2017, the DEC discussed IED s draft work program for and the 2018 budget. IED s proposed work program for , which was endorsed by DEC, is attached for Board approval.

6 3 INDEPENDENT EVALUATION DEPARTMENT S PROPOSED WORK PROGRAM, Program Area I. Cross-Cutting Evaluations Agriculture and Rural Development ADB Support for Agriculture (SE) Environment and Climate Change Governance, Public Sector Management, and Finance Sector ADB Support for Small, and Medium-Sized Enterprises (TE) ADB Support for State- Owned Enterprises (TE) Inclusive Growth and Social Development ADB Support for Gender and Development (TE) ADB Support for Education and Training (SE) ADB Support for Job Creation (TE) Infrastructure Development Contribution of ADB Sanitation Projects (TP) Experience with Donor Coordination: The Case of Water Supply and Sanitation in Sri Lanka (TP; Joint with World Bank Group and JICA) Impact Evaluation of Small Towns Water Supply and Sanitation Sector Project (Nepal) ADB s Transport Operations (SE) ADB Support for the Energy Sector (including the energy policy) (SE) Private Sector and/or Nonsovereign Operations Boosting ADB s Mobilization Capacity: The Role of Credit Enhancement Products (CE) ADB s Use of Private Sector Equity Investments (CE) ADB Support for Public Private Partnerships (TE) ADB Institutional Effectiveness and Instruments ADB s Results-Based Lending Pilot Policy requirement (CE) Knowledge, Finance, and the Quality of Growth: An Policy-Based Lending (CE) Multitranche Financing Facility (CE) ADF Operations (ADF 12), including Review of the Disaster Response Facility

7 4 Evaluative Perspective on Strategy 2030 (TP) and Regional Health Grants (CE) Development Effectiveness of ADB Processes Effectiveness of Safeguard Policy Statement Policy requirement (CE) One ADB: Approach to Sovereign and Nonsovereign Clients (CE) ADB s Procurement System for Operations (CE) Delivery of Knowledge for Development (CE) Annual Evaluation Review (with special chapter on Learning from Project Evaluation Lessons) II. Annual Report Annual Evaluation Review (with special chapter on Quality of Operations Design and Preparation for Impact) Annual Evaluation Review (with special chapter on ADB s Self-Evaluation System) Annual Evaluation Review (special chapter TBD) III. Knowledge Information Systems and Database Enhancement of evaluation Enhancement of evaluation Enhancement of evaluation information systems (Lessons, Management information systems (Lessons, Management information systems (Lessons, Management Action Records System, and Action Records System, Action Records System, Success Rates databases) Synthesis papers: (i) Lessons from ADB s Country Partnerships : A Synthesis of Evaluations; (ii) Sustainability of Irrigation Operations and Success Rates databases) and Success Rates databases) Working paper series: Increasing Penetration of Variable Renewable Energy: Lessons for Asia and the Pacific Enhancement of evaluation information systems (Lessons, Management Action Records System, and Success Rates databases)

8 5 IV. Learning and Accountability from Country, Sector, Project, and Technical Assistance Evaluations and Validations CAPE: India CAPE: Azerbaijan CAPE: Indonesia CAPE: Myanmar Country Assistance Program Evaluations (including associated sector assistance program assessments) Validation of CPS Final Reviews: Afghanistan, Kazakhstan, Mongolia, Philippines, Turkmenistan SAPE: Pakistan Energy Sector Sector synthesis note on energy Validation of CPS Final Reviews: Armenia, Bhutan, Fiji, Kyrgyz Republic, Solomon Islands, Thailand, Uzbekistan SAPE: Indonesia Finance Sector Sector synthesis note (TBD) Validation of CPS Final Reviews: Bangladesh, Cambodia, Georgia, Nepal, Pakistan, Timor Leste, Viet Nam SAPE: Myanmar Social Sectors Sector synthesis note (TBD) Validation of CPS Final Reviews: Afghanistan, Kazakhstan, Lao PDR, Papua New Guinea, PRC Project, Program, and TA Evaluations and Validations 12 PPERs Sovereign projects or programs 3 PPERs Nonsovereign operations 2 TA performance evaluation reports Validations of 100% of circulated PCRs and XARRs 9 PPERs Sovereign projects or programs 3 PPERs Nonsovereign operations 2 TA performance evaluation reports Validations of 100% of circulated PCRs and XARRs Feasibility of validation system for TA selfassessments 9 PPERs Sovereign projects or programs 3 PPERs Nonsovereign operations 2 TA performance evaluation reports Validations of 100% of circulated PCRs and XARRs Validations of circulated TA self-assessments 9 PPERs Sovereign projects or programs 3 PPERs Nonsovereign operations 2 TA performance evaluation reports Validations of 100% of circulated PCRs and XARRs Validations of circulated TA self-assessments Project Preparation Comment selectively on draft RRPs and concept papers Comment selectively on draft RRPs and concept papers Comment selectively on draft RRPs and concept papers Comment selectively on draft RRPs and concept papers Unprogrammed evaluations V. Contingency Resources for Unprogrammed Evaluations Topics TBD during the year Topics TBD during the year Topics TBD during the year

9 6 VI. Real-time Feedback, Capacity Development, Outreach, and Other Regular Activities (i) Training and evaluation capacity development program (ii) Evaluation event: TBD (2018, 2019, 2020) (iii) Annual meeting seminar (2018, 2019, 2020) (iv) Updates on methods and guidelines, i.e., CAPE, PPER, and PVR (continuing) (v) Continue learning curves, learning lessons, and evaluation alerts by Independent Evaluation Department Communication and Outreach Support Group (vi) Enhancement of the evaluation website and management action record system (vii) Provision of resource persons to Shanghai International Program for Development Evaluation Training as needed (viii) Support for ECG activities (e.g. chairing of ECG meetings in 2018) (ix) Participation in the Evaluation Network of Organisation for Economic Co-operation and Development Development Assistance Committee activities (x) RDTA TA cluster subproject preparation for select evaluation studies and outreach (subproject 3) (xi) KSTA TA cluster subproject preparation for select evaluation studies and outreach (subprojects 1, 2, and 3) (xii) KSTA TA cluster preparation for select evaluation studies and outreach (subproject 1) ADB = Asian Development Bank, ADF = Asian Development Fund, CAPE = country assistance program evaluation, CE = corporate evaluation, CPS = country partnership strategy, ECG = Evaluation Cooperation Group, JICA = Japan International Cooperation Agency, KSTA = knowledge and support technical assistance, Lao PDR = Lao People s Democratic Republic, PCR = project or program completion report, PPER = project or program performance evaluation report, PRC = People s Republic of China, PVR = PCR validation report, RDTA = research and development technical assistance, RRP = report and recommendation of the President, SAPE = sector assistance program evaluation, SE = sector-wide evaluation, TA = technical assistance, TBD = to be determined, TE = thematic evaluation, TP = topical paper, XARR = extended annual review report. Notes: 1. Number of evaluations for discussion by the Development Effectiveness Committee: 7 in 2018, 7 in 2019, and 8 in The current estimate of project validations (sovereign and nonsovereign) for 2017 is about 60. Indicative targets for 2018, 2019 and 2020 depend on the number of PCRs and XARRs to be circulated during the year. Source: Asian Development Bank Independent Evaluation Department.