Evaluation and Accountability. 21 September 2015 INTERPOL HQ, Lyon

Transcription

1 Evaluation and Accountability 21 September 2015 INTERPOL HQ, Lyon

2 Outline 1. Evaluation and accountability - overview 2. Evaluation and accountability in UNODC 3. Measuring GA Resolution 69/237: Building capacity for the evaluation of development activities at the country level

3 1. Evaluation and Accountability - Overview

4 In order to succeed, we need actions that can turn commitments into results. And we need to monitor, review, and evaluate those results and make them accessible to policy-makers and the public. Evaluation is critical for promoting accountability and for understanding what we are doing right or wrong. UN Secretary-General, Mr. Ban Ki-moon

5

6

7 Accountability includes achieving objectives and high-quality results in a timely and cost-effective manner, in fully implementing and delivering on all mandates to the Secretariat ( ); truthful, objective, accurate and timely reporting on performance results; ( ) all aspects of performance, including a clearly defined system of rewards and sanctions; and with due recognition to the important role of the oversight bodies and in full compliance with accepted recommendations. An assessment, as systematic and impartial as possible ( ). It focuses on expected and achieved accomplishments, examining the results chain, processes, contextual factors and causality, in order to understand achievements or the lack thereof.

8 Accountability Accountability: Feedback mechanisms play key role in International Organisations, Governments as well as private sector. Evaluation Quality Assurance Accountability Audit Monitoring

9 Sustainable Development Goals

10 SDGs and the cooperation UNODC-INTERPOL The agenda for SDGs 2030 includes e.g.: Goal 8: Promote sustained, inclusive and sustainable economic growth, full and productive employment and decent work for all Take immediate and effective measures to eradicate forced labour, end modern slavery and human trafficking Goal 16: Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels Significantly reduce illicit financial and arms flows and strengthen the recover and return of stolen assets and combat organized crime; Strengthen national institutions, including through international cooperation, for building capacity at all levels to prevent violence and combat terrorism and crime

11 SDGs, evaluation and accountability Evaluation and accountability mainstreamed in SDGs - the follow-up and review process will be: Rigorous and based on evidence, informed by country-led evaluations; Requiring enhanced capacity-building support for developing countries, including the strengthening of national data systems and evaluation programs Evaluation and accountability mainstreamed in Goals: Goal 16.6: Develop effective, accountable and transparent institutions at all levels. We commit to engage in systematic follow-up and review of implementation of this Agenda ( ). Operating at the national, regional and global level, it will promote accountability to our citizens, support effective international cooperation ( ) and foster exchanges of best practices and mutual learning (p. 27)

12 Lessons learned from Millennium Development Goals Evaluation ( ) has the potential to serve as a bridge between monitoring and accountability at key levels and junctures of deliberation and decision-making by stakeholders. This will enhance decision makers ability to make necessary improvements and mid-course corrections ( ). one key lesson learned is that the post-2015 sustainable development goals ( ) would benefit from a clear, overarching framework of monitoring and evaluation objectives ( ). In this regard, the Office of Internal Oversight Services makes one recommendation for consideration, namely, that the Secretary-General formulate an overarching strategy and action plan to support coherent, coordinated and timely monitoring and evaluation, together with relevant capacity development needed to support decision-making, ( ).

13 2. Evaluation and Accountability in UNODC

14 By increasing transparency, and enhancing learning and critical thinking, evaluation can empower people, and in this way support positive change. ( ) In short, evaluation can help us to provide clarity in a complex, interconnected world. Director-General/Executive Director UNOV/UNODC, Mr. Yury Fedotov

15 The Independent Evaluation Unit timeline Before 2010 Sporadic evaluations Internal reporting lines 2010 Evaluation Unit dissolved Member States Resolutions for fully independent evaluation Institutionalization of the Evaluation Function After 2010 Independent Unit - reporting lines to MS and ED Full coverage of all projects and programmes Knowledge and strategic products Evaluation capacity building for external stakeholders

16 The Independent Evaluation Unit current status Evaluation Function: Team: 3 professional and 1 support staff Reporting to Member States and the Executive Director Premise: All UNODC interventions are evaluated every four years and/or before completion Final evaluations: six months before the end of the interventions; focus on accountability and performance Mid-term evaluations: For continuing interventions every four years with a focus on learning and improving

17 Evaluation in UNODC Evaluation in UNODC Capacity Building Evaluation results Knowledge products

18 Evaluation portfolio in UNODC - examples UNODC Interventions Types of evaluations Knowledge Products Around 70 Programmes (geographic and thematic) In depth evaluations (four - eight annually) Portfolio management Around 260 Projects Evaluation Meta Analysis Policies, strategies, etc. Independent Project Evaluations (20-25 annually) Recommendations database; etc.

19 Evaluation in the project cycle Implementation of evaluation at HQ and FO Strategy Setting Evaluation results and lessons learned used for strategy setting Plan, budget and define scope of evaluation(s) consultations with IEU Generation of data for future evaluation Evaluation Implementation & Monitoring Project/ Programme Cycle Programme development Resource Mobilisation 2-3% of overall budget reserved for evaluation

20 Percentage of completed projects evaluated Evaluation status of completed/operationally completed projects % 22% Projects evaluated/planned to evaluate in 2015 Projects not evaluated Evaluations discontinued 75%

21 Percentage of completed/operationally completed projects evaluated Fully evaluated completed projects % 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% Series % 51.00% 52.00% 75.00%

22 Evaluation and Accountability services Evaluation and Accountability Evaluation Capacity Evaluation Results Knowledge Products Training of internal and external stakeholders Development evaluation norms and standards Project, programmatic, corporate and thematic evaluations Ex-Ante evaluations Meta analytical, strategic reports Portfolio assessments Accountability frameworks

23 Evaluation Knowledge Management in UNODC UNODC Evaluation Database (also online: Recommendations database (internal) Lessons Learned database (interal) Automated Evaluation Recommendation follow-up system

24 Evaluation Application 1/2

25 Evaluation Application 2/2

26 Portfolio Analysis

27 Meta Analysis 100 evaluations 105 evaluators / experts 1003 Recommendations 445 Lessons Learned 1 Meta Analysis evaluation/evaluation.html

28 Quality of evaluation reports Independent quality assessment of all evaluation reports January 2014 to June 2015 In line with best practices in other UN Organisations (e.g. UNWOMEN; UNDP; UNFPA; etc.)

29 Feedback survey to Member States and Project Managers 1 2 3

30 Services evaluation capacity building (external) Capacity Building for external stakeholders: Member States (in line with GA Resolution on evaluation capacity building) Partner Organisations NGOs/CSOs Tailored trainings on evaluation and accountability Trainings in relation to substantive topics Innovative approaches: cost-sharing evaluation positions for mutual learning GA Resolution 69/237: "Capacity building for the evaluation of development activities at the country level

31 3. Measuring You cannot manage what you cannot measure

32 1. Planning 2. Performance 3. Reporting Elements of accountability in implementation Specific Results Framework Plan for sustainability Operational implementation plan Risk Assessment Clear and specific indicators Stakeholder engagement and communication strategy Results-based data collection plan Performance measurement Evaluation capacity building Ex ante evaluations Mid-term evaluations after 2 and every 4 years Financial and Performance Reporting Final Outcome (Impact) evaluation

33 Planning-phase Specific Results Framework Plan for sustainability Operational implementati on plan Risk Assessment Clear and specific indicators Stakeholder engagement and communicati on strategy

34 Indicators a core facet of accountability Qualitative or quantitative variables to measure achievements Consultative process Baseline development Monitor progress at all levels of results-based systems - with respect to inputs, outputs, outcomes and goals Specific, measurable, attainable, realistic and time-bound Indicators Key question: Are we moving toward achieving our desired outcomes?

35 Illustrations of Indicators Objective: Through international cooperation, Norway will help to make countries and organisations better equipped to prevent, respond to and combat global security challenges Percentage of trained Law Enforcement personnel uses knowledge provided by UNODC/INTERPOL for border control in region X by Number of terrorism-related arrests in region X as a result of improved cooperation of Member States through UNODC/INTERPOL intervention XY. Partnership indicators: Number of new projects developed through UNODC/INTERPOL partnership by Clear roles and responsibilities of all partners identified in the Joint Action Plan.

36 Performance/Implementation-phase Results-based data collection plan Performance measurement Evaluation capacity building Ex ante evaluations Mid-term evaluations after 2 and every 4 years

37 Reporting Financial and Performance Reporting based on evidence Final Outcome (Impact) evaluation

38 Example: results of evaluations (crime portfolio) Human rights and Gender Engagement at highest level Measuring impact 30 Evaluations 40 independent evaluators Intelligence and data sharing Communication Collaboration with partners Collaboration with IOs and CSOs positive impact

39 What we can offer in building this partnership 1. Accountability to stakeholders Development of joint products, e.g. evaluations; knowledge products; etc. Ex-ante evaluations of the partnership and informing strategic development 3. Assessment of provisions for sustainable partnership 2. Management vis-á-vis results Development or transfer and application of norms, standards, instruments, etc.

40 Evaluation is not easy. Nor is it popular. But it is essential. UN Secretary-General, Ban Ki-moon, 9 March 2015

41 THANK YOU FOR YOUR ATTENTION! Please visit IEU s website: