The Role of Monitoring & Evaluation for Good Governance King Mongkut s University of Technology North Bangkok

Transcription

1 The Role of Evaluation for Good Governance King Mongkut s University of Technology North Bangkok Director of the Center for Evaluation, Saarland University, Germany

2 Content rational Outcomeoriented 3. instruments for rational

3 Traditional rational Control via input factors Financial resources personnel equipment Control via volume of resource allocation Puts wrong incentives ineffective activities Negative image of public administration 3

4 Management rational Aim: To improve performance and the process of service delivery in public administration. 4

5 Principles of NPM rational Customer orientation Competitive orientation Output and outcome orientation Quality orientation 5

6 Methodological Challenges rational Traditional control- and financing instruments are not suitable for outcome-oriented management M&E-instruments are essential for this new task Outcomes need to be measured and causes be identified = pre-condition for outcome-oriented management 6

7 Tasks of Monitoring rational Control of implementation according to the planning To supply management with data about program development (input, output) and goal achievement (impact) on a continuous basis as well as with other impact data continuous, routine activity 7

8 Monitoring examples rational Level Example Indicators entire system () -system (subsystem of ) social indicator system employment and unemployment happiness satisfaction in life medical support educational qualification employment rates employment by economic sector profession social position unemployment rate program-level development cooperation program improvement of educational opportunities for girls improved knowledge better job opportunities 8

9 Planning- and Steering Instrument rational result- oriented Comprises output, goal achievement and impacts of a program Monitoring processoriented Comprises control of implementation according to the planning Navigation 9

10 Evaluation rational a misused concept: monitoring reports, experts reports, reviews, judgments, assessments 10

11 Everyday Meaning rational Something is assessed by someone according to some criteria in some way. (Kromrey 2001: 106) 11

12 Definition of Evaluation rational Evaluations conducted scientifically or professionally distinguish themselves by being related to a clearly defined object by being conducted by people with specialized competences by intersubjectively verifiable criteria by objectifying empirical data collection methods by applying systematically comparative procedures in the assessment of that information 12

13 Criteria for Good Evaluations rational professional, fulfilling scientific standards useful for political or administrative decisions participative, including different stakeholders feasable, balance between demand / requirements and feasibility 13

14 Requirements of Evaluation rational An evaluation should produce and disclose valid and useful information and findings pertaining to the evaluation question (Accuracy). An evaluation should be guided by both the clarified purposes of the evaluation and the information needs of its intended users (Utility). In the course of the evaluation, all stakeholders should be treated with respect and fairness (Propriety). An evaluation should be planned and conducted in a realistic, thoughtful, diplomatic, and costeffective manner (Feasibility). 14

15 Constraints between the Requirements rational Scientific standards/ accuracy Utility Feasibility 15

16 Comparison M&E Monitoring Evaluation rational use continuous, routine task, often inhouse sporadically, special occasion, often external objective controls implementation process puts goals, objectives, program theory into question method descriptive data collection according to program indicators advanced methods, causal-impact analysis result process control strategic control 16

17 Combining M&E rational M&E form a unit Evaluation uses information of monitoring, thus synergies occur Both instruments do cover different information needs and are necessary for process and strategic control 17

18 Triple function of Evaluation Evaluation rational Implementation Programs Policy Policy strategies Society Norms and values opens up learning potential provides recommendations creates transparency helps supervision improves level of information opens up opportunities for democratic participation makes more effective, efficient and sustainable makes more credible allows to be assessd using sustainablity criteria Management Legitimacy Enlightenment 18

19 M&E in rational M+E cycle final evaluation external evaluation midterm review Program run end of sponsorship implementation planning survey of baseline data implementation formative evaluation continuous monitoring replanning (annual planning) 19

20 Benefits of M&E rational ex-ante on-going ex-post planning implementation impact preformative formative/ summative summativ 20

21 Benefits at Planning Stage rational M&E answer questions like Are program goals and objectives clearly formulated? What measures are needed by the target group? How can the measures be implemented? What general conditions have to be taken in mind? What resources for the program implementation are needed? 21

22 Benefits at Implementing Stage rational M&E answer questions like How has the program developed? Goals achieved? What kind of implementation-problems exist? Are the existing goals/objectives still the right objectives? Are the intended target groups reached? Who benefits from and who is harmed by the program? Were program-risks considered correctly? 22

23 Benefits at Impact Stage rational M&E answer questions like Which intended and non-intended effects occurred? Did intended effects arise among the target group? Are impacts caused by interventions? Are impacts relevant for the country s? Is there any diffusion of impacts? Are the impacts of the program sustainable? 23

24 Steering with the Help of M&E rational Objectives at level of operation M&E-system data collection + assessment with the help of deliver information for comparisons management assesses, decides steering Goal adjustment 24

25 Preconditions for M&E rational Lessons learned from implementing and using M&E systems: Five factors are important: (1) Acceptance (2) Clear Structure (3) Qualification (4) Utilization (5) Quality Standards 25

26 (1) Acceptance rational Define M&E as a priority task Make all stakeholders involved understand the whole purpose of an M&E-system Involvement of stakeholders in the development of an M&E-system from beginning on Regular informing of all stakeholders involved No misuse of the information for control or punishment Make open and constructive criticism possible Ensure that recommendations are implemented Establish a culture of learning 26

27 (2) Clear Structure rational Clear and transparent formulation of M&Egoals Tasks have to be identified Clarify communication channels Provision of personnel and financial resources Establish a manageable M&E-system Implementation into the organization structure Apply clear responsibilities 27

28 (3) Qualification rational Convince personnel of task and qualify it Convince decision-makers of an organisation that the M&E-system functions so that they use it Support by external experts upvalues work Monitoring and Evaluation are part of Capacity Development 28

29 (4) Utilization rational Reliable, valid, feasible, comprehensible and prompt results Results must be relevant for steering Clarify responsibilities for the use of the results Functioning relationship between monitoring-system and management 29

30 (5) Quality Standards rational Utility M&E serves the practical information needs Feasability Methods, organizational processes, timetable and implementation are realistic, carefully considered and cost-effective Propriety M&E activities are implemented legally, ethically and in regard to the context conditions Accuracy M&E outputs include and impart reliable information M&E is used by decision-makers 30

31 rational to strengthen the role of M&E for making: 1. Advocacy in state and 2. Development of M&E cultures in governmental institutions and nonprofit organizations in terms of learning cultures not control cultures 3. Stronger integration of interests of stakeholders and participants 4. Buildup of internal M&E capacities 5. Strengthening of independent external M&E capacity 6. Establishing of legal basic principles 7. Funding of research and development 31

32 for Activities rational to develop evaluation capacities: 1. Quantitative expansion and qualitative improvement of vocational training and further training of M&E experts 2. Development of training services for operative and management staff 3. Improving of professionalization of M&E 4. Development of networks and media for communication (e.g. by evaluation societies, journals, publications, conferences, etc.) 5. Increase of international cooperation and exchange 6. Development of Evaluation standards for evaluations, codes of conduct etc. 7. Further development of theories and methods of evaluation research 32

33 Thank you for your attention! 33