Building Monitoring and Evaluation Systems. Maddalena Honorati Economist Human Development Network The World Bank

Transcription

1 Building Monitoring and Evaluation Systems Maddalena Honorati Economist Human Development Network The World Bank

2 Outline Why M&E? What is a good M&E system? Monitoring vs. Evaluation Using Results Chain in SSN projects Key issues 3 Case studies

3 Why M&E? ACCOUNTABILITY to prove program performance demonstrate results against money spent different level of accountability relationships -between government and parliament, civil society and donors -within government: between central and sector ministries, agencies in LICs, intensified by continuing fiscal and macroeconomic pressures MANAGEMENT to Improve program design and implementation to deliver better results Inform what works, what doesn t and the reasons why Identify corrective actions to improve program performance Establishing links between M&E, policy formulation, and budgets help to allocate and prioritize resources

4 WHAT DOES A GOOD M&E SYSTEM LOOK LIKE? No simple answer It depends on country specific factors (availability and quality of existing M&E arrangements, staff and institutional capacity, demand for M&E information, budget for M&E..) 1. Intensive use of M&E information (demanddriven process) 2. Information meets standards for data quality and evaluation reliability 3. Sustainability of M&E systems

5 MONITORING MONITORING vs. EVALUATION A continuous process of collecting and analyzing information Tracks implementation processes indicators over time Often based on administrative data (universal data) RESPOND TO : Are outputs/outcomes moving in the right direction? EVALUATION A periodic, systematic, objective assessment of program implementation and results Tailored to processes, performance, impacts to generate lessons learned to inform the decision making process. Often based on specialized data/surveys (sample based) RESPOND TO : Is program being implemented efficiently? Is program targeting the right population? What is the effect of the program on outcomes? Is the program cost-effective? Monitoring and evaluation are separate but complementary functions, both are key to results-based management

6 M&E TOOLS MOSTLY USED IN SSN PROJECTS MONITORING Management Information Systems (MIS) Coverage Beneficiary Profile Costs Service delivery Compliance Financial data Operational and financial audits EVALUATION Participatory Evaluations Social audits Beneficiary Assessments Community Scorecards Targeting Assessments Spot checks Complaints and appeals systems Process Evaluation Impact evaluations Cost-Benefit Analysis Expenditure Tracking Studies

7 THE RESULTS CHAIN Inputs Financial and human resources Activities Actions to convert inputs into outputs Outputs Tangible products, information campaigns, trainings, studies Intermediate outcomes Use of outputs by intended population Take-up Use Final Outcomes Objective of the program Growth Social cohesion Employme nt status Implementation (SUPPLY SIDE) Results (DEMAND RESPONSE)

8 RESULTS CHAINS Are simple planning tool that help you outline the causal logic/theory of change and define: what you are doing and for what purpose what needs to be monitored what needs to be evaluated DEMAND RESPONSE MONITORING EFFICIENCY INPUTS OUTPUTS OUTCOMES EVALUATING EFFECTIVENESS SUPPLY

9 KEY ISSUES Effective approaches to reinforce and integrate to national M&E systems from the program platform The feedback process: how M&E results are used to inform desicion making inside the program (management and key stakeholders) and outside the program (civil society, researchers). Complementing and integrating different M&E tools : how well defined results frameworks and comprehensive M&E systems (including both quantitative and qualitative methods) can improve project performance. Implementation challenges in low capacity settings: how can M&E methods assist program managers in overcoming these challenges? The role of strategic impact evaluation in building evidence on what works best and whether programs are reaching their intended outcomes.

10 3 CASE STUDIES 1. Jamaica Program of Advancement Through Health and Education 2. Malawi Public Works Program - Local Development Fund 3. Palestinian National Cash Transfer Program

11 Conclusions Good M&E is crucial to for evidence-based decision making Use results chains to define a shared view of the project s theory of change, causal pathways to reaching main objectives Monitoring and evaluation are separate, complementary functions, but both are key to results-based management Have a good M&E plan BEFORE you roll out your project