Setting Up M&E and Reporting System

Transcription

1 Setting Up M&E and Reporting System

2 Monitoring - Evaluation Monitoring is the routine process of data collection and measurement of progress toward program objectives. Normally and internal process Answers the question, what are we doing? Interventions Evaluation is the use of specific study designs and special studies to measure the extent to which changes in desired health outcomes are attributable to a program s interventions. Answers the question, what have we achieved and what impact have we made Outcome

3 EFA Goal 1: Universal Coverage of Out-of-School Children, Youths and Adults in the Provision of Basic Learning Needs ALS Learners % % % Completers A & E Examinees A & E Test Passers % Passers over Examinees Number of learners are in Thousands Slide 3

4 EFA Goal 2: Universal school participation & elimination of drop-outs and repetition in the first three grades Net: Secondary and Elementary; Gross: Kinder SY , Actual SY , Actual SY , Targets Slide 4

5 EFA Goal 3: Universal completion of full cycle of basic educ. schooling with satisfactory achievement by all at every level Cohort Survival Grade 1 Grade 4 Elem graduates HS entrants HS graduates Slide 5

6 EFA Goal 3: Universal completion of full cycle of basic educ. schooling with satisfactory achievement by all at every level National Achievement Test, Elementary 65 MPS 66 MPS 68 MPS 68 MPS 67 MPS 75 Target 60 MPS SY SY SY SY SY SY SY Slide 6

7 EFA Goal 3: Universal completion of full cycle of basic educ. schooling with satisfactory achievement by all at every level National Achievement Test, Secondary 47 MPS 49 MPS 47 MPS 46 MPS 48 MPS 49 (Y4) MPS 75 MPS PDP Target SY SY SY SY SY SY SY Slide 7

8 Magallanes CES Challenges on Performance Indicators Attained EFA Goals by 2015 Increased Participation Rate to 100% Intensify Balik Eskwela Campaign Advocacy Campaign Family mapping

9 Basic Performance Indicators Basic Performance Indicators SY SY Level of Accomplishmen t Participation Rate Completion Rate Retention Rate Graduation Rate Drop-Out Rate 0 0 Achievement Rate Promotion Rate

10 Improving the Reading Skills of Children Reading Abilities Independent Level Instructional level Frustration Level Performance Level Gains Remarks Data shows that the reading abilities of children improved in all levels 17% 29% 12% 41% 51% 10% 42% 20% 16%

11 Monitoring vs Evaluation Monitoring Continuous: day-to-day Documents progress Focuses on inputs and outputs Alerts managers to problems Self-assessment Evaluation Periodic: important milestones In-depth analysis of achievements Focuses on outcomes and impacts Provides managers with strategy and policy options External analysis

12 Why monitor activities? Tracks inputs and outputs and compares them to plan Identifies and addresses problems Ensures effective use of resources Ensures quality and learning to improve activities and services Strengthens accountability Program management tool

13 Plan vs. Target 45.00% 40.00% 35.00% 30.00% 41.29% 28.83% Plan Actual Variance 25.00% 20.00% 15.00% 12.46% 10.00% 5.00% 0.00% Plan Actual Variance PRIME Overall Accomplishment

14 Why evaluate activities? Determines program effectiveness Shows impact Strengthens financial responses and accountability Promotes a learning culture focused on service improvement Promotes replication of successful interventions

15 What monitoring and evaluation (M&E) require? A plan with a monitoring and reporting framework (the logframe) Baseline studies A monitoring system Annual reports An evaluation study and report A project completion report

16 The Logical Framework Matrix Logical Framework Matrix Narrative Summary Objectively Verifiable Indicator Mean of Verification Important Assumption The logframe defines the objectives or desired results to be achieved by the program/project. Also provide the outputs the project will deliver. Goal Higher level objective Purpose Immediate objective A planning, monitoring and evaluation tool. Outputs Deliverables Inputs Resources

17 Three Primary Uses of Evaluation Findings Rendering judgments Summative evaluations of program s overall effectiveness e.g., audit, renewal, quality control, accreditation Facilitating improvements Formative evaluation to improve program e.g., program s strengths/weaknesses, progress Generating knowledge Conceptual use of findings e.g., generalization, theory building

18 Region IX Actual Region IX 34% 72% Plan Division of Zamboanga Del Sur 21% 56% Proponent Variance Division Zamboanga Del Norte 24% 35% Region IX -38% Division of -18% Zamboanga Sibugay Division Zamboanga -11% Del Norte Division of Zamboanga Sibugay 62% 80% Division of Zamboanga Del Sur -35% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Proponent PRIME Grant Utilized % of Utilization Unutilized Region IX 2,011,230 22,518 1% 1,988,712 Zamboanga del Sur 3,550,000 94,941 3% 3,455,059 Zamboanga del 3,606,620 46,500 3,560,120 Norte 1% Zamboanga Sibugay 22,518-0% 22,518

19 Region XII Region XII 10% Actual Plan Division of South Cotabato 28% Division of Sultan Kudarat Division of Sarangani 7% 34% 45% Proponent Variance Region XII 4% Division of -34% Sarangani Division of Sultan -11% Kudarat Division of South -25% Cotabato 0% 10% 20% 30% 40% 50% 60% Proponent PRIME Grant Utilized % of Utilization Unutilized Region XII 2,300,000-0% 2,300,000 Sarangani 2,627,500 93,482 4% 2,534,018 South Cotabato - - 0% - Sultan Kudarat - - 0% -

20 Rationale for M&E M&E help to make informed decisions regarding on-going programs They facilitate effective and efficient use of resources They determine whether a program is right on track and where changes need to be considered M&E help stakeholders conclude whether the program is a success M&E preserve institutional memory

21 Jan 12-Feb 12-Mar 12-Apr 12-May 12-Jun 12-Jul 12-Aug 12-Sep 12-Oct 12-Nov 12-Dec 13-Jan 13-Feb 13-Mar 13-Apr 13-May 13-Jun 13-Jul 13-Aug 13-Sep 13-Oct 13-Nov 13-Dec 14-Jan 14-Feb 14-Mar 14-Apr 14-May 14-Jun % Physical Accomplishment Indigenous Peoples Education 100% 90% 80% 70% 60% Plan Actual 50% 40% 30% 20% 10% 0% Implementation Period (in months)

22 Jan 12-Feb 12-Mar 12-Apr 12-May 12-Jun 12-Jul 12-Aug 12-Sep 12-Oct 12-Nov 12-Dec 13-Jan 13-Feb 13-Mar 13-Apr 13-May 13-Jun 13-Jul 13-Aug 13-Sep 13-Oct 13-Nov 13-Dec 14-Jan 14-Feb 14-Mar 14-Apr 14-May 14-Jun % Physical Accomplishment Muslim Education 100% 90% Plan 80% 70% Actual 60% 50% 40% 30% 20% 10% 0% Implementation Period (in months)

23 Types of Evaluation Type Formative Process Outcome Impact Purpose Initial assessment of the target populations and contextual environment. Determines concept and design Seeks to identify the extent to which planned activities have been achieved and assesses the quality of the activities/services Examines specific program outcomes and accomplishments. What changes were observed, what does it mean, and if changes are a result of the interventions? Gauges the program s overall impact and effectiveness. Aims to strengthen design and replication of effective programs and strategies

24 Evaluation areas Relevance Equity Efficiency Effectiveness Impact contribution to poverty reduction, empowerment and partnership Sustainability Replicability Lessons learned Information, Dissemination & Networking

25 Typical M&E Framework Level Description Frequency Inputs Outputs Resources that are put into the project. Lead to the achievement of the outputs Activities or services that the project is providing. Outputs lead to outcomes Outcomes Changes in behaviors or skills as a result of the implemented project. Outcomes are anticipated to lead to impacts Impacts Measurable changes in health status, particularly reduced STI/HIV transmission and reduced AIDS impact. Impact results are effects of the intervention Continuous Quarterly 2-3 years (short to medium term) 3-5 years (long term)

26 Organizational Performance Indicator Framework Societal Goal Sector Outcomes Inclusive Growth and Poverty Reduction Equitable Access to Adequate Quality Societal Services and Assets Sub-Sector Outcomes Organizational Outcomes Knowledge, skills, attitude and values of Filipinos to lead productive lives enhanced Improved Access to Basic Education Major Final Output (MFOs) Improved Quality of Basic Education Filipino Artistic & Cultural Traditions Preserved & Promoted 1 Public Kindergarten & Elementary Education Services 2 Public Secondary Education Services 3 Alternative Learning System Services 4 Basic Education Sector Managemen t Services 5 Regulatory and Devt. Services 6 Govt. Assist. to Students & Teachers in Private Educ. Services 7 Informal Education Services -Cultural Outreach Services (PHSA) -Children Television Devt. Services 8* Book Industry Devt. Services * This is an MFO of the National Book Development Board (NBDB), an attached agency of the DepEd, and does not contribute directly to the attainment of DepEd s organizational outcomes. Initial recommendation is for the NBDB to be attached to the Department of Trade and Industry (DTI) since its mandate is to develop and promote the local publishing industry 26

27 Intermediate Outcomes Organizational Outcomes Outcome Indicators Access to Basic Education Improved % Participation Rate of age 5 in Kindergarten before % Net Intake Rate of Grade 1 by % Participation Rate of ages 6-11 in Elementary by % Participation Rate of ages in Secondary by 2016 Quality of Basic Education Improved % Grade 1 Readiness Rate before % Completion Rate in Elementary by % Completion Rate in Secondary by % Completion Rate in Alternative Learning System Program annually Mean Percentage Score in National Achievement Test of Grade 6 and Year II by

28 Why M&E? M&E should be part of the design of a program Ensures systematic reporting Communicates results and accountability Measures efficiency and effectiveness Provides information for improved decision making Ensures effective allocation of resources Promotes continuous learning and improvement

29 Designing of intervention Start of Intervention End of Intervention Next Cycle Baseline situation Project Design Results Framework Implementation Plan Quarterly MEPA Appraisal Report Annual Documentation Most Significant Changes (Initial Gains) Completion Report Set Up M&E Mechanisms Start Up Review Mid-term Review End-of-Program Review Scope of M&E Progress M&E Benefit Monitoring Impact Evaluation (% of accomplishment based on Plan, Time, Cost, Scope and Quality) (application and utilization of outputs) (benefits to the organization)

30 It s not something new Intersection Point between planning, program implementation and monitoring Planning Implementation M&E

31 Pre-MEPA Complete Staff Work Reporting & Adjustment of Strategies Post- MEPA Actual MEPA Performance Review

32 1. Pre-MEPA The Pre-MEPA activity is the most critical activity in the MEPA technology. The quality of preparations, data and information you will gather will dictate the quality of the MEPA sessions and the quality of the status reports. Completed Staff Work (CSW) What preparations do we need? What activities do we need to do? What forms and reports do we need to accomplish? Session 3 The MEPA Technology

33

34 Actual MEPA Agenda (i) (ii) Managing Program Implementation Status of Projects (Grants) (iii) Other Matters Session 3 The MEPA Technology

35

36 2. Actual MEPA Agenda 1. Managing Program Implementation Scope/Time Cost/Time Objectives 1. Determine the efficiency RO/DO implementation 2. Identify issues & difficulties that contributed to delays in implementation 3. Agree on adjustment measures (input what type of adjustments they need to make) that will help implementation back on track. Session 3 The MEPA Technology

37 80% 70% 60% 50% 40% Plan Actual 30% 20% 10% 0%

38

39 2. Actual MEPA Agenda 2. Status of Projects Objectives 1. Determine the status of school/community projects funded under PRIME including updates on # of target beneficiaries - boys and girls, ethnic group 2.Assess implementation and administrative issues and difficulties encountered in project implementation 3.Provide suggestions/recommendations that will address issues & problems identified Session 3 The MEPA Technology

40 2. Actual MEPA Agenda 2. Status of Projects Discussion Points 1. Profile of school & community projects (SUMMARY) List of schools, project title, type of project Description of project/output, issues being addressed Target # beneficiaries (example PRIME - IP/Muslim children), all or percentage (population) FOR DOCUMENTATION NOT FOR PRESENTATION 2. Updates on school & community projects (SUMMARY) Status of projects Implementation issues and difficulties HIGHLIGHT THE CAPABILITY ISSUES OF DIVISION Session 3 The MEPA Technology

41 2. Actual MEPA Agenda 3. Other Matters Other matters as per Region agenda. (Varies from region to region) Session 3 The MEPA Technology

42 3. Post MEPA 1. Status Report 2. Adjustment Plan / CATCH UP PLAN (Technical Assistance) Session 3 The MEPA Technology

43 Osborne & Gaebler, 1992 If you do not measure results, you can not tell success from failure If you can not see success, you can not reward it If you can not reward success, you are probably rewarding failure If you can not see success, you can not learn from it If you can not recognize failure, you can not correct it If you can demonstrate results, you can win public support

44