II. Key Elements of Results-based Monitoring Systems

Transcription

1 II. Key Elements of Results-based Monitoring Systems DCED Standard for Results Measurement 4 November Page Seite 1

2 Overview Certain core elements common to all good monitoring systems GTZ uses its own internal guidelines, which are mandatory for all projects Participates in efforts for standardization among donor agencies DCED Results Measurement Initiative Specific focus on private sector development Sets minimum criteria 4 November Page Seite 2

3 GTZ guidelines: 7 Steps for setting up a results-based monitoring system 1. Examine Results Chains 7. Use findings 2. Clarify interests, expectations and contributions 6. Collect data 3. Determine major areas to be monitored 5. Establish structure and processes 4. Examine indicators at objectives level 4 November Page Seite 3

4 Results Measurement Initiative Background: Increased pressure on donor-financed programmes to prove results of private sector promotion (PSP) approaches Few documented, quantitative results in the field of PSP Measurement and communication of results a priority Objective: Development and establishment of a common donor M&E minimum standard Value for donors and implementing agencies Credible, summable numbers More effective implementation through clear priorities Common approach to results measurement -> efficiency gains 4 November Page Seite 4

5 Process: 2008 M&E Standard developed and tested with 7 pilot projects 2009/2010: Inclusion of more projects (regional and topical expansion) 2009/2010: training for auditors and project staff 4 November Page Seite 5

6 Long term vision: DCED Members Encourage compliance by being more interested in compliant programmes (or even require compliance) Market relationship Standard Organisation Guardian of Standard, responsible for signing off on audits, ensuring consistency, guidelines, case studies, KM, updating, handling complaints, etc. Training Payment Auditors auditors carrying out 150 audits p.a. Competing on cost and quality Audit Payment Programmes Countries and PSD programmes following methodology; audit after 1 year, then every 2 years Support to Programmes Training and consulting organisations in a competitive market, offering range of courses, advice etc. Training, consultancy and networks 4 November Page Seite 6

7 Key elements of the DCED Standard 1. Articulate Results chain 2. Define indicators for each step in the results chain 3. Measure changes in indicators 4. Attribute those changes to the program 5. Capturing wider change in the system or market 6. Relating impacts to programme costs 7. Report on results 8. Manage the results measurement system 4 November Page Seite 7

8 DCED Standard relates closely to GTZ M&E Approach 1. Defining Results Chains 7. Reporting Results 3. Measurement 8. Management of Results Measurement System 2. Defining Indicators 4. Attribution 5. Capturing Systemic Change 6. Relating Impacts to Costs 4 November Page Seite 8

9 Results chain Clear results chain showing program logic Staff using model in their work Model reviewed annually DCED Standard At a Glance 1. Results chain 2. Indicators of change 3. Measurement of indicators 4. Attribution 5. Capturing wider change in the system or market 6. Relating impacts to programme costs 7. Reporting results 8. Results measurement system management 4 November Page Seite 9

10 Defining indicators of change At least one relevant indicator for each key change in the results chain. The impact model(s) include the universal impact indicators Indicators assess likelihood of lasting impact (sustainability) DCED Standard At a Glance 1. Results chain 2. Indicators of change 3. Measurement of indicators 4. Attribution 5. Capturing wider change in the system or market 6. Relating impacts to programme costs 7. Reporting results 8. Results measurement system management 4 November Page Seite 10

11 Universal Impact Indicators Scale: Number of target enterprises who realize a financial benefit as a result of the programme s activities per year and cumulatively. Net income: Net additional income accrued to target enterprises as a result of the programme per year and cumulatively. Net additional jobs created: Net additional, full time equivalent jobs created in target enterprises as a result of the programme, per year and cumulatively. 4 November Page Seite 11

12 Measurement of indicators Measurement plan contains: What information will be gathered for each key indicator How the information will be gathered How each key indicator will be calculated or described At what interval each key indicator will be measured or assessed Baseline information for key indicators is collected DCED Standard At a Glance 1. Results chain 2. Indicators of change 3. Measurement of indicators 4. Attribution 5. Capturing wider change in the system or market 6. Relating impacts to programme costs 7. Reporting results 8. Results measurement system management **All research is in line with established good practices.** 4 November Page Seite 12

13 Attribution A clear and transparent system for estimating attributable changes in key indicators. All other contributing programmes acknowledged. DCED Standard At a Glance 1. Results chain 2. Indicators of change 3. Measurement of indicators 4. Attribution 5. Capturing wider change in the system or market 6. Relating impacts to programme costs 7. Reporting results 8. Results measurement system management 4 November Page Seite 13

14 Wider change in the system or market Recommended: Consideration of the results of systemic change, such as crowding in, copying, replication. Programmes distinguish between enterprises reached directly and those reached indirectly, i.e. as a result of systemic change. DCED Standard At a Glance 1. Results chain 2. Indicators of change 3. Measurement of indicators 4. Attribution 5. Capturing wider change in the system or market 6. Relating impacts to programme costs 7. Reporting results 8. Results measurement system management 4 November Page Seite 14

15 Relating impacts to programme costs Costs are tracked on a cumulative and annual basis. Reporting results The programme makes estimates of changes in key impact indicators at least annually Reported figures are disaggregated by gender Costs are reported together with impact DCED Standard At a Glance 1. Results chain 2. Indicators of change 3. Measurement of indicators 4. Attribution 5. Capturing wider change in the system or market 6. Relating impacts to programme costs 7. Reporting results 8. Results measurement system management 4 November Page Seite 15

16 Results measurement system management Clear and reliable system for measuring key indicators is established Tasks and responsibilities for impact assessment have been specified System is supported by human and financial resources DCED Standard At a Glance 1. Results chain 2. Indicators of change 3. Measurement of indicators 4. Attribution 5. Capturing wider change in the system or market 6. Relating impacts to programme costs 7. Reporting results 8. Results measurement system management 4 November Page Seite 16

17 Coffee Break Page Seite 17