Managing for Results Results Based Management in practice Trine Eriksen and Alex Stimpson Financial Mechanism Office Vilnius, 11 December 2013 1
Johannes Hahn (Oct. 2013) billions of euros of EU taxpayers' money has in the past been spent with no clear strategy or benchmarks "The progress has to be measured and, crucially, communicated," said the commissioner.
RESULTS BASED MANAGEMENT What is Results Based Management (RBM) RBM is a management strategy by which all actors, contributing directly or indirectly to achieving a set of results, ensure that their processes, products and services contribute to the achievement of desired results (outputs, outcomes and impact) What is a result and a risk? In the context of the Grants: A result is the output and the outcome of a development intervention A risk is an event that may occur and impede the objective 3
KEY CONCEPTS Results achievement Beneficiary oriented Managing for results and reduced risks 4
In practice RBM is about defining realistic expected results based on appropriate analysis; clearly identifying program beneficiaries and designing programmes to meet their needs; monitoring progress toward results and resources consumed with the use of appropriate indicators; identifying and managing risk; increasing knowledge by learning lessons and integrating them into decisions; and reporting on the results achieved and resources involved.
RBM is supposed to help us: clarify what we mean by results and lets us deal with differences of understanding before a programme/project begins build capacity, because it clarifies for us what we need to concentrate on produce more realistic schedules, forcing us to think through the preconditions and sequence for actions prepare programmes/projects for effective evaluations. better identify unplanned results, as they occur, and assess if these are desirable, or problematic, requiring support or coping strategies making the links between inputs, funded activities and the results they should be leading to, which reduces the potential for corruption
However Agreeing on clear goals and objectives is often difficult Using results-information to continually improve one s own work and operations requires time to reflect and acceptance from management Ensuring that staff members can employ their creativity and resourcefulness to engage on the quest for results requires a mind shift within the organisation In other words: red tape often remains and hampers the efforts of public agencies to increase their results-orientation
The challenge is to: Keep it simple Getting stakeholder buy-in Fostering organisational use of information Making outcome accountability operational and realistic Measure Ensure attribution Linking financial and performance information
The role of the PO is to keep the focus on the big picture during implementation To create a simple and clear roadmap that guides the project promoters What the donors (and we) want to be able to do, is to explain our results clearly to other people
Part I Quality at entry - Basis for good management 10
PROGRAMMING CYCLE The Programming Cycle and Continuous Feedback Planning Reporting Programming Evaluation Budgeting Monitoring Implementation 11
Identify main programme elements of EEA/N Grants Objective (impact) Widespread improvements in societ or a sector 32 predefined objectives Outcome(s) Intermediate effects on target groups and systems Outputs Products and services delivered Activities Tasks transforming inputs to outputs 160 predefined outcomes In DoRIS data base Self-defined (at least two outputs per outcome) Self-defined In DoRIS data base Not in DoRIS Inputs Financial, human and material resources Budget + Budget in DoRIS 12
Programme logic Contribute to Overall objectives which will contribute to the two overall objectives Contribute to Likely to happen Objective Expected outcomes which will contribute to the fulfilment of the objective if the outputs are produced, then the expected outcome is likely to be achieved Should be able to guarantee that this will happen Outputs Activities Inputs if the activities take place, then the outputs will be produced if the inputs are available, then the activities will take place 13
ASSUMPTIONS The intervention logic is an aim-means chain and a cause-effect chain. It is assumed that certain events need to occur at one level in order to achieve the desired effect at the next level. It is not certain that these events happen; they represent uncertainties. 14
Impact/Objective Outcome Outputs (deliverables) Activities Inputs There are Assumptions between each level: Identify Assess Monitor Manage and Mitigate 15
Identify important assumptions Assumptions are worded as positive conditions (see objectives) Assumptions are linked to the different levels of the Programme Matrix. Each level must contain the necessary and sufficient conditions for the next level above. Start from bottom of matrix and work upwards Are weighted according to importance and probability Example: Scholarship recipients return to assigned jobs 16
Programmes and projects Pre-defined projects Strategic approach Programme proposal Projects Pre-defined projects 17
EXAMPLE Green Industry Innovation Objective (pre-defined) Expected outcome(s) (pre-defined) Increased competitiveness of green enterprises, including greening of existing industries, green innovation and green entrepreneurship Realisation of the business opportunities of greening of the European economy Outputs Types of projects Environmental technologies in industry developed, improved or implemented by SMEs Environmental technology development Implementation or replacement of the existing technology Processing sequence and/or synthesis of these innovations in order to run the processes at a higher efficiency and minimize waste, energy and emission generation Green products or material in industry developed or improved by SMEs New green product and material development or modification of the existing product and material to minimize environmental impact during or after its use. 29
Call for proposals Be clear about which outcomes and outputs you want the projects to contribute to. Which projects are necessary and sufficient? Project template in DoRIS: http://www.eeagrants.org/id/3192 19
Logical relationship 20
PART IV Basis for results measurement 21
Basis for measuring results Result based management 22
"The starting point for a results-oriented approach is ex ante setting of clear and measurable targets and outcome indicators. Indicators must be clearly interpretable, statistically validated, truly responsive and directly linked to policy intervention, and promptly collected and publicised." From the conclusions of the 5th Cohesion Report, adopted by the European Commission in November 2010 QUALITY AT ENTRY! 30
INDICATOR A means to measure achievement, or to help assess a development 24
HOW SHOULD INDICATORS BE FORMULATED? Once indicators have been identified, they should be formulated to reflect measurable change. Indicators frequently utilize the following metrics to depict measurable amounts: Number; LESS IS MORE Ratio; Percentage; or Rate; Time Indicators shall whenever relevant, be broken down per gender In DoRIS, always specify unit of measurement! 25
EXAMPLES OF INDICATORS Example 1: Outcome: Cultural heritage restored, renovated and protected Indicator: Number of buildings of cultural heritage value restored or rehabilitated Example 2: Outcome: Awareness of cultural diversity raised and intercultural dialogue strengthened Indicator: Number of local cultural associations involved in the implementation of projects 26
INDICATORS To describe how results will be measured, output indicators must be specific, observable and come from reliable data. Does the indicator provide the most direct evidence of the condition or result to be measured? Is the indicator important, understood, and accepted by important stakeholders? 27
Think about Are data for the indicator collected in the same way over time? Will data be available for the indicator? Will data be collected frequently enough? Are data currently being collected? If data are not currently collected, can cost effective instruments for data collection be developed quickly? 28
Indicators related to Bilateral Relations All programmes should contribute to strengthened Bilateral Relations Use the Bilateral guideline. Four types of bilateral results expected: Extent of cooperation Shared results Wider effects Knowledge and mutual understanding 29
Indicators related to «Horizontal issues» A human-rights based approach and concerns for horizontal issues like fighting discriminiation, racism, hate speech etc. should guide defining the outputs and the indicators. Example: % of members of X Parliament are minorities Example: Number of annual incidences of hate speech in Y school 30
Programme matrix - indicators
Sources of Verification When defining the indicators, consider the data sources. Where can you easily get data that are reliable and timely? Specify your sources of information Assess where you can get easy access to data Example Ministry of Health s statistics on cancer (health status of population) and various statistics on cancer detection and treatment services
Programme matrix Sources of Verification
WHAT ARE BASELINES? Baselines provide context for the setting of targets and capture the situation before a development intervention begins, or at the beginning of a time period that will be monitored and assessed. Baselines describe the conditions prior to programming efforts. The present situation! Essential to make credible and meaningful assessments on progress towards outcomes. 34
Programme matrix Baseline data
WHAT ARE TARGETS? Targets set the level for the magnitude of change expected by the end of a predetermined time period. They reflect what organizations commit themselves to achieve by the end of some duration of time. Targets provide tangible and meaningful points of discussion with beneficiaries, stakeholders, and partners, and allow us to add further specificity to the outcomes from the results logic. 36
TARGETS When you have defined the indicators and know the baseline situation, you set the targets. What do you want the situation for the beneficiaries to be in 3 or 5 years? Examples: 1. 60% of W 40+ called in for cancer test in Southern Region by year 2016. We want to increase the % of W 40+ called in from 30% in 2010 to 60% in 2016. 2. Time lag between detection of cancer to operation is not more than 3 months in 2016. 37
BASELINES AND TARGETS SUMMARY A baseline is the value of the indicator before the implementation of the project starts Targets orients stakeholders to the tasks to be accomplished Targets help establish clear expectations Targets serve as the guideposts for monitoring whether progress is being made Targets promote transparency 38
Part IV Quality at implementation Monitoring Reporting on results and risks Evaluation 39
PROGRAMMING CYCLE The Programming Cycle and Continuous Feedback Planning Reporting Programming Evaluation Budgeting Monitoring Implementation 40
PERFORMANCE MONITORING Define processes and systems for performance measurement; Decide the intervals at which measurement will occur; Assign roles and responsibilities; Set standard requirements for project promoters Build in adequate time and resources for analysis of performance measurement data so that progress is reviewed and assessed. Use web site for communication with project promoters and other stakeholders 41
Performance monitoring Status regarding the output and outcome indicators is measured and reported towards the baseline situation ( where we were ), and also towards the future, desired situation ( the targets ).
Programme performance framework
Group work Results based management Questions for discussion: With the principles of results based management in mind, in your view: Is there a clear intervention logic (link between outputs/outcomes/objective)? Are the Outputs sufficient to achieve the Outcome? Are the indicators good measures of achievement? Is it clear what the indicators refer to/mean? 44