CARICOM. Institutional Partnership. Partnerships. Page 1

Transcription

1 SECTION 7 Results Based Management (RBM) Guidelines for Institutional Partnerships Institutional Partnership Manual for the CARICOM Education for Employment Program (C-EFE) Associatio on of Canadian Community Colleges (ACCC) August 2013 C-EFE Institutional Partnership Manual August 2013 Page 1

2 Page 2

3 SECTION 7 Results-Based Management Section 3 of the Institutional Partnership Manual gives a brief explanation of the key concepts of the results- Program (C-EFE). based management (RBM) approach which are applicable to the CARICOM - Education for Employment Results-based management applies to all stages of program management, from design through to evaluation. The RBM framework enables Canadian and Caribbean institutions to manage institutional partnerships effectively and efficiently, and to make good use of such tools as the Implementation Plan, Activity Planning and Report Summaries and Budgets, periodic financial reports, project statisticss and narrative reports. This section of the Manual includes: 7 Introduction 7.1 Introduction-Structure and Direction 7.2 Results Chain 7.3 Results 7.4 Performance Indicators 7.5 Critical Conditions 7.6 Planning Annual Activities 7.7 Monitoring 7.8 Evaluation 7.9 Conclusion Tools and resources on DFATD s RBM Approach can be found at: /acdi-cida.nsf/eng/nic kef C-EFE Institutional Partnership Manual August 2013 Page 3

4 7.1 Introduction In April 1996, the President of CIDA issued Results-Based Management (RBM) in CIDA (now known as the Department of Foreign Affairs, Trade and Development Canada or DFATD) is part of the Agency s commitment to becoming more results-oriented. The Agency Accountability Framework followed in July Both documents have served as the foundation for the design of a variety of management tools, frameworks and training programs that track the disbursement of public funds against the results achieved. Subsequently, a new revised version of the policy was issued in As such, RBM is a core project/program management tool for DFATD s partners. The CARICOM -Education for Employment Program (C-EFE) was developed using the DFATD s RBM framework and guidelines. RBM gives priority to results and retains this principle for all stages of project management. RBM essentially allows a project to be designed, planned, implemented, managed, monitored and evaluated with a clear, accurate view of the expected results, from the beginning through all stages of implementation. RBM is a practical tool for helping to maintain a clear direction and act effectively and efficiently in carrying out initiatives. This approach involves defining and categorizing results, developing performance indicators to assess results and identifying critical conditions that may affect the achievement of results. In this context, managing a project is primarily managing the process of transforming material, physical, human, intellectual, informational and organizational resources through activities that generate the expected results. This overall project management is carried out through the dynamic use of an integrated and coherent system, consisting of: An initial design and planning stage: this entails defining expected results, performance indicators and critical conditions; An implementation stage: this entails introducing methodologies and carrying out project activities and; Project monitoring and evaluation. The identification of indicators makes it possible to monitor and evaluate activities, to assess progress towards achieving results and to assess the resources and processes that enabled these results to be achieved. Throughout the life of the project, there is a complementary relationship amongst the project s planners, managers and evaluator. The particular concerns and areas of interests for these actors are at time different but they tend to come together. They share a common approach to setting priorities based on the needs of the partners, being concerned about the management of resources and activities, about how activities are being carried out and about the outcomes and the impact of activities. Page 4

5 Results-Based Management: Structure and Direction PIP STRATEGIC PLANNING Mission: Goal,.Purpose, Objectives, Institutional analysis CRITICAL CONDITIONS PERFORMANCE INDICATORS DEFINTION OF RESULTS Immediate Outcomes Intermediate Outcomes Ultimate Outcome PLANNING OF ACTIVITIES IMPLEMENTATION OF ACTIVITIES MONITORING Activity Report Summaries & Statistics Activity Planning Summaries and Annual Budget Periodic Financial Reports DESIGN Relevance, coherence, meeting needs OPERATIONAL PLAN Accessibility and availability of resources, efficiency, effectiveness Final Narrative Report EVALUATION Project relevance, assessment of performance, analysis of outcomes, analysis of needs met Evaluation Report C-EFE Institutional Partnership Manual August 2013 Page 5

6 7.2 The Resultss Chain In the past, project managers concentrated on inputs (what they had to spend), activities (what they weree doing) and outputs (what they produced). Today, using the RBM approach, work planning is based on a set of clearly-defined results, activities and performance indicators. To assess the progress made on a project, managers must continually monitor the project (ongoingg monitoring, operational reviews) in order to: ensure the project remains relevant; measure progress towards the achievement of results; make adjustments to activities, as necessary. RBM consists of a set of dynamic components that interact with each other and with the environment in orderr to accomplish the project s overall aim. The results chain can be represented as a continuum from inputs to impacts. This results chain is characterized by a sequencee of cause-effect logic of this results chain. Results chain (logic model) is a depiction of the causal or logical relationships between the relationships between various categories of results. The table below shows the inputs, activities, outputs, and outcomes of a given policy, program, or initiative. Results Chain Components C-EFE Institutional Partnership Manual Page 6 August 2013

7 Definition of Terms Result: A describable or measurable change in state that is derived from a cause-andeffect relationship. Results are defined as outcomes, which are further qualified as immediate, intermediate, or ultimate. Development results: Reflect the actual changes in the state of human development that are attributable, at least in part, to a DFATD activity. Inputs: The financial, human, material, and information resources used to produce outputs through activities and accomplish outcomes. Activities: Actions taken or work performed through which inputs are mobilized to produce outputs. Outputs: Direct products or services stemming from the activities of an organization, policy, program, or initiative. Immediate outcome (short term): This is a change that is directly attributable to the outputs of an organization, policy, program, or initiative. In terms of time frame and level, these are short-term outcomes, and are usually at the level of an increase in awareness/skills of or access to among beneficiaries. Intermediate outcome (medium term): This is a change that is expected to logically occur once one or more immediate outcomes have been achieved. In terms of time frame and level, these are medium-term outcomes, which are usually achieved by the end of a project/program and usually when there is a change of behaviour or practice level among beneficiaries. Ultimate outcome (long term): This is the highest-level change that can be reasonably attributed to a policy, program, or initiative in a causal manner, and is the consequence of one or more intermediate outcomes. An ultimate outcome usually represents the raison d'être of a policy, program, or initiative, and takes the form of a sustainable change of state among beneficiaries. At the end of the five-year implementation period, C-EFE as a program is expected to achieve outcomes up to the intermediate level. The C-EFE Institutional Partnerships are directly responsible for achieving results at the Output level and the Immediate Outcomes level. Institutional partnerships will contribute to the achievement of results at the Intermediate Outcome level these are programmatic results for the C-EFE for which ACCC, CANTA and the partner ministries, Technical and Vocational Education and Training Council/NTAs and Technical and Vocational Education and Training institutions in each country and the regional partner organizations are responsible for. The Ultimate Outcome relates to the overall goal of the program which is beyond the five-year implementation period. Page 7

8 7.3 Results As indicated in the above table, there are close, direct links between the various elements: the inputs (various resources), activities, outputs and three categories of results, i.e. immediate, intermediate and ultimate outcomes. The Program is directly and completely responsible for the immediate outcomes. The ability to predict and measure the depth of change attributable to the Program activities diminishes as one moves down the performance chain (i.e. towards intermediate and ultimate outcomes), as many outside factors come into play. The institutional partnerships (as one component of the C-EFE) will contribute to the achievement of intermediate and ultimate outcomes, which apply to the C-EFE in its entirety although they are directly responsible for achievement of outcomes at only the immediate level. In most institutional partnerships where the goal is individual and institutional capacity development, as well as for partnerships focussing on institutional strengthening, activities stimulate and support a process of change initiated by the Canadian and Caribbean partners an important premise of partnership. This joint approach must be reflected in all stages of the partnership, for all activities and throughout its life.. Immediate Outcomes Institutional partnerships are responsible for expected Outputs and Immediate Outcomes. The C-EFE itself, through its partners including ACCC, CANTA, ACTI and CARICOM are responsible for achievement of Immediate and Intermediate Outcomes. At the level of individuals, results are identified in terms of awareness, increased knowledge, changes in abilities and attitudes, the acquisition of new skills, the development of new capacities, etc. These changes can be seen in the short-term. The results may have a ripple effect, since the individuals involved have a direct effect on the institutions and organizations where they work. At the level of the institution, results often relate to changes in administrative systems, policies, rules, and programs; the introduction of new instructional approaches; and/or improvements in the production of goods and services for the community. The partner takes ownership of these new skills and expertise. Institutional life starts to change: teaching methodologies are modified, course content is reviewed and adapted in response to the new realities, faculty is retrained. These institutional results are supported by the participation and increased awareness of the people who contributed to the achievement of the individual results and who will, in turn, train their colleagues and make up the core group on whom the sustainability of the change depends. These reforms and changes are implemented by people supported by people. This is a fundamental aspect of the dynamic and the interaction between the components of the resultsbased management approach. The majority of these results are achieved within the time frame of the project. Page 8

9 Intermediate Outcomes Intermediate Outcomes occur in the medium-term and represent the sustainable benefits that accrue to the actual users of the institutional results. The Immediate Outcomes from the Institutional Partnerships will contribute to the achievement of programmatic Outcomes in each country and for the overall C-EFE. For the C-EFE, Intermediate outcomes relate to changes at the national level including systems for workforce development, strategies for the development and evaluation of competency-based programs, including gender and environment standards, national employer institutional partnerships and better trained human resources. The Office of the Senior Technical Advisor/Regional Coordinator based in Trinidad and Tobago, working with the STA offices in the other two centres, Jamaica and Guyana, is responsible for monitoring progress towards the Intermediate Outcomes and for monitoring how the Institutional Partnerships are contributing to this progress. The achievement of Intermediate Outcomes may occur beyond the Program s time frame. However, in many cases, it is possible to estimate and assess the contribution of the institutional partnership activities to the achievement of these results. Ultimate Outcome The Ultimate Outcome refers to long-term results that reflect the effects on local, regional and sometimes national communities. It is the highest-level change that can be reasonably attributed to a policy, program, or initiative in a causal manner, and is the consequence of one or more intermediate outcomes. For the C-EFE, the immediate and intermediate outcomes resulting from institutional partnerships (Component 3) as well as other activities undertaken as part of Component 1 and 2 (support at the level of the regional and national bodies) will all contribute toward the achievement of the program s Ultimate Outcome. ACCC and the partner organization, CARICOM, are responsible for the final results of the C- EFE program. In some cases, these results may be achieved after the Program is completed. In conclusion, it is important to remember that the definition of these results should include certain features: Realistic expected results, that are defined following an appropriate analysis; Results that are measurable quantitatively (as much as possible) and qualitatively; Results that can be achieved within the given time-frame; Results that reflect a sound philosophy of cooperation, aiming to foster ownership by the partner (support, facilitation, accompaniment, improvement, etc.), and a sound management philosophy (transparency, joint decision-making, accountability); Results that are relevant; Limited number of results. At times it is necessary, in light of documented evidence and discussions with partners, for results to be reviewed and adjusted. In such cases, ACCC must be involved, as soon as possible, in this reflection on the merits and the rationale for these changes. This will allow all Page 9

10 partners to gain an understanding of the different perspectives of the situation and the risks and, consequently, change the focus (if appropriate). ACCC will confirm in writing whether or not it concurs with this major change in the Program s directions and results. 7.4 Performance Indicators Performance indicators, as the name suggests, are chosen and assessed at regular, predefined times to determine whether or not results are being achieved. These performance indicators are identified during the institutional analysis. The institutional analysis is, therefore, the first tool used in assessing the relevance of the Program and in its design. At this stage of data collection, some observed and measured findings guide us in determining results and, in particular, give rise to questions regarding ways of accounting for the achievement of results. An indicator is used for this purpose. It is a number, a fact or a perception that indicates and measures progress toward the achievement of expected results. It may measure a qualitative or quantitative result. Indicators are chosen with care, are simple to use and understand, and are few in number. These indicators may relate to the operational level and assess the progress of the work. In other cases, indicators may be useful for measuring effectiveness by showing the relationship between the activities and the results achieved. Lastly, other indicators may relate to efficiency; measuring the relationship between the service or product delivered and the quantity of resources used. 7.5 Critical Conditions Critical conditions are factors external to a project that have a direct or indirect bearing on its success. For example, a training program designed to direct graduates towards a particular industry assumes the willingness of the industry to hire these graduates. This willingness is, in fact, a fundamental assumption. Careful identification of these assumptions is a step that guides the reflection on potential problems that can occur throughout a project and over which project managers have little or no control. The literature on Results-based Management refers to critical conditions as risks. Political instability of a country may, in some cases, represent a significant risk. A lack of stability can destroy any chance of achieving program results, even when a program is flawlessly designed, meets specific needs and is brilliantly administered. Generally speaking, governance problems, whether in public administration or partner institutions, are significant risk factors that can affect a project s smooth operation. Identifying critical conditions and risks is a special concern of RBM, since it emphasizes results, and results can be heavily influenced by what happens in the broader, sometimes turbulent world outside a project. RBM carefully identifies any factors that may potentially interfere with the achievement of project results. Page 10

11 In conjunction with partners, special attention must be paid not only to identifying specific risks and assumptions, but also to looking at strategies for avoiding risks as far as possible and for managing them so that the consequences cause as little disruption as possible to the project. 7.6 Planning Annual Activities The planning of annual activities is part of the project management process. It involves a set of decisions concerning the direction the project will take each year and an action plan. Activities are planned, taking into account the expected results, the needs shared by the partners, the adjustments noted during regular monitoring and the available resources. This planning process is used for establishing annual priorities and particularly for scheduling the project s activities. Results-based objectives are set, obstacles or risks are identified and analyzed, resources are distributed and the budget is allocated. All of these activities are documented and recorded in the Activity Planning and Reporting Summaries, which is found in Section 3 of this Institutional Partnership Manual. This document is submitted to ACCC for approval on the fourth Wednesday in March each year. 7.7 Monitoring Project monitoring is part of planning, which is central to project management. In RBM, the planning process is ultimately guided and shaped by the statement of results. Monitoring should, therefore, provide a framework for the formal examination of how activities are progressing and signal, on an on-going and as-needed basis, any corrective actions and adjustments required for the project to achieve the expected results. In some cases, monitoring may even identify the need to initiate a change of some results to take into account an unstable environment or lessons learned along the way. If such a process is necessary, it is important to involve ACCC in the discussions on the merits and rationale for such changes. Monitoring is a project management function, beginning from the first day of the project. It is concerned with activities and processes from a project s conception to its completion. Monitoring is a process that: is reflective, flexible and dynamic throughout the life of a project; regularly confirms the relevance of the project; tracks progress of actions toward results; spots potential risks; checks progress before moving on to a new stage; makes adjustments to meet requirements for efficiency and effectiveness; ensures activities are occurring as planned; Page 11

12 closely involves partners in order to facilitate their ownership of the management process and their capacity development. The Activity Planning and Reporting Summaries and Annual Progress Report (an example is in Section 3) UTH of this Institutional Partnership Manual are used to periodically record a project s progress and development. See Section the introduction to Section 4 for the submission dates for these documents. Periodic financial reports provide input for project monitoring (see Section 5UTH of this Institutional Partnership Manual). These reports are submitted to ACCC four times per year on pre-determined dates. The calendar noting the submission dates for these reports is included in USection 2 of this Manual. 7.8 Evaluation Evaluation goes beyond activities and results. It formally and fully measures and assesses all the cause-effect relationships between activities and results. It is carried out after the end of the project, and operates independently of project management. The evaluation provides an objective assessment of a project s relevance and measures the degree of achievement of all the results. It provides suggestions to partners about ways to maximize the project s impact, or sustainability. And lastly, it makes it possible to draw a series of conclusions or lessons regarding upcoming challenges facing the partners. An evaluation of the Institutional Partnerships will form part of the overall evaluation of the C-EFE in each country to identify important lessons for replicating the results from the institutional partnerships more broadly in each country and to determine next steps for the EFE, both in the C-EFE and in EFE programs in other countries around the world. The evaluation is based on: an analysis of information, data and observations; a validation of project relevance; an objective, transparent and realistic action; an outside, independent opinion; a formal process, scheduled and written, for providing a well-documented assessment of project success for direct and indirect partners; an examination of trends not visible during operational monitoring; and the identification of results not defined at the beginning of a project, if applicable. This process allows for the direct involvement of partners in order to facilitate their ownership of the evaluation process and to thus continue their capacity development. While distinct, monitoring and evaluation are closely related, since the information, data and observations that support a thorough evaluation come from project monitoring. It is almost impossible to evaluate a project that has not been carefully monitored. Page 12

13 7.9 Conclusion n The RBM approach emphasizes three essential operations: strategic planning, strategic management and performance analysis. While the RBM framework can, at first glance, seem static due to the linear illustration of the concepts, this approach is, in fact, fluid and dynamic. If information is gathered continuously regardingg the achievement of results and the use of resources, partners working together, will be in a better position to track a project s progress and performance and to be proactive in dealing with changing situations. RBM Application in Theory and Practice Source: Using RBM in Project Management, APPIAN Consulting Inc., July 2011 This approach is based on the commitment and constant cooperation of partners in the interests of ongoing, long-term improvement for all the parties involved in the process. C-EFE Institutional Partnership Manual August 2013 Page 13

14 BIBLIOGRAPHY AND RESOURCES FOR FURTHER READING Results-based Management Policy Statement 2008 : cida/acdi-cida.nsf/eng/ann gvj «Results-Based Management Tools at CIDA: A How-to Guide Results-based Management 2008 Policy Statement: Amended Terms and Definitions : Using RBM in Project Management, Appian Consulting Inc., July 2011 CIDA s Business Process Roadmap, September 2012, Page 14