WORLD INTELLECTUAL PROPERTY ORGANIZATION GENEVA INTERNAL AUDIT AND OVERSIGHT DIVISION INTERNAL REVIEW ON PROGRAM PERFORMANCE REPORTING PROCESS

Transcription

1 WIPO WORLD INTELLECTUAL PROPERTY ORGANIZATION GENEVA Reference: EV/01/2008 INTERNAL AUDIT AND OVERSIGHT DIVISION INTERNAL REVIEW ON PROGRAM PERFORMANCE REPORTING PROCESS Evaluation Section October 10, 2008

2 Page 2 TABLE OF CONTENTS EXECUTIVE SUMMARY INTRODUCTION FINDINGS AND CONCLUSIONS Strategic planning Project and Program Management Existing weaknesses in results-based monitoring and evaluation systems Existing deficiencies in the current results based budgeting within WIPO Management information systems have not been developed within WIPO Scarce Utilization Risk Management System among Programs RECOMMENDATIONS Start with the development of a medium and long term strategic plan Mainstream program and project management Strengthen results-based monitoring and evaluation systems Enhance the coherence of results based budgeting (RBB) Develop Management Information Systems (MIS) Further development of Risk Management System (RMS)... 16

3 Page 3 EXECUTIVE SUMMARY 1. This review on Program Performance Reporting was the result of various consultations with program implementers within the organization during IAOD s preparation of the Program Performance Report (PPR) for The purpose of the review is to identify areas in need of improvement and some lessons learnt that could assist the organization to further consolidate and integrate existing RBM systems and tools as well as to provide recommendations about required additional systems. When considering the results of this review it is important to keep in mind that like in many other institutions, some gaps are only visible once implementation of systems has started. 3. Overall, this review identifies that WIPO has made efforts to enhance the efficiency, transparency and accountability within the organization. Evidence of this has been found in tools like AIMS, the Program and Budget Report, the PPR, itself, RBM processes, the strategic framework and other tools. However, it has been noted that current systems are very much driven by the organization s budget systems rather then by the organization s strategy. Most UN organization shifted from an input budget approach in the 90 s to a more integrated results based system approach, this has not been the case for WIPO. Consequently, it remains a challenge to move towards a more integrated results and performance-based approach within current arrangements. 4. Taking into consideration the above mentioned findings and conclusions, this review recommends that the current program performance framework should require top management within the Organization to be actively involved in strategic performance planning accompanied by establishing and ensuring consultation and consensus building with the lower accountability levels. WIPO needs to focus more on its strategic performance planning process and client needs/problems analysis and on results at the various stages of implementation such as resource utilization (inputs), activity completion, output generation, and outcome/impact achievement. 5. More specifically, in this review six main findings and conclusion have emerged leading to six key recommendations: Finding and Conclusion 1: Formal strategic planning that uses a longer time frame and identifies systematic achievement of planned development results over a period of time has not been developed within the Organization. Consequently, it has been a challenge to keep the organization s focus on key priorities; Recommendation 1: Start with the development of a medium and long term strategic plan and consider redesigning strategic objectives and goals, corporate values, programs and results indicators (at the corporate strategic and program levels) to be included as part of the strategy; Finding and Conclusion 2: Project and Program management practices are not standardized throughout the organization. As a result, a program might not always be as effective as it could and resources are not always allocated in the most efficient manner to

4 Page 4 the highest priority activity. Deficiencies in planning are also accompanied by lack of accountability and ownership of program staff. Recommendation 2: Mainstream program and project cycle management within the organization better. This is particularly important for work related to the Development Agenda. Start linking program delivery and contribution towards strategic goals to resources allocation. Finding and Conclusion 3: Weaknesses in existing results-based monitoring and evaluation systems reduce the accuracy and reliability of data. Consequently information can be used only to a limited extend for decision making purposes. Recommendation 3: Improve the quality of existing results-based monitoring and evaluation systems and build the monitoring and evaluation capacity of program staff. Both financial and program monitoring need improvement. Finding and Conclusion 4: Results-based management and budgeting (RBMB) systems have been improved greatly in recent periods. However, they are still underdeveloped and do not link resources to results. Furthermore, the current RBMB system does not allow program managers to flexibly monitor utilization of allocated resources by project or activity. Consequently, program managers cannot be made accountable for managing allocated resources in the most efficient manner. Recommendation 4: Expand on existing RBMB systems and allocate responsibilities to program managers for monitoring allocated resources. By doing so the organization will not only be able to increase its accountability but also increase the transparency and understanding of the system. Finding and Conclusion 5: Management information systems (MIS) have not yet been fully developed within WIPO. Consequently, it is not possible to capture information from monitoring and evaluation systems at every level to enable managers and stakeholders to carry out effective and informed decision-making on a timely basis. Recommendation 5: Consider the development of a MIS (e.g. Enterprise Resource Management System) that can be updated by program staff on a regular basis and at the same time provide all key information for decision making a timely manner. Finding and Conclusion 6: The availability and application of risk management systems is not applied across WIPO. Recommendation 6: Develop risk management systems at the corporate level (achievement of strategic objectives) and where needed at lower institutional levels. It is noted that this is an aim of the current Organizational Improvement Program. 6. This report has been prepared for internal use and it is not intended for publication outside the organization. The report has made use of qualitative data gathered through interviews within the organization and available WIPO documentation during the PPR process.

5 Page 5 1. INTRODUCTION 7. WIPO has adopted an RBM approach and in this, program performance assessment, which is comparable to self-evaluation of program and projects within the results-based context, is an essential component. RBM is considered a valuable tool for program managers to contribute to the accountability and effectiveness throughout the UN system (Joint Inspection Unit, ). At WIPO the annual reporting of organizational progress in implementing the approved program is made primarily through the Program Performance Report (PPR). The PPR is the key accountability and progress reporting document for Member States and other stakeholders 8. Since the introduction of RBM within WIPO, a number of systems, mechanism and tools were put in place. For example, RBB, results-based strategic reporting (program performance reports) and some risk assessment systems. These systems are being implemented for at least ten years. WIPO prepared its first program performance report (PPR) in 1998 and since then ten-program performance reports have been presented. In addition, WIPO has modified and improved its strategic framework which was redesigned in 2005; its performance indicators for ; the financial management systems and introduction of updated financial rules and regulations; apart from other improvements the organization has experienced in the last years. 9. However, after nearly a decade it is imperative to look at past experiences in the implementation of such tools and identify areas for further improvements. This review is intended to assess the current process, strengths, weaknesses and limitations identified during the preparation of the most recent PPR for In addition, the review comes up with recommendations for improving future performance management, reporting and more specifically provide where possible recommendations to strengthen the organization s resultsbased monitoring and evaluation systems as wells as to assist with the implementation of WIPO s evaluation policy. 10. The findings and conclusions of this review expand on the conclusions identified during the preparation of the PPR for As requested by the Director General in 2008, the Internal Audit and Oversight Division (IAOD) based on Program managers contributions, Human Resources Management Department (HRMD) and the Office of Controller (OC) information, prepared Program Performance Report (PPR) During preparation of the PPR, several limitations and areas for improvements were identified. Hence a need to review the current process emerged, with the specific aim to provide explicit recommendations for overcoming the current shortcomings for validation, monitoring, evaluation and future RBM reporting. This review has specifically focused in six areas: the process of planning, programming, budgeting, monitoring and evaluation; human resources management; and management information systems. The here mentioned areas are the main pillars for the development of a solid integrated RBM system (Joint Inspection Unit, ).. 1 JIU/REP/2004/6: Implementation of results-based management in United Nations Organizations 2 JIU/REP/2004/6: Implementation of results-based management in United Nations Organizations

6 Page 6 2. FINDINGS AND CONCLUSIONS 11. WIPO has made efforts to enhance the efficiency, transparency and accountability within the organization. Evidence of these efforts have been found in recent improvements in tools like AIMS, the Program and Budget Report, The Program Performance Report, the strategic framework and other tools. 12. However and like in many other institutions, some gaps are only visible once implementation of systems has started. This has been also the case for WIPO. In this review it is noted that most of the gaps or weaknesses identified during the preparation of the Program Performance Report occur within the organization due to the following: a) Formal strategic planning that uses longer time frame and identifies systematic achievement of planned development results over a period of time has not been developed; b) Project and program management practices are not being fully implemented throughout the organization; c) Weaknesses of existing monitoring and evaluation systems; d) Results-based budgeting still under developed; e) Management information systems (MIS) have not been developed within WIPO; f) Scarce utilization of risk management system (RMS) among programs Strategic planning 13. The last medium-term strategy was presented to Member States in In 2003 the Director General presented a medium-term plan for WIPO programs and activities highlighting the vision and strategic direction of WIPO for the period of (document A/39/5). However, the Member States did not approve this strategy. Since then the organization has been operating without any formal long- and medium-term strategic guidance and has been mostly driven by it budget system rather then by its strategy. The Program and Budget Document provides shorter term programmatic guidance to the organization (within a set strategic framework), but not having a long- and medium term guidance implies the following: a) Priorities of the organization have not been formally defined in any officially approved strategic papers. This has partly resulted in inputs being planned without tailoring these according to where resources are most needed to deliver organizational goals;

7 Page 7 b) The current strategic framework does not include corporate level performance indicators linked to the five strategic goals approved in the Program and Budget Document (document A/44/2). c) Program objectives do not fulfill standard international SMART 3 criteria, for instance they are not time bound or beneficiary groups have not been defined (see publication No. 360E/PB0607); d) There are currently 32 programs within the organization, from which 12 are management and support services/functions. All of this could be classified as what they are: Management and support services/functions rather than programs delivering the core business of WIPO. In addition, having too many programs increase the complexity of the framework. e) Although the strategic framework has benefited the organization by providing short-term guidance, it is noted that existing guidance and a common format for preparing projects and programs are not being used. Furthermore, there are no formally articulated criteria for either the development of new programs and projects or management and support services/functions within the organization. This is reflected for example in the newly created Security Program which is actually a support function (see A/44/2); f) The organization has put a lot of efforts to increase its transparency, ownership and accountability by introducing as part of their framework indicators and expected results to each program against which program managers have to report on an annual basis. However, the Strategic Framework 4 (see publication No. 360E/PB0607) contains too many indicators and expected results (a total of 219 indicators and 134 expected results) most of which are activities/output indicators rather then outcome/impact indicators; g) Through the current strategic framework, it is not possible to analyze as to why differing degrees of progress have been made in relation to strategic goals (both internal and external to the organization) and it is not sufficient to formulate conclusions or recommendations that would be useful for decision making purposes. h) The strategic framework (mission, vision, goals) does not have properly stated values of the organization. Having well-defined mission, vision and goals is a must in every strategic framework since it helps to sharp corporate culture and behavior. This is much needed. 14. Overall, it remains a challenge for the organization to provide formal guidance and fulfill the needs and expectations of its stakeholders and beneficiaries without having an approved long- and medium-term strategy. 3 SMART: Specific, Measurable, Achievable, Realistic and Time-bound. 4 The strategic framework entails 137 expected results and 254 key performance indicators.

8 Page Project and Program Management 15. While there is very good guidance and common format for preparing projects and programs (see on WIPO intranet: WIPO Draft Project or Program Document Guidelines, February 2003), there are several reasons why program/project plans tools are not being coherently implemented across the organization, for instance: a) This report notes that the implementation of program and project plans is still not fully adopted within the organization. This has major implications on the appropriate allocation of resources, making the organization look less transparent and forecasting a challenge; b) There seems not to be sufficient internal and external demand for project and program planning or information on planning requirements. Consequently, there is lower accountability and ownership on programs within the organization; c) Project/Program Guidelines were developed but these were not linked to a capacity building plan on program and project management; d) Planning tools like the logical framework, action plans, monitoring and evaluation system as well as formal situation analysis or needs identification are not utilized. Activities and projects seem to evolve increasingly on an ad-hoc basis. This creates confusion in regards to key terms like objectives, indicators, results and validation of data. e) The weaknesses of program management leads to confusion about who ultimate beneficiary is and makes the program or project neglect the program impact while concentrating more on outputs/activities undertaken; f) External expertise to assist program implementers with the development of their program/project plans has not been provided; g) There are no sanctions for not complying with program/project plans; It can be concluded that weaknesses of program planning do have a direct negative effect on program delivery, especially on the effectiveness of programs and efficiency of resources utilization. Furthermore, deficiencies in planning are also accompanied by lack of program staff s accountability and ownership Existing weaknesses in results-based monitoring and evaluation systems 16. According to WIPO s evaluation policy the main role of IAOD when preparing the PPR is the validation 5 of information provided by program managers through their program reports. This automatically implies that all programs have monitoring 6 and information 5 Validation is the process of cross-checking to ensure that the data obtained from one monitoring method are confirmed by the data obtained from another method. Source: A guide for project M&E, IFAD Monitoring is defined as the regular collection and analysis of information to assist timely decision-making, ensure accountability and provide the basis for evaluation and learning. It is a continuing function that [Footnote continued on next page]

9 Page 9 systems in place that would enable cross checking. The PPR report is prepared by IAOD on request of the DG s office. Program managers are responsible for reaching agreement with financial and human resources data provided by the Controller and HRMD. 17. During the preparation and review of the PPR for 2006/07 some aspects affecting the preparation and initially envisaged validation process were identified and there is a need to improve on the issues identified further below in order to comply with international standards for RBM As part of the PPR preparation the Evaluation Section consulted program managers and staff with the purpose of collecting and reviewing data. However, little monitoring data or historical data was readily available. Often data was not systematically collected and accurate at the program level and even less at the project level. As a result, the collection of data and its review becomes a lengthy process and systematic cross-checking and referencing is often impractical. In future years, if IAOD are to carry out useful and effective validation of performance information and results, rather then editing of a report, such data will need to be properly collected and maintained at the program level. 19. Although Program managers were requested to self evaluate their programs and provide a report on expected result, these were mostly based on listing activities rather than the achievement of concrete outcomes. This has been an issue for previous PPRs as well. Some of the reasons for the, at times, limited quality of information are: (a) From the 210 key performance indicators, only 36 per cent did fulfill the SMART criteria and were developed using a baseline. The remaining 64 per cent of the indicators were neither SMART nor did have a baseline. Furthermore the vast majority of indicators are activities based rather the outcome based; (b) A vast amount of indicators have been developed. However, most indicators are not relevant to measure the expected results and in most cases indicators do only require the listing of activities; (c) Only few indicators have been developed to deliver information on effectiveness and efficiency. Consequently, it is not possible to measure if the achieved results are value for money; (d) The current monitoring system does not always include benchmarks and baselines. This is also reflected in the way administrative efficiency gains are measure within WIPO. For instance, WIPO does measure its administrative efficiency based on eight selected indicators. However, the selected indicators seem not to be representative enough and selection criteria for indicators have not been established at the time of reporting; [Footnote continued from previous page] uses methodical collection of data to provide management and their main stakeholders of an ongoing project or program with early indication of progress and achievement of objectives. Source: A guide for project M&E, IFAD CIDA/OECD DAC, 2003: Results-based management: Towards a common understanding among development cooperation agencies

10 Page 10 (e) From the 31 program objectives, only one was well defined. The majority of the indicators did not fulfill the basic SMART criteria nor describe the situation that should be observed at the end of the time period; (f) This report notes that more attention needs to be given to the 3Rs: Are we doing the Right things, for the right beneficiaries, in the Right way? (g) Insufficient monitoring and evaluation capacity among program implementers; (h) When monitoring and evaluating, there is a missing link between measuring progress against expected results and the contribution of those results to the strategic goals; (i) Monitoring systems are still scarce and scattered across the organization; and used only sporadically; (j) The lack of utilization of monitoring and evaluation tools across all programs reduces the effectiveness when designing strategic frameworks for subsequent budget periods; (k) Evaluations have not been undertaken since 2004 and their purpose seems unclear within the organization despite past efforts made by IAOD to raised awareness about evaluation; (l) A learning and knowledge sharing culture within the organization has not yet been fully established. 20. Overall, this review notes that by not having formal and well developed monitoring and evaluation systems, the credibility of data is questionable, inaccurate and increases the challenge of validation exercises. Consequently, it remains a challenge to provide management with reliable and accurate information that could serve the decision making process within the organization. 21. Due to weaknesses in the current strategic results-based framework the total time required for the report preparation, consultation and validation amounts to approximately six months. Programs do also request longer consultation processes. Although in some cases the maximum time provided per consultation round were 45 days. Some program managers still felt this was not enough. Nevertheless, the consultation did not prove to be helpful to validate data but rather stimulated programs to concentrate in editing their own reports. Even after six months some programs were still correcting their own program reports and expecting the evaluation section to play the role of editing these reports. 22. Some of the issues mentioned here are not new to the organization and have been raised before as part of an internal memorandum prepared by IAOD on June 15 th Some of them have started to be tackled already. For instance the quality of program narrative and performance indicators for the report has been improved. The evaluation section plan for 2008 provides some activities to strengthen results-based monitoring systems. However, there are still outstanding issues to be addressed.

11 Page Existing deficiencies in the current results based budgeting within WIPO 23. Some financial and human resources information is available in WIPO, for instance: the Administrative Integrated Management System (AIMS), the program and budget report, the financial management report. However, this is not sufficient and there are still many complains regarding the usefulness and accuracy of the data provided. In addition, program implementers were not always familiar with the financial or human resources information currently available. More specifically this review identifies the following: a) Managers are not fully focused on managing for results but more on inputs/activities completion; b) Existent systems are not sufficient to support managers to manage for results and to be accountable. For instance, managers have been provided with tools like AIMS to monitor their non-personnel financial resources but at the same time they cannot monitor their human resources budget. Consequently, it is not possible to make program managers accountable for managing their resources in the most efficient and effective manner; c) There is a limited systematic cascading (Strategic goals, programs, project and finally activities) of strategic macro development plans to delivery levels; d) There is currently a lack of an integrated approach to linking budgeting and systematic performance management. For instance, it has been noted that individual staff performance appraisal systems that link personal objectives to the strategic and corporate objectives were not in place; e) Some program staff are still not making proper use of the available financial information and tools since the capacity of staff have not been sufficiently developed to understand how to make the best use of the available information; f) It is evident that there are is an incoherent use of definitions and unless harmonized, this will continue creating confusion among staff and make the systems look less transparent; g) Financial information might not always be communicated to program managers or might not be communicated in an user friendly manner; h) In some cases financial data provided for the purpose of the PPR did differ from the data available on AIMS. This was the case in some of the programs and in such cases the Office of the Controller was helpful in providing more information; i) Budget adjustments or revisions are not necessarily linked to adjustments on the expected results and indicators; j) Post management is not always very clear or straight forward. Senior managers are considering a revision of the post management process.

12 Page It is possible to conclude that the reduced accountability among program managers is partly due to the lack of delegation of responsibility for monitoring their resources. Adequate monitoring tools have not been provided either. Furthermore, the limited communication of information linked to the, still insufficient, support offered by the organization in order to understand the various systems does neither help to reduce confusion, nor increase transparency, accountability and trust within the organization Management information systems have not been developed within WIPO 25. This report notes that within WIPO there is only one system (AIMS) that can be used by program managers to monitor the utilization of non-personnel resources. However, no other monitoring systems have been developed to facilitate the monitoring and evaluation of programs. 26. Although program implementers have been given the responsibility to monitor and undertake self-evaluations of their program progress and achievements, they have not been adequately equipped with the required knowledge and management information systems (MIS) to do so. Consequently, the implementation of programs is not being closely monitored to ensure it is being implemented as planned within the agreed parameters. The lack of a well-designed MIS reduces the accountability and ownership of program managers over their programs and does not facilitate managers and stakeholders to carry out effective and informed decision-making on a timely basis Scarce Utilization Risk Management System among Programs 27. It has been noted that apart from the construction program, which has a register for risk management and effective risk management, and the security program, which use the security risk assessment from the UN, there seem to be no other examples of good risk management within the organization. Fore example, there is no management of risk to achieving the organization s strategic goals. 28. Consequently, it is not possible to assess the risk that could affect achievement of strategic goals and stated program plans.

13 Page RECOMMENDATIONS The following recommendations of this internal report do mainly focus on the following: a) Start with the development of a medium and long term strategic plan; b) Mainstream program and project management within the organization; c) Strengthen results-based monitoring and evaluation systems; d) Enhance coherence of results-based budgeting; e) Develop management information systems as well as risk management systems Start with the development of a medium and long term strategic plan 29. It is most recommendable that WIPO starts with the development of a strategy and strategic framework that uses longer time frame and identifies WIPO s values, and systematic achievement of planned results (outcome and impact) over a period of time. The new strategy should take into consideration the following: a) Create linkages between project, program and the strategic level since it is essential for the organization to adopt standard practice used in other well-developed IP offices like the United States Patent and Trademark Office (USPTO) and the United Kingdom Intellectual Property Office (UKIPO); b) Develop fewer but more meaningful SMART program objectives and indicators at the program level, corporate/strategic level. The new indicators should be outcome/impact oriented (results focused) and provide information for decisionmaking. For instance indicators that provide information on sustainability, effectiveness and efficiency could be very useful; c) Reduce the number of programs and find synergies wherever possible. This could be done by defining what a program should be (e.g. the core business of the organization) and then separating programs from management and support services/functions; d) Separate strategic goals, which are mostly related to program delivery from management goals. Aiming at better management and administration is not seen as a strategic issue. An example of how other organizations position management goals within their strategic frameworks can be taken from the USPTO. USPTO strategic framework 8 consists of three strategic goals and one management goal, which has then been further, elaborated into 14 sub-objectives. The whole framework is measure based on 24 performance indicators; e) Develop clear guidance for development and classification of program and project. f) Find the balance between RBM and RBB when development programs. For instance, instead of creating programs for each new project/activity, which is currently being 8 See USPTO Strategic Framework under the following link:

14 Page 14 done due to budget requirements, do consider the pyramid framework and recommend the development of projects or better classification of services/functions. For instance, the construction project or the security service/function; g) Make the exercise as participative as possible involving key stakeholders; h) Include action plans with specific intermediate milestones and a system for monitoring and evaluation; i) Risk assessment and mitigation measures will need to be strengthened, accompanied by risk registers developed for each strategic goal. 30. The Management, Strategic Planning Unit and the RBM focal point in collaboration with the Office of the Controller, IAOD including monitoring and evaluation experts as well as other stakeholders that could assure the quality of the developed framework could drive the development of the new strategy Mainstream program and project management 31. WIPO urgently needs to strengthen the utilization of program and project management tools like the logical framework and action plans for implementation, which includes not only budget and timeframes but also sets clear roles, responsibilities for implementation as well as identification of end beneficiaries. Furthermore, written plans for each program and major projects, showing the linkage with the overall strategic goal, need to be developed. Having clear plans in place will allow the organization to enhance its accountability and ownership. 32. This report recommends to update existing program and project guidelines and tools; and start building program management capacity of program staff by providing training and technical support on the development of program and projects, 33. It is most advisable to start with the identification of incentives for improving program management e.g. resources should only be provided to programs once the quality of proposed plans has been certified. 34. The coordination and linkages of the array of strategic goals reflected in diverse projects and programs into specific sets of common goal actions is necessary in order to strengthen strategic reporting rather than focus in activity level reporting. 35. Equipping staff with policies and instruments will not be sufficient, if the organization is keen to see policies and systems implemented. Any new instrument needs to be well understood and owned by staff; and management needs to provide the necessary technical support for achieving stronger ownership and appropriate use of systems Strengthen results-based monitoring and evaluation systems 36. In line with increasing demand from Member States for enhancing the effectiveness and efficiency of the organization, WIPO is continuously working towards putting in place a stronger and more coherent results-based monitoring and evaluation system that will contribute towards an enhanced learning culture, transparency and accountability. Some tools

15 Page 15 like the PPR, work plans and some monitoring system have been developed over the years and more needs to be done in order to consolidate results-based monitoring and evaluation systems. For instance: a) Consolidate existing monitoring systems and include baselines and when possible benchmarks; b) Instead of developing monitoring and evaluation systems that focus only on inputs and outputs, start rather focusing on outcomes and impacts. Improvements on the existing results-based M&E could provide the organization with information as to why and how strategic goals have been achieved to a greater or lesser extent and what has been learned during the process in order to improve future planning and implementation of programs; Figure 2: Shift towards a more results based system Past and Current Situation Inputs and outputs related results Future Shift Start in 2009 with Outcome and Impact Indicators Development showing first signs of evidence in 2010 and 2011 c) It would be even more advisable to use the PPR as a tool to evaluate organizational performance and to look at why and how results were or were not achieved at the organizational level; d) Develop baseline, benchmarks and fewer but SMART indicators at program and project level and consider having a good balance between effectiveness, impact, sustainability and efficiency indicators as part of the strategic, and monitoring and evaluation framework. Think about why you assume the indicator is relevant and what the indicator is going to tell to the organization s stakeholders. 37. This proposed change in emphasis will require a shift in organizational culture. It is advisable to continue building the monitoring and evaluation capacity of program staff. Provide sufficient technical assistance to programs when developing monitoring and evaluation frameworks. Furthermore, it is key that WIPO starts using M&E systems as a tool not only for enhancing accountability, reliability of information but also a learning culture within the organization.

16 Page Enhance the coherence of results based budgeting (RBB) 38. This report recommends to further expanding on existing systems in order to facilitate program staff the monitoring of resources. Equipping program staff with adequate budgeting systems for monitoring utilization of allocated resources will make them accountable for managing their resources in an efficient and effective manner. In addition, program managers need to be trained in order to manage their resources and be accountable for the utilization of resources. More specifically, the organization will need to improve on the following: a) Systematic cascading of strategic goals to activities is not only necessary for planning and strategic purposes but also for budgeting purposes; b) Start considering allocation of resources based on achieved results and contribution towards organization strategic goals. Apart from measuring organizational performance, the organization could start linking individual staff performance appraisal systems that link personal objectives to the program and strategic objectives; c) Consulting and communicating resources allocation plans or adjustment to program implementers and especially informing program managers of any planned changes that could affect their programs; d) Provide accurate, consistent and reliable financial and human resources information on a monthly basis. This will help managers to know whether they are meeting, exceeding or failing to meet their projections, so that they can make any adjustments required; e) Harmonized financial and human resources definitions in order to avoid confusion among program implementers; f) Link budget adjustments to adjustments of expected results and indicators; 3.5. Develop Management Information Systems (MIS) 39. The role of MIS and M&E are closely knit, drawing on each other constantly to ensure that the right information for the right people at the right time is produced. 40. Management information systems within WIPO will need to be developed in order to allow program managers to monitor on-going processes based on key performance indicators at different program levels. This would allow for timely and appropriate steps to be taken to keep a program on track and to ensure that its objectives or goals are met in the most efficient and effective manner Further development of Risk Management System (RMS) 41. IAOD will be undertaking a Gap Analysis of the Internal Control Systems. The Office Improvement Project (OIP) will consider the findings and recommendations of the analysis. Recommendations regarding RMS could provide the organization with some guidance on how to proceed with the development of such a system for WIPO as a whole.