UNITED NATIONS EDUCATIONAL, SCIENTIFIC AND CULTURAL ORGANIZATION EXECUTIVE BOARD. Hundred and sixtieth Session

Transcription

1 160 EX/INF.6 PARIS, 22 September 2000 English & French only UNITED NATIONS EDUCATIONAL, SCIENTIFIC AND CULTURAL ORGANIZATION EXECUTIVE BOARD Hundred and sixtieth Session Item 6.5 of the provisional agenda PROPOSAL BY THE DIRECTOR-GENERAL TO SET UP A UNESCO INTERNAL OVERSIGHT SYSTEM DRAFT REPORT BY THE INSTITUTE OF INTERNAL AUDITORS (IIA) SUMMARY This document has been produced for the information of the Members of the Executive Board in connection with the Board s examination of document 160 EX/23. It contains the draft report presented by the Institute of Internal Auditors (IIA) following its Quality Assurance Review (QAR) of the internal audit, evaluation, investigation and related functions in UNESCO, undertaken at the request of the Director-General and as he so informed the Board at its 159th session (159 EX/5).

2 (i) CONTENTS EXECUTIVE SUMMARY... 1 Highlights of the recommendations... 1 Organizational issues for UNESCO management... 1 Priority issues for the internal oversight function... 2 Opinion as to conformity with the Standards... 2 Looking to the future... 3 GENERAL OVERVIEW OF QAR... 4 ENVIRONMENT FOR OVERSIGHT AT UNESCO... 5 OBSERVATIONS AND RECOMMENDATIONS... 6 Part I ORGANIZATIONAL ISSUES FOR UNESCO MANAGEMENT Define and measure risks; enhance management controls and accountability processes Improve the tools and processes to manage information technology Develop a charter for the new internal oversight (IO) function and define its structure Commence implementation of the IO function now Define the IO relationship and information links to the Executive Board Part II ISSUES SPECIFIC TO THE INTERNAL OVERSIGHT FUNCTION Define the oversight universe and assess the related risks Prepare long-range and annual IO plans Enhance evaluation oversight activities through their integration into IO Establish appropriate IO methodologies and a new policies and procedures manual Assess staff numbers, sources and skills; expand continuing professional development Page

3 (ii) 11. Assess report format(s), review process and distribution; improve follow-up of implementation of agreed remedial actions Give priority attention to strengthening the information technology (IT) audit capabilities of IO and coverage of activities throughout UNESCO Implement an internal quality assurance process within IO Enhance coordination with the external auditors and their reliance on IO ADDITIONAL OBSERVATION FOR UNESCO MANAGEMENT Page Annex I Annex II Annex III Model internal auditing department charter Framework for risk management and management control Sample policy statement for controlling an organization

4 160 EX/INF.6 EXECUTIVE SUMMARY The Institute of Internal Auditors (IIA) conducted a Business-Focused Quality Assurance Review (QAR) of the internal audit, evaluation, investigation and related functions primarily the Office of the Inspector General (IOM) and the Central Evaluation Unit (CEU) of the United Nations Educational, Scientific and Cultural Organization (UNESCO). UNESCO is planning to combine those internal audit, evaluation, investigation and related functions into an integrated internal oversight (IO) function in the near future and plans to take the results of QAR into account as part of that restructuring process. Prior to our on-site review in August 2000, we made a preliminary visit to become familiar with UNESCO, gather background information and conduct brief interviews with selected executives. Some relevant parts of the QAR Self-Study Report were completed by CEU and IOM, and a survey was sent to selected UNESCO executives. Results of the survey, including the comments returned with the survey responses and comments from our personal interviews with a representative sample of management and national delegations, were provided to UNESCO (without identification of information from individual respondents and interviewees). HIGHLIGHTS OF RECOMMENDATIONS We agree that the decision to integrate all of UNESCO s oversight functions into a single unit is appropriate and should result in increased effectiveness of oversight of the Organization s sector activities and support functions. Our recommendations are divided into two classifications: those that concern UNESCO as a whole and/or require action by senior management and those that relate to the IO organizational structure, processes, staffing, etc., that would be implemented within IO, with some direction and support from senior management. ORGANIZATIONAL ISSUES FOR UNESCO MANAGEMENT Enhance the processes for defining and measuring risks; establish/enhance the framework of management authorities, controls and accountability mechanisms, using a comprehensive risk management methodology. This comprehensive framework can provide a necessary discipline to achieve better operating effectiveness, consistent performance measurement, and accountability at all levels of the Organization. For the specific purposes of IO, it is also a precondition for defining a comprehensive oversight universe, prioritizing the deployment of IO resources, and preparing longrange and annual oversight plans. Improve the tools and processes to deal with the risks and opportunities related to the management of information technology (IT) and the implementation of an adequate management information system. This should include the updating and effective implementation of a long-range IT master plan, under the control of an in-house technology executive and the oversight of a permanent steering committee made up of major users. This enhanced control over IT resources is also important for IO to perform its function in a cost-effective manner.

5 160 EX/INF.6 page 2 Develop a charter for the integrated IO function which could be adopted on a provisional basis pending arrival of the new Director of IO, in line with the model in Annex I. We suggest that current CEU and IOM staff, under the close direction of senior management of UNESCO, begin outlining and drafting the main features of this charter immediately. There should also be due regard for the necessary transition time for fully integrating evaluation activities (separated from the evaluation functions under the control of sector management), as well as for the clear definition of the special investigation process and responsibilities. UNESCO should commence implementation of the new IO function now, even before the new Director of IO is appointed, at least with respect to identification of areas of common interest and oversight coverage, the design and testing of new methodologies, assessing and enhancing the skills of staff, determination of availability of appropriate technology, etc. Define the IO relationship and information links to the Executive Board (EB). Establish a process for furnishing to the Executive Board (EB) periodic summaries of significant IO report items and plans, advising the EB of major changes in IO authority, scope and resources, and for consulting on the hiring or removal of the Director of IO. PRIORITY ISSUES FOR THE INTERNAL OVERSIGHT FUNCTION Define the oversight universe and assess the related risks aligning this universe with that of risk management and management control for UNESCO as a whole. Prepare annual and long-range IO plans, prioritizing the use of IO resources based on an oversight risk assessment and consultation with sector and support units across UNESCO. Apply the Standards for the Professional Practice of Internal Auditing (Standards) to IO and adopt appropriate best practices relating to IO staffing, engagement administration, performance and documentation of field work, reporting, implementation follow-up, and overall management of the function, each of which is discussed in more detail in our report. Give priority attention to strengthening IT audit capabilities and coverage of IT activities including plans, operations, administration, security and other aspects of the management of IT and its utilization to provide management information throughout UNESCO. Enhance coordination with the external auditors (EA) and their reliance on IO, through such means as better communications, sharing of work plans and reports, IO follow-up to foster the prompt and effective implementation of EA recommendations, shared training and joint performance of audit work, along with demonstrating the enhanced professionalism of IO through implementation of the Standards and best practices mentioned above. OPINION AS TO CONFORMITY WITH THE STANDARDS It is our opinion that the current internal monitoring functions of UNESCO generally do not conform to the Standards for the Professional Practice of Internal Auditing.

6 160 EX/INF.6 page 3 In IOM, there are some areas of partial conformity with elements of the Standards and there have been improvements, notably in organizing and supervising audit work, in recent months. There are major opportunities for improvement in IOM, relating to virtually all areas covered by the Standards. In CEU, conformity to some elements of the Standards, such as professional proficiency, long-range plans and communicating results is evident. There are also major opportunities for improvement in some areas, such as independent oversight of evaluation processes, following up to ensure implementation of recommendations, management of the function through a formal structure, policies and processes, and appropriate continuing professional development. We do not believe it useful to describe in detail the past structures, policies and practices, which we took into account in reaching our opinion, but we have considered the opportunities for improvement, for both CEU and IOM, in developing our recommendations for the new IO function. LOOKING TO THE FUTURE We suggest UNESCO consider having a brief implementation review performed approximately six to 12 months after the new Director of IO is hired. This might be done inhouse by persons competent in IO matters and independent of IO, or could be done with external assistance. We also recommend UNESCO have another QAR about three years from now, as specified in the Standards. We appreciate this opportunity to be of service to UNESCO. We will be pleased to respond to further questions you may have concerning this report and to furnish you any information you desire about IIA publications, training, consulting, or its other services to internal oversight professionals and their organizations. Lew Burnham, CPA Project Manager Robert A. Ferst, CIA, CISA, CFE Vice President, Global Services Integration and Quality Programs The Institute of Internal Auditors Team members: José Bouaniche, CIA, CISA DavidKanja,CIA,CPA,ACA Gerard Scalabre, CIA, CISA David Woodward, Dir. Ext. Aud., United Nations

7 160 EX/INF.6 page 4 GENERAL OVERVIEW OF QAR The Institute of Internal Auditors (IIA) conducted a Business-Focused Quality Assurance Review (QAR) of the internal audit, evaluation, investigation and related functions primarily the Office of the Inspector General (IOM) and the Central Evaluation Unit (CEU) of the United Nations Educational, Scientific and Cultural Organization (UNESCO). UNESCO is planning to combine those internal audit, evaluation, investigation and related functions into an integrated internal oversight (IO) function in the near future and plans to take the results of our QAR into account as part of that process of restructuring its internal monitoring (IM) activities. Our QAR was conducted in accordance with the IIA s Business-Focused QAR Manual. We made appropriate adaptations of our work programme to accommodate the particular circumstances of UNESCO. We also made adaptations to accommodate UNESCO management requests that we direct our attention to specific areas of interest or perceived needs of UNESCO s management, both in the planning stages of the review and during the course of our field work. Our preliminary visit was in April 2000 and the field work was performed in the period of August, culminating in reviews of our findings and the expected content of our report with the Director-General and appropriate members of UNESCO senior management. Our preliminary visit and our subsequent review of a variety of relevant documentation served to help us become familiar with UNESCO, gather background information and conduct brief interviews with selected executives. As part of that preliminary work, some relevant parts of the QAR Self-Study Report were completed by CEU and IOM, and a survey was sent to selected UNESCO executives. Results of the survey, including the comments returned with the survey responses and comments from our personal interviews with a representative sample of management and national delegations, were provided to UNESCO (without identification of information from individual respondents and interviewees). The interviews, covering a wide range of management control, sector and support operational practices, along with interviews with IOM and CEU personnel and reviews of their policy guides, working papers, reports, etc., form the principal basis of the observations and recommendations in this report The objectives of QAR included the following: Assess the efficiency and effectiveness of UNESCO s recent internal monitoring activities, including the current level of satisfaction of the customers of CEU and IOM, and the plans for their restructuring into a new, integrated IO function. Identify opportunities for improving the performance and credibility of current internal monitoring and planned future IO activities, including coordination with the external auditors and their increased reliance on IO work. Evaluate the proposed organizational structure and IO framework and comment on these matters, as well as the needed staffing and other resources. Provide an opinion as to whether or not the current IOM activities conform to the Standards for the Professional Practice of Internal Auditing. The objectives shown in our QAR proposal to UNESCO also included review your audit universe and the method followed for annual risk assessment... annual and long-range audit plans..., as well as examine internal auditing techniques and methodology... policies and

8 160 EX/INF.6 page 5 practices.... We did only limited work relating to these objectives, as the relevant universes, risk assessments, plans, techniques and methodology do not exist, or their practical application was so limited that it was not useful to try to review or evaluate them. However, we have commented on these matters as they relate to the planned restructuring of IO and implementation of such processes and techniques in the future. In connection with the first objective above, we relied primarily on the survey and interviews of selected UNESCO executives (42 survey responses and about 20 interviews), interviews with representatives of national delegations (5), discussion with the external auditor, a representative of the Auditor General s Office of Canada (EA), a review of recent internal reports and documents prepared for the Executive Board (EB). With respect to the first three objectives, we took into account relevant best practices on internal oversight functions, as collected and evaluated by IIA, along with ideas from the diverse background and experience of the members of the QAR team. We have identified opportunities for application of best practices and other improvements to the planned integrated IO function. These are presented in our report under two headings: (1) organizational, policy and management control issues for UNESCO as a whole, to be resolved by the Director-General or under his direct guidance, and (2) issues to be dealt with in planning, organizing, staffing and managing the integrated IO function. ENVIRONMENT FOR OVERSIGHT AT UNESCO In performing our review and formulating recommendations, we have borne in mind UNESCO s uniqueness as an organization and the special circumstances and requirements under which it operates. The QAR team members were selected, in part, because they have a great variety of experience with international organizations. This has been useful in addressing the broad range of oversight activities that will be under the authority of the new IO function, as well as in understanding the evolving structure and management processes of UNESCO as a whole. Several strengths of UNESCO, important to its future, were apparent to us particularly in the context of the planned reform of management structures and controls, as well as the strengthening of its internal oversight activities. These include: A widely recognized mandate covering a broad range of important activities, along with a history of valuable work and good credibility with most of UNESCO s stakeholders. Extensive networks and contacts, along with the expectation that UNESCO will continue to make use of these in pursuit of its objectives. A broad base of knowledge and expertise among UNESCO s executives and staff, and the operational and administrative structures, capable of enhancement, to make effective use of that knowledge and expertise. The will and executive direction to reform and enhance the management structure and processes.

9 160 EX/INF.6 page 6 As will be discussed in further detail in our comments and recommendations, successful establishment of the new IO function is very dependent on implementation of the plans to reform the structure and management controls at UNESCO, including significant improvements in management information. The reforms include a new UNESCO focus on strategic planning, clearer delegation of authority and related accountability processes, results-based programme management, decentralization and improved coordination of field activities (including a more extensive staff-rotation programme), and related efforts to strengthen management controls at all levels. In effect, if these reforms are successfully implemented, there will be an almost completely new framework for identifying and managing the risks and opportunities of the Organization. UNESCO management should be the full owner of the processes of identifying, assessing and managing these risks. If the related management controls are well-designed, fully implemented, and followed consistently at the various management and operational levels, there can be a less extensive additional layer of control provided by an oversight function. This illustrates the principle that cost-effective oversight is applied by exception, as a testing, qualityassurance and special-investigation mechanism. It should not be considered as an integral part of the primary system of management controls and accountability mechanisms. In Annex II, there is a brief conceptual outline of our suggested framework for risk evaluation, management control and oversight for UNESCO. That outline discusses further the concept that risk management, management control and accountability mechanisms should be entirely the responsibility of ( owned by ) management. Internal oversight should be functionally and organizationally distanced from those management responsibilities, but at the same time should align with those management processes the definition of its oversight universe, risk assessment, planning and application of oversight resources. In this way, managers become more control conscious and make better use of management tools and control processes, while the oversight function can deliver optimum added value in a cost-effective manner. This concept is expanded in several of our detailed recommendations. We also suggest that UNESCO consider adopting a general policy statement on management control concepts and responsibilities, based on Annex III. OBSERVATIONS AND RECOMMENDATIONS Our observations and recommendations are presented in two parts: (I) broad issues that concern UNESCO as a whole, requiring significant management decisions and actions, and/or that depend on the implementation of broad reform initiatives, and (II) those more specifically directed at the planned establishment of an integrated IO function and, while some of them may also be dependent on resolution of broader management issues, can be undertaken within the new IO function, with some direction and support by senior management.

10 PART I ORGANIZATIONAL ISSUES FOR UNESCO MANAGEMENT 160 EX/INF.6 page 7 1. DEFINE AND MEASURE RISKS; ENHANCE MANAGEMENT CONTROLS AND ACCOUNTABILITY PROCESSES Enhance the processes for defining and measuring risks; establish/enhance the framework of management authorities, controls, and accountability mechanisms, using a comprehensive risk management methodology. As mentioned earlier, in formulating our recommendations, we have taken into account UNESCO s extensive plans to reform the structure and management processes of the Organization. To the extent we have had an opportunity to familiarize ourselves with them, we concur with those reform plans and have tried to align our recommendations with them. A comprehensive risk management framework, with complementary controls and processes, can provide a necessary discipline to achieve better operating effectiveness, consistent performance measurement, and accountability at all levels of the Organization. For the specific purposes of IO, it is also a precondition for defining a comprehensive oversight universe, prioritizing the deployment of IO resources, and preparing longrange and annual oversight plans. See Annex II for a conceptual outline of a suggested framework and how the IO universe, risk assessment and planning would be aligned with it. The external auditors have also included useful comments in their long-form reports on the objectives and processes of a management control framework. In this reform context, we emphasize an important issue that was pointed out to us by a number of interviewees from both UNESCO management and the Executive Board: there is a need to fully integrate the deployment and administration of extrabudgetary funds into the new management framework. These trust activities should be under the related sector management responsibility and covered by the same level of accountability as is applicable to the regular sector programmes and projects. They should also be included in the systems of management information, budgetary and expenditure controls, results measurement, review and reporting by sector evaluation functions, internal oversight coverage, etc.

11 160 EX/INF.6 page 8 Audit management s response 2. IMPROVE THE TOOLS AND PROCESSES TO MANAGE INFORMATION TECHNOLOGY Improve the tools and processes to deal with the risks and opportunities related to the management of information technology (IT) and the implementation of an adequate management information system. This process improvement should include the updating and effective implementation of a long-range IT master plan, under the control of an in-house technology executive and the oversight of a permanent steering committee made up of major users. This enhanced control over IT resources is also important for IO to perform its function in a cost-effective manner. We are aware that there have been IT master plans developed in the past and that, to some extent, various elements of them have been put into effect. Some users have tried to manage the IT resources for their particular sectors or functions, directly or through reliance on consultants. With respect to SAP, responsibilities have been assigned and extensive work, particularly by consultants, has been done. The Director of Information Technology (DIT) also has been involved in some of these activities, along with routine systems maintenance responsibilities. However, we do not believe that all of these efforts, individually or taken together, constitute adequate overall management of IT resources or of individual projects such as SAP and SISTER. We have concluded that UNESCO must significantly enhance its management information at all levels, in order for adequate systems of management control and accountability to function. Unfortunately, there is likely to be a period of up to two years in limbo before that can happen. The legacy systems are outdated and it is not feasible to enhance them or even maintain them adequately. SISTER, while very good in concept and implemented to a degree, does not appear to have access to the full range of data needed and its effective application across the Organization does not appear to be occurring. It appears that SAP implementation can fill a major part of UNESCO s management information needs, provided these needs can be defined in a consistent manner across the control and accountability framework and management and provided staff are properly trained to use the system at a reasonably advanced level. We emphasize that DIT should have more involvement in the management of IT resources, under the direction of a senior executive such as a Chief Information Officer and with an IT

12 160 EX/INF.6 page 9 steering committee of users to exercise strong oversight. On an ongoing basis, the steering committee and DIT should apply a strong central discipline to the major elements of the IT master plan. These elements include: the overall IT budget and user needs assessments (not just left to individual users), acquisition/development and implementation of significant new IT resources (enforcing requirements such as system compatibility, operating economy and effectiveness, and management of IT consultants particularly to ensure that their knowledge and tools are retained by UNESCO, not lost when the consultants leave), ongoing risk assessment and effectiveness evaluation of software currently in use, along with reviews of controls to ensure data integrity and logical and physical security, and overall IT risk management, including implementation and testing of a disaster recovery and business resumption plan. We have furnished to DIT extensive information of management of IT resources, particularly on the COBIT (Control Objectives for Information and related Technology) framework, from the Information Systems Audit and Control Foundation. See also the essentials of IT management shown in Annex II. Audit management s response 3. DEVELOP A CHARTER FOR THE NEW INTERNAL OVERSIGHT (IO) FUNCTION AND DEFINE ITS STRUCTURE Develop a charter for the integrated IO function and define its structure, which could be adopted on a provisional basis pending arrival of the new Director of IO (DIO), in line with the model in Annex I. We suggest that current CEU and IOM staff, under the close direction of senior management of UNESCO, begin outlining and drafting the main features of this charter immediately. In addition to the standard elements of the charter in Annex I, UNESCO should give consideration to the following: Define the authority and role of IO in the coordination and oversight of the evaluation functions that are to be the responsibility of management. Cover such matters as guidance for sector and support unit evaluators and the consultants they may hire (eventually in the form of a policy and procedure manual to be adopted by the sector and support unit evaluation functions), review of the two-year evaluation

13 160 EX/INF.6 page 10 master plan, tracking and follow-up of implementation of the recommendations and agreed action plans resulting from evaluation reports, preparation of a semi-annual summary of significant report matters, etc. Set out the responsibilities and boundaries of IO investigation activities, including when they should turn the responsibility over to the Legal Department (e.g. when IO finds evidence or has strong suspicions of serious violations of UNESCO regulations or criminal actions), when the Personnel Department should be involved, and how IO should continue to cooperate in such investigations. Describe the formal and informal processes by which IO should take into account the advice and requests of management, in assessing its oversight universe, setting priorities and planning work. One of the alternatives being considered is to have a formal internal oversight committee made up of the heads of support services with control oversight responsibilities (Bureau of the Budget, Strategic Planning, and Bureau of the Comptroller) and of the sector heads. This committee would function as an IO advisory panel and help IO gain wider acceptance of its work. It would also help spread best practice ideas arising from IO reports and ensure that agreed action plans are implemented. We think this internal oversight committee is a good concept, and, in fact, it is considered a best practice by many organizations. But, we wonder whether at UNESCO such a high-powered group would be willing to regularly attend meetings (or would send low-level delegates and defeat the committee s purpose). Also, such a committee might give the impression that IO is controlled, or unduly influenced by, the subjects of its oversight work. At UNESCO, it would appear that a better alternative may be to occasionally give IO a place on the agenda of regular management meetings. Those would be valuable opportunities to communicate with and receive advice from management as a group. More targeted communication and input from IO customers would be by means of periodic meetings of the DIO and/or designated IO liaison persons with individual heads of sectors and support services units. Establish a staff rotation policy to move high-potential people with operational (especially field) experience into IO. The charter should also mention IO s authority to obtain shortterm resources and cooperation, through temporary assignments of appropriate staff to partner with IO staff on team engagements in selected areas of the organization.

14 160 EX/INF.6 page 11 Further, it should mention, and complement with budget resources, IO s authorization to outsource some of its work, employ short-term consultants, secondees from other organizations, etc. In establishing the organization chart for IO, UNESCO should recognize that while multiple functions or disciplines can be described, it is not necessary to segment the Organization along those multiple lines. Given the likely priorities of IO, as is indicated by the outline of risks in Annex II, we believe IO will have to concentrate more on operational controls, programme effectiveness, quality assurance, process improvement, management information, etc. Also, it will need to recruit and/or train multidisciplinary staff and its senior staff will have to be able to work and supervise in all areas. The staff will be smaller initially, with uncertainties as to how many will stay and what will be the source and qualifications of their replacements. Consequently, we suggest that an interim sub-director be appointed only for the ongoing evaluation work, to ensure its continuity and credibility, and that other potential posts at that level be left open until the organizational picture is clearer. An important supplement to the charter will be the position description of the DIO in alignment with the charter and designed as a document to communicate to UNESCO management the importance of IO. With respect to the management level of the DIO, we believe the D-2 grade is appropriate, but suggest that future consideration be given to elevating the position to the ADG level. Audit management s response 4. COMMENCE IMPLEMENTATION OF THE IO FUNCTION NOW UNESCO should commence implementation of the new IO function now, even before the new DIO is appointed. Some areas where we believe this would be worthwhile, particularly for the Director of CEU and the acting Inspector General working together, would be: Consider areas of common interest and oversight coverage, both with respect to a tentative outline of a combined universe and identifying the potential synergies from using combined teams on oversight engagements.

15 160 EX/INF.6 page 12 Begin the process of identifying the complete audit universe and performing risk assessments which will lead to an appropriate schedule for Assistance may be required for this effort. Compare policies and procedures and identify gaps that will need to be covered when a new departmental manual is prepared. Design and test new methodologies. Assess the skills of staff, their needs for additional guidance or supervision, and determine the short-term training that may be feasible to enhance those skills, make them more marketable (if they are deemed not to fit into IO). Arrange to have the required short-term training conducted. Discuss potential internal and external sources of new staff. Determine IO s needs for new technology and the alternatives available to fill those needs (including the requisite training). Meet with IO stakeholders (principally management of sectors and support services) to explain how IO is changing and get their preliminary input to assist in identifying and evaluating elements of the IO universe. Consider what the Director-General should be asked to say to management at this time, both formally and informally, to help the IO reform process along. Audit management s response 5. DEFINE THE IO RELATIONSHIP AND INFORMATION LINKS TO THE EXECUTIVE BOARD UNESCO has been considering various alternatives in this matter, ranging from no direct relationship or reporting to the establishment of a formal audit committee to which the DIO would have a link (along with a primary reporting line to the Director-General). We believe a reasonable compromise, which improves transparency while leaving the reporting line to the Director-General essentially undisturbed, would have the following features: Have the DIO prepare a semi-annual report of significant IO recommendations and remedial actions. Address it to the Director-General, with copies widely distributed to other members of senior management, as a routine means of keeping them informed of IO activities and spread best practices and other good ideas across the Organization. Have this report coincide with the Executive Board meetings and

16 160 EX/INF.6 page 13 include it in the package of advance information to Executive Board Members. Include a cover letter of comments, additional information, etc., from the Director-General, if he deems this desirable. Similarly, have the DIO prepare an annual summary of the IO annual plan (including comments on adequacy of resources and accomplishment of the prior year s plan), also directed to the Director-General and senior management. Furnish a copy of this annual summary in the advance package for the Executive Board meeting. Advise the Executive Board of major changes in IO authority, scope, or resources. Advise the Executive Board of plans to appoint a new DIO or to remove the incumbent. PART II ISSUES SPECIFIC TO THE INTERNAL OVERSIGHT FUNCTION 6. DEFINE THE OVERSIGHT UNIVERSE AND ASSESS THE RELATED RISKS Define the oversight universe and assess the related risks aligning this universe with that of risk management and management control for UNESCO as a whole. The concept and steps to be taken are summarized in Annex II, along with the subsequent steps applicable to our recommendation No. 7 on oversight planning, below. After an initial generalized outline of the oversight risk universe, we suggest IO obtain an audit risk assessment software package to assist in evaluating and comparing oversight risks among the many different units of the universe. Our recommendation is based partly on numerous comments from the survey and interviews of UNESCO management. They said that IOM needs to redirect its resources to more important and higher risk work, by which the interviewees usually meant more operational audits, as opposed to transaction and administrative compliance audit work. It is also based on our conclusions as to what would likely be of most value to UNESCO in areas where the significant control risks are (as indicated in Annex II) as well as being in line with current best practice of the profession. The same comments on the subject of extrabudgetary funds in No. 1, above, would also apply to the process of defining the universe and assessing the risks for IO. These extrabudgetary activities would be incorporated into IO risk assessment and planning, in alignment with the way management responsibilities and accountability processes are restructured.

17 160 EX/INF.6 page 14 Audit management s response 7. PREPARE LONG- RANGE AND ANNUAL IO PLANS Prepare long-range and annual IO plans, prioritizing the use of IO resources based on an oversight risk assessment and consultation with sector and support units across UNESCO. IO should use a comprehensive structure of risk factors and weighting (which can be provided by an audit risk assessment software package, mentioned above) for overall setting of priorities and managing changes in the annual plan of IO engagements. This formal modelling process imposes a strong discipline both on the initial allocation of resources and on determining whether potential expansion of the scope of a planned engagement, agreeing to a management request for additional work, or substituting an unplanned engagement should take precedence over engagements already planned. For illustrative purposes, a description of some of the significant processes and information relevant to the universe and resulting plansisgivenbelow: The oversight universe should list its units in terms of potential oversight engagements (subsector activities, programmes, major projects, regions, offices, support service areas, systems, transaction cycles, etc.), with appropriate subclassifications by types of work (scope). An indication of relevant factors such as materiality, nature and magnitude of potential threats and opportunities, other relevant indicators of risk (of various types such as: strategic, operational, reputation/credibility, financial) and management concerns, should also be shown for each unit. Eventually, there should be built up a history for each unit, showing date of previous IO work (with information about scope, person in charge, major findings and/or other indicators of the quality of management controls). The long-range plan should show how all of the higher priority units will be covered over the long-range planning cycle and how the lower priority units will be tested by coverage of a representative sample of them. It need show only summary information. It should be updated ( rolled forward ) each year. The annual plan should show the scope of work, timing, type and amount (e.g. person days) of staff resources to be allocated, tentative assignment of IO staff person in charge, and similar summary information for each planned engagement.

18 160 EX/INF.6 page 15 As already mentioned in No. 3 above and Annex II, IO should consult frequently with management of the sectors and support services, both as a group and individually, with respect to preparation and subsequent modifications of its annual plan. Audit management s response 8. ENHANCE EVALUATION OVERSIGHT ACTIVITIES THROUGH THEIR INTEGRATION INTO IO The survey and interviews with management and Executive Board Members indicated that CEU and the sector evaluation and coordination activities generally are held in high regard. There was some concern expressed that the evaluation function might disappear into the combined IO function or might lose effectiveness or credibility through a weakening of its mandate (paraphrased). For this reason, it is essential that there be prompt communication to those concerned that these fears will not come to pass. And, on the contrary, that evaluation in UNESCO will be strengthened through a clear separation of sector evaluation to serve management process improvement and accountability purposes from evaluation oversight. The merging of evaluation into IO will be an orderly, well-thought-out process. In addition, there will be a continuation and eventual enhancement of the evaluation capabilities of IO (through better rotation of capable staff with sector and field experience) and it will exercise a stronger discipline on the process through: independent review and challenging of the two-year sector evaluation plan and expansion of oversight to such areas as review of strategic policy analysis, more specific and consistent guidance to sector evaluation and coordination staff (and the consultants they may employ) through the completion and issuance of an evaluation policies and procedures guidebook, participation in and supervision of selected evaluations that are in the two-year plan, as well as independent performance of a few additional evaluation reviews, extraction of significant issues, recommendations, practices applicable across other sectors, etc., from sector evaluation reports and publish them in a semi-annual report of significant oversight matters (which will be sent to the Director-General, widely distributed to UNESCO management, and furnished to the Executive Board as part of the document package for their regular meetings), and

19 160 EX/INF.6 page 16 tracking and implementation follow-up of recommendations and/or agreed action plans. In addition, we suggest the Director-General consider having IO attest annually as to the adequacy of UNESCO s evaluation processes for sector activities. Audit management s response 9. ESTABLISH APPROPRIATE IO METHODOLOGIES ANDANEW POLICIES AND PROCEDURES MANUAL These methodologies and the related policies and procedures should reflect adherence to professional standards and best practices (from the IIA and relevant Evaluation Association literature on hand or readily available to UNESCO). Some of the significant areas to be included in this upgrading effort are: planning individual engagements (preliminary discussions with relevant management and staff; preparation of a brief planning document covering items carried over from prior engagements, priorities, scope, objectives, timing, time budget, staffing, other relevant engagement administration matters and reporting matters), process for modifying the engagement plan (e.g. expanding/decreasing scope), direction and oversight of staff/consultants from sources outside IO, work paper standards including general form and content, indexing and cross-referencing, type of information to be retained, memoranda of work done and conclusions, and observations and potential recommendations, reporting matters and person assigned to review working papers and report draft, and engagement management, including travel policies and expense reporting, time budgeting and budget/actual control, performance evaluation and process improvement, etc. Some of these matters have been drafted and/or exist in final form (though not being consistently applied) in IOM and CEU. These should be collected, reviewed, considered for updating and/or combining, etc., so as to have them ready for review by the new DIO. Additional policy/procedure matters, relating to staffing, reporting and quality assurance are covered in Nos , below.

20 160 EX/INF.6 page 17 Audit management s response 10. ASSESS STAFF NUMBERS, SOURCES AND SKILLS; EXPAND CONTINUING PROFESSIONAL DEVELOPMENT We understand the current combined headcount of IOM and CEU is 10 (professional staff). We believe this is not a sufficient number to staff the new IO function. It is difficult to judge at this time how many of the current staff have the capabilities to perform satisfactorily in the new IO environment or, indeed, how many will wish to remain in IO. In No. 4, above, we mentioned some preliminary steps to review current staff, consider training, determine potential sources of new staff, etc., prior to the arrival of the new DIO. In addition, we have the following suggestions relating to longer range staff management: Prepare job descriptions of staff, with a focus on broader skill sets and with the target of having staff with capabilities to work on a wide variety of assignments. Cross-train staff with both internal audit (IIA and others) and evaluation (Evaluation Association) materials. Hold frequent (at least quarterly) staff meetings, for discussion of IO policies and practices and presentation of new ideas and techniques (by staff members or outside professionals). Make staff responsible for their own continuing professional education plans, subject to review by the DIO or a designated specialist, for reasonableness and consistency. Provide training opportunities above the IIA average (60 hours per year according to GAIN, the IIA s benchmarking service) for an initial catch-up period and at the level of hours per year thereafter. Provide all staff with IT tools and training, particularly for data extraction and analysis, general IT controls and effectiveness reviews (at a moderate technical level), and automated preparation of routine work papers. (See also No. 12, below.) Given the likely priorities and changes of emphasis of the new IO, we believe that new additions to its staff should come more from internal operational areas than from recruitment outside UNESCO (although initial recruitment of a few staff with broad internal audit backgrounds will be appropriate). IO should become an important career development step for high-potential operations and support services staff, as this will provide one of the best opportunities to learn about management controls and processes across the whole of UNESCO.

21 160 EX/INF.6 page 18 We have been asked to comment on the approximate number needed and potential sources of staff for IO. The numbers are dependent on several variables of reform actions and timing, but with that caveat, we would estimate that permanent professional staff should be in the range of 12-15, with sources outside IO providing staffing resources to bring the equivalent total to around Those outside sources could be: short-term assignment of staff from sectors and support service units to assist IO with specific evaluations, reviews, etc., in their own or other areas team audits or partnering with IO customers, short- or long-term secondment of staff from other United Nations agencies or member countries, joint engagements with oversight units of other United Nations agencies, in operations areas of common interest, external consultants and other professionals contracted on a short-term basis, and engagements outsourced to professional firms, under the terms of reference and other oversight of IO. Audit management s response 11. ASSESS REPORT FORMAT(S), REVIEW PROCESS AND DISTRIBUTION; IMPROVE FOLLOW-UP OF IMPLEMENTATION OF AGREED REMEDIAL ACTIONS In the professional literature of internal auditing and evaluation there are plenty of good ideas about report formats and processes. We do not propose to recommend any particular reporting style for UNESCO, except to suggest three ideas for consideration: Adopt the policy that the field work on an engagement is not to be considered complete until the recommendations have been finalized sufficiently to present them in the closing conference and discuss (if possible, agree) the remedial actions to be taken. Include the action plans in the report, to the extent they can be agreed; if they can t be agreed, at least set forth the factors that prevent this (budgetary considerations, further study to be undertaken, difference of opinion or interpretation, etc.) and plans to resolve these matters. Use the report to promote and reinforce management plans or desired actions (with which IO agrees); if these are in the form of recommendations or agreed action plans, be sure to give credit where it is due. We understand past policy/practice has been that reports are not issued without advance review and approval by the Director-

22 160 EX/INF.6 page 19 General, especially if there are recommendations. We agree that reports involving significant decisions and/or resources may need to be handled in this way. However, we believe most reports should be issued directly to the sector ADG or head of the support service unit, particularly where there is agreement on recommendations or actions. Such a change would be not only a step to make reporting more timely, but would also be a signal that the new decentralization and delegation of authority initiatives are taking effect. The concept and purposes of the semi-annual summary report of significant matters are covered in No. 5, above. In addition, we would suggest that this report will encourage implementation of recommendations (action plans), facilitate IO follow-up, promote transparency of management activities, and increase both the visibility and credibility of IO. Audit management s response 12. GIVE PRIORITY ATTENTION TO STRENGTHENING THE INFORMATION TECHNOLOGY (IT) AUDIT CAPABILITIES OF IO AND ITS COVERAGE OF IT ACTIVITIES THROUGHOUT UNESCO As can be inferred by our comments in No. 2 above, Annex I, and elsewhere in our report, IT resources and the related management information are very important to UNESCO. The management of these resources will be strengthened by the reform initiatives under way, including adoption of our recommendations in No. 2. As a complement to those control and quality improvements, they need to be part of the coverage of the new IO function. In particular, they need to be considered as an important part of the oversight universe, oversight risk assessment, and planning, discussed in Nos. 6 and 7 above. Only limited IT audit work has been performed during the past several years. There is a staff member with EDP audit background and experience. That staff member s capabilities and effectiveness need to be enhanced through training, supervision and direction of those capabilities into high-priority oversight work. This should include plans, system development and implementation, operations, administration, security and other aspects of the management of IT as well as the utilization of IT resources to provide management information throughout UNESCO. We believe the scope of IT audit work in UNESCO, including assistance to IO management and other staff in better using technology for performing IO work, will require the services of at least one more IT auditor. We suggest the necessary steps to obtain this additional staffing from within UNESCO or recruitment from outside be undertaken promptly.