Review of the Education Sector s Category I Institutes

Internal Oversight Service Evaluation Section IOS/EVS/PI/118 REV. Original: English Review of the Education Sector s Category I Institutes Synthesis Report IOS/EVS February 2013

Category I Institutes are important delivery mechanisms in key areas of UNESCO s mandate in Education, and their areas of work are likely to remain highly relevant beyond 2015. However, currently the majority of the six Institutes under review (with the exception of IIEP and to some extent UIL) lack a sound financial base and critical mass of expertise for becoming centers of excellence and for effectively delivering upon their global or regional mandates. For the short term, recommendations include: a need for strengthening the Institutes strategic focus; enhancing donor, host country and UNESCO support; and the implementation of a number of institutional reforms within the Institutes. In the medium term, the future of UNESCO s engagement with some of the Institutes should be conditional on their performance record and their ability to harness international support to increase the scale and outreach of their activities. Purpose and approach 1. A review of six Education Sector Category I Institutes (IIEP, UIL, IBE, IITE, IESALC, IICBA) was carried out in collaboration with the Education Sector. The aim of the review was to assess specific key aspects of the Institutes performance, their achievements and within the context of the implementation of the overall strategy for Category I Institutes and Centers. 1 In its findings and recommendations the review also considered the progress achieved in the implementation of the recommendations of the previous evaluations from 2005/2006. 2 The review was designed to inform the Education Sector s decision-making process towards strengthening the overall framework for cooperation with (and among) the Institutes as well as the strategic allocation of resources and capacities. In terms of scope the review covered the Institutes mandate and relevance, the results achieved, collaboration and interaction with partners within the UNESCO system and beyond, management and governance mechanisms, as well as sustainability (including staffing, budgetary issues and host country support). Data collection and analysis were based on in-depth document review, self-assessment, and interviews with staff members (including management), representatives of the Institutes Governing Bodies and Education Sector senior managers, as well as representatives from Field Offices, other Education Institutes and external stakeholders. Findings are based on triangulation between observation, interview data and documentary evidence. Key findings 2. Overall, the Institutes mandates and activities are in alignment with priority areas of the mandate of the Education Sector, and are likely to remain of high relevance beyond 2015. However, the Institutes (with the exception of IIEP) have not been able to position themselves clearly in their areas of expertise with negative implications for their visibility and reputation. Moreover, there has been little strategic guidance and a lack of clarity from the Education Sector as to the Institutes expected contributions in their respective areas of work. Within this perspective, the Institutes and the Education Sector (including Field Offices) so far have not been able to establish a clear division of roles and responsibilities, in some cases resulting in parallel and uncoordinated activities, information asymmetries, and a lack of clarity among UNESCO s partners and stakeholders. 3. Capacity development, (policy-oriented) research and knowledge sharing are the three core functions of the Institutes. Of these three, the Institutes have been relatively successful in the areas of capacity development and knowledge 1 33 C/19 and 171 EX/18. 2 See evaluation reports of Education Sector Category I Institutes, 2005/2006. 1

sharing. By contrast, with the exception of IIEP, the research function has been quite weak. In all three areas major remain. With respect to capacity development, key relate to the modalities, target audiences and outreach of activities. Moreover, capacity development often lacks a clear institutional development perspective, for example through a partnership-oriented approach to capacity development. Regarding knowledge sharing, several Institutes have built up rich repositories of knowledge and have been quite successful in making these available to different stakeholders. However, the Institutes lack a targeted knowledge management approach, which includes developing more effective means of communication as well as a focus on comparative and synthetic analyses of existing research with the explicit aim of influencing policy. IIEP is the only Institute with a credible in-house research capacity. While it is unrealistic to assume that all Institutes build up such capacities, research objectives can be realized through better and more collaboration with reputable research partners. 4. In all Institutes, the management function has proven to be a crucial variable in the overall success of an Institute. There have been cases where discontinuities in management, lack of performance targets and accountability mechanisms for the Director have negatively affected performance and reputation while in other instances the presence of key leadership and management skills at the (deputy)director level have proven to be instrumental to success. 5. The current size and composition of the Governing Bodies as well as the modalities of work are not the most optimal to ensure efficient and effective governance. Nonetheless, overall, the Governing Bodies of the Institutes are composed of high-level experts of international repute and have provided substantive guidance on the Institutes areas of work. 6. In terms of learning and accountability, most Institutes (with the exception of IIEP) lack a clear and coherent results-based framework (intervention logic) to consistently guide monitoring and reporting of their activities, outputs and achievement of expected results. 7. Given the recent overall decrease in UNESCO s core funding to the Institutes, the financial sustainability of all Institutes is at risk to various degrees. The situation is particularly critical for those Institutes (IBE, IESALC, IITE, IICBA) which are financially highly dependent on UNESCO funding as they have not managed to mobilize significant amounts of extrabudgetary funding. Limited capacities for fundraising, the absence of targeted fundraising strategies and a lack of coordination with fundraising activities at UNESCO Headquarters are among the main. The degree of host country support has proved to be a decisive variable in the financial sustainability of the Institutes. However, until recently, there have been few attempts to establish strategic dialogues between the Institutes, the Education Sector, host country governments and other potential donors with the aim of strengthening the Institutes financial sustainability. Concluding remarks and overall recommendations 8. Category I Institutes are important delivery mechanisms in key areas of UNESCO s mandate in Education, and their areas of work are likely to remain highly relevant beyond 2015. However, currently the majority of the six Institutes under review (with the exception of IIEP and to some extent UIL) lack a sound financial base and critical mass of expertise for becoming centers of excellence and for effectively delivering upon their global or regional mandates. The Institutes sustainability and capacity to make a difference depend on concerted efforts by three main stakeholders: the Institutes; 2

UNESCO s Education Sector; and UNESCO Member States, in particular the host countries of the Institutes. 9. The role of UNESCO Member States. Host country support for the Institutes is a critical variable for ensuring the financial viability of the Institutes activities. Enhanced support from host country governments and of the Member States is needed to strengthen the operating scale and financial sustainability of the Category I Institutes. The fundamental capacity for Institutes to deliver upon their mandates in a functionally autonomous manner depends on this support. Moreover, for UNESCO s catalytic support (in terms of funding and human resources) to be effective, a sound foundation needs to be in place. 10. The role of the Institutes. The individual reports of the reviews of the six Institutes include Institute-specific recommendations and are available on the IOS website. Annexes 1 and 2 present a summarized picture of, respectively, key findings and recommendations for each of the six Institutes. For the short term, recommendations include: a need for strengthening the Institutes strategic focus; enhancing donor support; and the implementation of a number of institutional reforms within the Institutes. 11. The Role of UNESCO s Education Sector. In the medium term, the future of UNESCO s engagement with some of the Institutes (i.e. IITE, IESALC and IBE) should be conditional on their performance record and their ability to harness international support to increase the scale and outreach of their activities. Below, we outline the principal recommendations regarding UNESCO s overall strategy and approach towards Institutes. The strategy should be determined by three criteria: first, the financial situation of the Organization, and more particularly the long-term availability of financial resources for supporting the Institutes; second, the extent of alignment and integration of the Institutes activities with the Education Sector s priority areas of work, with implications for the division of labor, complementarity and achievement of economies of scale; third, the performance and impact of the Institutes activities in key areas of the Education Sector s mandate. 12. Within the framework of these criteria, the review has arrived at the following overall conclusions for UNESCO s Education Sector. The current financial situation is critical, as a result of which, more than ever efficiency and effectiveness considerations are driving a process of prioritization of the Education Sector s overall resource allocation. Moreover, given the financial constraints, the Education Sector can only afford to support a network of Institutes if there is no overlap or competition between any of the UNESCO entities and if a clear and feasible modus of collaboration and division of labor can be realized. The current review has identified a number of comparative advantages of the Institutes. However, these have to be further strengthened, which can only happen if the division of labor and strategic roles of the Institutes within the overall Education strategy are clarified. Finally, the review has found that there are substantial differences in the levels of performance and impact among the Institutes, with most of the Institutes requiring (at least in the short term) further UNESCO support and strategic investments to take them to a level of operating scale and capacity that is needed to make a difference. 13. Given the above, there is a key role for the Education Sector to play in terms of supporting all the Institutes (at least in the short term), but continued support should be conditional on future performance and impact. Consequently, the review recommends that the overall budget allocated to the Category I Institutes should remain at the same level (in the short/medium term). At the same time, the Education Sector should consider providing other forms of support (e.g. clarification of roles and responsibilities, technical support from Headquarters, support for fundraising efforts) to the Institutes in line with the recommendations of the individual review reports and 3

improvement plans. In the long term, the Education Sector should invest its scarce resources in those areas and those Institutes where UNESCO can make the biggest difference. An assessment of the extent to which particular targets have been achieved (i.e. targets established in the improvement plans which have been or are going to be defined for some of the Institutes under review) should inform the Education Sector s decision-making to revise the future financial allocation in favor of those Institutes that have performed well. In addition, it should consider discontinuing financial support and reconsidering the Category I status for those Institutes which have not achieved (or are not close to achieving) the agreed upon targets. 14. More specifically, and taking into account the above, the review foresees the following scenarios for UNESCO financial support to each of the six Institutes under review. 15. IIEP (International Institute for Educational Planning, Paris) is the only Institute which lives up to the expectations of a global center of excellence. In terms of scale, focus, and capacity it is the best-performing UNESCO Education Institute. Notwithstanding its achievements the Institute is currently facing a difficult financial situation. The recommended scenario for UNESCO support is to maintain the current level of funding and to consider an increase in the future. 16. IBE (International Bureau of Education, Geneva) has been struggling to effectively deliver upon its global mandate. Moreover, its financial situation is unsustainable. Given the crosscutting importance of IBE s mandate in curriculum reform, UNESCO s General Conference approved a strategy designed to develop IBE into a center of excellence in curriculum. This strategy includes an improvement plan with clear targets. The current review reaffirms the importance of the measures laid out in the strategy, which are in line with the review s recommendations. The recommended scenario for UNESCO support is to maintain the current level of funding and pending assessment of achievement of the targets defined in the improvement plan revise the allocation. In case IBE does not meet these targets, UNESCO should reconsider IBE s status as a Category I Institute, e.g. by exploring alternative solutions with the host country. In case of meeting the targets, given the current relatively high allocation of financial resources to IBE (the highest compared to its total size), UNESCO should consider revising its financial allocation to IBE in the future. 17. With respect to IICBA (International Institute for Capacity Building in Africa, Addis Ababa), there is a clear mismatch between the scope and importance of IICBA s mandate on the one hand and IICBA s achievements on the other. IICBA is the only specialized institution within the UN system on teacher development in the African Region. Given UNESCO s Global Priority Africa and the importance of Teacher Development in the Education Sector s mandate, the review supports the case for strengthening the Institute and presents a number of recommendations for the Education Sector and IICBA on how to reinforce the Institute s capacities, performance and basis for sustainability. Within this perspective it is recommended that UNESCO increases the financial support to the Institute with a view to investing in its comparative advantages and tapping the full potential of the Institute s mandate. 18. UIL (UNESCO Institute for Lifelong Learning, Hamburg) has undergone a positive evolution in terms of strengthening its capacities and increasing the focus, outreach and effectiveness of its activities. Moreover, it has been successful in securing a sustainable funding base. Nevertheless, a number of remain on the path towards becoming a center of excellence in lifelong learning and ongoing UNESCO support is essential. In order to sustain the current positive development process, it is recommended to maintain the current level of funding and over time consider increasing its financial support to the Institute. 4

19. IESALC (International Institute for Higher Education in Latin America and the Caribbean, Caracas), until recently, has suffered from discontinuities in the Institute s management and, with limited human and financial resources, has not been able to effectively deliver upon its mandate. Moreover, its current financial situation is unsustainable. In the 34 C/4 period Higher Education has not been among the Education Sector s key priorities. Furthermore, the Institute s regional mandate seems hard to justify given the limited resources and the absence of specialized capacities in Higher Education in other Regions (e.g. Africa). Consequently, from a resource optimization perspective serious doubts can be raised about the rationale for continued UNESCO support to the Institute. IESALC in consultation with the Education Sector has been requested to develop a one-year improvement plan with clear targets. The recommended scenario for UNESCO support is to maintain the current level of funding and pending assessment of achievement of the targets defined in the improvement plan revise the allocation. In case IESALC does not meet these targets, UNESCO should reconsider IESALC s status as a Category I Institute, e.g. by exploring alternative solutions with the host country. In case of meeting the targets, given the relatively high allocation of financial resources to the Institute, UNESCO should consider revising its financial allocation to IESALC in the future. 20. IITE (Institute for Information Technologies in Education, Moscow) has undergone a positive evolution in recent years which has put the Institute on a more secure footing and the financial sustainability is currently not at risk. Nevertheless, IITE is far away from becoming a center of excellence and lacks the human and financial capacities to respond to its global mandate. Substantial efforts from all stakeholders (principally IITE, UNESCO and the host country government) are needed in order to enable the Institute to make a difference. IITE in consultation with the Education Sector has been requested to develop a one-year improvement plan with clear targets. It is recommended for UNESCO to maintain the current level of funding and pending assessment of achievement of the targets defined in the improvement plan to revise the allocation. In case IITE does not meet these targets, UNESCO should reconsider IITE s status as a Category I Institute, e.g. by exploring alternative solutions with the host country. In case of meeting the targets, UNESCO should maintain the current level of funding. 21. The Education Sector has actively participated in the review process and has already started developing and implementing reforms designed to address the identified in the review. The systemic and self-critical approach adopted by the Education Sector reflects the overall needs and reform priorities of the Organization in such areas as strengthening strategic guidance (to be reflected in UNESCO s Strategy for Education 2021) and governance. The review strongly commends the Sector s initiative and solicits the Institutes and UNESCO s Governing Bodies to endorse the reform proposals and support the ongoing efforts for improvement. 5

ANNEX 1: Summary of findings across the six Institutes under review IIEP UIL IBE IICBA IESALC IITE Scope global global global regional regional regional (formally global) Number of staff (2012) 114 32 20 15 17 15 Budget (2010-2011 biennium) in USD 36,72 million 9,31 million 10,30 million 3,25 million 2,85 million 2,41 million UNESCO allocation (2010-2011 biennium) in USD 5,30 million 2,00 million 4,80 million 2,50 million 2,00 million 0,90 million Main area of work Educational planning Lifelong learning Relevance The area of work is at the core of UNESCO s education strategy yes yes (increasing) Curriculum reform yes (crosscutting) Teacher development yes Higher education unclear (declining) ICT in education yes (increasing) The Institute s mandate within the Education Sector s strategy in the Institute s mostly mostly partly partly partly partly area of work is clear The Institute is the key UNESCO entity fulfilling UNESCO s mandate in area of yes yes partly no no no work in which Institute is operating UNESCO has a distinct comparative advantage / is occupying a niche position yes yes partly yes no partly within the area of work that the Institute is active The Institute enjoys global / regional recognition yes (global) yes (global) limited (global) limited (regional) limited (regional) limited (regional) The Institute s capacities and resources are commensurate with the Institute s realistic realistic unrealistic unrealistic unrealistic unrealistic mandate Results, outputs and activities The Institute has a clear strategic focus in its activities no no no no no no The Institute has a strong capacity development function The Institute has a strong laboratory of ideas function strong with strong with moderate moderate moderate weak weak weak weak weak weak weak The Institute has a strong standard setting function no incipient no no inactive incipient The Institute has a strong catalyst for international cooperation function moderate strong strong with The Institute has a strong clearing house function strong with strong with strong with weak weak weak weak moderate moderate Quality of coordination and collaboration Institutional (non-financial) support from Headquarters to the Institute has been strong low / low / low / low / 6

IIEP UIL IBE IICBA IESALC IITE Collaboration at programme / activity level between the Institute and considerable moderate moderate low low low Headquarters has been strong The division of labour between the Institute and Headquarters has been clear mostly / mostly no no partly no There has been a strong collaboration with other Institutes considerable moderate moderate low low low There has been a strong collaboration with Field Offices considerable increasing increasing moderate low moderate The division of labour between Institutes and Field Offices has been clear unclear unclear unclear unclear / attempts to clarify unclear UNESCO has been visible and has clear positioned itself among external clear mostly clear unclear unclear at regional stakeholders in the area of work in which the Institute operates level clear The Institute has a clear partnership strategy / approach yes yes no partly no yes unclear at regional level clear Management, governance and accountability Management has been highly effective yes yes no no no yes Functional autonomy has been an issue yes yes partly no no no The governing board has been highly efficient and effective high high low moderate low high Tools and mechanisms for evaluation and results-reporting are adequate yes / Tools and mechanisms for monitoring are adequate yes / Tools and mechanisms for planning are adequate moderate Staffing The Institute has a minimum critical mass of expertise in key areas of its mandate Staffing conditions are adequate (collaborative spirit, labour conditions, workload, recruitment and promotion) Budget no no no no no no no no no no / no no moderate yes partly no no no no mostly mostly partly (not w, r, p) no (not c, r, p) partly (not l c) partly (not c, l c) The UNESCO financial allocation is the principal funding source of the Institute no no yes yes yes no The UNESCO financial allocation is sufficient to cover UNESCO posts no no no under no under pressure pressure The Institute has managed to establish diversified and sustainable (extrabudgetary) funding lines (excl. host country support) yes / under pressure yes no / incipient improvement no no no / incipient improvement There is strong host country financial support for the Institute s premises and associated costs yes / but could be yes no / UNESCO owns premises yes / building under yes yes / maintenance There is strong host country financial support for programmes and (programme) staff costs strengthened partial / could be strenghtened partial / could be strenghtened construction inadequate increasing partial / could be strenghtened costs an issue partial / could be strenghtened 7

Annex 2: Summary of systemic recommendations concerning the six Institutes under review The following table presents key systemic recommendations, indicating for each recommendation to which entity (Education Sector, specific Institute) it applies. The wording of the recommendations may differ from the exact wording used in the Institute-specific recommendations, but captures the principal message of the latter. Education Sector IIEP UIL IBE IITE IESALC IICBA Institutes as delivery mechanisms of UNESCO s mandate in Education 1. Clearly position and align the Institute s mandate with UNESCO s Strategy for Education 2021 2. Rethink and define the Institute s role within the overall UNESCO network and decentralisation structure Effectiveness (strategic focus, functions, visibility) 3. Strengthen the strategic focus of the Institute (reduced number of flagship programmes in line with the Institute s mandate and comparative advantages) 4. Develop new modalities for knowledge sharing and ensure systematic monitoring of the visibility and usefulness of knowledge resources 5. Review capacity development modalities with a view to enhancing scale, outreach and effectiveness 6. Develop a research approach that builds on collaborative (long-term) research partnerships with selected institutions and academic partners 7. Develop more synthetic and comparative research products aimed at influencing policy agendas 8

Education Sector IIEP UIL IBE IITE IESALC IICBA Organisational building blocks (governance, management, accountability and learning, staffing) 1. Review the size, composition and modalities of work of the Institute s Governing Body in function of more efficient and effective governance 2. Define a clear accountability framework for senior management and revitalize leadership through periodic rotation 3. Strengthen the Institute s M&E framework by developing cost-effective tools and practices founded in a clear overall intervention logic 4. Introduce more flexible and diversified staffing arrangements in line with the funding sources of the Institute and to ensure a more dynamic staffing environment 5. Strengthen the human resource capacities (quantity and specialized expertise) of the Institute through secondments, staff exchanges and staff transfers 6. Clarify and develop a common understanding of the concept of functional autonomy Financial sustainability (core funding, extrabudgetary funding, host country support) 1. Develop a coordinated and targeted approach to fundraising 2. Engage in a strategic dialogue with the host country government, Member States and other stakeholders to increase core financial support for the Institute 3. Review the Education Sector s financial allocation to the Institute applying performance-based criteria 9