DG for Education and Culture

Transcription

1 DG for Education and Culture External evaluation of the European Centre for the Development of Vocational Training Cedefop Final Report May 2001

2 Contents Page 1. Introduction Brief history of Cedefop Background of the Evaluation Objective of the Evaluation Recommendations Executive summary Internal Efficiency Analysis main conclusions and recommendations The Management Board - main conclusions and recommendations Defining and implementing the medium-term priorities and the annual work programme External Effectiveness Analysis - main conclusions and recommendations The move to Thessalonica Internal Efficiency Analysis - Findings and Conclusions Role and quality of management Internal function of Cedefop Communication of information and decisions Allocation of financial and human resources Recruitment and allocation of human resources Development and Management of Human Resources Quality Control and Quality Development Defining and implementing objectives Co-operation with the Management Board Co-operation with the Commission The impact of the transfer from Berlin to Thessalonica The Management Board Findings and Conclusions The functioning of the Management Board and the Bureau The effectiveness of the Management Board The relationship between Cedefop and the European Commission, and the role of Cedefop The Medium-term Priorities and the Annual Work Programme68

3 6. External Effectiveness Analysis Findings and Conclusions General assessment How do stakeholders use Cedefop? Relations with external programmes and organisations at European and international level Cedefop s information and communication services The research activities of Cedefop The move to Thessalonica Annex A: The methodology of the evaluation Annex B: Basic results of the questionnaires Annex C: The findings of the case studies

4 1. Introduction This final report refers to contract no / ETU ETU of 13 November 2000 between PLS RAMBOLL Management and the European Commission, DG for Education and Culture regarding an External Evaluation of the European Centre for the Development of Vocational Training Cedefop. This chapter gives a brief history of Cedefop, and the background and the objective of the evaluation are presented. Chapter 2 presents the overall recommendations of the evaluation. Chapter 3 is an executive summary of the findings, conclusions and recommendations of the evaluation. Chapter 4 presents and discusses the findings and conclusions of the internal efficiency analysis of Cedefop. Chapter 5 presents and discusses the findings and conclusions of the functioning of the Management Board. Chapter 6 presents and discusses the findings and conclusions of the external effectiveness analysis of Cedefop. The following annexes are attached to the draft final report: Annex A, which presents the terms of reference and the methodology of the evaluation. Annex B, which presents the basic results of the questionnaires. Annex C, which presents the findings of the case studies. 1.1 Brief history of Cedefop Since its creation in 1975 Cedefop has responded to political developments at the national level within the Member States of the European Communities as well as at the international level. Both the pur- 1 External evaluation of Cedefop - Final Report

5 pose of Cedefop and the composition of the organisation can to a large extent be regarded as a product of the political situation during the past years. One of the demands following the severe worldwide criticism of education systems in 1968 was for more academic research into vocational training as well as better co-ordination between universities and practitioners. At the same time the European Community had started an integration process that over the years encompassed increasingly more policy areas and member states. In the beginning of the 1970s some European bodies, such as the Economic and Social Committee, suggested that some of the issues raised in 1968 should be resolved at a European level. The member states were reluctant to accept this proposal. The governments in the member states were accustomed to co-ordinating training policies at a national level in order to reflect national priorities and labour market conditions. Nevertheless, the European Community decided to investigate the possibility of creating a European body, and Cedefop was established in The subjects that Cedefop had to investigate also reflected the political discourse. As a result of increased focus on international cooperation, policy makers and interest groups began to consider European co-ordination, which resulted in new demands being made on Cedefop. At a general level there was a desire for better horizontal co-ordination of vocational education and training (VET) research between the member states, better exchange of knowledge and results, and the carrying out of research by Cedefop itself. Cedefop was to facilitate vertical co-ordination between academic researchers and practitioners. Accordingly, the European Centre for the Development of Vocational Training (Cedefop) was set up to assist the European Commission in encouraging the promotion and development of vocational education and training at Community level, and to contribute to the work of the Member States and its social partners through the promotion of in- 2 External evaluation of Cedefop - Final Report

6 formation exchange and the comparison of experiences on issues of common interest. In the years following its creation, Cedefop established itself as an institution where it was primarily possible to obtain comparative information from the other member states on VET subjects. With the further integration of the European Community, and especially following the establishment of the European Union in 1992, Cedefop had to adapt to new challenges. At policy level there were new demands for information on student exchanges and the labour force, for instance, as well as investigations of the transparency of qualifications. At a management level Cedefop has had to enlarge the size of the Management Board as new member states have been admitted into the European Union. Before the establishment of the European Union in 1992 Cedefop s objective was to contribute to the implementation of a common vocational training policy. With the establishment of the Union this changed to implement a vocational training policy. Today Cedefop mainly functions as a point of reference for providing policy-makers and practitioners at all levels in the EU with information to enable them to promote a clearer understanding of developments in vocational education and training, and thus permit them to take informed decisions about future action. Cedefop also aims to encourage researchers to identify trends and research themes relevant to policymakers. This implies that as an institution Cedefop ought to monitor developments and collect and furnish updated knowledge and information in relation to a wide range of actors with differing needs and objectives. In the beginning Cedefop was located in West Berlin. This was mainly for political reasons. Germany was divided in two, and the European Community wished to underline that despite its geographical location, West Berlin was a part of Western Europe. The Management Board was set up in accordance with the European political norms for the composition of European bodies. It was to comprise members from each member state to make sure that attention would be paid to national differences and needs. To ensure that 3 External evaluation of Cedefop - Final Report

7 the most important labour market partners had an influence on Cedefop s work, representatives from employees' and employers' organisations as well as government representatives from each country were included in the Management Board. Political developments in the former Eastern Bloc will also have a major impact on Cedefop. The European Union has initiated cooperation with the candidate countries in areas such as the labour market and educational policies. Therefore it was only natural for Cedefop to incorporate these states within the scope of its research, as well as into the different Cedefop networks. Another consequence of the end of the bipolar system and the unification of Germany was that there was no political reason for keeping Cedefop in Berlin. It has been a practice within the European Union to make sure that each member state hosts at least one major European institution. Since Germany was set to host the European Central Bank, and since Greece did not then have any European institutions, in 1995 it was decided to move Cedefop to Greece. The move meant large staff changes, since a lot of the employees could not or would not move to Greece. Another consequence of the move was that a completely new Cedefop building with more facilities was built in Thessalonica. A third consequence of the political changes during the past decade is that Cedefop, along with other European Union institutions, will have to adapt to the future admission of the candidate countries in the European Union. The Management Board will be in the same dilemma as other European organisations with one or more members from each state: either they appoint members from all states, with the risk of the Management Board growing very large, or they reduce the number of Management Board members at the risk of not reflecting all national priorities. Cedefop will also have to consider recruiting staff from the new member states. The subjects within the Cedefop policy area will have to either change or expand. The VET systems in the Eastern European countries are different from those in Western European countries, and Cedefop will have to reflect the needs of future members in its different activities. 4 External evaluation of Cedefop - Final Report

8 1.2 Background of the Evaluation The present external evaluation of Cedefop conforms to Article Two of the Financial Regulation applicable to the general budget of the European Communities, which specifies that all Community expenditures must be subject to periodic re-examination. In 1996 the Commission decided that every community action should be evaluated at least every six years. This also applies to its decentralised agencies, such as Cedefop. The last evaluation of Cedefop took palace before the Centre moved from Berlin to Thessalonica in The period from 1995 to the present was considered sufficient to allow it to reorganise and restructure its activities. Therefore in the autumn of 2000 the Commission initiated a tender procedure for an external evaluation of Cedefop. PLS RAMBOLL Management was assigned to carry out the evaluation. The evaluation has been carried out under the auspices of the European Commission s DG for Education and Culture with the involvement of representatives of the Management Board and Cedefop Directorate and Staff. 1.3 Objective of the Evaluation The main purpose of the evaluation is to assess the internal efficiency and external effectiveness of Cedefop compared to its statutory objectives, including Cedefop s policy guidelines and medium-term priorities in the period Concerning internal efficiency, the evaluation has assessed: The management information and decision-making systems The methods and transparency of allocating financial and human resources to essential activities and core tasks identified in the course of a priority-setting exercise 5 External evaluation of Cedefop - Final Report

9 The organisation of the agency and the recruitment and development of its human resources The extent to which quality control and quality development is carried out The extent to which the policy guidelines and annual programmes have been set in terms of measurable objectives and implemented in terms of concrete outputs The monitoring and assessment of the actual implementation of the priorities and programmes as a basis for the development of policy guidelines and priorities for The impact of the transfer of the Centre from Berlin to Thessalonica Concerning external effectiveness, the evaluation has assessed: The use of the policy guidelines and annual programmes as priority-setting and programming tools and as means to achieve the statutory objectives and tasks The extent to which the actual activities carried out and outputs produced have met the expectations of the main stakeholders The development, production, quality and relevance of the major outputs of the Centre, such as the Research report and the VET policy report The information, communication and publication strategy and activity, notably the use of ICT and the Internet The terms of reference for the evaluation emphasises that the evaluation should specifically address the following questions: 1. The Centre s activities in supporting the implementation of the Leonardo da Vinci programme, in particular the Study Visits measure. 2. The Centre s co-operation with the Commission in assisting it in implementing a vocational training policy at European level, and with the Social Partners and the Member States. 3. The Centre s co-operation with other EU bodies working in related fields, particularly the European Training Foundation, Eurydice and the European Foundation for the Improvement of Living and Working Conditions. 6 External evaluation of Cedefop - Final Report

10 4. The functioning of the Management Board. Points 1-3 have been addressed by the external evaluation, while Point 4 has been addressed both via the evaluation of the Centre s external effectiveness and its internal efficiency. To supplement the evaluation of both the internal efficiency and the external effectiveness of Cedefop, the following three case studies have been carried out: The research activities at Cedefop, in particular the Research Report The Study Visits programme The Information and Communication services of Cedefop The case studies were selected by the steering group comprising representatives from the Commission, the Management Board and the Cedefop directorate and staff monitoring the evaluation. In general the evaluation has been carried out with the overall purpose of producing prospective conclusions and recommendations to improve: The planning, implementation and management of the Centre Organisational and human resources development The performance, products and services of the Centre Accountability to stakeholders 7 External evaluation of Cedefop - Final Report

11 2. Recommendations Below PLS RAMBOLL presents a sample list of our recommendations based on the internal efficiency and the external effectiveness analyses of Cedefop. The main recommendations regarding the internal efficiency of the Centre in Thessalonica concern the organisation of the Centre, management skills, budgetary procedures, and the information strategy. PLS RAMBOLL recommends: that emphasis is put on developing the Centre s management skills both at top and middle management level of the organisation. that a management system for the whole organisation should be developed in order to improve the overall coordination of the organisation. that a transparent division of labour and responsibility between the Director and the Deputy Director should be developed. that a review of the organisation is performed in order to work out a clear structure and division of tasks. that a project organisation where the project managers refer directly to the management but are given greater responsibility than today, is considered. to map in-house competencies and formulate a strategy on how to nurture and develop staff competencies within the framework of a global strategy for the development of Cedefop. more transparency regarding recruitment policies. that guidelines be developed with regard to the budget procedure. that budget procedures are decentralised. that activity-based budgeting is implemented. 8 External evaluation of Cedefop - Final Report

12 that output indicators relating to the overall priorities of Cedefop be developed with regard to the allocation of financial and human resources. a greater use of external funding. that the work with the development of an internal information policy and strategy continues. a more efficient and targeted use of information technology internally within the Centre. to develop a strategy for the systematic quality control of both its products and services. The main recommendations regarding the Management Board of Cedefop concern the relation to the staff at the Centre, the division of work and responsibility between the Management Board and the Bureau, and the future organisation of the Management Board. PLS RAMBOLL recommends: that some effort is put into ways of getting the Management Board and the staff at the Centre to feel more closely interrelated. An opportunity might be to involve the expert staff more with the Management Board regarding the political and strategic discussions on vocational education and training. that the whole process regarding the formulation of the medium-term priorities and the annual work programme is revised in order to include the opinion of staff more. that there should be greater and more structured involvement of experts and stakeholders in the work on priorities and activities before presentation to the Management Board. that clearer output indicators for the priorities are established. that the Management Board requests better management tools from the Centre. that progress reports are brought into concordance with the annual work programmes. that the gaining of more time for the Management Board to consider strategic and political discussions by reducing 9 External evaluation of Cedefop - Final Report

13 the time spent on administrative matters should be considered. a reduction in the time spent on administrative matters by separating issues of administration into items of high relevance and items of lower relevance. that the consequences of enlargement regarding the work of the Management Board are discussed among the relevant partners as soon as possible. that the relationship between the Bureau and the Management Board is formalised. the exact purpose and competencies of the Bureau are made explicit. that strategies for the improved dissemination of information from the Bureau to the Management Board are ensured. The main recommendations regarding the external effectiveness analysis of Cedefop concern the public profile of Cedefop and dissemination and targeting of information. PLS RAMBOLL recommends: that a devoted effort is made to raise the public profile of Cedefop, for instance by continuing participating in international meetings. Cedefop examines the procedures with which it responds to external enquiries, in order to ensure that all enquiries are handled within a given period of time. extra effort is made to direct the publications at more specifically defined groups, and that more basic introductions, executive summaries and separate briefs are published. that the new initiatives taken to improve the dissemination of the next Research Report are supported as much as possible. that a formal quality control system with external peer reviews of publications is adopted. that ICT-based forms of co-operation and the already extensive use of the Internet are further developed. 10 External evaluation of Cedefop - Final Report

14 that the work on promoting and developing the Electronic Training Village is continued in order to further develop it as a site for the sharing and development of knowledge. that the Electronic Training Village closely is incorporate into the strategies of Cedefop. that Cedefop looks into its own role regarding the various networks and forums in order to find out how the Centre can best support and nurture them in the future. that the work related to the Transparency Forum should continue and be developed further. that the subject of whether and in what ways Cedefop could gain from strengthened co-operation with the European Training Foundation should be discussed by the Management Board and the Directorate of Cedefop. that the co-ordinator of the Study Visits programme participates more systematically in the meetings of the Leonardo da Vinci committee in order to strengthen the links further. By integrating the activities of the Study Visits programme more into other areas of the Leonardo da Vinci programme and Cedefop s work in general, it might be possible to create more synergy. 11 External evaluation of Cedefop - Final Report

15 3. Executive summary Cedefop, the European Centre for the Development of Vocational Training, has been through a period of reorientation and change since it moved in 1995 from Berlin, Germany, to Thessalonica in the northern part of Greece a move which was made mainly for political reasons. The relocation created problems for Cedefop because the Centre lost a huge number of its original staff. But the move also made possible a new beginning. As one observer puts it: The relocation nearly destroyed the Centre, but at the same time it was almost the best thing that ever happened to it. The hiring of new staff led to a revitalising of the Centre, and at the same time the relationship between the Centre and the European Commission, which had been under some strain during the Centre s years in Berlin, improved considerably. In recent years there has been an increasing political focus on vocational education and training, both with regard to the emphasis on actively seeking work which is being pursued in the employment policies of many member states, and the new focus on lifelong learning. On the one hand, this means greater demand for the services of the Centre, and on the other provides the Centre with the opportunity of positioning itself in relation to new areas of activity and co-operation partners. Overall, it could be said that Cedefop s scope is expanding but is simultaneously becoming more complex. It is the general assessment of PLS RAMBOLL that the Centre has undergone a very positive development since the move to Thessalonica, and that the Centre is responding well to the new demands in many ways. It is the assessment of PLS RAMBOLL that in relation to its external users and partners, Cedefop is doing important work to promote and develop vocational education and training in the European context. A survey among external stakeholders such as civil servants, social partner representatives, researchers and people working with vocational education and training suggests that the external stakeholders believe that Cedefop mostly focuses on the right 12 External evaluation of Cedefop - Final Report

16 issues in vocational education and training, that its products are being used, and that they are considered to be of high quality. However, there is room for improvement in terms of better visibility and a more targeted dissemination strategy. Before the move to Thessalonica the Centre was experiencing a variety of different organisational difficulties. Many of these persisted after the move, and although the move did offer solutions to some of them, the relocation also brought great challenges for the management of the Centre. It is the overall impression of PLS RAMBOLL that many aspects of the organisational efficiency of Cedefop offer some scope for improvement, but that even so, in many respects processes to improve the internal efficiency of the organisation are underway. The evaluation suggests that there is internal confusion surrounding the management of the Centre, and that the potentials of both the human and the financial resources of the Centre are not being fully realised. It is the opinion of PLS RAMBOLL that there needs to be a restructuring and possibly a rethinking of the way the Centre is led, particularly in clarifying the decision-making structures, both in relation to the role of the Director and the directorate, and in relation to the role of the Management Board. Furthermore, the tripartite nature of the Management Board puts Cedefop in a unique position to contribute to the development of vocational education and training and to the debate that centres on it, because it provides a forum for governments, social partners and the Commission to meet at a European level. In addition, the fact that Cedefop, which among other things functions as a reference point for the provision of research and knowledge about vocational education and training, is led by a tripartite body with members who work with this issue both in different countries and in different contexts, provides a rare opportunity to bridge the gap between those who provide research and those who make use of it. The organisations and the governments in the Management Board can tell the experts at the Centre what topics they believe they should concentrate on, and at the same 13 External evaluation of Cedefop - Final Report

17 time the experts can inform the policy makers of the Management Board about the results of current research. However, at present neither of these two things is being achieved, partly due to too much of the Management Board's time being consumed by administrative issues, partly to the unclear relationship between the Management Board and the Bureau, and partly to disagreements among the different parts of the Management Board. The relative status of the medium-term priorities revealed by the evaluation also suggest that although the introduction of the mediumterm priorities has meant a higher degree of focus in the work of Cedefop, the opportunity to use the medium-term priorities as a tool for setting the overall direction of the Centre is not being fully exploited. The present evaluation points to a number of ways in which the working procedures and the management of the Centre could be improved. If the organisational resources of both the Centre in Thessalonica and the Management Board are fully to be taken advantage of, it is the opinion of PLS RAMBOLL that these suggestions should be considered carefully and that an action plan to implement them should be prepared and implemented as soon as possible. Beyond this, on the basis of the present evaluation it is the opinion of PLS RAMBOLL that the Centre would benefit from a frank and open debate in the Management Board about what kind of agency Cedefop should become in the long term. Should the Centre continue to function primarily as a reference centre for information and research, or should the budding initiatives in working out policy implications be embraced more wholeheartedly? Should the Centre continue to focus on a wide range of different topics and areas, or should the focus be narrower? An executive summary of the evaluation s main conclusions and main recommendations on how the functioning of the Centre could be improved is presented below. 14 External evaluation of Cedefop - Final Report

18 3.1 Internal Efficiency Analysis main conclusions and recommendations Role and quality of management It is PLS RAMBOLL s overall assessment that the obvious strength of the directorate is that it is highly committed, has a clear focus on results, and is oriented towards the overall development of Cedefop. The management of the Centre is also characterised by a limited hierarchy, with the lack of formal heads in four of the five areas having both advantages and disadvantages. The advantage is efficient and competent decision-making. The disadvantage is the lack of formal structures to facilitate the implementation of specific plans according to schedule. It is PLS RAMBOLL s evaluation that both top and middle management are beset by a number of problems concerning the division of tasks and co-ordination among the different actors. It is also a weakness that the decision-making process lacks transparency as a result of the absence of clear and formal structures. On this basis PLS RAMBOLL recommends that: a transparent division of labour and responsibility between the Director and the Deputy Director should be developed. a management system for the whole organisation should be developed in order to improve the functioning of the different areas and units, as well as the overall co-ordination of the organisation. emphasis is put on developing the Centre s management skills at both top and middle management level. Internal function of Cedefop With regard to the internal function of Cedefop, it is PLS RAMBOLL s overall assessment that the internal co-operation in the individual units is functioning well, though it appears that some staff members would like to see more co-ordinating unit meetings. The overall weakness in the internal functioning of Cedefop is that there seems to be a lack of co-operation and co-ordination between the units, especially across areas, and that the formal internal division of labour and responsibility is unclear. Even though most staff members seem to 15 External evaluation of Cedefop - Final Report

19 know what they are supposed to be doing themselves, it is unclear to many what the responsibilities of their colleagues are, and hence to whom they should turn regarding decisions and assistance of different kinds. The organisation chart of the Centre does not represent the true structure of the organisation, and the system does not appear to be transparent. PLS RAMBOLL recommends that: a review of the organisation is performed in order to work out a clear structure and division of tasks. Instead of the present priority areas I-III, it is the assessment of PLS RAMBOLL that it is worth considering a project organisation where the project managers refer directly to the management but are given greater responsibility than today. Communication of information and decisions A problem closely related to that of the internal functioning and coordination at Cedefop is that there is no systematic internal information policy and information system in the organisation. However, a wide range of strategies and objectives is currently being developed at the Centre to improve the existing tools of internal communication. PLS RAMBOLL has been informed that the overall purpose of area IV Information, dissemination and external services is to raise the profile and impact of Cedefop s publications and services, as well as to improve the internal co-ordination of information and communication activities. It seems that inside this group dynamic and targeted work is being directed to this purpose, but it also appears that there is still much scope for improvement in this area. PLS RAMBOLL strongly recommends that: work to develop of information policy and strategy continues, since a well-structured information system is a prerequisite for a properly functioning organisation. Allocation of financial and human resources Concerning the allocation of financial and human resources, an obvious strength of Cedefop is that it administers 11 projects and a total budget of 5 million Euros without major problems. The budget procedure has several weaknesses, however, both with regard to the decision-making process and to budget allocation. The decision-making 16 External evaluation of Cedefop - Final Report

20 process is of low transparency, since the budget procedure is topdown and without guidelines. The Director has to approve almost all costs related to daily operations, which is inefficient and suggests that too much power is concentrated at the top of the organisation. The impression created is, firstly, that the financial allocation system contains no output indicators with regard to the overall priorities of Cedefop, and secondly, that the budgetary system is far from transparent. On the basis of these findings, PLS RAMBOLL recommends that: budget procedures are decentralised, meaning that the project leaders are given more budgetary responsibilities both with regard to proposals for budgets and their implementation. guidelines be developed with regard to the budget procedure. output indicators relating to the overall priorities of Cedefop be developed with regard to the allocation of financial and human resources. activity-based budgeting is implemented. external funding is used to a larger extent. Recruitment and allocation of human resources PLS RAMBOLL has found that there is clearly some imbalance among VET experts and administrative supportive staff in terms of nationalities. According to the findings of PLS RAMBOLL, it seems that there is a heavy overrepresentation of staff belonging to the Northern European' countries among categories A + LA, while there is a slightly lesser overrepresentation of Southern European staff belonging to categories B, C + D. 1 Currently the Centre has more staff than the total prescribed. This is due to the fact that more national experts and staff on short-term contracts are recruited compared to previously. This obviously gives the Centre more human resources. A threat, however, is that too many national experts, and too many staff hired on a short-term basis, 1 The terms Northern and Southern should be seen more as an indication of language communities than of the geographical location, that is Northern European Countries covers Scandinavian, Dutch/German/Austrian and English/Irish speaking countries, while Southern European Countries covers French, Spanish, Italian, Portuguese and Greek speaking countries. 17 External evaluation of Cedefop - Final Report

21 might disturb the general development of a corporate spirit and engagement in the overall objectives of Cedefop. Given that it seems to PLS RAMBOLL that there are different understandings at Cedefop regarding the recruitment procedures, it might be a good idea for the Director to communicate his wish to recruit expert staff from Portugal and Spain, and that in general an official plan or policy for the recruitment and allocation of human resources is developed and communicated externally and to the staff at the Centre. More transparency regarding recruitment policies might prevent the development of different understandings among staff members about which categories of staff are preferred to others. Development and Management of Human Resources PLS RAMBOLL has noticed both strengths and weaknesses at the Centre regarding the development and management of human resources. Some work has been initiated to ensure that most staff members receive training. But the fact that there is no global strategy for the development and management of the human resources at the Centre is one of the chief weaknesses that PLS RAMBOLL has noticed about Cedefop. From the top management s side there seems to be vague focus and a lack of competence in human resource development at a more strategic level. An obvious opportunity is to map in-house competencies and formulate a strategy on how to nurture and develop staff competencies within the framework of a global Cedefop development strategy. PLS RAMBOLL recommends that Cedefop initiate this work. Quality Control and Quality Development It is in the area of quality control and quality development that PLS RAMBOLL has found some of the greatest weaknesses in the internal efficiency of Cedefop. At the global level, no general quality policy or strategy encompassing all the activities and services at the Centre exists, and actual quality assurance activities seem to be initiated not from the Directorate but from those directly involved. As only minimal quality control and reviewing of products and services takes place, and given the fact that only very little of the quality control and review 18 External evaluation of Cedefop - Final Report

22 that does take place is the result of formalised procedures, PLS RAMBOLL highly recommends that: Cedefop develops a strategy for the systematic quality control of both its products and services, such as systematic peer reviews and output measures/indicators for its projects and work in general. Co-operation with the Management Board The predominant feeling among Cedefop staff seems to be that the Management Board is too big, and is therefore an inefficient body with members who are not really engaged with the content of the work being carried out at Cedefop. However, there is also a large segment of Cedefop staff in favour of the fundamental tripartite principle that the Management Board is built upon, and who do not see any other way to structure it. The Bureau of the Management Board seems to be perceived much more as a co-operation partner for Cedefop. PLS RAMBOLL recommends that: some effort is put into developing ways of getting the two bodies to feel more closely interrelated. One suggested possibility is to increase the involvement of the expert staff with the Management Board in the political and strategic discussions concerning vocational education and training. Co-operation with the Commission In the assessment of PLS RAMBOLL, in recent years the relationship to the Commission seems to have improved considerably. There seem to be good opportunities for supporting the trend of positive developments in the relationship between the Commission and Cedefop. On the other hand, these positive developments might be at the expense of the relationship with the Management Board, which is a more direct vehicle for member state influence. 19 External evaluation of Cedefop - Final Report

23 3.2 The Management Board - main conclusions and recommendations The functioning of the Management Board and the Bureau It is the overall assessment of PLS RAMBOLL that a major strength of the Management Board is its high number of highly enthusiastic members. A weakness is that the Management Board spends too much of its time on administrative details, particularly on those concerned with the relocation of the Centre, rather than on focusing on the strategic development of Cedefop. The existence of the Bureau is an opportunity to make the decisionmaking process in the Management Board more efficient. However, the lack of a formal decision regarding the role and the competencies of the Bureau, and the lack of any systematised diffusion of information from the Bureau to the Management Board members, are viewed as major weaknesses. On the background of these findings it is the recommendations of PLS RAMBOLL that: the relationship between the Bureau and the Management Board is formalised. the exact purpose and competencies of the Bureau are made explicit. strategies for the improved dissemination of information from the Bureau to the Management Board are ensured. the gaining of more time for the Management Board to consider strategic and political discussions by reducing the time spent on administrative matters should be considered. a reduction in the time spent on administrative matters by separating issues of administration into items of high relevance and items of lower relevance. The effectiveness of the Management Board PLS RAMBOLL considers that the size of the Management Board reduces its efficiency as a decision-making body. This issue becomes even more relevant when taking the future enlargement of the European Union into consideration. 20 External evaluation of Cedefop - Final Report

24 The different possible models include enlarging the Management Board in the same way as hitherto, having fewer representatives per country in the Management Board, and/or making the Bureau more active. Each of these models has its strengths and weaknesses and fundamentally it is a political decision, which affects not only the composition of the Cedefop Management Board, but all the institutions and agencies of the European Union. PLS RAMBOLL finds it paramount that this highly political and sensitive issue is addressed, and therefore strongly recommends that: the consequences of enlargement on the work of the Management Board are discussed among the relevant partners as soon as possible. The relationship between Cedefop and the European Commission, and the role of Cedefop It is the impression of PLS RAMBOLL that both the Commission and the majority of the Management Board members believe that the relationship between Cedefop and the European Commission has improved considerably in recent years. In addition, it is the impression that the Centre now lives up better to its aims as defined in the statutory objectives. A divide seems to exist internally in the Management Board between those would like to see the Centre becoming more active in highlighting policy implications, and those who believe that the Centre should concentrate on disseminating research and information. For its part, the Commission sometimes has the feeling that Cedefop is trying to push the boundaries of what it considers to be its role, and sometimes oversteps the competencies of the Centre when making explicit policy recommendations. While the difference in opinion between part of the Management Board and the Commission may be characterised as a weakness which could affect the co-operation between the partners and make it less fruitful, the fact that the Centre is seen as trying to push its boundaries can be seen as a strength if it is interpreted as a sign of 21 External evaluation of Cedefop - Final Report

25 engagement and the Centre's desire to become a more active partner. 3.3 Defining and implementing the medium-term priorities and the annual work programme PLS RAMBOLL considers that a core strength of Cedefop is its use of its medium-term priorities as a goal-setting tool. In relation to the Centre s work on the priorities for , it is PLS RAMBOLL s overall assessment that the process has improved compared to the one, which was applied to the work on the priorities. However, it is a weakness that the stakeholders to be involved are not defined beforehand, and that the Centre has no clear strategy concerning the involvement of stakeholders. Also PLS RAMBOLL considers it a weakness that the process for developing the priorities is too top-down. The medium-term priorities are formulated in very general terms. The advantage of this is that it leaves room for flexibility, and thus allows the Centre to respond to a constantly changing agenda. The downside is that the breadth of the medium-term priorities weakens them as an instrument for setting priorities, or as guidelines for the annual work programme. The limitation most commonly mentioned by staff regarding the setting of realistic priorities is that the medium-term priorities have therefore become a political document, which is too broad and muddled. Based on these observations, PLS RAMBOLL recommends that: the whole process regarding the formulation of the mediumterm priorities and the annual work programme is revised in order to include the opinions of staff more widely in the process by creating better links between the staff and the Management Board, while simultaneously creating medium-term priorities that both retain their flexibility and increase their strategic focus, and thus enhance their usefulness as guidelines. 22 External evaluation of Cedefop - Final Report

26 progress reports are brought into concordance with the annual work programmes. there should be greater and more structured involvement of experts and stakeholders in the work on priorities and activities before presentation to the Management Board. time and energy is devoted to establishing clearer output indicators for the priorities, as this would make it easier for the Management Board to scrutinise the performance of Cedefop, and it would motivate the process of defining the priorities for setting realistic and attainable goals for Cedefop s work, and hence ease the process of establishing a feeling among staff at Cedefop of having some common goals. the Management Board requests better management tools from the Centre in terms of better follow-up of the implementation of the work programme, perhaps by using more specific output criteria. 3.4 External Effectiveness Analysis - main conclusions and recommendations General assessment of Cedefop The general assessment of PLS RAMBOLL is that Cedefop does important work in the promotion and development of vocational education and training in the European context. PLS RAMBOLL considers that a strength in the work of Cedefop is that the Centre mostly focuses on the right issues in vocational education and training. A weakness is that the Centre is trying to do too much, often being apparently unable to set negative priorities. It is the assessment of PLS RAMBOLL that by becoming more focused, the Centre might increasingly be able to set the agenda in the VET area and raise the visibility of the Centre. PLS RAMBOLL recommends that: an effort is made to raise the public profile of Cedefop. Relations with external programmes and organisations at the European and international levels It is the assessment of PLS RAMBOLL that the co-operation between Cedefop and various external programmes and organisations functions well in general, but that international organisations and Cedefop 23 External evaluation of Cedefop - Final Report

27 could benefit from even closer co-operation concerning both joint projects and networks, and the co-ordination of activities. The assessment of PLS RAMBOLL is that Cedefop might benefit from closer cooperation at levels other than at the Directorate level. PLS RAMBOLL recommends that: it be discussed within the Management Board and the Directorate of Cedefop whether Cedefop could gain from strengthened co-operation with the European Training Foundation. PLS RAMBOLL finds that the relations and synergy between the Leonardo da Vinci programme and the Study Visits programme are to some extent ensured in the present structure, and that the Study Visits programme significantly strengthens the mobility aspects of the Leonardo da Vinci programme. PLS RAMBOLL recommends that: the Study Visits co-ordinator participates more systematically in the meetings of the Leonardo da Vinci committee. the potential of the Study Visits programme as a tool for preparing the accession and increasing integration of the candidate countries into Cedefop s work in general is considered carefully. the themes of the Study Visits programme should reflect the medium-term priorities of Cedefop to a larger degree in order to enhance links to specific research projects and areas within Cedefop. during future staff recruitment, the Study Visits department should place continuing emphasis on enhancing relations with the other departments within Cedefop. the Study Visits department initiates a follow-up system to make people stay in contact. The creation of networks and the formation of synthesis seminars are possible tools for creating multiplier effects from the Study Visits. the subject rather than the time frame for the visits should be the focus for attracting appropriate target groups, and the duration of the visits should be more flexible, depending on the host country and the subject. the formation of working groups including both National Liaison Officers/Technical Assistants and Cedefop staff members 24 External evaluation of Cedefop - Final Report

28 should be continued as a way of further improving the organisation and implementation of the Study Visits programme. the work on integrated and consistent survey tools is continued and supported in order to generate relevant tools for the future planning and implementation of the Study Visits programme. the actual and planned extensive use of the website as a tool for the dissemination of information and administration of the Study Visits programme is continued, and that Cedefop places emphasis on training the users of the website. PLS RAMBOLL finds that Cedefop is generally acknowledged for its contribution in bringing people together in order to break down the barriers between different professions and countries. However, the support, which the different networks receive, seems to differ substantially. PLS RAMBOLL therefore recommends that: Cedefop looks into its own role regarding the various networks and forums in order to find out how the Centre can best support and nurture them in the future. that the work related to the Transparency Forum should continue and be developed further. Cedefop s information and communication services There is widespread satisfaction with the information material produced by Cedefop and the way it is communicated. Nevertheless, it is the assessment of PLS RAMBOLL that the overall information and communication strategy of the Centre has scope for improvement. PLS RAMBOLL considers that there is a large potential to widen the audience, and recommends that: extra effort is made to direct the publications at more specifically defined groups, and that more basic introductions, executive summaries and separate briefs are published. Cedefop examines the procedures with which it responds to external enquiries, in order to ensure that all enquiries are handled within a given period of time and none are lost. the work on promoting and developing the Electronic Training Village is continued in order to raise the visibility and user op- 25 External evaluation of Cedefop - Final Report

29 portunities of the site, and to further develop it for the sharing and development of knowledge. that the Electronic Training Village closely is incorporate into the strategies of Cedefop. The research activities of Cedefop PLS RAMBOLL believes that the quality of Cedefop s research activities is quite high, has improved in recent years, and is better related to the general EU agenda. However, some people believe that the quality of research is too variable. PLS RAMBOLL considers that this may be the result of the absence of a formal quality control system and obligatory peer reviews, and therefore recommends that: a formal quality control system with external peer reviews of publications is adopted. new initiatives taken to improve the dissemination of the next Research Report are supported as much as possible. 3.5 The move to Thessalonica It is the assessment of PLS RAMBOLL that the move from Germany to Greece has had an impact on the work of Cedefop in several ways. Among others, the commitment of the staff working in Cedefop and the improvements in the quality of work are seen as consequences of the redirection that followed the shift to Greece. The collective understanding among both staff and the Management Board is that the move to Greece caused a lot of difficulties, but that most of them are now resolved. It is the assessment of PLS RAMBOLL that in the long run Cedefop seems to have gained from the move, as it has had a chance to redefine objectives and working practices, and has thereby caused its external reputation to rise. A negative aspect of the transfer is that it is more difficult to recruit staff, because transport to Thessalonica is difficult and because many people, particularly those from North or Central Europe, might find the location in Thessalonica fairly remote. 26 External evaluation of Cedefop - Final Report