3. Executive summary. 12 External evaluation of Cedefop - Final Report

Transcription

1 3. Executive summary Cedefop, the European Centre for the Development of Vocational Training, has been through a period of reorientation and change since it moved in 1995 from Berlin, Germany, to Thessalonica in the northern part of Greece a move which was made mainly for political reasons. The relocation created problems for Cedefop because the Centre lost a huge number of its original staff. But the move also made possible a new beginning. As one observer puts it: The relocation nearly destroyed the Centre, but at the same time it was almost the best thing that ever happened to it. The hiring of new staff led to a revitalising of the Centre, and at the same time the relationship between the Centre and the European Commission, which had been under some strain during the Centre s years in Berlin, improved considerably. In recent years there has been an increasing political focus on vocational education and training, both with regard to the emphasis on actively seeking work which is being pursued in the employment policies of many member states, and the new focus on lifelong learning. On the one hand, this means greater demand for the services of the Centre, and on the other provides the Centre with the opportunity of positioning itself in relation to new areas of activity and co-operation partners. Overall, it could be said that Cedefop s scope is expanding but is simultaneously becoming more complex. It is the general assessment of PLS RAMBOLL that the Centre has undergone a very positive development since the move to Thessalonica, and that the Centre is responding well to the new demands in many ways. It is the assessment of PLS RAMBOLL that in relation to its external users and partners, Cedefop is doing important work to promote and develop vocational education and training in the European context. A survey among external stakeholders such as civil servants, social partner representatives, researchers and people working with vocational education and training suggests that the external stakeholders believe that Cedefop mostly focuses on the right 12 External evaluation of Cedefop - Final Report

2 issues in vocational education and training, that its products are being used, and that they are considered to be of high quality. However, there is room for improvement in terms of better visibility and a more targeted dissemination strategy. Before the move to Thessalonica the Centre was experiencing a variety of different organisational difficulties. Many of these persisted after the move, and although the move did offer solutions to some of them, the relocation also brought great challenges for the management of the Centre. It is the overall impression of PLS RAMBOLL that many aspects of the organisational efficiency of Cedefop offer some scope for improvement, but that even so, in many respects processes to improve the internal efficiency of the organisation are underway. The evaluation suggests that there is internal confusion surrounding the management of the Centre, and that the potentials of both the human and the financial resources of the Centre are not being fully realised. It is the opinion of PLS RAMBOLL that there needs to be a restructuring and possibly a rethinking of the way the Centre is led, particularly in clarifying the decision-making structures, both in relation to the role of the Director and the directorate, and in relation to the role of the Management Board. Furthermore, the tripartite nature of the Management Board puts Cedefop in a unique position to contribute to the development of vocational education and training and to the debate that centres on it, because it provides a forum for governments, social partners and the Commission to meet at a European level. In addition, the fact that Cedefop, which among other things functions as a reference point for the provision of research and knowledge about vocational education and training, is led by a tripartite body with members who work with this issue both in different countries and in different contexts, provides a rare opportunity to bridge the gap between those who provide research and those who make use of it. The organisations and the governments in the Management Board can tell the experts at the Centre what topics they believe they should concentrate on, and at the same 13 External evaluation of Cedefop - Final Report

3 time the experts can inform the policy makers of the Management Board about the results of current research. However, at present neither of these two things is being achieved, partly due to too much of the Management Board's time being consumed by administrative issues, partly to the unclear relationship between the Management Board and the Bureau, and partly to disagreements among the different parts of the Management Board. The relative status of the medium-term priorities revealed by the evaluation also suggest that although the introduction of the mediumterm priorities has meant a higher degree of focus in the work of Cedefop, the opportunity to use the medium-term priorities as a tool for setting the overall direction of the Centre is not being fully exploited. The present evaluation points to a number of ways in which the working procedures and the management of the Centre could be improved. If the organisational resources of both the Centre in Thessalonica and the Management Board are fully to be taken advantage of, it is the opinion of PLS RAMBOLL that these suggestions should be considered carefully and that an action plan to implement them should be prepared and implemented as soon as possible. Beyond this, on the basis of the present evaluation it is the opinion of PLS RAMBOLL that the Centre would benefit from a frank and open debate in the Management Board about what kind of agency Cedefop should become in the long term. Should the Centre continue to function primarily as a reference centre for information and research, or should the budding initiatives in working out policy implications be embraced more wholeheartedly? Should the Centre continue to focus on a wide range of different topics and areas, or should the focus be narrower? An executive summary of the evaluation s main conclusions and main recommendations on how the functioning of the Centre could be improved is presented below. 14 External evaluation of Cedefop - Final Report

4 3.1 Internal Efficiency Analysis main conclusions and recommendations Role and quality of management It is PLS RAMBOLL s overall assessment that the obvious strength of the directorate is that it is highly committed, has a clear focus on results, and is oriented towards the overall development of Cedefop. The management of the Centre is also characterised by a limited hierarchy, with the lack of formal heads in four of the five areas having both advantages and disadvantages. The advantage is efficient and competent decision-making. The disadvantage is the lack of formal structures to facilitate the implementation of specific plans according to schedule. It is PLS RAMBOLL s evaluation that both top and middle management are beset by a number of problems concerning the division of tasks and co-ordination among the different actors. It is also a weakness that the decision-making process lacks transparency as a result of the absence of clear and formal structures. On this basis PLS RAMBOLL recommends that: a transparent division of labour and responsibility between the Director and the Deputy Director should be developed. a management system for the whole organisation should be developed in order to improve the functioning of the different areas and units, as well as the overall co-ordination of the organisation. emphasis is put on developing the Centre s management skills at both top and middle management level. Internal function of Cedefop With regard to the internal function of Cedefop, it is PLS RAMBOLL s overall assessment that the internal co-operation in the individual units is functioning well, though it appears that some staff members would like to see more co-ordinating unit meetings. The overall weakness in the internal functioning of Cedefop is that there seems to be a lack of co-operation and co-ordination between the units, especially across areas, and that the formal internal division of labour and responsibility is unclear. Even though most staff members seem to 15 External evaluation of Cedefop - Final Report

5 know what they are supposed to be doing themselves, it is unclear to many what the responsibilities of their colleagues are, and hence to whom they should turn regarding decisions and assistance of different kinds. The organisation chart of the Centre does not represent the true structure of the organisation, and the system does not appear to be transparent. PLS RAMBOLL recommends that: a review of the organisation is performed in order to work out a clear structure and division of tasks. Instead of the present priority areas I-III, it is the assessment of PLS RAMBOLL that it is worth considering a project organisation where the project managers refer directly to the management but are given greater responsibility than today. Communication of information and decisions A problem closely related to that of the internal functioning and coordination at Cedefop is that there is no systematic internal information policy and information system in the organisation. However, a wide range of strategies and objectives is currently being developed at the Centre to improve the existing tools of internal communication. PLS RAMBOLL has been informed that the overall purpose of area IV Information, dissemination and external services is to raise the profile and impact of Cedefop s publications and services, as well as to improve the internal co-ordination of information and communication activities. It seems that inside this group dynamic and targeted work is being directed to this purpose, but it also appears that there is still much scope for improvement in this area. PLS RAMBOLL strongly recommends that: work to develop of information policy and strategy continues, since a well-structured information system is a prerequisite for a properly functioning organisation. Allocation of financial and human resources Concerning the allocation of financial and human resources, an obvious strength of Cedefop is that it administers 11 projects and a total budget of 5 million Euros without major problems. The budget procedure has several weaknesses, however, both with regard to the decision-making process and to budget allocation. The decision-making 16 External evaluation of Cedefop - Final Report

6 process is of low transparency, since the budget procedure is topdown and without guidelines. The Director has to approve almost all costs related to daily operations, which is inefficient and suggests that too much power is concentrated at the top of the organisation. The impression created is, firstly, that the financial allocation system contains no output indicators with regard to the overall priorities of Cedefop, and secondly, that the budgetary system is far from transparent. On the basis of these findings, PLS RAMBOLL recommends that: budget procedures are decentralised, meaning that the project leaders are given more budgetary responsibilities both with regard to proposals for budgets and their implementation. guidelines be developed with regard to the budget procedure. output indicators relating to the overall priorities of Cedefop be developed with regard to the allocation of financial and human resources. activity-based budgeting is implemented. external funding is used to a larger extent. Recruitment and allocation of human resources PLS RAMBOLL has found that there is clearly some imbalance among VET experts and administrative supportive staff in terms of nationalities. According to the findings of PLS RAMBOLL, it seems that there is a heavy overrepresentation of staff belonging to the Northern European' countries among categories A + LA, while there is a slightly lesser overrepresentation of Southern European staff belonging to categories B, C + D. 1 Currently the Centre has more staff than the total prescribed. This is due to the fact that more national experts and staff on short-term contracts are recruited compared to previously. This obviously gives the Centre more human resources. A threat, however, is that too many national experts, and too many staff hired on a short-term basis, 1 The terms Northern and Southern should be seen more as an indication of language communities than of the geographical location, that is Northern European Countries covers Scandinavian, Dutch/German/Austrian and English/Irish speaking countries, while Southern European Countries covers French, Spanish, Italian, Portuguese and Greek speaking countries. 17 External evaluation of Cedefop - Final Report

7 might disturb the general development of a corporate spirit and engagement in the overall objectives of Cedefop. Given that it seems to PLS RAMBOLL that there are different understandings at Cedefop regarding the recruitment procedures, it might be a good idea for the Director to communicate his wish to recruit expert staff from Portugal and Spain, and that in general an official plan or policy for the recruitment and allocation of human resources is developed and communicated externally and to the staff at the Centre. More transparency regarding recruitment policies might prevent the development of different understandings among staff members about which categories of staff are preferred to others. Development and Management of Human Resources PLS RAMBOLL has noticed both strengths and weaknesses at the Centre regarding the development and management of human resources. Some work has been initiated to ensure that most staff members receive training. But the fact that there is no global strategy for the development and management of the human resources at the Centre is one of the chief weaknesses that PLS RAMBOLL has noticed about Cedefop. From the top management s side there seems to be vague focus and a lack of competence in human resource development at a more strategic level. An obvious opportunity is to map in-house competencies and formulate a strategy on how to nurture and develop staff competencies within the framework of a global Cedefop development strategy. PLS RAMBOLL recommends that Cedefop initiate this work. Quality Control and Quality Development It is in the area of quality control and quality development that PLS RAMBOLL has found some of the greatest weaknesses in the internal efficiency of Cedefop. At the global level, no general quality policy or strategy encompassing all the activities and services at the Centre exists, and actual quality assurance activities seem to be initiated not from the Directorate but from those directly involved. As only minimal quality control and reviewing of products and services takes place, and given the fact that only very little of the quality control and review 18 External evaluation of Cedefop - Final Report

8 that does take place is the result of formalised procedures, PLS RAMBOLL highly recommends that: Cedefop develops a strategy for the systematic quality control of both its products and services, such as systematic peer reviews and output measures/indicators for its projects and work in general. Co-operation with the Management Board The predominant feeling among Cedefop staff seems to be that the Management Board is too big, and is therefore an inefficient body with members who are not really engaged with the content of the work being carried out at Cedefop. However, there is also a large segment of Cedefop staff in favour of the fundamental tripartite principle that the Management Board is built upon, and who do not see any other way to structure it. The Bureau of the Management Board seems to be perceived much more as a co-operation partner for Cedefop. PLS RAMBOLL recommends that: some effort is put into developing ways of getting the two bodies to feel more closely interrelated. One suggested possibility is to increase the involvement of the expert staff with the Management Board in the political and strategic discussions concerning vocational education and training. Co-operation with the Commission In the assessment of PLS RAMBOLL, in recent years the relationship to the Commission seems to have improved considerably. There seem to be good opportunities for supporting the trend of positive developments in the relationship between the Commission and Cedefop. On the other hand, these positive developments might be at the expense of the relationship with the Management Board, which is a more direct vehicle for member state influence. 19 External evaluation of Cedefop - Final Report

9 3.2 The Management Board - main conclusions and recommendations The functioning of the Management Board and the Bureau It is the overall assessment of PLS RAMBOLL that a major strength of the Management Board is its high number of highly enthusiastic members. A weakness is that the Management Board spends too much of its time on administrative details, particularly on those concerned with the relocation of the Centre, rather than on focusing on the strategic development of Cedefop. The existence of the Bureau is an opportunity to make the decisionmaking process in the Management Board more efficient. However, the lack of a formal decision regarding the role and the competencies of the Bureau, and the lack of any systematised diffusion of information from the Bureau to the Management Board members, are viewed as major weaknesses. On the background of these findings it is the recommendations of PLS RAMBOLL that: the relationship between the Bureau and the Management Board is formalised. the exact purpose and competencies of the Bureau are made explicit. strategies for the improved dissemination of information from the Bureau to the Management Board are ensured. the gaining of more time for the Management Board to consider strategic and political discussions by reducing the time spent on administrative matters should be considered. a reduction in the time spent on administrative matters by separating issues of administration into items of high relevance and items of lower relevance. The effectiveness of the Management Board PLS RAMBOLL considers that the size of the Management Board reduces its efficiency as a decision-making body. This issue becomes even more relevant when taking the future enlargement of the European Union into consideration. 20 External evaluation of Cedefop - Final Report

10 The different possible models include enlarging the Management Board in the same way as hitherto, having fewer representatives per country in the Management Board, and/or making the Bureau more active. Each of these models has its strengths and weaknesses and fundamentally it is a political decision, which affects not only the composition of the Cedefop Management Board, but all the institutions and agencies of the European Union. PLS RAMBOLL finds it paramount that this highly political and sensitive issue is addressed, and therefore strongly recommends that: the consequences of enlargement on the work of the Management Board are discussed among the relevant partners as soon as possible. The relationship between Cedefop and the European Commission, and the role of Cedefop It is the impression of PLS RAMBOLL that both the Commission and the majority of the Management Board members believe that the relationship between Cedefop and the European Commission has improved considerably in recent years. In addition, it is the impression that the Centre now lives up better to its aims as defined in the statutory objectives. A divide seems to exist internally in the Management Board between those would like to see the Centre becoming more active in highlighting policy implications, and those who believe that the Centre should concentrate on disseminating research and information. For its part, the Commission sometimes has the feeling that Cedefop is trying to push the boundaries of what it considers to be its role, and sometimes oversteps the competencies of the Centre when making explicit policy recommendations. While the difference in opinion between part of the Management Board and the Commission may be characterised as a weakness which could affect the co-operation between the partners and make it less fruitful, the fact that the Centre is seen as trying to push its boundaries can be seen as a strength if it is interpreted as a sign of 21 External evaluation of Cedefop - Final Report

11 engagement and the Centre's desire to become a more active partner. 3.3 Defining and implementing the medium-term priorities and the annual work programme PLS RAMBOLL considers that a core strength of Cedefop is its use of its medium-term priorities as a goal-setting tool. In relation to the Centre s work on the priorities for , it is PLS RAMBOLL s overall assessment that the process has improved compared to the one, which was applied to the work on the priorities. However, it is a weakness that the stakeholders to be involved are not defined beforehand, and that the Centre has no clear strategy concerning the involvement of stakeholders. Also PLS RAMBOLL considers it a weakness that the process for developing the priorities is too top-down. The medium-term priorities are formulated in very general terms. The advantage of this is that it leaves room for flexibility, and thus allows the Centre to respond to a constantly changing agenda. The downside is that the breadth of the medium-term priorities weakens them as an instrument for setting priorities, or as guidelines for the annual work programme. The limitation most commonly mentioned by staff regarding the setting of realistic priorities is that the medium-term priorities have therefore become a political document, which is too broad and muddled. Based on these observations, PLS RAMBOLL recommends that: the whole process regarding the formulation of the mediumterm priorities and the annual work programme is revised in order to include the opinions of staff more widely in the process by creating better links between the staff and the Management Board, while simultaneously creating medium-term priorities that both retain their flexibility and increase their strategic focus, and thus enhance their usefulness as guidelines. 22 External evaluation of Cedefop - Final Report

12 progress reports are brought into concordance with the annual work programmes. there should be greater and more structured involvement of experts and stakeholders in the work on priorities and activities before presentation to the Management Board. time and energy is devoted to establishing clearer output indicators for the priorities, as this would make it easier for the Management Board to scrutinise the performance of Cedefop, and it would motivate the process of defining the priorities for setting realistic and attainable goals for Cedefop s work, and hence ease the process of establishing a feeling among staff at Cedefop of having some common goals. the Management Board requests better management tools from the Centre in terms of better follow-up of the implementation of the work programme, perhaps by using more specific output criteria. 3.4 External Effectiveness Analysis - main conclusions and recommendations General assessment of Cedefop The general assessment of PLS RAMBOLL is that Cedefop does important work in the promotion and development of vocational education and training in the European context. PLS RAMBOLL considers that a strength in the work of Cedefop is that the Centre mostly focuses on the right issues in vocational education and training. A weakness is that the Centre is trying to do too much, often being apparently unable to set negative priorities. It is the assessment of PLS RAMBOLL that by becoming more focused, the Centre might increasingly be able to set the agenda in the VET area and raise the visibility of the Centre. PLS RAMBOLL recommends that: an effort is made to raise the public profile of Cedefop. Relations with external programmes and organisations at the European and international levels It is the assessment of PLS RAMBOLL that the co-operation between Cedefop and various external programmes and organisations functions well in general, but that international organisations and Cedefop 23 External evaluation of Cedefop - Final Report

13 could benefit from even closer co-operation concerning both joint projects and networks, and the co-ordination of activities. The assessment of PLS RAMBOLL is that Cedefop might benefit from closer cooperation at levels other than at the Directorate level. PLS RAMBOLL recommends that: it be discussed within the Management Board and the Directorate of Cedefop whether Cedefop could gain from strengthened co-operation with the European Training Foundation. PLS RAMBOLL finds that the relations and synergy between the Leonardo da Vinci programme and the Study Visits programme are to some extent ensured in the present structure, and that the Study Visits programme significantly strengthens the mobility aspects of the Leonardo da Vinci programme. PLS RAMBOLL recommends that: the Study Visits co-ordinator participates more systematically in the meetings of the Leonardo da Vinci committee. the potential of the Study Visits programme as a tool for preparing the accession and increasing integration of the candidate countries into Cedefop s work in general is considered carefully. the themes of the Study Visits programme should reflect the medium-term priorities of Cedefop to a larger degree in order to enhance links to specific research projects and areas within Cedefop. during future staff recruitment, the Study Visits department should place continuing emphasis on enhancing relations with the other departments within Cedefop. the Study Visits department initiates a follow-up system to make people stay in contact. The creation of networks and the formation of synthesis seminars are possible tools for creating multiplier effects from the Study Visits. the subject rather than the time frame for the visits should be the focus for attracting appropriate target groups, and the duration of the visits should be more flexible, depending on the host country and the subject. the formation of working groups including both National Liaison Officers/Technical Assistants and Cedefop staff members 24 External evaluation of Cedefop - Final Report

14 should be continued as a way of further improving the organisation and implementation of the Study Visits programme. the work on integrated and consistent survey tools is continued and supported in order to generate relevant tools for the future planning and implementation of the Study Visits programme. the actual and planned extensive use of the website as a tool for the dissemination of information and administration of the Study Visits programme is continued, and that Cedefop places emphasis on training the users of the website. PLS RAMBOLL finds that Cedefop is generally acknowledged for its contribution in bringing people together in order to break down the barriers between different professions and countries. However, the support, which the different networks receive, seems to differ substantially. PLS RAMBOLL therefore recommends that: Cedefop looks into its own role regarding the various networks and forums in order to find out how the Centre can best support and nurture them in the future. that the work related to the Transparency Forum should continue and be developed further. Cedefop s information and communication services There is widespread satisfaction with the information material produced by Cedefop and the way it is communicated. Nevertheless, it is the assessment of PLS RAMBOLL that the overall information and communication strategy of the Centre has scope for improvement. PLS RAMBOLL considers that there is a large potential to widen the audience, and recommends that: extra effort is made to direct the publications at more specifically defined groups, and that more basic introductions, executive summaries and separate briefs are published. Cedefop examines the procedures with which it responds to external enquiries, in order to ensure that all enquiries are handled within a given period of time and none are lost. the work on promoting and developing the Electronic Training Village is continued in order to raise the visibility and user op- 25 External evaluation of Cedefop - Final Report

15 portunities of the site, and to further develop it for the sharing and development of knowledge. that the Electronic Training Village closely is incorporate into the strategies of Cedefop. The research activities of Cedefop PLS RAMBOLL believes that the quality of Cedefop s research activities is quite high, has improved in recent years, and is better related to the general EU agenda. However, some people believe that the quality of research is too variable. PLS RAMBOLL considers that this may be the result of the absence of a formal quality control system and obligatory peer reviews, and therefore recommends that: a formal quality control system with external peer reviews of publications is adopted. new initiatives taken to improve the dissemination of the next Research Report are supported as much as possible. 3.5 The move to Thessalonica It is the assessment of PLS RAMBOLL that the move from Germany to Greece has had an impact on the work of Cedefop in several ways. Among others, the commitment of the staff working in Cedefop and the improvements in the quality of work are seen as consequences of the redirection that followed the shift to Greece. The collective understanding among both staff and the Management Board is that the move to Greece caused a lot of difficulties, but that most of them are now resolved. It is the assessment of PLS RAMBOLL that in the long run Cedefop seems to have gained from the move, as it has had a chance to redefine objectives and working practices, and has thereby caused its external reputation to rise. A negative aspect of the transfer is that it is more difficult to recruit staff, because transport to Thessalonica is difficult and because many people, particularly those from North or Central Europe, might find the location in Thessalonica fairly remote. 26 External evaluation of Cedefop - Final Report

16 The location of the Centre in Thessalonica means that due to its rather physical distant location in a European VET context, the Centre has to enhance its visibility in other ways. PLS RAMBOLL therefore recommends that: ICT-based forms of co-operation and the already extensive use of the Internet are further developed to overcome the geographical isolation. 27 External evaluation of Cedefop - Final Report