ESF Ex post Evaluation Synthesis

Transcription

1 ESF Ex post Evaluation Synthesis Country Report Estonia Contract VC/2015/0098 Written by Metis GmbH, Fondazione Brodolini and Panteia October 2016 Social Europe

2 EUROPEAN COMMISSION Directorate-General for Employment, Social Affairs and Inclusion Directorate G Audit and Evaluation Unit EMPL. G4 Evaluation & Impact Assessment EMPL-G4-UNIT@ec.europa.eu European Commission B-1049 Brussels

3 EUROPEAN COMMISSION ESF Ex post Evaluation Synthesis Country Report Estonia Directorate-General for Employment Social Affairs and Inclusion European Social Fund 2016

4 Europe Direct is a service to help you find answers to your questions about the European Union. Freephone number (*): (*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you). LEGAL NOTICE This document has been prepared for the European Commission however it reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein. More information on the European Union is available on the Internet ( Luxembourg: Publications Office of the European Union, 2016 Catalogue number: KE EN-N ISBN: doi: /47459 European Union, 2016 Reproduction is authorised provided the source is acknowledged. For any use of photo which are not under the European Union copyright, permission must be sought directly from the copyright holder(s) indicated.

5 Glossary of key concepts A2E Access and sustainable integration into employment one of the Priorities of Article 3 Scope of assistance of the ESF Regulation No 1081/2006 Action Adaptability Annex XXIII categories Category of expenditure (CoE) Certified expenditure Cluster Community added value (CAV) Effectiveness Efficiency Gender sensitivity ISCED Intermediate Body (IB) Intervention The second level in the OP architecture, usually the Priority Axis (see below) consists of several actions A key policy area in the ESF, consisting of activities to increase the adaptation of workers and enterprises to the changing economic circumstances and labour market demands - one of the Priorities of Article 3 Scope of assistance of the ESF Regulation No 1081/2006 The socio-economic characteristics of ESF participants reported in the ESF monitoring systems, relating to the participant gender, labour market status (employed (of which self-employed), unemployed (of which long-term unemployed), inactive of which in education and training), age (young people aged and older people aged 55-64), disadvantaged status (migrants, minorities, disabled, other disadvantaged) and educational attainment status (by ISCED levels) Categorisation of the Structural Fund expenditure; cf ANNEX IV of COUNCIL REGULATION (EC) No 1083/2006 of 11 July ESF expenditure relates to Categories 62 to 74. Expenditure incurred in the implementation of the ESF activities which has been approved by the Managing Authority of the Operational Programme and the European Commission A group of actions or interventions with common objectives and activities The extent to which the ESF activities provided effects additional to the national / regional activities The extent to which the set aims and objectives have been reached The comparison between the achieved outputs and results and the costs incurred The extent to which the planning, design, implementation and monitoring reflects the gender issues International Standard Classification of Education, an international standard classification used to classify the education levels: 1 Primary education 2 Lower secondary education 3 Upper secondary education 4 Post-secondary non-tertiary education 5 Short-cycle tertiary education 6 Bachelor or equivalent The organisation charged by the Managing Authority to implement the ESF funds in the OP The third level in the OP architecture, usually the Actions 5

6 Human capital Managing Authority (MA) Operational Programme (OP) Output Participant Priority Axis (PA) Project promoter Result Strengthening Institutional Capacity Sustainability in the OP consist of several interventions A key policy area in the ESF, consisting of activities to develop the skills and knowledge of human resources across the different stages of the education and training system cycle, this theme is subject to another thematic evaluation - one of the Priorities of Article 3 Scope of assistance of the ESF Regulation No 1081/2006 and the focus of this evaluation The institution in each Member State OP responsible for the strategic direction and financial management of the OP The means through which the ESF support was implemented in the Member States, as agreed between the European Commission and the Member States. Each OP consists of several Priority Axes, which in turn consist of several actions, which in turn consist of several interventions The immediate reach of the ESF activity (e.g. number of participants reached, number of schools or enterprises supported) The person who participated in the ESF funded activity. The first level in the OP architecture, usually the OP consists of several Priority Axes (concepts of priorities, areas and others are also used in the OPs), which in turn consist of several actions and each action of several interventions The organisation in charge of implementing specific ESF funded projects The change achieved through the activity upon leaving to long term achievements of ESF activities (e.g. number of qualifications acquired by participants, number of enterprises providing training) A key priority focussing on the efficiency of public administrations and public services at national, regional and local level by promoting mechanisms to improve good policy and programme design, monitoring and evaluation, and capacity building in the delivery of policies and programmes in the relevant fields. SIC is one of the Priorities of Article 3 Scope of assistance of the ESF Regulation No 1081/2006. The extent to which the achieved results last 6

7 Table of Contents EXECUTIVE SUMMARY 10 INTRODUCTION 16 1 BACKGROUND AND CONTEXT Challenges and role/position of ESF Challenges faced by the country in the different policy fields at the time of programming and during the programming period CSR addressed by the ESF ESF response to the crisis Support and Contribution to the Lisbon Strategy and Europe 2020 strategy and targets SCALE OF ESF INVESTMENT 20 3 EFFECTIVENESS BY PRIORITY Outputs and results Outputs Results Evidence of effectiveness EFFICIENCY BY THEME 29 5 SUSTAINABILITY 32 6 GENDER SENSITIVITY 33 7 COMMUNITY ADDED VALUE Volume effects Scope effects Role effects Process effects SOCIO-ECONOMIC IMPACT 38 9 KEY LESSONS LEARNED What are the key lessons in terms of policy choices? What are the key lessons in terms of target groups? What are the key lessons in terms of the appropriate programming? What are the key lessons in terms of the effective implementation? What are the key lessons in terms monitoring? What are the key lessons in terms of evaluation? ANNEXES 46 7

8 List of tables and figures Table 1. ESF Ex post evaluation, main figures by policy theme Table 2. Financial allocation and implementation rate of the ESF programming period Table 3. Cost per participant/participation of some interventions Table 4. Cost per output by theme Table 5. Share of women by theme Table 6. Socio-economic indicators, Table 7. Financial allocation by Priority Axes and policy field Table 8. Data on participants of the OP Table 9. Output indicators by policy field Table 10. Result indicators by policy field Table 11. Impact indicators by Priority Table 12. Inventory of evaluations Table 13. CSRs by theme Table 14. Mapping of Strengthening institutional capacity Figure 1. Labour market status of participants in ESF (entering) Figure 2. Education status of participants in ESF (entering) Figure 3. Vulnerable target groups participating in ESF (entering) Figure 4. Share of participants by policy field

9 Acronyms A2E AIR ALMP CAV CoE CF CSR EC EEN ESF EU FEI HC IB ISCED LMP MA MS NEET NSRF OP PA SFC SI Access to employment and sustainable integration into employment Annual Implementation Report Active labour market policy Community Added Value Category of expenditure Cohesion Fund Country Specific Recommendation European Commission Expert Evaluation Network European Social Fund European Union Financial engineering instrument Human Capital Intermediary Body International Standard Classification of Education Labour market policy Managing Authority Member State Not in Employment, Education or Training National Strategic Reference Framework Operational Programme Priority Axis Structural Funds Common Database Social Inclusion 9

10 Executive summary Headline figures and key findings at the end of 2014 Activities were implemented under the ESF Operational Programme Human Resources Development (OP HRD) Total financial allocation: EUR million (ESF contribution: 85%) 83.5% or EUR 385 million was incurred by the end of % of the total ESF investment was attributed to the Adaptability and Human Capital (HC) theme, 34% to Access to Employment (A2E). 1.1 million participations were registered by the end of 2014, of which 74% participated under the HC theme. 44% of the participants were employed, 38% with tertiary education, 36% with lower education (ISCED 1 and 2), 41% were young people (15-24) and 10% were minorities. Aggregation of results: o Participants in employment: 104,744 o Participants gaining qualification: 123,633 o Other positive results: 279,290 o Self-employment: 4,153 o Products: 66 o Entities: 1,981 Main findings: The main policy challenges facing Estonia in 2007 were the large number of inactive people outside the labour market and the skills gap. The ESF OP HRD addressed these issues by investing over half the funding in developing human capital and a third in access to employment. The interventions for employed people and for inactive people, especially NEETs were effective in helping people remain in and get closer to the labour market. There was very high coverage of registered unemployed and new target groups were reached. Support was also provided for innovative ALMP activities and for entrepreneurship. Background The country synthesis report covers the programming period up until the end of 2014 and includes the analysis of expenditure, outputs and results in Estonia across all policy themes. It builds on the Human Capital, Social Inclusion and Access to Employment ex post evaluations, Annual Implementation Reports and on other available ESF evaluations carried out in Estonia. 1. Policy challenges addressed by the ESF One of the main challenges in terms of access to employment, recognised in the Estonian National Strategic Reference Framework (NSRF), is that unemployment in Estonia is of a structural nature. Inclusion of inactive persons in the labour market is also one of the challenges mentioned in the NSRF. These include older people, people with disabilities and people who care for a family member (16% of inactive people aged years - 10,600 people in 2005). The issue of linking training and education more effectively to the needs of the labour market and increasing opportunities for low-skilled workers to improve their access to lifelong learning were among the country specific recommendations (CSRs) issued by the European Commission (EC) in 2011, 2012 and These were addressed by Priority Axis (PA) 1: Lifelong learning and PA 3: Good-quality and long working life. The OP HRD shows good alignment with the reform agenda of the Lisbon Strategy and the Europe 2020 Strategy and the CSRs. The main focus is on education and employment with an emphasis on the integration of the long-term and young unemployed into the labour market and the development of their skills. 10

11 2. The scale of ESF investment 2.1 Funding Overall, EUR 461 million (85% of which was ESF funding) was allocated to Estonia, of which about 83.5% was spent by the end of The HC policy theme received 59% of the funding. The implementation rate was higher for A2E (93%) than for HC (78%) Participations In total, there were approximately 1.1 million participations 1 in the interventions funded by the Estonian ESF OP HRD in the programming period , of which 74% participated in the interventions under the HC theme. The vast majority (74%) was supported under the HC Priority Axes (PA 1 and PA 2). Almost half of all participants were employed. The second largest group was the inactive in education or training (e.g. pupils, students). Lower qualified (ISCED 1 and 2) (36%) and higher qualified (ISCED 5 and 6) (38%) people make up almost 74% of all participants. Estonia has a very high share of young people (41%) supported by the OP and was one of the few countries to explicitly support minorities. 3. The effectiveness of interventions / What works for whom? The results set by the OP were mostly achieved, with PA 4 as the only PA to underachieve. However, the low fulfilment levels in this PA can be explained by the ambitious target levels set and changes in the external environment. Those factors aside, implementation of this PA was still considered to be satisfactory. All result indicators within PA 1 were overachieved. The majority of interventions of PA 2 Developing the Human Resources for R&D were achieved. The result indicators of the policy field Access to Employment show good performance, as most were either achieved (or even overachieved) or close to achievement. The result indicators of the policy field Strengthening Institutional Capacity also over-performed. Concerning effectiveness, within the HC policy field, PA 2 showed the most effective and successful measures according to the evaluations. The evaluators strongly recommend that these measures be continued in order to guarantee the sustainability of the investments made in research and infrastructure and soft measures. In the case of the policy field Access to Employment, it was difficult to assess the impact of the interventions. However, evaluations were mostly positive regarding effectiveness. Concerning the policy field Enhancing Administrative Capacity, evaluations highlight that different training programmes have had a positive impact on the professionalism of public sector employees, but this career development and training approach is not yet systematic. Aggregation of results Participants in employment: 104,744 Participants gaining qualification: 123,633 Other positive results: 279,290 People in self-employment: 4,153 Products: 66 Entities: 1, The efficiency of interventions Information about the efficiency/cost effectiveness is very scarce. However, data for certain interventions are available. For example, the cost-effectiveness of the measure providing wage subsidies is considered high. Furthermore, for most interventions, the actual cost per participant was lower than planned. The average cost per participant was EUR 363. The cost per output in the field of Access to Employment was lower than the cost per ALMP participant (EUR 389 vs. EUR 436). 1 As one person could participate more than once, the term `participations is used here. In the rest of the document, `participants is used, but may include double counting of individuals. 11

12 5. The sustainability Sustainability of results was measured for five interventions, the majority of which had positive sustainable results. Those with sustainable results included the intervention providing wage subsidies to employers, the development of services supporting working and reducing the care burden and the development of the nation-wide network of care institutions for the elderly. Those with negative results included labour market training programmes and measures in favour of R&D. Regarding the sustainability of interventions, the continuing case management training and the nation-wide network of care institutions can be mentioned. On the other side, in the field of R&D and higher education, the ESF funding has often been focused on the provision of services for which no alternative funding sources have been found. So there is a risk of discontinued development in some policy areas if funding stops and alternative funding sources are not available. 6. The gender sensitivity of interventions The Estonian situation with regard to gender can be assessed as ambivalent. On the one hand, the educational and employment levels of women are high. On the other, the gender pay gap and horizontal and vertical segregation are also high. In the OP HRD , only one measure is dedicated specifically to gender sensitivity. This measure was deemed appropriate, reasonably effective, reasonably efficient and sustainable; however, it did not help to improve work-related gender equality. Nevertheless, public discourse on gender equality has become more prevalent, an action plan to decrease the gender pay gap was approved by the Estonian government in 2012, and the Gender Equality Council was established in October 2013 to advise the government on strategies to improve gender equality. The national surveys carried out have provided a basis for an evidence-based approach to handling gender equality issues in society, are widely usable and have led to specific political steps to promote gender equality. Expression of positive attitudes towards gender equality has increased significantly, but the actual gender equality in enterprises has not changed. Concerning participation in the OP HRD, women are generally over-represented at 62%. However, there are differences across the policy fields, with Increasing adaptability having the lowest share of women (42%) and Strengthening institutional capacity having the highest share of women (70%). The latter can be explained by the fact that in Estonia more women generally work in the public sector. The low share of women in interventions in the field of Increasing adaptability is due to the higher share of welleducated women to men in Estonia. 7. Community added value of ESF investment (for each type, specify in which theme(s) the CAV has been predominant) presented by order of importance. Regarding volume, evaluation reports claim considerable added value in terms of increased volume of outputs that would not have been possible without the ESF support. The overall volume of people reached is substantially larger than it would have been without support, and many activities could not have been sustained at the present level and/or new measures developed without the ESF. For example, within the theme Access to Employment, 190,000 participants were supported, while in the same period there were only 28,000 ALMP participants. Participants in the activities regarding entrepreneurship for small and recently established enterprises benefited from support especially during the economic crisis where banks were particularly careful in offering finance to enterprises. Furthermore, the ESF support has also added to the volume effect of the services that are generally provided by local governments. In terms of scope, the ESF funding widened the scope of the national policies on access to employment and sustainable integration into employment and investment both in terms of target groups and policy areas. 12

13 In terms of role, the ESF actions in Estonia have resulted in the implementation of new activities in the field of access to employment and sustainable integration into employment that would otherwise not have been implemented. In addition, to those activities that are provided typically under national programmes and measures, several innovative instruments and approaches (such as job clubs and psychological counselling) were established. Lastly, in terms of process, the ESF actions in Estonia have improved the way some services in the access to employment and sustainable integration into employment area are offered to the target groups, contributed to the establishment of new partnerships and strengthened the capacity of various organisations and institutions. 8. Socio-economic impact 8.1 Macro level The ESF has made an important contribution to Estonian s economic development, as well as strengthened its competitive position. It may be concluded that implementing the ESF has helped to mitigate the negative impact of the economic crisis (e.g., restrained unemployment and promoted entrepreneurship). Entrepreneurial activity has increased year-by-year, indicating that an increasing number of people have become entrepreneurs either alongside their day job or instead of it 2. According to the EEN Country Report, PA 4 measures created more than 500 new working places and the export activities have grown by more than EUR 30 million. However, according to AIR 2014, the progress in indicators related to poverty, material shortage and gender pay-gap can be considered poor (same level as in ). 8.2 Meso level The percentage of graduates in higher education in the natural sciences, engineering, production and construction has significantly increased in the past few years. The AIR 2014 concludes that ensuring competitive income and up-to-date working conditions, as well as promoting science and internationalisation, are important prerequisites for the increase in the number of scientists and thus, to the achievement of the general objectives of the R&D and innovation strategy. One of the impacts regarding system improvement is related to the integration of health and welfare system (which also coordinates care and nursing care in Estonia) in order to ensure that it is easier for people to receive the necessary services provided under both systems. In addition, in the field of access to employment, the ESF has contributed to the improvement of the labour market services offered (according to the agreed quality requirements and on the basis of actual needs of target groups) by the Estonian Unemployment Insurance Fund (EUIF). 8.3 Micro level Through the ESF OP HRD, a significant number of people were able to successfully complete training and upgrade/update their qualifications. A large number of teachers were trained and a large number of participants in ALMP measures remained in work or found employment. 9. Overall assessment of all evaluation criteria The indicator targets set by the OP were mostly achieved, with PA 4 Increasing Adaptability being the only intervention to see underachievement. PA 2 showed the most effective and successful measures according to the evaluations. The ESF has made an important contribution to Estonian s economic development, as well as strengthened its competitive position. 2 Ibid., p. 9 13

14 10. Key lessons One lesson learned was that it is very important to increase active labour market measures and measures which will increase entrepreneurship and the competitiveness of economy. This helped Estonia to recover more quickly from the economic crisis, to restructure the economy within a shorter period of time and should hopefully have positive long-term effects. Evaluations strongly recommend that the measures be continued in order to guarantee the sustainability of the investments made in research and infrastructure and soft measures. The target groups addressed with the OP HRD can in general be considered appropriate. As an example, feedback from the target group (received within the PA3 intervention Development of counselling services supporting inclusion to the labour market inventory) on the services provided within the intervention was overall positive. The services offered (e.g. counselling) were regarded as necessary and, as a result, supported the everyday coping of the target group. The experience gained within the implementation of A2E related activities only confirmed that appropriate target groups had been chosen (in general high participation rate). Due to the changes in the labour market caused by the economic crisis, there was a need to include additional target groups, such as short-term unemployed, people at risk of losing their job and those who have received redundancy notice. The evaluation process has underlined the difficulties in the interpretation of indicators resulting in double counting, target setting (e.g. under PA 4) and a monitoring system with respect to the assessment of the contribution of Structural Funds measures to broader strategic goals. More result indicators measuring the long-term effects of the implemented activities should be used, which would provide a good input in planning the activities of subsequent periods and allow the assessment of which supported activities are more sustainable. The implementation of projects has slowed down due to a number of factors related to the bureaucratic regulation of funding and modest support from the I Intermediate Body on national level. There is a need to increase support to applicants during the application process and assist participants in resolving more complex procedural issues. 3 3 Expert Evaluation Network Country Report Estonia (2014), p

15 Table 1. ESF Ex post evaluation, main figures by policy theme Human Capital Access to Employment Social Inclusion SIC Partner ships Total Allocated in million EUR (EU + national funding) of which EU (%) Certified expenditure in million EUR (and % of allocation) (80%) (93%) 21.2 (87%) (82%) Participations (thousands) ,067 Women % Young people (16-24)% Unemployed % Inactive % Disadvantaged% Low skilled (ISCED 1-2)% Aggregated results Participants in employment 10,646 98, ,897 Participant gaining qualification 123, ,633 Other positive result 279, ,290 Products Entities 179 1,802 1,981 15

16 Introduction This country synthesis report for Estonia is part of Evaluation Task 2B: National level analysis of the ESF Ex post Evaluation Synthesis. The purpose of the report is to summarise the most recent and most robust evaluation evidence available, analyse the data and provide a succinct overview of the outputs, results and impacts of the ESF under the main ESF Priorities as defined in the ESF Regulation 4. These are: increasing adaptability of workers, enterprises and entrepreneurs with a view to improving the anticipation and positive management of economic change enhancing access to employment and the sustainable inclusion in the labour market of job seekers and inactive people; preventing unemployment, in particular long-term and youth unemployment; encouraging active ageing and longer working lives; and increasing participation in the labour market reinforcing the social inclusion of disadvantaged people with a view to their sustainable integration in employment and combating all forms of discrimination in the labour market enhancing human capital promoting partnerships, pacts and initiatives through networking of relevant stakeholders, such as the social partners and non-governmental organisations, at the transnational, national, regional and local levels in order to mobilise for reforms in the field of employment and labour market inclusiveness And, additionally in Convergence regions: expanding and improving investment in human capital strengthening institutional capacity and the efficiency of public administrations and public services at national, regional and local level and, where relevant, of the social partners and non-governmental organisations, with a view to reforms, better regulation and good governance especially in the economic, employment, education, social, environmental and judicial fields The ESF achievements under each Priority have been evaluated in the following sections: Extent to which resources were used, Effectiveness, Efficiency/cost-effectiveness, Socioeconomic impact, Community Added Value (CAV), Gender Sensitivity and Sustainability. For this country synthesis report, existing reports have been used emanating from EU level evaluations, in particular the recent thematic ex-post evaluations on Human Capital, Social Inclusion and Access to Employment and the reports of the ESF expert evaluation network (EEN) as well as national evaluation material and other relevant reports. This information has been updated with the AIRs from Given the reliance of the country synthesis reports on existing material for the most part, there is more detailed information on some policy fields than others. This concerns mainly Human Capital, Access to Employment and Social Inclusion. Given the financial volume, however, of the Priorities covered thoroughly by thematic ex post evaluations, this difference seems acceptable. For example, 59% of the OP Human Resources Development resources are used in the Priority Axes concerning Human Capital and Increasing adaptability and they supported 78% of the total number of participants. Some 34% were allocated to the policy field Access to Employment, covering 18% of the participations while Social Inclusion is considered a horizontal priority. 4 REGULATION (EC) No 1081/2006 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 5 July 2006 Article 3, Scope of assistance 16

17 1 Background and Context 1.1 Challenges and role/position of ESF Challenges faced by the country in the different policy fields at the time of programming and during the programming period The Estonian labour market was affected by the economic crisis but recovered quickly. GDP dropped by 14.7% from 2008 to The unemployment rate increased rapidly and markedly from 4.6% in 2007 to 16.7% in 2010, affecting in particular the low-skilled (ISCED 0-2) whose rate increased to 32.4% in Since then, the overall unemployment rate dropped to 7.4% in The share of long-term unemployed had a quite similar development with a peak of 7.6% in 2010, dropping to 3.6% in The youth unemployment rate has been very high since 1999 and higher than that of other age groups. During the economic crisis youth unemployment was 32.9% in 2010 but recently the unemployment rate has dropped (to 15% in ). The main reasons for youth unemployment in Estonia is the high dropout rate (especially from vocational education) and the mismatch between the education acquired and labour market demand. The share of young people not in employment, education or training (NEET) also rose significantly as an effect of the crisis from 8.9% in 2007 to 14% in 2010 (15-24 yearolds), but it dropped to 11.7% in 2014, which is slightly below the EU-28 average of 12.4%. Over the same period the share of unemployment decreased for women and increased for men. This can be attributed to the economic boom which encouraged young men to leave education but then resulted in unemployment during the crisis years. The decline in women s NEET status can be partially explained by changed preferences in family planning, which has increased the age of first-time mothers significantly 6. The rate of tertiary education attainment of the age group (46.6% in 2014) is above the average of the EU-28 and the national target of the Europe 2020 strategy of 40% as it increased by 10% in this programming period. The share of women with tertiary qualifications is substantially higher than for men in this age group (58.2% vs. 35.6%). The share of people at risk of poverty or social exclusion is slightly below the EU average (26% in 2014 compared to 24.4% for the EU-28), but it is higher than at the beginning of this programming period (22.0% in 2007). Surprisingly, the crisis had a relatively modest and short-lived effect on social exclusion and risk of poverty. The risk of poverty for the low-skilled is higher than the population average and is also higher than the EUaverage. Research shows 7 the spending on labour market policy increased rather rapidly in Estonia to fight the strong increase in unemployed people during the economic crisis. Until 2008 the labour market policy costs were rather low with a share of only 0.2% of the GDP, but rose to % of the GDP thereafter. One of the main labour market challenges recognised in the National Strategic Reference Framework (NSRF) is that unemployment in Estonia is of a structural nature. There are certain risk groups, such as young people (aged 15-24), long-term unemployed, people with disabilities and non-estonians whose integration into the labour market is more difficult due to the radical changes in the structure of the Estonian economy, insufficient knowledge of the Estonian language and the lack of active labour market measures for people with disabilities. Inclusion of inactive persons in the labour market is also one of the challenges mentioned in the NSRF. In addition to older people, people with disabilities constitute a significant 5 Overview of the employment in Ministry of Social Affairs, ESF Ex-post evaluation: Supporting the integration of disadvantaged groups into the labour market and society Estonia 7 The role of active labour policy in flexicurity design in Estonia. PRAXIS Centre of policy studies (2010) 17

18 group of inactive people as do people with care duties (16% of inactive people aged years - 10,600 people in 2005) CSR addressed by the ESF In the context of the Semester, since 2011 the Commission undertakes a detailed analysis of EU Member States' plans of budgetary, macroeconomic and structural reforms and provides them with country-specific recommendations for the next months. These recommendations also contribute to the objectives of the EU's long-term strategy for jobs and growth, the Europe 2020 strategy. Before 2011 Member States received country-specific recommendations in the context of the Lisbon Strategy for growth and jobs. The challenges mentioned above are also addressed by the EC country specific recommendations (CSR) for Estonia (from 2004 and ). A detailed overview of the CSRs can be found in the Annex. For example, challenges regarding the need to increase the participation of specifically young people and long-term unemployed in the labour market have been highlighted each year in the recommendation documents. Strengthening active labour market measures has also been one of the recommendations pointed out each year. As the provision of active labour market measures to different target groups has been one of the main activities implemented within the PA 3 Goodquality and long working life, it can be concluded that the PA 3 has reflected this issue to a great extent 8. The issue of linking training and education more effectively to the needs of the labour market and increasing opportunities for low-skilled workers to improve their access to lifelong learning were part of the recommendations of 2011, 2012 and This is addressed by PA 1 Lifelong learning and PA 3 Good-quality and long working life. Applying measures for fostering economic development and entrepreneurship in regions faced with high unemployment was recommended to Estonia in Recommendations related to enhancing the cooperation between universities and enterprises and the internationalisation of the research and innovation systems have emerged in recent years ESF response to the crisis The economic crisis induced some changes in the OP, which were introduced in The nature of the activities remained the same (no activities were cancelled or added); however, due to the increase in the unemployment rate, the range of the target group was enlarged by including also e.g. short-term unemployed people. Also the budgets of the measures were slightly changed by allocating more resources to certain activities (e.g. wage subsidies, work practice, labour market training). Moreover, it was decided to use the financial resources within the period All those changes proved to be successful. 1.2 Support and Contribution to the Lisbon Strategy and Europe 2020 strategy and targets In 2011, the National Reform Programme Estonia was approved. It describes the objectives for 2015 and The main focus in the coming years is on education and employment, with an emphasis on integration of long-term and young unemployed people in the labour market and on the development of their skills. This matches the CSRs of the last years and the main objectives of the ESF All measures of PA 3 (Access to Employment) and also the Social Inclusion investment contributed directly to increasing labour market participation as an objective of the Lisbon strategy and the Europe 2020 economic growth targets. In 2014, the overall participation in the labour market in the age group was 74.3%. The overall target set for 2015 (72%) is already achieved. The PA has thus had a positive impact on achieving the Europe ESF Ex-post evaluation Access and sustainable integration into employment National Template Estonia

19 targets and has been the contributed most to alleviating the impact of the economic crises with active labour market measures 10. The HC interventions also targeted EU2020 and ET2020 benchmarks through increasing participation in lifelong learning and employability. The rate of tertiary education attainment increased by 10% in this programming period (46.6% in 2014), and already exceeds the set target of 40% for 2020 for the age group ESF Ex-post evaluation Access and sustainable integration into employment National Template Estonia 19

20 2 Scale of ESF investment Estonia was allocated EUR 391,517,329 ESF funding for the programming period Including national funding (EUR 53 million) and private funding (EUR 17 million) this meant EUR 461 million, as illustrated in Table 2. Public expenditure on active labour market policy (ALMP) was EUR 276 million (Cat. 2-7) according to the Ministry of Social Affairs 11. The ESF HC investment comprised around 5% of the national education and training expenditure in the period 12. According to the SCF 2007 data, about 83.5% of the allocated financial volume was spent by the end of The policy field Increasing adaptability and enhancing human capital received the highest share (59% or EUR 271 million) of the total financial volume in this programming period. Absorption varied between the Priority Axes. The policy field Access to Employment (OP PA 3) reports the highest implementation rate (93.3%). The reason for the high implementation rate compared to the other policy fields is that due to the economic crisis there was a need to expand the range of the target group by including for example, short-term unemployed people. It was also decided to accelerate the use of financial resources (all the support was used between 2009 and 2011) 13. The PA with the lowest implementation rate was PA 2 Developing the human resources for R&D of the policy field Enhancing human capital at 65%. A detailed overview with the information for each policy field and PA is available in the Annex. A synthetic table is presented here. Table 2. Financial allocation and implementation rate of the ESF programming period OP Policy field Financial Allocation Share of total funding OP HRD Implementation rate (%) OP HRD OP HRD Increasing adaptability and enhancing human capital Enhancing access and sustainable integration into employment 271,600,501 59% ,718,852 34% 93.3 OP HRD Strengthening institutional capacity 24,465,675 5% 86.7 Total OP HRD 461,337, Source: own figure on the basis of SFC2007 (AIR2014) 11 Eurostat reports 126 million expenditure on ALMP (cat 2-7) for this period. 12 ESF Ex-post evaluation Investment into human capital, National Template Estonia 13 ESF Ex-post evaluation Access and sustainable integration into employment Country report Estonia 20

21 3 Effectiveness by Priority 3.1 Outputs and results Outputs In total, there were approximately 1.1 million participations 14 in the interventions funded by the Estonian ESF OP HRD in the programming period The vast majority (74%) was supported under the policy field Enhancing human capital (PA 1 and PA 2). Overall, 89% of the output indicators exceeded the target set with all themes reaching at least 80% (and A2E reaching 100%). The following figures provide an overview of the employment and educational status of all participants entering (measures) during this programming period. Almost half of all participants were employed. The second largest group was the inactive in education or training (e.g. pupils, students). The share of unemployed and long-term unemployed supported by the ESF is quite low compared to other countries. Figure 1. Labour market status of participants in ESF (entering) 3,3% Employed Self-employed 37,8% 43,6% Unemployed LTU 6,7% 8,0% 0,6% Inactive in education/training Inactive others Source: own figure on the basis of SFC2007 (AIR2014) Lower qualified (ISCED 1 and 2) and higher qualified (ISCED 5 and 6) people make up almost 74% of all participants, as illustrated in Figure 2. This reflects the focus of the OP on increasing the support for graduates in science and research and the integration into the labour market and the support of the long-term unemployed and vulnerable, low qualified groups. Figure 2. Education status of participants in ESF (entering) 38,0% 10,5% 15,1% 35,6% Primary or lower secondary education (ISCED 1 and 2) Upper secondary education (ISCED 3) Post-secondary non tertiary education (ISCED 4) Tertiary education (ISCED 5 and 6) Source: own figure on the basis of SFC 2007 (AIR2014) 14 As one person could participate more than once, the term `participations is used here. In the rest of the document, `participants is used, but may include double counting of individuals. 21

22 The Estonian OP supported above all young people (4 out of 10 participants are between 15 and 24 years) and to a lesser extent minorities and older people, as illustrated in Figure 3. The share of migrants and people with disabilities is very low. Figure 3. Vulnerable target groups participating in ESF (entering) 50% 40% 30% 20% 10% 0% 41,1% Young people (15-24 years) 9,5% Older people (55-64 years) 9,8% 0,4% 1,9% 3,7% Migrants Minorities Disabled Others Share of total participants Source: own figure on the basis of SFC 2007 (AIR2014) The policy field Enhancing human capital supported the vast majority of the participants (74%). It is also the largest policy field in terms of financial volume. Together with the policy field Increasing adaptability, it covers 78% of the participants. Figure 4. Share of participants by policy field 18% 4% Enhancing human capital 4% Increasing adaptability Access to Employment 74% Strenghthening institutional capacity Source: own figure on the basis of SFC 2007 (AIR2014) Looking at the outputs by policy field, with reference to Table 8 in the Annex, the following can be noted. Increasing adaptability and enhancing human capital This policy field supported about 827, participants in the framework of 3 PAs. The vast majority of the participants (95%) participated in the interventions of the policy field Enhancing Human Capital, where activities such as training for individuals, the modernisation of the competence-based qualification system, training for youth workers, development of integrated counselling services and capacity building for teachers were implemented. Activities related to R&D were e.g. grants for foreign researchers, the introduction of new teaching methods with a focus on ICT, training and short-term courses for higher education staff, activities to popularize science, development of networks of higher education institutions and enterprises and innovation. 15 The total number includes also the participants of the measure Health at work 22

23 The policy field Increasing adaptability has the lowest number of participants (37,000 or 3.5%) but 18.5% of the ESF funding. Due to the nature of activities, the share of employed persons (teachers, academic staff, specialists, etc. - 38%) and inactive people (pupils and students - 60%) is very high and the share of unemployed is marginal. Under this PA, four general groups of activities were supported: training, counselling, awareness raising and support for start-ups. The interventions exceeded the targets for the number of participants/entities/products in most of the cases. With regard to the theme enhancing human capital all targets set were reached (except the interventions targeting vocational education and training (VET) teachers and VET centres). The number of teachers trained even exceeded the target value by 1200%, upgrading the qualification of about 13,000 teachers. However, this number has to be treated with caution as, at the beginning of the programming period, the specific definition of training was missing and therefore different beneficiaries have called different events training. Also, there is no information available about specific participants and one person could be counted several times if he/she participated in different training courses. Due to these problems the implementing body does not treat this indicator as trustworthy 16. The output indicators related to research and development (R&D) show good results in supporting a new generation of scientists and researchers. The ESF co-financed long term grants, attracted 43 international lecturers and scientists, about four times more than targeted, which had positive effects on the internationalisation of the Estonian R&D sector. Access to Employment In this policy field, about 190,000 people were supported during this programming period (compared to 28,305 participants in ALMP measures from 2007 to ). Activities included counselling services and improvement of access to high-quality career services and direct employment mediation; relocation and transport schemes to reduce regional differences on the labour market; care services for children for a better combination of work and family life; measures to raise health awareness, including emphasis on increasing people s physical activity level, improving healthy eating habits, reducing substance abuse and risky sexual behaviour. The rehabilitation system of people with disabilities was reorganised to offer career, work-based and psychological rehabilitation services and opportunities for adjusted workplaces and sheltered work. Different measures also targeted non-estonian speaking unemployed (opportunity to learn Estonian, professional training, counselling), young unemployed (work practice, volunteer work, etc.) and older workers (including raising the awareness of employers regarding flexible forms of work). Employers and employees were targeted with awareness raising activities (seminars, information materials) about safe working environment and publicity campaigns helped increase appreciation of the protection of the health of workers. Overall, maintaining the competencies and competitive ability of people helped react to the changes of the economic situation in the country. About 76% (of which 50% long-term) of the participants were unemployed and 19% employed. Some 30% of the participants were from minorities, which reflects the aim of social inclusion and access to employment for those having difficulties being integrated into the labour market. Since the beginning of the programming period, more women (around 60%) than men have participated. The share of unemployed people participating in the measures is declining: in 2009, their share of the total number of participants was 97% and it declined to 53% in At the same time, the share of non-active persons has increased from 1% to 5%. An even more remarkable increase can be noticed among the employees - from 1% in 2009 to 39% in The reason might lie in the changes in the context and application rules for the measures as people at risk of losing their jobs were later also eligible to apply for different active labour market measures (primarily 16 Expert Evaluation Network Country Report Estonia. 17 Eurostat 23

24 people who have been made redundant) and more budget was allocated to certain activities (e.g. wage subsidies, work practice, labour market training) 18. All targets set were reached. There is a significant over-fulfilment of these (between 130% and 283%). The reason for the over-fulfilment is the worsening economic situation and increase in unemployment, leading to a significant increase in the number of people participating in active labour market measures and/or receiving services that support finding a job 19, 20. Reinforcing Social Inclusion No PA is dedicated to this policy field as it was set up as horizontal priority in the OP and different programmes and measures across the OP have a social inclusion (SI) focus. Several interventions of PA 1 and the PA 3 were closely linked to SI (e.g. preventing school-drop-outs, teaching Estonian language, counselling and training activities). Strengthening institutional capacity Overall, 43,434 people were supported under this policy field. All of the participants in this policy field are employed and 70% have tertiary education as the measures target employees of the Estonian administrative system. Overall, 25,674 people were trained during this programming period (civil servants and workers trained with the organisational development training programmes and horizontal training programme). In the intervention targeting people employed at NGOs, an important aspect for the policy field SI, 11,404 people were trained (compared to a targeted 2,100). One explanation behind this over fulfilment of the output targets is that one person can participate in several training events and thus be counted more than once Results Overall, 66% of the result indicators exceeded the targets set, ranging from 57% within A2E up to 100% in Strengthening institutional capacity. Nevertheless, results are good for A2E as the result indicators which did not reach the target set have a success rate of at least 85%. Aggregation of results: Participants in employment: 104,744 Participants gaining qualification: 123,633 Other positive results: 279,290 Self-employment: 4,153 Products: 66 Entities: 1,981 Enhancing human capital Almost all of the result indicators of PA 1 of this policy field characterise changes related with participants four indicators are connected with completed training and two with required qualification of teachers. The set target of the indicator number of youth workers who successfully finished training is fulfilled by 175% (in a total 6,997 participants) as there was great interest towards participation in different types of training, but also flexibility on the part of the training organisers has allowed participation of more youth workers. The results of the aforementioned activities are also assessed by the number of holders of the new qualification certificate and almost doubled from 39,000 in 2012 to 77,854 in The 18 Ibid. 19 Ex-post evaluation of ESF investment in Access and sustainable Integration into employment - EU27 national template Estonia 20 Expert Evaluation Network Country Report Estonia. 21 Ibid. 24

25 over-fulfilment is partly related to the decision that all new certificates which have a European Qualification Framework (EQF) level reference are taken into account. The activities related to the objective increasing competitiveness of non-ethnic Estonians have been popular among the target group leading to an over achievement (4,747 compared to 3,750 targeted). There is a substantial over achievement of the indicator proportion of schools which systematically carry out internal evaluation activities while the target was set at 50%, by the end of 2014, 98% of the total number of schools were carrying out systematic evaluations. This obligation comes from the Law of Primary Schools and Gymnasiums and the Law of Vocational Education Centres which states that schools have to carry out internal evaluations every third year. The interventions of the PA 2 Developing the human resources for R&D are targeted to increase and support the number of people in tertiary education, to increase the quality of it and to increase the new generation of researchers and engineers needed for knowledge-based society and to foster internationalisation. As illustrated in Table 9 in the Annex, the majority of the indicators achieved the target set. Particularly positive is that all PhD students (both financed either through PhD study vacancies or foreign PhD grants) were positively evaluated and can continue their studies and research. Participants on complementary training are successfully completing their training (98% out of the 90% targeted). The number of participants who successfully complete e-courses at higher education levels was achieved by the end of 2014 (45,779 vs. 45,000 targeted). The main problem is with the effectiveness of continuing education (the ratio of students who complete their studies in continuing education to students admitted). Achievement levels at the end of 2012 were 21.5% for undergraduate and graduate students and 25% for doctoral students. The respective figures by the end of 2014 were 40% for undergraduate and graduate students and 60% for doctoral students, which can be explained by the high drop-out rate. Looking at the dropout rate it also has to be considered that that the target group of this intervention were ex-students who had already discontinued their studies before the economic crises and then were brought back to complete their studies. The high drop-out is caused by a combination of different reasons, the most common one being the inappropriateness of the chosen specialty, indicating the need for career counselling. 22 Regarding PA 4, the result indicators are not directly related to key results (job entries, self-employment, gained qualification), but cover broader training and renewal of skills, maintenance of jobs due to trained employees gaining additional qualifications, increasing the effectiveness of enterprises, etc. The low fulfilment levels can be explained by the fact that fairly ambitious target levels were set at the beginning of the programming period and also by the changes in the external environment (economic recession and its repercussions). In several cases, the indicators that have been chosen are difficult to measure 23. However, according to the AIR 2012, the implementation of the PA can be considered to be satisfactory despite the aforementioned problems as general enterprise activity (which could be one of the impact indicators outside the OP characterising this axis) has increased year-by-year with the biggest increase occurring in Access to Employment The result indicators of this policy field underline the good performance of the interventions. The first two indicators are related to key results job entries and job entries sustained. Some 42.5% out of targeted 50% of those who participated in active labour market measures started to work after participation. The share has increased by 11.5 percentage points from 2011 to 2013 due to a better economic situation. 22 ESF Expert evaluation network Country report Estonia (2014) 23 AIR 2012, p.8 25

26 However, there are still problems for people with disabilities in finding employment as the majority of enterprises is reluctant to invest in changing the working environment and employing people from vulnerable groups. However, 41% of the participants found work or kept their jobs after receiving support, i.e. 6% more than targeted. The share has increased in the last years from 15% in 2010 to 30% in 2011 up to 41% in 2014, which indicates an increasing willingness of enterprises to employ this target group. The result indicator share of people referred to active labour market services with the help of services supporting finding work, as a proportion of all those who received the relevant services was at 26%, almost reaching the set target of 30%. The share has increased by 8% since 2012 and there is still a need to expand the awareness of the situation of people from vulnerable groups and to diminish employers fears related to employment of people from these groups. One of the objectives of this policy field was to improve the quality of life. The over achievement of the result indicators (share of total municipal governments who have prepared a local health profile; number of enterprises and institutions who have joined the network of promoting health jobs) demonstrate good progress in this field. Strengthening institutional capacity The result indicators of this policy field show an over performance of each intervention, especially the number of training programmes, that was developed (66 out of 12 targeted). The activities carried out within this policy field involved mainly development activities. These would possibly have been carried out at some point using the institutions own funds but given the need to solve daily operational problems (particularly during the economic recession), development related activities tend to remain in the background. On the positive side, due to the programmes planned within the priority Enhancing administrative capacity, the resources were allocated to development activities, which were planned a long time ago. In general, this means that the added value is good on average 24. The most successful activities were the centralized training and the training programmes of NGO county support structures, the programme of more effective public services and the programme of better regulation. Within these programmes the planned activities have been implemented most effectively. In general, the effectiveness of this priority axis is good and the target levels of output and result indicators have been achieved (as of 2011). However, the achievement of results was negatively affected by several unforeseen external factors including the economic crisis 25 which had consequences for training and development activities in the public sector. In addition, the reduction of staff levels has increased workloads and reduced possibilities to engage in development activities. Furthermore, the evaluation saw as an important factor potentially limiting the impact of the interventions under the priority axis the increasing influence of partisan politics on policy making. The same factor was also said to potentially influence the implementation in practice of the new competence model and recruitment system for higher civil servants. The fragmentation of activities is also a problem in the context of this priority axis. There was an overlap between various activities initiated in the context of different programmes or projects. The clearest example can be found in the framework of training programmes, where similar courses were sometimes offered to the same target groups at the same time. In addition, in some cases, the same types of activities were aimed at different target groups (civil servants, employers organisations, trade unions) without paying enough attention to the integrated character of these activities. An example is the involvement of social partners in the policy development process, which would have 24 AIR 2012, p The evaluation of the effectiveness and the impact of the PA 5 Raising Estonian administrative capacity (in Estonian), p

27 greatly benefited from more joint activities for employers, trade unions and civil servants. 3.2 Evidence of effectiveness Enhancing human capital In this policy field, several impact indicators were set for PA 1 (none for PA 2) to assess the effectiveness of the implemented measures. The participation rate of adults (25-64 years) in lifelong learning increased by 4.5% up to the set target of 11.5% in The participation rate of the adult non-ethnic Estonian population (25-64 years) in lifelong learning deals with the availability of further training (especially language training) the target set of 10% was almost achieved in 2012 but achievement decreased to 7.1% in The Mid-term evaluation of the implementation of measures in favour of R&D and higher education in the framework of the EU co-financed Structural Funds during the period found that the centres of excellence and researcher mobility measures can be regarded as the most effective and successful measures in this PA. There is a strong recommendation that these measures should be continued in future to guarantee the sustainability of the investments made in research, infrastructure and soft measures (doctoral schools, mobility schemes). For most of the result indicators under PA 4, the 2014 achievement levels are not available. The next small and medium sized enterprises survey will be carried out in The last survey was carried out in Two result indicators previously introduced survival rate amongst new enterprises that have received support; Proportion of study programmes where enterprise training is taught (universities and vocational education schools) helps in getting an idea of the effectiveness of the interventions. In the case of the survival rate of those enterprises that have received support, this indicator has achieved its target level (98.1%) 26. Access to Employment In the case of this policy field, four evaluations were initiated by the Estonian Ministry of Social Affairs. All evaluations commented that inaccurate indicators make it difficult to assess the impacts and allow different interpretations of the results obtained. The evaluation of the welfare measures supporting employment also highlights that the programme activities could not directly influence the recipients employment. In addition, the economic downturn worsened the opportunities to find employment among those who belong to vulnerable groups. At the same time, the evaluations state that activities offered within these programmes have a significant influence on improving people s wellbeing, confirmed by the interviewees, which is related to employment. The evaluation Business Start-up Subsidy, Work Practice and Coaching for Working Life found that from those people who were employed thanks to a wage subsidy, 74% have not registered themselves as unemployed, a good indicator for the effectiveness of this measure. The wage subsidy recipients were also participating in other measures career counselling, labour market training, work practice etc. The results of the counterfactual evaluation of the services in the programme Increasing the Supply of Qualified Labour carried out in 2012 are positive regarding the effectiveness and sustainability of the impact of the programme as the difference in the employment rates between participant and control groups shows. Six months after exiting the programme the probability of being employed is about 56% higher for participants than for the control group. Over time, the difference diminishes, but even 18 months after exiting the programme, the participants employment rate remains over 30% higher. 26 ESF Expert Evaluation Network Country Report Estonia (2014) 27

28 The evaluation of the effectiveness and the impact of the PA 5 `Raising Estonian administrative capacity highlights that different training programmes have an impact on the professionalisation of public sector employees, but this career development and training approach is not systematic yet. Participation is voluntary and the effectiveness of training is analysed in the minority of the cases. Regarding the effectiveness of the interventions assessed in- depth in the thematic expost evaluation on Access to employment, the following can be concluded: The effectiveness of the interventions is evaluated as good regarding performance against targets. All selected interventions have achieved or exceeded their target values. The targets initially set were reviewed in some interventions during the implementation. The exceeding of target values of set indicators at intervention level may be largely due to the economic crisis at the time of the implementation that increased the volume of unemployed people in general (in %, in % and in %). But also, the implementation of the interventions facilitated innovation in the approaches, providing new delivery of active labour market services for the young and the elderly. Implementation of the intervention Raising the administrative capacity of labour market institutions was to a certain extent influenced by the structural changes undertaken in the beneficiary organisation in parallel, causing changes in priorities and as a consequence a part of the indicators were over achieved but at the same time a part were under achieved. Regarding the interventions related to welfare measures, the majority of indicator target levels were achieved, but delays in launching the social services data register caused under performance connected with the training of the relevant employees. Regarding the activities carried out in the framework of open calls for proposals, the indicators are not set on the intervention level, which makes it difficult to evaluate their effectiveness at all ESF Ex-post evaluation Enhancing access and sustainable integration into employment Country report Estonia 28

29 4 Efficiency by Theme Information about efficiency / cost effectiveness is very scarce in the available documents. The information available was found in Estonian evaluations carried out by the Ministry of Social Affairs. Regarding the measure `Providing wage subsidies to employers`, the efficiency calculations (in the cost-benefit framework) of the Evaluation of services in the programme increasing the supply of qualified labour showed that EUR 1 invested in wage subsidies, resulted in EUR 7 return, of which EUR 2.8 is revenue for the state in direct taxes. Such a cost-benefit ratio can be considered as high compared to similar studies conducted in Estonia 28, but the results have to be treated with an understanding that the study does not take into account the product market displacement effects, which can diminish the results 29. According to the report 30, the relaxed eligibility conditions that were applied from January to December 2010 widened the focus of the programme from long-term unemployed to include those who had been unemployed for a relatively short period of time. In an economic crisis, the situation of the long-term unemployed is even worse than before, since the pool of all unemployed now includes people with fresh labour market experience who have an advantage in competing for available jobs. Keeping the focus of wage subsidies strictly on risk groups would have compensated for the risk groups disadvantage and their reduced employment opportunities. This is primarily an issue of value judgments, however, and it should also be kept in mind that despite expanding the target group of the programme, the overall amount of funding for the programme increased more than tenfold compared to and thus, opportunities for subsidised employment increased significantly also for risk groups. In general, wage subsidies should be targeted to risk groups who are in the worst situation in the labour market and these should be temporary. Table 3 compares the planned and the actual cost per participant/participation of some interventions. For most of the interventions the actual cost per participant was lower than planned, e.g. the cost of the intervention targeting young people (EUR 1,116,092) and the number of young people who actually participated in active labour market measures (6,375) within the intervention giving an average cost per participant EUR 175. Considering the initial plan to offer services to 5,219 young people (EUR 214 per participant), the intervention proved to be rather efficient. The average expenditure per participant is also lower for the ESF interventions compared to average cost per participant of the total labour market policy (LMP) measures in Estonia (EUR 296 vs. EUR 436) Studies addressing active labour market measures (for example Leetmaa et al, 2003 and Lauringson et al, 2011) have previously come to the cost-benefit ratio ranging from Extract from the Evaluation of services in the programme Increasing the supply of qualified labour , Centar, Interact, April 2012, p. 3-4, There are, however, important limitations regarding the conclusions. Data available to us enabled to estimate the difference between participants and nonparticipants employment rates. In reality, the impact of wage subsidies could be wider since the amount of the wage subsidy offered by the Unemployment Insurance Fund is substantial in case of low-wage jobs (making up to 50% of the wage), subsidies may crowd out other employed people with similar levels of qualification. It is also possible that people who were hired with wage subsidies, would have been hired even without it. It was not possible to take these effects into account in this study (data requirements for estimating such effects would substantially exceed the possibilities of this study), but their presence can significantly reduce the actual benefit-cost ratio of the programme. It should also be mentioned that although the data used by researchers included the most important socio-demographic characteristics that should ensure the comparability of the participant and control groups, but it cannot be excluded that there are other important unobservable characteristics (e.g. personality traits, motivation, cognitive ability) that affect both participation in the programme and subsequent labour market performance. Not taking these into account could potentially lead to overestimating the efficiency of the programme. 30 Evaluation of services in the programme Increasing the supply of qualified labour , Centar, Interact, April Ex-post evaluation of Access and Sustainable Integration into Employment, Country Report Estonia 29

30 Table 3. Cost per participant/participation of some interventions Activity Planned service cost per participant/participation (in EUR) 32 Intervention: Improving access to employment for young people Provision of labour market services for young people Intervention: Improving access to employment for the unemployed non-estonians Provision of labour market measures for non-estonians Intervention: Improving access to employment for older people Provision of labour market measures for older people Actual service cost per participant/participation (in EUR) 33 Intervention: Development (at local government level) of services supporting working and reducing the nursing burden Training for social workers/development managers of local governments Training for social workers of local governments on case management STAR and case management training Training for child minders (profound and severe disabilities) Training for support persons Intervention: Development of integrated care and nursing care system in order to reduce the nursing burden Trainings for the personnel of nursing institutions for elderly Source: Ex-post evaluation of Access and Sustainable Integration into Employment, Country Report Estonia Overall, the average cost per participant was EUR 363. Looking at the table below, we find that cost per participant was lowest within the theme Human Capital, while Strengthening institutional capacity had the highest costs. The average cost per output of the theme Access to Employment (EUR 389) are lower than the average cost of the national ALMP measures (EUR 436). Variations can be explained by the difference in the nature of the interventions and the different services offered to specific target groups, such as the provision of labour market measures was EUR 534 for Non-Estonians and EUR 242 for older people. However, interpretation of this rough calculation dividing certified expenditure and participations have to be done carefully. 32 The planned budget of the intervention divided through the planned number of people participating in the activities 33 The actual budget of the intervention divided through the actual number of people participating in the activities. 30

31 Table 4. Cost per output by theme Theme Cost per output (in EUR) Enhancing Human Capital 256 Access to Employment 389 Strengthening institutional capacity 488 Total 363 Source: SFC2007; Authors calculation The ESF OP contributions paid to the FEIs (Financial engineering instruments) for enterprises and invested in final recipients at the end of 2014 in Estonia was EUR 6.01 million DG REGIO, Summary of data on the progress made in financing and implementing financial engineering instruments reported by the managing authorities in accordance with Article 67(2)(j) of Council Regulation (EC) No 1083/2006 (2015) 31

32 5 Sustainability Sustainability of result for individuals and entities The measure providing wage subsidies to employers gave sustainable employment results, according to the Evaluation of services in the programme Increasing the supply of qualified labour (see Chapter 3.2). The evaluation report also highlights that the relaxed eligibility conditions that were applied from January to December 2010 widened the focus of the programme from the long-term unemployed to also include those who had been unemployed for a relatively short period of time. This had its side effects: during the economic crisis, the situation of the long-term unemployed has become even worse than before, since the pool of all unemployed now includes people with more recent labour market experience who have an advantage in competing for available jobs. The impact evaluation Business Start-up Subsidy, Work Practice and Coaching for Working Life 35 found that the results for those participating in labour market training programmes were weak on sustainability. Just over half of those people who were participating in labour market training programmes (54%) entered employment, but almost 80% of those who entered employment were registered after some time 36 again as unemployed. As more than half of the participants in labour market training were long-term unemployed (they might belong also to other risk groups), the achievement of sustainable employment is even more important 37. Sustainability of interventions The intervention Development of services supporting working and reducing the care burden involves several sustainable aspects. For example, the social services data register (STAR 38 ) as a tool associated with the case management approach is currently in use. The sustainability of the intervention is also shown by the fact that the case management training is now part of the curriculum in Tartu University 39. Another good practice example for the sustainability of an intervention is the development of the nation-wide network of care institutions for the elderly, which is integrated in the activities of the relevant development plans for each county (and was elaborated within the intervention). The continuation of the development of the financing scheme for the care system of elderly on the basis of the analysis performed within the intervention is under way. Also the training of the personnel of the care institutions continued in the next programme that related to welfare services ( ). However, the mid-term evaluation of the implementation of measures in favour of R&D and higher education pointed out that, in several cases, sustainability of the measures has not been secured. The ESF funding has often been focused on the provision of services for which no alternative funding sources have been found. Hence, despite the fact that there is an intention to continue the funding of the most important activities under the new ESF programming period, there is a risk of discontinued development in some policy areas if funding stops and alternative funding sources are not available. 35 Praxis Center for Policy Studies, The Evaluation of the Business Start-up Subsidy, Work Practice and Coaching for Working Life (2010) 36 No detailed information about the period was available in the EEN report. 37 Expert Evaluation Network Country Report Estonia. 38 Training on using the DB was also an activity of the intervention 39 Ex-post evaluation of Access and Sustainable Integration into Employment, Country Report Estonia 32

33 6 Gender sensitivity According to the country report in Estonia occupies an ambiguous position in respect to gender disparities: The educational and employment levels of women are high but at the same time the gender pay gap and gender horizontal and vertical segregations are equally high. In the EU Gender Equality Index, Estonia is ranked 16 th. The Estonian NSRF also outlines the gender inequality referring to salaries, reconciliation of work and family life, low participation of women in management levels, tackling stereotypical gender roles and limited administrative capacity to implement both positive special measures for reducing inequality as well as integrating the gender aspect into all activities. In the OP HRD there is only one measure (1.3.5 Promoting of Gender Equality) related to gender sensitivity. The document raises the issue of the gender gap on a general level, presenting a very limited number of gender sensitive indicators on the impact level. The approach to target groups is mostly gender neutral, without gender specific targets. According to the EC country report 2011 the indicative allocation of ESF funding from the OP for category 69 (measures targeting women) was EUR 4.1 million which makes up 1% of the total amount and is also below the EU average (3.72%). Measure Promotion of Gender Equality focused on the application of legislative provisions on gender equality, increasing the awareness of employees and employers about the issue, and reducing the negative impact of gender stereotypes. During the programming period the gender equality mainstreaming strategy on the employment and education market was implemented, contributing mainly to creating the environment and preconditions for reducing the gender gap in accessing employment 41. In 2013, an evaluation of this measure was undertaken by the University of Tartu. They found that the measure as a whole has been appropriate, reasonably effective, reasonably efficient and sustainable. Work-related gender equality has not improved significantly during the implementation period of the measure. Changes in the actual situation through higher awareness are a longer term process, where activities must be systematic and continuous. Nevertheless, discussions on gender equality have become more prevalent and founded in society 42. One of the most successful activities was national surveys, especially the gender-related wage gap survey and the gender equality monitoring. These have provided a basis for an evidence-based approach to handling gender equality issues in society, are widely usable and have led to specific political steps to promote gender equality. Expression of positive attitudes towards gender equality has increased significantly, but the actual gender equality in enterprises has not changed. Employers interest in dealing with gender equality issues in their enterprises has been modest because people do not consider discrimination to be a problem and do not understand what the enterprise could gain from more equality 43. The interventions with the aim of reducing the care burden (establishment of an information and aid centre aggregating information on assistive technology, etc.; development of counselling centres for the people with physical, mental and social special needs and their families) were expected to have potential positive effects on the gender gaps, supporting the informal carers who are mostly women. There are no gender equality-related indicators on output and result level within the OP. On the impact level, the indicator Gender segregation in the labour market (segregation index) has been determined. The AIR 2014 acknowledges that in general the indicator value has changed only slightly during the programming period. This index cannot be 40 Evaluation of the European Social Fund s support to Gender Equality, 2011: Country report, Estonia. European Commission, Directorate-General Employment, Social Affairs and Equal Opportunities Estonia, Task 2, Phase 2 Country Report

34 related only to the implemented activities under the above mentioned measure as it is influenced by several factors. Promotion of gender equality is a long-term process. In 2012, an action plan to decrease the gender pay gap was approved by the Estonian government and in October 2013 the Gender Equality Council was established to advise the Government in relation to strategies to improve gender equality (policies). As illustrated in the table below, the overall share of women in the participants of the OP HRD is 62%. Looking at the themes, there are substantial differences, with Increasing adaptability having the lowest share of women (42%) and Strengthening institutional capacity having the highest share of women (70%). The latter can be explained by the fact that in Estonia generally more women work in the public sector. The lowest share of women in interventions increasing adaptability can be explained by the higher share of well-educated women than men in Estonia. As the activities in the field of increasing adaptability helped mainly to bring low-skilled and low-educated workers back to the education system, fewer women participated in the interventions. Table 5. OP Share of women by theme Theme Share of women (in % of total participants) OP HRD Increasing adaptability 42 OP HRD Enhancing human capital 64 OP HRD Access to Employment 56 OP HRD Strengthening institutional capacity 70 OP HRD Total 62 Source: SCF 2007; Author s calculation 34

35 7 COMMUNITY ADDED VALUE 7.1 Volume effects According to the EEN, evaluations carried out in 2009 at the NSRF and OP level in Estonia, as well as further evaluation reports of specific measures, considerable added value has been achieved in terms of an increased volume of outputs that would not have been possible without the ESF support. Although similar services are also provided and interventions are activated in Estonia without ESF support, the overall volume of people reached is substantially larger than it would have been without the support. For example, the relaxed eligibility conditions of participation in different active labour market services offered by the Unemployment Insurance Fund that were applied from January to December 2010 widened the focus of the programme from long-term unemployed to include also those who had been unemployed for a relatively short period of time. This helped to slow down the increase of unemployment, and especially longterm unemployment, as it is usually easier and quicker to find a new job if there have been no long unemployment periods where people lose their skills and also motivation to work. However, those who were long-term unemployed already faced more difficulties in finding a job. According to the representative of the Intermediate Body (IB), the interventions proved to be especially beneficial during the period of economic crisis, when the unemployment rate increased remarkably. Many providers highlight that their activity could not have been sustained at the present level and/or new measures developed without the ESF (especially beneficiaries of the PA 4 Knowledge and skills for innovative entrepreneurship ). Usually one of the biggest problems for small and recently established enterprises is to receive additional financing, even more during the economic crisis where banks were particularly careful in offering finance to enterprises. Also, several training possibilities would have been lost or significantly diminished in the absence of additional resources available from the ESF. The ESF support has also added to the volume effect of the services that are generally provided by local governments. The capacities of local governments in Estonia are very different, and they often lack resources to provide necessary services which, for example, help to improve possibilities for the inactive to participate in the labour market. For this reason, the ESF interventions Development (at local government level) of services supporting working and reducing the care burden and Development of integrated care and nursing care system in order to reduce the care burden helped to reach a significantly higher number of people than would have been possible with the resources of local governments only. 7.2 Scope effects The ESF funding widened the scope of the national A2E policies and investment both in terms of target groups and policy areas. The ESF intervention Development (at national level) of counselling services supporting inclusion to the labour market provided support in an A2E policy area that was not systematically addressed by non-esf funded national instruments. The intervention aimed to assist people with disabilities and their families in entering the labour market, which is, in general, supported by local governments. Due to very big differences in the capacities of Estonian local governments, not all the people who belong to the target group received the support they actually needed before the ESF intervention was introduced. In addition, several interventions Raising the administrative capacity of labour market institutions and Development of integrated care and nursing care system in order to reduce the nursing burden provided the opportunity to train the workers of the public employment services and personnel of care institutions in new and more specific fields. Moreover, the ESF intervention Development (at local government level) of services supporting working and reducing the care burden enabled outreach to target groups who 35

36 had not received sufficient attention before, such as the child minders and support persons for children with profound and severe disabilities. 7.3 Role effects The ESF actions in Estonia have resulted in the practical implementation of new activities in the A2E area that would otherwise not have been implemented. In addition to those activities that are provided typically under national programmes and measures, several innovative instruments and approaches were established. Key examples of added value through role effects include: 7.4 The interventions Improving access to employment for young people and Improving access to employment for older people involved two services which were inherently new job clubs and psychological counselling. The service of job clubs aims to offer additional opportunities for young and older inactive people to exchange experience with, and acquire new knowledge and skills from people who are in a similar situation. The service of psychological counselling aims to support the continuation of their job search, especially of those with no experience or who have been inactive for a long time, through solving personal problems and helping to overcome their crisis of motivation. Within the intervention Development (at national level) of counselling services supporting inclusion to the labour market, the Centre of Disability Related Information and Assistive Technology was integrated into The Astangu Vocational Rehabilitation Centre. The new centre enables the accumulation of the unbiased and relevant information and knowledge necessary for the disabled in order to improve their possibilities to cope with everyday life and to increase their chances to access the labour market. Within the intervention Development of integrated care system in order to reduce the nursing burden the health and welfare system (which also coordinates care and nursing care in Estonia) was integrated in order to ensure that it is more convenient for people to receive the necessary services provided under both systems. Additionally, within the intervention a basis was created for using a sustainable financing scheme for the care system of elderly. The aim of the measure Promotion of gender equality is the application of legislative provisions on gender equality, enhancing employees and employers awareness, implementation of the gender mainstreaming strategy on the labour and education market and reducing the negative impacts of gender stereotypes. Within the action several media campaign were carried out, e.g. the awareness raising campaign on gender stereotypes. Within the measure Health awareness several awareness-raising activities which aimed at reducing health related work absence by raising the health awareness were implemented. During the programming period there were actions taken to increase work aged populations health awareness, but there were also actions to increase specialist level knowledge of health promotion. To achieve this goal there were media campaigns and information spread toward the population and health promotion specialists. Process effects The ESF actions in Estonia have improved the way some services are offered to the target groups, contributed to the establishment of new partnerships and strengthened the capacity of various organisations and institutions. In particular, one of the objectives of the intervention Raising the administrative capacity of labour market institutions was to improve the professional skills of the specialists of the public employment service (Estonian Unemployment Insurance Fund) in order to ensure that the services are offered according to the agreed quality requirements. The same intervention also helped to strengthen cooperation between employers and the public employment service. Furthermore, the intervention Development of integrated care and nursing care system in order to reduce the nursing burden has raised the capacity of local government to provide good quality services in the fields of care and 36

37 nursing care and promoted the establishment of more cooperation between neighbouring local authorities. In addition, after the establishment of the services of job clubs and psychological counselling within the interventions Improving access to employment for young people and Improving access to employment for older people the first feedback was collected from the target groups. The feedback gave a valuable input to how to improve the way the services are offered in future (for example, the number of sessions of job clubs was increased in the next period of the programme). Also cooperation between state institutions and non-governmental partner organisations has become more efficient. In the programming period more structural instruments were provided to non-governmental actors (including local authorities, social partners, umbrella organisations in various fields, etc.) for the development of their general administrative capacity on the one hand, to improve efficiency in administration and the ability of non-governmental actors to participate in the decisionmaking processes. On the other hand, these organisations form a vital group amongst the possible applicants for structural instruments, and an increase in their capacity would assist in the more successful implementation of the Structural Funds. Several different training programmes have been developed to reach this goal and a large number of people have participated in these trainings. 37

38 8 Socio-economic impact Macro-level The ESF has made an important contribution to Estonia s economic development, as well as strengthening its competitive position. It may be concluded that implementing the ESF has helped to mitigate the negative impact of the economic crisis (e.g., restrained unemployment and promoted entrepreneurship). According to the Structural Funds Country Report 2012, Estonian GDP per capita increased from 64% of the EU-27 average in 2010 to 67% in Labour force productivity per employee amounted to 67.6% in 2011 of the EU-27 average 44. However, according to AIR 2014, the progress in indicators related to poverty, deprivation and gender pay-gap can be considered poor (same level as in ) 45. The table below illustrates that there is a common pattern across the indicators, showing a strong increase/decrease in 2009, and a slow recovery until 2013, almost reaching precrisis level in some cases. Table 6. Socio-economic indicators, Access to Employment Source: Eurostat When analysing the impact indicators of the OP HRD, AIR 2014 indicates that the objectives of the PA Lifelong Learning have been achieved for the most part. However, it is more difficult to reach target levels of the (impact) indicators which reflect the overall socio-economic situation and the state educational system and which are largely dependent on demographic and other factors. Approximately 12.5% (in 2013) of adults are taking part in lifelong learning to improve their job prospects twice as many as in earlier years (6.5% in 2006). This rate is higher than the EU-27 average 46. Meso-level One of the impacts regarding the system improvement is related to the integration of health and welfare system (which also coordinates care and nursing care in Estonia) in order to ensure that it is more convenient for people to receive the necessary services provided under both systems. In addition, in the field of access to employment, the ESF has contributed to the improvement of the labour market services offered (according to the agreed quality requirements and on the basis of actual needs of target groups) by the Estonian Unemployment Insurance Fund (EUIF). Micro-level Trendline Employment rate 69,8 70,1 63,8 61,2 65,3 67,1 68,5 69,6 Unemployment rate 4,6 5,5 13,5 16,7 12,3 10,0 8,6 7,4 Young people NEET, 15-24y. 8,9 8,7 14, ,6 12,2 11,3 11,7 Employment rates, Females, 15-64y. 66,2 66,6 63,2 60,8 63,0 64,7 65,7 66,3 Human capital Early school leaving 14, , ,6 10,3 9,7 11,4 Educational attainment 33,5 34,4 36,3 40,2 40,2 39,5 42,5 43,2 Social Inclusion Population at risk of poverty 22 21,8 23,4 21,7 23,1 23,4 23,5 26 About 80,000 people started to work after participation in active labour market measures. In general the theme Access to Employment had a strong impact due to the 44 Structural Funds Country Report, Estonian Ministry of Finance, December 2012, in Estonian, 45 Annual Implementation Report (monitoring report) reflects the implementation of the OPHRD in with state of , p Ibid., p

39 number of people participating compared to the number of ALMP participants, as mentioned above. One of the largest ESF interventions in Estonia is Increasing the Supply of Qualified Labour Force It aims to increase employment through active labour market measures by improving the availability and quality of employment services. The AIR 2014 states that as a result of the implementation of these active labour market services, the proportion of the recipients of the services compared to all people is steadily increasing (less than 30% in 2009 vs. over 42% in 2014) 47. One of the services provided was a wage subsidy. According to the evaluation of the services, the efficiency calculations (in the cost-benefit framework) indicate that one euro invested in wage subsidies results in seven euros in return, of which EUR 2.8 is revenue for the state as direct taxes. Such a benefit-cost ratio can be considered very high 48. Another labour market service intervention, provided in the framework of the programme, was labour market training. The evaluation report states further: The analysis (on those trained in January-September 2010) of data from Estonian Tax and Customs Board and Unemployment Insurance Fund reveals that six months after the start of the training the probability of being employed is about 5.8% higher for participants than those who did not take part in the training. The impact of training on the employment is sustainable as it remains statistically relevant also 12 and more months after the start of the training 49. According to AIR 2014, for the impact indicators of PA Good-quality and long working life, the target level can be considered as 100% achieved for activity rate (target and achievement level 74.8%). This is mainly due to the improved economic situation and the 94% achievement of the indicator the number of people in the age group who are inactive and whose reason for the inactivity is illness or injury. There are larger shortcomings regarding impact indicators reflecting gender equality: the employment gap between people in the age group with no children and those who have small children and the gender segregation of the labour market. Compared to the start of the period the situation has improved little, if at all. The report indicates that although the programme activities continue, the target levels set for the end of the period are likely not to be achieved. Considering that the gender gap issue is deeply rooted in society and influenced by several factors and the progress of gender equality is a long-term process, the achievement of objectives largely depends on the changing of society's way of thinking. It has been concluded that at the same time one cannot say that the measure Promoting gender equality has not had any positive effect on achieving the set objectives 50. According to AIR 2014, the achievement of the objective new generation of scientists and engineers for knowledge-based society is provided is somewhat defined by an indicator not included in the OP, Number of Estonian scientists and engineers, which has increased strongly in the past few years. Another good impact indicator representative of good higher education and science is the percentage of graduates in the natural sciences, engineering, production and construction in higher education, which has significantly increased in the past few years. AIR 2014 concludes that ensuring competitive income and up-to-date working conditions, as well as promoting science and utilisation of measures of internationalisation, are important prerequisites for the increase in the number of scientists and thus, to the achievement of the general objectives of the R&D and innovation strategy Ibid., p Evaluation of services in the programme Increasing the supply of qualified labour , Centar, Interact, April 2012, p.3-4, 49 Evaluation of services in the programme Increasing the supply of qualified labour , Centar, Interact, April 2012, in Estonian, p (CENTAR%20InterAct)%20final.pdf 50 Annual Implementation Report (monitoring report) reflects the implementation of the OPHRD in with state of , p Ibid., p. 8 39

40 AIR 2014 further points out that general entrepreneurial activity has increased year-byyear. This could be taken as an indication of the achievements of the Priority Axis Knowledge and skills for innovative entrepreneurship. It indicates that the Estonian people have become more entrepreneurial and an increasing number of people have started to realise their potential as an entrepreneur either alongside their day job or instead of it 52. According to the EEN Country Report and recent data from the Ministry of Economic Affairs, at the end of the programming period PA 4 measures created 6,493 new jobs, established 4,153 new companies and the export activities have grown by more than EUR 30 million. On the negative side, it should be stated that out of the 720 enterprises which have received support, 41 enterprises became bankrupt or went into liquidation. However, those who have survived are more innovative than the average enterprises and more active in different cooperation activities with different parties Ibid., p Expert Evaluation Network Country Report Estonia. 40

41 9 Key lessons learned 9.1 What are the key lessons in terms of policy choices? The activities in the Operational Programme for Development of Human Resources for (OP HRD) focused on promoting education, research and development, youth work, labour market, entrepreneurship and improvement of administrative capacity. Social inclusion is regarded as a horizontal priority in the OP and different programmes and measures across the OP focus on social inclusion. As described in section 1.1, the interventions of the OP were generally designed on the basis of actual needs and bottlenecks inherent to the relevant areas in Estonia at the time of programming. As an example, interventions related to improving access to employment for different target groups addressed several commitments and bottlenecks highlighted in the Estonian Action Plans for Growth and Jobs ( ; ): the Estonian labour market is characterized by low youth participation in employment ; there is a need to develop measures to support national labour market services (such as social rehabilitation, career-related psychological counselling, job clubs) and implement them for risk groups (e.g. young people) of the unemployed and job-seekers. The policy choices were also to a large extent related to the commitments and objectives set on the national and EU level (e.g. the Lisbon strategy, EC CSRs). This ensured a good take-up (as shown by the over-achievement of the majority of the output indicators) and good results due to the relevance of the interventions. In 2009, an evaluation of all OPs of Estonia was carried out in order to provide feedback and propose appropriate treatment of the problems in the OPs, to evaluate the need for reviewing the activities and measures and/or diverting funds. The evaluation concluded that the National Strategic Reference Framework s goals are still relevant and the main problems in Estonia have not changed since the formation of the strategy. These include low productivity, low level of knowledge-base and high value added industries in the economy, low ability to export and low level of R&D activities. The proposals about changes in funded measures and priorities, as well as shifts between measures, were presented to the government and monitoring committees. Taking into account the results of the evaluation of the OPs in May 2010, the government, after consulting with different institutions (social partners, MAs and implementation authorities, etc.), approved the changes in the OP 54. In the situation of economic decline (with a significant rise in the unemployment rate), there was a need to extend active labour market measures (PA3) and further promote career services (PA1) 55. One of the lessons learned was that it was very important to increase active labour market measures and measures to increase entrepreneurship and the competitiveness of economy as a reaction to the crisis. This appeared to help Estonia recover more quickly from the economic crisis, restructure the economy within a shorter period of time and should hopefully have positive long-term effects 56. The Operational Programme for Human Resource Development also raises the issue of gender equality. A separate action 57 addressing the promotion of gender equality was designed under the Priority axis 3 of the OP for HRD, focussing on the development of legal norms and regulations of gender equality, increasing the awareness of employees and employers about the issue, and reducing the negative impact of gender stereotypes. However, according to the EU level country report in 2011, Estonia is still at an early stage of embedding the gender equality agenda and gender mainstreaming in the national policy-making. Although gender equality is formulated as a horizontal theme, its integration in the policy cycle has been relatively modest and it has not been fully developed in the majority of the priority axes. Furthermore, the interim evaluation of the 54 Final Country Report: Main ESF achievements in Estonia. Outputs and results of the ESF in the programming period (Metis GmbH, University of Glasgow; 2014) 55 AIR Final Country Report: Main ESF achievements in Estonia. Outputs and results of the ESF in the programming period (Metis GmbH, University of Glasgow; 2014) 57 Measure Promotion of Gender Equality 41

42 measure Gender mainstreaming 58 states that it is critical to explain to employers what benefits they receive from promoting gender equality, to educate educators and to discuss the topic in the system of education from the beginning of preschool. In addition, it is necessary to increase the awareness of society through systematic and continuous dissemination of information on gender equality and discrimination. The support by political measures to the development of gender equality is critical as well. If the state stresses that gender equality is a priority, it fosters understanding of the importance of the topic by employers and employees as well. Aggregation of information and keeping it up to date is important. 9.2 What are the key lessons in terms of target groups? As shown in section 3, the target groups addressed with the OP HRD can, in general, be considered appropriate. As an example, feedback from the target group (received within the PA3 intervention Development of counselling services supporting inclusion to the labour market inventory) on the services provided within the intervention was overall positive. The services offered (e.g. counselling) were regarded as necessary and, as a result, supported the everyday coping of the target group. A lesson is that care should be taken to ensure that the target group itself benefits from the activities and not just the institutions dealing with them. The evaluation on the implementation of ESF measures in the area of general education and youth work reveals that, for instance, the programmes often set students or youth in general as their target group but the direct beneficiaries of the activities are teachers, youth workers, etc. Career services development programme is a positive exception, as a significant part of the resources were directly targeted at youth. The experience gained within the implementation of A2E related activities confirmed that appropriate target groups had been chosen (high participation rate overall). Due to the changes in the labour market caused by the economic crisis, there was a need to include additional target groups, such as short-term unemployed, people at risk of losing their job and those who have received redundancy notice. Furthermore, the majority of target groups (especially young people and elderly, but also e.g. people with special needs) were addressed later in the programming period. In the situation of economic downturn, the target group of people with special needs proved to be especially relevant (counselling was highly valued by the target group). The main lesson learnt in terms of target groups was that instead of approaching one specific group of people it is more important to focus on activities and services that would match specific needs. According to the experience of the Intermediate Body (IB), another lesson learnt was that the open calls targeting only one small-scale sensitive target groups (e.g. victims of human trafficking) will likely fail and the most effective activities have proven to cover several target groups at the same time. 9.3 What are the key lessons in terms of the appropriate programming? The ex-ante evaluation of the OPs highlighted that while the structure of the programming system can be considered as a strength in the way that all the priority axes contribute to the thematic objectives (and the budget is divided accordingly), it is also a weakness of the system, because it inhibits achieving greater impacts by giving more focus to a small number of prioritised objectives. The evaluation process has underlined the inappropriate nature of the indicators, target setting and monitoring system with respect to the assessment of the contribution of Structural Funds measures to broader strategic goals. The large number of measures led to an excessive administrative burden. According to the Intermediate Body (IB) this resulted in a lack of understanding of the potential for synergies between different measures. Another aspect is the uncertainty for the applicants to identify for which 58 Interim evaluation of the measure Gender mainstreaming (University of Tartu, 2013), Executive summary, p. 3, 42

43 measures and institutions to apply for funds, as the financing decisions of, for example, infrastructure investments and scientific needs are made separately. More attention should be paid to the investment into human resources. 59 The implementation of projects has slowed down due to a number of factors related to the bureaucratic regulation of funding and modest support from the 2nd level IB. A lesson is that there is a need to increase support to applicants during the application process and assist participants in resolving more complex procedural issues. Over-precise rules on eligible activities limit the applicants flexibility in designing and planning the most appropriate actions. The slow process of refunding participant costs is penalising smaller organisations with limited financial liquidity and hence, inadvertently, limits participation in the programme to the usual suspects. There is also insufficient attention given to complementary efforts to optimise the return on investment. The existing fragmented financing schemes do not guarantee the coherent and sustainable financing of the infrastructure and science capacities. 60 Several lessons can be drawn from the Interim evaluation of Structural Funds for the period , for example: More result indicators measuring the long-term effects of the implemented activities should be used, which would provide a good input in planning the activities of subsequent periods and allow the assessment of which supported activities are more sustainable. Additional training should be provided before the beginning of the period in order to ensure that all stakeholders (e.g. intermediate bodies) are capable of performing their work as efficiently as possible and commence the implementation of activities at the time planned. Stakeholders interested in the measures should be involved as much as possible in the preparation of the regulations for the measures. In fields where local authorities are the beneficiaries, it is necessary to involve the county level administrations more in the decision-making process. Due to the impact of the economic crisis, there was a need to make some changes within the OP priorities in the middle of the programming period. Although no activities were cancelled or added, there was for example a need to enlarge the range of target groups. Also, application requirements for certain open calls for proposals needed to be adjusted and some budget reallocations between activities had to be performed. Therefore, it is important that a certain extent of flexibility (e.g. changing indicator target levels, the composition of the target group and the activities) would be allowed when implementing the interventions in order to respond to unforeseen circumstances that may appear and influence the effectiveness of interventions. One of the main lessons learned from the implementation of the interventions related to improving access to employment in regarding the appropriate programming, is connected with the structuring of activities under the measure level (several target groups received the same universal types of services). It appeared that the target-based approach is not always the best choice from the perspective of improving the target group s situation in the labour market. It is more important to approach target groups with tailor-made services in order to meet the target group s specific needs. 9.4 What are the key lessons in terms of the effective implementation? According to the AIR 2014, the majority of the objectives are moving towards the predicted levels or have exceeded them or the possibility of exceeding most of the targets by the end of the period was deemed likely. It must be noted that the reference and target level estimates of several OP HRD indicators have been conservative, especially considering the relatively large volume of financing which shows that the true 59 ESF Expert Evaluation Network - Country Report Estonia (2014), p ESF Expert Evaluation Network - Country Report Estonia (2014), p Interim Evaluation of Structural Funds for the Period (CPD and University of Tartu, 2011). 43

44 implementation situation or specific volumes of activities could not be accurately predicted. The same problem has also affected significant under-achievement, which is also often affected by unpredictable external factors (e.g. the economic crisis). Also, it can be said that not all indicators accurately reflect the achievement of the actual objectives of the measures. Achievement of the objectives of the whole OP is significantly influenced by PA 4 Knowledge and skills for innovative entrepreneurship where achievement of reference levels has been most difficult and the fulfilment level of several indicators is unknown. This PA has the highest number of indicators that will most probably not be fulfilled by the end of the period. All OP HRD measures had been started by the end of the reporting period; at the same time the financial progress as well as the productivity between priority axes and measures still varies considerably. The PAs Developing human resource for Research & Development, Lifelong learning and Enhancing Administrative Capacity have done especially well in fulfilling the indicators. The most successful PA in terms of financial progress has been the PA Good-quality and long working life. It can be explained by the fact that in the most sizeable measure Increasing the supply of qualified labour, 94% of the payments have been made which can be attributed to the crisis with the increased need for these measures. Five very successful open application rounds have taken place. The lesson here might be that the OP should be flexible enough to respond adequately to the changing needs. The changes introduced during the years of economic crisis were more oriented to the future development of Estonia and its economy. New measures and changes in the priorities were in accordance with the general development of economy. Some of the changes broadening the target groups of different active labour market measures, simplified conditions for application of start-up and micro loans programme, development grants, etc. aimed to fulfil the priorities set out in the Estonian Strategy for Competitiveness and in this sense they were more general. Due to the accelerated spending more people could participate in different measures, especially in active labour market measures and in different training/retraining measures. These changes were partly done as a response to the crisis and reached the expected outcome. Furthermore, without the additional funding different measures would not have been developed and the mitigation of the crisis would not have been possible 62. One of the most important lessons was the importance of increasing a number of active labour market measures and measures which increase the entrepreneurship and competitiveness of economy. This hopefully has a positive effect also in the long-run and helps to restructure the economy within a shorter period of time. 9.5 What are the key lessons in terms monitoring? To track the level of attainment of the OP s objectives, annual monitoring reports were prepared for each separate PA as well as for each OP. The monitoring reports provide an overview of the success in attaining the objectives measured through the indicator achievement rate, and list the progress and obstacles in the implementation of measures. The Mid-term Evaluation of Structural Funds during the Programming Period (2011) reveals that the quality of information gathered through monitoring was uneven at all levels. The structural funds administration survey and the interviews showed that the coordination of the preparation of monitoring reports at the OP and priority levels was not efficient enough and not properly planned. The evaluation also indicated shortages in the strategic view at the national level. With a more elaborated strategic planning and systematic monitoring it could be possible to increase the effectiveness, 62 Final Country Report: Main ESF achievements in Estonia. Outputs and results of the ESF in the programming period (Metis GmbH, University of Glasgow; 2014) 44

45 efficiency and sustainable allocation of resources. Existing indicators do not sufficiently allow the assessment of the movements towards stated goals 63. Elaboration of appropriate indicators proved to be rather difficult according to the project promoters and the IA. In some cases the target levels had to be changed during implementation and several interventions over performed, largely due to the economic crisis, and not necessarily having comprehensive explanations in the final reports. The Mid-term review also suggests that there has been excessive focus on output indicators. More result indicators measuring the long-term effects of the implemented activities should be used, which would provide a good input to planning the activities of subsequent periods and allow the assessment of which activities are more sustainable. The substantial amount of data required for monitoring was regarded as problematic to secure from the participants. The data received from persons benefitting from the services was in some cases incomplete, due to the unwillingness of the participants to submit the necessary data (for example, for the more vulnerable target groups, such as the disabled). Another highlighted point was the lack of a system to provide data enquiries from a public institution that would guarantee the reliability of the data. During the new programming period ( ), it is foreseen that Statistics Estonia will provide support in the field of data collection and management. 9.6 What are the key lessons in terms of evaluation? During the programming period, the Ministry of Finance, as the Managing Authority, was responsible for the coordination of evaluations. The implementation of evaluations concerning OP HRD, the PAs or measures (actions) within it, was the responsibility of the 1st level Intermediate Bodies (primarily ministries). The evaluations were mostly carried out by external evaluators and a large part of the assessments were implemented at measure level (reflecting certain areas of the measure). Today there are six impact evaluations on active labour market policies available in Estonia. The existing evaluation studies have focused on evaluating the impact of labour market training, wage subsidies, work practice and business-start-up grants. The Evaluation report for Interim Evaluation of Structural Funds for the period indicates a number of bottlenecks in the indicator system, e.g. the indicators are only weakly linked to the content of the measures. Also, too much focus was on output indicators, which do not provide sufficient information about the progress towards goals. The focus should be on result and impact indicators, which are the key to evaluating long-term effects of the implemented activities and would provide good input to planning the activities of subsequent periods and allow the assessment of sustainability of the supported activities. Also, the evaluation of the effectiveness and impact of the PA 5 Raising Estonian administrative capacity in the OP HRD in 2011 indicated that some target levels selected were overly ambitious and should be planned on a more realistic level. 63 Final Country Report: Main ESF achievements in Estonia. Outputs and results of the ESF in the programming period (Metis GmbH, University of Glasgow; 2014) 45

46 10 Annexes Table 7. Financial allocation by Priority Axes and policy field OP Priority Financial Allocation Increasing adaptability and enhancing human capital Corresponding public contribution Implementation rate (in %) OP HRD 1 Lifelong learning 68,679,947 60,987, OP HRD 2 Developing the human resources for R&D 117,598,503 76,401, OP HRD 4 Knowledge and skills for innovative entrepreneurship 85,322,051 74,259, Overall, EUR 271,600,501 (=59%) were allocated to this policy field, of which EUR 211,648,114 (=80%) were spent. Enhancing access and sustainable integration into employment OP HRD 3 Good-quality and long working life 155,718, ,201, Overall, EUR 155,718,852 (=34%) were allocated to this policy field, of which EUR 145,201,315 (=93%) were spent. Strengthening institutional capacity OP HRD 5 Enhancing administrative capacity 24,465,675 21,210, Overall, EUR 24,465,675 (=5%) were allocated to this policy field, of which EUR 21,210,585 (=87%) were spent. Technical Assistance OP HRD 6 Horizontal technical assistance 7 Technical assistance 9,161, ,011 6,896, , Total OP Human Resources Development 461,337, ,321, Source: SFC

47 Table 8. Data on participants of the OP PA 1 PA 2 PA 3 PA 4 PA 5 PA 6 Total Share of women (in%) Total number of participants 596, , ,841 37,216 43,434 5,987 1,066, Employed 37.6% 65.3% 18.8% 98.6% 100.0% 100.0% 44.2% 70.6 of which self-employed 1.7% 0.8% 3.8% 0.8% 0.0% 0.0% 1.4% 56.7 Unemployed 2.0% 0.1% 76.0% 0.5% 0.0% 0.0% 14.7% 53.6 of which LTU 11.5% 0.9% 48.5% 8.4% % 53.5 Inactive 60.4% 34.7% 5.2% 0.9% 0.0% 0.0% 41.0% 55.8 of which in education/training 94.2% 89.9% 30.2% 54.0% % 55.5 Young people (15-24 years) 55.9% 35.4% 16.5% 7.7% 5.8% 2.2% 41.1% 56.1 Older people (55-64 years) 7.9% 12.0% 11.5% 5.4% 14.7% 9.5% 9.5% 69.9 Migrants 0.4% 0.6% 0.6% 0.0% 0.0% 0.1% 0.4% 67.6 Minorities 6.5% 3.5% 30.8% 0.3% 0.0% 0.9% 9.8% 55.9 Disabled 0.4% 0.3% 8.7% 0.0% 0.0% 0.0% 1.9% 55.2 Others 1.2% 0.0% 16.8% 0.0% 0.0% 0.0% 3.7% 60.1 Primary or lower secondary education (ISCEC 1 and 2) 55.1% 6.6% 19.0% 2.3% 1.5% 0.0% 35.6% 53.5 Upper secondary education (ISCED 3) 11.0% 9.6% 32.6% 20.1% 15.5% 1.6% 15.1% 55.0 Post-secondary non tertiary education (ISCED 4) 5.7% 13.0% 21.3% 16.9% 13.5% 1.1% 10.5% 66.3 Tertiary education (ISCED 5 and 6) 26.7% 70.7% 27.1% 60.7% 69.5% 97.1% 38.0% 71.5 Source: SCF

48 Table 9. Output indicators by policy field Type Indicator Final Target Implementation rate Increasing adaptability and enhancing human capital Output Number of VET centres and non-formal adult education centres providing training for adults % Output Number of professional qualifications worked out according to modernised requirements % Output Number of trainings for youth workers % Output Number of trainings for non-ethnic Estonians % Output Output Percentage of teachers and VET teachers from the institutions providing vocational education participating in trainings Number of materials which meet the needs of national curriculum developed for SEN pupils under the OP % % Output Number of teachers who participated in trainings 1,000 13, % Output Number of trained advisers % Output Number of pedagogues trained on the ICT area 12,000 13, % Output Number of ESF co-financed long-term grants for high-level international lecturers and scientists in prioritised fields % Output Number of ESF financed PhD study vacancies % Output The number of undergraduate students admitted to continuing education in the higher education level , % Output The number of graduate students admitted to continuing education in the higher education level % Output Number of ESF financed foreign PhD grants in Estonian universities % Output Number of lecturers participating in short-term complementary training co-financed by ESF 400 1, % Output E-courses of higher education level financed by ESF 3,000 3, % 48

49 Type Indicator Final Target Implementation rate Output Percentage of enterprises that have received training grant out of potential beneficiaries (selfemployed entrepreneurs not included) % Output Number of enterprises that have received mobility support for hiring highly qualified personnel % Output Number of enterprises per year who have received consulting support % Output Number of start-ups supported per year % Output Number of education employees per year who have participated in entrepreneurship education programme. Access and sustainable integration to Employment % Output Number of people participating in active labour market measures 91, , % Output Number of persons receiving services supporting finding work 15,000 42, % Output The number of new work approaches including health and counselling services that prevent fall out % Output The number of information campaigns raising health awareness to prevent employment loss % Output The number of recipients that for health reasons received training to prevent dropout of employment 4,000 8, % Output Number of employees of Labour Inspectorate taking part in training % Output Number of people taking part in working environment training or information days 7,550 16, % Output Organisation of publicity campaigns contributing to the improvement of the quality of working life % Output Carrying out surveys that contributes to improve the quality of working life % Strengthening institutional capacity Output Number of financed studies and analyses % Output Number of participants who participated in the training 11,900 25, % Output Number of NGOs (incl. social partners) employees trained. 2,100 11, % Source: SCF

50 Table 10. Result indicators by policy field Type Indicator Increasing adaptability and enhancing human capital Final Target Implementation rate Result Number of adults successfully completed the trainings 73,000 78, % Result Number of people whose qualifications are updated by modernised requirements (number of new qualification certificates) 30,000 77, % Result Number of youth workers who successfully finished trainings 4,000 6, % Result Number of non-ethnic Estonians who completed the trainings 3,750 4, % Result Result Proportion of trained VET teachers who evaluate passed training as necessary and useful Proportion of risk group students who does not attend school, but study on basis of individual curriculum % % Result % of pedagogues who correspond to demands of qualifications % Result Result Result Result Proportion of schools which systematically carry out internal evaluation activities Proportion of teachers using actively ICT in their lessons, train and support other teachers in using ICT and create study materials electronically Number of MSc students supervised by high-level international lecturer and scientist grant-holders Number of PhD students supervised by high-level international lecturer and scientist grant-holders % % % % Result Share of ESF financed positively evaluated PhD students filling the vacancies % Result Result Result Effectiveness of continuing education the ratio of students who completed their studies in continuing education to the number of students admitted. Undergraduate and graduate studies Effectiveness of continuing education the ratio of students who completed their studies in continuing education to the number of students admitted. Doctoral studies Share of evaluated positively foreign PhD students filling the vacancies financed by ESF foreign PhD grants % % % Result aggregation other positive result qualification other positive result other positive result other positive result other positive result 50

51 Type Indicator Final Target Implementation rate Result Share of participants that successfully complete complementary training % Result Result Number of participants that successfully complete e-courses of higher education level Proportion of enterprises that have trained at least half their workers within the last 12 months 45,000 45, % % Result Proportion of enterprises that have trained a worker during the last 12 months % Result Result Result Growth of innovation expenditure in enterprises that have received mobility support for hiring highly qualified personnel Proportion of enterprises that have used outside consultancy services during the last 12 months Proportion of enterprises whose managers have participated in longer-term management training % % % Result Rate of survival amongst new enterprises that had received support % Result Result Result Proportion of enterprises saying that they have no problem finding enterprise related information Number of pupils who have received enterprise teaching (classes 1-12 in Estonian education system) Proportion of study programmes where enterprise training is taught (universities and vocational education schools) Access and sustainable integration to Employment Result Result Result Result Result Share of people starting work after participation in active labour market measures, as a proportion of all those who participated in active measures Share of people starting work or who kept their jobs after receiving services supporting finding work, as a proportion of all those who received the relevant services Share of people referred to active labour market services with the help of services supporting finding work, as a proportion of all those who received the relevant services The share of total municipal governments who have prepared a local health profile, as a proportion of all municipal governments The number of enterprises and institutions who have joined the network of promoting health jobs % % % % % % % % Result aggregation other positive result qualification employment employment entities 51

52 Type Result Result Indicator The average grades received at the assessment of the civil servants competencies at the Labour Inspectorate Share of employees working in companies of which specialist has completed training or attended the thematic day organized by Labour Inspectorate Strengthening institutional capacity Result Number of organisations (or organisation groups in the same field or hierarchy) that have changed their management systems or procedures Final Target Implementation rate % % % Result Number of simplified and codified legal fields % Result aggregation Result Number of NGOs per year who have received advice 1,500 1, % entities Result Personnel turnover % Result Number of developed training programs % product Source: SCF 2007 entities 52

53 Table 11. Impact indicators by Priority Type Indicator Priority Impact Impact Impact Impact Participation rate of the adult population (25-64 year olds) in lifelong learning Participation rate of the adult non-ethnic Estonian population (25-64 year olds) in lifelong learning Proportion of young people in the year age group with a low level of education and who do not participate in trainings Proportion of young people with at least secondary education in the year old age group Initial Baseline Final Target 2014 Achievement Implementation rate % % % % Impact Activity rate % Impact Impact Employment gap between childless people and parents of young children in the age group Employment gap between childless people and parents of young children in the age group % % Impact Inactivity due to illness or injury - age group ,400 13,000 15, % Impact Gender segregation in the labour market (or the segregation index) % Impact Effectiveness of work in the government sector % Impact Impact Impact Source: SCF Professionalism in the public service Source: annual study by AS Emor, commissioned by the State Chancellery Trustworthiness in the public service Source: Index of dedication in the public service Source: % % % 53

54 Table 12. Inventory of evaluations Title of evaluation Author Type of evaluation Date Scope Evaluation of services in the programme Increasing the supply of qualified labour , Evaluation of the European Social Fund s support to Gender Equality, 2011: Country report, Estonia. Interim Evaluation of Structural Funds for the Period Mid-term evaluation of the implementation of measures in favour of R&D and higher education in the framework of the EU co-financed Structural Funds during the period Assessment of program Welfare measures supporting employment The evaluation of the effectiveness and the impact of the PA 5 Raising Estonian administrative capacity Centar, Interact, Process and impact evaluation including counterfactual approach 2012 The evaluation focuses on two labour market services: wage subsidies and labour market training. In the report description of the services is given and relevance, effectiveness, sustainability and efficiency of services are discussed. FGB, GHK Process evaluation 2011 The evaluation focusses on the following topics: suitability of the activities supported under the measure for the objective and subobjectives of the measure; suitability of the indicators used; effectiveness of the activities conducted; levels of achievement of the indicators; factors affecting performance of the activities; efficiency of the performance of the activities; sustainability of the results achieved CPD and University of Tartu Institute of Baltic Studies, Praxis Center for Policy Studies, Technopolis group University of Tartu and Ernst & Young CPD Process evaluation 2011 The goal of the evaluation was to get analytical support for correction of plans within the current programming period and for the preparation to the next programming period. Process evaluation 2011 The goal is to assess the relevance, applicability and efficiency of the implementation of measures in favour of R&D and higher education. N/A 2010 The evaluation analysed following activities: counselling services provided by Counselling Centres in Pärnu and Jõhvi, counselling services provided by Center for Disability related Information and Technical Aids, training of the disabled child minders Process and impact evaluation including counterfactual approach 2011 Assessment to the activities: how much they have contributed to the achievement of goals set up in the OP of HRD and priority axes and how these activities have influenced the administrative capacity in Estonia. 54

55 Title of evaluation Author Type of evaluation Date Scope The Evaluation of the Business Start-up Subsidy, Work Practice and Coaching for Working Life Praxis Center for Policy Studies, Process and impact evaluation including counterfactual approach 2010 The evaluation of three labour market services. As the program is still running and the time period for achieving results has been short, the evaluation focused on the aspects of implementation and efficiency 55

56 Table 13. CSRs by theme Year Human Capital ESF contribution Access to Employment ESF contribution Social Inclusion ESF contribution 2004 Provide greater access to training for the unemployed Reduce levels of inactivity and raise further the participation of women, older workers and the low-skilled 1) Strengthen active labour market measures, provide greater access to training for the unemployed and ensure that the labour market becomes more inclusive, particularly with regard to disadvantaged people, such as the longterm unemployed, young people, persons with disabilities, older jobseekers, those belonging to an ethnic minority; Increase use of flexible forms of work Raise the participation of long-term unemployed, young people, persons with disabilities, older jobseekers; those belonging to an ethnic minority and women. Increase access to training for the unemployed, active labour market policies. 2) Reduce the tax wedge on labour, especially on lower wage earners, and promote contractual and working time diversity; improve the tax systems and transform undeclared work into regular jobs. It is also important that wage trends remain in line with productivity developments; ) Reduce levels of inactivity and raise further the participation of women, older workers and the low-skilled No formal CSRs, But call for raising the skills level of the labour force further by reinforcing the reforms in education and lifelong learning - 1) Focus on improving labour market flexibility through further progress in the renewal of labour laws; 2) Reinforcing active labour market policies; (PA3 involves analysis of legal acts) Increase use of flexible forms of work; modernise the labour law. Raise the skills level of the labour force by reforms in the education and LLL. Reinforcing ALMP No formal CSRs, but call reinforcing active labour market policies and 3) Raising the skills level of the labour force further by reinforcing the reforms in education and lifelong learning. - 1) Focus on reinforcing active labour market policies; - (this issue is addressed by PA1) Promotion of flexible forms of work; need for the modernisation of the 56

57 Year Human Capital ESF contribution Access to Employment ESF contribution Social Inclusion ESF contribution increasing the supply of skilled labour by implementing a comprehensive lifelong learning strategy that responds to labour market needs 2) Increasing the supply of skilled labour by implementing a comprehensive lifelong learning strategy that responds to labour market needs; - (lifelong learning is addressed by PA1) labour law. Raise the supply of skilled work force by improving LLL opportunities including active labour market policies Speed up the implementation of the new labour law package ) Reducing labour market rigidities by urgent progress towards labour law modernisation and by promoting flexible forms of work. 1) Speed up the implementation of the new labour law package; 2) Increase the efficiency of public employment services, in particular by well targeted active labour market policies aimed at facilitating labour market transition. - Labour market rigidness and high longterm unemployment should be dealt with through speeding up the implementation of the new labour law package and increasing the efficiency of public employment services. Further raise the supply of skilled work force by improving LLL opportunities including active labour market policies 2011 While implementing the education system reform, give priority to measures improving the availability of preschool education Enhance the quality and availability of professional education. Focus education outcomes more on labour market needs, and provide opportunities for low-skilled workers to take part in lifelong learning Link training and education more effectively to the needs of the labour 1) Take steps to support labour demand and to reduce the risk of poverty by reducing the tax and social security burden in a budgetary neutral way; 2) Take steps to support labour demand and to reduce the risk of poverty through improving the effectiveness of active labour market policies, including by targeting measures on young people and the long-term unemployed, especially in areas of high unemployment; 3) Focus education outcomes more on labour market needs, and provide opportunities for low-skilled workers to take part in lifelong learning. 1) Improve incentives to work by streamlining the social benefits system - - (this issue is addressed by PA1) Focus on reducing the unemployment of young people and long-term unemployed. Reduce risk of poverty by relieving tax and social security burden. Improve human capital. Focus education outcomes more on labour market needs, and provide opportunities for low- skilled workers to take part in lifelong learning. Improve ALMP effectiveness. - Increase the participation of the young and the long-term unemployed in the 57

58 Year Human Capital ESF contribution Access to Employment ESF contribution Social Inclusion ESF contribution market Enhance cooperation between businesses and academia. Increase opportunities for low-skilled workers to improve their access to lifelong learning. Foster prioritisation and internationalisation of the research and innovation systems Continue efforts to improve the labourmarket relevance of education and training systems, including further involvement of social partners and implementation of targeted measures to address youth unemployment. Significantly increase the participation of low-skilled workers in lifelong learning. Intensify efforts to prioritise and internationalise the research and innovation systems Enhance cooperation between businesses, higher education and research institutions. and increasing flexibility in the allocation of disability, unemployment and parental benefits, while ensuring adequate social protection; 2) Increase the participation of the young and the long-term unemployed in the labour market; 3) Link training and education more effectively to the needs of the labour market. 1) Strengthen activation measures to facilitate the return to the labour market of the long-term unemployed and people receiving disability benefits and incapacity-for-work benefits; 2) Establish coordinated measures for fostering economic development in regions affected by high unemployment; 3) Continue efforts to improve the labour-market relevance of education and training systems, including further involvement of social partners and implementation of targeted measures to address youth unemployment; 4) Significantly increase the participation of low-skilled workers in lifelong learning. (PA3 addresses only the training aspect) - (PA3 addresses only the training aspect) - (this issue is addressed by PA1) labour market. Improve incentives to work by streamlining the social benefits system and increasing flexibility in the allocation of disability, unemployment and parental benefits. Improve delivery of social services, while better targeting family and parental benefits and removing distortionary income tax exemptions related to children. Link training and education more effectively to the needs of the labour market, Increase opportunities for low-skilled workers to improve their access to lifelong learning Improve incentives to work through measures targeted at low income earners. Target activation efforts by ensuring the timely adoption and implementation of the work capacity reform. Increase the efficiency and costeffectiveness of family policy while improving the availability and accessibility of childcare. Deploy coordinated measures for fostering economic development and entrepreneurship in regions faced with high unemployment. Ensure the labour market relevance of education and training systems, improve skills and qualification levels by expanding lifelong learning measures and systematically increasing participation in vocational education and training, including in apprenticeships. 58

59 Table 14. Mapping of Strengthening institutional capacity OP/PA/Action Main dimension(s) of the intervention (see explanatory appendix) Indicate the applicable dimension(s) - Structures and processes - Human resources - Service delivery - Judiciary - Social partners - NGOs - Other, please specify Type of support (see explanatory appendix) Indicate the applicable category(ies) of support and indicate their relative importance: - judicial -e-governance - territorial planning - PES - education - other, please specify Beneficiaries List the beneficiaries distinguishing between the following three categories: - public entities - social partners - NGOs Please also indicate their relative importance. Level of intervention Indicate the level(s) of intervention and their relative importance: - local - regional - national Operational Programme for Human Resource Development Priority Axis 5 - Enhancing administrative capacity Measure Enhancement of strategic management of the public sector and NGOs - Structures and processes (process dimension) - Human resources (training) - Social partners - NGOs - Service delivery - Governance (incl. local and municipal) (territorial planning) - NGOs and social partners capacities - General administration support - Other Capacity building outside the administration (researchers from universities) - public entities - social partners - NGOs The main beneficiaries are ministries, local governments, social partners and NGOs, but also universities - national Measure Better regulation - Judiciary - Structures and processes - Human resources - Judicial (system improvement through codification and development of support structures; e.g. development of common guidelines and the system of the impact analysis) - General administration - public entities - social partners - NGOs The main beneficiaries are ministries, but also to a certain extent NGOs and - national 59

60 OP/PA/Action Main dimension(s) of the intervention (see explanatory appendix) Indicate the applicable dimension(s) - Structures and processes - Human resources - Service delivery - Judiciary - Social partners - NGOs - Other, please specify Type of support (see explanatory appendix) Indicate the applicable category(ies) of support and indicate their relative importance: - judicial -e-governance - territorial planning - PES - education - other, please specify Beneficiaries List the beneficiaries distinguishing between the following three categories: - public entities - social partners - NGOs Please also indicate their relative importance. Level of intervention Indicate the level(s) of intervention and their relative importance: - local - regional - national support social partners Measure Training and development of employees of the State, local authorities and NGOs - Structures and processes (process dimension) - Human resources - Service delivery - Social partners - NGOs - NGOs and social partners capacities - Local and municipal governance (territorial planning) - General administration support - public entities - social partners - NGOs The main beneficiaries are ministries, local governments, social partners and NGOs - national Measure Supporting countylevel support structures - Human resources - Social partners - NGOs - NGOs and social partners capacities - social partners - NGOs The main beneficiaries are social partners, NGOs and county development centres - national - local (county level) 60

61 HOW TO OBTAIN EU PUBLICATIONS Free publications: one copy: via EU Bookshop ( more than one copy or posters/maps: from the European Union s representations ( from the delegations in non-eu countries ( by contacting the Europe Direct service ( or calling (freephone number from anywhere in the EU) (*). (*) The information given is free, as are most calls (though some operators, phone boxes or hotels may charge you). Priced publications: via EU Bookshop ( Priced subscriptions: via one of the sales agents of the Publications Office of the European Union (

62 doi: /47459 KE EN-N