Armenia WORKFORCE DEVELOPMENT. SABER Country Report 2014

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Armenia WORKFORCE DEVELOPMENT Dimensions 1. Strategic Framework The strategic framework is assessed to be at a high emerging level. The political support for workforce development (WfD) as an asset for economic progress is strong; the challenges of aligning the workforce development system to the needs of the economy are recognized and addressed in the country s strategic documents. However, the roles and responsibilities of key stakeholders in WfD remain fragmented and the influence of businesses and industries in shaping and implementing WfD priorities is modest. 2. System Oversight System oversight is assessed to be at the emerging level. Competency based testing and certification await implementation, arrangements for institutional accreditation are still being formulated, and measures for articulation among training programs have to be further improved. Funding is also an issue: Government funding is modest; there is no strong linkage between the allocation of funds to institutions and institutions performance, and the funding of educational institutions is generally not supplemented by other sources. 3. Service Delivery Service delivery is assessed to be at the emerging level, suggesting that although formally all stakeholders have influence over training curricula, in a large number of cases, non state stakeholders involvement is limited. Private training providers are allowed to operate within the existing legal framework quite freely, but they have almost no incentives to meet quality standards, and a culture of monitoring and evaluation is not well developed. SABER Country Report Status 2014

2 Table of Contents Executive Summary Introduction Country Context... 7 Economic Trends... 7 Employment... 7 Supply of Skills Overview of Findings and Implications Overview of the SABER WfD Scores Policy Implications of Findings Aligning Workforce Development to Key Economic and Social Priorities Strategic Socioeconomic Aspirations and Priorities SABER WfD Ratings on the Strategic Framework Policy Goal 1: Articulating a Strategic Direction for WfD Policy Goal 2: Articulating a Strategic Direction for WfD Policy Goal 3: Strengthening Critical Coordination for Implementation Implications of the Findings Governing the System for Workforce Development Overall Institutional Landscape SABER WfD Ratings on System Oversight Policy Goal 4: Ensuring Efficiency and Equity in Funding Policy Goal 5: Assuring Relevant and Reliable Standards Policy Goal 6: Diversifying Pathways for Skills Acquisition Implications of the Findings Managing Service Delivery Overview of the delivery of training services SABER WfD Ratings on Service Delivery Policy Goal 7: Incentivizing Diversity and Excellence in Training Provision Policy Goal 8: Fostering Relevance in Public Training Programs Policy Goal 9: Enhancing Accountability for Results Implications of the Findings Annex 1: Acronyms Annex 2: The SABER WfD Analytical Framework Annex 3: Rubrics for Scoring the SABER WfD Data Annex 4: References and Additional Sources Annex 5: SABER WfD Scores Annex 6: Validation Workshop Report Annex 7: Authorship and Acknowledgements SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 2

3 Executive Summary Armenia is a small, open economy country that is strategically located in the South Caucasus, at the crossroads of Western Asia and Eastern Europe. Since 2004, economic growth has meant that Armenia has crossed the threshold from a low to middle income country. In 2012, the GNI per capita in PPP terms (constant 2005 international dollars) was 5,540 and the HDI value was registering in the high human development category positioning the country to rank 87 th out of 187 countries and territories. Between 1990 and 2012, Armenia s HDI value increased from to 0.729, an increase of 16 percent or average annual increase of about 0.7 percent. Like many countries worldwide, Armenia has made considerable efforts to reinforce its education and training systems capacity to respond to the pressures of globalization and the challenges of a knowledge based economy. For countries like Armenia that have few natural resources, the most important resource for economic development is human capital in the form of a well educated, creative labor force. The only way to secure this capital is by creating a globally competitive education system, which is the goal of wide ranging reforms. 1 The achievement of this goal requires strong leadership at all levels of the workforce development (WfD) system, consistency in setting and implementing long term national goals, and coherence in policies targeted at decreasing the gaps between skills demand and supply. This report presents a comprehensive diagnostic of the country s WfD policies and institutions to inform policy dialogue on this important issue. The results are based on a new World Bank tool designed for this purpose. Known as SABER WfD, the tool is part of the World Bank s initiative on Systems Approach for Better Education Results (SABER), 2 the aim of which is to provide systematic documentation and assessment of the policy and institutional factors that influence the performance of education and training systems. It examines three crucial dimensions of the WfD system: strategic framework, system oversight, and service delivery. The SABER WfD tool encompasses initial, continuing, and targeted vocational education and training that are offered through multiple channels and focuses largely on programs at the secondary and post secondary levels. This report documents the results of the use of the SABER WfD tool in Armenia for the first time. Leaders play an important role in crystallizing a strategic vision for WfD that is appropriate given a country s unique circumstances and opportunities, and in Armenia there are some visible champions advocating for WfD. It is well recognized that effective advocacy for WfD requires credible assessments of the demand for skills, engagement of employers in shaping the WfD agenda, and the provision of incentives for employers to support skills development. To align the efforts of multiple stakeholders involved in WfD with the country s key socioeconomic priorities, in Armenia the roles and responsibilities of all government and non government stakeholders are formally defined by policy along with corresponding institutional mechanisms for coordinating government entities and non government WfD stakeholders. Despite this effort, however, coherence is lacking and communication and interaction remain weak. There is a disconnect between the long term strategic goals and current vision for WfD, on the one hand, and the industry involvement in setting strategic priorities for WfD, on the other. Moreover, the measures undertaken by the Government of the Republic of Armenia to provide means for employers to be engaged in the strategy making process to address skills constraints, even in priority sectors, are limited. 1 Economy and Values (EV) Research Center and EV Consulting, National Competitiveness Report of Armenia 2010: Higher Education Report (Yerevan: 2010), p For details on SABER, see SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 3

4 The efficient and equitable funding of investments in WfD is important. Although the government allocates funds from the state budget according to existing procedures, the current principle of funding does not promote efficiency in resource use, and private sector involvement in providing resources for training is very limited. In general, the linkages in Armenia between the education and training systems and the labor market are weak. There is a clear mismatch between the type and level of skills needed by the labor market and what is being supplied by education and training. A key trend is that while the WfD system is producing an increasingly highly educated workforce, graduates do not possess the specific skills required by the labor market, which are often lower level skills and attainable through shorter periods of education. Therefore, it is important to continue with the system reforms that aim to increase the quality of professional education and to bring it into compliance with labor market demands as well as the strategic educational and development objectives of the country. In order to keep training program offerings relevant to current market conditions, training institutions need reliable information on current and emerging skills needs. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 4

5 1. Introduction Strengthening microeconomic fundamentals, broadening job opportunities, and enabling workforce development (WfD) are all viewed as important means for improving Armenia s socioeconomic prospects. To inform policy dialogue on the important issue of WfD, this report attempts to present a comprehensive diagnostic of the country s WfD policies and institutions. The results are based on a new World Bank tool designed for this purpose. Known as SABER WfD, the tool is part of the World Bank s initiative on Systems Approach for Better Education Results (SABER), 3 the aim of which is to provide systematic documentation and assessment of the policy and institutional factors that influence the performance of education and training systems. The SABER WfD tool encompasses initial, continuing, and targeted vocational education and training that are offered through multiple channels, and focuses largely on programs at the secondary and post secondary levels. The SABER WfD tool was used in Armenia for the first time. The tool is based on an analytical framework that identifies three functional dimensions of WfD policies and institutions: (1) Strategic framework, which refers to the praxis of advocacy, partnership, and coordination in relation to the objective of aligning WfD in critical areas to priorities for national development; (2) System oversight, which refers to the arrangements governing funding, quality assurance, and learning pathways that shape the incentives and information signals affecting the choices of individuals, employers, training providers, and other stakeholders; and (3) Service delivery, which refers to the diversity, organization, and management of training provision, both state and non state, that delivers results on the ground by enabling individuals to acquire market and job relevant skills. Taken together, examining these three dimensions allows for a systematic analysis of the functioning of a WfD system as a whole. The focus in the SABER WfD framework is on the institutional structures and practices of public policymaking and what they reveal about the system s capacity to conceptualize, design, coordinate, and implement policies in order to achieve results on the ground. Each dimension is composed of three policy goals that correspond to important functional aspects of WfD systems (see Figure 1). Policy goals are further broken down into discrete policy actions and topics that reveal more detail about the system. Figure 1. Functional Dimensions and Policy Goals in the SABER WfD Framework Source: Tan et al Information for the analysis is gathered using a structured SABER WfD Data Collection Instrument, which was designed to collect, to the extent possible, facts rather than opinions about WfD policies and institutions. The Data Collection Instrument defines three functional dimensions, nine policy goals, and 56 system topics. For each topic, it poses a set of multiple choice questions that are answered based on documentary evidence and interviews with knowledgeable informants. The answers allow each topic to be scored on a four point scale against standardized rubrics based on available knowledge on global good practice (See Figure 2). Topic scores are averaged to produce Policy Goal scores, which are then aggregated into Dimension scores. The results are 3 For details on SABER, see SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 5

6 finalized following validation by the relevant national counterparts, including the informants themselves. Figure 2. SABER WfD Scoring Rubrics Source: Tan et al The rest of this report summarizes the key findings of the SABER WfD assessment and also presents the detailed results for each of the three functional dimensions. To put the results into context, the report begins below with a brief profile of the country s socioeconomic makeup. The collection of data using the SABER WfD instrument was led by a principal investigator and co principal investigator. The sources for the SABER WfD study include the documentary evidence as well as interviews with key informants including stakeholder and provider representatives. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 6

7 2. Country Context Economic Trends Armenia s transition from a centrally planned to a market oriented economic system formally began in From 1995 to 2001 the Armenian economy grew by an average of 6 percent annually, and from 2002 to 2007 it registered double digit rates of annual growth (see Figure 3). Due to the global economic crisis, in 2008 the economy slowed down, registering 6.8 percent growth, and in 2009 it suffered a deep decline ( 14.4 percent). The recent economic indicators point to a moderate economic recovery, with GDP growth of 2.2 percent in 2010 and 4.7 percent in The most productive sectors in Armenia are a few exportoriented industries, such as mining, as well as sectors with a high presence of foreign ownership, such as the financial sector. Currently the top three most productive sectors financial intermediation, mining, and construction account for just 9 percent of the total employment in the country. By contrast, exactly half of the labor force is concentrated in the three sectors with the lowest productivity levels: food and accommodation services, agriculture, and education. 4 Figure 3. Armenia: Real GDP Growth (%, year over year) Employment As of 2011, Armenia s population was million. Between 1991 and 2013, the share of the population 7.2 under working age has declined from 30.3 percent to 20.2 percent, while the share of the population above working age has increased from 6 percent to 10.3 percent 5. The liberalization of Armenia s economy in and the adoption of the first Employment Act in 1992 laid the foundations that formed the labor market. Employment in Armenia steadily declined between 1991 and 2004 (from million to million persons) as a result of the economic transition. The economic growth between 2003 and 2008 also could not support an adequate increase in employment, since the growth in the newly formed economic sectors was assured mainly as a result of higher productivity, and consequently higher incomes from employment. 6 One of the major reasons for this economic growth without increased employment was the absence of adequate coordinated policies aimed at increasing jobs. In Armenia s current strategic program, employment growth and the development of human capital are recognized as key priorities, and the program highlights activities aimed at the creation of quality, well paid jobs. Table 1. Share of Employed, Unemployed and Economically Non Active Population in the Working age Population, (percent, based on LFS data) Employed Unemployed Economically non active Source: Armenia, National Statistical Service, Labor Market in the Republic of Armenia, 2012 ( Note: LFS = Labor Force Survey Due to transitional changes, employment in all sectors of economy has declined except in agriculture. 7 The greatest change since transition has been the shift from stable wages and salaried jobs to casual and less formal jobs and self employment. The share of self employed in 2011 comprised 29.8 percent, the vast majority of which 4 Economy and Values (EV) Research Center and EV Consulting, National Competitiveness Report of Armenia: Agenda for Upgrading Management Practices ( ), p Note that the working age population is defined as people ages 15 to While economic growth has failed to have a positive net effect on job creation so far, sustained market reforms can pay off in terms of greater job creation and increased productivity (Arias and Sánchez Páramo, 2014). As growth continues, one challenge for the education and training system will be to ensure that emerging skills demands are met. 7 Land reform initiated in 1991 resulted in the emergence of subsistence agriculture, which absorbed 23.3 percent of labor employed in the economy in 1991 (compared to 17.7 percent in 1990). Although the ratio of employed in subsistence agriculture starting mid 2000s showed a declining trend, it was still as high as 38.9 percent in SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 7

8 (80 percent 8 ) were employed in subsistence agriculture. Although the share of labor resources with tertiary, postgraduate, secondary specialized, or incomplete tertiary education was high in Armenia at 49.6 percent, 53.3 percent of this group was unemployed. Of those who were unemployed, 40.2 percent had vocational or upper secondary education. 9 Despite high levels of education, employers find it difficult to fill job vacancies due to a lack of adequately skilled candidates. Results from a recent World Bank STEP skills measurement employer and household survey shows the employers are able differentiate between individuals education and the actual skills, and that a candidates possessed of a variety of skills can command a premium on the labor market. 10 Results show that higher cognitive, technical and socioemotional increase the probability of several desirable labor market outcomes including being employed and earnings. The same survey indicates that employers see the technical and vocational education system as part of the skills problem. Roughly two out of three employers feel that technical and vocational training does not meet there skills needs. 11 Employers dissatisfaction is most pronounced with respect to practical skills. Over three in four report that technical and vocational education do an inadequate job of instilling these skills. The youth unemployment rate is high, at 39.2 percent in 2011, according to the Labor Force Survey (LFS). Taken together, these extremely high unemployment rates and unemployment duration (about one fourth of unemployed were looking for jobs for more than four years) and the high share of the employed in subsistence and informal jobs raise questions about the sustainability of the country s economic performance. They also raise questions as to whether Armenia can realize its vision of emerging as a competitive, knowledge based, and skillintensive economy. Supply of Skills Education. Since independence, the literacy rates in Armenia have been as high as nearly 99 percent. However, the transition period has had a negative impact on the education system. In particular, the reduction of public spending on education has led to a deterioration of the quality, relevance, and efficiency of education services. Despite this fact, the general level of educational attainment, particularly among labor resources (both the economically active and non active population) remains high, as suggested by the LFS data below (see Figure 4). Reforms that were undertaken in Armenia in the field of education mainly concerned general and higher education, giving little attention to technical and vocational education and training (TVET). Only in recent years have a number of documents been adopted that focus on the development of national TVET with regard to Initial TVET (ITVET), lifelong learning, and social partnership. The educational attainment of the population reveals a constant share of vocational and secondary specialized education of about 25 percent. Compared to general education (43.9 percent in 2010), the participation in VET is low, as its attractiveness did not change significantly. Education system. The formal education system comprises four years of primary schooling, followed by middle school, adding up to five years of lower secondary education. These two levels comprise general basic education in Armenia, which is followed by three years of upper secondary education (high school) or of primary vocational (craftsmanship) education. The VET system is characterized by two different pathways: 1. Preliminary VET schooling, which culminates in a craftsman qualification. This may start after nine years of general basic/compulsory or secondary (12 years) education, and the duration may be from six months to three years. 2. Middle vocational education (colleges), which prepares middle vocational specialists. This may start after compulsory general or secondary education, with a duration of two to five years to be qualified as a specialist. 8 Armenia, National Statistical Service, Labor Market in the Republic of Armenia, 2012 ( 9 Ibid. 10 Valerio, et. al., Armenia Skills toward Employment and Productivity (STEP) Survey Findings (Urban Areas), (2015). Survey included urban firms only. 11 The general education system enjoys slightly more esteem, but not by much. A little fewer than half of firms surveyed felt that the general education system met their skills needs. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 8

9 Both routes offer vocational qualification (access to the labor market) and opportunities for a secondary general diploma (Matura), and therefore provide the option to pursue higher education (see Figure 8). Preliminary professional education is offered by state institutions. At this level, TVET professions related to the spheres of services, trade, and the food industry are the most popular ones. Middle level professional education also is provided by state institutions. Private colleges are not allowed to issue state model diplomas, however, so the number of such colleges has decreased in recent years. For students entering craftsman school or college after completing compulsory general education (grades 1 to 9), preliminary and middle TVET offer both the vocational qualification and a secondary general diploma (Matura), and therefore provide the option to pursue higher education. As for continuing education, a number of initiatives with regard to continuing technical vocational and education training programs have been proposed by outside donors and have been welcomed by the national government, but in practice little progress has been made in this area. As a result, this dimension of TVET is developing rather slowly and no particular structure is assigned to it. Figure 4. Composition of Labor Resources by Education, 2011 (percent of total) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 9

10 3. Overview of Findings and Implications This chapter highlights findings from the assessment of Armenia s workforce development (WfD) system based on the SABER WfD analytical framework and tool. The focus is on policies, institutions, and practices in three important functional dimensions of policymaking and implementation: strategic framework, system oversight, and service delivery. Because these aspects collectively create the operational environment in which individuals, firms, and state and non state training providers make decisions with regard to training, they exert an important influence on observed outcomes in skills development. Strong systems of WfD have institutionalized processes and practices for reaching agreement on priorities, for collaboration and coordination, and for generating routine feedback that sustain continuous innovation and improvement. By contrast, weak systems are characterized by fragmentation, duplication of effort, and limited learning from experience. The SABER WfD assessment results summarized below provides a baseline for understanding the current status of the WfD system in Armenia as well as a basis for discussing ideas on how best to strengthen it in the coming years. Overview of the SABER WfD Scores Figure 5 on the next page shows the overall results for Armenia in the three functional dimensions of the SABER WfD framework. The score for each functional dimension is an aggregation of the scores for the underlying policy goals associated with it. The strategic framework, system oversight, and service delivery dimensions are all rated at an Emerging level. The detailed results will show that although Armenia has appropriate policies and institutions for WfD and there has been some success when it comes to policy conceptualization and implementation, the task of putting effective and responsive institutions into place has been much less successful across all three dimensions. Strategic Framework. The strategic framework dimension is concerned with setting the direction and the overall authorizing environment for WfD. As mentioned above, in Armenia it is assessed to be at the Emerging level, with a score for this dimension of 2.1 (out of 4). This finding suggests that while political support for WfD as an asset for economic progress is strong, the roles and responsibilities of key stakeholders in WfD remain fragmented, and the influence of businesses and industries in shaping and implementing WfD priorities is modest. System Oversight. System oversight refers to the standards and quality assurance that guide the functioning of the system. For this dimension, Armenia s score is 1.9, which falls at the low end of the Emerging level of development. The results revealed that competency based testing and certification await implementation, arrangements for institutional accreditation are still being formulated, and measures for articulation among training programs need to be further improved. Funding is also an issue: Government funding is modest; there is no strong linkage between the allocation of funds to institutions and the institutions performance; and the funding of educational institutions has generally not been supplemented by other sources. Service Delivery. Service delivery refers to the set up for training provision that equips individuals with marketand job relevant skills. Armenia s score for this dimension is 2.3, meaning that the service delivery is a bit above the Emerging level of development. A key finding in this dimension is that although formally all stakeholders have influence over training curricula, in a large number of cases non state stakeholders involvement is limited. Private training providers are allowed to operate within the existing legal framework quite freely, but they have almost no incentives to meet quality standards, and a culture of monitoring and evaluation is not well developed. While service delivery scores relatively well compared to the other two dimensions at this time, given the nature of the challenges continued improvement in service delivery will likely be difficult to achieve without marked improvement in linking WfD to broader socioeconomic goals and strengthening of the arrangements for funding and quality assurance. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 10

11 Figure 5. Armenia: Benchmarking Results Dimension Strategic Framework System Oversight Service Delivery Source: Authors analysis based on SABER WfD data Policy Implications of Findings The findings presented in this report reinforce the government s recent identification of priority issues it is seeking to address. Armenia s Development Strategy for , Employment Strategy for , and Strategy of National Security all highlight target goals that include the following: To ensure a competitive and better quality education system, To ensure a sustainable and continued improvement of labor force competitiveness, Policy Goal 1. Direction 2. Demand led 3. Coordination 4. Funding 5. Standards 6. Pathways 7. Excellence 8. Relevance 9. Accountability Note: Key to Scoring: 1 = Latent; 2 = Emerging; 3 = Established; 4 = Advanced; see Figure 2 for definitions. To mitigate the imbalance between the labor force supply and demand, and To create job opportunities for young people and employment opportunities for disadvantaged groups of the population To achieve these goals, not only do well designed policies need to be in place, but these policies also have to be translated into actions. By systematically documenting existing policies, institutions, and practices, this report provides a foundation for identifying policy and institutional gaps and moving forward towards actions for strengthening Armenia s workforce. The results indicate areas of particular system strengths, which could be used as platforms for wider actions, as well as weaknesses that need to be overcome for a better functioning WfD system. In particular, the current assessment highlights a number of areas that may provide a starting point for dialogue around strengthening Armenia s WfD policies and institutions. The recommendations here are not listed based on priority, but rather linked to each policy dimension in the SABER WfD framework. In the end, the prioritization of actions will be the result of an on going dialogue and assessment by policymakers. On Strategic Framework. A strong strategic framework for WfD requires visible leadership and advocacy that is driven by an understanding of, and conviction about, the role and nature of WfD in national development. It is critical to identify strategic priorities and to mobilize collaborative effort to overcome challenges. Therefore, critical efforts are required to strengthen cooperation among all stakeholders. Although in recent years some positive developments were registered at the strategic governance level in the identification of primary goals, targets, and priorities, well coordinated policy actions for improving the strategic framework of WfD need to be considered further. Recommendation 1: Strong political leadership is necessary, and it needs to be backed by an apexlevel WfD body that can coordinate the design and implementation of strategic WfD initiatives. With the wide range of stakeholders in the area of WfD, strong leadership is essential to ensure the coherence of policies and effectiveness of program implementation. There is therefore a critical role for an apex level body with a legally defined mandate to coordinate the implementation of key priorities. This apex level body should be able to connect promote strategic cooperation among businesses, relevant government agencies, educational and training institutions and civil society. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 11

12 One important consideration is promoting a demanddriven approach to WfD led by industry and the private sector generally. The time lag between labor market signals and actual employers needs requires stronger intervention from different labor market players, particularly from employers organizations, as well as better communication and cooperation among all social partners and the government agencies concerned. Therefore, the partnership with industry and the private sector is key to the progress of the WfD agenda. The appropriate institutional arrangement for this coordination function is a subject that requires more study. One possible option is strengthening and expanding the National Council for VET Development (NCVD) s roles and mandates. Together with the National Center for VET Development (NCVETD), various sectoral committees for educational and training standards, and boards of vocational education institutions, NCVD could assume a larger role in coordinating WfD strategy implementation at different levels. However, the capacity of NCVD needs to be much strengthened in the areas of carrying out comprehensive assessments of national economic prospects; identifying their implications for skills supply and demand; and aligning education and training supply, labor market policies, and broad economic policies. To play the apex level role effectively also requires NCVD to be able to collaborate with a wide range of stakeholders, prepare an agreed implementation plan of needed reforms, and oversee and support each implementation agency that is involved. On System Oversight. The main objectives of oversight include the facilitation of efficient and effective skills acquisition by individuals to improve their employability and productivity and the ability of employers to meet their demand for skilled workers in a timely manner. The goal is to minimize mismatches in skills supply and demand, thereby enhancing the contribution of WfD to economic growth and social progress. Based on the diagnosis, system oversight is the weakest element in Armenia s WfD system and thus requires particular attention. Recommendation 2: The government should pilot performance based funding mechanisms that ensures diversification in resources and promotes non state investments in TVET. Funding arrangements (how money is mobilized, allocated, and channeled) are critical in shaping incentives and consequently the decisions made by individuals, training providers, and employers. In Armenia, the WfD system relies on limited resources, mainly from the state. There are no explicit criteria to encourage performance or efficiency in spending. In fact, the current system often promotes inefficiencies in allocation and inequalities among training providers. Funding principles are not tied to performance, which results in few investments in innovation for service delivery. Based on the assessment, the following could be considered in terms of improving system financing: (i) increasing overall public funding allocated to WfD, but the increased funding nevertheless needs to be utilized in a more effective way, particularly with regard to further leveraging private financing; (ii) introducing performance based funding to training institutions for better training efficiency and effectiveness, particularly as measured by graduates job placement; (iii) building appropriate capacity into training institutions so that their autonomy, as currently guaranteed by the legal framework, goes hand in hand with accountability, which is a prerequisite for performance based funding to be relevant and to enable them to become more flexible in responding to local market needs; (iv) introducing competitive funding into the system, with both public and private training institutions eligible, as a driver for greater incentives for better quality and innovation; and (v) promoting innovation and scaling up of best practices at the sector level through strong monitoring and evaluation mechanisms across all programs. Recommendation 3: Strengthen and expand current efforts to put in place quality assurance system (accreditation, standardization, and certification) that is uniformly enforced. Creating reliable standards for quality in service provision and skills acquisition is an integral part of having efficient and effective system oversight. In 2011, the government adopted the Armenian Qualification Framework, which facilitates a comparison of national qualifications with other European frameworks. The level descriptors of the national qualification framework also form the basis for the development of state educational/training standards. The existing educational/training programs are being reviewed in accordance with the descriptors. Since 2009, about 120 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 12

13 competency based state educational/training standards and modular curricula have been developed for TVET, which were reviewed by the sectoral committees and approved by the Ministry of Education and Science (MoES). About 100 standards have already been introduced to educational and training institutions. Regulations of the State Accreditation of Educational Institutions providing professional education programs and their professions have also been adopted by the national government. The National Center for Professional Education Quality Assurance Foundation (ANQA) is an external body monitoring quality assurance in higher education. Following European standards and guidelines, ANQA compiled a Strategic Plan (for the transitional period ) that aims to establish a quality assessment culture at the tertiary level. However, ANQA only issues accreditation to tertiary institutions, not specific programs. In addition, ANQA does not cover quality assurance for TVET institutions below tertiary level. Going forward, the following measures should be considered by policymakers: (i) At the tertiary level, allocating adequate resources to and strengthening ANQA to fulfill its role as a quality assurance body, and further fostering a closer link between institution accreditation and institution s contribution to the labor market and skills; (ii) charting out a roadmap for developing a quality assurance system for TVET institutions and programs below the tertiary level and for priority professional programs at the tertiary level; (iii) piloting skills assessment and certification protocols to grant competency based qualifications within and beyond the existing formal pathways, particularly for selected occupations and skills with high labor market demand. Recommendation 4: Increase pathways from VET to higher education. The underlying aim for diversified and flexible pathways is to keep the skills of the workforce up to date and adapted to changing economic conditions and opportunities. Given that labor market conditions are always evolving, it is important that all available options for acquiring relevant skills, particularly those funded publicly, remain responsive and sensitive to emerging trends and opportunities for employment. The demand for skills upgrading is particularly going to dominate in the longer term, following the economic growth and higher demand for high order skills. In Armenia, despite the adequate number of institutions and programs covering diverse disciplines and levels, the system provides limited opportunities for progression within vertical pathways. This is due to the fact that the perception of VET was and still is of a second chance education, one for less privileged pupils who had no alternatives to this dead end education, one that would not lead to university or to attractive jobs in the labor market. Even though there are no policy barriers for graduates of preliminary and middle VET to enter tertiary education, in practice it is difficult for them to get back on track once the focus of learning has shifted toward vocational training under the VET curriculum. Without additional private tutoring and extra learning outside the schools, they have little chance to pass the tertiary education entrance exams. Students coming from low income families are particularly disadvantaged, since their families would not be able to afford the extra tutoring required. There is a general policy framework in place to guide further development in this area. The key national policies and strategies are all designed to contribute to diversifying the pathways for skills acquisition in Armenia. These include VET System Reform Program with Action Plan and Implementation Timeline ; Concept of Lifelong Learning in Armenia adopted in October 2009; Concept on Social Partnership in the Field of Preliminary Professional (craftsmanship) and Middle Professional Education adopted in May 2009; and Memorandum of Understanding on Cooperation in the field of VET, signed in September 2009 by MoES, the Republican Union of Employers of Armenia, and the RA Chamber of Commerce and Industry. Looking ahead, policies should be refined to address the following needs: (i) Strengthening the learning of foundation skills at the preliminary and middle level VET schools, which will likely need related curriculum reform at these levels giving added emphasis on core subjects; (ii) assessing the feasibility, together with the costs of benefits, of postponing tracking into vocational education and training after 12 years of broad based general education; and (iii) improving the assessment and introducing recognition of prior learning within the SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 13

14 qualifications framework, which can be piloted in selected areas. On Service Delivery. This functional dimension of policy choices shifts our focus from the strategic and systemic to the operational. Service delivery addresses implementation challenges in WfD that stand between a country s ambition for WfD to support economic development and what materializes in reality. Training providers, both state and non state, are the main channels through which the country s strategic and systemic policies are translated into operational applications. Recommendation 5: Increase provider accountability for results through robust monitoring and evaluation. Training providers (both state and non state) in Armenia are regulated, but there is a gap in terms of accountability for results at the WfD sector level to meet quality standards. This in part stems from: (i) limited tonearly nonexistent monitoring and evaluation of the delivery of WfD services; (ii) a lack of evidence based assessments; and (iii) a weak information base for communicating labor market needs. Therefore, in this area the following is suggested: (i) in selected and strategic sectors/industries, enhancing the monitoring and evaluation system at the sector level and linking this to accreditation and certification of training providers and programs, with one central agency possibly MoES serving as a clearinghouse of all information; (ii) introducing and enforcing accountability measures for both state and non state training providers by setting clear performance indicators to be used in accreditation and quality assurance processes; (iii) instituting a graduate tracking database, which can be piloted in selected institutions, and in the longer term creating a centralized database for continuous system monitoring and improvement; (iv) building a strong labor market information system and career counseling services based on labor market information. The labor market information system can be tailored to specific users students and parents, job seekers, employers, policymakers, education and training institutions and ultimately to address mismatches in skills supply and demand. Recommendation 6: Public private partnerships should be broadened and relevance of programs vis à vis labor market demand more carefully ensured. Currently, a large share of curriculum and training programs has been developed despite the absence of an adequate level of input from industry and the private sector. Linkages between industry and training providers exist, but they are largely sporadic and have yet to be institutionalized across the system as whole. The large potential of training provision by the private sector partners has not been fully tapped. The assessment thus highlights the following recommended actions: (i) introducing incentives to attract private sector investments in the delivery of training (financial and/or non financial), through either pre employment training or on the job training; (ii) exploring public private partnership potentials in delivering training, which could be through government contracting of private training providers that have track records of effectiveness and efficiency or have training expertise in specific skill domains or industry knowledge; and (iii) linking to innovative financing instruments, such as competitive grants for training, in order to introduce competition that would incentivize innovation in delivering high quality programs relevant to the labor market. The competition should be open to both public and private providers. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 14

15 4. Aligning Workforce Development to Key Economic and Social Priorities Workforce development (WfD) is not an end in itself, but an input toward achieving broader objectives: boosting employability and productivity, relieving skill constraints on business growth and development, and advancing overall economic growth and social well being. This chapter briefly introduces Armenia s socioeconomic aspirations, priorities, and reforms, which are relevant for contextualizing the next section s presentation of the SABER WfD ratings. It summarizes the key features of the environment in which apex level strategy is made: (i) strategic priorities for economic development; (ii) national WfD priorities; and (iii) key laws that define the procedures and context for setting economic and WfD strategy. Strategic Socioeconomic Aspirations and Priorities The current stage of Armenia s socioeconomic development poses a set of complex challenges for the country, making it urgent to address both immediate political demands and institutional and structural reforms for enabling sustained growth and macroeconomic stability. The government has declared building a knowledge based economy as its long term strategic objective, and it has adopted several policies to guide the achievement of this objective. Most notable among these policies is the Armenia Development Strategy (ADS), , which provides a vision for the country s socioeconomic and political development. The strategy defines the increase of the employment through creation of quality and high productivity jobs and elimination of human poverty and assuring human development as the main objectives. In order to achieve the mentioned objectives, the ADS updated the government s 2008 Sustainable Development Program strategy paper, which envisaged three sets of priority strategies: (i) economic policy for ensuring sustainable and accelerated economic growth; (ii) active social and income policy for vulnerable population groups (including the poor); and (iii) 12 Armenia, Ministry of Economy, Republic of Armenia, Export Led Industrial Policy Framework (Yerevan: 2011), page The Employment Strategy was developed taking into account the RA Constitution, the regulatory (legislative) framework of the sphere, as well as the International Labor Organization s (ILO s) employment related modernization of the governance system, including improving the effectiveness of state governance and ensuring accelerated growth of the resource envelope at the disposal of the state. The government s Export Led Industrial Policy Framework reflects the government s active industrial policy, which is intended to support the expansion of the exportable sector of the economy by identifying and utilizing export potential. The guiding vision for the industrial policy is to turn Armenia into a country producing high value and knowledge intensive goods and services with creative human capital at its core. Achieving this vision requires a phased approach that entails a gradual transition from resource intensive production to capacity and skills intensive production and subsequently knowledge based production to play a dominant role in the long term outlook. 12 At the core of creating such an economy is the challenge of matching employment, skills development, and WfD. Lastly, the government s Employment Strategy ( ) 13 provides the country s vision on implementing investment, fiscal and monetary policies, education (including higher education), vocational training, and social welfare policies through the lens of employment, with a vision to reduce the gap between labor market and educational institutions through continuously increasing the quality of education, implementing the national qualification framework, providing opportunity for and assuring the accessibility of lifelong learning, and implementing effective career guidance services. Policies and specific measures for promoting employment implemented by the government should have clear procedures for effectiveness evaluation. Technical and Vocational Education and Training (TVET) Reforms. A strong education sector is considered to be important as one of the preconditions for the country s sustainable development and development of the human capital. Thus, the development of this sector is one of the development priorities of the country. Increasing the quality and effectiveness of education at all levels of the educational system, increasing the relevance of different levels to each other and matching them to international standards, and ensuring conventions and the recommendations of the European Social Charter Requirements of the EU European Neighborhood policy program (RA Employment Strategy, , page 4). In addition, it refers to other country strategy papers where it is appropriate. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 15

16 affordable/accessible education for all groups of the population are all priorities for the development of the sector. In order to achieve the mentioned goals, reforms are implemented at all levels of education. By the end of the 1980s, the preliminary (craftsmanship) and middle vocational system of Armenia had lost its significance due to the socioeconomic crisis and the absence of appropriate legislative and normative instruments. After the fall of the Soviet Union, Armenia, like most former Soviet countries, faced challenges related to the emerging private sector and declining public sectors. The restructuring of the economy and the closing down of large state enterprises led to a level of unemployment not seen before, with serious consequences for poverty and social exclusion. As a result, vocational and technical schools saw their main clients disappearing from the market, and they had to rebuild themselves (their teachers, curricula, and pedagogical approach) to meet the new emerging needs of the labor market. Starting in 1999, by adopting the Law on Education, the government undertook broad reforms in the field of education. These reforms mainly concerned general and higher education, giving little attention to TVET. Only in recent years have a number of policies have been adopted that focus on the development of national TVET with regard to Initial TVET (ITVET), lifelong learning, and social partnership. In 2004, to make the content of vocational education relevant to labor market requirements, the government approved the Preliminary (Craftsmanship) and Middle Vocational Education and Training Strategy, which has served as a basis for implementing reforms in the sphere of preliminary and middle vocational education. The document Concept on Social Partnership in the Field of Preliminary (Craftsmanship) and Middle Vocational Education in RA adopted by the Government in 2008 identifies the following priorities for Armenian TVET: (i) optimize TVET financing and improve TVET governance; (ii) introduce competency based TVET standards; (iii) increase the effectiveness of the TVET system and improve educational outcomes; (iv) modernize quality monitoring mechanisms; and (v) strengthen and institutionalize social partnership. In 2008, two important steps were taken for VET governance: namely, the establishment of the National Council for VET Development (NCVD), a tripartite body to drive system reform, and the creation of the National Center for VET Development (NCVETD), an instrument to implement the decisions made by the NCVD. SABER WfD Ratings on the Strategic Framework Based on data collected by the SABER WfD questionnaire, Armenia receives an overall rating of 2.1 (Emerging) for the Strategic Framework dimension (see Figure 6). This score is the average of the ratings for the underlying policy goals: Policy Goal 1: Articulating a strategic direction for WfD Policy Goal 2: Fostering a demand driven approach to WfD Policy Goal 3: Strengthening critical coordination among key WfD leaders and stakeholders The ratings for these policy goals are presented and explained below, followed by a reflection on their implications for policy dialogue. Figure 6. SABER WfD Ratings of the Strategic Framework Dimension Strategic Framework Overall Direction 2. Demand led 3. Coordination Note: Key to Scoring: 1 = Latent; 2 = Emerging; 3 = Established; 4 = Advanced; see Figure 2 for definitions. Source: Authors analysis based on SABER WfD data. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 16

17 Policy Goal 1: Articulating a Strategic Direction for WfD. Leaders play an important role in crystallizing a strategic vision for WfD that is appropriate to the country s unique circumstances and opportunities. Their advocacy and commitment attract partnership with stakeholders for the common good, build public support for key priorities in WfD, and ensure that critical issues receive due attention in policy dialogue. Taking these ideas into account, Policy Goal 1 assesses the extent to which apexlevel leaders in government and in the private sector provide sustained advocacy for WfD priorities through institutionalized processes. Armenia is rated at the Emerging (2.5) level on this policy goal, reflecting the fact that there are some visible champions advocating for WfD to support economic development. Although the advocacy is on an ad hoc and limited basis, nonetheless WfD champions have taken specific action on strategic WfD priorities through a range of interventions, and implementation progress is monitored, albeit through ad hoc reviews. The prioritization of WfD across all national plans demonstrates a level of government support and willingness to reform WfD. The country s strategic documents identify as integral to national security and economic and social development the following goals: enhancing the skills and competitiveness of the labor force, mitigating the imbalance between the labor force supply and demand, creating job opportunities for young people, and ensuring employment opportunities for noncompetitive groups of the population. These goals are articulated in policy and planning documents such as ADS, Employment Strategy, Armenia Export Led Industrial Policy Framework, Concept on the Development of Preliminary Professional (craftsmanship) and Middle Professional Education in RA with Action Plan for , Adult Education Concept Paper and Strategy, Concept of Lifelong Learning in Armenia Education. The reforms of tertiary educational programs are underway and aim to improve the quality of vocational, higher, and continuing education, bringing them in line with European standards. Despite the fact that the vision of WfD and advocates for WfD are there and the vision is reflected in key strategic documents, there is a disconnect between these longterm strategic goals and the current vision for WfD, which focuses on combating unemployment, improving the readiness of students for the current labor market, and addressing the unmet demand for skills of current, often non strategic industries. There appears to be a consensus in Armenia that there is a mismatch between supply and demand for qualified labor and that there is little up to date information on the actual demand. The National Competitiveness Report Armenia 2010: Higher Education Challenge concluded in its analysis that: What is needed is better information on the labor market, policies that promote economic growth and labor absorption, and more academic counselors to advise on career choices or act as liaisons between industry and the institution. The process of connecting labor market supply and demand, businesses, the economy, and the educational institutions started years ago and is still underway. The government, business chambers and guilds, and VET institutions (schools, colleges, and universities), assisted by international donors, have undertaken numerous initiatives to deal with the challenge. The establishment of the NCVETD and NCVD, of sectoral committees for educational standards, and of boards of vocational education institutions that serve as bodies for coordinating WfD strategies at different levels these are all mechanisms that were created to pull the various elements of WfD into a coherent national plan aligned with the country s economic challenges and priorities. However, stronger and more consistent leadership is required to bring efforts of multiple national and international stakeholders together, mainstream good practice to create a critical mass for improvement, and make these mechanisms work to link WfD to the broader economic development goals of the country. Stronger leadership is also needed to convert ad hoc actions focused on improving areas of the WfD system into regular actions that can leverage WfD as a tool for economic development. At present there is no consensus among leaders about how to achieve greater coherence among socioeconomic goals and WfD. Despite the existence of the abovementioned strategic documents, the efforts of those within various government bodies remain fragmented, indicating that there is a need for leaders in the government to provide more sustained guidance on concrete goals for WfD and how different agencies can SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 17

18 harmonize activities in pursuit of them. In addition, limited, ad hoc advocacy from the business community has not yet led to a comprehensive and widely supported strategy for leveraging WfD as a means to support broader economic and social development plans. This has taken place partly because the business community (which could serve as influential champions) is largely absent in advocating for a strategic vision for WfD. Although institutional mechanisms have been created and seem to be functioning, the private sector s involvement in and thus ownership of the decisions on WfD priorities is weak, sporadic, and only at the local and sectoral levels. Policy Goal 2: Articulating a Strategic Direction for WfD. Effective advocacy for WfD requires credible assessments of the demand for skills, engagement of employers in shaping the country s WfD agenda, and incentives for employers to support skills development. Policy Goal 2 incorporates these ideas and benchmarks the system according to the extent to which policies and institutional arrangements are in place to: (i) establish clarity on the demand for skills and areas of critical constraint; and (ii) engage employers in setting WfD priorities and in enhancing skills upgrading for workers. Armenia is rated at the Latent (1.4) level on this policy goal, reflecting the fact that industry is not involved in setting strategic priorities for WfD nor does the government take measures to include industry, either by providing means to engage industry in the strategymaking process or by encouraging employers to take an active role in training workers. Measures to identify and address skills constraints, even in priority sectors, are also limited. Formal assessments of Armenia s economic prospects and their implications for the labor market are done periodically. Occasional studies have identified the gaps between the skills and knowledge that exist and those that are required, the training needs of the workforce in a particular sector of the economy, and the shortcomings of the educational system in contrast with the actual demand in the long term. However, the findings of those studies have not been addressed appropriately or in a timely way. One reason for this is that the skills and knowledge needs assessments are occasional and are not sufficient to provide a complete picture of the country s economic prospects and economy wide skills implications. Another reason could be the weak cooperation among stakeholders. There is also a lack of consensus among apex level leaders on the appropriate policies and, crucially, a roadmap for their implementation. This is in part related to limited industry input at the apex level and, despite limited examples of productive engagement, the failure of the business community as a whole to take a proactive approach to identify and address the emerging skills gap. While clear roles and responsibilities are assigned by law to ministries and agencies such as the Ministry of Education and Science (MoES), the Ministry of Economy (MoE), and the Ministry of Labor and Social Issues (MLSI) with regard to strategy development and implementation, Armenian law neither assigns analogous responsibilities to the private sector nor contains provisions to incentivize skills upgrading by employers. Efforts to facilitate more robust connections among businesses, those involved in economic planning, and educational institutions have been on the government s agenda for years, but measures to do so have been ad hoc and slow moving. Such efforts have included utilizing new and upgraded channels to link supply and demand for qualified labor, and establishing and strengthening relationships with institutions such as career centers, the NCVD, sectoral committees for educational standards, boards of vocational education institutions, and training and competitiveness/innovation funds. Policy Goal 3: Strengthening Critical Coordination for Implementation. Ensuring that the efforts of multiple stakeholders involved in WfD are aligned with the country s key socioeconomic priorities is an important goal of strategic coordination. Such coordination typically requires leadership at a sufficiently high level to overcome barriers to cross sector or cross ministerial cooperation. Policy Goal 3 examines the extent to which policies and institutional arrangements are in place to formalize roles and responsibilities for coordinated action on strategic priorities. Armenia is rated at the Emerging (2.3) level on this policy goal, meaning that the coordination of government entities in the pursuit of WfD strategies and programs is weak. Non governmental WfD stakeholders, such as employers and trade unions, have a legally defined role SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 18

19 and responsibility. Furthermore, the coordination between them and government entities occurs through ad hoc albeit institutionalized mechanisms, which at the national level extends to WfD strategy. The assessment suggests that each of the ministries and agencies has a legally defined role and responsibility, and there are almost no overlaps in the mandates of the entities responsible for WfD. The involvement of all parties is assured by the existence of corresponding institutions at the national and local level. The NCVD, created in 2008, is the main institution in this sphere; it is made up equally of governmental ministries (MLSI and MoES) and employers and trade unions. According to its charter, the NCVD holds both regular (at least four times per year) and ad hoc meetings. Direct participation by council members in the meetings/sessions is required. Other non government stakeholders (that are not members of the NCVD) do not have legally defined roles and responsibilities; however, they have some nonformal engagement. The document Concept on Social Partnership in the Field of Preliminary (Craftsmanship) and Middle Vocational Education in RA presents the strategic issues regarding the development of social partnership in the sphere of preliminary and middle vocational education, and proposes solutions deriving from the interests of all parties. It is aimed at promoting the formation of a tripartite consensus on cooperation between the government, employers, and trade unions as well as the institutional composition of social partnership. In the field of VET it defines the following four levels of social partnership: national, local (institutional), sectoral, and regional. On the national level, the social partnership is realized within the framework of the NCVD, as established by a 2008 decision of the prime minister. 14 In order to ensure social partnership on the institutional level, collegial governance bodies VET governing boards (VETGBs) have been established in VET institutions. Taking into consideration the relatively small territory and limited resources of Armenia, structures (temporary or permanent sectoral or regional councils or committees) of social partnership have been established on sectoral and regional levels, but only to tackle issues specific to the relevant sector or region. Despite the well established roles and responsibilities of WfD stakeholders and some formal mechanisms in place that bring WfD stakeholders together, there is still a lack of coherence, and the communication and interaction remains weak. Implications of the Findings The relationship between education, on the one side, and the economy, business, and the labor market, on the other, is a matter of debate in Armenia. It has received constant attention both from the government and from employers, who underline the need for a skilled labor force in order to make companies competitive in the internal and external market. The process of connecting labor market supply and demand, businesses, the economy, and the educational institutions started years ago, when the government, to deal with the abovementioned challenges, began undertaking numerous initiatives. However, the findings of this analysis for Dimension 1 reveal a number of challenges yet to be addressed. Although the country s main strategic documents define the increase of the employment through creation of quality and high productivity jobs and elimination of human poverty and assuring human development as the main objectives, advocacy and dialogue among the champions in order to collaborate on the WfD policy agenda remains challenging. 14 Armenia, Office of the Prime Minister, Decision of the RA Prime Minister, No A, 11 December SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 19

20 5. Governing the System for Workforce Development Fostering efficient and equitable funding of investments in workforce development (WfD), facilitating effective skills acquisition by individuals, and enabling employers to meet their demand for skilled workers in a timely manner are important functions of WfD authorities. The overall objective is to minimize systemic impediments to skills acquisition and mismatches in skills supply and demand. This chapter begins with a brief description of how the WfD system in Armenia is organized and governed and provides a summary description of: (i) the organizational and governance structures of the WfD system (i.e., the main ministerial agencies and mandates); and (ii) key characteristics of WfD funding mechanisms. It begins with a brief overview of the institutional landscape for governance of the WfD system, then presents the detailed SABER WfD results, and concludes with a discussion of the policy implications of these results. Overall Institutional Landscape 15 The Ministry of Education and Science (MoES) bears responsibility for the overall coordination of curricula and learning outcomes for all types of education. Overall, the technical and vocational education and training (TVET) system can be considered centralized when it comes to decision making but fragmented when it comes to the distribution of authority. A new division under MoES, called TVET Policy Making and Strategy, was established in 2008 as a result of restructuring of the TVET Department under MoES. In the same year, the National Council for TVET Development (NCVD) was set up as a tripartite body aimed at supervising the TVET reform. Representatives of national government and social partners, that is, employers and trade unions, are all participating in the council. In order to provide methodological support to the VET system development (e.g., the development of standards), improve curricula and learning materials, and provide teacher training, the National Center for TVET Development (NCVETD) was established as a part of the National Institute for Education. The National Center for Professional Education Quality Assurance was established in 2008, responsible for both higher and vocational education. It has been taking the first steps towards the definition of quality assessment standards and criteria. In 2011 the government established a National Training Fund with the objective of raising the competitiveness of individuals in the labor market and reducing the level of poverty by imparting vocational knowledge and other working abilities and skills. The TVET system can be categorized into two different levels: (i) preliminary (craftsmanship) vocational education, which has a duration of six months to three years and leads to a Craftsman qualification; and (ii) middle vocational education, which has the duration of two to five years and leads to a Specialist qualification (see Figure 7). Post completion of compulsory general education (grades 1 to 9) students are offered the option of pursuing higher education through either vocational training or pursuit of a secondary general diploma. Successful middle level TVET graduates can then proceed to a second year of higher education. However, resolutions adopted by the national government rearranged a number of preliminary (Craftsmanship) and middle level vocational educational institutions. In December 2011, 13 craftsmanship vocational schools functioning in Armenia were transferred to the subordination of the MoES. Currently, all 25 such schools are managed by this body. According to the measures adopted by the government, the middle level educational institutions are to be transferred under the authority of MoES by the following stages: in 2012, seven colleges being managed by the Ministry of Agriculture, one under the Ministry of Culture, and one under the Ministry of Energy, and in 2013, all colleges managed by the Ministry of Healthcare. When it comes to lifelong learning, the State Employment Services Agency (SESA), acting under the authority of the Ministry of Labor and Social Issues (MLSI), provides services all over the country through its 15 This section draws from UNESCO UNIVOC s World TVET Database ( 3=&disp=3). SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 20

21 network of local employment services, including training for the unemployed. The National Institute of Labor and Social Research (NILSR) in cooperation with NCVETD, under MoES, began identifying occupations as a first step to establishing standards and qualifications in The competency standards for each occupation are being developed by a specially formed working group: Appropriate ministries, social partners, and VET institutions are participating in developing the standards. The working group consists of five to seven representatives of a government specialized agency (NCVETD), training providers, employers, or industry associations. Even though a number of initiatives have been proposed by outside donors with regard to programs in continuing TVET and have been welcomed by the national government, in practice little progress has been done in this area. As a result, this dimension of TVET is developing rather slowly and no particular structure is assigned to it. Non formal learning activities are not registered in Armenia (mainly because no license is required for their provision), and no relevant official statistics on them are collected. According to some experts estimations, however, tens of thousands of people are involved in these activities. Figure 7. The Armenian Education System LABOR MARKET Preliminary Vocational Education TVET Institutions Craftsmanshi p Middle Vocational Education First stage (not leading to entry to advanced research programmes) Lower Secondary Education (5 years) Middle Schools Primary Education (4 years) Elementary Schools Higher Education - Master (Second or further) Degree Upper Secondary Education (3 years) High Schools Researcher Second stage (leading to advanced research qualifications) Higher Education - Bachelor (First) Degree Public spending on education is very limited and has been decreasing. Despite the fact that the MoES and the Ministry of Economy (MoE) have recognized TVET Basic General Educatioan First stage (leading to entry to advanced research programmes) Secondary General Education development as a priority, the Ministry of Finance (MoF) has been against any increase in the budget for TVET. Therefore, the main funding of TVET institutions comes from fee paying students, whereas the funds for modernization and improvement of TVET come from donor organizations (the European Union, the United Nations, and governments and institutions of various countries). The share of expenditure on education in Armenia in 2012 comprised 2.6 percent of GDP, compared to 3.2 percent in 2010 and 2.8 percent in Thus, the budget allocation for education is quite low. Out of the total 2012 education budget, only 5.2 percent went to VET (although this is higher than the 4.8 percent in 2011), including preliminary and middle VET. In the school year, the number of students attending initial and middle VET was 34,821, of which 27 percent attended with free seats; thus 73.5 percent of students were privately funding their education. This type of reliance on household contributions understandably limits disadvantaged populations from accessing public training. The number of free seats has decreased, putting an even higher financial burden on young people and their families. SABER WfD Ratings on System Oversight The SABER WfD framework identifies three pertinent policy goals corresponding to oversight mechanisms for influencing the choices of individuals, training providers, and employers: Policy Goal 4: Ensuring Efficiency and Equity in Funding Policy Goal 5: Assuring Relevant and Reliable Standards Policy Goal 6: Diversifying Pathways for Skills Acquisition. The ratings for these Policy Goals are presented and explained below, followed by a reflection on their implications for policy dialogue. Based on data collected by the SABER WfD questionnaire, Armenia receives an overall rating of 1.9 (Emerging) on the System Oversight (see Figure 8). This score is the average of the ratings for the underlying three Policy Goals: Ensuring Efficiency SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 21

22 and Equity in Funding (1.4), Assuring Relevant and Reliable Standards (2.1), and Diversifying Pathways for Skills Acquisition (2.2). The explanation for these ratings and their implications follow below. Figure 8. SABER WfD Ratings of the System Oversight System Oversight Overall Funding 5. Standards 6. Pathways Note: Key to Scoring: 1 = Latent; 2 = Emerging; 3 = Established; 4 = Advanced; see Figure 2 for definitions. Source: Authors analysis based on SABER WfD data. Policy Goal 4: Ensuring Efficiency and Equity in Funding. WfD requires a significant investment of resources by the government, households, and employers. To ensure that these resources are effectively used, it is important to examine the extent to which policies and institutional arrangements are in place to: ensure stable funding for effective programs in initial, continuing, and targeted VET; monitor and assess equity in funding; and foster partnerships with employers for funding WfD. Therefore, Policy Goal 4 focuses on the government s role in funding WfD, ensuring efficient and effective use of the available funds, and in fostering partnerships that can multiply the resources available to encourage and support investment in WfD by individuals and employers. Armenia is rated at the Latent (1.4) level on this Policy Goal, reflecting funding allocation procedures that do not promote efficiency in resource use and the lack of the private sector involvement in providing resources for training. The government determines the priorities for the state budget expenditure and allocates finances. Since Soviet times, both preliminary and middle VET has not been considered very important and most of the limited budget resources for education have been allocated to general education. The VET system has undergone several phases of reform, in particular after the adoption of the Law on Education in 1999, followed by the rationalization programs aimed at reducing the number of less relevant VET schools. In order to ensure better use of the limited resources, the rationalization of the system is ongoing with the reduction of the number of colleges and preliminary VET schools (ETF, 2013, p. 12). The current input based budgeting processes for TVET do not take into account performance indicators; they are based on historic trends and enrollment, with some consideration of alignment to emerging WfD priorities. However, most public programs do not draft or distribute annual performance documentation, which results in a lack of accountability and incentives to utilize resources efficiently or to become more self reliant. There does not appear to be much cost consciousness in the TVET system; based on the current budget process, the prevailing incentives are to spend allocated budgets in full and to obtain more financing. Lack of available data on cost benefit analysis and cost comparisons across programs and institutions could be one reason for this. Another effect of this lack of data is that it is difficult to determine the impact on equity of funding. Despite the fact that formal and institutional partnerships between training providers and employers have been designed, the systemic arrangements for the WfD authority to collaborate with industry and other key stakeholders are absent. Although the government facilitates the partnership between industries and other stakeholders through formal arrangements and the partnership is institutionalized on the national, regional, and individual levels, industry involvement is very limited. While industry experts occasionally provide technical inputs in setting standards and testing, at the college level the industry involvement often ends at providing internship opportunities. Policy Goal 5: Assuring Relevant and Reliable Standards. The WfD system in Armenia comprises a wide range of training providers offering courses at various levels in diverse fields. An effective system of standards and accreditation would enable students to document what they have learned and employers to identify workers with the relevant skills. For Policy Goal 5, it is therefore important to assess the status of policies and institutions to set reliable competency standards, assure the credibility of skills testing and certification, and develop SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 22

23 and enforce accreditation standards for maintaining the quality of training provision. Therefore, this particular policy goal focuses on assuring the quality of both the services offered by training providers and the skills acquired by individuals through credible procedures for accreditation and certification. Armenia is rated at the Emerging (2.1) level on this policy goal, suggesting that although the competency standards are developed for a large number of occupations and all key stakeholders are engaged in setting competency standards for major occupations, the policy dialogue on competency standards and on Armenia s national qualification framework (NQF) engages numerous stakeholders through institutionalized processes. Still, competency based testing is not used; skills testing for major occupations is mainly theory based and the certificates awarded have little impact on employment and earnings. Since 2009, more than 100 competency based state educational standards and modular curricula have been developed for the TVET, all of which were reviewed by the sectoral committees and approved by the MoES. Of these 100 standards, 80 have already been introduced at educational institutions and the remaining 20 were supposed to be introduced at the beginning of the academic year. Here it should be mentioned that after the state educational standards for qualifications are developed, the next step is to develop the modular programs and educational plans based on which educational institutions build their educational programs; these programs should be tested and, after that, implemented. Therefore, given that the occupations standards are still in the early stages of development, Armenia currently does not carry out competency based testing. To regulate the process of testing, the MoES developed a system for educational institutions to report on the process of testing. According to the procedure of reporting, the educational institutions submit a report on testing twice a year, at the end of each semester. After the educational standards and modular programs are introduced to educational institutions, they should be tested and, after that, implemented. Therefore, since the test results based on existing standards will be viewed in coming years, the development of competency based testing is still in its initial stage of development. In 2011, the government adopted the NQF and the schedule for its implementation. The NQF is described in official documents as an eight level Lifelong Learning Framework (with one being the lowest and eight the highest level) designed to be consistent with the European Qualifications Framework for Lifelong Learning. The objectives of the NQF are (i) to link different levels of qualifications (including general education) and educational degrees awarded in a hierarchy from the lowest to the highest; (ii) to link Armenian qualifications to those of other countries that have implemented a national qualifications framework; (iii) to enable learners and workers to access a qualification, transfer between qualifications, and progress to the next level, in the spirit of trust, mobility, and lifelong learning. Because the educational standards started being developed in 2009, whereas the NQF was adopted in 2011, most of the current educational standards should be reviewed and approved in accordance with the specialties and educational levels by December The National Center for Professional Education Quality Assurance Foundation (ANQA) is the agency under the government that establishes accreditation standards. ANQA was established in November 2008 and started its functioning in March The responsibility for running and maintaining ANQA falls under the MoES. ANQA conducts external quality assurance processes in accordance with the norms set by government legislation and regulations, as well as with the European Standards and Guidelines. ANQA also develops guidelines, criteria, and standards for quality assurance, taking into account the local needs and international good practice. The accreditation of middle and higher professional educational institutions only began in 2011, while preliminary vocational education institutions are not subject to accreditation. Accreditation will include institutional accreditation, which focuses primarily on the adequacy of infrastructure, and the accreditation of educational programs by profession/specialty. Institutional accreditation is mandatory for all educational institutions functioning in the territory of Armenia and must be renewed periodically. The accreditation of educational programs is voluntary and can be sought if the results of institutional accreditation are positive. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 23

24 To assure quality at the tertiary level of education, ANQA has developed a comprehensive Quality Assurance Framework, which revolves around four mechanisms: (i) an institutional audit; (ii) academic program accreditation; (iii) ANQA recognition; and (iv) information provision. This process is already underway in three state TVET colleges: Yerevan Regional State College 1, Yerevan Regional State College 2, and Yerevan State Armenian American "Erebuni" Medical College. In addition, in 2012 the ANQA started mandatory accreditation processes for three VET institutions (regional colleges): the three chosen VET institutions have already passed the first stage the stage of selfanalysis based on the specially developed form after which the process of accreditation will begin. Policy Goal 6: Diversifying Pathways for Skills Acquisition. In dynamic economic environments, workers need to acquire new skills and competencies as well as keep their skills up to date throughout their working lives. They are best served by a system of initial and continuing education and training that promotes lifelong learning by offering clear and flexible pathways for transfers across courses, progression to higher levels of training, and access to programs in other fields. For those already in the workforce, schemes for recognition of prior learning are essential to allow individuals to efficiently upgrade their existing skills and learn new ones. Policy Goal 6 examines the diversity of programs and ease of movement between them, whether or not the system facilitates skills upgrading by providing information on emerging trends and recognition of prior learning, and how well the system is able to adapt to changing skills demand. Armenia is rated at the Emerging (2.2 points) level on this policy goal, meaning that while pathways and recognition of prior learning are stated as priorities, the current system does not effectively address obstacles to lifelong learning related to the articulation of certification, recognition of informally acquired skills, or access to career guidance services. The learning pathway is quite diversified; graduates from middle school have a choice to attend high school or obtain a preliminary vocational education in colleges and academies that can provide secondary and professional education. Also, graduates of secondary vocational/technical streams can easily progress to post secondary levels of study. The portion of this Policy Goal focused on skills certification and recognition of prior learning lags other aspects of the Pathways goal and is accordingly rated quite low at the Latent level. This reflects both the fact that the AQF is still being developed and also the fact that while students within the formal education system enjoy open pathways for progression, there are vocational institutions under other ministries that do no issue certificates that are recognized by institutions under the MoES, and therefore such institutions are effectively dead ends. The system is being reformed and modernized, with most institutions being consolidated under the oversight of the MoES, something that should improve the portability of qualifications issued by these institutions. In addition, the issues pertaining to the recognition of prior learning have not been addressed sufficiently. The assessment and validation of non formal learning is in its very initial stages, and the concept and policy issues linked with this are just being discussed in government. Therefore, the NQF is expected to become a key reform instrument for improving quality in the national education and training system. To fulfill the agenda described above, the NQF should be revised and further developed. Although the government is strongly committed to providing lifelong learning, systematic career counseling has a long way to go. The absence of a unified system and clear definition of authorities and responsibilities for each agency, of age specific and need specific methods and instruments, and of a qualification and training system for service providers creates serious obstacles for effective organization and provision of continuing VET services. Though pathways and opportunities do exist for upgrade and acquiring skills, the public perception of TVET hinders using this opportunity, since it has traditionally been associated with low academic performance, limited social possibilities, and low skilled jobs. The creation of pathways into and out of the VET system, as well as steps to improve its quality, have helped to improve public perceptions of VET. However, despite the fact that public perception of the VET system is changing, it is still a serious challenge for VET institutions to become more attractive for enrollment. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 24

25 Implications of the Findings In Armenia, there exist a number of training providers, both state and non state, offering courses at various levels in diverse fields. One of the main deficiencies identified within Dimension 2 is that although a system of competency standards and accreditation mechanisms has been developed in consultation with key stakeholders, the competency based testing is not used for skilled and semi skilled occupations (it is mainly theory based), the certificates awarded have little impact on employment and earnings, and the accreditation standards are not sufficiently developed. Despite the fact that the public perception of the VET system is changing, it is still a serious challenge for VET institutions to become more attractive for enrollment. Though pathways and opportunities to upgrade and acquire new skills exist, the current public perception of TVET hinders using this opportunity, as it has traditionally been associated with low academic performance, limited social possibilities, and low skilled jobs. In a society marked by a very strong aspiration toward obtaining university degrees (regardless of their use in practical life or their value in the labor market), it is fundamental to take coherent action on multiple fronts to improve public perception of TVET. Moreover, to address negative perceptions on TVET, action must be taken to increase its stature among stakeholders on both the supply and demand side. Not only should the general population be better informed about the quality of vocational education and training provided and about available career paths/options, employers should also be aware of WfD s outcomes and play a major role in setting accreditation standards to overcome the gap in recognition from the standpoint of providers versus that of employers. The funding of the system raises the question of sustainability and equity. The government determines recurrent funding for VET institutions based largely on the previous year s budget. Public resources allocated for WfD are scarce and always competing with other socioeconomic policy priorities. Improving efficiency and sustaining an adequate level of public resources for WfD would benefit from a clear strategy and mechanisms for continuous monitoring and evaluation. Incentives should be introduced in the budget processes as a way to use financial mechanisms to stimulate better performance. A common characteristic of advanced WfD systems is regular assessments and timely reviews of the impact of funding on a range of training and labor market outcomes. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 25

26 6. Managing Service Delivery Training providers are the main channels through which the country s policies in workforce development (WfD) are translated into results on the ground. This chapter provides a brief overview of the composition of providers and the types of services available in the system before presenting the detailed SABER WfD findings on service delivery and their policy implications. It briefly summarizes the key features of service delivery, as reflected in data on (i) enrollment trends by level and type of training program initial vocational education and training (IVET), continuing vocational education and training (CVET), and active labor market programs (ALMPs) and by type of provider (state and non state); (ii) prevalence of firm based training; and (iii) the labor market outcomes of training. Overview of the delivery of training services The Ministry of Education and Science (MoES) provides the bulk of vocational training through different channels. The vocational track starts after the end of compulsory basic education (after grade 9). At this point, students attend high school, where they earn a certificate in Secondary General Education (Matura) after three years of study or attend a dedicated vocational institution. There are two levels of vocational education: preliminary and middle vocational education. Students may enter both directly after completion of compulsory education. Thus, these levels are differentiated by the level of skill possessed by graduates, not by the level of educational attainment required for admission. Accordingly, the time required to complete these levels varies. Preliminary courses last from six months to three years, and middle courses last from two to five years. Students studying in both preliminary and middle vocational education and training (VET) institutions may also obtain a Matura, and therefore are able to pursue higher education. General high schools can also offer a vocational stream, though it should be mentioned that not all high schools do due to the lack of required resources to run such programs. The educational attainment of the population in Armenia reveals a constant share of vocational and secondary specialized education of about 25 percent. Preliminary professional education is offered at 51 state institutions: 25 vocational craftsmanship schools, 20 middle technical colleges, four educational complexes and one university. At this level of TVET, professions in the services, trade, and food industry are the most popular ones. For economic reasons, the number of colleges has recently been reduced by from 101 in 2010 to 91 public and private middle VET in 2011 in an attempt to rationalize the system. There are 71 public middle vocational colleges, five universities providing middle VET, and 13 private middle VET colleges (which, however, are not allowed to issue a state diploma, something that has contributed to their decline in numbers in recent years). In , the number of preliminary/craftsmanship VET schools has decreased from 30 to 25, aiming at increasing efficiency in the system. In 2011, 51 public institutions provided preliminary professional (vocational) education (25 craftsmanship vocational schools, 20 middle vocational institutions providing preliminary education, four educational complexes, one university, and one foundation). Since December 2011, all 25 public preliminary (craftsmanship) vocational schools have been managed by the MoES. At this level of preliminary VET, the professions related to the spheres of services, trade, and the food industry are the most popular ones. Females enroll mainly in traditional fields like pedagogy, health, and the arts. Provision of Adult Learning. The legislative and political frameworks for adult education and learning are reflected in the number of legislative measures passed by the government. The Law on Education, adopted in 1999, in addition to establishing the basic framework, approaches, and principles of the management, financing, and methodology for the whole field of education, refers to the issue of additional education as well (Article 26). The government also adopted a strategic document in October 2009, titled The Concept of Lifelong Learning in Armenia, which lays out relevant principles and concepts related to lifelong learning, the main problems in the field, and possible solutions. However, investment in lifelong learning is extremely limited and the government provides support for lifelong learning mainly through its active employment measures organized by the State Employment Services Agency (SESA). Under the SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 26

27 EU Sector Policy Support Programmes 2007, a study was carried out in 2009 to assess the feasibility of establishing a National Training Fund to promote adult learning. The study was based on a review of existing documents and on interviews with local stakeholders, including social partners, public institutions, companies, school training providers, and employment centers. The study found that the annual offering of training per year includes: (i) employment services (provided to approximately 1,300 unemployed persons); (ii) short term courses in colleges (1,000 trainees); (iii) trade union organizations (1,000 people); (iv) NGOs (1,000 trainees); (v) large enterprises such as ArmenTel, Viva Cell, Grand Candy (1,000 people each); (vi) adult education organizations (300 people); (vii) international programs and projects (number of trainees not available, but the study reports that computer technologies and foreign languages are the first and second most subscribed training courses); and (viii) the Centre for Professional Orientation (approximately 1,000 people trained, free consultations provided). There are a few privately owned or managed training providers. Other forms of training include industry attachments (apprenticeships schemes) and in service training and re training of both employed and unemployed workers in the labor force. Information on this type of training is scarce. In general, the linkages between the education and training system and the labor market could be described as weak. There is a clear mismatch between the type and level of skills needed by the labor market and what is being supplied by the education and training system. A key trend is that while the WfD system is producing an increasingly highly educated workforce, graduates do not possess the skills required by the labor market, even though they are often lower level and attainable through shorter periods of education. SABER WfD Ratings on Service Delivery The policy goals for this dimension in the SABER WfD framework focus on the following three aspects of service delivery: Policy Goal 7: Incentivizing Diversity and Excellence in Training Provision Policy Goal 8: Fostering Relevance in Public Training Programs Policy Goal 9: Enhancing Accountability for Results. The ratings for these three Policy Goals are presented below and are followed by a reflection on their implications for policy dialogue. Based on data collected by the SABER WfD questionnaire, Armenia received an overall rating of 2.3 points (Emerging) for the Service Delivery Dimension (see Figure 9). This score is the average of the ratings for the underlying Policy Goals mentioned above. The explanation for these ratings and their implications follow below. Figure 9. SABER WfD Ratings of the Service Delivery Dimension Note: Key to Scoring: 1 = Latent; 2 = Emerging; 3 = Established; 4 = Advanced; see Figure 2 for definitions. Source: Authors analysis based on SABER WfD data. Policy Goal 7: Incentivizing Diversity and Excellence in Training Provision Because the demand for skills is impossible to predict with precision, having a diversity of providers is a feature of strong WfD systems. Among non state providers, the challenge is to temper the profit motive or other program agendas with appropriate regulation to assure quality and relevance of institutions. Among state providers, a key concern is their responsiveness to the demand for skills from employers and students. Striking the right balance between institutional autonomy and accountability is one approach to address this concern. Policy Goal 7 takes these ideas into account and benchmarks the system according to the extent to which policies and institutional arrangements are in place to: (i) encourage and regulate non state provision of training and (ii) foster excellence in public training provision by combining incentives and autonomy in the management of public institutions. Armenia is rated at the Emerging (2.1) level on this policy goal, reflecting the fact that there are active non state SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 27

28 providers in the training market despite almost no government incentives to encourage non state provision. While most of these providers are registered and licensed, few measures for quality assurance are in place. Regarding public training provision, the government grants considerable autonomy to the institutions. There are provisions for oversight of public institutions, but little is done to create appropriate incentives for performance. The government does not require public institutions to meet explicit performance targets or provide financial or non financial incentives for performance. In Armenia, training, re training, and re qualification requirements are fulfilled by both state and non state providers. Under the current arrangements, the nonstate providers do not benefit from government training grants, and incentives for entry do not exist. The Law on Licensing 16 does not prescribe any limitation with respect to organizations providing supplementary educational programs, which leaves open a free avenue for any organization that would like to be engaged in the training provision. Under these circumstances, issues such as preparedness of the educational methodological training courses and the availability of respective specialized staff and materials or technology do not have significant meaning; the only requirement for implementation is that the given organization s charter must establish the possibility of preparedness. The issue of quality in the preparation of learners/trainees is left completely to the conscience of the organization providing the training, because the process is not inspected by any organization. As a result, the exact number of organizations engaged in the business of training provision remains unclear. A number of governmental and non governmental organizations provide retraining and level raising courses for the specialists in their respective fields/sectors; their educational programs are provided upon the request of organizations and are aimed exclusively at raising the level of specialists knowledge and capacities to conform with the demands of their work functions. Therefore, for non state providers the main challenge is to bring the profit motive together with appropriate regulation in order to ensure the quality and relevance of training provision. Among state training providers, a key concern is their responsiveness to the demand for skills from employers and students. Aiming at increasing VET effectiveness and relevance to the regional and local level needs, the VET governing boards (VETGB) were established in all VET colleges in accordance with the law adopted in 2005 (it took three years to establish them, with lengthy discussion about representation). According to the law, a governing board is established for a period of five years and has representatives from pedagogical staff, students, and the public administration body handling education. It has the power to approve budgets, strategic programs, and tuition fees, as well as the power to elect a director. Starting in 2011, the membership of the governing boards of the regional colleges was extended by involving new representatives of social partners and regional employment services. Changes were also introduced into the management boards of all VET schools by involving more representatives of social partners, local authorities, NGOs, and regional employment services in order to promote cooperation within the community and increase the relevance of education within the community context. In 2009 the document Concept on Social Partnership in the Field of Preliminary (Craftsmanship) and Middle Vocational Education was endorsed by the government and, based on this, a Memorandum of Understanding and Cooperation in the field of VET was signed by the MoES and social partners in the same year. The aim of the memorandum was to improve cooperation and support for the future development of VET as a modernized system that provides a qualified labor force suited to the requirements of employers. The amount of effective control these boards exercise in practice, however, is questionable, as the existing measures for accountability are not sufficient and those measures that exist are not being adequately enforced. The government grants VETGBs considerable autonomy in personnel, equipment, income generating activities, and program offerings. While the VETGBs in principle exercise control over the operations of schools, the VETGBs do not set explicit performance targets, meaning that while VETGBs can exercise considerable control over inputs, there is little accountability for outcomes achieved. The provision of financial or non financial 16 The Law of the Republic of Armenia on Licensing, SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 28

29 incentives for performance are weak, both at the level of the VETSBs and at individual institutions. Policy Goal 8: Fostering Relevance in Public Training Programs In order to keep their program offerings relevant to current market conditions, public training institutions need reliable information on current and emerging skills needs. As such, these training institutions would benefit from establishing and maintaining relationships with employers, industry associations, and research institutions. Such partnerships are a potential source for institutions to receive information about skills competencies and expertise, as well as advice on curriculum design and technical specifications for training facilities and equipment. They could also help create opportunities for workplace training for students and continuing professional development for instructors and administrators. Policy Goal 8 considers the extent to which arrangements are in place for public training providers to: (i) benefit from industry and expert input in the design of programs and (ii) recruit administrators and instructors with relevant qualifications and support their professional development. Armenia is rated at the Emerging (2.3) level for this policy goal, reflecting the fact that informal and sporadic links exist between public training institutions and industry, with limited involvement of industry in curriculum design and the specification of standards for training facilities. Links between public training providers and research institutions are rare. The process of connecting labor market supply and demand, businesses, the economy, and educational institutions has begun and strong progress has been made, starting in 2008 with the restructuring of the VET Department in the MoES and establishment of the new division of VET Policymaking and Strategy. The implementation of VET reform came under the responsibility of the National Council for VET Development (NCVD) and has included significant tasks in revising curricula, developing of learning material, and preparing and providing teacher and staff training within the system. The NCVD was established in 2008 as a tripartite consultative body that includes an equal number (seven each) of representatives from the three parties (public, employers, trade unions) and ensures the achievement of social partnership in the VET sector at the national level. It meets on a regular basis and has two main objectives: (i) to promote VET system reforms and assist in further developing the VET sphere; and (ii) to provide consultancy to the MoES on decision making, design, and performance of development programs related to vocational education and training policy. Recently, NCVD made the decision to reorganize the 13 existing sectoral committees into 15. The role of the sectoral committees is to review the educational standards and curricula as developed by the National Center for TVET Development (NCVETD). 17 New and upgraded channels to link supply and demand for qualified labor and to establish and strengthen relationships with institutions are currently being utilized. The NCVETD, sectoral committees for educational standards, and boards of vocational education institutions have all played a part in this process. Despite the fact that the institutional mechanisms exist to ensure effective links between the training institutions and industry, in practice, industry s involvement at the level of educational institutions is mostly limited to providing internship opportunities. The NCVETD is in charge of, and periodically organizes, trainings for heads and deputies of educational institutions, lecturers, and training masters. For example, since the concept of modular education 18 is new in Armenia, the NCVETD organized a training series in 2010 for about 183 heads and deputies of educational institutions, as well as 2,544 lecturers and training masters. Heads of education institutions are encouraged to participate in different seminars, workshops, and conferences, especially when they are organized in Armenia. The above mentioned activities are implemented through a capacity building approach, aimed at improving the knowledge, understanding, and 17 Armenia, VET TA Program, Quarterly Progress Report, 2nd quarter In Armenia, as the VET system is being developed and gradually implemented, capacity or results oriented educational criteria are being applied. This is an important but not sufficient factor for the VET system to prepare skilled, qualified, and competitive personnel to meet labor market demand. The next important challenge for the VET institutions is to be able to meet the requirements set by the above mentioned criteria. Before the introduction of the new state education standards, educational programs were developed based on the subjects, while now instead of subjects, especially in vocational training groups, the modules are being taken into consideration. The difference between the two approaches is that now the trainee is required to be able to perform an action instead of only having some knowledge about the action. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 29

30 consensus on VET reform objectives from the beneficiaries side. In 2009, the Concept on Social Partnership in the Field of Preliminary (Craftsmanship) and Middle Vocational Education was endorsed by the government and, based on this, a Memorandum of Understanding and Cooperation in the field of VET was signed by the MoES and social partners in the same year. The aim of the memorandum is to improve cooperation and support for the future development of VET as a modernized system that provides a qualified labor force suited to the requirements of employers. Policy Goal 9: Enhancing Accountability for Results Systematic monitoring and evaluation of service delivery mechanisms and processes is valuable for both quality assurance and ongoing system improvement. Accomplishing these objectives requires gathering and analyzing data from a variety of sources. The reporting of institution level data enables the relevant authorities to ensure that providers are delivering on expected outcomes. Such data also enable these authorities to identify gaps or challenges in training provision or in areas of good practice. Additionally, periodic surveys and evaluations of major programs generate complementary information that can help enhance the relevance and efficiency of the system as a whole. Policy Goal 9 considers these ideas when assessing the system s arrangements for collecting and using data to focus attention on training outcomes, efficiency, and innovation in service delivery. Armenia is rated at the Emerging (2.3) level on this policy goal, reflecting the fact that all training providers are required to collect and report basic administrative data. The public training providers produce an annual report as a necessary part of their intradepartmental accountability requirements. These reports are for internal use only and are submitted to the MoES, where they are then consolidated into one centralized database that is publically available and systematically used for analysis of system inputs by the MoES. This data is occasionally used by different stakeholders to assess institutional performance as well as to analyze systemlevel trends and issues. The development and implementation of the VET management information system is ongoing, and it is expected to improve the data collection process and produce the relevant information on the educational system. According to Armenia s Law on State Statistics, 19 all educational institutions report administrative measurements (enrollments, staffing, budgets, etc.) and educational attainment, as well as graduation data, to the National Statistical Service (NSS), where the data is summarized and published. Besides this, the public training providers produce and submit to the MoES an annual report as a necessary part of intradepartmental accountability. While basic administrative data on both public and private service provision is widely available, there is less information on labor market outcomes as attempts to measure this are limited to a few ad hoc skills related surveys or evaluations of specifically targeted programs. Thus, even though there is some evidence of data collection, there is no system for feedback to improve skills, develop policies, or deliver services. Occasional or ad hoc tracer studies, conducted in the context of projects financed by development partners, do not adequately address or overcome this challenge. Such studies are carried out in collaboration with the MoES and NCVETD, but funding often comes from donors. This suggests that one impediment to creating a more robust information base one useful as a basis for decisions about systemlevel performance is inadequate funding. Implications of the Findings To build a coherent and well functioning VET system requires shifting away from government defined programs towards a well regulated market of private and public providers that deliver training services to both working and unemployed adults. In general, successful systems require a high degree of coordination and partnership among government agencies and the private sector, in addition to ensuring that the demand side of training (businesses and individuals) has a strong voice in the determination of training policy. Therefore, it is important to continue with the system reforms that aim to increase the quality of professional education and to bring them into compliance with labor market demands as well as the strategic educational and development objectives of the country. To address this concern, one 19 Law on State Statistics, Charter 1, Article 2, Sphere of Law Activity. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 30

31 approach would be to strike the right balance between institutional autonomy and accountability. The findings for Dimension 3 reveal that although the training provision is quite diversified, the effectiveness and relevance of training provisions are a challenge. The government does not provide adequate financial and non financial incentives to foster non state training provision. In addition, the measures to assure quality of training provision by training institutions are limited and require further improvement. For a training institution to be able to respond to local needs, decentralization and the provision of adequate autonomy are necessary preconditions. Despite the fact that the VETGBs are granted considerable autonomy, their ability to set explicit performance targets and become accountable for outcomes is a challenge. The key concern today is the responsiveness of training institutions, particularly government institutions, to the demand for skills from employers and students. The integration of industry and expert input into the design and delivery of public training programs is high on the agenda now. Effective monitoring and evaluation systems require appropriate tools for measuring quality and performance, and robust information systems. The availability and use of policy relevant data is very limited. The responsiveness of public training institutions to the demand for skills is in part hindered by a lack of incentives for results. In turn, performance targets for public training providers and corresponding incentives for meeting those targets are currently underutilized, partly because of the lack of appropriate feedback information. Therefore, a more systematic approach to collecting data on training outcomes is necessary (covering not only enrollment or graduation data, but also employer and trainee satisfaction, job placement rates based on accurate tracer studies, etc.). A more systematic approach will also help ensure that policymaking and state budget funding are evidencebased. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 31

32 Annex 1: Acronyms ADS ALMP ANQA CAPS CCI CRRC CTVET EC EDMC EIF ETF GD Fund ITVET LFS MLSI MoE MoES MoF MTEF NCVETD NCVD NGO NIE NILSR NQF NSS PPP PRSP SESA SMEs TVET UNDP USAID VETGB Armenia Development Strategy Active Labor Market Programs National Center for Professional Education Quality Assurance Foundation Competitive Armenia Private Sector Chamber of Commerce and Industry Caucasus Research Resource Centers Continuing Technical and Vocational Education and Training European Commission Enterprise Development and Market Competitiveness (USAID) Enterprise Incubator Foundation European Training Foundation Global Developments Fund Initial Technical and Vocational Education and Training Labor Force Survey Ministry of Labor and Social Issues Ministry of Economy Ministry of Education and Science Ministry of Finance Midterm Expenditure Framework National Center for VET Development National Council for VET Development non governmental organization National Institute of Education National Institute for Labor and Social Research National Qualification Framework National Statistical Service public private partnership poverty reduction strategy paper State Employment Services Agency small and medium enterprises technical and vocational education and training United Nations Development Program United States Agency for International Development VET Governing Board SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 32

33 Annex 2: The SABER WfD Analytical Framework Policy Goal Policy Action Topic in DCI 2.5 FINAL G1 Setting a Strategic Direction Provide sustained advocacy for WfD at the top leadership level G1_T1 G1_T2 Advocacy for WfD to Support Economic Development Strategic Focus and Decisions by the WfD Champions Dimension 1 Strategic Framework G2 G3 Fostering a Demand Led Approach Strengthening Critical Coordination Establish clarity on the demand for skills and areas of critical constraint Engage employers in setting WfD priorities and in enhancing skills upgrading for workers Formalize key WfD roles for coordinated action on strategic priorities G2_T1 G2_T2 G2_T3 G2_T4 G2_T5 G3_T1 G3_T2 G3_T3 Overall Assessment of Economic Prospects and Skills Implications Critical Skills Constraints in Priority Economic Sectors Role of Employers and Industry Skills Upgrading Incentives for Employers Monitoring of the Incentive Programs Roles of Government Ministries and Agencies Roles of Non Government WfD Stakeholders Coordination for the Implementation of Strategic WfD Measures G4_T1 Overview of Funding for WfD G4 Ensuring Efficiency and Equity in Funding Provide stable funding for effective programs in initial, continuing and targeted vocational education and training G4_T2 G4_T3 G4_T4 Recurrent Funding for Initial Vocational Education and Training (IVET) Recurrent Funding for Continuing Vocational Education and Training Programs (CVET) Recurrent Funding for Training related Active Labor Market Programs (ALMPs) Monitor and enhance equity in funding for training G4_T5 Equity in Funding for Training Programs Facilitate sustained partnerships between training institutions and employers G4_T6 Partnerships between Training Providers and Employers Dimension 2 System Oversight G5 Assuring Relevant and Reliable Standards Broaden the scope of competency standards as a basis for developing qualifications frameworks Establish protocols for assuring the credibility of skills testing and certification Develop and enforce accreditation standards for maintaining the quality of training provision G5_T1 G5_T2 G5_T3 G5_T4 G5_T5 G5_T6 G5_T7 G5_T8 Competency Standards and National Qualifications Frameworks Competency Standards for Major Occupations Occupational Skills Testing Skills Testing and Certification Skills Testing for Major Occupations Government Oversight of Accreditation Establishment of Accreditation Standards Accreditation Requirements and Enforcement of Accreditation Standards G5_T9 Incentives and Support for Accreditation Promote educational progression and permeability through multiple pathways, including for TVET students G6_T1 G6_T2 Learning Pathways Public Perception of Pathways for TVET G6 Diversifying Pathways for Skills Acquisition Facilitate life long learning through articulation of skills certification and recognition of prior learning Provide support services for skills acquisition by workers, job seekers and the disadvantaged G6_T3 G6_T4 G6_T5 G6_T6 Articulation of Skills Certification Recognition of Prior Learning Support for Further Occupational and Career Development Training related Provision of Services for the Disadvantaged SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 33

34 G7_T1 Scope and Formality of Non State Training Provision G7 Enabling Diversity and Excellence in Training Provision Encourage and regulate non state provision of training Combine incentives and autonomy in the management of public training institutions G7_T2 G7_T3 G7_T4 G7_T5 G7_T6 Incentives for Non State Providers Quality Assurance of Non State Training Provision Review of Policies towards Non State Training Provision Targets and Incentives for Public Training Institutions Autonomy and Accountability of Public Training Institutions Dimension 3 Service Delivery G8 Fostering Relevance in Public Training Programs Integrate industry and expert input into the design and delivery of public training programs Recruit and support administrators and instructors for enhancing the market relevance of public training programs G7_T7 G8_T1 G8_T2 G8_T3 G8_T4 G8_T5 G8_T6 Introduction and Closure of Public Training Programs Links between Training Institutions and Industry Industry Role in the Design of Program Curricula Industry Role in the Specification of Facility Standards Links between Training and Research Institutions Recruitment and In Service Training of Heads of Public Training Institutions Recruitment and In Service Training of Instructors of Public Training Institutions G9 Enhancing Evidence based Accountability for Results Expand the availability and use of policy relevant data for focusing providers' attention on training outcomes, efficiency and innovation G9_T1 G9_T2 G9_T3 Administrative Data from Training Providers Survey and Other Data Use of Data to Monitor and Improve Program and System Performance SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 34

35 Annex 3: Rubrics for Scoring the SABER WfD Data Policy Goal G1: Setting a Strategic Direction for WfD Functional Dimension 1: Strategic Framework Level of Development Latent Emerging Established Advanced Visible champions for WfD are either absent or take no specific action to advance strategic WfD priorities. Some visible champions provide adhoc advocacy for WfD and have acted on few interventions to advance strategic WfD priorities; no arrangements exist to monitor and review implementation progress. Government leaders exercise sustained advocacy for WfD with occasional, ad-hoc participation from non-government leaders; their advocacy focuses on selected industries or economic sectors and manifests itself through a range of specific interventions; implementation progress is monitored, albeit through ad-hoc reviews. Both government and nongovernment leaders exercise sustained advocacy for WfD, and rely on routine, institutionalized processes to collaborate on wellintegrated interventions to advance a strategic, economy-wide WfD policy agenda; implementation progress is monitored and reviewed through routine, institutionalized processes. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 35

36 Policy Goal Functional Dimension 1: Strategic Framework Level of Development Latent Emerging Established Advanced G2: Fostering a Demand-Led Approach to WfD There is no assessment of the country's economic prospects and their implications for skills; industry and employers have a limited or no role in defining strategic WfD priorities and receive limited support from the government for skills upgrading. Some ad-hoc assessments exist on the country's economic prospects and their implications for skills; some measures are taken to address critical skills constraints (e.g., incentives for skills upgrading by employers); the government makes limited efforts to engage employers as strategic partners in WfD. Routine assessments based on multiple data sources exist on the country's economic prospects and their implications for skills; a wide range of measures with broad coverage are taken to address critical skills constraints; the government recognizes employers as strategic partners in WfD, formalizes their role, and provides support for skills upgrading through incentive schemes that are reviewed and adjusted. A rich array of routine and robust assessments by multiple stakeholders exists on the country's economic prospects and their implications for skills; the information provides a basis for a wide range of measures with broad coverage that address critical skills constraints; the government recognizes employers as strategic partners in WfD, formalizes their role, and provides support for skills upgrading through incentives, including some form of a levy-grant scheme, that are systematically reviewed for impact and adjusted accordingly. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 36

37 Functional Dimension 1: Strategic Framework Policy Goal G3: Strengthening Critical Coordination for Implementation Level of Development Latent Emerging Established Advanced Industry/employers have a limited or no role in defining strategic WfD priorities; the government either provides no incentives to encourage skills upgrading by employers or conducts no reviews of such incentive programs. Industry/employers help define WfD priorities on an ad-hoc basis and make limited contributions to address skills implications of major policy/investment decisions; the government provides some incentives for skills upgrading for formal and informal sector employers; if a levygrant scheme exists its coverage is limited; incentive programs are not systematically reviewed for impact. Industry/employers help define WfD priorities on a routine basis and make some contributions in selected areas to address the skills implications of major policy/investment decisions; the government provides a range of incentives for skills upgrading for all employers; a levy-grant scheme with broad coverage of formal sector employers exists; incentive programs are systematically reviewed and adjusted; an annual report on the levy-grant scheme is published with a time lag. Industry/employers help define WfD priorities on a routine basis and make significant contributions in multiple areas to address the skills implications of major policy/investment decisions; the government provides a range of incentives for skills upgrading for all employers; a levy-grant scheme with comprehensive coverage of formal sector employers exists; incentive programs to encourage skills upgrading are systematically reviewed for impact on skills and productivity and are adjusted accordingly; an annual report on the levy-grant scheme is published in a timely fashion. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 37

38 Functional Dimension 2: System Oversight Policy Goal G4: Ensuring Efficiency and Equity in Funding Level of Development Latent Emerging Established Advanced The government funds IVET, CVET and ALMPs (but not OJT in SMEs) based on ad-hoc budgeting processes, but takes no action to facilitate formal partnerships between training providers and employers; the impact of funding on the beneficiaries of training programs has not been recently reviewed. The government funds IVET, CVET (including OJT in SMEs) and ALMPs; funding for IVET and CVET follows routine budgeting processes involving only government officials with allocations determined largely by the previous year's budget; funding for ALMPs is decided by government officials on an ad-hoc basis and targets select population groups through various channels; the government takes some action to facilitate formal partnerships between individual training providers and employers; recent reviews considered the impact of funding on only training-related indicators (e.g. enrollment, completion), which stimulated dialogue among some WfD stakeholders. The government funds IVET, CVET (including OJT in SMEs) and ALMPs; funding for IVET is routine and based on multiple criteria, including evidence of program effectiveness; recurrent funding for CVET relies on formal processes with input from key stakeholders and annual reporting with a lag; funding for ALMPs is determined through a systematic process with input from key stakeholders; ALMPs target diverse population groups through various channels and are reviewed for impact but follow-up is limited; the government takes action to facilitate formal partnerships between training providers and employers at multiple levels (institutional and systemic); recent reviews considered the impact of funding on both training-related indicators and labor market outcomes; the reviews stimulated dialogue among WfD stakeholders and some recommendations were implemented. The government funds IVET, CVET (including OJT in SMEs) and ALMPs; funding for IVET is routine and based on comprehensive criteria, including evidence of program effectiveness, that are routinely reviewed and adjusted; recurrent funding for CVET relies on formal processes with input from key stakeholders and timely annual reporting; funding for ALMPs is determined through a systematic process with input from key stakeholders; ALMPs target diverse population groups through various channels and are reviewed for impact and adjusted accordingly; the government takes action to facilitate formal partnerships between training providers and employers at all levels (institutional and systemic); recent reviews considered the impact of funding on a full range of trainingrelated indicators and labor market outcomes; the reviews stimulated broad-based dialogue among WfD stakeholders and key recommendations were implemented. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 38

39 Functional Dimension 2: System Oversight Policy Goal G5: Assuring Relevant and Reliable Standards Level of Development Latent Emerging Established Advanced Policy dialogue on competency standards and/or the NQF occurs on an ad-hoc basis with limited engagement of key stakeholders; competency standards have not been defined; skills testing for major occupations is mainly theory-based and certificates awarded are recognized by public sector employers only and have little impact on employment and earnings; no system is in place to establish accreditation standards. A few stakeholders engage in adhoc policy dialogue on competency standards and/or the NQF; competency standards exist for a few occupations and are used by some training providers in their programs; skills testing is competency-based for a few occupations but for the most part is mainly theory-based; certificates are recognized by public and some private sector employers but have little impact on employment and earnings; the accreditation of training providers is supervised by a dedicated office in the relevant ministry; private providers are required to be accredited, however accreditation standards are not consistently publicized or enforced; providers are offered some incentives to seek and retain accreditation. Numerous stakeholders engage in policy dialogue on competency standards and/or the NQF through institutionalized processes; competency standards exist for most occupations and are used by some training providers in their programs; the NQF, if in place, covers some occupations and a range of skill levels; skills testing for most occupations follows standard procedures, is competency-based and assesses both theoretical knowledge and practical skills; certificates are recognized by both public and private sector employers and may impact employment and earnings; the accreditation of training providers is supervised by a dedicated agency in the relevant ministry; the agency is responsible for defining accreditation standards with stakeholder input; standards are reviewed on an adhoc basis and are publicized or enforced to some extent; all providers receiving public funding must be accredited; providers are offered incentives and limited support to seek and retain accreditation. All key stakeholders engage in policy dialogue on competency standards and/or the NQF through institutionalized processes; competency standards exist for most occupations and are used by training providers in their programs; the NQF, if in place, covers most occupations and a wide range of skill levels; skills testing for most occupations follows standard procedures, is competency-based and assesses both theoretical knowledge and practical skills; robust protocols, including random audits, ensure the credibility of certification; certificates are valued by most employers and consistently improve employment prospects and earnings; the accreditation of training providers is supervised by a dedicated agency in the relevant ministry; the agency is responsible for defining accreditation standards in consultation with stakeholders; standards are reviewed following established protocols and are publicized and routinely enforced; all training providers are required as well as offered incentives and support to seek and retain accreditation. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 39

40 Policy Goal G6: Diversifying Pathways for Skills Acquisition Functional Dimension 2: System Oversight Level of Development Latent Emerging Established Advanced Students in technical and vocational education have few or no options for further formal skills acquisition beyond the secondary level and the government takes no action to improve public perception of TVET; certificates for technical and vocational programs are not recognized in the NQF; qualifications certified by non- Education ministries are not recognized by formal programs under the Ministry of Education; recognition of prior learning receives limited attention; the government provides practically no support for further occupational and career development, or training programs for disadvantaged populations. Students in technical and vocational education can only progress to vocationally-oriented, nonuniversity programs; the government takes limited action to improve public perception of TVET (e.g. diversifying learning pathways); some certificates for technical and vocational programs are recognized in the NQF; few qualifications certified by non-education ministries are recognized by formal programs under the Ministry of Education; policymakers pay some attention to the recognition of prior learning and provide the public with some information on the subject; the government offers limited services for further occupational and career development through stand-alone local service centers that are not integrated into a system; training programs for disadvantaged populations receive ad-hoc support. Students in technical and vocational education can progress to vocationallyoriented programs, including at the university level; the government takes some action to improve public perception of TVET (e.g. diversifying learning pathways and improving program quality) and reviews the impact of such efforts on an ad-hoc basis; most certificates for technical and vocational programs are recognized in the NQF; a large number of qualifications certified by non- Education ministries are recognized by formal programs under the Ministry of Education, albeit without the granting of credits; policymakers give some attention to the recognition of prior learning and provide the public with some information on the subject; a formal association of stakeholders provides dedicated attention to adult learning issues; the government offers limited services for further occupational and career development, which are available through an integrated network of centers; training programs for disadvantaged populations receive systematic support and are reviewed for impact on an adhoc basis. Students in technical and vocational education can progress to academically or vocationally-oriented programs, including at the university level; the government takes coherent action on multiple fronts to improve public perception of TVET (e.g. diversifying learning pathways and improving program quality and relevance, with the support of a media campaign) and routinely reviews and adjusts such efforts to maximize their impact; most certificates for technical and vocational programs are recognized in the NQF; a large number of qualifications certified by non-education ministries are recognized and granted credits by formal programs under the Ministry of Education; policymakers give sustained attention to the recognition of prior learning and provide the public with comprehensive information on the subject; a national organization of stakeholders provides dedicated attention to adult learning issues; the government offers a comprehensive menu of services for further occupational and career development, including online resources, which are available through an integrated network of centers; training programs for disadvantaged populations receive systematic support with multi-year budgets and are routinely reviewed for impact and adjusted accordingly. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 40

41 Policy Goal Functional Dimension 3: Service Delivery Level of Development Latent Emerging Established Advanced G7: Enabling Diversity and Excellence in Training Provision There is no diversity of training provision as the system is largely comprised of public providers with limited or no autonomy; training provision is not informed by formal assessment, stakeholder input or performance targets. There is some diversity in training provision; non-state providers operate with limited government incentives and governance over registration, licensing and quality assurance; public training is provided by institutions with some autonomy and informed by some assessment of implementation constraints, stakeholder input and basic targets. There is diversity in training provision; non-state training providers, some registered and licensed, operate within a range of government incentives, systematic quality assurance measures and routine reviews of government policies toward non-state training providers; public providers, mostly governed by management boards, have some autonomy; training provision is informed by formal analysis of implementation constraints, stakeholder input and basic targets; lagging providers receive support and exemplary institutions are rewarded. There is broad diversity in training provision; non-state training providers, most registered and licensed, operate with comprehensive government incentives, systematic quality assurance measures and routine review and adjustment of government policies toward non-state training providers; public providers, mostly governed by management boards, have significant autonomy; decisions about training provision are time-bound and informed by formal assessment of implementation constraints; stakeholder input and use of a variety of measures to incentivize performance include support, rewards and performancebased funding. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 41

42 Policy Goal Functional Dimension 3: Service Delivery Level of Development Latent Emerging Established Advanced G8: Fostering Relevance in Public Training Programs There are few or no attempts to foster relevance in public training programs through encouraging links between training institutions, industry and research institutions or through setting standards for the recruitment and training of heads and instructors in training institutions. Relevance of public training is enhanced through informal links between some training institutions, industry and research institutions, including input into the design of curricula and facility standards; heads and instructors are recruited on the basis of minimum academic standards and have limited opportunities for professional development. Relevance of public training is enhanced through formal links between some training institutions, industry and research institutions, leading to collaboration in several areas including but not limited to the design of curricula and facility standards; heads and instructors are recruited on the basis of minimum academic and professional standards and have regular access to opportunities for professional development. Relevance of public training is enhanced through formal links between most training institutions, industry and research institutions, leading to significant collaboration in a wide range of areas; heads and instructors are recruited on the basis of minimum academic and professional standards and have regular access to diverse opportunities for professional development, including industry attachments for instructors. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 42

43 Policy Goal Functional Dimension 3: Service Delivery Level of Development Latent Emerging Established Advanced G9: Enhancing Evidence-based Accountability for Results There are no specific data collection and reporting requirements, but training providers maintain their own databases; the government does not conduct or sponsor skills-related surveys or impact evaluations and rarely uses data to monitor and improve system performance. Training providers collect and report administrative data and there are significant gaps in reporting by nonstate providers; some public providers issue annual reports and the government occasionally sponsors or conducts skills-related surveys; the government does not consolidate data in a system-wide database and uses mostly administrative data to monitor and improve system performance; the government publishes information on graduate labor market outcomes for some training programs. Training providers collect and report administrative and other data (e.g., job placement statistics, earnings of graduates) and there are some gaps in reporting by non-state providers; most public providers issue internal annual reports and the government routinely sponsors skills-related surveys; the government consolidates data in a system-wide database and uses administrative data and information from surveys to monitor and improve system performance; the government publishes information on graduate labor market outcomes for numerous training programs. Training providers collect and report administrative and other data (e.g., job placement statistics, earnings of graduates) and there are few gaps in reporting by non-state providers; most public providers issue publicly available annual reports and the government routinely sponsors or conducts skills-related surveys and impact evaluations; the government consolidates data in a system-wide, up to date database and uses administrative data, information from surveys and impact evaluations to monitor and improve system performance; the government publishes information on graduate labor market outcomes for most training programs online. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 43

44 References Annex 4: References and Additional Sources AM Partners Consulting Company Skills and Knowledge Needs Assessment for Pharmaceutical Industry. Copublished by USAID CAPS Project and AM Partners Consulting Company. Arias, Omar and Sánchez Páramo Back to Work. Washington, DC: The World Bank. Armenia, Ministry of Economy Republic of Armenia, Export Led Industrial Policy Framework. Yerevan. Armenia, Ministry of Education and Science The Development and State of the Art of Adult Learning and Education National Report of the Republic of Armenia. Hamburg, Germany: UNESCO UIL. Armenia, Ministry of Education and Science, National Centre for VET Development Concept on Social Partnership in Preliminary (Craftsmanship) and Middle Vocational Education in Armenia. Yerevan: European Training Foundation and Global Developments Fund. Armenia, Ministry of Finance. Undated. Medium Term Public Expenditure Framework of the Republic of Armenia for Armenia, National Statistical Service Labor Market in the Republic of Armenia, Available at Armenia, Office of the Prime Minister The Concept of Lifelong Learning in the Republic of Armenia Extract From the Minutes of the RA Government Meeting, October 15, 2009 (No. 43). Yerevan. Armenia: Quality Assurance (QA) and National Qualifications Framework (NQF) PowerPoint presentation from the Tbilisi Regional Seminar on Quality Management in the Context of National Qualifications Framework, Tbilisi, November Avagyan, Aram Development and Introduction of National Qualification Framework in Armenia. Presentation sponsored by the Global Developments Fund AE & LLL Association, Armenia. Corradini, Milena Torino Process, 2010: Armenia. European Training Foundation. Dolgova Dreyer, Katia Presentation on Development of a National Qualification Framework in Armenia: Background and Cooperation with Council of Europe. Economy and Values (EV) Research Center and EV Consulting National Competitiveness Report of Armenia 2010: Higher education report. Yerevan National Competitiveness Report of Armenia: Agenda for Upgrading Management Practices. Yerevan. European Training Foundation (ETF) and GDF Data Base of the Donors and Projects in the Field of Vocational Education and Training in Armenia. Yerevan Support to Mobility Partnership Project Implementation Plan (Work Programme). Turin Tracer Study of Recent Graduates From Vocational Education Institutions in Kotayk and Ararat Marzes of Armenia. Draft Report. Turin: European Training Foundation & Caucasus Research Resource Centers Armenia Torino Process 2012: Armenia. Working paper. Turin. Global Developments Fund (GDF). Undated. Draft Report on the Analysis of the Adequacy of Competences Provided by VET System for Labour Market Requirements. Hergnyan, Manuk, Hovhannisyan, Sevak, Malumyan, Gohar, Atabekyan, Hovhannes, Arzumanyan, Ashot National Competitiveness Report of Armenia 2010: Higher Education Challenge (in English, Armenian). Yerevan, Armenia: Economy and Values Research Center. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 44

45 National Center for Professional Education Quality Assurance, Foundation (ANQA) ANQA Strategic Plan : Transitional Period. Yerevan, Armenia: _eng.pdf Prokopenko, Joseph The Role of Republican Union of Employers of Armenia in Promoting Youth Employment and Strengthening Professional Education Institutions. Geneva: ILO, ACT/EMP. Sargsyan, Arevik, Aram Manukyan, and Milena Corradini Educational Policies for Economic Competitiveness: The Case Study of Lori and Tavush Marzes. Yerevan: European Training Foundation and Global Developments Fund. Tan, Jee Peng, Kiong Hock Lee, Alexandria Valerio, and Robert McGough What Matters for Workforce Development: A Framework and Tool for Analysis. SABER Working Paper Series, No. 6. Washington, DC: The World Bank. WfD.pdf. USAID Armenia Armenian Engineering Workforce Survey. Yerevan. USAID Armenia, RA Ministry of Economy, and Enterprise Incubator Foundation Armenian Information Technology Sector: Software and Services, 2009 Industry Report. Yerevan. USAID CAPS Project Tourism Skills and Knowledge Needs Assessment. Yerevan. Valerio, Alexandria, Katia Herrera Sosa, Sebastian Monroy Taborda and Dandan Chen. Armenia Skills toward Employment and Productivity (STEP) Survey Findings (Urban areas). Washington, DC: The World Bank. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 45

46 List of Informants The analysis is based on the legal framework documents and sector studies and analysis conducted in recent years. During the study, 25 stakeholders were interviewed. These included experts, representatives, and decision makers from the Ministry of Education, Ministry of Labor and Social Issues, Ministry of Economy, educational institutions, public organizations, NGOs and civil society. Name Institution Position Astghik Mirzakhanyan Gov. of Armenia Head of Social Affairs Department Karine Harutyunyan Ministry of Education and Science Deputy Minister Robert Stepanyan Ministry of Education and Science Head of Development Programs and Monitoring Department Robert Abrahamyan Ministry of Education and Science Head of VET Department Zara Grigoryan Ministry of Education and Science Chief of Vocational Education and Training Policy and Strategy Division Tatevik Gasparyan The National Center of VET Development Director Anetta Babayan Ministry of Economy Economic Development Policy Department Tadevos Avetisyan Ministry of Labor and Social Issues Head of Labor and Employment Department Haykuhi Gevorgyan Youth Professional Orientation Center (YPOC) Director Karine Simonyan UNDP Armenia Vocational Education and Training (VET) System Project Coordinator Vahagn Hovhannisyan Chamber of Commerce and Industry of RA Head of Education and Training Department Bagrat Yengibaryan Enterprise Incubator Foundation Director Aram Avagyan Global Developments Fund Director Paruyr Santrosyan Yerevan Regional State College #1 Director Hrach Grigoryan Yerevan Regional State College #2 Director Hasmik Sahakyan, International Accounting Training Center Educational Fund Executive Director Tigran Tananyan USAID Armenia EDMC Project Deputy Team Leader, Workforce Development Component Lusine Kharatyan Adult Education Development Cooperation Head of Branch Office/National Coordinator Melik Gasparyan AVAG Solutions Ltd Deputy CEO Sevak Hovhannisyan EV Consulting Lusine Kalantaryan National Statistical Service (NSS) Head of Labour Statistics Division Nelly Baghdasaryan National Statistical Service (NSS) Head of Social Sphere and Nature Protection Division Artak Simonyan State Employment Services Agency Deputy head SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 46

47 Annex 5: SABER WfD Scores Dimension 1 Dimension 2 Dimension Policy Goal Policy Action Topic G1 2.0 Provide sustained advocacy for WfD at the top leadership level 2.0 G2 1.8 G3 1.3 G4 1.4 G5 2.1 G6 1.7 G7 2.3 G8 1.8 G9 1.7 Establish clarity on the demand for skills and areas of critical constraint Engage employers in setting WfD priorities and in enhancing skillsupgrading for workers Formalize key WfD roles for coordinated action on strategic priorities Provide stable funding for effective programs in initial, continuing and targeted vocational education and training Monitor and enhance equity in funding for training 1.0 Facilitate sustained partnerships between training institutions and employers Broaden the scope of competency standards as a basis for developing qualifications frameworks Establish protocols for assuring the credibility of skills testing and certification Develop and enforce accreditation standards for maintaining the quality of training provision Promote educational progression and permeability through multiple pathways, including for TVET students Strengthen the system for skills certification and recognition 1.5 Enhance support for skills acquisition by workers, job seekers and the disadvantaged Encourage and regulate non state provision of training 2.3 Combine incentives and autonomy in the management of public training institutions Integrate industry and expert input into the design and delivery of public training programs Recruit and support administrators and instructors for enhancing the market relevance of public training programs Expand the availability and use of policy relevant data for focusing providers' attention on training outcomes, efficiency and innovation G1_T1 2.0 G1_T2 2.0 G2_T1 2.0 G2_T2 2.0 G2_T3 1.0 G2_T4 2.0 G2_T5 2.0 G3_T1 2.0 G3_T2 1.0 G3_T3 1.0 G4_T1 info G4_T2 2.0 G4_T3 2.0 G4_T4 1.0 G4_T5_IVET 1.0 G4_T5_CVET 1.0 G4_T5_ALMP G4_T G5_T1 1.0 G5_T2 1.0 G5_T3 2.0 G5_T4 3.0 G5_T5 3.0 G5_T6 info G5_T7 2.0 G5_T8 3.0 G5_T9 2.0 G6_T1 2.0 G6_T2 1.0 G6_T3 1.0 G6_T4 2.0 G6_T5 3.0 G6_T6 1.0 G7_T1 3.0 G7_T2 2.0 G7_T3 3.0 G7_T4 1.0 G7_T5 2.5 G7_T6 2.5 G7_T7 2.0 G8_T1 3.0 G8_T2 1.5 G8_T3 1.0 G8_T4 1.0 G8_T5 2.0 G8_T6 2.0 G9_T1 2.0 G9_T2 1.5 G9_T3 1.5 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 47

48 Annex 6: Validation Workshop Report Congress Hotel, Yerevan, February 28, 2013 Number of Participants: 27 (including Caucasus Research Resource Centers staff and guest presenter M. Mulatu, World Bank) Divided into two groups, the participants discussions on the results from the SABER Workforce Development (WfD) data collection instrument resulted in the following: Participants didn t agree with about 45 percent of scores given to different topics, arguing that scores should ve been lower. In the group where non government representatives dominated, the share of scores to be lowered was higher. There were some areas where the scores given to the topics were found to be too low, such as the coordination of strategic policy towards WfD, the actions to assure the quality of education provided by nonstate institutions, and use of administrative data. Here, the participants said the scores could be a bit higher. The key message from the discussions was this: Although strategic documents and laws are in place and the direction of WfD is identified, what is written is not always and in most cases is not translating into adequate action. The participants made the following comments and suggestions: Comments: The strategic directions of WfD are formally identified, but actions to support the implementation are not sufficient and the score 3 is high for this topic. The leaders of WfD are formally identified, but none of them is really active enough to make a difference. Although the economic perspectives are well identified, the skills implication is not assessed at all: the government thinks that business does not share information openly, which makes it difficult to do an impact assessment. The NGO and international organizations representatives said the mechanisms for policy implementation, monitoring, and impact assessment are weak or even nonexistent. Although the economic development priorities are identified, the skills needed are not well. The role of government versus private sector is identified somewhat, but it has a more formal nature and works only at the higher national level. The employers representatives said employers shouldn t be engaged in identifying facility standards for training institutions. Although the state educational institutions, according to the current regulations, have autonomy, in practice its realization is not sufficient. Employers and their associations participation and cooperation are almost nonexistent at the tertiary level. Although it is institutionalized, the cooperation between employers and educational/training institutions and coordination is weak and in most cases based on personal relationships. One of the comments that was made by both working groups is that the first dimension should clearly separate the issue of strategy setting, which is generally reflected in documents and agenda, from the issue of decision making, which is likely to be quite different. At present, incentives for employers do not encourage them to help increase employee motivation. Suggestions and priorities: The policy implementation and impact assessment mechanisms development and realization is a priority. Reviewing the developed standards of occupations in accordance with the economic priorities is a priority, along with provision of incentives for employers to update skills. The development of other mechanisms of standards assessment is a priority, too. Check whether the existing/developed standards correspond to the priority directions of the economy. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 48

49 The role of social partnership should be increased, as well as the participation and role of employers in governing the educational institutions should be strengthened. One of the priorities and recommendations to be addressed is to include suggestions on how to address the sub national coordination, which is noted as an issue in the presentation. Financial resources allocated by the state budget are scarce and disconnected with performance of training institutions; therefore implementation of performance based financing is a priority, and the development and implementation of a monitoring and assessment mechanism to support the performance based financing is critical. Real autonomy for educational institutions is important; therefore the improvement of mechanisms to translate the law into action in this regard is a priority. Creating incentives for the real engagement of employers especially at the sub national level is an issue to be addressed. To summarize the findings from the workshop, we could state: The key message is that although the strategies of economic development are there and priorities have been identified, the assessment of skills needed for the economy in general and in priority sectors in particular is not sufficient, if not nonexistent. The coordination between stakeholders and government agencies requires further improvement and strengthening. The main issue is the discrepancy between the declared intentions for WfD and the decision making and actual action. As the latter is the weakest link in the Armenian reality, the scores should be lower. There is disagreement with many of the scores, with the group suggesting downgrading in almost all cases (with a couple of exceptions), but this is improved a bit by a discussion of the rubrics by thematic area. There appears to be general agreement that the weight given to theoretical and documented intentions is greater than what is rated with regard to actual practice. Education standards evaluation is being done only at the end of cycles of training for VET, while the best option would be to do evaluations mid term (so that corrective action can be taken before they graduate). SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 49

50 Annex 7: Authorship and Acknowledgements This report is the product of collaboration between Susanna Karapetyan and Heghine Manasyan of the CRRC Armenia Center and staff at the World Bank, comprising Meskerem Mulatu, Dandan Chen as well as Jee Peng Tan, Ryan Flynn and Sankalpa Dashrath, leader and members, respectively, of the SABER WfD team based in the Education Global Practice. Susanna Karapetyan and Heghine Manasyan collected the data using the SABER WfD data collection instrument, prepared initial drafts of the report, and finalized the report; the World Bank team scored the data, designed the template for the report, and made substantive contributions to the final write up. Thanks and appreciations also go to Andrew Loizeaux for editing the initial report, Anna Olifir and Xiaoyan Liang of the World Bank for their useful comments, as well as to those colleagues from CRRC Armenia and different local and international organizations and institutions in Armenia, who assisted in the project. The research team acknowledges the support of all who have contributed to the report and its findings, including informants, survey respondents, participants at various consultation workshops, as well as other members of the SABER WfD team at the World Bank: Rita Costa, Viviana Gomez, Rijak Grover, Kiong Hock Lee, Joy Yoo Jeung Nam, Brent Parton, and Alexandria Valerio. The research team gratefully acknowledges the generous financial support of the Government of the United Kingdom through its Department of International Development s Partnership for Education Development with the World Bank, which makes it possible for the SABER WfD team to provide technical support to the principal investigators in the form of standardized tools for and guidance on data collection, analysis, and reporting. The team also acknowledges the contribution by the World Bank and the Government of Armenia in supporting the countryspecific aspects of this research. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 50

51 The Systems Approach for Better Education Results (SABER) initiative produces comparative data and knowledge on education policies and institutions, with the aim of helping countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence based global standards, using new diagnostic tools and detailed policy data. The SABER country reports give all parties with a stake in educational results from administrators, teachers, and parents to policymakers and business people an accessible, objective snapshot showing how well the policies of their country's education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of Workforce Development. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 51