Vida Beresnevičiūtė Arūnas Poviliūnas Rūta Žiliukaitė. Guidelines for research of a field of professional activity

Transcription

1 Vida Beresnevičiūtė Arūnas Poviliūnas Rūta Žiliukaitė Guidelines for research of a field of professional activity

2 Development of the Concept of the European Credit Transfer and Accumulation System (ects) at the National Level: Harmonization of the Credit and Implementation of the Learning Outcomes Based Study Programme Design (Vp1-2.2-Šmm-08-v ) Vida Beresnevičiūtė Arūnas Poviliūnas Rūta Žiliukaitė Guidelines for research of a field of professional activity Vilnius 2012

3 Approved by the Council of the Faculty of Philosophy, Vilnius university, April 25, 2012 Vida Beresnevičiūtė Arūnas Poviliūnas Rūta Žiliukaitė Guidelines for research of a field of professional activity Reviewed by dr. Almeda Kurienė (Lithuanian University of Educational Sciences) dr. Vidmantas Tūtlys (Vytautas Magnus University) Vilniaus universitetas, 2012 ISBN

4 Table of Contents Introduction Team of PROFESSIONAL ACTIVITies FIELD research. BRIEF SUMMARY OF STAGES OF THE MAKING LISTS OF SUBJECT-SPECIFIC AND GENERIC COMPETENCES SURVEY OF EMPLOYERS FOCUS GROUPS WITH GRADUATES...21

5 Introduction When designing new degree programmes, higher education institutions justify the need for such programmes by labour market demands and by employers certification that their companies and institutions need specialists with certain competences, which shall be developed by the new degree programme. Designers of new degree programmes often rely on the research of a field of professional activities of a wider or lesser extent, or that are more or less reliable. However, such study is not a common practice when upgrading degree programmes. Is the research of professional activity necessary? And how can it be useful when designing or upgrading degree programmes? The main goal of research of professional activities is to define subject-specific and generic competences as a combination of knowledge, skills, capacities, values and attitudes that a specialist of a certain field must have in order to properly perform activities 1. Such list of subjectspecific and generic competences, which is developed by integrating professional activity field experience rather than based on the unilateral assessment of professional activity needs given by degree programme designers, forms the basis for the definition of learning outcomes. Learning outcomes are statements specifying the level of competences that graduates should have upon the completion of the programme, i.e. what they should know and understand and what skills and capacities they should have upon graduation from the degree programme 2. When designing degree programmes, research of the field of professional activities provides an opportunity to move from a more traditional approach, which emphasises knowledge and content related to the preferences of professors of higher education institutions, to the approach of the European Credit Transfer and Accumulation System (ECTS) and the project of Tuning Educational Structures in Europe (hereinafter referred to as the Tuning project), which emphasises the student-centred approach and the development of students competences in order to provide the graduates of the degree programme with the maximum opportunities to integrate into the labour market. Research of the field of professional activities studies professional activity from the point of view of social relations. It means that the characteristics of professional activity also depend on the place the person performing professional activities occupies with regard to social relations. Research of professional activities covers not only certain positions but also the rules regulating professional activities, which can be formalised and non-formalised. Such rules are more or less visible compromises among the key players of the professional activity field: employers, representatives of education institutions and graduates. Therefore, the research of the field of professional activities should first of all be considered as a tool of a constructive and productive dialogue among higher education institutions, employers and graduates. Unsystematically gathered information about the graduates satisfaction with the completed degree programme or about the employers opinion regarding which knowledge, skills and capacities the young specialists lack upon their graduation from certain degree programmes 1 Taking into consideration the existing variety of the definitions of the competence s term, it is important to indicate that hereby the definition of competence provided by TUNING methodology is used: competences and skills are understood as including knowing and understanding (theoretical knowledge of an academic field, the capacity to know and understand), knowing how to act (practical and operational application of knowledge to certain situations), knowing how to be (values as an integral element of the way of perceiving and living with others and in a social context). Competences represent a combination of attributes (with respect to knowledge and its application, attitudes, skills and responsibilities) that describe the level or degree to which a person is capable of performing them. (Tuning, 2003, p. 68). There are two types of competences: competences which are subject-area related, they are intimately related to specific knowledge of a field of study [...] these give identity and consistency to the particular degree programme and generic competences which could be general to any degree (ibid). 2 Ibid. 4

6 will hardly provide the designers of degree programmes with sufficient precise knowledge how to amend degree programmes in order for them to meet the needs of the professional activity field. Systematic surveys of employers and graduates performed applying the methods of social research may provide with reliable information about professional capacities of graduates and prevent the designers of degree programmes from making mistakes resulting from the lack of such information. On the other hand, if the research of the field of professional activities is viewed as a tool of striking up a dialogue with representatives of the professional activity field, it is important to recognise not only the importance of the input of employers and graduates when designing and upgrading degree programmes, but also the role of teaching staff of higher education institutions in charge of such degree programmes. Not all expectations of employers and graduates are necessarily reasonable and not all of them have to be taken into consideration; therefore, professional activity research systematically performed by designers of degree programmes may be an instrument that helps these social partners understand what a higher education institution can achieve and what it cannot achieve while training specialist of a certain qualification for the labour market, and what cooperation is needed between employers and higher education institutions in order to get the most optimal outcome of studies. Guidelines of Research of the Field of Professional Activities (hereinafter referred to as the Guidelines) have been prepared during the implementation of national project Development of the Concept of the European Credit Transfer and Accumulation System (ECTS) at the National Level: Harmonisation of the Credit and Implementation of the Learning Outcomes Based Study Programme Design (No. VP1-2.2-ŠM-08-V ) (hereinafter referred to as the ECTS project). The implementation of the ECTS project included the conducting of the research of professional activities the goal of which was to perform the analysis of the professional activity field of the eight study areas stated in the project in order to provide important information to project participants who upgrade degree programmes. Evaluating the benefits of the research for upgrading degree programmes while summarising the research results, teachers of various higher education institutions who participated in the research of the field of professional activities under the ECTS project indicated the following aspects 3 : The research of the field of professional activities gives an insight into a degree programme as a whole and facilitates better assessment of the objectives of the degree programme. The research of the field of professional activities provides strong arguments for upgrading degree programmes: changes in a degree programme (e.g. an updated content of the degree programme, inclusion or non-inclusion of course units in the degree programme, etc.) are made on the basis of systematically collected data; in other words, the research of the field of professional activities facilitates critical evaluation and going beyond the personal experience of the authors of degree programmes. The results of the research of the field of professional activities provide information on the professional activities of graduates and facilitate detailed evaluation of the contribution of studies to professional activities. 3 Two focus groups with the teachers of higher education institutions who participated in the research of the field of professional activities (members of study committees) were organised when summarising the research results. The group discussion with the teachers who participated in the research of the field of professional activities was organised and conducted by Vaida Dagytė, a member of the professional field research group. 5

7 The results of the research of the field of professional activities contribute to the integration of the intended learning outcomes of course units in the objectives of an entire degree programme. The results of the research of the field of professional activities provide arguments for evaluating the importance of the development of relevant competences or, vice versa, facilitate the determination of competences that are not important to the future professional activities of a student. 6

8 1. Team of PROFESSIONAL ACTIVITies FIELD research. BRIEF SUMMARY OF STAGES OF THE Professional activity field team When performing a research of the field of professional activities, it is recommended to have a team consisting of experts and specialists of diverse professional experience. In the ECTS project, the study team consisted of the following: professional field research experts in charge of the development of the professional field research methods, the coordination of the project, the analysis of the study data and the reporting; professional field researchers who can gather and generalize information and materials required for the research, and draw up reports; professional field researchers representatives of subject area groups of the ECTS project (hereinafter referred to as specialised researchers) who can prepare the digest of positions held by graduates from degree programmes, also document digests, the list of information sources on professional activity and competences required for such professional activity, and compile initial lists of competences, analyse the research data, provide contact information on graduates who completed differed levels of studies, etc.; professional field representatives employers who participate whether directly or indirectly in the designing or consideration of degree programmes, or who are otherwise familiar with the study process and can represent a certain professional group (professional field), programmes of respective study areas, and who can be involved in the compilation of initial lists of competences and the analysis of the study data. Representatives of the above four groups may have different functions in various stages of the research, to share different responsibilities with due consideration to the needs of the study, available sources, etc. Brief summary of stages of the study The study of professional activity is carried out in stages, each of which can be an independent research in itself or which can be coordinated according to the need and possibility. The following key stages of the study are recommended: The analysis of documents based on which the initial lists of subject-specific and generic competences of study areas are developed. The set of documents to be analysed may consist of various Lithuanian and foreign documents, such as regulations of degree programmes, descriptions of the degree programmes, descriptions of a course unit, AIKOS (Open Information, Advice and Guidance System (Internet website), the Lithuanian Classifier of Professions, developed professional standards, applicable regulations of professional regulations, specialist job descriptions, descriptions of qualification requirements, the qualification framework of the sector (e.g. the qualification framework of the European informatics sector), foreign qualification standards, descriptions of foreign degree programmes, documents of the Tuning project and other material related to the analysed study area. 7

9 The survey of employers, which aims at finding out the opinion of Lithuanian employers about subject-specific and generic competences of higher education institution graduates employed in their companies who completed programmes of respective study area. Instruments of the study include the developed lists of subjectspecific and generic competences. Quantitative and qualitative research methods can be employed in gathering and processing of information. Focus groups of graduates, which is advisable to carry out through focus groups with young specialists who completed programmes of respective study areas and who are practising professionally. 8

10 2. Making lists of subject-specific and Generic competences A research of the field of professional activity starts with the analysis of documents. The goal of document analysis is to draw up lists of subject-specific and generic competences. Such lists are later used in other stages of the field of professional activity, such as the survey of employers and focus groups with graduates having professional experience. Competence is the key term used for the description of the field of professional activity. The analysis of various documents regulating the field of professional activity leads to the formulation of subject-specific and generic competences that are important for the field of professional activity. Not all documents describing the field of professional activity use the concept of competence; therefore, it is important that the researchers who analyse documents know how to describe the content of the documents using the concepts of subject-specific and generic competences. The analysis of documents consists of three stages. First of all, a list of documents containing relevant information is made, and then subject-specific and generic competences describing the field of professional activity are selected from such documents; finally, the list of selected subject-specific and generic competences is revised by uniting competences selected from various documents. Three groups of professional field researchers, i.e. professional field research experts, professional field researchers and representatives of subject area groups of the ECTS project specialised researchers that had to upgrade degree programmes, actively cooperate in all stages of document analysis. It is worth while discussing the drawn up initial lists of competences with professional field representatives employers. In the ECTS project, the list of documents providing information related to subject-specific and generic competences covered several major groups of sources. Such sources may be divided into three groups. The first major source of documents consists of various documents regulating the education system and documents of individual educational establishments. This group includes degree programmes of higher education institutions the brief descriptions of which are available in the Open Information, Advice and Guidance System (AIKOS; and in Internet websites of higher education institutions; regulations of studies of respective study areas; professional training standards ( other documents related to the activity of higher education institutions (self-evaluations of degree programmes, various studies of graduates, etc.). The second source of documents consists of documents directly describing the field of professional activity, such as the Lithuanian Classifier of Professions ( mod/klasifikatorius/?p=0), professional standards (e.g. European professional standards of informatics), various job descriptions, professional qualification requirements approved by the state (e.g. Lithuanian Medical Standard MN 14:2005 Family physician. Rights, functions, competences and responsibility (Official Gazette, 2006, No 3-62); a description of qualification requirements for public health care specialists providing health care services in schools (Official Gazette, 2007, No ), documents, if any, which regulate professional activity and which are approved by national and international professional associations, etc. The third source of documents has been singled out somewhat artificially, as it covers foreign documents of the types mentioned in the first two groups of sources; therefore, such 9

11 foreign documents could be included into the first two groups of sources. They have been classified as a separate group because of two reasons. First, in spite of the widely discussed trends of globalisation, the field of professional activity has national character. Second, a set of foreign documents will inevitably be fragmented, as documents will be included by chance rather than because of the goals and objectives of a systematic study. When selecting sources, an important role shall be played by the language that researchers can read, also by the experience of professional activity researchers, the availability of documents, etc. However, in spite of its unsystematic nature, this source is important, as it provides the opportunity to check, learn and look at the performed work from outside. During the document analysis in the ECTS project the language factor determined that the most use was made of documents in English. For instance, when developing a list of subject-specific and generic competences of a social worker, U.K., Irish and Australian professional standards of a social worker have been analysed; in addition to more general standards, learning outcomes of English philology of several universities have been reviewed in order to describe competences of a philologist of English; chemistry subjectspecific competences of the degree programme in chemistry were the only ones of the study fields covered by the ECTS project to be formulated in the Tuning project, etc. (Table 1). Table 1. Examples of document sources of different study areas English philology What has been done The 2007 self-evaluation summary of the study field of philology of Vilnius Pedagogical University has been analysed Descriptions of professional bachelor s, bachelor s and master s degree programmes in English philology of Lithuanian higher education institutions have been analysed Descriptions of professions falling in the professional subgroup of philologists and translators/interpreters, which correspond to professional education levels 5 and 6, have been analysed A document on language-related industries and professions developed by the European Language Council has been analysed A document on English philology studies in Lithuania developed by CQAHE has been analysed Degree programmes in English philology offered by several foreign universities have been analysed Teaching standards developed in the United Kingdom have been analysed Source Internal document of Vilnius Pedagogical University Open Information, Advice and Guidance System (AIKOS) Lithuanian Classification of Occupations (LCP) Languages for language-related industries and professions: the National Report of Lithuania. European Language Council, 2004 Review report on the study field English philology in Lithuania. Centre for Quality Assessment in Higher Education Degree programmes in English philology offered by the University of Helsinki, Clark University (USA), Johannes Gutenberg University (Germany) The UK Professional Standards Framework for teaching and supporting learning in higher education 10

12 Chemistry What has been done Various university degree programmes in chemistry have been analysed Descriptions of bachelor s and master s degree programmes in chemistry and applied chemistry of Lithuanian higher education institutions have been analysed The standard of training a technologist of chemical processes has been analysed Descriptions of professions falling in the professional subgroup of chemists (Code 2113), which correspond to professional education levels 5 and 6, have been analysed Documents of the Tuning project have been analysed Source Bachelor s and master s programmes of Vilnius University, Kaunas University of Technology and Vilnius Pedagogical University Open Information, Advice and Guidance System (AIKOS) The Ministry of Education and Science of the Republic of Lithuania, the Ministry of Social Security and Labour of the Republic of Lithuania (2008) Lithuanian Classification of Occupations (LCP) Tuning. Reference points for the design and delivery of degree programmes in chemistry. Publicaciones de la Universidad de Deusto, 2009; Tuning. Tuning chemistry subject area brochure. European Chemistry Thematic Network, 2008 Social work What has been done Descriptions of degree programmes of three study cycles in social work of Lithuanian higher education institutions have been analysed Descriptions of various specialisations of the profession of a social worker have been analysed The regulation of social work studies has been analysed Documents of the self-evaluation of master s programme in social work have been analysed The standard of training a social worker has been analysed The Directory of Social Services has been analysed Source Open Information, Advice and Guidance System (AIKOS) Lithuanian Classification of Occupations (LCP) The Ministry of Education and Science of the Republic of Lithuania (2008) Internal document of Vilnius Pedagogical University Methodological Centre for Vocational Education and Training (2008) The Ministry of Social Security and Labour of the Republic of Lithuania (2006) U.K., Irish and Australian standards of training a social worker have been analysed It is important to note that these lists were made in cooperation. Study experts were aware of the variety of documents regulating the field of professional activity, professional field researchers had skills on analysing such documents and were familiar with documents regulating the field of professional activity, while specialised researchers assessed their relevance to the design and upgrade of degree programmes. Study experts and professional field researchers provided 11

13 the following documents regulating the field of professional activity to specialised researchers for document analysis: descriptions of professions provided in the Lithuanian Classification of Occupations, descriptions of degree programmes published in the Open Information, Advice and Guidance System, descriptions of various jobs, while specialised researchers shared the experience of their foreign colleagues and documents regulating professional activity of national and international professional associations. The procedure of making a list of documents regulating the field of professional activity can be summarised as follows: different groups of researchers, study experts, professional field researchers and specialised researchers cooperate and develop a list of documents regulating the field of professional activity, which can be considered as being rather exhaustive or at least as containing more information than one group of researchers initially had on documents regulating the field of professional activity. When making a list of documents, it is important that in cooperation of all researchers each group substantially broadens its knowledge about documents regulating the field of professional activity and that the set of documents provided for further analysis of the field of professional activity be as comprehensive as possible. Once lists of documents regulating the field of professional activity were ready, the work was taken over by professional field researchers. It was them who reviewed all documents and selected information based on which the initial lists of subject-specific and general competences were drawn up. The lists were further analysed by specialised researchers and experts of the Tuning project who were international consultants of the ECTS project. Table 2. Change in the number of subject-specific competences during the analysis of documents Study field Initial number of subjectspecific competences Final number of subjectspecific competences Subject-specific competences remaining in the final list (%) Chemistry English philology Art Informatics Medicine (general practitioners) Music 46 (73 in Version 2) Public health Social work The average number 35 Table 2 shows the change in the number of subject-specific competences during the analysis of documents regulating the field of professional activities. In all cases, the initial list of subject-specific competences was the longest. On average the final list of the subject-specific competences includes 35% of the subject-specific competencies of the initial list. The comparison of lists of competences shows that the initial list, which was fragmentary, random and full of repetitions, has been transformed into the conceptual and justified final list of 12

14 competences. This improvement of the list was an important intellectual exercise for the entire group of researchers 4. During the intense analysis, which lasted for over a month, the analysis, synthesis and classification of competences has been mastered. Table 3 shows the change in competences related to teaching chemistry in higher education institutions and in secondary schools. The initial list features as many as 13 competences describing this activity. Professional field researchers who compiled this list were not competent to decide whether the organisation of activity outside school time, the knowledge of the teacher s work, the ability to prepare textbooks, the knowledge of child and teen psychology, etc., should be included into the list of subject-specific competences. After all, they analysed documents from the approved list. Table 3. Change of chemistry teaching-related subject-specific competences of the chemistry subject area First list of subject-specific competences To be able to teach the subject, to develop the skills of chemistry knowledge of schoolchildren and students To be able to organise work in class and exercises in class To be able to examine and assess the knowledge of schoolchildren and students To be able to organise activity outside school time To be able to draw up teaching plans To be able to communicate with colleagues, parents and schoolchildren To train students for final theses and examinations To prepare students and schoolchildren for future studies and career To prepare textbooks To know the methodology of the teacher s work To be able to select and apply the methods of the teacher s work To know the subject of teaching To have knowledge of child and teen psychology Second list of subject-specific competences To be able to teach chemistry in general education schools To be able to teach chemistry in higher education institutions To be able to prepare and apply methodological materials and guides To know the methodology of the teacher s work To know the subject of teaching Ability to teach chemistry in general education schools Third list of subject-specific competences Ability to prepare and apply methodological materials and guides in chemistry Source: Results of the professional field research of chemistry study area: Guidelines for upgrading degree programmes ( lt/files/file/chemija_tyrimu%20technine_ataskaita.pdf). 4 More detailed information about the improvement process of competences is available at the Internet website of the ECTS project. The results of the research of the field professional activities are available under Project results at the website ( Each of the eight study reports contains chapter Analysis of documents of professional activity, which provides the lists of analysed documents and at least two tables of the lists of competences one containing the initial lists of competences and the other the final lists of competences. 13

15 It is important to note that the initial list of competences is not surplus information. The final list of competences covers and integrates the entire experience of drawing up the list, thus also including competences that were later omitted. If there is a need to specify competences of the final list in more detail, the reconstruction of the procedure of drawing up the final list should be useful a major part of the omitted competences would become elements of competences of the final list. Initial lists of competences were made with reference to national, foreign and international sources. National sources provide information that describes professional activity, which is inconsistent and in need of systematisation. Furthermore, the major part of information contained in such documents is irrelevant; therefore, it was criticised and corrected during discussions with specialised researchers. The experience of other countries helped systematise national information. Such cooperation with specialised researchers and reference to better developed documents resulted in the second list of subject-specific and generic competences. This list was discussed with experts of the Tuning project. The discussions led to the development of the third list of subject-specific and generic competences, which was employed in further professional field studies. The specification of competence lists was aided by all experts of the Tuning project who participated in the ECTS project and have vast international experience. Thus, the national lists of competences of the field of professional activity integrated the international experience in line with the priorities of the common European higher education area. The summary of the document analysis performed within the framework of the ECTS project reveals several points: First, professional field researchers noted that information contained in national documents directly regulating the field of professional activity is inconsistent and in it needs to be systematised more than information contained in international documents. A major part of information stated in national documents is dated and irrelevant. Information contained in foreign documents was valuable for the analysis of professional activity and its synthesis into individual activity areas, which were later united into competences by specialised researchers. Second, the descriptions of professions provided by the Lithuanian Classification of Occupations and the descriptions of degree programmes provided by AIKOS differ from European sources; the formulation of competences lack points that are more global and related to more general things (socially responsible activity, equal opportunities, etc.). In comparison with sources of other European universities, the Lithuanian sources focus more on subject-specific competences, yet they discuss generic competences to a lesser degree. Third, professional standards that regulate the educational system were more of a benefit. They contained information that was better structured, was more explicit, detailed and more useful for the drawing up of lists of subject-specific and generic competences. The experience of other European universities and the Tuning project was especially valuable. These sources were especially systematic and exhaustive. Four, the analysis of documents regulating the field of professional activity was carried out by combining individual work and group discussions, which was a big advantage. Discussions revealed that the lists of competences, which were drawn up individually and based on the analysed documents, could be specified in more detail. Group discussions not only helped better understand the core and the correlation of individual competences, but also set the guidelines 14

16 for further work. Such methods of document analysis revealed the following advantages of the analysis: a shorter duration of the analysis of documents, faster assimilation of information, accelerated grouping of synonymous competences, more efficient group work. However, if the work of any group of researchers was not so good, it influenced the joint result. For instance, the failure of specialised researchers to participate made it more difficult to identify areas of professional activity, to reasonably group competences or to assess which competences were more important than others. This confirms again that the analysis of the field of professional activity is a complex study requiring subject-specific knowledge, the knowledge of how to carry out a social research, as well as purposeful cooperation between specialised researchers and professional field researchers, which alone can ensure as adequate reconstruction of the field of professional activity as possible. EACH COMMITTEE OF THE DEGREE PROGRAMME CAN: draw up the list of the most important national and international sources defining the field of professional activity and related to the degree programme and its graduates; analyse information contained in the sources by singling out competences to be developed, summarise and classify such competences; discuss the developed list of competences with a most wide circle of academics and representatives of the professional activity field, i.e. with professors, members of the research staff and employers; draw up the final list of competences developed by the degree programme. 15

17 3. Survey of employers The goal of the survey of employers is to find out the opinion of Lithuanian employers about subject-specific and generic competences of higher education institution graduates employed in their companies who completed certain degree programmes. This goal can be achieved with the help of both quantitative and qualitative research methods. Subject to the selected research strategy, the results and the nature of conclusions, which can be drawn from the research data, will differ. When conducting the quantitative research, i.e. the representative survey of employers according to the standardised questionnaire, answers to the following questions of the designers of degree programmes may be given: which subject-specific and generic competences the employers point out as being important for the activities of their employees, and which are considered to be less important; which competences the employers agree on, and regarding which competences their opinions differ; whether the employer s assessment of the relevance of competences is subject to the area of his/her company s operations; how employers assess the level of the competence possessed by the employee when starting his/her work at the company; whether the assessment given by the employer is subject to the qualification (professional bachelor, bachelor or master) acquired by his employee. The results of the quantitative research are numerical values of the evaluation of subject-specific and generic competences. In turn, the designers of degree programmes may employ these results for justifying the list of competences used for the definition of learning outcomes. Such research allows drawing a conclusion whether and to what extent the list of competences drawn up by the designers of degree programmes and used by them when designing or upgrading degree programmes is accurate. The qualitative research, which may employ the methods of a qualitative interview or a focus groups, may also reveal the opinion of employers regarding which subject-specific and generic competences the specialists employed in their companies need. The qualitative research allows drawing up as exhaustive a list of subject-specific or generic competences as possible, and being sure that no competence of importance for the field of professional activities, which can be developed by the degree programme, was omitted from the initial list of competences discussed with the employers during the research. However, unlike the application of the methods of the quantitative research, this research will not allow drawing conclusions regarding quantitative differences of evaluation of competences and ranking competences according to the importance attributed to them by employers. Due to a small sample of employers participating in the qualitative research, each competence included into the list, each proposal and each opinion of the employers will have equal weight. Conclusions about the trends of the field of professional activity may be provided only as assumptions the validity of which could be proven by a subsequent quantitative research. Designers and/or deliverers of degree programmes may choose to what extent they wish to study the opinion of employers. Several approaches can be employed in this respect, viz. both quantitative and qualitative, or either quantitative or qualitative research can be performed. Also, several strategies may be used when combining research methods: researchers may perform a qualitative research during the preparatory stage of a quantitative research in order to develop a more accurate instrument for measuring (i.e. the questionnaire), or a qualitative research may be performed after a quantitative research in order to get better understanding of the obtained results. 16

18 A quantitative survey of employers was performed within the framework of the ECTS project during which employers were provided with lists of subject-specific and generic competences developed by the researchers from the study programmes (representatives of different subject areas) participating in the project jointly with foreign experts. The qualitative interviews of employers were conducted during the final stage of the research in order to better understand the results of both the quantitative survey of employers and focus groups of graduates. The goal of the quantitative survey of employers performed within the framework of the ECTS project was to find out the opinion of Lithuanian employers about subject-specific and generic competences of higher education institution graduates employed in their companies who completed programmes of eight study areas. The research covered the following study areas: 1. English Philology, 2. Social Work, 3. Informatics, 4. Chemistry, 5. Public Health, 6. Medicine, 7. Arts 8. Music With due consideration to its goal, the distinguishing feature of the research of the field of professional activities lies in the fact that when researching the opinion of employers it is impossible to perform a single representative survey of employers of a certain country using the questionnaire, which is the same for all employers, the results of which could be applied to the designing of different degree programmes. Subject-specific competences are subject to a specific area or areas of professional activity, which serve as targets when training specialists. It is impossible from the methodological point of view to develop a single comprehensive list of competences suitable for all areas of activity from which employers would select competences relevant to the operations of their company. Therefore, the research of the opinion of employers regarding subject-specific and generic competences that should be developed by a certain degree programme will inevitably include two tasks: 1. the definition of a sample of potential employers of graduates who completed degree programmes of a certain study area, e.g. who are the employers of graduates who complete degree programmes in English Philology in Lithuanian higher education institutions (translation services, general education schools, companies providing tourist services, governmental institutions, etc.); 2. the development of a questionnaire customised for the employers of graduates completing degree programmes of a certain study area. During the implementation of the ECTS project, eight independent samples of employers were interviewed using a questionnaire adapted to each specific study area. A survey agency carried out the survey using the methods developed by research experts of the ECTS project. Employers were surveyed using a face-to-face interview method and Internet survey. A respondent was asked to do a face-to-face interview first; if the respondent objected or if she/he requested so, she/he was provided with an e-questionnaire. 17

19 A properly formed sample of respondents is one of the main conditions of the quality of quantitative research data. The representativeness is the key quality criterion of sampling. A sample shall be considered to be representative if the survey of sample units (in this case employers selected for the research) yields similar results to those that would be obtained if the entire population (in this case all potential employers of graduates of a certain study area) is surveyed. There are different ways of sampling; however, not all of them provide the same guarantee regarding the representativeness of the sample. The representativeness of the sample is best ensured by probability sampling. However, the application of such sampling requires a comprehensive and accurate list of population. Unfortunately, a list grouping national companies according to study areas of employed graduates is not available to researchers, and the development of such list would have increased the cost of research considerably. Non-probability sampling has been chosen during the implementation of the ECTS project. Just as in the case of probability sampling, the quota sample is simulated in order to make it correspond to the population with regard to the parameters of importance for the research, and that the sample survey would allow drawing conclusions pertaining to the entire population. However, unlike in the case of a probability sample, there is a risk of the researcher having inaccurate data on population parameters that would be used for the quota sampling, which means that the sample will not be representative of the entire population. This must be taken into consideration when analysing data and formulating conclusions of the research. The ECTS project used four criteria of the quota definition for the sampling of employers, viz. the degree programme completed by employees (the qualification of employees), the area of corporate operations, the sector of corporate operations, the location of corporate headquarters. It is also important to note that sampling for the survey was extended only to companies and institutions that have employees with qualification degrees of a professional bachelor, bachelor or master from a certain study area acquired in higher education institutions not earlier than five years ago (i.e. degree programme graduates who graduated in ). Employers or employer representatives (direct managers or persons informed about subjectspecific and generic competences of the employee) from the selected companies were surveyed. The questionnaire consisted of the following four question groups: factual questions about the employee and the company; the assessment of the general employee s qualification acquired at higher education institution; subject-specific competences (the importance of each competence to professional activity and the employee s level of each competence at the start of employment); generic competences (the importance of each competence to professional activity and the employee s level of each competence at the start of employment). Questionnaires of the Tuning project were employed for the development of questions aimed at finding out the employers assessment of the importance of subject-specific and generic competences to professional activity and the employee s level at the start of employment. The wording of these questions is presented in Fig

20 Q1. A list of subject-specific skills and capacities that may be useful for professional activity (e.g. of a chemist) is provided below. Please asses each capacity: Q1_A. Is this capacity completely unimportant, unimportant, important or very important for the employee s professional activity in your company? Q2_B. What was the level of this capacity that the employee had at the start of employment at your company: the capacity was non-existent, low, average or high? INSTRUCTIONS FOR THE PERSON CONDUCTING THE POLL: PLEASE SHOW CARD 1 FEATURING THE LIST OF SUBJECT-SPECIFIC SKILLS AND CAPACITIES. PLEASE ASK Q1_A AND Q1_B REGARDING EACH CAPACITY. Skills and capacities Q1_A Importance of the capacity to the employee s professional activity in the company Completely unimportant Unimportant Important Very important Q1_B Capacity level of the employee at the star of his/her employment at the company Non-existent Low Average High 1 Competence Competence Q2. A list of generic skills and capacities that may be useful for successful professional activity of a graduate from a higher education institution (e.g. interpersonal competences, technological skills, language skills, cognitive capacity) is provided below. Please asses each capacity: Q2_A. Is this capacity completely unimportant, unimportant, important or very important for the employee s professional activity in your company? Q2_B. What was the level of this capacity that the employee had at the start of employment at your company: the capacity was non-existent, low, average or high? INSTRUCTIONS FOR THE PERSON CONDUCTING THE POLL: PLEASE SHOW CARD 2 FEATURING THE LIST OF GENERIC SKILLS AND CAPACITIES. PLEASE ASK Q2_A AND Q2_B REGARDING EACH CAPACITY. Q2_A Importance of the capacity to the employee s professional activity in the company Q2_B Capacity level of the employee at the star of his employment at the company Skills and capacities Completely unimportant Unimportant Important Very important Non-existent Low Average High 1 Competence Competence Fig. 1. Wording of questions about subject-specific and generic competences 19

21 The assessment of subject-specific competences included in the survey of employers conducted within the framework of the ECTS project allowed ranking these competences according to their importance to professional activity of employees and the preparedness of employees for employment 5. In turn, the designers of degree programmes could employ these data, on one hand, to justify the need to include or omit the development of this or that subjectspecific competence by the degree programme, and on the other hand to consider which competences are regarded by employers as being less irrelevant. EACH COMMITTEE OF THE DEGREE PROGRAMME CAN: conduct a quantitative survey of employers with the help of survey agencies or persons experienced in conducting and qualified to conduct social research, provided there are sufficient financial resources. The committee of the degree programme must provide the persons conducting the research with the list of competences that it wants to assess. The cooperation with other committees of degree programmes in the same study area in the same or another higher education institution is recommended in order to reduce financial costs; arrange a focus group or conduct qualitative interviews with the employers of the graduates of degree programmes and with representatives of their associations, if any, with the help of persons experienced in conducting and qualified to conduct social research. The focus group should be limited to the maximum of 10 representatives of employers (the recommended size of the group is 6-8 participants) with whom the developed list of competences and the preparedness of graduates for their professional activity could be discussed. 5 Detail results of the survey of employers are provided by the reports of the professional field research available at the website of the ECTS project 20

22 4. Focus groups with graduates Focus groups with graduates of study areas aim at revealing the graduates opinions and assessment of degree programme and learning outcomes, their application in professional activity and in the labour market. They also seek to discus subject-specific and generic competences of graduates that they acquired during studies, the need or absence of certain professional knowledge, skills and capacities, proposals regarding the upgrading of the completed degree programmes. It is recommended to invite the following professionally active graduates of degree programmes to attend focus groups: graduates who graduated not earlier than five years ago; young specialists engaged in professional activity and holding various positions in different sectors; graduates of degree programmes of the same level (professional bachelors, bachelors or masters) graduates of degree programmes can take part in the focus group. The specialised researchers (who compile information about further professional activity of graduates) will have the best chances to ensure a constructive contact with graduates and their interest in committing some of their time for discussions in the focus groups. The results of the survey of employers as well as the lists of subject-specific and generic competences applied in the professional activity research can be used in discussion. In the ECTS project it was the professional field researchers who developed guidelines for the focus groups and moderated discussions with due consideration to the progress of the research and interim results. The key topics and issues discussed with graduates of degree programmes of the surveyed study area were as follows: the assessment of the preparedness for employment and professional activity (the assessment of degree programme and learning outcomes); expectations of employers regarding graduates of certain degree programmes: on what competences the employers focused when hiring; what competences the employers demand from their newly hired employees; knowledge, skills and capacities acquired on-the-job: what competences were acquired on-the-job; whether they think it would have been better to acquire these competences at the higher education institution; whether it is possible to acquire these competences at the higher education institution, or whether they are specialised competences that can and must only be acquired on-the-job; lacking knowledge, skills and capacities: what competences they missed or currently miss at work; proposals for the upgrading of degree programmes of the study area: what proposals they have regarding any supplementation/improvement of the degree programme that they themselves completed; maybe course units should be supplemented with new topics, or the degree programme should be supplemented with new course units, maybe the methods of teaching should be slightly amended; what competences should get more attention according to their opinion; the discussion of lists of subject-specific and generic competences: which competences are required in their employment, and which competences are less relevant; whether they think that during their studies they acquired the required and relevant competences. 21

23 Data of the qualitative section of the professional field study carried out within the framework of the ECTS project provided valuable information about the results of degree programmes in the labour market 6. Even though it was difficult for all participants of focus groups to discuss degree programmes at the competence level, the developed lists of subjectspecific and generic competences provided the discussion with a certain direction and content, and they were like a tool for structuring the discussion. It was relatively easy for graduates of degree programmes to identify knowledge and skills that they miss in the labour market or in professional activities, and to state the areas in which they feel professionally competent. Recent graduates are able to constructively reflect on the expression of degree programme outcomes within the context of their professional experience. Data from this stage of the professional field study complemented the results of the quantitative survey of employers with the graduates perspective: they assessed degree programmes within the context of both further education and work experience, and discussed competences of relevance to their professional activity. It should be noted that graduates were inclined to attribute relatively more relevance to the development of generic competences during studies. EACH COMMITTEE OF THE DEGREE PROGRAMME CAN: arrange a focus with graduates of degree programmes with the help of persons experienced in conducting and qualified to conduct social research. The focus group should be limited to the maximum of 10 graduates (the recommended size of the group is 6-8 participants) with whom the developed list of competences and the preparedness of graduates for their professional activities could be discussed. It is recommended to invite a person that the graduates are not familiar with to moderate the discussion. 6 Data of the focus groups with graduates are provided by the reports of the professional field research available at the website of the ECTS project Reports of the professional field research summarise the insights and results of focus groups held with graduates of degree programmes of one study area. 22

24 Published BY Vilniaus universitetas Universiteto str. 3, LT Vilnius Tel.: (8 5) , fax: (8 5) Prepared by Publishing company Kriventa V. Pietario str. 5-3, LT Vilnius Tel./fax: kriventa@takas.lt

25