Project. findings, results and. conclusions. Are you a Personal Assistant? Learning is not only classroom KAI. Developed by: Contributors:

Transcription

1 Are you a Personal Assistant? is not only classroom Developed by: Project KAI findings, results and Contributors: conclusions This project has been funded with support from the European Commission. This publication [communication] reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

2 The consortium of EuPA would like to express their gratitude to all those that made EuPA such a successful project. Special thanks to: The members of EuPA groups (A, B, C) that participated in the pilots and the assessments. EACEA for their continuous support and assistance during the development of the project. The EuPA team: all the members of the partners that believed in the project and worked hard to produce a valuable result. Everyone who has participated in EuPA activities or in any way expressed interest in the project. This project has been funded with support from the European Commission. This publication [communication] reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

3 EXECUTIVE SUMMARY 6 INTRODUCTION 8 THE EUROPEAN QUALIFICATION FRAMEWORK AT A GLANCE 9 EUPA BACKGROUND 11 Aims and objectives of EUPA 12 The stages used by EUPA 12 EUPA Participants 14 Description of the methodology for each of the stages 15 1) Development of the qualification framework 15 2) Development of EuPA assessment tool 16 3) First Assessment 22 4) Training and Development 22 5) Second Assessment 27 PROJECT FINDINGS, RESULTS AND CONCLUSIONS 28 Lessons learned: The toolkit for development of a qualification framework 28 Evaluations of the trainings 29 Results from the assessments 32 1) comments on the ASSESSMENTS 32 2) Indicators used for the analysis of results 33 3) First assessment results and conclusions 34 4) Conclusions from the first assessment 39 5) Second assessment results and conclusions 40 6) Comparison between the first and the second assessment 45 Proposed path from the development of Outcomes for personal assistants: Level Two (2) 49 In summary 57 3 P a g e

4 Areas for improvement and final remarks 58 References 59 BIBLIOGRAPHY 59 List of figures Figure 1: EUPA Stages Figure 2: EUPA Assessment tool Figure 3: Sample from the assessment tool Figure 4: EUPA Current Situation Map Figure 5: EUPA Personal Development Path Figure 6: Percentage of success from the first assessment (presented per country) Figure 7: Percentage of success from the first assessment (presented per group) Figure 8: Percentage of success shown per country (graphical representation of second assessment) Figure 9: Percentage of success - shown per group (graphical representation of second assessment) List of tables Table 1: EQF Levels Table 2: EUPA Groups and profiles Table 3: Training classes implemented during EUPA Table 4: Dimensions used for the evaluations of trainings Table 5: Evaluations of the trainings in Cyprus, Greece and Slovakia Table 6: outcomes added on the final qualification framework in level 2 (after the first assessment).. 33 Table 7: Analysis of the results of the first assessment for EUPA Group A Table 8: Analysis of the results of the first assessment for EUPA Group B Table 9: outcomes that can be developed through Table 10: outcomes that can be developed through (evidence from two out of the three countries) Table 11: outcomes that may be developed to some extent through informal Table 12: outcomes that cannot be developed through or informal Table 13: Analysis of the results of the second assessment for EUPA Group A P a g e

5 Table 14: Analysis of the results of the second assessment for EUPA Group B Table 15: Analysis of the results of the second assessment for EUPA Group C Table 16: outcomes that can be developed through formal Table 17: outcomes that can be developed through formal (evidence from two out of the three countries) Table 18: outcomes that remained undeveloped after formal Table 19: Comparison between the first and second assessment using two indicators for EUPA Group A Table 20: Comparison between the first and second assessment using two indicators for EUPA Group B Table 21: Development of outcomes for personal assistants level P a g e

6 Executive Summary Report: "Project findings, results and conclusions" is the sole deliverable of Work package 9 of EuPA: Analysis of findings from work packages 1-8, project results and conclusions. The objective of this work package is to analyse the project findings from workpackages 1-8 so as to draw some conclusions regarding certifications and validation of formal (FL), (NFL) and informal (IL). The terms formal, and informal were used in EuPA in accordance to their definition by CEDEFOP: Formal that occurs in an organised and structured context (in a school/training centre or on the job) and is explicitly designated as (in terms of objectives, time or support). Formal is intentional from the learner s point of view. It typically leads to certification. Source: Cedefop, Informal resulting from daily work-related, family or leisure activities. It is not organised or structured (in terms of objectives, time or support). Informal is in most cases unintentional from the learner s perspective. It typically does not lead to certification. Source: Cedefop, Non-formal which is embedded in planned activities not explicitly designated as (in terms of objectives, time or support), but which contain an important element. Non-formal is intentional from the learner s point of view. It typically does not lead to certification. Source: Cedefop, The objectives of our research and pilot actions are the following: To define the qualifications framework. This is achieved through the initial research. (work package 1) To identify dimensions of knowledge/skills/competences that can be acquired through Nonformal or Informal. This was achieved using the first assessment of the 60 personal assistants or potential personal assistants. These two groups have no formal and their assessment identifies dimensions of knowledge/skills/competences even with 6 P a g e

7 no experience or formal training. This has significant implications for the methodological tools designed for the training of these competences. As far as formal is concerned: To identify those dimensions of knowledge/skills/competences that cannot be easily acquired through and informal and it is easier to acquire them through formal. This is achieved as follows: For EuPA groups A and B: By comparing their results from the first and second assessment. For EuPA group C: By analysing their results from the second assessment only (they did not participate in the first assessment). It is important to note here that one of the objectives of this analysis is to see whether some dimensions of knowledge/skills/competence cannot be acquired through formal only, but as a combination of formal/non formal and informal. Our general objective was to identify types/categories of knowledge, skills and competences that can be acquired through previous experience and through other life activities, as opposed to those that can be acquired through formal. This then enabled the assignment of credits to work experience, and the development of a path through which a personal assistant could collect the credits and be certified. This process took into account of the existing approaches to the allocation of credits at national and at European levels ECTS and ECVET. European Credit Transfer and Accumulation System (ECTS) ECTS makes teaching and in higher education more transparent across Europe and facilitates the recognition of all studies. The system allows for the transfer of experiences between different institutions, greater student mobility and more flexible routes to gain degrees. It also aids curriculum design and quality assurance. The European Credit system for Vocational Education and Training (ECVET) The European Credit system for Vocational Education and Training (ECVET) aims to give people greater control over their individual experiences and make it more attractive to move between different countries and different environments. The system aims to facilitate the validation, recognition and accumulation of work-related skills and knowledge acquired during a stay in another country or in different situations. It should ensure that these experiences contribute to vocational qualifications. ECVET aims for better compatibility between the different vocational education and training (VET) systems in place across Europe and their qualifications. 7 P a g e

8 Introduction This report is the sole deliverable of Work package 9, Analysis of findings from work packages 1-8, project results and conclusions. The aim of work package 9 is the development of a final report that summarizes all the results of the project, evaluates whether the initial objectives have been met and suggests further improvements. The current report presents: 1. An introduction to the European Qualification Framework 2. The background of EuPA 3. The objectives of the project 4. The methodology used during the project 5. Lessons learned in terms of the design of a qualification framework for the area of personal assistants 6. Limitations and areas for the expansion of the project The methodology used for the composition of this report is: 1. Communication with the partners in the consortium 2. Questionnaires to the participants of the project (EuPA Group A, B and C) 3. Evaluation of the project by Employers 4. Communication with the trainers of classroom training as well as the assessors using the EuPA Assessment tool 8 P a g e

9 The European Qualification Framework at a glance The objective of this section is to present the necessary information that one should have and be able to understand in order to proceed with either designing, or implementing a qualification framework. In other words, in this section the consortium presents what is useful in understanding the philosophy of the European Qualification Framework. EQF is a common European reference system that links different countries national qualification systems and frameworks together. EQF shifts the focus from input (lengths of experience, type of institution) to what a person knows and is able to do ( outcomes). outcomes are expressed in terms of Knowledge, Skills and Competences. Knowledge: In the context of EQF knowledge is described as theoretical and/or factual Skills: In the context of EQF skills are described as cognitive (involving the use of logical, intuitive and creative thinking) and practical (involving manual dexterity and the use of methods, materials, tools and instruments.) Competence: In the context of EQF competence is described in terms of responsibility and autonomy Moreover EQF uses 8 reference levels. EQF can be summarised in the following diagram: (only the levels used in EuPA are presented) 1 1 See section EUPA background. 9 P a g e

10 Level 5 Level 4 Level 3 Level 2 Project Findings, Results and Conclusions KNOWLEDGE SKILLS COMPETENCE Basic factual knowledge of a field of work or study Knowledge of facts, principles, processes and general concepts in a field of work or study Basic cognitive and practical skills required to use relevant information in order to carry out tasks and to solve routine problems using simple rules and tools A range of cognitive and practical skills required to accomplish tasks and solve problems by selecting and applying basic methods, tools and materials and information Work or study under supervision with some autonomy Take responsibility for completion of tasks in work or study Accept own behaviours to circumstances in solving problems Factual and theoretical knowledge in broad contexts within a field of work or study A range of cognitive and practical skills required to generate solutions to specific problems in a field of work or study Exercise self management within the guidelines of work or study contexts that are usually predictable but are subject to change Supervise the routine work of others, taking some responsibility for the evaluation and improvement of work or study activities Comprehensive, specialised, factual and theoretical knowledge within a field of work or study and an awareness of the boundaries of that knowledge A comprehensive range of cognitive and practical skills required to develop creative solutions to abstract problems Exercise management and supervision in contexts of work or study activities where there is unpredictable change Review and develop performance of self and others Table 1: EQF Levels P a g e

11 EUPA Background Based on the EU policies related to the recognition of qualifications, EuPA's main objective is to develop a model for the recognition and validation of the qualifications of the sector of personal assistants based on outcomes. The sector of Personal Assistants (PAs) has been selected (as a case study) for the following reasons: 1. PAs often do not have academic qualifications, which makes the validation of other skills and competences a necessity. 2. Most of the inactive women, when entering the labour market, request a position as a secretary or a PA and in many cases they have no formal qualifications. Evaluation of their and informal will ease their access to the employment market. 3. PAs are of vital importance in every EU Company. Their role has been upgraded during the last decade. EuPA is supporting the improvement of flexible pathways, through the implementation of the European Qualifications Framework and systems for the validation of and informal and lifelong guidance. In this framework, EuPA developed a qualification framework based on outcomes and a certification based on the validation of formal, and informal outcomes. EuPA consists of several steps through which the qualification framework was developed, based on research that identified not only the knowledge, skills and competences needed by a personal assistant but also the importance of different outcomes. An assessment tool was also developed to assess the knowledge, skills and competences of PAs. Additionally training modules were designed, developed and implemented for different outcomes. It should be noted that, through research, four different levels have been identified to match EQF levels 2,3,4 and 5 for PAs. Although designing a qualifications framework in 4 levels is an enormous amount of work, the consortium decided to design the complete qualification framework but focused only on the assessments, design of training material and reassessment of level P a g e

12 Aims and objectives of EUPA The main objective of EuPA is to develop a model for the recognition and validation of the qualifications of the sector of personal assistants based on outcomes. As far as the specific objectives of the program are concerned, EuPA aims: 1. To develop a European Certification for PAs, based on outcomes (LOs) since no national certifications exist. 2. To identify outcomes that may be easier to be developed through and informal and to use this information to design methodological tools to be used during formal training for these outcomes. 3. To develop, with the aid of an Assessment Tool, a European certification for secretaries and PAs. The Assessment Tool is two-dimensional using different methodological tools for the assessment of different outcomes. Therefore, when customized, it can be used across sectors. 4. To develop a modular curriculum and training material that will assign credits to the modules of this curriculum and hence enable PAs to certify. The stages used by EUPA The main emphasis of EuPA was to develop a qualification framework for personal assistants, as well as an assessment tool to facilitate the assessment of PAs towards this framework, and training material for the development of personal assistants. The stages of the project are demonstrated by the figure 1: 12 P a g e

13 Development of Qualification framework Second assessment Development of EUPA assessment tool Training and development First assessment Figure 1: EUPA Stages 13 P a g e

14 EUPA Participants Three groups of participants participated in the EuPA project as presented by the table. EUPA Group Profile EUPA Group A EuPA Group A consists of ten participants from each partner country who have to comply with the following requirements: a. Previous work experience of 3-5 years in a secretarial position b. No previous formal training EUPA Group B EuPA Group B consists of ten participants from each partner country who have to comply with the following requirements: a. Inactive women - women outside the labour market who have not been actively searching for a job during the last 5 years b. No previous formal training c. Interested in becoming a personal assistant EUPA Group C EuPA Group C consists of ten participants from each partner country that: a. Are graduate school leavers b. Have no work experience c. Have no formal training Table 2: EUPA Groups and profiles 14 P a g e

15 Description of the methodology for each of the stages 1) DEVELOPMENT OF THE QUALIFICATION FRAMEWORK Development of a qualification framework is not an easy task. This section deals with the development of the qualification framework. The following are being described: The stages the consortium followed for the development The key issues identified and the way they were resolved Stages towards the development of a qualification framework 1. Review of the European Qualification Framework policy as described by the European Union The first step, to develop a qualification framework in accordance to the European Union policy, is, of course, to review the policy. Several documents have been reviewed related to the European Qualification Framework, to the assignment of credits, to the progress of the EU countries (and especially of the countries in the consortium) with regards to the EQF 2 etc. Credits on outcomes have been assigned based on research into the importance of tasks. Differences between countries were minor. In general, workload is a significant aspect of credit allocation across all the Levels of the EQF. 3 It should be noted here that nothing has been identified in the literature as a practical set of guidelines on how to develop a qualification framework, on how the terminology matters in the development of levels and even on how specific tasks relate to specific levels (e.g. the difference between basic troubleshooting towards creative problem solving etc.). 2. Review of similar work on development of qualification frameworks The investigation focused on two main issues: Methodologies and qualification frameworks dealing with other sectors outcomes from qualification frameworks of other sectors that could and should be used in the current qualification framework. It is noted that no qualification framework has been identified for the area of personal assistants, although qualification frameworks have been developed for other areas such as receptionists and administration personnel. 4 2 All literature reviewed is listed in the references section 3 For a discussion of workload and credits, see the 2009 ECTS User Guide all the activities required to achieve the expected outcomes, including the time spent on independent work, compulsory work placements, preparation for assessment and the time necessary for the assessment. 4 References for all the research are also included in the references sections 15 P a g e

16 3. Identification of work areas, units and outcomes through research To identify the work areas and outcomes for personal assistants, the previous experience of the coordinator on the field was taken into account together with the expert opinion of EUMA members (the professional association of personal assistants). Based on the above, a questionnaire was designed. The questionnaire identified the main areas of work, but at the same time enabled the research group to add areas of work. Moreover, the questionnaire has identified several outcomes from different levels and asked the research group to assign weights to different outcomes. Three research groups have been used, namely: personal assistants, their managers and other stakeholders who have interaction with personal assistants. It should be noted that the research revealed 4 (four) different levels of outcomes from level 2 (two) to level 5 (five). Although the design of a qualification framework in 4 different levels is a huge amount of work, the consortium decided to design the qualification framework for all levels and then focus on one level (level 2) for the design of the assessment tool and training materials, as well the actual assessments and trainings. 4. Finalisation of the qualification framework Based on the findings of the steps mentioned above, the first draft of the qualification framework was designed for levels 2,3,4 and 5. Then a workshop took place in Slovakia where the partners formed groups (one group for each level) and discussed in detail: Whether the outcome was expressed correctly Whether it was in conformance with the specific level Whether it was covered from another LO And so on The qualification framework was modified during this workshop and was sent to the partners for final evaluation and adjustments. Minor adjustments were identified and corrected and after that the first assessment took place 5. 2) DEVELOPMENT OF EUPA ASSESSMENT TOOL The EuPA assessment tool is a tool to assess the personal assistants on each of the outcomes. The assessment tool consists of an introduction to the assessors (although an assessors manual has also been developed), the factors that must be taken into account while using the assessment tool, as well as the types of evidence used. The following should be noted here: 5 The assessment tool was adapted each time to match the qualification framework. 16 P a g e

17 1. Observation is a method that needs to be used for achieving reliability and validity. However, during the pilot assessments observation was not used, because this is an exploratory project (that is the certificate is not yet of value to the market), therefore it will be difficult to persuade companies or organisations to accept this specific method. 2. As the project involves also inactive (unemployed) women, observation cannot be used as a method of assessment. 3. Therefore, all assessments were performed in the offices (or other locations) of the partners. The following types of evidence are included in the EuPA assessment tool: Direct Observation of the learners performance by their assessor Outcomes from oral or written questioning Products of the learner s work Personal Statements and /or reflective accounts Outcomes from simulation, where permitted by the assessment strategy Professional Discussion Assignment, project/case studies Authentic statements/witness testimony Expert witness testimony Evidence of recognition of Prior The following is a sample page of the EuPA assessment tool. Figure 2: EUPA Assessment tool 17 P a g e

18 In other words, the page on the left hand side includes the instructions for the assessors, while the page on the right hand side includes comments for the assessors (related to how they should proceed in marking this outcome). A sample from the left hand side is demonstrated below: WORK AREA CODE 2.3 WORK AREA TITLE COMMUNICATION, TELEPHONE HANDLING, SOCIAL AND RELATIONSHIPS UNIT Use telephone techniques effectively Theme Outcome LO 8 Telephone Techniques 8. Be able to recall the main principles of effective telephone communication and demonstrate competence in applying these principles in a clear and effective way. Be able to perform Public Relations activities to improve the workplace relations for example acknowledging birthdays, name days and other special occasions. Credits assigned 2 Number of assessments used 2 (out of 3) 18 P a g e

19 Assessment Criteria Assessment tools (Guidance) Be able to use effectively telephone techniques. Explain the advantages and limitations of telephone communication. Describe organisational procedures when receiving or making a telephone call. Identify and describe different stages involved in telephone (incoming and outgoing) and techniques that should be used in each of them. Identify positive and negative words and phrases. Identify strategies for improving your vocabulary. Describe information that could affect confidentiality and security and how to handle these. ROLE PLAY (1 credit) Role-play can be used on any scenario for testing the assessment criteria. [Role Play on any Scenario] [SELECT ONE OF THE TWO EXERCISES] Evidence collected (1 credit) Evidence for testing that the participant uses the correct telephone techniques, might be collected from memos and telephone logs. Observation that the telephone system is used correctly. OR Professional Discussion (1 credit) Discuss with learners: Advantages and limitations of telephone communication. The key stages of an inbound as well as an outbound telephone conversation together with the techniques needed in each. Issues that must be treated with confidentiality. Positive and negative words and phrases and strategies for improving one s vocabulary. Figure 3: Sample from the assessment tool 19 P a g e

20 The assessment tool is accompanied by two very important tools: 1. The current situation map which, as its name indicates, presents the current level of the personal assistant on each of the LOs. LO30 LO32 LO31 LO33 EUPA Current Situation Map LO37 LO36 LO35 LO LO1 LO2 LO3 LO4 LO5 LO6 LO7 LO8 LO9 LO29 LO28 0 LO10 LO11 LO27 LO26 LO25 LO24 LO23 LO22 LO21 LO20 LO15 LO16 LO17 LO18 LO19 LO14 LO12 LO13 Figure 4: EUPA Current Situation Map The current situation map presents the current situation (in blue) versus an ideal situation (red) 2. The EuPA personal development map (next page) presents the ideal and the assessed situation together with the gap between them. Therefore, the learner can see the amount of development they have to undergo to reach the requested level. 20 P a g e

21 LO1 LO2 LO3 LO4 LO5 LO6 LO7 LO8 LO9 LO10 LO11 LO12 LO13 LO14 LO15 LO16 LO17 LO18 LO19 LO20 LO21 LO22 LO23 LO24 LO25 LO26 LO27 LO28 LO29 LO30 LO31 LO32 LO33 LO34 LO35 LO36 LO37 Project Findings, Results and Conclusions 12 EUPA Personal Development Path Ideal (total number of credits) Assesed Gap Figure 5: EUPA Personal Development Path 21 P a g e

22 3) FIRST ASSESSMENT Ten members from group A and group B participated in the first assessment in each country. Group A learners have no formal but have experience in the area, that is, they have. Group B learners have no formal but they have informal, which is through life experiences. The results of the assessment are presented in the section First assessment results and conclusions 4) TRAINING AND DEVELOPMENT Training material has been designed and developed for each of the units identified. The following trainings have been implemented in each of the countries: Country Unit Date Started Date Finished Duration Number of participants Cyprus Handle Mail 22/09/ /09/ :00 15:00 7h 19 Cyprus Handle Stock 23/09/ /09/ :00 12: h 21 Cyprus Use office equipment 23/09/ /09/ :00-15:00 2h 18 Cyprus Basic Business Communication 12/10/ /10/ :00 14: h 23 Cyprus Use telephone techniques effectively 31/10/ /10/ :00 14: h 16 Cyprus Building Relationships 05/12/ /12/ :00-12: h 11 Cyprus Basic Written Communication 07/12/ /12/ :00 14: h P a g e

23 Cyprus Maintenance of an established filing 26/09/ /09/ :30 12: h 17 system Cyprus Basic Business 26/09/ /09/ :00 15:00 18 Documentation 2h Cyprus Making business 09/12/ /12/ :00 12:15 14 transactions and 4.15h Payments Cyprus Organize business travel and accommodation 14/12/ /12/ :00 14: h Organize and support meetings Cyprus Handling Diary 30/01/ /01/ :00-14:30 17 Systems 6.30h Cyprus Word-processing 17/02/ /02/ :00 13:00 5h 13 Cyprus Word-processing 21/02/ /02/ :00 13:00 5h 13 Cyprus Spreadsheets 22/02/ /02/ :00 13: Presentation 5h Cyprus Internet 24/02/ /02/ :00 13: h 23 P a g e

24 Greece Handle Mail 07/12/ /12/ :00-17:00 7h 22 Greece Handle Stock 08/12/ /12/ :00-17:00 7h 12 Greece Use office equipment 09/12/ /12/ :00-18:00 2h 11 Greece Basic Business Communication 14/12/ /12/ :00-16:00 6h 17 Greece Effective Telephone Handling 20/12/ /12/ :00-19:00 9h Building relationships Greece Basic Written Communication 05/01/ /01/ :00-18:00 8h 28 Greece Filing Skills Dealing with 11/01/ /01/ :00-15:30 05:30h 26 business documentation Greece Making business transactions and 11/01/ /01/ :00-17:00 4h 15 Payments Greece Organize Business travel and 18/01/ /01/ :00-15:00 5h 20 accommodation Greece Organize and support meetings 25/01/ /01/ :00-16:00 6h 22 Greece Handling Diary Systems 1/02/2012 1/02/ :00-17:00 7h P a g e

25 Greece Word-processing Spreadsheets 15/02/ /02/ :00-18:30 10:30 h 17 Greece Presentations 28/02/ /02/ :00-19:00 9h 27 Greece Internet 29/02/ /02/ :00-16:00 6h 27 Greece /03/ /03/ :00-15:00 5h 27 Slovakia Handle Mail 28/12/ /12/2011 9:00 18:00 9 hr 18 Slovakia Handle stock 29/12/ /12/ :00 19:00 5 hr 14 Slovakia Use office equipment 29/12/ /12/ :00 17:00 4 hr 18 Slovakia Basic Business 10/01/ /01/2012 9:00 13:00 15 Communication 14:00 19:00 9 hr Slovakia Building 12/01/ /01/2012 9:00 14:00 5 hr 14 relationships Slovakia Effective 17/01/ /01/ :00 19: Telephone Handling hr 25 P a g e

26 Slovakia Basic Written 16/01/ /01/2012 9:00 13:00 18 Communication 14:00 19:00 9 hr Slovakia Filing Skills 12/01/ /01/ :00 19:00 4 hr 19 Slovakia Organize 18/01/ /01/2012 9:00 13:00 4 hr 13 Business travel and accommodation Slovakia Organize and support meetings 18/01/ /01/ :00 19:00 5 hr 14 Slovakia Dealing with business documentation 20/02/ /02/ :00 17:00 4hr 22 Slovakia Making business 20/02/ /02/ :00-20: transactions and hr Payments Slovakia Handling Diary Systems 22/02/ /02/2012 9:00-13:00 4hr 19 Slovakia Word-processing 21/02/ /02/ :00-19:00 9hr 10 Slovakia Internet 22/02/ /02/ :00-17:00 3hr 10 Slovakia /02/ /02/ :30-16:30 2hr 10 Slovakia Presentations 23/02/ /02/2012 9:00-14:00 5 hr 21 Slovakia Spreadsheets 23/02/ /02/ :00-19: P a g e

27 hr Table 3: Training classes implemented during EUPA Group A and Group B participated in training modules where they needed further development, while Group C participated in all training modules. 5) SECOND ASSESSMENT After the completion of the trainings, a second assessment was implemented. All participants from Group A, B and C participated in the second assessment. The results of the second assessment are presented in Second assessment results and conclusions 27 P a g e

28 Project Findings, Results and Conclusions Based on its gained experience on designing qualification frameworks, the consortium has developed a procedure to help other learners designing qualification frameworks. This is demonstrated by the figure below: Lessons learned: The toolkit for development of a qualification framework Study the sector deeply to idenify tasks performed by its members Refine the list of tasks. Remove duplicates and ensure that the tasks are well expressed DO NOT WORRY FOR LOS at this stage Once the list is complete, try to express each work task as a outcome. Identify the outcome on which the job responds to as well as the outcome where this job takes its more advanced form in terms of Knowledge, skills and completence as stated by the EQF Have a look at the EQF and express each of the jobs in terms of LOs for all the applicable levels. Ensure that you check the EQF and that your terminology is in accordance to the EQF Be carefull with the progress of the LO between the levels and how this is expressed Once you have finalised the list of LOs you need to assign credits to each of the outcomes so that they can be assessed. Credits should somehow express the importance of a particular LO, for one to be able to perform its work Go over the procedure again to ensure that everything is OK. You may need to repeat the procedure several times. 28 P a g e

29 Evaluations of the trainings The following dimensions have been used to evaluate the trainings (in terms of content, structure, duration and trainer). D1: Did the workshop meet your expectations? D2: Was the length of the workshop adequate? D3: Were you satisfied with the structure and flow of the workshop? D4: Do you feel that the level of the content was suited to the group? D5: How would you rate the balance between theory (models, tools) and practical exercises (case studies, group work)? D6: How confident are you that you will be able to apply what you have learned in your job? D7: How would you rate the course documentation and why? D8: How would you rate the methodological tools used with regards to innovation? D9: What is your overall evaluation of this workshop? D10: How well did the speaker communicate his/her knowledge of the subject? D11: How would you rate the presentation style of the speaker? D12: Was sufficient opportunity given for questions and answers? D13: Did the speaker allocate enough time for interaction and group work? D14: What is your overall evaluation of the speaker? Table 4: Dimensions used for the evaluations of trainings 29 P a g e

30 The evaluation results per country are shown in table below (average rates out of 5): Dimension of evaluation Cyprus Greece Slovakia Did the workshop meet your expectations? Was the length of the workshop adequate? Were you satisfied with the structure and flow of the workshop? Do you feel that the level of the content was suited to the group? How would you rate the balance between theory (models, tools) and practical exercises (case studies, group work)? How confident are you that you will be able to apply what you have learned in your job? How would you rate the course documentation and why? How would you rate the methodological tools used with regards to innovation? What is your overall evaluation of this workshop? How well did the speaker communicate his/her knowledge of the subject? Were the examples given during the workshop relevant and up to date? Was sufficient opportunity given for questions and answers? Did the speaker allocate enough time for interaction and group work? What is your overall evaluation of the speaker? Did the workshop meet your expectations? Was the length of the workshop adequate? P a g e

31 Dimension of evaluation Cyprus Greece Slovakia Were you satisfied with the structure and flow of the workshop? Do you feel that the level of the content was suited to the group? How would you rate the balance between theory (models, tools) and practical exercises (case studies, group work)? How confident are you that the personal assistants will be able to apply what they have learned in their job? How would you rate the course documentation and why? How would you rate the methodological tools used with regards to their effectiveness (i.e. whether they helped PAs to learn)? How would you rate the methodological tools used with regards to innovation? What is your overall evaluation of this workshop? Table 5: Evaluations of the trainings in Cyprus, Greece and Slovakia 31 P a g e

32 Results from the assessments 1) COMMENTS ON THE ASSESSMENTS It should be noted that the two assessment tools used for the first and the second assessment were not identical. The reason for this was that the consortium continued working with the qualification framework after the implementation of the first assessment. However, to facilitate comparison, the consortium kept track of the changes made. Changes included: Rephrasing or changing the expression of some outcomes Adding new outcomes. The first qualification framework consisted of 31 outcomes in level 2 and the second of 37 outcomes in the respective level. The addition was a result of one of the following: o A completely new outcome that was not identified prior to the first assessment o The decomposition of a outcome to two or more outcomes to better facilitate its assessment The following outcomes have been added 6 : LO9 Be able to understand the organisation s Codes of Conduct within the social, economic and cultural context. 6 outcome numbers mentioned in this document refer to the identification number in the final qualification framework (the one used in the second assessment) 32 P a g e

33 LO10 Be able to recall the key principles of data protection, security and confidentiality within the workplace and apply these principles effectively and efficiently. LO18 Be able to enter accurate and relevant data in an existing database system (electronic or manual). LO20 Be able to make payments to suppliers and receive payments from customers in conformity with organisational procedures LO30 Be able to welcome visitors and customers in a professional manner, identify the purpose of their visit and make them feel welcome during their period of waiting. LO31 Be able to use appropriate tone and body language when dealing with visitors and customers. LO32 Demonstrate a basic understanding of customer care principles and be able to apply these in an effective way, including the ability to solve routine problems using simple rules to the satisfaction of visitors and customers. Table 6: outcomes added on the final qualification framework in level 2 (after the first assessment) It should also be noted that two LOs were merged into one. Comparability between the results of the first and the second assessment has been maintained because: When two LOs were merged into one, their credits merged as well The credits at the unit level were maintained. Therefore, when an LO was added in a unit the credits of that unit were reallocated through a small scale research on the importance of the LOs within that unit. 2) INDICATORS USED FOR THE ANALYSIS OF RESULTS Two indicators have been used to analyse the results of the assessments. The number of outcomes identified as needing further development. The smaller the value of this number the less development needed The percentage of success (that is the total scoring of the candidate in percentage format). The higher the value of this indicator, the less development needed 33 P a g e

34 3) FIRST ASSESSMENT RESULTS AND CONCLUSIONS In terms of percentage of success The following table shows the values of the two indicators for each of the three countries for EuPA Group A and EuPA Group B EUPA GROUP A No of Los needing Number of Los needing Country development development (percentage) Percentage of success Cyprus % 87.43% Greece % 88.20% Slovakia % 94.60% Table 7: Analysis of the results of the first assessment for EUPA Group A The average percentage of success for the three countries in the assessment of EuPA Group A is 86.44% EUPA GROUP B No of Los needing Number of LOS needing Country development development (percentage) Percentage of success Cyprus % 46.59% Greece % 79.95% Slovakia % 77.60% Table 8: Analysis of the results of the first assessment for EUPA Group B The average percentage of success for the three countries in the assessment of EuPA Group B is 68.05% The following two graphs present the differences between Group A and Group B. 34 P a g e

35 Percentage of success for EUPA Group A and B (first assessment) [shown per country] % 90.00% 80.00% 87.43% 88.20% 79.65% 94.60% 77.60% 70.00% 60.00% 50.00% 40.00% 30.00% 46.59% 40.84% Group A Group B Difference between Group A and Group B 20.00% 10.00% 8.55% 17.00% 0.00% Cyprus Greece Slovakia Figure 6: Percentage of success from the first assessment (presented per country) Percentage of success for EUPA Group A and B (first assessment) [shown per group] % 50.00% 0.00% Group A Group B Difference between Group A and Group b Cyprus Greece Slovakia Figure 7: Percentage of success from the first assessment (presented per group) 35 P a g e

36 In terms of outcomes Analysis of the results of EUPA Group A The following three outcomes had zero gaps (that is no development was needed) in all three countries in EuPA Group A, which indicates that these outcomes can probably be developed through. outcome ID outcome LO22 Be able to produce and collate travel documentation within an appropriate timescale in order to meet traveller s needs and legal requirements (e.g. visa procedures). LO23 Be able to state the importance of checking travel plans immediately prior to the journey and carry out the process in a clear and effective way. LO36 Demonstrate ability to connect to internet sites safely and securely using browser software and search tools including identification of relevant business sites and communicating information online as required by the office. Table 9: outcomes that can be developed through The following outcomes also had zero gaps in two out of the three countries, which again indicates that they can be developed to a great extent through. outcome ID outcome Comments LO11 Be able to recall the key principles of good relationships with customers and other external stakeholders, demonstrate competence in applying these principles in a clear and effective way. Be able to perform Public Relations activities to improve workplace relations, for example acknowledging birthdays, name days and other special occasions. Cyprus and Slovakia (Gap in Greece is also only 0.2 out of 2, that is 10%) LO21 Be able to recommend business travel and accommodation arrangements in accordance with a deadline and in line with budget requirements and organisational procedures. Cyprus and Slovakia. (Gap in Greece is only 0.05 out of 1, that is 5%) 36 P a g e

37 outcome ID outcome Comments LO33 Demonstrate ability to use word processing software safely and securely to produce simple documents that meet the requirements of the office. Greece and Slovakia (Gap in Cyprus is only 0.14 out of 2, that is 7%) LO37 Demonstrate ability to use software tools and techniques to compose and send messages; manage incoming efficiently, safely and securely to meet the requirements of the office. Cyprus and Greece (Gap in Slovakia is only 0.11out of 2 that is 5.5%) Table 10: outcomes that can be developed through (evidence from two out of the three countries) The above demonstrates that these LOs can be developed through. Therefore, for example, PAs that have experience in booking flights and accommodation, using the Internet, building relationships and performing PR activities or sending s, do not need formal to learn these areas. As noted before, the European Qualifications Framework (EQF) shifts the focus from input to output. What is of interest is what a person knows and is able to do (LO).The EQF is an overarching qualifications framework that links the qualifications of different countries together. It acts as a translation device to make qualifications easier to understand across different countries and systems in Europe. This will help learners and workers who are moving between countries, from job to job and from one educational or training institution to another. The aims of the EQF are to help develop a European-wide workforce that is mobile and flexible, and to aid lifelong. Analysis of the results of EUPA Group B None of the LOs of Group B demonstrated a zero gap that would indicate that these LOs could be developed through informal. The following two LOs demonstrated a lower gap (less than 20%) that indicates that some development of this LO may happen through informal. outcome ID outcome LO1 Be able to describe the procedure through which the organisation handles incoming and 37 P a g e

38 outcome ID outcome outgoing mail. LO3 Be able to collect, arrange and dispatch outgoing mail accurately. LO9 Be able to understand the organisation s Codes of Conduct within the social, economic and cultural context. Table 11: outcomes that may be developed to some extent through informal Other comments It should also be noted that some LOS needed development for all of the participants of EuPA group A and B. The unit "Use written communication skills to produce a variety of business documents" is very characteristic. This unit consists of three LOs, namely: outcome ID outcome LO13 Be able to recall the main principles of effective forms of written communication and demonstrate competence in applying these principles in a clear and effective way. LO14 Be able to produce a variety of routine business documents (based on predefined templates). LO15 Be able to communicate in writing at a level that is appropriate for business (use business terms, avoid jargon etc.). Table 12: outcomes that cannot be developed through or informal LO13 and LO15 were two units where all participants needed development, demonstrating that these units should be developed using formal. The results were slightly better in terms of level of achievement for LO14 which asks the participants to be able to produce routine documents, through completion of standard templates and did not require knowledge and skills in using principles. 38 P a g e

39 4) CONCLUSIONS FROM THE FIRST ASSESSMENT Conclusions from the first assessment The following list of conclusions can be drawn from the analysis: Non-formal (on the job ) is extremely important since the lowest percentage of success for group A is 87.43% (in Slovakia it is 94.60%) which is extremely high. As is so important it should definitely be validated. Informal is also very important since the lowest percentage of success of group B is 46.59%, that is almost 50% of the knowledge, skills and competences can be reached through informal (while this percentage in Slovakia reaches 77.60%). There are specific outcomes that can easily be developed through and these are LOS that are learned through experience (such as making travelling arrangements). On the other hand there are outcomes that cannot be developed through. Probably expert input is required for the development of these LOs. LOs related to written communication need the support of an expert for their development. As far as informal is concerned, it seems that the development through informal is much slower. However, it is obvious that a significant amount of development does take place through informal (noted from the average percentage of success of EuPA Group B which is 68%). Two conclusions may be drawn from this: o Although significant development does happen through informal, the application of knowledge in the profession is still necessary for the learners to certify on level 2. o Informal is strongly related to reflection, that is the ability of the person to gain knowledge from lifetime situations, and this is probably another skill one has to develop in order to gain from informal. 39 P a g e

40 5) SECOND ASSESSMENT RESULTS AND CONCLUSIONS In terms of percentage of success The following table shows the values of the two indicators for each of the three countries for EuPA Group A, EuPA Group B and EuPA Group C. EUPA GROUP A No of Los needing Number of Los needing Country development development (percentage) Percentage of success Cyprus % Greece % 98.56% Slovakia % Table 13: Analysis of the results of the second assessment for EUPA Group A The average percentage of success for the three countries in the assessment of EuPA Group A is 96.18% EUPA GROUP B No of LOs needing Number of LOs needing Country development development (percentage) Percentage of success Cyprus Greece % 93.10% Slovakia % Table 14: Analysis of the results of the second assessment for EUPA Group B The average percentage of success for the three countries in the assessment of EuPA Group B is 85.27% 40 P a g e

41 EUPA GROUP C No of Los needing Number of Los needing Country development development (percentage) Percentage of success Cyprus % Greece % 96.39% Slovakia % 87.70% Table 15: Analysis of the results of the second assessment for EUPA Group C The average percentage of success for the three countries in the assessment of EuPA Group C is 91.76% The following two graphs present the percentage of success for Groups A, B and C (by country and by group). 41 P a g e

42 Percentage of success for EUPA Group A, B and C (second assessment) [shown per country] % 90.00% 96.84% 87.63% 95.89% 93.10% 93.10% 98.60% 91.30% 87.55% 80.00% 70.00% 71.40% 60.00% 50.00% 40.00% Group A Group B Group C 30.00% 20.00% 10.00% 0.00% Cyprus Greece Slovakia Figure 8: Percentage of success shown per country (graphical representation of second assessment) Percentage of success for EUPA Group A, B and C (second assessment) [shown per group] % 80.00% 60.00% 40.00% 20.00% 0.00% 96.84% 98.60% 93.10% 93.10% 95.89% 91.30% 87.63% 87.55% 71.40% Group A Group B Group C Cyprus Greece Slovakia Figure 9: Percentage of success - shown per group (graphical representation of second assessment) 42 P a g e