Brussels 6/07/2015 Meeting Minutes ESCO Mapping Pilot Workshop 3 Version: document version ESCO Mapping Pilot

Transcription

1 ESCO MAPPING PILOT WORKSHOP 3 MEETING MINUTES Brussels 6/07/2015 Meeting Minutes ESCO Mapping Pilot Workshop 3 Version: document version ESCO Mapping Pilot

2 1 REPORT STRUCTURE Attendees see section 2 Minutes of the meeting see section 3 Page 1 of 8

3 2 ATTENDEES 2.1 PES Gerd Goetschalckx (VDAB) (MAI) Milada Stalker (Czech) Alain Dupuch (Pole Emploi) (MAI) Jean-Christophe Bonnin (Pole Emploi) Julius De Zeeuw (UWV. WERKbedrijf) Wilm Van Dam (UWV. WERKbedrijf) 2.2 DG EMPL EN ESCO SEC Martin Le Vrang, Project leader EC Barend Jan de Jong, Project leader ESCO SEC Team Nuria Moyes Prellezo, ESCO team qualification team Laura Visan, ESCO SEC team 2.3 TENFORCE Honza Förster, Taxonomy Expert Group Joeri Moreno, Developer Johan Delaure, Tenforce Program Manager Page 2 of 8

4 3 MINUTES OF THE MEETING 3.1 AGENDA 10:00 : Welcome and intro 10:15 : ESCO project status update, pilots and fit-for-purpose testing 10:45 : Creating ESCO mappings: How we envision creating ESCO mappings 11:15 : Introducing the mappings 11:30 : Evaluating the draft occupations mappings: NL <-> ESCO, FR <-> ESCO, ES <-> ESCO, CZ <-> ESCO 12:15 : Lunch 12:45 : Evaluating the draft occupations mappings: (continued) 13:15 : Evaluating the draft skills mappings: NL <-> ESCO, FR <-> Mappings 13:45 : Lessons learned from the mappings 14:00 : Next Steps & Planning Validating the Mappings Cross Country Job Matching 14:45 : Concluding remarks 3.2 INTRODUCTION The 3 th ESCO Mapping Pilot workshop will focus on the envisioned process for mapping National classifications on to ESCO and share the first results of the mappings. It has taken 10 months since the previous workshop, since it was agreed with all the members that the translations needed to be organized first. This work has now been completed for the part of the classification for Hospitality and Tourism (HOSP), and as consequence the timeline to proceed with workshops could de-blocked.. As a consequence we can demonstrate the mappings between the HOSP and the 4 national classifications. The mappings between ESCO and the NOCs are absolutely crucial in supporting the countries to comply with the EURES legislation that is currently being proposed. With the pilot the European Commission wants to learn about what are the issues, what is needed and what is the effort to make the mapping effective and what is the level of semantic interoperability that can be obtained. Since last meeting there were a number of changes in the staffing of the team. Johan De Smedt is deceased due to illness. Agis Papantoniou left Tenforce and is replaced by Johan Delaure. Barend Jan de Jong takes over from Katrien Vander Kuylen, who is on maternity leave. Karel Kremer is replaced by Joeri Moreno. Further it was mentioned that a Member State Expert Group for ESCO will be created. Each Member State is invited to participate with two delegates. Unlike the Maintenance Committee and ESCO Board, delegates in the Member State Expert Group will formally represent their country. Page 3 of 8

5 3.3 ESCO PROJECT UPDATE We refer to the presentation that is included as a genuine part of these Minutes. Information from the presentation is not repeated. These minutes reflect the clarifications that are made on top off the presentation THREE PILLARS Apart from the Occupations (1) and the pillar of Knowledge, Skills and Competences (2), there is the third pillar of Qualifications (3). The data for the Qualifications will not be developed or maintained by/in ESCO. The data for Qualifications are owned by the Awarding Bodies and regularly maintained in national qualification databases. At ESCO we will harvest the data from the Awarding Bodies and national qualification databases and integrate them, however the ESCO team will not curate the data themselves DEVELOPING CONTENT The ESCO11 and the ESCO16 (with the online consultation) are the two tracks used for developing the classification for the individual the sectors. These processes end beginning of 2016, when the effort shifts towards integration and clean-up that transforms the sectors into one integrated ESCO classification. During the integration and clean-up process the translations will be added. This work will be completed at the end of 2016, when ESCO v1 will be published. The sectors Hospitality and Tourism (HOSP) and ICT will get an accelerated treatment in order to make their content available for use in pilots. (Depending on the pilot, the content will be provided in the SKOS format or as a local API.) At the meeting it was discussed that during the second half of this year the procedures around maintaining ESCO will be discussed. The EU Commission will draft a policy and discuss it with the ESCO Board. The ESCO Board consists of senior representatives of various stakeholder groups. Members of the ESCO Board are appointed on the basis of their personal expertise and do not officially represent organisations or Member States PILOTS An update was provided on the various ESCO pilots that the European Commission is involved in (see presentation). During the presentation an omission was found, a slide on the pilot with Academy cube was omitted. The pilot was discussed and the slide was added to the presentation after the workshop and is part of the distributed version. 3.4 MAPPINGS The envisioned process to create mappings was presented and discussed. Key for the process is how the model of the national classification is mapped onto the ESCO model. In the current state the tool proposes possible mappings in two directions, i.e. from NOC to ESCO and visa versa. It is in the plan to have these proposed mappings qualified in a manual step. A mock-up of the interface to approve and qualify the mappings was provided. (See presentation). Page 4 of 8

6 For the platform that proposes the mappings tools like e.g. ITM and SILK were investigated (see previous workshops). In the end Apache SOLR was chosen as the tool to generate the initial mappings. Apache SOLR is a search engine with techniques as stop-word extraction, stemming, word adjacency, weighting dependent on frequency and rating the matches with statistical methods. The SOLR search engine is Open Source, almost an industry standard, integrable and multilingual. More on Apache SOLR: About Apache SOLR language support: EVALUATING THE MAPPING: OCCUPATIONS Key discussions happened around the following topics: The process works with proposed mappings that are validated and qualified in a manual step. As a consequence of having a limited list of proposed mappings to deal with, the process of creating the mappings becomes a task that does not take too much time. Precision and Recall: the tool currently provides a good indication on the precision aspect of its query results. Wrt the recall aspect, this still needs to be investigated. Availability of preferred terms and synonyms: the tool showed that the availability of synonyms in the National Classifications highly supports the quality of the results. Semantic alignment/misalignment of the underlying schema s of ESCO and the national classifications influences the quality of the results significantly. (NL NOC is well aligned with ESCO, FR NOC schema is not ease to align with ESCO). Further investigation is needed. In order to ensure full coverage of the occupations in ESCO as well as in the NOC, mappings need to be qualified from both the NOC perspective as well as the ESCO perspective. The final mapping can be constructed by merging the two perspectives. As a consequence of differences between the underlying schema s and vocabularies used by the various PESs data-loss will always occur when translating data between systems. This is independent of ESCO. ESCO supports data exchange quality by providing a reference vocabulary for occupations, knowledge, skill and competences, and qualifications as to empower the PESs not to have to create separate mappings to all the individual NOCs available in Europe. The quality of the mappings of the skills needs further work. Matching the skill phrases proves not to be straightforward. To resolve this is especially relevant since the number of skill concepts is significant. At the same time it must be understood that there are only a limited PES organisations making use of skills at this moment. Some observations per PES/NOC/language: DUTCH 1. After a quick glance at a few mappings a reasonably good result seemed to be obtained for mapping the occupations. 2. Possibly the mapping will be improved further through by developing an approach that matches on the basis of the combination of occupations and skills. 3. The NL PES showed interest to go and investigate the quality of the mappings further. 4. For skills more investigation is required. Page 5 of 8

7 5. NL PES does not register skills in client files. Tasks, skills and competencies are only used to exemplify the NOC; they do not completely cover all aspects of an occupation but solely serve an informational purpose. 6. Besides the mapping application the Dutch participants requested mapping tables in Excel format in order to have a proper overview and to be able to involve third parties in the Netherlands FRENCH 1. The mapping result is less convincing; ESCO occupations proved to be mapped to occupations groups in the French system and not on occupations. 2. It should be investigated if the mapping rate can be improved by matching against a lower level. 3. A partial scope-mismatch was observed wrt the occupations belonging to Hospitality and Tourism. (This is of course an issue that does not exist when considering the whole of ESCO and the whole of the national classification.) 4. It is pointed out that we are here concentrating on mapping occupations, while in the case study this mapping is used to map actual jobs to to actual vacancies. Just matching occupations does not add significant additional information compared to just using ISCO-08 (assuming the same level of granularity). To augment the matching and use the value of ESCO the addition of matching the related skills has to be added. 5. In practice, the CVs are tagged at the occupation group level, at the occupation level, but most importantly at the level of skills. It is this level that is required to obtain good matches. 6. The mapping should be realised at following levels; ROME appellations (and not FEM title) with ESCO OCC ROME activités and ROME Savoirs théroriques et procéduraux (Theoretical and procedural knowledge) and Savoirs de l action (Practical knowledge) with ESCO SKC 7. The traditional way in matching CVs is tagging them to occupations. The new way of tagging is to describe the content with tasks, and/or skills and competences. Europe is moving country by country towards the new approach CZECH 1. Mismatches due to ISCO Some bad mappings because of the ISCO code. Example: the ISCO code for cleaning industrial areas is the same as a cleaner of hotel rooms (room attendant). ISCO has a generic code for cleaning, in whatever context that is (e.g. hotel, industry, street, camping, etc.) 2. Error due to ESCO and NOC using different ISCO code: (73.) 3. Wrong match due to NOC concept out of scope of HOSP in ESCO: : (22.) 4. Broader NPT maps on an ESCO concept which is wrong when there are better maps: (40.) 5. Lower matching due to different translation: (77.) SPANISH 1. Same kind of mismatches appears here. 2. A low mapping rate is also due to the origin dataset not having NPT. Page 6 of 8

8 3. The Spanish PES will provide the ESCO Secretariat with a) the Spanish extended NOC (8 digits) and b) The Spanish VET skills framework A) As presented on the first meeting, there is also an extended version for the Public Employment Services called CO-SISPE (Tabla de Ocupaciones del Sistema de Información de los Servicios Públicos de Empleo) with base on the ONC-11 and 4 more digits for certain occupations. The CO-SISPE compiles a total of 2925 occupations (744 occupations of the ONC are included). B) The list of Skills units related to the National Catalogue of Qualifications from the National Institute of Professional Qualifications (INCUAL) WHAT TO DO NEXT? We will try to improve the mapping with ROME. Starting tomorrow, ROME will become open data. This might help to improve the mapping. In a next step, a platform will be provided where the mapping for each of the 4 states/languages is provided in such a way that the automatic proposed mappings (see slides) can be validated. Question is if the mapping has to be improved first. This is at least required for FR. Aim is to provide the Mapping Validation Website by the end of the summer. The PES will then receive sufficient time to review the proposed mappings for their countries classification system, if they are willing to do this. Not anywhere else classified : If national classification uses a more precise term, we can bring it under a category Miscellaneous. However such a category is also to be considered problematic. 3.1 EVALUATING THE MAPPING: SKILLS There are more skills then occupations. The vocabulary to describe skills is broader than the one for occupations DUTCH Rate of matches is much lower. Opstellen is a generic term, unfortunately it is recognized as a significant word to strengthen a mapping. The other words in a string with opstellen are more significant FRENCH The mapping should be done on ROME activités and ROME Savoirs théroriques et procéduraux (Theoretical and procedural knowledge) and Savoirs de l action (Practical knowledge) with ESCO SKC. Now the dataset is available to anyone in CSV format at : Page 7 of 8

9 3.2 LESSONS LEARNED There are two major challenges left: 1) Improve the mapping between ESCO and ROME (The team had not properly interpreted the received data of the ROME scheme. This will be investigated in more detail.) 2) Improve the mapping of skills (Skills are captured as phrases and do not comply to any standard scheme. Also, Scheme concepts do not have multiple phrases that may well have synonyms. It could probably help to use the association with occupations to improve on the skill mappings.) 3) The quality of the translations is very important. To evaluate in more detail the current mapping, the PES should be given the possibility to investigate in more detail the mappings. Insights that could help us to refine the mapping logic would be very helpful. Every PES knows its own data best. Datasets in English and as translated will be provided. To decide in the mappings, knowledge is required in following areas: NOC, ISCO, ESCO knowledge required. What criteria do we need to handle to do a quality control? To get better insight in current matching practice: 1) Every Pes to provide more info, flowchart, on how CVs are tagged. 2) We could follow a rich CV of NL and FR and see how it is now transferred. A case between the Dutch and the French could be studied CONCLUDING REMARKS We have realized an important step in the mapping. The results are better for the occupations of e.g. NL. We will continue along this path. Where required we will invest if we can improve the mapping. There are challenges ahead. We refer again to the PPT. 4 ACTIONS Ref Action description Resp 4.1 Investigate match between language support of Apache SOLR and European TF languages 4.2 Provide the ESCO HOSP classification and preferably also the translations to the SEC members 4.3 Investigate if the mapping with ROME can be improved TF 4.4 Provide a Mapping Validation Website to validate the occupation mapping TF 4.5 Validate the occupation mapping: optional for PES (PES) TF 4.6 Investigate if the Czech Skills can be mapped TF+CZ 4.7 PES to provide insight in how and at what level they currently tag CVs. PES 4.8 Make a definitive prototype mapping that can be used for the matching workflow TF 4.9 Select a test set of CVs and JVs TF 4.10 Run matching tests TF 4.11 Plan next workshop SEC+TF Page 8 of 8