Brussels 15/12//2015 Meeting Minutes ESCO Mapping Pilot Workshop 5 Version: document version ESCO Mapping Pilot

Transcription

1 ESCO MAPPING PILOT WORKSHOP 5 MEETING MINUTES 15-DEC-2015 Brussels 15/12//2015 Meeting Minutes ESCO Mapping Pilot Workshop 5 Version: document version ESCO Mapping Pilot

2 1 REPORT STRUCTURE Attendees see section 2 Minutes of the meeting see section 3 2 ATTENDEES 2.1 PES Milada Stalker (Czech Republic) Alain Dupuch (Pole Emploi - France) (MAI) Julius De Zeeuw (UWV. WERKbedrijf - The Netherlands) Catalina Cantero Talavera (State Public Employment Service Spain) Gerd Goetschalckx (VDAB - Belgium) (observer MAI) 2.2 DG EMPL AND ESCO SEC Pedro Chavez, Principal Administrator at European Commission (Introduction) Barend Jan de Jong, Project leader ESCO ESCO Secretariat 2.3 TENFORCE Honza Förster, TenForce Taxonomy Expert Group Kathleen Van Hoebroek, Knowledge Engineer Johan Delaure, TenForce Program Manager 2.4 EXCUSED Martin Le Vrang, Project leader EC Katrien Van der Kuylen, Taxonomy Consultant ESCO Secretariat 1 / 7

3 3 MINUTES OF THE MEETING 3.1 AGENDA A proposal of agenda was communicated with the invitation on November 26 th On request of participants the start of the meeting was moved to 10h30. At the start of the meeting the agenda is agreed to be: 10:30: Start session: introduction and agenda; approval minutes meeting November 3th :50: Status update mapping pilot 11:00: Matching approach 11:30: Matching results 12:30: Lunch 13:30: Information loss evaluation 14:30: Evaluation 15:00: Conclusions and next steps 16:00: End of meeting 3.2 INTRODUCTION Johan Delaure opens the meeting. Mr Pedro Chavez, EC, welcomes the attendees. A summary on the reporting to the Maintenance Committee about the Mapping Pilot is given. The objective of the Mapping Pilot Project is not to deliver a final tool, final mappings, neither to deliver complete translations. The objective is to understand what the issues are when creating mappings and what is required to organize the support for this process. A tool was made to precipitate a mapping process so the issues and aspects could be observed and discussed. The pilot resulted in a better understanding of the domain as was the objective of the pilot. The findings can result in the design of solutions. The meeting minutes of the workshop of the 3th of November 2015 were distributed and the participants have provided their feedback. An updated version was redistributed last week. This version is accepted with following remark. Regarding section 3.3.1, reflecting the understanding that the mapping tool is only a supporting tool for the experts to make the mappings, the phrase The human validator can select from the resulting set the ones that are actually relevant is appended with and/or adds manually correct mappings. The latest version of the minutes for the meeting of November 3th is: Mapping-pilotws4_minutes_ _v docx. 3.3 PROGRESS REPORT AND STATUS UPDATE MAPPING PILOT The actions as agreed upon in the last workshop are checked and confirmed. Ref Action description Resp 4.1 ES PES to share a cross table linking the Spanish occupations and skills ES 4.2 Investigate reported bugs in tool TF 4.3 Requirements specification for mapping tool TF 4.4 Share the current mapping in Excel format with PES TF 4.5 Check the mapping for the specific concepts relevant in the matching PES 2 / 7

4 4.6 Annotate the CV and JV PES 4.7 Report on the matching scenario s TF 4.8 Edit meeting minutes TF 4.9 Review meeting minutes PES 4.10 Book for the 5th Workshop on December 15 th ALL 3.4 MATCHING APPROACH AND RESULTS PRELIMINAR REMARKS We refer to the presentation that was used during the workshop. An updated version (with correction of agenda) is part of these minutes. (See: ESCO_Mapping_Workshop_ _ c.pdf ) The 5 curriculum vitae (CVs) and 5 job vacancies (JVs), translated for and tagged by each PES were printed out. This allows to validate the tagging and the execution of scenarios. The printouts are as documentation part of these minutes. (See: ESCO_MapPilot_Workshop5_CVJV_Printouts.zip ). With the data provided TenForce checked the different scenarios and their variations. Given the limitations of the workshop itself it is decided to concentrate on a couple of the most significant scenarios. An approach where we demonstrate and inspect the steps for the most likely scenario and some alternate cases will clarify better the assumptions and implications then a grid with numbers can do MATCHING DATA The PES services register a diversity of information in the client file both for a job seeker as for an employer. Throughout the exercise of this pilot for cross-border matching only a subset of that information is considered relevant. 1) Not all partners/pes register today information at the level of skills. 2) At the service centres of the PES more information is captured in the client file then will be used. Indeed, also the JV information that is exchanged through EURES is also a subset of what is registered at PES level. Personal information like name, gender, age are not considered. For the matching scenarios we use essentially the job titles (previous, current, aimed for) of the CV and JV. These relate to Occupations (OCC) concepts in ESCO. When registered at the PES we also take into account concepts regarding skills that relate to the Knowledge, Skills, and Competence (KSC) concepts of ESCO. The CV and JV information being provided in the different languages and tagged by PES, we highlight the information for the specific case. This is done in respect of the different classifying schemas: 1) the NOC tagging 2) ISCO tagging 3) ESCO OCC mapping 4) ESCO KSC mapping 3 / 7

5 Regarding the KSC mapping, it is noted that in practise the skills (KSC) can and should not be inferred from the occupation that is linked to one or another stated job. In the presentation this has been done in several cases. Although KSC are structured in essential and optional, we can and may not assume that a person has even by default the essential skills. The PES services that work with KSC-like tagging of client profiles do the assignment of skills only in an assessment with the client; not just by theoretical inference. So while at this moment the KSC tagging of clients is not done in the practise of the project partners it is less relevant in any matching scenario. As will be demonstrated further on, the KSC matching may well provide background to a matching suggestion, a ranking of different profiles and/or a recommendation to candidates. The quality of these additional suggestions depends on how these results are obtained and an indication on the assumed quality or origin of these suggestions should accompany the suggestions. It is also noted that the generalisation from a job title to an occupation is not that evident in real world context. PES services asses the information that the client provides. In the setup of this matching exercise we assume the information in the CV is correct and that we understand the content of each job and that we may infer the occupation. Work context would be a useful addition to take into account when tagging the information of the CV and JV. E.g. working as a waiter in a context for disabled people brings significant more information to the table then just waiter MATCHING The matching at the level of ISCO proofs to be at a more generalised level that leads to the fact that a Sommelier is mapped with Waiter. This case demonstrates ISCO is not really designed to do job matching. Indeed, in practise the prime use of ISCO is for statistical metrics of the labour market. This implies also that ISCO may not change too much as that would break the lines for historical analysis and description of market evolutions. The matching at the level of ESCO occupations provides a deeper granularity that allows more accurate matching (or indeed not matching in case of the pair Sommelier-Waiter). The matching at the level of skills would allow to detect if somebody with certain skills could fit in a different type of occupations and/or in a different work context. A Sommelier and a Waiter have some skills in common. At most of the partner PES skills matching is however not done yet in practise. The matching at skill level would allow to provide more information/support to a matching suggestions at the level of occupations. That would be also useful and informative while an occupation remains an abstract grouping of actual jobs. Skill matching brings more concepts to the matching. But this potential benefit requires- as already mentioned a more elaborated screening of the client up front. 3.5 INFORMATION LOSS To evaluate information loss in cross-border exchange of CV and/or JV, we have to define a norm, the ideal case. We agree that the norm is country specific. So when we want to assess if the transformation through ESCO leads to information loss, we cannot measure that in absolute numbers, but it will always be expressed in respect of a certain PES/NOC. 4 / 7

6 The ideal case of interoperability is that a foreign CV or JV is translated to the local language so that the CV or JV can be treated as if it is a local CV or JV. The controlled environment of this pilot allows to do exactly that, whereas in reality this is not possible because of the translation costs. To measure the information loss during mapping/cross border exchange we follow following steps: 1. A CV in original language A is translated to language B. 2. The CV in language B is tagged according to PES B classification 3. The PES B tagged CV is mapped to ESCO. 4. The ESCO mapped CV is mapped back to PES A classification 5. This is compared to the result of the CV A tagged without middle steps to PES A classification This scenario will highlight: 1. Difference in information tagging between different PES, possibly with information loss of one compared to the other 2. Possible information loss when mapping from a NOC to ESCO 3. Possible information loss when mapping from ESCO to a NOC 4. Possible information loss during the translation. We found that ESCO allows to retain on average 75 % of the concepts of the smallest NOC tagged set that is present in the cross-border pair. ESCO mapping may result in more concepts but it is likely these are more narrow concepts (e.g. Cook in one more general classification is mapped upon the more specialised occupations Chinese cook, Italian cook, etc. in another classification). Indeed there are cases where ESCO would provide narrower terms to concepts of a NOC. Given the variety between the NOC we will be confronted with e.g. specialisations for certain occupations in certain cross border regions or economic sectors that are not relevant for other countries. These more narrow concepts cannot be used in an automated matching. At the level of occupations ESCO cannot compensate for the differences in granularity between different NOC. In a classification the general term waiter can have more specific terms as sommelier, caviste, zythologist, barist,etc. In others the sommelier will be at the same level as waiter and barista. In some the concept of sommelier will be non-existent and most closely related term would be waiter. A mapping and matching form sommelier to waiter could be ok for some cases but not for other cases. When the original mapping at the original PES has only the more general concept waiter, it is not ok to add in the mapping the narrower terms by default. A dimension of potential information loss is also the case where ESCO performs a role in bridging between the economic labour reality and education. This dimension has to be covered when developing the qualification pillar. 5 / 7

7 3.6 CONCLUSIONS To create a mapping between a NOC and ESCO, involvement of labour market experts (National Classification expert and/or sectoral experts)is essential. The experimental tool of the pilot should be redesigned so it serves as a mapping cockpit for the expert, i.e. a place that is optimised to support an expert to create and manage mappings in an efficient and user friendly manner. A mapping tool can only eventually, trough different techniques, provide suggestions but the experts need to get the choice to use the suggestions part of the tool or not. Machine generated mappings are at best useful in the same way as machine translation is used by a translator. It can help or hint to a possible solution, but it is the expert that will have to create the mapping. As each NOC vocabulary reflect the national labour market (terms usually used on it), the mapping will provide interesting inputs for the translation process. For a machine based (using NLP technics) mapping, the translation of the concepts is very useful. However a mapping provided by experts could also facilitate the translation while through the mapping translated terms of the target classification can be inherited and added to the origin classification. It is noted that ESCO, designed to bridge national classifications, will also be used to bridge between intra-national decentralised public employment services. Some PES supported the idea that the mapping platform facilitates sharing and collaborating among experts to create a mapping, where others strongly objected and wanted to work offline and only share the result. Once a classification is recognized as bridge for interoperability, the exchange of tagged information becomes easy. Only the identification (ID numbers or IRIs) need to be exchanged. If such classification for exchanging would additionally also be the same classification that is used for internal operation of a PES (like e.g. ROME), then no mapping and no bridging would be needed and no information would be lost. This ideal case for interoperability requires however that all PES use one and the same classification. The EC judges such an ambition as unrealistic. ESCO aims to cover the interoperability and bridge between the autonomous NOC of the sovereign member states. The EC assumes every NOC is the best suited classification for the local labour market it covers. This is reflected in the guidelines for ESCO that stipulate that occupations that are specific for regional market are not to be covered in ESCO. ESCO want to be good enough to allow bridging. The meeting agrees that there is not one guideline that can rule all mapping issues. It is clear there will be a continuous improvement requiring a community forum to raise, discuss and settle issues. ESCO will evolve in function of the needs of the PES. A platform for the mapping between NOC and ESCO would support this process. The mapping efforts would benefit from a platform that: serves the expert in his mapping task allows the expert to navigate at will between the classifications and associated classifications (e.g. ISCO) supports the discussion/collaboration on mapping principles (and if requested by the experts working on a mapping: on the actual mappings) allows to version the changes with features to signal the changes allows download and upload (supporting classifications as well as mapping tables) 6 / 7

8 It noted that uploading NOC data will be optional and will not be made obligatory. Comparisons with existing national classifications need to be done as early as possible to verify the coverage and granularity of ESCO, permitting to solve issues with the coverage of ESCO before the release of version 1 (end 2016). 3.7 NEXT STEPS A final report summarizing the outcomes of the mapping pilot will be distributed for feedback, also as soon as possible. The Commission needs now to consult NOC owners and PES at a strategical / political level to ensure they are OK with the proposed mapping process. Reviewing the outcome of the pilot, EC is deciding on how to proceed with the challenge to create mappings given the outcome of present pilot. The most likely follow up is twofold: - The EC decides to proceed with coverage checks of ESCO occupations using a number of complete mappings between ESCO and a number of NOC. This process will identify possible gaps in ESCO. This process will make use of an evolved version of the mapping tool and the experience of these mapping will be fed back to the tool development. - The EC will decide on how best to develop a mapping platform that supports the creation and management of mappings with ESCO. This platform would become available from with the EC in the summer of In respect of the latter the members of this pilot will likely be contacted to seek advice on requirements and when possible to assess designs and or development versions. TenForce thanks all the participants for their collaboration and sharing their insights and experience that brought the pilot initiative to its results. TenForce thanks the EC for facilitating the project and the opportunities given and the guidance in the navigation towards the results. The EC thanks all participants for their contributions and plans to develop a service that meets the expectancies. <end of minutes> 7 / 7