Item 5 of the Agenda. Education data collections: issues

Size: px
Start display at page:

Download "Item 5 of the Agenda. Education data collections: issues"

Transcription

1 EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social Statistics Unit D-5: Education and Culture Doc. ESTAT/D5/2004-ETS EN Version Item 5 of the Agenda Education data collections: issues Meeting of the Education and Training Statistics Working Group Luxembourg, 22 and 23 January 2004

2 Education data collections: issues Summary This document presents some issues related to the process of collection, verification and dissemination of education data collected through the joint UOE (UNESCO Institute for Statistics-OECD-Eurostat) data collection and the Eurostat data collection on regional enrolment and foreign language learning in schools. It is a continuation of the reflection presented in the 2003 Education and Training Statistics Working Group in document EUROSTAT/E3/2003/ETS14 1 on Dissemination of education data where issues related to data collection and validation were presented. This reflection is taking into account ongoing discussions in the framework of other international groups related to the UOE data collection (like the OECD INES Technical Group) as well as among the 3 UOE data requesters in view of a future revision of the UOE data collection. The role of the different actors, i.e. data providers (countries) and data requesters (international organisations and especially Eurostat), as well as the importance of correctly using the available definitions and tools are underlined. Steps already taken or to be taken to improve the process as well as the quality of the data collected are indicated. The issues and solutions presented in this paper are related to the data collection process, to the improvement of the input (collected data and metadata) and to the data validation and dissemination process. The conclusion is that clear agreements are needed with all actors involved. This agreement could take the form of a legal basis for the data collection (i.e. one or more legal acts defining the contents of the data (definitions, variables), the role of the actors and the planning of the data collection exercise) and the form of a revised guidance (manuals) which would be covering also checking, validation and estimation rules. The ETS WG is invited: - to comment on the issues presented and propose additional issues which should be taken into account in the overall discussion on the education data collections (UOE and Eurostat) in view of a future revision of the data collection instruments and processes (possibly in 2005) - to advise Eurostat on the conclusions, including on the feasibility of a legal basis for the sustainable collection of education data - to communicate this document and other relevant discussions and documents from the ETS WG and other forums to the national actors involved at different levels in the UOE data collection system, in order to raise their awareness on the issues raised. 1 The document is available on the ETCS circa site at the address: /working_documents 2

3 Education data collections: issues 1. Data collection process 1.1. Unique address For the 2003 UOE data collection a unique functional address has been established. This address is: estat-uoe-joint@cec.eu.int This address is managed by Eurostat on behalf of the 3 UOE data requesters. Messages are forwarded to UIS and OECD and the mailbox, which includes all the messages received, is available practically online to UIS and OECD using a dedicated password protected circa site. All UOE data submissions and resubmissions should be addressed ( to: ) the unique address while all message exchanged between the data requesters and data providers during the validation phase should be copied ( cc: ) to the unique address. Although the UOE2003 data collection is not completed yet, we may say that the main objective which was to avoid duplicate requests to countries during the validation, to monitor the data submissions and share in real time the information submitted and exchanged in the framework of the UOE data collection has been broadly attained. It has also been very useful to avoid loss of messages due to staff turnover in the data requester organisations. A functional address has also been established for the Eurostat education tables. This is: estat-educ@cec.eu.int 1.2. Deadlines The deadlines have not been respected in many cases. The issue was raised several times also in the past. The delay in the submission of final data (i.e. including validation) creates problems of planning and co-operation for international organizations. This does not only concern the production of publications with complete, good quality data and metadata, which is a common objective for all 3 data requesters. For the European Commission and Eurostat it results in a delay in the update of the NewCronos domain, the key source of statistical information for Commission reports, which is becoming more and more important in a context where demand for education indicators is increasing. It is also essential for the efficiency of the whole process of production of good quality international education data to receive feedback by countries in a timely manner, an issue which was already discussed in the document on dissemination presented at the 2003 ETS WG. The approach presented in that document still indicates the intention for the future work, although it the exact deadlines for data submission and feedback would need to be revised to become more realistic. However, especially for this year, there is a significant delay also on the side of Eurostat which is not only due to the delay of data submissions but also to resource problems in Eurostat which delay the processing of data submitted by non-oecd countries and data submitted as part of the Eurostat education data collection The Education data collection guidance The Education data collection guidance includes the following parts: a. The joint UOE data collection guidance which consists of 2 volumes: the Concepts and definitions volume (the so-called UOE manual) and the How to use the electronic questionnaires volume, which includes also the description of the missing codes and their use as well as the data collection tables. 3

4 b. The guidance for the Eurostat data collection tables which consists of the additional Concepts and definitions needed for the regional tables and the foreign language learning tables and is complementary to the UOE manual and the specific Eurostat tables. All data collection tools for the 2003 education data collections are available on the public ETCS circa site 2. It has been recognized by all actors that the UOE guidance needs to be improved in the sense of simplifying and clarifying definitions and instructions. That is why the Eurostat, OECD and UIS have decided to update the UOE manual taking into account the results of recent methodological discussions and surveys but also the discussions held during the preparation of 2 user guides related to the UOE data collection: the Eurostat Short guide on education expenditure statistics released in and the OECD Handbook on Comparative Education Statistics expected to be released later in The new UOE manual should be ready for the 2004 UOE data collection which is expected to be launched by the end of March Therefore it will be particularly important for data providers to study this revised manual before starting to complete the 2004 questionnaires as this will be the unique reference for all international education data (UOE and Eurostat tables) The Education data collection tables 2. Input The data collection tables have been standardized to facilitate the collection and processing of data. Informatics tools have been added to facilitate the work both of the data providers and the data requesters. It is essential that countries use these tools according to the instructions included in the How to use the electronic questionnaires volume of the UOE guidance. In order to avoid confusion Eurostat tables have been formatted in the same way as the joint UOE tables and some of the features (e.g. the changes sheet) have been implemented. It is important that countries respect the format of the tables because the arbitrary addition of rows or columns may create delays in the processing of the file or the incorrect use of data in case these interventions are not identified in time. The data requesters have maintained the requested stability of the data collection tables avoiding any changes when this was possible. This will be the case also in No major changes will be implemented in the UOE questionnaires in Some changes may be implemented in the ISCMAP table, but these will be clearly indicated in the revised UOE manual, after they are agreed by the data requesters. On the contrary the necessary changes will be implemented in the Eurostat regional data collection tables to reflect the adoption of the new classification (NUTS ). Countries should already adapt their data collection system accordingly. It is essential to improve the input of international education statistics, that is the quality of the UOE and Eurostat education data and metadata collections. Some important points to improve area: Eurostat Working Paper 3/2003/E/N 24 and available on the public ECTS site at the address:

5 2.1. Table ENRL 1a presents a summary picture of the education system as it combines the highest level of detail for ISCED levels with system level information. It is important that this table is completed with particular care (especially as far as the missing codes are concerned) and that it is consistent with the rest of the tables in the data collection. This should correspond to what is included in the ISCMAP table Table ENRL2 provides information on enrolment adjusted to education finance and education personnel and should always be completed. It is impossible to calculate with confidence key indicators like expenditure per pupil and pupil/teacher ratio without this information, which the data providers are in a much better position to estimate/confirm than the data requesters Missing codes should be used consistently following the instructions in the How to use the electronic questionnaires volume of the UOE guidance. Codes like not applicable should also be consistent across tables including the ISCMAP table. The metadata available for the UOE and Eurostat education data collections need to be improved. Data providers should be aware of the important role that the ISCMAP table as well as past and ongoing surveys carried out by the data requesters (like the Eurostat survey on ISCED 6 or the more recent Finance Country profiles exercise) play in this context. It is particularly important for countries to contribute this information and for data requesters to make it available in a useful form to data users. Although Eurostat has not been able to complete the tasks related to the evaluation of ISCMAP tables and the analysis of the ISCED 6 quick survey in 2003, the work done already in these fields, together with the work done by OECD on the ISCMAP table will need to be taken into account. It is also necessary for data requesters to improve the transparency of the methods used for the calculation of the indicators published as part of this metadata system, an area where there is still a lot of space for improvement. The tools that have already been made available in the questionnaires for the rationalization of additional meta-information necessary to understand the figures included in the questionnaires, should also be used. These are: the CHANGES worksheet which has been included in all UOE and Eurostat tables and the extract notes functionality (separate worksheet) which allows data providers to see all the notes included in a set of tables and harmonise or reduce them. 3. Data validation process To make efficient use of the available resources, data requesters need to present clearly the checking, validation and estimation rules used during data validation. These should be common to all data requesters and should be transparent for data providers giving them the opportunity to implement them on their side before sending the data. This set of common rules which should be agreed by the data requesters before the 2005 UOE data collection would be elaborated on the basis of: - what is included under data verification in Chapter 3 Instructions concerning the completion of the tables of the UOE 2003 manual - the checks that have been implemented through macros in the electronic questionnaires - the validation procedures that are implemented by the three requester organisations This is essential also for the further automation of procedures on both the data requesters and providers sides and for the development of a common culture about the questionnaires both among data providers and among data requesters. These Verification rules could be implemented as a separate volume of the UOE and Eurostat education data collection guidance as well as part of checking tool which would be offered to data providers for use at their end. 5

6 4. Dissemination 5. The dissemination strategy presented at the 2003 ETS WG the same in its broad lines. NewCronos, the Eurostat reference data base, remains the main dissemination tool. All European Commission staff, international organizations and education data providers may have free access to this data base. The short Statistics in focus report Education in Europe - Key Statistics will be maintained. Some key indicators on education finance will also be included. Thematic Statistics in Focus reports will be produced on issues of special interest. The publication series European Social Statistics has been abandoned. In 2004 there will be no detailed tables publication produced by on Education across Europe. Indicators will be provided for the European Commission publication Key Data on Education in Europe 2005, although the involvement of Eurostat and the Education data providers has changed compared to previous editions (see item 6 on the agenda). Education data will be provided for inclusion in other Eurostat Publications. In 2004 a special chapter on education will be included in the Regional Yearbook. Education indicators have been included also in the new way of free dissemination of data (which practically replaces the Eurostat Yearbook ) through the so-called 1000 tables Conclusion Already at the 2003 ETS WG meeting the need for formalization of procedures for the production and dissemination of education and training data was discussed. The minutes (doc EUROSTAT/E3/2003/ETS30) read: Under Point IV.2: Functioning and role of the ETS working group and its subgroups: (Eurostat) will propose written agreements with other actors on both the data development side (e.g. OECD, UIS) and the indicator development side (e.g. SGI). Under Item VII.3: Dissemination of education data Countries were in general favourable to the proposal. However several points were made on possible risks during its implementation. Some countries considered the deadlines given by Eurostat for the validation process too short and rigid. It was also proposed to make sure that contact with the right person in the country has been established before starting the countdown for deadline. The validation should be done in a way that would guarantee that all requester organisations use finally the same data. The usefulness of validation macros integrated in the UOE tables was also questioned. Eurostat will use the feedback by countries and establish a written agreement for the future This written agreement could take the form of a legal basis as is the general guideline for all activities of the European Statistical System. This legal act would commit both the Commission (Eurostat) and the EU Member States concerning the content and the deadlines for the production of statistical data on education, as well as the role of the different actors concerned. The feasibility of this approach and the form of such a legal act will be evaluated, as the education data collections involve also other international organizations on the side of data requesters and other bodies than national statistical institutes on the side of the data providers. However some steps can already be undertaken to document better the production and dissemination process even under the current gentlemen s agreement for the collection of education data. The first results of this attempt should be implemented for the 2005 education data collections. 5 Available on the internet at the address: uage=en 6