ESS Standards for reference metadata and quality reporting (ESMS, ESQRS, SIMS)

Size: px
Start display at page:

Download "ESS Standards for reference metadata and quality reporting (ESMS, ESQRS, SIMS)"

Transcription

1 ESS Standards for reference metadata and quality reporting (ESMS, ESQRS, SIMS) Giorgia Simeoni, Istat, Head of project Design and implementation of quality reports to accompany data dissemination and transmission ESTP Training Course Information standards and technologies for describing, exchanging and disseminating data and metadata Rome, April 2016 THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION

2 Outline Introduction to quality reporting User and producer oriented quality reporting ESS quality reporting: committment ESS quality reporting: standards The Single Integrated Metadata Structure (SIMS) ESS standard structures for metadata and quality reporting ESS implementation process Reference metadata in SDMX

3 Introduction to quality reporting Definition of Quality Reporting: Quality reporting is the preparation and dissemination, on a regular or irregular basis, of reports conveying information about the quality of a statistical product or survey (, 2003b). A quality report provides information on the main quality characteristics of a product so that the user should be able to assess product quality. In the optimal case quality reports are based on quality indicators (Bergdahl et al., 2007)

4 Introduction to quality reporting Main features of a quality report: Object: a statistical product or statistical process Target recipients: users or producers of official statistics Purposes: to accompany official statistics dissemination, to support quality assessment Contents: quality dimensions, quality indicators Further characteristics: standard structure, different levels of detail tailored on recipient and purpose

5 Introduction to quality reporting Some examples: ONS Information on quality and methodology DESTATIS Quality reports INE Spain Standardised Methodological Report

6 User and producer oriented quality reporting User oriented quality reports To communicate quality to users To report on strengths and limits in order to support a proper use of statistics To enhance transparency Concise, not much technical and focussed on the statistical output

7 User and producer oriented quality reporting Producer oriented quality reports To support management monitoring To identify points for further improvements To produce reference documentation for quality assessment (e.g.: audit and self-assessment) Detailed, comprehensive, supplemented with quality indicators and focussed more on the statistical process, even if following product quality dimensions

8 ESS Quality reporting: commitment European statistics Code of Practice Indicator 4.3: Product quality is regularly monitored, assessed with regard to possible trade-offs, and reported according to the quality criteria for European Statistics Quality Assurance Framework of the European Statistical System Indicator Methods at product/survey level User oriented quality reports. User oriented quality reports are made available to the public. Producer oriented quality reports. Producer oriented quality reports are published regularly [ ]. Product quality monitoring. Users and producers quality reporting are used for regular quality monitoring over time.

9 ESS Quality reporting: commitment European Statistical Law Member States shall provide the Commission () with reports on the quality of data transmitted (REGULATION (EU) 2015/759 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 29 April 2015 amending Regulation (EC) No 223/2009 on European statistics)

10 ESS quality reporting: standards Historically, ESS requirements for reference metadata ad quality reporting derive from EU regulations Very detailed and domain specific information required, agreed in domain specific WGs Transmission to was made in several ways (e.g. file word, excel sent by ) A lot of efforts spent in last years trying to harmonise and standardise ESS quality reporting

11 ESS quality reporting: standards

12 The Single Integrated Metadata Structure (SIMS) SIMS is a dynamic and unique inventory of ESS standard reference metadata statistical concepts with definitions and reporting guidelines.

13 The Single Integrated Metadata Structure (SIMS) The first version of SIMS was obtained by integrating and harmonising the concepts of the two already existing ESS standard templates: Euro SDMX Metadata Structure (ESMS) recommended for useroriented quality reporting ESS Standard for Quality Reports Structure (ESQRS) recommended for producer-oriented quality reporting

14 The Single Integrated Metadata Structure (SIMS) ESMS and ESQRS have been mapped and merged creating a unique structure based on the following principles: All the concepts in the existing structures are included; The statistical concepts appear only once; The same concept names and the same quality indicators are always used in the different metadata and quality report structures; The descriptions and the guidelines for the compilation of the concepts and sub-concepts have been reviewed and harmonised

15 SIMS 2.0 Some minor discrepancies between ESMS and ESQRS, mostly in terms of section headings, still remained Version 2.0 of SIMS, ESMS and ESQRS: - the coherence between ESMS and ESQRS has been increased mainly by adding to ESQRS the conceptual metadata from ESMS that were missing in ESQRS - the alignment with ES Code of Practice has been improved by appropriately merging and re-ordering the statistical concepts 15

16 SIMS 2.0

17 ESMS 2.0 Commission Recommendation of 23 June 2009 on reference metadata for the European Statistical System: ESMS is to be used by NSIs when compiling reference metadata in the different statistical areas and exchanging them within the ESS or beyond.

18 ESQRS 2.0

19 ESS Handbook for quality reports

20 ESS Quality and Performance Indicators Data completeness - rate Sampling error - indicators Over-coverage - rate Common units - proportion Unit non-response - rate Item non-response - rate Data revision - average size Imputation - rate Time lag - first results Time lag - final results Punctuality - delivery and publication Asymmetry for mirror flows statistics coefficient Length of comparable time series Data tables consultations Metadata - consultations Metadata completeness - rate

21 ESS Implementation process 1. Implementation of ESMS files at level 2. ESMS files dissemination through database 3. Gradual implementation (domain by domain) of ESMS/ESQRS at national level supported by the ESS Metadata Handler (next presentation)

22 ESS Implementation process 4. grants to support national implementation of ESMS/ESQRS 5. Definition of SIMS, big efforts in conceptual harmonisation and standardisation 6. Effort on defining rules for interoperability between systems 7. Improvement of standards: version 2.0 for SIMS, ESMS and ESQRS

23 ESS Implementation process 8. Endorsement of SIMS 2.0, ESMS 2.0 and ESQRS 2.0 by the ESSC SIMS will be the standard for quality reporting according to Article 12 of Regulation 223/2009 on European statistics. The new reporting structures should be implemented in the different European statistical domains by 2018.

24 Reference metadata in SDMX SDMX: the technical standard for Statistical Data and Metadata exchange Based on the exchange of xml messages (SDMX-ml), but including an Information Model and an IT infrastructure Introduction of SDMX: supported quality reporting harmonisation influenced the structure for ESS quality reports 24

25 SDMX Information Model* * SDMX self-learning package No. 2 Student book

26 Metadata Structure Definition The Metadata Structure Definition is made up of 2 main parts: - Identifiers - Report structure: a set of report attributes ESMS and ESQRS have been reported in MSD. Identifiers are usually time period, metadataflow, data provider The report attributes correspond to the statistical concepts required from the template.

27 Metadataflow Metadataflow is the conjuction between the template and the statistical domain (reported as a CATEGORY of a CATEGORY SCHEME in SDMX language): e.g. it says that for the domain TOURISM DEMAND the quality report follows ESQRS template (MSD) It reports also the reporting frequency (annual, quarterly )

28 Metadataset Metadataset is the filled in quality report It should refer to a metadataflow and report the values for the identifiers of the MSD

29 References (2015) Single Integrated Metadata Structure v 2.0 ESMS 2.0 and ESQRS (2015) Single Integrated Metadata Structure and its Technical Manual Technical-Manual.pdf/6013a162-e8e2-4a8a e3318cbb39 (2014) ESS Handbook for Quality reports pdf/d a a e0ac7c655 (2014) ESS Guidelines for the implementation of the ESS quality and performance indicators (QPI) pdf/5c b770-4a7c-9c2f-bf733e6b1f31 ESS Standard for Quality Reports Structure (release 1, October 2010) Euro SDMX Metadata Structure (release3, March 2009) (2009). Methodologies and Working papers ESS Standards for Quality Reports. (2003a), Definition of Quality in Statistics. Working Group on Assessment of Quality in Statistics, Luxembourg, October 2-3. (2003b), Glossary. Working Group on Assessment of Quality in Statistics, Luxembourg, October 2-3.