SDMX. The basis for renovating the ESS Metadata Systems. August Götzfried Head of Unit EUROSTAT

Size: px
Start display at page:

Download "SDMX. The basis for renovating the ESS Metadata Systems. August Götzfried Head of Unit EUROSTAT"

Transcription

1 SDMX The basis for renovating the ESS Metadata Systems August Götzfried Head of Unit EUROSTAT 1

2 The ESS Vision for the next decade The Vision for the European Statistical System (ESS) for the next decade the basic ideas: The ESS vision wants to overcome statistical stove pipes and move towards more integrated statistical production processes; It wants to better integrate the ESS in terms of IT infrastructure, software, data quality, metadata, methodology etc. (both in terms of horizontal and vertical integration); It aims to stimulate the better use of administrative data in the statistical data production processes; It also calls for better integrated statistical cross-cutting legislation in covering larger statistical domains. This vision also impacts the ESS Metadata Standards and the ESS Metadata Systems. 2

3 The starting point Until 2009 The Eurostat and ESS metadata systems were not fully SDMX compliant, i.e. Reference (explanatory) metadata structures were used containing statistical concepts not sufficiently harmonised or not clearly explained; Different reference metadata structures were used in different statistical domains (e.g. different quality report structures); In 2009 The SDMX Content-oriented Guidelines were published. These SDMX Content-oriented Guidelines were the basis for upgrading the Eurostat and ESS metadata systems and making them SDMX compliant. 3

4 Using the SDMX Content-oriented Guidelines The SDMX Content-oriented Guidelines (SDMX COG) comprise the following 4 annexes: Annex 1: Cross-domain Concepts; Annex 2: Cross-domain Code Lists; Annex 3: Statistical Subject-Matter Domains; Annex 4: Metadata Common Vocabulary; Two SDMX compliant Metadata Structure Definitions were defined for reference (explanatory) metadata using a sub-set of the 66 SDMX Cross-domain Concepts (and sub-concepts) listed in annex 1 of the SDMX COG. 4

5 The Euro SDMX Metadata Structure The Euro SDMX Metadata Structure (ESMS) is the main report structure for reference (explanatory) metadata; is fully SDMX compliant; uses 21 concepts (plus additional sub-concepts) taken from the SDMX cross-domain concepts; is a metadata structure definition which is used across all statistical domains; is user oriented (the ESMS files are normally published on web-pages); incorporates the main criteria measuring data quality; when fully implemented, will lead to more than 2000 Eurostat and national ESMS files produced and disseminated. 5

6 The Euro SDMX Metadata Structure Example: ESMS Harmonised indices of consumer prices Bulgaria 6

7 The ESS Standard Quality Report Structure The ESS Standard Quality Report Structure (ESQRS) is the main report structure for reference metadata related to data quality; is SDMX compliant (however: some limited quality related concepts should be included in the SDMX COG in the future, e.g. imputation rate); uses 8 main concepts taken from the SDMX cross-domain concepts plus more detailed sub-concepts measuring data quality; is a metadata structure definition which is used across all statistical domains; is data producer oriented and therefore sometimes not publicly released; will harmonise quality reporting within the European Statistical System. 7

8 The ESS Standard Quality Report Structure 8

9 Using the ESMS and the ESQRS within the European Statistical System The implementation of the ESMS and the ESQRS: The two Metadata Structure Definitions are stored in the Euro SDMX Registry (= main SDMX Registry used within the ESS); The ESMS is fully used at Eurostat and more and more used when national reference metadata are produced; The ESQRS gets now more and more used at Eurostat and when national quality reporting is done. Attention The existing Eurostat and national reference metadata and quality reports need to be converted to these ESS standards. This needs to be done for more than 50 statistical domains. 9

10 The ESS Metadata Handler as facilitator for implementing the SDMX compliant metadata standards The ESS Metadata Handler (ESS MH): comprises the IT applications managing the production, exchange and dissemination of ESS metadata; accommodates the ESMS and the ESQRS structures for production, exchange and dissemination of the national and Eurostat metadata files; allows the on-line production of the national ESMS and ESQRS files and their exchange within the ESS; stimulates the use of the SDMX based ESS metadata standards within the ESS and the further integration of the ESS metadata systems. 10

11 The ESS Metadata Handler as facilitator for implementing the SDMX compliant metadata standards National Statistical Institute ESS Metadata Handler ESS Metadata Handler Eurostat Website National dissemination Eurostat Metadata Files National Metadata Files (ESMS, ESQRS ) edamis National + Eurostat Metadata Files (ESMS, ESQRS ) National Metadata Files (ESMS, ESQRS ) PROCESSING PROCESSING and ANALYSIS DISSEMINATION 11

12 What is next? Our further work will concentrate on: The full and broad implementation of the ESMS and the ESQRS in the European Statistical System; The further deployment and opening of the ESS Metadata Handler; The definition of additional Metadata Structure Definitions based on the SDMX COG such as the Euro Process Metadata Structure (EPMS) using the GSBPM; The improvement of the international exchange of the ESS metadata with other international organisations in order to reduce the burden on EU Member states; The contributions to the updating of the SDMX Contentoriented Guidelines. 12