SDMX Global Conference Washington D.C., 2 4 May 2011 The European Census Hub Changing the Business Model Adam Wroński, Marco Pellegrino, Nadezhda Vlahova, Bengt-Åke Lindblad Eurostat, Quality, Methodology and Information Systems Directorate 1
The European Census Hub provides: Data from the 2011 population and housing censuses in the European Statistical System Methodologically harmonised data, structured according to tables agreed with Member States (E.U. Census Regulation) Easy access to detailed census tables (data and metadata) Management of massive amounts of data produced and controlled by Member States SDMX Global Conference, Washington D.C., 2-4 May 2011 2
E.U. Regulatory System: Census Implementing measures Regulation (EC) 763/2008 on population and housing censuses authorises the European Commission to adopt implementing measures on: technical specifications of the topics and their breakdown (Regulation (EC) 1201/2009) programme of the statistical data and metadata to be transmitted to Eurostat (Regulation (EU) 519/2010) quality reporting and technical format of data transmission (Regulation (EU) 1151/2010) SDMX Global Conference, Washington D.C., 2-4 May 2011 3
Driving force: Statistical working group on Census Eurostat Census task-force, April 2007 Definition of the main objectives Analysis of different technical possibilities (pros & cons) Agreement that the pull with hub approach could be tested: launch of the pilot project SDMX Global Conference, Washington D.C., 2-4 May 2011 4
How do we exchange data? SDMX Global Conference, Washington D.C., 2-4 May 2011 5
The Hub concept Hub mode of the pull approach: based on the concept of data sharing, where a group of partners agree on providing access to data according to standard processes, formats and technical standards. The Hub mode offers the following benefits: decoupling of NSIs' architecture / systems from the central Hub via standard data model, formats and techniques Limited investment for data providers, as the SDMX infrastructure is made available by Eurostat Re-usability: build once, use it many times (with minimum extra effort) SDMX Global Conference, Washington D.C., 2-4 May 2011 6
The Census Hub architecture Pull approach Hub mode SDMX-RI (Reference infrastructure) SDMX Global Conference, Washington D.C., 2-4 May 2011 7
How the Hub works National Statistical Institute Eurostat Census Hub National Statistical Institute SDMX Global Conference, Washington D.C., 2-4 May 2011 8
SDMX Global Conference, Washington D.C., 2-4 May 2011 9
Status as of April 2011 System of 60 DSDs (built on SDMX 2.0) in use for 31 countries of the European Statistical System SDMX cross-sectional data format and queries Public user registration, offline query execution system Multilingual Hub interface Tests on-going with real tables Infrastructure will be maintained (and data stored) at least until 2025 SDMX Global Conference, Washington D.C., 2-4 May 2011 10
Example: DSD for Table 6 (Marital Status) Dimensions ID CONCEPT CODELIST TIME Time period or range CL_TIME GEO Geographical area CL_GEO SEX Sex CL_SEX FST Family status CL_FST LMS Legal marital status CL_LMS ID ATTACHMENT LEVEL CODELIST OBS_STATUS Observation CL_OBS_STATUS OBS_LEVEL Observation CL_OBS_LEVEL OBS_NOTE HC_NOTE Attributes Observation Series CAS Current activity status CL_CAS POB Country/place of birth CL_POB COC Country of citizenship CL_COC AGE Age CL_AGE FREQ Frequency CL_FREQ Measures ID NAME OBS_VALUE Observation value SDMX Global Conference, Washington D.C., 2-4 May 2011 11
Elements for SDMX DSD and data message Data Structure Definitions (Key families) Concepts Codelists Geo Codelists SDMX data message example (cross-sectional), SDMX Query message example and Dummy hypercube in CSV format MIG XML schema (XSD file) Explanatory notes Census Hub WS Implementation Guideline Census Hub WS Security Guideline Census 2011 Regulation Census 2011 Explanatory notes SDMX Global Conference, Washington D.C., 2-4 May 2011 12
Where are we - ESS? Countries already integrated (12) Czech Republic, Germany, Ireland, Italy, Luxembourg, Poland, Portugal, Spain, Norway, Sweden, Slovenia, Belgium Not yet integrated but in various stages of implementation (12) Bulgaria, Cyprus, Estonia, France, Greece, Hungary, Latvia, Lithuania, Malta, Netherlands, Switzerland, United Kingdom Asking for more information (7) Austria, Denmark, Finland, Iceland, Liechtenstein, Romania, Slovakia SDMX Global Conference, Washington D.C., 2-4 May 2011 13
Next steps: 2012-2013 More user-friendly interface Search Support to explanatory and quality metadata Demo tour of the application On-line user guide Web-services administration LAU management tool (allowing Member States to update codes for municipalities) Software published as Open Source SDMX Global Conference, Washington D.C., 2-4 May 2011 14
Lesson learnt and benefits in participating Statistical needs first. Then, technological aspects. Capacity-building is a must Participating organisations are gaining a good in-house experience in SDMX and its implementation A system of distributed databases is harmonised through the use of SDMX standards and content guidelines SDMX-RI can be reused for sharing data in other domains Limited cost for installations, development costs can be reduced Step forward towards generic solutions for statistical domains SDMX Global Conference, Washington D.C., 2-4 May 2011 15
For more information: Nadezhda.Vlahova Marco.Pellegrino @ ec.europa.eu Bengt-Ake.Lindblad Adam.Wronski Eurostat, Unit B5, Statistical Information Technologies SDMX Global Conference, Washington D.C., 2-4 May 2011 16