Implementation of SDMX in the National Institute of Statistics and Geography of Mexico. SDMX Global Conference 2013 Paris, France, Sep 2013

Transcription

1 Implementation of SDMX in the National Institute of Statistics and Geography of Mexico SDMX Global Conference 2013 Paris, France, Sep 2013

2 The way we started implementing SDMX SDMX initiative in 2004, mainly from the IT side, with very low participation of the data producing units. Centered on awareness, development of technical skills, planning of possible implementations and completion of small projects as proofs of concept. Collaboration with the OECD and participation in the groups developing the standard. SDMX has been perceived as a useful tool for domestic and international exchanges.

3 SDMX implementation as a priority for INEGI By the end of 2009, SDMX implementation became a priority for INEGI, it was included in its strategic work program. A multidisciplinary working group was set up with clear goals and responsibilities. Support from the high level authorities of the Institute is essential: understanding the relevance and potential of SDMX as a coordination tool for the development and strengthening of the National Statistical and Geographical Information System. Fully committed to SDMX implementation.

4 SDMX Implementation Group Multidisciplinary departments coordinated by a statistical unit: Statistics data integration (conceptual issues) Information technologies (technical issues) International affairs (agreements) Involvement of data producers

5 Training and skills development Staff members have attended several workshops organized by international organizations to improve their knowledge of the standard. 2010: Internal training workshop for the development of conceptual and technical skills. 2012: International seminar organized in collaboration with EUROSTAT to promote knowledge in national statistics offices from the Economic Commission for Latin America and the Caribbean region.

6 International Collaboration Participating since 2004 in the OECD Expert Group in Data and Metadata Exchange Collaborating since 2010 in the Statistical Working Group (SWG) and in the Technical Working Group (TWG) Collaborating in the SDMX-DDI dialog core group Planned participation in the SDMX-RI User Group coordinated by Eurostat Collaboration with the Andean Community, ECLAC and other Latin-American countries in the implementation of the standards

7 SDMX flows 2010: Agreement with the OECD to replace Excel files by SDMX flows: Short Term Economic Indicators Merchandise Trade Statistics Quarterly Gross Domestic Product at constant prices OECD pilot on Short Term Economic Statistics, Millennium Development Goals, with the UNSD in the adoption of an international DSD. This flow is published in CountryData site. Foreign Trade of Mexico (COMTRADE), looking to establish a regular exchange this year with UNSD. All these flows are available in INEGI s website, along with information about SDMX and its use:

8 SDMX flows Statistical data flows converted to SDMX Foreign trade on merchandises, annual flow. Short term economic indicators, monthly flows. GDP, quarterly. MDG Indicators. STES_OCDE, monthly (pilot test). INFRA_ANNUAL_LABOUR_OECD, monthly and quarterly (pilot test).

9 Technological Infrastructure Based on the SDMX Reference Infrastructure provided by Eurostat Development of extensions for RESTful web services and SOAP web services delivering SDMX streaming flows Export of SDMX-ML to CSV, JSON, DSPL and JSONP formats Collaboration with Istat to send SDMX flows to the OECD s Statistics explorer

10 SDMX sub-site

11 Generates (RI) Reference Infrastructure DSW Mapping store Mapping Assist DSD MA 2.9 NSI SERVICE Imports Dissemination DB DB BaseX Service SOAP RESTful CSV Json Jsonp

12 Generates Generates Replication among instances Development Dataflows Replicator Production DSW DSW Mapping store Mapping DSD MA 2.9 Imports Dissemination DB DB BaseX Service Mapping store DSD MA 2.9 Imports Mapping Dissemination DB DB BaseX Service NSI SERVICE NSI SERVICE SOAP RESTful SOAP RESTful CSV Json Jsonp CSV Json Jsonp

13 Generates Generates INEGI SDMX Data collection architecture Generates DSW Mapping store DSD MA 2.9 NSI SERVICE Imports Mapping Dissemination DB DB BaseX Service SOAP DB Management RI INEGI CSV Json Jsonp SOAP RESTful CSV Json Jsonp SOAP RESTful DSW Mapping store DSD MA 2.9 NSI SERVICE Imports Mapping Dissemination DB DB BaseX Service REST COLLECTOR NSI SERVICE DSD DSW Imports Mapping Assist MA 2.9 Mapping store Dissemination DB BaseX Service DB SOAP RESTful CSV Json Jsonp CSV FILES DB Integration ETL Process (DWH Integration)

14 Key factors for the success of SDMX implementation Skilled staff from both conceptual and technical units. Well defined and internationally agreed DSD. Conceptual and technical support for the implementation of the DSD. Structured data and stored (recommendable to have database which are continually updated). Fully operational technological infrastructure.

15 Some learned lessons Statistical work is critical to obtain the benefits from SDMX implementation. Efficient communication between sender and receiver. Exchanged data must be fully compliant with guidelines and definitions. Having an internationally agreed DSD will make it easier to standardize concepts and comparability. SDMX must be considered as an element of the whole statistical process Limitations in the use of SDMX are the result of different realities that must be faced in the international context, it doesn t mean that SDMX is not useful!

16 Foreseen challenges Implementing SDMX must not be seen as an additional workload. Get the right and best use of the standard in the organizations, recognizing it as one of the tools for modernizing the statistical process. Adoption of global DSD s to our specific context.

17 Next moves Incorporation of SDMX to domestic data exchanges. Develop the concepts and IT infrastructure to disseminate SDMX flows. Built software tools to visualize and analyze information in SDMX formats.

18 Questions and Feedback

19 THANK YOU FOR YOUR TIME Knowing Mexico INEGI Informa