Using SDMX To Exchange Development Indicators Abdulla Gozalov United Nations Statistics Division SDMX Expert Group Meeting Seoul, 27-30 Oct 2014
MDG DSD and CountryData DSD MDG DSD developed by Interagency and Expert Group for Millennium Indicators Used for data exchange between UNSD, UNESCO, World Bank CountryData DSD developed for UNSD-DfID project, to support data exchange with National Statistical Offices Same dimensionality; CountryData codelists are supersets of MDGs Both are used for dissemination
UNSD-DfID Project Improving the collation, availability and dissemination of development indicators Improve the coherence and clarity of development indicators Improve accessibility and visibility Enhance knowledge 11 participating countries from Africa, Asia, Middle East Burundi, Cambodia, Ghana, Lao, Liberia, Morocco, Palestine, Rwanda, Thailand, Uganda, Viet Nam
SDMX in UNSD-DfID Project SDMX data sharing established with all project countries Several other countries also joined data exchange Indicators from National Offices delivered to UNSD s CountryData platform National data is then compared to int l indicators to identify discrepancies and their sources
SDMX Registry: Subscriptions and Notifications SDMX Registry is used to facilitate data exchange Country exports and registers data after database update Registry sends a notification message to subscribers, incl. UNSD Subscriber gets and processes updated data Proved to work very well across multiple platforms
Encoding Time Series MDG DSD is inherently heterogeneous, which often causes ambiguity in how a time series should be encoded What should be the value in the SEX dimension for indicator Births attended by skilled health personnel? In addition to inconsistencies, if a country makes an error then corrects the value, this may result in a duplicate time series at the receiver Developed and enforced coding sheets for valid time series
Governance of SDMX Artefacts UNSD maintains all artefacts centrally Repeated requests by the countries to expand supported indicator set All requests must be channeled through UNSD causing significant delays Difficult to enable the countries to independently expand the codelists in the existing model
Metadata Exchange Project established metadata exchange with some of the countries Had to overcome a number of challenges Poor support in SDMX 2.0 compared to 2.1 Metadata often stored outside a database Difficulties exchanging formatted text (format, security) A very useful and positive experience overall Metadata key to explaining discrepancies
Software Tools 9 out of 11 project countries use the DevInfo platform for their indicator database 2 use home-grown solutions A number of modules were developed for DevInfo to support SDMX Exchange Mapping Tool for MDG/CountryData DSD Integrated SDMX Registry These modules are part of the standard DevInfo package, which extends SDMX support to approx. 100 countries
Capacity Building Rather than use a black box approach, project provided training and other capacity development to participating countries Mapping the structures and updating the data is the country s responsibility Country has full control over data exchange, which is carried out without UNSD intervention Fusion Registry operated by Thailand, Morocco SDMX-RI was introduced to the countries
Dissemination Both official MDG indicators and national MDG indicators are disseminated in the SDMX format through UNSD s UNdata platform Using MDG and CountryData DSDs, respectively UNdata SDMX dissemination is powered by SDMX Reference Infrastructure and supports API
Thank you for your attention http://data.un.org/countrydata http://data.un.org UNdata SDMX API http://data.un.org/ws http://unstats.un.org