StatDCAT-AP. Virtual meeting 4. 3 June 2016 ISA Programme Action 1.1

Transcription

1 StatDCAT-AP Virtual meeting 4 3 June 2016 ISA Programme Action 1.1

2 Opening, agenda, tour de table

3 Agenda 1. Opening, agenda, tour de table 2. Outcomes Rome meeting, 13 May Objectives of this meeting 4. Overview of DCAT-AP and StatDCAT-AP 5. Extensions: Dimensions and attributes, Quality aspects, Visualisation, Other extensions 6. Mapping theme vocabulary (Eurostat MDR) 7. SDMX Transformation mechanism 8. Any other business 9. Next steps

4 Tour de table

5 Outcome of Rome meeting 13 May 2016

6 Key points from Rome meeting (1) Decisions on extensions: o Not include Number of observations and Dimensions as keywords o Include Number of data series o Further discussion on Link to visualisation, Dimension as property, Quality aspects, Statistical population, Statistical unit and Length of time series o Discuss proposals for RDF terms

7 Key points from Rome meeting (2) SDMX transformation mechanism o Not part of the specifications (relevant for SDMX implementers migrating to StatDCAT-AP) move to annex o Slight preference for the use of a Metadata Structure Definition (recommendation?) Theme vocabulary mapping o Publications Office to propose mapping from Eurostat themes to Metadata Registry (MDR) data themes

8 Objectives of this meeting

9 Intended outcome Agree on proposed extensions Review mapping of themes Eurostat Metadata Registry (MDR) Reach common understanding of proposed SDMX transformation mechanism Identify further issues for the future Prepare for public review period

10 Overview of DCAT-AP and StatDCAT-AP

11 Model diagram DCAT-AP

12 Catalogue Mandatory Recommended Optional dcat:dataset dct:description dct:publisher dct:title foaf:homepage dct:language dct:license dct:issued dcat:themetaxonomy dct:modified dct:haspart dct:ispartof dcat:record dct:rights dct:spatial

13 Mandatory Recommended Optional Dataset dct:description dct:title dcat:contactpoint dcat:distribution dcat:keyword dct:publisher dcat:theme adms:identifier adms:sample adms:versionnotes dcat:landingpage dct:accessrights dct:accrualperiodicity dct:conformsto dct:hasversion dct:isversionof dct:identifier dct:issued dct:language dct:modified dct:provenance dct:relation dct:source dct:spatial dct:temporal dct:type foaf:page owl:versioninfo StatDCAT-AP to add optional properties: dqv:hasqualityannotation qb:attribute or stat:attribute qb:dimension or stat:dimension schema:population stat:numseries stat:statunit

14 Distribution Mandatory Recommended Optional dcat:accessurl dct:description dct:format dct:license adms:status dcat:bytesize dcat:downloadurl dcat:mediatype dct:conformsto dct:issued dct:language dct:modified dct:rights dct:title foaf:page spdx:checksum StatDCAT-AP to add optional property: dct:type

15 Dimensions and attributes

16 Dimensions and attributes Requirement to expose information about: o Dimensions: e.g. observations related to sex, age, etc. o Attributes: e.g. observations expressed in certain units Option 1: re-use properties qb:dimension and qb:attribute from Data Cube Vocabulary Option 2: define new properties stat:dimension and stat:attribute in StatDCAT-AP namespace

17 Option 1: Data Cube properties Expected values: URI of qb:dimensionproperty and qb:attributeproperty However, these properties are not directly attached to qb:dataset This may lead to confusion for datasets that are published as qb:datasets

18 Option 2: StatDCAT-AP properties Allows semantics to be precisely defined to meet StatDCAT-AP requirements Expected values can still be: URI of qb:dimensionproperty and qb:attributeproperty For Data Cube datasets, values can be derived (copied) from qb:dimension and qb:attribute

19 Quality aspects

20 Short-term vs. longer-term approach Quality aspects are very important for datasets in general and statistical datasets in particular Due to time and resource constraints we cannot fully address the issue now Propose to address it in two phases: o Short-term: provide mechanism to link to existing quality information in StatDCAT-AP, version 1 o Longer-term: consider integrated quality framework as basis for extensions to StatDCAT-AP, version 2

21 Combining ESMS and ESQRS: the Single Integrated Metadata Structure (SIMS) of the European Statistical System

22 Short-term: annotation Link to existing document/webpage with quality information, or provide plain text Use property: dqv:hasqualityannotation from W3C Data Quality Vocabulary (in development) Expected value: URI (e.g. webpage) or plain text conformant to specification of oa:annotation :Dataset-001 a dcat:dataset ; dqv:hasqualityannotation :Annot-001. :Annot-001 a dqv:qualityannotation ; oa:hasbody <URL> ; oa:hastarget :Dataset-001 ; oa:motivation oa:commenting. :Dataset-001 a dcat:dataset ; dqv:hasqualityannotation :Annot-001. :Annot-001 a dqv:qualityannotation ; oa:hasbody [ oa:text "Some text" ] ; oa:hastarget :Dataset-001 ; oa:motivation oa:commenting.

23 Longer-term: Quality aspects of SIMS Eurostat's Single Integrated Metadata Structure includes specific quality aspects: o e.g. Accessibility and clarity; Quality management; Relevance; Accuracy and reliability; Timeliness and punctuality; Coherence and comparability This set of aspects can form the basis for future extensions to StatDCAT-AP, or even to DCAT-AP

24 Property: Statistical unit ESMS concept STAT_UNIT: o o Defined as entity for which information is sought and for which statistics are ultimately compiled Usage note: list the basic units of statistical observation for which data are provided. These observation units (e.g. the enterprise, the local unit, private households,...) can be different from the reporting units used in the underlying statistical surveys New property in StatDCAT-AP namespace: stat:statunit Expected value: free text Or: defer to deeper discussion on quality aspects?

25 Property: Statistical population ESMS concept STAT-POP: o o Defined as: total membership or population or "universe" of a defined class of people, objects or events Usage note: describe the target statistical population (one or more) which the data set refers to, i.e. the population about which information is to be sought Use property: schema:population, Any characteristics of the population used in the study, e.g. 'males under 65'. Expected value: free text Or: defer to deeper discussion on quality aspects?

26 Visualisation

27 Link to visualisation Visualisation can be seen as a type of Distribution Use dcat:accessurl and dct:type Type: MDR../distribution-type/VISUALIZATION (to be added) Example: :Visual-001 a dcat:distribution ; dcat:accessurl <URL of page> ; dct:type <MDR distribution type>.

28 Type of distribution To support modelling approach for visualisation Use property: dct:type Expected value: URI of type (e.g. visualisation) Also to be discussed for DCAT-AP as part of larger discussion on Distributions that are not files, see:

29 Other extensions

30 Number of data series Information on how values in the Dataset are grouped o Dataset contains data for three regions with three values for each region number of series is three while the number of observations is nine New property in StatDCAT-AP namespace: stat:numseries Expected value: integer

31 Time coverage of the data series Proposed to express this as start and end of data series, e.g rather than two years Use property: dct:temporal Expected value: time period with schema:startdate and schema:enddate Already in DCAT-AP; no extension necessary

32 Mapping theme vocabulary

33 Eurostat themes MDR Data themes Eurostat Metadata Registry (MDR) Themes Title Title Code Type Theme 1 General and regional statistics general folder Theme 2 Economy and finance Economy and finance economy folder Theme 3 Population and social conditions Population and society popul folder Theme 4 Industry, trade and services Economy and finance icts folder Theme 5 Agriculture, forestry and fisheries Agriculture, fisheries, forestry,foods agric folder Theme 6 International trade Economy and finance external folder Theme 7 Transport Transport transp folder Theme 8 Environment and energy Environment Energy envir folder Theme 9 Science and technology Science and technology science folder *Legend Red clear: no equivalence Red: partial equivalence Orange: equivalence by combing two terms Green: equivalence

34 SDMX Transformation mechanism

35 How to produce DCAT-AP metadata Organisations are free to choose how to create DCAT-AP from their systems o For SDMX users the specification defines a mapping between SDMX-ML structural metadata and DCAT-AP o For those not wishing to use SDMX, the organisation must make its own map between the metadata in its system and DCAT-AP Two approaches are under consideration: o SDMX structural metadata to DCAT-AP o SDMX metadata set to DCAT-AP

36 SDMX Structural Metadata (Example) SDMX Dataflow DCAT-AP Dataset DCAT-AP Dataset DCAT-AP Dataset

37 SDMX Metadata Set valid content defined by Metadata Structure Definition (MSD) Metadata Attributes Defined in MSD Metadata Report Metadata Target Attribute Values

38 DCAT-AP Transformation Mechanism Data Publisher Organisation These components can be developed in Java and.net and integrated into SDMX systems or used in SDMX conversion tools SDMX Data Reader SDMX Data Writer SDMX Structural Metadata Repository Other Metadata Sources Intermediary File or Data Stream Choices SDMX-ML Structure SDMX Metadata Set MSD SDMX Data Validator DCAT-AP

39 Any other business

40 More issues? Comments, questions?

41 Next steps

42 Planning December 2015: invitations to stakeholders, set up collaboration infrastructure January 2016: collect requirements and suggestions 5 February 2016: Familiarisation Webinar February 2016: first draft based on initial analysis and issues raised 11 March 2016: first virtual WG meeting to discuss first draft 15 April 2016: second meeting; to discuss draft mapping and implementation options 6 May 2016: second draft available for review, incorporating comments and further development 13 May 2016: third meeting (face-to-face plus Adobe Connect) in Rome; to discuss mapping issues in practice End of May 2016: third draft, including full mapping proposal and usage of controlled vocabularies 3 June 2016: fourth virtual WG meeting to agree schedule for public review June 2016: preparation of final draft for public review July and August 2016: public review period Mid-September 2016: fifth virtual WG to discuss and resolve public comments received End of September 2016: approval of StatDCAT-AP version 1 for publication

43 Public review period planning Preparation of final draft early to mid-june Review of final draft by ISA mid- to late June Announcement (Joinup, mailing lists) late June Gathering comments July and August Final WG meeting to resolve received comments mid-september (Doodle to be issued) Publication StatDCAT-AP 1.0 end September

44 Project Officers Visit our initiatives Get involved on Twitter Join the SEMIC group on LinkedIn Join the SEMIC community on Joinup