Evolutions in the Copernicus Coordinated Data Access System

Transcription

1 Evolutions in the Copernicus Coordinated Data Access System for EO Satellite Data J. Martin, R. Knowelden, G. Vingione Ground Segment and Mission Operations Department, Earth Observation Programmes Directorate, European Space Agency

2 Overview 1. The Copernicus Programme and the CSCDA: from pre operations to operations, data access, data policy and growing demands, service charter and user communities 2. The Copernicus Ground Segment and the Coordinated Data Access System (CDS). CDS data flows, functionality and tools. CDS current status and ongoing evolutions. 3. Next generation Earth Observation (ngeo) functionality and integration with CDS 4. Bulk dissemination via Data Hub Service (DHuS) Software, functionality and access examples. 2

3 The Copernicus Programme and the CSCDA Pre Operations (since 2008 until Sentinels): more than 10 different ground segment and 20 EO space mission and has served a growing user community with a rapidly increasing library of EO products. Space Component Operations: with the Sentinel satellites, outstanding increases gradually expected in number and volume of data products as well as in number of users. Copenicus Space Component Data Access (CSCDA) evolutions shall support a smooth and seamless transition to the routine operations In Situ Component Services Component 3

4 Copernicus Space Data Access A major undertaking A major European Earth Observation Ground Segment design challenge: Systematic processing and availability of the complete acquired data corresponding to a sustained generation rate (24h/7d) of a continuous stream of 500Mb/s of user products (Sentinel 1, 2, 3 A series) Distributed interconnected network of centres (improved redundancy) Local centre access capacity built on top of a powerful and scalable network With First Results! 4

5 Northern Antartic Peninsula Coastal Islands (Sentinel-1, 13 April 2014) Sentinel 1 A+B will be the backbone of future ice charting Leif Pedersen, MyOcean

6 Sentinel 1 provides a new dimension for maritime monitoring " Dr. Olaf Trieschmann, EMSA Rotterdam Harbour, Netherlands (Sentinel-1, 15 April 2014)

7 Copernicus Data Policy Free, Full and Open Users shall have free, full and open access to Copernicus dedicated Sentinel data and Copernicus service information The CSC Ground Segment architecture implements this policy, and includes an evolutionary approach to further enhance the data exploitation by the broad user community The CSC Ground Segment features dedicated data access infrastructure solutions, tailored to the needs of the various user typologies 7

8 CSC Ground Segment Data Access Tailored infrastructures adapted to different User Typologies CSC GS Data Access Copernicus Services Access Scientific / Other Access International Agreements Access MSs Collaborative Access Operated by int. partners Operated by MSs Copernicus Services International International Mirrors Mirrors Int l (Inc local Mirrors station) National Mirrors (Inc National local National station) Mirrors (Inc local station) Mirrors Scientific/ Other users International users MSs CollGS Users e.g. public, scientific, commercial ESA funded R&D projects 8

9 Common User Service Charter The CSC Ground Segment provides a set of main services to all users, independently from the access infrastructure User REGISTRATION User ACCESS MANAGEMENT DISCOVERY Service Management of user account creation with dedicated username and password Management of user requests for category assignment (upon agreement of Terms and Conditions) Dataset and product search (including visibility of planned acquisitions) VIEW Service DOWNLOAD Services BULK DISSEMINATION Service EMERGENCY Service OFFLINE MAPPING Service HELP DESK Service RSS and HOSTED PROCESSING Service Visualization of browse image Interactive Download (via web browser or using a specific download manager) Systematic background download based on defined filters (e.g. area, time window, product type, ) Dissemination of large volume of products on physical media, by land mail or via mirroring on dedicated data hub. Opening of an emergency dossier upon user call and management of data collection (tasking, selection from archive) Coordination of data collection for ad hoc requests Support and communications to users. Management of inquiries, complains, and suggestions. Advertisement of datasets. Escalation of issues to dedicated mission level specialists and help desks. Research Support Service and Hosted Processing Service operations 9

10 Data Flows from CCMs to Copernicus Users INSPIRE Conference 2014 Aalborg, 19 June 2014

11 CDS Current Status The CDS is currently in routine operations in version 2.1 Most of the CCMs are already integrated, including recent upgrades to finalize machine to machine interfaces or include new missions (SPOT 6/7, PROBA V). Completion of currently tasked CCMs expected Q Major datasets and ad hoc orders routinely delivered to users

12 Major Current CDS Tools Datawarehouse

13 CDS On going Evolutions Overall evolutions of the CDS to version 3 and beyond have recently commenced (Q1 2014). These are foreseen to last 2 years and will prepare the system for: Sentinels integration and increase in data volumes thus entailed, Next Generatation Earth Observation (ngeo) integration ngeo (see later) Data Hub Service (DHuS) Software (see later) The aforementioned Services Charter All evolutions will ensure continuity with the current baseline. A clear objective is to design the CDS system taking into account the INSPIRE Directive, ensuring compliancy with Discovery and View Services and aiming to maximize the accessibility of the geospatial data and services it provides.

14 next generation EO (ngeo) Evolutions A core evolution of the CDS will be the integration of ngeo into version 3, providing: Core data access services offered through a uniform web based interface, supported on major browsers Comprehensive (INSPIRE compliant) DISCOVERY, VIEW and DOWNLOAD services provided Friendly to Web and OGC standards for Earth Observation Detailed product image as map layers (in selected cases) Sharable shopcarts, dataset search, browse and product URIs Systematic user defined downloads using performant download manager

15 next generation EO (ngeo) Evolutions A core evolution of the CDS will be the integration of ngeo into version 3, providing: Core data access services offered through a uniform web based interface, supported on major browsers Comprehensive (INSPIRE compliant) DISCOVERY, VIEW and DOWNLOAD services provided Friendly to Web and OGC standards for Earth Observation Detailed product image as map layers (in selected cases) Sharable shopcarts, dataset search, browse and product URIs Systematic user defined downloads using performant download manager INSPIRE Conference 2014 Aalborg, 19 June 2014

16 ngeo Service Highlights DISCOVERY Service Search the catalogue for available or planned products, using user selected filters Subscription to catalogue search with RSS notification of new products Sharing of catalogue searches as URIs Interferometric search support VIEW Service Viewing of browse images of products found in the catalogue as a map overlay, allowing panning and zooming Support up to full resolution image for selected products/datasets Compatible with third party map clients supporting the standards Web Map Service (WMS) protocol from the Open Geospatial Consortium DOWNLOAD Service Download Manager for download of individual products selected via DISCOVERY service Catalogue search results provide a download address for each product Download Manager is an open source multi platform application provided by ESA

17 Bulk Dissemination: Data Hub Software (DHuS) BULK DISSEMINATION Service Online Access to Rolling Archive of Sentinel Data Products Simple Open and Free Registration Search, Preview, Inspection and Download Intuitive Web and Scripting Interface for bulk download via http open data protocol

18 DHuS: Search bash$ curl u user p password = sentinel AND germany

19 DHuS: Online Product Inspection bash$ curl u user p password ontent?$path=annotation/*/product/qualityinf ormation/qualitydatalist/qualitydata/downlin kquality/*

20 DHuS: Individual and Bulk Download #!/bin/bash # Disclaimer: API will change backwards compatibility to be maintained fom Data Hub version 1 echo starting download curl u user p password -metalink sentinel AND germany &$format=metalink echo done!

21 Further Information CSCDA Homepage: Sentinel Technical Web: Sentinel Data Hub: Free and Open (feel free to register) Sample Data Until Sentinel 1 In Orbit Commissioning Review (9/2014 TBC) Full data access and detailed API to be published accordingly Salar de Uyuni, Bolivia (Sentinel-1, 20 April 2014)