MONTHLY OPERATIONS REPORT

Transcription

1 MONTHLY OPERATIONS REPORT MOR#019 Reporting period from 16-Jun-2015 to 15-Jul-2015 Reference: PROBA-V_D5_MOR-019_ _v1.0 Author(s): Erwin Wolters, Dennis Clarijs, Sindy Sterckx, Roger Kerckhofs Version: 1.0 Date: 17/07/2015

2 DOCUMENT CONTROL Signatures Author(s) Erwin Wolters, Dennis Clarijs, Sindy Sterckx, Roger Kerckhofs Reviewer(s) Dennis Clarijs Approver(s) Dennis Clarijs Issuing authority Change record Release Date Pages Description Editor(s)/Reviewer(s) /07/2015 All Initial version PROBA-V_D5_MOR-019_ _v1 Page 2 of 18

3 TABLE OF CONTENT 1. Summary System Infrastructure Image Processing Services Ingested and archived products Generated and archived products Backup and archiving service Dissemination service End-user activity Image Calibration services Radiometric Calibration Geometric Calibration Anomalies System related issues Image processing issues Scheduled activities for the next period(s) Operational remarks PROBA-V_D5_MOR-019_ _v1 Page 3 of 18

4 1. Summary In the past month, all syntheses were incomplete, mainly due to decompression errors and missing spectral bands. Two syntheses show a major gap, 16 a large gap, 9 a medium gap and 3 a minor gap. A new release of the DIF component is currently under test. Target is to minimize the amount of data that is discarded if a decompression error occurs. Release is expected in week 29. ESA is investigating the cause of the corrupt data on the platform side. In this reporting period, five transfer frame files were missing, 4 due to an antenna malfunction and 1 due to a scheduling mistake. Furthermore, the switch to a new FTP server in Kiruna was successful. All data is now received via the new server with a 100% increase in download speed as opposed to the old FTP server. There were no issues with the image quality during this reporting period, but for both radiometric and geometric calibration, a new set of ICP files were uploaded. Currently GeoTIFF generation is ongoing for products of the last year. Once finished, a public announcement will be made. Currently developments are ongoing to improve the cloud detection and to make the data INSPIRE compliant. 2. System Infrastructure Category % Up Time % Down Time Switches Database Servers Mid Term File Servers Short Term File Servers Master Servers (1) Worker Nodes (2) PDF (3) Table 1: System Infrastructure availability for this reporting period (1) Pmaster11_LTDA_NOM_NC has been down for a planned relocation and network maintenance. (2) Worker nodes: The system disc of some worker nodes have been put in RAID1 for redundancy reasons. (3) Data server ORA4 was physically moved. PROBA-V_D5_MOR-019_ _v1 Page 4 of 18

5 3. Image Processing Services 3.1. Ingested and archived products Product Type Total Received Missing data, Archived ingested by VITO METEO TFF (*) 295 Table 2: Ingested and archived products for this reporting period (*) 4 missing TFFs due to pass lost (antenna problems at Inuvik) (*) 1 missing TFF due to pass lost (scheduling mistake Inuvik) 3.2. Generated and archived products Product Type Total Processed Error Archived PROBAV_L1A - Calibration PROBAV_L1A - Nominal (*) 2404 PROBAV_L1C PROBAV_L3_S1_TOA_100M PROBAV_L3_S1_TOC_100M PROBAV_L3_S1_TOC_NDVI_100M PROBAV_L3_S5_TOA_100M PROBAV_L3_S5_TOC_100M PROBAV_L3_S5_TOC_NDVI_100M PROBAV_L3_S1_TOA_300M PROBAV_L3_S1_TOC_300M PROBAV_L3_S10_TOC_300M PROBAV_L3_S10_TOC_NDVI_300M PROBAV_L3_S1_TOA_1KM PROBAV_L3_S1_TOC_1KM PROBAV_L3_S10_TOC_1KM PROBAV_L3_S10_TOC_NDVI_1KM (*) 45 x L1A decompression error Table 3: Generated and archived products for this reporting period PROBA-V_D5_MOR-019_ _v1 Page 5 of 18

6 3.3. Backup and archiving service Product type Total Files Total File Size (GB) TFF L1A Database transaction logs Database incremental back-up Database full back-up Table 4: Back-up data volumes for this reporting period Product type Total Files Total File Size (GB) PROBAV_TRANSFERFRAMES PROBAV_L1A PROBAV_L1C PROBAV_L3_S1_TOA_100M PROBAV_L3_S1_TOC_100M PROBAV_L3_S1_TOC_NDVI_100M PROBAV_L3_S5_TOA_100M PROBAV_L3_S5_TOC_100M PROBAV_L3_S5_TOC_NDVI_100M PROBAV_L3_S1_TOA_300M PROBAV_L3_S1_TOC_300M PROBAV_L3_S10_TOC_300M PROBAV_L3_S10_TOC_NDVI_300M PROBAV_L3_S1_TOA_1KM PROBAV_L3_S1_TOC_1KM PROBAV_L3_S10_TOC_1KM PROBAV_L3_S10_TOC_NDVI_1KM ICP_GEOMETRIC_CENTRE ICP_ GEOMETRIC _LEFT ICP_ GEOMETRIC _RIGHT ICP_RADIOMETRIC_CENTRE ICP_RADIOMETRIC_LEFT ICP_RADIOMETRIC_RIGHT METEO_ECMWF METEO_METEOSERVICES POLARMOTION Table 5: Archived data volumes for this reporting period PROBA-V_D5_MOR-019_ _v1 Page 6 of 18

7 3.4. Dissemination service Product type Added to catalogue Ordered Delivered PROBAV_L1C PROBAV_L3_S1_TOA_100M PROBAV_L3_S1_TOC_100M PROBAV_L3_S1_TOC_NDVI_100M PROBAV_L3_S5_TOA_100M PROBAV_L3_S5_TOC_100M PROBAV_L3_S5_TOC_NDVI_100M PROBAV_L3_S1_TOA_300M PROBAV_L3_S1_TOC_300M PROBAV_L3_S10_TOC_300M PROBAV_L3_S10_TOC_NDVI_300M PROBAV_L3_S1_TOA_1KM PROBAV_L3_S1_TOC_1KM PROBAV_L3_S10_TOC_1KM PROBAV_L3_S10_TOC_NDVI_1KM Table 6: Ordered and delivered products for this reporting period 3.5. End-user activity 20 new user(s) were registered in this reporting period. The total number of users registered for PROBA-V data and that have ordered data is 477 with 81 different nationalities representing 396 different companies/universities. Product type Africa Asia Europe N-America Oceania S-America PROBAV_L1C PROBAV_L3_S1_TOA_100M PROBAV_L3_S1_TOC_100M PROBAV_L3_S1_TOC_NDVI_100M PROBAV_L3_S5_TOA_100M PROBAV_L3_S5_TOC_100M PROBAV_L3_S5_TOC_NDVI_100M PROBAV_L3_S1_TOA_300M PROBAV_L3_S1_TOC_300M PROBAV_L3_S10_TOC_300M PROBAV_L3_S10_TOC_NDVI_300M PROBAV_L3_S1_TOA_1KM PROBA-V_D5_MOR-019_ _v1 Page 7 of 18

8 PROBAV_L3_S1_TOC_1KM PROBAV_L3_S10_TOC_1KM PROBAV_L3_S10_TOC_NDVI_1KM Table 7: Data download (GB) in total per Origin of the User for the reporting period Product Type Global L1C PROBAV_L3_S1_TOA_100M PROBAV_L3_S1_TOC_100M PROBAV_L3_S1_TOC_NDVI_100M PROBAV_L3_S5_TOA_100M PROBAV_L3_S5_TOC_100M PROBAV_L3_S5_TOC_NDVI_100M PROBAV_L3_S1_TOA_300M PROBAV_L3_S1_TOC_300M PROBAV_L3_S10_TOC_300M PROBAV_L3_S10_TOC_NDVI_300M PROBAV_L3_S1_TOA_1KM PROBAV_L3_S1_TOC_1KM PROBAV_L3_S10_TOC_1KM PROBAV_L3_S10_TOC_NDVI_1KM Table 8: Data download (GB) in total for the reporting period Company # of downloaded products University of Leicester 3009 VITO 800 Geoville 723 Zrc 679 IFSULDEMINAS 608 TUSUR 429 Geosys 411 Geoinformation Systems 378 Centre de Suivi Ecologique 365 Institute of Remote Sensing and Digital Earth 350 Table 9: Top 10 user companies for the reporting period PROBA-V_D5_MOR-019_ _v1 Page 8 of 18

9 Company # of registered users BELGIUM 53 CHINA 36 ITALY 28 FRANCE 21 NETHERLANDS 20 UNITED STATES 20 UNITED KINGDOM 17 BRAZIL 15 ARGENTINA 15 INDIA 15 Table 10: Top 10 countries with most registered users List of issues raised by users: ProbaV - Proba-V: provide any thermal data?? - Information on the quality flag of the Proba-V S10 product for academic purpose. - PROBA-V 100 m and 300 m resolution: evaluation of the possibilities of using either 100 or 300 m PROBA-V NDVI data sets - PROBA-V 100 m data in Belge Lambert 1972 projection - PROBA-V 300 m syntheses, question on NDVI conversion factors in Python script PDF portal - PROBA-V ProductCustomization Tool Error (VRG file that was not found) - username forgotten - PROBA-V data and information request PROBA-V_D5_MOR-019_ _v1 Page 9 of 18

10 4. Image Calibration services 4.1. Radiometric Calibration Calibration request type Total Processed Not received Error CLOUDS DARK CURRENT MOON RAYLEIGH SNOW SUN_GLINT Table 11: Calibration Image requests for this reporting period Calibration image type Total Valid Invalid (*) PROBA_V_L1A_CALIBRATION 2 2 PROBA-V_L1B_CALIBRATION PROBA-V_L1B_INTERSECTION PROBA-V_L1B_OVERLAPREGION Table 12: Processed calibration images for this reporting period (*) Due to insufficient overlap with the calibration region of interest, not enough pixels (e.g. clouds contamination), site not sufficiently uniform (illumination), etc. No major changes are observed in the absolute radiometry of the different strips since previous reporting period. Long-term monthly Libya-4 mean plots for different cameras are given in Figure 1, Figure 2 and Figure 3. The deep convective clouds calibration results show a slight decrease in the NIR interband calibration results voor LEFT and CENTER camera as shown in Figure 4. This trend is however not confirmed by the other calibration methods. Similar as in previous reporting periods sudden jumps in the dark current values of a few SWIR pixels, distributed over the different strips and cameras, have been observed which explains the presence of a few stripes in the SWIR images. An update of the dark current values will therefore be done in the next days. Radiometric ICP file ICP SWIR dark values will be updated in the coming days, the current ICP files are PROBAV_ICP_RADIOMETRIC#LEFT_ _V01 PROBAV_ICP_RADIOMETRIC#CENTER_ _V01 PROBAV_ICP_RADIOMETRIC#RIGHT_ _V01 PROBA-V_D5_MOR-019_ _v1 Page 10 of 18

11 Figure 1. Libya-4 desert calibration results: LEFT monthly averaged results PROBA-V_D5_MOR-019_ _v1 Page 11 of 18

12 Figure 2. Libya-4 desert calibration results: CENTER monthly averaged results PROBA-V_D5_MOR-019_ _v1 Page 12 of 18

13 Figure 3. Libya-4 desert calibration results: RIGHT monthly averaged results PROBA-V_D5_MOR-019_ _v1 Page 13 of 18

14 Figure 4. DCC inter-band calibration results: LEFT, CENTER and RIGHT camera PROBA-V_D5_MOR-019_ _v1 Page 14 of 18

15 4.2. Geometric Calibration Calibration image type Total Processed Error PROBA-V_L1C_INTERSECTION Table 9: Processed calibration images for this reporting period During the previous month, the average ALE was < 66 m (σ < 52 m). In the first two weeks, daily ALE values were generally below 60 m. From 2/7 onwards, the ALE sharply increased to a maximum value of 103 m at 7/7 for the BLUE channel. At 9/7, the geometric ICP files were updated and the daily ALE decreased to ~50 m at 10/7. This minimum was followed again by an increase to a peak value of 83 m at 14/7. We will closely monitor these fluctuations over the next weeks. The average daily compliance was > 99.7%. Daily values decreased from 2/7 7/7, with a minimum of 98.7% for the BLUE channel at 7/7. This minimum was followed by an increase to values well above 99% until 15/7. Geometric ICP file PROBAV_ICP_GEOMETRIC#LEFT_ _V01 PROBAV_ICP_GEOMETRIC#CENTER_ _V01 PROBAV_ICP_GEOMETRIC#RIGHT_ _V01 Figure 5 ALE this reporting period PROBA-V_D5_MOR-019_ _v1 Page 15 of 18

16 5. Anomalies 5.1. System related issues Figure 6 - Long term ALE: 15 Jan July 2015 A detailed description of each issue is available in the issue tracking system Key Summary Status Created Component/s PROBAVUS-7 Very small images fail to process Resolved 10/01/2014 General PROBAVUS-31 new ICP file generation Resolved 17/04/2014 Geometry PROBAVUS-56 S5 rejection -> validation status rejected, but processing status remains 'Processed' Resolved 06/07/2015 Software 1 new issue was logged during this reporting period 2 issue(s) was resolved and closed during this reporting period 0 issues are resolved but remain to be closed formally 2 issues are resolved but remain in the list logging purposes 0 issue(s) is open and remain to be solved PROBA-V_D5_MOR-019_ _v1 Page 16 of 18

17 5.2. Image processing issues Currently GeoTIFF generation for products until July 2014 is still running. Once all products are generated, a public announcement will be done. A detailed description of each issue is available in the Weekly Report and the image processing tracking system The below table gives an overview of the S1 s of this reporting period: # S1 Dates Major Gaps (> km² (missing TFF)) 2 26/06, 28/06 Large Gaps (< km²) 16 04/07, 14/07, 29/06, 24/06, 13/07, 23/06, 27/06, 07/07, 02/07, 16/06, 21/06, 06/07, 15/07, 25/06, 30/06, 10/07 Medium Gaps (< km²) 9 09/07, 19/06, 11/07, 01/07, 03/07, 08/07, 18/06, 20/06, 12/07 Minor Gaps (< 3600 km²) 3 05/07, 22/06, 17/06 Negligible Gaps (< 1000 km²) 0 Complete synthesis (no gaps) 0 Table 10: Overview of S1 for this reporting period PROBA-V_D5_MOR-019_ _v1 Page 17 of 18

18 6. Scheduled activities for the next period(s) Software upgrades: o A new release of the DIF component is currently under test. Target is to minimize the amount of data that is discarded if a decompression error occurs. Release is expected in week 29. Hardware: o Some servers in the PROBA-V processing chain are end-of-life. An assessment will be done to determine which servers can and will be upgraded. Development: o An improvement of the cloud detection algorithm is under investigation. o Developments to make the metadata INSPIRE compliant are ongoing. o Developments are ongoing to update the PF component to integrate the geometric error checker tool. This is now a separate tool that needs to run daily. No other activities scheduled. 7. Operational remarks On 30/06 a leap second was introduced in the processing facility. PROBA-V_D5_MOR-019_ _v1 Page 18 of 18