Executive Test Suite explained: Belgium - Flanders experience

Transcription

1 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 1/ 20 Executive Test Suite explained: Belgium - Flanders experience Pavel Milenov, Piotr Wojda, Sara De Santis EC Joint Research Centre Institute for Protection and Security of the Citizen Monitoring of Agriculture Resources Pieter Roggemans

2 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 2/ 20 Outline Overview of ETS Objective and basic concept Key quality elements and measures Workflow: Sampling, Inspection procedure, reporting Performance issues and future challenges Belgium - Flanders experience with ETS Feasibility, technical constraints Time and resources needed Observations and suggestions Status of the ETS pilots

3 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 3/ 20 Objective of ETS To measure the performance of the LPIS on the ability to fulfil two explicit LPIS functions: unambiguous localisation of all claimed agricultural parcels quantification of all eligible area for crosschecks Through evaluation of the of of a number of pre-defined and agreed LPIS quality elements Which values are derived though a certain set of data quality measures, specific for each LPIS quality element

4 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 4/ 20 Quality Elements and Data Quality Measures doc Quality topic Description Observation and Reporting Measures Expectation 1.1 Total eligible area maximum quantity of land that can be claimed by farmers (SAPS) or eligible hectares upon which entitlements can be activated (SPS) Reference Parcel RP true eligible area informative Reference Parcel RP diversity Informative Reference Parcel RP landscape elements informative Reference Parcel 10104_2 RP landscape elements area informative LPIS Sample LPIS eligible area 1.2 Proportion of RP allowing payment on ineligible land indication of the statistical size of the quality problem. Reference Parcel RP Ineligible land cover features informative Reference Parcel 10102_2 - RP Area purity LPIS Sample LPIS nonconformant RP Distribution of RP allowing payment on ineligible land indication of the statistical spread and nature of the quality problem LPIS Sample LPIS eligibility rates

5 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 5/ 20 Quality Elements and Data Quality Measures (2) 1.3 Categorization of the RP allowing payment on ineligible land occurrence of parcel categories based on the cause of the contained problem LPIS Sample LPIS cause of anomaly Categorization of the RP allowing payment on ineligible land abundance of parcel categories based on the cause of the contained problem LPIS Sample 10204_2 - LPIS number of anomaly causes 1.4 Critical defects Occurrence of non-compliances that obstruct the use of the parcel Reference Parcel RP critical defects informative LPIS Sample LPIS critical defects 1.5 Claimed area inside the reference parcel Proportion of claimed farmed land to the reference area N/A in ETS: claim register All unclaimed area identified 1.6 Effectiveness of the update processes the occurrence and abundance of transactions originating from farmer/ N/A in ETS claim register the occurrence and abundance of transactions originating from OTSCinspector To be developed the occurrence and abundance of transactions originating from LPIScustodian N/A in ETS claim register 1.7 Relation with error rates identified during OTSC field inspection Irregularities identified during the OTSC can reflect shortcomings of the LPIS To be developed N/A Boundary accuracy Rate of boundary segments with inaccurately positioned nodes Reference Parcel RP positional accuracy

6 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 6/ 20 Data Flow and Processes in the ETS Inspect certain number of randomly selected reference parcels on the most recent orthoimagery

7 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 7/ 20 Basic properties of ETS Quality measures and procedures based on ISO Most recent orthoimagery as a reference Satellite Imagery collected under the CwRS program compliant with the JRC VHR specifications for the CwRS Inspection based on manual photointerpretation and delineation Flexible mapping rules Application Schema or Feature Catalogue of the implementation under test Identification and quantification of features, based on the difference in land cover No link with ATS yet No follow-up visits on the field planned Model Conformance Test Conformance Statement (ICS) Abstract Test Suite (ATS) Additional Information for Testing Data Conformance Test Executable Test Suite (ETS) Analysis of results Conformance Test Report

8 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 8/ 20 Structure and Content of the ETS ISO compliant data quality measures, at parcel and sample Annex I ISO/TS and ISO Inspection procedure at the Reference Parcel Annex II UML Activity Diagram Flow of events and non-functional requirements Application of the land cover concept to describe eligibility of land Annex III The concept of eligibility profile introduced ISO based Sampling Scheme Annex IV ISO SQL statements for database analysis and reporting

9 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 9/ 20 ETS Sampling scheme A population of reference parcels defined for each country/region Isolated lot of reference parcels A minimum number of reference parcels to be tested Sample size based on ISO 2859/ (Procedure A, Limiting Quality = 2%) A larger number of reference parcels Sample pre-selection based on clipped reference parcels

10 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 10 / 20 Practical sampling procedure Reference parcels are represented by points located inside each parcel Reference parcels are clipped with CwRS zones only zones, where VHR data is successfully acquired Lot Clipped Pre-selection Sample size

11 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 11 / 20 Data Quality Measures - RP true eligible area

12 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 12 / 20 ETS Inspection Procedure Select the reference parcel from the selection list Locate the reference parcel Decide if it can be inspected or not. If yes, proceed with the inspection. If not, select the next one from the ordered list Identify and delineate the eligible land cover features on the land represented by the reference parcel. Derive the area. Identify the eligible landscape features on the land represented by the reference parcel. Derive the area. Identify the ineligible land cover features on the land represented by the reference parcel. Identify if the reference parcel has a critical defect, which obstruct its use in the LPIS

13 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 13 / 20 Factors affecting the ETS performance Type of reference parcels and structure of the LPIS model ETS more complicated for some types of LPIS ATS could provide important information for ETS Quality of the input orthoimagery Poor image quality will jeopardize the whole ETS Additional data or processing might be needed Operator experience and skills Good knowledge and experience in land cover mapping Special training might be necessary (CAP context, largescale mapping)

14 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 14 / 20 Future Challenges Mapping instruction for landscape features Decision rules for the land cover features in the range of 0.01 and 0.1 ha? To map them or not? Link between ATS and ETS: ETS could use information from ATS Refine Land cover classes, according to FAO LCCS Elaboration of similar ISO compliant quality measures to assess the orthoimagery Evaluation of the suitability of the VHR data from CwRS to be used for the ETS AOB: MMU, visual scale, vectorization,

15 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 15 / 20 Continued by Pieter Roggemans

16 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 16 / 20 Status of the ETS pilots LPIS sample Work done by Date of the preselection sample sent from JRC Minimum number of Reference parcels to check Reference parcels inspected LPIS 1 MS LPIS 2 JRC LPIS 3 MS LPIS 4 MS

17 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 17 / 20 ETS-scoreboard (partial and provisional) % LPIS1 LPIS2 LPIS3 LPIS4 1. LPIS eligible area rate Proportion impure RP >3% Distribution impure RP 0-2% % % % % % >50% Proportion defective RP

18 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 18 / 20 First feedback from the ETS pilots ETS might require more efforts than expected, boundary accuracy and landscape elements considered complex measures Parcels found to have critical defects, are not necessary erroneous Additional analysis of these cases in the national context is necessary The land cover mapping in that large scale is not a straightforward process Specific mapping rules needed to be defined, especially for landscape features. National CAP rules should be considered. VHR Orthoimagery might not be sufficient source for ETS certain land cover features are problematic to map time factor needs to be considered when archive data is used

19 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 19 / 20 Conclusions on ETS Provides lot of objective information on the status of LPIS Quality indicators defined seems to be meaningful and allows comprehensive analysis Feasible to be done in relatively short time frame Applicable in any type of LPIS Compliant with International Standards (ISO, LCCS, UML)

20 LPIS Workshop: Applications and Quality, 6-8th October, 2009, Tallinn 20 / 20 Thank you for your attention!