Changes to LPIS QA: Analysis of the sampling representativeness and impact on 2017

Transcription

1 The European Commission s science and knowledge service Joint Research Centre Changes to LPIS QA: Analysis of the sampling representativeness and impact on 2017 W. DEVOS and D. FASBENDER

2 PART1: Sampling --- Trigger ECA Special report (25/2016) to assess the reliability, effectiveness and impact of LPIS/GIS across the EU Recommendation 5: The Commission should, before the start of the QA exercise 2017, carry out a cost-benefit analysis to determine whether the representativeness of QA samples could be improved so that a better coverage of the population of parcels in the LPIS can be achieved. EC (DG JRC) acknowledged the recommendation since The monitoring of sample representativeness is part of a continuous process. 2017: Experience of 2 years dedicated LPIS QA image provision Image allocations unchanged in 2016 update with latest population data 2

3 A. A new statistic 3

4 What is representativeness? Short definition: A representative sample is a sample that shows similar characteristics compared to the population from which it has been issued. Population Representative of what? In LPIS QA context: representative of the system s quality (i.e. presence of non-conformities, MEA correctly recorded ) How to proceed with analysis? For this analysis, reference area is the only observable parameter (i.e. proxy parameter)! Note: larger RP may behave different than smaller RP Use simulations for verifying that the whole sampling procedure tends to produce samples in which no particular portion of the population is over-/underrepresented. Representative sample Unrepresentative sample 4

5 Simulation (Improving representativeness of LPIS samples) Population statistics: Min, max, mean, var, range, intervals of 95%, 100 x sampled each time random zones Each sample statistics: Min, max, mean, var, range, intervals of 95%, With 1, 2, 3, zone(s) until acceptable match 5

6 Which characteristics (stats) to compare? Z-test Chi-square test Kolmogorov- Smirnov test PCPI (% of pop. in central prob. interval) Comparison of sample and pop. means Comparison of sample and pop. variances Full comparison of distribution s shapes Comparison of «95% imprints» Detection of bias Fails to assess the variability 6 Fails to assess the bias Detection of abnormal variance Tests all aspects of the sample So idealistic that it becomes too restrictive Detection of shifts Detection of abnormal variance while keeping the objective realistic!

7 Acceptance rate of the tests [%] Tests results on the past LPIS samples. 95%=Producer error Proposed tests were applied to the actual samples of previous years ( ) No particular trend in the acceptance rates (all systems together) Two distinct periods and PCPI ideal Impact of at least 3 images could be identified (green and red lines) Old rule-of-thumb reintroduced from km 2 7

8 Simulation to pass KS: test too restrictive? Hypothesis: What if we increased the current ( ) image allocation: threefold fivefold The KS test still fails 43 systems! Number of images PS: nonsense to calculate the largest systems. 8

9 PCPI simulation: steps 2: sample 1: zones 3: interval 9 The working definition of representativeness: «The interval with 95% of the sample should contain 95% of the population». Statistical tolerance / margin :±2% 4: PCPI

10 B. A new density mask Threshold of 2 RPs/km² for low density Using 2016 population 2 km resolution regular grid (instead of 10 km) Improved coastline delineation

11 C. Low RP density map modifications Low RP density = less than 2 RPs/km² Simulated campaigns to establish the maximum percentage of low RP density mask in the control zones MSs with maximum percentage < 20% are regarded as homogeneous (not considered for the PanEU zone) Values are the number of images per LPIS system 11 N.B. Size of MS also taken into account (FR in PanEU, BE-WA and DE- ST not) Total of 15 LPIS systems in the PanEU zone

12 D. A new image allocation map 1 image 2 images 3 images PanEU (2 images) LPIS name BE-WA CY DE-BB DE-MV DE-NW DE-SN DE-ST DE-TH DK EE HR LU LV PL PT SK UK-SC TOTAL

13 E. A new RP sampling procedure Simple random sampling (2016) Stratified random sampling (2017) Example: Total of 10,000 RPs covered spread over 4 zones Need to select 3750 RPs (i.e. 3 times 1250) The 10,000 RPs are pooled together The 3750 RPs selected proportionally in each zone and then the 3750 RPs are selected at once and then pooled together 13

14 Why stratified random sampling? Sampling design better in line with the spatial component of the control zones: Each individual sample is generated per zone before to be pooled together No missed control zone : Each zone is covered with at least one inspection (Particularly noticeable if the density of RPs is drastically different between the zones) Independent samples between the zones: Each sample will generally be representative of its own zone (not always true for zones with very low density) so it allows to analyze the difference between control zones 14

15 F. adapted inspection loop (?) Proposal 1 «scientific» DG JRC generates one sample pre-selection list per control zone MSs make sure that the required number of inspections per control zone is reached Fully in line with the sampling procedure Instructions are clear (work organized per zone) There is up to 6 lists for some MS (update of software?) Proposal 2 «pragmatic» DG JRC generates one unique sample pre-selection list MSs make sure that the required number of inspections per control zone is reached Required number of inspections is garanteed Instructions are more complicated (manual jumping until required number of inspections) There is one list (No need to update the software) Proposal 3 «lowest burden» DG JRC generates one unique sample pre-selection list with a particular structure that maximizes the chance to reach the required number of inspections per control zone Required number of inspections is not garanteed Instructions are clear (=Business As Usual) There is one list (No need to update the software) 15 In all the cases, no increase of inspections!

16 Proposal 1: Scientific Example: 3 zones RPs covered (10% in control zone 1, 30% in control zone 2, 60% in control zone 3) 1250 inspections (pre-select 3750 RPs) 125 for control zone 1, 375 for control zone 2, 750 for control zone 3 Control zone 1: 125 inspections RP1.1 RP1.2 Once 125 inspections reached, start with the 2 nd list RP1.375 Control zone 2: 375 inspections RP2.1 RP2.2 Once 375 inspections reached, start with the 3 rd list RP Control zone 3: 750 inspections RP3.1 RP3.2 Once 750 inspections reached, full stop RP

17 Proposal 2: Pragmatic Example: 3 zones RPs covered (10% in control zone 1, 30% in control zone 2, 60% in control zone 3) 1250 inspections (pre-select 3750 RPs) 125 for control zone 1, 375 for control zone 2, 750 for control zone 3 RP1.1 RP1.2 Once 125 inspections reached, jump manually to the 2 nd block RP1.375 RP2.376 RP2.377 RP RP RP Once 375 extra (sub-total of 500) inspections reached, jump manually to the 3 rd block Once 750 extra (sub-total of 1250) inspections reached, full stop 17 RP3.3750

18 Proposal 3: Lowest burden RP1.1 RP1.125 RP2.126 RP2.500 Example: 3 zones RPs covered (10% in control zone 1, 30% in control zone 2, 60% in control zone 3) 1250 inspections (pre-select 3750 RPs) - known skipping rate = 10% 125 for control zone 1, 375 for control zone 2, 750 for control zone 3 RP3.501 RP RP Required number without skipping RP RP RP Once 1250 inspections reached, full stop RP RP RP Expected end of loop end from skipping in the past RP RP RP

19 2017: Improved representativeness The objective of 2% margin for the PCPI is now generally met (except for CY and LU) Classes of PCPI Good (less than 1% difference) Intermediate (between 1-2% difference) Poor (more than 2% difference) 7 2 Score system (with classes of PCPI) shows good costbenefit improvements Score (Good=2, Intermediate=1, Poor=0) (+16%) Total images (+17%)

20 Conclusions (on part 1) Better establishment of the location of the agricultural areas (i.e. improved spatial resolution of density masks, improved coastline delineation ) Representativeness of the samples assessed and remedial actions were taken where necessary Biggest changes are: - addition of 21 images spread over 17 LPIS systems - change of RP sampling procedure (internal for DG JRC) - repartition of the inspections imposed across the control zones (ETS inspections by MS) through adapted inspection loop (?) Required (?) number of inspections is provided per control zone but total number of inspections (500/800/1250) is unchanged! 20

21 PART 2: Clarifications on some previous communications 1. Parcels in scope ( active parcel ) 2. Counting multiple non-conformities in a single nonconforming item 3. Mapping / reporting small artificial surfaces THESE ARE NO CHANGES!!! Merely clarification / typo corrections to avoid erroneous inspection recycled slides 21

22 Belief The LPIS QA scope contains all RPs on agricultural land Fact Only the active RPs are in the scope reference parcel reference parcel reference parcel Agricultural parcel 1 declared n years Agricultural parcel 1 declared n years Agricultural parcel 1 Agricultural parcel 2 Agricultural parcel 2 Agricultural parcel 2 Agricultural parcel 2 Agricultural parcel 2 declared Agricultural parcel 2 declared Active Active Not active 22 n 2

23 RP2 remains completely active RP 1 RP 2 Declared IN SCOPEparcels 1. 5,00 ha 2. 2,00 ha P1 3. 2,00 ha 4. 4,00 ha 5. 2,00 ha P2 P3 P4 P5 Claim year 2014 RP1-1. 5,00 ha RP2-2. 2,00 ha RP2-3. 2,00 ha RP2-4. 4,00 ha RP2-5. 2,00 ha Claim year 2015 RP1-1. 5,00 ha RP2-2. 2,00 ha RP RP2-4. 4,00 ha RP2-5. 2,00 ha Claim year 2016 RP1-1. 5,00 ha RP2-2. 2,00 ha RP RP2-4. 4,00 ha RP ALERT 23

24 Non-conformities (cf. EU 640/2014 art 6.2) Conformance class 1 assess the quality of LPIS, counts nonconforming items (RP or crop aggregate) QE1, QE2, QE3 factual assessment Straightforward link with RP upkeep processes expectation = 5% (QE2) or 1% (QE3) non-conforming items NO CHANGE FROM ETSv Conformance class 2 identify possible weaknesses, requires a broader system wide analysis, QE4 analysis on the LPIS processes and design Example: a single, large parcel is contaminated, includes ineligible land and its land is wrongly classified this represents 1 NC RP but 3 different weaknesses! QE4: expectation = <5% non-conformities per 100 items where item RP/aggregate 24

25 In practice 1 Illustration of observation change (fictitious) ETS v6.0: If area-conforming then locate contaminating road and building [x,y] 25 25

26 26 Illustration of processing change test observation ETS v Area conformance m 2 >< Fail Fail m 2 ETS v Contamination 1 road, 1 shed n/a n/a * Area correctness PG: m 2 >< 8925 m Fail QE2 Any fail above Fail Fail QE4 (ex-qe3) Count QE2/QE3 fails 1 non conforming item 2* non conformities *: any unrelated road and shed would be individually counted i.e. 2 counts 26

27 27 Mapping and reporting of small nonagriculture features Reporting of the artificial sealed features in ETS is regardless their size for accounting: any potential triggers for contamination following the LPIS guidance of AGRI, stating that manmade constructions. should be excluded from the RP by delineation However, ETS does not require a delineation of all nonagriculture features only those larger than or equal to 0.03 ha or 0.01 ha depending on the orthoimage and nature of land feature other smaller features are reported as points only 27