The FAO crop forecasting Philosophy

Size: px

Start display at page:

Download "The FAO crop forecasting Philosophy"

Bryce Smith
6 years ago
Views:

FAO Training on crop yield forecast strengthening for Armenian Experts (November 2011- Arlon

1 FAO Training on crop yield forecast strengthening for Armenian Experts (November Arlon Belgium) The FAO crop forecasting Philosophy Bernard TYCHON (with R. Gommes large inspiration)

2 Variability of cereal yields 7 Kyrgyzstan 6 5 Egypt Romania SAUDI USA Yield (tons/ha)

3 Factors of yield variability Yield (arbitrary units) Factors F1 to F5 (arbitray units) Yield F1 (innovation) F2 (policy) F3 (trend) F4 (extreme factor) F5 (weather)

4 Tricky trends 45 F3 Technology trend F3, F5, F6 and price of nitrogen fertiliser F5 weather Price of N fertiliser F6 (nitrogen) Yield Year

5 Rice yield and Tong-Il in S. Korea

6 Relation between Evapotranspiration and assimilation [F(mesophyll resistance)]

7 Agricultural statistics District Yield District ETA Station water balance NDVI or other grid ETA grid

9 General Methodology INPUTS Explanatory Variables INDEPENDENT VARIABLES Initial Water Holding Content OUTPUT ETP Soil Water Satisfaction Index AMS Water excess, deficit Actual rainfall NOAA GAC Temperature, RR, RH, V A S T W I N D I S P METEO Actual ETA Starting date NDVI max Time peak NDVI Cumulated actual rainfall S T A T I S T I C A M A T L A B Yield prediction model at departmental level Yield agregation at national level Yield prediction model at national level CROP YIELD DATA Historical crop yield data at departmental level

10 Data types used in crop forecasting TECHNICAL FIELD (DATA CATEGORIES) MINISTRY (SOURCE) DATA TYPE (VARIABLES) GEOGRAPHIC UNIT SAMPLE SIZE (AREA) (TIME) SAMPLING FREQUENCY AGRO- METEOROLOGY SOILS & TERRAIN REMOTE SENSING MARKETING AGRICULTURAL STATISTICS Transport, Environment, Nat. resources, Defense, Agriculture... Agriculture, Nat. resources Transport, Defense, Research Commerce, Agriculture Agriculture, Statistics, Planning Weather data Phenology Crop condition Phenology Soil WHC DTM NDVI CCD (Rainfall) DTM Station 1+ Region 1000 Point/Pixel 1-10 Station 1 Polygon Pixel 7 x 7 km 1 x 1 km 1 Day 10 Week 10 Week 10 Week 50 Month 10 Week years Hour, then aggregated to week Price data Market Bi-monthly Grain stocks Warehouse 5+ 1 Day Grain Village Year Cropped area Yield Production Admin. unit, Agric. dev. district, or polygon to 12-monthly DEMOGRAPHY Planning, Central statistics Population NUTRITION Health Children's weight, other indicators Administrative Unit, or "polygon 2" years Dispensary Weekly to monthly

11 Yield and future weather

12 Technical options Use ETA as main forecasting variable Integrate RS and ground data at data and product level Spatialise after modelling Calibrate against agricultural statistics Scale independence

13 Philosophical options Central development of tools (technology, data and demand driven) Distribution at no cost for users Flexible/modular data processing chain Main advantages reduced costs (of development, maintenance, training, translation, operation, e.g. file formatting ) comparable results in a regional context.

14 Parametric methods

15 coef.de determination variance total explained variance 2 = = R = = = = n t n t t y t n y S y 1 n 1 variance = = = n t y t y t 1 2 ^ n 1 MS mean square error 2 y 2 y 2 S MS S R = error z t b b t a y = Models comparison (Palm R., 1996) Model choice : 156 combinations crops-countries = 28 years Linear trend for many crops but for some crops, a quadratic trend improve clearly the R 2 Cf. Table 1 : distributions of determination coefficients, R²

16 Table 1. Determination coefficient distribution R² (% of the number of fittings) R² Annual crops Permanent crops R² = < R² < < R² < < R² < < R² < Total

17 Table 2. Fitting distribution according to the shape of the retained relation (in % of the number of fittings) Shape of the relation Annual crops Permanent crops No trend 7 36 Growing trend Linear Non linear Concavity upward Concavity downward 5 3 Decreasing trend 1 20 Linear 0 12 Non linear Concavity upward 0 0 Concavity downward 1 8 Total

18 Hypothesis : yield depends not only on time but also on weather conditions 87 European stations (Tmean, Tmax, Tmin, RR) Model y = f 1 (t) + f 2 (m) + e f 1 (t) = general trend component f 2 (m) = weather component removal of the general trend and study on the residues (z) z = y f 1 (t) = b 1 x b p x p + e error x i = meteo variables i (i=1 à 4) dekadal data, 10 stations, 27 dekades 1080 variables Selection of x i variables by simple correlation analysis and then, by stepwise procedure.

19 Model Validity Given y^ t, the forecasting for year t for a given crop and a given country, this forecasting being calculated from information of the t-1 former years yt e = relative _ error = t t ^ y y t.100 e q n 2 _ error = et on 6 years (83 88) t= n 6 6 = quadratic 1

20 Table 3. Means of e q for the complete set of crops Country Trend Trend + meteo variables Belgium Denmark Germany France Ireland Italy GD of Luxemburg The Netherlands United Kingdom The introduction of meteo data in the model increases the error!!!

21 Use of a set of meteorological variables ETP, RR, Rad, T max, T min Application to maize yield forecast in France Selected variables by agronomist (Models type a) ex : ETP cumulated from July to August Cumulated radiation in June Sum of mean tempertures > 6 C in June etc. (in total, 13 new variables). Raw variables selected statistically by fitting like before (Models type b) Maize crop on 24 decades

22 Comparaison criteria ( ) = = = = = n t t n t t Z Z Z e n MS Z n S avec S MS S R Jackkniffe explain the method Z j Z j S MS S R = to avoid that estimation for one year is linked to observations of that particular year Model based on the last 8 years. One forecasts yield by using a fitted model on the last 8 years. These last approach is the one that is the closest to a true validation. This method will have the most value Z s Z S S MS S R =

23 Table 4. Models type a and b comparison : mean values for 19 departments Criteria R² R² j R² s Models type a Models type b Cf. Table 4 no prediction capacities of these models!!!

24 2. DHC-CP (AGRHYMET, 1998) ETR j (mm) Hr : demande évaporative ETM j (mm) Hr : 0.9 Hr : 0.8 Hr : 0.7 situation favorable Hr : 0.5 situation de stress hydrique

25 DHC-CP ETA d = 0,732-0,05xETM d + (4,97xETM d - 0,661xETM d ² )xhr d - (8,57xETM d - 1,56xETM d ²)xHR j ² + (4,35xETM d -0,88xETM d ² )HR d 3 (EAGLEMAN Algorithm, 1971) ETM d = maximum evapotranspiration of crop = Kc x ETP d Kc = crop coefficient ETP d = ETP Penman of day j calculated from dekadal ETP issued of the Sahelian ETP Atlas (MOREL) (average ). HR d = humidity rate in the considered soil layer for day d, ie, ratio between the water content in the root zone (S d ) and the maximum water content in this same root zone(rur d ).

26 The first day of the pentade, water stock variation in the studied soil layer (DS) is calculated by : (S d -S d-1 ) = DS = R p - Ruis p - Dr p - ETA d S d-1 = available stock of water the last day of the preceding decade S d = available stock of water the first day of the new pentade R p = pentade rainfall Ruis p = surface runoff (not taken into account) Dr p = drainage below root zone calculated once per pentade

27 Next days, water balance is equal to : S d = S d-1 - ETA j

28 kinetic of water into soil and root front RUR = water holding capacity RUM = maximum water holding capacity

29 Crop parameters : phenological stages, K c Coefficient cultural K cmin Interpolation K cmax Interpolation K cfin Cycle de la plante φ végétation φ floraison φ maturation Intervalles des coefficients culturaux Stades phénologiques de la plante

30 Lengths of phenological stages for each crop included into DHC-CP Tableau 1: durée des stades phénologiques pour chaque culture proposées dans DHC-CP. Culture RIZ MIL SORGHO MAïS ARACHIDE NIEBE Stades phéno. Phase 1 : IDV 15 jours 15 jours 15 jours 15 jours 15 jours 15 jours Phase 2 : croissance Durée variable selon la longueur du cycle de la plante: entre 20 et 60 jours Phase 3 FL1 Durée variable selon le cycle : entre 15 et 25 jours FL2 Durée variable selon le cycle : entre 15 et 25 jours Phase 4: MAT 20 jours 20 jours 20 jours 25 jours 30 jours 15 jours

31 DHC-CP ETA ETA IRESP % = ( cycle) ( sensitive _ phase) ETM ETM Yield (Kg/ha) = 11,3 x IRESP r² = 0,66

32 User Board meeting 2 of the Global Monitoring for Food Security Project FAO Offices, Rome February 2006 Senegal : Millet Yield forecast in 2005 with agrometeorological models Bernard TYCHON and Damien ROSILLON

33 Objectives Objective : Predict the agricultural yields in Senegal at national level for the agricultural season 2005 Methodology : Estimate yield at third national level (department) in the bassin archidier and combine the results at national level Statistical study to find a correlation between yield and agrometeorological explanatory variables (AMS outputs, VAST outputs, rainfall)

34 Data Actual rainfall ( ) One station per department Normal ETP Known for 6 stations Interpolation by Thiessen polygons method Historical agricultural statistics ( ) Yield, production, surface NOAA degraded GAC ( )

35 Vast outputs

36 INPUTS Methodology OUTPUTS - INPUTS OUTPUTS ETP Initial Water Holding Content AMS Sol Water satisfaction Index Water excess, deficit Actual rainfall Actual ETA... Sowing date Statistica Yield prediction model NOAA GAC VAST NDVI max Time peak Temp, RR, RH, METEO Actual rainfall (simple and combination)

37 Statistical study Software : Statistica 7.1 Stepwise regression and cross validation (to calculate the generalization error of the model) Example :Thiès Model of millet yield prediction Y = * X * X2 With : Y : yield (kg/ha) X1 : sum of rainfall occurring during the 7 th -10th dekads following the sowing date X2 : Water excess at vegetative stage R² = 0.77 R² (cross validation) = 0.69

38 Millet yield forecast 2005 at departmental scale

39 Millet yield forecast at national scale Predicted yield : 768 kg/ha Variation between predicted production and DAPS data : 8% Millet production (t) Predicted millet production vs official forecast (DAPS) 0 Prediction DAPS

40 Estimated yield at national scale vs Official data Estimated yield vs FAO yield FAO yield Estimated yield Millet yield (kg/ha) Year Correlation coefficient between estimates and FAO yield R² = 0.58 Similar trend model able to simulate extreme cases RMSE = 40 kg /ha

41 Limits of the methodology Reliability of yield statistics Reliability, number of input data (meteo, RS, crop, ) Unforeseeable weather conditions (abnormal extension of the season in 2006) The statistical approach itself

42 Limits of the method : quality of the inputs Time series not complete especially for rainfall data Comparison between millet yield from field survey and agricultural statistics in 2004 Reliability of agricultural statistics (millet yield 2004, evolution of agricultural surface in some departments) millet yield (kg/ha) Agricultural statistics Field survey Bambey Fatick Foundiougne Kaffrine Kaolack Kébémer Mbacké Nioro Tamba Tivaouane Department

43 Non-Parametric methods

Yield Forecast Zimbabwe 1960/61-2001-2002 Gommes R., 2002 Map of Southern Africa.

44 Yield Forecast Zimbabwe 1960/ Gommes R., 2002 Map of Southern Africa. The hatched area corresponds to the main maize growing areas. The background map shows vegetation densities as estimated from satellite indices (light, medium and heavy vegetation)

45 Weather Conditions Zimbabwe 1960/ Rainfall and ETP (evapotranspiration potential) patterns in Zimbabwe between and : average monthly values, maximum and minimum recorded for each month, as well as rainfall profiles of driest and wettest years. Monthly rainfall (mm) Jul Sep Nov Jan Mar May

46 Historical yields Zimbabwe 1960/ Maize yields in Zimbabwe, together with their trend and the and detrended value (departure of actual values from the trend) Tons/Ha Yield -1 Trend Detrended

47 Rainfall-Yield Relation Zimbabwe 1960/ Relation between detrended maize yield (expressed as standard deviations from average) and average July-June rainfall in maize growing areas of Zimbabwe between 1982 and 2002 Yield (Std from average) y = x R 2 = Average m onthly rainfall (m m)

48 ETa Yield Relation Zimbabwe 1960/ Relation between detrended yield and actual maize evapotranspiration in maize growing areas of Zimbabwe between 1982 and Detrended Yield (T/Ha) R 2 = Actual evapotranspiration (mm)

49 Threshold method Zimbabwe 1960/ Correlations between maize yield and monthly rainfall for the 11 years between and in which January rainfall was between 75 and 155 July Aug. Sep. Oct. Nov. Dec. Jan. Feb. Mar. Apr. May June July 1.00 Aug Sep Oct Nov Dec Jan Feb Mar Apr May June Y obs

50 Threshold method Zimbabwe 1960/ Criteria 1 January rainfall (mm) 75 to to to 327 Yield Criteria 2 Threshold and yield < 120 mm >120 mm February to rainfall to to < 170 mm > 170 mm February to rainfall to to < 190 mm > 190 mm December 0.35 to rainfall 0.23 to to 1.25

51 Threshold method Zimbabwe 1960/ Comparison of estimated and observed yields in Zimbabwe between 1961 and 2001 using the threshold method. Yields are expressed in standard deviations from the average. 1 Estimated Yield R 2 = Observed Yield

52 Rainfall profiles Zimbabwe 1960/ Rainfall amount mm Class 1 Class 4 Class 10 Yield = StD Yield = 1.19 StD Yield = 0.21 StD 50 0 July Aug. Sep. Oct. Nov. Dec. Jan. Feb. Mar. Apr. May June

53 Classes with yield intervals Zimbabwe 1960/ /2002

54 Rainfall profile Zimbabwe 1960/ Comparison of estimated and observed yields in Zimbabwe between 1961 and 2001 u sing the rainfall profile method. Yields are expressed in standard deviations from the average. Observed yield R 2 = Estimated yield

55 Advantages of nonparametric forecasting Calibration using combination of mix of time-series and crosssectional data independent of type of functional relation between variables and yield (non-parametric) confidence intervals are easy to derive (may) require little data processing

56 Comparison of methods Method Average Rainfall Water Balance R 2 Trend Method Total Threshold Clustering

57 Summary: main functions required Models for extreme factors Improved simple models Phenological data Grid processing and spatial statistics Future data Support for nonrainfall limited conditions Regression techniques Clustering techniques

58 Thank you! 1634 etching by Rembrandt

Introduction to yield forecasting with CST

Introduction to yield forecasting with CST H. Kerdiles (JRC), H. Boogaard & S. Hoek (Alterra) Harare, Zimbabwe 26 Oct 2016 Yield variability: can you rank the maize fields according to their yield? Same