Will Canadian Precipitation Analysis System improve precipitation estimates in Alberta?

Transcription

1 1 Will Canadian Precipitation Analysis System improve precipitation estimates in Alberta? Xinli Cai 1, Piyush Jain 1, Xianli Wang 2, Mike Flannigan 1 1 University of Alberta, Western Partnership of Wildland Fire Science 2 Natural Resources Canada, Canadian Forest Service

2 2 Canadian Fire Weather Indexes (FWI) System 1 Precipitation >0.5mm Precipitation >1.5mm Precipitation >2.8mm 1 Development and structure of the Canadian Forest Fire Weather Index System Van Wagner, C.E. Canadian Forestry Service, Headquarters, Ottawa. Forestry Technical Report p.

3 3 Challenges The highly variable nature of spatial distribution of precipitation (PCP) is the biggest challenge for the spatial interpolation of FWI indexes 2,3. 2 Flannigan, M. D., and Wotton, B. M., (1989) A study of interpolation methods for forest fire danger rating in Canada. Can. J. For.Res. 19: Flannigan, M. D., Wotton, B. M., and Ziga, S., (1998) A study on the interpolation of fire danger using radar precipitation estimates. International Journal of Wildland Fire. 8:

4 4 Challenges The highly variable nature of spatial distribution of precipitation (PCP) is the biggest challenge for the spatial interpolation of FWI indexes 2,3. Radar observed PCP on July 18, 2015 PRECIPT Rain , 12:50 UTC, 1/13 2 Flannigan, M. D., and Wotton, B. M., (1989) A study of interpolation methods for forest fire danger rating in Canada. Can. J. For.Res. 19: Flannigan, M. D., Wotton, B. M., and Ziga, S., (1998) A study on the interpolation of fire danger using radar precipitation estimates. International Journal of Wildland Fire. 8:

5 5 Challenges The highly variable nature of spatial distribution of precipitation (PCP) is the biggest challenge for the spatial interpolation of FWI indexes 2,3. Radar observed PCP on July 18, 2015 PRECIPT Rain , 12:50 UTC, 1/13 Interpolated PCP on July 18,2015 using inverse distance weighting (IDW) 2 Flannigan, M. D., and Wotton, B. M., (1989) A study of interpolation methods for forest fire danger rating in Canada. Can. J. For.Res. 19: Flannigan, M. D., Wotton, B. M., and Ziga, S., (1998) A study on the interpolation of fire danger using radar precipitation estimates. International Journal of Wildland Fire. 8:

6 6 Potential Solutions 1: Canadian Precipitation Analysis (CaPA) System 4 (produced by Environment and Climate Change Canada) Station observations CaPA Radar Use optimal interpolation procedure to combine the three sources Produce near real time 10km gridded PCP GEM forecasts 4 Mahfouf et al. (2007). A Canadian precipitation analysis (CaPA) project: Description and preliminary results. AtmosphereOcean,45:1-17.

7 7 Potential Solutions 1: Canadian Precipitation Analysis (CaPA) System 4 (produced by Environment and Climate Change Canada) Station observations CaPA Radar Use optimal interpolation procedure to combine the three sources Produce near real time 10km gridded PCP GEM forecasts 4 Mahfouf et al. (2007). A Canadian precipitation analysis (CaPA) project: Description and preliminary results. AtmosphereOcean,45:1-17.

8 Potential Solution 2: use other interpolation methods 8

9 9 Potential Solution 2: use other interpolation methods Thin Plate Spline 5 (TPS): smooth and non-smooth A Thin Plate Splines surface A field is fitted with minimum mean-square error under a smooth curvature constraint. 5 Cressie, N., (1993) Statistics for Spatial Data. John Wiley and Sons, 900 pp.

10 10 Potential Solution 2: use other interpolation methods Thin Plate Spline 5 (TPS): smooth and non-smooth Ordinary Kriging 5 (ok) A Thin Plate Splines surface Exponential Model A field is fitted with minimum mean-square error under a smooth curvature constraint. A geostatistical method that models spatial variability using regression analysis on the covariance structure as a function of distance (i.e., variogram modeling). 5 Cressie, N., (1993) Statistics for Spatial Data. John Wiley and Sons, 900 pp.

11 11 Potential Solution 3: A combination of solution 1 and solution 2 Regression Kriging with CaPA (rk_capa) Fit a regression model using observed precipitation and CaPA outputs Then, build the variogram model using the residuals

12 12 Research Questions 1. Is CaPA superior in estimating PCP in Alberta? What about radar versus non-radar areas? 2. What are the implications of improved PCP estimates on FWI indexes? 3. How sensitive is the PCP and FWI to weather station density?

13 13 Methodology Leave-one-out cross-validation (LOOCV)

14 14 Methodology Leave-one-out cross-validation (LOOCV) Areas are divided into validated areas to remove the edge effect Data: Alberta fire weather station observations for 2014 (14 th July 31 st August) and 2015 (1 st May to 31 st August). Source: Alberta Agriculture and Forestry (AAF)

15 15 Methodology Leave-one-out cross-validation (LOOCV) Candidate Methods Transformation of precipitation Acronym Canadian precipitation analysis system CaPA Inverse distance weighting idw Thin-plate spline non-smoothing tps_ns Thin-plate spline non-smoothing square root tps_ns_s Thin-plate spline smoothing tps_s Thin-plate spline smoothing square root tps_s_s Ordinary kriging ok Ordinary kriging square root ok_s Regression kriging with CaPA rk(capa) Regression kriging with CaPA square root rk(capa)_s Areas are divided into validated areas to remove the edge effect Data: Alberta fire weather station observations for 2014 (14 th July 31 st August) and 2015 (1 st May to 31 st August). Source: Alberta Agriculture and Forestry (AAF)

16 16 Results 1.1: Performance rank of the 10 PCP prediction methods Best Worst ( )=,, = = n= number of weather stations (i.e.81); k= number of days (i.e.123)

17 17 Results 1.1: Performance rank of the 10 PCP prediction methods Best Worst ( )=,, = = n= number of weather stations (i.e.81); k= number of days (i.e.123) Mean error (B ) = = = (,, )

18 18 Results 1.2: Statistical tests of the PCP prediction methods A non-parametric resampling hypothesis test 6 was applied for the spatial and temporal autocorrelation of the data. Stationary block bootstrapping 7 (SBB) was implemented (n=1000 times). Resampling ANOVA test followed by resampling Post-hoc paired t-tests with Holm-Bonferroni p-value adjustment 5 % CI 95 % CI Test statistic: MAE(capa) MAE(idw) 6 Politis &Romano. (1994). The Stationary Bootstrap. Journal of the American Statistical Association, Vol.89: Hamill, T. (1999). Hypotehsis test for evaluating numerical precipitation forecasts. America Meteorological Society,

19 19 Results 1.2: Statistical tests of the PCP prediction methods A non-parametric resampling hypothesis test 6 was applied for the spatial and temporal autocorrelation of the data. Stationary block bootstrapping 7 (SBB) was implemented (n=1000 times). Resampling ANOVA test followed by resampling Post-hoc paired t-tests with Holm-Bonferroni p-value adjustment Overall MAE (mm) for 2014 and 2015 (test statistics) Method ok idw tps 5 % CI _ns tps _s rk (capa) CaPA tps 95 % CI _ns_s tps _s_s rk (capa)_s ok_s ok Test statistic: idw MAE(capa) MAE(idw) tps_ns tps _s rk(capa) CaPA tps_ns_s tps_s_s rk(capa)_s ok_s : significant in year 2014 : significant in year Politis &Romano. (1994). The Stationary Bootstrap. Journal of the American Statistical Association, Vol.89: Hamill, T. (1999). Hypotehsis test for evaluating numerical precipitation forecasts. America Meteorological Society,

20 20 Results 1.3: Spatial variation between the PCP prediction methods Spatial distribution of MAE in 2015

21 21 Results 1.3: Spatial variation between the PCP prediction methods MAE Performance Ranking: Radar versus Non-radar Performane rank of PCP candidate methods in radar area ok_sqrt tps_s_sqrt rk(capa)_sqrt tps_ns_sqrt CaPA tps_s rk(capa) tps_ns IDW Performance rank of PCP candidate methdos in non-radar area ok CaPA, rk(capa), and rk(capa)_s were the most improved methods in radar covered regions.

22 22 Results 1.3: Spatial variation between the PCP prediction methods MAE Performance Ranking: Radar versus Non-radar Performane rank of PCP candidate methods in radar area ok_sqrt tps_s_sqrt rk(capa)_sqrt tps_ns_sqrt CaPA tps_s rk(capa) tps_ns IDW Performance rank of PCP candidate methdos in non-radar area ok CaPA, rk(capa), and rk(capa)_s were the most improved methods in radar covered regions.

23 23 Results 1.4: PCP prediction methods for fuel moisture codes PCP category: 0-0.5mm, mm, mm, >2.8mm Equitable Threat Score (ETS) = H-H R / (H+M+F-H R ) + ( + ) Where H R = Measures the accuracy of forecast events, best score is 1. Frequency Bias Index(FBI)=(H+F)/(H+M)-1 Measures the frequency of forecast events, best score is 0.

24 24 Results 1.4: PCP prediction methods for fuel moisture codes PCP category: 0-0.5mm, mm, mm, >2.8mm Equitable Threat Score (ETS) = H-H R / (H+M+F-H R ) + ( + ) Where H R = Measures the accuracy of forecast events, best score is 1. Frequency Bias Index(FBI)=(H+F)/(H+M)-1 Measures the frequency of forecast events, best score is 0.

25 25 Results 1.4: PCP prediction methods for fuel moisture codes PCP category: 0-0.5mm, mm, mm, >2.8mm Equitable Threat Score (ETS) = H-H R / (H+M+F-H R ) + ( + ) Where H R = Measures the accuracy of forecast events, best score is 1. Frequency Bias Index(FBI)=(H+F)/(H+M)-1 Measures the frequency of forecast events, best score is 0.

26 26 Results 2.1:Implications to FWI indexes MAE of FWI indexes calculated with estimated PCP and observed RH, WS, Temp for 2015 (1 st May to 31 st August). Methods PCP FFMC ISI FWI capa idw tps_ns tps_ns_sqrt tps_s tps_s_sqrt rk(capa) rk(capa)_sqrt ok ok_sqrt Worst Best

27 27 Results 2.1:Implications to FWI indexes MAE of FWI indexes calculated with estimated PCP and observed RH, WS, Temp for 2015 (1 st May to 31 st August). Methods PCP FFMC ISI FWI DMC DC BUI capa idw tps_ns tps_ns_sqrt Remember, CaPA had high negative bias for PCP > 2.8mm tps_s tps_s_sqrt rk(capa) rk(capa)_sqrt ok ok_sqrt FBI >2.8mm Worst Best

28 28 Results 3: the sensitivity of PCP/ FWI estimates to fire weather station density Weather station density by randomly selecting weather stations from the study area (rounded up) Scenario No. of stations No. of stations per 10000km 2 10% selected % selected % selected % selected % selected Weather station density = 2.69 wstn per 10,000 km 2 Note: The ECCC weather station density of CaPA was constant in the study area (~1.58 wstn per 10,000 km 2 in 2015)

29 29 Results 3: the sensitivity of PCP/ FWI estimates to fire weather station density Weather station density by randomly selecting weather stations from the study area (rounded up) Scenario No. of stations No. of stations per 10000km 2 10% selected % selected % selected % selected % selected Weather station density = 2.69 wstn per 10,000 km 2 Note: The ECCC weather station density of CaPA was constant in the study area (~1.58 wstn per 10,000 km 2 in 2015)

30 30 Results 3: the sensitivity of PCP/ FWI estimates to fire weather station density Weather station density by randomly selecting weather stations from the study area (rounded up) Scenario No. of stations No. of stations per 10000km 2 10% selected % selected % selected % selected % selected Weather station density = 2.69 wstn per 10,000 km 2 Note: The ECCC weather station density of CaPA was constant in the study area (~1.58 wstn per 10,000 km 2 in 2015)

31 31 Conclusions and implications 1. CaPA had a mid-tiered performance amongst the 10 candidate methods and was significantly better than IDW (6%-7% improvement). However, CaPA had a large tendency to underestimate PCP>2.8mm. 2. ok_sqrt, rk_capa_sqrt, and tps_s_sqrt were the top three methods and were significantly better than other methods (i.e., 23%, 21%, and 19% improvement compared to IDW). 3. CaPA related methods greatly improved under radar and rk_capa_sqrt was even better than CaPA. 4. Improved PCP estimates significantly improved FWI indexes. rk_capa_sqrt was the top method for quick drying indexes: FFMC, ISI, and FWI, while rk_capa was the top method for slow drying indexes: DC, DMC, and BUI. 5. CaPA is better than interpolation methods when there are <0.6 wstns per km 2. CaPA is a great seed value in regression kriging but should not be used directly (especially if wstn density >0.6 stns/10 000km 2 )

32 32 Take home PCP / FWI decision chart Is the area under radar? yes no rk(capa)_sqrt Is the density > 0.6 wstn / km 2? yes no rk(capa)_sqrt CaPA

33 Thank you, any questions? 33

34 34 Appendix Sensitivity of DMC/DC estimates to fire weather station density Use rk(capa) regardless of wstn density!

35 35 MAE Appendix Performance ranking of 18 PCP prediction methods between year 2014 and Performance rank of candidate methods ok idw tps_ns tps_s rk(capa) CaPA tps_ns_ln tps_ns_s tps_ns_c tps_s_ln tps_s_s tps_s_c ok_ln rk(capa)_ln rk(capa)_s ok_s rk(capa)_c ok_c Performance rank of candidate methods The best performing method (lowest MAE) had a rank of 1, while the worst performing method (biggest MAE) had a rank of 18. Performance rank of the 18 PCP candidate methods were similar (stable) between 2014 and 2015.

36 36 Appendix Histogram of monthly precipitation (mm/month) for all the active weather stations for 2014, 2015, and 30-year average ( ) in the overall study area. Both 2014 and 2015 were drier than the 30-yrs average and 2014 was especially dry.

37 37 Appendix Averaged temporal autocorrelation of daily PCP and FWI indexes for fire weather stations (n=81) in Averaged temporal autocorrelation indicate the mean block length (m) that requires for the Stationary Block Bootstrapping (see section a) to remove the temporal correlation of PCP and FWI indexes in the resampling hypothesis test. In this study, we chose m of 3 days for PCP; m of 4 days for FFMC; m of 5 days for FWI; m of 12 for DMC.