Francesca Pianosi and Thorsten Wagener Department of Civil Engineering University of Bristol! credible.bris.ac.uk NE/J017450/1

Transcription

1 Global Sensitivity Analysis to support model calibration, evaluation, uncertainty propagation and robust decision-making: a toolbox for access to methods and workflows Francesca Pianosi and Thorsten Wagener Department of Civil Engineering University of Bristol credible.bris.ac.uk NE/J017450/1 francesca.pianosi@bristol.ac.uk

2 Global Sensitivity Analysis is a set of statistical techniques that provide a structured approach to tackle several types of uncertainty associated with the development and application of numerical models GSA is useful for: :: more efficient model calibration :: better understanding of model response :: prioritizing efforts for uncertainty reduction (e.g. acquisition of new data) :: assessing robustness to modeling assumptions :: francesca.pianosi@bristol.ac.uk

3 Global Sensitivity Analysis investigates how the variation in the output of a numerical model can be attributed to variations of its input factors time forcing inputs parameters pdf BETA LP FC resolution NUMERICAL MODEL output parameters structure (equations) forcing inputs resolution interactions francesca.pianosi@bristol.ac.uk

4 Outline :: Key concepts underlying GSA techniques :: Examples of GSA applications :: SAFE: a Matlab/Octave/R toolbox for GSA francesca.pianosi@bristol.ac.uk

5 Key concepts underlying GSA techniques

6 Steps of sampling-based GSA 1 INPUT SAMPLING Latin-hypercube Quasi-random sampling BETA LP FC 2 MODEL EVALUATION pdf output 3 POST PROCESSING sensitivity Sensitivity indices x 1 x 2 x 3 francesca.pianosi@bristol.ac.uk

7 Steps of sampling-based GSA 1 INPUT SAMPLING 2 MODEL EVALUATION Sensitivity measured by: :: multiple-start derivatives/differences (e.g. Morris method, DELSA, ) :: correlation between inputs and outputs 3 POST PROCESSING :: properties of input distributions after conditioning outputs (Monte Carlo filtering) :: properties of output distribution after conditioning inputs (e.g. Sobol method, density-based methods, ) sensitivity Sensitivity indices x 1 x 2 x 3 francesca.pianosi@bristol.ac.uk

8 Steps of sampling-based GSA 1 INPUT SAMPLING 2 MODEL EVALUATION 3 POST PROCESSING :: Computational cost of post-processing is negligible wrt to model evaluation :: Required number of model evaluations grows with number of input factors :: Growth rate differs from method to method sensitivity Sensitivity indices x 1 x 2 x 3 francesca.pianosi@bristol.ac.uk

9 Choice of GSA method: classification system Pianosi et al Env.Mod&Soft Specific purpose Screening Ranking Mapping M = number of input factors implemented in the SAFE Toolbox Number of model evaluations >1000 x M >10 x M >100 x M Multiple-starts derivatives Monte-Carlo filtering Correlation & Regression Analysis Variance- based & Density- based EET (Morris) FAST VBSA (Sobol ) Regional Sensitivity Analysis CART PAWN francesca.pianosi@bristol.ac.uk

10 Examples

11 Supporting calibration of a land surface model with J. Iwema, R. Rosolem Which parameters mostly affect the model performance? Which parameters have little influence and can be set to default values? transpiration rainfall water on canopy evaporation rainfall evaporation WATER runoff ENERGY latent heat sensible heat surface skin temperature soil heat soil moisture runoff soil temperature Application to the Santa Rita creosote site in the US drainage The Joint UK Land Environment Simulator (JULES) francesca.pianosi@bristol.ac.uk

12 Supporting calibration of a land surface model with J. Iwema, R. Rosolem Sensitivity of RMSE of different simulated variables to uncertain parameters and initial conditions sensitivity Sensible heat 2. Latent heat sensitivity Soil moisture (TDT) 4. Soil moisture (CRNS) : parameters: b sathh satcon sm_sat sm_crit sm_wilt hcap hcon albsoil 10-12: initial conditions: tsar_tile sthuf t_soil francesca.pianosi@bristol.ac.uk

13 Supporting calibration of a land surface model with J. Iwema, R. Rosolem Sensitivity of RMSE of different simulated variables to uncertain parameters and initial conditions sensitivity Sensible heat 2. Latent heat sensitivity Soil moisture (TDT) 4. Soil moisture (CRNS) : parameters: b sathh satcon sm_sat sm_crit sm_wilt hcap hcon albsoil 10-12: initial conditions: tsar_tile sthuf t_soil francesca.pianosi@bristol.ac.uk

14 Supporting calibration of a land surface model with J. Iwema, R. Rosolem b sathh satcon sm sat sm crit sm wilt hcap hcon albsoil tstar tile sthuf t soil parameters initial conditions samples that improve model performances wrt default set-up default set-up values francesca.pianosi@bristol.ac.uk

15 Supporting calibration of a land surface model with J. Iwema, R. Rosolem b sathh satcon sm sat sm crit sm wilt hcap hcon albsoil tstar tile sthuf t soil parameters initial conditions samples that improve model performances wrt default set-up default set-up values francesca.pianosi@bristol.ac.uk

16 Investigating uncertainties in a flood inundation model with J. Savage, P. Bates, J. Freer How important is the choice of the model s spatial resolution for flood simulations with respect to other uncertain factors? LISFLOOD-FP model applied to Imera Basin, Italy francesca.pianosi@bristol.ac.uk

17 Investigating uncertainties in a flood inundation model with J. Savage, P. Bates, J. Freer Flood extent = percentage of wet cells (water depth > 0.10 m) Uncertain input factors: - spatial Resolution - Channel friction (parameter) - Floodplain friction (parameter) - Forcing Hydrograph (boundary condition) - DEM: Digital Elevation Model uncertainty in predicted flood extent time (hours) francesca.pianosi@bristol.ac.uk

18 Investigating uncertainties in a flood inundation model with J. Savage, P. Bates, J. Freer Flood extent = percentage of wet cells (water depth > 0.10 m) Uncertain input factors: - spatial Resolution - Channel friction (parameter) - Floodplain friction (parameter) - Forcing Hydrograph (boundary condition) - DEM: Digital Elevation Model uncertainty in predicted flood extent time (hours) most important contributor to uncertainty (Sobol ) time (hours) francesca.pianosi@bristol.ac.uk

19 Investigating uncertainties in a flood inundation model with J. Savage, P. Bates, J. Freer Uncertain input factors: spatial Resolution Channel friction Floodplain friction Forcing Hydrograph DEM francesca.pianosi@bristol.ac.uk

20 Finding key drivers of slope failure with S. Almeida and L. Holcombe What are the dominant drivers of landslides in a slope with properties known at different level of certainty? Slope properties (geometry&soil): Thickness of top soil (m) [2,6] etc. (25 in total) Design-storm > deeply uncertain: Slope angle (degrees) [27,30] Duration Time Intensity rainfall water table evaporation runoff slip circle the Combined Hydrology And Slope Stability Model (CHASM) francesca.pianosi@bristol.ac.uk

21 Finding key drivers of slope failure with S. Almeida and L. Holcombe Cohesion/ Thickness top soil Results of CART analysis <2.0 >2.0 duration Cohesion/ Thickness top soil <11 >11 Thickness top soil duration <3 >3 <1.5 >1.5 Cohesion/ Thickness top soil <7.5 >7.5 <3.2 >3.2 intensity <18 >18 intensity Depth WT <47 >47 Fail <80 >80 Fail Fail intensity <5 >5 Fail

22 Finding key drivers of slope failure with S. Almeida and L. Holcombe Cohesion/ Thickness top soil Results of CART analysis :: the dominant drivers of landslides in this slope are: 1 cohesion of the top soil 2 thickness of the top soil 3 rainfall duration 4 rainfall intensity 5 depth of water table duration <3 >3 <47 Cohesion/ Thickness top soil <1.5 intensity >47 >1.5 <11 duration Fail >11 Cohesion/ Thickness top soil <7.5 >7.5 <2.0 Thickness top soil <3.2 >3.2 <18 Depth WT <80 >80 >2.0 intensity >18 Fail Fail intensity <5 >5 Fail francesca.pianosi@bristol.ac.uk

23 Finding key drivers of slope failure with S. Almeida and L. Holcombe Cohesion/ Thickness top soil Results of CART analysis :: the dominant drivers of landslides in this slope are: 1 cohesion of the top soil 2 thickness of the top soil 3 rainfall duration 4 rainfall intensity 5 depth of water table :: thresholds for these drivers that would lead to slope failure duration <3 >3 <47 Cohesion/ Thickness top soil <1.5 intensity >47 Fail >1.5 <11 duration Fail >11 Cohesion/ Thickness top soil <7.5 >7.5 <2.0 Thickness top soil <3.2 >3.2 <18 Depth WT <80 >80 >2.0 intensity intensity >18 Fail <5 >5 Fail francesca.pianosi@bristol.ac.uk

24 SAFE: a Matlab/Octave/R toolbox for GSA francesca.pianosi@bristol.ac.uk

25 Characteristics of the SAFE Toolbox :: It works under Matlab/Octave (an R version is also available) :: flexible, modular structure > easy to integrate with models running outside matlab :: tutorial scripts (workflows) to get started more in our introductory paper on Env. Mod & Soft (2015) francesca.pianosi@bristol.ac.uk

26 Uptake in academia Freely available for non-commercial use since December, Introductory paper published on Env. Mod & Soft in May, 2015 About 300 academic users so far About 300 academic users so far

27 Uptake in industry Download requests for a closed-code 3-months trial version by: - E.ON Energy (Uncertainty in yield prediction for wind farms) - Pfizer (Physiological Based Pharmacokinetic models) - EDF (Thermochemical Heat Storage) Ongoing collaboration with: - Risk Management Solutions > support calibration of rainfall-runoff models > find key drivers of loss models - Airbus > uncertainty in aircraft design models - JBA Trust > support long-term investment plans for flood risk reduction francesca.pianosi@bristol.ac.uk

28 Conclusions Our is very active in GSA research, contributing to both methodological advances and development of GSA tools The SAFE Toolbox is freely available for non-commercial use from: