Assessing internal variability with few ensemble runs

Transcription

1 Assessing internal variability with few ensemble runs Dorit Hammerling Data Analytics and Integrative Machine Learning Section Technology Development Division National Center for Atmospheric Research (NCAR) April 24, 2018 Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

2 Collaborators Stefano Castruccio Ben Sanderson Alicia Karspeck Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

3 The National Center for Atmospheric Research (NCAR) An NSF funded research and development center Mission: To understand the behavior of the atmosphere and related Earth and geospace systems Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

4 NCAR s Community Earth System Model a virtual laboratory to study past, present and future climate states describes interactions of the atmosphere, land, river runoff, land-ice, oceans and sea-ice complex! Large code base: approx. 1.5 Millions lines of code Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

5 Variability in Climate Models Climate models are typically implemented in a deterministic way. Chaotic systems: small changes lead to large effects Initial conditions ensembles are used to study internal variability. Many different sources of variability: choice of process representations and parameterization, resolution, numerical implementation, tuning,... Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

6 Variability in Climate Models Climate models are typically implemented in a deterministic way. Chaotic systems: small changes lead to large effects Initial conditions ensembles are used to study internal variability. Many different sources of variability: choice of process representations and parameterization, resolution, numerical implementation, tuning,... Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

7 Variability in Climate Models Climate models are typically implemented in a deterministic way. Chaotic systems: small changes lead to large effects Initial conditions ensembles are used to study internal variability. Many different sources of variability: choice of process representations and parameterization, resolution, numerical implementation, tuning,... Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

8 Variability in Climate Models Climate models are typically implemented in a deterministic way. Chaotic systems: small changes lead to large effects Initial conditions ensembles are used to study internal variability. Many different sources of variability: choice of process representations and parameterization, resolution, numerical implementation, tuning,... Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

9 Variability in Climate Models Climate models are typically implemented in a deterministic way. Chaotic systems: small changes lead to large effects Initial conditions ensembles are used to study internal variability. Many different sources of variability: choice of process representations and parameterization, resolution, numerical implementation, tuning,... Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

10 Ensembles are powerful Provide sheer endless opportunity for studying processes of interest, complex relationships and internal variability BUT, they require a lot of effort in terms of setup computational execution and verification storage... Not feasible to conduct very often. Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

11 Ensembles are powerful Provide sheer endless opportunity for studying processes of interest, complex relationships and internal variability BUT, they require a lot of effort in terms of setup computational execution and verification storage... Not feasible to conduct very often. Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

12 Ensembles are powerful Provide sheer endless opportunity for studying processes of interest, complex relationships and internal variability BUT, they require a lot of effort in terms of setup computational execution and verification storage... Not feasible to conduct very often. Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

13 The CESM Large Ensemble () 30 global climate simulations conducted using a nominal 1 spatial configuration to 2100: historical emissions from and RCP 8.5 scenario for Identical radiative forcing and thus ensemble mean can be considered as an estimate of the forced response of the model, key interest: variability around the mean 10 million CPU hours and over 300 TB in storage Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

14 The CESM Large Ensemble () 30 global climate simulations conducted using a nominal 1 spatial configuration to 2100: historical emissions from and RCP 8.5 scenario for Identical radiative forcing and thus ensemble mean can be considered as an estimate of the forced response of the model, key interest: variability around the mean 10 million CPU hours and over 300 TB in storage Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

15 The CESM Large Ensemble () 30 global climate simulations conducted using a nominal 1 spatial configuration to 2100: historical emissions from and RCP 8.5 scenario for Identical radiative forcing and thus ensemble mean can be considered as an estimate of the forced response of the model, key interest: variability around the mean 10 million CPU hours and over 300 TB in storage Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

16 The CESM Large Ensemble () 30 global climate simulations conducted using a nominal 1 spatial configuration to 2100: historical emissions from and RCP 8.5 scenario for Identical radiative forcing and thus ensemble mean can be considered as an estimate of the forced response of the model, key interest: variability around the mean 10 million CPU hours and over 300 TB in storage Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

17 Where do statistical models come in? Maybe some simple questions can be answered by training a statistical model on fewer runs? Idea: train the statistical model using only a few runs, simulate new values and then compare the distribution from the statistical model and the original ensemble. Question: Is the internal variability from the statistical model equivalent (i.e. indistinguishable) from the internal variability of the actual ensemble for some specific property of interest? Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

18 Where do statistical models come in? Maybe some simple questions can be answered by training a statistical model on fewer runs? Idea: train the statistical model using only a few runs, simulate new values and then compare the distribution from the statistical model and the original ensemble. Question: Is the internal variability from the statistical model equivalent (i.e. indistinguishable) from the internal variability of the actual ensemble for some specific property of interest? Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

19 Where do statistical models come in? Maybe some simple questions can be answered by training a statistical model on fewer runs? Idea: train the statistical model using only a few runs, simulate new values and then compare the distribution from the statistical model and the original ensemble. Question: Is the internal variability from the statistical model equivalent (i.e. indistinguishable) from the internal variability of the actual ensemble for some specific property of interest? Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

20 Climatological regions Area codes ARL ARO ALA HBO CGI NNA NEU WNA CNA ENA SEU NNE MED NAT CAM CAR SAH NPE WAF EPW EPE EAF EAT AMZ SAF SPW SPE SAT SSA NAS CAS TIB NNW EAS SAS SEAWPN NPW WPE IND WPS NAU SAU SIO AOP AOI ANL Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

21 Regional temperature trends and trend variability Winter temperature trends ( C/34 years) for the time period. Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

22 The time series model: notation and main ideas Notation: T (t): monthly temperature for each region f (t) = log ( [CO 2e ](t)/[co 2e ] (B)) : forcing Basic idea: present and past forcing determines present temperature T (t) = µ(f (t), f (t 1),...) + ε(t), technical term: infinite distributed lag model µ is the mean temperature, error is ε(t) = φ 1 ε(t 1) + φ 2 ε(t 2) + η(t) Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

23 The time series model: notation and main ideas Notation: T (t): monthly temperature for each region f (t) = log ( [CO 2e ](t)/[co 2e ] (B)) : forcing Basic idea: present and past forcing determines present temperature T (t) = µ(f (t), f (t 1),...) + ε(t), technical term: infinite distributed lag model µ is the mean temperature, error is ε(t) = φ 1 ε(t 1) + φ 2 ε(t 2) + η(t) Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

24 The time series model: notation and main ideas Notation: T (t): monthly temperature for each region f (t) = log ( [CO 2e ](t)/[co 2e ] (B)) : forcing Basic idea: present and past forcing determines present temperature T (t) = µ(f (t), f (t 1),...) + ε(t), technical term: infinite distributed lag model µ is the mean temperature, error is ε(t) = φ 1 ε(t 1) + φ 2 ε(t 2) + η(t) Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

25 The time series model: some intuition on the components The mean is expressed as and T (t) = µ + ε(t), µ = β 0 + β 1 C(t) + K k=1 + K k=1 { γ k C(t) cos ( 2πtk 12 C(t) = { γk cos ( 2πtk 12 ) + ζ k C(t) sin ( 2πtk 12 (1 ρ)ρ m f (t m). m=0 base temperature, slope, monthly oscillation, change in monthly oscillation ) + ζk sin ( )} 2πtk 12 )}, Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

26 Parameter estimation in 2-step procedure 1. The month-specific variance of ε(t), as well as the autoregressive coefficients φ 1 and φ 2 are estimated from the training runs. This is obtained by computing the sample standard deviation for every month, and then performing a linear regression to obtain ˆφ 1 and ˆφ Conditionally on the parameters estimated in the first step, the profile likelihood of the model is then maximized with respect to ρ. Since all other model parameters are linear, the optimization is fast and can be performed within a few seconds on a modest laptop. No improvements using more than four training runs. Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

27 Fitted time series model for two regions C) T anomaly ( Year Year (a) (b) T anomaly ( C) C) (c) (d) C) T anomaly ( T anomaly ( Year Year -2-3 Climate model (magenta) vs statistical model (black) for the entire period and zoomed for 34 years. Top: Eastern North America, Bottom: North Atlantic. Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

28 Boxplot comparison of monthly distribution Jan Feb Mar Apr T anomaly ( C) May Jun Jul Aug Sep Oct Nov Dec Three runs of Large Ensemble compared against three runs from the generated from the statistical model for Eastern North America for Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

29 Boxplot comparison of monthly distribution T anomaly ( C) Jan Feb Mar Apr May Jun Jul Aug Sep 2.5 Oct Nov 0 Dec Three runs of Large Ensemble compared against three runs from the generated from the statistical model for the North Atlantic for Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

30 Comparison of trend estimates (a) (b) 5 4 T trend ( C/34 years) T trend ( C/34 years) (c) (d) NS NS 0.6 Winter surface air temperature trends ( C/34 years) for the 30 large ensemble runs (magenta) and 30 runs generated from the statistical model (black). Top: Eastern North America, Bottom: North Atlantic. Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

31 Results for all regions istical model Winter temperature trends ( C/34 years) for the time period. Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

32 Some problems with ocean regions ARO,1 ARL,0.7 ALA,1.3 CGI,1 HBO,1.2 NNA,0.7 NEU,1.2 NAS,0.9 NNW,0.7 NNE,1 WNA,1.1 CNA,1.2 ENA,1.2 NAT,0.8 SEU,0.8 CAS,0.8 TIB,0.8 NPE,1.5 CAM,1.2 CAR,0.9 MED,0.7 SAH,0.8 SAS,1.1 EAS,0.7 WPN,1.6 EPW,3.2 EPE,2.8 EAT,1.4 WAF,1.6 IND,1.8 SEA,1.8 NPW,1 SPW,0.7 SPE,0.9 AMZ,2.1 SAT,0.6 SAF,1 WPE,3.1 WPS,1.4 NAU,1.6 AOP,1.1 SSA,1.1 AOI,0.8 SIO,0.8 ANL,0.9 SAU,1.2 Comparison of the first five years for all regions using functional boxplots. Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

33 Summary and future work Summary: Distribution of monthly distribution and long term trends indistinguishable (for almost all regions) between Large Ensemble and statistical model Training the statistical model on four climate model simulations sufficient, no further gain using more Estimating parameters and creating new simulations takes less than a minute on a laptop Future Work: Model dependency between locations to use full spatial information Consider a more complex time series model to capture inter-annual variability Test modeling other properties Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

34 Summary and future work Summary: Distribution of monthly distribution and long term trends indistinguishable (for almost all regions) between Large Ensemble and statistical model Training the statistical model on four climate model simulations sufficient, no further gain using more Estimating parameters and creating new simulations takes less than a minute on a laptop Future Work: Model dependency between locations to use full spatial information Consider a more complex time series model to capture inter-annual variability Test modeling other properties Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

35 Summary and future work Summary: Distribution of monthly distribution and long term trends indistinguishable (for almost all regions) between Large Ensemble and statistical model Training the statistical model on four climate model simulations sufficient, no further gain using more Estimating parameters and creating new simulations takes less than a minute on a laptop Future Work: Model dependency between locations to use full spatial information Consider a more complex time series model to capture inter-annual variability Test modeling other properties Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21

36 Interested? If you are interested or have ideas, send us a note! dorith@ucar.edu Thanks! Hammerling et al. (NCAR) Internal Variability Assessment April 24, / 21