Error in Ecosystem Models

Transcription

1 Error in Ecosystem Models Observational Support for Model Complexity Jerry Wiggert USM - Dept. of Marine Science jerry.wiggert@usm.edu

2 Outline Assessment and Optimization of Ecosystem Model Parameters Application of Variational Adjoint Method - Single Site & 3 Ecosystem Models - Multiple Sites with 12 Ecosystem Models Take Home Messages from Exploration of Ecosystem Model Robustness Observational Program Support of Regional Ecosystem Model - Example from Chesapeake Bay GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

3 Data Assimilative Ecosystem Modeling Testbeds Goal: To objectively compare the performance of ecosystem models characterized by varying levels of complexity Which ecosystem structures are most robust? How much complexity is justified? Is it feasible to develop models that are applicable over many diverse ecosystems? How do we compare models objectively? Method: Apply Variational Adjoint within an assimilation framework GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

4 Single Site & 3 Ecosystem Models GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

5 Source of Assimilated Data JGOFS Arabian Sea Process Study 1995 X = WHOI Mooring = Sediment Trap = S7 Station Assimilated Data: Cruise Observations: nitrate (6) chlorophyll (5) primary production (5) zooplankton (5) Sediment Trap: 1-year time series of export flux Forcing Data: Mooring (1-year time series) Mixed Layer Velocity PAR Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

6 Physical Forcing: Arabian Sea (A) Depth [m] (B) Depth [m] F1 = F2 = F3 = O N D J F M A M J J A S O N D F1 = F4 = 1994/1995 F2 = F5 = Year Day 1995 F3 = MLD time series F1: Mooring F2: McCreary et al. model (21) F3: Murtugudde/Busalacchi model (1998) Cruises Data Black diamonds represent cruise times MLD Subset (lower panel) is from period denoted by black bar (upper panel) ttn-5 is sole cruise from this period Two additional idealized time series (F4 and F5) Modification to F3 F4: Match thermocline depth of F2 F5: Match timing of ML shoaling Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

7 Ecosystem Models Employed Designator Structure Descripon Source Literature EM4 EM5 EM8 4- component model Classic diatom mesoz system N - P - Z - D Designed for Arabian Sea 5- component model Microbial Loop Emphasis N - P - Z - D - DON Designed for BATS ( 8- component model Large & Small Size Classes for P, Z, D 2N - 2P - 2Z - 2D Designed for Indian Ocean (No DFe) McCreary et al., 1996, 21 Hood et al., 23 Hood et al., 21, 24 Coles et al., 24 Wiggert et al., 26, 27 Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

8 Cost Functions J = 1 N M T W 2 ij i=1 j=1 ( a ij â ) 2 ij W ij = 4C ij σ i aij: âij: modeled values observed values N: # of assimilated observations M: # of variables with available observations T: # of time steps Cij: availability switch J: Standard Cost Function JP: Predictive Cost Function Withheld used (seasonal subsets) Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

9 Pre-Assimilation (Chl) F3 F2 F1 depth [m] depth [m] depth [m] EM4 EM5 EM Pre-assimilation solutions primarily set by applied ecosystem model EM4 develops no Deep Chlorophyll Maximum (DCM) during Spring Intermonsoon (SIM) EM5 and EM8 develop DCM during the SIM for all three forcing fields Elevated chlorophyll extends to 1 m during SW Monsoon (SWM) in all three models when F3 is applied 1 ONDJ F MA MJ J OND 1994/ ONDJ F MA MJ J OND mgchl m -3 1 ONDJ F MA MJ J OND 1994/1995 Pre-assimilation distributions of phytoplankton chlorophyll 3 ecosystem models (EM4, EM5, EM8) and 3 forcing fields (F1 F3) Red diamonds denote times of assimilation availability Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

10 Post-Assimilation (Chl) - Experiment 1 EM4 EM5 EM8 Post-assimilation solutions primarily set by applied forcing field F3 F2 F1 depth [m] depth [m] depth [m] ONDJ F MA MJ J OND 1994/ ONDJ F MA MJ J OND mgchl m ONDJ F MA MJ J OND 1994/1995 All three models forced by F3 reveal NEM and SWM blooms DCM (EM5, EM8) under F2 forcing no longer develops EM4 develops DCM during SIM when F1 and F3 forcing are applied; EM5 and EM8 maintain this feature Application of F2 forcing leads to oscillatory behavior in EM4 and EM5 Presumably, with assimilation of higher resolution time series (mooring or satellite based), this would be mitigated Experiment 1: Obtain all model parameters using variational adjoint Experiment 2: Obtain objectively chosen subset of non-correlated parameters Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

11 Experiment 1 - Cost Function Summary J J JP J: Pre-assimilation Cost J: Post-assimilation Cost JP: Post-assimilation Predictive Cost Assimilation dramatically reduces the cost function (J) Reduction is 37 68% for all model/forcing combinations As expected, JP is consistently higher than J Reproducing unassimilated is more challenging In some cases, JP > J Over all three models, all cost functions are lower when F3 forcing is applied EM8 always produces highest J and JP EM4 always shows lower JP than J F1 F2 F3 Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

12 Experiment 2 - Cost Function Summary J J JP J: Pre-assimilation Cost J: Post-assimilation Cost JP: Post-assimilation Predictive Cost Assimilation dramatically reduces the cost function (J) Reduction is 24 57% for all model/forcing combinations Improvement is not as good as for Expt. 1 As for Expt. 1, JP is consistently higher than J JP < J (Always true) Average of 21% lower JP from Expt. 2 are always lower than those from Expt. 1 F1 F2 F3 Over all three models, all cost functions are lower when F3 forcing is applied (Same as for Expt. 1) EM4 and EM8 produce lowest JP over cases F3 forcing Value of JP is nearly identical Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

13 Insights from Experiment 2 Experiment 1 - Uncertainties in the parameter values computed from the Hessian matrix are high because of correlations between many of the ecosystem model parameters Experiment 2 - All optimized parameters are well constrained and are associated with relatively small uncertainties - Specific parameters included in the optimized parameter subset varies with ecosystem model and forcing field - Certain parameters are consistently well-constrained Remineralization rates Phytoplankton growth rates (µ) Due to cost functions being highly sensitive to chlorophyll, production and export Nutrient kinetics parameters (1/2 saturations, KN) also a sensitive parameter, Cost function is somewhat less sensitive than for growth rates Since µ and KN are highly correlated, only the former was used in Expt. 2 Assimilation efficiency elicits greater cost function sensitivity than maximum grazing rate Mortality rates of P and Z rarely appeared in the objectively chosen parameter subsets GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

14 Export Flux Results F1 F2 F3 ST Flux [mg N m -2 d -1 ] ST Flux [mg N m -2 d -1 ] ST Flux [mg N m -2 d -1 ] Pre-assimilation Experiment # N D J F M A M J J A N D J F M A M J J A 1994/1995 = = EM5 = EM4 = EM Experiment #2 N D J F M A M J J A 1994/1995 Export time series demonstrate that neither the EM4 nor the EM5 models produce reasonable sediment fluxes EM4 shows oscillating export flux EM5 shows negative fluxes EM8 does not exhibit these issues With F3, EM8 shows greater capacity to capture episodic flux events NOTE: Weights assigned in cost function will control the influence of various solution aspects on total cost GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

15 Multiple Sites & 12 Ecosystem Models GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

16 Source of Assimilated Data JGOFS Arabian Sea/EqPac Process Studies Mooring Data: Temperature PAR Sediment Trap: 1 year time series Cruise Observations: nitrate chlorophyll production Larry Anderson (WHOI) Rob Armstrong (Stony Brook) Fei Chai (U. ME) Jim Christian (U. Victoria) Scott Doney (WHOI) John Dunne (NOAA GFDL) Jeff Dusenberry (WHOI) Masahiko Fujii (U. ME) Raleigh Hood (U. MD) Keith Moore (U. CA. Irvine) Dennis McGillicuddy (WHOI) Markus Schartau (GKSS, Germany) Yvette Spitz (OSU) Jerry Wiggert (ODU) Wide Range of Model Complexity: (NPZD -> 5N,3P,3Z,2D) Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

17 Multiple Oceanic Sites w/ 12 Models Expand to EQPAC and Arabian Sea sites - Experiment 1 Optimize subset of model parameters for each site Compute cost when applying optimized parameters at coinciding site (Individual Cost) - Experiment 2 Optimize subset of model parameters using from both sites Compute cost obtained when applying this blended set of parameters (Simultaneous Cost) - Experiment 3 Optimize subset of model parameters at each site Compute cost when applying optimized parameters from other site (Cross-Validation Cost) Expand to a suite of 12 Ecosystem Models - Model complexity ranges from 4-component N-P-Z-D ecosystem to 24-compartment model with multiple planktonic functional groups - In presented results, single-p and multi-p models are distinguished Normalized Cost => Normalize by cost of Mean Model GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

18 Optimization Experiments Expt. 1 Expt. 2 Expt. 3 simultaneous Individual cost Simultaneous cost Cross validation cost Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

19 Expt. 1 Experiment 1 Expt. 2 Expt. 3 simultaneous Individual cost Simultaneous cost Cross validation cost Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

20 Experiment 1: Individual Cost No Assim Individual Assim: Expt. 1 No improvement with additional complexity Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

21 Expt. 1 Experiment 2 Expt. 2 Expt. 3 simultaneous Individual cost Simultaneous cost Cross validation cost Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

22 Experiment 2: Simultaneous Assimilation Expt. 1 Expt. 2 More complex (multi-p) models can fit at both locations simultaneously better than simpler models Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

23 Expt. 1 Experiment 3 Expt. 2 Expt. 3 simultaneous Individual cost Simultaneous cost Cross validation cost Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

24 Experiment 3: Cross Validation Worse than mean Model Complexity = Single P = Multi-P Better than mean Only five models do better than mean More complex (multi-p) models do better in cross validation, i.e. are more portable Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

25 Take Home Messages GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

26 Summary & Conclusions: Modeling Testbeds The variational adjoint method can be used to objectively and quantitatively compare ecosystem model performance Simple models can fit well at individual sites Multiple P compartment models are best able to fit simultaneously at both sites Models can fit similarly well, but do so via very different pathways - Need to incorporate that better constrain the model flows and dynamics Data assimilation is crucial to objectively and quantitatively compare/assess ecosystem model performance Need to assess model performance by more than just how well the model reproduces the used to tune the model Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

27 Paramount Needs for Developing an Ecosystem Model with Trusted Fidelity 1) Comprehensive & Ongoing Sampling Program (Space and Time) 2) Diversity in Observations (Concentrations, Rates & Fluxes) GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

28 Observational Support Enabling Development of Chesapeake Bay Ecosystem Model GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

29 CBP Sampling Sites CB3.3C (upper bay) CB5.3 (middle bay) CB6.3 (lower bay) Chesapeake Bay Program ( Data Used For: Initial Conditions River Boundary Conditions Solution Validation Sites (following Xu & Hood, 26) CB3.3C (Upper Bay) CB5.3 (Mid-bay) C6.3 (Lower Bay) Chlorophyll Dissolved Oxygen DON, PON Freshwater Flux NO3/NO2/NH4 TSS Map Courtesy of Chesapeake Bay Program GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

30 Biogeochemical Flows In ChesROMS Atmospheric Deposition Diffuse Sources N 2 River Inputs O 2 NO 3 NH 4 DON Chl P Z ISS D L D S NO 3 NH 4 Wiggert et al., in prep. Aspects of the Mechanistic Dissolved Oxygen Model Implementation (DO is Indicated by the Blue-Green Background) 1) Light Model Based on Xu et al (25) 2) Water Column NTR follows Olson (1981) 3) Water Column DNF follows Oguz (22) 4) Partition of Oxic/Anoxic Sediments based on Seitzinger & Giblin (1996) i) 14% DNF : 86% NTR ii) % of DNF ramps up as overlying DO tends toward hypoxia 5) Benthic NH4 Efflux of NO3 Uptake also ramp up as overlying DO decreases 6) N Inputs from Atmospheric Deposition and Point & Diffuse Sources (N-Loading Promotes O2 Demand & NH4 Efflux) 7) Grazing f(temp); Based on Huntley and Lopez 1992 (Improved Bloom Dyn.) 8) Reduce POM sinking in bottom layer i) Promote O2 Demand in Water Column ii) Promote BGC link to Estuarine Circulation GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

31 Shift in BGC Processes w/ [O2] Mid-Bay Measurements (Cowan & Boynton 1996) NH4 Flux Oxygenated overlying waters Mid-Bay[O2] High NO3 (~2 um) Low O2 (<8 um) in overlying waters from Middelburg et al GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

32 CHL and DO Comparisons Chlorophyll Upper Bay (4.1C) Mid-Bay (5.3) Lower Bay (6.3) 3 tst26: chlorophyll at stationcb4.1c for year=1999 Model Data 35 3 tst26: chlorophyll at stationcb5.3 for year=1999 Model Data 25 tst26: chlorophyll at stationcb6.3 for year=1999 Model Data chlorophyll 2 15 chlorophyll 2 15 chlorophyll Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Dissolved Oxygen Upper Bay Mid-Bay Lower Bay tst26: oxygen at stationcb4.1c for year=1999 tst26: oxygen at stationcb5.3 for year=1999 tst26: oxygen at stationcb6.3 for year= Model Data 4 Model Data 4 Model Data oxygen 3 25 oxygen 25 2 oxygen Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Wiggert et al., in prep. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

33 NO3 and NH4 Comparisons NO3 Upper Bay (4.1C) Mid-Bay (5.3) Lower Bay (6.3) 35 tst26: NO3 at stationcb4.1c for year=1999 Model Data 12 tst26: NO3 at stationcb5.3 for year=1999 Model Data 3 tst26: NO3 at stationcb6.3 for year=1999 Model Data NO3 2 NO3 6 NO Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec NH4 Upper Bay Mid-Bay Lower Bay 25 tst26: NH4 at stationcb4.1c for year=1999 Model Data 2 18 tst26: NH4 at stationcb5.3 for year=1999 Model Data 15 tst26: NH4 at stationcb6.3 for year=1999 Model Data NH4 NH4 1 NH Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Wiggert et al., in prep. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

34 Skill (New Base 26, AqSci Base 15) Model Skill Comparison: Chlorophyll Basline: Test26 Baseline Test15 Test159 Test168 Test174 Test Model Skill Comparison: Ammonium Basline: Test26 Baseline Test15 Test159 Test168 Test174 Test197 Index Station CB2.2 Model Skill Model Skill CB3.1 3 CB3.2 4 CB3.3C 5 CB4.1C CB4.1W 7 CB4.2W CB4.3C CB4.3W Downbay > Downbay > 1 CB CB CB Model Skill Comparison: Nitrate Basline: Test26 Baseline Test15 Test159 Test168 Test174 Test Model Skill Comparison: Dissolved Oxygen Basline: Test26 13 CB CB5.4W 15 CB CB CB CB6.4 Model Skill Model Skill CB7.1 2 CB7.1N 21 CB7.1S 22 CB CB7.2E Baseline Test15 Test159 Test168 Test174 Test CB Downbay > Downbay > GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

35 CBP - Enabled Model Objective Chesapeake Bay Hypoxic Volume from CBP Interpolator Hypoxic Volume (km 3 ) Decimal Month Year Data holdings obtained by CBP provide opportunity for developing a coupled physicalbiogeochemical model with fully mechanistic dissolved oxygen that can simulate full range of oxic conditions over seasonal to interannual time frame. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

36 Extras GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

37 ChesROMS: Bottom DO Animation Wiggert et al., in prep. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

38 Post-Assimilation (Chl) - Experiment 2 EM4 EM5 EM8 XXX F3 F2 F1 depth [m] depth [m] depth [m] ONDJ F MA MJ J OND 1994/ ONDJ F MA MJ J OND mgchl m -3 1 ONDJ F MA MJ J OND 1994/1995 Experiment 1: Obtain all model parameters using variational adjoint Experiment 2: Obtain objectively chosen subset of non-correlated parameters Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

39 Comparison 1: Optimize all parameters When all parameters are optimized, Eco_4 and Emp_4 fit the equally well Important to objectively optimize models prior to comparison Friedrichs, Hood & Wiggert, Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

40 Note: Parameters selected on the basis of cost function sensitivity The mechanistic model fits the better than an empirical model with the same number of degrees of freedom Friedrichs, Hood & Wiggert, Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

41 Variational Adjoint Method Expt. 1 Expt. 2 Expt. 3 simultaneous Individual cost Simultaneous cost Cross validation cost Friedrichs, et al., JGR - Oceans, 27. GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

42 Cost Ratio (Mechanistic/Empirical) Eco_ 4 =.32 ±.2 Eco _ 5 =.28 ±.3 Eco _ 8 =.28 ±.2 If only 2 to 4 parameters are optimized, the mechanistic models all fit the ~7% better than the empirical model GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214

43 Summary: Three comparisons Cost Function Optimize all parameters Cost Ratio Optimize key parameters Predictive Cost Ratio Optimize all parameters There is no improvement in model- fit with the addition of more ecosystem model complexity Friedrichs et al., Deep-Sea Res., 26 GCOOS-MTT Ecosystem Modeling Workshop; 7-9 April, 214