Fundamentals of modelling: The process and tools

Transcription

1 Fundamentals of modelling: The process and tools Water Modelling Overview Geoff Podger 10:00-10:45 (4:00-4:45) am Friday, 4 Nov 2016 BASIN MANAGEMENT OUTCOMES LAND AND WATER

2 Contents Best Practice Modelling 10 basic steps to problem definition Modelling tools and data Building confidence and model acceptance Interpreting outputs and telling the narrative 2 Fundamentals of Modelling Geoff Podger

3 Best Practice Modelling Guidelines 4 four major components Project administration Problem definition Option modelling Preferred option Needs to be done in the knowledge of the issues to be addressed Needs to be done before considering modelling and solutions It guides the nature and form of the conceptual model Stakeholder engagement is key 3 Fundamentals of Modelling Geoff Podger

4 Best practice modelling framework 4 Fundamentals of Modelling Geoff Podger

5 Problem definition 1. Problem statement 2. Objectives 3. Understanding the problem domain 4. System definition 5. Conceptual models 6. Metrics and criteria 7. Decision variables 8. Uncertainty and risk 9. Preliminary assessment 10. Getting the narrative right 5 Fundamentals of Modelling Geoff Podger

6 Problem definition Defining the purpose and scope of work Setting performance criteria Getting early stakeholder buy-in and support Required for both new projects or an extension to existing work Is likely to be an iterative process Budget and Terms of Reference may be provided but are rarely sufficient for a project to be implemented 6 Fundamentals of Modelling Geoff Podger

7 1. Problem statement Defining the problem is the most important step in any solution finding strategy Requires consultation with water managers/policy officers, modellers and the wider community Should be initially considered free of time resource and budget constraints so all problems and solutions are considered The constraints on possible ways forward can be clearly explained Technical Budget Time Resource Transparent priority setting for multiple options Is there a role for modelling in the project and what is this role? 7 Fundamentals of Modelling Geoff Podger

8 2. Objectives Project goals and objectives should be identified in a consultative process There may be multiple goals It might be useful to describe objectives in a hierarchy Consideration should be given to possible future objectives and goals as this might influence the design of the initial project. The decision on the best option will be guided by how well it meets the agreed objectives 8 Fundamentals of Modelling Geoff Podger

9 3. Understanding the problem domain The range of disciplines required to the understand the problem space needs to be identified and agreed: Water managers and policy officers Discipline experts Modellers Wider community/stakeholders Need to consider the environmental and economic impacts Social acceptance and adoption: just because the modelling says it is a good option does not mean that it will be adopted 9 Fundamentals of Modelling Geoff Podger

10 4. System definition Should be done in consultation with stakeholders Is critical to model applicability Requires identification of system Components Behaviour Feedbacks Boundaries Forcing states Outputs Need to decide what is in and out of scope Selection of the appropriate temporal and spatial scale and extent, which may be constrained by available data Documented in the conceptual model 10 Fundamentals of Modelling Geoff Podger

11 5. Conceptual model An agreed understanding of how the system works and the relative importance of different aspects Assumptions should be clearly stated (simplifications, exclusions) u/s gauge Q in-trib(obs) Q in(obs) Q in-rain(obs) -Q out-evp(obs) (in-stream) It is important that agreement is reached with stakeholders on the conceptual model. This may be an iterative procedure. Should consider multiple alternative conceptual models to understand model uncertainty Existing models should be considered in the context of the required conceptual model. What are limitations and how can these be addressed? d/s gauge Loss & delay due to dead storage Loss & delay due to over-bank flow (Evap & GW) (above threshold) Q in-ug-tribs(sim) Q loss(sim) Q out(sim) vs Q out(obs) 11 Fundamentals of Modelling Geoff Podger

12 Conceptual Model Design Climate Rainfall/Snow/Ice Surface/groundwater Water quality River systems modelling Biophysical modelling Ecosystems Socio-economic modelling Potable water Hydropower Agriculture Ecosystem services Social benefits Gender Geoff Podger Page 12 Fundamentals of Modelling Geoff Podger 12

13 Integrating Conceptual Framework Risk management Threats Levers for trade-off scenarios Socioeconomic Modelling Social Decision Modelling Bio-physical Modelling Hydrological systems and assets Services Wellbeing

14 Integrating Conceptual Framework Risk management Threats Climate Change Variability Land use Forest cover Landslides Bushfires Population Growth Over-allocation Surface water Groundwater Pollution Point Non point Levers for trade-off scenarios Policy/Management, Build Infrastructure, Treatment, Targets/Standards, Education Socioeconomic Modelling Services Social Decision Modelling Wellbeing Food Security Cyrosphere / Snow melt models River Flows Water Quantity Rainfall / Runoff models Groundwater Transport and fate Water Quality Sediment Nutrients Organics Pathogen Treatment Assets Environment Irrigation Hydropower Navigation Food production Potable WaterEnergy Security Energy production Water Security Energy Security Economic Security Storages Floodplains Heavy metals Urban water Hazard/Risk Sense of Security / Place Industrial Bio-physical Modelling Recreational Ecosystem Services Environmental Security

15 Model choice There are lots of different models available (too many acronyms to remember) and in many cases the underlying algorithms are similar There is no one model that can do everything Problem Space Model choice is a trade-off Data Model Choice Parsimony: Choose the simplest model that best answers the question Take into consideration uncertainty. Is the model telling us something useful or is it noise? Complexity Geoff Podger Page 15 Fundamentals of Modelling Geoff Podger 15

16 Model choice questions Problem Space (What are the issues to be considered?) Planning (scale, sectors, sharing rules, WQ, SW/GW) Operations (dams, structures, hydro, irrigation, environment, culture, WQ, GW pumping) Forecasts (scale, lead time, extent, floods, allocations) Data (What is needed and available?) Global/Local, Observed/Inferred, Historic/Real-time What is the uncertainty in the data? Complexity (What is justified given data and problem space?) Spatial and temporal scale Process description Run times Number or parameters 16 Fundamentals of Modelling Geoff Podger

17 Conceptual model check list Is the conceptual model domains, boundary conditions spatial/temporal scales and extents considered? Are all relevant processes/dependencies addressed and assumptions and limitations clearly stated? Has the need for an alternative conceptual model been addressed? Is the conceptual model sound and defensible? Is the model complexity supported by the available data? Has this conceptual model been applied elsewhere and lessons learnt have been considered? Is the conceptual model consistent with the project objectives? 17 Fundamentals of Modelling Geoff Podger

18 6. Metrics and criteria Model performance criteria and metrics need to be agreed up front else the goal post will change as results emerge Performance criteria and indicators that demonstrate compliance with agreed objectives need to be identified and agreed between modellers and stakeholders The metrics and criteria should reflect the purpose and objectives of the model e.g. different criteria for flood models compared to planning models Some performance criteria may be set by legislation e.g. drinking water quality standards A baseline period and associated development conditions needs to be agreed where performance against a baseline are considered Positive change might be a sufficient socially acceptable metric Risk around meeting performance criteria needs to be considered 18 Fundamentals of Modelling Geoff Podger

19 7. Decision variables Decision variables include anything that the stakeholders can adjust to influence the performance of the system Decision variables should be agreed early in the project in consultation with stakeholders Might include social and economic instruments and incentives, as well as institutional arrangements and the biophysical components of the system Different solutions are generated by considering different states of decision variables 19 Fundamentals of Modelling Geoff Podger

20 8. Uncertainty and risk Uncertainty Sources of uncertainty should be identified Considered for deciding on metrics and criteria Considered for decision variables May be qualitative in the problem definition phase Risk Considers the likelihood (frequency/certainty) and consequence (impact on assets) Understanding uncertainty is a essential part of risk assessment Likelihoods may be expressed qualitatively (high, med, low) and derived from expert knowledge 20 Fundamentals of Modelling Geoff Podger

21 9. Preliminary assessment Comprises a simple appraisal of the likely results from the planned modelling such that the results could be used as independent check on detailed modelling results. Where modelling results differ from the preliminary assessment further investigation of the model will be required to determine if the model is behaving correctly May lead to a review of project objectives and metrics 21 Fundamentals of Modelling Geoff Podger

22 10. Getting the narrative right So what? What does this mean for people $ Impact for farmers $ Recreation Impact of energy on business Loss/health of species People flooded/loss of life Sustainability of industry Impacts on the vulnerable Impacts on women and children Cease to flow days Flood level 22 Fundamentals of Modelling Geoff Podger

23 Water Resource Management Technology Remote sensing (LIDAR, DEM, ET and Land use) Climate surfaces, GCMs and downscaling Water Resource Information Systems Flood forecasting systems Flexible hydrological modelling frameworks Workflow tools (integrating hydrological, environmental and economic models) Modelling uncertainty and risk Technology in the cloud and web 23 Fundamentals of Modelling Geoff Podger

24 The spectrum of DEM products 9 second SRTM 3 second DEM-H 1 second Lidar 5 m 24 Geoff Fundamentals of Modelling Geoff Podger Podger Page 24

25 Catchment delineation Error in delineated watershed area 5% 1990 s 0.5% 2000 s 1 : 30 m 3 arcsec: 90 m From: Maidment (2011) DEM Cell Size 25 Geoff Fundamentals of Modelling Geoff Podger Podger Page 25

26 Climate data Climate variables Precipitation (rainfall, snow) Temperature Potential evaporation Data sources BoM daily observations BoM climate data sets Silo rainfall Humidity Solar radiation Wind speed/run Local data Global products (NOAA) 26 Fundamentals of Modelling Geoff Podger Geoff Podger Page 26

27 Climate change impact on water Change in mean annual runoff for 2 o C warming 2080 flood frequency for 20 th century 100-year flood 27 Geoff Fundamentals of Modelling Geoff Podger Podger Page 27

28 Water Resource Information Systems: Geofabric 28 Fundamentals of Modelling Geoff Podger

29 Seasonal streamflow forecasting 3 month probabilistic outlook of unregulated total streamflow volumes Ensemble forecasts at 74 sites in 32 river basins Uses CSIRO developed statistical model (BJP) Further testing on sites in all states and territories Extend to 200 sites by the mid

30 Flood & short-term forecasting Ensemble forecast of a flood event in the Stanley River 30 Geoff Fundamentals of Modelling Geoff Podger Podger Page 30

31 Short term streamflow forecasting Flow forecasts up to 10 days ahead Unregulated inflows to regulated systems Includes rainfall forecasts R&D conducted through CSIRO Ovens River pilot for registered users 31 Geoff Fundamentals of Modelling Geoff Podger Podger Page 31

32 Flexible hydrological modelling frameworks LAND USE CLIMATE ECOLOGICAL ASSETS DAMS & WEIRS IRRIGATION CITIES 32 Geoff Fundamentals of Modelling Geoff Podger Podger Page 32

33 Tamor daily rainfall runoff model Area: 4058 km 2 Elevation: m Glacier area: 13% Rainfall runoff model GR4J+snow+glacier 7 parameters 44 HRUs Bias 2% NSE 0.87

34 Indus River System Model Structure

35 Dam break modelling CSIRO 2 and 3d hydrodynamic models Geheyan Dam in China Different dam break scenarios Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 5 Scenario 6 35 Fundamentals of Modelling Geoff Podger

36 Urban water management 36 Geoff Fundamentals Podger Page 36 of Modelling Geoff Podger

37 Irrigation modelling Farm scale bio-physical models (e.g. APSIM) consider: Physical properties of different crops over growing period Soil type Water and heat stress Irrigation efficiency Fertiliser application Yield and production Regional scale crop models (e.g. Source) consider: Supply storages both regional and local Water access rules Multiple water sources (surface and groundwater) Distribution and return systems (irrigation districts) FAO 56 crop factors to drive current use and future demands Losses (channel, escapes, deep percolation) Crude yield and production estimates 37 Geoff Fundamentals Podger Page 37 of Modelling Geoff Podger

38 Floodplain modelling hydrologic vs 2d hydrodynamic a) Simulated inundation by LiDAR based approach b) Simulated inundation by 2D HD model 22 month event 10 minute run time 38 Geoff Fundamentals Podger Page 38 of Modelling Geoff Podger 55 day event 10 day run time

39 SW-GW interaction Establishing relationships in losing and gaining streams Preliminary results from Paroo Validating the relationships 0 Integrating with the river system model Ungauged Loss Function Adjusted Loss Function Revised annualised Ungauged Losses and Adjusted Losses (ML): Willara to Wanaaring. (Positive values are losses, negative values are gains) Raw Flow (Fit + UGL) Groundwater Loss Ungauged Loss Function Adjusted ULF (ULF - GWL) SW-GW connectivity map (MDBSY) 0 1/01/2000 1/04/2000 2/07/2000 1/10/2000 1/01/ Revised daily flow components for the year 2000 (ML/d), Willara to Wanaaring. River flow and GW loss are on left-axis, loss/gain functions on the right-axis. 39 Geoff Fundamentals Podger Page 39 of Modelling Geoff Podger

40 Physically based groundwater models 40 Geoff Fundamentals Podger Page 40 of Modelling Geoff Podger

41 Ecological Systems: Environmental Flow Modelling Understanding Ecosystem Complexity Species, habitats and refugia Predicting Ecological Outcomes FF_last_ten_yrs (Low Floodplain) one in one 0 one in two 100 one in five 0 one in ten 0 less one in ten 0 Time Since Last Wet (Low_Floodplain) current 0 up to 05y 0 btn 05 to 1 0 btn 1 to btn 2 to 5 0 btn 5 to 10 0 greater 10 0 Duration1 irrelevant 100 up to 1mo 0 up to 2mo 0 up to 4mo 0 up to 6mo 0 up to 12mo 0 Ecological Response Models Driver-Pressure-Stressor-Impact-Response Suitabiliy_FF1 Extreme Dry 0 Dry 0 Optimal 100 Wet 0 Suitabiliy_TSLW1 Extreme Dry 0 Dry 0 Optimal 100 Redgum_Forest Extreme Dry 0 Dry 0 Optimal 100 Wet 0 Suitabiliy_Duration1 Extreme Dry 0 Dry 0 Optimal 100 Wet 0 Integration: Models & Assessment Scenario-based tools Optimisation-based tools 41 Fundamentals of Modelling Geoff Podger

42 Predicting Ecological Outcomes: Models that are Fit for Purpose No Data, High uncertainty Some Data, Some Knowledge Lots of Data, Good Knowledge Data complete, System well defined Participation Conceptual, Hydrologic Alteration Expert system, Habitat models, Uncertainty analysis Population dynamics, Function dynamics Dynamic ecosystem models 42 Geoff Fundamentals of Modelling Geoff Podger Podger Page 42

43 Economic modelling Local scale (farm, hydro station) Regional scale (irrigation districts, countercyclical trade) National scale (computable general equilibrium) Trans-boundary scale (trade between countries) Pareto surface Price, $ Supply P e Demand Q e Quantity, GL 43 Geoff Fundamentals of Modelling Geoff Podger Podger Page 43

44 Getting confidence Applies to both modellers and stakeholders A model that is not agreed by all stakeholders is useless Reproducing history may be an important step Be careful not to overfit models Model uncertainty vs data uncertainty Ensuring the model is robust Model parameters are stable There is no equifinality Ensure parsimony: every parameter needs to prove its right to be there Calibration and validation Uncertainty and sensitivity 44 Fundamentals of Modelling Geoff Podger

45 Choosing an objective function Need to reflect what the model is going to be used for Flood models need to represent: Level/depth Extent High Flows Operation models Daily volume and flow Low to medium flows Planning models Overall mass balance Flux attribution Average flows 45 Fundamentals of Modelling Geoff Podger

46 Sensitivity and uncertainty Quantifying rating uncertainty Quantifying river model uncertainty Quantifying climate uncertainty Manilla Black Springs Quantifying groundwater model uncertainty BATEA Uncertainty analysis

47 Examples of custom interfaces Brahmani system interface: 16 scenarios + climate change Indus system Lower Murray operations support system 47 Fundamentals of Modelling Geoff Podger

48 Interpreting outputs B0H MIN B0M MIN B0D MIN 1 It is not just about the physical outputs 0.01 Flow Groundwater level Water quality Hydropower and irrigation production Reliability of supply It is not just about the environmental and economic outputs Wetland health Fish health Economic benefit for irrigation and power production It is about people The stories and narratives from their perspective B0H MED B0M MED B0D MED BOH MAX BOM MAX BOD MAX Fundamentals of Modelling Geoff Podger

49 Thank you Land and Water Basin Management Outcomes Geoff Podger Project Director SDIP t e geoff.podger@csiro.au w CSIRO LAND AND WATER FLAGSHIP

50 Problem Definition Check List Problem statement Objectives Problem domain System definition Conceptual model Metrics and criteria Decision variables Uncertainty and risk Preliminary assessment Have the issues to be addressed been discussed internally? Have the issues to be addressed been discussed with stakeholders? Is a model required? Have different modelling options been explored? Have objectives been agreed internally? Have objectives been agreed with stakeholders? Is there a clear objective hierarchy? Have the important disciplines been identified? Is there a plan for the community acceptance process? System components/modules identified? Is the existing system behaviour understood? At the various biophysical and socioeconomic interactions understood? Have temporal scale data constraints been considered? Have spatial scale data constraints been considered? Has data analysis and quality assurance been undertaken? Has prior knowledge been elicited and used? Has the relative importance of components been considered? Have functional relationships been considered? Are the model assumptions clearly stated? Do you have stakeholder agreement? Have alternatives been assessed and documented? Has model soundness been considered? Does the model have a track record? Does the conceptual model meet project objectives? Has model complexity and parsimony been considered with respect to data? Have been defined and agreed? Match the project objectives? Have local and international standards been considered? Analysis periods have been defined and agreed? Clearly defined and agreed? Was uncertainty considered in the conceptual model? Has uncertainty been considered for option modelling (will the model tell us anything useful)? Has risk assessment be considered? Are the expected model outcomes clearly defined? 50