December, A report to the Australian Government from the CSIRO South-West Western Australia Sustainable Yields Project

Transcription

1 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' South-west Western Australia Sustainable Yields Project: Technical Report. Silberstein, R.P., Aryal, S.K., Pearcey, M., Durrant, J., Braccia, M., Boniecka, L., McCallum, S., Smith, K., Bari, M.A., Hodgson, G.A. and McFarlane, D.M. December, 21 A report to the Australian Government from the CSIRO South-West Western Australia Sustainable Yields Project

2 South-West Western Australia Sustainable Yields Project acknowledgments The South-West Western Australia Sustainable Yields project was undertaken by CSIRO under the direction of the Australian Government Department of the Environment, Water, Heritage and the Arts. Important aspects of the work were undertaken by the Department of Water Western Australia. The Water Corporation and the Western Australia Department of Agriculture and Food provided expert advice and data. A contract to develop a groundwater model for part of the region was undertaken by URS Australia Pty Ltd. Additional technical input was provided by CyMod Systems, Jim Davies and Associates, Resource Economics Unit and Geographic Information Analysis. Valuable feedback was received during the review process from Tony Jakeman, Andy Pitman, Don Armstrong, Peter Davies and Murray Peel. South-West Western Australia Sustainable Yields Project disclaimers Derived from or contains data and/or software provided by the Organisations. The Organisations give no warranty in relation to the data and/or software they provided (including accuracy, reliability, completeness, currency or suitability) and accept no liability (including without limitation, liability in negligence) for any loss, damage or costs (including consequential damage) relating to any use or reliance on that data or software including any material derived from that data and software. Data must not be used for direct marketing or be used in breach of the privacy laws. Organisations include: Department of Water Western Australia, Bureau of Meteorology, Water Corporation and the Western Australia Department of Agriculture and Food. CSIRO advises that the information contained in this publication comprises general statements based on scientific research. The reader is advised and needs to be aware that such information may be incomplete or unable to be used in any specific situation. No reliance or actions must therefore be made on that information without seeking prior expert professional, scientific and technical advice. To the extent permitted by law, CSIRO (including its employees and consultants) excludes all liability to any person for any consequences, including but not limited to all losses, damages, costs, expenses and any other compensation, arising directly or indirectly from using this publication (in part or in whole) and any information or material contained in it. Data is assumed to be correct as received from the Organisations. Citation Silberstein, R.P., Aryal, S.K., Pearcey, M., Durrant, J., Braccia, M., Boniecka, L., McCallum, S., Smith, K., Bari, M.A., Hodgson, G.A. and McFarlane, D.M. (21). Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model'. CSIRO, Australia, pp Publication Details Published by CSIRO 21 all rights reserved. This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced by any process without prior written permission from CSIRO. ISSN X

3 Table of Contents Summary Background Models Used Streamflow data Rainfall and APET data Catchment runoff behaviour Geographical layout of catchments Model calibration Warm-up Parameter variability Selection of adopted model based on NSE Calibration results Model bias Performance of the adopted model Issues of stationarity Model uncertainty Data adequacy Comparison of projected change versus model uncertainty Model structural uncertainties and parameter selection Discussion References Tables Table 3-1. Number of gauged and ungauged catchments and grid cells in the basins and regions in the surface water modelling area Table 3-2. Percentage of catchments which meet different Nash-Sutcliffe efficiency criteria for all 31 combinations of the 5 models. The column labels indicate which combination of the five models were included in each combination... 5 Table 3-3. Percentage of catchments which meet different Nash-Sutcliffe efficiency criteria for different summation intervals Table 3-4 Percentage of catchments which meet model bias criteria Figures Figure 3-1. Locations of streamflow gauging stations and ungauged SRNs across the project area. The grey areas were not modelled Figure 3-2. Frequency distribution of the record completeness of the 16 gauged catchments Figure 3-3. Record completeness of the 16 gauged catchments showing presence of daily streamflow measurements Figure 3-4. Maps of catchment runoff coefficient, measured runoff and rainfall; and bias and Nash-Sutcliffe efficiency of the adopted model Figure 3-5. Distribution of observed runoff coefficient in the gauged catchments across the project area Figure 3-6. (a) Spread of observed mean annual runoff for all sites used in the model calibrations, and (b) scatter plot of runoff ratio and catchment area Figure 3-7. Distribution of (a) catchment inter-centroid distances and (b) closest neighbour inter-centroid distances Figure 3-8. Illustration of the effect of warm up period on each model for two selected catchments. The plots show (a) and (c) monthly hydrographs for the four calibration runs for two selected catchments, and (b) and (d) the difference between three shorter calibrations (commencing in 197, 1972 and 1974) and the baseline commencing in Figure 3-9. The spread of parameters for the Sacramento model in all 16 calibrated catchments in the project area. Parameters Pctim, Rserv, Side, and Ssout were kept constant for all calibrations Figure 3-1. The cumulative frequency distributions of parameters for the Sacramento model in all 16 calibrated catchments in the project surface water modelling area. Parameters Pctim, Rserv, Side, and Ssout were kept constant for all calibrations Figure The spread of parameters for the IhacresClassic model in all 16 calibrated catchments in the project area. Parameters P and Tref were kept constant for all calibrations Figure The cumulative frequency distributions of parameters for the IhacresClassic model in all 16 calibrated catchments in the project surface water modelling area. Parameters P and Tref were kept constant for all calibrations Figure Frequency distribution of Nash-Sutcliffe Efficiency for the 31 combinations of models Figure Frequency distribution of Nash-Sutcliffe Efficiency for the 5 models and the 'adopted model' Figure Frequency distribution of model bias (or net percentage error) for the 5 models and the 'Adopted model' Figure Nash-Sutcliffe Efficiency plotted against (a) catchment area and (b) latitude of catchment centroid Figure Daily Nash-Sutcliffe efficiencies in calibrated catchments from north-west to south-east for the adopted model results. The fitted linear regression given shows no linear trend. The numbers along the x-axis are the Australian Water Resources Council (AWRC) catchment numbers, of which the first three digits are the AWRC basin numbers CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' i

4 Figure Comparison between observed and adopted model annual runoff in representative calibration catchments whose AWRC numbers are shown as the title of each panel. The NSE listed is for the annual runoff series as plotted Figure Flow hydrograph of observed and modelled annual runoff (left column), and cumulative error (modelled less observed runoff right column) for a representative calibration catchment in each of the 13 surface water basins, plus 634 in the Hay River basin adjacent to the Denmark Figure 3-2. Relationship between monthly and annual Nash-Sutcliffe efficiencies for all 16 gauges for the baseline Scenario A, and the Change-Uncertainty (CU) for Scenario Cmid Figure 3-21 Relationship between monthly and annual change-uncertainty ratios for all 16 gauges under Scenario Cwet and Cdry Figure Distribution of change-uncertainty (a) for Cwet, Cmid and Cdry scenarios as cumulative frequency distributions, and (b) spatial distribution of CU_Cdry across the surface water modelling area. The dashed line in panel (a) indicates CU= ii Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

5 1 Silberstein, R.P., Aryal, S.K., Pearcey, M., Durrant, J., Braccia, M., Boniecka, L., McCallum, S., Smith, K., Bari, M.A., Hodgson, G.A. and McFarlane, D.M. This chapter describes the selection and performance of the adopted model used to project streamflow at the selected Streamflow Reporting Nodes (SRNs) within the surface water basins in the surface water modelling area for the South- West Western Australia Sustainable Yields project (CSIRO, 29). 3.1 Summary 16 gauged catchments were used in model calibration using streamflow data from within the period 1975 to 27. The calibrated catchments comprise 67 percent of the area of the surface water basins in the project area, are distributed well over the surface water modelling area, and account for a little more than 67 percent of the total flow. Although observed rainfall and simulated areal potential evaporation (APET) data were available for the full 33 years, the streamflow data length ranged from 1 years upwards. Mean annual runoff in the calibration catchments ranged from 5 to 348 mm, with a median value of 112 mm. Five rainfall runoff models were calibrated for the gauged catchments. Calibration results from weighted means of all 31 linear additive combinations of the five models were examined for their ability to reproduce the measured streamflows. The mean of the daily runoff from the Sacramento and IHACRES models consistently resulted in a better model fit than any other combination of the individual models and this adopted model was used for scenario modelling. For both the selected models values of several of the optimised parameters occurred at or very close to one or other of the allowed range boundaries of the parameters, suggesting that the optimiser range may have been too constrictive to allow a global optimum to be reached. This suggests a limitation of the implementation of the optimisers that could be improved in future developments. In some of the calibration catchments the models over estimate runoff in the last 5 to 1 years of record, and under estimate in the first 1 years of the record. Our interpretation of this is that it is due to the changing groundwater status in these catchments resulting from the recent drying climate not being well represented in the current models. However, it could also be due to increasing vegetation cover in catchments subject to large scale plantation development in the last decade, or to changes in rainfall characteristics and seasonal distribution not adequately represented in the previous decades. Whatever the cause, the implication is that our modelling may over estimate projected streamflows. 3.2 Background The SWSY rainfall-runoff simulations were carried out on 24 fresh and brackish water catchments in the south-west of Western Australia. Model calibration was undertaken for 16 of these catchments for which streamflow data were available. Locations of the streamflow gauging stations used are depicted in Figure 3-1 showing a reasonable distribution of stations across the surface water modelling area. The areas indicated in grey in Figure 3-1 were not included in the rainfall-runoff modelling because they had no harvestable surface flows. Chapter 6 gives the calibration results from all the gauged catchments. 1 Citation: Silberstein, R.P., Aryal, S.K., Pearcey, M., Durrant, J., Braccia, M., Boniecka, L., McCallum, S., Smith, K., Bari, M.A., Hodgson, G.A. and McFarlane, D.M. (21). Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model'. CSIRO, Australia, pp , CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 35

6 The surface water modelling area was different from the whole project surface water basin area as some parts of the basins were excluded from the modelling. These were the coastal areas without definable catchment boundaries which either discharge directly to the sea or consist of sand dunes that do not produce any surface runoff, or are areas draining directly into the lower Avon River or the Swan River estuary which are affected by urban or industrial runoff or are otherwise impractical to harvest. Figure 3-1. Locations of streamflow gauging stations and ungauged SRNs across the project area. The grey areas were not modelled Models Used Five lumped, conceptual rainfall-runoff models (Sacramento, IHACRES, AWBM, SIMHYD and SMARG) were calibrated using SILO Data Drill climate data from 1975 to 27 (< Jeffrey et al., 21). All five conceptual models require only daily rainfall and daily potential evaporation as input data. Further details for these models are given in Chapter 4 of the accompanying Surface Water Main Report (CSIRO, 29). The models were applied to the catchments represented as a matrix of cells on a.5º longitude x.5º latitude (approximately 5 x 5 km) grid across the surface water basins in the project area (See Chapter 2 of this report, Silberstein et al., 21). 36 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

7 3.2.2 Streamflow data Streamflow data from 16 streamflow gauging stations in the surface water modelling area were used for calibrating the models. Data for all but two of the streams were obtained from the Department of Water Western Australia b with the other two supplied by the Western Australian Water Corporation. The duration of the streamflow data records ranged from 1 to 56 years with a median of 3.5 years, although data from only the 33 years between 1975 and 27 were used in model calibration. Figure 3-2 shows the frequency distribution of completeness (in percent) of the recorded streamflow data in the 16 gauged catchments while Figure 3-3 shows the completeness record of each of the gauges. 1% 9% Data record completeness 8% 7% 6% 5% 4% 3% 2% 1% % Number of stations Figure 3-2. Frequency distribution of the record completeness of the 16 gauged catchments Rainfall and APET data The observed daily rainfall and calculated areal potential evaporation (APET) data for the period rainfall were available. These data were obtained for a.5º longitude x.5º latitude grid from the Queensland Government Environmental Protection Agency SILO gridded data (Data Drill) derived from selected stations (< Jeffrey et al., 21). Figure 3-4 shows the mean annual rainfall and mean annual APET across the surface water modelling area. CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 37

8 Figure 3-3. Record completeness of the 16 gauged catchments showing presence of daily streamflow measurements 38 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

9 Figure 3-3. ctd. Record completeness of the 16 gauged catchments showing presence of daily streamflow measurements Catchment runoff behaviour Runoff characteristics are not uniform across the surface water modelling area (Figure 3-4) distribution. The runoff coefficient in the project area varies from 1-2 percent to more than 3 percent (Figure 3-5). Figure 3-6 shows the spread of observed mean annual runoff for all sites used in the model calibrations, and (b) scatter plot of runoff ratio and catchment area. It shows the frequency distribution of the observed annual runoff and runoff coefficients for all 16 CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 39

10 catchments used in the model calibrations. The distribution is slightly skewed towards larger values probably caused by the wetter climate in the south of the surface water modelling area. For example, 75 percent of the runoff values are less than 176 mm (median 112 mm) with the top 1 percent of values being greater than 261 mm. There is no obvious relationship between runoff coefficient and catchment area (Figure 3-6b). 4 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

11 Figure 3-4. Maps of catchment runoff coefficient, measured runoff and rainfall; and bias and Nash-Sutcliffe efficiency of the adopted model CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 41

12 Runoff coefficient (a) mm Percentage of catchments Figure 3-5. Distribution of observed runoff coefficient in the gauged catchments across the project area (b).4 Runoff coefficient Area (km 2 ) Figure 3-6. (a) Spread of observed mean annual runoff for all sites used in the model calibrations, and (b) scatter plot of runoff ratio and catchment area Geographical layout of catchments Figure 3-7 gives the distributions of catchment inter-centroid distances and closest catchment inter-centroid distances. About 7 percent of catchments are less than 1 km apart. The closest catchments were 632 and 63136, in the Denmark basin, which had 2.1 km between their centroids, and the closest gauges were and in the Harvey basin, separated by 1.9 km. The greatest separation between catchment centroids was 448 km between catchment 6312 and 6173 km. The distribution of closest neighbour distances shows that the most distant nearest neighbours were separated by 36.5 km, where gauge 681 was the closest donor catchment to Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

13 a) b) 5 4 Distance betewen centroids (km) Percentage of catchments Distance to closest neighbour (km) Percentage of catchments Figure 3-7. Distribution of (a) catchment inter-centroid distances and (b) closest neighbour inter-centroid distances 3.3 Model calibration Five conceptual models (AWBM, IHACRES, Sacramento, SIMHYD and SMARG) and LUCICAT were calibrated using observed streamflow for 16 gauged catchments and rainfall and APET data from 1975 to 27 (Figure 3-1). The calibrated catchments make up 67 percent of the surface water modelling area, are distributed reasonably well over the surface water modelling area, and account for more than 67 percent of the runoff. Table 3-1 shows the number of catchments and grid cells in each region. Table 3-1. Number of gauged and ungauged catchments and grid cells in the basins and regions in the surface water modelling area Number of grid cells Gauged Ungauged Total Northern (Gingin to Murray) region Gingin Swan Coastal Murray Sub-total Central (Harvey to Preston) region Harvey Collie Preston Sub-total Southern (Busselton Coast to Denmark) region Busselton Coast Lower Blackwood Donnelly Warren Shannon Kent Denmark Sub-total Surface water modelling area Total CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 43

14 The calibration runs used the Catchment Yield Modelling Framework developed by CSIRO Land and Water. Three optimisation algorithms were used to maximise the objective function based on their Nash-Sutcliffe efficiency (NSE) and a constraint to facilitate the difference between observed and modelled flows to be within 5 percent. The LUCICAT model was applied only to selected catchments and was calibrated manually. The LUCICAT results are presented in Chapter 5 and Appendix B of this report Warm-up The models were run with 1-, 3-, 5- and 15-year warm-up periods to check the modelling timesteps required before initial conditions ceased to have an impact on the simulated runoff. It was found that a warm-up period of more than 5 years was required to achieve this condition (Figure 3-8), and hence to be sure of unaffected simulations a 15-year warm-up period was adopted for all calibration runs. Thus all calibration runs were started in 196, 15 years ahead of the start of the 1975 to 27 calibration record. All scenario simulation runs were started in 1975 so that from the end of 27, being the beginning of the model output period, scenario runoff results were unaffected by the initial conditions. 44 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

15 (a) Monthly runoff (mm) Jan-6 Jun-65 Dec-7 Jun-76 Nov-81 May-87 Nov-92 May-98 Oct-3 (b) Monthly runoff difference (mm). (c) Monthly runoff (mm). Jan-7 Jun-75 Dec-8 Jun-86 Nov-91 May-97 Nov Jan-6 Jun-65 Dec-7 Jun-76 Nov-81 May-87 Nov-92 May-98 Oct-3 (d) Monthly runoff difference (mm). Jan-7 Jun-75 Dec-8 Jun-86 Nov-91 May-97 Nov Figure 3-8. Illustration of the effect of warm up period on each model for two selected catchments. The plots show (a) and (c) monthly hydrographs for the four calibration runs for two selected catchments, and (b) and (d) the difference between three shorter calibrations (commencing in 197, 1972 and 1974) and the baseline commencing in 196 CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 45

16 3.3.2 Parameter variability The optimised model parameter values were evaluated for their range and variability in the 16 calibrated catchments. For the Sacramento model, four of the 18 parameter values were kept constant and a further three parameters were found to vary across a narrow range. The rest of the parameter values showed a wide spread within the prescribed range with a few outliers (Figure 3-9 and Figure 3-1). With more than one-third of the parameters constant or nearly constant, the degrees of freedom in fitting the model were effectively somewhat fewer than the full 18 in the 18-parameter model. It is clear that for some parameters there is a wide spread of values across the catchments, while for others a significant number of catchments share values. For example, Sacramento parameter Rexp, restricted to the range 1 to 3, has a value of exactly 3 in 17 of the 16 catchments, very close to 3 (above 2.95) in another 17, and the value 1 in eight catchments, and less than 1.5 in another nine. The parameter Lzfpm, allowed the range to 1, has value in no catchments, but 1 in 17 catchments and greater than in another Adimp Lzfpm Lzfsm Lzpk Lzsk Lztw m Pctim Pf ree Rexp Rserv Sarva Side Ssout UH1 Uzfw m Uzk Uztw m Zperc Figure 3-9. The spread of parameters for the Sacramento model in all 16 calibrated catchments in the project area. Parameters Pctim, Rserv, Side, and Ssout were kept constant for all calibrations 46 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

17 .7 Adimp 12 Lzfpm 6 Lzfsm.6 Lzpk Lzsk 7 Lztwm.6 Pfree 3.5 Rexp Sarva 5 1 Uztwm UH1 5 1 Zperc Uzfwm Uzk Figure 3-1. The cumulative frequency distributions of parameters for the Sacramento model in all 16 calibrated catchments in the project surface water modelling area. Parameters Pctim, Rserv, Side, and Ssout were kept constant for all calibrations For IHACRES, two of the nine parameters were kept constant and the rest show a reasonable range (Figure 3-11 and Figure 3-12). Similarly to the Sacramento model, the degrees of freedom for IHACRES were fewer than that apparent for its nine parameters. Parameter T q, constrained between and 5, had the value 5 for six catchments and very close (>4.95) for another 19 catchments, and T s, constrained between 5 and 5, had exactly 5 in 3 catchments and greater than 499 in another 24. These analyses suggest that in some cases the parameter range given to the optimisers may have been too restrictive for these catchment conditions, and if a broader range had been allowed optimum parameter sets may have converged in other parts of the parameter space. The analyses also invite an exploration of the links between parameter values and catchment and climatic characteristics, and also suggest that a greater range of allowed values should be available to the optimisers. This could be coded into improved implementations of them in future. CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 47

18 F InverseC L P Tq Tref Ts Tw Vs Figure The spread of parameters for the IhacresClassic model in all 16 calibrated catchments in the project area. Parameters P and Tref were kept constant for all calibrations 48 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

19 16 F 3 InverseC 35 L 6 Tq Ts Tw Vs Figure The cumulative frequency distributions of parameters for the IhacresClassic model in all 16 calibrated catchments in the project surface water modelling area. Parameters P and Tref were kept constant for all calibrations 3.4 Selection of adopted model based on NSE There are many permutations that could be made from a linear combination of the models used in this project. We assessed the results of each of the five models in calibration and then the average daily flows of the 26 possible additive combinations of these models, thus the combination model flow sequence is the daily mean flow of the (two, three, four or five) sequences in the combination. This section describes the results of all 31 permutations Calibration results Overall the calibration results for the Sacramento model were slightly better than the results for the other rainfall-runoff models, possibly because there are more parameters in the Sacramento model. On the other hand SMARG was consistently the worst performing model with annual NSE values of less than.6 in 27 percent of catchments and monthly NSE values less than.8 in 36 percent of catchments. The SMARG, SIMHYD and AWBM models all had daily NSE values of less than.8 in at least 65 percent of catchments. The ability of the models to estimate runoff across the whole region was also a consideration. SMARG and SIMHYD could not adequately reproduce the observed runoff series in some catchments of the northern region. Daily runoff series were compiled from all 31 combinations of all models. These were derived by taking the average of each daily runoff value of the models in the combination. The total of 31 combinations comes from the five single models, 1 combinations of two models, 1 combinations of three models, five combinations of four models and one combination of all five models. The NSE values calculated for daily series of all 31 combinations are given in Table 3-2 and the distribution of NSEs of these combinations in Figure From these data it is clear that combination 3 & 5, which is Sacramento and IHACRES, gave the best NSE result, with 81% of catchments having an NSE above.8. It was on this basis we selected this combination as our adopted model. CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 49

20 Table 3-2. Percentage of catchments which meet different Nash-Sutcliffe efficiency criteria for all 31 combinations of the 5 models. The Nash- Sutcliffe column labels indicate which combination of the five models were included in each combination AWBM SIMHYD Sacramento SMARG IHACRES NSE>.8 24% 34% 74% 32% 54%.6<NSE.8 73% 48% 24% 49% 45%.4<NSE.6 4% 11% % 11% 1% <NSE.4 % 4% 2% 2% % NSE % 3% 1% 6% % 1 & 2 1 & 3 1 & 4 1 & 5 2 & 3 2 & 4 2 & 5 3 & 4 3 & 5 4 & 5 NSE>.8 51% 69% 49% 54% 7% 51% 55% 65% 81% 6%.6<NSE.8 45% 3% 45% 45% 28% 37% 44% 3% 18% 36%.4<NSE.6 3% 1% 2% 1% 1% 8% % 2% 1% 1% <NSE.4 % % 3% % % 3% % 1% % 1% NSE 1% % 1% % 1% 2% 1% 2% % 2% NSE>.8 7% 52% 61% 68% 7% 62% 66% 76% 59% 75%.6<NSE.8 29% 42% 38% 28% 3% 34% 28% 23% 37% 22%.4<NSE.6 % 5% 1% 3% % 3% 5% % 3% 3% <NSE.4 1% % % % % % % 1% % % NSE % 1% % 1% % 1% 1% % 1% 1% NSE>.8 67% 71% 61% 76% 73% 71%.6<NSE.8 31% 28% 36% 23% 26% 28%.4<NSE.6 1% 1% 2% % % % <NSE.4 % % % % % % NSE 1% % 1% 1% 1% 1% There is a clear spread in the distribution of model efficiencies visible in Figure 3-13, with some combinations, indeed some individual models, performing relatively poorly, but most combinations had NSE above.8 for over 6% of catchments. The selected model combination (designated 35 in the figure) is clearly better than all other combinations for the vast majority of catchments. The distribution of NSE for the five individual models and the adopted model combination is shown separately for greater clarity in Figure 3-14, which shows not only the better performance of the model but also that Sacramento was the best of the individual models for the majority of catchments. 5 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

21 1.9.8 Nash-Sutcliffe efficiency (AWBM) 2 (SIMHYD) 3 (Sacramento) 4 (SMARG) 5 (IHACRES) Figure Frequency distribution of Nash-Sutcliffe Efficiency for the 31 combinations of models Nash Sutcliffe model efficiency IhacresClassic Sacramento AWBM SimHydWithRouting SMARG Adopted model Number of stations Figure Frequency distribution of Nash-Sutcliffe Efficiency for the 5 models and the 'adopted model' The NSE values were also calculated for daily, monthly and annual runoff calibration periods for the individual models and the combination selected as the adopted model and shown in Table 3-3. The results indicate that of the five rainfallrunoff models, Sacramento and IHACRES performed the best. CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 51

22 Table 3-3. Percentage of catchments which meet different Nash-Sutcliffe efficiency criteria for different summation intervals Nash-Sutcliffe efficiency criteria Annual SMARG SIMHYD AWBM IHACRES Sacramento Adopted NSE>.8 24% 44% 47% 69% 73% 75%.6<NSE.8 49% 38% 41% 25% 18% 15%.4<NSE.6 13% 1% 8% 1% 4% 4% <NSE.4 6% 5% 5% 4% 4% 4% NSE 8% 3% % 1% 2% 2% Monthly NSE>.8 65% 76% 86% 97% 95% 96%.6<NSE.8 21% 13% 14% 3% 2% 3%.4<NSE.6 8% 7% % % % % <NSE.4 1% 1% % % 2% 1% NSE 6% 3% % % 1% % Daily NSE>.8 32% 34% 24% 54% 74% 81%.6<NSE.8 49% 48% 73% 45% 24% 18%.4<NSE.6 11% 11% 4% 1% % 1% <NSE.4 2% 4% % % 2% % NSE 6% 3% % % 1% % Using the same models across the whole region is likely to ensure that any differences obtained were due primarily to differences in climate and development scenarios and less to the models or calibration methods used. Therefore even if a model performed well in a catchment but not well overall it was not adopted. Although the structure and implementation of LUCICAT differed from the five conceptual models and its performance may not be directly comparable, its overall performance based on NSE was less than both Sacramento and IHACRES. Thus LUCICAT was excluded from the scenario modelling, but its performance is examined in Chapter 5 of this report Model bias Model bias is the total difference between the simulated daily runoff and the observed daily runoff, expressed as a fraction of the total observed runoff. Table 3-4 gives the proportional bias for each individual model and the adopted model, and Figure 3-15 shows the distribution of bias from each model and the adopted model across all 16 calibration catchments. Clearly, some models showed quite undesirable bias (Figure 3-15), most particularly SMARG in a few catchments in which a good calibration could not be reached. It is interesting that both SIMHYD and SMARG showed a positive bias (5 percent of mean annual flow) in over 7 percent of gauged catchments. Our preliminary interpretation of this is that these models did not cope well with the ephemeral nature of streams in SWWA, being more usually applied in perennial catchments. Both Sacramento and IHACRES showed the best (most balanced, and least net) bias, although AWBM was pretty good also. 52 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

23 AWBM Sacramento IHACRES SIMHYD SMARG Adopted model Model bias Number of stations.1 Figure Frequency distribution of model bias (or net percentage error) for the 5 models and the 'Adopted model' Table 3-4 Percentage of catchments which meet model bias criteria Model bias SMARG SIMHYD AWBM IHACRES Sacramento Adopted Percent of catchments Daily -.5<Bias -.2 5% 6% 24% 24% 47% 33% -.2<Bias.2 8% 1% 43% 67% 43% 64%.2<Bias.5 8% 83% 33% 8% 1% 4% >.5 7% 1% % % % % Performance of the adopted model The range of daily NSEs for the adopted model for the calibration results was.93 to.63, with a mean of.84. The adopted model performance is better in almost all cases than any of the individual models because the averaging improves the prediction from two models individually under estimating and over estimating the observed runoff. Although the forcing climate data varied from north to south and the catchments undoubtedly have different characteristics across the surface water modelling area, the selected model gave a better calibration than other model combinations across all regions. This, together with the high model calibration efficiencies obtained, gives confidence in the robustness of the models for use in the climate range used in the calibration. There was no apparent dependence of model efficiency on any catchment characteristic we tested. The relationship against catchment area and latitude are shown in Figure CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 53

24 (a) (b) NSE Catchment area (km2) NSE Latitude ( o S) Figure Nash-Sutcliffe Efficiency plotted against (a) catchment area and (b) latitude of catchment centroid Despite the strong gradients in rainfall and APET across the surface water modelling area, the adopted model s performance was consistent across the regions. Figure 3-17 shows the Nash-Sutcliffe efficiency for annual flows from the adopted model for 16 calibration catchments, plotted from north-west (Australian Water Resources Council basin 617) to south-east (Australian Water Resources Council basin 63). The fitted linear regression shows no significant trend implying no geographic trend in the distribution of Nash-Sutcliffe efficiency across the surface water modelling area. As the x-axis shows the continuous increment of catchment number from right to left, the data points next to each other are most likely from similar areas. Nash-Sutcliffe Efficiency (-) Catchments from north-w est to south-east Figure Daily Nash-Sutcliffe efficiencies in calibrated catchments from north-west to south-east for the adopted model results. The fitted linear regression given shows no linear trend. The numbers along the x-axis are the Australian Water Resources Council (AWRC) catchment numbers, of which the first three digits are the AWRC basin numbers. Figure 3-18 shows the comparison between observed and simulated annual runoff from the adopted Sacramento- IHACRES model in a representative calibration catchment from each of the basins of the surface water modelling area, plus an extra catchment (Hay River) from the Denmark basin. It shows that there is good agreement between observed and adopted model flows over a wide range of values for all 13 basins. The gauges chosen for this comparison are those with the largest catchment area in each of the basins, which were selected as therefore being the most representative on each basin. However, similar results were obtained from almost all 16 gauges. 54 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

25 6 NSE = NSE = NSE = NSE = Modelled annual runoff (mm) Observed annual runoff (mm) Modelled annual runoff (mm) Observed annual runoff (mm) Modelled annual runoff (mm) Observed annual runoff (mm) Modelled annual runoff (mm) Observed annual runoff (mm) 15 NSE = NSE = NSE = NSE = Modelled annual runoff (mm) Modelled annual runoff (mm) Modelled annual runoff (mm) Observed annual runoff (mm) NSE = Observed annual runoff (mm) NSE = Observed annual runoff (mm) Modelled annual runoff (mm) Modelled annual runoff (mm) Modelled annual runoff (mm) Observed annual runoff (mm) NSE = Observed annual runoff (mm) NSE = Observed annual runoff (mm) Modelled annual runoff (mm) Modelled annual runoff (mm) Observed annual runoff (mm) NSE = Observed annual runoff (mm) Modelled annual runoff (mm) Modelled annual runoff (mm) Observed annual runoff (mm) NSE = Observed annual runoff (mm) Figure Comparison between observed and adopted model annual runoff in representative calibration catchments whose AWRC numbers are shown as the title of each panel. The NSE listed is for the annual runoff series as plotted Issues of stationarity This project used long climate data series to estimate parameters through calibration on gauged runoff,, and then estimated runoff under future climate projections. It is implicitly assumed in doing this that the physical characteristics of the catchment and the hydrological and climate conditions remain similar during the scenario period (Gupta et al., 22). The rainfall change experienced in the project area over recent years raises the possibility that the climate data series are non-stationary and the runoff generation processes within the catchments may be non-stationary. Non-stationarity in this context, if it appears to occur, is really a reflection that the models used may be insufficient and not adequately represent physical processes in the catchment. The physics of runoff generation, after all, has not changed, although the balance of different processes and the proportional contribution to runoff have may have. The suitability of parameters determined using past climate sequences for future modelling may, therefore, be questioned. Since the future climate series used for scenario modelling were derived by scaling historical data, any trends in the original series are repeated in the projected future series. The approach undertaken in this project is that results of future scenarios are based on comparisons of future projections against a base case (Scenario A) using the historical climate record, not the historical CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 55

26 runoff record itself. Thus if the data trend is assumed to be the same in the future projections and the historical base case, then by reporting the difference between future scenarios and Scenario A any error due to these non-stationarities is minimised. In a number of the calibration catchments the models over estimate runoff in the last 5 to 1 years of record, and under estimate in the 1 years prior to that. An example catchment in each of the 13 surface water basins, plus one in the Hay River catchment adjacent to the Denmark basin are shown in Figure The annual hydrographs of observed record and the modelled runoff are shown with the cumulative error shown on the right. In three of the examples given the cumulative error (modelled less observed runoff) show the curve trending downward till about 1992 and then upward to the end of the period. The downward trend indicates a consistent under-estimation of runoff during that period and the upward trend an over-estimation. The systematic nature of these opposing trends indicates a change of status in the catchments around the early to mid 199s not being well represented in the current models. This may be due to the changing moisture storage and declining groundwater, to changes in the nature of rainfall, or to changes in vegetation water balance in these catchments. The implication is that our modelling may over estimate projected streamflows if the underestimation in recent years continues. The implications for this project are not major as the under estimation is likely to be similar in all scenarios, and results and interpretations are made on the basis of comparisons of runoff under scenarios B, C and D against that under Scenario A. 56 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

27 Annual runoff (mm) Observed 8 Modelled Cumulative error (mm) Observed 8 Modelled Annual runoff (mm). Annual runoff (mm) Observed 8 Modelled Cumulative error (mm)... Cumulative error (mm) Annual runoff (mm) Observed 4 Modelled Cumulative error (mm) Figure Flow hydrograph of observed and modelled annual runoff (left column), and cumulative error (modelled less observed runoff right column) for a representative calibration catchment in each of the 13 surface water basins, plus 634 in the Hay River basin adjacent to the Denmark CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 57

28 Annual runoff (mm) Observed Modelled Cumulative error (mm) Annual runoff (mm). Annual runoff (mm) Observed 2 1 Modelled Observed 2 1 Modelled Cumulative error (mm).. Cumulative error (mm) Annual runoff (mm) Observed Modelled Cumulative error (mm) Figure ctd. Flow hydrograph of observed and modelled annual runoff (left column), and cumulative error (modelled less observed runoff right column) for a representative calibration catchment in each of the 13 surface water basins, plus 634 in the Hay River basin adjacent to the Denmark 58 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

29 Annual runoff (mm) Observed Modelled Cumulative error (mm) Annual runoff (mm). Annual runoff (mm) Observed 15 1 Modelled Observed 4 Modelled Cumulative error (mm)... Cumulative error (mm) Annual runoff (mm) Observed Modelled Cumulative error (mm) Figure ctd. Flow hydrograph of observed and modelled annual runoff (left column), and cumulative error (modelled less observed runoff right column) for a representative calibration catchment in each of the 13 surface water basins, plus 634 in the Hay River basin adjacent to the Denmark CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 59

30 Annual runoff (mm) Observed 15 1 Modelled Cumulative error (mm) Annual runoff (mm) Observed Modelled Figure ctd. Flow hydrograph of observed and modelled annual runoff (left column), and cumulative error (modelled less observed runoff right column) for a representative calibration catchment in each of the 13 surface water basins, plus 634 in the Hay River basin adjacent to the Denmark 3.5 Model uncertainty Uncertainties in modelling are related to data and model structure and are categorised as external and internal uncertainties respectively. Some uncertainties are difficult to account for. For example, the set of parameter values used to model runoff across the project area for scenarios A, B and C using 33 years of daily climate inputs does not take into account the effect of global warming and enhanced CO 2 concentrations on forest water use or other catchment processes. This effect could be significant, but it is difficult to estimate because of potentially offsetting positive and negative impacts and complex climate-biosphere-atmosphere interactions. Bushfire risks are also likely to change under the future climate. In areas where bushfires occur, runoff could reduce significantly as forests regrow. However, the impact on runoff averaged over an entire region is unlikely to be significant. Cumulative error (mm) Catchment model uncertainty can be assessed based on: 1. the adequacy of the water resource observation network supplying the input calibration data 2. an assessment of model performance, based on model reports and a comparison of modelled water balance components with a set of best estimate streamflows for 1975 to a comparison of projected change versus model uncertainty (the change-uncertainty ratio) 4. model structural uncertainties, including the manner in which key processes are described and the possibility of model surprises 5. appropriateness of regionalisation of model parameters, including assessment during the calibration phase on the test catchments. 6 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

31 3.5.1 Data adequacy Although the observations used to produce the rainfall and APET gridded data have been quality checked by the Bureau of Meteorology and have been subject to error checking by the Queensland Climate Change Centre of Excellence, it is inevitable that errors will remain. As the point observations of rainfall are highly discontinuous in space and time, interpolation of these data using a tri-variate thin plate spline has a potential to introduce data errors in spatial and temporal dimensions (Charles et al., 21). The effect of these potential data errors have not been investigated in the project. In general, the data accuracy is expected to be higher in areas where the observation density is high relative to the climate gradients. Thus the adequacy of data also varies from location to location as discussed by Charles et al.(21). As presented earlier, the gauge network in SWWA is comprehensive, with a good coverage across the project surface water modelling area (Figure 3-1). Although many gauges have been closed in the last decade, it is considered the data network is adequate for our purposes. This is also borne out by the excellent calibration results presented earlier, with uniformly high Nash-Sutcliffe efficiencies across the modelling domain Comparison of projected change versus model uncertainty The uncertainty associated with the observation network and model performance must be interpreted in the light of the predicted change. A small change makes it more likely that the uncertainty in predictions is too great for the model result to be used in decision making. The robustness of the predicted change can be evaluated by comparing it to the mismatch between the baseline model and the observations. Streamflow projections may be compared against the observed series and against the baseline case, and if these explain the observed flows as well, or better, than the calibrated model runs then it is concluded that these scenarios are within the natural variability of the data and/or the model. In this case, the scenarios are within the model noise and are not significantly different from the undisturbed system, and our conclusion should be that there is no discernible change. In the case where the scenario displays markedly different behaviour to the observed system and the calibrated model of this system, then the conclusion would be that the change induced by the scenario is greater than the system noise and it is therefore a legitimate interpretation that the change is due to the scenario. This degree of noisiness, or signal to noise ratio (Bormann, 25) can be quantified in a number of ways, similar to that for model efficiency using the Nash-Sutcliffe statistic (NSE) (Nash and Sutcliffe, 197). We calculate the changeuncertainty (CU) as a ratio which indicates how much greater the projected change is than the uncertainty in model performance as the ratio of the NSE for the projected model result under the scenarios to that for the calibration, or the baseline scenario (Scenario A). CU is expressed as: Q Q 1 NSE CU Q 2 O X X 2 1 O Q NSE A A Where Q is the flow, the subscript O refers to the observed floe sequence, and the subscript X refers to the B, C or D scenario, and NSE X is the Nash-Sutcliffe Efficiency calculated for the scenario X flow sequence relative to the observed sequence. Mathematically, the equation above compares the additional unexplained variation in the scenario model with that in the baseline model. In the extreme case, the scenario model may describe observations better (that is, the NSME is higher) than the baseline model, which necessarily affects the confidence that can be put in the models adequacy in simulating observations. There are three value sets of interest for CU (Van Dijk et al., 28). CU<<1 When the CU is less than 1, the scenario is a better representation of the observed data than the baseline model. This should not be possible, if the original calibration is the best possible, but may be true if the baseline calibration has a systematic bias that is removed by the trend in a scenario. For example, if the baseline overestimates the runoff, and a drier scenario removes this bias. CU ~ 1 When CUR is around 1 then the projected scenario is not significantly different from the baseline and hence is within model and observation noise and there is no significant change due to the scenario. CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 61

32 CU >>1 If CUR is greater than 1 the conclusion that the scenario has a meaningful impact on the system is justified, and in this case a changed runoff regime under the climate scenarios. The further above 1 the CU, the more significant is the result. It should be noted that this analysis and this statistic as a measure of confidence in projections of changed runoff characteristics depend on the fact that we have used transformed data sequences as input to the models, so the individual scenarios are directly comparable. This is not the case of Scenario B which, apart from the last 11 years of the scenario, is completely out of phase with the baseline Scenario A historical sequence. Nor would it be the case if stochastically generated sequences were to be used for the future climate projection scenarios, as once again there would be no synchronous relationship between the sequences being compared. The determination of CU, as with NSE, can be made on the basis of any time interval; in our case daily, monthly or annual time intervals are the obvious candidates. First we test whether there is any difference between CU at monthly and annual intervals (Figure 3-2). Monthly NSE NSE A Annual NSE Monthly CU 1 CU_Cmid 1:1 line Annual CU Figure 3-2. Relationship between monthly and annual Nash-Sutcliffe efficiencies for all 16 gauges for the baseline Scenario A, and the Change-Uncertainty (CU) for Scenario Cmid What we see is that the monthly NSE are generally much higher than the annual values, indicating that the seasonal distribution of flow is being well represented by the adopted model. It was stated above that if CU is less than 1 the modelled scenario is doing a better job of simulating the observed sequence than the calibrated model. This is not shown for any of the Cmid scenario runs (Figure 3-2), and was present in only a few Cwet runs (Figure 3-21) which, as can be seen in the main report (CSIRO, 29), are generally not significantly different from the baseline Scenario A. This analysis has been undertaken on annual and monthly runoff totals to test whether there was a significant difference in seasonal behaviour of the models. As is seen in Figure 3-21 there was no significant difference. It is also clear that our scenario results for Cmid and Cdry show CU well above 1 for almost all cases (Figure 3-22), indicating that the projected change in flow regime is most likely due to the climate scenarios and is not just model noise or observational uncertainty. The CU for Cwet is only slightly above 1 for the majority of cases indicating that this scenario is only having a minor influence on streamflow in those cases, and is above 1.5 for 14 catchments. Furthermore, from Figure 3-22b we see that the change-uncertainty ratio is higher in the northern part of the surface water modelling area indicating greater confidence in the projections of changed runoff here. This is commensurate with the greater decline in runoff projected for these catchments. 62 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

33 1 CU_Cwet 1:1 line 1 CU_Cdry 1:1 line 1 1 Monthly CU Monthly CU Annual CU.1 Annual CU Figure 3-21 Relationship between monthly and annual change-uncertainty ratios for all 16 gauges under Scenario Cwet and Cdry (a) 1 CU monthly 1 1 CU_Cwet CU_Cmid CU_Cdry (b) Latitude Longitude CU_Cdry<2 2<CU_dry<5 5<CU_dry<1 1<CU_dry Figure Distribution of change-uncertainty (a) for Cwet, Cmid and Cdry scenarios as cumulative frequency distributions, and (b) spatial distribution of CU_Cdry across the surface water modelling area. The dashed line in panel (a) indicates CU= Model structural uncertainties and parameter selection The performance of a catchment model in explaining historical water resource observations does not provide a full picture of its likely ability to predict future water resource patterns given assumed scenarios. This has several reasons, all of which are virtually impossible to quantify. In this project we assessed the model process complexity from the point of view of ensuring the models were fit for purpose and only as complicated as necessary while as simple as possible. There was always the likelihood that the real system response to the scenarios would be very different from the model because an as yet unknown but potentially important aspect of behaviour is inadequately captured. This is more likely to occur in complex, poorly measured systems than in simple systems. An important uncertainty relates to unknown aspects of vegetation response to climate change and attending changes (CO 2, bush fires), and its impact on water resources. It is virtually impossible to assess these uncertainties within the constraints of the project, primarily because it would require much more detailed data on processes and catchment development. Some progress could also be made with a more detailed model. CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 63

34 It is also possible that the climate projections explored in this project take the catchments outside the current range of responses and therefore compromise our catchment calibrations. This can partially be estimated as a function of the length of the period and the range of climate circumstances over which (different parts of) the model was calibrated. The data shown in Figure 3-19 show that indeed some of the catchments appear to have changed their responses to rainfall and this change is not captured by the models. This has a minor impact on the project conclusions as these were based on comparative responses between scenarios, but it does raise the question of how simple models such as those used could be generalised to capture these changes in circumstances. The model structural uncertainties arising from serial correlation of parameters, insufficient parameters, a lack of representation of fundamental runoff processes and unsound modelling formulations can produce uncertain model results. All six rainfall-runoff models used in this project are based on simple structures which may induce model uncertainty as they have a limited ability to incorporate all physical processes involved in estimating runoff from climate and catchment inputs. To minimise the effect of model structural uncertainty, only those models which consistently performed better in calibration were chosen for use (see Section 4.3.4). This use of a multiple model ensemble is expected to help minimise some of the errors due to model structural uncertainty. In addition there is the uncertainty of the process of regionalisation of parameters. In the project we need to project runoff in ungauged catchments. To do this parameters were translated from gauged catchments on the basis of hydrological similarity (SKM, 28). This was different from the approach taken by the other Sustainable Yields projects which based their regionalisation purely on proximity. These two approaches are assessed in detail in Chapter 4 of this report (Aryal et al., 21). 3.6 Discussion The adopted Sacramento-IHACRES model consistently produced better calibration results, as assessed using the Nash- Sutcliffe efficiency (NSE), than any of the individual models, and any of the other 26 possible arithmetic mean combinations of the five models. This result was consistent across the surface water modelling area despite the variation in climate and catchment characteristics. It is therefore concluded that the adopted model is likely to be robust within the range of inputs applied. It is interesting that the two models that yielded the best NSEs are the most detailed (Sacramento) and, arguably, the simplest model (IHACRES) that has no process representation expressly stated. The Sacramento model has 16 parameters, many more than the other models, allowing a more detailed process representation and consequently might be expected to best represent catchment responses. IHACRES is a transfer function model that could be used for any physical system with a few forcing inputs and a single output. While IHACRES has origins in time-series analysis, its structure used is tailored to suit river catchments whose efficient parameter structure results in generally very good fits to data sets. The trend in model error with under estimation in the early part of the calibration period and over estimation in the latter part in some catchments may be due to a number of factors. Some catchments, particularly in the southern region, have undergone plantation development, agro-forestry and environmental plantings on agricultural land beginning in the early 199s. These plantings grew rapidly through to the end of our modelling period, and as they did they were likely to progressively intercept more water leading to a reduction in stream flow. This suggests the catchments were not in steady state therefore the parameter values may have been optimised for a state that is evolving. Furthermore, in catchments with mature forest a decline in groundwater levels has been observed in almost every catchment with monitoring wells in some cases creating a disconnect between surface and groundwater. This has been attributed to a declining rainfall experienced over the last few decades, and has also led to a reduction in runoff coefficient through this period. The groundwater representation in the models is may not be rigorous enough to represent such a mechanism and optimisation based on the whole record necessarily leads to under calculation in the early period and over calculation in the latter. The Nash-Sutcliffe modelling Efficiency (NSE) is only one measure of model capability to match observed data. It is biased towards higher flows which means it is more useful when the interests are floods and overall water volume, the latter being the focus of this study. Hence it is suitable as a measure of model performance for this project. The optimal model parameters will depend on many factors which influence catchment response to rainfall. The catchments simulated in this project range from very wet (annual rainfall over 12 mm with perennial flow) to drier (annual rainfall 64 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

35 less than 6 mm with occasional years with no flow at all), the sizes range from 1 km 2 to 4, km 2, there are steep upland catchments and flat coastal plain catchments, and catchments with total native forest cover and some almost completely cleared for agriculture. With such variability it is not surprising that the optimised models have a large range in parameter values shown in Figure 3-9 to 3-12, and indeed it is perhaps more surprising that the selected models do as well as they do in representing such a disparate range of responses. One catchment was consistently the hardest to simulate, it was the northern most catchment, with a low runoff coefficient and was influenced by groundwater abstraction and irrigation interacting with the stream alluvial aquifers and stream flows. Some of the values in the future climate series, especially for Scenario Cdry, are outside the range in the historical climate series which was used for calibrating the models. Therefore it is possible that the model s performance will be worse than that indicated by the NSE, especially if rainfall-runoff processes change under the extreme climate. The input rainfall and APET data for the rainfall-runoff models were obtained from gridded SILO patched point data. The accuracy of these data primarily depends on the density and quality of the meteorological observation network, which is not uniform across the surface water modelling area. There are likely to be orographic effects that are not accounted for in the gridded data, but as the relief across the surface water modelling area is relatively low the resultant inaccuracies are likely to be limited in area and severity. The observed streamflow records are generally good with a long period of record and few data gaps for many stations. The stations with significant data gaps were those which were decommissioned during the mid-199s and restarted in the mid-2s. Eight catchments calibrated in this project were downstream of dams which do not regularly release water into the stream. Three of the eight dams have never been overtopped, but the rest have overflowed at least eight times during the calibration period. As neither the overflow volumes nor the downstream releases were available, the dams were treated as terminal points, that is no flow was routed further downstream in the models. It was assumed that the dam overtopping occurs during the high flow period when the overflow volume is a relatively small portion of the flow recorded at the gauged catchments, and calibrations were run ignoring these upstream contributions. This is not expected to produce major errors. In a few cases calibration was not possible and parameters for these reaches were derived using the hydrological similarity approach described earlier. As the estimated streamflows for reaches below dams ignore dam releases, detailed estimations of flow volumes would need to be added to the modelled streamflow to get a more realistic assessment of in-stream conditions. With the exception of one gauge, calibrations for those catchments downstream of the dams were quite satisfactory with daily NSE values ranging from.73 to.92 justifying the treatment of dams as terminal points. These calibration results give confidence in the model s ability to simulate the observed streamflow. The gauge that could not be adequately calibrated was removed from the calibration set and flows were modelled on the basis of similarity parameters described earlier. Despite good quality data, software modelling of natural systems is prone to uncertainties. Only some of those are measurable and can be accounted for. For example, the uncertainty in modelling due to different combinations of parameter values producing the right results a phenomenon called equifinality is hard to quantify. This is closely related to the fact that model results can be right for wrong reasons due to compensating errors, e.g. different wrong combinations of surface and baseflows may produce the correct modelled streamflow. The results from internal subcatchments can also compensate each other such that despite incorrect response from individual subcatchments the overall flow from the catchment could be correct. Also, the model calibration can also be affected by the choice of coefficient of efficiency in objective function for the automated optimised calibration. Errors due to the quality of the hydrological observation network are not discussed here. These affect the reliability of modelling results since the errors in parameter values can propagate from calibration to the future scenario projection. Furthermore, good reproduction of historical data in calibration does not guarantee a reliable future projection, especially in a non-stationary climate regime. 3.7 References Aryal, S.K., Silberstein, R.P. and Hodgson, G.A., 21. Rainfall-Runoff Modelling in South-west Western Australia. 4. Catchment similarity and cross-verification of parameters, CSIRO Water for a Healthy Country Flagship, Australia, pp.67-74, Bormann, H., 25. Evaluation of hydrological models for scenario analyses: Signal-to-noise-ratio between scenario effects and model uncertainty. Advances in Geosciences, 5, CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 65

36 Charles, S.P., Silberstein, R.P., Teng, J., Fu, G., Hodgson, G.A., Gabrovsek, C., Crute, J., Chiew, F.H.S., Smith, I.N., Kirono, D.G.C., Bathols, J.M., Li, L.T., Yang, A., Donohue, R.J., Marvanek, S.P., McVicar, T.R., Van Niel, T.G. and Cai, W., 21. Climate analyses for the South-West Western Australia Sustainable Yields Project. A report to the Australian Government from the CSIRO South-West Western Australia Sustainable Yields Project, CSIRO Water for a Healthy Country Flagship, Australia, CSIRO, 29. Surface water yields in south-west Western Australia. A report to the Australian Government from the CSIRO South-West Western Australia Sustainable Yields Project, CSIRO Water for a Healthy Country Flagship, Australia, 171+xx, Gupta, H., Bastidas, L., Vrugt, J. and Sorooshian, S., 22. Multiple criteria global optimisation for watershed model calibration. In: Q. Duan, H. Gupta, S. Sorooshian, A. Rousseau and R. Turcotte (Editors), AGU Monograph: Advances in Automatic Calibration of Watershed Models. Jeffrey, S.J., Carter, J.O., Moodie, K.B. and Beswick, A.R., 21. Using spatial interpolation to construct a comprehensive archive of Australian climate data. Environmental Modelling and Software, 16, Nash, J. and Sutcliffe, J., 197. River flow forecasting through conceptual models, 1: a discussion of principles. Journal of Hydrology, 1, Silberstein, R.P., Aryal, S.K., Hodgson, G.A., McFarlane, D.M., Pearcey, M., Durrant, J. and Bari, M.A., 21. Rainfall-Runoff Modelling in South-west Western Australia. 2. Rainfall-runoff modelling Methodology, CSIRO Water for a Healthy Country Flagship, Australia, pp.9-34, SKM, 28. Estimation of Sustainable Diversion Limits for catchments in South West Western Australia. Regionalisation of Sustainable Diversion Limits for catchments, Sinclair Knight Mertz for the Department of Water, WA, 131+vii. Van Dijk, A.I.J.M., Kirby, J.M., Paydar, Z., Podger, G., Mainuddin, M., Marvanek, S. and Peña Arancibia, J., 28. Uncertainty in river modelling across the Murray-Darling Basin. A report to the Australian Government from the CSIRO Murray-Darling Basin Sustainable Yields Project, CSIRO Australia, Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' CSIRO 21

37 CSIRO 21 Rainfall-Runoff Modelling in South-west Western Australia. 3. Performance of the 'adopted model' 67

38