Proceedings of the 13 th International Conference of Environmental Science and Technology Athens, Greece, 5-7 September 2013 IDENTIFICATION OF UNCERTAINTY IN HYDROLOGICAL MODELLING USING SEVERAL MODEL STRUCTURES, OPTIMISATION ALGORITHMS AND OBJECTIVE FUNCTIONS IN A CYPRIOT WATERSHED GKILIMANAKIS E. 1, VASILIADES L. 1, and LOUKAS A. 1 1 Laboratory of Hydrology and Aquatic Systems Analysis, Department of Civil Engineering, University of Thessaly, Pedion Areos, 38334 Volos, Greece e-mail: lvassil@civ.uth.gr EXTENDED ABSTRACT The aim of this study is to identify and assess different types of hydrological model uncertainty in daily runoff simulation at Yermasoyia watershed, Cyprus. This is achieved with the aid of an open software available in the R statistical computer environment, called HYDROMAD, (Hydrological Model Assessment and Development) based on work by Jakeman and Croke at The Australian National University. The proposed modeling framework rests on the unit hydrograph theory of hydrological cycle components separation and consists of a generic two-component structure: a soil moisture accounting (SMA) module and a routing or unit hydrograph module. The SMA module converts rainfall and temperature into effective rainfall. The routing module converts effective rainfall into streamflow. Model calibration can be approached in many different ways. The usual strategy is to jointly optimize all model parameters or alternatively, to estimate the unit hydrograph directly from streamflow data, using inverse filtering or average event unit hydrograph estimation (Andrews et al., 2011). Several well-known lumped Rainfall-Runoff (R-R) models (such as the GR4J, the IHACRES models and the AWBM) have been applied and tested using the split-sample test for estimation of model uncertainty. Several local non-linear and global optimization algorithms (i.e. SCE Shuffled Complex Evolution, DREAM DiffeRential Evolution Adaptive Metropolis, DE - DiffeRential Evolution, SANN - Simulated Annealing and quasi-newton) have been deployed for model calibration and then compared for the different hydrological structures in an attempt to identify sources of optimization error and model parameter uncertainty. Finally, several objective functions, (i.e. Nash-Sutcliffe Efficiency and variations or adaptations), addressing different parts of the hydrograph have been used to assess both the skill and the robustness of the selected models to perform consistent streamflow predictions. Model performance has been evaluated with the use of fit statistics for calibration and validation periods. A total of 224 simulations have been performed in this study comprising of all possible combinations from four (4) hydrological models, four (4) objective functions, seven (7) optimisation algorithms and two (2) calibration periods. Application of the R-R models in Yermasoyia watershed showed that the primary source of uncertainty in R-R modeling was the choice of the hydrological model structure followed by the parameter uncertainty caused by the optimization algorithm and the choice of an objective function. Overall, to reduce R-R modeling uncertainty an ensemble of R-R models using benchmark comparisons in testing optimisation algorithms and objective functions is suggested. These findings could be useful in water resources management at the study area using operational lumped rainfall-runoff models. KEYWORDS: Hydrological models, rainfall-runoff modelling, model structure uncertainty, optimisation algorithms, objective functions, parameter uncertainty, streamflow modelling.
1. INTRODUCTION Nowadays, most Rainfall-Runoff (R-R) models bear a relatively simple structure and use not too complex mathematical equations to explain the physical processes that take place in catchments. Parsimony and simplicity in terms of model parameters and structure are essential but it is also required to ensure a realistic behavior that best reflects nature. Concerning the lumped R-R models, their parameters cannot be measured directly on site and therefore, their values or ranges of values ought to be determined through calibration techniques. Initially, calibration was being performed by manual or trial and error approaches but these soon proved to be a rather laborious task and time consuming (Madsen et al., 2001). Inevitably, scientific research shifted its focus towards the development of more automated approaches. These generally involve the selection of either a single measure of goodness of fit (objective function), focusing on particular parts of the hydrograph (low or high flows) or multi-objective calibration as proposed by Gupta et al., (1998), focusing on the overall hydrograph behavior. Automated calibration also includes the selection of an appropriate parameter search method (optimization algorithm). Until recently, calibration has been performed using local-search optimization techniques, such as the Nelder and Mead simplex algorithm. In 1992, Duan et al. addressed the limitations of local search algorithms producing large numbers of parameter values trapped as local optima within the optimum space of a given objective function. Their studies suggested the use of global search techniques, such as the Shuffled Complex Evolution, (SCE) and similar (i.e. Differential Evolution, DE) and proved that these algorithms are very effective and efficient, they find consistently the region of global optimum solutions and require not too many function evaluations, (Duan et al., 1992). This study examines different conceptual lumped R-R model structures and compares the modeled streamflows to the observed runoff measured at the outlet of Yermasoyia catchment. Streamflow simulation requires that model parameters are estimated (calibrated) in a way that modeled flows are as closely related as possible to the observed flows. This implies that during modeling, uncertainty resulting in poor predictions must be identified, evaluated and reduced. Uncertainty in R-R modeling is often described as the degree of confidence that resides in the predictions after a model has been calibrated. In general, there exists an amount of inherent uncertainty due to randomness and variability in nature and thus cannot be reduced. However, what can be reduced is the epistemic uncertainty attributed to the model structure, the selection of an appropriate objective function and an optimization algorithm for parameter estimation. 2. STUDY AREA AND DATABASE The watershed of Yermasoyia is located in the Troodos Mountain, near Limassol, Cyprus and has an area of approximately 157Km 2 (Fig. 1). The elevation of the catchment ranges from 70m to 1400m above mean sea level (a.m.s.l.) and the Mediterranean climate found in the region produces mild winters and hot and dry summers. Thus, the stream of the watershed is found to be transient and ephemeral with rainfall-induced peak flows being observed during winter months. The mean annual areal precipitation is about 640mm (450mm at lower elevations up to 850mm at higher elevations) and the mean annual runoff at the catchment outlet is 0.42 m 3 /s. The available data consists of 11 hydrological years (Oct. 1986 Sept. 1997) of daily precipitation from three rainfall gauges at 70m, 100m and 995m a.m.s.l., daily maximum and minimum temperature values from one meteorological station at 70m a.m.s.l. and daily streamflow measured at the outlet of the catchment. Areal precipitation and potential evapotranspiration was estimated by daily precipitation gradients and the Hargreaves method, respectively.
Figure 1: Yermasoyia watershed, Cyprus. 3. METHODOLOGY Four different lumped R-R models have been selected, namely the GR4J (Perrin et al., 2003), the AWBM (Boughton, 2004) and two versions of the IHACRES model: the CMD (Catchment Moisture Deficit, Croke and Jakeman, 2004) and the CWI (Catchment Wetness Index, Jakeman and Hornberger, 1993). Based on observations of streamflow, two different temporal periods have been identified. A wet period, (Oct 1987 Sep 1992), with mean annual flow 0.45 m 3 /sec and a dry period, (Oct 1992 Sep 1997), with mean annual flow 0.35 m 3 /sec. Using the split-sample test the models have been calibrated for half of the years during both wet and dry periods leaving the rest half of the years for validation. Figure 2 shows the flowchart of the methodology. The analytical process steps of the applied method can be found in Gkilimanakis (2013). Overall, a total of 224 simulations have been performed in this study (4 models, 4 objective functions, 7 optimisation algorithms, 2 temporal periods). For each model, parameter estimation is based on combination of objective function and optimisation algorithms. Optimisation techniques involved the choice of several local and global search algorithms. Finally, the models have been evaluated with the use of fit statistics. 3.1 Rainfall Runoff models The lumped GR4J is a four-parameter model and the ranges of its parameters have been initially taken from the 80% confidence intervals (Perrin et al., 2003). For the groundwater exchange coefficient the ranges have been extended from (-5.0, 3.0) to (-25.0, 3.0) to avoid optimisation algorithms converging to local optima during calibration. Also, the proposed fixed split of 10% and 90% of water entering into the Routing module as slow and quick flow respectively has been selected. The 3-bucket structure of the AWBM model has been selected. This configuration uses different storing capacities with weighted areas, to simulate partial areas of runoff. Excess water is calculated independently for each capacity and all of it is then divided into two stores: surface and baseflow. These stores are routed into two components arranged in parallel. The parameters of the routing components have been calculated by fitting a linear transfer function with exponentially decaying components. In other words, an ARMAX-like model has been specified, (Autoregressive, Moving Average), with autoregressive terms = n and moving average terms = m, (i.e. the transfer function order). To ensure a good choice of order for the ARMAX model, different combinations (with respect to n, m and the delay
between effective rainfall and runoff) have been tested. ARMAX models have been fitted using the SRIV (Simple Refined Instrumental Variable) algorithm as described by Young et al. (2008). Different model structures shown in Table 1 have been evaluated by comparing the criterion of determination (R 2 ) and the Average Relative Parameter Error (%ARPE). In this study, a configuration of two exponential stores in parallel with a transfer function of order (2, 1) has been selected. Figure 2: Study outline and main calibration strategies. Table 1: AWBM. Specification of routing structure. ARPE and fit statistics calculated by fitting several unit hydrograph transfer functions of different orders. Transfer function order ARPE R 2 (n=0, m=0, d=0) 0.000 0.277 (n=1, m=0, d=0) 0.000 0.689 (n=1, m=1, d=0) 0.006 0.483 (n=2, m=0, d=0) N/A 0.501 (n=2, m=1, d=0) 0.000 0.756 (n=2, m=2, d=0) 0.012 0.684 The IHACRES model consists of two modules. A non-linear or rainfall loss module through which Rainfall P (t) is transformed into Excess or Effective Rainfall U (t). The linear module converts / routes the excess rainfall U (t) into streamflow. For the CWI version of IHACRES model, at a time step (t), the excess or effective rainfall U (t) is proportional to rainfall P (t) and scaled by a soil moisture index s (t) so that: U (t)= c * s (t)* P (t) (1) where s (t) is the catchment wetness index, which decays exponentially backwards in time and P (t) is the observed rainfall. For the CMD version of IHACRES model, at a time-step (t), the portioning of input rainfall into drainage (or effective rainfall), evapotranspiration and changes in catchment moisture (Croke and Jakeman, 2004), is given by the following equation: U (t) = M (t) M (t-1) ET (t) + P (t) (2)
where M (t) and M (t-1) are the CDM at timesteps (t) and (t-1), P (t) is the catchment areal rainfall and ET (t) is the evapotranspiration, all in millimeters. Similar to the specification of the AWBM routing module parameters, a structure of two runoff components arranged in parallel has been used for both IHACRES models. Once more fitting of ARMAX models with SRIV algorithm indicated that the best transfer function was of order (2, 1) for both CWI and CMD versions. 3.2 Optimisation algorithms Seven different optimization methods, using both local and global search algorithms, have been tested at each calibration for all R-R models. Local search algorithms were set to perform sampling of parameter sets on a single-mode, instead of a multi-start mode, given that the later choice required a lot more time whereas it did not improve significantly the algorithm performance. The maximum iterations were set to 1000 and algorithms were set to start their search from 100 different parameter samples. In this study the following algorithms have been evaluated: The Nelder-Mead, a non-linear direct search method that identifies points (simplices) in the parameter space producing an objective function with one local minimum, the PORT, a gradient search method for finding local minimum, the BFGS, a quasi-newton non-linear optimisation method that uses both values of the objective function and gradients to identify local minimum, the SANN, a probabilistic method of optimisation of a given objective function that returns a global minimum solution, the SCE, a probabilistic population-sampling evolutionary algorithm that converges to a global optimum solution, the DE, a stochastic global optimization algorithm and the DREAM, a multiple Markov Chains Monte Carlo method that searches for a global solution. Mathematical details of the study algorithms could be found in HYDROMAD package (http://hydromad.catchment.org). 3.3 Objective functions Objective functions (OF) measure how well the modeled streamflows match with the available observations. In this study, the well known Nash-Sutcliffe Efficiency (NSE) and NSE modifications and/or adaptations have been used as objective functions. The selected OF are: (1) the NSE, (2) the NSE 3, a transformation of NSE with the absolute residuals raised to the cubic power, (3) the Viney OF as suggested by Viney et al., (2009) and (4) the OF proposed by Bergstrom et al., (2002). The mathematical formulations are: Qobs i is the observed flow on day i, Qsim i is the simulated flow on day i and Qobs is the average observed flow for the simulation period. All OF range from - to 1 (optimal value). Viney and BL OFs are modifications of NSE with a penalty constraint. For Viney OF, a log-bias constraint term is subtracted from the NSE. This penalty is assigned in a multiplicatively symmetrical way (i.e. the same penalty is applied to a prediction that is twice or half of the observation volume), Viney et al., (2009). The BL OF combines the NSE and the relative bias. To assess model performance, several evaluation metrics have been calculated for both dry and wet validation periods. These are: The NSE and the relative Bias as described before, the Root Mean Square Error (RMSE, in mm) between simulated and observed flows given by the formula: n 2 Qobs i Qsimi RMSE i 1 n
and the Average Percent Error of the Maximum Annual Flows, AMAFE, expressed as k 1 MaxQsim j MaxQobs j % AMAFE 100. j k j MaxQobs 1 j flow of year j, MaxQsim is the simulated maximum annual MaxQobs is the observed maximum annual flow of year j, and k is the j number of hydrological years of the simulation period. Parameter estimates obtained during calibration have been used in validation, assuming stationarity in the state variables over the entire time-series. Assessment of model performance outside the period of calibration and over periods with different climatic patterns indicated the sensitivity of the models to the hydrological variability. 4. RESULTS In general, modeled streamflows produced by different model structures, are closely related to the observed runoff. All optimisation algorithms are capable to optimize the study hydrological models and the evaluation metrics yielded plausible results. As an example, Figure 3 shows the derived streamflow hydrographs (in log-scale) using the AWBM model. The model has been calibrated over the dry period (1992-1997) and validated over the wet (1987-1992) using all algorithms and the Viney as objective function. It is shown that the SCE and DE algorithms slightly outperform the others. Similar results have been produced also for the other hydrological structures (results not shown due to paper length limitations). NSE = 0.734 Validation (wet) Calibration (dry) NSE = 0.798 NSE = 0.744 NSE = 0.793 NSE = 0.734 NSE = 0.798 NSE = 0.709 NSE = 0.773 NSE = 0.754 NSE = 0.798 NSE = 0.754 NSE = 0.797 Figure 3: AWBM efficiency (NSE) results, (objective function used: Viney). Table 2 presents the evaluation metrics of the validation period for all R-R models. These results have been derived using the SCE algorithm and compares the optimisation of the different objective functions. The table shows that the GR4J model obtained the highest efficiency (NSE) values irrespective of the objective function used. The IHACRES-CMD structure achieved the second highest efficiency followed by the AWBM and the IHACRES-CWI models. In terms of volume losses, (RMSE and rel. Bias), again the GR4J and the CMD models perform slightly better than the other two and, optimisation of the two bias-constrained objective functions (Viney and BL) yields results of minimal volume errors. Finally, in terms of peak flow assessment (AMAFE criterion), the AWBM and the IHACRES-CWI models have derived the lowest values. Figures 4a and 4b show the scatter plots between modeled and observed streamflows only for validation period. The
plots demonstrate the degree of fitness of the points to the 1:1 line shown in black color, (identity line). A visual inspection indicates that there is a high agreement between calculated and observed flows and, in all models, the majority of the scatters concentrate close to the 1:1 line. Also, all models appear to have the skill to perform homogeneously within the prediction range of flows and are not influenced by the magnitude of the simulated flows. Finally the red line shows the linear correlation between modeled and observed flows as well as whether the predictions of the models are under-estimated or over-estimated (model bias). It is demonstrated that models perform better during the wet period.. Table 2: Validation statistics for all models and objective functions using the SCE optimisation algorithm Objective Function MODEL NSE RMSE rel. BIAS AMAFE GR4J 0.836 0.309-0.020-0.128 Viney AWBM 0.742 0.397 0.076-0.009 CMD 0.771 0.374-0.004-0.125 CWI 0.726 0.404 0.052-0.051 GR4J 0.845 0.309-0.127-0.161 NSE AWBM 0.741 0.397 0.078-0.008 CMD 0.771 0.374-0.006-0.128 CWI 0.724 0.403 0.077-0.055 GR4J 0.827 0.319 0.001-0.184 BL AWBM 0.740 0.395 0.075-0.009 CMD 0.771 0.374 0.001-0.122 CWI 0.726 0.402 0.053-0.052 GR4J 0.820 0.326 0.047-0.161 NSE 3 AWBM 0.739 0.399 0.083-0.003 CMD 0.770 0.374 0.008-0.117 CWI 0.711 0.412 0.202-0.220 a. (dry period) b. (wet period) Figure 4: Scatter plots of Qobs and Qsim for all study models during validation period 6. DISCUSSION AND CONCLUDING REMARKS The selected hydrological model structures conceptualize and explain the physical processes in a similar way. However, as demonstrated, they have performed differently under the same optimization techniques and objective functions used. Depending on the
purpose of study, different efficiency criteria emphasize different hydrologic behaviors. In this study, the NSE and the NSE 3 place emphasis to the high peaks at the expense of the low flows, while the Viney and the BL objective functions demonstrate a finer model behavior in all hydrograph segments. The choice of an optimization technique also affects the uncertainty degree in the model parameters, but in a lower degree than the model structure itself. For example, global search algorithms appear to outperform the local search algorithms in most cases, with the SCE and the DE algorithms being almost in all cases the most effective ones. In general, local search algorithms also performed in most cases, very satisfactorily. The split sample test of different calibration / validation periods, (wet / dry period split), demonstrated that hydrologic variability affects significantly the performance of the models. Different calibration periods produced different parameters estimates. The more unstable and disperse the parameter values are, the worse the model performs. For a model to be considered reliable and robust, it should perform consistently during hydrologically different calibration periods. This is termed as parameter transposability in time (Gharari et al., 2012) and is viewed as one of the most important elements in R-R modeling. Application of the R-R models in Yermasoyia watershed showed that the primary source of uncertainty in R-R modeling was the choice of the hydrological model structure followed by the parameter uncertainty caused by the optimization algorithm and the choice of the objective function. Overall, to reduce R-R modeling uncertainty an ensemble of R-R models using benchmark comparisons in testing optimisation algorithms and objective functions is suggested. These findings could be useful in water resources management at the study area using operational lumped rainfall-runoff models. REFERENCES 1. Andrews F.T., Croke B.F.W. and Jakeman A.J. (2011) An open software environment for hydrological model assessment and development, Environ. Modell. Softw., 26, 1171-1185. 2. Bergstrom S., Lindstrom G. and Pettersson A. (2002) Multi-variable parameter estimation to increase confidence in hydrological modeling, Hydrol. Process., 16, 413 421. 3. Boughton W. (2004) The Australian water balance model, Environ. Modell. Softw., 19(10), 943-956. 4. Croke B.F.W. and A.J. Jakeman (2004) A Catchment Moisture Deficit module for the IHACRES rainfall-runoff model, Environ. Modell. Softw., 19(1), 1-5. 5. Duan Q., Gupta V.K. and Sorooshian S. (1993) A shuffled complex evolution approach for effective and efficient global minimization, J. Optimization Theory Appl., 76(3), 501-521. 6. Gharari S., Hrachowitz M., Fenicia F. and Savenije H.H.G. (2012) An approach to identify time consistent model parameters: sub period calibration, Hydrol. Earth Syst. Sci., 17, 149 161. 7. Gkilimanakis E. (2013). Identification of uncertainty in hydrological modeling using several model structures, optimisation algorithms and objective functions. M.Sc. Thesis, Department of Civil Engineering, University of Thessaly, Volos, Greece. 8. Gupta H.V., Sorooshian S. and Yapo P.O. (1998) Toward improved calibration of hydrologic models: Multiple and non commensurable measures of information, Water Resour. Res., 34, 751 763. 9. Jakeman A.J. and G.M. Hornberger (1993) How much complexity is warranted in a rainfallrunoff model? Water Resour. Res., 29, 2637-2649. 10. Madsen H., Wilson G. and Ammentorp H.C. (2001) Comparison of different automated strategies for calibration of rainfall-runoff models, J. Hydrol., 261, 48-59. 11. Perrin C., Michel C. and Andreassian V. (2003) Improvement of a Parsimonious Model for Streamflow simulation, J. Hydrol., 279(1-4), 275-289. 12. Viney N.R, Vaze J., Chiew F.H.S., Perraud J., Post D.A and Teng J. (2009) Comparison of multi-model and multi-donor ensembles for regionalization of runoff generation using five lumped rainfall-runoff models, Proceedings, 18th World IMACS / MODSIM Congress, Cairns, Australia 13-17 July 2009, 3428-3434. 13. Young P.C. (2008) The refined instrumental variable method, Journal Européen des Systèmes Automatisés, 42(2-3), 149-179.