Use of the extended Fourier Amplitude Sensitivity Test to assess the importance of input variables on urban water supply system yield a case study

Transcription

1 Use of the extended Fourier Amplitude Sensitivity Test to assess the importance of input variables on urban water supply system yield a case study D. M. King and B. J. C. Perera 1 1 School of Engineering and Science, Victoria University, Melbourne, Australia PO BOX Melbourne, VIC 8001 David.King@zoho.com, Chris.Perera@vu.edu.au Keywords: Extended Fourier Amplitude Sensitivity Test (efast), Sensitivity Analysis, Uncertainty, Variability, Yield estimation, Barwon water supply system, Abstract: This paper presents a Sensitivity Analysis (SA) of the input variables used in the estimation of yield, considering multiple 20 year hydroclimatic scenarios of system inflow, rainfall, evaporation and demand. The Barwon urban water supply system in Australia was considered as the case study, whilst the extended Fourier Amplitude Sensitivity Test (efast) method was used for the SA. Input variables of the simulation model of the Barwon system were divided into two categories for use in SA analysis: the climate dependant variables (i.e. system inflow, rainfall, evaporation and demand) used to generate various climate scenarios, and the system policy variables which were assumed to have knowledge deficiency in relation to their optimum values. The security of supply thresholds were found to be the most important input variables, followed by the upper restriction rule curve position. The remaining variable did not show a discernible trend, indicating sensitivity to system inflow variability rather than total system inflow volume. The yield estimate was found to increase as the total system inflow increased. However the yield estimate had a wide range of variability as the total system inflow increased showing that the model behaviour and the yield estimate is particularly sensitive to climate variability. Introduction Balancing water demand with water supply is a challenging task for water authorities in Australia and throughout the world. This is particularly the case with urban water supply systems, where growing population in urban cities and the effect of climate change have put immense pressure on the management and operation of these systems. Demand can be decreased via water saving measures and schemes, and education, whilst water supply can be increased through optimisation of system operation and/or augmentation with additional water sources. The estimation of yield is a fundamental component in the management and operation of an urban water supply system. It represents the performance of the physical system and the optimum operation and management of the system. Yield is defined throughout much of the Australian water industry and in this study as: the maximum average annual volume of water that can be supplied from the water supply system subject to climate variability, operating rules, demand pattern and adopted level of service (or security criteria). Operating rules include water restriction rules, environmental flow rules and storage targets. The security of supply criteria are defined by thresholds that safeguard against system failure, such as minimum storage threshold and reliability of supply threshold. If one or more security criteria threshold is violated, the system is deemed to have failed in the definition of system yield. The yield of a water supply system is typically estimated by simulating the water supply system using a computational model of the physical system considering the entire length of available climate data. Although the exact realisation of yield is impossible to obtain (due to the uncertainty

2 inherent in climate variability), uncertainty in its estimation can be reduced by identifying highly influential input variables, and investigating and refining their knowledge. This will improve the confidence of its estimation, leading to optimised management procedures and policies. By identifying the most important input variables used in the estimation of yield via a sensitivity analysis, resources can be allocated and research prioritised so that water authorities can improve the knowledge of these important input variables, and hence decrease their uncertainty and variability, and finally increase the confidence in the yield estimate. In Australia, yield is commonly estimated by iteratively increasing the average annual demand until system failure. This is done using a computational model such as REALM (REsource ALlocation Model), which is used extensively in Australia (Perera et al., 2005). Sensitivity Analysis (SA) is the study of input variable uncertainty and variability and its propagation (through a computational model) to the model output. SA attempts to measure the weighting of the sources of uncertainty and variability found in input variables of a model. In other words, SA determines which of the model s input variables carry the most importance to the model output, i.e. to which variables are the model and hence the model output, most sensitive to variation. Commentary regarding SA theory, and applications from hydrology and water resources, and various other scientific fields can be found in Frey and Patil (2002), Saltelli et al. (2000, 2004), Ratto et al. (2008), Pappenburger et al. (2006a, 2006b, 2006c), and references therein. In a proof-of-concept case study, King and Perera (2010) performed SA on a REALM model of a simple urban water supply system, considering the Morris Method (Morris, 1993) and the extended Fourier Amplitude Sensitivity Test (efast Cukier et al., 1973). It was found that streamflow was the most important variable to the estimation of yield. Although the SA procedure was successful, it was found that the methodology and SA framework used in King and Perera (2010) needed improvement. King and Perera (submitted) presented a refined SA framework that considers two types of inputs and their respective variability type (i.e. climate dependent variables and system policy variables), with different scenarios of hydroclimatic variables of various lengths. The climate dependent input variables were considered to be subject to natural variability due to inherent climate uncertainty and the system policy variables were assumed to have knowledge deficiency in terms of their optimum values. Twenty hydroclimatic scenarios with different lengths were generated from the historical data of climate dependent variables. The Morris Method of SA (Morris, 1991) was used to provide qualitative ranking information on the importance of system policy variables over various climate scenarios. This paper uses the same SA framework of King and Perera (submitted). The efast SA technique was employed to provide quantitative importance measures of system policy variables on the system yield of an urban water supply system, instead of the qualitative ranking information obtained from the Morris method in King and Perera (submitted). The SA was performed over seven 20 year climate scenarios to test the effect of system inflow on the importance of input variables on system yield. The Barwon urban water supply system is described first in this paper. It is followed by a further description of the SA framework together with a brief introduction to the efast method, including sensitivity measures. The results of the SA of input variables used in the estimation of yield is then described, with conclusions of the paper are presented in the final section. Barwon Urban Water Supply System The Barwon urban water supply system in Australia is owned and managed by Barwon Water Corporation and considered as the case study in this paper. It is situated on a regional and coastal

3 area in south east Australia, 60 km south west of Melbourne. It supplies over 43,000 ML a year to 285,000 permanent residents in an 8,100 km 2 area around Greater Geelong. The Barwon system headworks consists of six major reservoirs, 5,000 km of pipes, six major storages, six water treatment plants and nine water reclamation plants. Water is sourced from two rivers and their catchments, and a number of groundwater sources. A REALM model of the Barwon urban water supply system was available from Barwon Water Corporation for this study (SKM, 2006). Input variables for the Barwon system REALM model include streamflow (to model inflow into the system), evaporation and rainfall (to model the losses and gains from storages), demand pattern and system policy variables (that dictate the system behaviour). Geelong is the only demand centre considered in this study. Environmental flows and other demands were assumed compulsory and excluded from the scope of this study. Fourteen system policy variables were used in the operation of the Barwon system. These are two supply security thresholds, a variable describing target storage curves and 11 variables used to model a four-level demand restriction policy. A conceptual schematic of the Barwon system REALM model can be seen in Figure 1. Figure 1. Basic Headworks Schematic of the Barwon Water system Barwon Water Corporation considers two supply security thresholds in managing the Barwon water supply system, and its system model: the reliability of supply and the minimum level of storage. The reliability of supply is the ratio of the number of time steps not subject to water consumption restrictions to the total number of time steps considered in the planning period. A typical value for this is 95%, which allows the system to have water restrictions of up to 5% of the planning period. The minimum storage level threshold is triggered by a low volume of total system storage, considering the six main reservoirs of the Barwon system. If either of these criteria is violated, the system is deemed to have failed under the given conditions in the Barwon Water Corporation planning studies. The estimation of yield is therefore directly linked to these SA variables which can fail individually or at the same time. A four-level (5 stage) demand restriction policy is implemented for the Barwon system, denoted as Restriction Rule Curves (RRC). Consisting of upper and lower rule curves, three intermediate

4 restriction zones (with definitions of relative positions and percentage restrictable levels) and a base demand curve, it is used to restrict the outdoor water demand during low storage volume periods. See King (2009) and King and Perera (submitted) for further details and nominal values of the RRC used in this study, including the values for the upper, lower, and intermediate zone curves, and the percentage restrictable in each zone. The Target Rule Curves (TRC) are defined by a single set of five-point curves for all months of the year, indicating the preferred distribution of individual storage volumes for various total system storage volumes. These curves impose inter-storage transfers to distribute water in the system so as to supply the required demands at various demand points. See King (2009) and King and Perera (submitted) for further details. Methods and Techniques The input variables used in the estimation of yield of an urban water supply system are sorted into two groups, according to two types of uncertainty, as climate dependent variables and system policy variables. The climate dependant variables (i.e. streamflow, rainfall, evaporation and demand) are subject to natural variability due to their inherent climate uncertainty. The system policy variables, consisting of the security of supply thresholds, restriction rule curves, and target rule curves, are controlled by water authorities and in this study were deemed to be subject to knowledge deficiency in regards to their optimum values. Furthermore, the system policy variables are in turn subject to natural variability through their dependence on the climate sequence. Whereas knowledge deficiency can be measured and refined, natural variability will always be present and largely cannot be reduced. Since natural variability cannot be reduced, only the knowledge deficiency of the system policy variables can be assessed. See King (2009) and King and Perera (submitted) for details. The Extended FAST (efast Saltelli and Bolado, 1998) method is a more efficient derivative of the Fourier Amplitude Sensitivity Test (FAST Cukier et al., 1973). In FAST and efast, input variables are assigned incommensurate angular frequencies to transform the input variables into an approximately space filling curve from which the samples are selected. The sampling density of this curve is called the resolution. The resolution and frequencies contribute to the required number of model simulations. Fourier principles of frequency analysis are used to estimate the first-order sensitivity index: S i. The first-order sensitivity index S i, as given in Equation 1, is the ratio of the output variance due to the i-th variable to the variance due to all variables. Whereas FAST can only determine S i, efast uses a more efficient sampling procedure to determine S i, and S Ti in addition. Given in Equation 2, S Ti is the total sensitivity index that estimates the sum of all effects involving the i-th variable. V V ( E Y X i ) i S i = = V ( Y) V ( Y ) Eq 1 S = Ti V V ~ i 1 Eq 2 where V i is the partial output variance due to the i-th input variable only, V(Y) is the total output variance and V ~i is the partial output variance due to all variables except the i-th input variable. The S i index provides the first-order importance of the i-th input variable, free of interaction and higher-order effects with other variables. The S i indices are standardised within an experiment, therefore for a purely linear model, ΣS i =1. The S Ti index measures the combined first- and all

5 higher-order effect of the i-th input variable, including all interaction effects involving the i-th input variable. When compared to S i, S Ti can indicate higher-order effects or interactions with other variables. By efast design, S i <S Ti and S i 1. Numerical errors can occur due to aliasing and interaction errors in the Fourier analysis. To overcome these errors, the resolution of the input variable sampling should be increased until non-erroneous results are obtained. See Saltelli et al. (2000, 2004) and King (2009), and references therein for details. Weekly historic climate data (streamflow, rainfall, evaporation and demand) for a 77 year period beginning on 1 st January 1927 was available. A 20 year moving window of total streamflow volumes (of the system) was calculated. Five sequences were selected with a range of total system inflow volume. For each streamflow sequence selected, the same period of the remaining climate dependant variables (i.e. rainfall, evaporation and demand) were also selected to complete each scenario and maintain cross correlations. Two extra 20 year scenarios were selected that match the total streamflow volume of the previously selected scenario 2, namely scenarios 2b and 2c. The starting time step and total streamflow volumes of the selected 20 year scenarios are given in Table 1. Table 1. Selected Scenarios Scenario Total Streamflow (Ml) Starting Year.Week 1 3,665, ,351, ,137, ,925, ,487, b 3,355, c 3,353, As stated earlier, the 14 system policy variables consist of the two security of supply thresholds, a variable describing target storage curves and 11 variables that control the RRC policy. The TRC variable is a discrete representation of possible storage behaviour. The 11 RRC variables model the RRC policy by changing the position and the curvature of upper, intermediate, lower and base demand curves, and the percentage of demand restrictable in each stage. The upper and lower RRC curvature variables modify the shape of the trigger levels over 12 months year, the upper and lower RRC position variables changes the whole curve positions against the total system storage while the three relative positions determine the positions of the intermediate curves between the upper and lower curves; creating four restriction levels. The three percentage restrictable variables model the outdoor demand restricted in levels 1 to 3, with level 4 being 100% outdoor use restriction. The base demand curve models the indoor water use which is not restricted. See King (2009) and King and Perera (submitted) for details on their nominal values, and permutation ranges and strategies. Using each of the seven 20 year climate scenarios, SA using efast was performed on 14 system policy variables using 1918 model simulations. This number of model simulations produced nonerroneous results, i.e. ΣS i not greater than 1 and the S i < S Ti. Results and discussion Table 2 presents the S i results for the 14 system policy variables simulated over the seven 20 year climate scenarios. The corresponding S Ti results are given in Table 3. The importance measures of

6 Table 2 and Table 3 indicate that one of the two security criteria considered in this study (the minimum storage threshold and the supply reliability threshold) is the most important input variable (first- and total-order) in all scenarios. These two security criteria dominate S i with the remaining variables one or two orders of magnitude less than the critical security criteria. Similar results can be seen from the S Ti indices, where the security of supply thresholds are the most important while there is no other clear trend in the remaining variables. Table 2. S i results of the efast SA using 1918 Simulations Scenario b 2c Relative Position Relative Position Relative Position Percent. Restrict Percent. Restrict Percent. Restrict Upper RRC Curv Upper RRC Position Lower RRC Curv Lower RRC Position Base Demand Target Curves Minimum Storage Threshold Reliability Threshold Sum of S i for all variables

7 Table 3. S Ti results of the efast SA using 1918 Simulations Scenario b 2c Relative Position Relative Position Relative Position Percent. Restrict Percent. Restrict Percent. Restrict Upper RRC Curv Upper RRC Position Lower RRC Curv Lower RRC Position Base Demand Target Curves Minimum Storage Threshold Reliability Threshold The reliability threshold is the most important variable in all 20 year climate scenarios experiments except scenario 3, in which the minimum storage threshold is most important. Scenario 3 inflow includes a large and severe drought which produced a rapid system drawdown, triggering the minimum storage threshold before the reliability threshold is violated. The S i results show that when the reliability threshold is most important variable, the upper RRC position shows notable importance. This is expected since the storage volume has gone below the upper RRC position several times to cause the reliability threshold to be violated. Interestingly, the upper RRC curvature does not have a high importance. There is no discernible trend with the remaining variables. The S i results for the remaining variables given in Table 2 vary across scenarios, showing that the importance of the input variables to the yield estimate changes under different inflow volumes. The varying S i values for the 2, 2b and 2c scenarios (which have equivalent total inflow) shows the importance of input variables is also dependent on the inflow variability, not just inflow volume. The ΣS i given in Table 2 indicates that the model tends to have a linear input to output relationship, with scenario 3 causing the least linear relationship. Table 4 shows partial output variance, V i, for each system policy variable in the SA results given in Tables 2 and 3. The V i measure is the non-standardised measure of only the i-th input variable s effect on the output. Understandably, the V i shown in Table 4 is dominated by the security criteria. The V i measures shows that as the streamflow volume increases V i of all system policy variables tends to increase. In other words, as the streamflow volume increases the yield becomes more sensitive to changes in the system policy variables. Table 4. Vi measures of the efast SA using 1918 Simulations, expressed as standard deviation. Scenario b 2c Relative Position Relative Position Relative Position Percent. Restrict Percent. Restrict

8 Percent. Restrict Upper RRC Curv Upper RRC Position Lower RRC Curv Lower RRC Position Base Demand Target Curves Minimum Storage Threshold Reliability Threshold As shown in Table 5, both the total output variance, V(Y), and the average yield estimate tends to increase as the streamflow increases. This indicates that as streamflow volume increases, the yield estimate increases but becomes more sensitive to the changes in the system policy variables. Scenarios 2, 2b and 2c have similar average yield but a wide V(Y) range, showing that the model and yield estimate is sensitive to streamflow variability. Table 5. Average Yield and total output variance Y(V) Scenario V(Y) x 10 3 Average Yield (Y) 1 27,384 67, ,602 61, ,284 49, ,667 50, ,416 49,475 2b 45,846 60,028 2c 22,024 60,031 Average 56,954 Range 18,015 Conclusions A quantitative sensitivity analysis method was successfully used to identify the importance of input variables used in the estimation of yield of the Barwon urban water supply system in Australia. Input variables were considered to be subject to natural variability (climate dependent variables) or knowledge deficiency (system policy variables). Climate dependent variables were used to select seven 20 year length climate scenarios of various total system inflow. The extended Fourier Amplitude Sensitivity Test (efast) technique was used as the SA method on the 14 system policy variables over the seven hydroclimatic scenarios to uncover system and yield estimate behaviour. Results of the 20 year climate scenarios show that the yield estimate of the Barwon system is highly sensitive to the security of supply thresholds; the reliability threshold and the minimum storage threshold. When the reliability threshold is most important, the upper Restriction Rule Curve (RRC) position is also important, but not the upper RRC curvature. The average yield estimate indicates that in general, as the streamflow volume increases, the yield increases. The efast results also show that the importance of input variables to the estimation of yield changes due to inflow variability. The lack of trend in the total output variance measures, V(Y), indicates that the model and the yield estimate are sensitive to inflow variability. The results given in this paper are only for the data set used and the system characteristics under

9 which the model was run. Future work will further the analysis of the yield estimate by considering various simulation lengths to provide insight into how simulation length affects the system and yield estimate behaviour. Acknowledgements The authors gratefully acknowledge Barwon Water Corporation, in particular Dr. Mee-Lok Teng for providing data and information of the Barwon water supply system. References Cukier, R.I., Fortuin, C.M., Shuler, K.E., Petschek, A.G. and Schaibly, J.H. (1973). Study of the Sensitivity of Coupled Reaction Systems to Uncertainties in Rate Coefficients. Part I: Theory, Journal of Chemical Physics, 59(8), pp Frey, H.C. and Patil, S.R. (2002). Identification and Review of Sensitivity Analysis Methods, Risk Analysis, 22(3), pp King, D.M. (2009). On the Importance of Input Variables and Climate Variability to the Yield of Urban Water Supply Systems. PhD Thesis, Victoria University, Melbourne, Australia. King, D.M. and Perera B.J.C. (2010). Sensitivity Analysis of Yield Estimate of Urban Water Supply Systems, Australian J. of Water Resources, 14(2), pp King, D.M. and Perera B.J.C. (Submitted). Morris Method of sensitivity analysis applied to assess the importance of input variables on urban water supply yield - a case study, Journal of Hydrology. Morris, M.D. (1991). Factorial Sampling Plans for Preliminary Computational Experiments, Technometrics, 33(2), pp Pappenberger, F. and Beven, K.J. (2006a). Ignorance is Bliss: Or Seven Reasons not to use Uncertainty Analysis, Water Resources Research, 42(5), WO5302, doi: /2005WR Pappenberger, F., Harvey, H., Beven, K.J., Hall, J. and Meadowcroft, I. (2006b) Decision Tree for Choosing an Uncertainty Analysis Methodology: A Wiki Experiment, Hydrological Processes, 20(17), pp Pappenberger, F., Iorgulescu, I. and Beven, K.J. (2006c). Sensitivity Analysis Based on Regional Splits and Regression Trees (SARS-RT), Environmental Modelling and Software, 21(7), pp Perera, B.J.C., James, B. and Kularathna, M.D.U. (2005). Computer software tool REALM for Sustainable Water Allocation and Management, Journal of Environmental Management, 77(4), pp Ratto, M., Young,.P.C., Romanowicz, R., Pappenberger, F., Saltelli, A. and Pagano, A. (2007). Uncertainty, Sensitivity Analysis and the Role of data Based Mechanistic Modeling in Hydrology, Hyrdology and Earth System Sciences, 11(4), pp Saltelli, A. and Bolado, R. (1998). An Alternative Way to Compute Fourier Amplitude Sensitivity Test (FAST), Computational Statistics and Data Analysis, 26(4), pp Saltelli, A, Chan, K. and Scott, E.M. (2000). Sensitivity Analysis. Wiley, Chichester. Saltelli, A., Tarantola, S., Campolongo, F. and Ratto, M. (2004). Sensitivity Analysis in Practice. Wiley, New York, NY. Sinclair Knights Merz (SKM). (2006) Central Region Sustainable Water Strategy, Configuration of a Basin Wide REALM Model for the Barwon/ Moorabool Basin. Prepared for Department of Sustainability and Environment, Malvern, Australia.