NEURAL NETWORK SIMULATION OF KARSTIC SPRING DISCHARGE

Transcription

1 NEURAL NETWORK SIMULATION OF KARSTIC SPRING DISCHARGE 1 I. Skitzi, 1 E. Paleologos and 2 K. Katsifarakis 1 Technical University of Crete 2 Aristotle University of Thessaloniki ABSTRACT A multi-layer perceptron (MLPs)-neural-network- back-propagation algorithm (BP) has been developed to simulate the daily discharges of two springs that lie in a karstic environment. The correlation of the precipitation and runoff series for different time lags was initially estimated to define the input and output parameters relation. The performance of several types of neural networks with different training functions was assessed based on a mean square error criterion between observed and simulated discharge values. The simulation results showed that our MLP network provides excellent predictions for karstic spring discharges, capturing both local minima and maxima, and resulting in low mean square errors.

2 1. INTRODUCTION The last two decades the application of artificial intelligence algorithms has found broad use in engineering applications [1, 2, 3, 4, 5]. Artificial intelligence refers to simulation techniques that aim to mimic the process of developing general rules from repeated trials and training from known cases, followed by testing of the rules validity in new cases. Neural networks hold promise for complex subsurface hydrologic problems, where the development of constitutive physical relations appears to be elusive. The use of black box models to describe complex processes is not new in hydrology, and in at least surface hydrology it has a long and productive record [6, 7]. Subsurface flow in karstic environments is characterized by a complex interaction of geological, physical, and chemical processes and its study appears to be a natural candidate for the application of neural networks. 1.1 Neural Network s philosophy Artificial neural networks simulate the constitution and function of biological neural networks. The model of an artificial neuron is comprised of a summing and an activation function. The worth of each input value is assessed through synaptic weights, and then, all the weighted inputs are added. To correct for a linearity assumption a distributive value of bias is further added to the summing function (equation 1). The result forms the argument of an activation function φ that acts as a filter and which yields the neuron s response as a single number (equation 2). For hydrologic processes the procedure for a single neuron k, where the input parameters j (e.g., precipitation, temperature, evaporation etc.) are given as time series, x j (t), can be described at each time interval as follows: and φ Here x j (t) is the input value of parameter j at time-step t; w kj (t) is the weight assigned by neuron k to the input value of parameter j at time t; φ is a nonlinear activation function; b k (t) is the bias of the k-neuron at time t, and y k (t) is the output signal from neuron k at time t. The Figure 1. The structure of an artificial neuron k

3 process can be repeated for all entries of the time series and yields an output vector y k. The procedure for one time step if a single artificial neuron were to be used is shown in Figure 1. Neurons can form layers that are fully interconnected creating networks. A typical network consists of three types of layers: a) input layer, b) hidden layer and c) output layer. The input layer refers to the available data that enter the system and the number of neurons that constitute this layer is equal to the number of the parameters that contribute to the simulation. The number of the network s hidden layers can be more than one, according to the problem s complexity. Finally, the output layer returns the output vectors, which are the final responses of the neural network [8]. A typical artificial neural network is shown in Figure 2. Figure 2. A typical artificial neural network The most important issue about artificial neural networks is the phase of learning, which is achieved through examples, mimicking the experience phase of human learning. There are three types of training: 1) Supervised - Associative, 2) Unsupervised - Self-organization and 3) Semi - supervised [8]. The process of learning refers to the adjustment of weights, through which the inputs are linearly related, in order to minimize the error between the network s prediction and the actual response. It is an iterative procedure that adjusts the synaptic weights values as the network gains extra knowledge, after each iteration. The learning rule is described through the following equation: where w kj represents the weight assigned by neuron k on parameter j, and Δw is the adjustment in this weight from time t n to time t n+1. The most common procedures for determination of the value are the [8, 9]: 1) Hebb rule: where is neuron s k output response, is neuron s j input signal, and η is the network s learning rate.

4 2) Delta rule: where is the error :. Here is the predicted value and is the actual response from a neuron k; is a neuron s j input signal, and η is the network s learning rate. 2. SITE DESCRIPTION The Municipality of Rouva is a semi-mountainous and agricultural area of 62,725 km 2 with a population of 2,324 inhabitants, situated at central-southern Crete in the prefecture of Heraklion, Greece. The region is characterized by a large number of natural springs, as Rouva lies at the hydrologic basin of Geropotamos. The area of Rouva is part of the hydrogeologic unit of the Psiloriti Talea Mountain, which is one of the main karstic systems in Crete. Simulating karstic aquifers with traditional mathematical methods, which are based on continuum approaches, is problematic and artificial neural networks have been used as alternative tools for these types of problems [10, 11]. The objective of the present study was to develop a neural network model to predict the discharge from two karstic springs in the region; the first spring is located at Mai Vrisi and the second at Pera Vrisi. The neural network utilized the measured rainfall and discharge data, which were collected on a daily basis between the years (until 14/07/09). 3. MODEL DEVELOPMENT Initially, in order to simulate the two karstic springs, it was necessary to set an appropriate neural network for the specific natural system. The type of network utilized in this study was a back propagation feed-forward Neural Network with one hidden layer. Selection of a single hidden layer was motivated by the fact that the network was to simulate discharges only, by means of precipitation data. A basic condition for training with back propagation was the availability of patterns and targets, since training was supervised during simulation of known cases [8]. Patterns were considered to be all the available data that entered the network, while targets were the responses that the network was to provide as output. In our study daily rainfall and discharge values formed the respective input and output vectors. In order to improve the network s efficiency an extra parameter was added in the input layer. This was a serial number that indicated the date that the precipitation and discharge data were obtained, designated as the day number. Finally, the number of neurons in the hidden layer was determined through trial and error. 3.1 Definition of the time lag An important issue in processing the available data is defining the right time lag [10] between precipitation and spring discharge. This designates the delay between input excitation and output response in a subsurface hydrologic system as water travels from recharge to discharge areas at a rate that depends on an aquifer s hydro-geologic characteristics and the intensity and duration of a precipitation event [12]. The determination of a suitable time lag contributes to the appropriate correlation of input and output variables and improves the efficiency of a neural network. The correlation coefficient of rainfall and discharge time series was used in order to define the time lag. The correlation coefficient is given by:

5 where A and B, two time series, i and i their components and and their average values, respectively. Calculation of the correlation coefficients for time lags from zero to thirty days yielded the appropriate time lag, taken as the one that provided the maximum correlation. 3.2 Selection of an optimum network In order to select an optimum network structure five different training algorithms were tested, comprising of one to ten neurons in a single hidden layer. Training algorithms adjusted weights and biases with the goal to minimize a performance function. In feed-forward neural networks the performance function to be minimized is taken to be the mean square error, between model-predicted output and actual response. The mean square error (MSE) is given by: where d i is the measured data, y i is the model prediction, and N the number of available samples. In each synapse between connected neurons the training function calculates the output error and determines the adjustments to the network s weights and bias. The five training algorithms that were tested were as follows [13]: 1. Gradient descent (GD) 2. Gradient descent with momentum (GDM) 3. Gradient descent momentum and an adaptive learning rate (GDX) 4. Levenberg-Marquardt (LM) 5. Bayesian regularization (BR) Tables 1 and 2 show in the fourth column the MSE from the five training algorithms as well as the number of neurons in the hidden layer that correspond to the minimum MSE values. TABLE 1. Mai Vrisi Spring: Optimum MSE values & neuron number in the hidden layer 1 newff, traingd, learngdm Η=9 MSE= (l/s) 2 2 newff, traingdm, learngdm Η=4 MSE= (l/s) 2 3 newcf, traingdx, learngdm Η=4 MSE= (l/s) 2 4 newcf, trainlm, learngdm Η=4 MSE=0.003 (l/s) 2 5 newcf, trainbr Η=3 MSE= (l/s) 2 The optimum value of MSE for the Mai Vrisi spring was obtained by the Levenberg Marquardt training algorithm with four neurons in the hidden layer and is shown with bold characters at the table above.

6 TABLE 2. Pera Vrisi Spring: Optimum MSE values & neuron number in the hidden layer 1 newff, traingd, learngdm Η=6 MSE= (l/s) 2 2 newff, traingdm, learngdm Η=9 MSE= (l/s) 2 3 newcf, traingdx, learngdm Η=8 MSE= (l/s) 2 4 newcf, trainlm, learngdm Η=4 MSE= (l/s) 2 5 newcf, trainbr Η=7 MSE=0.009 (l/s) 2 The optimum value of MSE for the Pera Vrisi spring was obtained again by the Levenberg Marquardt training algorithm with four neurons in the hidden layer and is shown with bold characters at the table above. The Levenberg Marquardt training algorithm is a popular method that is simple, fast, and efficient [14, 15] for determining a function s minimum via the Gauss-Newton and Steepest Descent methods. It is used in moderate-sized feed-forward neural networks [13]. Based on these tests the same neural network structure was used for both springs, and this is depicted in the following figure: 4. RESULTS Figure 3. Optimum neural network structure for both springs 4.1 Optimum network response The optimum neural network architecture and training parameters resulted from testing five different algorithms in a Matlab environment. The available data were normalized between the values of 0 and 1, and then they were portioned into three subsets: a) the training set, b) the validation set, and c) the testing set. The validation phase in learning is needed to supervise the performance of the trained neural network, and to assess the network s behavior with other sets during simulation. The results of the neural network containing the training, validation and testing periods are presented in Figures 4 and 5, for the two springs, respectively. The values of normalized testing mean square error were found to be equal to (l/s) 2 and (l/s) 2 for the Mai Vrisi, and the Pera Vrisi springs, respectively. The neural network was able to capture in a very satisfactory way the trends as well as predict the observed discharge data.

7 Spring Discharge (l/s) Spring Discharge (l/s) Mai Vrisi Spring training validation testing Measured Data Model Prediction Data Points Figure 4. Measured Data and Model Prediction for Mai Vrisi Spring Discharge Pera Vrisi Spring training validation testing Measured Data Model Prediction Data Points Figure 5. Measured Data and Model Prediction for Pera Vrisi Spring Discharge 5. CONCLUSION Neural networks provide a promising alternative in flow problems where the applications of models that are based on physical constitutive relations, developed through a continuum assumption appear to be questionable. Studies of carbonate karstic aquifers, where there exists a complex interaction of geologic, hydrologic, and geochemical processes on the geometry of the porous medium and the flow in it, suggest that artificial neural networks result in excellent simulation results [10, 11]. The optimum structure of a neural network, determination of input and output vectors, the number of hidden layers, as well as the number of neurons in each hidden layer, are significant issues that affect a network s performance. Hydrologic problems with many variables require complicated neural networks with many neurons in the hidden layers in order to provide an adequate prediction. Our study dealt with the simulation of daily spring discharge in a karstic environment in Crete. The appropriate network structure was found by testing five different types of

8 algorithms in the Matlab environment and the Levenberg-Marquardt algorithm proved to be the most efficient algorithm, for the two springs considered. The optimum network resulted in simulated discharges that followed closely the patterns of the measured discharge data, and provided predictions of high accuracy. Ongoing research includes investigation of the effect of additional parameters to the input space, with the aim to improve model prediction. Furthermore, we plan to examine the optimization of a neural network s behavior by using as input the precipitation series that correspond to the maximum correlation coefficient for different time lags. ACKNOWLEDGEMENT The authors are indebted to N. Darivianakis for providing the field data. REFERENCES 1. Katsifarakis K.L. and Z. Petala (2006) Combining genetic algorithms and boundary elements to optimize coastal aquifers management, Journal of Hydrology, Vol. 327 (1 2), pp Karamperidou C. (2007) Coastal Aquifer s Management with Artificial Neural Networks, M.Sc. Thesis, Department of Civil Engineering, Aristotle University Thessaloniki. 3. Chen J. and B. J. Adams (2006) Integration of artificial neural networks with conceptual models in rainfall-runoff modeling, Journal of Hydrology, Vol. 318, pp Affandi A., K. Watanabe and H. Tirtomihardjo (2007) Application of an artificial neural network to estimate groundwater level fluctuation Journal of Spatial Hydrology, Vol. 7 (2), pp Hu Z., G. Huang, and C. Chan (2003) A fuzzy process controller for in situ groundwater bioremediation, Engineering Applications of Artificial Intelligence, Vol. 16, pp Abbott M.B. and J.C. Refsgaard (1996) Distributed Hydrological Modelling, Kluwer Academic Publishers. 7. Singh, V. P., and D. A. Woolhiser (2002) Mathematical Modeling of Watershed Hydrology, Journal of Hydrologic Engineering, Vol. 7 (4), pp Haykin S. (1994) Neural Networks: A Comprehensive Foundation, Macmillan Publishing Company, N.Y. 9. Hebb D.O. (1949) "The organization of behavior - A neurophysiological theory", Wiley. 10. Trichakis I., I. Nikolos, G. Karatzas (2009) Optimal selection of artificial neural network parameters for the prediction of a karstic aquifer s response Hydrological Processes, Vol. 23, pp Lallahem S. and J. Mania (2003) A nonlinear rainfall-runoff model using neural network technique: example in fractured porous media Mathematical and Computer Modeling, Vol. 37, pp Hao Y., D. Huang and C. Hu (2005) Response of Karst Spring Discharge to Precipitation in Semiarid Region of China Systems, Man and Cybernetics, 2005 IEEE International Conference, Vol. 2, pp Mathworks, Matlab, R2008a, Tutorial. 14. Affandi A., K. Watanabe, H. Tirtomihardjo (2007) Application of an artificial neural network to estimate groundwater level fluctuation Journal of Spatial Hydrology, Vol. 7 (2), pp Hagan M. T. and B. M. Menhaj (1994) Training Feedforward Networks with the Marquardt Algorithm IEEE Transactions on Neural Networks, Vol. 5 (6), pp