Application of Hybrid FFNN-Genetic Algorithm for Predicting Evaporation in Storage Dam Reservoirs

Transcription

1 AGRICULTURAL COMMUNICATIONS, 2014, 2(4): Application of Hybrid FFNN-Genetic Algorithm for Predicting Evaporation in Storage Dam Reservoirs MOHAMMAD ALI IZADBAKHSH 1 AND HOSAIN JAVADIKIA 2 1 Department of Irrigation, College of Agriculture, Kermanshah Branch, Islamic Azad University, Kermanshah, Iran. 2 Department of Mechanical Engineering of Agricultural Machinery, College of Agriculture, Razi University, Iran. *Corresponding Author: maizadbakhsh@yahoo.com (Accepted: 3 July 2014) ABSTRACT Evaporation is one of the most important parameter to manage dam reservoirs water regulations. Average annual evaporation losses from storage dam reservoirs in arid and semi-arid areas are about 2 meters. It is necessary to predict how much evaporation losses occur from water surface of dam reservoirs. For estimating the evaporation, direct measurement methods such as pan evaporation can be used. Pan evaporation is one of the popular instruments to measure evaporation. Direct measurement methods required installing meteorological stations in various stations for measuring evaporation. In this study, the evaporation was estimated by using hybrid FFNN - Genetic Algorithm system. Primary model was simulated based on Feed forward and then the model structure was optimized through Genetic Algorithm. Daily data from the meteorological station in Kermanshah statistical period was used as input to the model. Network inputs include minimum and maximum temperature, wind speed and sunshine hours. The best structure had only one hidden layer with 14 neurons and it had mean square error (MSE) with correlation coefficient of (R) Carefully fitted coefficients (R 2 ) for this model was 0.9. The results show the suitable capability and acceptable accuracy of hybrid FFNN Genetic Algorithm system in estimation of daily evaporation. Keywords: Feed forward, genetic algorithm, hybrid method, neural network, pan evaporation. Abbreviations: ANN: Artificial Neural Networks; ETO: one neuron; FFNN: Feed Forward Neural Network, GA: Genetic Algorithms; MAE: Mean Absolute Error, MLP: Multilayer Perceptron; MSE: Mean Squared Error; MSNE: Meansquared Normalized Error; R: correlation coefficient. INTRODUCTION Evaporation is transferred as water vapour to atmosphere. Evaporating is an important parameter in designing storage dam reservoirs. It is necessary to predict how much evaporation losses occur from water surface of dam reservoirs. Research on application of artificial intelligence to estimate daily pan evaporation indicates that Genetic Algorithms method is more precise considering other methods. (Shiri et al 2011). Research show that The Artificial Neural Networks (ANN) method gave better estimates than the conventional method that requires wind speed and humidity data (Rahimi Khoob, 2008). Evaporation pan (class A pan, US Weather Bureau) is used to estimate evaporation (Irmak et al., 2002). In another research Bruton et al. (2000) estimated pan evaporation by application of artificial neural networks. Daily pan evaporation was also estimated using multiple linear regression and the Priestley-Taylor method and was compared to the results of the ANN models. The measured variables included daily observations of rainfall, temperature, relative humidity, solar radiation, and wind speed. They found that the ANN model of daily pan evaporation with all available variables as inputs is the most accurate with root mean square error of 1.11 mm for the independent evaluation data set. There are several equations to calculate evaporation that depending upon weather data. Artificial Intelligence has shown an acceptable ability for modelling complex and nonlinear systems. An alternative approach to process dynamic modelling is the application of Artificial Neural Networks (ANN). ANN is composed of adaptive non-linear simple processing elements called neurons or nodes equivalent to neurons in a 30 biological system capable of performing parallel computations for data processing (Hertz et al., 1991). Genetic algorithms (GAs) belong to a family of algorithms called evolutionary algorithms. Other major algorithms of this family include

2 AGRICULTURAL COMMUNICATIONS. evolutionary programming, evolutionary strategies and genetic programming. Evolutionary algorithms have been inspired from the principles of evolution in the nature. Genetic algorithms were developed by Holland, his colleagues and students in the 1960s and 1970s (Holland, 1975). Genetic Algorithm (GA) is an iterative random search algorithm for nonlinear problem based on mechanics of natural selection and natural genetics (Liang and, 1977). A neural network can predict true outputs for corresponding inputs only when network parameters are selected properly. Link weight is one of the network parameters. A process in which proper values for weights are selected is called network training. Back propagation is a neural network training method. In this method, inputs are presented to neural network and then network output is calculated; the difference between desired outputs (target outputs) and network outputs (real outputs) shows network error. At the next step, value of weights is revised according to network error. Each interaction of revising weights values is called an epoch. Neural Network training can be considered as a minimization problem which seeks for weights that minimize network error index (Haykin, 1994). Genetic algorithm is an evolutionary method which is inspired from evolution of creatures in nature. In this method chromosomes and genes are utilized to solve problems. First, solutions are coded into chromosomes. Then an initial generation of chromosomes is produced. In the next steps, new generations of chromosomes are produced using genetic operators and positive characteristics of each generation are transferred to new generations (Haupt, 2004). Moreover, antecedent moisture conditions are always changing and depend upon both present and past hydrological complex processes and large volumes of data. In particular, a back propagation neural network (BPNN) is useful for handling real-time, non-stationary and non-linear natural phenomena (Nishimura and Kojiri, 1996). In this study, the neural network structures were optimized by use of a hybrid method. Efficiency of hybrid feed forward and genetic algorithms were examined in estimation of evaporation using meteorological data of minimum and maximum temperature, wind speed and sunshine hours in order to predict evaporation of a dam reservoir in Kermanshah, Iran. MATERIAL AND METHODS Daily meteorological data of a 14 years period (from 1994 to 2007) were obtain from Kermanshah Meteorological Stations in 47 9 longitude and latitude. Weather records of solar radiation or sunshine duration, minimum and maximum air temperature, humidity (minimum and maximum relative humidity) and wind speed were obtained from the mentioned station and used as input data. To ensure the integrity of computations, the weather measurements were made at 2m (or converted to that height). Then, each of the features was normalized into range of [0 1] by using the following formula: Where x is the data which should be normalized, the x max and x min are maximum and minimum of the original data respectively and x norm is the normalized data which transformed. After the standardization of data, 70% of the data were used for network training and the rest of 30 % were used for network testing. In during of study, estimating error of testing were done. By using these calculated values as target outputs various for feed forward-genetic algorithm was trained. Finally, the capabilities of the system were analysed. Artificial Neural Network: Neural networks are composed of simple elements operating in parallel. These elements are inspired by biological nervous systems. As in nature, the connections between elements largely determine the network function. You can train a neural network to perform a particular function by adjusting the values of the connections (weights) between elements. Different types of artificial neural network are available [e.g. support vector machine (SVM), self-organization map (SOM), and multilayer perceptron (MLP)]. The third one (MLP) is the most widely used (Assidjo et al., 2008). A multilayer feed-forward neural network, have three type layers that including input, hidden layer and output. The first layer has weights coming from the input. Each subsequent layer has a weight coming from the previous layer. All layers have biases. The last layer is the network output. A MLP can have several layers. Each layer has a weight matrix (W), a bias vector (b), and an output vector (a). To distinguish between the weight matrices, output vectors, etc., for each of these layers we append the number of the layer as a superscript to the variable of interest (Demuth and Beale, 2002). Adaption is done with trains, which updates weights with the specified learning function. Training is done with the specified training function. Performance is measured according to the specified performance function. For creating a multilayer feed-forward neural network we can do some settings to get best result of modelling, for example number of hidden layer, number of neurons in each hidden layer, transfer function in each layer, training function, learning function, performance function and number of epochs. If it is 58

3 IZADBAKHSH AND JAVADIKIA assumed that in best result of modelling, network maybe have maximum 8 layers and maximum 20 neurons in each layer so with attention to the number of other settings for multilayer feedforward neural network, and the number of inputs in our research, it estimated that we have 4 20^ different networks to test. In fact we have almost networks with different settings. Absolutely it is not possible to test all of these networks manually and also with machine (PC) it will take a lot of times, for example for testing 2000 multilayer feed-forward neural networks by means of machine, it takes 4 hours, so it s possible to calculate that for all of networks how long it maybe need time. To solve the mentoned problem we can use Genetic Algorithm. The input layer consists of four neurons (max and min temperature, sunshine, wind speed), and the output layer contains one neuron (ETO). Totally data were experimentally collected and used. In this research, we designed a chromosome with 20 Genes. The first gene represents the number of inputs, because we tried to enter the influence of each input parameters to modelling the output. Genetic Algorithm: A genetic algorithm (GA) is a search heuristic that mimics the process of natural evolution. This heuristic is routinely used to generate useful solutions to optimization and search problems. Genetic algorithms belong to the larger class of evolutionary algorithms (EA), which generate solutions to optimization problems using techniques inspired by natural evolution, such as inheritance, mutation, selection, and crossover. In a genetic algorithm, a population of strings (called chromosomes or the genotype of the genome), which encode candidate solutions (called individuals, creatures, or phenotypes) to an optimization problem, evolves toward better solutions. The evolution usually starts from a population of randomly generated individuals and happens in generations. In each generation, the fitness of every individual in the population is evaluated, multiple individuals are stochastically selected from the current population (based on their fitness), and modified (recombined and possibly randomly mutated) to form a new population. The new population is then used in the next iteration of the algorithm. Commonly, the algorithm terminates when either a maximum number of generations has been produced, or a satisfactory fitness level has been reached for the population. If the algorithm has terminated due to a maximum number of generations, a satisfactory solution may or may not have been reached (Haupt and Haupt, 2004). The block diagram of GA and ANN actions is shown in Fig. 1. In order to decrease model complexity and also improve model generalization ability, the optimum layout of hidden layer connection links is designed using genetic algorithm. During optimization process, some links will be omitted. This could change model to a partially connected neural network. Fig. 2 depicts the optimization process. Fig. 1. Block diagram of genetic algorithm and artificial neural networks. In this research, we designed a chromosome with 20 genes. The first gene represents the number of inputs, because we tried to enter the influence of each input parameters to modelling the output. It means that GA will remove unvalued inputs. 2 to 9 genes represent two different types of important information for structure of ANN that are the number of hidden layers of the network and the number of neurons of each hidden layer, which ranged from 1 to 20 neurons. 10 to 17 genes specify type of transfer function for each hidden layer, which ranged from 0 to 12. Transfer functions applied were Competitive, Hard limit, Symmetric hard limit, Log-sigmoid, Inverse, Positive linear, Linear, Radial basis, Saturating linear, Symmetric saturating linear, Hyperbolic tangent sigmoid and Triangular basis. Eighteenth gene depicts the type of training, which ranged from 0 to 18. Training functions included batch training with weight and bias learning rules, quasi-newton back propagation, quasi-newton back propagation for use with NN model reference adaptive controller, Bayesian regularization, Batch unsupervised weight/bias training, cyclical order incremental update, Powell-Beale conjugate gradient back propagation, Fletcher-Powell conjugate gradient back propagation, Polak-Ribiere conjugate gradient back propagation, Gradient descent back propagation, Gradient descent with adaptive learning rule back propagation, Gradient descent 59

4 AGRICULTURAL COMMUNICATIONS. with momentum back propagation, Gradient descent with momentum and adaptive learning rule back propagation, Levenberg-Marquardt back propagation, One step secant back propagation, random order incremental training with learning functions, resilient back propagation, sequential order incremental training with learning functions and Scaled conjugate gradient back propagation. Nineteenth gene specifies the Learning, which ranged from 0 to 14. Learning functions were Conscience bias, Gradient descent weight/bias, Gradient descent with momentum weight/bias, Hebb weight, Hebb with decay weight learning rule, Instar weight, Kohonen weight, LVQ1 weight, LVQ2 weight, Perceptron weight and bias, Normalized perceptron weight and bias, Selforganizing map weight, Batch self-organizing map weight and Widrow-Hoff weight and bias learning rule. Twentieth gene determines the number of epochs to train of each network, which ranged from 10 to 500. Some statistical parameters such as mean squared error (MSE), mean-squared normalized error (MSNE), mean absolute error (MAE), correlation coefficient (R) and P value (P), were calculated and compared with the performance of Feed Forward Neural Network (FFNN). About the GA settings, the best generation number was set to 50 (Heckerling et al., 2004; Izadifar and Zolghadri Jahromi, 2007; Mohebbi et al., 2008, Mohebbi et al., 2010) Therefore, the termination criterion of 100 was chosen. The Tournament selection with size of 5 based on ranking algorithm was applied for the selection operator. Two point and constraint dependent default is selected for cross solver function and mutation operators, respectively. All of optimization program was developed in m-file of MATLAB software by version of RESULTS AND DISCUSSION: In this research FFNN (Feed Forward Neural Network) was programmed in m-file of MATLAB software to model the ETO, and it was optimized with Genetic Algorithm. In another word, the Genetic Algorithm supervised the Feed Forward Neural Network and it was parallel with FFNN. It is necessary to test all the settings of FFNN, but checking all the settings (about networks) took a lot of times. So the Genetic Algorithm was used to get the best networks in very little time and with only checking the 5000 networks. It took almost 5 hours with a notebook [core 2 Duo CPU 2.20 GH GH)]. The result of program was a network with only one hidden layer that had 14 neurons, W1, W2 and B1, B2 are the weights and biases of network, respectively. W1 and B1 are the matrixes of hidden layer and W2 and B2 are for output layer that the row number of matrixes indicates the number of neurons in each layer and the column number indicates the number of inputs. Fig. 2. The process of optimizing the studied network. The structure of the best obtained network is shown in Fig. 3. The properties of the best network are mentioned in Table 1. The changes of some statistical parameters (MSE, MSNE, MAE and R) is shown in Table 2, and plot regression of each generate in genetic algorithm is shown in Fig W = = `= =

5 IZADBAKHSH AND JAVADIKIA Fig. 3. The structure of the best network obtained by GA. Inputs Table 1. Properties of best network obtained by GA First hidden layer Output layer Network Neurons Transfer Neurons Transfer Training 4 14 Hyperbolic tangent sigmoid 1 Hyperbolic tangent sigmoid Levenberg marquardt back propagation Weight Bias Learning Gradient descent with momentum Performance Mean squared error Epochs 277 Table 2. The results of the best network to test the practical data. Result of the best network with obtained weight/bias MSE MSNE MAE r R 2 p In a study. Chung et al. (2012) estimate pan evaporation with hybrid ANFIS (Adaptive Neural Fuzzy Inference Systems) and Kriging. The daily estimation of evaporation obtained from hybriding ANFIS and kriging model provides a root mean square error (RMSE) of 1.09 mm day -1, whereas in this paper, the estimation accuracy of the model was 0.14 mm day -1. Jadeja (2011) estimate reference evapotranspiration from pan evaporation with artificial neural network. Selected ANN model estimated the daily ETO with a R 2 value of 0.84 and RMSE of 1.46 mm day -1, whereas in this paper, R 2 was 0.99 mm day -1. CONCLUSION Artificial neural networking is one of the methods to estimate evaporation. Hybrid FFNN- Genetic Algorithm is developed for accurately estimating pan evaporation at ungagged sites. But it takes a long time to find the best network structure. Therefore the neural network structures were optimized by use of genetic algorithm. In this research, the FFNN-Genetic Algorithm is used to predict evaporation. Evaporation is modelled via FFNN and the GA optimized the FFNN to get best result. In other worlds, the genetic-neural network to modelling was created. The best model had only one hidden layer with 14 neurons and it had MSE, MSNE, and MAE of , and respectively. Correlation coefficient and coefficient of determination (R 2 ) for this model were 0.99 and 0.98, respectively. The results had the acceptable accuracy of hybrid FFNN Genetic Algorithm system by using input data include minimum and maximum temperature, wind speed and sunshine hours to estimate of daily evaporation. Hybrid FFNN-Genetic Algorithm model showed very good performance when compared with the other research. From the above discussion, it was inferred that the selected artificial neural network model are reliable approaches for estimation of evaporation from dam reservoirs and lakes in semi-arid areas to manage water regulations. 61

6 AGRICULTURAL COMMUNICATIONS. ` ACKNOWLEDGEMENTS The authors greatly acknowledge the financial and scientific support of Islamic Azad University, Kermanshah Branch through a research project. Fig. 4. Plot regression of the best network to test the practical data. Assidjo, E. Yao, B. Kisselmina and Amane, K Modelling of an industrial drying process by artificial neural networks. Brazilian Journal of Chemical Engineering. 25(3): Bruton, J.M., R.W. Mc Clendon and G. Hoogenboom Estimating daily pan evaporation with artificial neural network. Journal of the American Society of Agricultural Engineers. 43(2): Chung, H.A Spatial neural fuzzy network for network for estimating pan evaporation at ungauged sites. Earth System Sciences. 16(1): Demuth, H and M. Beale Neural Network Toolbox for use with MATLAB. The Math Works Inc. Natick, MA, USA. 310 p. Haupt, R.L. and S.E. Haupt Practical genetic algorithms. John Wiley and Sons Inc. Hoboken, New Jersey, USA. pp: Haykin, S Neural Networks. A Comprehensive Foundation. New York, McMillan. USA. pp: Heckerling, P., B.S. Gerber, T.G. Tape and R.S. Wigton Use of genetic algorithms for neural networks to predict community-acquired pneumonia. Artificial Intelligence in Medicine. 30: Hertz, J., A. Krogh and R.G. Palmer Introduction to the theory of neural computation. Addison Wesley. USA. pp: Holland, J.H Adaptation in natural and artificial systems. University of Michigan Press. Ann Arbor, USA. pp: Irmak, S., R.G. Allen and E.B. Whitty Daily grass and Alfalfa reference evapotranspiration estimates and alfalfa-to-grass evaporation ratios in Florida. Jarring Drain Engineering. 129(5): Izadifar, M. and M. Zolghadri Jahromi Application of genetic algorithm for optimization of vegetable oil hydrogenation process. Jpurnal of Food Engineering. 78: 1-8. Jadeja, V Artificial neural network estimation of reference evapotranspiration from pan evaporation in a semi-arid environment. National conference on recent trends in engineering and technology May 2011, B.V.M. Engineering College, V.V. Nagar, Gujarat, India. REFERENCES Liang, B. and S.U. Pillai Two-dimensional blind deconvolution using a robust GCD approach Proceedings. International Conference on Image Processing ICIP 97, vol. 1, pp Mohebbi, A., M. Taheri and A. Soltani A neural network for predicting saturated liquid density using genetic algorithm for pure and mixed refrigerants, International journal of Refrigerating. 31: Mohebbi, M., F. Shahidi, M. Fathi, A. Ehtiati and M. Noshad Prediction of moisture content in preosmoses and ultra-sounded dried banana using genetic algorithm and neural network. Journal of Food and Bio Products Processing. 13: Nishimura, S. and T. Kojiri Real-time rainfall prediction using neural network and genetic algorithm with weather radar data, 10 th Congress of the Asia and Pacific Division of the International Association for Hydraulic Research. pp: Rahimi Khoob, A Artificial neural network estimation of reference evapotranspiration from pan evaporation in a semi-arid environment. Irrigation Science. 27(1): Shiri, J. and O. Kisi, O Application of Artificial Intelligence to Estimate Daily Pan Evaporation Using Available and Estimated Climatic Data in the Khozestan Province (South Western Iran). Journal of Irrigation and Drain Engineering. 137(7):