TRAINING FEED FORWARD NEURAL NETWORK USING GENETIC ALGORITHM TO PREDICT MEAN TEMPERATURE

Transcription

1 IJRRAS 29 (1) October TRAINING FEED FORWARD NEURAL NETWORK USING GENETIC ALGORITHM TO PREDICT MEAN TEMPERATURE Manal A. Ashour 1,*, Somia A. AlZahby 2 & Mahmoud I. Abdalla 3 1,2 Al-Azher University, Faculty of Science, Math. Department 3 Zagazig University, Faculty of Engineering, Electronics and communication Department ABSTRACT The genetic algorithm has been used to train fixed neural networks to predict the mean temperature. In this study the implementation of a genetic algorithm to neural network for the weather parameters prediction is considered. Various genetic operators to design a genetic algorithm are examined, with an objective of minimizing the cumulative square error. The effect of the crossover and mutation operators and population size are taken into account. By computer s simulations, the effectiveness of different crossover and mutation operators in training the network are shown. The appropriate crossover operators are described with the different selection methods which are used. A design of experiment (DOE) approach is used to find the value of the genetic algorithm parameters through determining the probability of the crossover, the probability of mutation, and the population size. p c This study discusses three cases. The first case is when the roulette wheel selection RWS method and power mutation with SPOX, TPOX, AMOX, HOX, BLX, and LX crossover operators are applied one at a time. The second case is when the tournament selection TOR method and power mutation with the previously mentioned crossover operators are applied one at a time. And finally the third case is when the rank based fitness BRF method and power mutation with the previously mentioned crossover operators one at a time. Results show improved performance of TPOX+TOR+POWRE model over other models. Keywords: artificial neural network (ANN), genetic algorithm (GA), genetic algorithm operators, genetic algorithm parameters, design of experiment (DOE). p m p s 1. INTRODUCTION Neural networks and genetic algorithms are two techniques for optimization and learning, each with its own strength and weaknesses. Each technique has generally evolved along separate paths. This study was presented by a hybrid method consists of two intelligent systems; artificial neural networks and genetic algorithms. It is an effective system in solving various issues. Every system suffers from some problems which reduce its efficiency so these systems were merged with each other to take the system benefits and get rid of its disadvantages. The hybridization method resulted from the combination of neural networks and genetic algorithms. There are many ways to apply GAs to neural networks. Davis Montana Lawrence [1] took the first approach evolving the weights in a fixed network. They were using GA instead of backpropagation to find the network weights for a fixed set of connections. Genetic algorithm ensures the best weights to be used to train the network and make them converge faster and reduce the error between desired output and the actual output and hence increases the efficiency of the network. The hybridization of artificial neural networks with the genetic algorithm make in many forms and different ways. However, we will focus on the hybridization process through adjusting the network weights. 2. ARTIFICIAL NEURAL NETWORK In its simplest form, the network is made up of cells. These cells begin its work when it receives signal that comes from another cell. Neuron receives that signal through the dendrites. Neural network consists of a group of cells that communicate with each other. Each cell of these cells has the ability to calculate weights coming to it from the previous cells plus the ability to compute output of each cell. The weights can usually be excite or inhibit; that it may strengthen the signal (if positive weight) or it may weaken the signal (if negative) that come to it. Network training acquire (adjustable weights) by knowing the difference between the desired and actual output. 3. GENETIC ALGORITHM Genetic algorithm is one of evolutionary algorithms methods that depends on the nature of the tradition of the work of Darwin's perspective. Genetic algorithm can be classified as a global search heuristics and it is a certain class of evolutionary algorithms. Genetic algorithms are considered to be one of the important techniques in the search for the perfect choice of set solutions available for a particular design. Genetic algorithm is a search algorithm relies on repetition, starts with a population of individuals (solutions) which generated randomly to reach the optimal solution by genetic operators in each genetic cycle; it produces children to be good solutions that are replaced with bad 19

2 IJRRAS 29 (1) October 216 solutions. The idea of the genetic algorithm is to generate some random solutions and then examine these solutions and comparing with certain criteria set by the algorithm designer. Genetic algorithms are implemented as a computer simulation where chromosomes are used as individuals in operations which are carried out to find the best solutions. In general chromosome represents by binary ( and 1), real encoding and by other encoding methods. The process of evolution usually starts from selecting chromosomes at random and this happens in other generations. In every generation, fitness function is calculated for each individual (chromosome) then chooses the best individuals which have the best fitness values pursuant to rule "the survival of the fittest". The individuals then undergo crossover and mutation operators. With the passage of generations, chromosomes fitness reaches to a high level. Termination is the criterion by with the genetic algorithm decides whether to continue searching or to stop the search. The termination condition is checked after each generation to see if it is time to stop the termination condition. Chromosomes breeding process pass through three important stages: (1) parent selection, (2) crossover operator and (3) mutation operator. It should be noted that the method of chromosome reproduction is the force point of the algorithm because it reaches the search to the global solutions; it shall not be suspended in local solutions what is the case in most of the known methods of the survey. 3.1 SELECTION OPERATOR Reproduction process begins by choosing the parents and the only measure in this choice is fitness. Although the selection operators like other operators are randomly, however, the selection of particular chromosome opportunities closely linked to its fitness. Most chromosomes fitness to be, a candidate for selection many times will be. The weak chromosome fitness may not be selected at all. During each successive generation, there is a percentage of the current chromosome that is selected to produce a new generation. There are many methods to select the best chromosomes [2]; tournament selection [8], roulette wheel selection [3], rank based roulette wheel selection [4]. 3.2 CROSSOVER OPERATOR It presents reproduction process. After parent selection, crossover takes place. Crossover operator mates the parents to produce offspring. This process does not take place with all parents, but they are defined by the designer. It may apply to % of parents and do not apply to the rest. The crossover is controlled by a crossover probability ( ) [9]. Depending on the applied crossover operator the number of parents and offspring may vary. In this study we use single point crossover operator (SPOX) [1], two point crossover (TPOX) [16, 17], arithmetic crossover operator (AMOX) [1, 11], laplace crossover operator (LX) [12] and BLX crossover operator (BLX) [18] as a crossover operators. 3.3 MUTATION OPERATOR With some low probability, a portion of the new individuals will have some of their bits flipped. This process is important because it adds some new features that may not exist in the parents. This study uses power mutation (PM) as a mutation operator [13]. 4. DESIGN OF GENETIC ALGORITHM PARAMETERS Because of the various parameters of the genetic algorithm, it is difficult to apply the genetic algorithm to successfully solve some particular global search problem unless a proper choice of the GA parameters is considered. An approach for this choice is by using a design of experiments (DOE) method employing pilot genetic algorithm runs [14]. It can be noted that the optimization of different GA parameters itself is a global search and one that must be dealt in the problem domain of interest before the probed GA is applied. In order to illustrate the use of DOE in GA parameterization the case of network configuration is used. The objective is the minimization of the cumulative squared error. Table 1 gives results obtained from 8 different values of the three parameters together with the corresponding cumulative square error. These results are obtained by using SPOX as a crossover operator and Power mutation as a mutation operator. Table 1 Design of experiment Exp. No p c p m p Calculated output MSE Fitness Exp. M. s p c 2

3 IJRRAS 29 (1) October 216 ps=3, pc=.7, cumulative square error cumulative square error ps Figure 2. Effect of population size on GA pc Figure 3. Effect of crossover operator on GA pm=.1, cumulative square error Figure 4. Effect of mutation operator on GA pm Initialization; generate an initial population P consists of p s individuals. Compute the fitness function (the inverse of the cumulative square error) F of individuals in P DO DO Select two random individuals. (Step 1) Generate a random number prob between (,1) for each selected pair. 1 prob < p c If ( ) then 1 Perform crossover operator on the selected individuals. Compute the fitness function of offspring. End if Otherwise; go to step1 Until Generating a population of of offsprings. Do Generate a random number If ( prob < 2 p m ) then ps Perform mutation operator to the individual in P ' Pop Compute the fitness function of the mutated individual End if Otherwise; go to step 2. Until End of the population P. prob between (, 1) for each individual. (Step 2) 2. F m. Augment the population of offsprings P` and P together. Select the individuals with the lowest fitness to get a new population with p s individuals. Use the new population as the initial population P. Until The maximum number of generation is reached. Algorithm 1 The proposed algorithm. RESULTS AND DISCUSSION Starting with randomly 3 networks and applying the different operators produce next generations of networks. A new generation is obtained from a current generation by applying SPOX, TPOX, AMOX, HOX, BLX, and LX together with power mutation operators. The generation process continues. Fitness function is measured for each of the 2 generations and hence the minimum along the 2 is obtained. This process of generating and evaluating the 2 generations with their minimum values is considered as a run. 21

4 IJRRAS 29 (1) October SPOX RWS Pow er operators Actual Calculated (a) TPOX RWS Pow er operators Actual Calculated (b) AMOX RWS Power operators HOX RWS Power operators Actual Calculated Actual Calculated (c) (d) BLX RWS Power operators Actual Calculated LX RWS Power operator Actual Calculated (e) (f) Figure. The predicted and the actual mean temperature (RWS+Power) 22

5 IJRRAS 29 (1) October 216 SPOX Tor Power TPOX Tor Power operators Actual Calculated Actual Calculated (a) (b) 3 3 AMOX Tor Power operators 3 3 HOX Tor Pow er operators Actual Calculated Actual Calculated (c) (d) LX Tor Power operators Actual Calculated BLX Tor Power operators Actual Calculated (e) (f) Figure 6. The predicted and the actual mean temperature (TOR+Power) 23

6 IJRRAS 29 (1) October 216 SPOX BRF Pow er operators Actual Calculated (a) TPOX BRF Power operators Actual Calculated (b) 3 3 AMOX BRF Power operators 3 3 HOX BRF Power operators Actual Calculated (c) Actual Calculated (d) BLX BRF Power operators Actual Calculated LX BRF Power operators Actual Calculated (e) (f) Figure 7. The predicted and the actual mean temperature (RBF+Power) Table 2 shows mean error, mean absolute error, root mean square error, prediction error and correlation coefficient for the testing data using roulette wheel and power mutation as the selection operator and the mutation operator respectively. 24

7 IJRRAS 29 (1) October 216 Table 2 comparison of the performance of forecasting models for mean temperature parameter (Roulette Wheel Selection) Techniques Bias MAE RMSE PE CC SPOX+RWS TPOX+RWS AMOX+RWS HOX+RWS BLX+RWS LX+RWS Table 3 shows mean error, mean absolute error, root mean square error, prediction error and correlation coefficient for the testing data using tournament and power mutation as the selection operator and the mutation operator respectively. Table 3 comparison of the performance of forecasting models for mean temperature parameter (Tournament selection) Techniques Bias MAE RMSE PE CC SPOX+TOR TPOX+TOR AMOX+TOR HOX+TOR BLX+TOR LX+TOR Table 4 shows mean error, mean absolute error, root mean square error, prediction error and correlation coefficient for the testing data using rank based fitness and power mutation as the selection operator and the mutation operator respectively Table 4 comparison of the performance of forecasting models for mean temperature parameter (Rank Based Fitness selection) Techniques Bias MAE RMSE PE CC SPOX+RBF TPOX+RBF AMOX+RBF HOX+RBF BLX+RBF LX+RBF From the above tables, the bias for testing data is the least for TPOX+TOR model than that obtained from the other models. The MAE further explains that TPOX+TOR model is more precise than other models. TPOX+TOR model shows a smaller value for RMSE as compared to those of the other models. The PE obtained for testing data from BLX+TOR, LX+RWS, HOX+RWS, LX+TOR, TPOX+RWS, LX+RBF, AMOX+RBF, AMOX+RWS, AMOX+TOR, BLX+RWS, HOX+RBF, HOX+TOR, BLX+RBF, TPOX+RBF, SPOX+RWS, SPOX+RBF and SPOX+TOR models (power mutation) is.11726,.1166,.11194,.1131,.1836,.186,.1197,.91291,.984,.983,.8648,.83614,.7861,.7721,.76326, and.619 respectively, while for TPOX+TOR is which is the least and indicating it as precise prediction model. Further, the correlation coefficient is found to be highest for TPOX+TOR+POWER model (with the learning rate of.7 and with momentum.3) in comparison to other models. Thus, the graphical representation as well as the numerical estimates both favored, TPOX+TOR+POWER model as preferred performance in comparison to the other models, concluding that this technique can be used as an effective mean temperature forecasting tool.

8 IJRRAS 29 (1) October 216 REFERENCES [1]. J. David Montana and Lawrence Davis BBN Systems and Technologies Corp. 1 Mouiton St. Cambridge, MA 2138 [2]. A. EibenE. et al. "Genetic algorithms with multi-parent recombination". PPSN III: Proceedings of the International Conference on Evolutionary Computation. The Third Conference on Parallel Problem Solving from Nature: ISBN , [3]. T. Blickle, L. Thiele, A Comparison of Selection Schemes used in Genetic Algorithms. TIK-Report, Zurich, 199. [4]. D. Whitley, The genitor algorithm and selection pressure: Why rank based allocation of reproductive trials is the best, In Proceeding of the 3rd International Conference on Genetic Algorithms, []. M. Noraini Razali, John Geraghty, " Genetic Algorithm Performance with Different Selection Strategies in Solving TSP" Proceedings of the World Congress on Engineering 211 Vol II WCE 211, July 6-8, 211, London, U.K. [6]. S. Warren Mcculloch And Walter Pitts "A Logical Calculus Of The Ideas Immanent In Nervous Activity" Bulletin Of Mathematical Biophysics Volume, [7]. L. James McClelland & Axel Cleeremans " Connectionist Models" In: T. Byrne, A. Cleeremans, & P. Wilken (Eds.), Oxford Companion to Consciousness. New York: Oxford University Press, 29. [8]. D. E. Goldberg and K. Deb, A comparative analysis of selection schemes used in genetic algorithms, in: G. J. E. Rawlins (Ed.), Foundations of Genetic Algorithms, Morgan Kaufman, Los Altos, 1991, pp [9]. M. Arumugan S., Rao M.V.C. and Palniappan R. New hybrid genetic operators for real coded genetic algorithm to compute optimal control of a class of hybrid system. Applied Soft Computing, Volume 6, Issue 1, 38-2,. [1]. P. Kaelo and Ali M.M. Integrated crossover rules in real coded genetic algorithms. European Journal of Operational Research, Volume 176, Issue 1, 6-76, 27. [11]. Z. Michalewicz Genetic algorithms + data structures = evolution programs. Springer-Verlag, New York, 387 p, [12]. K. Deep and Thakur M. A new crossover operator for real coded genetic algorithms. Applied Mathematics and Copmutation, volume 188, issue 1, , 27. [13]. K. Deep and Thakur M. A new crossover operator for real coded genetic algorithms. Applied Mathematics and Copmutation, volume 193, issue 1, , 27. [14]. T. Bagchi and Deb K., Calibration of GA Parameters: The Design of Experiments Approach. Computation Sc. And Inf., 26(3), [1]. D. E. Goldberg and K. Deb, A comparative analysis of selection schemes used in genetic algorithms, in: G. J. E. Rawlins (Ed.), Foundations of Genetic Algorithms, Morgan Kaufman, Los Altos, 1991, pp [16]. L. Booker, Improving search in genetic algorithms, In Genetic Algorithms and Simulated Annealing, L. Davis (Ed.). Morgan Kaufmann Publishers, [17]. M. Kaya, The effects of two new crossover operators on genetic algorithm performance, Applied Soft Computing, 11, , 211. [18]. L.J. Eshelman, K.E. Mathias and J.D. Schaffer: Crossover Operator Biases: Exploiting the Population Distribution, Proc. ICGA97, 34/361,