Landscape Ruggedness in Evolutionary Algorithms

Transcription

1 Persona use of this materia is permitted. However, permission to reprint/repubish this materia for advertising or promotiona purposes or for creating new coective works for resae or redistribution to servers or ists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE. Contact: Manager, Copyrights and Permissions / IEEE Service Center / 445 Hoes Lane / P.O.Box 1331 / Piscataway, NJ , USA. Teephone: + Int Landscape Ruggedness in Evoutionary Agorithms Krasimir Koarov Interva Research Corporation, 1801-C Page Mi Road, Pao Ato, CA E-mai: koarov@interva.com 1. Introduction This paper describes a mode for the exporation of the dynamics of interaction in a popuation of individuas using an evoutionary approach. In particuar we anayze the effect of the compexity of the fitness andscape and the popuation size on the performance of an evoutionary agorithm in terms of speed of fixation and fixation to suboptima individuas. Our evoutionary mode resembes those in the fied of Popuation Genetics and approaches evoution as a process of adaptation rather than an optimization. The simuation resuts from our experiments are justified with a theoretica probabiistic anaysis of the dynamics of the popuation with and without recombination. The paper introduces severa new insights into the interaction and roes of the different parameters of an evoutionary system. There have been very few attempts to theoreticay anayze the dynamics of interaction between the different operators and the performance of GA (e.g. Hoand's buiding bocks and schema [8], [1]). The probem of finding the appropriate (and in some cases the optima) popuation size have been discussed for certain appications in the GA iterature [5]. There has aso been research of the roe of different seection schemes on GA performance [8]. Interesting work on anaytica toos for GA incude the approach in [13] based on the Perron-Frobenius theorem, and the Markov chain anaysis described in [12]. We wi give a detaied description of our mode in Section 2. The experiment for which we vary the ruggedness of the andscape is described in Section 3. To better understand the resuts in this section, we performed a theoretica anaysis of the mode and derived recursive formuas describing the dynamics of interaction in the mode. These resuts are expained in section 4. Section 5 draws some concusions from our work. 2. The mode We performed experiments with a simpe GA type system. Our mode incudes a finite number of diaeic hapoid organisms. The chromosome consists of one string of bits and we chose each genotype or bit string to be of fixed ength 20. Each aee can have two vaues: 0 or 1. The GA operators are crossover and mutation and there is random mating within non-overapping generations. The initia popuation is seected at random with equa probabiity of 0's and 1's. Our mode has fixed rates of recombination and mutation. We use singe (one-point) recombination with the break point seected at random, uniformy across the chromosome. The offspring are subjected to mutation and seection. New offspring are accumuated unti the fixed popuation size is reached. At that time we have formed the new generation and the current one becomes its parenta generation. Thus we do not have expicit eitism - the new generation is entirey formed by appying the genetic operators to the previous one. For each case (i.e. random initia popuation) this process continues unti a the individuas within the same generation have the same genotype - this is the condition for fixation. For statistica test we run each set of input parameter settings for 100 cases. As expained in [9] this method of constructing the offspring generation where every member of the new popuation is added as a resut of individua "seection" after appication of the genetic operators to the previous generation, is a characteristic of popuation genetic anaysis. In other words we impement fitness based proportiona seection, independent of the fitness vaues for the rest of the popuation. 3. Fitness Landscape with Randomy Controed Ruggedness To contro the ruggedness of the fitness andscape we use the same formaism as proposed by Bergman and Fedman 1992 [3], i.e.: Ψ( x).sin( xπ i M F(i) = ) (1) x=1 x where is the number of oci in the genotype (20 in our case). Ψ(x), x = 1,2,..., M are random numbers that are uniformy distributed in the interva [0,1]. The number of random eements M is the andscape variabe in this experiment. It determines the ruggedness of the fitness function andscape. When M = 1 there is ony one wave of the sine function and its peak is the maximum we are ooking for. The arger is M, the more rugged and compicated is the fitness function. The fitness function is cacuated for a particuar genotype by counting the number of 1's in the genotype to give the index i used in formua (1) to find the fitness vaue

2 for that individua. Thus it is cear that we can have maximum 21 different fitness vaues in our environment. Note that the optima genotype does not have a of its aees equa to 1. In addition, as can be seen from the simuation resuts, for different cases of the same run, the genotypes at the fixed point can be different. For exampe if the optima individua has ten 1's, i.e. i=10, the corresponding fixed point can be in one case and in another one. There are possibe genotypes with the same optima fitness. i In a experiments the mutation rate was set to zero, because our goa was to anayze fixation, and if there is any bi-directiona mutation, fixation (as defined in our mode) does not occur. However our framework can aso be used for anayzing other processes that incude mutation. As we mentioned aready for this first experiment we varied the PopSize, the number of random eements M and in few cases the recombination rate. The first two tabes of resuts correspond to the case when the recombination rate is equa to 0.4, i.e. there is 40% probabiity of one recombination event over the entire chromosome. This vaue for the recombination rate is often used in popuation genetics iterature and aows us to compare our resuts to those in [2] and [3]. Tabe 1 dispays the number of generations to fixation to any point in the search space for three popuation sizes: 100, 150 and 200 individuas, and six fitness andscapes: for M=1, 6, 10, 13, 16 and 20. The mean, minimum and maximum are taken over the tota number of cases -> 100. The standard deviation for a cases in Tabe 1 and 2 foow the trend of the mean. We aso performed χ 2 tests (see [4]) to anayze the resuts from a experiments. The tests confirmed the significance of the reported data with respect to the nu hypothesis. As we can see from Tabe 1 the more compicated the fitness function the faster fixation occurs. In addition a arger popuation generay takes onger to reach fixation. For comparison we performed the same simuation with the Recombination Rate equa to zero. In this case we have neither mutation, nor recombination and the system performance is imited by the fitness of the most fit individua in the initia popuation. Tabe 2 shows that for simpe andscapes the mode without recombination takes significanty ess generations to fixate. For more rugged andscapes (M > 6) the recombination causes faster fixation. However if we ook at the overa computing time the differences are not statisticay significant for arge M's. Generations to fixation: mean (min, max) from 100 cases; Recombination Popuation (159,648) 283(41,749) 193(28,643) 171(44,360) 160(37,508) 154(13,390) (174,104) 374(57,760) 244(33,628) 236(18,544) 226(71,676) 220(36,677) (288,1580) 441(48,1160) 294(35,885) 274(16,487) 257(21,630) 260(77,678) Tabe 1. Generations to fixation for rugged andscape with recombination Generations to fixation: mean (min, max) from 100 cases; No Recombination Popuation (46,745) 164(25,560) 153(31,617) 156(15,436) 180(48,471) 187(42,786) (71,813) 294(145,629) 255(49,1360) 277(71,998) 260(83,864) 280(68,820) (75,1357) 348(163,623) 379(58,1019) 333(82,976) 372(111,1493) 372(110,1206) Tabe 2. Generations to fixation for rugged andscape without recombination The next Tabe 3 anayzes the cases when the popuation fixates to a vaue that is different from the goba optimum. The numbers in parenthesis indicate the number of 1's in the genotype on which different cases fix. For exampe for M=6 and Popuation Size of 100 most of the cases fixated to a genotype with eight 1's, whie few cases (here the number is five) fix on a genotype with nine 1's and one case fixated to a genotype with two 1's. The tota number of cases considered is again 100. For a andscapes the mode fixed on the oca maximum that is cosest to the mean of the distribution of maximum as we. Thus the numbers in Tabe 3 are smaer for this case compared to the rest of the tabe. However, the number of cases that do not fix on oca maximum is smaer for more rugged andscapes (M > 1). One possibe expanation of this phenomenon is that GA doesn't do we for functions with reativey fat tops when there is not enough seection pressure to drive it to the oca optimum (it stops at 1- ε for some sma ε rather than at 1). From Tabe 3 we can aso derive that, as the popuation size increases there is generay ess fixation on suboptimum with a other conditions been the same. the initia popuation. When there was ony one random A simiar anaysis for the case of no recombination is eement in the fitness function (a be-shaped or a U-shaped iustrated in Tabe 4. Ceary the presence of recombination andscape), this oca maximum happens to be the goba

3 eads to ess fixation on suboptimum, X, especiay in the case of a simpe, non-rugged andscape (M = 1). In fact athough as we pointed out above, it takes ess generations for the mode without recombination to fix in that case, it actuay fixates more often on the "wrong" genotype. We can aso compare the mean fitness of a 100 cases after fixation for the andscapes. Consistenty this fitness is higher for the mode with recombination than that without for a parameter sets. We woud ike to emphasize one feature of recombination in our mode. Typicay in GA it is assumed that recombination increases variabiity in the environment because it introduces new individuas. However in our experiments this is not the case and the reasoning goes as foows. We start with a Gaussian distribution of the initia popuation, i.e. most of the individuas have the 10 ones (out of 20 maxima). - If there is no recombination (and no mutation) the next generation wi typicay again have individuas with number of ones cose to 10 and there wi be a arge variety of them. This trend wi propagate throughout the simuation, thus maintaining arge variabiity in the popuation. This is aso the reason why it takes onger for this mode to fixate and X is arger than when recombination is present. Number of Cases that do not Fixate to the Goba Optimum; Recombination Popuation (8,9,2) 98 (8) 100(13) 100(12,15,7) 99(12) (8,9,2) 96 (8) 99(13) 100(12,15) 100(12) (8,9,2) 94 (8) 99(13) 99(15,12,7) 100(12) Tabe 3. Suboptima Fixation for rugged andscape with recombination Number of Cases that do not Fixate to the Goba Optimum; No Recombination Popuation (8,9,2,3) 100(8,2) 100(13,1) 100(12,15,7) 100(12) (8,9,2,3) 100(8) 100(13) 100(12,15,7) 100(12) (8,9,3,2) 100(8) 100(13) 100(12,15) 100(12) Tabe 4. Suboptimum fixation for rugged andscape without recombination - With recombination, individuas with ower fitness are produced which because of the distribution of the initia popuation, are ess statisticay represented than the individuas with number of ones around 10. Thus in the second generation we wi actuay have ess variabiity than in the initia one. In other words in future generations the rare individuas with higher and ower fitness than the median ones wi be reinforced. As a resut the popuation fixes faster and with higher overa fitness than the case without recombination. However recombination in this case reduces the variabiity. Figure 1. Fitness Landscape for the case M = 16 The advantage of recombination for faster fixation to an individua with higher fitness is further iustrated in the exampe in Figure 1. When the initia popuation has a 50% chance of 1's and 50% chance of 0's for the vaues of the aees the popuation invariaby fixes to an individua with the ocay maxima fitness F(13) (see eq.(1)) when M = 16. The predominant point of fixation moves as we vary the percentage of 1's in the initia popuation. For exampe if we have initiay 90% 1's, the popuation fixes at F(16) in most of the cases (with the mode with recombination performing better than the one without it). As seen in Figure 1, this is a oca maximum with a smaer fitness than F(13). On the other side with 25% initia 1's the popuation with recombination fixes in 97% of the cases to the goba optimum F(1). Note that in the figure F(1) (and resp. the rest of the indices) corresponds to the case of zero number of 1's in the genotype). The corresponding mode without recombination fixes to the goba optimum ony in 27% of the cases, with the rest fixing on a oca maximum F(8). Thus the mean fitness at fixation is ower than the case with recombination. In addition to the anaysis mentioned above, we did imited experiments with varying the recombination rate. It is not cear from this imited data what are the exact

4 correations. However, we can conjecture that with increase in recombination, the genetic drift decreases and the speed of fixation increases. The question of the roe of the recombination is very interesting and we wi present some resuts on that topic in a future pubication. This question was extensivey discussed for dipoid individuas in [3]. 4. Theoretica Anaysis In this section we wi extend our discussion of the roe of the shape of the fitness andscapes and derive iterative reationships that describe the dynamics of interaction in our system. Let us denote with F(s) the vaue of the fitness function for an individua with s number of 1's. With a k (s) we denote the expected number of individuas with fitness F(s) in the popuation at generation k. As we have mentioned before N is the popuation size and is the ength of the genotype (in our exampes N = 100 and = 20 ). Because we aways keep constant popuation size, we have: ϖ k (s) = N for every k ( 2 ) s= 0 The average expected fitness of the popuation at generation k, is: = 1 F(s)ϖ k (s) for every k ( 3 ) N s= 0 We want to find a recursive reation describing the expected number of individuas with fitness F(s) at generation k+1 in terms of the number of individuas with different fitness at generation k. For simpicity et us first consider the case with no recombination, i.e. the recombination rate r = Derivation without recombination. The ony way to add an individua with fitness F(s) to the new generation without recombination is to choose it from the parent generation. The agorithm picks an individua with that fitness, compares it to a uniformy distributed random number and adds it to the new generation if it is arger than the random number. If it is not arger, the same cyce is repeated unti an individua has been added to the new generation. We can derive the foowing formua: ϖ k +1 (s) = F(s) ϖ k (s) ( 4 ) This is the main recursive reationship that cacuates the expected number of individuas with fitness F(s) in a new generation as a function of the expected number of individuas with that fitness in the parent generation, the expected average fitness of the parent generation and the fitness F(s). This reation is simiar to the Fisher's Fundamenta Theorem of Natura Seection (see [4] and [7]). In the GA iterature an exampe of simiar anaysis is described in [6]. Using (4) we can express the expected number of individuas with fitness F(s) at generation k, as a function of this number for the initia randomy generated popuation and the expected average fitness of a generations before k. We can aso derive a recursive reationship for the expected average fitness of the new generation in terms of the data from the parenta generation: +1 = 1 F 2 (s)ϖ k (s) ( 5 ) N s =0 Equations (4) and (5) describe the dynamics of interaction of our mode. Theoreticay for infinite popuation size and appropriate recombination rate the popuation shoud aways tend to the optimum. If there is no recombination the popuation shoud fixate to the most fit individua initiay present in the popuation. However we have observed that in numerous cases the finite popuation size in the simuations causes the mode to fixate on oca optima. Athough we can not predict with certainty the direction of fixation of the popuation based on the initia conditions, we wi describe a simpe rue that ets us predict the concentration of individuas in a given generation based on the distribution of its parent generation. We wi iustrate this rue for the case of the initia distribution in the simuation. Let us denote with p I the initia probabiity of 1's in the first randomy generated popuation. In our discussion so far we have assumed that every aee for each individua in the initia popuation has equa chances (0.5) of being a 1 or a 0. In genera however we can choose different distribution of 1's and 0's and we can vary that using p I. Let us aso denote with X the brake point p I. (i.e. X = p I. ). Then we can form the foowing quantities, which we wi ca "eft" and "right moment of F with respect to X": X Φ + 0 = 1 F(s)ϖ 0 (s) and N s =0 Φ 0 = 1 F(s)ϖ 0 (s) ( 6 ) N s= X Using reationships (4) and (6) we can show that the condition Φ + 0 >Φ 0 is equivaent to the condition: X ϖ 1 (s) > ϖ 1 (s) ( 7 ) s= 0 s = X In other words if the eft moment of F with respect to X" is arger than the right moment in the initia popuation, in the first generation there wi be arger expected number of individuas whose fitness is to the eft of X than those to the right. This property is aso true for any two consecutive generations but it does not necessariy propagates throughout a generations. In particuar we might encounter a situation, where even though the eft moment with respect to a fixed point is arger than right one, these moments are actuay smaer than the corresponding moments in the previous generation. In that case the fixation point is ikey

5 to be on the right side of X, contrary to what the first generation wi suggest. For competeness we wi point out that for our mode the expected number of individuas with fitness F(s) in the initia popuation is: ϖ 0(s) = N s p s I (1 p I ) s ( 8 ) 4.2 Derivation with recombination In this section, we wi provide some formuas for the genera case when the recombination rate r 0. There are two ways to generate an individua with fitness F(s) to the new generation. The first one is if as a resut of a recombination, a new individua with fitness F(s) is created and it is added to the new generation. The probabiity of a recombination event is r because we compare the recombination rate with a uniformy distributed random number. The second way of seecting an individua with fitness F(s) corresponds to the case when there is no recombination and an individua is chosen as in Section 4.1. The probabiity of no recombination is 1-r thus in anaogy to the previous section we can derive: ϖ k +1 (s) = N[r.PR + (1 r ). PN] ( 9 ) Here PR is the probabiity of seecting an individua with fitness F(s) with recombination and adding it to the new generation, whie PN is the corresponding probabiity without recombination. If the individua that is seected is not added to the new generation, then we perform a second try to add new individua, then third try, etc. unti a new individua is added. Thus using the same reasoning as in the previous section, we can show that the overa probabiity of adding an individua with fitness F(s) is F(s). Using that and the reasoning about the probabiity of seecting individuas with fitness F(s) without recombination, we can derive: ϖ k +1 (s) = N[r.Pr(s)+(1 r) 1 N ϖ F(s) k (s)] ( 10 ) where Pr(s) is the probabiity of generating an individua with fitness F(s) with recombination. At this point if we had mutation in the mode, the probabiity Pr(s) woud have been simiary subdivided to account for the effect of mutation on the individua genotype. However as we pointed out earier, in that case there woud be no fixation in the finite popuation mode. Thus we assume that the Mutation Rate is set to zero. We can derive the foowing formua for P r (s): 1 2 ( + 1)N y y () s x z x [ x = 0 z = x p = s x y = 0 y y () () s x ( p s + x) () z() p ].ω k (z)ω k ( p) s 1 = B(x,z, p, s) ( + 1)N ω 2 k (z)ω k ( p) x = 0 z = x p= s x ( 11 ) Here the coefficients B(x,z,p,s) are numerica constants for given x, z, p and s. From (11) we can see that the probabiity P r (s) is a inear combination with constant coefficients of terms of the type ω k (z)ω k (p) where the parameters z and p vary over the range [0,]. We can add the coefficients for terms that have permuted indices and arrange the resuting coefficients in ower trianguar matrices where each entry r ij of the matrix R k+1 (s) corresponds to the coefficient in (11) in front of the termω k (i)ω k ( j). We can see from (11) that with the increase of s we get ess representation in the R-matrices of the ω k (z) for z < s and stronger representation of those with z > s. At the same time if Φ + 0 >Φ 0 then higher weight is given to the individuas with fitness ess than the mean of the initia popuation X. Because those individuas are more represented in the R-matrices to the eft of X, that means that in the next generation those individuas wi be even more represented and thus by induction the popuation in the next generation wi be moving toward the eft of X. By anaogy if Φ + 0 <Φ 0 the individuas to the right of X get more weight and representation. For arger we can ceary see the structure of this argument because for a given X, a the R-matrices for s > X wi have upper eft rectanges of zeros. Accordingy the R- matrices for s < X wi have zeroes in there bottom right rows. These properties confirm the vaidity of our concusion for the genera case. We shoud note however that this effect is not as strong as in the case of no recombination, where there was a cear tendency toward one direction or the other. In this case athough the R-matrices have a ot of zeroes for say sma s to the right of a given vaue, not a of the coefficients are 0, and those that are represented have a neutraizing infuence. An interesting question that is beyond the scope of this paper, is to anayze the infuence of r on the dynamics of the system. Finay we want to point out that athough the derivation referred to the "front" positions of the first parent and the "back" positions of the second, in fact the exact ocation or even adjacency of the positions is not important here. In fact our resuts wi be vaid with sight modifications for any singe event recombination (not necessariy simpe one-point ones). 5. Summary and Concusions

6 With respect to fixation on suboptima fitness in the simuations we can concude that: - The increase of the popuation size decreases the rate of "wrong" fixation due to ess samping error. This resut is in accord with the resuts in popuation genetics that are due to samping error. As described in Chapter 5 of [11] seection strength has to be above certain vaue for given popuation size, in order for the system to overcome the "genetic drift". Genetic drift in that context refers to the samping error by the gametes - an interna source of chance effects in evoution. We have shown that: - More rugged andscapes fix faster and with higher overa fitness to the oca optimum that is cosest to the mean vaue of the fitness of the initia popuation. The extent to which this resut hods depends on the recombination rate. As shown in Section 3 our genera concusion is that: - The presence of recombination in the mode eads to a decrease in the fixation on suboptimum fitness especiay for a simpe, Gaussian andscape due to introduction and maintenance of diversity. For a Gaussian distribution of the initia popuation, recombination reduces the variabiity in the popuation. The resuts for rate of fixation refect the speed for a fixations, correct or incorrect. We can summarize the resuts from Section 3 as foows: - The more compicated the fitness andscape, the faster fixation occurs. Popuations with compicated, rugged fitness terrain fixate faster with recombination rather than without it. Increased Recombination increases the speed of fixation. - As expected arger popuations take onger to fixate. These resuts aso foow from the equations (4), (10) and (11) derived in Section 4. Since the rate of seection F(s) does not depend on the popuation size, ceary arger popuations wi take onger to bring a except one of the ϖ k (s) to zero (i.e. to fixate). In addition when F is reativey fat on the top (arger s), for Gaussian initia distribution, there are severa suboptima vaues of s around the optimum for which both F(s) and ϖ k (s) (and thus their product) are cose to the optima ones, i.e. seection pressure is weak. From (4) and (11) foows that in future generations the number of these suboptima individuas wi be cose to the number of the optima. Thus on average it wi take onger to fixate on the optima individua and the probabiity of wrong fixation wi be higher. In addition to the above resuts we emphasize the foowing new insight that is a direct resut from this work: It comes from the rue that we derived experimentay and proved theoreticay regarding the expected distribution of the popuation at a given generation based on the distribution at the previous generation. We showed that if the average fitness (the moment) of the individuas at one side of the mean of the initia popuation is arger than the corresponding one at the other, then in the next generation there wi be more individua on that side than on the other. As we pointed out, this property is ony vaid for one generation at a time and it does not necessariy propagate throughout the entire simuation. In addition the formuas in Section 4 describe theoreticay the dynamics of interaction within our system and aow for further exporation of the infuences of different parameters and conditions. Finay we woud ike to mention that the ruggedness of the fitness function have simiar effect as the strength of the seection pressure. If we compare the anaysis in Section 3 with the resuts in [9] we can see that both the increase of the ruggedness (arger N ) and the increase of the seection pressure ead to faster fixation and ess "wrong" fixation. Acknowedgments I thank Dr. Aviv Bergman and Professor Marc Fedman for the detaied and usefu comments and corrections on the paper. References [1] L. Atenberg, The Schema Theorem and Price's Theorem, FOGA-3 (eds. Whitey and Vose), 1995, Morgan Kaufmann, pp [2] A.Bergman and M.Fedman, More on Seection for and against Recombination, Journa of Theoretica Popuation Bioogy, Vo. 38, No.1, August [3] A.Bergman and M.Fedman, Recombination Dynamics and the Fitness Landscape, Physica D 56, 1992, p [4] W. J. Ewens, Mathematica Popuation Genetics, Springer-Verag, [5] D.Godberg, Sizing Popuations for Seria and Parae Genetic Agorithms, Proceedings of the Third Internationa Conference on Genetic Agorithms, 1989, p [6] J. Grefenstette, Conditions for Impicit Paraeism, FOGA-1 (ed. Rawins), 1991, Morgan-Kaufmann, p [7] D.Hart and A.Cark, Principes of Popuation Genetics, Sinauer Associates, Inc., [8] J.Hoand, Adaptation in Natura and Artificia System, MIT Press, [9] K.Koarov, The Roe of Seection in Evoutionary Aagorithms, Proceedings of the 1995 IEEE Internationa Conference on Evoutionary Computation, Perth, 1995, p [10] B.Manderick, M. de Weger and P.Spiessens, The Genetic Agorithm and the Structure of the Fitness Landscape, Proceedings of the Fourth Internationa Conference on Genetic Agorithms, 1991, p [11] J.Roughgarden, Theory of Popuation Genetics and Evoutionary Ecoogy: An Introduction, Macmian Pubishing Co., [12] G. Rudoph,, Proceedings of the 1996 IEEE Internationa Conference on Evoutionary Computation, 1996, p [13] M. Vose and C. Liepens, Punctuated Equiibria in Genetic Search, Compex Systems 5, 1991, p