The Genetic Algorithm CSC 301, Analysis of Algorithms Department of Computer Science Grinnell College December 5, 2016

Transcription

1 The Genetic Algorithm CSC 301, Analysis of Algorithms Department of Computer Science Grinnell College December 5, 2016 The Method When no efficient algorithm for solving an optimization problem is available, the genetic algorithm heuristic can often be used to find a near-optimal solution. (However, it is not guaranteed to work; the justification for using it is entirely pragmatic.) The metaphor underlying the genetic algorithm is biological adaptation through selective breeding. Think of the problem to be solved as a constraint imposed by a natural environment and of potential solutions to that problem as organisms. As the genetic constitution of an organism partially determines whether it thrives or languishes in a particular environment, so the internal structure and contents of each potential solution determine how good or bad it is. The genetic algorithm simulates the natural selection of organisms that are well adapted to their environment. Initially, a population of possible solutions is constructed, either at random (as in this implementation) or by some method that tends to generate good but not necessarily optimal solutions. This initial population is used to breed a second generation of slightly better solutions, which is used to breed a third, and so on, until some specified number of generations have been simulated. At the end of the process, the best solution in the final generation is returned. The process by which each new generation replaces its predecessor comprises three main steps. First, a large number of new potential solutions are bred by combining elements of randomly selected individuals of the old generation. Secondly, mutations are applied to these new solutions, replacing some of their elements with randomly selected values. Finally, the brood of new solutions is culled: Most of the individuals are discarded as less fit, that is, farther from the desired optimal solution. The rest make up the new population. The algorithm takes seven arguments: random-organism is a procedure of zero arguments that, when invoked, constructs and returns a randomly constructed possible solution to whatever problem is to be solved. population-size is the number of organisms in the breeding pool in each generation. Its value is a trade-off between the need for genetic diversity and variation in the population and the need for the program to run quickly. splice is a procedure that takes two organisms as arguments and returns a new organism comprising some genetic material from each parent, randomly selected. mutate is a procedure that takes an organism and returns a similar organism, except that a random change may have been made in some element of its genetic material. brood-size is the number of organisms generated from the breeding pool, before culling. It must be greater than or equal to population-size and is ordinarily several times greater. fitness is a procedure that takes an organism as argument and returns an exact real number indicating its degree of fitness (i.e., its success as a solution to the problem). number-of-generations is the number of cycles of selection that the algorithm will simulate before delivering an answer. Again, its value is a trade-off between running time and the likelihood of obtaining a near-optimal solution. (define genetic-algorithm (lambda (random-organism population-size splice mutate brood-size fitness number-of-generations) (let ((initial-population (make-vector population-size)))

2 The Genetic Algorithm page 2 (do ((index 0 (+ index 1))) ((= index population-size)) (vector-set! initial-population index (random-organism))) (let generation-loop ((chronon (population initial-population)) (report chronon fitness (vector-ref population (- population-size 1))) (if (= chronon number-of-generations) (vector-ref population (- population-size 1)) (let offspring-loop ((count (brood ())) (if (= count brood-size) (generation-loop (+ chronon 1) (cull fitness population-size brood)) (let ((alpha (random-element population)) (beta (random-element population))) (offspring-loop (+ count 1) (cons (mutate (splice alpha beta)) brood)))))))))) The random-element procedure selects a random element from a non-empty vector. (define random-element (lambda (vec) (vector-ref vec (random (vector-length vec))))) The report procedure gives a progress report, displaying a specified organism. (define report (lambda (chronon fitness best-of-breed) (display "Generation ") (display chronon) (display ": ") (newline) (display best-of-breed) (newline) (display "Fitness: ") (display (fitness best-of-breed)) (newline) (newline))) To cull the brood, evaluate the fitness of each new organism and keep track of the n fittest, where n is the desired population size. (define cull (lambda (fitness population-size brood) (let loop ((rest brood) (fittest ()) (counter ) (if (null? rest) (list->vector (map car fittest)) (let ((score (fitness (car rest))))

3 The Genetic Algorithm page 3 (let ((updated (cond ((< counter population-size) (insert (cons (car rest) score) fittest)) ((< (cdar fittest) score) (insert (cons (car rest) score) (cdr fittest))) (else fittest)))) (loop (cdr rest) updated (+ counter 1)))))))) The insert procedure expects to receive (1) a pair consisting of an organism and its fitness score and (2) a list of such pairs, in ascending order of fitness. It returns a similarly ordered list, to which the new pair has been added. (define insert (lambda (new ls) (cond ((null? ls) (list new)) ((<= (cdr new) (cdar ls)) (cons new ls)) (else (cons (car ls) (insert new (cdr ls))))))) Example: The Virtual Ant As a small example of the genetic algorithm, let s try to breed a virtual ant that can instinctively find its way from its home to a food source and back again, avoiding obstacles and bringing some food back with it. Here s a map of the ant s world:.. H * * * *... o o... * * * E.. F..... P H marks the position of the ant s home, F the position of the food source. E is an ersatz food source that actually contains no nourishment for the ant. P is a source containing a poison that kills the ant if it happens to eat it. The letter o indicates a pit from which the ant cannot escape. An asterisk indicates a hill that the ant can traverse, but only at an effort and only if it is not carrying anything. (define home? (and (= row 2) (= column 2)))) (define distance-from-home (+ (abs (- row 2)) (abs (- column 2))))) (define food-source?

4 (and (= row 9) (= column 9)))) The Genetic Algorithm page 4 (define distance-from-food-source (+ (abs (- row 9)) (abs (- column 9))))) (define ersatz-food-source? (and (= row 9) (= column 6)))) (define poison-source? (and (= row 1 (= column 3)))) (define pit? (and (= row 6) (<= column 1)))) (define hill? (or (and (= row 4) (<= 8 column 9)) (and (= row 5) (<= 7 column 8)) (and (= row 6) (<= 5 column 7))))) An ant has an age, which is an integer in the range from 0 to 4095, and a health level, which is an integer in the range from 0 to The ant dies when its age is 4095 or its health level is zero. (define full-health 1023) (define maximum-age 4095) The ant always sets out from home at age 0 and health level It travels around on the grid, moving from one grid cell to the next in any of the four cardinal directions. At each stage in its journey, its age increases by 1, and its health level decreases, unless it consumes some food, in which case its health level is reset to The decrease in the health level is normally 1, but changes to 2 if the ant is carrying anything or moving onto a grid cell containing a hill. Falling into a pit or consuming poison resets the ant s health level to zero. At each stage in its journey, the ant can try to move one position north, south, east, or west on the grid, pick up something, or consume something; we ll represent these activities by the letters N, S, E, W, P, and C, respectively. Each of these possible actions has a precondition: N: The ant is not in the top row of the grid, and is either not just south of a hill or not carrying anything. S: The ant is not in the bottom row of the grid, and is either not just north of a hill or not carrying anything. E: Theant is not in the right column of the grid and is either not just west of a hill or not carrying anything. W: The ant is not in the left column of the grid and is either not just east of a hill or not carrying anything.

5 The Genetic Algorithm page 5 P: The ant is in a grid cell containing either the food source, the ersatz food source, or the poison source. C: The ant is either in a grid cell containing either the food source, the ersatz food source, or the poison source, or is carrying something. If the precondition for an action is not met, the action has no effect on the ant (except that its age increases and its health level decreases by 1, or by 2 if it is carrying something). An ant is represented by its instinct, which is a string containing the letters N, S, E, W, P, and C, in any combination and any order. An instinct is a program of steps for the ant, telling it, stage by stage, what to do. The fitness of an ant is an integer representing the degree of success that an ant has when it follows its instinct. It is computed when the ant dies, when its program runs out, or when it returns home carrying something for the first time. In any of these cases, its journey is over. Let s construct the instinct for an ant by giving it some number of randomly chosen actions at least 16, at most 512. (define random-ant (lambda () (let ((len (+ 16 (random 496)))) (let ((result (make-string len))) (do ((index 0 (+ index 1))) ((= index len) result) (string-set! result index (random-action))))))) (define random-action (lambda () (string-ref "NSEWPC" (random 6)))) Our splicing procedure will choose a substring of one parent(selecting the end-points randomly) and replace the corresponding substring of the other parent with it. (define splicer (lambda (alpha beta) (let ((alpha-len (string-length alpha)) (beta-len (string-length beta))) (let ((one-end (random (+ alpha-len 1))) (other-end (random (+ alpha-len 1)))) (let ((start (min one-end other-end)) (finish (max one-end other-end))) (string-append (substring beta 0 (min start beta-len)) (substring alpha start finish) (substring beta (min finish beta-len) beta-len))))))) Our mutation procedure passes over each character of the string, occasionally replacing it with a randomly selected character (with probability 1/1024), deleting a character (with probability 1/2048), or inserting a randomly selected character (with probability 1/2048). It also occasionally appends a new character at the end (with probability 1/128). (define mutator (lambda (instinct) (let ((len (string-length instinct))) (let loop ((remaining len)

6 The Genetic Algorithm page 6 (so-far (if (zero? (random 128)) (list (random-action)) (list)))) (if (zero? remaining) (list->string so-far) (let ((next (- remaining 1))) (loop next (cond ((zero? (random 1024)) (cons (random-action) so-far)) ((zero? (random 2048)) so-far) ((zero? (random 2048)) (cons (random-action) (cons (string-ref instinct next) so-far))) (else (cons (string-ref instinct next) so-far)))))))))) The fitness evaluator runs the ant through its program and computes its fitness score. If instinct demands that the ant pick up something when it is already carrying something, and picks up whatever is available on site instead. If instinct requires it to consume something, it will first try to consume whatever is available on site, then (if nothing is available on site) whatever it is carrying. (define ant-fitness (lambda (instinct) (let ((len (string-length instinct))) (let loop ((age (health full-health) (row 2) (column 2) (carrying #f) (away-from-home #f)) (if (or (= age (min len maximum-age)) (zero? health) (and (home? row column) carrying away-from-home)) (final-score age health row column carrying away-from-home len) (case (string-ref instinct age) ((#\N) (let ((ok-to-move (and (positive? row) (or (not carrying) (not (hill? (- row 1) column)))))) (let ((new-row (if ok-to-move (- row 1) row))) (loop (+ age 1) (cond ((or (= health 1) (pit? new-row column)) ((or (hill? new-row column) carrying) (- health 2)) (else (- health 1)))

7 The Genetic Algorithm page 7 new-row column carrying #t)))) ((#\S) (let ((ok-to-move (and (< row 12) (or (not carrying) (not (hill? (+ row 1) column)))))) (let ((new-row (if ok-to-move (+ row 1) row))) (loop (+ age 1) (cond ((or (= health 1) (pit? new-row column)) ((or (hill? new-row column) carrying) (- health 2)) (else (- health 1))) new-row column carrying #t)))) ((#\E) (let ((ok-to-move (and (< column 11) (or (not carrying) (not (hill? row (+ column 1))))))) (let ((new-column (if ok-to-move (+ column 1) column))) (loop (+ age 1) (cond ((or (= health 1) (pit? row new-column)) ((or (hill? row new-column) carrying) (- health 2)) (else (- health 1))) row new-column carrying #t)))) ((#\W) (let ((ok-to-move (and (positive? column) (or (not carrying) (not (hill? row (- column 1))))))) (let ((new-column (if ok-to-move (- column 1) column))) (loop (+ age 1) (cond ((or (= health 1) (pit? row new-column)) ((or (hill? row new-column) carrying) (- health 2))

8 The Genetic Algorithm page 8 (else (- health 1))) row new-column carrying #t)))) ((#\P) (loop (+ age 1) (- health (if carrying 2 1)) row column (cond ((food-source? row column) food) ((ersatz-food-source? row column) ersatz-food) ((poison-source? row column) poison) (else carrying)) away-from-home)) ((#\C) (cond ((food-source? row column) (loop (+ age 1) full-health row column carrying away-from-home)) ((ersatz-food-source? row column) (loop (+ age 1) (- health 1) row column carrying away-from-home)) ((poison-source? row column) (loop (+ age 1) 0 row column carrying away-from-home)) (carrying (loop (+ age 1) (case carrying ((food) full-health) ((ersatz-food) (- health 1)) ((poison) ) row column #f away-from-home)) (else (loop (+ age 1) (- health 1) row column #f away-from-home)))))))))) The final-score procedure inspects the ant at the end of its program and awards points, positive or negative, for fitness. (define final-score (lambda (age health row column carrying away-from-home len) (let ((home-distance (distance-from-home row column)) (food-distance (distance-from-food-source row column))) There is a severe penalty for never leaving home. (+ (if away-from-home 0-256)

9 The Genetic Algorithm page 9 There is a modest reward for carrying food, an even more modest reward for reaching the food source while not already carrying food, and a still smaller reward for getting close while not already carrying food. (if (eq? carrying food) 128 (if (zero? food-distance) 64 (quotient 8 food-distance))) There is an enormous reward for bringing food home (particularly while remaining young and healthy) and a pretty good reward for getting it most of the way home. (if (eq? carrying food) (if (zero? home-distance) ( (quotient health 4) (- age)) (quotient 1024 home-distance)) There is an enormous penalty for bringing poison home and a modest penalty for bringing it close to home. (if (eq? carrying poison) (if (zero? home-distance) (quotient -256 home-distance)) There is a modest penalty for being burdened with a program of superfluous actions. (quotient (- age len) 2) Finally, there is a very modest reward for longevity. (quotient age 256))))) Filling in the rest of the parameters with plausible values (population size 400, brood size 2000, number of generations 200, let s see what a few million ants can come up with: (display (genetic-algorithm random-ant 400 splicer mutator 2000 ant-fitness 200) (newline) The optimal instinct conducts the ant directly to the food, tells it to eat some of the food for maximum health and to pick up some more food, and then leads it directly back home (avoiding the hill). There are actually many instincts that fit this general description and all result in a score of On most runs, the genetic algorithm comes up after only a few dozen generations with an instinct that gets the ant safely home, carrying food with it. The algorithm then refines its solution, pruning away superfluous steps, for another thousand or twelve hundred generations.

10 The Genetic Algorithm page 10 Generation 0: WSPPNWSCPSESCWNCSNWNSCEPCWSCCCWCNSEWEPSPSCNWEPSWNSCSEPSNSNSSSNENWCPEPNNPWNW SESNNWNNEWSPNNWPCPPESPSCSPESPEPPPPPSENSCPNESEWCECCSWEWCEWSWCSCSPESCPNPWENWN CWWNPEPPCEEESSNSWCEEEENSEWSSWCCCCCWNCSPSNWPNENPCNNENWPECENNEPCNCNESCNWSSNPN ECWPWPSSCEPPEPNNENEECCNECESCSCWPPNEESSENSNSEPPPCWSCPPCPWSWWSWCEWENSENSWSPEC SCSNEESEPCPNENWSNPCSSSSPNNCPWCCPWPPPWNPPCPPCSPPWCPSWSWPWENNEECEPNNNSCNNPCPN PSECSNWWPPPNSCPSNENSNECWNEECCSNSSEWNNSCCCCNWCCCPEPEPCNNSNCWENNCSWENSWSCSECW EESPESSEPPWWPEWWPESESSSEPCC Fitness: -219 Generation 1: SSSECCCPSSSCWENWCEWSNEEENPNEEPCPWPECWNNWPSPEPNNCCESWPWSCSECCSWWSNSNWPPEENWW NNWSNNPSSNPWPNPWNESPWSPPWSNSCPENEEEWESSNCNEEPCNSNEEPSSWWWNPPNEWPWESCNEEPPNE SSCNWEEPEEECSSNPSPWWNWNPNPSNNPPWCSCSCECESNSWNSWNSSECPESPCCSWNWCSWWSCPNCNNEC WSSWWPPSNPSNCEENNNWSSSSSECEWWNEPENWECCCESNEESSNENNCESECPCWWSSNWNSCEWSPSCPCE NSEEWSPSCPPSWNCEEESWECSESSNCWSNNENEEPENPEECPCENWCNECPPWWPWSCSPCCPCSPCE Fitness: 65 Generation 2: NCSSWSPNCNWCWPWWENCCCNEWWNCWCENSWCSEWEPWCSWNNNPEECSEECNPPEEEESSPNWWEEENWEEP SCPCWPWPNCNNNECCNENENPCENSECPEWPPNCNPNWNNWSESCWSSNNCSEPSNPCPPCNESSESSWSPSNN WNWCPWCCNNCEEWSPECPCEENNWPWPWCWCPEPCCCPSSPSESSWSNCSNNPEEWCPESWSSWPESNPWWPPW CNP Fitness: 230 Generation 3: WNNPNEWPCEPNCPNEWSNNSPNWENPCCCENCSCNEENCCPSPPEEPSSSSEESPCSCWPCNCSWEEPWENPCN WWPNPCSNNSWPEWWWNPEPPCSNWSCEWWPCEECCEEEPPSSPNWWESPNSPECSPSPCCPSESNPNENCSSPW PPESCNWWCCSSNPWEWEWNSSNNWWENEPPESSSPSCSPESENWESPPSEWCCNPPWENCSSPNCNSNSWCWSN CECEENNEWPWWEPWCWNWCNNPNWEPPSWSPPWSNPSPECSECPPCEPPSWWCESWENNESNENPSSCECWWEP NPCCENCWECSPSWNNPWNP Fitness: Generation 15: WEEESEPPCPNEENNPCPNNWSWSPWCEPWSEWCESENPSEWWWCCNECCSPSNWPWPEEWPNPEEPSNCPWESE EESCNSNCSPCPCNCWPPCCWNNPCPSECSCCNCPPCNSESSECCPCSSECWEENPNNWNPNSPCSEPSPCCWPP ESCNSCCSENSSWNNSPNEPPSCENWNNEWSCPCNSSESSCPWWNPSCEWCENECNSECSEEWSCNPSNCEWCEC PSCCNSPNNSWCPSCSWWSNSPSESPPSENPCENPPWPCNSCSCWWCPCSESNPCPCWESCECECSNEPWECSPE WENESPEPWENCENPNENCWSPPSCSPNNNWSPPNWWWNWWNSPWSPWPNENNNNPPP Fitness: 641 Generation 16: WEESEPPCPNEEENPCPNNWSWSPWCESSENSCCPWNSSPEWWNEWEWECPEPWWNPSNSCPNESNPCEWNSWES WPENNWCWCWCPSCSEEWPSNNWSPNNSSSNCPEEWEPPWPCEWPPWNSNWWECCNECPSEECSSNWEECNEWSE CCNPNESNNPWNPCPEWSCPCEWCCNNNPPWCSCSWEWCESSESSSNSSECPNSSECSEEWSCNPSNCEWCECPS SSPCSPWSCNNNECPCPWSPPSESPPPSENSSWEWECPWPCNSCWCESNPCPCWESCECECSEENEPWECSPEWE NESPEPWENCENPNENCWSPPSCSPNNNWSPPNWWWNWWWNSPPWSPWPNENNNNPPP

11 The Genetic Algorithm page 11 Fitness: 1153 Generation 17: WEEESEPPCNSSNPCPNCWPCEESSEECESSENSCCPWNSSPEWWNECCSPSNWPWPEEWPNPEEPSNCPWESEE ESCNSNCSPCPCNCWPPCCWNNPCPSECSCCPCPPCNSESSECCPCSSECWEENPNNWNPNSPCSEPSPCSENCC EWEESPSENNNPESWCCCESPPECNEPWPCCWCPSWCPPWEEWSCECCCSEWPCWNPSNWCCSSCCEEPPSSPEN CWCNPNEPCWCPSCSWWSNSPSESPPSENPCENPPWPCNSCSCWWCPCSESNPCPCWESCECECSNEPWECSPEW ENESPEPWENCENPNENCWSPPSCSPNNNWSPPNWWWNWWNSPWSPWPNENNNNNPPPW Fitness: Generation 200: WEESPCCNENWPNEPEPENNCENNNNNNESEEESESCECPPECPSPWNSESSNESEEEEECCSSENWPSECECCS PPWNNEESEEEPPPPCEPSNWSWEECCSCSESWENESNEECCSESESSPPPWCSWEEEECECECCSCEPWCWSWE SSWCCWPPENCESCCSPEENENNENWSPSSNNNWSCPNWWWNWWNNNWWNNNNP Fitness: Generation 500: PCESECEEEESPESESSESSSSSSEWNNNWCPNWWWNWWNNNWWNNN Fitness: Generation 1000: SEEEEESESSSESESSWNCPNWWWWWNNWNNNNWN Fitness: Generation 1999: SEEEEESESSSESSPCNWWWWWWNNNNWNNN Fitness: 8538 Generation 2000: SEEEEESESSSESSPCNWWWWWWNNNNWNNN Fitness: 8538 Copyright c 2016 by John David Stone This work is licensed under the Creative Commons Attribution ShareAlike 4.0 International License. To view a copy of this license, visit US or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.