Evolutionary Algorithms. LIACS Natural Computing Group Leiden University

Transcription

1 Evolutionary Algorithms

2 Overview Introduction: Optimization and EAs Genetic Algorithms Evolution Strategies Theory Examples 2

3 Background I Biology = Engineering (Daniel Dennett) 3

4 Background II DNA molecule Carries genetic information Human DNA: Base pairs Combinations Genotype

5 Thought Experiment: Elementary particles in the universe Time steps since big bang Possible computations in the universe is faaaaaar larger!

6 Optimization f : objective function High-dimensional Non-linear, multimodal Discontinuous, noisy, dynamic M M1 M2... Mn heterogeneous Restrictions possible over Good local, robust optimum desired Realistic landscapes are like that! Local, robust optimum Global Minimum 6

7 Dynamic Optimization Dynamic Function 30-dimensional 3D-Projection 7

8 Optimization Creating Innovation Illustrative Example: Optimize Efficiency Initial: Evolution: 32% Improvement in Efficiency! 8

9 Evolutionary Algorithm Principles

10 Model-Optimization-Action quality Function 15 i = weighti i= 1 scalei calculated desired 2 i Model from Data Simulation Experiment Function(s) r ( y) =... f i Subjective Business Process Model Evaluation Optimizer 10

11 Evolutionary Algorithms Taxonomy Evolutionary Algorithms Evolution Strategies Genetic Algorithms Other Mixed-integer capabilities Emphasis on mutation Self-adaptation Small population sizes Deterministic selection Developed in Germany Theory focused on convergence velocity Discrete representations Emphasis on crossover Constant parameters Larger population sizes Probabilistic selection Developed in USA Theory focused on schema processing Evolutionary Progr. Differential Evol. GP PSO EDA Real-coded Gas 11

12 Generalized Evolutionary Algorithm t := 0; initialize(p(t)); evaluate(p(t)); while not terminate do P (t) := mating_selection(p(t)); P (t) := variation(p (t)); evaluate(p (t)); P(t+1) := environmental_selection(p (t) Q); t := t+1; od 12

13 Genetic Algorithms

14 Genetic Algorithms: Mutation Mutation by bit inversion with probability p m. p m identical for all bits. p m small (e.g., p m = 1/n). 14

15 Genetic Algorithms: Crossover Crossover applied with probability p c. p c identical for all individuals. k-point crossover: k points chosen randomly. Example: 2-point crossover. 15

16 Genetic Algorithms: Selection Fitness proportional: f fitness, λ population size Tournament selection: Randomly select q << λ individuals. Copy best of these q into next generation. Repeat λ times. q is the tournament size (often: q = 2). 16

17 Permutation Representations in GAs

18 Mutation operators for permutations Normal mutation operators lead to inadmissible solutions e.g. bit-wise mutation : let gene i have value j changing to some other value k would mean that k occurred twice and j no longer occurred Therefore must change at least two values Mutation parameter now reflects the probability that some operator is applied once to the whole string, rather than individually in each position After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

19 Insert Mutation for permutations Pick two allele values at random Move the second to follow the first, shifting the rest along to accommodate Note that this preserves most of the order and the adjacency information After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

20 Swap mutation for permutations Pick two alleles at random and swap their positions Preserves most of adjacency information (4 links broken), disrupts order more After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

21 Inversion mutation for permutations Pick two alleles at random and then invert the substring between them. Preserves most adjacency information (only breaks two links) but disruptive of order information After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

22 Scramble mutation for permutations Pick a subset of genes at random Randomly rearrange the alleles in those positions (note subset does not have to be contiguous) After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

23 Crossover operators for permutations Normal crossover operators will often lead to inadmissible solutions Many specialised operators have been devised which focus on combining order or adjacency information from the two parents After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

24 Order 1 crossover Idea is to preserve relative order that elements occur Informal procedure: 1. Choose an arbitrary part from the first parent 2. Copy this part to the first child 3. Copy the numbers that are not in the first part, to the first child: starting right from cut point of the copied part, using the order of the second parent and wrapping around at the end 4. Analogous for the second child, with parent roles reversed After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

25 Order 1 crossover example Copy randomly selected set from first parent Copy rest from second parent in order 1,9,3,8,2 After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

26 Partially Mapped Crossover (PMX) Informal procedure for parents P1 and P2: 1. Choose random segment and copy it from P1 2. Starting from the first crossover point look for elements in that segment of P2 that have not been copied 3. For each of these i look in the offspring to see what element j has been copied in its place from P1 4. Place i into the position occupied j in P2, since we know that we will not be putting j there (as is already in offspring) 5. If the place occupied by j in P2 has already been filled in the offspring k, put i in the position occupied by k in P2 6. Having dealt with the elements from the crossover segment, the rest of the offspring can be filled from P2. Second child is created analogously After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

27 PMX example Step 1 Step 2 Step 3 After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

28 Cycle crossover Basic idea: Each allele comes from one parent together with its position. Informal procedure: 1. Make a cycle of alleles from P1 in the following way. (a) Start with the first allele of P1. (b) Look at the allele at the same position in P2. (c) Go to the position with the same allele in P1. (d) Add this allele to the cycle. (e) Repeat step b through d until you arrive at the first allele of P1. 2. Put the alleles of the cycle in the first child on the positions they have in the first parent. 3. Take next cycle from second parent After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

29 Cycle crossover example Step 1: identify cycles Step 2: copy alternate cycles into offspring After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.

30 Evolution Strategies

31 Evolution Strategy Basics Mostly real-valued search space IR n also mixed-integer, discrete spaces Emphasis on mutation n-dimensional normal distribution expectation zero Different recombination operators Deterministic selection (µ, λ)-selection: Deterioration possible (µ+λ)-selection: Only accepts improvements λ >> µ, i.e.: Creation of offspring surplus Self-adaptation of strategy parameters. 31

32 Representation of search points Simple ES with 1/5 success rule: Exogenous adaptation of step size σ Mutation: N(0, σ) v a = ( x 1,..., x n ) Self-adaptive ES with single step size: One σ controls mutation for all x i Mutation: N(0, σ) v a = (( x 1,..., ), σ ) x n 32

33 Representation of search points Self-adaptive ES with individual step sizes: One individual σ i per x i Mutation: N i (0, σ i ) v a = (( x1,..., x n ),( σ1,..., σn)) Self-adaptive ES with correlated mutation: Individual step sizes One correlation angle per coordinate pair Mutation according to covariance matrix: N(0, C) v a = (( x1,..., x n ),( σ1,..., σ n),( α1,..., α n( n 1)/ 2)) 33

34 Evolution Strategy: Algorithms Mutation 34

35 Operators: Mutation one σ Self-adaptive ES with one step size: One σ controls mutation for all x i Mutation: N(0, σ) Individual before mutation v a = (( x1,..., xn), σ ) Individual after mutation v a = (( x,..., x ), σ ) x = i x i 1 σ = σ exp( τ N(0,1)) n 0 + σ N i (0,1) 1.: Mutation of step sizes 2.: Mutation of objective variables Here the new σ is used! 35

36 Operators: Mutation one σ Thereby τ 0 is the so-called learning rate Affects the speed of the σ-adaptation τ 0 bigger: faster but more imprecise τ 0 smaller: slower but more precise How to choose τ 0? According to recommendation of Schwefel*: τ 0 = 1 n *H.-P. Schwefel: Evolution and Optimum Seeking, Wiley, NY,

37 Operators: Mutation one σ Position of parents (here: 5) Contour lines of objective function Offspring of parent lies on the hyper sphere (for n > 10); Position is uniformly distributed 37

38 Pros and Cons: One σ Advantages: Simple adaptation mechanism Self-adaptation usually fast and precise Disadvantages: Bad adaptation in case of complicated contour lines Bad adaptation in case of very differently scaled object variables -100 < x i < 100 and e.g. -1 < x j < 1 38

39 Evolution Strategy: Algorithms Recombination 39

40 Operators: Recombination Only for µ > 1 Directly after Selektion Iteratively generates λ offspring: for i:=1 to λ do choose recombinant r1 uniformly at random od from parent_population; choose recombinant r2 <> r1 uniformly at random from parent population; offspring := recombine(r1,r2); add offspring to offspring_population; 40

41 Operators: Recombination How does recombination work? Discrete recombination: Variable at position i will be copied at random (uniformly distr.) from parent 1 or parent 2, position i. Parent 1 Parent 2 Offspring 41

42 Operators: Recombination Intermediate recombination: Variable at position i is arithmetic mean of Parent 1 and Parent 2, position i. x r1,1 x r2,1 Parent 1 Parent 2 Offspring ( x + x ) / 1 r1, 1 r2, 2 42

43 Operators: Recombination Global discrete recombination: Considers all parents Parent 1 Parent 2 Parent µ Offspring 43

44 Operators: Recombination Global intermediary recombination: x r1,1 Considers all parents x r2,1 Parent 1 Parent 2 Parent µ x rµ,1 Offspring 44 1 µ x ri, 1 µ i= 1

45 Evolution Strategy Algorithms Selection 45

46 Operators: (µ+λ)-selection (µ+λ)-selection means: µ parents produce λ offspring by (Recombination +) Mutation These µ+λ individuals will be considered together The µ best out of µ+λ will be selected ( survive ) Deterministic selection This method guarantees monotony Deteriorations will never be accepted = Actual solution candidate = New solution candidate Recombination may be left out Mutation always exists! 46

47 Operators: (µ,λ)-selection (µ,λ)-selection means: µ parents produce λ >> µ offspring by (Recombination +) Mutation λ offspring will be considered alone The µ best out of λ offspring will be selected Deterministic selection The method doesn t guarantee monotony Deteriorations are possible The best objective function value in generation t+1 may be worse than the best in generation t. 47

48 Operators: Selection Example: (2,3)-Selection Parents don t survive Parents don t survive! but a worse offspring. Example: (2+3)-Selection now this offspring survives. 48

49 Operators: Selection Possible occurrences of selection Exception! (1+1)-ES: One parent, one offspring, 1/5-Rule (1,λ)-ES: One Parent, λ offspring Example: (1,10)-Strategy One step size / n self-adaptive step sizes Mutative step size control Derandomized strategy (µ,λ)-es: µ > 1 parents, λ > µ offspring Example: (2,15)-Strategy Includes recombination Can overcome local optima (µ+λ)-strategies: elitist strategies 49

50 Evolution Strategy: Self adaptation of step sizes 50

51 Self-adaptation No deterministic step size control! Rather: Evolution of step sizes Biology: Repair enzymes, mutator-genes Why should this work at all? Indirect coupling: step sizes progress Good step sizes improve individuals Bad ones make them worse This yields an indirect step size selection 51

52 Self-adaptation: Example How can we test this at all? Need to know optimal step size Only for very simple, convex objective functions Here: Sphere model Dynamic sphere model f n * ( x) = ( x i xi i= 1 Optimum locations changes occasionally v ) 2 x v * : Optimum 52

53 Self-adaptation: Example Objective function value According to theory ff optimal step sizes Largest average and smallest step size measured in the population 53

54 Self-adaptation Self-adaptation of one step size Perfect adaptation Learning time for back adaptation proportional n Proofs only for convex functions Individual step sizes Experiments by Schwefel Correlated mutations Adaptation much slower 54

55 Mixed-Integer Evolution Strategies

56 Mixed-Integer Evolution Strategy Generalized optimization problem: 56

57 Mixed-Integer ES: Mutation Learning rates (global) Learning rates (global) Geometrical distribution Mutation Probabilities 57

58 Some Application Examples

59 Example 1 Courtesy of

60 MDO Crash / Statics / Dynamics Minimization of body mass Finite element mesh Crash ~ elements NVH ~ elements Independent parameters: Thickness of each unit: 109 Constraints: 18 Algorithm Avg. reduction (kg) Max. reduction (kg) Min. reduction (kg) Best so far NuTech ES

61 MDO Production Runs (I) Minimization of body mass Finite element mesh Lateral impact rear impact high speed Crash ~ elements NVH ~ elements Independent parameters: Thickness of each unit: 136 front impact high speed MDO Front impact low speed rear impact low speed Statics Dynamic Constraints: 47, resulting from various load cases 180 (10 x 18) shots ~ 12 days No statistical evaluation due to problem complexity

62 MDO Production Runs (II) NuTech s Evolution Strategy Initial Value Mass Generations 13,5 kg weight reduction by NuTech s ES. Beats best so far method significantly. Typically faster convergence velocity of ES. Reduction of development time from 5 to 2 weeks allows for process integration. Still potential for further improvement after 180 shots.

63 Example 2 Courtesy of

64 MDO ASF Front Optimization Pre-optimized Space-Frame-Concept improvement possible? Goal: Minimization of structural weight Degrees of freedom: Wall thicknesses of the semi-finished products sheet & profile Material characteristic profile Limitation of design space: Semi-finished products technology Technique for joining parts

65 MDO ASF Disciplines Damage according to insurance classification, Component Model, 2 CPUs Global dynamic stiffness, Trimmed Body, 1 CPU Front Crash (EURO NCAP), Complete Body 4 CPUs Resources per Design: 7 CPUs, approx. 23h

66 MDO Run Comparison Initial design, constraints violated Optimum (exp. 924), Constraints satisfied Masse Best so far optimizer Increased weight! Optimum (exp. 376), Constraints satisfied Mass Decreased weight!

67 MDO Optimization Run Fitness All Individuals Infeasible Feasible 107,13% 105,05% 100,00% Number of Individuals 67

68 Optimization of metal stamping process Objective: Minimization of defects in the produced parts. Variables: Geometric parameters and forces. ES finds very good results in short time Computationally expensive simulation 68

69 Bridgeman Casting Process Objective: Max. homogeneity of workpiece after Casting Process Variables: 18 continuous speed variables for Casting Schedule Computationally expensive simulation (up to 32h simulation time) Turbine Blade after Casting Initial (DoE) GCM (Commercial Gradient Based Method) Evolution Strategy 69

70 Steel Cube Temperature Control Minimize the deviation of temperture at the cube s Surface to 1000 C! Input: Temperature Profile (12 Variables 7 Temperatures and 5 Time-Steps) Algorithm 100 OFE 200 OFE 1000 OFE SQP Conjugate directions Pattern search DSCP/Rose nbrock Complex strategy T max (1,5)-DES (1,7)-DES (1,3)-DES (1,10)-DES MAES

71 Siemens C Reactor - Maximisation of growth speed - Minimisation of diameter differences FLUENT: Simulation of fluid flow CASTS: Calculation of Temperature and concentration field Optimsation of 15 process parameters: Production time 35 hours 30 hours Reaction Gas TCS Growing High Purity Silicon Rod 0 m/s 1.5 m/s 3 m/s 71

72 Multipoint Airfoil Optimiziation Objective: Find pressure profiles that are a compromise between two given target pressure distributions under two given flow conditions! Variables: 12 to 18 Bezier points for the airfoil Low Drag! Cruise High Lift! Start 72

73 Traffic Light Control Optimization Objective: Minimization of total delay / number of stops Variables: Green times for next switching schedule Dynamic optimization, depending on actual traffic Better results (3-5%) Higher flexibility than with traditional controllers 73

74 Optimization of elevator control Minimization of passenger waiting times. Better results (~10%) than with traditional controller. Dynamic optimization, depending on actual traffic. 74

75 Optimization of network routing Minimization of end-to-end-blockings under service constraints. Optimization of routing tables for existing, hard-wired networks. 10%-1000% improvement. 75

76 Automatic battery configuration Configuration and optimization of industry batteries. Based on user specifications, given to the system. Internet-Configurator (Baan, Hawker, NuTech). 76

77 Optimization of reactor fueling. Minimization of total costs. Creates new fuel assembly reload patterns. Clear improvements (1%-5%) of existing expert solutions. Huge cost saving. 77

78 Optical Coatings: Design Optimization Nonlinear mixed-integer problem, variable dimensionality. Minimize deviation from desired reflection behaviour. Excellent synthesis method; robust and reliable results. MOC-Problems: anti-reflection coating, dialetric filters Robust design: Sharp peaks vs. Robust peaks 78

79 Example: Intravascular Ultrasound Image Analysis Real-time high-resolution tomographic images from the inside of coronary vessels: 79

80 Intravascular Ultrasound Image Analysis Detected Features in an IVUS Image: Shadow Sidebranch Plague Vessel Border Lumen 80

81 Experimental Results on IVUS images Parameters for the lumen feature detector 81

82 Intravascular Ultrasound Image Analysis: Results On each of 5 data sets algorithm ran for 2804 evaluations 19,5h of total computing time Performance of the best found MI-ES parameter solutions A paired two-tailed t-test was performed on the difference measurements for each image dataset using a 95% confidence interval (p=0.05) The null-hypothesis is that the mean difference results of the best MI-ES individual and the default parameters are equal. Significant improvement over expert tuning 82

83 Case-Study: QUANTUM CONTROL EXPERIMENTS

84 Altering the Course of Quantum Dynamics Phenomena

85 Quantum Control Experiments (QCE) Yield, or success-rate, correspond to a measurement; no Hamiltonian required. Control (field) shaped through phase (freq. vs. time) Several levels of experimental uncertainties From QC theoretical perspective: severely constrained landscape: limited bandwidth, limited fluence, resolution, proper basis, etc. In practice: local traps, hard landscapes Topology versus Local Structures

86 Quantum Control Experiments

87 The Optical Table: Shaping the Pulse Figure courtesy of Jonathan Roslund

88 Figure courtesy of Jonathan Roslund

89 Single-Objective Experiments CMA-ES was observed to perform extremely well with small population sizes Recombination is necessary (Genetic Repair Hypothesis) [Arnold and Beyer] Robust, reproducible, reliable solutions

90

91 QCE Systems: ES vs. GA O. M. Shir, J. Roslund, T. Bäck, and H. Rabitz, Performance Analysis of Derandomized Evolution Strategies in Quantum Control Experiments, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO New York, NY, USA: ACM Press, 2008.

92 Evolutionary Biology and EAs Evolutionary Biology Individual Population Crossover, Mutation Fitness Adaptive landscape Evolutionary Computation Representation of a candidate solution Multiset of candidate solutions Probabilistic variation operators Objective function value Objective function 92

93 Literature H.-P. Schwefel: Evolution and Optimum Seeking, Wiley, NY, I. Rechenberg: Evolutionsstrategie 94, frommannholzboog, Stuttgart, Th. Bäck: Evolutionary Algorithms in Theory and Practice, Oxford University Press, NY, Th. Bäck, D.B. Fogel, Z. Michalewicz (Hrsg.): Handbook of Evolutionary Computation, Vols. 1,2, Institute of Physics Publishing,