Evolutionary Algorithms
Overview Introduction: Optimization and EAs Genetic Algorithms Evolution Strategies Theory Examples 2
Background I Biology = Engineering (Daniel Dennett) 3
Background II DNA molecule Carries genetic information Human DNA: 3. 10 9 Base pairs 310 4 9 Combinations Genotype
Thought Experiment: 10 80 Elementary particles in the universe 10 40 Time steps since big bang 10 120 Possible computations in the universe 310 4 9 is faaaaaar larger!
Optimization f : objective function High-dimensional Non-linear, multimodal Discontinuous, noisy, dynamic M M1 M2... Mn heterogeneous Restrictions possible over Good local, robust optimum desired Realistic landscapes are like that! Local, robust optimum Global Minimum 6
Dynamic Optimization Dynamic Function 30-dimensional 3D-Projection 7
Optimization Creating Innovation Illustrative Example: Optimize Efficiency Initial: Evolution: 32% Improvement in Efficiency! 8
Evolutionary Algorithm Principles
Model-Optimization-Action quality Function 15 i = weighti i= 1 scalei calculated desired 2 i Model from Data Simulation Experiment Function(s) r ( y) =... f i Subjective Business Process Model Evaluation Optimizer 10
Evolutionary Algorithms Taxonomy Evolutionary Algorithms Evolution Strategies Genetic Algorithms Other Mixed-integer capabilities Emphasis on mutation Self-adaptation Small population sizes Deterministic selection Developed in Germany Theory focused on convergence velocity Discrete representations Emphasis on crossover Constant parameters Larger population sizes Probabilistic selection Developed in USA Theory focused on schema processing Evolutionary Progr. Differential Evol. GP PSO EDA Real-coded Gas 11
Generalized Evolutionary Algorithm t := 0; initialize(p(t)); evaluate(p(t)); while not terminate do P (t) := mating_selection(p(t)); P (t) := variation(p (t)); evaluate(p (t)); P(t+1) := environmental_selection(p (t) Q); t := t+1; od 12
Genetic Algorithms
Genetic Algorithms: Mutation 0 1 1 1 0 1 0 1 0 0 0 0 0 1 0 0 1 1 1 0 0 0 1 0 1 0 0 0 1 0 Mutation by bit inversion with probability p m. p m identical for all bits. p m small (e.g., p m = 1/n). 14
Genetic Algorithms: Crossover Crossover applied with probability p c. p c identical for all individuals. k-point crossover: k points chosen randomly. Example: 2-point crossover. 15
Genetic Algorithms: Selection Fitness proportional: f fitness, λ population size Tournament selection: Randomly select q << λ individuals. Copy best of these q into next generation. Repeat λ times. q is the tournament size (often: q = 2). 16
Permutation Representations in GAs
Mutation operators for permutations Normal mutation operators lead to inadmissible solutions e.g. bit-wise mutation : let gene i have value j changing to some other value k would mean that k occurred twice and j no longer occurred Therefore must change at least two values Mutation parameter now reflects the probability that some operator is applied once to the whole string, rather than individually in each position After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Insert Mutation for permutations Pick two allele values at random Move the second to follow the first, shifting the rest along to accommodate Note that this preserves most of the order and the adjacency information After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Swap mutation for permutations Pick two alleles at random and swap their positions Preserves most of adjacency information (4 links broken), disrupts order more After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Inversion mutation for permutations Pick two alleles at random and then invert the substring between them. Preserves most adjacency information (only breaks two links) but disruptive of order information After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Scramble mutation for permutations Pick a subset of genes at random Randomly rearrange the alleles in those positions (note subset does not have to be contiguous) After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Crossover operators for permutations Normal crossover operators will often lead to inadmissible solutions 1 2 3 4 5 5 4 3 2 1 1 2 3 2 1 5 4 3 4 5 Many specialised operators have been devised which focus on combining order or adjacency information from the two parents After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Order 1 crossover Idea is to preserve relative order that elements occur Informal procedure: 1. Choose an arbitrary part from the first parent 2. Copy this part to the first child 3. Copy the numbers that are not in the first part, to the first child: starting right from cut point of the copied part, using the order of the second parent and wrapping around at the end 4. Analogous for the second child, with parent roles reversed After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Order 1 crossover example Copy randomly selected set from first parent Copy rest from second parent in order 1,9,3,8,2 After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Partially Mapped Crossover (PMX) Informal procedure for parents P1 and P2: 1. Choose random segment and copy it from P1 2. Starting from the first crossover point look for elements in that segment of P2 that have not been copied 3. For each of these i look in the offspring to see what element j has been copied in its place from P1 4. Place i into the position occupied j in P2, since we know that we will not be putting j there (as is already in offspring) 5. If the place occupied by j in P2 has already been filled in the offspring k, put i in the position occupied by k in P2 6. Having dealt with the elements from the crossover segment, the rest of the offspring can be filled from P2. Second child is created analogously After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
PMX example Step 1 Step 2 Step 3 After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Cycle crossover Basic idea: Each allele comes from one parent together with its position. Informal procedure: 1. Make a cycle of alleles from P1 in the following way. (a) Start with the first allele of P1. (b) Look at the allele at the same position in P2. (c) Go to the position with the same allele in P1. (d) Add this allele to the cycle. (e) Repeat step b through d until you arrive at the first allele of P1. 2. Put the alleles of the cycle in the first child on the positions they have in the first parent. 3. Take next cycle from second parent After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Cycle crossover example Step 1: identify cycles Step 2: copy alternate cycles into offspring After: Eiben and Smith, Introduction to Evolutionary Algorithms, Springer, Berlin, 2003.
Evolution Strategies
Evolution Strategy Basics Mostly real-valued search space IR n also mixed-integer, discrete spaces Emphasis on mutation n-dimensional normal distribution expectation zero Different recombination operators Deterministic selection (µ, λ)-selection: Deterioration possible (µ+λ)-selection: Only accepts improvements λ >> µ, i.e.: Creation of offspring surplus Self-adaptation of strategy parameters. 31
Representation of search points Simple ES with 1/5 success rule: Exogenous adaptation of step size σ Mutation: N(0, σ) v a = ( x 1,..., x n ) Self-adaptive ES with single step size: One σ controls mutation for all x i Mutation: N(0, σ) v a = (( x 1,..., ), σ ) x n 32
Representation of search points Self-adaptive ES with individual step sizes: One individual σ i per x i Mutation: N i (0, σ i ) v a = (( x1,..., x n ),( σ1,..., σn)) Self-adaptive ES with correlated mutation: Individual step sizes One correlation angle per coordinate pair Mutation according to covariance matrix: N(0, C) v a = (( x1,..., x n ),( σ1,..., σ n),( α1,..., α n( n 1)/ 2)) 33
Evolution Strategy: Algorithms Mutation 34
Operators: Mutation one σ Self-adaptive ES with one step size: One σ controls mutation for all x i Mutation: N(0, σ) Individual before mutation v a = (( x1,..., xn), σ ) Individual after mutation v a = (( x,..., x ), σ ) x = i x i 1 σ = σ exp( τ N(0,1)) n 0 + σ N i (0,1) 1.: Mutation of step sizes 2.: Mutation of objective variables Here the new σ is used! 35
Operators: Mutation one σ Thereby τ 0 is the so-called learning rate Affects the speed of the σ-adaptation τ 0 bigger: faster but more imprecise τ 0 smaller: slower but more precise How to choose τ 0? According to recommendation of Schwefel*: τ 0 = 1 n *H.-P. Schwefel: Evolution and Optimum Seeking, Wiley, NY, 1995. 36
Operators: Mutation one σ Position of parents (here: 5) Contour lines of objective function Offspring of parent lies on the hyper sphere (for n > 10); Position is uniformly distributed 37
Pros and Cons: One σ Advantages: Simple adaptation mechanism Self-adaptation usually fast and precise Disadvantages: Bad adaptation in case of complicated contour lines Bad adaptation in case of very differently scaled object variables -100 < x i < 100 and e.g. -1 < x j < 1 38
Evolution Strategy: Algorithms Recombination 39
Operators: Recombination Only for µ > 1 Directly after Selektion Iteratively generates λ offspring: for i:=1 to λ do choose recombinant r1 uniformly at random od from parent_population; choose recombinant r2 <> r1 uniformly at random from parent population; offspring := recombine(r1,r2); add offspring to offspring_population; 40
Operators: Recombination How does recombination work? Discrete recombination: Variable at position i will be copied at random (uniformly distr.) from parent 1 or parent 2, position i. Parent 1 Parent 2 Offspring 41
Operators: Recombination Intermediate recombination: Variable at position i is arithmetic mean of Parent 1 and Parent 2, position i. x r1,1 x r2,1 Parent 1 Parent 2 Offspring ( x + x ) / 1 r1, 1 r2, 2 42
Operators: Recombination Global discrete recombination: Considers all parents Parent 1 Parent 2 Parent µ Offspring 43
Operators: Recombination Global intermediary recombination: x r1,1 Considers all parents x r2,1 Parent 1 Parent 2 Parent µ x rµ,1 Offspring 44 1 µ x ri, 1 µ i= 1
Evolution Strategy Algorithms Selection 45
Operators: (µ+λ)-selection (µ+λ)-selection means: µ parents produce λ offspring by (Recombination +) Mutation These µ+λ individuals will be considered together The µ best out of µ+λ will be selected ( survive ) Deterministic selection This method guarantees monotony Deteriorations will never be accepted = Actual solution candidate = New solution candidate Recombination may be left out Mutation always exists! 46
Operators: (µ,λ)-selection (µ,λ)-selection means: µ parents produce λ >> µ offspring by (Recombination +) Mutation λ offspring will be considered alone The µ best out of λ offspring will be selected Deterministic selection The method doesn t guarantee monotony Deteriorations are possible The best objective function value in generation t+1 may be worse than the best in generation t. 47
Operators: Selection Example: (2,3)-Selection Parents don t survive Parents don t survive! but a worse offspring. Example: (2+3)-Selection now this offspring survives. 48
Operators: Selection Possible occurrences of selection Exception! (1+1)-ES: One parent, one offspring, 1/5-Rule (1,λ)-ES: One Parent, λ offspring Example: (1,10)-Strategy One step size / n self-adaptive step sizes Mutative step size control Derandomized strategy (µ,λ)-es: µ > 1 parents, λ > µ offspring Example: (2,15)-Strategy Includes recombination Can overcome local optima (µ+λ)-strategies: elitist strategies 49
Evolution Strategy: Self adaptation of step sizes 50
Self-adaptation No deterministic step size control! Rather: Evolution of step sizes Biology: Repair enzymes, mutator-genes Why should this work at all? Indirect coupling: step sizes progress Good step sizes improve individuals Bad ones make them worse This yields an indirect step size selection 51
Self-adaptation: Example How can we test this at all? Need to know optimal step size Only for very simple, convex objective functions Here: Sphere model Dynamic sphere model f n * ( x) = ( x i xi i= 1 Optimum locations changes occasionally v ) 2 x v * : Optimum 52
Self-adaptation: Example Objective function value According to theory ff optimal step sizes Largest average and smallest step size measured in the population 53
Self-adaptation Self-adaptation of one step size Perfect adaptation Learning time for back adaptation proportional n Proofs only for convex functions Individual step sizes Experiments by Schwefel Correlated mutations Adaptation much slower 54
Mixed-Integer Evolution Strategies
Mixed-Integer Evolution Strategy Generalized optimization problem: 56
Mixed-Integer ES: Mutation Learning rates (global) Learning rates (global) Geometrical distribution Mutation Probabilities 57
Some Application Examples
Example 1 Courtesy of
MDO Crash / Statics / Dynamics Minimization of body mass Finite element mesh Crash ~ 130.000 elements NVH ~ 90.000 elements Independent parameters: Thickness of each unit: 109 Constraints: 18 Algorithm Avg. reduction (kg) Max. reduction (kg) Min. reduction (kg) Best so far -6.6-8.3-3.3 NuTech ES -9.0-13.4-6.3
MDO Production Runs (I) Minimization of body mass Finite element mesh Lateral impact rear impact high speed Crash ~ 1.000.000 elements NVH ~ 300.000 elements Independent parameters: Thickness of each unit: 136 front impact high speed MDO Front impact low speed rear impact low speed Statics Dynamic Constraints: 47, resulting from various load cases 180 (10 x 18) shots ~ 12 days No statistical evaluation due to problem complexity
MDO Production Runs (II) NuTech s Evolution Strategy Initial Value Mass Generations 13,5 kg weight reduction by NuTech s ES. Beats best so far method significantly. Typically faster convergence velocity of ES. Reduction of development time from 5 to 2 weeks allows for process integration. Still potential for further improvement after 180 shots.
Example 2 Courtesy of
MDO ASF Front Optimization Pre-optimized Space-Frame-Concept improvement possible? Goal: Minimization of structural weight Degrees of freedom: Wall thicknesses of the semi-finished products sheet & profile Material characteristic profile Limitation of design space: Semi-finished products technology Technique for joining parts
MDO ASF Disciplines Damage according to insurance classification, Component Model, 2 CPUs Global dynamic stiffness, Trimmed Body, 1 CPU Front Crash (EURO NCAP), Complete Body 4 CPUs Resources per Design: 7 CPUs, approx. 23h
MDO Run Comparison Initial design, constraints violated Optimum (exp. 924), Constraints satisfied Masse Best so far optimizer Increased weight! Optimum (exp. 376), Constraints satisfied Mass Decreased weight!
MDO Optimization Run Fitness All Individuals Infeasible Feasible 107,13% 105,05% 100,00% Number of Individuals 67
Optimization of metal stamping process Objective: Minimization of defects in the produced parts. Variables: Geometric parameters and forces. ES finds very good results in short time Computationally expensive simulation 68
Bridgeman Casting Process Objective: Max. homogeneity of workpiece after Casting Process Variables: 18 continuous speed variables for Casting Schedule Computationally expensive simulation (up to 32h simulation time) Turbine Blade after Casting Initial (DoE) GCM (Commercial Gradient Based Method) Evolution Strategy 69
Steel Cube Temperature Control Minimize the deviation of temperture at the cube s Surface to 1000 C! Input: Temperature Profile (12 Variables 7 Temperatures and 5 Time-Steps) Algorithm 100 OFE 200 OFE 1000 OFE SQP 148 67.7 59 Conjugate directions Pattern search DSCP/Rose nbrock Complex strategy 135 135 135 157 147 48 88 38 29 152 152 111 T max (1,5)-DES 53 48 48 (1,7)-DES 110 58 58 (1,3)-DES 91 67 67 (1,10)-DES 152 80 28 MAES 122 65 12 70
Siemens C Reactor - Maximisation of growth speed - Minimisation of diameter differences FLUENT: Simulation of fluid flow CASTS: Calculation of Temperature and concentration field Optimsation of 15 process parameters: Production time 35 hours 30 hours Reaction Gas TCS Growing High Purity Silicon Rod 0 m/s 1.5 m/s 3 m/s 71
Multipoint Airfoil Optimiziation Objective: Find pressure profiles that are a compromise between two given target pressure distributions under two given flow conditions! Variables: 12 to 18 Bezier points for the airfoil Low Drag! Cruise High Lift! Start 72
Traffic Light Control Optimization Objective: Minimization of total delay / number of stops Variables: Green times for next switching schedule Dynamic optimization, depending on actual traffic Better results (3-5%) Higher flexibility than with traditional controllers 73
Optimization of elevator control Minimization of passenger waiting times. Better results (~10%) than with traditional controller. Dynamic optimization, depending on actual traffic. 74
Optimization of network routing Minimization of end-to-end-blockings under service constraints. Optimization of routing tables for existing, hard-wired networks. 10%-1000% improvement. 75
Automatic battery configuration Configuration and optimization of industry batteries. Based on user specifications, given to the system. Internet-Configurator (Baan, Hawker, NuTech). 76
Optimization of reactor fueling. Minimization of total costs. Creates new fuel assembly reload patterns. Clear improvements (1%-5%) of existing expert solutions. Huge cost saving. 77
Optical Coatings: Design Optimization Nonlinear mixed-integer problem, variable dimensionality. Minimize deviation from desired reflection behaviour. Excellent synthesis method; robust and reliable results. MOC-Problems: anti-reflection coating, dialetric filters Robust design: Sharp peaks vs. Robust peaks 78
Example: Intravascular Ultrasound Image Analysis Real-time high-resolution tomographic images from the inside of coronary vessels: 79
Intravascular Ultrasound Image Analysis Detected Features in an IVUS Image: Shadow Sidebranch Plague Vessel Border Lumen 80
Experimental Results on IVUS images Parameters for the lumen feature detector 81
Intravascular Ultrasound Image Analysis: Results On each of 5 data sets algorithm ran for 2804 evaluations 19,5h of total computing time Performance of the best found MI-ES parameter solutions A paired two-tailed t-test was performed on the difference measurements for each image dataset using a 95% confidence interval (p=0.05) The null-hypothesis is that the mean difference results of the best MI-ES individual and the default parameters are equal. Significant improvement over expert tuning 82
Case-Study: QUANTUM CONTROL EXPERIMENTS
Altering the Course of Quantum Dynamics Phenomena
Quantum Control Experiments (QCE) Yield, or success-rate, correspond to a measurement; no Hamiltonian required. Control (field) shaped through phase (freq. vs. time) Several levels of experimental uncertainties From QC theoretical perspective: severely constrained landscape: limited bandwidth, limited fluence, resolution, proper basis, etc. In practice: local traps, hard landscapes Topology versus Local Structures
Quantum Control Experiments
The Optical Table: Shaping the Pulse Figure courtesy of Jonathan Roslund
Figure courtesy of Jonathan Roslund
Single-Objective Experiments CMA-ES was observed to perform extremely well with small population sizes Recombination is necessary (Genetic Repair Hypothesis) [Arnold and Beyer] Robust, reproducible, reliable solutions
QCE Systems: ES vs. GA O. M. Shir, J. Roslund, T. Bäck, and H. Rabitz, Performance Analysis of Derandomized Evolution Strategies in Quantum Control Experiments, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO-2008. New York, NY, USA: ACM Press, 2008.
Evolutionary Biology and EAs Evolutionary Biology Individual Population Crossover, Mutation Fitness Adaptive landscape Evolutionary Computation Representation of a candidate solution Multiset of candidate solutions Probabilistic variation operators Objective function value Objective function 92
Literature H.-P. Schwefel: Evolution and Optimum Seeking, Wiley, NY, 1995. I. Rechenberg: Evolutionsstrategie 94, frommannholzboog, Stuttgart, 1994. Th. Bäck: Evolutionary Algorithms in Theory and Practice, Oxford University Press, NY, 1996. Th. Bäck, D.B. Fogel, Z. Michalewicz (Hrsg.): Handbook of Evolutionary Computation, Vols. 1,2, Institute of Physics Publishing, 2000. 93