1 Design of Experiments (DOE) Instructor: Thomas Oesterle
2 Instructor Thomas Oesterle thomas.oest@gmail.com
3 Agenda Introduction Planning the Experiment Selecting a Design Matrix Analyzing the Data Modeling the Data and Optimizing Input Exercises
4 Goals After this course you will be able to.. prepare for an experiment select a design (matrix) for your application analyze data using Minitab identify significant factors of your process judge the validity of your data use DOE tools to reduce variation
5 Course Guidelines Four 3-hour sessions Suggest 1 break per session (10 minutes) You are responsible for learning Introductory Course Focus on application Fun and an environment of learning Please participate in the quizzes, polls, and questions!
What is DOE and why should you use it? 6
7 Typical Challenges 90 s 2000 s Today
8 How to increase success rate Customer Focus According to PDMA the leading cause of product failure is not understanding the customer Establish effective NPD processes Quality Systems (ISO) Use of effective tools (Six Sigma) Data driven decision making Develop metrics to measure progress
9 What is DOE? DOE is an efficient procedure for planning and systematically executing experiments. Goal is to obtain knowledge on how the process inputs affect the process outputs. Input Process Output
10 Two Components of DOE Experimental Design Screening, characterization, optimization Analytical Procedure Statistical Significance Input Process Output
11 Use to establish capable processes Step 1: Mistake Proofing Systematically eliminate all process errors Step 2: Process Optimization Use Tools to Move process to hit a target Reduce variation
12 Best time to start with DOE tools? Early in the process to improve quality and reduce cost (Dr. Taguchi) to confirm important design inputs to produce robust designs
13 Principles of Good Experimentation Understand Variables 2 to 5 replicates to increase precision Process Replication Randomize Control Avoid false conclusions Change some variables while carefully controlling others
14 A Few Definitions Uncontrolled Co-Factors (Noise) Different Machine Different Raw Material Lot Controlled Factors (Input) Discrete Responses (Outputs) PROCESS Continuous Temperature, Humidity
15 Effectiveness of DOE Information on this page is based on instructor s interpretation of [1].
16 If used correctly, DOE allows you to learn the most with the fewest possible runs identify the factors you should focus on determine the effect of controlled factors learn about interactions between factors predict factor settings with lowest variation 50% improvement in efficiency and effectiveness
17 Obstacles to using DOE Perceived complexity of set-up and analysis No software to crunch numbers Corporate cultures Don t Need, I am a Technology Guru
18 Why not One-Factor-at-a-Time Even simple processes are very time consuming and expensive You will miss the optimum combination of factors You will miss interactions
19 Getting Payback from this Training 5/10 rule Hands-On Training Sessions Discuss with Peers/Monthly Reviews Numerous Applications
Engineering versus Experimentation The Steps of a DOE Prerequisites, Assumptions Proper Preparation is Key 20
21 Engineering vs. Experimental Design DOE in not a substitute for technical knowledge DOE incorporates current understanding Physics first
22 Example from Injection Molding Relative Viscosity versus Relative Shear Rate THICK High sensitivity to machine fill speed fluctuation VISCOSITY Crossover point Low sensitivity to machine fill speed fluctuation THIN SLOW FLOW RATE FAST
23 DOE Steps Plan Select Design Matrix Conduct Analyze Confirm Define Objective, Select Factors, Levels, Responses, Overall Strategy Screening or Modeling? Setup is automated by software Train Personnel, Be present (at least during first few experiments) Evaluate Measurement System Statistical Analysis, Graphs, Predict Responses Demonstrate that the prediction from transfer function is useful
24 Prerequisites and Assumptions Plan Select Design Matrix Conduct Understand process before you start a DOE. Is the process stable or does it drift? Is your measurement system capable of measuring the changes you hope to see? (Validate!) Analyze Confirm
25 Proper Planning is Key Plan Select Design Matrix Conduct Analyze Plan several smaller experiments instead of one large experiment (25% Rule) Define the Objective Determine factors (Check FMEA) Determine factor levels Determine Responses Are there more than one? Are all responses equally important? Confirm
26 Defining the Objective OBJECTIVE Manufacturing Troubleshooting Which option is best? Screening What are the important factors? Robust Design What factor settings make the product/process least sensitive to noise Modeling To predict process performance
27 Defining the Objective DOE Choices Types of Experiments Manufacturing Troubleshooting Which type of supplier/material is best? Reduce variation, adjust a target. Screening to Identify Key Factors Which factors affect mean and/or variation Factor Significance Reduce factors to between 2 and 5 to better manage
28 Defining the Objective DOE Choices Types of Experiments Characterization To study simple effects with up to 4 factors Surface Response Modeling To hit a target To minimize or maximize a response To reduce variation To make a process robust Seeking multiple goals (desirability function) Regression Modeling
29 Defining the Objective DOE Choices Types of Experiments Robust Design To find process or product settings that are least sensitive to noise (establish robust designs/processes)
30 Manufacturing Troubleshooting Factors Low High Mold Temp 100 150 F Barrel Temp Low High Cure Time 40 50 sec Inj. Velocity 1 3.1 Hold Pressure 200 1100 psi
31 Manufacturing Troubleshooting Responses Type Appearance Discrete 1 to 5
32 Modeling Effects Plot Response Surface Plot
33 Factors and Responses Objective: Grill Hamburgers as fast as possible Controlled Factors Amount of Flame Meat Temperature Noise Factors Ambient Temperature Patty Thickness Responses Time to Grill PROCESS Constant Factors Distance from Flame
34 Four Types of Controlled Factors 50 45 40 Scatter Plot Impacts average only R e s p o n s e 35 30 25 20 15 10 5 1(-) 2(+) A(A) Factors 40 30 Scatter Plot Impacts variation only R e s p o n s e 20 10 0 1(-) 2(+) B(B) Factors
35 Four Types of Controlled Factors 60 Scatter Plot Impacts both 50 R e s p o n s e 40 30 20 10 0 1(-) 2(+) C Factors 20 Scatter Plot No impact R e s p o n s e 15 10 5 1(-) 2(+) D Factors
36 Example Factor A shifts the mean Factor B effects variance Run A B R1 R2 R3 R4 1-1 -1 9 11 10 10 2-1 1 5 10 15 10 3 1-1 20 20 19 21 4 1 1 25 20 15 20 Mean Standard Deviation 50 Main Effects 5 Main Effects R E S P 40 30 20 10 S R E S P 4 3 2 1 0-1(-) 1(+) A(A) -1(-) 1(+) B(B) Factors -1(-) 1(+) AB 0-1(-) 1(+) A(A) -1(-) 1(+) B(B) Factors -1(-) 1(+) AB
37 Selecting and Scaling Factors Team Approach to identify (input) factors Use engineering judgment to determine factors Cause and Effect Diagram Review FMEA Be bold but not foolish setting factor levels Rule of Thumb: 20% beyond specification limit Select levels such that process still works! Verify selected levels before starting experiment
38 Range of Interest and Level Selection L2 L1
39 Area suited for 2 levels Over the levels of 100 to 130 the linear approximation appears to be useful. How about 130 to 160?
40 Area where 3 levels are required Over the range of 130 to 160 a three-level design would provide a much better approximation than a two-level design
41 Sample Size Rule of Thumb: n>40 per experiment for variable data 2 to 5 replications Multiple category response, n>10 per run Attribute (Pass/Fail) n must be large, not recommended for DOE It is far more important to consider the breadth of conditions rather than number of replicates!
42 Minitab, Sample Sizes Minitab Option STAT -> Power and Sample Size -> General Factorial Design
43 Minitab, Sample Sizes Goal: Determine the number of samples we need to get a power of 80% (0.8) Challenge: We don t know S! Suggestion: Look at a ratio between minimum difference and standard deviation. In this example we look at 4 ratios (0.5, 1, 1.5 and 2) 1: Enter Levels for all factors 2: Try 2,3 and 4 replicates 3: Enter 1,2,3 and 4 4: Enter 2 Entries in 3) and 4) give you a ratio of 0.5, 1, 1.5 and 2 -> make judgment! 1 4 3 2
44 Minitab, Sample Sizes We choose to detect one standard deviation. If we run 24 samples -> 63.3% Power If we run 32 samples -> 77.5% Power Power = 1-β β=probability of rejecting hypothesis when it is false (i.e. correctly detecting the difference) 3 H 0 : Not Guilty (Hypothesis) H A : Guilty (Hypothesis) True Result Not Guilty Guilty Accept H 0 Reject H 0 Correct Decision (Confidence Level) Incorrect Decision Type I Error Producer's Risk Incorrect Decision Type II Error Consumer Risk Correct Decision (Power of Test)
45 Sample Size, Simple Equation Approximates 80% Power and a desired detection of 1 standard deviation Example: Specification: 800 +/- 18 mm S: 3 mm You want to detect a shift of 2 mm n = (32* 3 2 ) / 2 2 = 288 /4 = 72
Two-Level Designs 46
47 The Design Matrix Plan Select Design Matrix Conduct Two-Level Designs are most popular. Why? Ideal for screening Simple Economical Analyze Confirm
48 The simplest two-level design matrix Displays the levels of each factor for all the run combinations in the experiment For Analysis, factor values are coded (-1,1 or +,-) (Coefficients) Vertically Balanced if Sum of Coded factors = 0 Factor Factor Run Temp (A) Press (B) Run Temp (A) Press (B) 1 100 600 1-1 -1 2 150 600 2 1-1 3 100 900 3-1 1 4 150 900 4 1 1 = 0 = 0
49 Orthogonal Analysis Matrix Vertically balanced Horizontally balanced Coded Factor Interaction Run A B A * B 1-1 -1 1 2 1-1 -1 3-1 1-1 4 1 1 1 = 0
50 Orthogonal Arrays Vertical Balance Horizontal Balance Run A B C 1-1 -1-1 2-1 -1 1 3-1 1-1 4-1 1 1 5 1-1 -1 6 1-1 1 7 1 1-1 8 1 1 1 Sum: 0 0 0 Run A B C 1-1 -1-1 2-1 -1 1 3-1 1-1 4-1 1 1 5 1-1 -1 6 1-1 1 7 1 1-1 8 1 1 1 Run A B C 1-1 -1-1 2-1 -1 1 3-1 1-1 4-1 1 1 5 1-1 -1 6 1-1 1 7 1 1-1 8 1 1 1
51 Orthogonal Arrays Run A B C AB AC BC ABC 1-1 -1-1 1 1 1-1 2-1 -1 1 1-1 -1 1 3-1 1-1 -1 1-1 1 4-1 1 1-1 -1 1-1 5 1-1 -1-1 -1 1 1 6 1-1 1-1 1-1 -1 7 1 1-1 1-1 -1-1 8 1 1 1 1 1 1 1 Sum: 0 0 0 0 0 0 0 Correlation between variables is zero r = ( x x)( y y) /( n 1) S i i x S y
Correlation in Minitab 52
53 Orthogonal Analysis Matrix Horizontally balanced versus unbalanced Coded Factor Interaction Run A B A * B 1-1 -1 1 2 1-1 -1 3-1 1-1 4 1 1 1 SUM 0 0 0 Coded Factor Interaction Run A B A * B 1-1 -1 1 2-1 -1 1 3 1 1 1 4 1 1 1 SUM 0 0 4
54 Example 1 Problem Statement An enzyme was processed and is stored in a buffered solution. The goal is to maintain the highest level of activity while in storage. Our process knowledge tells us that two ingredients of the buffered solution affect the activity: NaCl and EDTA. We want to know what happens if we vary the NaCl from 5% to 10% and the EDTA from 1% to 10%. Two samples were counted per run: Run NaCl A EDTA B Activity Y1 1 5 1 27 28 2 10 1 33 35 3 5 10 24 24 4 10 10 31 30 Activity Y2
55 Minitab Analysis (Main Effects) Is this a significant shift? We don t know yet.
56 Minitab Analysis (Interaction Plot) Lines Parallel: No Interaction Lines Intersect: Interaction Two factors interact if the influence of one factor is impacted by the level of another factor.
57 Comparative Surface Plots With Interaction No Interaction
58 Interactions Maximum height of projectile released from a trebuchet. W1=weight of rock Question: Is there an interaction between weight and release angle? The answer is no if we are considering release angles of greater than 150, yes if we are considering release angles less than 150.
59 Minitab Analysis (Pareto Chart) Red Line is Threshold. Anything beyond red line is significant
60 Minitab Analysis (Contour Plot) Question: Suppose your specification is A >= 32. Please provide a mixture of the NaCl and EDTA that would yield in activity levels greater than 32!
Minitab Analysis (Response Surface) 61
Minitab Tutorial 62
63 Minitab Analysis p <= 0.05 -> factor significant R-Sq: Sum of Sq. Residuals R.O.T.: > 80%
64 Model for this experiment Use coded numbers (-1,1) for A and B Three important assumptions Two levels Orthogonal Array Coded numbers Minitab use MLR to generate transfer function
65 Model Verification = 29.000 + 3.250 * (1) + [-1.750 * (-1)] = 34
66 Last Step: Confirmation Run Run between 5 and 50 samples Sample size depends on ratio of standard deviation and desired detection interval Measure Results and Calculate Average Calculate Confidence Interval Does the average confirm the model?
67 Question Pareto Chart of the Effects (response is L, Alpha = 0.05) Main Effects Plot for L Data Means Term A B C AC AB 0.5646 F actor A B C Name Pressure Temp Time Mean 3.2 3.0 2.8 2.6 2.4 3.2 1100 A C 1900 210 B 290 ABC 3.0 BC 0.0 0.1 Lenth's PSE = 0.15 0.2 0.3 0.4 0.5 Effect 0.6 0.7 0.8 0.9 2.8 2.6 2.4 1 2 A manufacturing associate asks for your help with interpreting these two charts. Which factor is significant? Please answer Poll on next page!
68 Question Contour Plot of L vs Temp, Pressure 285 2.8 3.2 3.4 Hold Values Time 1 270 2.4 Temp 255 240 225 2.2 2.6 3.0 210 1100 1200 1300 1400 1500 1600 Pressure 1700 1800 1900 The manufacturing specification is 2.6 +/- 0.1. What process settings would you recommend to make good parts? Please answer poll on next page!
69 Design Selection Guideline Number of Factors Screening Objective 2 4 Full Factorial Fractional Factorial > 5 Fractional Factorial Plackett-Burman Response Surface Objective Central Composite Box-Behnken Screen first to reduce number of factors
70 Full Factorial Designs A design with all possible high/low combinations of all the input factors is called a full factorial design in two levels. Run X1 X2 X3 1-1 -1-1 2 1-1 -1 3-1 1-1 4 1 1-1 5-1 -1 1 6 1-1 1 7-1 1 1 8 1 1 1
71 Full Factorial Designs If there are k factors each at 2 levels, this design has 2 k runs. Number of Factors Number of Runs 2 4 3 8 4 16 5 32 6 64 7 128
72 Example of Full Factorial (3 Factors) Injection Molding Challenge A part for the stage of a microscope is molded. Since the microscope allows precise positioning of samples, the process must produce a part with a length of 12.007 ± 0.003 mm.
73 Exercise Set up a Design Matrix in Minitab with 3 replicates Create Main Effects, Pareto, and Interaction Charts Identify significant factors Using a contour plot identify 2 factor settings that will result in a 12.007 long part. The process technician established the following parameters and levels:
74 Measurement Results Run Mold Temp Inj. Time Hold Press L1 L2 L3 StDev 1 90 3 5000 12.0039 12.0038 12.0039 0.0001 2 110 3 5000 12.0020 12.0000 12.0040 0.0020 3 90 5 5000 12.0052 12.0050 12.0051 0.0001 4 110 5 5000 12.0030 12.0080 12.0000 0.0040 5 90 3 7000 12.0147 12.0148 12.0149 0.0001 6 110 3 7000 12.0150 12.0190 12.0100 0.0045 7 90 5 7000 12.0151 12.0153 12.0154 0.0002 8 110 5 7000 12.0140 12.0190 12.0100 0.0045