Bradley Novic, Ph.D. PhaseTwo Analytics, LLC Data Knowledge Intelligence.

Transcription

1 Bradley Novic, Ph.D., LLC Data Knowledge Intelligence

2 Reluctant to analyze manufacturing data Requests to analyze plant data (circa 2000) Influence of J. S. Hunter PARC Analysis skepticism Why am I seeing useful relationships? The bad news & the good news Bad news: Processes are out of control Good news: Lots of strong signals in the data Bad news: Data assembly is painful so no one is looking at it Good news: LOTS of low hanging fruit!!! Big opportunity!!! Probability of successful discovery higher than I anticipated High return on investment -> increased demand Realization: Mining mfg data has perils mitigated by JMP Case studies will highlight perils related to: Data: Time scale issues manufacturing specific Data integration with non-matching timestamps Integration of static & time-based data Known relationships UNKNOWN TO YOU! value of visualization! Prediction vs Control the need for guidance in tree building!

3 Data Mining Would be Great if it weren t for the Data 70-80% of time is spent wrestling with data issues Data Access multiple, disparate databases Completeness having all the right data observability! Data cleaning Missing data Data integration & alignment joining, concatenating, stacking Time scale differences Aligning datasets with non-matching time stamps Integrating Static data + time-based data (batch processes)

4 A rolling process produces coils about 1 every 20 minutes The rolling process timestamp reflects the start of coil processing The rolling process dataset contains process variables, process performance metrics & product quality Lubricant, sprayed on rolls and the coil, is influential but quantification of effects is needed Problem: Lubricant chemistry data are measured less frequently Lube timestamp will not match the coil process timestamp but it varies smoothly Alignment is required to model results as a function of both process and chemistry Here s a way of aligning data with non-matching timestamps in JMP using spline predictions

5 Objective: Align variable LubeChem1 ( measured 1/day ) with Rolling Process data measured 3/hr LubeChem1 raw data flexible spline fit Next: Save the prediction formula Note: use the same name for the time stamp as in the rolling data set Create a new variable in the rolling data set ( LubeChem1_Pred ) Use the spline prediction formula as the formula for LubeChem1_Pred Here are the aligned (predicted) LubeChem1 data:

6 Time-based variables such as: Static variables such as: An Ingot Casting Example Static Data (1/cast) static conditions single point meas Holding furnace temp Coolant Additive % Grain refiner set point Drop cast start Steady state drop rate Ingot head ml set point Metal treatment set point Ambient dew point, % humidity Temperature Time-based Data per cast trajectories about 1,000 data points per cast Filter Temperature Cast trough temp Coolant temperature Coolant flow Dist Trough Metal level Mold level (5) Mold Controller(5) Casting rate How do we combine the data to model the impact of all of these data on some response???

7 Mold Level Coolant temp Cast Temp Filter temp Integrating Static Data (1/batch) with Time-based batch trajectories An Ingot Casting Example Static Data (1/cast) Holding furnace temp Coolant Additive % Grain refiner set point Start drop rate Steady state drop rate Ingot head ml set point Metal treatment set point Ambient dewpoint, % humidity Temperature Time-based Data (cast trajectories) FilterTemperature Cast temp Coolant temperature Coolant flow Dist Trough Metal level Mold level (5) Mold Controller(5) 1000 data points but not 1000 pieces of information!!!

8 Mold Level Coolant temp Cast Temp Filter temp Integrating Static Data (1/batch) with Time-based batch trajectories through feature extraction An Ingot Casting Example Static Data (1/drop) Holding furnace temp X1 Coolant Additive % X2 Grain refiner set point X3 Start drop rate X4 Steady state drop rate X5 Ingot head ml set point X6 Metal treatment set point X7 Ambient dewpoint, X8 % humidity X9 Temperature X10 Time-based trajectory features Filter Temperature F1, F2, F3, F4 Cast temp F5, F6, F7 Coolant temperature F8, F9 Mold level F10, F11, F12, F13 Coolant flow etc Dist Trough Metal level etc Mold Controller(5) etc MaxFiltTemp( L<4) F1 Init_FiltTemp( L<4) F3 MaxCASTTemp( L<4) F5 MinCoolTemp F8 MinFiltTemp( L>4) F4 MaxFiltTemp( L>4 F2 MaxCASTTemp( L>4) F6 MinCASTTemp( L>4) F7 MaxCoolTemp F9 Model Y = F(X1, X2, X10, F1, F2,,Fi) Lvl Max F10 Lvl Slope2 F11 Lvl Final F13 Lvl Slope1 F12

9 Tool Friction You Can Observe a Lot Just By Watching Yogi Berra Clean, merge, align, characterize your data THEN EXPLORE IT!!! Fit Y by X, time plots, Scatterplots, Graph builder, PCA Case Study 3: Hidden Effects of Lubrication on Extrusion Process Friction Parts are extruded through a die and lubricant is sprayed on the part as it enters the die. Friction variation can occur and cause part surface damage & scrap. Lube components are measured but are difficult to control tightly. Lube effects are suspected qualitatively but never quantified A plant study was conducted. Key lube components were set at specific levels for periods of time with changes made from time period to time period. Data were thrown over the fence for analysis ~9000 obs., ~30 predictors Research question #1: Do we see significant changes in mean measured die friction from time period to time period? Naïve answer (based on a Fit Y by X of the response versus test periods: Not much! Unusually high level of variability But wait! There s More!

10 Die Friction Die Friction Resids Die Friction Die Friction Die Friction Closer Inspection/Visualization of The Data Reveals More Plot of the response vs Time Magnification reveals a repeating systematic downward trend Further visual exploration reveals the culprit is Die Life. Die Life is the primary driver of the response The effect of Die Life was known(qualitatively) but not communicated to me It s effect obscures the effects of studied variables If partitioning were used to model Die Friction, the tree would primarily split on die life and miss other important variables Solution??? Regress out the known die life effect A smoothing spline was used to regress out the effect CompareTest Periods and the effects of other studied variables using Residuals Comparisons are now clearer Facilitated follow-on modeling

11 Peril #3: Decision Trees Selecting Candidates that are Predictive But not Useful or Provide Incomplete Control Guidance In manufacturing prediction is not good enough! The end game in manufacturing is control For control purposes, a predictor must be causative Decision trees can select variables that are predictive but make no sense or are not useful to a process engineer from a control standpoint Tree growth needs to be guided by process knowledge JMP permits guiding the partitioning process enabling reality checks!!! Decision trees may also miss important variables. They may be correlated with the top choice in a split, can share prediction/control capabilities but partitioning may ignore them In non-manufacturing classification tree applications we re not looking to control but to classify even if the predictor isn t root cause it may be good enough! Common Issues: Nonsense correlations Non-actionable predictors Predictors that are actually responses After further review Important control variables are missed because of correlation between key predictors

12 Nonsense Correlations Storks vs population in Oldenburg, Germany So, would getting rid of storks be an effective mechanism for population control??? The reality is that increased population causes the increase in the number of Storks Does watching TV increase life expectancy? "Televisions, Physicians, and Life Expectancy" in the _Journal of Statistics Education_ (Rossman 1994) Data from a number of countries were gathered from The World Almanac and Book of Facts 1993 So, to improve life expectancy do we ship boat loads of TVs to those countries with low life expectancy???

13 Case Study 4: Sheet Scrap Integrating Process Knowledge With Partitioning Sheet scrap was a >$1MM/yr problem The problem was seasonal Engineers weren t sure if the cause was related to ingot casting, heat treatment or rolling A swat team of metallurgists, process engineers and a data scientist was formed Data were being recorded throughout the manufacturing flow path Data collection, assembly, characterization/feature extraction was a HUGE task Shortly after assembling & cleaning the data a quick partitioning exploration yielded the following:

14 Partitioning Step 0 Occurrence rate ~10% Partitioning Step 1: Problem Solution Emerges! Although the root cause emerged quickly and required verification some additional exploration of the data took place that provided further discovery insights

15 Sheet Scrap (cont.) Step 3 Over-ruled by the process engineers and MC1 is locked out Interesting but can t be implemented Step 3.1 Tells us that the remainder of the problem would go away if dew point<63.3 BUT we can t control dew point in a plant Step 4 Also over-ruled by the process engineers and MC2 is locked out Step 4.1 Produced an interesting countermeasure Step 5 Was a what-if scenario where we locked out the root cause to explore additional process countermeasures Step 6 identified a temperature countermeasure that made total sense to our metallurgists!!!

16 Sheet Scrap Case Study Summary Points 1. Data assembly was a PAINFUL task!!! 2. Once data assembly was completed discovery was FAST!!!! This problem was ongoing for more than 2 years!!! Once the data were assembled discovery occurred within minutes!!! 3. Discovery was validated through a designed experiment 4. JMP permitted control of the partitioning steps with the ability to override split selection based on engineering judgment and 1 st principles Nonsense correlations can be easily dismissed using JMP Splits that are not practically useful (but perhaps interesting nonetheless) can be noted and then locked out 5. Interaction between the Data Scientist and Engineer during the partitioning process is critical to getting to root cause understanding

17 Case Study 5: Root Cause Discovery In Part Fabrication with Correlated Predictors **Guiding The Discovery Process, Part II** Key Learnings are: The Value of JMP Visualization Must be attentive to correlated Predictors when partitioning May be identified in Candidate list.. pay attention! Important (root cause) predictors can be missed Partitioning can obscure their contribution Only one may be selected as Predictive BUT More than One may actually be Needed for Control Engineering judgment/1 st Principles Should guide Partitioning Complement Partitioning with Multivariate Methods Once the data are assembled, Discovery is FAST!!!

18 Problem: Dimensional Tolerance In A Part Fabrication Process Process entails production of a highly engineered part Processing irregularities produced thick and off tolerance parts (6% scrap and 50% rework) Process data were in multiple, disparate databases that required extraction, alignment, characterization and cleaning Attempts over many months by engineering to identify root cause using standard one variable at a time statistical tools were unsuccessful and time consuming 68 different process parameters studied against the product data

19 Initial Visual Exploration Critical part dimensions both with USL=3.0 PROBLEM LOCATION 6% Scrap 50% Rework Different Location OK! Problem improved over time but they didn t understand why and were still incapable Furnace differences???

20 Guided Partitioning The 1 st Splits Appear to be Promising BUT Engineering Questioned The Validity and Usefulness of Some Predictors selected This was interesting but is not actionable This was a correlated response NOT a predictor V_WDY and S_KW_P were locked for the rest of the analysis

21 Continue Partitioning With Guidance Guided partitioning yielded actionable rules The rules were easily implemented and the predicted results were validated in production!!! Partitioning required engineering guidance Guidance provided a reality check on root cause Yearly savings ~$300,000/yr. Time to discovery SEVERAL HOURS!!! Return on investment VERY HIGH Engineering spent months looking at data for a solution

22 Correlated Predictor Considerations Candidates That Are Close Choices Require Scrutiny!!! 1. Initial group of Candidates were very close choices based on Logworth/SS 2. Be aware that minor error (e.g. measurement) would affect their ranking 3. The group is closely ranked because they re strongly correlated (in this case) 4. Partitioning only selects one of them although all are important!!! 5. Because of this, careful scrutiny is advised and candidate selection based on engineering judgment 6. Diagnostics from a PLS suggests they may be influential IN TANDEM 7. This should factor into partitioning rule implementation with engineering input Are SW_8 and W_7 worthy of closer consideration?

23 A Case For Considering a Candidate Not Ranked 1 st W_8 and SW_8 were very close choices based on both Logworth and SS Can we be sure W_8 is more significant than SW_8? Let s examine what would have happened if W_8 were measured with a small amount of additional error Replace W_8 with W_8 err where W_8 err is W_8 with.5% of error added The Candidate list now reveals that SW_8 has moved into the 1 st ranking and W_8 is third Perhaps a case exists to give SW_8 consideration based on the sensitivity of the ranking to errors

24 PLS Useful In Vetting Influence of Predictors Augment a partitioning study with PLS modeling to guard against missing important influences For the Part Fab data set a Correlation Loading Plot can be used to identify influential predictors Correlated predictors will tend to cluster The distance from the center to a predictors normal projection onto the diagonal line measures its influence PLS Validated Partitioning Contributors But Augmented The List & Suggests the Combined Influence of W & SW variables Engineering agreed that the correlation between the Ws and SWs needed to be maintained in implementation

25 There are Perils When Mining Manufacturing Data Data assembly Data with differing time scales alignment & feature extraction Known effects, unknown to you & the value of visualization Prediction vs Control getting to root cause & the need for engineering guidance (permitted by partitioning platform) Correlated predictor issues important predictors can be missed JMP Tools & Platform Features help Mitigate the Perils After data assembly discovery is fast It s worth it!

26 QUESTIONS