QUICK & DIRTY GRR PROCEDURE TO RANK TEST METHOD VARIABILITY

Transcription

1 QUICK & DIRTY GRR PROCEDURE TO RANK TEST METHOD VARIABILITY Mike Mercer, Quality Engineering Specialist, 3M, St. Paul, MN Steve Cox, Lean Six Sigma Coach, 3M, St. Paul, MN Introduction One of the first steps in a process improvement effort is to assess the stability of the process. One of the simplest methods is to use some type of control chart. Once a baseline has been established some thought must be given to how much of the variability is actually process variability and how much is due to test method measurement error. This becomes important in any improvement effort. It is not uncommon that for many processes, especially those using destructive testing like the PSA industry, that a large portion of variability can be due to the test method measurement. Processes with high test method variability can often achieve major improvement by just improving the test. In any event trying to improve a process without at least a good estimate of test method error is frustrating at best and potentially quite costly. Performing experimentation without knowing what minimum effect can be detected can be costly in both resources and morale and possibly more important can lead to erroneous conclusions and chasing down dead ends. The situation can be worse if experimentation follows a Design of Experiments (DOE) methodology, in that many runs can fail to show any factors as significant. In this case frustration abounds and the damage done extends well past the process under study. As an example Figure shows the results of a typical study to select the material that demonstrates the highest CD Tensile. The chart shows the average result of 5 measurements for each formulation. Figure is taken from ASTM D3759 and displays that the width the 95% confidence limits for a single operator testing different materials is +/- 6.7%. So instead of selecting the best material, what we really did was reject the worst one, as depicted in Figure 3. This paper will demonstrate an easy way for characterizing the test method impact on process variation. Control Charting PSA Tape Processes Where a process has only one quality characteristic value per coated roll, the classic Individual Moving Range Control Charts are a valuable method to judge stability of the process. See Figure 4. Often times the Individual Moving Range Control Chart is used to plot quality characteristics for lanes. Figure 4 depicts the coat weight for the left side of the coater, a second chart is used to depict the right side coating weight also. When more than one quality characteristic values are collected per roll, we can have an entirely different situation if we apply the classic Xbar Range Control Chart. By their very nature tape coating processes can violate the very assumptions required for successful implementation of classic control charting techniques. Classic Xbar Range control charts are based on the assumption that the variability within the 83

2 subgroup is the same as the variability between subgroups. When crossweb measurements are used to represent a subgroup (i.e., within variability), their variability is often much less than the downweb variability (i.e., between variability). When classic control charts are used, the process often looks wildly out of control. See for instance Figure 5 which depicts a classic Xbar Range control chart where the subgroups are composed of two coat weight measurements, one from each side of the web, plotted for each coated roll. The crossweb variation is by coater design optimized to be uniform, whereas the downweb variation is influenced by other factors. Luckily for us, Don Wheeler has described a procedure for situations where the within variation (crossweb) is different from the between variation (downweb). Luckily also, Minitab has implemented a Control Chart for this situation, see Figure 6. The top chart depicts the stability of the process average. The middle range chart depicts the downweb Range stability. The bottom range chart depicts the crossweb stability. Figure 7 shows the standard deviation computations that accompany these between/within control charts. The estimate of the within standard deviation is the variation due to crossweb, the between standard deviation is the variation due to downweb. The total is the square root of the sum of squares of the two components, and is the standard deviation that represents the total variation due to the process. It is the number that can be used to calculate a Cpk index for instance. It is not our intention to teach you about the use of the Between/ Control Charting techniques, since Wheeler has already done that for us. However it is this very structure of between/within variation that we will use to obtain estimates of test method variation from typical tape coating processes. Figure. Selection of the best formulation 84

3 Figure. ASTM D3759 confidence intervals Figure 3. Rejection of the worst 85

4 I-MR Chart of Left CW 34 UC L=33.95 Individual Value _ X=3.760 LC L= Observation Moving Range 3 UC L=.69 MR= LC L= Observation Figure 4. Individual Moving Range Control Chart for Coat Weight Figure 5. Xbar Range Control Chart for Tape Coat Weight 86

5 Figure 6. Between/ Control Chart for Coat Weight Figure 7. Standard Deviation Calculations from Between/ Control Chart Gage R&R Studies In cases where one process is under study, it is common to use a Gage R&R study to assess the test method measurement system health. The criteria for the action level for the Gage R&R method is often quoted from the Automotive Industry Action Group (AIAG) Measurement Systems Analysis Manual 3 (MSA). Donald Wheeler 4 has published a critique of this action level and makes a good case that at least for control charts the criteria is far too conservative. In any event we will use the Gage R&R ratio as another metric used to rank the health of the measurement system. A gage study has the advantage of looking at operators, their repeatability and reproducibility. There are some issues with gage studies, probably the most prominent being that the samples must be carefully selected so they bracket the process width. It is sometimes difficult to do this and if not done properly the resulting Gage R&R ratio does not reflect what is really occurring because the estimate of process variation is incorrect. 87

6 GageR & R ratio Test Method Equation Process Thus time can be spent fixing an adequate measurement system when that effort would be much better spent in process improvement. The difficulties of gage studies can be overcome with careful planning and preparation. Tests that are destructive can represent additional challenges which can usually be compensated for by careful planning. Gage R&R studies have the disadvantage that they require additional experimentation above and beyond what is routinely collected. Typically ten samples are collected for testing, three operators perform the tests and repeat the testing three times for each sample. This amounts to 90 data points, which is a considerable amount of additional work. The resulting statistical output is comprised of two components for the test method standard deviation, the repeatability portion and the reproducibility portion. Repeatability is the result one person gets testing the same material repeatedly. Reproducibility is the representation of the variability due to different operators. Repeatability is always present. Reproducibility, if present, is the result in many cases of poor training. In the best of cases it is negligible. Many tape processes require that multiple specimens be tested and averaged to represent the reported result of the quality measurement. In some cases the test is such that multiple specimens are needed to average out test variability to an acceptable level. In may be that multiple specimens are required as a customer requirement, it may also be that it is simply the way things have always been done. PSTC test methods routinely require multiple specimens to be tested and the average or median reported, see Figure 8. Figure 8. PSTC Test Methods requiring multiple specimens 88

7 If a Gage R&R study were performed on PSTC 07 where five specimens are required to represent the reported result of the test method then the Gage R&R study now requires 450 individual tests (90 times 5 specimens per average.) Clearly if there were a simpler method to obtain an estimate of the test method repeatability standard deviation it would be welcome. Concept for Quick and Dirty Gage R&R Studies Often times the individual values obtained to calculate the reported result of the quality characteristic represented by a test method are discarded and only the average value recorded. The subsequent quality characteristic is often charted on the classic Individual Moving Range Control Chart and gives a picture of the process stability. The lost information due to discarding the individual results of the quality characteristic is in actuality the information needed to calculate the repeat standard deviation from a Gage R&R study. If we can calculate the standard deviation of the individual results we can estimate the standard deviation of the test method itself since the two are related as follows. Individuals Test Method n Individuals Equation Instead of discarding the individual values of the quality characteristic if we use them in conjunction with the Between/ Control Chart for one lane we can obtain an estimate of test method variability from the within standard deviation. Individual s n Equation 3 The between standard deviation is a representation of the true process variability. However the observed process variation standard deviation is a combination of both process and test method # Process Substituting equations and 3 into equation 4 gives Equation 4 Between Test Process # n Method Between Equation 5 Substituting these relations into the definition of the Gage R&R ratio from equation gives 89

8 GageR & R ratio Between n # n n Between # Equation 6 We now have a way to calculate the Gage R&R ratio directly from the output of a Between/ Control Chart when we use the individual values of the test method. Process Capability Calculations The next thing to consider is what is the capability of the process and what portion of the variability is due to the test method. Fortunately again Minitab has a procedure to help us analyze this in the presence of Between/ variability. For illustration purposes only, assume that the data represented earlier in Figure 6 is in fact from two repeats of the coating weight which are averaged to be the coating weight reported test result instead of from the right and left sides of the web. We do this to show the calculations of the Between and standard deviations as calculated before. Between/ Capability Sixpack of Coat Weight Individual Value Individuals Chart of Subgroup Means UCL= _ X=3.807 LCL=9.677 LSL Capability Histogram USL Specifications LSL 9.5 U SL Moving Range Chart of Subgroup Means Normal Prob Plot A D: 0.49, P: 0.4 Moving Range UCL=.67 MR=0.80 LCL= Sample Range Range Chart of All Data UCL=.76 _ R=0.360 LCL=0 StDev Betw een B/W O v erall Capability Plot Figure 9. Minitab Between/ Capability Sixpack B/W O v erall Specs C apa Stats C p. C pk.03 Pp 0.9 Ppk 0.83 C pm * This chart contains a lot of information about our process as well as the test method. The values of the Between and standard deviations are given and are identical as seen from Figure 7. The Gage R&R ratio from these data is 90

9 .39 GageR & R ratio n # x.673 #.39 Between.3 Equation 7 In addition we have Cp, Cpk, Pp, and Ppk metrics. The Cpk metric (calculated from the control chart standard deviation estimates) and Ppk (population standard deviation) would be examined. In this example the Cpk of. is greater than the Ppk of.83 indicating there is special cause variation present. Any difference between the Cp and the Cpk and similarly the Pp and the Ppk is due to the process being off center from the specifications, in this case the process is slightly off center. Quick and Dirty Ranking In many process improvement situations there are many processes and many quality characteristics to work on. Capability metrics are relatively easy to compute and gage metrics less easy, in some cases much more difficult. In the situation where repeats are present the described method allows a quick method to do a gage study. It allows many processes to be triaged so resources are focused on processes most in need of work. In the above example the process is not capable (Ppk<) and test method is marginal, so this would be a potential candidate for improvement. In the factory that produced this product there would likely be other processes more in need of improvement. There are situations where a quick estimate of test variation can be very useful even the case of very capable processes. In the case of process improvement there can be several criteria used to evaluate the potential for improvement activities. Process capability metrics, test variation, resource availability, and economic factors. In an actual case study there were two processes and a total of 9 products with many quality characteristics per product which are represented as the result of a test method. They were analyzed in two days using the described procedures. The individual test data had been recorded in the quality records and was available for analysis. A sixpack was run for each process/product/quality characteristic. The Gage R&R ratio was computed and tabulated along with the other measures depicted in Table. The data developed was then put in a cause and effect matrix (C&E) and scored against criteria selected by the improvement team. The scored matrix was then sorted into rank order and the top ten were selected for improvement work. 9

10 Process Table. Quick and dirty process capability improvement opportunities Product Quality Characteristic Cpk Cp Cpk ratio Ppk Cpk Ppk ratio Gage R&R ratio Description of Improvement activity => Potential for process improvement. >.33 is better Potential for process improvement via centering. Bigger is better Actual process performance >.33 is better Potential for process improvement via removing special causes Potential for process improvement via test method. Smaller is better SDC A Adhesion SDC A Coat Weight SDC B Adhesion MCM C Adhesion MCM C Shear MCM D Tack In situations where repeats are used in testing, the described technique can greatly reduce time required to achieve results. It is particularly effective in ranking processes for improvement. This allows the most needed improvements to be done first so the greatest value can be achieved. It should be noted that in all cases discussed the metric described would be for individual values. In the case where specifications are based on averages the calculated standard deviations would need to be corrected for sample size. Literature Citations Donald Wheeler Advanced Topics in Statistical Process Control, SPC Press, 995 Minitab Software, Revision 5 3 Measurement Systems Analysis, 3 rd Edition, 00, AIAG, Automotive Industry Action Group 4 Donald Wheeler, Good Data, Bad Data and Process Behavior Charts. 9

11 #$%&'()*+,)-./(0,+-/(-3( #$%&'(&)#&)*(+,-*(.-/*(.-*(0( ( ( #$%&'(&)#&)*(+,-*(.-/*(.-*,?C5-.,89(+D,+((,+D?(),-.,?8E,8*8*(F+>5?8,-. -+>-+?=8-8,=+G5?,+(??-8&@H%(*-?-B=->B-)3D=5+>,+C5-.,89,+).5>,+D C5-.,890-+-D(0(+8I?(J,+D'K9(-?-?F/LMKK'8(-0.(->(N=8*(F+>5?8,-. O>*(?,J(-+>-(?7,J,?,=+-+>?8-8,?8,)?IE,8*(0*-?,?=+8(?80(8*=>J-,-B,.,89-+> )=+8=.)*-8,+DH@()()5(+8.9,?>(J(.=,+D-8(?80(8*=>$(+8(=N#P)(..(+)(N= 8*(F+>5?8,-.-+>-+?=8-8,=+G5?,+(??H%((-+(>-GH/H>(D((,+)*(0,)-. (+D,+((,+DN=08*(/8-8(Q+,J(?,89=NR(ES=3-8G5NN-.=I-+>5?5(>->>,8,=+-. (>5)-8,=+,+)*(0,)-.(+D,+((,+DN=08*(Q+,J(?,89=N@,++(?=8-H 8