The Importance of Understanding Type I and Type II Error in Statistical Process Control Charts. Part 1: Focus on Type 1 Error

Transcription

1 The Importance of Understanding Type I and Type II Error in Statistical Process Control Charts Part 1: Focus on Type 1 Error Phillip R. Rosenkrantz, Ed.D., P.E. California State Polytechnic University Pomona ASQ Riverside Section May 17, 2017

2 n Illustrate how the improper use of decision rules creates excessive Type I error and creates mistrust in the use of Goals 2 n Inspection and Statistical Process Control (SPC) are the major tools for monitoring production. Each is susceptible to error. n The purpose of this presentation is to provide a brief review of SPC and the types of error of concern when sampling using SPC. n Explain Type I and Type II error with colorful examples n Give examples of Type I and Type II error for common decision rules

3 Assignable vs. Common Cause Variation 3 n Dr. Walter Shewhart developed Statistical Process Control (SPC) during the 1920s. Dr. W. Edwards Deming promoted SPC during WWII and after. n Premise is that there are three types of variation n Common Cause Variation Caused by the system n Assignable (or Special Cause) variation Controllable by the work group n Tampering (or over-adjusting) n Each types of variation require a different type of action.

4 Common Cause vs. Assignable Cause Variation 4 n According to Dr. Deming s research, more than 85% of problems are the result of common cause variation. Management is responsible for the system and it is their responsibility to work on reducing this type of variation. Later research puts the estimate at over 94%. n The work group is responsible for preventing and reducing assignable or special cause variation. n Management needs to understand these concepts.

5 Tampering The Third Type of Variation 5 n Tampering is over-adjusting the system caused by a lack of understanding of variation. n Sometimes large built in variation is mistaken for a process going out of calibration and needing adjustment n Over adjusting actually increases variation by adding more variation each time the process is changed n Tampering is a difficult habit to break because many machine operators consider it their job to constantly adjust their machine. n SPC reduces or eliminates unnecessary adjustments.

6 Major Concept #1: Process Capability 6 nthe ability of a process to produce within specification limits n Able to produce within specifications process is capable n Not able to produce within specifications not capable noften quantified with process capability indices n Cp, Pp Ability to stay within specs if centered n Cpk, Ppk Ability based on current center & spread

7 Major Concept #2: Process Control 7 Process Control refers to how stable and consistent the process is. n In-control - stable and predictable. Only experiencing systematic or common cause variation. n Not in-control Process is not stable. Mean and variation are changing due to identifiable or special causes (usually controllable by those running the operation). Represents <10% of the variation.

8 Process Capability What it is Process Control Note - no reference to specs! In Control (Special Causes Eliminated) Out of Control (Special Causes Present) Process Capability Lower Spec Limit Upper Spec Limit In Control and Capable (Variation from Common Causes Reduced) In Control but not Capable (Variation from Common Causes Excessive)

9 Control Charts 9 n Walter Shewhart developed control charts that help management and workers identify common cause and special cause variation n Management s responsibility to reduce common cause variation n The work group is primarily responsible for controlling special or assignable cause variation n Small samples are taken periodically with statistics (e.g., average, range) plotted on charts and reveal the amount and type of variation. Control limits are traditionally +/- 3 standard deviations from the process average.

10 Sample Statistical Process Control (SPC) Chart 10

11 11 Use of Control Charts nwhen the process remains within control limits with only a random pattern, process variation can be attributed to common cause variation (random variation in the system) and is deemed in control. The process is stable and continues. nwhen the process goes beyond control limits or is non-random, it is assumed that an assignable cause is present and deemed out of control. The process is not stable and predictable. Find and eliminate the assignable

12 Implementing SPC 12 n SPC was designed to be a tool for first line workers to monitor for the presence of assignable causes n Requires that management not to use results for evaluating performance, but rather only for improving processes--otherwise data will be biased n Implies that the work group and support personnel take time from their other duties to permanently eliminate assignable causes that reoccur

13 Where to Use SPC 13 n Use strategically on: n Critical customer requirements n Major problems n Six Sigma project related processes n Use tactically on: n Processes that are not capable and need to be monitored closely

14 Managing SPC 14 n A CQE, Black Belt or Master Black Belt should be able to set up the proper SPC Charts and monitor them. n Issues to address when designing SPC charts: n Proper type of chart to use for the situation n Sample size and sample frequency n Sampling method n Decision rules being used n How assignable causes will be resolved n Is the process capable or not capable?

15 Decision or Sensitizing Rules 15 n Decision Rules (a.k.a. Sensitizing rules) are used by operators to determine if a pattern of points indicates a process is no longer stable, that is: outof-control. n Some rules are designed to detect changes or shifts in the process center (mean) n Some rules are designed to detect changes in the process variation (standard deviation) n Some rules are designed to detect a non-normal patterns (e.g. trends or cycles)

16 Zone C is nearest the centerline in this chart. The first eight rules are also known as the Nelson Rules if you change the number of points in Rule 4 from 8 to 9. 16

17 Types of error when you use sampling 17 n Control charts are based on sampling. Sampling is subject to two kinds of error: n Type I error (α): False Alarm The sample indicates the process is out-of-control but is not n Type II error (β): Failure to detect The sample indicates the process is stable, but it really is outof-control n In most quality situations the larger concern is avoiding Type II error: Failure to detect. However, with SPC probably the larger concern is Type I error: False alarms

18 Types of Error 18 Ho: Part is good Ha: Part is bad Test Says H 0 True H 0 False State of Reality H 0 True H 0 False No error Type II error: b Failure to detect, consumer s risk Type I error: a False alarm, producer s risk No error

19 Additonal Examples 19 n Ho: Person did not commit the crime Ha: Person did commit the crime n Ho: The appendix is good Ha: The appendix is bad n Ho: The process is in control Ha: The process in not in control

20 Type of Error Consequences Main Causes Solutions Type I False alarm. Sampling error with probability α. SPC indicates an assignable cause is present but none found. Type II Failure to detect. Sampling error with probability β. Slow response to presence of some assignable causes. Results are deadly to SPC efforts. Operators quickly lose faith in SPC and cease to search for assignable causes. Time wasted collecting data Not getting full benefit of SPC. Missed opportunity to improve, especially when capability is low. Some defects passed on prior to detection. Some temporary defects never detected. Some loss of confidence. Too many decision rules. Each additional rule adds to overall Type I error. Using I, MR-charts on nonnormal data Poor measurement capability Using the wrong chart. Wrong sampling method Wrong Sampling frequency. Poor choice of sample size. Slow detection of small shifts in processes with low Cp, Cpk. Not using R or MR-charts. Using I-chart on Non-normal data. Not recognizing non-normal patterns. Low Defect Level Monitoring an attribute instead of a variable. 20 Reduce number of decision rules used (recommend two decision rules). Test for normality. Gage R&R studies Use correct chart. Use better sampling method. Increase sampling frequency. Use appropriate sample size. Add CUSUM, EWMA, or Moving Average Chart. Switch from I, MR-charts to X-bar, R-charts. Test for normality. Learn to interpret control chart patterns.

21 A look at two decision rules and the probability of Type I and Type II errors 21

22 The Central Limit Theorem is the basis for assuming that a process in control follows a Normal Distribution 22

23 Probability zones for the normal distribution 23

24 Rule 1 Any point outside the 3σ control limits (probability shown for a sequence of 8 points) False Alarm Failure To Detect Failure To Detect

25 Rule 4 A run of 8 points on the same side of the centerline but within the 3σ control limits False Alarm Failure To Detect Failure To Detect

26 Focus of Part 1 is Type I Error

27 Overall Type I Error for both rules 27

28 Cumulative effect of Type I error on a sequence of 8 points as decision rules are added 28 Suppose you added more rules with α i 0.02 each? The overall Type I error as the number of decision rules used would increase: k α T The probability of a False Alarm increases dramatically as decision rulesare added. It does not take too many false alarms before operators begin to lose faith in control charts and start to ignore them

29 Type I Error - A Common Problem That Makes SPC Ineffective 29 n Too much Type I error eventually renders SPC ineffective. People get tired of chasing false alarms. n Many experts recommend using two decision rules (three at the most) to minimize Type I error. Rules 1 and 4 are commonly used. n Often, upon set up, software installers toggle on all decision rules thinking that is desirable. n If you use SPC software, ask to see which rules are in effect.

30 Tactics for Managers 30 n Ask to see SPC Charts n Ask how it was decided which type of chart to use. n Ask which decision rules are being used. n Look for out-of-control points on the chart and what the response was in removing the causes. n Ask if the work group is having trouble resolving assignable causes. Were Pareto Charts, Cause & Effect Diagrams, or other tools used to prioritize efforts?