HIMSS ME-PI Community. Quick Tour. Sigma Score Calculation Worksheet INSTRUCTIONS

Transcription

1 HIMSS ME-PI Community Sigma Score Calculation Worksheet INSTRUCTIONS Quick Tour Let s start with a quick tour of the Excel spreadsheet. There are six worksheets in the spreadsheet. Sigma Score (Snapshot) This tab is the main tool for estimating Sigma Score. It allows you to enter your customer and process assumptions and measures, and it provides your Sigma Score values and related metrics. It is a one-time snapshot. Order Example (Snapshot) An example of the snapshot tool completed for an Order example. The example data is filled in, showing the entire snapshot tool in use (although most of the rows weren t used in the example). Order Example (Snapshot-H) The order snapshot example with unused rows hidden from view for simplicity. This is the typical view that you ll present to people in your project reporting. DPMO Chart A chart for quickly converting defect rates into rough Sigma Scores. The first tab is the actual calculator tool that you ll be using for making sigma score estimates for your process. The next two are the Order Example that we ll use here to illustrate the use of the tool. The term snapshot refers to the fact that the Sigma Score estimate provided by this tool is simply a snapshot of the process capability given the data that is entered into the tool. For longer term analysis, this snapshot data, completed over many time intervals, would be combined into a time series version that shows defect rates and sigma score estimates over time. An SPC control chart could then be used to establish whether or not the defect rate and Sigma Score estimates for the process are statistically stable. Trying to estimate a Sigma Score is never done in isolation from the use of other Six Sigma tools. Using this estimating spreadsheet requires fairly extensive knowledge of the process domain being analyzed. Two core Six Sigma tools that can be helpful in completing this tool include Failure Mode & Effect Analysis (FMEA) and Cause & Effect (Ishikawa) Diagrams, both of which help in differentiating customerfocused failure modes (left side of this tool) from process-oriented causes (right side of this tool). Instructions Page 1

2 Order Example Let s take a look at the Order Example (Snapshot-H) worksheet. The H in the tab name stands for Hidden because unused rows have been hidden from view, leaving only what you are most likely to be really after in using this tool. (More about that later.) Here s the order example: The top left shows that we re looking at an estimate for the Order Processing process, and that it is being estimated on a Monthly timescale, specifically for January 2011 in this example. The left side of the tool concentrates on estimating a Customer Sigma Score; the level of process performance from the customer s perspective, concentrating on aspects that would cause the customer to consider process outputs defective. The right side of the tool concentrates on estimating Process Sigma Score; the level of process performance from the organization s perspective, concentrating on aspects that represent mistakes (defects) in the execution of the process. These defects might or might not cause customer defectives but, if chosen well (see below), they will correlate strongly with customer defectives. Organizationally (usually through our Six Sigma program), our goal is to improve the Customer Sigma Score. Knowing that what the customers see as failures are typically caused by process defects, we actually attack the Process Sigma Score as our mechanism of improvement. That s a core philosophy of Six Sigma quality improvement: Fixing the process side results in improvements in the customer side. Determining Customer Units One of the key challenges in estimating Sigma Score is making sure that all of the units being measured are the same. Since Sigma Score is a function of the opportunities for defects within the units being measured, it is important that each unit measured have (more or less) the same number of defect opportunities. This is sometimes referred to as the apples-and-oranges problem. Instructions Page 2

3 Imagine a discuss among process stakeholders in the Order Processing example having a discussion about what kind of units they should be measuring for the Customer Sigma score: Stakeholder 1: Stakeholder 2: Stakeholder 3: All the customer cares about is that they received their order error free, so we should simply measure orders. If there s something wrong with their order, we ve messed up. Yes, but not all orders are the same. Your customers tend to order one thing, and it usually goes fine. My customers tend to order multiple things, and they act like the whole order is messed up if we make a mistake on one thing. That s fine for you two, but my orders are really complex. My customers sometimes order things that have to be delivered through multiple shipments for a single product. Most of those shipments are fine, so my orders should be considered completely defective if just one shipment gets messed up. I ll take the hit for that shipment, but not the whole order. This discussion could go on for quite some time, and might even involve some additional research in order to resolve some of what s being discussed. But within that discussion will be the information needed to identify the unit of interest in this worksheet. Keep in mind that we re looking for a unit of detail such that every unit has (more or less) the same number of opportunities for defects to occur. From the above discussion, it s clear that everyone is talking about orders. However, clearly different orders have different numbers of opportunities for defects to occur, so orders by themselves can t serve as units for our estimate. Stakeholder 2 raised the issue of the number or products on an order, so perhaps the number of line items on each order could serve as the unit of detail. An order with more lines would have more opportunities for defects compared to single-line orders. Order Lines would likely satisfy the first two stakeholders in this discussion, but Stakeholder 3 raised an additional issue that makes some lines more complicated than others. If we add Shipments per Line to the model, the problem might be solved. Orders with more lines, particularly if some of those lines involve multiple shipments, have many more opportunities for defects than a single-line single-shipment order. We ve found our grain for estimating Customer Sigma Score: the Shipment of a Line Item on an Order. To complete the left side of this tool, those entries will be entered as the dimensions of the process in Column B as dimensions A, B, and C. (The tool supports four dimensions, but this example only needs three of them.) In Column C, we need to quantify the units within the time period being analyzed. In this example, we measured 1,000 orders in the month, each with an average of 5 line items, with an average of 1.1 shipments per line. (These averages need to be agreeable to all stakeholders as a reasonable representation of the process. If they don t or can t agree, it could be that you ve actually got multiple distinct processes, in which case this tool would be used separately for each.) Instructions Page 3

4 With these measurements provided, the tool has calculated that 5,500 Order-Item-Shipment units were processed in January Determining Process Units The units used on the right side of the tool don t need to be the same as the customer units and, in fact, they rarely are. In the order example, it might turn out that the process is also driven by Orders and Line Items, so the model might be the same in the first two dimensions. Again, we return to our stakeholders: Stakeholder 2: Stakeholder 3: Stakeholder 1: Stakeholder 2: Stakeholder 3: Why don t we just measure our process based on shipments? Our customers measure us that way, so why shouldn t we? That would be ridiculous! We can t evaluate our process just by looking at shipping. Most of the things that go wrong happen long before we ship our products. If the warehouse picked the orders correctly, our shipments would be fine. Don t blame the warehouse guys. It s the sales people who keep taking orders that we can t possibly fulfill without errors every time. Forget sales. Look at all the stuff that marketing does to mess up the customer location and pricing assignments. I guess we d have to agree that we run the risk of introducing defects every time somebody touches the order. Why don t we agree that each transaction in the pipeline could introduce an error? Again, as long as all stakeholders buy into the model, this will work. If they can t or won t, then consider splitting the process and estimating Sigma Score for the differentiated processes. To complete the right side of this tool, those entries will be entered as the dimensions of the process in Column F as dimensions W, X, and Y. (This example doesn t use dimension Z.) In Column G, we need to quantify the units within the time period being analyzed. In this example, we again measured 1,000 orders in the month, each with an average of 5 line items. The stakeholder agreed that the average Order-Line gets touched by 12 transactions, each of similar complexity. With these measurements provided, the tool has calculated that 60,000 Order-Item-Transaction units were processed in January Determining Customer Defectives Having identified the customer and process units being measured, the next steps involve identifying the opportunities for defects to impact these units, starting with customer-oriented defect that would cause a customer to identify the unit as defective. It isn t critical to identify all possible defects that can occur for this tool to be useful. What we re after are the defects that would be significant to the customer, Instructions Page 4

5 and that can be reasonably measured. Identifying defects that can t be measured would actually impede the use of this tool. If there are defects like that that seem significant, then they need to be addressed in some other way than simply entering them in this tool. However, usually even the most obscure defects can be estimated accurately enough to make this tool both useful and meaningful. In our Order Example, we see that four different defect types have been identified from our customer perspective: receiving incorrect products, late arrivals, damaged products, or shipments that arrive at the wrong location: This means that there are four opportunities for a defect to occur in each Order-Line-Shipment, for a total of 22,000 opportunities in January 2011 for the 5,500 units processed. We ve identified that there are four things that can go wrong on each unit that would directly upset the customer. That doesn t mean that there are only four defect opportunities per shipment. This data works at the unit level. If a single shipment of 50 products arrives late, damaged, and in the wrong place; that would be 150 defects out of the 200 opportunities. Remember: Large complicated shipments would have far more opportunities for defects than small simple shipments. This worksheet has been designed with the intention that calculations will be based on defect rates derived from measured samples rather than actual defect counts from the entire population. Thos rates are entered as percentages (blue) to the right of each defect type. The tool then calculates the number of defects as those percentages of the opportunities, and totals those defects at the bottom of the list. In the Order Example, there were 1,100 defects in January 2011 against the 22,000 opportunities. Determining Process Defects Identifying the possible defects on the process (right) side of the tool is a bit more involved because we chose a fairly generic unit for processing at the transaction level. That means that the defects listed in this tool need to be generic enough that they could reasonable happen during any of the 12 transactions included in the model. In the Order Example, those transaction-based defects were identified as providing incorrect quantities, incorrect assignments, delayed processing, incorrect qualities, or incorrect processing: Instructions Page 5

6 This means that there are 5 opportunities for a defect on each Order-Line-Transaction unit, or 300,000 opportunities in January 2011 against the 60,000 units processed. Transaction defects that affect an entire order would have a disproportionate impact on the calculated Sigma Score since those transactions trigger a defect against each line on the order. For example, assigning the order to the wrong logistics warehouse during order confirmation of a 30-line order would count as 30 defects. Entering an incorrect pull quantity for a single line on that order during picking would constitute only one defect. The sampling estimates of each of the 5 defect types indicated that there were 39,000 defects in January 2011 against the 300,000 opportunities. The validity of this model presupposes that each transaction in the order process actually has these five opportunities for defects, meaning that the process transactions represent similar types of processing complexity. If that is not the case, the orientation of this example would not be appropriate. If stakeholders become focused too much on distinct defects in distinct transactions, and seem unwilling or unable to generalize, then a separate Sigma Score calculation might have to be developed for each distinct transaction. Such an approach is more labor intensive, and usually does not end up giving a more accurate or clearer picture of the quality level of the process. Determining Sigma Scores Enough data has been provided to the worksheet at this point for the final calculation of the Customer Sigma Score and related metrics. In January 2011, there were 1,100 customer defects against 22,000 units, for a Defect per Unit (DPU) of.2, and Defects per Million Opportunities (DPMO) of 50,000. This equates to a 3.14 Customer Sigma Score in the short-term. On the process side, there were 39,000 defects out of 300,000 opportunities in January This corresponds to a Defect per Unit (DPU) of.65, and Defects per Million Opportunities (DPMO) of 130,000. This equates to a 2.63 Customer Sigma Score in the short-term. Instructions Page 6

7 Using the Blank Worksheet The tool you ll use to do these calculations in your arena is in the Sigma Score (Snapshot) tab of the Excel worksheet. It initially looks very different than the example we ve been discussing because the example had unneeded rows hidden from view. The full spreadsheet is complicated because the dimensions of the units in analysis haven t been defined yet. The worksheet allows for units that are up to four dimensions on each side of the model (they were 3 on each side of the sample). Step-by-Step 1. Establish Dimensions. Enter the Customer dimensions into cells B7:B10 and Process dimensions into F7:F10. The < dimension name > value should be deleted from any unused dimensions. Enter representative dimensional counts into C7:C10 and G7:G10. Use 1 in unused dimensions. 2. Note Maximum Dimensionality. On the Customer left side, you ll have defined a model that is one (A), two (A-B), three (A-B-C) or four (A-B-C-D) dimensional. Likewise, your Process right side will be one (W), two (W-X), three (W-X -Y) or four (W-X -Y -Z) dimensional. The two sides don t have to be the same number of dimensions. 3. Establish Defects. As you work down the spreadsheet, you ll provide your identified defect names in the places where you see the < defect name > cells. Do not enter defects for higher dimensions that you have defined above. In fact, the majority of the defects will be at the level you defined above (e.g. the 3-dimensional order example had defects at the three dimensional A-B-C and X-Y-Z levels). a. An example of a lower-dimensional defect in the Order Example might have been if customer sometimes complained that order confirmation weren t received correctly when they placed the order. This kind of defects affects the order as a whole, independent of the number of lines or shipments, so it would be entered in cell B15 as a defect at the one-dimensional level. 4. Establish Opportunities. Empty any remaining cells that still contain the < defect name > literal so that the COUNT function doesn t include them as opportunities for defects (i.e., cell C12). 5. Provide Defect Rates. From your measurement data used to establish your defects, enter your defect rates for each defect entered above (in Column D or H). The level of accuracy or precision of these measures is a function of your measurement program, and so this tool will only reflect that level of precision. Reasonable data will usually result in a reasonable Sigma Score estimate. Once these steps have been completed, your Sigma Score estimates will be present in Row 76. Instructions Page 7