HIGH MATURITY CONFERENCE. Christian Hertneck June 2014

Transcription

1 HIGH MATURITY CONFERENCE Christian Hertneck June 2014

2 CONTENT > Motivation > Statistical Thinking > Capability and Maturity Level 4 & 5 > Overview of Statistical Tools > Workshop Topics > Examples > Exercises > Discussions > Workgroups 2

3 3

4 TYPICAL REPORTING SYSTEM Quality July Actual Value Monthly Average Value % Diff % Diff from July Last Yr Actual This YTD Plan or Average % Diff as % of Last YTD On-time Shipments (%) First Time Approval (%) Pounds Scrapped (per 1000 lbs production) Production Monthly Report Yr-to-Date Production Volume (100s/lbs) Material Costs ($/100 lbs) Manhours per 100 lbs Energy & Fixed Costs/100 lbs Total Production Costs/100 lbs In-process Inventory (100's lbs)

5 TIME SERIES FOR MONTHLY IN-PROCESS INVENTORY July Actual Value Monthly Average Value % Diff In-Process Inventory % Diff from July Last Yr In-process Inventory (100's lbs) Actual Yr-to-Date Plan or Average % Diff This YTD as % of Last YTD Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Yr Yr Yr No long term trends No other systematic patterns Average Does not say whether July value is exceptional Is the July value a signal---or is it just noise? Need to filter out the month to month variation. J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J 5

6 UPPER CONTROL LIMIT FOR MOVING RANGE CHART Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec = upper limit for difference between Monthly values J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J Moving range directly measures the month-to-month variation. Upper control limit for Moving Range = 3.27 x Avg = 14.2 Result: If the amount of in-process inventory changes (up or down) by more than 1420 lbs from one month to the next, then one should look for an explanation. 6

7 TIME SERIES FOR MONTHLY IN-PROCESS INVENTORY Result of monitoring the process behavior chart: No problem! 7

8 PROBLEMS WITH PERCENT DIFFERENCES AS A BASIS FOR INTERPRETING VALUES Size of % difference partially depends upon the magnitude of the base number 10 unit change from 100 to 110 is 10% change 10 unit change from 300 to 310 is 3.3% change Comparing lines in a monthly report by comparing the size of the percent differences assumes that all lines should show the same amount of relative variation month-to-month Differences based upon comparison of the current value with last year s value, a large % difference may be due to an unusual value in the past rather than an unusual value in the present 8

9 INTERPRETATION Month-to-month variation Inventory change > 1420 lbs from one month to the next, one should look for an explanation likely due to the direct result of some special / assignable cause. Actual Values Monthly Chart Limits on the individual values chart define how large or small a single monthly value must be before it represents a definite departure from the historic average value in excess of 31.6 would be a signal that the amount of inventory had shifted upward. Value of 28 is not, by itself, a signal. There is not real evidence of any real change in the in-process inventory. 9

10 TYPICAL REPORTING SYSTEM Quality July Actual Value Monthly Average Value % Diff % Diff from July Last Yr Actual This YTD Plan or Average % Diff as % of Last YTD On-time Shipments (%) First Time Approval (%) Pounds Scrapped (per 1000 lbs production) Production Monthly Report Yr-to-Date Production Volume (100s/lbs) Material Costs ($/100 lbs) Manhours per 100 lbs Energy & Fixed Costs/100 lbs Total Production Costs/100 lbs In-process Inventory (100's lbs)

11 TIME SERIES FOR PERCENTAGE ON-TIME SHIPMENTS 11

12 INTERPRETATION Six of the individual values and one of the moving ranges fall outside the limits. The six values below the limit should be treated as signals. The process is trying to tell you it has a problem. If you concentrate on the percent differences in the monthly report, you are not likely to ever be aware of this problem until it is already too late Control Charts are about separating signals from noise. 12

13 CONTENT > Motivation > Statistical Thinking > Capability and Maturity Level 4 & 5 > Overview of Statistical Tools > Workshop Topics > Examples > Exercises > Discussions > Workgroups 13

14 PROCESS VARIATION Shewhart s notion of dividing variation into two types: > common cause variation > variation in process performance due to normal or inherent interaction among process components (people, machines, material, environment, and methods) > represents the noise of the process > assignable/special cause variation > variation in process performance due to events that are not part of the normal process. > represents sudden or persistent abnormal changes in one or more of the process components > represents a signal of the process 14

15 COMMON CAUSES 15

16 BY CHANGING THE PROCESS YOU HAVE TO ADAPT THE CONTROL CHARTS 16

17 PREDICTABILITY IS THE KEY FOR HIGH MATURITY > A process is said to be predictable when, through the use of past experience, we can describe, at least within limits, how the process will behave in future. > A unpredictable process will display exceptional variation that is the result of assignable/special causes. > A predictable process will display routine variation that is characteristic of common causes. Image courtesy of Microsoft Cliparts 17

18 REMOVING SPECIAL CAUSES OF VARIATION > Difficult to do well! > Identify special/assignable causes as soon as possible > checks/alerts/triggers in data entry/measurement tools > checks within the measured process > use charts/diagrams for daily review > assign individuals to monitor process data > provide template/form to fill out initial details of special variance > Maintain data base of special cause investigations > Use data base reports to monitor corrective actions > Train individuals for the investigations and problem solving techniques. 18

19 IDENTIFYING SPECIAL CAUSES OF VARIATION - EXAMPLE 19

20 FROM PROCESS STABILITY TO PROCESS PREDICTABILITY Process is stable! Determine stability of the process Process is predictable! Derive prediction model based on an appropriate* probability model appropriate* means the model fits the reality 20

21 Managing Quantitatively? - Example Therefore, just having nice charts and diagrams DOES NOT mean you are statistically controlling anything. 21

22 SIMPLE DETECTION TESTS FOR INSTABILITIES 22

23 REMOVING SPECIAL CAUSES OF VARIATION - REVISITED > Use traditional, effective, problem solving techniques to select corrective action > team of experts > addressing root causes of problems > obtain approval, pilot, deploy, train, monitor, inform > Beware of false alarms > data errors, inconsistencies > improperly calculated limits, wrong assumptions > lack of data grouping > too many tests > nothing wrong with the process > Beware of a lack of any special variation > control limits may be too wide > Improving data quality and data grouping is a continuous effort 23

24 LISTENING TO VOICES USL/LSL = Upper/Lower Specification Limit UCL/LCL = Upper/Lower Control Limit Voice of the process = the natural bounds of process performance Voice of the customer = the goals established for the product and process performance (e.g. specifications) Capable process = stable process + product conformance UCL USL UCL USL LCL LSL LCL LSL Stable Capable Process Capability DOES NOT EQUATE TO a capable process. 24

25 ALIGNING PROCESS PERFORMANCE TO PROCESS REQUIREMENTS Reduce Variation Upper Spec Mean Shift the Aim Upper Spec Upper Spec Change the Specs 25

26 CONTENT > Motivation > Statistical Thinking > Capability and Maturity Level 4 & 5 > Overview of Statistical Tools > Workshop Topics > Examples > Exercises > Discussions > Workgroups 26

27 CAPABILITY LEVELS 4 & 5 (SPICE) PA 4.1 Process measurement attribute > Measure of the extent to which measurement results are used to ensure that performance of the process supports the achievement of relevant process performance objectives in support of defined business goals. > Result of full achievement > process information needs in support of relevant defined business goals are established; > process measurement objectives are derived from process information needs; > quantitative objectives for process performance in support of relevant business goals are established; > measures and frequency of measurement are identified and defined in line with process measurement objectives and quantitative objectives for process performance; > results of measurement are collected, analyzed and reported in order to monitor the extent to which the quantitative objectives for process performance are met; > measurement results are used to characterize process performance. ISO/IEC Optimizing 4 Predictable 3 Established 2 Managed 1 Performed 0 Incomplete 27

28 CAPABILITY LEVELS 4 & 5 (SPICE) PA 4.2 Process control attribute 5 Optimizing > Measure of the extent to which the process is quantitatively managed to produce a process that is stable, capable, and predictable within defined limits. > Result of full achievement > analysis and control techniques are determined and applied where applicable; > control limits of variation are established for normal process performance; > measurement data are analyzed for special causes of variation; > corrective actions are taken to address special causes of variation; > control limits are re-established (as necessary) following corrective action. ISO/IEC Predictable 3 Established 2 Managed 1 Performed 0 Incomplete 28

29 CAPABILITY LEVELS 4 & 5 (SPICE) PA 5.1 Process innovation attribute > Measure of the extent to which changes to the process are identified from analysis of common causes of variation in performance, and from investigations of innovative approaches to the definition and deployment of the process. > Result of full achievement > process improvement objectives for the process are defined that support the relevant business goals; > appropriate data are analyzed to identify common causes of variations in process performance; > appropriate data are analyzed to identify opportunities for best practice and innovation; > improvement opportunities derived from new technologies and process concepts are identified; > an implementation strategy is established to achieve the process improvement objectives. 5 Optimizing 4 Predictable 3 Established 2 Managed 1 Performed 0 Incomplete 29

30 CAPABILITY LEVELS 4 & 5 (SPICE) PA 5.2 Process optimization attribute 5 Optimizing > Measure of the extent to which changes to the definition, management and performance of the process result in effective impact that achieves the relevant process improvement objectives. > Result of full achievement > impact of all proposed changes is assessed against the objectives of the defined process and standard process; > implementation of all agreed changes is managed to ensure that any disruption to the process performance is understood and acted upon; > effectiveness of process change on the basis of actual performance is evaluated against the defined product requirements and process objectives to determine whether results are due to common or special causes. ISO/IEC Predictable 3 Established 2 Managed 1 Performed 0 Incomplete 30

31 CMMI CONNECTING CAPABILITY AND MATURITY ON ORGANIZATIONAL LEVEL Organizational Performance Management Causal Analysis and Resolution Organizational Process Performance Quantitative Project Management Requirements Development Technical Solution Product Integration Verification Validation Organizational Process Focus Organizational Process Definition Organizational Training Integrated Project Management Risk Management Decision Analysis and Resolution Requirements Management Project Planning Project Monitoring and Control Supplier Agreement Management Measurement and Analysis Process and Product Quality Assurance Configuration Management Maturity Level 5 Optimizing Maturity Level 4 Quant. Managed Maturity Level 3 Defined Maturity Level 2 Managed Capability Level

32 WHAT AN ORGANIZATIONAL LEVEL 3 HAS ACHIEVED» Managing a company by means of the monthly report is like trying to drive a car by watching the yellow line in the rear-view mirror «[Myron Tribus] 32

33 HIGH MATURITY REQUIRES LEADING INDICATORS 33

34 SUMMARY OF TERMINOLOGY Improvement Capability Improve process continually to reduce variability and improve quality, cost and cycle time. Make the process capable by changing the process performance. Focus of Level 5 Predictability Stability A process is said to be predictable when through the use of past experience, you can describe, at least within limits, how the process will behave in future. Ensure that process behaviour is stable by removing assignable causes. Focus of Level 4 Performance Measuring of attributes of the process Focus of ML 3 34

35 CONTENT > Motivation > Statistical Thinking > Capability and Maturity Level 4 & 5 > Overview of Statistical Tools > Workshop Topics > Examples > Exercises > Discussions > Workgroups 35

36 PURPOSE AND BENEFITS OF SPC Provide quantitative information that improves decision making in time to positively affect the business outcome Decisions regarding adaptations may be necessary due to changing circumstances on project, product, process, or business level Suitable information is necessary to be able to decide what needs adaptation on business, process, or project level Early identification of critical project situations and problems Early identification of critical project situations and problems Obtain leading (measures to decide) instead of lagging (measures to learn) indicators Consistent prediction and monitoring across different levels (project, multi-project, organizational) 36

37 WHAT IS STATISTICAL PROCESS CONTROL? SPC is an analytical / statistical technique used to identify and understand sources of process performance variations in order to quantitatively monitor, control, and predict the process. Typical questions to be answered using SPC Is process x in control and predictable? (Is the quality of work products generated in the design phase predictable?) What is the average value and range for process x? (How many hours do we typically spend on peer reviews of design documents?) Is the latest measurement typical or did something unusual happen (need for corrective action)? 37

38 QUANTITATIVE/STATISTICAL TOOLS AND TECHNIQUES - 1 Statistics Inferential Statistics Precondition Descriptive Statistics Inferential statistics comprises Descriptive the statistics use of statistics a branch to make of statistics inferences concerning some unknown that aspect denotes any of the many techniques used to (usually a parameter) of summarize a population a set of data. In a sense, we are using the data on members of a set to describe the set. Watching Statistics monitor variation, i.e. distinguish between usual random and abnormal change. Watching statistics are applied to assess the nature of variation in a process and to facilitate forecasting and management (monitor variation, i.e. distinguish between usual random from abnormal change).

39 QUANTITATIVE/STATISTICAL TOOLS AND TECHNIQUES - 2 Statistics Inferential Statistics Precondition Descriptive Statistics Distributions (the shape of the process) Hypothesis Testing Numerical Graphical Regression Analysis Prediction Models Parameter Estimation Location Mean Median Mode Dispersion Range, IQR Standard Deviation Variance Pareto Chart Histogram Run Chart Contingency T. Scatter Plot Fishbone Box Plot Watching Statistics monitor variation, i.e. distinguish between usual random from abnormal change. 39

40 MISPERCEPTIONS ABOUT HIGH MATURITY - 1 > If we measure more things, and involve more people in reviewing and using the measures, we will eventually achieve High Maturity... > The key to achieving high maturity is measuring the right things, and using the correct techniques to analyze and interpret the measures... > We need to wait until we have more of the right kind of data before we can attempt to implement High Maturity Practices... Image courtesy of Microsoft Cliparts 40

41 MISPERCEPTIONS ABOUT HIGH MATURITY - 2 > Adding the use of Control Charts to the practice of measurement and analysis results in High Maturity > All I need to do is to use Control Charts to analyze the outcome of our critical subprocesses and we can control them... > Using threshold based on specification limits is a high maturity practice.. Image courtesy of Microsoft Cliparts 41

42 MISPERCEPTIONS ABOUT HIGH MATURITY - 3 > All the things we need to understand about high maturity practices can be adequately explained in a single conference presentation or workshop. [10] Image courtesy of Microsoft Cliparts 42

43 SUMMARY PROCESS BEHAVIOR MEASUREMENT FRAMEWORK Clarify business goals Yes Level 3 Identify and prioritize issues Select and define measures No No New goals, strategy? Yes New issues? Level 4 Collect, verify, and retain data No Yes New measures? Analyze process behaviour Level 5 Yes Process stable? Process capable? No No Remove assignable causes Change process Yes Continually improve 43

44 HIGH MATURITY IS ABOUT IDENTIFYING WASTE AND IMPROVEMENT OPPORTUNITIES QUANTITATIVELY 44

45 CONTENT > Motivation > Statistical Thinking > Capability and Maturity Level 4 & 5 > Overview of Statistical Tools > Workshop Topics > Examples > Exercises > Discussions > Workgroups 45

46 SAMPLES FOR WORKGROUP DISCUSSIONS > High Maturity case studies > Selecting processes and data for statistical analysis > Connecting business improvement to quantitative data > Creating performance models and their use in project management. > Tools for use in statistical analysis of data 46

47 QUESTIONS 47

48 THANK Y0U! Christian Hertneck Anywhere.24 GmbH Lindberghstr Puchheim

49

50 Version Draft / for review / released Date Comments/Change History Author 0.01 Draft Layout; Contents; Summary... C.Hertneck 0.02 For review Contents C.Hertneck 1.00 Released Final touches C.Hertneck Capability Maturity Model, Carnegie Mellon, CMM, and CMMI are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University. sm CMM Integration; IDEAL; Personal Software Process; PSP; SCAMPI; SCAMPI Lead Assessor/ Appraiser; SEPG; Team Software Process; and TSP are service marks of Carnegie Mellon University.

51 REFERENCES > [1] Measuring the Software Process, William Florac and Anita Carleton, Addison- Wesley, > [2] Understanding Variation: Key to Managing Chaos, Donald Wheeler, SPC Press, > [3] Understanding Statistical Process Control, Donald Wheeler and David Chambers, SPC Press, > [4] Practical Software Metrics for Project Management and Process Improvement, Robert Grady, Englewood Cliffs, > [5] Statistical Methods for Software Quality - Using Metrics to Control Process and Product Quality, Adrian Burr and Mal Owen, International Thomson Computing Press. > [6] Goal-Driven Software Measurement - A Guidebook, Robert Park, Wolfhart Goethert and William Florac, CMU/SEI-96-HB-002, Carnegie Mellon University. > [7] Metrics and Models in Software Quality Engineering, Stephen Kan, Addison Wesley, > [8] Software Metrics: A Rigorous & Practical Approach, Norman Fenton, Shari Pfleeger, Thomson, > [9] Capability Maturity Model Integration (CMMI-DEV v1.3). > [10 ]SEPG Presentations, e.g., High Maturity Misconceptions, Will Hayes,