Measure Twice, Cut Once

Transcription

1 Measure Twice, Cut Once IMPROVING YOUR IMPROVEMENT THROUGH BETTER METRICS Michael Cardinal ITSM Academy ITIL Expert, CPDE, Educator, Amateur Woodworker 2 1

2 A chance to WIN! Measure Twice, Cut Once Always measure many, before you cut any -John Florio, Second Frutes 1591 Common mistakes in woodworking all relate to measurement Using imprecise or incorrect tools for measuring Inaccurate or inconsistent measuring Thinking, working or measuring too fast Poor measurement math Measuring the wrong pieces These mistakes apply to measurements and metrics in business and IT 3 2

3 Imprecise or Inaccurate tools 4 Imprecise or Incorrect Tools Confusing terminology Choosing the wrong measurement tool Using a ruler when a digital caliper is needed Not marking in pencil Being unable to erase mistakes and correct them Eyeballing 5 3

4 What is a Metric? A metric is the result of a performance measure. Effective metrics consider 4 process aspects Compliance - Are we following the process? Quality - Are we executing effectively? Performance - Are we executing efficiently? Value - Is what we are doing facilitating business goals and objectives? What are the proper terms and tools to use? 6 MEASUREMENT (Quantification) METRIC (Result-Point Data) TARGET KPI CSF (Metric over Time; (Required Elements) Directionality) 7 4

5 Process Metric Types Efficiency (Cost: Value) Inputs Economy (Costs) Process Activities Outputs Effectiveness (Value) 8 Inaccurate or Inconsistent Measurements 9 5

6 Inaccurate or Inconsistent Measurements Failing to follow a metrics program Using the wrong scale for measurement Maximum scales when minimum are needed and vice versa Forgetting the kerf (blade width) Using tools with too much wobble or variation 10 Producing Meaningful Metrics Best practice Implement a metrics program Use metrics to achieve business and process improvement goals Automatically monitor metrics and publish reports Common practice Measure what you can Focus on operational performance Manually produce reports 11 6

7 Implement a Metrics Program A metrics program describes The metrics needed to achieve business goals How to collect them How to use metrics to continually improve performance An effective metrics program focuses on What you should measure to achieve business goals Individual process performance Process interfaces Examples of metrics programs include the Balanced Scorecard and Results that Matter approaches. 12 Ten Steps for Creating a Metrics Program Step 1 - Determine management s vision Step 2 - Identify critical success factors Step 3 - Identify key performance indicators Step 4 - Identify metrics Step 5 - Verify that metrics are SMART Step 6 - Identify required data elements Step 7 - Test and pilot your metrics and reports Step 8 - Document your metrics and reports Step 9 - Place approved metrics and reports under change management control Step 10 - Continually review reports for effectiveness 13 7

8 Thinking, Working or Measuring Too Fast 14 Thinking, Working or Measuring Too Fast Failing to understand the importance of meaningful metrics Skipping measurements altogether Seeing the tree not the lumber Not using metrics for better integration Failing to write down the results 15 8

9 Using Metrics to Achieve Goals Vision CSF KPI Strategy, goals and objectives Deliver cost-effective services in a competitive timeframe What must happen to achieve process success Make changes quickly and accurately Key metric used to manage a process Reduce the number of emergency changes Increase the number of standard changes Metric What is measured to manage a process Number of changes by type 16 The Importance of Process Integration Measuring a single process may not be enough The output of one process is input to another Consider the effect of changes on integrated processes Monitor, measure and improve interfaces Communicate process integration success to Recognize accomplishments Show proof of concept Provide incentive to continually improve 17 9

10 Measuring Process Integration # of Incidents Problem Management Change Management Availability, Capacity, Security, Continuity Management Release Management Process Maturity Every ITSM process has a relationship with Every other ITSM process Other IT processes such as project management Other business processes such as financial management 18 Poor Measurement Math 19 10

11 Poor Measurement Math Not using the Seven Tools of Quality Focusing only on the numbers Not focusing on analysis What will we do as a result of the analysis? Focusing on outliers rather than correlation # of Changes Scatterplot-Change vs Incident # of Incidents 20 Seven Basic Tools of Quality Cause-and-effect diagrams Check sheets Control or run charts Flowcharts Histograms Pareto charts Scatter diagrams 22 11

12 Automatically Monitoring Metrics and Publishing Reports Automated reporting Reduces the effort to manually produce reports Reduces the number of reporting errors Increases the timeliness of reports Understand What data is available? What is the definitive source of that data? Is the data located in disparate systems? What tools are available or needed to aggregate and summarize the data? What tools are available or needed to produce reports? 22 To Automate Metrics and Reports Identify the data sources Identify and document algorithms Design reports that Meet audience needs Aid understanding Control reports via change management Determine the distribution medium Schedule and automate distribution 23 12

13 Measuring the Wrong Pieces 24 Measuring the Wrong Pieces Making metrics about people To err is human Measuring at the wrong time or for the wrong reasons Following the action, not as a driver of behavior 26 13

14 Roles Involved in CSFs, KPIs, Metrics Service management program owner Responsible for overall program Process owners Responsible for process specific metrics Process stakeholders Validate metrics and participate in baselining Data analysts Ensure needed data is captured, summarized and assembled into appropriate reports and graphs Technical specialists Facilitate tool implementation and administration Process practitioners Capture data and use to demonstrate and improve performance 26 Measuring Individual Performance IT staff complaints I m entering all this data and no one is using it The data I m entering is being used as a stick to criticize and find fault with my work The data being collected is not useful to me Process efficiency is just another word for downsizing Management must Use data to improve service quality and job satisfaction Communicate honestly about efforts to optimize staffing levels Avoid using metrics to make the numbers right Create a spirit of learning, growth and continual improvement 28 14

15 Frameworks and Metrics The Deming Cycle (PDCA) is an integral aspect of ISO/IEC All best practice frameworks and models stress metrics The check step of the Deming Cycle requires Methods for monitoring and measuring processes The ability to show that processes achieve results Planned reviews and audits to ensure processes Conform to the ITSM plan Are effectively implemented and maintained 29 Metric Usage Well designed and meaningful metrics can Justify existing or additional resources Compare performance against A baseline A target or goal Another organization Demonstrate proof of concept Initiate corrective action when needed The ability to measure process performance is one of the characteristics the PMF uses to assess process maturity

16 Questions? 30 Contact info: MJ_Cardinal Michael Cardinal 16

17 Thank you for attending this session. Don t forget to complete the evaluation! Session #302 Improving your Improvement Through Better Metrics 33 17