"Meaningful Metrics for Agile Teams and Organizations"

Transcription

1 KT2 Keynote 11/18/2010 4:30:00 PM "Meaningful Metrics for Agile Teams and Organizations" Presented by: Niel Nickolaisen EnergySolutions Brought to you by: 330 Corporate Way, Suite 300, Orange Park, FL

2 Niel Nickolaisen EnergySolutions Niel Nickolaisen is one of the founders of Accelinnova, a think tank focused on improving organizational and IT agility. Niel has held technology executive and operations executive positions typically in turnaround roles. His strategic and tactical alignment model significantly improves returns on technology and business initiatives. Niel writes an enterprise CIO column for TechTarget and is the co-author of Stand Back and Deliver: Accelerating Business Agility. CRN Magazine named Niel one of the top 25 IT Executives of 2008.

3 Meaningful Metrics for Agile Teams and Organizations The BIG Idea We should carefully select metrics because...

4 Metrics Influence Behavior Meaningful metrics lead to meaningful behavior this is a good thing. Meaningless metrics lead to meaningless behavior we do better if we avoid this. Why Be Careful?

5 How About An Example? Manufactured Stone The Situation Two types of stone. Flats Corners Primary production metric is: Square feet produced per person hour. It takes 3 times more person hours to produce a square foot of corners than it does flats. Total cycle time (production and curing) is about one day.

6 How Does This Metric Influence Behavior? Actual Results The company has combined inventory turns of 4 per year. But, there are not enough corners to meet customer demand. As a result, On Time Shipments are lower than 50%. But, each plant exceeds its square foot per person hour goal Hooray!

7 In Other Words... How Can We Solve This Problem?

8 First Option Change that troublesome On Time Shipment metric! Old On Time Order is late if a single line item does not ship.

9 New On Time Order is on time if a single line item ships. The Results? On Time Score Soars to over 95%!

10 What is it good for? Absolutely Nothing - War The Big Quiz Did the square feet per person hour lead to meaningful behavior? Did it improve the company s production processes? Its competitive advantage? Is this a meaningful metric? What would have been the characteristics of a more meaningful metric?

11 ANOTHER EXAMPLE The company wants to encourage employee involvement. Company defines a metric and goal for: Five employee suggestions per month. What happened? A labeling frenzy.

12

13 How About One More? How meaningful is this measure of development productivity? How many of the lines were usable? How many of the lines were high priority? How many of the lines generated value? OK, So What Do We Do? My over arching goal is to use metrics that: Measure accomplishment rather than activity. Lead to continuous process improvement.

14 Acceptance Criteria Meaningful Metrics: 1. Represent reality 2. Measure processes not people 3. Are few in number 4. Are mostly non financial 5. Align with strategy 6. Show trends 1. Represent Reality According to the great philosopher Polonius... To thine own self be true And, the most important characteristic of successful leaders? Self Awareness

15 2. Measuring Processes Not People Some philosophical mumbo jumbo: 85% of a person's effectiveness is determined by the system he works within, only 15% by his own skill. Edward Deming Our real aim is to bring out the capabilities of each individual. The ultimate aim is to draw out people s motivations. Inside the Mind of Toyota If we measure people, we miss the majority of the improvement opportunities and crush motivation. Let s Return to An Example Does lines of code per day (or per developer day / hour) measure processes or people? If processes, what processes does it tell us to improve?

16 3. Are Few In Number If we want to measure (and improve) processes and not people, how many do we need to drive meaningful development behavior? Lines of Developer Code Per Day Code Coverage On Time On Budget Defects Per Line Customer Request Completion Bugs Fixed % Requirements Tested Code Merge Conflicts % Security Mapped To Code Ratio of Internal / External Bugs Found % Technical Debt Remaining Estimate Accuracy % Documentation Complete % Data Redundancy Maintenance Cost Per Line of Code Are Few In Number... To avoid metric conflicts (lines of code per developer day / % documentation complete). So that we can focus our continuous process improvements on those that generate the most value. The Only Metric You Will Ever Need The Net Promoter Score How many of our customers would recommend us to someone they know? How many of our customers if given the choice would use us again?

17 4. Are Mostly Non Financial What choice do we have? What about revenue generated per line of code? What is the direct, measurable link? How about product profitability? How meaningful is this for our compliance module? Our security module? 5. Align With Strategy We favor Accomplishment over Activity But, what defines Accomplishment?

18 6. Show Trends Is this contribution margin performance good or bad? June Total Variable Costs as a % of Revenue 59.07% Materials as a % of Revenue 32.70% Direct Labor as a % of Revenue 6.00% Variable Overhead as a % of Revenue 8.10% Commissions as a % of Revenue 3.40% Does This Help?

19 Start with strategy So, What Do We Do? Purpose Based Alignment Model High Partner? Differentiate Market Differentiating Who cares? Parity Low Low Mission Critical High

20 So What? Two big benefits: First, in our market, what creates competitive advantage? How can we measure how well we are doing this? Second, in our market, how do we compare to parity? This also defines Accomplishment over Activity Mapping Differentiating to Measures How do we measure the processes that directly generate competitive advantage?

21 Specialty Retailer Differentiating = Product Selection How to measure software development? Product Line Growth Decision Quality Quality and Timeliness of Business Analytics Healthcare Software Company Differentiating = Product Innovation Measure? Percent of revenue from products released in the past two years.

22 Mapping Parity to Measures What is barely sufficient? How does this map to metrics? Specialty Retailer Starting Point: Overly complex business rules (better than parity) In order to improve this process, measure % of required customizations in ERP, POS, CRM systems.

23 Healthcare Software Company Differentiating = Product Innovation Goal: Free up development to work on new stuff Measure? % of staff supporting 3+ year old releases drove improvements in lifecycle management Some Metric Areas To Consider

24 Cycle Time Concept to First Iteration Feature Request to Deployment Defect to Patch Business Case Product Performance Market Share Growth % of Revenue From New Products

25 Customer Service Net Promoter Score Do We Keep Our Commitments? Helpful versus Right Customer Retention Recap Meaningful Metrics: Represent reality Measure processes not people. Are few in number. Are mostly non financial. Align with strategy. Show trends.

26 Getting Started Brainstorm A process (not people) to improve

27 Map The process to purpose differentiating / parity to establish the design goal Meaningful metrics that will monitor improvement s to the process Select

28 The metrics. If they work, the process will improve. If not, they won t. If they stop working, stop using them. Pilot When Good Enough Move to the next process improvement candidate

29 A Final Thought Think it through. Goal, reduce calls to the service desk by 50%. Results? Unplug half the phone lines. Questions?