What Is The Urgency Index? The Power and Limits of Agile Metrics. Others Talk, We Listen

Transcription

1 What Is The Urgency Index? The Power and Limits of Agile Metrics Others Talk, We Listen

2 Learning Outcomes A better understanding of how to use metrics more effectively to drive change

3 My Outcome To Learn from You

4 Agile Transformation Coach at CapTech 5+ years as PM, RTE, or Coach 20+ years in technology Talk to me Twitter: LinkedIn:

5 Points This is a Story Urgency Index Commitment Delivery Sprints (Time)

6 The Urgency Index Commitment Delivery = Urgency

7 The Urgency Index = 1.67

8 Points What Is the Effect of Higher Urgency Index to People and Teams? Urgency Index Commitment Delivery Sprints (Time)

9 What Is The Effect of the Urgency Index? Waste! Grooming too soon/too much Context switching High WIP Quality? Stress

10 Why Is This Happening? Leadership committed to a solution and date for delivery of it without knowing the complexity Management wants the teams to feel endangered The team is ambitious and really wants to push themselves

11 The Agile Metrics Challenge Velocity Lead time Overtime Team NPS WIP Predictability Cyclomatic Complexity Defect Leakage Variability Cycle Time Unplanned work Build Failures Unit test coverage %

12 System Performance PI3.4 Velocity and Predictability System Velocity Average Commitment Completion % Sprint 1 Sprint 2 Sprint 3 Sprint 4 UM System Velocity 80% 70% 60% 50% 40% 30% 20% 10% 0% Sprint 1 Sprint 2 Sprint 3 Sprint 4 UM Average Commitment Completion %

13 System Performance PI3 - Features Ideal burndow n rate Actual burndow n PI3 Start

14 System Performance PI3 June Release Committed/Open/ReOpened, 245, 11% Refining, 53, 3% In Review, 53, 2% Resolved, Ready for Release, Closed, 935, 44% Ready, 146, 7% In Progress, 96, 5% Integration Test Prep, 68, 3% Test Ready, 391, 18% Testing, 160, 7% Committed/Open/ReOpened Refining In Review Ready In Progress Integration Test Prep Test Ready Testing Resolved, Ready for Release, Closed

15 UM System Performance PI3 Post June (Prediction) Release Information Defect breakdown included in release Prod Defects Found Between Releases Total Defects Fixed In Release Dev Defects in Release Prod Defects in Release Cont. Integ Defects in Release End2End Defects In Release Performance Defects In Release blank / training / stage defects in release Total Production Defects found after Release What happens if we hit our average number of prod defects/story with this release? Valid Production Defects found after Release % Valid Defects to stories found after release FixVersion Date of Release Stories in Release Defects / Stories SIT Defects in Release UAT Defects in Release Oct % % Nov % % Dec % % Jan % % Feb % % Mar % % Apr % % May % % totals % % % of Defect Source 0.13% 19.55% 0.43% 2.96% 0.57% 62.40% 13.53% 0.43% /29/ % % 12.04% 0.08% 0.08% 0.24% 84.53% 2.87% 0.16%

16 System Performance PI3 Defects & Stories Defects Found In Oct-17 Nov-17 Dec-17 Jan-18 Feb-18 Mar-18 Apr-18 May-18 Test - Cont. Integration Test - End to End Test - Performance Test - SIT Test - UAT Defects Found in Production Defects Found in Production - Warranty

17 System Performance PI3 Stories/Defects by Release 1200 Resolved - Stories vs. Defects Oct-17 Nov-17 Dec-17 Jan-18 Feb-18 Mar-18 Apr-18 May-18 Stories Defects

18 UM System Performance PI3 Defects Root Cause May 2018 Defect Root Cause Req Error, 87, 9% Test Error, 5, 0% Blank, 1, 0% Human Error, 12, 1% Env. Config. Error, 88, 9% Design Error, 11, 1% Oct May 2018 Defect Root Cause Req Error, 423, 9% Test Error, 41, 1% Blank, 3, 0% Human Error, 37, 1% Env. Config. Error, 448, 9% Design Error, 71, 1% Deployment Error, 146, 3% Deployment Error, 36, 4% Data Config. Error, 438, 9% Data Config. Error, 56, 6% Code Error, 3299, 67% Code Error, 687, 70%

19 Story Points Commitment Completion Story Points Train X Train and Teams Train X - Total Velocity Train X Predictability Trend Train X -Scope Change % % 60% 50% 40% 30% 20% 76% 61% 71% 40% % 2 0 Sprint 1 Sprint 2 Sprint 3 Sprint 4 0% Sprint 1 Sprint 2 Sprint 3 Sprint 4 0 Sprint 3 Sprint 4 Key Observations Velocity stabilizing across some teams on the train, however Commitment Completion dropping as stories pile up Commitment to starting work without intention of completing in-sprint, resulting in significant carry-over Off-train dependencies prevent delivery of sprint commitments

20 Train X Train and Teams

21 Train Y Kanban Teams Observations Teams are doing a better job of visualizing their work and putting work in Jira Stories are still too big so they don t flow and have the stair steps as above Too much WIP where stories queue then complete in big batches

22 The Signal and the Noise The signal is the truth. The noise is what distracts us from the truth. -Statistician Nate Silver

23 Driving Change with Metrics Connect Metrics to the Vision Build Systems Thinking Introduce New Mental Models Education Through Repetition Pick the Right Metrics But Not Too Many

24 Driving Change with Metrics Connect Metrics to the Vision Build Systems Thinking Introduce New Mental Models Education Through Repetition Pick the Right Metrics But Not Too Many

25 Vision We Are Going Agile

26 Vision: Why Do We Want to Change? So We Can: Keep Our Best Employees Increase Sales by Entering a New Market Increase Customer Results (and make them happier)

27 Points How Does This Connect with the Vision? Urgency Index Commitment Delivery Sprints (Time)

28 Driving Change with Metrics Connect Metrics to the Vision Build Systems Thinking Introduce New Mental Models Education Through Repetition Pick the Right Metrics But Not Too Many

29 Here

30 Local View

31 Broader View

32 Map The System

33 Points How Does This Build Systems Thinking? Urgency Index Commitment Delivery Sprints (Time)

34 Driving Change with Metrics Connect Metrics to the Vision Build Systems Thinking Introduce New Mental Models Education Through Repetition Pick the Right Metrics But Not Too Many

35 Mental Models

36 Mental Models

37 Mental Models

38 Mental Models

39 Mental Models Analysis Design Build Support

40 Mental Models Collaborative, Cross-Functional Delivery

41 Establish Mental Model: Delivery System Idea Support

42 Points What s the Mental Model here? Urgency Index Commitment Delivery Sprints (Time)

43 Driving Change with Metrics Connect Metrics to the Vision Build Systems Thinking Introduce New Mental Models Education Through Repetition Pick the Right Metrics But Not Too Many

44 Education Through Repetition

45 Present Your Models Over and Over Again

46 Highlight Small Changes

47 The System Review Same Set of Metrics All Quarter

48 Points How Does Repetition Fit in Here? Urgency Index Commitment Delivery Sprints (Time)

49 Driving Change with Metrics Connect Metrics to the Vision Build Systems Thinking Introduce New Mental Models Education Through Repetition Pick the Right Metrics But Not Too Many

50 Not Too Much Information at Once

51 Get Close to the Metal with Metrics

52 Business Speed Culture Quality $$ Velocity Team NPS Customer Interaction Failure Churn Avg Idea Lead Time Team Health Metrics Defects/Release NPS Deployment Cycle Time Attrition Rate Customer Complaints Lifetime Customer Value Key Metric Change Rate Skill Development Surveys Unit Test Coverage

53 Points Is the Urgency Index a Key Metric For You? Urgency Index Commitment Delivery Sprints (Time)

54 What s Your Take?

55 Final Words Remember: A hammer is a useful tool for all sorts of things like putting nails through wood but you can also use it to kill someone.

56 Connect with Me

57 Points What Is the Urgency Index? Urgency Index Commitment Delivery Sprints (Time)