LEAN IN THE SOFTWARE TEST LAB

Transcription

1 LEAN IN THE SOFTWARE TEST LAB Kathy Iberle Iberle Consulting Group, Inc. Pacific Northwest Software Quality Conference October 15, 2013

2 HELLO, MY NAME IS KATHY IBERLE Kathy Iberle has been working with agile software development and Lean development teams for many years. She has developed unusual expertise in the challenging area where hardware and software meet. Kathy recently retired from Hewlett-Packard after a long career as a programmer, quality engineer, and process improvement expert and is now the principal consultant and owner of the Iberle Consulting Group. Visit her website at 2

3 INTRODUCTION What is Lean? History of Lean Methods How Lean Methods Work Batch Size WIP Controls Other Lean Tools 3

4 LEAN: THREE BODIES OF KNOWLEDGE Lean Science: Batch Size, WIP Limits, Pull, Flow, Cadence Lean Knowledge Stewardship: A3s, Five Whys, Kaizen Lean Management: Visual Controls, Leadership, Education Alan Shalloway in Scaling Software Agility 4

5 THE SYSTEMS VIEW Lean Science views your development system as a machine turning ideas into saleable features. 5

6 LEAN: A NEW WAY TO SEE THINGS See the work as discrete batches See the organization as a set of activities, not as an org chart PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 Waiting in line 6

7 THE WORK MOVES THROUGH A SYSTEM PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE Queue: A waiting line

8 START OF DAY 1 PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE

9 END OF DAY 1 PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE

10 START OF DAY 2 PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE

11 START OF DAY 2 PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE

12 MIDDLE OF DAY 2 PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE

13 END OF DAY 2 PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE

14 END OF DAY 2 PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE

15 START OF DAY 3 PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE

16 START OF DAY 3 PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE

17 AN AVERAGE DAY PEOPLE DOING STEP 1 PEOPLE DOING STEP 2 WORK QUEUE QUEUE DONE This system deals with: unpredictably arriving batches of work of variable size using resources of fixed capacity

18 TELEPHONE SYSTEMS How Many? Phone lines Switchboard operators PHONE CALLS QUEUE QUEUE DONE

19 MATH TO THE RESCUE! Ågner Krarup Erlang applied statistical methods to solve the problem. He invented queueing theory. Ågner Krarup Erlang Denmark 19

20 WHAT ERLANG (AND OTHERS) LEARNED It is possible to do more with the same capacity. Which bank will serve more customers today? Why? 20

21 TODAY, QUEUEING THEORY IS USED FOR: Telecomm systems Internet design Traffic control Manufacturing Lines 21

22 SO WHAT? Queueing theory lets us accurately predict the behavior of a system. How many batches will get done per week - throughput How long it will take to do the average batch cycle time What to change to get different results This method can be used to manage work. Better predictability Higher throughput Less stress on people 22

23 LEAN AND QUEUEING THEORY Stagewise model Bennington 1956 NASA, IBM 1960s Waterfall Royce (1970) Mil-Std 671 (1979) Evo: Gilb 1980s Agile Software Methods Plan-Driven Software Methods Scrum: Schwaber, Beedle 1997 XP: Beck 1999 Agile Manifesto 2001 Poppendieck 2003 Anderson Kanban 2007 Leffingwell Shalloway present Erlang s Theorems 1909 Packet Switching Kleinrock 1960s Little s Law 1961 Queueing Theory Matrix Analytic Method, etc. Henry Ford 1912 Toyota Production System: Toyoda, Ohno 1940s Kanban: Ohno 1950s Lean Methods JIT manufacturing 1980s The Machine That Changed the World Womack 1980s Managing the Design Factory Reinertsen 1997 Alan Ward nd -Generation Lean

24 WATCH THE WORK, NOT THE WORKER Split work into batches which deliver value. Make your activities and queues visible. Track progress of batches through your system. Optimize system via Batch size Work-in-process controls Other tools 24

25 WHAT IS A BATCH? Batch = a set of work Has a distinct start and a distinct end Delivers value when finished Why must a batch deliver value? Because we want to maximize our output of value not our effort 25

26 WHAT VALUE DOES A TEST LAB PRODUCE??? 26

27 WHAT VALUE DOES A TEST LAB PRODUCE? Prediction of project end date Information! Likelihood that user will encounter a failure 27

28 TYPICAL BATCHES IN A SOFTWARE TEST LAB Testing batches Feature 1 Feature 3 Other work Feature 2 Improve automated test framework Performance test Final report Move equipment to building 2 28

29 A BATCH HAS A START AND AND AN END When is a testing batch done? Regression: a pre-defined suite has been run Functionality: No new critical defects found in last 20 hours of testing Spot-check: 10 hours of testing have been performed Other possible criteria for done Customer agrees to accept batch as-is Measurable attributes are within predetermined values All items on checklist are green We have used up the allocated time 29

30 WATCH THE WORK, NOT THE WORKER Split work into batches which deliver value. Make your activities and queues visible. Track progress of batches through your system. Optimize system via Batch size Work-in-process controls Other tools 30

31 MAKE YOUR ACTIVITIES VISIBLE Stuff to test NOT STARTED QUEUE WRITING TESTS RUNNING TESTS WRITTEN QUEUE DONE In Process Show handoffs between different groups or activities. Show queues when there is significant waiting. 31

32 VISIBILITY: A VISUAL PLANNING BOARD Not Started Writing Written Executing Done Each sticky note represents a batch. Each column represents a state or queue. 32

33 WATCH THE WORK, NOT THE WORKER Split work into batches which deliver value. Make your activities and queues visible. Track progress of batches through your system. Optimize system via Batch size Work-in-process controls Other tools 33

34 TRACK PROGRESS OVER TIME Each week, count how many batches are Not Started In Process Done Graph as a Cumulative Flow Diagram (CFD) 34

35 WHAT YOU CAN SEE ON THE CFD Arrival Rate (Demand) Departure Rate (Throughput) 35

36 EXTRAPOLATION OF PROGRESS Not Started Being Tested Being Fixed Done 36

37 THE HOCKEY STICK EFFECT 37

38 WATCH THE WORK, NOT THE WORKER Split work into batches which deliver value. Make your activities and queues visible. Track progress of batches through your system. Optimize system via Batch size Work-in-process controls Other tools 38

39 A PREDICTABLE SYSTEM CAN BE OPTIMIZED By eliminating bottlenecks By analyzing each step for waste 39

40 VALUE-STREAM MAPPING 40

41 OPTIMIZE A VARIABLE SYSTEM Monitor and adjust in real-time Things to watch: Batch size Work-in-process Cadence Feedback channels 41

42 BATCH SIZE Batch = set of things moving in lockstep through your process One large batch Batch 1 Batch 2 Two smaller batches 42

43 SOFTWARE TEST A BATCH Setup: Create account and user Step 1) Log in 2) Find account 3) Enter deposit 4) Enter withdrawal 5). 6) 7). Teardown File Defect Reports Expected Results Actual Results This could be one long test OR two shorter tests What are the advantages of one large test? 43

44 SMALL BATCHES MORE FLEXIBILITY When this test is done, I can choose to run any of these three tests 44

45 SMALL BATCHES PREDICTABILITY At the start of the program, uncertainty in end date is always high. The only way to reduce uncertainty is to finish a batch which is an actual end product. Then you know how long that one took. One Large Batch Uncertainty in End Date (weeks) Lots of Small Batches Uncertainty in End Date (weeks) 45

46 WHAT S THE BEST SIZE FOR A SOFTWARE TEST? Transaction Cost: Setup: Enter test as started Set up account and user Teardown: Delete account and user Enter total time spent on test, who ran it, etc. Holding Cost: Delay in value: Don t get defect reports until entire test is over. Lack of flexibility: Can t decide to start a different test. Other cost of delay: More likely that staffer is interrupted. Test becomes inactive but test station still in use. 46

47 TRANSACTION COST VS. HOLDING COST SWEET SPOT 47

48 WORK IN PROCESS Time and money is spent on maintaining lists and status of partially finished work. Partially finished work can become obsolete mid-stream. Workers spend time switching between tasks 48

49 WIP CAUSES MULTI-TASKING Psychology studies show: Task switching for easy tasks: 10 seconds and up Task switching during knowledge work: up to 10 minutes This cost is paid every time you switch between jobs. Task-switching is a huge hidden cost in knowledge work. Cohn, Mike; Agile Estimating and Planning; 2006, p.15; Original data from Clark and Wheelwright s Managing New Product and Process Development: Text and Cases; 1993

50 WORK IN PROCESS (WIP) Work in Process 50

51 week 1 week 2 week 3 week 4 week 5 week 6 week 7 week 8 week 9 week 10 week 11 week 12 week 13 week 14 week 15 week 16 week 17 week 18 WHAT HAPPENS WHEN WIP GROWS? Doing more jobs concurrently increases cycle time Not Started In Process Done

52 A WIP LIMIT CONTROLS WIP Not Started Writing Tests Executing Tests Done WIP Limit = 8 One gate at the start. Don t start new things until there is room. 52

53 CADENCE IS A POWERFUL TOOL Meetings or actions on a regular schedule Reduces overhead, reduces decision points Week 0 Week 2 Week 4 Week 6 Week 8 Week 10 Week 12 Week 16 53

54 SLOW CADENCE CAN CAUSE HIGH WIP NOT STARTED WIP DONE 54

55 OTHER TOOLS OF LEAN SCIENCE Cadence Clean, well-defined feedback channels Standup meetings Pull workflow via Kanban 55

56 LEAN SCIENCE IN A NUTSHELL Split work into batches which deliver value. Make your activities and queues visible. Track progress of batches through your system. Optimize system via Batch size Work-in-process controls Other tools 56

57 REFERENCES Principles of Product Development Flow; Reinertsen, Donald G; 2009; Kanban: Successful Evolutionary Change for Your Technology Business; Anderson, David; 2010 Iberle Consulting Group resource page: Visual Planning Boards for Small Groups More Techniques for Visual Planning Boards Visualizing Workflow in Your Organization Cadence: Increase Efficiency by Using a Rhythm Finding the Best Frequency for a Recurring Activity 57

58 Questions? 58