"Agile Testing: What Would Deming Do?"

Transcription

1 T10 Class 4/19/ :15:00 AM "Agile Testing: What Would Deming Do?" Presented by: Mark Strange Wood Cliff Consulting Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL

2 Mark Strange Wood Cliff Consulting IT management consultant Mark Strange focuses on training, coaching, and implementing agile/scrum for software development organizations. With his background in engineering, quality management, and software development, Mark has helped many companies in a wide range of industries to improve their software development efficiencies. Contact Mark for tailored agile training, coaching, or consulting at mstrange@woodcliffconsulting.net, or visit the Wood Cliff Consulting website at woodcliffconsulting.net.

3 Agile Testing: WWDD? (What Would Deming Do?) STAREAST 2012 April 19, 2012 About Mark Strange Background in manufacturing and engineering Over 15 years in IT development and management Certified ScrumMaster and member of Scrum Alliance Registered Project Management Professional (PMP) Main job is as a project manager, team manager and Agile facilitator Has made LOTS of mistakes! 1

4 Today s Objectives Objective 1: Briefly review Deming's management principles and their impact on Japanese and American business. (14 points, P-D-C-A, Definition of Quality) Objective 2: Discuss ways in which current software testing methods do / do not align with Deming's principals. Objective 3: Discuss ideas for improving software quality and decreasing our reliance on end-state regression testing based on Deming's teachings. Software Development Challenges We struggle with release dates. We go over budget. We build features that our customers do not want. We spend more and more on increased inspection, but somehow the above problems get worse, not better. These issues are nearly identical to the issues faced by American manufacturing in the 1960s and 1970s! 2

5 Dr. W. Edwards Deming - Background William Edwards Deming (10/14/ /20/1993) Made a significant contribution towards the rise of Japan s economic power beginning in the 1950s Deming s message to Japan: Improving quality will reduce expenses while increasing productivity and market share. Dr. W. Edwards Deming More Background Began working with Ford in By 1986, Ford was the most profitable American auto company. Stressed new management concepts and the use of statistical techniques to improve quality. Taught the concepts of special cause and common cause variation in a system. 3

6 Deming s 14 Points ( Out of the Crisis ) 1. Create constancy of purpose toward improvement of product and service, with the aim to become competitive and stay in business, and to provide jobs. 2. Adopt the new philosophy. We are in a new economic age. Western management must awaken to the challenge, must learn their responsibilities, and take on leadership for change. 3. Cease dependence on inspection to achieve quality. Eliminate the need for massive inspection by building quality into the product in the first place. 4. End the practice of awarding business on the basis of price tag. Instead, minimize total cost. Move towards a single supplier for any one item, on a long-term relationship of loyalty and trust. 5. Improve constantly and forever the system of production and service, to improve quality and productivity, and thus constantly decrease costs. 6. Institute training on the job. Deming s 14 Points (continued) 7. Institute leadership (see Point 12 and Ch. 8 of "Out of the Crisis"). The aim of supervision should be to help people and machines and gadgets to do a better job. Supervision of management is in need of overhaul, as well as supervision of production workers. 8. Drive out fear, so that everyone may work effectively for the company. (See Ch. 3 of "Out of the Crisis") 9. Break down barriers between departments. People in research, design, sales, and production product management, design, development, and QA must work as a team, to foresee problems of production and in use that may be encountered with the product or service. 10. Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity. Such exhortations only create adversarial relationships, as the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force. 4

7 Deming s 14 Points (continued) 11. a. Eliminate work standards (quotas) on the factory floor. Substitute leadership. b. Eliminate management by objective. Eliminate management by numbers, numerical goals. Substitute leadership. 12. a. Remove barriers that rob the hourly worker of his right to pride of workmanship. The responsibility of supervisors must be changed from sheer numbers to quality. b. Remove barriers that rob people in management and in engineering of their right to pride of workmanship. This means, inter alia," abolishment of the annual or merit rating and of management by objective (See Ch. 3 of "Out of the Crisis"). 13. Institute a vigorous program of education and self-improvement. 14. Put everybody in the company to work to accomplish the transformation. The transformation is everybody's job. Plan-Do-Check-Act (Deming Circle) Plan Establish objectives & processes Implement the plan Act Do Measure, study the results, compare to objectives Check Take corrective action on differences between planned & actual results Repeat! 5

8 Deming s Definition of Quality Quality = Software teams tend to focus on results, with little regard for cost Results of Work Efforts Total Cost Management teams tend to focus on Cost rather than improving the system Some Questions to Ask about Software Quality How do I define Quality in my product / process? What should I measure? (Defects? If so, how?) What metrics should I keep, and how should I use them to make decisions? With Agile techniques gaining acceptance, how should I use the traditional QA function (or should we use it at all)? 6

9 Balloon Exercise In your groups, Produce as many balloons as you can in 1 minute (2 eyes, nose, mouth) Balloon Exercise In your groups, Produce as many balloons as you can in 1 minute (2 eyes, nose, mouth)

10 Balloon Exercise In your groups, Produce as many balloons as you can in 1 minute (2 eyes, nose, mouth) Scrum (One of the Agile Processes) Daily Standup Sprint Backlog Sprint weeks Prioritized Product Backlog (User Stories) Sprint Backlog (broken down by team) New Functionality (demonstrated at end of sprint) Retrospective / Action Items (Continuous Improvement) 8

11 Agile... is based on Deming s Plan Do Check Act concept. brings software teams into closer alignment with their customers. helps us build only features that are desired. seeks to reduce costs by finding defects earlier in the development process. But we continue to... struggle to get all testing done within the sprint. allow testing to accumulate against our better judgment. spend more and more $$$ on inspection, without a corresponding decrease in overall costs. struggle to change management behavior, which is critical for successful Scrum implementation. 9

12 What Would Deming Do? Make product and process quality everyone s responsibility (make it a management issue) Define quality in detail for your product and process (and decide how to measure it) Strive to reduce costs Expand the role of the QA team (collect data, analyze, improve) Use statistics to analyze data Example: Measuring Defects Sprint New Defects Defects 10

13 Measuring Defects - The Problem Sprint New Defects Story Points Story Points Defects Measuring Defects - Normalized Data Sprint New Defects Story Points Defects/SP

14 Control Charts Upper Limit Lower Limit Upper Limit Lower Limit Limits of variation attributable to the system: = x ± 3 * (Std Deviation) = x ± 3 Variance Upper Limit Lower Limit n 2 Control Limits Upper Limit Lower Limit 2 Upper Limit Lower Limit Upper Warning Lower Warning Upper Warning Lower Warning = ± 3 Standard Deviations = ± 3 Standard Deviations = ± 3 Standard Deviations

15 Defects / Story Point - Control Chart UCL LCL Control Chart Scenarios What does each chart signal? 13

16 Scenario 1 Scenario 1 Data points outside control limits special cause variation 14

17 Scenario 2 Scenario 2 Upward trend - non-random variation 15

18 Scenario 3 Scenario 3 String of data points above mean upward shift in mean 16

19 Scenario 4 Scenario 4 All points +/- 1σ : Recalculate Control Limits 17

20 Scenario 5 Scenario 5 Process in statistical control! 18

21 Main Points Software QA must evolve for software development teams to remain competitive Low Defect Rate + High Cost == High Quality Testing burden must be pushed upstream to development QA s job is to define, measure, monitor, and improve quality Control charts are one tool to measure and monitor quality Please feel free to me if you have any questions. mstrange@woodcliffconsulting.net Good luck with your Agile projects! 19