Realistic, Actionable Schedules Truthful Alignment of Prediction and the Future

Size: px

Start display at page:

Download "Realistic, Actionable Schedules Truthful Alignment of Prediction and the Future"

Elisabeth Gaines
6 years ago
Views:

1 Realistic, Actionable Schedules Truthful Alignment of Prediction and the Future

Realistic Schedules How Long Will Testing Take What makes an estimate a good

included, accurately estimated, risks understood and mitigated Actionable:

dependencies Estimation process Ask experts and owners Consult metrics and

2 Realistic Schedules How Long Will Testing Take What makes an estimate a good one? Accurately predicts and guides the project s future Realistic: All tasks included, accurately estimated, risks understood and mitigated Actionable: Clear ownership by committed individual contributors, assigned resources, known dependencies Estimation process Ask experts and owners Consult metrics and industry averages Negotiate with managers and stakeholders Copyright (c) RBCS Page 2

3 Manager and dstakeholder Expectations ti Cover the test basis s completely, regardless ess of cost and schedule implications Cover the test basis as completely as possible within schedule constraints (cost not an issue) Cover the test basis as completely as possible within schedule and cost constraints Break the iron triangle (ignore interaction of schedule, budget, quality, features) Realistic Schedules Copyright (c) RBCS Page 3

4 Developing a Work-Breakdown-Structure kd t What are the major stages of a testing subproject? Planning Staffing (if applicable) Test environment acquisition and configuration Test development Test execution Break down to discrete tasks within each stage What test suites are required for the critical risks? E.g., functionality, performance, error handling, etc. For each suite, what tasks are required? Tasks should be short in duration (e.g., a few days) Inch-pebbles as well as milestones Realistic Schedules Copyright (c) RBCS Page 4

5 AHpthti Hypothetical lcase Study Assume you re the test manager for Grays and Blues, a US Civil War-based video game The test team consists of yourself plus Two test engineers Five test technicians Your team is responsible for System Test Your organization uses the V model Test strategy is primarily scripted manual test cases with some automated load and reliability tests An initial work-breakdown-structure might look like Realistic Schedules Copyright (c) RBCS Page 5

6 Realistic Schedules Copyright (c) RBCS Page 6

7 Tapping the Team s Wisdom for Estimation Delphic Oracle Technique 1. Gather estimates from each team member 2. Ask high and low estimators to explain estimates 3. Repeat twice, then use average Three-Point Technique 1. Have team estimate bestcase, worst-case, and expected-case 2. Use expected-case 3. Variances between best and worst useful to gauge estimate accuracy These techniques can be combined (which is called the Wideband technique) and also used with other sources of estimation like historical project data, expert advice, and rules of thumb Realistic Schedules Copyright (c) RBCS Page 7

8 Planning Session with Team Project team estimates effort and duration Finds missing activity: Smoke test Variance in best-case/worst-case indicates schedule could be 25% off Risks identified and listed Because it s the team s estimate, commitment is higher The sized WBS might look like Realistic Schedules Copyright (c) RBCS Page 8

9 Realistic Schedules Copyright (c) RBCS Page 9

10 Ud Understanding di Dp Dependencies Assign Dependencies 1. Identify tasks with no predecessors 2. Identify tasks dependent only on previouslyidentified tasks 3. Repeat step two until dependencies all identified Small projects: Can use project management tools (e.g., MS Project) Medium to large projects: Use index cards or sticky notes and whiteboard Realistic Schedules Analyze Critical Paths A set of dependent tasks where delay in any task delays the project end Near-critical path exist where significant delays will delay yproject end What affects phase entry and exit criteria? During test execution: How many test passes, releases, and cycles? (More later) External dependencies: a frequent cause of delay Copyright (c) RBCS Page 10

11 Assigning i Resources People Test engineers and test technicians, contractors and employees, outside test resources Using less-skilled people increases task effort, duration People s skill with a given task or tool determines estimate accuracy Test environments Hardware, software, networks, facilities, etc. Especially important to include expensive or long-lead-time items like large servers, test tools, lab space Test tools and testware Custom (new or reused) Commercial off-the-shelf Don t assume two people can get the job done in half the time Don t overload tools or the test environment Don t forget setup and support tasks for environments and tools Realistic Schedules Copyright (c) RBCS Page 11

12 Realistic Schedules Copyright (c) RBCS Page 12

13 Realistic, Actionable But t Too Long? We have a realistic, actionable schedule but the executive management team says We don t have until April 30! We need to ship sooner than that What options do we have Realistic Schedules Copyright (c) RBCS Page 13

14 Allow Overlap of Phases Classic waterfall model says one phase must end before next begins Suppose we let an almost complete build in to start system test, waiving the feature complete entry criteria? Voila, April 16 is new end date (16% earlier) but overlap may increase risks to quality Can developers fix bugs and finish system at same time? Testing against an unready system often is less efficient Realistic Schedules Copyright (c) RBCS Page 14

15 Realistic Schedules Copyright (c) RBCS Page 15

16 Add Staff Another solution is to add staff Increase test and programmer staff to run test passes in a week and fix bugs quickly Now, March 26 is new end date (40% earlier) but the human resource budget is doubled Can we hire new people and ramp up them up in time? Realistic Schedules Copyright (c) RBCS Page 16

17 Realistic Schedules Copyright (c) RBCS Page 17

18 Arbitrary Reductions in Test Execution? Maybe we can accept lesser level of quality Cut test execution in half (i.e., run one and a half passes and three cycles)? This gets us done on March 26 too but will we really be done testing??? Cutting test execution is a very risky strategy Schedule: System not good enough at end date Quality: Bugs found late in testing, no time to remove them Budget: Schedule slips during testing are expensive Features: May have to drop buggy functions late in the game Realistic Schedules Copyright (c) RBCS Page 18

19 Realistic Schedules Copyright (c) RBCS Page 19

20 Use Risk to Reduce Test Subproject Scope Eliminate test coverage Identify lowest-risk ik areas Drop coverage or Use opportunity or report bugs approach Reduce extent of testing Realistic Schedules Identify the least risky of the highest-risk areas Adopt a balanced (broad, not deep) test approach Other options Postpone automation ti or tool development Outsource some of the testing ti to reduce costs, speed schedule Be creative to come up with ways that work. Make sure that stakeholders approve reductions in scope. Be ready to iterate and negotiate. Copyright (c) RBCS Page 20

21 Realistic Schedules Copyright (c) RBCS Page 21

22 Drop Some System Functionality Suppose we drop multiplayer functionality, support for arcade, Mac, and Linux platforms, and non-us versions? This reduces test effort by about 25% Depending p gon effect on development, we could be done by March 26 However, more frequent releases increase regression quality risks so these features should be bundled with next regular release Realistic Schedules Copyright (c) RBCS Page 22

23 Realistic Schedules Copyright (c) RBCS Page 23

24 Realistic Schedules Realistic, Actionable Schedules Apply best practices of project estimation and management Develop a work-breakdown-structure that Is complete and accurate Captures and balances risks Has committed team and individual ownership Takes into account dependencies and critical path Allow executive management team to balance competing risks Guide the project to success! Features Schedule Quality Budget In a well-managed project, features, quality, schedule, and budget converge to a successful delivery realistic, actionable schedules are a foundation of good project management Copyright (c) RBCS Page 24

25 To Contact t RBCS For over a dozen years, RBCS has delivered consulting, outsourcing and training services for software and hardware testing. ti Employing the industry s most experienced and recognized consultants, RBCS conducts product testing, builds and improves testing groups and hires testing staff for hundreds of clients worldwide. Ranging from Fortune 20 companies to start-ups, RBCS clients save time and money through improved product development, decreased tech support calls, improved corporate reputation and more. To learn more about RBCS, visit. Address: RBCS, Inc Beck Road Bulverde, TX USA Phone: +1 (830) info@rbcs-us.com Web: Realistic Schedules Copyright (c) RBCS Page 25

Test Estimation Seeing the Future of Your Test Effort

Test Estimation Seeing the Future of Your Test Effort How Long Will Testing Take What makes an estimate a good one? Accurately predicts and guides the project s future Realistic: All tasks included, accurately