ASPE Technology Toll-Free

Transcription

1 APE Technology Toll-Free Related Training Courses: Managing Real World Projects & Processes with Metrics oftware Testing & Quality Assurance Techniques Planning Effective oftware Testing eveloping & Writing Testable Business Requirements oftware Project Management Agile oftware evelopment Agile & High peed oftware Testing Techniques **There are no PMI PU s for this web seminar.

2 Managing oftware evelopment With Metrics Al orkowitz A. orkowitz (2006)

3 Capturing and Using Metrics to et New Goals

4 /W Metrics Provides a top-level management overview of the software development status Visibility into development process Compare actual progress to plans Early detection of potential problems Trends uncovered can be projected to evaluate potential impacts Points out inconsistencies in interrelated activities

5 oftware ize ize RR R R PR CR

6 oftware ize Metrics Tracks changes in the software size ize is a major variable used to estimate developmental effort and schedule Unplanned increases can adversely effect schedule and cost. oftware undersizing has been a problem due to optimistic estimates, lack of historical data on size growth, and an incomplete understanding of requirements. Tailoring - Each ystem/ubsystem

7 oftware ize Corrective Actions Increase over 5% per reporting period, or over 20% between end of phase reviews requires a cost/schedule impact. Analysis /W size should not exceed thresholds without a detailed understanding of the reasons for the change, and an understanding of the cost, and schedule impacts. Changes in size often result from a better understanding of the requirements.

8 * 0 oftware Test Personnel Jan Feb Mar Apr May Jun Jul Aug ep Oct Nov ec PLANNE ACTUAL

9 oftware Personnel Metrics Tracks ability to maintain planned staffing levels Corrective Action Changes of 20% against planned levels, measured over the complete software life cycle requires a cost/schedule impact, and revised plan. Tailoring idea Report staffing separately for senior people, unique skills. Analysis This metric can provide an early indication of schedule slips. Adding staffing late in a project may not improve schedule Intelligent use of overtime, and alternative shifts may help A high turnover rate in an indication of problems, which should be identifies and managed.

10 Tracking Testing Progress Module Level Unit Testing ystem/ubsystem Level Functional Testing

11 Unit/Module evelopment Progress CCI X * CU' EIGNE COE & TETE PLANNE ---- ACTUAL INTEGRATE 0 JAN FEB MAR APR MAY JUN JUL AUG EP OCT NOV EC JAN

12 Module evelopment Progress Metrics Track ability to keep unit design, code, test and integration activities on schedule. Corrective Actions A 20% change in total milestones completed, requires replanning Analysis The rate of planned progress should be reviewed to ensure that it is reasonable and not unusually steep.

13 Module evelopment Progress Once an actual trend line is established, it is difficult to modify the rate of completion unless a corrective action is applied, or the process was altered. Other metrics to review which may provide insight into problems include Requirements Volatility, oftware ize, and taffing. The availability (or non-availability) of appropriate historical productivity data may be a problem.

14 CU Metric Updated 800 * CU's esigned Plan 1 Plan 2 Actuals Jan Feb March April May June

15 CU Metric Update (cont) 800 * CU's esigned Plan 1 Plan 2 Actuals Jan Feb March April May June

16 ystem Testing Progress P LANNE CCI YTEM TE T 200 ACTUAL 0 TRR Fe b Apr Jun Aug Oc t

17 Testing Progress Metric Measures the ability to maintain progress during the ystem Testing Phase. Corrective Action A 20% change, (planned/actuals), requires replanning. Analysis The rate of planned progress should be reviewed to ensure that it is reasonable and not unusually steep. Once an actual trend line is established, it is difficult to modify the rate of completion unless a corrective action is applied, or the process was altered.

18 Testing Progress Progress may slip when a critical number of problems have been identified by the Testing Team and must be resolved by the development group before significant testing can continue. Unsuccessful tests reflect the additional amount of rework needed. This may impact cost/schedule. Early testing tend to discover more problems. If the number of unsuccessful tests does not decrease each reporting period, there may be serious problems with the quality of the software Other metrics to review which may provide insight into problems include Requirements Volatility, oftware ize, and taffing

19 Incident Tracking and Management oftware Test Incident - Any event occurring during the execution of a software test that requires investigation. (IEEE td 610)

20 FAULT PROFILE * OVERALL/OPEN TREN OPEN TOTAL JAN FEB MAR APR MAY JUL AUG EP OCT NOV

21 Error Correc tion Pe rformance The positive slope of the re d line indicates the number of unre solved e rrors is s till increasing. Total Bugs Re ma ining Total Closed ate

22 Incident/Fault Metrics Provides insight into the quality of the software as well as the developers ability to fix known bugs. Plots the number of problems. And the number of open problems against time. Open defects represent rework, and defect trends provide insight into product maturity. A large number of open defects may be a leading indicator of future budget and schedule slips.

23 Incident/Fault metrics efects should be tracked throughout the project, using a single defect tracking system. /W should not be released for ystem Testing if open priority one or two problem s are present. A leveling off of the cumulative number of total problem s may indicate that the s/w has reached maturity, or that testing has decreased. A low number of faults may indicate a good product or poor testing.

24 Metric Benefits Metrics by themselves do not control or improve anything. They give insights for planning, managing, and a basis for objective communications. Historical metric data helps estimations, predictions and planning Actual versus plan data helps determine progress and support decision making

25 Metric Benefits Analyzing trends helps identify and focus on problem areas. Project outcomes can be predicted by projecting current trends as straight lines. The variance to date can predict future performance. For example, if progress to date has been 20% below plan, it may be assumed that future progress will lag by 20%, unless some specific action is taken.

26 Mapping oftware Issues To Metrics Are we making progress towards schedule? Requirements/esign Progress metrics Program Progress Metrics ystem Progress Metrics Incremental Build Progress Metrics Are changes impacting the project? oftware ize metrics Requirements/esign volatility metric Is the staffing adequate? oftware Personnel Metric

27 Mapping /W Issues to Metrics Is a high quality product being developed? epth of Testing Metric Fault Profile Metric efect ensity Metric efect Containment Metrics Breadth of Testing Metric Rework Metric Requirements Traceability Metric esign/code Complexity Metric

28 Metrics and oftware Quality Quality Principle No.1 To produce quality s/w, developers must manage the quality of their work. Quality s/w must be managed while it is being built. If not, it is to late to do anything but fix defects. Quality Principle No.2 To manage s/w quality, developers must measure quality. Quality must be measured during development.

29 Metrics and oftware Quality Quality Principle No.3 The quality of a product is determined by the quality of the process used to develop it. To evaluate a process, you must be able to measure the process. The three things we can measure about a process stage are (1) time in stage, (2) defects removed in stage, and (3) size of product produced by that stage. (Planned/Actuals are used) Quality Principle No.4 While measurements and quality practices are not difficult, they represent a significant behavioral change for most organizations.

30 We Will Now Begin the Q&A ession If you do not have a question, Please be sure that your phone is muted