Quality Management Lessons of COQUALMO (COnstructive QUALity MOdel) A Software Defect Density Prediction Model

Transcription

1 Quality Management Lessons of COQUALMO (COnstructive QUALity MOdel) A Software Defect Density Prediction Model AWBrown and Sunita Chulani, Ph.D. {AWBrown, sdevnani}@csse.usc.edu} -Center for Systems & Software Engineering (-CSSE) 1

2 Outline Behavioral Underpinnings Hidden Factory Defect Types COQUALMO Framework The Defect Introduction Sub-Model Expert-Judgment Model + Some Initial Data Results The Defect Removal Sub-Model Expert-Judgment Model (Result of COQUALMO Workshop) COQUALMO Integrated with COCOMO II 2

3 Modeling Methodology 3

4 Operationally Wideband Delphi Final values (for each parameter) Max=(Hmax + 4*AVEmax + Lmax)/6 4

5 Fig 11., Pg 170 SwCEwCII A-posteiori Bayesian Update 5

6 Model Framework Defect Introduction pipes Code Defects Residual Software Defects Design Defects Requirements Defects Defect Removal pipes 6

7 Software Devel. Process(es) 7

8 Software Devel. Process(es) (cont.) 8

9 Software Devel. Process(es) (cont.) 9

10 Quality Issues What's a defect? Orthogonal Defect Categorization specialists forced to conclude the ONLY objective way to know there is/was a defect is because of [compound] change in the artifact or code! ONLY exception is Fagan's Inspections Defects are determined ONLY by the responsible Author of an artifact Typically start out as concerns in informal or agile reviews What's an "issue" Concerns that can NOT be fixed by the author of the artifact under review In developments with large numbers of people or cycles, issues are usually tracked to closure. 10

11 Empirical Data Shows CeBASE Empirical Data Shows Factor-of-100 Growth in Software Cost-to-Fix (Requirements through Test) Technique Selection Guidance: Peer Reviews vs. Test Other Empirical Data Shows Factor-of-1000 Growth in Software Cost-to-Fix (Requirements through Test) Factor's "through the roof" for Delivered Software Cost-to-Fix Requirements thru In-field fixes for embedded systems Unexpected costs Product rebates (in lieu of recall) Law suites; fines(?) 11

12 Classification of Defects - I Based on origin Requirements Defect Example: leaving out a Cancel option in an Input screen Design Defect Example: Error in algorithm Code Defect Example: Looping nine instead of ten times 12

13 Classification of Defects - II Based on severity (only non-trivial defects) Critical: Causes a system crash or unrecoverable data loss or jeopardizes personnel High: Causes impairment of critical system functions and no workaround solution exists Medium: Causes impairment of critical system functions although a workaround solution exists 13

14 Outline Model Framework The Defect Introduction Sub-Model Expert-Judgment Model + Some Initial Data Results The Defect Removal Sub-Model Expert-Judgment Model (Result of COQUALMO Workshop) COQUALMO Integrated with COCOMO II 14

15 The Defect Introduction (DI) Sub-Model Software Size estimate Software product, process, computer and personnel attributes (subset of COCOMO II factors) Defect Introduction Sub-Model Number of non-trivial requirements, design and code defects introduced 15

16 Modeling Effects of COCOMO II Cost Drivers on DI Defects Introduced/ KDSI or 10FPS Requirements Design Code Baseline Now, If ACAP is VH & RELY is VH How does baseline change? As compared to ACAP-VL & RELY-VL 16

17 An Example DI Rate Driver AEXP (Applications Experience) level VH Requirements Design Code Fewer Requirements defects due to less learning and fewer false starts Fewer Requirements understanding defects 0.81 Fewer Design defects due to less learning and fewer false starts Fewer Requirements traceability defects Fewer defects introduced in fixing requirements, preliminary design fixes 0.82 Fewer Coding defects due to less learning Fewer Coding defects due to requirements, design shortfalls Nominal Nominal level of defect introduction 1.0 VL More Requirements defects due to extensive learning and more false starts More Requirements understanding defects 1.24 More Design defects due to less learning and fewer false starts More Requirements traceability defects More defects introduced in fixing requirements, preliminary design fixes 1.22 More Coding defects due to extensive learning More Coding defects due to requirements, design shortfalls 1.13 Behavioural analysis; relative significance Expert-judgment Range Delphi Round 1 Median Expert-judgment Range Delphi Round 2 Median

18 A-Priori Expert-Judgment Based Code DI Ranges FLEX RUSE STOR TIME DATA ACAP AEXP PEXP DOCU SITE TEAM SCED LTEX PVOL PREC TOOL PCON PCAP CPLX RESL RELY PMAT

19 DI Model Equations Estimated Number of Defects Introduced = 3 21 B j j= 1 i= 1 For each artifact j, Quality Adjustment Factor (QAF) QAF A (Size) (DI driver) j identifies the 3 artifact types (requirements, design and coding). A is the multiplicative calibration constant. B is initially set to 1 (DI-driver) ij is the Defect Introduction driver for the j th artifact and the i th factor. j = 22 DIR - driver ij ij i = 1 Estimated Number of Defects Introduced = 3 B j j Copyright j= 1 -CSSE A (Size) QAF j 19

20 Initial Data Analysis on the DI Model Type of Artifact 1970 抯 Baseline DIRs Quality Adjustment Factor Predicted DIR Actual DIR Calibrated Constant (A) 1990 抯 Baseline DIRs Requirements Design Code DIR = Defect Introduction Rate 20

21 Outline Model Framework The Defect Introduction Sub-Model Expert-Judgment Model + Some Initial Data Results The Defect Removal Sub-Model Expert-Judgment Model (Result of COQUALMO Workshop) COQUALMO Integrated with COCOMO II 21

22 Integrated COQUALMO Software Size estimate COCOMO II COQUALMO Software development effort, cost and schedule estimate Software platform, project, product and personnel attributes Defect Introduction Model Number of residual defects Defect density per unit of size Defect removal profile levels Defect Removal Model 22