COCOMO II.2003 Calibration Status USC-CSE 1

Transcription

1 COCOMO II.2003 Calibration Status USC-CSE 1

2 Outline Introduction to COCOMO II COCOMO II.2003 Calibration Conclusion USC-CSE 2

3 COCOMO II Model Usage COCOMO II Estimation Endpoints I R R L C O L C A I O C P R R MBASE/RUP Inception Elaboration Construction Transition Waterfall Plans and Requirements Preliminary (Product) Design Detailed Design Code and Unit Test Integration and Test L C R S R R P D R C D R U T C S A R Most likely model to use: Early Design Model Post-Architecture Model USC-CSE 3

4 Early Design and Post-Arch Models Nominal-Schedule Estimated Effort (PM NS ): (excludes Required Development Schedule cost driver) PM NS = A (Size) where E = B A = 2.94 B = 0.91 Size (thousands of lines of code, KSLOC, or function points) EM: Effort Multipliers (6 for ED, 16 for PA) SF: Scale Factors (5 for both models) E 5 j= 1 n i= 1 SF j EM i USC-CSE 4

5 Size Estimated New KSLOC New KSLOC Unadjusted Function Points Adapted KSLOC C R A A M Equivalent KSLOC R E V L Size KSLOC: Thousands of Source Lines of Code CR: Conversion Ratios REVL: Requirements Evolution AAM: Adaptation Adjustment Modifiers USC-CSE 5

6 Early Design and Post-Arch Models Nominal-Schedule Estimated Duration (TDEV NS ): (excludes Required Development Schedule cost driver) C = 3.67 TDEV NS = C (PM where F = D PM NS = from nominal-schedule effort estimation D = 0.28 SF: Scale Factors (5 for both models) NS ) F 5 j= 1 SF j USC-CSE 6

7 Post-Arch Model Example Suppose we had a 100 KSLOC, average project Sum EM s = 1.0 (all nominal ratings) Sum SF s = 24 (mostly low ratings) PM NS =2.94(100) ( *24) = person months TDEV NS =3.67(586.6) ( *0.01*24) =29.7 months Average number of staff = PM NS /TDEV NS = people USC-CSE 7

8 COCOMO II.2000 Driver Values Driver VL L N H VH XH PR P R EC FLEX RESL TEAM P MAT RELY DATA CP LX R US E DOCU TIME S TOR P VOL ACAP P CAP PCON AEXP P EXP LTEX TOOL SITE SCED USC-CSE 8

9 Outline Introduction to COCOMO II COCOMO II.2003 Calibration Conclusion USC-CSE 9

10 COCOMO II.2003 Calibration As of this Annual Research Review there are 204 project data points 43 new project data points The 2000 calibration used 161 data points New project data sources Airborne systems Financial systems USC-CSE 10

11 Distribution of Size # of Projects >600 Size 2003 DB 2000 DB Size = Thousands of Source Lines of Code (KSLOC) USC-CSE 11

12 Distribution of Effort # of projects >6000 Effort 2003 DB 2000 DB Effort = Person Months (PM) USC-CSE 12

13 Distribution of Duration # of Projects >60 Duration 2000 DB 2003 DB Duration = Calendar Months USC-CSE 13

14 CII.2003 Bayesian Calibration Approach Model Building Steps 2 & 3 Initial Model Driver Values Expert - Based Delphi & Assessment A-Priori Coefficients, b 1 Variation, v1 Data Collection Data Repository (Symbolic) Numeric Data Data Coefficients, b 2 Variation, v 2 Accuracy Analysis Bayesian Coeffici ents, b 3 Variation, v USC-CSE 14

15 Bayesian Calibration Technique Combine two sources of information A-Priori Information (b 1 ) Sampling + Data (b 2 ) A-Posteriori = Model (b 3 ) Influence of two information sources on result Data Bayesian Experts More variation; Less influence Less variation; More influence b 2 =1.41 b 3 =1.48 b 1 = USC-CSE 15

16 A-Priori Information (b 1, v 1 ) Log Transformed Drivers b1 v1 ONES LN_SIZE PMAT_LN_SIZE PREC_LN_SIZE TEAM_LN_SIZE FLEX_LN_SIZE RESL_LN_SIZE LN_PCAP LN_RELY LN_CPLX LN_TIME LN_STOR LN_ACAP LN_PEXP LN_LTEX LN_DATA LN_RUSE LN_DOCU LN_PVOL LN_AEXP LN_PCON LN_TOOL LN_SITE USC-CSE LN_SCED

17 Linear Regression Results (b 2, v 2 ) Coefficient Estimates Label Estimate Std. Error t-value p-value ONES LN_SIZE PMAT_LN_SIZE PREC_LN_SIZE TEAM_LN_SIZE FLEX_LN_SIZE RESL_LN_SIZE LN_PCAP LN_RELY LN_CPLX LN_TIME LN_STOR LN_ACAP LN_PEXP LN_LTEX LN_DATA LN_RUSE LN_DOCU LN_PVOL LN_AEXP LN_PCON LN_TOOL LN_SITE LN_SCED USC-CSE 17

18 Bayesian Coefficients: b 3 Log Transformed Drivers B3 ONES LN_SIZE PMAT_LN_SIZE PREC_LN_SIZE TEAM_LN_SIZE FLEX_LN_SIZE RESL_LN_SIZE LN_PCAP LN_RELY LN_CPLX LN_TIME LN_STOR LN_ACAP LN_PEXP LN_LTEX LN_DATA LN_RUSE LN_DOCU LN_PVOL LN_AEXP LN_PCON LN_TOOL LN_SITE LN_SCED A = 3.13 B = USC-CSE 18

19 A-Posteriori Values (Not Finalized) Driver VL L N H VH XH PREC FLEX RESL TEAM PMAT RELY DATA CPLX RUSE DOCU TIME STOR PVOL ACAP PCAP PCON AEXP PEXP LTEX TOOL SITE SCED USC-CSE 19

20 COCOMO II.2003 Accuracy Results Effort Prediction Accuracy PRED(0.20) PRED(0.25) PRED(0.30) Before Stratification 37% 47% 56% After Stratification 54% 61% 70% Compare to COCOMO II.2000 Accuracy Results Effort Prediction Accuracy PRED(0.20) PRED(0.25) PRED(0.30) Before Stratification 57% 65% 71% After Stratification 66% 74% 76% USC-CSE 20

21 Outline Introduction to COCOMO II COCOMO II.2003 Calibration Conclusion USC-CSE 21

22 Conclusion -1 Need to investigate the large differences between the 2003 non-stratified and stratified results. Need to handle correlated parameters from the 204 data points TIME & STOR: ACAP & PCAP: USC-CSE 22

23 Conclusion -2 Conduct experiments on other calibration approaches Experimenting calibration for each application domain Remove older data from project data set Experiment with SCED calibration Need to resolve existence of outliers in the dataset DEMO: Outlier Analysis in ARC (analysis tool) USC-CSE 23

24 Backup Slides USC-CSE 24

25 COCOMO II Effort Effort is measured in Person Months (PM) = 152 labor hours per month Phases included: Preliminary Design, Detailed Design, Code and Unit, [CSCI] Integration and Test Activities for all phases included: Requirements analysis (4-6%) Product design (12-14%) Programming (45-48%) V&V (10-14%) Others USC-CSE 25

26 COCOMO II SLOC Definition COCOMO II data collection specified logical source line of code (SLOC) However, most project data are Non-commented, Non-blank SLOC COCOMO II uses a detailed Reuse model (next slide) to compute equivalent SLOC from Adapted SLOC (reused code that is modified) USC-CSE 26

27 COCOMO II Reuse Model ESLOC = ASLOC [AA + AAF( (SU)(UNFM))] AAF < ESLOC = ASLOC [AA + AAF + (SU)(UNFM)] AAF > Where: AAF = 0.4 (DM) (CM) (IM) SU = Software Understanding (zero when DM = 0 & CM = 0) UNFM = Programmer Unfamiliarity AA = Assessment and Assimilation ASLOC = Adapted SLOC ESLOC = Equivalent new SLOC USC-CSE 27

28 Data Age for 2000 vs Calibration # of Projects DB 2000 DB End Year of Development USC-CSE 28