COCOMO III. Brad Clark, PhD USC Center for Systems and Software Engineering 2017 Annual Research Review April 4, 2017

Transcription

1 COCOMO III Brad Clark, PhD USC 2017 Annual Research Review April 4, 2017

2 The COCOMO III Project COCOMO (COnstructure COst MOdel) is the most widely used, free, open source software cost estimation model in the world. Registered Trademark for intellectual property protection COCOMO 81 and COCOMO II models are open and free for anyone to use Models have been commercialized It has been 17 years since the COCOMO II model has been updated and calibrated to new Software Engineering practices. Apr 5, 2017 Copyright USC-CSSE 2

3 COCOMO III Project Purpose Broaden audiences of COCOMO and address scope of modern projects: mobile devices, web/internet, big data, cloud-targeted, and multi-tenant software Modernize model size inputs Consider the impact of modern development processes (e.g. Agile) Improve the accuracy and realism of estimates Improve driver definitions New and updated software cost drivers and adjust their ratings as needed Quality estimation capability Point and range estimates based on risk Improve value of COCOMO in decision-making Apr 5, 2017 Copyright USC-CSSE 3

4 COCOMO III Project Scope COCOMO III will product estimates for: Effort, Schedule, Cost, Quality Level (Defects) COCOMO III can be applied at various moments in a project s lifecycle: Early Estimation, Initial Project Estimation, Project Re-estimation COCOMO III s functional vision Single and Multiple component estimate Analysis of alternatives Analysis with Size-Effort-Schedule as independent variables Support for different lifecycle processes Lifecycle cost estimation Legacy system transformation Apr 5, 2017 Copyright USC-CSSE 4

5 Model Scope Model Development Process Compatibility Application Domain Model Breath and Depth Model WBS Workload Sizing Cost estimating relationships Parametric models Simple relationships Apr 5, 2017 Copyright USC-CSSE 5

6 Model ICSM Common Cases Since 2000, a plethora of development processes have arisen Non-Developmental Item such as COTS Agile Development Brownfield Development One size fits all estimation model is no longer feasible 11 common development cases are presented next with indications on where COCOMO III is suitable Source: Boehm, B., Lane, J., Koolmanojwong, S. and Turner, R. The Incremental Commitment Spiral Model, Addison-Wesley 2014, Chapter 11. Apr 5, 2017 Copyright USC-CSSE 6

7 Model Match to ICSM Common Cases-1 Case 1: Use Non-Developmental Item (NDI) Example: Small accounting system Size variable, complexity low Typical Change Rate/Month: Negligible Criticality: n/a NDI Support: Complete Personnel Capability: NDI-experienced (medium) Activities: Acquire NDI, Use NDI Time/Build: n/a Time/Increment: Vendor-driven Case 2: Agile Example: E-services Size, Complexity: Low Typical Change Rate/Month: 1-30% Criticality: Low to medium NDI Support: Good, in place Personnel Capability: Agile-ready, medium-high experience Activities: Skip Valuation and Architecting phases; Scrum plus agile methods of choice Time/Build: <= 1 day Time/Increment: 2-6 weeks Apr 5, 2017 Copyright USC-CSSE 7

8 Model Match to ICSM Common Cases -2 Case 3: Architected Agile Example: Business data processing Size, Complexity: Medium Typical Change Rate/Month: 1-10 % Criticality: Medium to high NDI Support: Good, most in place Organizational Personnel Capability: Agile-ready, medium to high experience Activities: Combine Valuation, Architecting phases. Complete NDI preparation. Architecturebased Scrum of Scrums Time/Build: 2-4 weeks Time/Increment: 2-6 months Case 4: Formal Methods Example: Security kernel; Safety-critical LSI chip Size, Complexity: Low Typical Change Rate/Month: 0.3% Criticality: Extra high NDI Support: None Organizational Personnel Capability: Strong formal methods experience Activities: Precise formal specification. Formally-based programming language; formal verification Time/Build: 1-5 days Time/Increment: 1-4 weeks Apr 5, 2017 Copyright USC-CSSE 8

9 Model Match to ICSM Common Cases -3 Case 5: Hardware with Embedded Software Component* Example: Multi-sensor control device Size, Complexity: Low Typical Change Rate/Month: % Criticality: Medium to very high NDI Support: Good, in place Organizational Personnel Capability: Experienced, medium-high * Means this process is suitable for COCOMO III Activities: Concurrent hardware/software engineering. CDR-level review. IOC development, LRIP, FRP. Concurrent version N+1 engineering Time/Build: Software 1-5 days Time/Increment: Market-driven Case 6: Indivisible IOC* Example: Complete vehicle platform Size, Complexity: Medium to high Typical Change Rate/Month: 0.3 1% Criticality: High to very high NDI Support: Some in place Organizational Personnel Capability: Experienced, medium to high Activities: Determine minimum-ioc likely, conservative cost. Add deferrable software features as risk reserve. Drop deferrable features to meet conservative cost. Strong award fee for features not dropped. Time/Build: Software: 2-6 weeks Time/Increment: Platform: 6-18 months Apr 5, 2017 Copyright USC-CSSE 9

10 Model Match to ICSM Common Cases -4 Case 7: NDI-Intensive Example: Supply chain management Size, Complexity: Medium to high Typical Change Rate/Month: 0.3 3% Criticality: Medium to very high NDI Support: NDI-driven architecture Organizational Personnel Capability: NDI-experienced, medium to high Activities: Thorough NDI-suite life cycle cost-benefit analysis, selection, concurrent requirements/architecture definition. Pro-active NDI evolution influencing, NDI upgrade synchronization Time/Build: Software: 1-4 weeks Time/Increment: Systems: 6-18 months Apr 5, 2017 Copyright USC-CSSE 10

11 Model Match to ICSM Common Cases -5 Case 8: Hybrid Agile/Plan-Driven System* Example: C4ISR system Size, Complexity: Medium to very high Typical Change Rate/Month: Mixed parts; 1-10% Criticality: Mixed parts; Medium to very high NDI Support: Mixed parts Organizational Personnel Capability: Mixed parts * Means this process is suitable for COCOMO III Activities: Full ICSM, encapsulated agile in high change, low-medium criticality parts (Often HMI, external interfaces). Full ICSM, three-team incremental development, concurrent V&V, next-increment re-baselining Time/Build: 1-2 months Time/Increment: 9-18 month Case 9: Multi-Owner System of Systems* Example: Net-centric military operations Size, Complexity: Very high Typical Change Rate/Month: Mixed parts; 1-10 % Criticality: Very high NDI Support: Many NDIs, some in place Organizational Personnel Capability: Related experience, medium to high Activities: Full ICSM; extensive multi-owner team building, negotiation. Full ICSM; large ongoing system/software engineering effort Time/Build: 2-4 months Time/Increment: months Apr 5, 2017 Copyright USC-CSSE 11

12 Model Match to ICSM Common Cases -6 Case 10: Family of Systems Example: Medical device product line Size, Complexity: Medium to very high Typical Change Rate/Month: 1-3% Criticality: Medium to very high NDI Support: Some in place Organizational Personnel Capability: Related experience, medium to high Activities: Skip Valuation and Architecting phases. Scrum plus agile methods of choice Time/Build: 1-2 months Time/Increment: 9-18 months Case 11: Brownfield Example: Incremental legacy phaseout Size, Complexity: High to very high Typical Change Rate/Month: 0.3-3% Criticality: Medium-high NDI Support: NDI as legacy replacement Organizational Personnel Capability: Legacy re-engineering Activities (Incremental Definition): Re-engineer/refactor legacy into services. Incremental legacy phaseout Time/Build: 2-6 weeks/refactor Time/Increment: 2-6 months Apr 5, 2017 Copyright USC-CSSE 12

13 Best Fits of Estimation-Types to ICSM Common Cases Pure Agile: Planning Poker, Agile COCOMO III Architected Agile COSYSMO for architecting; Planning Poker, CAIV-SAIV for sprints, releases; IDPD for large systems Formal Methods: $/SLOC by Evaluated Assurance Level NDI/Services-Intensive: Oracle, SAP, other ERP RICE Objects: (R)eports, (I)nterfaces, (C)onversions, (E)nhancements COCOTS, Value-Added Function Points, Agile for portions Hybrid Agile/Plan-Driven Expert Delphi, COCOMO III, Agile for portions; IDPD Systems of Systems COSYSMO for Integrator; Hybrid Agile/Plan-Driven for component systems (COCOMO III) Family of Systems: COPLIMO Brownfield: COSYSMO for refactoring; above for rebuilding 8/23/2016 Copyright USC-CSSE 13

14 COCOMO III Suite of Models Concept AGILE COCOMO III COPROMO Model Extensions to address other ICSM Common Cases COCOMO III COPSEMO COPLIMO COINCOMO Legend: Model has been calibrated with historical project data and expert (Delphi) data Model is derived from COCOMO III Apr 5, 2017 Copyright USC-CSSE 14

15 New Feature: Application Domain Types Real-Time Sensor Control and Signal Processing Vehicle Control Vehicle Payload Real Time Embedded Mission Processing Engineering Systems Software Automation and Process Control Simulation & Modeling Automated Information Systems Mission Planning Training Test Data Processing Selecting an Application Domain pre-sets model drivers 15 Apr 5, 2017 Copyright USC-CSSE

16 Model Breadth There are a number of different activities in software development: Requirements analysis Architecting Detailed Design Assembling or Coding Integration Testing System Testing Acceptance Testing Deployment Training COCOMO III will cover a subset of these activities Apr 5, 2017 Copyright USC-CSSE 16

17 Model Depth Development activities include/exclude different types of work: Management Requirements analysis Product design Programming Test and evaluation Configuration Management / Quality Assurance Documentation COCOMO III covers a number of work types (next slide) The work covered is an indicator for whether the model is suitable for estimating a development process (re: 11 cases discussed earlier) Apr 5, 2017 Copyright USC-CSSE 17

18 COCOMO III Depth Subsystem Work Breakdown Structure Management Engineering Programming Test & Evaluation Data Cost, Schedule, Performance Management Contract Management Subcontract Management Customer Interface Branch Office Management Management Reviews & Audits Software Requirements Product Design Configuration Management End Item Acceptance Quality Assurance Detailed Design Code and Unit Test Integration Product Test Plans Procedures Test Reports Acceptance Test Plans Procedures Test Reports Test Support Test beds Test tools Test data Manuals Apr 5, 2017 Copyright USC-CSSE 18

19 Workload Sizing The amount of development work to be done is expressed as either a functional or product size Software Requirements Feature Points Function Point Use Case Points SNAP Points Story Points (Agile Fast Function Points Development) COSMIC Points Source Lines of Code Automated Function Points The desire is for COCOMO III to use different size types organically as a size input Want to move away from converting one size type to another, e.g. Function Points to Source Lines of Code Apr 5, 2017 Copyright USC-CSSE 19

20 Reused Functionality Currently COCOMO III uses the reuse model from COCOMO II Model is based on source lines of code AAF: Adaption Adjustment Factor DM: percent design modified CM: percent of code and unit test modified IM: percent of integration and test modified AAM: Adaption Adjustment Multiplier SU: Software Understanding UNFM: Programmer Unfamiliarity AA: Assessment and Assimilation AAF = ( 0.4 DM) + ( 0.3 CM) + ( 0.3 IM) #[AA + AAF(1+ (0.02 SU UNFM))], for AAF 50 % AAM = 100 $ %[AA + AAF + (SU UNFM)], for AAF > 50 &% 100 Equivalent KSLOC = Adapted KSLOC AAM Apr 5, 2017 Copyright USC-CSSE 20

21 Software product size estimate Software product, platform, personnel & project attributes Software reuse, maintenance, and increment parameters Defect removal profile levels COCOMO III Model Software development and maintenance estimates for: Effort Cost & Schedule distributed by: o Phase o Activity o Increment Quality Local calibration to organization s data COCOMO is an open and free model Apr 5, 2017 Copyright USC-CSSE 21

22 COCOMO III Model Concept Software product size estimate Software product, platform, personal & project attributes Labor Rates Defect removal profile levels Schedule Model Effort Model Defect Introduction Model Defect Removal Model Schedule (Months) Staffing Levels Effort (Person Months) Costs ($$) Number of est. non-trivial defects for Requirements, Design, & Code Number of est. residual defects and the residual defect density Apr 5, 2017 Copyright USC-CSSE 22

23 COCOMO III Effort & Schedule Estimation Model Effort (PM) = A * Size E * Product(19 Cost Drivers) E = B + Sum(5 Cost Drivers) Schedule (M) = C * PM F * SCED%/100 F = D + 0.2(E-B) Where: A, B, C, D are constants determined by calibration E represents (dis)economies of scale and project-wide scale factors 23 8/23/2016 Copyright USC-CSSE

24 COCOMO III Defect Introduction and Removal Model Defect Introduction (DI) = A * Size E * Product(DI Drivers) E = Initially set to 1.0 Residual Defects = C * DI * Product(1 DRF) DRF: Defect Removal Fraction from 3 profiles: 1. Automated Analysis 2. People Reviews 3. Execution Testing 8/23/2016 Copyright USC-CSSE 24

25 Defect Estimation Scope Estimates only Nontrivial defects Critical: causes a system to crash or unrecoverable data loss or jeopardizes personnel High: causes impairment of critical system function and no workaround solution exists Medium: causes impairment of critical system function, through a workaround solution does exist Defect estimates are for only 3 artifacts: Requirements Design Code Apr 5, 2017 Copyright USC-CSSE 25

26 COCOMO III Cost Drivers -1 Product Attributes Impact of Software Failure (FAIL) (formerly RELY) Product Complexity (CPLX) Developed for Reusability (RUSE) Required Software Security (SECU) - New Dropped: Documentation Match to Lifecycle Needs Database Size Platform Attributes Platform Constraints (PLAT) New Platform Volatility (PVOL) Apr 5, 2017 Copyright USC-CSSE 26

27 COCOMO III Cost Drivers -2 Personnel Attributes Analyst Capability (ACAP) Programmer Capability (PCAP) Personnel Continuity (PCON) Applications Experience (APEX) Language and Tool Experience (LTEX) Platform Experience (PLEX) Apr 5, 2017 Copyright USC-CSSE 27

28 COCOMO III Cost Drivers -3 Project Attributes Precedentedness (PREC) Development Flexibility (FLEX) Opportunity and Risk Resolution (RESL) Stakeholder Team Cohesion (TEAM) Process Capability & Usage (PCUS) (formerly PMAT) Use of Software Tools (TOOL) Multisite Development (SITE) Defect Removal Profile Automated Analysis People Reviews Execution Testing and Tools Apr 5, 2017 Copyright USC-CSSE 28

29 Cost Driver Values -1 Product Attributes Driver VL L N H VH XH PR FAIL CPLX RUSE SECU Platform Attributes Driver VL L N H VH XH PR PLAT PVOL Apr 5, 2017 Copyright USC-CSSE 29

30 Cost Driver Values -2 Personnel Attributes Driver VL L N H VH XH PR ACAP PCAP PCON APEX LTEX PLEX Project Attributes Driver VL L N H VH XH PR PREC FLEX RESL TEAM PCUS TOOL SITE Apr 5, 2017 Copyright USC-CSSE 30

31 Defect Removal Drivers Defect Removal Profile Artifact VL L N H VH XH Automated Analysis People Reviews Execution Testing Requirements Design Code Requirements Design Code Requirements Design Code Apr 5, 2017 Copyright USC-CSSE 31

32 The New Nominal Cost Driver definition refinement The New Nominal Study of productivity trends over the past 40 years reveals a shift in nominal ratings Following slides show the shift in ratings for selected cost drivers, i.e., the new nominal Apr 5, 2017 Copyright USC-CSSE 32

33 Impact of Productivity Trends Kendall's Rank Correlation Coefficients between the Completion Year and COCOMO II Cost Drivers (sorted by degrees of correlation) Cost driver Kendall s τ p-value TOOL Use of Software Tools E-16 PMAT Process Maturity (PCUS) E-13 STOR Main Storage Constraint E-11 TIME Execution Time Constraint E-10 PLEX Platform Experience E-05 PVOL Platform Volatility E-05 APEX Applications Experience E-05 LTEX Language and Tool Experience E-04 DATA Database Size E-03 RELY Required Software Reliability E-02 CPLX Product Complexity E-02 PREC Precedentedness of Application E-02 ACAP Analyst Capability E-02 Apr 5, 2017 Copyright USC-CSSE 33

34 Use of Software Tools Apr 5, 2017 Copyright USC-CSSE 34

35 Process Maturity (Now PCUS) Apr 5, 2017 Copyright USC-CSSE 35

36 Execution Time Constraint-TIME Apr 5, 2017 Copyright USC-CSSE 36

37 Main Storage Constraint-STOR Apr 5, 2017 Copyright USC-CSSE 37

38 Platform Experience-PLEX Apr 5, 2017 Copyright USC-CSSE 38

39 Applications Experience-APEX Apr 5, 2017 Copyright USC-CSSE 39

40 Language and Tool Experience-LTEX Apr 5, 2017 Copyright USC-CSSE 40

41 Next Steps Refine the definition for Required Software Security Shift cost driver rating definitions to accommodate changes in nominal Create a Rosetta Stone for converting COCOMO II data to the COCOMO III format Test the model on past data Setup data collection Apr 5, 2017 Copyright USC-CSSE 41

42 Required Software Security (SECU)* Is this correlated to Process Maturity? Should it be an extension to FAIL Should it address the threat? Malicious hackers Foreign governments Should it address exposure On the network Under guard, separated from outside environment Where are the descriptors: Confidentiality Integrity Availability Apr 5, 2017 Copyright USC-CSSE 42

43 Invitation to Participate CSSE invites you to collaborate on model development Review model formulation Submit data for model calibration Actual Size Effort Schedule Defects Model Parameters Review of COCOMO III model Want to Participate? Make suggestions and be a model reviewer! If you contribute data for model calibration, you will receive: An advanced copy of the new model Comparison of your data with respect to other data points used to calibrate the model Please talk with me afterwards if you are interested Apr 5, 2017 Copyright USC-CSSE 43

44 For more information, contact: Brad Clark Questions? 44 Apr 5, 2017 Copyright USC-CSSE