MTAT Software Economics. Session 6: Software Cost Estimation

Transcription

1 MTAT Software Economics Session 6: Software Cost Estimation Marlon Dumas marlon.dumas ät ut. ee

2 Outline Estimating Software Size Estimating Effort Estimating Duration 2

3 For Discussion It is hopeless to accurately estimate software costs. Most often than not, such estimates are wrong. So why should we bother? We have 6 months and 10 analysts/developers, so it will take 6 months and 60 person-months. Why bother about estimating the cost? 3

4 There are lies, dammed lies and statistics. What about a method to estimate software costs from a high-level architecture, that is: within 20% of the actual size 50% of the time within 30% of the actual size 66% of the time Is this good enough? Can we do better? 4

5 Cone of Uncertainty 5

6 Estimating Size From the early design, we can count FPs Can we estimate the size (LOC) from there? Yes, through backfiring Capers Jones s database: > 9000 projects with both function-points and actual LOC Cobol, C, Cobol, Fortran LOC/FP Pascal, Ada LOC/FP OO Languages 30 LOC/FP QSM Function Point table: 6

7 Estimating Effort Parkinson's Law If we have 600 personmonths, it will take 600 person-months Estimation by analogy This project is about 20% more complex than the previous one Expert judgement Wideband Delphi Planning Poker Based on Work-Breakdown Structures (WBS) See Six Forms of Software Cost Estimation by Capers Jones (Reading for week 6) 7

8 Estimating Effort (cont.) Parametric cost models and tools SLIM (Putnam model) COCOMO 81 and COCOMO II.2000 (Boehm et al.) Costar and Cost Xpert (based on COCOMO II) Construx Estimate KnowledgePlan No method is perfect consider combinations 8

9 Example: Expert-based estimation Wide-Band Delphi Ask each team member their estimate Apply personal experience, Look at completed projects, Extrapolate from modules known to date Collect and share in a meeting: discuss why/ how different people made their estimate Repeat When stable, Size = (H + 4 X Ave. + L)/6 See: 9

10 Productivity estimates Real-time embedded systems: LOC/PM Systems programming (e.g. games, graphics): LOC/PM Commercial applications: LOC/PM Web apps with simple business logic > 500 Heavy transactional business logic, high scalability requirements < 500 Similar estimates exist for FPs (cf. cost estimation tools) From I. Sommerville s Software Engineering 10

11 Estimation Models It took me one month to fully develop (end-toend) a small software application of 1000 LOC Can I develop an application of LOC in 10 months? I have four friends with similar experience as mine, can we develop an application of LOC in 2 months? Hints: Brook s law, Farr & Nanus study 11

12 Non-Linear Productivity There is overwhelming evidence that, except for simple projects, development effort goes up exponentially with size, so this is probably wrong: Effort = P x Size This might be closer to the mark: Effort = A x M x Size B where A is a constant derived from historical data, and M is dependent on each project (effort multiplier), and B is dependent on the complexity of the project 12

13 Diseconomy of Scale Nonlinear relationship when exponent > 1 (c) USC CSSE 13

14 COCOMO Stands for Constructive Cost Model Developed at USC (Barry Boehm et al.) based on a database of projects First version of COCOMO (now COCOMO 81) Most recent version COCOMO II.2000 Based on statistical model building (fitting actual data to equation) Can be calibrated based on company-specific historical data 14

15 Basic COCOMO 81 Complexity Formula Description Organic PM = 2.4 (KLOC) 1.05 Well-understood applications developed by small teams with strong prior experience in related systems. Semi- Detached PM = 3.0 (KLOC )1.12 More complex projects where team members may have limited experience of related systems. Embedded PM = 3.6 (KLOC) 1.20 Complex projects where the software is constrained by hardware limitations (embedded), needs to respond in real-time, or is critical. 15

16 Intermediate COCOMO 81 E = a KLOC b x EAF EAF is the product of 15 factors Check out Cocomo 81 calculator a Organic Semi-detached Embedded b 16

17 Estimating Time The Cocomo model is calibrated under the assumption of nominal time Nominal time in Cocomo 81 model: D = c E d c d Organic Semi-detached Embedded

18 Nominal versus Optimal Time 18

19 Warm-up Exercise See exercise Cocomo I on course web page Use the Cocomo 81 calculator (see link on Readings page) 19

20 COCOMO 81 limitations Over time, Cocomo 81s database became outdated by new tools, languages and practices Cocomo 81 was designed for the waterfall model, which was largely superseded by incremental, iterative methods Cocomo 81 had only three possible exponents could not explain for various factors affecting non-linearity of productivity Did not take into account different levels of information available throughout the lifecycle 20

21 Cocomo II.2000 Designed for an iterative development method (MBASE) More refined set of cost drivers (6-17) Multiple exponential scale drivers: PM = a x Size b x Π EM i (i = 1 to 6 or 17) where a = 2.94 b = x Σ SF j (j = 1 to 5) 21

22 COCOMO II models COCOMO II incorporates a range of sub-models that produce increasingly detailed software estimates. Sub-models in COCOMO II: Application composition model. Used when software is composed from existing parts. Early design model. Used when requirements are available but design has not yet started (6 cost drivers). Reuse model. Used to compute the effort of integrating reusable components. Post-architecture model. Used once the system architecture has been designed and more information about the system is available (17 cost drivers). From I. Sommerville s Software Engineering 22

23 Use of COCOMO II models From I. Sommerville s Software Engineering 23

24 Cost Factors Significant factors of development cost: scale drivers are sources of exponential effort variation cost drivers are sources of linear effort variation product, platform, personnel and project attributes effort multipliers associated with cost driver ratings Defined to be as objective as possible Each factor is rated between very low and very high per rating guidelines relevant effort multipliers adjust the cost up or down (c) USC CSSE 24

25 Scale Drivers Precedentedness (PREC) Degree to which system is new/past experience applies Development Flexibility (FLEX) Need to conform with specified requirements Architecture/Risk Resolution (RESL) Degree of design thoroughness and risk elimination Team Cohesion (TEAM) Need to synchronize stakeholders and minimize conflict Process Maturity (PMAT) SEI CMM process maturity rating (c) USC CSSE 25

26 Scale Factors Sum scale factors SF i across all of the factors to determine a scale exponent, B, using B = Σ SF i (c) USC CSSE 26

27 Precedentedness (PREC) and Development Flexibility (FLEX) (c) USC CSSE 27

28 Architecture / Risk Resolution (RESL) Use a subjective weighted average of: (c) USC CSSE 28

29 Team Cohesion (TEAM) Use a subjective weighted average of the characteristics to account for project turbulence and entropy due to difficulties in synchronizing the project's stakeholders. Stakeholders include users, customers, developers, maintainers, interfacers, and others (c) USC CSSE 29

30 Process Maturity (PMAT) Two methods based on the Software Engineering Institute's Capability Maturity Model (CMM) Method 1: Overall Maturity Level (CMM Level 1 through 5) Method 2: Key Process Areas (see next slide) (c) USC CSSE 30

31 Key Process Areas Decide the percentage of compliance for each of the KPAs as determined by a judgment-based averaging across the goals for all 18 Key Process Areas. (c) USC CSSE 31

32 Example of Scale Factors A company takes on a project in a new domain. The client has not defined the process to be used and has not allowed time for risk analysis. The company has a CMM level 2 rating. Precedenteness - new project 0.4 Development flexibility - no client involvement - Very high 0.1 Architecture/risk resolution - No risk analysis - V. Low 0.5 Team cohesion - new team nominal 0.3 Process maturity - some control nominal 0.3 Scale factor = From I. Sommerville s Software Engineering 32

33 Cost Drivers (Post-Architectural Model) Product Factors Reliability (RELY) Data (DATA) Complexity (CPLX) Reusability (RUSE) Documentation (DOCU) Platform Factors Time constraint (TIME) Storage constraint (STOR) Platform volatility (PVOL) (c) USC CSSE Personnel factors Analyst capability (ACAP) Program capability (PCAP) Applications experience (APEX) Platform experience (PLEX) Language and tool experience (LTEX) Personnel continuity (PCON) Project Factors Software tools (TOOL) Multisite development (SITE) Required schedule (SCED) 33

34 Example Cost Driver - Required Software Reliability (RELY) Measures the extent to which the software must perform its intended function over a period of time. Ask: what is the effect of a software failure? (c) USC CSSE 34

35 Example Effort Multiplier Values for RELY Very Low Low Nominal High Very High Slight Inconvenience Low, Easily Recoverable Losses Moderate, Easily Recoverable Losses High Financial Loss Risk to Human Life 0.75 E.g. a highly reliable system costs 39% more than a nominally reliable system 1.39/1.0=1.39) or a highly reliable system costs 85% more than a very low reliability system (1.39/.75=1.85) 35 (c) USC CSSE

36 COCOMO II Schedule Estimation D = c x E d x SCED%/100 where c = 3.67 d = x [b ] SCED% = percentage of required schedule compression 36

37 Cocomo II Exercise See separate handout Use COCOMO II Data Sheet, Model Definition Manual and online cost Cocomo II calculator (see list of Cocomo Resources under the course s Readings page) 37

38 Software Costing vs Pricing Caution: All of the above is about effort and schedule estimation From effort and schedule, one can estimate cost Estimate technical effort cost based on PM x monthy total salary cost Add licensing costs and overhead cost for administrative support, infrastructure, etc. But cost price Price depends on many other factors: Risk margin, requirements volatility, competitive advantage, market opportunity, need to win the bid 38

39 Final Word of Caution COCOMO and similar models are just MODELS COCOMO comes calibrated by a set of projects that might not reflect a particular project s context Should be combined with expert assessment for example, combine Cocomo with estimates based on the Work Breakdown Structures Cost estimation should be followed by continuous cost control (more on this next week) 39

40 Homework (5 points) In teams of 2-4 (same teams as homework 1) Take the Function-Point estimate of homework 1 Make a size estimate If you have actual size data, compare your estimate with the actual size and explain the difference Prepare an effort and schedule estimation using Cocomo II (post-architectural). Explain your choice of cost and scale drivers Be ready to answer this question: Is the estimate credible/realistic? Presentations on 26 Sept. 40