Effective Software Sizing A Galorath Web Seminar 01 September PDF Free Download

Effective Software Sizing A Galorath Web Seminar 01 September 2004 1

Effective Software Sizing What are we estimating? What is software? Why do we care? What do we mean by software size? How do we measure it? What are the units? Lines of code Function Points Other measures Total vs. Effective Size Size Estimate Growth Size Estimate Uncertainty How do we estimate? Direct measurement Function points Requirements Other design artifacts Code Counts (pre-existing) Historical comparison methods Single analogy Database analysis Delphi Methods Structured comparisons (AccuScope) Developing a high-fidelity size estimate Galorath process 2

How Big is My Software Project? 50,000 what? 50,000 according to whom? Why not 30,000? 50,000 plus or minus what? 50,000 with or without requirements creep? 3

Why We re Concerned about Software Size: The Size Effort Relationship Development effort varies in direct proportion to the size of a program Knowing size allows estimators to determine effort (cost) Better size estimates = better cost estimates Software Size is the main driver of software development cost, effort and schedule -- use the best available estimate of size range! Development Schedule Months 40 35 30 25 20 15 10 5 0 1000 900 800 700 20000 40000 60000 600 80000 500 100000 Size (SLOC) 400 300 200 100 0 Development Effort Months 20000 40000 60000 80000 100000 Size (SLOC) 4

Galorath Observations from Reviewing Program Estimates Sizing Mistake Not enough time/effort spent on software sizing in general No clear definition of size, or lack of common understanding of definition Size as an Independent Variable technique changing the size estimate to get the desired cost or schedule estimate, without considering impact on functionality Historical actuals ignored as analogies due to differences in language and methodology Consequence Unbelievable estimates results don t match the program and are too optimistic or pessimistic Inconsistent estimates results don t pass the sanity check, unreliable output, blame the model Unrealistic size (and effort and schedule) estimates, unexecutable projects Lost opportunity to better forecast future from past 5

What is it you re trying to estimate? The most critical step in estimating anything is to first understand what it is. Peter Korda 6

What Do We Mean by Software Size? Size (work units) Desire Software Development Process Software Size (work units) Viewpoint: Essence (the what) Viewpoint: Implementation (the how) 7

Common Size Metrics Functional Function Points Requirements Design Objects Use Cases Technical Source Lines of Code Executable Lines of Code 9

Which Size Metric at Which Phase? Concept Requirements Design Completion Analogies Let you size items when size is still a fuzzy concept. Can translate directly into lines Functional Metrics Map functionallyoriented requirements documents directly into a sizable basis whether lines or functions These also may be appropriate at an earlier stage, depending on project and design circumstances Design Metrics As the software assumes a form, seek a metric that mirrors the developers perspective, such as OO metrics, lines, effective lines Lines or similar Grammatical Metrics Unarguably, a bedrock metric if obtainable, they can be counted precisely. At this point, function based size can be counted within 10% 10

Total vs. Effective Size Size (work units) Desire Software Development Process Software Size (work units) Viewpoint: Essence (the what) Total software size reflects the total functionality of the program Viewpoint: Implementation (the how) Effective Size is more directly related to the amount of effort required to develop the software, considering Redesign, reimplementation and/or retesting of pre-existing code Use of existing design or concept Use of code generators, GUI Builders, etc. 11

Total vs. Effective Size Many programs include a combination of new and pre-existing software New code needs to be designed, coded, tested, integrated Pre-existing code May require some degree of design change, recoding, retesting, reintegration May also require reverse engineering to discover how it works Misconceptions about reuse will impact the estimate 12

Total vs. Effective Size Size (Total SLOC) 120000 100000 80000 60000 40000 20000 0 Change in size mix may have no impact on functionality Total Size Requirements Design Coding Size (SLOC) Phase 120000 100000 80000 60000 40000 20000 0 Total Size Total Size Requirements Design Coding Phase No change in Total Size Pre- Existing Lines of Code New Lines of Code 13

Total vs. Effective Size 120000 100000 80000 Total vs. Effective Size Growth Effective Size grows 54% Size 60000 40000 20000 Effort and Schedule Growth 0 Requirements Design Coding Total Size Phase Effort Estimate +61% Schedule Estimate +27% No additional functionality 120 100 Effective 80 Size 60 40 20 0 Requirements Design Coding Phase Effort (Thousands of Hours) Schedule (Months) 14

Total Size Growth Source: http://www.sei.cmu.edu/sema/pdf/baumert.pdf In addition to effective size growth, the total size estimate tends to grow over time Some sharp growth periods due to addition of functionality, better understanding of requirements or design changes Steady growth from approximately development midpoint through acceptance testing can occur due to response to trouble reports Some size growth can also be attributed to institutional optimism Bottom Line: You must understand size, and factor code growth in your estimates, or your effort & schedule projections probably will be low 15

Expressing Uncertainty The actual, final size of the finished program is a point value. However Estimates of size and difficulty expressed as single point values don t tell the whole story: How confident am I in this value; i.e., what is the probability of not exceeding this value? How certain am I in this value; i.e., how wide is the probability distribution? Three-point estimates are better: LEAST: 1% Probability; I can t imagine the result being any smaller than this. LIKELY: Best Guess; If I were forced to pick one value, this would be it. MOST: 99% Probability; I can t imagine the result being any larger than this. 16

Measurement or Estimation? Some software size metrics can be directly measured at certain stages of development (but may be estimated at any stage) Function Points based on detailed requirements Objects or Use Cases based on design artifacts Requirements based on structured requirements documents Lines of Code (pre-existing software) Other metrics or stages require estimation Lines of Code (before they re written) Function Points (prior to detailed requirements) Object-oriented or use case metrics (prior to design) 18

Measured Metrics Count key elements of the available abstraction(s) Business Requirements documented during a Goals and Objectives sessions Actors and Use Cases that form a Unified Modeling Language (UML) Use Case Diagram Structured Text requirements Function Points Programs Modules Source Lines of Code (SLOC) if the code already exists 19

Estimated Metrics Virtually any metric which can be measured can also be estimated Fortunate, since effort and schedule estimates are frequently required before the requisite information is available to be measured Estimated metrics can include: Source Lines of Code (either total or effective) Function Points Requirements Actors / Use Cases 20

Estimation Techniques Expert Judgment What does the development team think? Delphi Techniques Structured process for collecting and distilling knowledge from a group of experts Analogies Compare to a program with known size and similar functionality Database Comparison Evaluate size range for a number of similar programs Structured Pairwise Comparison Analytical Hierarchy technique incorporated in SEER-AccuScope 21

Expert Judgment Advantages: Can be very accurate and effective, especially if the expert has a great deal of domain knowledge and experience If the expert is part of the development organization, the project team is more likely to buy into the size estimate and resulting effort/schedule estimate Specific historical data is not required Challenges: True experts may be hard to find New or different types of application may be difficult to estimate accurately Hard to tell the difference between a solid expert judgment and a SWAG 22

Delphi Methods Delphi techniques use a panel of knowledgeable persons, responding anonymously to a series of questions. Each panelist is provided with a summary of opinions before each series of questions. In each series, it is expected that the range of estimates will decrease and the median will approach the correct answer Advantages: Incorporates a range of expertise, not just one individual Results in a statistically derived range with a known confidence level Specific historical data is not required Challenges: Need to assemble a panel of experts Additional time, effort and coordination required to conduct the study 23

Analogy Methods Specific program for which actual size information is available is selected, and the program to be estimated is compared. Judgements are made as to whether the unknown program is bigger, smaller or about the same as the analogous program. Advantages: Based on specific historical data Takes advantage of familiarity with previous projects Challenges: Specific historical data must be available Normalization of metrics may be required Depends on a thorough understanding of the similarities between the programs 24

When is an analogy an analogy? Data points that are size analogous have similar: Platform Application Acquisition method Development method Development standard Language Functionality Implementation date More characteristics in common is better, although a data point should not necessarily be thrown out if it lacks a particular common characteristic Often the estimator will need to use judgment as to whether a data point is analogous or not. Be sure to choose analogies that you can explain why it is an analogy! 25

Maintaining Analogy Data Points Match proposed products, features, and characteristics to products, features, and characteristics of known size Mutually exclusive choices Combinations of choices Deviation from choice(s) Maintain products, features, and characteristics lists with associated actual sizes in a history repository SEER-AccuScope provides this capability 26

Database Analysis A sizing database can be an extremely powerful tool for using past data to predict the future. Data points which are similar to the unknown program are selected from the database, and a statistical analysis of the range and median of those points is conducted. Advantages: Incorporates more data to achieve better statistical significance Results in a statistically derived range with a known confidence level Outstanding sanity check on other methods Challenges: Significant effort may be required to develop and maintain a high-quality software size database Normalization of size metrics may be required Selection of appropriate data points for comparison is important May produce wide ranges 27

Database Analysis Size (normalized SLOC) 100000 90000 80000 70000 60000 50000 40000 30000 20000 10000 0 0 5 10 15 20 25 30 Data Point + Std Dev Median - Std Dev 28

Maintaining Historical Data SEER Repository Maintains historical data in SQL database format Integrates with SEER-SEM and SEER-AccuScope SEER-SEM and SEER-AccuScope handle normalization of different metrics 29

Sizing by Pairwise Comparison Compare each and every possible couple combination Analyze logical consistency of comparison results Compute relative magnitudes of each element Bind unknown elements to an absolute size 30

Which Technique to Use? Depends on: Available expertise Available data Do we have information about analogous programs? Do we have access to a size database? Available time Hunting for experts Hunting for data More fidelity = more time and effort to estimate size Requirement for fidelity Is this a ROM planning estimate, a budgetary estimate, a contractual estimate? 31

There is no single best method for estimating software size Any method for assessing software size for any given development project will have advantages and disadvantages Since this is estimation, no single correct answer is possible Always consider size estimates as a range which reflects the level of confidence A comprehensive, organized sizing study which considers multiple methods can be used to derive a high-fidelity size estimate 32

Size Study Methodology Evaluate All Sources of Software Size New Size Preexisting Size (rework) Generated Code COTS/GOTS Glue Code Integrated Code Total Size Estimates Least Likely Most Expert Judgement 12000 15500 17000 Relevant Range by Analogy 19850 24750 32540 Sizing Database 8000 32000 46000 Functional Analysis 19680 27540 35400 SEER-AccuScope 15450 22650 29850 Delphi Analysis 16788 19750 22713 Expert Judgment Functional Analysis Multiple Comparisons Counts for Pre-existing Sizing Databases Using Multiple Methods Analogies 34

Why do a sizing study? By analyzing the size several ways, your estimate will be less influenced by outliers When multiple methods are used, often the results will cluster around particular values, thus giving credibility to your size estimate Using multiple methods and a selection criteria is usually more reliable and defendable than using a single sizing method 35

Document, document, document! Document all data sources! Document which experts were used, where the analogies came from, source of line/function counts, organizational size growth data Be certain to document all assumptions relating to: Language Language conversion factors Functionality included in the size estimate Rework Size growth Ownership of functionality (ex. will the security be handled by the purchased OS?) Document as you go if you don t you ll forget your rationale before you get it written down! 36

Sizing Study Example Task: Develop a size estimate for a particular program Resources: Development team Other development teams in the same company Software Process Improvement Group with expertise in the organization Relatively firm design in Rationale Rose Decent history of performance on similar programs in the past 37

Select Methods and Establish Groundrules Based on available time, data and other resources, the following methods have been selected: Expert Judgment Functional Measurement Delphi Analysis Analogy with existing programs Database Analysis AHP Comparison (SEER-AccuScope) Groundrules: All size estimates using various methods will be normalized to the intended development language, C++ SLOC estimates will be normalized to logical lines of code (standard SEER-SEM definition) Normalized Lines of C++ Method Least Likely Most Expert Judgement Functional Measurement Delphi Analysis Relevant Range by Analogy Database Analysis SEER-AccuScope Composite 38

Expert Judgment Lead developer was consulted and expressed opinion that this project will be about 50,000 lines of C++. Feels that this estimate is correct within plus or minus 10,000 lines of code. Normalized Lines of C++ Method Least Likely Most Expert Judgement 40000 50000 60000 Functional Measurement Delphi Analysis Relevant Range by Analogy Database Analysis SEER-AccuScope Composite 39

Functional Measurement Use cases were counted based on available design documentation. Metrics from the table below were used to convert use cases into C++ SLOC. The metrics from the table are felt to be accurate within -10/+20% The team is not confident that all use cases have been properly defined Use Case Size Number of Use Cases Lines of C++ per Use Case C++ SLOC Small 8 500 4000 Medium 12 1200 14400 Large 4 2267 9068 Total 24 27468 Normalized Lines of C++ Method Least Likely Most Expert Judgement 40000 50000 60000 Functional Measurement 24721 27468 32962 Delphi Analysis Relevant Range by Analogy Database Analysis SEER-AccuScope Composite 40

Delphi Study A Delphi study was conducted using a panel of six experts from within the development team, other development teams and the process improvement group. Five iterations were conducted, with results as shown below. Normalized Lines of C++ Method Least Likely Most Expert Judgement 40000 50000 60000 Functional Measurement 24721 27468 32962 Delphi Analysis 32865 53270 75940 Relevant Range by Analogy Database Analysis SEER-AccuScope Composite 41

Analogy Several analogous programs were selected based on similar platform, application and development environment. Comparison resulted in below range estimate. Normalized Lines of C++ Method Least Likely Most Expert Judgement 40000 50000 60000 Functional Measurement 24721 27468 32962 Delphi Analysis 32865 53270 75940 Relevant Range by Analogy 45320 59450 78950 Database Analysis SEER-AccuScope Composite 42

Database Analysis Thirty data points were selected from the sizing data repository based on similarity in application and development environment. Standard language conversion factors were used to normalize to C++. A normal distribution was assumed in selecting least, likely and most estimates. Normalized Lines of C++ Method Least Likely Most Expert Judgement 40000 50000 60000 Functional Measurement 24721 27468 32962 Delphi Analysis 32865 53270 75940 Relevant Range by Analogy 45320 59450 78950 Database Analysis 23150 72350 115670 SEER-AccuScope Composite 43

SEER-AccuScope Analysis Pairwise comparison was conducted using SEER-AccuScope. Six historical programs from the repository were selected which were similar in scope of functionality. Representatives from the development teams assessed comparisons. Results are as shown below. Normalized Lines of C++ Method Least Likely Most Expert Judgement 40000 50000 60000 Functional Measurement 24721 27468 32962 Delphi Analysis 32865 53270 75940 Relevant Range by Analogy 45320 59450 78950 Database Analysis 23150 72350 115670 SEER-AccuScope 46546 54760 62974 Composite 44

140000 120000 100000 80000 60000 40000 20000 0 Analyze and Compare Methods 45 Expert Judgement Functional Measurement Delphi Analysis Relevant Range by Analogy Database Analysis SEER- AccuScope

140000 120000 100000 80000 60000 40000 20000 0 Assess Least, Likely and Most Most Likely Least 46 Expert Judgement Functional Measurement Delphi Analysis Relevant Range by Analogy Database Analysis SEER- AccuScope

Sanity check Does your size estimate make sense? Sanity check the size of each module Look for outliers Pay attention (but don t totally rely on) developer s gut Sanity check the sum of the size of each module against the expected system size Is the sum of the pieces relatively in line with the expected size of the entire system? Sanity check the relative sizes of each module of the system under estimation Do the smallest pieces have the smallest size estimate? 47

How Big is My Software Project? 54,015 what? C++ logical SLOC Why not 30,000? Because there s no reason 54,015 plus or minus what? 24,721 to 78,950 to believe so 54,015 according to whom? Well, lots of people 54,015 with or without requirements creep? Without assuming built as specified 48

Thanks For Your Interest Contact Information: www.galorath.com lnelson@galorath.com 310-414-3222 x635 thohmann@galorath.com 310-414-3222 x644 Copyright 2004, Galorath Incorporated. ALL RIGHTS RESERVED Reproduction, copy, or excerpt in whole or in part is strictly prohibited without written permission SEER is a trademark of Galorath Incorporated 100 N. Sepulveda Bl., Suite 1801, El Segundo, CA 90245 49