Understanding Cost & Affordability Using SEER as a Common Benchmark, Driven By Facts

Size: px

Start display at page:

Download "Understanding Cost & Affordability Using SEER as a Common Benchmark, Driven By Facts"

Berniece McBride
6 years ago
Views:

2011 Copyright Galorath Incorporated 2 An Estimate Defined An estimate is the most knowledgeable statement you can make at a particular point

2 2011 Copyright Galorath Incorporated 2 An Estimate Defined An estimate is the most knowledgeable statement you can make at a particular point in time regarding: Effort / Cost Schedule Staffing Risk Reliability Estimates more precise with progress A WELL FORMED ESTIMATE IS A DISTRIBUTION

3 Viable Estimation Is Critical Estimating is critical for all kinds of systems Yet many treat is as a second rate process Everyone estimates. Just most get it wrong and don t have a process Having a repeatable estimation process is critical to both estimating AND to successful projects Estimation and measurement go hand in hand

4 Repeatable Process Necessary 10 Step System Estimation Process Establish Estimate Scope 10. Track Project Throughout Development 2. Establish Technical Baseline, Ground Rules, Assumptions 9. Document Estimates and Lessons Learned 8. Generate a Project Plan 4. Refine Technical Baseline Into Estimable Components 6. Validate Business Case Costs & Benefits (go / no go) 4. Collect data / estimation inputs 6. Quantify Risks and Risk Analysis 4 5. Estimate Baseline Cost, Schedule, Affordability Value

5 Software & IT Systems Are About Business Value Software economics should provide methods for analyzing the choices software projects must make. Leon Levy Business economics should provide methods for analyzing choices in which projects to fund Total Ownership Cost Value

6 Affordability 2011 Copyright Galorath Incorporated

USD (AT&L) Efficiency 5 Initiatives Carter Memo https://acc.dau.mil/bbp 1. Target Affordability 2. Incentivize Productivity & Innovation in Industry 3. Promote Real Competition 4.

7 USD (AT&L) Efficiency 5 Initiatives Carter Memo 1. Target Affordability 2. Incentivize Productivity & Innovation in Industry 3. Promote Real Competition 4. Improve Tradecraft in Services Acquisition 5. Reduce Non-Productive Processes & Bureaucracy Director, Acquisition Resources & Analysis will establish tracking system to monitor progress and compliance 2011 Copyright Galorath Incorporated 7

8 Impact On The Cost Community Not just the Carter Memo Weapon Systems Acquisition Reform (WSARA) DoD Budget Reductions Economic Changes/Uncertainty Change is being driven by continued and sometimes surprising growth in cost and schedule. We know we must: Rise above business as usual Ensure estimates and estimating process are realistic, reliable, and meaningful for today's decision makers

9 Why Do Cost Estimates: OSD Prospective (Source Steve Miller) Why do we do cost estimates? COMPARE: To realistically compare options for decision making BASELINE: To establish baselines to measure execution performance and provide accountability MANAGEMENT: To enable the allocation of current and future resources in such a way that it enables efficient execution of the program while providing the minimal amount of turbulence to the greater portfolio so other programs can also execute efficiently Arguably reason three is the most important and gets directly to the concept of Affordability 2011 Copyright Galorath Incorporated 9

10 Affordability: Making Cost A Requirement Speed Accuracy Turn Rate Roll Rate Acceleration Stealth EW Fuel Consumption Drag Traditional Milspecs Milstds (Adapted From: Col Mike Kaye) Level of Requirements Definition Allocated Requirements X X X X Measureable TOC Analysis Supports Trade Space Potential Affordability Capabilities Based on Affordable Requirements Performance Spec Kill Lethality Speed EW Stealth Survive Challenge: Defining, Prioritizing, Vetting Requirements 10

11 11 Should Cost Vs Will Cost: Trade Study Flow Requirements And Features Analysis Preliminary Design Process Human Factors Security Reliability Availability Survivability Other Factors Alternative 1 Alternative 2 Alternative 3 Alternative 4 Supportability Testability Producibility Reuse Transportability Iterate SEER Assessment Performance Schedule Risk Assessment Life Cycle Cost Yes 1 No Selection Process OK? Optimized? Cost Performance Schedule Risk Bottoms Up Estimation as Required

12 System Alternatives Analysed Relative Time In Months SEER Saves Analysis Effort & Increase Alternatives With Galorath / SEER Save up 17+ Times Planning Process Effort Of Manual Methods 1 Manual With Galorath / SEER With Galorath / SEER Analyse 5 To 10 Major Solutions To Optimize Cost, Schedule, Reliability Manual Methods with Galorath / SEER SEER Provides a 17 to 1, to 20 to 1 Time Savings In Planning Process Organizations Can Analyze 5 to 10 Major Solutions SEER identifies Candidates For Cost Avoidance / Reduction Traditional industry practices often mean a part or assembly in early evaluation stages has to be sent out for several bids - a two to three week process. With SEER, I can work with an engineer to specify a part s cost in 10 minutes Michael Walter, Advanced Manufacturing Group, Case Corp. 12

13 Common Language 2011 Copyright Galorath Incorporated

14 System Description (Parametrics Can Estimate More, Earlier) Adapted from CEBOK If you can t tell me what it is, I can t tell you what it costs. -Mike Jeffers If you can tell me the range of what it might be I can tell you the range of cost, schedule & probability -Dan Galorath 14

15 Parametric Approach Parametric estimating: Employs cost estimating relationships or equations Based on technical or physical characteristics of the product and the manufacturing environment Allows estimation based on measurement of key parameters What do you measure? Bottom-up Approach (Detailed Work Measurement) Measure everything directly and exactly Little or no analysis of alternatives Predictive value? Parametrics Measure or estimate key parameter values and ranges Understand (mathematically) their effect on time, effort Once relationships are known, parameter values can be changed and the effects evaluated. 15

16 DCMC Comments on SEER Parametrics As A Common Language Use SEER For Trades & Detailed Analysis Driving Out 1.5 pounds (Sculpt With Spars) Quadrupled cost Answers Questions Such As How many do you have to produce before a process makes sense? DCMA Comment: This Is What We Are Looking For. It simplifies The negotiation. The Parameters Define The Part. Easier to negotiate change SEER breaks down to detail so both sides think it is fair Parameters (Common Language) define the part 16

17 Manual Estimates: Human Reasons For Error (Metrics Can Help) Manual Task estimates yield SIGNIFICANT error Desire for credibility motivates overestimate behavior (80% probability?) So must spend all the time to be reliable Better approach: force 50% probability & have buffer for overruns Technical pride sometimes causes underestimates Lack of common language for uncertainty causes overruns and credibility issues 17

18 How SEER Supports Affordability Analysis Should Cost / Will Cost

19 Estimates Provide Insight & Decision Making Support Effort / Cost Estimating Process Schedule Risk TRADEOFFS Reliability Defects

20 Software Estimation Basic Model & Associated Metrics Effective Technology Cte Effort K Effective complexity D People Staffa & Constraints Technology Process Defects Count (Qi Qr) Development Legacy, Maintenance Specifics & Constraints and/or Block Changes As Redevelopment Maintenance/ Block Change Development Process Size St Stakeholder ReuseDIT Requirement Size Se s (work units) Start Software Development Process Calendar Time On-going Iterations of Effort (ACWP or Spent) Progress (BCWP or Earned Value) Defects (Qi Qr ) Growth (Sg) Delivered Software Finish 20 Size (Effective Se & Total St)

21 Estimated Function Points Can Provide Excellent Results (Source Pam Morris) Best depends on estimate scope Must be clearly identified 2011 Copyright Galorath Incorporated 21

Status Indicator shows status and trends from the previous snapshot

22 Other Key Software Specific Metrics Can Help Track Project Performance Track defect discovery and removal rates against expected rates Heath and Status Indicator shows status and trends from the previous snapshot Thresholds are user definable Increased defect reporting rate shows a worsening trend

23 Data Doesn t Have To Be Perfect To Be Useful: But Is Has To Be Viable 80 Calories per serving 2.5 Servings per can 4 Ounces, Condensed, 8 Ounces With Water

24 Normalization: Adjustments for homogeneity Data comes in all sizes and shapes Development of models and calibration of models require that format and scale of data used be internally consistent Inches versus centimeters Full dollars versus K dollars versus Euros Labor phases and activities Handling of overhead and the list goes on A Common language is supported by normalized data 26

25 Lesson Learned Estimate Total Ownership Costs IT Example Software Development Software Maintenance IT Infrastructure IT Services Total Cost of Ownership should cover all costs 2009 Copyright Galorath Incorporated 27

26 Lesson Learned: Estimate Must Quantify Risk & Uncertainty Firm Fixed Price? Feel lucky? What is likely to happen Understand the risk before you commit! 28 28

27 Vendor Evaluation: Scrutinize Vendors With SEER 29 In-House Project Vendor Proposal Verify productivity claims and measure them against industry averages. Using industry average productivity levels, the vendor has a 20% probability of achieving their proposed 11 month schedule.

28 Staffing Strategy Evaluation You may want to apply dedicated teams to projects, or you may want to shift staff from project to project, as required... Optimal Staffing Profile: 200 effort months 18 calendar months...seer-sem s staffing analysis capability lets you model each alternative. Fixed Staffing Profile: 213 effort months 20 calendar months 30

29 SEER-SEM Provides How Much Can Be Built Within a Cost or Schedule 31

30 And How Much Can Be Built Within A Forced Schedule and/or Cost 32

31 SEER Evaluates Total Ownership Costs From Both Software Development & Maintenance 33

32 Validate Estimates Against TRUSTED Historical Data

33 Providing Your Organization Access to Data You Trust SEER Historical Database SEER Enterprise Database

34 Estimates Are Validated Against Data (Yours and/or Other) Galorath Benchmark Trendline SEER-SEM Estimate Your Data Regression Trendline Your History Data 36

35 Common Language With Uncertainty and Consequences 37

Historical software project data SEER-SEM Data Driven

36 SEER-SEM s ProjectMiner Model ProjectMiner is a completely data-driven compliment to SEER-SEM parametric estimate Historical software project data SEER-SEM Data Driven Estimate ProjectMiner was industry s FIRST fully data driven model. 38

37 SEER Comparative Sizing: Sizing From Your Prior Actuals Performs Sophisticated Pairwise comparison Evaluate results Generates likely size of unknown software 40

38 Estimates and Sizing Informed By Your Experience 2010 Copyright Galorath Incorporated 41 41

Cost IQ - Obtaining Cost Directly From Limited Requirements A Case Based Reasoning system that can transform high level requirements and specifications into a cost

39 Cost IQ - Obtaining Cost Directly From Limited Requirements A Case Based Reasoning system that can transform high level requirements and specifications into a cost modeling workup within a sophisticated cost estimating tool. This approach can provide even concept planners with an advanced understanding of the cost of potential designs. 42

40 Looking At Maintenance and Technical Debt 2011 Copyright Galorath Incorporated 44 44

41 Process To Estimate Cost of Clearing Violations (Perfective Maintenance) 2011 Copyright Galorath Incorporated 45 45

42 CAST /SEER Preliminary Example Analysis to define a ratio of CAST projected $/KSLOC to SEM projected $/KSLOC Ratio developed calibration to perfective maintenance projection Validates applying SEER-SEM to provide estimated cost of clearing technical debt 47

43 SEER / CAST Sample Projects Internal Application project CMMI Level 4 Size 3,150KLOC COBOL Critical violations per KSLOC 0.44 Estimate of SEM vs CAST $ per KLOC $1070 vs $1006 (6%) Data Warehouse project CMMI Level 4 Size 106KLOC JAVA Estimate of SEM vs CAST $ Per KLOC $828 vs $833 (0.6%) Customer Facing WEB application Size 1152 KLOC CMMI Level 3 Language JAVA Estimate SEM vs CAST in $/KLOC $4614/$4083 (13%) 48

44 Backup Slides

45 2011 Copyright Galorath Incorporated 50 50

Software Total Ownership Cost: Development Is Only Part of the Equation

Software Total Ownership Cost: Development Is Only Part of the Equation Daniel Galorath An Estimate Defined An estimate is the most knowledgeable statement you can make at a particular point in time regarding: