Your logo here. Webinar: Estimation, Planning & Control Can Make the Difference Between Project Success and Failure. June 2012 Time ITMPI005

Transcription

1 Your logo here Webinar: Estimation, Planning & Control Can Make the Difference Between Project Success and Failure June 2012 Time 1 ITMPI005

2 Dan Galorath Founder & CEO Galorath Incorporated Michael Milutis Director of Marketing Computer Aid, Inc. (CAI) 2

3 About Presenter s Firm Galorath is the developer of the SEER models for parametric estimation of Software, Hardware, Information Technology & Manufacturing SEER provides Estimation, Analysis, Planning, Control of critical systems More projects fail due to lack of planning than any other reason 3

4 CAI Achieves IT Operational Excellence 4

5 PDU CREDITS FOR THIS WEBINAR The Project Management Institute has accredited this webinar with PDUs 5

6 NOW AVAILABLE! ONLINE WEBINAR RECORDINGS ANYTIME ACCESS! WWW. ITMPI.ORG / LIBRARY 7 Day Free Access For All Recordings ITMPI 6

7 Human Nature Optimism in Estimates HBR Article explains this Phenomenon: Problem: Humans seem hardwired to be optimists Routinely exaggerate benefits and discount costs Delusions of Success: How Optimism Undermines Executives' Decisions (Source: HBR Articles Dan Lovallo, Daniel Kahneman Jul 01, 2003) Solution: Temper with outside view Viable estimation is tempering Past Measurement Results traditional forecasting w/ statistical Parametrics Can help Don t remove optimism, but balance optimism & realism 7

8 Example of Tempering: (Source How To Measure Anything) German Mark V Tanks Allies estimated production by analyzing serial numbers Used the captured tanks as a random sample and predicted probability of various production levels

9 Estimation Is Critical To Project Success The single most important task of a project: setting realistic expectations Unrealistic expectations based on inaccurate estimates are the single largest cause of (project) failure. Futrell, Shafer and Shafer, Quality Software Project Management

10 An Estimate Defined An estimate is the most knowledgeable statement you can make at a particular point in time regarding: Effort / Cost Schedule Staffing Risk Reliability Estimates more precise with progress A WELL FORMED ESTIMATE IS A DISTRIBUTION

11 Viable Estimation Is Critical Estimating is critical for all kinds of systems Yet many treat is as a second rate process Everyone estimates. Just most get it wrong and don t have a process Repeatable estimation process is critical estimating AND successful projects Estimation & measurement go hand in hand

12 Rating Projects Against Business Criteria (adapted from CAST) 1. Business Value Support effectively one or several Business Strategies Meet legal & regulatory demands Reduce operating cost Produce adequate ROI 2. IT Value Reduce application TCO Replace obsolete or non standard systems Improve reliability, scalability and performance of existing systems 3. Level of Risk IT resource availability User commitment Budget & schedule Technical challenges High Effectiveness (Business Value) Low To be considered To be cancelled Low Project Cost To be approved To be considered if Generates profits Efficiency (IT Value) Level of Risk High

13 Viable Estimating Provides Dependable Decision Support Management Level What is the best business decision Can I believe the numbers What is the risk Technical Leaders What is the most viable effort & schedule What is the optimal development approach What happens when the project changes How can I avoid death marches

14 Poor Estimates Effect on Projects Inaccurate estimates can reduce project success: Poor implementations Critical processes don t scale Emergency staffing Cost overruns caused by underestimating project needs Scope creep Forever changing project goals Frustration Customer dissatisfaction Cost overruns and missed schedules Project Failures Important project business decisions made early with minimum knowledge & maximum uncertainty Poor estimates are a root cause of subsequent program failure 14

15 Estimation Process 15

16 Estimation, Planning & Control Best Practices Quantitative Project Management Estimate & Plan Measure & Analyze Monitor & Control A Foundation of Risk Management

17 10 Step Software Estimation Process: Consistent Processes = Reliable Estimates 1. Establish Estimate Scope 10. Track Project Throughout Development 2. Establish Technical Baseline, Ground Rules, Assumptions 9. Document Estimate and Lessons Learned 8. Generate a Project Plan 3. Collect Data 7. Quantify Risks and Risk Analysis 4. Estimate and Validate Software Size 6. Review, Verify and Validate Estimate 5. Prepare Baseline Estimates 17

18 Estimation Organizational Maturity V1.7 Level 0 Informal or no estimating Manual effort estimating without a process Level 1 Direct Task Estimation Spreadsheets Ad Hoc Process Level 2 Formal Sizing (e.g. function points) Direct Task Estimation Simple model (Size * Productivity) or informal SEER Use Some measurement & analysis Informal Process Level 3 Formal Sizing Robust Parametric estimation (SEER) Estimate vs. actual capture Formalized Multiple Estimate Process Rigorous measurement & analysis Parametric planning & Control Risk Management Repeatable process Level 4 Formal sizing Repeatable process Robust parametric estimating (SEER) Rigorous measurement & analysis Parametric estimation with tracking & control Risk Management Process improvement via lessons learned Level 5 Formal sizing Repeatable process Robust parametric estimating (SEER) Rigorous measurement & analysis Parametric estimation with tracking & control Risk Management Continuous process improvement

19 Estimating Core Governance Component - A standard Corporate Governance Model (Source: K. Aguanno) Great Idea Gate 1 Gate 2 Gate 3 Gate 4 Opportunity Analysis Preliminary Business Case Committed Business Case Achieve Business Case Concept - Describe Idea & Possible benefits Marketing Analysis -Determine customer acceptance -Interviews focus groups, etc. Feasibility Study -Design solution -Estimate cost / schedule -Analyze risk - Determine feasibility / ROI - Pilot or proof of concept - Validate & commit to design & approach -Revised estimates & schedule -Risk reduction -Baselined plan Full Execution or Deployment -Build solution -Deploy -Achieve -business case -Capture lessons learned -incl: estimating

20 Gap Between IT & Management Estimation Needs (Source: Fraunhofer) Financial Business/ Organization Level Customer Vision & Strategy Business Process IT Governance Customer Focus 6σ GQM PSM Innovation / Growth Goals Goal Attainment Questions Answers Metrics Measurement Data Collection Software Development & Maintenance CoBIT IT Governance

21 Lesson Learned: Estimate Must Quantify Risk & Uncertainty Firm Fixed Price? Feel lucky? What is likely to happen Understand the risk before you commit! 21 21

22 Lesson Learned Estimate Total Ownership Costs IT Example Software Development is highest risk but IT infrastructure & IT services can be the biggest cost

23 An ROI Analysis of A New System: ESTIMATION SHOULD BE INVOLVED Can we do better? Will stakeholders tolerate a loss for 3 years? What is the risk?

24 Death March Projects Are Likely To Fail 24

25 Reducing Costs & Spending

26 Software & IT Systems Are About Business Value Cost Value Economics is primarily a science of choice software economics should provide methods for analyzing the choices software projects must make. Leon Levy

27 Many IT Failures Are Due to Planning Issues Cutter Consortium Software Project Survey: 62% overran original schedule by more than 50% 64% more than 50% over budget; 70% had critical product quality defects after release Standish Group CHAOS Report 46% challenged 19% failed 35% successful ~$875 billion spent on IT ~$300 billion spent on IT projects ~$57 billion wasted annually

28 Estimates Provide Insight & Decision Making Support Effort / Cost Estimating Process & Models Schedule Risk TRADEOFFS Reliability Defects

29 What Size Metric Appropriateness Software type in this column Traditional Information Technology Algorithmic Processing is best characterized by Lines Functions Simplified FBS Use Cases X X X X (ROM) X X X X (ROM) COTS Auto-gen Code X X X (ROM) COTS Integration X X (ROM) X Non-Line Based X X X (ROM) X 29

30 Function Points Define WHAT (ISO IFPUG) Source IFPUG External Input Application Being Considered External Interface Files External Output External Inquiry Internal Logical File External Input External Output External Inquiry Other Applications Functionality as viewed from the user s perspective 30

31 Reuse Can Be Measured Even With Functional Analysis Reuse or Heritage: applying existing software to a new mission (or additional innovation in its current mission) Effort to reuse software is routinely under estimated Test Implementation Design We can apply Design, implementation & test even if there is no added functionality 31

32 Key Components Of A Software Project That Uses COTS COTS Software: Purchased functionality Direct Cost component of COTS integration COTS Cognition: Required functionality within the COTS software that must be understood Glue Code: Code written to bind COTS to developmental software Development effort must be captured Developmental Software Customization Glue COTS Software COTS Cognition 32

33 Sophisticated Schedule & Effort Modeling: Essential For Viable Project Plans For a given Size, Complexity and Technology Software Solution Work Expands To Fill Time (Work expands due to lack of pressure) Effort Months Minimum Time To Complete Reasonable Solution Range Optimal Effort for staff (Lower Effort for Longer Schedule) Effort Increase due to Longer Schedule Calendar Time

34 Data Doesn t Have To Be Perfect To Be Useful: But Is Has To Be Viable 80 Calories per serving 2.5 Servings per can 4 Ounces, Condensed, 8 Ounces With Water

35 Use Historical Measurement to evaluate your estimate! It s easy to dig deeper and deeper to justify an estimate! Galorath Incorporated

36 Cost of Technical Debt (Source: CAST) Future cost of defects remaining in code at release (component of cost of ownership) Principal Cost of fixing problems remaining in the code after release that must be remediated Interest -Continuing IT costs attributable to the violations causing technical debt, including higher maintenance costs, greater resource usage, etc. Liability business costs related to outages, breaches, corrupted data, etc. Opportunity cost benefits that could have been achieved had resources been put on new capability rather than retiring technical debt Large programs often have in excess of $1m technical debt

37 Estimation Best Practices

38 Manual Estimates: Human Reasons For Error (Metrics Can Help) Manual Task estimates yield SIGNIFICANT error Desire for credibility motivates overestimate behavior (80% probability?) So must spend all the time to be reliable Better approach: force 50% probability & have buffer for overruns Technical pride sometimes causes underestimates 38

39 Estimation Best Practices Decide Why You Want An Estimate Map Estimation Goals To Estimate Process Maturity & Develop Plan To Achieve The Maturity Have A Documented, Repeatable Estimation Process Make The Estimating Process As Simple As Possible; But No Simpler Be Proactive: The Process Is Important, The Tools Go Along With The Process Get Buy-in From Program Managers Hold People Accountable: Center Of Excellence Can Prepare Estimate But Program Managers Must Own Them Tie The Estimate To The Plan 39

40 Estimation Best Practices 2 Evaluate Total Ownership Cost; Not Just Development Estimate A Range And Pick A Point For The Plan Re-estimate The Program When It Changes Avoid Death Marches: Programs With Unachievable Schedules Are Likely To Fail And Drain Morale Keep A History: Start An Enterprise Database NOW Business Case: Evaluate ROI In Addition To Costs Convert Expert Spreadsheets Into A Common Language 40

41 Estimation Best Practices 3 Track Progress Vs. Estimate Throughout The Life Cycle Estimate Schedule As Well As Effort (Cost) For Complete Picture Tie The Business Case Into The Estimating Process Attack Non-productive Rework As Part Of The Process 41

42 Estimation Best Practices 4 Have clear definitions: What does complete mean What activities are included and excluded (E.g. development only or total ownership; help desk included or excluded, etc.) Which labor categories are included and excluded in the estimate (e.g. are managers included? Help desk? Etc.) Don t ignore IT infrastructure and IT services costs Tracking defect sources can go along with the process 42

43 Estimation Process ROI Loss avoidance from understanding expected outcomes Savings via improved negotiations Reduced time required to estimate work Reduced resources required to accomplish a project Reduced failure from overcommitting portfolio 43

44 Actions Have a Repeatable estimation process Ensure data are understood and normalized Bring ROI under the estimation umbrella Quantify affordability requirements and use estimating to plan and execute them

45 Questions? 45

46 CAI Sponsors The IT Metrics & Productivity Institute: Clearinghouse Repository of Best Practices: Weekly Educational Newsletter: / SUBSCRIBE Weekly Webinars Hosted by Industry Leaders: / WEBINARS ACCESS WEBINAR RECORDINGS ANYTIME AT / LIBRARY Follow Us on TWITTER at / ITMPI Join Our Network on LINKED IN at LINKEDIN Follow Us on FACEBOOK at FACEBOOK Find Out About Our CONFERENCES at EVENTS 46 46

47 Dan Galorath Founder & CEO Galorath Incorporated Michael Milutis Director of Marketing Computer Aid, Inc. (CAI) 47