Fifteen Undeniable Truths About Project Cost Estimates, or Why You Need an Independent Cost Estimate

Size: px
Start display at page:

Download "Fifteen Undeniable Truths About Project Cost Estimates, or Why You Need an Independent Cost Estimate"

Transcription

1 iparametrics, LLC Headquarters 2325 Lakeview Parkway, Suite 200 Alpharetta, GA Fifteen Undeniable Truths About Project Cost Estimates, or Why You Need an Independent Cost Estimate iparametrics, LLC Washington Regional Office Random Hills Rd., Suite 821 Fairfax, VA 22030

2 Introduction All project cost estimates are wrong, some more than others It is difficult to accurately estimate the cost of a large project Cost estimates are usually too low Uncertainty everywhere cost is really a probability distribution Attempting to predict the future with imperfect information Early estimates are the most likely to be too low Cost estimates are only as good as the timeliness of the project s technical description Immature design parameters generally lead to low estimates Unanticipated events will usually drive cost upward Bad assumptions almost always lead to increased cost later on The most comprehensive cost estimates are typically perceived to be too high Include provisions for what is likely to happen beyond the scope Can kill a project before it even starts, so we don t want to know the truth! But wouldn t you rather know now than later?

3 Project Estimates Independent Estimates Requirements Systems Engineering Technical Design Technical Design Cost Uncertainty Technical and Schedule Risks Informed by: Engineering Analysis, Expert Opinion Informed by: Historical Experience, SMEs, Typical Gotchas Driving Factors: Success Oriented, Optimistic, Can Do Driving Factors: Pessimistic, Risks, Uncertainty If competitive award, expect downward pressure on proposed cost Usually higher than project estimate, but still may miss unknown unknowns Engineering Buildup Cost Estimate Bottoms Up Plausible Estimate Should Cost Parametric Cost Estimate Top Down Historically Based Will Cost What the project ought to cost What the project is likely to cost

4 15 Undeniable Truths About Project Cost Estimates

5 1. Project Cost Estimates Are Almost Always Too Low Project cost estimates are produced by project advocates Desire to sell the project to decision makers May assume away foreseeable problems downstream and assume the best case scenario No schedule perturbations, test failures, technical difficulties, etc. Difficulties may be under assumed But events that occur over acquisition life cycle will almost surely lead to increased costs Design changes Funding perturbations Schedule perturbations Unknown unknowns

6 2. Even Independent Cost Estimates Tend to be Too Low Independent estimates get you closer to the truth, but can still be too low Depend on currently agreed upon technical description rather than final actual technical configuration Non advocate estimate, but still pressure to ignore black swans Stakeholders will push back against added costs for predictable events for which nobody is planning The best independent estimates will assume the most likely scenario Allow for occurrence of some foreseeable risk events But even then, may miss some major hits to the project Problem: The estimate may be perceived as unrealistically high

7 3. The More Independent the Estimate, the Better Estimates that are immune from stakeholder pressure tend to be most accurate No pressure to assume away risk events that are likely to occur Willing to report the bad news But even then, likely to miss one or more major hits to the project The best estimates result from independent organizations Recognize the inherent uncertainty in cost estimation Might still be too low, but much closer to the truth Anticipate more extreme risk events Include allowance for funding and schedule perturbations Include potential for design changes and their associated costs

8 4. Cost Growth is the Rule, Not the Exception

9 5. It s Impossible to Know the True Cost Until After the Project is Delivered Nobody knows the true cost of the project until it is delivered and it has experienced all of its difficulties Cost estimates are produced based on the project description at the time But few projects survive their acquisition life cycle without experiencing changes Estimates produced early in the life cycle are most prone to inaccuracy Estimates produced near the end of the life cycle are most accurate, because there is not much else that can go wrong Finally, once the project is delivered, only then do we know the actual cost and it is usually higher than originally thought Some agencies have a no cost overrun policy Forces awardees to de scope to meet budget when difficulties arise De scopes can include things like reduced performance, reduced testing or premature delivery which usually leads to increased cost post delivery

10 6. Your WBS is Probably Missing Something Standard Work Breakdown Structures (WBS) are available for most government projects (MIL STD 881C) Can be complex, and are usually tailored to project at hand Make sure the WBS reflects the entire project effort For example, one WBS element is called Testing But have you added one below that called Re test in the event of a test failure? Strive to ensure that the WBS includes all costs that your project may encounter Typical omissions include: Non recurring costs GFE costs (GFE is not FREE) Design costs associated with COTS integration

11 7. COTS = Certain Overrun To System Commercial Off The Shelf (COTS) hardware and software integration are often assumed to be low cost just plug and play Experience has shown that this is rarely true COTS hardware and software almost always require a substantial amount of non recurring design effort to make the system compatible with the COTS equipment But most project managers consider this effort to be negligible Therefore, these costs are frequently ignored COTS is no easier to integrate into a complex system than custom designed hardware or software Always assume COTS integration is just as expensive as integration of a new design

12 8. Shortening the Schedule Increases Cost Cost cannot be reduced by forcing the project to deliver earlier than planned Compressing the delivery schedule means that the awardee must do the same amount of work in less time, leading to Overtime costs, extra shifts, parallel operations, reduced testing, less time for quality assurance, greater chance for mistakes Corollary: An artificially short schedule will lead to a cost overrun If awardee promises delivery in a time frame that is shorter than feasible, then cost growth is likely to occur when reality sets in Either the awardee will have to compress the schedule internally (see above) Or, the awardee will seek an engineering change order that extends the schedule, which will lead to added cost

13 9. Extending the Schedule Increases Cost Not surprisingly, cost tends to grow when awardees are given extra time to deliver This often manifests when government funding constraints cause awardees to go into standing army mode Awardees must keep the team on standby while awaiting the green light to start up again Keeping them busy with non value added tasking Adding cost to the project After obtaining the notice to start back up, the project proceeds as before, and maybe even farther back along the learning curve Corollary: If it is found that the project needs extra time then refer to Truth #8

14 10. Withholding Funding Increases Cost When government agencies withhold funding from a project, for any reason, then project cost will end up increasing Standing Army effect cost due to non value added effort Potential for parts obsolescence may need to purchase new Loss of industrial base manufacturers may move on to greener pastures Awardee loses cash flow, may have to divert resources to other tasks loss of trained team or manufacturing facilities Upon restoration of funding, awardee may have to start up a new team higher up the learning curve This is sometimes done to punish an awardee who is falling behind But withholding funding only pushes the awardee further behind

15 11. Providing Additional Funding Increases Cost This may seem self explanatory, but Sometimes government tries to solve problems by throwing money at it If it is necessary to throw money at it then the project is in trouble anyway, and costs are likely to grow just for that reason But also, awardees will use the money to get well then will proceed as before from a new baseline Once you ve thrown money at it you ll never see that money again It is likely that the project really needed it in the first place May happen in the form of an Engineering Change Order

16 12. Contracts Cost Estimates A program manager once argued with me, saying, I know exactly what this program will cost because I have a contract right here in my hand! (true story) So what happens when the money runs out before the project is delivered? The project is de scoped? maybe, but performance may suffer, and operations and maintenance costs may have to increase to fix the problems Presumably the government still wants the project to be completed so usually, the government ultimately provides the needed funding The point is, a project costs what it costs regardless the value stated in the contract Truth is necessary on both sides so that the government is not disappointed, and the awardee can execute

17 13. The Best Estimate is a Probability Distribution The best cost estimates are those that recognize and reflect uncertainty Remember we are trying to predict years into the future Quantification and reporting of uncertainty helps bound the problem Point estimates are guaranteed to be wrong at least with a probability distribution, we can see the range of possible costs The funding amount can only be determined AFTER the cost probability distribution is developed Probability Density Function 50 th th percentile $115M Mean $120M Lognormal Lognormal Cost Cost Distribution Distribution Mean Mean = $120M $120M Std. Std. Dev Dev = $35M $35M Probability Density 5 th th percentile $72M 80 th th percentile $146M 95 th th percentile $184M $M

18 14. But Even a Distribution May Miss Some Important Events Initial cost estimates are almost always too low Here is what frequently happens Cost is estimated based on the baseline program description Requirements change and/or schedule slips Cost distribution shifts to the right Problems are identified that require ECOs to resolve Cost distribution narrows, but shifts further to the right Repeat previous two steps numerous times Cost Distributions Eng. Change: Mean = $175M Other Disaster: Mean = $200M Final Actual Cost = $210M Density Baseline Estimate Mean = $100M Rqmt Change: Mean = $125M Sched Slip: Mean = $150M Baseline 80th %ile = $122M $M

19 15. If You Think the Estimate is Too High, Think Again Sticker shock for complex projects is common But we really do ask awardees to produce complex projects on a shoestring budget The systems we buy are expensive by their nature, and willing them to be less expensive does not work When a cost estimate is perceived to be too high, ask yourself Where does the estimate fall on the probability distribution? Is this an awardee s estimate, or an independent estimate? Do we realize that awardees estimates are always too low, and even independent estimates tend to be too low? Have we anticipated inevitable cost growth? Have we ever seen a project that was not more expensive than we originally thought?

20 So What Can We Do?

21 Be highly skeptical of project cost estimates They are almost always too low You need a solid independent cost estimate This will get you closer to the truth So What Can We Do? Embrace uncertainty, then assume the worst case A cost probability distribution is helpful, but don t plan on the cost falling at the 5 th percentile Anticipate cost growth and budget for it All projects experience cost growth anticipate it Let the estimate drive the budget, and not vice versa Trying to force the estimate to match the available funding will lead to disappointment

22 Not beholden to key stakeholders Benefits of an Independent Cost Estimate But be forewarned, you may suffer sticker shock A good independent cost estimate will identify potential cost growth that you have not considered The cost probability distribution is a tool that can be used for risk and resource management You will be prepared to deal with the real cost when it materializes An independent estimate provides backup for requesting necessary funding for your project Independent estimates help you make good decisions

23 Be skeptical of project cost estimates What the project ought to cost usually too low Independent cost estimates get you closer to the truth Summary What the project is likely to cost may appear to be too high, but probably not high enough Cost growth is a fact of life in government acquisition Anticipate it include it in the budget Always think of cost estimates as probability distributions Then assume the cost will fall in one of the two upper quartiles The Heisenberg Uncertainty Principle applies to cost estimates too It is impossible to determine with perfect accuracy both the position and momentum of a project s cost at any given point in time