Anthony A. DeMarco President, PRICE Systems, L.L.C.

Transcription

1 Presented by Anthony A. DeMarco President, PRICE Systems, L.L.C. September 26, 2006

2 Headlines! Inadequate Project Management at DOE Results in Frequent Cost Overruns, Delays - National Academy of Sciences FBI's Trilogy Program Draws Senate Ire FBI Case Management System Marked for Termination Modern casemanagement capabilities remain atop the FBI's IT Most Wanted List as the bureau's $170 million Virtual Case File (VCF) project faces imminent termination. - INTERNET News Of the 250 projects analyzed, about 25 were deemed successful in that they achieved their schedule, cost, and quality objectives. About 50 had delays or overruns below 35 percent, while about 175 experienced major delays and overruns, or were terminated without completion. Capers Jones, Software Project Management Practices: Failure Versus Success, Crosstalk Vol. 17 No. 10, October 2004 UNITED STATES COAST GUARD Improvements Needed in Management and Oversight of Rescue System Acquisition -GAO Report 2

3 The Affordability Challenge Affordability is the biggest issue facing agencies, organizations and contractors today. Clear evidence is seen in high profile failures (inaccurate estimates, cost overruns, major project delays, project cancellations) Why? Although there are clear program management guidelines, programs do not focus on the cost engineering discipline and cost engineering processes and technologies stove-piped The Consequences Government oversight has increased. Credibility, competence, and funding are challenged. The Solution Program Affordability Management How do you get the funding you need, manage risks and eliminate surprise cost overruns? There are six key steps to successful Program Affordability Management 3

4 Anthony A. DeMarco President, Managing Member and co-founder of PRICE Systems, L.L.C. 25 years experience in project cost engineering and program management analysis Award winning creator of cost and schedule models and management methods Patent author of cost estimating techniques Member of the NASA International Space Station Management and Cost Estimating task force (Young task force) Board of Director of the National Defense Industrial Association Delaware Valley Chapter 4

5 Definitions Program A system of services, opportunities, or projects, usually designed to meet a social need Affordability ability to be able to bear the cost of Management the conducting or supervising of something Program Affordability Management The set of supervised and coordinated activities that determine whether or not an organization will be able to bear the cost of a program over the course of its life 5

6 Total Program Life Cycle 6

7 Program Affordability Management There are key steps to Program Affordability Management, fortifying the practice with the expertise, tools and best practices and discipline to meet specific oversight requirements and build credibility, competence and success. Program Affordability Management integrates traditionally stove-piped activities, such as : Engineering/Design Estimating/Cost Management Earned Value Management Knowledge Management into a cohesive, iterative, lifespan view. 7

8 Assemble the right team Percent of effort spent on Affordability Management 8.0% 7.0% Complexity 6.0% 5.0% 4.0% Priority 3.0% Complex Program $100M/year 30 people Simple Program $5M/year 1 person Statistical Technical Leadership Financial You must size the affordability management effort commensurate with program size, complexity, and priority You must ensure that you have the right skill mix for the program 8

9 Use a credible life cycle cost model Decision Criterion = Effectiveness / LCC Simple cost model (< 10,000 function points) LCC = Design + Build + Support = ($200 x FP) + ($400 x FP) + ($50 x FP x Yrs) where LCC = life cycle cost, and FP = function point count Understanding common language among the team from Management throughout the supply chain Focus common performance criteria helps guide management actions Performance Based Contracts People operate with beliefs and biases. To the extent that you can eliminate both and replace them with data, you gain a clear advantage. Speed rapidly evaluate, negotiate and agree on changes Confidence relieve fear of mistrust and inaccuracy Moneyball is the best business book Lewis has written. It - John Henry may (Florida be the best Marlins business Owner book 1999) anyone has written. - Mark Gerson, The Weekly Standard 9

10 Use a credible life cycle cost model Decision Criterion = Effectiveness / LCC Robust Cost Models (>10,000 function points) Understanding common language among the team from Management throughout the supply chain Focus common performance criteria helps guide management actions Performance Based Contracts Speed rapidly evaluate, negotiate and agree on changes Project cost/ effort over time Credible: offering reasonable grounds for being believed Confidence relieve fear of mistrust and inaccuracy Project Cost by Resource there are commercially available models for ERP and COTS implementations, Data Center Operations, custom hardware and software development and more. 10

11 Get the estimate right the first time FINAL PROGRAM COST 2x s 1 UNDERESTIMATES LEAD TO DISASTER - insufficient resources - panicked decisions - unrealistic expectations REALISTIC ESTIMATES MINIMIZE FINAL COST THE FREIMAN CURVE OVERESTIMATES LEAD TO DISASTER - under-utilized resources - excess capacity - un-competitive pricing 1 2x s ESTIMATED COST 11

12 Get the estimate right the first time Size, Productivity, Complexity Design Cost = SIZE x Productivity x Complexity = SIZE x (Hours/SIZE x K) where K is determined by benchmark comparison with other projects and/or operations research on project characteristics, for example Design Cost = (FP x $/FP x K) = 5,000FPs x $200 x 1.4 = $1,400,000 where FP = function point count SIZE, and $200 = average normalized cost per FP for a web-based user interface, and 1.4 = empirical factor derived from OR on non co-located teams 12

13 OMB OEGIT Directive 13

14 Create, document, train, and reuse knowledge, metrics and processes 1. Program Affordability Management Manual 2. Cost Estimating Guide 3. Earned Value Management System Guide 4. Risk Management Guide 5. Design to Cost Guide Life Cycle Cost Estimating Process Flow 1 Understand the customer needs and capture the program mission and goals. Aircraft, Ship, Missile Software, electronics etc. 4 Collect applicable and necessary technical, programmatic, and cost data COTS Models & Databases such as Price H, Price S, ACEIT, Excel. Custom Built Applications 7 Identify cost drivers and determine risk/uncertainty. What If Analysis Cost, Schedule, Tech. Parameters Sensitivity Analysis Monte Carlo Simulation Analysis of Variance (ANOVA) 2 Identify program requirements and establish timeline OMB Circular A76 or A94 DAD 5000 etc. 3 Establish a common Work Breakdown Structure (WBS) RDT&E Procurement Operations & Support Disposal 5 Determine appropriate estimating tools & techniques to employ. ACEIT, PRICE H, PRICE S SEER-SEM COCOMO MS Excel, etc. 6 Develop a cost estimate Current Year $ Then-Year $ Discounted $ 8 Validate/Verify cost estimate and assumptions Use Data from Analogous programs for cross checks. 9 Present findings and defend cost estimate Documentation 14

15 Integrate cost, project control and knowledge management and ensure program is incentivized on forward looking measures As is Cost Estimating Cost Estimating Cost Estimating Cost Estimating Program Control (Earned Value Management) Cost Estimating Should be Cost Estimating & Program Control (Earned Value Management) Knowledge Management 15

16 Integrate cost, project control and knowledge management and ensure program is incentivized on forward looking measures Program Affordability Dashboard You cannot manage by only looking into the rearview mirror Integrating estimating models with program control enables forward-looking performance based measures of cost, schedule and effectiveness. TOTAL PROGRAM Level 1: Visitor Recognition and ID System Today PRE-SYSTEM ACQUISITION Concept Refinement & Technology Development SYSTEM ACQUISITION SDD & Production O&S Acq Pre-Acq Budget Targets SUSTAINMENT Operation & Support

17 Persist! Repeat steps 1 through 5 throughout the program life cycle People, Processes, Tools 1. Assemble the right team 2. Use a credible life cycle cost model 3. Get the estimate right the first time 4. Create, train, document and reuse knowledge: metrics and processes 5. Integrate Cost, Project Control, and Knowledge Management and ensure that the team is incentivized on forward looking measures 17

18 Federal Program Office Case Study Large Government Program Office Program Office Improves Business Modernization Management Controls and Capabilities Through Independent Validation of Cost and Performance The Challenge: Controls and capabilities related to configuration management, human capital management, cost and schedule estimating, contract management, and postimplementation reviews have not yet been fully implemented or institutionalized. Weaknesses in these controls and capabilities have contributed, at least in part, to Business Systems Modernization (BSM) project cost and schedule shortfalls. Solution: Partner with an expert who could produce accurate, independent cost estimates and analyses quickly and reliably, improving the credibility and reliability of program cost and budgeting. Results: The initial effort was focused on one project in the form of an independent cost estimate and assessment for the next increment. The results were successfully received by the program office and the oversight agencies. Additional projects of various sizes have been taken on with similar results. The program office is currently evaluating proposals for a five year effort to continue this type of support. Highlights: 1. Program Office gained credibility with oversight committee. 2. Initial findings valuable enough to warrant a long-term contract. 18

19 Defense Program Office Case Study Department of Defense (DoD) Standard Procurement System Gains Balanced Scorecard through Accurate Cost Estimating and Knowledge Transfer The Challenge: Overcome SPS software integration failures and functionality shortcomings and eliminate the ineffective resulting work-arounds that triggered a strategic pause in the project and a lack of confidence by end users and oversight boards. Solution: Partner with an expert who could produce accurate, independent cost estimates and analyses quickly and reliably, maximizing efficiency of existing DoD tools and processes. Results: A process of knowledge transfer and set of reusable best practices for project control that enabled SPS to interface with 32 systems across DoD and ended the strategic pause by regaining the confidence their customers, users, and the oversight agencies. More than 23,000 DoD professionals use SPS to purchase over $48 billion in goods and services; currently it s deployed in 800 sites worldwide. Highlights: SPS achieved Upper-Management s Recognition as well as gained credibility with their oversight committee 19

20 Federal Program Office Case Study National Oceanic & Atmospheric Administration s, National Weather Service (NWS) NWS Improves Ability to Define, Analyze, and Defend Budget and Performance Requirements with Program Affordability Management Approach The Challenge: NWS determined the need to modernize our nation's ability to monitor weather and climate with greater resolution and frequency, including the modernization of their COOP program, which actually supports more remote and densely populated stations throughout the United States. Determining the process for modernization was a challenge that had never been undertaken at this level. Solution: Working with PRICE Systems TrueMethods best practices and TruePlanning tools, NWS was able to adopt an Affordability Management to planning and control, to improve their ability to manage costs in their multiple-program portfolio environment. Results: By integrating the budget process, the cost estimating efforts, and the earned value activities all into one framework, NWS is able to measure performance essential for oversight organizations such as Office of Management and Budget. Highlights: President Bush asked NWS to take the lead in an international Tsunami Detection system, NWS was ready and implemented the framework immediately. 20

21 Defense Contractor Case Study BAE Systems (formerly United Defense) Armament Systems Division Achieves Measurable Competitive Edge, Proven by Repeat Contracts and an Increased Win Rate The Challenge: Develop a cost-modeling tool that will immediately establish credibility with new customers by quickly producing data-defensible proposals and cost estimates. This new tool needs to respond fast to complex what if scenarios and make proposal and project adjustments seamlessly. Over time, a high level of confidence must be maintained through ongoing accuracy of quotes in the prototype, test and production phases, ultimately enabling BAE to effortlessly execute contracts on time and within budget at all times. Solution: Establish a disciplined cost methodology based on a robust knowledge base of cost management throughout the life cycle of a program. BAE s knowledge base is built on an internal history and industry-wide cost data using PRICE tools and consulting best practices. Results: Achieved a measurable competitive edge, proven by repeat contracts and an increased win rate. With cost-modeling from PRICE Systems, United Defense has established credibility with its customers and contractors. Profitability is now sustained, allowing its systems to sell at competitive costs. Overall, BAE is in a stronger negotiating position with suppliers,has improved workflow and reduced risk. Highlights:. We have to be profitable in order to develop new product lines, go after new business and execute DoD contracts. Our work using PRICE gives us the confidence we need to invest in new areas, deliver our projects at a fair price and move forward profitably. Customers keep coming back as a result. - Jim Unterseher Director of Army Programs at ASD 21

22 Systems Integrator Case Study Computer Science Corporation CSC Wins New Business by Employing PRICE Tools and Services The Challenge: The AirForce s Expeditionary Combat Support System (ECSS) program is an Oracle-based ERP commercial off-the-shelf logistics support system, part of the Air Force s elog21 long-term overhaul of its logistics system. ECSS is intended to integrate financials, order management, purchasing, inventory management, distribution and other business functions onto one platform. In order to receive award, Computer Science Corporation (CSC) would need to provide a response that addressed costs and time required for ERP implementation. Solution: Working with PRICE Systems TrueMethods best practices and TruePlanning tools, CSC was able to accurately provide a credible cost estimate that supported the proposed COTS software and addressed the AirForce s requirements. Results: The Air Force Materiel Command s 554th Electronic Systems Group awarded a firm fixed-price task order worth $627.8 million to CSC for systems integration support to configure, deploy, and conduct training and change management activities for the Expeditionary Combat Support System (ECSS). Highlights: With the completion of ERP blueprinting by May 2008, this selection will enable the first ECSS capability to be implemented soon thereafter, with deployment of full operational capabilities by January

23 Banking Industry Case Study Credit Agricole SA The Company's principal activity is the operation of the largest commercial bank in France and one of the largest in Europe. The Challenge: Every IT project, with associated operational justification and cost estimation, are reviewed as part of the elaboration of the IT multi-year Budgetary Plan. With projects at very different stages, there was a need to prioritize projects to make the decision of which projects will be launched and when. Solution: Partner with an expert who could produce accurate, independent cost estimates and analyses quickly and reliably, improving the credibility and reliability of program cost and budgeting. Results: Crédit Agricole received a Proof of Concept which covers the whole process of budgetary planning enabling them to better manage their Information Systems by providing a flexible cost estimation framework for all their IT projects and a means to rationalize and improve consistency of the data requested from the projects. Highlights: Crédit Agricole achieved a roadmap to make credible decisions regarding their IT multi-year Budgetary Plan 23

24 Banking Industry Case Study ING The organization's corporate governance and transparency have been improving continuously since ING was created in ING's governance moves away from a typically Dutch governance structure toward a structure that is better adapted to the needs of an international company. ING has an annual IT spend of about 2.5 billion about 25 percent of total operating expenditure and about 40 percent of net operating income. It employs about 15,000 IT staff, and invests around 1 billion on IT-enabled capital programs. The Challenge: ING is totally dependent on information technology, not just to support and enhance its business, but increasingly to enable it. ING realized the need for excellent governance structures and processes to determine the right investments were made and the organization's resources were properly employed resulting in a return on investment. Solution: ING adopted a dashboard and benchmarking process to provide information necessary to enable ING, over time, to develop and compare the most appropriate metrics on IT spend, performance and value. Results: ING use of the knowledge and tools improved the IT investment process and help to better manage IT expenditure and providing a far clearer and more complete picture of where its major areas of project risk are. Highlights: The ING case study indicates the co-variance between ING s superior business performance and the robustness of the IT governance structure supported by innovative IT portfolio analysis (investment fund management approach of enterprise IT) combined with strong execution capabilities. 24

25 Summary Affordability is the biggest barrier to program mission success Government oversight is demanding more affordability discipline Program Affordability Management is a practice that satisfies government oversight and helps ensure program success There are Six key steps to successful Program Affordability Management Program Affordability Management is practiced by successful programs today Among the most important software development practices leading to success are those of planning and estimating before the project starts, absorbing changing requirements during the project, and successfully minimizing bugs or defects. Successful projects always excel in these critical activities: planning, estimating, change control, and quality control. By contrast, projects that run late or fail typically had flawed or optimistic plans, had estimates that did not anticipate changes or handle change well, and failed to control quality. Capers Jones, Software Project Management Practices: Failure Versus Success, Crosstalk Vol. 17 No. 10, October

26 Upcoming Webinars Project Sizing Independent Government Cost Estimates Thursday, October 26, 2006 Thursday, November 30, :00 AM 11:00 AM Size of the project is a critical element How to determine the size of your project Common mistakes in determining size Methods for sizing White paper available soon We will offer a free workshop and 30 day trial of software sizing tools for all who attend Why is an Independent Estimate needed? Does an Independent baseline really gain you anything? What is the industry best practice? White paper available soon Also presented will be the results of an independent Market Survey of government leaders To learn more and register: 26

27 Thank you for your time QUESTIONS? If you would like to discuss further or find out more Anthony A. DeMarco President, PRICE Systems, L.L.C N. Moore Street Suite Commerce Parkway Suite A Arlington, VA Mt. Laurel, NJ (Office) Anthony.DeMarco@PRICESystems.com ( ) (Website) 27