Data-Driven Estimating Lessons Learned

Transcription

1 Data-Driven Estimating Lessons Learned Melissa Winter Solutions Architect Canadian Cost Symposium October 21, PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 1

2 Knowledge Based Estimation Systems Integration DATA CAPTURE DATA MINING & ANALYSIS KNOWLEDGE MANAGEMENT PARAMETRIC MODELS ESTIMATION FRAMEWORK ESTIMATE MAPPING PRICE Consulting Services PRICE Consulting Services PRICE TrueFindings PRICE Models PRICE TruePlanning PRICE TrueMapper Private or Public Data Data dictionary Data Quality Standard Define Collection Cycle Define Collection scope Selective automation Supplier Training Determine Taxonomy Build Database Data Cleansing Categorization Normalization Calibration Selective automation Historical data repository Statistical tools Perform Data Studies Create Findings, metrics, CER s Inject historical perspective into cost models Foundation for data-driven estimates Proven, reusable Parametric Models Effort Materials Activity Duration Metrics Activity-based models Integrated Custom Model Development Platform Estimate cost, schedule and risk Estimate parts to programs Comprehensive Burdening Escalation Schedule Risk Systems & Integration level Integrated with Common Tools COM API Map your estimate into your customers preferred format, WBS, CES, OBS, etc. PRICE Cost Analysis Framework DATA INTEGRATION Program Execution (PLM, EVM, PPM, PM) Bid or Should Cost ESTIMATE (Cost, Schedule, Risk) Bottom Up Top Down Estimate Reconciliation 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 2

3 Data Collection Roadmap Raw data Do we collect it? Is it out there somewhere? Is it usable? Is it relevant? Estimating databases For analogy project level? Component level? For derived metrics? For parametric models? For multiple purposes? One or many? 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 3

4 What data do we want? Within past 5 years Business we will be pursuing in the future (relevant) Data that is available Data from projects with cognizant personnel still available Contract restrictions on data use is understood 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 4

5 What do we want to use it for? Project level analogy Analogy at lower levels (product or component or CI) Derived metrics? Productivity metrics? To drive parametric models? Which ones? Sizing by analogy, estimating by parametric database must support both methods Quick Look against Price-To-Win what data does this require? 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 5

6 Data needs throughout the project lifecycle Project or product level data for top down estimating methods Detailed cost history, metric data, calibrated estimating models Detailed cost history and metric data by development phase, calibrated estimating models High level & detailed cost history, metric data, calibrated estimating models In support of: ROM & Life Cycle estimates, price to win and feasibility analysis In support of: Firm Cost Estimates/ Independent Cost Evaluations (ICE), LCCE, Engineering bid estimation In support of: Cost estimating for Project baselining and rebaselining (budget estimates), ETCs / EACs / ECP development, Risk Impacts & Mitigations, ROMs In support of: ECP Development, ROMs & Life Cycle estimates Prepare Bid/No Bid Information Prepare Proposal Contract Accepted Mobilize Initial Design Design System Integration of System System Qualification Production, Installation and Commissioning In-Field Support Disposal Phase Reviews at selected points Identify Options Develop Option Definitions Assess Each Option Down Select Options Detail Remaining Options Prepare Proposal Proposal Accepted Deploy Technology Produce System Specification Agree System Specification Freeze Engineering Definition Product Manufacturing Definition Validate, Build & Clear for Op. Trials Prove Op. Capability Certify, Build, & Product Operational Clearance Support Product and Service Decom- Missioning Procedures INNOVATION SELECTION SPECIFICATION DEFINE/REFINE VALIDATION PROD. SUPPORT DISPOSAL Design Reviews SRR SDR/PDR CDR TRR PRR 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 6

7 What are we starting with? Do you have the right data collected? Does it match the requirements? Does the WBS collect the data that you need for estimating? Do projects adhere to the WBS? Cost history and metrics need to align with intended use Don t need to collect everything possible Some data is key to support estimating methodologies Let use cases identify the data needs of your organization 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 7

8 The Process Designing the data access Levels of access: Summary/Dashboard Summary for the project Detailed for the project Combinations of data Derived data Data Markings: Are there any restrictions? How many flavors? No access Limited permissions No restrictions within RFP constraints Unlimited permissions, no restrictions 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 8

9 The Process Your data must have a gatekeeper Data review is critical For consistency For assumptions For right level of detail For nothing inappropriate No finger pointing, just facts 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 9

10 Derived metrics, which ones are useful? Average and standard deviation of: Allocations by discipline (SW, HW, Systems) Allocations by Engineering life cycle phase Concept Definition, Design/Implement, Integration & Verification, Transition & Validation Allocations by WBS elements Top level only Third level All levels? Multiple program sets of data Selectable Entire DB segments 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 10

11 Partial/incomplete project data sets Are they still meaningful? For life cycle phase information In support of ETCs What do they tell us? Compare against original estimate for phase Estimate ETC and compare to total estimate Early indicator of a cost overrun 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 11

12 Data output from the database What do we need for output? Analogous program data sets Derived metrics for analogous programs How will it be used? In Basis of Estimate justification Referenced by program task code Referenced by bid code mix 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 12

13 Many uses Price-to-Win assessment From an architecture diagram Building block representation Cost history must support this Or Productivity metrics for common blocks Also need factors for supportive activities And don t forget management Does your project data collection support all these needs? 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 13

14 Parametric Models Existing models require specific input parameters Does project data collection include these parameters? Part of closed loop metrics process Painful to collect after the fact Less complete, less accurate Inherent in database, can be used for analogous sizing Drives new inputs into parametric model with lower risk/variance 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 14

15 Parametric Models the trust issue General reluctance to trust parametric models Bidding is too important to make a mistake Don t understand how they work Trust in old ways, bottoms up Suggest using as validation of estimate Baseline estimate using parametric, use it for iterations after that Cost savings from bottoms up iterations Use for analogy of input parameters, less risk in input assessments 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 15

16 Care and feeding of the database Aging of the program data Changes in data have to be propagated Populating is part of business rhythm Automated import data to the extent possible Access control Communication/training Maintain expert at Org. level Governance/ownership Sustained management championship and support resources and funding 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 16

17 Questions? 2013 PRICE Systems, LLC All Rights Reserved Decades of Cost Management Excellence 17