BOE Training Material The Drive Toward data driven Estimating Boeing/Lockheed Martin/Northrop Grumman
|
|
- Camilla Stewart
- 6 years ago
- Views:
Transcription
1 BOE Training Material The Drive Toward data driven Estimating Boeing/Lockheed Martin/Northrop Grumman Original March 2011 Updated February
2 Background/Objectives Background: There is a significant industry effort to increase the quality of contractor estimating processes. The industry has applied the term data driven estimating to this new method of developing and documenting our basis of estimates(boes). data driven Estimating methods have certain characteristics that are starting to emerge as standards within our industry. Objectives: What is driving our industry toward the use of data driven estimating methods and why is this a business imperative Review the current primary estimating methods Be able to identify the characteristics of good data driven estimating processes and how they are represented in our BOEs What are objective-scope-measures and how are they used in a data driven estimate Provide examples of data driven versus non-data driven BOEs 2
3 data driven Estimating - Contents BOE - Description data driven Estimating - Definition The Future Fundamentals of data driven Estimating Developing Relationships from History The Cost Estimating Relationship (CER) Process data driven Estimate Examples We are the Customer Summary Backup Charts/Supplemental Resources 3
4 Purpose of a BOE Documents and substantiates effort required to perform tasks Documents basis and assumptions Details understanding of requirements Conveys credibility by clearly defining tasks and assumptions Demonstrates that estimates are credible, defendable and realistic (historical data) Provides evidence that estimate is consistent with other proposal volumes Time-phased resources consistent with IMP/IMS Allocates skill mix appropriate for task Aligns with Work Breakdown Structure (WBS), SOW, and other product structures Meet customer funding, timing, and/or scheduling constraints 4
5 BOEs The Components A BOE consists of the following main elements: 1. Work Element ID WBS, SOW, Location, CLIN, NRE/Rec 2. Description of the products or services delivered and the tasks to be performed 3. Estimating method, rationale, assumptions, and computations used to develop the estimate Method of quoting 4. Relevant historical data and comparability of the historical data to the current program Explanation of any adjustments made to historical cost or programmatic data (i.e., use of modifiers) 5. Estimated labor resources to perform the defined tasks 6. Expenditure profile (time-phasing) of labor resources 7. Other elements as required by the cost proposal instructions 5
6 BOE Estimating Methods Labor estimates are developed using one or these typical estimating methods Judgment based on subject matter expert (SME) experience with similar efforts. Level-of-Effort/Support - uses resource-based estimates that are often based on a predetermined level of support for a given period of time Analogy/Comparison Method estimate based on historical frame of reference from a similar effort with adjustments for programmatic or technical differences Standard Time Estimating - Labor Standards and Realization/ Performance Factors - time necessary to complete a defined element of work following a prescribed (recognized) technique or method. Unit Method generally a simple ratio (not always statistically validated) Follow-on Based On Actuals extrapolation of actuals (CEAC, production) Parametric Cost Models use of calibrated, internally developed, or commercially available cost models Cost Estimating Relationships (CERs) - equations developed based on historical, statistically correlated relationships Note: Material or Other Direct Costs may be estimated by data such as quotes or catalogs. See FAR for specific cost element and degree of competition requirements 6
7 Customer Expectations Customers/DCAA/DCMA expect contractors to begin adopting improved estimating methods to support TINA-compliant proposals and other estimating products Least Preferred Judgment Most Preferred Fact Based Estimating Methods Move To Fact Based ASAP 7
8 BOE Methodologies Credibility Scale Customer Credibility High Low Labor Follow-on Projections based on Actuals Analogy/Comparison CER s Parametric Estimates / Cost Models Standards Unit Methods Level Of Effort Judgment Historical resource based estimates are easier to support than judgmental type estimates. 8
9 Customer Consequences Poor Estimating Failure to be responsive to customer estimating processes could be grounds for Non-compliance with RFP or government regulations Rejection of proposal and possibly defective pricing Estimating System Inadequacy Penalties for bad BOEs include: Questioned costs DCAA/Government Deficiency Report Negotiation losses Negative factor in competitive evaluations Defective pricing and fraud prosecution Suspension of bidding and debarment Holdbacks 9
10 data driven BOEs Basic Definition Fact Based Estimating: Estimating processes/boes that provide the supporting information and rationale necessary for accurate, credible and traceable estimates Information includes: Use of defined estimating methods Historical data from past programs Use of objective scope measures and complexity factors where appropriate Rationale addresses: Complete description of the estimating approach Explanation of the applicability of the historical information Estimating process must be documented, consistent and repeatable 10
11 Fact Based BOEs The Fundamentals 1. Estimate develop using a defined and documented estimating processes 2. Methods rely on scope measures (attributes) and complexity factors (parametrics) when applicable 3. Complete estimate development process is explained and documented in the BOE Professional judgments, modifiers/adjustment factors, and historical context is adequately explained Process is consistent and repeatable 4. Estimate relies on relevant historical data and context that is accessible and verifiable Shows use of latest and most current relevant data Supports all resource estimates with verifiable data 5. Estimating processes are monitored for learning and improvement as required to support future estimates 11
12 Objective Work-Scope Measures (OSM) Scope Measures: estimating predictive measures technical measures relating to the size of the effort (attributes) Generally physical and technical measures Sized at different product levels - system, subsystem, box, etc. Example: Work Products: # drawings by type, sheet counts, # documents & sheet count, SLOC by category, # requirements, # reviews, # test & type, weight, quantities, etc. Traits/Complexity Factors: characteristics that can improve the accuracy of relationships developed for the OSMs Example: Scope Measures = Drawings Traits: (numeric and descriptive): # Drawings, Type of Drawing; Drawing Heritage; Item Next Higher Assembly/Element; Number of Sheets; # of Revisions Example: Scope Measures = Source Lines of Code (SLOC) Traits: Software Language; New SLOC; Modified SLOC; Reuse SLOC; Actual Total Delivered SLOC, Deleted SLOC, Auto Generated Code, etc 12
13 Why Use Scope Measures and Traits in BOEs? Scope Measures - help to scope the size of the effort to be estimated Objective quantitative measures of scope to be performed Directly correlate to (predictors) resource estimates for work products and tasks Understanding the size (scope) of the proposed effort is critical to developing an accurate estimate (BOE) The BOE author should explain the derivation, source, and application of the scope measures in the BOE Traits/Complexity Modifiers address unique characteristics of the scope measures that can affect required effort to produce Capture the effects of technical and programmatic differences between historical programs and the current proposed effort. The produceability can vary widely for any scope measure - traits can be used to explain the differences Sometimes requires judgment 13
14 Objective Scope Measures - Examples Software Hardware Systems Engineering Candidate Estimating Attributes Source Lines of Code (SLOC) Function points Computer Software Configuration Item (CSCI) Number of drawings Power requirements Number/type of subsystems, subassemblies, components Volume (Dimensions) Weight Number/type of reviews Number of end items Number/type of tools Number of Contract Data Requirements Lists (CDRLs) Number of Subcontract Data Requirements Lists (SDRLs) Number of system requirements Number of Interface Control Drawings ( ICDs) Number of payloads Number of program segments Number of internal interfaces Number of external interfaces Number of risks Number of architectural trades 14
15 Traits/Complexity Factors - Examples Software Hardware Systems Engineering Estimating Modifier Design reuse Performance factors Complexity Reuse and pedigree New technology Components/functional complexity Schedule Requirements maturity Technology Readiness Level (TRL) requirements/maturity path Multi functional analyses Testing (qualification; prototype protoqualification; delta qualification; static; modal) Assembly, Integration and Test (AI&T)) Training requirements Mission profile and environments Mission design (data rates, pointing stabilities, Technical Performance Measurements (TPMs)) Oversight and analyses of subcontracted elements Risk Program maturity/life cycle Complexity Schedule Heritage Required documentation 15
16 Explaining Adjustment to a BOE In order to use an objective or subjective adjustment, an estimator must explain it thoroughly in the BOE. Provided below are some examples of adjustments that may be required to improve the accuracy of the estimate. The list is not meant to be totally inclusive of adjustments an estimator has available in their tool box. Technical Complexity ~ Labor Estimate Improvement Curves ~ Labor Estimate Schedule adjustments ~ Labor Estimate Quote Decrement ~ Material Estimate Quantity Adjustment ~ Material Estimate Escalation ~ Material Estimate Exchange Rate ~ Material Estimate 16
17 Using Historical Data in a BOE Use of Historical Data Must Address: Source of data generally from the organization s certified historical data repositories Complete identification of the nature, source and details of relevant historical data Ensure compliance with A.C.T.T.R. by documenting the following: Actual historical hours Charge numbers included Technical data used Time frame of historical data Repository or source of historical data 17
18 Developing a Relationship from Historical Data The statistical analysis process used in the development of a cost estimating relationship (CER) is documented in numerous industry and statistical sources The objective of the analysis is to validate that a significant statistical relationship exists for the CERs and to demonstrate that the CERs accurately predict effort or costs CER Strengths Past performance reflective of future costs Lean and credible estimating method in preparing BOEs More scientific and objective than judgement 18
19 Developing a data driven Relationship (CER) numerator denominator item to estimate known item pool base dependent variable independent variable Processs Step: 1. Identify Need 2. Establish Estimate Structure 3. Identification of Effort/Cost Drivers 4. Selection of Equation Form 5. Selection of Population 6. Data Collection 7. Data Normalization 8. Generate the Cost Estimating Relationship (CER) 9. Validate the Results 10. Document 19
20 Developing a data driven Relationship (CER) Regression Analysis: Develop mathematical relationship to make inferences about a future probabilistic event based on a sample of historical data. If relationship exist between two or more variables, such as weight and hours of effort/cost required, regression analysis is a powerful tool for the cost estimator/analyst. Regression analysis consists of curve fitting and the analysis of the descriptive and inferential statistics that result from the fit of the data. 20
21 Developing a data driven Relationship (CER) Y y b b x 0 1 Y y ax ( b 1) b Y b y ax ( 0 b 1) X Positive Linear Relationship X Positive Increasing Curve-linear Relationship X Positive Decreasing Curve-linear Relationship Y Y y b b x 0 1 y ax ( b 1) b X Negative Linear Relationship Negative Increasing X Curve-linear Relationship 21
22 Preparation of the Historical Data (CER) Step 1: Select the dependent variable to be estimated (e.g. results = hours, cost) Step 2: Determine the auditable source of historical data (both cost and technical measures) Step 3: Define the population(s), subpopulations, and sample(s) of data considering methods of collection (sites, programs, stratifications) Step 4: Select logical and measurable independent variables and determine their auditable sources of data (SLOCs, Drawings, Pages, Hours) Step 5: Extract the data and verify its accuracy and completeness using tests of reasonableness Step 6: Prepare the data for use in EXCEL or a statistical tool to perform the regression analysis Step 7: Test the resulting relationship for accuracy 22
23 Interpretation Decision Criteria of Statistical Results Statistical Measure Purpose of Measure Target Value for Good Fit Correlation Coefficient Coefficient of Variation Usability Criterion Measures the fit of the CER model. Gives a quick look at the percentage of the total variation that has been explained by the regression line. Measures the target goal for predictability within one standard error (68% of the data). The C.V. is calculated at the mean of Y, allowing the comparison of one CER to another. Calculated from the Standard error which measures the dispersion of the data. Judges reliability or usefulness of the CER measured on data points in analysis determining confidence of the results based on F-test when R 2 should be.80. R < R 2 < R 2 C.V. 50% 50% < C.V. < 100% C.V. 100% Population 10 6 Pop < 10 Pop < 6 Meets guidelines Below guidelines, acceptable with proper understanding Below guidelines 23
24 Data Driven Estimating - Examples We are all confronted in our daily lives with estimates for products and services E.g. Home remodel, auto repair We evaluate the credibility of these estimates based upon the content of the estimate We need to apply this customer mentality when we review our own proposal cost volumes and BOEs Example: Everyone has dealt with an automobile repair estimate that we struggle to feel comfortable with. Evaluate the following BOEs for automobile repair service how do we as consumers know that we are getting a fair price for these services 24
25 D&G Auto Poor, Fair or Good BOE Criteria Description of the products or services delivered and the tasks to be performed Estimated labor resources to perform the defined tasks Time-phasing of labor resources The estimating method, rationale, assumptions, and computations used to develop the estimate Identification of the relevant historical data and comparability of the historical program to the current program Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers) 25
26 D&G Auto Poor, Fair or Good BOE Criteria Description of the products or services delivered and the tasks to be performed Estimated labor resources to perform the defined tasks Time-phasing of labor resources The estimating method, rationale, assumptions, and computations used to develop the estimate Identification of the relevant historical data and comparability of the historical program to the current program Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers) 26
27 D&G Auto Poor, Fair or Good BOE Criteria Description of the products or services delivered and the tasks to be performed Estimated labor resources to perform the defined tasks Time-phasing of labor resources The estimating method, rationale, assumptions, and computations used to develop the estimate Identification of the relevant historical data and comparability of the historical program to the current program Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers) 27
28 data driven Estimating The Challenges Ahead Focus Improve estimating products across all estimating stakeholders (e.g. BOEs, Earned Value, Cost Analysis and Program Execution (CAPE), Independent Cost Evaluation (ICE), Business Case Analysis) The debate as we evolve to our future At what level will we be generating estimates (e.g. subsystem, assembly) What scope measures (OSM) will we be using How will we document our estimating processes Same or different approaches for firm business, ROMs/NTE, EAC/ETC? What measures/approaches will be cost credible with our customers Historical databases, processes and strategies need to be flexible as we evolve toward the future improved estimating products Estimating, Cost Modeling and Engineering teams working together As the desired characteristics of these high quality estimating methods evolve our methods must evolve also. Immediate needs: Documented estimating process Good historical data Libraries of BOEs demonstrating data driven estimating approaches 28
29 Backup Charts/Supplemental Resources 29
30 Estimating Method - Judgment Judgment: Least preferred, least credible estimating method very subject to bias Generally not based upon historical data, scope measures or other sound estimating approaches Relies on Subject Matter Expert (SME) estimate s of small work packages aggregated to complete hardware, software, etc tasks State author s experience from which to make an estimate Programs worked; Years in industry Decompose effort into well defined tasks and subtasks Disclose if historical methods were investigated and not used, and why they were not used Rarely are we asked to quote something that does not have some similar historical likeness! 30
31 Estimating Method - LOE Level of Effort (LOE) - Use when customer specifies effort to be provide over a stated period of time on work that can be stated only in general terms Requirement for specific skills/talents that tend to be headcount driven Used when product cannot be well defined Contract is for delivery of hours not product. Payment based on effort expended not results achieved. Restricted LOE Contracts Statement of Work (SOW) that specifically calls out LOE by customer What is NOT LOE Level loading of labor not specifically directed by customer Staffing a position (i.e. Program Manager) as 160 hrs/month during period of performance on a completion type contract 31
32 Estimating Method Analogy/Comparison Analogy/Comparison: This method is based on historical context or frame of reference from a similar effort. Compares items being estimated to items of similar configuration and known cost to produce estimate Includes use of historical data Addresses differences between historical data and proposed solution being estimated May use relevant estimating attributes and complexity or adjustment factors (modifiers) to reflect programmatic, technology, requirements or technical differences Clearly document in BOE why and how factors are used and validated Estimated Cost (New Design) = Historical Cost (Similar Design) + Complexity or Scaling Factor 32
33 Estimating Method Standard Time Labor standard - time necessary to complete a defined element of work following a prescribed method, for an average (qualified) worker working at a 100% efficiency with no hindrances, breaks, delays or fatigue over extended periods of time Realization/Performance factors recognize actual experience measured against the standards This method is most often used in manufacturing or production related estimates. Examples: Manufacturing operation: Time required to complete a drilling operation on a product adapter Car servicing: Factory maintenance manual sets a standard number of minutes for each type of repair 33
34 Estimating Method Unit Method Unit Method: Relies on an accumulation of past experience divided by cost driver to produce cost per unit Example: Labor cost of developing and fielding entire ground station and costing using number of ground stations to be fabricated Example: ABC widget has 11 square feet of surface area. Actual design hours for XYZ widget (10 square feet of surface area) were 80 hours. 80 hours/10 square feet = 8 hours per square foot. Therefore ABC widget will take 88 (8 X 11) hours to design A Simple Ratio (not statistically validated) of X to Y Is A Unit Method 34
35 Estimating Method Follow-on Effort (Actuals) Follow-on based upon program actuals Commonly used when developing an estimate of labor or material for continuing efforts (usually) Production hardware Spare parts, Repairs, or Retrofit Sustaining Engineering Estimate At Complete (EAC) Definition: An estimate that uses historical data (hours, material dollars, weights, etc.) to project continuing future efforts of the same product line, process or part. 35
36 Estimating Method Parametric Model Parametric Model: Utilizes mathematical calibrated models that derive project cost or effort from attributes / metrics of project Source lines of code (SLOC), function points (FPs), COTS product characteristics, Weight, Size The measurement is based on the technical, physical, or other end item characteristics Facilitates alignment of program baselines (e.g. IMS and Staffing plans) Customer RFP often dictates use Utilized by customer to evaluate BOEs and bid overall Used by ICE to evaluate bid position & performance risk Performed by an experienced cost modeler Commercial parametric models include: PRICE, SEER 36
37 Estimating Method Parametric Model, Cont Parametric Model: is a group of cost estimating relationships used together to estimate entire cost proposals or significant portions thereof. These models are often computerized and may include many interrelated CER s both cost-to-cost and cost-to-non-cost estimating techniques estimating relationships to predict complex Some models use a very limited number of independent estimated values and a series of Parametric inter-related cost-to-noncost estimating techniques relationships to predict complex proposal cost structures 37
38 Estimating Method - CER Cost Estimating Relationships: Statistical estimating methods that produce mathematically fitted functions Developed by mathematically relating effort or time estimates to OSM, traits or other feature of product or manufacturing environment Relationship may be based on single or multiple measures (independent variables) representing estimating attributes, complexity measures, or other measures of effort. The results from application of the CERs (dependent variable) is frequently a labor estimate expressed in hours/staff-months or an estimate of time in days/weeks/months. Example: Based on actuals of four projects (design hrs/ sq ft) using linear regression, this estimating relationship is created: Design Hours = 9.32 * sq ft
39 Hardware Cost Modeling Estimate system acquisition costs based upon: Quantitative parameters such as complexity, quantity, weight, and size; and Quantitative parameters such as environmental specification, type of packaging, and level of integration; and schedule parameters such as months to first prototype, manufacturing rate, and amount of new design Electronic hardware items are characterized by the electronic application and type of componentry Mechanical and structural elements can be described in terms of their construction, type of material, functionality, machinability and manufacturing process 39
40 data driven Estimating Why Basis of Estimate (BOE) Focus? BOE - Description data driven Estimating - Definition Fundamentals of data driven Estimating data driven Estimate Examples We are the Customer Non-Proprietary BOE Training Material Available to All 40
41 Background/Objectives Background: There is a significant industry effort to increase the quality of contractor estimating processes. The industry has applied the term data driven estimating to this new method of developing and documenting our estimates/boes. data driven Estimating methods have certain characteristics that are starting to emerge as standards within our industry. Objectives: What is driving our industry toward the use of data driven estimating methods and why is this a business imperative Be able to identify the characteristics of good data driven estimating processes and how they are represented in our BOEs Provide examples of data driven versus non-data driven BOEs Raise the bar by providing access to non-proprietary BOE training 41
42 Mapping of DFARS Cost Estimating System Requirements to Major Estimating System Components 13. Provide for internal review of, and accountability for, the acceptability of the estimating system, including the budgetary data supporting indirect cost estimates and comparisons of projected results to actual results, and an analysis of any differences. Roles 1. Establish clear responsibility for preparation, review, and approval of cost estimates and budgets. 2. Provide a written description of the organization and duties of the personnel responsible for preparing, reviewing, and approving cost estimates and budgets. 12. Require management review, including verification of the estimating and budgeting policies, procedures, and practices. Internal Controls Responsibilities 3. Ensure that relevant personnel have sufficient training, experience, and guidance to perform estimating and budgeting tasks in accordance with the established procedures. Estimate Approval 5. Provide for adequate supervision throughout the estimating and budgeting process. 6. Provide for consistent application of estimating and budgeting techniques. 7. Provide for detection and timely correction of errors. 8. Protect against cost duplication and omissions. 10. Require use of appropriate analytical methods. 14. Provide procedures to update cost estimates and notify the Contracting Officer in a timely manner throughout the negotiation process. 16. Provide estimating and budgeting practices that consistently generate sound proposals that are compliant with the provisions of the solicitation and are adequate to serve as basis to reach a fair and reasonable price. 17. Have an adequate system description, including policies, procedures, and estimating and budgeting practices, that comply with the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement. Estimate Review Estimate Preparation Estimate Data Sources Training 4. Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets. 9. Provide for the use of historical experience, including historical vendor pricing information, where appropriate. 11. Integrate information available from other management systems. 15. Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable. 4 Well Supported Data Driven Estimating in Key to Estimating System Health 42
43 DFARS Cost Estimating System Requirements 4. Identify and document the sources of data and the estimating methods and rationale used in developing cost estimates and budgets. 9. Provide for the use of historical experience, including historical vendor pricing information, where appropriate. 11. Integrate information available from other management systems. 6. Provide for consistent application of estimating and budgeting techniques. 10. Require use of appropriate analytical methods. 15. Provide procedures that ensure subcontract prices are reasonable based on a documented review and analysis provided with the prime proposal, when practicable. Provide for adequate supervision throughout the estimating and budgeting process. Estimate Data Sources Estimate Preparation Estimate Review 43
44 Purpose of a BOE Documents and substantiates effort required to perform tasks Documents basis and assumptions Details understanding of requirements Conveys credibility by clearly defining tasks and assumptions Demonstrates that estimates are credible, defendable and realistic (historical data) Provides evidence that estimate is consistent with other proposal volumes Time-phased resources consistent with IMP/IMS Allocates resources such as skill mix appropriate for task Aligns with Work Breakdown Structure (WBS), SOW, and other product structures Meet customer funding, timing, and/or scheduling constraints 44
45 BOEs The Components A BOE consists of the following main elements: 1. Work Element ID WBS, SOW, Location, CLIN, NRE/Rec 2. Description of the products or services delivered and the tasks to be performed 3. Estimating method, rationale, assumptions, and computations used to develop the estimate Method of quoting 4. Relevant historical data and comparability of the historical data to the current program Explanation of any adjustments made to historical cost or programmatic data (i.e., use of modifiers) 5. Estimated labor and ODC resources to perform the defined tasks 6. Expenditure profile (time-phasing) of resources 7. Other elements as required by the cost proposal instructions 45
46 BOE Estimating Methods Estimates are developed using one or more of these typical estimating methods Judgment based on subject matter expert (SME) experience with similar efforts. Level-of-Effort/Support - uses resource-based estimates that are often based on a predetermined level of support for a given period of time Analogy/Comparison Method estimate based on historical frame of reference from a similar effort with adjustments for programmatic or technical differences Standard Time Estimating - Labor Standards and Realization/ Performance Factors - time necessary to complete a defined element of work following a prescribed (recognized) technique or method. Unit Method generally a simple ratio (not always statistically validated) Follow-on Based On Actuals extrapolation of actuals (CEAC, production) Parametric Cost Models use of calibrated, internally developed, or commercially available cost models Cost Estimating Relationships (CERs) - equations developed based on historical, statistically correlated relationships (See full training package for more details) 46
47 Customer Expectations Customers/DCAA/DCMA expect contractors to begin adopting improved estimating methods to support TINA-compliant proposals and other estimating products Least Preferred Judgment Most Preferred Fact Based Estimating Methods Move To Fact Based ASAP 47
48 BOE Methodologies Credibility Scale Customer Credibility High Low Labor Follow-on Projections based on Actuals Analogy/Comparison CER s Parametric Estimates / Cost Models Standards Unit Methods Level Of Effort Judgment Historical resource based estimates are easier to support than judgmental type estimates. 48
49 Customer Consequences Poor Estimating Failure to be responsive to customer estimating processes could be grounds for Non-compliance with RFP or government regulations Rejection of proposal and possibly defective pricing Holdbacks Penalties for bad BOEs include: Questioned costs DCAA/Government Flash Report Negotiation losses Negative factor in competitive evaluations Defective pricing and fraud prosecution Suspension of bidding and debarment 49
50 data driven BOEs Basic Definition Fact Based Estimating: Estimating processes/boes that provide the supporting information and rationale necessary for accurate, credible and traceable estimates Information includes: Use of defined estimating methods Historical data from past programs Use of objective scope measures and complexity factors where appropriate Rationale addresses: Complete description of the estimating approach Explanation of the applicability of the historical information Estimating process must be documented, consistent and repeatable 50
51 Fact Based BOEs The Fundamentals 1. Estimate developed using a defined and documented estimating processes 2. Methods rely on scope measures (attributes) and complexity factors (parametrics) when applicable 3. Complete estimate development process is explained and documented in the BOE Professional judgments, modifiers/adjustment factors, and historical context is adequately explained Process is consistent and repeatable 4. Estimate relies on relevant historical data and context that is accessible and verifiable Shows use of latest and most current relevant data Supports all resource estimates with verifiable data 5. Estimating processes are monitored for learning and improvement as required to support future estimates 51
52 Objective Work-Scope Measures (OSM) Scope Measures = estimating predictive measures technical measures relating to the size of the effort (attributes) Generally physical and technical measures Sized at different product levels - system, subsystem, box, etc. Example: Work Products: # drawings by type, sheet counts, # documents & sheet count, SLOC by category, # requirements, # reviews, # test & type, weight, quantities, etc. Traits/Complexity Factors: characteristics that can improve the accuracy of relationships developed for the OSMs Example: Scope Measures = Drawings Traits: (numeric and descriptive): # Drawings, Type of Drawing; Drawing Heritage; Item Next Higher Assembly/Element; Number of Sheets; # of Revisions Example: Scope Measures = Source Lines of Code (SLOC) Traits: Software Language; New SLOC; Modified SLOC; Reuse SLOC; Actual Total Delivered SLOC, Deleted SLOC, Auto Generated Code, etc 52
53 Why Use Scope Measures and Traits in BOEs? Scope Measures - help to scope the size of the effort to be estimated Objective quantitative measures of scope to be performed Directly correlate to (predictors) resource estimates for work products and tasks Understanding the size (scope) of the proposed effort is critical to developing an accurate estimate (BOE) The BOE author should explain the derivation, source, and application of the scope measures in the BOE Traits/Complexity Modifiers address unique characteristics of the scope measures that can affect required effort to produce Capture the effects of technical and programmatic differences between historical programs and the current proposed effort. The produceability can vary widely for any scope measure - traits can be used to explain the differences Sometimes requires judgment 53
54 Explaining Adjustment to a BOE In order to use an objective or subjective adjustment, an estimator must explain it thoroughly. Provided below are some examples of adjustments that may be required to improve the accuracy of the estimate. The list is not meant to be totally inclusive of adjustments an estimator has available in their tool box. Technical Complexity ~ Labor Estimate Improvement Curves ~ Labor Estimate Schedule adjustments ~ Labor Estimate Quote Decrement ~ Material Estimate Quantity Adjustment ~ Material Estimate Escalation ~ Material Estimate Exchange Rate ~ Material Estimate 54
55 Using Historical Data in a BOE Use of Historical Data Must Address: Source of data generally from the organization s certified historical data repositories Complete identification of the nature, source and details of relevant historical data Ensure compliance with A.C.T.T.R. by documenting the following: Actual historical hours Charge numbers included Technical data used Time frame of historical data Repository or source of historical data 55
56 data driven Estimating - Examples We are all confronted in our daily lives with estimates for products and services E.g. Home remodel, auto repair We evaluate the credibility of these estimates based upon the content of the estimate We need to apply this customer mentality when we review our own proposal cost volumes and BOEs Example: Everyone has dealt with an automobile repair estimate that we struggle to feel comfortable with. Evaluate the following BOEs for automobile repair service how do we as consumers know that we are getting a fair price for these services 56
57 D&G Auto Poor, Fair or Good BOE Criteria Description of the products or services delivered and the tasks to be performed Estimated labor resources to perform the defined tasks Time-phasing of labor resources The estimating method, rationale, assumptions, and computations used to develop the estimate Identification of the relevant historical data and comparability of the historical program to the current program Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers) 57
58 D&G Auto Poor, Fair or Good BOE Criteria Description of the products or services delivered and the tasks to be performed Estimated labor resources to perform the defined tasks Time-phasing of labor resources The estimating method, rationale, assumptions, and computations used to develop the estimate Identification of the relevant historical data and comparability of the historical program to the current program Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers) 58
59 D&G Auto Poor, Fair or Good BOE Criteria Description of the products or services delivered and the tasks to be performed Estimated labor resources to perform the defined tasks Time-phasing of labor resources The estimating method, rationale, assumptions, and computations used to develop the estimate Identification of the relevant historical data and comparability of the historical program to the current program Explanation of any adjustments made to the historical data or estimate (i.e., use of modifiers) 59
60 data driven Estimating The Challenges Ahead Focus Improve estimating products across all estimating stakeholders (e.g. BOEs, EV, CAPE, ICE, Business Case Analysis) The debate as we evolve to our future At what level will we be generating estimates (e.g. subsystem, assembly) How will we document our estimating processes Historical databases, processes and strategies need to be flexible as we evolve toward the future improved estimating products Estimating, Cost Modeling and Engineering teams working together As the desired characteristics of these high quality estimating methods evolve our methods must evolve also. Immediate needs: Documented estimating process Good historical data Libraries of BOEs demonstrating data driven estimating approaches 60
61 Back Up 61
62 Estimating Method Analogy/Comparison Analogy/Comparison: This method is based on historical context or frame of reference from a similar effort. Compares items being estimated to items of similar configuration and known cost to produce estimate Includes use of historical data Addresses differences between historical data and proposed solution being estimated May use relevant estimating attributes and complexity or adjustment factors (modifiers) to reflect programmatic, technology, requirements or technical differences Clearly document in BOE why and how factors are used and validated Estimated Cost (New Design) = Historical Cost (Similar Design) + Complexity or Scaling Factor 62
63 Estimating Method - Judgment Judgment: Least preferred, least credible estimating method very subject to bias Generally not based upon historical data, scope measures or other sound estimating approaches Relies on SME estimate s of small work packages aggregated to complete hardware, software, etc tasks State author s experience from which to make an estimate Programs worked; Years in industry Decompose effort into well defined tasks and subtasks Disclose if historical methods were investigated and not used, and why they were not used Rarely are we asked to quote something that does not have some similar historical likeness! 63
64 Estimating Method - LOE Level of Effort - Use when customer specifies effort to be provide over a stated period of time on work that can be stated only in general terms Requirement for specific skills/talents that tend to be headcount driven Used when product cannot be well defined Contract is for delivery of hours not product. Payment based on effort expended not results achieved. Restricted LOE Contracts SOW that specifically calls out LOE by customer What is NOT LOE Level loading of labor not specifically directed by customer Staffing a position (i.e. Program Manager) as 155 hrs/month during period of performance on a completion type contract 64
1.0 BASIS OF ESTIMATE (BOE) HOW TO
1.0 BASIS OF ESTIMATE (BOE) HOW TO 1.1 Definition A Basis of Estimate (BOE) is a document that identifies the logic, data, methodology and calculations used to estimate the resources required to perform
More informationIntroduction to Cost Estimation - Part I
Introduction to Cost Estimation - Part I Best Practice Checklists Best Practice 1: Estimate Purpose and Scope The estimate s purpose is clearly defined The estimate s scope is clearly defined The level
More informationMORS Introduction to Cost Estimation (Part I)
MORS Introduction to Cost Estimation (Part I) Mr. Huu M. Hoang Slide 1 Disclosure Form Slide 2 Module Four Module #(s) Items covered Presenter Start Stop One (1) Overview and Background (1) Define Estimate
More informationRisk Based BOE Analysis PMAG Approach
2010 ISPA/SCEA Joint Annual Conference Risk Based BOE Analysis PMAG Approach Mr Imran Ahmed Dr David L Wang 1 Dr Mun Kwon 02/Apr/2010 1 Agenda Introduction Common Challenges and Findings PMAG Approach
More informationMORS Introduction to Cost Estimation (Part I)
MORS Introduction to Cost Estimation (Part I) Mr. Huu M. Hoang Slide 1 Disclosure Form Slide 2 Learning Objectives of Module Two 1. Understand how to define a program using the various documents and other
More informationSoftware Technology Conference
30 April 2003 Costing COTS Integration Software Technology Conference Salt Lake City Linda Brooks 1 Objective Provide a roadmap for doing an estimate for a Commercial Off-the-Shelf (COTS) software intensive
More informationHow to Prepare a DCAA Compliant Cost or Price Proposal. Bag Lunch Webinar November 17, 2016
How to Prepare a DCAA Compliant Cost or Price Proposal Bag Lunch Webinar November 17, 2016 Presenter: Paul H. Calabrese Rubino & Company, CPAs & Consultants Senior Manager Tel: 301 214 4137 pcalabrese@rubino.com
More informationPT 03 - Linear Regression
Parametric Cost Estimating Training Track For The Parametric Estimating Handbook Chapter 8 (PT 11) Other Parametric Applications Presenter: David Eck 1 Contents Objective Overview Tailoring Applications
More informationContractor Estimating Systems: System Adequacy Standards and Key Compliance Issues
McKenna Government Contracts, continuing excellence in Dentons Contractor Estimating Systems: System Adequacy Standards and Key Compliance Issues Joe Martinez, Esq. Dentons US LLP Michael Steen, CPA Redstone
More informationCost Estimating Documentation Guide
Cost Estimating Documentation Guide Prepared by Deputy Assistant Secretary of the Navy (Cost & Economics) Naval Center for Cost Analysis Purpose: Provide Naval Center for Cost Analysis (NCCA) policy on
More informationUNCLASSIFIED SE/IT/PM Factor Development Using Trend Analysis 4/2/2007
SE/IT/PM Factor Development Using Trend Analysis 4/2/2007 Funded by the NRO Cost Group Alex Wekluk Raymond.Wekluk@ngc.com (703) 633.2173 Nathan Menton nmenton@mcri.com (703) 584.7129 1 Abstract SE/IT/PM
More informationTOPIC DESCRIPTION SUPPLEMENT for the SYSTEMS ENGINEERING SURVEY DESCRIPTION
1 2 Objectives of Systems Engineering 3 4 5 6 7 8 DoD Policies, Regulations, & Guidance on Systems Engineering Roles of Systems Engineering in an Acquisition Program Who performs on an Acquisition Program
More information25 January 2010 Software Maintenance Cost Worksheet
WBS Title Description Government Or Contractor 1.0 Operations, Maintenance & This entry collects the total cost associated with maintaining a system after it has been accepted by the customer (DD-250).
More informationLessons Learned in Estimating the Software Cost of a Ground Station with COTS Integration. Kathy Bradford 22 February 2001
Lessons Learned in Estimating the Software Cost of a Ground Station with COTS Integration Kathy Bradford 22 February 2001 1 Short History of an Integrated COTS Procurement RFP requested a mostly COTS ground
More informationCLB 029 Wrap Rate Projections
CLB 029 Wrap Rate Projections Lesson Defense Acquisition University Table of Contents Wrap Rate Projections... 3 Introduction... 3 Wrap Rate Projection Methods... 4 Wrap Rate Projections Applied... 6 Summary...
More informationREQUIREMENTS DOCUMENTATION
REQUIREMENTS DOCUMENTATION Project Title: Date Prepared: Stakeholder Requirement Category Priority Acceptance Criteria REQUIREMENTS DOCUMENTATION Project Title: Date Prepared: Stakeholder Requirement Category
More informationANALYSIS OF PARAMETRIC AND DATABASE DRIVEN COST ESTIMATES IN THE TRANSIT INDUSTRY
ANALYSIS OF PARAMETRIC AND DATABASE DRIVEN COST ESTIMATES IN THE TRANSIT INDUSTRY L. Brian Ehrler Project Management Oversight Cost and Risk Manager Burns Engineering, Inc. 4925 Greenville Ave Dallas,
More informationBOEMax Feature List. Configuration, Setting Up Core and Historical Data ProjStream 1
BOEMax List Configuration, Setting Up Core and Historical Data Set up an unlimited number of user defined fields to support project unique data views and reporting requirements Import from anywhere using
More informationExploring the Estimation of Production Breaks Using EVM. Vincent Sipple Naval Center for Cost Analysis
Exploring the Estimation of Production Breaks Using EVM Vincent Sipple Naval Center for Cost Analysis 1 Goal of the Brief This briefing explores using recurring manufacturing EVM data to help estimate
More informationDFARS Business Systems Compliance. March 11, 2015
DFARS Business Systems Compliance March 11, 2015 Speakers Steven Tremblay Ernst & Young Executive Director Government Contract Services Jack Gay Ernst & Young Senior Manager Government Contract Services
More information73R-13: Basis of Estimate
73R-13: Basis of Estimate As Applied for the Software Services Industries Slides are used with permission from Nesma, All rights reserved. Acknowledgements For transforming AACEi 34R-05: Basis of Estimate
More informationSTUDENT GUIDE. CON 170 Fundamentals of Cost & Price Analysis. Unit 5, Lesson 1 Cost Analysis Process, Players, and Business Systems
STUDENT GUIDE CON 170 Fundamentals of Cost & Price Analysis Unit 5, Lesson 1 Cost Analysis Process, Players, and Business Systems January 2017 CON170, Unit 5 Lesson 1 Cost Analysis Process and Players
More informationLVC-IA EC WBS/Dictionary Dictionary
WBS 1.0 LVC-IA EC 1.1 LVC-IA Products 1.1.1 Live Integration Products 1.1.2 Virtual Integration Products 1.1.3 Constructive Integration Products 1.1.4 Gaming Integration Products 1.1.5 Integrated Architecture
More informationCMMI V2.0 MODEL AT-A-GLANCE. Including the following views: Development Services Supplier Management. CMMI V2.0 outline BOOKLET FOR print.
CMMI V.0 MODEL AT-A-GLANCE Including the following views: Development Services Supplier Management CMMI V.0 outline BOOKLET FOR print.indd CMMI V.0 An Integrated Product Suite Designed to meet the challenges
More informationLVC-IA EC WBS/Dictionary Dictionary
WBS 1.0 LVC-IA EC 1.1 LVC-IA Products 1.1.1 Live Integration Products 1.1.2 Virtual Integration Products 1.1.3 Constructive Integration Products 1.1.4 Gaming Integration Products 1.1.5 Integrated Architecture
More informationInternational Diploma in Project Management. (Level 4) Course Structure & Contents
Brentwood Open Learning College (Level 4) Page 1 Unit 1 Overview of Project Management The unit 1 covers the following topics: What is A Project? What is Project Management? Project Constraints Tools and
More informationMORS Introduction to Cost Estimation (Part I)
MORS Introduction to Cost Estimation (Part I) Mr. Huu M. Hoang Slide 1 Disclosure Form Slide 2 Module Three Module #(s) Items covered Presenter Start Stop One (1) Overview and Background (1) Define Estimate
More informationSTATEMENT OF WORK SMALL SPACECRAFT PROTOTYPING ENGINEERING DEVELOPMENT & INTEGRATION (SSPEDI) Space Solutions (SpS)
SSPEDI SpS J.1(a), Attachment 1 80ARC018R0007 National Aeronautics and Space Administration Ames Research Center Moffett Field, CA 94035-0001 STATEMENT OF WORK SMALL SPACECRAFT PROTOTYPING ENGINEERING
More informationCMMI-DEV V1.3 CMMI for Development Version 1.3 Quick Reference Guide
processlabs CMMI-DEV V1.3 CMMI for Development Version 1.3 Quick Reference Guide CMMI-DEV V1.3 Process Areas Alphabetically by Process Area Acronym processlabs CAR - Causal Analysis and Resolution...
More informationReport of the Reliability Improvement Working Group (RIWG) Volume II - Appendices
Report of the Reliability Improvement Working Group (RIWG) Volume II - Appendices Appendix 1 Formulate Programs with a RAM Growth Program II-1 1.1 Reliability Improvement Policy II-3 1.2 Sample Reliability
More informationCMMI-SVC V1.3 CMMI for Services Version 1.3 Quick Reference Guide
processlabs CMMI-SVC V1.3 CMMI for Services Version 1.3 Quick Reference Guide CMMI-SVC V1.3 Process Areas Alphabetically by Process Area Acronym processlabs CAM - Capacity and Availability Management...
More informationRisk assessment checklist - Acquire and implement
Check Yes or No or N/A (where not applicable). Where a No is indicated, some action may be required to rectify the situation. Cross-references (e.g., See FN 1.01) point to the relevant policy in the First
More informationChapter 6 Field Work Standards for Performance Audits
Chapter 6 Field Work Standards for Performance Audits Introduction 6.01 This chapter contains field work requirements and guidance for performance audits conducted in accordance with generally accepted
More informationHow Raytheon Meets the Challenge of Developing Systems with Evolving Requirements
How Raytheon Meets the Challenge of Developing Systems with Evolving Requirements Linda Martz Raytheon IIS Rocky Mountain Engineering Director April 2, 2008 Copyright 2008 Raytheon Company. All rights
More informationDefective Pricing- Material Pricing Deficiencies. Next Slide
Defective Pricing- Material Pricing Deficiencies Table of Contents Risk Assessment Research and Planning Risk Assessment - Initial Data Obtained from Contractor Risk Assessment Overrun or Underrun Analysis
More informationIndependent Government Estimates A Fact Sheet
1. Introduction Independent Government Estimates A Fact Sheet An Independent Government Estimate (IGE) is an unbiased cost estimate based upon the specifications and without the influence of potential
More information8.0 PRE-ENVIRONMENTAL REVIEW (PER)
GSFC-STD-1001 Page 36 of 52 8.0 PRE-ENVIRONMENTAL REVIEW (PER) At the PER, the project discloses to the Integrated Independent Review Team (IIRT) the complete and comprehensive project status in order
More informationDRAFT. Effort = A * Size B * EM. (1) Effort in person-months A - calibrated constant B - scale factor EM - effort multiplier from cost factors
1.1. Cost Estimation Models Parametric cost models used in avionics, space, ground, and shipboard platforms by the services are generally based on the common effort formula shown in Equation 1. Size of
More informationM3 Playbook Guidance. 1.1 Establish Initial Customer PMO and Processes. Human Resources (HR)/Staffing Plan
M3 Playbook Guidance Phase 1: Readiness This guidance is intended for use by organizations to confirm and validate that their plans are comprehensive and have adequate level of detail for proper migration
More informationI n t e g r i t y - S e r v i c e - E x c e l l e n c e 1
Measurement in Cost Analysis Richard Hartley Deputy Assistant Secretary of the Air Force (Cost and Economics) 23 Jun 2009 1 Tough Choices It is one thing to speak broadly about the need for budget discipline
More informationBASIS OF ESTIMATE AS APPLIED FOR THE SOFTWARE SERVICES INDUSTRIES TCM Framework: 7.3 Cost Estimating and Budgeting
AACE International Recommended Practice No. 74R-13 BASIS OF ESTIMATE AS APPLIED FOR THE SOFTWARE SERVICES INDUSTRIES TCM Framework: 7.3 Cost Estimating and Budgeting Rev. Note: As AACE International Recommended
More informationPrice Analysis and Reasonable Pricing. government contracting
Price Analysis and Reasonable Pricing 1 Bill Walter, CPA Managing Director DHG GovCon Advisory Bill.Walter@dhgllp.com 1410 Spring Hill Road, Suite 500 Tysons, VA 22102 (703) 970.0509 Steve Masiello Partner
More informationWM2012 Conference, February 26 March 1, 2012, Phoenix, Arizona, USA. Improving DOE Project Performance Using the DOD Integrated Master Plan 12481
Improving DOE Project Performance Using the DOD Integrated Master Plan 12481 Glen B. Alleman, DOD Programs, Project Time & Cost and Michael R. Nosbisch, Managing Principle, Western Region, Project Time
More informationPreparing for the Future: Electronic Data Delivery and the DCMA Automated Surveillance Process
Preparing for the Future: Electronic Data Delivery and the DCMA Automated Surveillance Process Dave Scott, BDO USA, dmscott@bdo.com Date: January 18, 2018 Learning Objectives Understand the new DCMA automated
More informationSoftware Maintenance, Sustaining Engineering, and Operational Support
Software Maintenance, Sustaining Engineering, and Operational Support Estimating Software Maintenance Costs for U.S. Army Weapons Systems Office of the Deputy Assistant Secretary of the Army for Cost and
More informationSummary of 47 project management processes (PMBOK Guide, 5 th edition, 2013)
Summary of 47 project management processes (PMBOK Guide, 5 th edition, 2013) Integration Management: processes & activities needed to properly coordinate all aspects of the project to meet stakeholder
More information19A NCAC 02E.0702 Solicitation and Award of Contract 19A NCAC 02E.0703 Prequalifying to Award - Professional Services Firms
FISCAL NOTE Rule Citation: Rule Topic: NCDOT Division: 19A NCAC 02E.0702 Solicitation and Award of Contract 19A NCAC 02E.0703 Prequalifying to Award - Professional Services Firms Revision of the Requirements
More informationIncreasing Bid Success Through Integrated Knowledge Management
Increasing Bid Success Through Integrated Knowledge Management CIMdata Commentary Key takeaways: Program success begins in the proposal phase when the focus is on both minimizing risk and defining the
More informationCommand and Control Software Development Lessons Learned. Lt Col Michael D. Sarchet Deputy Director, Space Systems Command and Control Division
Command and Control Software Development Lessons Learned Lt Col Michael D. Sarchet Deputy Director, Space Systems Command and Control Division 1 UNCLASSIFIED Agenda Two real world case studies Lessons
More informationProject+ Examination Blueprint Version June 1, 2003
Introduction The Project + examination is designed for business professionals involved with projects with a technology component. The examination is designed for candidates possessing at least 12 months
More informationRequirements for Estimation Purposes
Requirements for Estimation Purposes William Roetzheim william@level4ventures.com (619) 917-4917 www.level4ventures.com 1 Instructor Introduction Mr. William H. Roetzheim Lead Scientist, Founder & CEO,
More informationRequirements for Estimation Purposes
Requirements for Estimation Purposes William Roetzheim william@level4ventures.com (619) 917-4917 www.level4ventures.com 1 Instructor Introduction Mr. William H. Roetzheim Lead Scientist, Founder & CEO,
More informationUnderstanding Cost & Affordability Using SEER as a Common Benchmark, Driven By Facts
Understanding Cost & Affordability Using SEER as a Common Benchmark, Driven By Facts Dan Galorath galorath@galorath.com Copyright 2012 Galorath Incorporated 2011 Copyright Galorath Incorporated 2 An Estimate
More informationShipyard Cost Estimating (Using PERCEPTION Version 7)
Shipyard Cost Estimating (Using PERCEPTION Version 7) Phone: (410) 263-8593 Fax: (410) 266-0503 Email: Info@SPARUSA.com Web Site: www.sparusa.com EM-GD-009 Rev. 06/12/2003 PERCEPTION is a registered trademark
More informationDetermining the to be cost of using COTS SW
Determining the to be cost of using COTS SW Lt April Morgan 3/27/2003 4:22 PM March 2006 TSAT Program MILSATCOM Joint Program Office OVERVIEW Background/Goal of Work Methodology Government Challenge Big
More informationProject Time Management
Project Time Management Project Time Management Project Time Management includes the processes required to manage timely completion of the project. Plan schedule management The process of establishing
More informationPrinciples of Contract Pricing
Principles of Contract Pricing Breakout Session # F09 Janie L Maddox, Lecturer, U.S. Naval Postgraduate School Erin Phalen, Sr. Supply Chain Analyst BAE Systems Date July 29, 2014 Time 4:00pm 5:15pm Contract
More informationThe Role of the IGCE in Source Selection. Cobec Consulting Annette Barlia, Senior Consultant
The Role of the IGCE in Source Selection Cobec Consulting Annette Barlia, Senior Consultant Table of Contents Objective Background Defining the Independent Government Cost Estimate (IGCE) Acquisition Process
More informationDEPARTMENT OF DEFENSE STANDARD PRACTICE
NOT MEASUREMENT SENSITIVE 5 April 2012 SUPERSEDING 28 January 2008 DEPARTMENT OF DEFENSE STANDARD PRACTICE DOCUMENTATION OF VERIFICATION, VALIDATION, AND ACCREDITATION (VV&A) FOR MODELS AND SIMULATIONS
More informationSection M: Evaluation Factors for Award HQ R LRDR Section M: Evaluation Factors for Award For HQ R-0002
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 LRDR Section M: Evaluation Factors for Award For 1 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 Table of
More informationQUALITY ASSURANCE PLAN OKLAHOMA DEPARTMENT OF HUMAN SERVICES ENTERPRISE SYSTEM (MOSAIC PROJECT)
QUALITY ASSURANCE PLAN OKLAHOMA DEPARTMENT OF HUMAN SERVICES ENTERPRISE SYSTEM (MOSAIC PROJECT) MOSAIC Quality Assurance Plan v04.02 Prepared by: Approved by: QUALITY ASSURANCE PLAN APPROVALS QA/QC Program
More information"Change is inevitable; except in vending machines."
Configuration Management Change is inevitable. In acquisition programs, missions, requirements, technologies, and environments change. In response, the system design will change as it evolves through the
More informationGROUND RULES AND PRICING INSTRUCTIONS:
GROUND RULES AND PRICING INSTRUCTIONS: F-22 REPLENISHMENT SPARES PRICING CY2013 CY2017 REQUIREMENTS CONTRACT (MRC) WITH VARIABLE QUANTITIES I. GENERAL INSTRUCTIONS: The objective of this Spares Requirements
More informationAppendix. Process Inputs and Outputs
3601cA.fm Page 493 Wednesday, October 26, 2005 12:47 AM Appendix A Process and Throughout this book, PMP: Management Professional Study Guide, 3rd Edition, I ve discussed the inputs and outputs to the
More informationCompiled by Rajan Humagai
Compiled by Rajan Humagai www.pavementengineering.com.au Project Processes Inputs, Tools & Techniques and Outputs. (Based on PMBOK 5 th Edition) 1. Project Integration 1. Develop 1. Project Statement Of
More informationBest Practices for a FAR 15 Procurement PART 3 PREPARING AWARD DECISION DOCUMENTS
Best Practices for a FAR 15 Procurement PART 3 PREPARING AWARD DECISION DOCUMENTS Agenda Prepare Source Selection Decision Document Price Analysis Typical FAR Part 15 Source Selection Process Source: DISA
More informationMake or Buy Decisions in a Proposal Audit. Next Slide
Make or Buy Decisions in a Proposal Audit Table of Contents Risk Assessment Research and Planning Risk Assessment Initial Review of Proposal Risk Assessment Coordination with the PCO and ACO Preliminary
More informationTesting 2. Testing: Agenda. for Systems Validation. Testing for Systems Validation CONCEPT HEIDELBERG
CONCEPT HEIDELBERG GMP Compliance for January 16-17, 2003 at Istanbul, Turkey Testing for Systems Validation Dr.-Ing. Guenter Generlich guenter@generlich.de Testing 1 Testing: Agenda Techniques Principles
More informationDI-MGMT-XXXXX (Technical Data DID) April 13, 2017
DRAFT DATA ITEM DESCRIPTION DRAFT Title: Technical Data Report (DRAFT) Number: DI-MGMT-XXXXX Approval Date: DRAFT AMSC Number: XXXX Limitation: DTIC Applicable: No GIDEP Applicable: No Preparing Activity:
More informationGAO Best Practices Guides
GAO Best Practices Guides Karen Richey 2017 EVMP Forum August 24 th & 25 th 1 Learning Objectives Become familiar with various GAO best practices guides used to develop audit findings Introduction to Cost
More informationDEPARTMENT OF DEFENSE HANDBOOK ACQUISITION OF SOFTWARE ENVIRONMENTS AND SUPPORT SOFTWARE
NOT MEASUREMENT SENSITIVE MIL-HDBK-1467 10 DECEMBER 1997 SUPERSEDING SEE 6.2 DEPARTMENT OF DEFENSE HANDBOOK ACQUISITION OF SOFTWARE ENVIRONMENTS AND SUPPORT SOFTWARE This handbook is for guidance only.
More informationSample Reliability Language for DoD Acquisition Contracts
Sample Reliability Language for DoD Acquisition Contracts The single most important step necessary to correct high suitability failure rates is to ensure programs are formulated to execute a viable systems
More informationInstrument Control System Project Management
Project Documentation Document PMCS-0021 Revision A Instrument Control System Project Management Bret Goodrich Software September 2015 Advanced Technology Solar Telescope 950 N. Cherry Avenue Tucson, AZ
More informationRepetitive Bidding of Duplicative Material Costs. Next Slide
Repetitive Bidding of Duplicative Material Costs Table of Contents Risk Assessment Research and Planning Risk Assessment Initial Review of Subcontract Proposal Preliminary Analytical Procedures Entrance
More informationPresented at the 2008 SCEA-ISPA Joint Annual Conference and Training Workshop -
DEVELOPMENT AND PRODUCTION COST EQUATIONS DERIVED FROM PRICE-H TO ENABLE RAPID AIRCRAFT (MDO) TRADE STUDIES 2008 Society Cost Estimating Analysis (SCEA) Conference W. Thomas Harwick, Engineering Specialist
More informationá1058ñ ANALYTICAL INSTRUMENT QUALIFICATION
USP 41 General Information / á1058ñ 1 á1058ñ ANALYTICAL INSTRUMENT QUALIFICATION INTRODUCTION A large variety of analytical instruments, ranging from a simple apparatus to complex computerized systems,
More informationETASS II SKILL LEVEL AND LABOR CATEGORY DESCRIPTIONS. Skill Levels
ETASS II SKILL LEVEL AND LABOR CATEGORY DESCRIPTIONS Skill Levels Level Entry I Intermediate II Senior III Principal IV Knowledge/Skill Description Applies fundamental concepts, processes, practices, and
More informationCMMI for Acquisition Quick Reference
AGREEMENT MANAGEMENT PROJECT MANAGEMENT (ML2) The purpose of Agreement Management (AM) is to ensure that the supplier and the acquirer perform according to the terms of the supplier agreement. SG 1 The
More informationImprovement Curves: Beyond The Basics
Improvement Curves: Beyond The Basics September 20, 2017 Kurt Brunner Kurt.r.brunner@leidos.com 310.524.3151 Agenda Objective Background Learning Curves Do They Even Exist? What Can Impact Them? Theory
More informationThe Work Breakdown Structure in the Systems Engineering Process. Abstract. Introduction
The Work Breakdown Structure in the Systems Engineering Process Mark A. Wilson Strategy Bridge International, Inc. 9 North Loudoun Street, Suite 208 Winchester, VA 22601-4798 mwilson@strategybridgeintl.com
More informationLife Cycle Cost Estimate (LCCE)
Life Cycle Cost Estimate (LCCE) Assessments 2013 ICEAA Annual Conference New Orleans, Louisiana June 17-21, 2013 Casey D. Trail Colleen Craig David Brown Motivation A particular program support and cost
More informationNew Opportunities for System Architecture Measurement
New Opportunities for System Architecture Measurement System Engineering Conference October 2012 Paul Kohl Lockheed Martin Dr. Ronald S. Carson -- Boeing 1 Background The United States Government Accountability
More informationCode 40 Tomahawk Information System (TOMIS) Software Engineering Support Services Pre-Solicitation Conference
Code 40 Tomahawk Information System (TOMIS) Software Engineering Support Services Pre-Solicitation Conference NUWC Division Newport Undersea Collaboration & Technology Outreach Center (UCTOC) April 2,
More informationPrinciples of Verification, Validation, Quality Assurance, and Certification of M&S Applications
Introduction to Modeling and Simulation Principles of Verification, Validation, Quality Assurance, and Certification of M&S Applications OSMAN BALCI Professor Copyright Osman Balci Department of Computer
More informationUsing your EAC as a GPS 6 Characteristics of a Good EAC. Kim Koster, Senior Director IPM
Using your EAC as a GPS 6 Characteristics of a Good EAC Kim Koster, Senior Director IPM Top Ten Reasons Projects Fail BAD plans and inadequate planning processes Proposal not adequate Lack of scope comprehension
More informationWORK PLAN AND IV&V METHODOLOGY Information Technology - Independent Verification and Validation RFP No IVV-B
1. Work Plan & IV&V Methodology 1.1 Compass Solutions IV&V Approach The Compass Solutions Independent Verification and Validation approach is based on the Enterprise Performance Life Cycle (EPLC) framework
More informationProject Controls Expo
Project Controls Expo 09/10 Nov London 2011 What is Cost Engineering and Cost Estimating Carl Dalton Director Polaris Consulting Speaker Profile Carl Dalton Carl is a Fellow of the Association of Cost
More informationUnderstanding Manufacturing Execution Systems (MES)
Understanding Manufacturing Execution Systems (MES) What is a Manufacturing Execution System (MES)? AMR Research, a Boston-based industry and market analysis firm, defines a Manufacturing Executing System
More informationHeadquarters U.S. Air Force. Building CERs & SERs for Enterprise Resource Planning (ERP) Programs
Headquarters U.S. Air Force Building CERs & SERs for Enterprise Resource Planning (ERP) Programs Wilson Rosa, Travis Packard, James Bilbro Brian Kolstad, Max Hodal 2011 ISPA/SCEA Conference June 9, 2011
More informationPHASE 4 - Post-Award. Type of Feedback Type of Contract Feedback Category Feedback
PHASE 4 - Post-Award Type of Feedback Type of Feedback Category Feedback Commodity (Competitive) Use of the Bid Evaluation Model (BEM), when validated, is a best practice approach to evaluating competitive,
More informationCOST LOADED CONSTRUCTION SCHEDULES PART 1 - GENERAL
SECTION 01310 COST LOADED CONSTRUCTION SCHEDULES PART 1 - GENERAL 1.01 SCOPE: A. COST LOADED CONSTRUCTION SCHEDULE (Construction Schedule): The WORK under this Contract shall be planned, scheduled, executed,
More informationFlorida Department of Transportation Office of Freight, Logistics and Passenger Operations Part 4 QUALITY ASSURANCE PROVISIONS #TRIPS-17-CA-RFP
QUALITY ASSURANCE PROVISIONS #TRIPS-17-CA-RFP 4.1.0 CONTRACTORS IN-PLANT QUALITY ASSURANCE REQUIREMENTS 4.1.1 QUALITY ASSURANCE ORGANIZATION Manufacturer shall establish and maintain an effective in-plant
More informationIntegrating Systems Engineering, Risk and Earned Value Management
Integrating Systems Engineering, Risk and Earned Value Management CPM Spring Conference 2001 San Diego, CA 24 May 2001 Presented by: Paul Solomon Northrop Grumman Corp. (661) 540-0618 solompa@mail.northgrum.com
More informationBiometrics Enterprise Architecture Systems Engineering Management Plan (BMEA SEMP)
Biometrics Enterprise Architecture Systems Engineering Management Plan (BMEA SEMP) Version 1.0 Prepared by: Date: November 24, 2009 Revision History Purpose Revision Date Level 11/17/2009 First Draft 1.0
More informationDefense Cost & Resource Center (DCARC) Executive Overview Brief
Defense Cost & Resource Center (DCARC) Executive Overview Brief March 2011 Agenda Introduction Mission & Function Organizational Structure Infrastructure Key Products and Services Efficiency Innovations
More informationPSP A/ March 28, PSP-003(R)
DEFENSE CONTRACT AUDIT AGENCY DEPARTMENT OF DEFENSE 8725 JOHN J. KINGMAN ROAD, SUITE 2135 FORT BELVOIR, VA 22060-6219 IN REPLY REFER TO PSP 730.5.1.A/2012-004 March 28, 2013 13-PSP-003(R) MEMORANDUM FOR
More informationThis presentation will briefly discuss the three methods of estimating; analytical, analogous and parametric before studying parametric cost and
Abstract This presentation will briefly discuss the three methods of estimating; analytical, analogous and parametric before studying parametric cost and schedule estimating in more depth. It will consider
More informationQuality Assurance / Quality Control Plan
Quality Assurance / Quality Control Plan Table of Contents MANAGEMENT APPROACH... 3 SUBCONTRACT MANAGEMENT... 3 QUALITY MANAGEMENT APPROACH... 3 METHODOLOGY... 4 CONCEPT OF OPERATIONS... 5 QUALITY MANAGEMENT
More informationPerformance-Based Earned Value
Performance-Based Earned Value NDIA Systems Engineering Conference San Diego, CA October 25, 2006 Paul J. Solomon, PMP Performance-Based Earned Value Paul.Solomon@PB-EV.com 1 Agenda DoD Policy and Guidance,
More information