How can Parametrics Improve: the Missing Element

Transcription

1 How can Parametrics Improve: the Missing Element Paolo Ponzio European Space Agency SCAF Annual Conference 2011 Royal Institution of Naval Architects Victoria, London

2 About the Author Paolo Ponzio ISPA Certified Parametric Practitioneer 2000 Master Degree in Aerospace Engineering Production Control Lineaform srl (Italy) MTM Cost Analysis and Quality Mgmt System and Cost Wellstood BV / ESA (NL) System Engineering and Cost Estimating Principal Cost Astrium Ltd (UK) Bids Cost and Price compilation and vetting, Parametric Methodology Development Cost ESA-ESTEC (NL) Cost Estimating and Offers Price Analysis Parametric Models Development ECOS Mgr Deputy and Users Trainer Industrial Cost Analyst and ESA-ESTEC (NL) Industry Rates and Overheads analysis and negotiations

3 Contents Parametric Cost Analysis Additional Cost Analysis Rates and Overheads Improving Parametric Cost Estimating

4 The Three Elements of Cost Cost is founded on three core elements The effort required to develop and manufacture or execute The cost of a unit of effort The developer markup Focus of Parametric Estimating EFFORT EFFORT UNIT BASIC COST MARKUP Labour Facilities Material Rates Investments Purchasing Value Indirect Costs

5 Parametric Cost Analysis

6 The Parametric approach The parametric method uses regression analysis of a database of two or more similar systems to develop cost estimating relationships (CERs) which estimate cost based on one or more system performance or design characteristics (e.g., speed, range, weight, thrust). Source: acc.dau.mil (Defense Acquisition University) Parametric estimating is a technique that develops cost estimates based upon the examination and validation of the relationships which exist between a project's technical, programmatic, and cost characteristics as well as the resources consumed during its development, manufacture, maintenance, and/or modification. Source: ISPA Handbook, 4 th Ed.

7 Data Normalization Normalization is one of the most important keywords in Parametrics. Data normalization is a process whereby a CER builder attempts to correct for dissimilarities in historical data by putting the data into uniform format. (Source: ISPA Handbook, 4 th Ed.) Purpose: if all of the historical data comes from the same organization, if that organization is not changing or learning, if the organization is repetitively doing the same kind of work with no improvements in efficiency or technology, and if the national currency does not fluctuate in value, historical data would not need to be normalized.

8 No need for saying: if a CER or a Cost Model is derived from a set of data not adequately normalized it may be rubbish.

9 Typical Normalization System of Units (IS, US, British ) Very Simple for quantifiable parameters May be difficult for qualitative parameters (complexity, novelty ) Economic Conditions / Currency It seems very easy: FT, ECB, OECD and other sources provide us with tables of indices and their variations In reality the actual variation of costs relevant to our product may be significantly different from the averaged publications (see later) A lot of estimators stop there more experienced ones consider also Production quantity and rate / learning Quality levels Contract conditions

10 Neglected normalizations Producer Capabilities Was the product (with its declared characteristics) at the cutting edge of the producer technology? Or was it business as usual? Producer Productivity Was the producer workload optimized? Can we compare the number of hours and rates among the different producers of our dataset? Economic Environment Was the market at an upturn or downturn and can that have an effect on the product cost? Has the Purchasing Power of the firm changed?

11 The Lost Element in normalization Cost of the Unit of Effort Is the cost of 1 hour of labour comparable among the producers of our dataset? IF THE DATASET INCLUDES PRODUCTS FROM DIFFERENT PRODUCERS IT IS VERY LIKELY THAT THE BASIC COST OF 1 HOUR OF LABOUR IS NOT COMPARABLE IF THE DATASET INCLUDES PRODUCTS FROM A SINGLE PRODUCER IT IS STILL POSSIBLE THAT THE BASIC COST OF 1 HOUR OF LABOUR IS NOT COMPARABLE (see later)

12 Example Evaluation of offers for a common spacecraft equipment 5 bidders: Similar designs Similar level of novelty Similar experience / heritage 5 very different European companies Bidders 1, 4 and 5 are slightly non-compliant to requirements (minor cost effect ~2-300 k ) Other important consideration are irrelevant for our example

13 Example Price vs Mass k kg

14 Example Parametric Price Analysis result: Bidders 1, 3, 4 appear giving a fair offer (slightly above expected values). Bidder 2 seems still in line with the average offer, however a significantly cheaper price would be expected. Bidder 5 offers an unrealistic price. Has the Evaluation Board been advised properly?

15 Additional Cost Analysis

16 Example Bidders 2, 3, 5 offer a similar amount of man-hours Bidders 1, 4 offer much less man-hours (?) Hours Comparison Price and Hours k Hours k

17 Example Bidder 5 has a much higher Material cost Labour cost is not proportional to the number of hours Hours Comparison Price and Hours k Hours Material Labour

18 Example Although bidders have different rates, 1, 2, 4, 5 offer similar value-for-money Bidder 3 offers the best value for money? Hours Comparison Labour Price and Hours k Hours Labour

19 Example Although bidders have different rates, 1, 2, 4, 5 offer similar value-for-money Bidder 3 offers the best value for money? The sentences above would be valid if the bidders had labour rates based on the same productivity. Productivity: Average employee works 1700 hours per year (workable hours leaves, bank holidays and absences) Average employee spends 20% of his time on activities that cannot be invoiced to customers Average employee spends 1700*80%= 1360 hours per year on activities invoiced to customers, producing operating income.

20 Example The productivity analysis shows us a different picture: Bidder 3 offers by far the best value for money Bidder 1 and 4 declare to be able to complete the work with less than 50% resources than the other three. Is that credible? FTE Comparison Labour Price and FTE k Hours Labour

21 Example Although bidders have different rates, 1, 2, 4, 5 offer similar value-for-money Bidder 3 offers the best value for money Hours Comparison Labour Price and Hours k Hours Labour

22 k Example Price vs Mass kg Parametric Price Analysis result: Bidders 1, 3, 4 appear giving a fair offer (slightly above expected values). Bidder 2 seems still in line with the average offer, however a significantly cheaper price would be expected. Bidder 5 offers an unrealistic price. Improved Analysis - Bidder 5 overestimates the Material Cost - Bidder 3 offers the best value for money - Bidders 1 and 4 offer <50% of the labour effort than the rest

23 Remarks Out of the given example, - Is the Parametric Analysis useful? Certainly yes, it can give us a reliable reference point. - Is the Parametric Analysis sufficient? Possibly, however when parametrics models are used to estimate the total price, it leaves important uncertainties

24 Rates and Overheads

25 Elements of Cost Effort EFFORT Labour Typically the largest part in Hi-Tech development costs It fits very well the Parametric approach It is normalized of productivity and indirect costs Facilities The use of special facilities is relevant to a few MAIT activities Effort can be parametrized Material The quantity of material is the easiest element to be parametrized

26 Elements of Cost Effort unit cost EFFORT UNIT BASIC COST Rates Large variance among organizations Rarely comparable Can be negotiated Investments to improve capabilities Special needs Very difficult to handle in parametrics Market value of Material The can be generically handled in parametrics using published indices

27 Elements of Cost Company Markup MARKUP Every company is different Indirect Personnel Senior Management Admin, Finance, HR General Expenses Office supplies, Consumables External Services (Professional, Legal, Maintenance Depreciation Assets acquisition allocated over years

28 Rates and Overheads at ESA The European Space Agency negotiates individually with its suppliers for rates and company markup (overheads) Responsibility of Procurement, Finance and Legal Directorate Industrial Matters Department Industrial Cost Auditing Division Audit Plan for handling about audits per year Standard Procedures and Implementing Rules regulate the audits Negotiation and Agreement of forward rates

29 Rates Structure In the ESA practice, the normal rate structure of each company is mostly maintained. Some companies compile projects cost based on Cost Centers, other based on generic rates per function, other on very detailed cost accounting. Such normal practice is not disrupted. ESA Audit analyses company productivity and costs Indirect cost recovery regulated by ESA General Conditions

30 Cost Reimbursement The general principle of the industrial cost analysis and rates calculation is: A contractor is entitled to recover (be paid for) full cost incurred, plus a profit. Condition: costs are reasonable The analysis uses Financial Statements and supporting documents, and Staff statistics to calculate a fair cost to recover salaries and indirect costs

31 Rates Interpretation Income Statement Sales Revenues - Expenses - Personnel - Material and Services - Depreciation Operating Result (Profit/Loss) - Other incomes and expenses - Financials - Taxes Net Result (Profit/Loss) Labour Rates and Overheads E P M D Pd Pi Md Mi D Pd Pi Mi D rates overheads

32 Example Pd Direct Personnel = 200 FTE = 10 M Total 350,000 worked hours 280,000 hours invoiced to customers 70,000 hours on indirect tasks (P&T, AR, Admin, Training ) not invoiced Average Cost rate = 10 M / 350 khr = /hr 28.57*280 khr invoiced = 8 M 2 M not invoiced Pi Indirect Personnel = 50 FTE = 2.5 M Mi Indirect Material and Sevices = 2 M D Depreciation = 0.8 M Total indirect cost = = 7.3 M = 91.25% of invoiced salaries = Overhead % Average Gross Rate = * ( %) = /hr check recovery: * 280,000 = 15.3 M

33 Productivity Using the previous example, 200 direct staff, selling 80% of the worked hours 50 indirect staff 200*0.8/(200+50) = 64% The higher the productivity, the lower are the rates. Negotiating better productivity is often the best way to reduce company rates and overheads. Value for money is linked to productivity: low productivity means paying inefficiencies.

34 Productivity There is no real lower limit, however one should expect >65% Productivity is also linked to Company size Company business (engineering, manufacturing, services) Origin (Institutional, Commercial) Productivity Services Manufacturing Engineering Rates

35 Productivity Also, productivity is linked to the specific company organization: Direct vs Indirect tasks there si no

36 Improving Parametric Cost Estimating

37 The missing element Often the missing or neglected element in parametrics is attention to the company-specific nature of the cost The same equipment or similar equipments (with very close performance and physical parameters) can be produced with the same effort, therefore being indistinguishable by a simple parametric analysis, yet being offered by different companies at very different prices.

38 Improving Parametric Cost Analysis We can improve the Parametric Analysis adequate effort to normalize data keeping in mind it should estimate the effort, rather than the total cost. EFFORT EFFORT UNIT BASIC COST Labour Facilities Material MARKUP

39 Improving Parametric Cost Analysis Such approach is already implemented in some commercial cost estimating tools. We don t have to forget it, a part of the many general settings. In custom-developed models, we need to extract the effort from the historical data beforehand. This is a difficult task.

40 New generation of parametric models In order to enable a new generation of parametric models that don t neglect the importance of rates and overheads it is necessary to collect adequate data ESA requests for many years detail of Labour, Material, Overheads etc, but the historical data is not stored, except for a few major proposals.

41 New generation of parametric models With management support, it is indeed possible to start collecting effort data, simply by fostering the implementation of rules and requirements already existing (ESA GCC). The price to pay is that the development of such tools requires probably more time and dedication from experinced analysts.

42 THANK YOU