Economic Impact of Reuse on Systems Engineering Dr. Ricardo Valerdi Massachusetts Institute of Technology rvalerdi@mit.edu 4 th Annual IeMRC Conference Loughborough University September 2, 2009 IeMRC 4 th Annual Conference Loughborough University 1
Roadmap (1) Cost estimation & systems engineering fundamentals; (2) Explanation of COSYSMO size and cost drivers; (3) Reuse in systems engineering; (4) Next steps; IeMRC 4 th Annual Conference Loughborough University 2
All models are wrong IeMRC 4 th Annual Conference Loughborough but some University of them are useful. 3
IeMRC 4 th Annual Conference Loughborough University The Delphic Sybil Michelangelo Buonarroti Capella Sistina, Il Vaticano (1508-1512)
Contract Engineering WBS Based On Standards 1.0 System/Project 1.1 Integrated Project Management (IPM) 1.2 Systems Engineering 1.3 Prime Mission Product (PMP) 1.3.1 Subsystem / Configuration Item (CI) 1 n (Specify Names) 1.3.2 PMP Application Software 1.3.3 PMP System Software 1.3.4 PMP Integration, Assembly, Test & Checkout (IATC) 1.3.5 Operations/Production Support 1.4 Platform Integration 1.5 System Test & Evaluation (ST&E) 1.6 Training 1.7 Data Management 1.8 Peculiar Support Equipment 1.9 Common Support Equipment 1.10 Operational / Site Activation 1.11 Industrial Facilities Product-oriented construct, by tailoring MIL- HDBK 881A and ANSI/EIA 632 Six Functions: 1. Systems Engineering 2. Software Engineering 3. Electrical Engineering 4. Mechanical Engineering 5. Support Engineering 6. Project Engineering Management IeMRC 4 th Annual Conference Loughborough University
How is Systems Engineering Defined? Acquisition and Supply Product Realization Supply Process Implementation Process Acquisition Process Transition to Use Process Technical Management Technical Evaluation Planning Process Systems Analysis Process Assessment Process Requirements Validation Process Control Process System Design System Verification Process Requirements Definition Process End Products Validation Process Solution Definition Process EIA/ANSI 632, Processes for Engineering a System, 1999. IeMRC 4 th Annual Conference Loughborough University 6
Why measure systems engineering? Cost Overrun as a Function of SE Effort NASA Data Honour, E.C., Understanding the Value of Systems Engineering, Proceedings of the INCOSE International Symposium, Toulouse, France, 2004. IeMRC 4 th Annual Conference Loughborough University 7
COSYSMO Scope Addresses first four phases of the system engineering lifecycle (per ISO/IEC 15288) Conceptualize Develop Oper Test & Eval Transition to Operation Operate, Maintain, or Enhance Replace or Dismantle Considers standard Systems Engineering Work Breakdown Structure tasks (per EIA/ANSI 632) Valerdi, R., The Constructive Systems Engineering Cost Model: Quantifying the Costs of Systems Engineering Effort in Complex Systems, VDM Verlag, 2008 IeMRC 4 th Annual Conference Loughborough University 8
COSYSMO Operational Concept # Requirements # Interfaces # Scenarios # Algorithms + 3 Adjustment Factors Size Drivers Effort Multipliers COSYSMO Effort - Application factors -8 factors - Team factors -6 factors - Schedule driver Calibration WBS guided by EIA/ANSI 632 IeMRC 4 th Annual Conference Loughborough University 9
4 Size Drivers 1. Number of System Requirements 2. Number of System Interfaces 3. Number of System Specific Algorithms 4. Number of Operational Scenarios IeMRC 4 th Annual Conference Loughborough University 10
14 Cost Drivers Application Factors (8) 1. Requirements understanding 2. Architecture understanding 3. Level of service requirements 4. Migration complexity 5. Technology Risk 6. Documentation Match to Life Cycle Needs 7. # and Diversity of Installations/Platforms 8. # of Recursive Levels in the Design IeMRC 4 th Annual Conference Loughborough University 11
14 Cost Drivers (cont.) Team Factors (6) 1. Stakeholder team cohesion 2. Personnel/team capability 3. Personnel experience/continuity 4. Process capability 5. Multisite coordination 6. Tool support IeMRC 4 th Annual Conference Loughborough University 12
Model Form PM NS = A k ( w e, kφ e, k + wn, kφ n, k + wd, kφ d, k ) E 14 j= 1 EM j Where: PM NS = effort in Person Months (Nominal Schedule) A = calibration constant derived from historical project data k = {REQ, IF, ALG, SCN} w x = weight for easy, nominal, or difficult size driver Φ x = quantity of k size driver E = represents diseconomy of scale EM = effort multiplier for the j th cost driver. The geometric product results in an overall effort adjustment factor to the nominal effort. IeMRC 4 th Annual Conference Loughborough University 13
COSYSMO Data Sources Boeing Raytheon Northrop Grumman Lockheed Martin General Dynamics BAE Systems SAIC Integrated Defense Systems (Seal Beach, CA) Intelligence & Information Systems (Garland, TX) Mission Systems (Redondo Beach, CA) Transportation & Security Solutions (Rockville, MD) Integrated Systems & Solutions (Valley Forge, PA) Systems Integration (Owego, NY) Aeronautics (Marietta, GA) Maritime Systems & Sensors (Manassas, VA; Baltimore, MD; Syracuse, NY) Maritime Digital Systems/AIS (Pittsfield, MA) Surveillance & Reconnaissance Systems/AIS (Bloomington, MN) National Security Solutions/ISS (San Diego, CA) Information & Electronic Warfare Systems (Nashua, NH) Army Transformation (Orlando, FL) Integrated Data Solutions & Analysis (McLean, VA) L-3 Communications Greenville, TX IeMRC 4 th Annual Conference Loughborough University
Policy & Contracts Deploy Commercial Implementations Collaborate Enable Create COSYSMO Model PM NS = A k ( w E e, kφ e, k + wn, kφ n, k + wd, kφ d, k ) 14 j= 1 EM j 10 Academic Theses Academic Curricula Proprietary Implementations SEEMaP COSYSMO-R SECOST Systems Eng. Cost Tool IeMRC 4 th Annual Conference Loughborough University
Systems Engineering Reuse Systems engineering activities are support-focused Do not produce physical products (HW, SW, etc.) Produce architectures, requirements, test plans, and other technical documents Systems engineering work products can be viewed as artifacts Encapsulation of systems engineering knowledge in a document or process Representative of systems engineering effort Reuse of an artifact should reduce the expected systems engineering effort for the development of a new system IeMRC 4 th Annual Conference Loughborough University
Systems Engineering Reuse (cont.) Observations 1. Reuse is done for the purpose of economic benefit, intending to shorten schedule, reduce cost, and/or increase performance 2. Reuse is not free, upfront investment is required 3. Products, processes, and knowledge are all reusable artifacts 4. Reuse needs to be planned from the conceptualization phase of programs 5. Reuse is as much of an organizational issue as it is a technical one 6. The benefits of reuse are limited to related domains and do not scale linearly IeMRC 4 th Annual Conference Loughborough University
Reuse Continuum New: Artifacts that are completely new Modified: Artifacts that are inherited, but are tailored New 1.0 Adopted: Artifacts that are incorporated unmodified, also known as black box reuse Managed: Artifacts that are incorporated unmodified and untested Deleted: Artifacts that are removed from a system Deleted Adopted Modified vs. New Threshold Modified 0.65 0.51 0.43 Reuse weight Managed 0.15 0 IeMRC 4 th Annual Conference Loughborough University
Example COSYSMO Estimate Estimated as 129.1 Person-Months by COSYSMO (without reuse) a 30.4% difference IeMRC 4 th Annual Conference Loughborough University
Collaboration in the U.K. IdMRC Industrial Day (November 2008) Joint projects with University of Bath (ongoing) Using prediction markets to improve cost estimation A descriptive model for cost estimation decision making Costing capabilities Special issue on cost estimation (2010) Journal of Computer Integrated Manufacturing IeMRC 4 th Annual Conference Loughborough University 20
Contact Ricardo Valerdi MIT Lean Advancement Initiative rvalerdi@mit.edu (617) 253-8583 www.valerdi.com/cosysmo IeMRC 4 th Annual Conference Loughborough University 21