Life Cycle Model-Based Improvement of SEER- SEM TM Schedule Estimates

Transcription

1 Life Cycle Model-Based Improvement of SEER- SEM TM Schedule Estimates Dr. Peter Hantos and Nancy Kern The Aerospace Corporation 2009 SEER by Galorath North American User Conference 8 October 2009 The Aerospace Corporation

2 Acknowledgements This work would not have been possible without the following: Reviewers Suellen Eslinger, Software Engineering Subdivision Dr. Dan Houston, Software Acquisition and Process Department Dr. Leslie Holloway, Software Acquisition and Process Department Sponsors Marilee Wheaton, Systems Engineering Division Rosalind Lewis, Acquisition and Planning Subdivision Mary A. Rich, Software Engineering Subdivision Funding Source The Aerospace Corporation s Independent Research & Development Program 2

3 Outline Objectives Work Breakdown Structure for the Example Traditional Estimation Approach Life Cycle Models The Waterfall Incremental and Evolutionary Life Cycle Patterns Unified Life Cycle Modeling (ULCM ) Modeling Shift Patterns of Concurrent Increments Estimation of Concurrent Increments Staff Adjustment Conclusions Acronyms References ULCM is registered in the U.S. Patent and Trademark Office by The Aerospace Corporation 3

4 Objectives Use life cycle modeling insights to improve software system development schedule estimates Make recommendations for a better staff adjustment approach 4

5 Work Breakdown Structure (WBS) for the Example Software System Component 1 (SLOC 1 ) Component 2 (SLOC 2 ) Modeling abstractions/simplifications for the example to be shown: Only two components The size of the components is the same (SLOC 1 = SLOC 2 ) For more complex systems the introduced concepts are used recursively Legend: SLOC Source Lines of Code 5

6 Traditional Estimation Approach % of Average Manpower Level 150% 100% 44% 32% 50% 0% 6% 18% 29% 56% 79% 100% Schedule Chart shows the effort-distribution of a typical system reflecting embedded mode* software development [Boehm 1981] The shaded area is an approximation of a Rayleigh curve that is the basis for computing effort-distribution [Galorath 2007] For our example, the traditional approach would be to simply aggregate the size of the increments before the estimation model/tool is used * Embedded mode represents highly constraint-driven development, in contrast to other, more flexible modes like organic or semi-detached 6 Effort = F(SLOC 1 + SLOC 2 ), where the F function depends on the model/tool used for estimation

7 Rayleigh Interpretation of the Distribution The Rayleigh approximation of the effort profile reflects the gradual build-up of development effort over time This notion was first introduced without considering software [Norden 1958] However, since 1978, based on Putnam s research [Putnam 1978] it is widely accepted by software cost estimators and most of the popular cost estimation models, including SEER-SEM TM, are using it Critique of the Rayleigh approach Putnam s interpretation and the mapping of the curve to software development phases is based on a sequential life cycle model, also referred to as the Waterfall model. However, the use of the Waterfall is very limited in large scale, embedded mode development One of the most critical deficiency of the Waterfall approach is the diseconomy of scale; the indiscriminate use of the Rayleigh approximation hides and obfuscates the circumstances leading to this concern Recommended approach is to introduce more life cycle modeling details into the planning and estimation process TM: SEER-SEM is a trademark of Galorath Incorporated 7

8 Introduction to Software Life Cycle Modeling Models are frameworks, providing a conceptual frame of reference Their objective is to clarify relationships, identifying key elements A software life cycle model is a project management framework It covers the activities involved in the development, operation, and maintenance of the software (system), It spans the life of the software (system) from the definition of its requirements to the termination of its use. Software Engineering activities are classified as Product-oriented (e.g., Requirements Definition, Design, Coding, Unit Test) Integral (e.g., Project Management, Quality Assurance) Integral efforts are distributed across the life cycle and concurrent with product-oriented activities. From a cost estimation perspective they are modeled as level of effort* A software build is defined as: A version of the software (system) that implements a specified subset of the requirements that the completed software (system) will meet. * In this presentation we are going to model only product-oriented software engineering activities of development 8

9 Life Cycle Patterns The Waterfall Model System Requirements Definition System Design and Requirements Allocation Software Requirements Def. High-level Design Detailed Design Programming & Unit Test Software Integration & Software Qualification Test Big Bang Waterfall characteristics All system requirements are defined/allocated to software prior development Software intended to be developed all at once (One single build) Assumes an overly simplified, static view of requirements and architecture Activities are the same as phase-names Minimal overlap between development phases Typically marred by late problem discovery and resolution 9

10 Incremental and Evolutionary Life Cycle Patterns System Requirements Definition System Design and Requirements Allocation Software Detailed Requirements Design Re-Assessment Fabrication High-level Test Design Detailed Design System Requirements Definition System Design and Requirements Allocation Software Detailed Requirements Design Definition Fabrication High-level Test Design Detailed Design Software Requirements Def. Software Increment 1 Programming & Unit Test Software Integration & Regression Test Software Increment 1 Software Increment 2 Programming & Unit Test Software Integration & Regression Test Software Increment 2 Sequential Implementation Pattern Software Increment 3 Sequential Implementation Pattern Incremental Software Increment n Evolutionary Software Increment n System Qualification Testing System Qualification Testing Operations and Maintenance Re-validation/Re-verification Operations and Maintenance Re-validation/Re-verification Incremental Model System requirements are allocated to software Software requirements for all increments are specified up-front Evolutionary Model System requirements are allocated to software The development of each increment begins with requirements definition Overlap of builds allowed in both models From our perspective we do not need to distinguish between the two patterns and will only refer to Incremental Development 10

11 Unified Life Cycle Modeling ULCM is an intuitive, pattern-based approach for specifying, constructing, visualizing and documenting the life cycle processes of software-intensive system development ULCM is also a research platform It provides a foundation for a consistent and universal system development methodology and the modeling of various development processes Selected, key first principles of ULCM The fundamental building block of life cycle models is an increment All life cycle models are constructs or derivatives of a small number of basic life cycle modeling patterns Life cycle modeling is synergistic with architecture, architecture concepts, and architectural modeling Concurrent processes are synchronized via anchor points For more details on ULCM see [Hantos 2007] ULCM is registered in the U.S. Patent and Trademark Office by The Aerospace Corporation 11

12 Anchor Points in ULCM IIR IDR IED IEL Phase 1 Phase 2 Phase 3 Phase 4 Phase 5 Phase 6 Legend: IIR Increment Inception Readiness Increment Life Cycle Objectives Increment Life Cycle Architecture Increment Operational Readiness IDR Increment Delivery Readiness IED Increment End-Of-Life Decision IEL Increment End-Of-Life APs are intermediate milestones with unique product and process-related objectives - Facilitate a Stop, Stabilize, and Regroup approach to development* Life cycle phases in ULCM are intentionally not named Specifying both phase content and anchor points is redundant Phase content stays flexible; phase activities are not pre-determined Focus is on achieving AP objectives * This characteristic of APs is the reason why they are well suited for concurrent increment synchronization 12

13 ULCM Mapping of the Waterfall Process % of Average Manpower Level 150% 100% 44% 32% 50% 18% 6% Schedule SDLC Phases (Waterfall) ULCM Anchor Points ULCM Phases 0% Requirements Definition 29% 56% 79% 100% Product Programming Integration & Design & Unit Test Regression Test IIR Phase 1 Phase 2 Phase 3 Note that the ULCM increment structure is generalized; many different process models, in addition to the Waterfall, can be mapped 13

14 Modeling Shift Patterns of Concurrent Increments L Phase 1 Phase 2 Phase 3 L Phase 1 Phase 2 Phase 3 T Phase 1 Phase 2 Phase 3 T Phase 1 Phase 2 Phase 3 Very High Coupling High Coupling L Phase 1 Phase 2 Phase 3 L Phase 1 Phase 2 Phase 3 T Phase 1 Phase 2 Phase 3 T Phase 1 Phase 2 Phase 3 Medium Coupling Low Coupling Model simplifications for the example Size of the modeled WBS components is the same (SLOC1 = SLOC2) System-level and Integral activities are not modeled Δ = ((Effort (SLOC1+SLOC2)) (Effort (SLOC1)+Effort (SLOC2)) is not modeled Legend: L Leading Increment T Trailing Increment 14

15 The Relationship with Schedule/Cost Risks A main source of schedule/cost risks is architecture volatility stemming from concurrent engineering The classic Iron Triangle of Cost-Schedule-Performance does not apply anymore (Explanation on the next chart) The coupling of the process increments (The cohesion of the life cycle model) would be determined on the basis of the dependency-characteristics of the architectural components Selected considerations Release planning and architectural maturation Benefits of code availability of the leading increment Opportunities for early detection and correction of defects Learning opportunities for the development of the trailing increment We need to differentiate between variables and constraints Note that only the mentioned considerations represent independent variables Despite of the traditional naming conventions in acquisition, cost and schedule are never really independent variables; they are constraints 15

16 Estimation of Concurrent Increments (Medium Coupling) Increment 1 Phase 1 Phase 2 Phase 3 Increment 2 Phase 1 Phase 2 Phase 3 44% Effort Distribution 1 6% 18% 32% 44% Effort Distribution 2 6% 18% 32% 24% 22% Effort Distribution 1+2 3% 9% 18% 9% 16% The Rayleigh curve is inappropriate to model the combined effort profile 16

17 Effects of Changing Staff Levels % of Average Manpower Level Peak Staffing Productivity Loss 150% Small Productivity Loss No Productivity Loss Average 100% Staffing No Productivity Loss Productivity Gain 50% Schedule 0% Requirements Definition 29% 56% 79% 100% Product Programming Integration & Design & Unit Test Regression Test SDLC Phases (Waterfall) In terms of productivity loss, this, traditional view only accounts for the increased coordination overhead due to excess manpower 17

18 Staff Adjustment Considerations The natural tendency amongst project managers is to pursue maximal concurrency to achieve the shortest schedule Experience shows that Very High Level Coupling of increments is risky; Theoretically this pattern yields the shortest schedule; in reality it becomes uncontrollably extended due to the problems associated with handling large code sizes, integration challenges, and belated defect resolution High or Medium Coupling of increments might be more optimal Due to the following issues, staff adjustment is recommended before aggregating the concurrent increment effort profiles The ability of schedule compression is phase dependent and limited; certain activities cannot be further sped-up or parallelized There are skill availability differences even between organizations of the same company There are skill-level and productivity differences between collaborating contractors even in the same skill category 18

19 Sample Contractor Structure for a Military Space Acquisition Commercial Customers Government Organizational Barriers of Staff Exchange Prime Contractor Contracts Main Contract Commercial Sector Commercial Organization Government Sector Contractor Internal Formal Agreement Contractor s Program Office Contractor Internal Formal Agreement Ground Software Development Subcontractor Division A Program Office and Payload Software Development Subcontract Division B Contractor Internal Formal Agreement Spacecraft Software Development Staff cannot be quickly and seamlessly moved between divisions and contractors to achieve a smooth and balanced effort profile 19

20 Conclusions Most traditional estimation methods do not account for the true life cycle structure of projects Rayleigh curve is used to approximate effort distribution without acknowledging that this approximation was originally applied for the Waterfall life cycle model only Unified Life Cycle Modeling approach is recommended to define an enhanced, more detailed life cycle model for the project before estimation begins The promise of this approach is that on the basis of documented product and process requirements a higher fidelity life cycle model can be created; in turn, the use of this model will yield higher fidelity estimates A second aspect of the recommended estimation approach is to estimate increments independently and then aggregate the results However, staff adjustment should be done before the aggregation of the increment-estimates takes place to account for further organizational and acquisition constraints. 20

21 Acronyms AP Anchor Point CDR Critical Design Review DAPA Defense Acquisition Performance Assessment DoD Department of Defense F Function IDR Increment Delivery Readiness IED Increment End-of-Maintenance Decision IEL Increment End-of-Life IIR Increment Inception Readiness Increment Life Cycle Architecture Increment Life Cycle Objectives Increment Operational Readiness L Leading (Increment) PDR Preliminary Design Review SDLC Software Development Life Cycle SFR System Functional Review SLOC Source Lines Of Code SRR System Requirements Review T Trailing (Increment) TRR Test Readiness Review 21

22 References Boehm 1981 Software Engineering Economics, Prentice-Hall, 1981 DAPA 2006 Defense Acquisition Performance Assessment (DAPA) Report, March 2006 DoD 2008 DoD Instruction on the Operation of the Defense Acquisition System, December 8, 2008 Galorath 2007 SEER-SEM TM, the SEER Software Estimation Model User s Manual, Galorath Inc., April 2007 Hantos 2007 Hantos, P., Unified Life Cycle Modeling Tutorial Version 1.0, Aerospace Report No. TOR-2007(8550)- 6966, August 31, 2007 Norden 1958 Norden, P.V., Curve Fitting for a Model of Applied Research and Development Scheduling, IBM Journal, July 1958 Putnam 1978 Putnam, L.H., A General Empirical Solution to the Macro Software Sizing and Estimating Problem, IEEE Transactions on Software Engineering, Vol. SE-4, No.4, July 1978 SEER is registered in the U.S. Patent and Trademark Office by Galorath Incorporated 22

23 Backup Slides 23

24 Anchoring Basic Software Development Processes Phase Phase 1 1 Requirements Requirements Phase Phase 2 2 Design Design High- High- Level Level Design Design Detailed Detailed Design Design Phase Phase 3 3 Coding Coding Integration Integration & Test & Test Phase Phase 1 1 Modeling Modeling Requirements Requirements Analysis and Design Analysis and Design Implementation Implementation Test Test Phase Phase 2 2 Modeling Modeling Requirements Requirements Analysis and Design Analysis and Design Implementation Implementation Test Test Phase Phase 3 3 Modeling Modeling Requirements Requirements Analysis and Design Analysis and Design Implementation Implementation Test Test Waterfall Iterative Analysis Analysis Phase Phase 1 1 Phase Phase 2 2 Analysis Analysis Simulation Simulation Phase Phase 3 3 Automated Code Automated Generation Code Generation MODEL MODEL CODE CODE 24 Model-Based

25 Use of Trademarks, Service Marks and Trade Names Use of any trademarks in this material is not intended in any way to infringe on the rights of the trademark holder. All trademarks, service marks, and trade names are the property of their respective owners. 25