COSOSIMO Parameter Definitions Jo Ann Lane University of Southern California Center for Software Engineering

Transcription

1 Constructive System-of-Systems Integration Cost Model COSOSIMO Parameter Definitions Jo Ann Lane University of Southern California Center for Software Engineering Introduction The Constructive System-of-Systems (SoS) Integration Cost Model (COSOSIMO) is designed to estimate the effort associated with the Lead System Integrator (LSI) to define the SoS, identify sources to either supply or develop the required SoS component systems, and eventually integrate and test these high level component systems. For the purposes of this cost model, an SoS is defined as an evolutionary net-centric that allows geographically distributed component systems to exchange information and perform tasks within the framework that they are not capable of performing on their own outside of the framework. The component systems may operate within the SoS framework as well as outside of the framework, and may dynamically come and go as needed or available. In addition, the component systems are typically independently developed and managed by organizations/vendors other than the SoS sponsors or the LSI. Results of recent COSOSIMO workshops have resulted in the definition of three COSOSIMO sub-models: a planning/requirements management/ (PRA) sub-model, a source selection and supplier oversight (SS) sub-model, and an SoS integration and testing (I&T) sub-model. The conceptual effort profiles for each sub-model are shown in Figure 1. This technical report is an update to the COSOSIMO parameter definitions dated March 2006 and describes the parameters for each of the COSOSIMO sub-models. The parameters include a set of size drivers that are used to Figure 1. Conceptual Overview of COSOSIMO Sub-Models. calculate a nominal effort for the sets of associated with the sub-model and a set of cost drivers that are used to adjust the nominal effort based on related SoS, process, and personnel characteristics. Each size driver description includes a definition of the parameter as well as associated counting rules and guidance for assigning complexity ratings. Each cost driver description includes a definition of the parameter as well as guidance for assigning the appropriate rating factor. Finally, COSOSIMO workshop findings indicate that some of the SoS LSI are similar to systems engineering addressed by the Constructive Systems Engineering Cost Model (COSYSMO) and have similar size and cost drivers. Therefore, some of the COSOSIMO parameter definitions are adapted from the COSYSMO definitions in [Valerdi 2005] and are indicated by a footnote. COSOSIMO Parameter Definitions USC CSE 1

2 COSOSIMO PRA Parameters The LSI PRA are those associated with SoS concept development; requirements identification, analysis, and evolution; SoS development and evolution, as well as the long term planning for providing incremental SoS capabilities in accordance with the SoS sponsor s cost and schedule targets. PRA Size Drivers Number of SoS-Related Requirements 1 : This driver represents the number of requirements for the SoS of interest at the SoS level. Requirements may be functional, performance, feature, or service-oriented in nature depending on the methodology used for specification. They may also be defined by the customer or contractor. SoS requirements can typically be quantified by counting the number of applicable shalls, wills, shoulds, and mays in the SoS or marketing specification. Note: some work may be required to decompose requirements to a consistent level so that they may be counted accurately for the appropriate SoS-of-interest. Table 1 contains the complexity definitions for the SoS-related requirements. Table 1. SoS-Related Requirements Complexity Ratings. Easy Nominal Difficult Simple to implement Familiar Complex to implement or engineer Traceable to source Can be traced to source with Hard to trace to source some effort Little requirements overlap Some overlap High degree of requirements overlap Number of SoS Interface Protocols: The number of distinct net-centric protocols to be provided/supported by the SoS framework. Note: This does NOT include s internal to the SoS component systems, but it does include s external to the SoS and between the SoS component systems. Also note that this is not a count of total s (in many SoSs, the total number of s may be very dynamic as component systems come and go in the SoS environment in addition, there may be multiple instances of a given type of component system), but rather a count of distinct protocols at the SoS level. Table 2 contains the complexity definitions for the SoS protocols. Table 2. Interface Protocol Complexity Ratings. Easy Nominal Difficult Simple protocol Moderately complex protocol Highly complex or new protocol(s) May already be supported by several SoS component systems May already be supported by some SoS component systems Currently supported by few if any SoS component systems Uncoupled Loosely coupled Highly coupled Well understood Predictable behavior Not easily predicable PRA Cost Drivers Requirements Understanding 1 : This cost driver rates the level of understanding of the SoS requirements by all of the SoS s including the SoS customers and sponsors, SoS PRA team members, component system owners, users, etc. Primary sources of added systems engineering effort are unprecedented capabilities, unfamiliar domains, or capabilities whose requirements are emergent with use. Table 3 defines the various rating values for the requirements understanding cost driver. (Note: These rating definitions are the same as the ones for the SS and I&T requirements understanding cost drivers, but should be evaluated in terms of the level of understanding between the all of the SoS s with emphasis on the SoS PRA team members.) 1 Adapted to SoS environment from COSYSMO. COSOSIMO Parameter Definitions USC CSE 2

3 Poor: emergent requirements or unprecedented capabilities Table 3. PRA Requirements Understanding Ratings. Minimal: many Reasonable: some Strong: few undefined areas undefined areas undefined areas Full understanding of requirements, familiar capabilities Level of Service Requirements 1 : This cost driver rates the difficulty and criticality of satisfying the ensemble of level of service requirements or Key Performance Parameters (KPPs), such as security, safety, transaction speed, communication latency, interoperability, flexibility/adaptability, and reliability. Table 4 defines the various rating values for the level of service requirements cost driver. (Note: These rating definitions are the same as the ones for the SS and I&T level of service requirements cost drivers, but should be evaluated in terms of their impacts to the PRA.) Table 4. PRA Level of Service Requirements Ratings. Aspect Difficulty Simple; single dominant KPP Criticality Slight inconvenience Low, some coupling among KPPs Easily recoverable losses Moderately complex, coupled KPPs Difficult, coupled KPPs or some conflicts between KPPs maybe requiring tradeoffs Very complex, tightly coupled KPPs or significant conflicts between KPPs requiring tradeoffs Some loss High financial loss Risk to human life SoS Stakeholder Team Cohesion 1 : Represents a multi-attribute parameter which includes leadership, shared vision, diversity of s, approval cycles, group dynamics, Integrated Product Team (IPT) framework, team dynamics, trust, and amount of change in responsibilities. It further represents the heterogeneity in community of the end users, customers, implementers, and development team. Table 5 defines the various rating values for the SoS team cohesion cost driver. (Note: These rating definitions are the same as the ones for the SS and I&T team cohesion cost drivers, but should be evaluated in terms of potential impacts to the PRA.) Table 5. SoS Stakeholder Team Cohesion Ratings. Aspect Very Low Low Nominal High Very High Culture Stakeholders with diverse expertise, task Heterogeneous community Shared project culture Strong team cohesion and project culture Virtually homogeneous nature, communities language, Some similarities Multiple culture, in language and similarities in Institutionalized infrastructure culture language and project culture expertise Highly heterogeneous Compatibility Familiarity communities Highly conflicting Lack of trust Converging Willing to collaborate, little Compatible Some familiarity Clear roles and responsibilities Extensive successful Strong mutual advantage to Very high level of familiarity LSI PRA Team Capability: Represents the anticipated level of PRA team cooperation and cohesion, personnel capability and continuity, as well as PRA personnel with the relevant domains, COSOSIMO Parameter Definitions USC CSE 3

4 applications, language, and tools for SoS LSI personnel working on the PRA. Table 6 defines the various rating values for the LSI PRA team capability cost driver. Table 6. LSI PRA Team Capability Ratings. Aspect Very Low Low Nominal High Very High PRA Cohesion Highly conflicting Converging Compatible Clear roles and responsibilities Strong mutual advantage to PRA Capability Specific SoS PRA-Relevant Experience Expected Annual PRA Turnover Lack of trust Willing to collaborate, little Some familiarity Extensive successful Very high level of familiarity and trust 15 th percentile 35 th percentile 55 th percentile 75 th percentile 90 th percentile Less than 2 months 1 year, other technical in similar job 3 years of 5 years of 48% 24% 12% 6% 3% 10 years of LSI PRA Process Maturity: A parameter that rates the maturity level and completeness of the LSI s PRA processes and plans. Table 7 defines the various rating values for the LSI PRA process maturity cost driver. PRA Assessment Rating (Capability or Maturity) Table 7. LSI PRA Process Maturity Ratings. Aspect Extra High Level 0 (if Level 1 Level 2 Level 3 Level 4 Level 5 model) PRA Team Behavioral Characteristics Ad Hoc approach to process performance Performed PRA process, driven only by immediate contractual or customer requirements, PRA focus limited Managed PRA process, customer and needs in a suitable manner, PRA focus is a projectcentric approach not processes Defined PRA process, benefit to project, PRA focus process approach processes tailored for the project Quantitatively Managed PRA process, PRA benefit Optimizing PRA process, improvement, system engineering and benefit PRA Tool support 1 : Coverage, integration, and maturity of the PRA tools in the SoS engineering and management environments. Table 8 defines the various rating values for the PRA tool support cost driver. COSOSIMO Parameter Definitions USC CSE 4

5 Table 8. PRA Tool Support Ratings. No PRA tools Simple PRA tools, little integration Basic PRA tools moderately integrated throughout the systems engineering process Strong, mature PRA tools, moderately integrated with other disciplines Strong, mature proactive use of PRA tools integrated with process, modelbased SE and management systems PRA Cost/Schedule Compatibility: The extent of business or political pressures to reduce the cost and schedule associated with the PRA and processes. Table 9 defines the various rating values for the PRA cost/schedule compatibility cost driver. (Note: These rating definitions are the same as the ones for the SS and I&T cost/schedule compatibility cost drivers, but should be evaluated in terms of potential impacts to the PRA.) Estimates are greater than 100% of budgeted cost and schedule Table 9. PRA Cost/Schedule Compatibility Ratings. Very low Low Nominal High Estimates are Estimates are between 50% and between 20% and 100% greater than 50% greater than the budgeted cost the budgeted cost and schedule and schedule Budgeted cost and schedule are within 20% of estimates SoS PRA Risk Resolution: A multi-attribute parameter that represents the number of major SoS PRA risk items, the maturity of the associated risk management and mitigation plan, compatibility of schedules and budgets, expert availability, tool support, and level of uncertainty in SoS PRA risk areas. Table 10 defines the various rating values for the SoS PRA risk resolution cost driver. Table 10. SoS PRA Risk Resolution Ratings. Aspect Number and criticality of PRA > 10 critical 5-10 critical 2-4 critical 1 critical <10 non-critical risk items PRA risk mitigation None Little Some Risks generally Risks fully covered covered Schedule, budget, and internal None Little Some Generally Mostly milestones compatible with PRA Risk Management Plan % of top system engineers and 20% 40% 60% 80% 100% integrators available to support PRA Tool support available for None Little Some Good Strong tracking PRA issues Level of uncertainty in PRA risk areas Extreme Significant Considerable Some Little COSOSIMO SS Parameters The LSI SS are those associated with the identification of potential component system suppliers or vendors, the development of Requests for Proposals (RFPs) and statements of work for candidate suppliers/vendors, the evaluation of supplier/vendor responses, the selection of suppliers/vendors, and then the on-going oversight of supplier/vendor performance through delivery and validation/verification of the desired component system. COSOSIMO Parameter Definitions USC CSE 5

6 SS Size Drivers Number of Independent Component System Organizations: The number organizations managed by the LSI that are providing SoS component systems. Table 11 contains the complexity definitions for component system organizations. Vendor/supplier has previously worked closely with LSI and/or many of the other vendors/ suppliers working on the SoS of interest No current competition between vendor/supplier and the LSI or other vendors/suppliers on the system of interest Table11. Component System Organization Ratings. Easy Nominal Difficult Vendor/supplier has previously worked to some extent with the LSI and/or some of the other vendors/suppliers working on the SoS of interest No current significant competition between vendor/ supplier and the LSI or other vendors/suppliers on the system of interest Vendor/supplier has not worked with the LSI in any significant way May be competing with the LSI or other SoS vendors/suppliers for significant related work Number of Unique Component Systems: The number of types of component systems that are planned to operate within the SoS framework. If there are multiple versions of a given type that have different s, then the different versions should also be included in the count of component systems. Table 12 contains the complexity definitions for component systems. Component system is a relatively open system, with many external s compatible with the SoS Table 12. Component System Ratings. Easy Nominal Difficult Component system is somewhat open and has some compatible external s, but will require additional s key to SoS operations Component system is currently a closed, stove-pipe system that has few or no external s compatible with the SoS Or component system is only in the planning stages or under initial development SS Cost Drivers Requirements Understanding 1 : This cost driver rates the level of understanding of the SoS requirements between the LSI and the component system suppliers/vendors. Primary sources of added systems engineering effort are unprecedented capabilities, unfamiliar domains, or capabilities whose requirements are emergent with use. Table 13 defines the various rating values for the SS requirements understanding cost driver. (Note: These rating definitions are the same as the ones for the PRA and I&T requirements understanding cost drivers, but should be evaluated in terms of the level of understanding between the LSI and the suppliers/vendors.) Poor: emergent requirements or unprecedented capabilities Table 13. SS Requirements Understanding Ratings. Minimal: many Reasonable: some Strong: few undefined areas undefined areas undefined areas Full understanding of requirements, familiar capabilities Architecture Maturity: A parameter that represents the level of maturity of the SoS. It includes the level of detail of the protocols and the level of understanding of the performance of the protocols in the SoS framework. Table 14 defines the various rating values for the SoS maturity cost driver. (Note: These rating definitions are the same as the ones for the I&T maturity cost driver, but should be evaluated in terms of potential impacts to the SS.) COSOSIMO Parameter Definitions USC CSE 6

7 Incomplete specification, especially in unprecedented areas Table 14. SS Architecture Maturity Ratings. Relatively complete Relatively Complete Complete complete specification, but specification, specification, unprecedented areas specification moderately detailed relatively detailed at a high level of level of specification specification in high specification risk areas Many To Be Determined (TBD) elements in the / specifications, especially in unprecedented areas No feasibility analyses or prototypes developed for high risk areas Little understanding about expected SoS scalability and performance Some TBD elements in the / specifications Few feasibility analyses or prototypes developed for high risk areas Scalability and performance aspects not investigated significantly Few TBD elements in the / specifications Some feasibility analyses or prototypes developed for high risk areas Scalability and performance aspects understood to a limited extent No TBD elements in the / specifications Feasibility analyses and prototypes developed for many unprecedented areas Most scalability and performance aspects understood reasonably well No TBD elements in the / specifications Feasibility analyses and prototypes developed for most/all unprecedented areas Scalability and performance aspects well understood Level of Service Requirements 1 : This cost driver rates the difficulty and criticality of satisfying the ensemble of level of service requirements or KPPs, such as security, safety, transaction speed, communication latency, interoperability, flexibility/adaptability, and reliability. Table 15 defines the various rating values for the SoS level of service requirements cost driver. (Note: These rating definitions are the same as the ones for the PRA and I&T level of service requirements cost drivers, but should be evaluated in terms of potential conflicts or incompatibilities between component systems and the SoS-level requirements.) Table 15. SS Level of Service Requirements Ratings. Aspect Difficulty Simple; single dominant KPP Low, some coupling among KPPs Moderately complex, coupled KPPs Criticality Slight inconvenience Easily recoverable losses Some loss Difficult, coupled KPPs or some conflicts between KPPs maybe requiring tradeoffs High financial loss Very complex, tightly coupled KPPs or significant conflicts between KPPs requiring tradeoffs Risk to human life LSI/Supplier Team Cohesion 1 : Represents a multi-attribute parameter which includes leadership, shared vision, diversity of s, approval cycles, group dynamics, IPT framework, team dynamics, trust, and amount of change in responsibilities. It further represents the heterogeneity in community of the end users, customers, implementers, and development team. Table 16 defines the various rating values for the LSI/supplier team cohesion cost driver. (Note: These rating definitions are the same as the ones for the PRA and I&T team cohesion cost drivers, but should be evaluated in terms of potential impacts to the SS.) COSOSIMO Parameter Definitions USC CSE 7

8 Table 16. LSI/Supplier Team Cohesion Ratings. Aspect Very Low Low Nominal High Very High Culture Stakeholders with diverse expertise, task Heterogeneous community Shared project culture Strong team cohesion and project culture Virtually homogeneous nature, communities language, Some similarities Multiple culture, in language and similarities in Institutionalized infrastructure culture language and project culture expertise Highly heterogeneous Compatibility Familiarity communities Highly conflicting Lack of trust Converging Willing to collaborate, little Compatible Some familiarity Clear roles and responsibilities Extensive successful Strong mutual advantage to Very high level of familiarity LSI SS Team Capability: Represents the anticipated level of SS team cooperation and cohesion, personnel capability and continuity, as well as SS personnel with the relevant domains, applications, language, integration tools, and integration platform(s) used by the various suppliers/vendors. Table 17 defines the various rating values for the LSI SS team capability cost driver. Table 17. LSI SS Team Capability Ratings. Aspect Very Low Low Nominal High Very High SS Cohesion Highly conflicting Converging Compatible Clear roles and responsibilities Strong mutual advantage to Extensive successful Willing to Some familiarity Very high level Lack of trust collaborate, little of familiarity and trust SS Capability 15 th percentile 35 th percentile 55 th percentile 75 th percentile 90 th percentile Specific SoS SS-Relevant Experience Expected Annual SS Staff Turnover Less than 2 months 1 year, other technical in similar job 3 years of 5 years of 48% 24% 12% 6% 3% 10 years of LSI SS Process Maturity: A parameter that rates the maturity level and completeness of the LSI s SS processes and plans. Table 18 defines the various rating values for the LSI SS process maturity cost driver. Table 18. LSI SS Process Maturity Ratings. Aspect Extra High SS Assessment Rating (Capability or Maturity) Level 0 (if model) Level 1 Level 2 Level 3 Level 4 Level 5 COSOSIMO Parameter Definitions USC CSE 8

9 Aspect Extra High Ad Hoc approach to process performance SS Team Behavioral Characteristics Performed SS process, driven only by immediate contractual or customer requirements, SS focus limited Managed SS process, customer and needs in a suitable manner, SS focus is a projectcentric approach not processes Defined SS process, benefit to project, SS process approach processes tailored for the project Quantitatively Managed SS process, SS benefit Optimizing SS process, improvement, system engineering and benefit SS Tool Support 1 : Coverage, integration, and maturity of SS tools in the SoS engineering and management environment. Table 19 defines the various rating values for SS tool support cost driver. Table 19. SS Tool Support Ratings. No SS tools Simple SS tools, little integration Basic SS tools moderately integrated throughout the systems engineering process Strong, mature SS tools, moderately integrated with other disciplines Strong, mature proactive use of SS tools integrated with process and management systems SS Process Cost/Schedule Compatibility: The extent of business or political pressures to reduce cost and schedule. Table 20 defines the various rating values for the SS process cost/schedule compatibility cost driver. (Note: These rating definitions are the same as the ones for the PRA and I&T cost/schedule compatibility cost drivers, but should be evaluated in terms of potential impacts to the SS.) Table 20. SS Process Cost/Schedule Compatibility Ratings. Very low Low Nominal High Estimates are Estimates are between 50% and between 20% and 100% greater than 50% greater than the budgeted cost the budgeted cost and schedule and schedule Estimates are greater than 100% of budgeted cost and schedule Budgeted cost and schedule are within 20% of estimates SoS SS Risk Resolution: A multi-attribute parameter that represents the number of major SoS SS risk items, the maturity of the associated risk management and mitigation plans, compatibility of schedules and budgets, expert availability, tool support, and level of uncertainty in SoS SS risk areas. Table 21 defines the various rating values for the SoS SS risk resolution cost driver. Table 21. SoS SS Risk Resolution Ratings. Aspect Number and criticality of SS > 10 critical 5-10 critical 2-4 critical 1 critical <10 non-critical risk items SS risk mitigation None Little Some Risks generally covered Risks fully covered COSOSIMO Parameter Definitions USC CSE 9

10 Aspect Schedule, budget, and internal None Little Some Generally Mostly milestones compatible with SS Risk Management Plan % of top system engineers and 20% 40% 60% 80% 100% integrators available to support SS Tool support available for None Little Some Good Strong tracking SS issues Level of uncertainty in SS risk area Extreme Significant Considerable Some Little COSOSIMO I&T Parameters The LSI I&T are those associated with the SoS component system integration and the verification/validation testing at the SoS level. These include integration and test planning, set up of the integration and test environments and tools, development of test data and procedures, and the actual execution and tracking of integration and verification/validation tests. (Note: the effort to develop I&T tools and simulators should be estimated using the COCOMO II cost model described in [Boehm et al, 2000].) I&T Size Drivers Number of SoS Interface Protocols: The number of distinct net-centric protocols to be provided/supported by the SoS framework. Note: This does NOT include s internal to the SoS component systems, but it does include s external to the SoS and between the SoS component systems. Also note that this is not a count of total s (in many SoSs, the total number of s may be very dynamic as component systems come and go in the SoS environment in addition, there may be multiple instances of a given type of component system), but rather a count of distinct protocols at the SoS level. Table 22 contains the complexity definitions for the SoS protocols. (Note: This is the same size driver that is part of the PRA sub-model. It is also included here in the I&T section for completeness.) Table 22. Interface Protocol Complexity Ratings. Easy Nominal Difficult Simple protocol Moderately complex protocol Highly complex or new protocol(s) May already be supported by several SoS component systems May already be supported by some SoS component systems Currently supported by few if any SoS component systems Uncoupled Loosely coupled Highly coupled Well understood Predictable behavior Not easily predicable Number of Operational Scenarios 1 : This driver represents the number of operational scenarios that an SoS must satisfy. Such scenarios include both the nominal stimulus-response thread plus all of the offnominal threads resulting from bad or missing data, unavailable processes, network connections, or other exception-handling cases. The number of scenarios can typically be quantified by counting the number of SoS states, modes, and configurations defined in the SoS concept ofoperations or by counting the number of sea-level use cases [Cockburn 2001], including off-nominal extensions, developed as part of the operational. Table 23 contains the complexity definitions for the SoS operational scenarios. Table 23. Operational Scenario Complexity Ratings. Easy Nominal Difficult Well defined Loosely defined Ill defined Loosely coupled Moderately coupled Tightly coupled or many dependencies/ conflicting requirements Timelines not an issue Timelines a constraint Tight timelines through scenario network Few, simple off-nominal threads Moderate number or complexity of off-nominal threads Many or very complex off-nominal threads COSOSIMO Parameter Definitions USC CSE 10

11 Number of Unique Component Systems: The number of types of component systems that are planned to operate within the SoS framework. If there are multiple versions of a given type that have different s, then the different versions should also be included in the count of component systems. Table 24 contains the complexity definitions for the SoS component systems. Component system is a relatively open system, with many external s compatible with the SoS Table 24. Component Systems Complexity Ratings. Easy Nominal Difficult Component system is somewhat open and has some compatible external s, but will require additional s key to SoS operations Component system is currently a closed, stove-pipe system that has few or no external s compatible with the SoS Or component system is only in the planning stages or under initial development I&T Cost Drivers Requirements Understanding 1 : This cost driver rates the level of understanding of the SoS requirements by all of the SoS s including the SoS customers and sponsors, SoS I&T team members, component system owners, users, etc. Primary sources of added systems engineering effort are unprecedented capabilities, unfamiliar domains, or capabilities whose requirements are emergent with use. Table 25 defines the various rating values for the requirements understanding cost driver. (Note: These rating definitions are the same as the ones for the PRA and SS requirements understanding cost drivers, but should be evaluated in terms of the level of understanding between the all of the SoS s with emphasis on the SoS I&T team members.) Poor: emergent requirements or unprecedented capabilities Table 25. I&T Requirements Understanding Ratings. Minimal: many Reasonable: some Strong: few undefined areas undefined areas undefined areas Full understanding of requirements, familiar capabilities Architecture Maturity: A parameter that represents the level of maturity of the SoS. It includes the level of detail of the protocols and the level of understanding of the performance of the protocols in the SoS framework. Table 26 defines the various rating values for the maturity cost driver. (Note: These rating definitions are the same as the ones for the SS maturity cost driver, but should be evaluated in terms of potential impacts to the I&T.) Incomplete specification, especially in unprecedented areas Table 26. I&T Architecture Maturity Ratings. Relatively complete Relatively Complete Complete complete specification, but specification, specification, unprecedented areas specification moderately detailed relatively detailed at a high level of level of specification specification in high specification risk areas Many TBD elements in the / specifications, especially in unprecedented areas Some TBD elements in the / specifications Few TBD elements in the / specifications No TBD elements in the / specifications No TBD elements in the / specifications COSOSIMO Parameter Definitions USC CSE 11

12 Few feasibility Some feasibility Feasibility analyses Feasibility analyses analyses or analyses or and prototypes and prototypes prototypes developed prototypes developed for many developed for for high risk areas developed for high unprecedented areas most/all risk areas unprecedented areas No feasibility analyses or prototypes developed for high risk areas Little understanding about expected SoS scalability and performance Scalability and performance aspects not investigated significantly Scalability and performance aspects understood to a limited extent Most scalability and performance aspects understood reasonably well Scalability and performance aspects well understood Level of Service Requirements 1 : This cost driver rates the difficulty and criticality of satisfying the ensemble of level of service requirements or KPPs, such as security, safety, transaction speed, communication latency, interoperability, flexibility/adaptability, and reliability. Table 27 defines the various rating values for the I&T level of service requirements cost driver. (Note: These rating definitions are the same as the ones for the PRA and SS level of service requirements cost drivers, but should be evaluated in terms of their impacts to the I&T.) Table 27. I&T Level of Service Requirements Ratings. Aspect Difficulty Simple; single dominant KPP Criticality Slight inconvenience Low, some coupling among KPPs Easily recoverable losses Moderately complex, coupled KPPs Some loss Difficult, coupled KPPs or some conflicts between KPPs maybe requiring tradeoffs High financial loss Very complex, tightly coupled KPPs or significant conflicts between KPPs requiring tradeoffs Risk to human life I&T Team Cohesion 1 : Represents a multi-attribute parameter which includes leadership, shared vision, diversity of s, approval cycles, group dynamics, IPT framework, team dynamics, trust, and amount of change in responsibilities. It further represents the heterogeneity in community of the end users, customers, implementers, and development team. Table 28 defines the various rating values for the I&T team cohesion cost driver. (Note: These rating definitions are the same as the ones for the PRA and SS team cohesion cost drivers, but should be evaluated in terms of potential impacts to the I&T.) Table 28. I&T Team Cohesion Ratings. Aspect Very Low Low Nominal High Very High Culture Stakeholders with diverse expertise, task Heterogeneous community Shared project culture Strong team cohesion and project culture Virtually homogeneous nature, communities language, Some similarities Multiple culture, in language and similarities in Institutionalized infrastructure culture language and project culture expertise Highly heterogeneous Compatibility Familiarity communities Highly conflicting Lack of trust Converging Willing to collaborate, little Compatible Some familiarity Clear roles and responsibilities Extensive successful Strong mutual advantage to Very high level of familiarity COSOSIMO Parameter Definitions USC CSE 12

13 SoS I&T Team Capability: Represents the anticipated level of SoS I&T team cooperation and cohesion, personnel capability and continuity, as well as I&T personnel with the relevant domains, applications, language, integration tools, and integration platform(s) needed to integrate the SoS system components and test the SoS. Table 29 defines the various rating values for the SoS I&T team capability cost driver. Table 29. SoS I&T Team Capability Ratings. Aspect Very Low Low Nominal High Very High I&T Cohesion Highly conflicting Converging Compatible Clear roles and responsibilities Strong mutual advantage to I&T Capability Specific SoS I&T-Relevant Experience Expected Annual I&T Turnover Lack of trust Willing to collaborate, little Some familiarity Extensive successful Very high level of familiarity and trust 15 th percentile 35 th percentile 55 th percentile 75 th percentile 90 th percentile Less than 2 months 1 year, other technical in similar job 3 years of 5 years of 48% 24% 12% 6% 3% 10 years of LSI I&T Process Maturity: A parameter that rates the maturity level and completeness of the LSI s processes and plans, and in particular, those associated with I&T and the SOS integration lab. Also indicates the level of automation/tool support. Table 30 defines the various rating values for the LSI I&T process maturity cost driver. I&T Assessment Rating (Capability or Maturity) Table 30. LSI I&T Process Maturity Ratings. Aspect Extra High Level 0 (if Level 1 Level 2 Level 3 Level 4 Level 5 model) I&T Team Behavioral Characteristics Ad Hoc approach to process performance Performed I&T process, driven only by immediate contractual or customer requirements, I&T focus limited Managed I&T process, customer and needs in a suitable manner, I&T focus is a projectcentric approach not processes Defined I&T process, benefit to project, I&T focus is a process approach processes tailored for the project Quantitatively Managed I&T process, I&T benefit Optimizing I&T process, improvement, system engineering and benefit I&T Tool Support 1 : Coverage, integration, and maturity of the tools in the SoS I&T environment. Table 31 defines the various rating values for the I&T tool support cost driver. COSOSIMO Parameter Definitions USC CSE 13

14 Table 31. I&T Tool Support Ratings. No I&T tools Simple I&T tools, little integration Basic I&T tools moderately integrated throughout the systems engineering process Strong, mature I&T tools, moderately integrated with other disciplines Strong, mature proactive use of I&T tools integrated with process, modelbased SE and management systems I&T Process Cost/Schedule Compatibility: The extent of business or political pressures to reduce the cost and schedule associated with the I&T processes and. Table 32 defines the various rating values for the I&T process cost/schedule compatibility cost driver. (Note: These rating definitions are the same as the ones for the PRA and SS cost/schedule compatibility cost drivers, but should be evaluated in terms of potential impacts to the I&T.) Table 32. I&T Process Cost/Schedule Compatibility Ratings. Very low Low Nominal High Estimates are Estimates are between 50% and between 20% and 100% greater than 50% greater than the budgeted cost the budgeted cost and schedule and schedule Estimates are greater than 100% of budgeted cost and schedule Budgeted cost and schedule are within 20% of estimates SoS I&T Risk Resolution: A multi-attribute parameter that represents the number of major SoS I&T risk items, the maturity of risk management and mitigation plan, compatibility of schedules and budgets, expert availability, tool support, and level of uncertainty in SoS I&T risk areas. Table 33 defines the various rating values for the SoS I&T risk resolution cost driver. Table 33. SoS I&T Risk Resolution Ratings. Aspect Number and criticality of I&T > 10 critical 5-10 critical 2-4 critical 1 critical <10 non-critical risk items I&T risk mitigation None Little Some Risks generally Risks fully covered covered Schedule, budget, and internal None Little Some Generally Mostly milestones compatible with I&T Risk Management Plan and integration scope % of top system engineers and 20% 40% 60% 80% 100% integrators available to support I&T Tool support available for None Little Some Good Strong tracking I&T issues Level of uncertainty in I&T risk areas Extreme Significant Considerable Some Little Component System Maturity and Stability: A multi-attribute parameter that indicates the maturity level of the component systems (number of new component systems versus number of component systems currently operational in other environments), overall compatibility of the component systems with each other and the SoS protocols, the number of major component system changes being implemented in parallel with the SoS framework changes, and the anticipated change in the component systems during SoS integration. Table 34 defines the various rating values for the component system maturity and stability cost driver. COSOSIMO Parameter Definitions USC CSE 14

15 Table 34. Component System Maturity and Stability Ratings. Aspect Very Low Low Nominal High Very High % of total number of Less than 30% 30-50% 50-75% 75-90% % unique component systems that are expected to be existing/legacy systems (vs. new) Compatibility of component systems Less than 30% 30-50% 50-75% 75-90% Greater than 90% % of major component system changes done in integration release related to SoS capabilities Less than 60% 60-70% 70-80% 80-95% Greater than 95% Anticipated average component system change during integration period Greater than 20% change 10-20% change 5-10% change 2-5% change Less than 2% change Component System Readiness: Indicates readiness of component systems for integration. User evaluates level of Verification and Validation (V&V) that has/will be performed prior to integration and the level of subsystem integration that will be performed prior to integration into the SoS integration lab. Table 35 defines the various rating values for the component system readiness cost driver. Table 35. Component System Readiness Ratings. Very Low Low Nominal High Very High Minimally V&V d Some V&V Moderate V&V Considerable V&V Extensive V&V No pre-integration Minimal preintegration Some preintegration Moderate preintegration Considerable preintegration References Boehm, B., C. Abts, A. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, B. Steece, Software Cost Estimation with COCOMO II, Prentice Hall, Cockburn, A., Writing Effective Use Cases, Addison-Wesley, Valerdi, R., The Constructive Systems Engineering Cost Model (COSYSMO), Unpublished PhD Dissertation, University of Southern California, Los Angeles, CA, May Glossary of Terms component system cost driver integrated product team key performance parameter size driver system of systems Independently developed and managed systems that can dynamically come and go from the SoS Cost model parameter used to adjust nominal effort based pre-defined rating scale, usually from very high to very low Multi-disciplinary team (often from multiple organizations) responsible for the design/development of a system (or SoS) product Desired system (or SoS) performance attribute, often related to transaction speed, communication latency, security, safety, reliability, etc. A quantitative cost model parameter used to determine the nominal effort associated with the indicated System developed by creating a framework or to integrate new and existing component systems COSOSIMO Parameter Definitions USC CSE 15

16 Glossary of Acronyms COSOSIMO Constructive System of Systems Integration Cost Model COSYSMO IPT I&T KPP LSI PRA RFP SoS SS TBD V&V Constructive Systems Engineering Cost Model Integrated Product Team Integration and Testing Key Performance Parameter Lead System Integrator Planning, Requirements Management, and Architecting Request for Proposal System of Systems Source Selection and Supplier Oversight To Be Determined Verification and Validation COSOSIMO Parameter Definitions USC CSE 16