Factors Influencing System-of-Systems Architecting and Integration Costs

Transcription

1 Paper # (unknown) Factors Influencing System-of-Systems Architecting and Integration Costs Jo Ann Lane University of Southern California Center for Software Engineering 941 W. 37th Place, SAL Room 328 Los Angeles, CA jolane@usc.edu Abstract Today s need for more complex, more capable systems in a short timeframe is leading more organizations towards the integration of existing systems into networkcentric, knowledge-based system-of-systems (SoS). Software and system cost model tools to-date have focused on the software and system development activities of a single software system. As we view the new SoS architectures, we find that the effort associated with the integration of these SoSs is not handled well, if at all, in current cost models. USC s Center for Software Engineering (CSE) began work on a SoS cost model, the Constructive SoS Integration Model (COSOSIMO), in late This model has evolved using feedback obtained from USC CSE affiliates and other experts in industry and academia. This paper presents an overview of the COSOSIMO cost model, descriptions of the size drivers and cost factors currently in the model, a summary of survey feedback received from USC CSE affiliates and other interested experts from industry, and the impact of survey findings on the current COSOSIMO cost model. It concludes with future plans for the COSOSIMO model. History of COSOSIMO Why a COSOSIMO? We are seeing a growing trend in industry and DoD to quickly incorporate new technologies and expand the capabilities of legacy systems by integrating them with other legacy systems, Commercial-Off-the-Shelf (COTS) products, and new systems. With this development approach, we see new activities being performed to define the new architecture, identify sources to either supply or develop the required components, and then to integrate and test these high level components. Along with this system-of-systems (SoS) development approach, we have seen a new role in the development process evolve to perform these activities: that of the Lead System Integrator (LSI). Today, there are fairly mature tools to support the estimation of the effort and schedule associated with the lower-level SoS component systems. For software development activities, there are the COCOMO II, Cost Xpert, Costar, PRICE S, and SEER-SEM cost models. At the single system level, there is COSYSMO to estimate the system engineering effort and PRICE H and SEER-H to estimate hardware development costs. For COTS implementation and integration, there is COCOTS to estimate the effort associated with the assessment, 1

2 tailoring, and glue-code implementation of COTS software products. [Boehm 2005] However, none of these models includes LSI activities such as the definition of the SoS architecture, the solicitation and procurement process for the SoS components, and the integration of the SoS components into the SoS framework. Many LSI organizations often estimate these costs using a percentage of the lower level system component development costs. As we see more and more of this type of development, it is important to get a handle on such questions as: How much should an organization budget for SoS integration activities? Is an extra 10%, 20%, or 50% sufficient? Too much? What factors or characteristics make actual effort higher or lower? COSOSIMO is a parametric model to compute just this effort. It is designed to estimate the up-front LSI effort associated with SoS abstraction, architecting, source selection, systems acquisition, as well as the effort associated with the later activities of integration, test, and change management. With the addition of COSOSIMO to the COCOMO suite of estimation tools, there will be more complete coverage of the development activities associated with implementing a system-of-systems: COSOSIMO: Designed to estimate the effort associated with designing the SoS architecture communications mechanisms, protocols, and interfaces as well as the effort required to integrate, tune, and test the SoS architecture. COSYSMO: Designed to estimate the system engineering effort associated with system development or system modifications required to enable the SoS architecture (effort typically performed by independent organizations responsible for the component systems). PRICE H or SEER-H: Designed to estimate any hardware development costs associated with system development or system modifications required to enable the SoS architecture (effort typically performed by independent organizations responsible for the component systems). COCOMO II: Used to estimate the software development effort associated with software enhancements and modifications required to enable the SoS architecture (effort typically performed by independent organizations responsible for the component systems). COCOTS: Used to estimate the effort associated with identifying, assessing, and incorporating COTS software products into the SoS framework. What is an SoS? Key to developing a cost model such as COSOSIMO is understanding what a system-of-systems is. Early literature research [Jamshidi] shows that the term system-of-systems means many things to many different people and organizations. In the business domain, it is the enterprise-wide integration and sharing of core business information across functional and geographical areas. In the military domain, it is a dynamic communications infrastructure to support operations in a constantly changing, sometimes adversarial, environment. In either case, users and nodes in the SoS network may be either fixed or mobile. Communications may be either pointto-point or broadcast. Networks may tie together other networks and well as nodes and users. SoS component systems typically come and go over time. These component systems can operate both within the SoS framework and independent of the SoS framework. Andrew Sage best summed it up when he states that SoS typically contain a majority of the following characteristics: operational independence of individual systems, 2

3 managerial independence of individual systems, geographic distribution of SoS component systems, emergent SoS behaviour not contained in any one component system, and evolutionary development over time, with functionality continually being added, deleted, and modified. [Sage] COSOSIMO Development Methodology: The development of COSOSIMO is following the standard cost model development methodology used at USC CSE. It is a methodology that builds on current practices, analysis of those practices, as wells as a combination of historical data and expert judgment to develop a statistically significant model for predicting future effort and costs associated with system and software development activities. These seven steps, outlined in Figure 1, can be summarized as: 1) analyze existing literature, 2) perform behavioural analysis, 3) identify relative significance of size drivers and scale factors, 4) perform expert-judgment Delphi assessment, formulate a-priori model, 5) gather project data, 6) determine Bayesian A- Posteriori model, and 7) gather more data; refine model. [Boehm 2000] Current COSOSIMO development activities are focused on steps 3 and 4, as highlighted in Figure Literature Search 2. Behavior Analysis 3. Parameter Significance 4. Expert Judgment 5. Gather Data 6. Bayesian Model 7. More Data/Model Refinement Figure 1. Cost Model Development Methodology. Factors Affecting SoS Architecting and Integration Costs Initial Model Form. The first version of COSOSIMO [Lane 2004] was based upon expert judgment of key members of the COCOMO Working Group and a hierarchical view of SoS architectures as described in [ISO/IEC 15288] and shown in Figure 2. Level 1 S 1 Level 0 SOS S 2 S m S 11 S 12 S 1n S 21 S 22 S 2n S m1 S m2 S mn Figure 2. SoS Architecture View for Initial COSOSIMO It is a two-tiered model to calculate the number of integration person months (IPM) required for the SoS LSI effort: n i Level 1 IPM(S i ) = A 1 [Σ S ij ] Bi m Level 0 IPM(SoS) = A 0 [Σ IPM(S i )] B0 i=1 where: IPM Integration effort in personmonths S ij The size ij th subsystem within the SoS. Determined by computing the weighted average of the ij th subsystem size driver(s). A Constant derived from historical project data. n i Number of Subsystem level 2 components comprising the ith subsystem. j=1 3

4 m Number of Subsystem level 1 components comprising the SoS. B i Effort exponent for the ith subsystem based on the subsystem s exponential scale factors. The geometric product of the scale factors results in an overall exponential effort adjustment factor to the B 0 nominal effort. Effort exponent for the SoS based on the SoSs exponential scale factors. The geometric product of the scale factors results in an overall exponential effort adjustment factor to the nominal effort. The size drivers in the initial version focus on quantifiable sizes associated with the system components and the relationships between them. They were defined as some combination of: software size of interfacerelated software, number of major interfaces, number of component systems, and number of operational scenarios. The initial scale factors include: Integration Simplicity: A parameter that represents the degree of system component coupling, processing criticality, scope of key performance parameters, and system precedentedness. Integration Risk Resolution: A multiattribute parameter that represents the number of major integration risk items, the maturity of risk management and mitigation plan, compatibility of schedules and budgets, expert availability, tool support, and level of uncertainty in integration risk areas. Integration Stability: Indicates anticipated change in integration components during system of system integration activities. Component Readiness: Indicates readiness of component systems for integration. User evaluates level of verification and validation that has/will be performed prior to integration and the level of subsystem integration activities that will be performed prior to integration into the SoS integration lab. Integration Team Capability: Represents the anticipated level of integration team cooperation and cohesion, integration personnel capability and continuity, and integration personnel experience with the application, language, integration tools, and integration platform(s). Maturity of the Integration Processes: A parameter that rates the maturity level and completeness of an integration team s integration processes, plans, and the SOS integration lab. Feedback from CSE Affiliates and Industry Experts. In July and October 2004, the USC CSE conducted workshops to discuss the COSOSIMO model developed to date and to obtain feedback from experts in industry and academia. [Lam] Workshop attendees included representatives from Aerospace Corporation, BAE Systems, Boeing, CIA, Disciplined SW Consulting, FAA, Galorath, General Dynamics, Lockheed Martin, Northrop Grumman, PRICE Systems, Raytheon, SAIC, Sparta, Inc., University of North Carolina, US Army Tacom, and USC CSE. In addition, representatives from the Software Engineering Institute (SEI) provided feedback in other review sessions outside of the workshop. As a part of the workshops, the COSOSIMO development team conducted a survey with respect to the relative importance and applicability of the proposed COSOSIMO size drivers and scale factors. Several 4

5 interesting themes were present in the feedback received. First, the concept of an SoS varies considerably across the multiple organizations participating in the workshop. For some, it may be a multi-system architecture that is planned up-front by an LSI. For others, it is an architecture that evolves over time, often driven by organization needs, new technologies appearing on the horizon, and available budget and schedule. The evolutionary SoS architecture is more of a network architecture that grows with needs and available resources. Because the concept of an SoS varies so much, it was suggested that the CSE may want to better define what an SoS is with respect to the COSOSIMO model. Second, there was considerable discussion about the selected size drivers and scale factors. Many agreed that the initial drivers and scale factors were relevant. However, they may not be complete or they may be difficult to assess in the early stages of the architecture development. Key comments and recommendations provided either during discussions at the workshops or in survey responses included: Software size is probably not known in the early stages. There needs to be an early-design version that does not depend on software size. The COSOSIMO model team should consider the number of types of components and types of interfaces. Size drivers should focus on the number of interface protocols and the number of independent system component organizations to coordinate with in the development of the SoS. Compatibility of existing interfaces should also be considered. Other technical factors to consider should include the complexity of the SoS protocols, the maturity of the SoS architecture, and the level of system component changes being implemented in parallel with the SoS architecture modifications. Management decisions, such as cost and schedule compression, may also have a significant impact on costs. There should be factors to adjust effort for political influences and the level of required negotiations and collaboration between the SoS and component system organizations. Upon further investigation and follow-up discussions, it became clear that the actual work in developing an SoS architecture is in the interfaces and the types of communications supported between the system components. This leads to a more netcentric view of an SoS, illustrated in Figure 3, and a need to measure the size of interfaces and communications. Net-Centric SoS Figure 3. Net-Centric View of SoS. Impact of Feedback on COSOSIMO. As a result of this feedback, the initial version was identified as the post-architecture version and an alternative early-design version of the COSOSIMO model was developed. The early-design model, illustrated in Figure 4, is of the form: where: m SoS L PM = A[Σ C IPi + Σ C SCOj ] B i=1 j=1 n 5

6 Size Drivers Number of SoS interface protocols Number of independent system component organizations Exponential Scale Factors SoS architecture maturity Cost/schedule compression Integration risk resolution Component system maturity and stability Component system readiness Integration team capability Integration processes COSOSIMO SoS LSI Effort Figure 4. Early-Design COSOSIMO Operational Concept. L PM LSI architecture definition and integration effort in personmonths. A Constant derived from historical data. C IPi Complexity factor associated with the i th SoS interface protocol. C SCOj Complexity factor associated with the j th SoS component system organization. m Number of interface protocols supported by the SoS architecture. n Number of organizations providing independently developed and maintained system components for the SoS. B Effort exponent for the SoS based on the SoSs exponential scale factors. The geometric product of the scale factors results in an overall exponential effort adjustment factor to the nominal effort. The early-design size drivers shown in Figure 4 focus on quantifiable sizes associated with the SoS interface protocols and the political complexities of negotiating agreements with other organizations to support the required system component changes needed to enable the SoS protocols. They are defined as: Number of SoS Interface Protocols: The number of distinct interface protocols to be provided by the SoS framework. Number of Independent System Component Organizations: The number organizations that are providing system components that will operate within the SoS framework. Each of these size inputs is weighted with respect to expected complexity. The interface protocol complexity is determined by factors such as number of protocol layers, desired security, and whether this is a new protocol 6

7 being implemented for the first time or an existing protocol. The independent system component organization complexity is determined by the expected level of cooperation between organizations and the competing needs of the SoS versus the needs and schedules of the independent component system provider or sponsor. The early-design scale factors, also shown in Figure 4, have been selected to capture the affects of key development processes and business/political decisions. They are defined as: SoS Architecture Maturity: A parameter that represents the level of maturity of the SoS architecture. It includes the level of detail of the interface protocols and the level of understanding of the performance of the protocols in the SoS framework. Cost/Schedule Compression: The extent of business or political pressures to reduce cost and schedule. Integration Risk Resolution: Same as the initial post-architecture version. Component System Maturity and Stability: A multi-attribute parameter that indicates the maturity level of the system components (number of new component systems versus number of component systems currently operational in other environments), overall compatibility of the system components with each other and the SoS interface protocols, the number of major component system changes being implemented in parallel with the SoS framework changes, and the anticipated change in the component systems during SoS integration activities. Component System Readiness: Same as the initial post-architecture version. Integration Team Capability: Same as the initial post-architecture version. Maturity of the Integration Processes: Same as the initial post-architecture version. Conclusions The current focus of the COSOSIMO model is to determine the key size drivers and scale factors associated with the LSI architecture and integration effort. Initial efforts were somewhat successful, as indicated in survey responses from representatives from industry and academia involved in SoS development. However, this same input indicated that while many of the initial drivers and factors selected influenced the SoS effort, they would be difficult to estimate in the early stages of development. Efforts should be more focused on an early-design version of the COSOSIMO model. Workshop discussions and surveys also indicated that there should be a considerable influence from political aspects of the SoS development effort, business decisions, and the complexities of people-communications across the development organizations. The early-design version of the model presented in this paper reflects the recommendations of workshop participants. It drops size drivers and scale factors that are relatively unknown or difficult to estimate in the early on. It replaces them with other relatively independent drivers and factors that are thought to have significant influence on the SoS LSI effort and are more readily discerned in the early stages of SoS inception and concept development. Future Plans Current plans are to continue to evolve both the initial post-architecture and the earlydesign versions of the COSOSIMO model. The next steps will be the review of the updated COSOSIMO model versions at another workshop with the USC CSE affiliates in March In addition, candidate sources 7

8 for actual data have been identified and efforts to capture this data for model validation and calibration have started. References Boehm, B., Abts, C., Brown, A. W., Chulani, S., Clark, B. K., Horowitz, E., Madachy, R., Reifer, D., Steece, B., Software Cost Estimation with COCOMO II. Prentice Hall, Boehm, B., Valerdi, R., Lane, J., and Brown, W., COCOMO Suite Methodology and Evolution, CSE Working Draft, January ISO/IEC 15288, Systems Engineering System Life Cycle Processes Jamshidi, M., System-of-Systems Engineering - a Definition, IEEE SMC 2005, Hawaii, October Lam, A. and Lane, J., COSOSIMO Workshop Minutes, 19 th Forum on COCOMO and Software Cost Modeling, USC CSE, October 26, Lane, J., Constructive Cost Model for Systemof-Systems Integration, International Symposium on Empirical Software Engineering, Los Angeles, California, August Maier, M. W., Architecting Principles for Systems-of-Systems, Systems Engineering, 1:4, pp , Sage, A., and C. Cuppan. "On the Systems Engineering and Management of Systems of Systems and Federations of Systems." Information, Knowledge, and Systems Management 2 (2001): years of software engineering and development experience in software development, software project management, software process definition and implementation, and metrics collection and analysis. Ms. Lane earned her BA degree in Mathematics and her MS degree in Computer Science from San Diego State University. Biography Ms. Jo Ann Lane is currently a PhD student at USC working in the area of System Architecting and Engineering. Prior to joining the USC PhD program, Ms. Lane was a key technical member of Science Applications International Corporation s (SAIC) Software and Systems Integration Group. She has over 8