Methodology for Selecting the Preferred Networked Computer System Solution for Dynamic Continuous Defense Missions

Transcription

1 Methodology for Selecting the Preferred Networked Computer Solution for Dynamic Continuous Defense Missions San Diego Dr. Glenn S. Tolentino Command & Control and Enterprise Engineering Department SPAWAR s Center Pacific Dallas, TX, USA Dr. Jeff Tian Computer Science and Engineering Department Southern Methodist University Dallas, TX, USA Dr. Jerrell T. Stracener Engineering Management, Information, and s Department Southern Methodist University Abstract This paper presents a methodology for addressing the challenges and opportunities in defining and selecting the preferred Networked Computer (NCS) solution in response to specified United States Defense mission planning requirements. The identified set of mission requirements are aligned with existing computer system capabilities allowing them to be acquired and processed as candidates to be included as part of the preferred NCS solution. In performing the proper selection process, decision making process is required in being able to properly select the preferred NCS by utilizing associated models for analysis. The models will then be applied towards NCS mission planning in analyzing an NCS solution s effectiveness in terms of operational availability, mission reliability, capability sustainment and lifecycle cost. The analysis and models were developed in response to the need to develop defense mission planning capability solutions by utilizing existing computer systems enabling the Department of Defense acquisition professionals to perform a practical approach in selecting and defining the preferred NCS for satisfying a mission. Keywords Mission reliability; sustainment reliability; operational availability; basic reliability; networked computer system; system of systems I. INTRODUCTION There s currently a methodology framework as part of this research that provides the process for modeling and analysis of an NCS s cost-effectiveness that provides the basis for ranking candidate computer systems and for selecting the preferred NCS solution. In this paper, we present models and describe the associated analysis for NCS mission reliability, sustainment reliability, operational availability and lifecycle cost that provide estimates for DoD acquisition managers to use during the decision-making process of defining the preferred NCS solution in response to defense mission planning requirements [6]. In developing this paper, there were ground rules that were determined and assumed. The following ground rules are described as part of the NCS costeffectiveness modeling and analysis: The constituent computer systems are currently operational, or will be within the required acquisition time. The NCS solution requires existing information from each of the owners of the constituent systems as input to cost-system effectiveness analysis. The NCS solution executes on a computer network whose availability and reliability is not considered as part of the NCS cost-effectiveness analysis. Each of the computer systems assumes data required to perform their respective capability is available. II. METHODOLOGY FACTORS AND THEIR RELATIONSHIPS NCS systems solutions that are formulated in response to DoD defense mission planning requirements may be compared in terms of systems cost effectiveness. NCS system cost effectiveness is a function of two factors, namely, system effectiveness and system cost. While there is not a standard definition of system effectiveness, we use a generally accepted definition consisting of three factors; system availability, system performance (form, fit, and function) and system mission reliability. In terms of system availability, it depends upon two factors: system sustainment reliability and system capability sustainment (Integrated Logistics Support plus Capability Upgrades) [3]. cost is the total cost of ownership over the system life cycles as specified by the requirements for a needed NCS for DoD mission planning. In general, system cost consists of Acquisition Cost plus the Operating and Sustainment Cost plus the Disposal Cost. Note that all NCS system cost effectiveness factors are correlated and related variables. For example, the mission reliability for a candidate NCS solution, a of s (SoS), can be increased by adding redundancy, i.e., by putting an identical system in parallel to a NCS system. In adding a redundant system, that specific change in mission reliability configuration in term will increase NCS mission reliability, and may, or may not, increase system effectiveness. But, it will increase system cost while decreasing NCS availability as a result of decreasing NCS Sustainment Reliability and increasing NCS Capability Sustainment. These relationships are indicated in Fig P a g e

2 Fig. 1. NCS cost effectiveness relationships. Two major contributions of the methodology that this study made to the current body of knowledge, and specifically to the DoD, are the capability to quantify (a) NCS costeffectiveness for required capability so that candidate solutions may be objectively compared and (b) the impact of trades among the NCS cost-effectiveness variables (see Fig. 2). Fig. 2. NCS cost-effectiveness interrelationships. III. NCS FORM, FIT, AND FUNCTION WITH RESPECT TO REQUIREMENTS Form, Fit, and Function is the identification and description of specific characteristics of a part, component, and/or assembly of a component or in this case the NCS. By developing the NCS based on requirements, it would allow for the possibility of making certain changes to the NCS while keeping the form, fit, and function of the solution. This is a critical consideration in selection of the computer systems and defining preferred NCS solution such that computer systems is unique but relevant, fits into the overall NCS, and functions as required in providing its capabilities. The form refers to specific dimensions, size, and physical characteristic of a NCS. In this case, the form is the computer systems and network architecture that makes up the NCS. In describing the overall characteristic of a NCS, the form is a comprehensive number of computer systems that is made up of hardware and software in performing a specific mission. The network architecture is also considered as part of the form as it provides the physical connectivity of the computer systems. The reference to an NCS fit refers to the ability for the computer systems that are parts of the NCS to be able to interconnect and interface with other computer systems. In this case of fit, computer systems communicate with one another through computer networking providing its capability at some time period. The ability for each of these computer systems to be able to connect, interface, interoperate, and communicate with one another satisfies the definition of the term fit in the case of an NCS architecture. In translating function of the NCS, the function refers to the purpose of each of the computer systems of the NCS in being able to perform its own capabilities. This defines that each computer system capability that is expected to perform in fulfilling its purpose. Furthermore, the capabilities performing its own function can be extended to the overall NCS mission as the overarching function. The NCS form, fit, and function requirements are based on overall requirements for an NCS for mission planning. The form, fit, and function of each candidate NCS is evaluated to determine whether or not the NCS for mission planning are met. If not, the candidate NCS is eliminated from consideration. IV. NCS MISSION RELIABILITY AND ANALYSIS There are methodologies for addressing mission reliability modeling and analysis being performed for individual systems [3], [4]. However, the methodology developed for this research and dissertation has been modified to accurately model constituent computer systems that are composed of both hardware and software while interconnected through computer networking in order to function as a SoS. In addressing mission reliability of a single constituent computer system, software and hardware reliability is provided as single reliability measures [2], [7]. Future research is considered in the area of the interrelationship between software and hardware reliability with respect to mission reliability. However, in this research of NCS mission reliability, the overall system reliability is considered as one measure for both hardware and software. Furthermore, the methodology 26 P a g e

3 developed addresses mission reliability in terms of a single NCS solution. The analysis starts with an overview of the NCS and its mission planning to develop the mission description, Concept of Operations (CONOPS), and mission definition and description. It is followed in describing the NCS mission profile, mission essential functions, success criteria, mission essential hardware and software for each phase. The following ground rules are described and assumed as part of the NCS Mission Reliability modeling and analysis: The NCS mission is deemed successful when all NCS phases are completed individually by each computer system capability. The NCS and its mission are considered failures when any of the constituent computer system in its respective NCS systems architecture phase fails during the mission. The constituent computer systems may have redundancy or failover capabilities if designed as a redundant system. The length of each of the NCS mission phases is determined prior to performing the mission. No failures are repaired at any time during the NCS mission. Each of the computer systems reliability measures is provided by each of the system owners. A. Overview and Process The model development and analysis overview is described and depicted with the following process (see Fig. 3). B. NCS Description, CONOPS, and Mission Definition and Description An NCS is described as a number of constituent computer system working and associated equipment for communicating with one another in order to accomplish a defense mission over some period of time. Each of the constituent computer systems is an integral part of the overall NCS by providing essential capabilities required during a phase or phases within the mission. Each of the capabilities are provided independently as individual computer systems. However, since each of the computer systems play a key role in the success of the mission, any computer system failure may be classified as a failure of the mission of the NCS depending on the type of failure and the functional configuration. While an NCS is described as a number of computer systems interconnected through computer networking in providing capabilities throughout the mission, there is typically one computer system that is considered to be the main system for operating and orchestrating through each of the capabilities. This command and control computer system is where an NCS user would utilize a mission staging system for collecting the results of the capability outputs and compiling the information as a final product. In the case of using an NCS during mission planning, the product would be an operational planning document that is used for a Department of Defense (DoD) operational purpose in both strategic and tactical execution. A concept of operation is described using the following notional CONOPS diagram (see Fig. 4). Fig. 4. NCS notional CONOPS. Fig. 3. Process for NCS mission reliability analysis. The model development and analysis of the NCS mission reliability starts with the mission definition and mission profile. The initiation of these steps in the process allows for the understating of the mission specific phases and times, functions, and success criteria of the NCS mission. In addition, these steps also characterize the various mission essential functions performed by the software and hardware equipment during each phase. The development of an NCS mission reliability mathematical model requires an understanding of the NCS equipment configuration and estimates of associated failure rates. Fig. 4 describes the characteristics of an NCS system by a user who will interact with the NCS during mission planning. This interaction with the NCS by the user is specified by the CONOPS and is the viewpoint on how the NCS will be utilized by that user. The NCS supports the DoD mission planning requirement to be responsive to any mission planning efforts given by the DoD. The NCS enhances the DoD s ability to support rapid and emerging response to capabilities necessary for crisis and deliberate planning. The NCS provides the interconnection between varying capabilities that enables mission planning to accomplish critical tasks in areas of intelligence information, operational capabilities, imagery integration, and command and control that is vital in executing search and rescue, combat readiness, physical disruption, and 27 P a g e

4 cyber operations. The NCS provides the means to continue the time sensitive synthesis of information allowing mission planning products to be developed with accurate information and performed in a timely manner. In order to perform mission reliability modeling and analysis of a NCS, a mission definition and description must be developed. In defining a mission plan, the NCS mission follows the format of the Joint Operation Plan (JOP) publication process. Joint operation planning begins when an appropriate authority recognizes potential for military capability to be employed in response to a potential or actual crisis. At the strategic level, that authority the President, Secretary of Defense, or Chairman of the Joint Chief of Staff initiates planning by deciding to develop military options. The JOP published a process consisting of activities associated with joint military operation planning performed by military leadership in response to deliberate and crisis action activities. The publication process is a well-defined process in order to address challenging military operations and activities. The JOP provided a framework in which an NCS is developed in order to execute a defense mission plan. Table I describes an NCS mission planning process in in leveraging the JOP. In determining the capabilities required by the NCS, it is important to understand the mission through the mission analysis phase. During the mission analysis phase, the process determines what systems are required to be able to provide the required capabilities in planning the right mission with the best capabilities. In this phase, mission analysis is utilized to research and study the assigned tasks and their objectives along with identifying all the required tasks necessary to accomplish the mission. Therefore, the mission analysis phase clearly defines the mission, the objectives the mission is trying to satisfy during each of the phases. In developing a generalized NCS mission, the following table (see Table II) depicts a mission characterized with i number of phases with an n number of systems. The table describes how each of the systems is utilized throughout the various phases of the mission. The notional mission description provides a generalization of an NCS mission and provides a holistic approach in describing a process to model and analyze the NCS. Furthermore, the NCS mission can be characterized as a number of phases that are performed sequentially to satisfy the mission objectives. In order to measure the NCS mission quantitatively, one must understand when each of the computer systems will be used and at what phase(s) of the NCS mission. This method will result in a mission reliability mathematical model given an NCS mission, the essential computer system capabilities, and the phases requiring each of the capabilities over some time phased period. The failure of the NCS to be able to perform its mission means that capability performed by one or more of the constituent system was not successful or failed to perform. In an NCS architecture, if a capability in any of the phases is not performed, then the reliability of the overall NCS mission is jeopardized. A mission description includes elements in each phase, the length of each of the phases, essential functions, and success criteria for each of the critical systems involved in the overall mission of the NCS. TABLE I. GENERAL NCS MISSION PLANNING PROCESS Planning Initiation Mission Analysis Course of Action Development Course of Action Analysis Course of Action Comparison Course of Action Approval Plan/Order Development TABLE II. GENERAL NCS MISSION DESCRIPTION WITH I PHASES Phase No i Mission Phase Titles P1 P2 P3 P i Phase Length T 1 T 2 T 3 T i 1 S 1 Required Required Required Required 2 S 2 Required Not Required Not Required Not Required Constituent s of NCS : : : : : Not Required Required Not Required Not Required : Not Required Not Required Required Not Required N S N Not Required Not Required Not Required Required 28 P a g e

5 C. NCS Mission Profile, Mission Essential Functions by Mission Phase, Success Criteria, and Mission Essential Equipment (Hardware and Software) by Phase The NCS mission profile must be developed with respect to the duration of the mission, mission phases, and the duration of each of the phases. In addition, a success criteria must be defined in ensuring the mission essential functions at each of the phases. The NCS mission profile involves both hardware and software mission essential functions that must be considered individually and overall holistically. This functional dependency implies a level of coupling between software and hardware, making the two components to be highly dependent on each other. This in fact is the systems architecture typical of computer systems. Therefore, computer systems functionality should take in for account that hardware and software reliability should both be considered when quantifying the systems reliability in addressing the overall mission reliability [8]. Therefore, in this methodology application, it will be assumed that each of the computer system reliability has been calculated such that each computer system addresses both hardware and software reliability as one complete system reliability. Based on the mission description, a mission profile is developed for the NCS mission (see Fig. 5) and is represented by phase in terms of the constituent computer systems delivering their required capabilities during the specified phased time period. Fig. 5. Mission profile for a general NCS. Fig. 5 depicts a mission profile of an NCS where each of the constituent systems is used in sequence to perform its function during a phase of the mission. In ensuring that each computer system is ready to provide its intended function, an initial pre-check occurs prior to the start of the mission for that system. The purpose of the system pre-check is a formal verification that the computer system is all up, including any redundant elements prior to the start of the mission. In providing a specific capability, each constituent computer system is required to provide its Mission Essential Functions (MEF) during each phase of the mission in order to be successful. In an NCS mission planning scenario, the mission phases have success criteria during each of the phases, including mission analysis, course of action development, model and simulation, and approval process [9]. The success criteria describe that each of the computer systems was able to perform its capabilities during its phase. In order for these MEFs to satisfy the defined success criteria successfully, they depend on mission essential equipment of an NCS solution (see Fig. 6). In the case of an NCS, the mission essential equipment consists of both hardware and software elements. At a minimum, the software includes the operating system and the software application to provide the defined capability, whereas the hardware components consists of computer hardware elements such as a computer desktop, laptop, rack server, and tower, to name a few. Fig. 6. Mission phases, mission essential functions, and success criteria. D. Mission Reliability Block Diagram In developing an NCS mission reliability block diagram (RBD), the mission as a whole is analyzed in depicting mission success utilizing mission essential equipment as well as any alternate modes of operation and redundancy. Therefore, in order to properly develop a RBD, the following must be considered; system functional block diagrams, mission definition and profile, mission success criteria, and mission essential functions and equipment. The mission definition and profile along with the mission essential functions and equipment were described in the previous sections. In this section, a system functional block diagram and well defined mission are developed for use in developing the RBD. The NCS functional block diagram describes the interrelationships between the computer systems during the mission (see Fig. 7). In this diagram, it depicts which computer system provides commands and variables to another specific computer system providing output variables in order for the next computer to accept the output variables as input variables. This interrelationship continues during each phase of the mission requiring for the computer systems to successfully perform their objectives and eventually satisfying the criteria successfully. Fig. 7. NCS functional block diagram. 29 P a g e

6 The functional block diagram depicts the interrelationship of the computer systems of the NCS and how they are connected together in being able to receive input variables and provide output variables. In order to fully depict the successful satisfaction of the objectives of the computer systems during each phase, a comprehensive success criteria must be clearly defined up front. This will provide a means to measure whether the computer system or systems were able to provide the necessary capabilities as part of the mission. Table III presents success criteria during each phase of the mission. This table shows which systems are required for each of the phases and the success criteria to satisfy mission success. Based on the NCS configuration, each phase requires two computer systems collaboratively performing their capabilities satisfy the mission objectives. TABLE III. NCS MISSION SUCCESS CRITERIA Name Phase Mssion Success Criteria Joint-Coordinated Planning Special Operations Mission Analysis Joint-Coordinated Planning Planning and Effects Based Joint-Coordinated Planning Mission Modeling and Simulation Joint-Coordinated Planning Joint Mission Planning Request and Approval S1 S2 S1 S3 S1 S4 S1 S Mission phase accomplishment in describing the following based on mission intent:- Leadership goal- Military impact - Political impact- Policy impact Development accomplishment of course of actions with the following information:- Mission objectives- Mission tasks- Mission capabilities required- Mission sustainment concept- Mission deployment concept- Mission interagency task Successful simulation of COA to define the following:- Verifying COA flow- Mission risks- Asset estimation Successful approval workflow process and passing the following:- Submit COA to leadership for review- Leadership selection of COA for execution- Validating requirement of Leadership Intent with COA- Receive Leadership recommendation In understanding the mission as a whole, the system functional block diagrams, mission definition and profile, mission success criteria, and mission essential functions, mission essential hardware and software, provided inputs in developing a diagram for depicting how the computer systems contributes to the success or failure of the NCS solution [1]. The following NCS RBD is developed to better understand the mission phases, capability needs, and the notional redundancy that must be performed throughout the mission to accomplish mission success through mission reliability (see Fig. 8). Fig. 8. NCS reliability block diagram. E. Mission Reliability Mathematical Model In general, mission reliability,, is the conditional probability that a specific system will successfully perform its essential functions during a specified mission, given that all mission essential equipment is up at mission start. An NCS mission reliability mathematical model is used to evaluate system reliability in terms of: Probability of mission success Expected number of mission losses per 1,000 missions An NCS mission reliability mathematical model describes the mathematical relationship between probability of mission success and mission critical failure rates and configuration of mission essential equipment, including software, and success criteria [4]. The development of an executable math model requires the type of time to failure probability distributions. F. Time to Failure Probability Distribution In developing an executable NCS mission reliability model, the model requires a time to failure probability distribution. The default probability distribution for reliability analysis of computer systems and NCSs is the Exponential Probability Distribution, i.e., the random variable Time to Failure, T, has the Exponential Probability Distribution with parameter. In this application, is the failure rate. Therefore, the probability of an NCS element surviving a period of time t during a mission is ( ), and, is the mission critical failure rate. If all NCS elements, i.e. computer systems, time between failures can be characterized by the exponential probability distribution with failure rate, then the NCS mission reliability model was developed from the RBD in Fig. 8. The mission Reliability Block Diagram by mission phase is as follows (see Fig. 9): 30 P a g e

7 , Fig. 9. NCS mission reliability block diagram. where is the mission critical failure for i=system number=1,2,3,4,5 and j= phase number =1,2,3,4. We derived the following NCS Mission Reliability Model from the above RBD as follows: where ( ) ( ) ( ) is the probability of NCS mission success for the defined mission of duration, and ( ) is the probability of NCS success for mission phase for k=1,2,3,4 where ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ), and ( ) ( ) ( ) Note that use of the above formula for NCS mission reliability overstates the true value since the per phase reliabilities are conditional probabilities of success, given that are mission essential systems are all up at the beginning of a phase. The mission reliability block diagram and mission reliability model may be developed for a general NCS having a well-define mission by utilizing the above methodology. G. Estimation of Failure Rates for Mission Essential Equipment Hardware and Software If the time to failure probability distribution for a NCS element, E, is Exponential with parameter,, its failure rate, then the failure rate of element E is estimated as follows where is the cumulative number of mission critical failures, associated with both hardware and software, that occurred in cumulative time. H. Mission Reliability Analysis NCS mission reliability analysis provides an estimate of the probability successfully completing a well-defined specified NCS mission, given that a successful permission NCS check is completed prior to mission start. NCS mission reliability for a specified mission depends on the NCS reliability configuration, mission time and mission critical failure rates of mission essential hardware and software. Since the methodology that we developed is applicable to NCS solutions utilizing existing systems to the extent feasible, the failure rates of mission essential equipment must be provided by the computer system owners as estimates. And for a specified mission, the mission time,, is a specified value. Therefore, the only variable in NCS mission reliability is the reliability configuration. Redundant reliability configurations, such as active parallel and standby, can be considered as alternatives in NCS mission reliability analysis. In general, use of redundancy will increase mission reliability. But with an associated penalty in, and Lifecycle Cost [6]. V. NCS SUSTAINMENT RELIABILITY MODEL AND ANALYSIS sustainment reliability,, in general, is the mean time between failures, over a specified period of calendar time. Time, T, and failure, F, must be defined for a particular system type. Time, T, is often defined to be system operating time for a specified system or for all systems of that type. Failure, F, is usually defined to be an event that results in a response to an indicated malfunction. Under this definition, failure can range from no fault found to loss of system function due to a physical failure of a part. Failure may be associated with systems hardware, software and hardwaresoftware interaction. The formula for calculating depends on the type of probability distribution of the random variable T, the time between failures. The default distribution for NCSs is exponential. In which case, then where is the failure rate of the i th element of the j th computer system for j=1,2,,k and 31 P a g e

8 The sustainment reliability of the NCS is: relative short calendar time. provides a mean time, a mathematical expectation of operating time between indicated malfunctions. The is a factor in calculating NCS operational availability and life cycle cost. It is not a factor in calculating mission reliability since only mission critical failures of mission essential equipment (hardware and software) may cause mission failure. VI. NCS OPERATIONAL AVAILABILITY MODEL AND ANALYSIS operational availability, in general, is defined to be the likelihood or probability that the system is capable of initiating its required mission at any given point in calendar time. Operational availability is a measure of system readiness. For a NCS for mission planning, the operational availability,, is defined as, where is NCS sustainment reliability and MDT is the mean down time of the NCS for all causes including time associated with preventative unscheduled and scheduled maintenance, downtime due to supply, administrative downtime, and support equipment downtime. MDT is also considered in the equation as a basic measure of maintaining and repairable items. In a case of NCS hardware requiring repair time, the hardware is either repaired or replaced which requires some down time as far as operations. Since the NCS also has software components, MDT also applies to software if required in the form of reboot, rejuvenation, or installation time requiring down time of the software. MDT is also an important consideration as average time a system is awaiting maintenance which requires time in troubleshooting, remediation, while following process and procedures to address any maintenance requirements. In using the obtained parameters, the and MDT estimates the availability of the NCS capable of its assigned mission at a given time. Since the methodology that we developed is based on using existing systems to define candidate NCS solutions,, and MDT are estimated from data provided by their owners. VII. NCS LIFE CYCLE COST MODEL AND ANALYSIS NCS life cycle cost is the total cost of a candidate NCS over its lifecycle as defined by DoD in Fig. 10. Since the methodology that we have developed is applicable only to NCS development using existing computer systems to the extent feasible in meeting NCS requirements, the activity in the first four phases involves planning, architecture development, identification, of existing computer systems and definition of candidate systems, acquisition planning and integration of existing systems. This activity is generally minimal comparted to new development NCS with Fig. 10. DoD NCS life cycle. Therefore, the first four life cycle (LC) phase is associated with the preferred NCS acquisition and its cost is nonrecurring. The NCS operations and support phase consists of two activities, namely, (1) providing the mission planning capability required and (2) supporting the NCS to sustain the required capability [5]. The cost associated with this phase is recurring. VIII. LIFE CYCLE COST MODEL The lifecycle cost model for a specific DoD required NCS capability is where is the NCS acquisition cost, nonrecurring, that is required (estimated) to accomplish the following activities by DoD life cycle phases (see Table IV): and is the NCS operations and support cost, recurring, that is required (estimated) to provide NCS operations and the associated support cost over this DoD life cycle phase consisting of the following activities (see Table V): The NCS support cost,, depends on maintenance cost and logistics support cost, and can be estimated as follows: ( ), where is the NCS sustainment reliability defined in 5.4 and is the average cost of an indicated malfunction of the NCS, including cost of manpower, material and other logistics support resources. 32 P a g e

9 TABLE IV. Material Solution Analysis Phase NCS ACQUISITION PHASES Analyze and define mission description and goals Analyze and define mission objectives Analyze and define mission requirements Analyze contractual cost of existing computer systems capabilities Technology Development Phase Determine requirements can be met by existing computer systems capabilities Develop preliminary design of the NCS Develop configuration items from existing computer systems capabilities as initial candidates for considerations Finalize preliminary design based on configuration items Engineering & Manufacturing Phase Develop master integration plan for developing an NCS Identify candidate computer systems Develop NCS candidate systems Select the preferred NCS solution Develop operations and support plan Production & Development Phase Develop initial working NCS prototype to ensure communication and computer system capabilities are functional across the computer network Integrate existing computer systems to obtain the full working NCS Perform operational test and evaluation Acquire NCS operations and support resources TABLE V. NCS Capability Sustainment NCS OPERATIONS AND SUPPORT PHASES Manage the system overall to ensure its operating and providing users with their needs Operate the system in terms of support with a help desk center (tier 1 to tier 3 help desk engineering support) Ensure the computer systems are working properly by replacing and upgrading equipment as required and ensure the computer network is functioning Support the NCS in terms of required training, maintenance, and logistics support NCS Disposal Develop plan for NCS disassemble Approve contractual disposal agreement Disassemble NCS capabilities Complete with engineering efforts in ensuring computer system capabilities are returned or disposed by system owner plan A. Life Cycle Cost Analysis An analysis is performed to determine (estimate) the life cycle cost of each candidate NCS for a specified DoD mission planning NCS [6], [9]. The life cycle cost may be used in conjunction with corresponding to characterize the cost effectiveness for each candidate NCS [6]. NCS life cycle cost analysis utilizes input date from various sources, ranging from actual recorded costs to forecasts based on extrapolation, to obtain a best estimate. Since the uncertainty associated with the estimated NCS life cycle cost may be large, sensitivity analysis may be utilized to identify life cycle cost drivers and to assess the effect of variation of input data on NCS life cycle cost. B. Life Cycle Cost Summary NCS life cycle cost analysis must be performed to ensure that a candidate NCS solution meets the cost requirement for a specific DoD required capability for mission planning. In addition, life cycle cost analysis results in an estimate, that when used in conjunction with and provides an estimate of the cost effectiveness of each candidate NCS solution. IX. SUMMARY AND CONCLUSION In the development of this paper, we discussed specific factors and their relationships with respect to NCS system cost effectiveness. These specific factors in cost effectiveness helped identify the critical elements in producing an NCS solution that could be modeled and analyzed. In addition, the research modeled specific factors in dealing with mission reliability, sustainment reliability, operational availability, and lifecycle cost that are essential drivers in producing a costeffective NCS solution. Also, during the mission reliability modeling, the scope of mission description, profile, and success criteria, along with reliability block diagrams was produced in understanding NCS mission as whole. This process provided a complete overall picture of the NCS mission and thus providing a detailed model to be used for analysis. The development of the mission reliability model and analysis provided an overarching perspective on the understanding of NCS mission and the factors that are important and critical for mission success. This is essential in establishing a measurement for mission success and is the contributing factor as part of the decision making process of NCS stakeholders in determining the preferred NCS in support of U.S. DoD Mission Planning missions. REFERENCES [1] Chelson, P. O. and Eckstein, R. E., "Reliability Computation from Reliability Block Diagrams," Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California1971. [2] Council, N. R., Reliability Growth: Enhancing Defense Reliability. Washington, D.C.: National Academy Press, [3] Defense, D. o., "ELECTRONIC RELIABILITY DESIGN HANDBOOK," D. o. Defense, Ed., MIL-HDBK-338B ed. Washington, DC, [4] Defense, D. o., "Reliability Modeling and Prediction," vol. MIL-STD- 756B, D. o. Defense, Ed., ed. Washington, DC: Department of Defense, [5] Defense, D. o., "Reliability Program for s and Equipment Development and Production," vol. MIL-STD-785B, D. o. Defense, Ed., ed. Washington, DC: Department of Defense, [6] Defense, O. o. t. S. o., "Operating and Support Cost-Estimation Guide," O. o. t. S. o. Defense, Ed., ed. Washington, DC: Cost Assessment and Program Evaluation (CAPE), [7] Friedman, M. A., Tran, P. Y., and Goddard, P. I., "Hardware/Software Reliability Modeling," in Reliability of Software Intensive s (Advanced Computing and Telecommunications Series), 1st Edition ed: William Andrew, [8] Prasad, M., Flowrence, L., and Srikrishna, C. V., "Overview of Software Reliability Models," International Journal of Engineering and Management Research, vol. 3, pp , October [9] Staff, J. C. o., "Joint Operation Planning," D. o. Defense, Ed., 5-0 ed. Washington DC, P a g e