Designing a Supportable System Design decisions made early in the systems engineering process can have a significant long-term impact on the operational effectiveness and cost of a system once it is deployed. For this reason, early in the design process, acquisition professionals need to take into account how the system will ultimately be used and supported in the field. In this lesson, you will learn about reliability and maintainability, two of the most important characteristics that impact a system's supportability. You will see that reliability, maintainability, and supportability need to be established for the system early in the acquisition process and kept in focus as a program moves through the Acquisition Lifecycle Framework (ALF). The goal is to balance higher operational effectiveness with lower life cycle costs. You will also learn how reliability and maintainability impact the availability of a system. You will also learn how to use a decision matrix, a decision analysis process that can help you make choices and recommendations on a wide range of issues. You may print the Designing a Supportable System lesson or save it for future reference. Page 1 of 31
Objectives Upon completion of this lesson, you should be able to: Define reliability and explain how to measure it Define maintainability and explain how to measure it Define supportability and explain how to evaluate it Define availability and explain how to measure it Explain how to use a decision matrix to make system design decisions Page 2 of 31
Reliability, Maintainability, and Supportability (RMS) Reliability is a measure of the probability that an item can perform its intended function for a specified interval under stated conditions. Colloquially speaking, it is can be thought of as how long a system is likely to work before it breaks. Maintainability is the measure of how long, on average, it takes to repair and service a system. It indicates the item's ability to be retained in, or restored to, a specified condition when skilled personnel perform maintenance using correct procedures and resources. Supportability is the ability of a system's design and planned logistics resources to meet requirements for operations and readiness at an affordable cost throughout the system's life. Reliability and maintainability both impact supportability. Supportability is also impacted by activities and resources that are necessary whether the system fails or not, such as fuel or personnel. Page 3 of 31
Elements of Reliability Reliability is a measure of the probability that an item will perform its intended function under stated conditions for either a specified interval or over its useful life. Reliability has four key elements: Quantifiable (probability): Reliability is expressed in probabilistic terms, with the highest probability being 1 and the lowest 0. For example, the bulb in this projector has a 0.8 probability of success (reliability) for 30 hours of operation. System performance (intended function): To design for reliability, we must know the exact function an item is meant to perform. For example, the intended function of the bulb is to provide 600 lumens of light in order to project words and pictures onto a screen. Conditions and environments (stated conditions): To design for reliability, we must specify the kind of environment and conditions under which the item must function. For example, if the projector is a portable unit, then it would likely be subjected to greater variances in temperature and vibration than a permanently mounted one. The design of the bulb should take these factors into account. Time period (specific interval): Finally, we must consider how long the item needs to perform its function. For example, if our projector will be operated in a classroom for 50 hours, and the light bulb has a reliability of 30 hours, we can expect system failure to occur around 30 hours of use. The reliability of the end item is dependent upon the reliability of the critical sub-components. Page 4 of 31
Elements of Reliability (continued) In colloquial terms, reliability may be thought of as how long the system is likely to operate without failing. For example, Customs and Border Protection (CBP) needs an all-terrain vehicle that can operate in the desert under harsh conditions and extreme temperatures. To do that, the engine has to be durable, the cooling system has to handle extremely high temperatures, and all the critical components have to function under a wide range of environmental conditions. The longer the vehicle operates between repairs, the more reliable the vehicle, and the more satisfied the user. Page 5 of 31
Measure of Reliability: Mean Time Between Failures One way to measure reliability is mean time between failures (MTBF), which indicates how long an item or system will work before it fails. MTBF is the average time interval between failures of repairable equipment. In other words, MTBF is the average amount of time between one failure, its correction, and the onset of a second failure of the same component, subassembly, or system based on the entire population of equipment. There are many variations of MTBF, such as mean time between system abort (MTBSA) or mean time between critical failures (MTBCF). Such terms are used to differentiate among types of failures. For example, in an automobile, the failure of the FM radio does not prevent the operation of the vehicle, so we may want to differentiate the failure rates of critical versus non-critical components. In cases where a system is replaced after a failure, the term is mean time to failure (MTTF). Let's look at an example of MTBF. Page 6 of 31
Reliability: MTBF Example The flight control system of the Coast Guard's C-130J aircraft is comprised of nearly one hundred subsystems. To determine the reliability of the flight control system, C-130J aircraft are flown for a total of 100,000 hours. During that time, 20 failures of the flight control system occur. What is the MTBF for the C-130J aircraft flight control system? This indicates that the flight control system will fail, on average, once every 5,000 hours of operation. Page 7 of 31
Knowledge Review The question, "How long will it work before it fails?" relates to which of the following? A. Reliability B. Maintainability C. Availability D. Supportability Correct! Reliability is a measure of the probability that an item can perform its intended function for a specified interval under stated conditions. In other words, reliability indicates how long a system will work before it breaks. Page 8 of 31
Elements of Maintainability Maintainability refers to how quickly, easily, and cost-effectively a system can be returned to operational status after preventive or corrective maintenance is performed. As a design characteristic, maintainability includes two primary elements: Ease of locating failures or potential failures. This is mainly accomplished by designing and installing diagnostics (reactive failure identification) or prognostics (predictive failure identification). Ease of repairing failures. This is where the "human factor engineering" aspects of the system's design play a key role. The ability to quickly find and repair failures reduces a system's down time and lowers maintenance costs. Page 9 of 31
Maintainability and Human Systems Integration One determinant of maintainability is human systems integration, which has several aspects: Accessibility can the part be easily accessed for repair? Visibility how easily can you see the part being worked on? Testability how easy is it to test and detect faults? Standardization are parts interchangeable, and can standard tools be used? The more user-friendly the design, the faster the repair and upkeep can be performed. Page 10 of 31
Measure of Maintainability: Mean Time to Repair A basic measure of maintainability is mean time to repair (MTTR), the average time it takes to fully repair a failed system. It indicates the ease, accuracy, and economy of performing a maintenance action. MTTR is the sum of corrective maintenance time divided by the total number of repairs. Corrective maintenance typically includes fault isolation, removal and replacement of failed item(s), and checkout. MTTR is calculated as follows: Let's look at an example. Page 11 of 31
Maintainability: MTTR Example Over the past five years, a Transportation Security Administration (TSA) passenger screening station has accumulated 1,200 hours of total corrective maintenance time. During this same period, a total of 60 repairs (corrective maintenance actions) have been required. What is the MTTR? This indicates that it takes an average of 20 hours to repair the passenger screening station. Page 12 of 31 Corrective maintenance times typically include fault isolation, removal, replacement of failed items, and checkout.
Maintainability versus Maintenance Don't confuse maintenance with maintainability. Maintenance is an activity that is performed on a system, an item of hardware, or software. It refers to those actions performed by an operator or maintainer to retain an existing system in, or restore it to, an operable condition. Maintainability is a characteristic of a system or item. It's a result of the actions taken by the system designer to incorporate design features that enhance ease of maintenance. Good maintainability allows for rapid diagnosis and removal of faults. For example, when your car does not start because of a simple, 10-cent, 15-amp fuse, you can quickly locate the problem by using the manufacturer's car manual to diagnose what's wrong. Then you can replace the fuse in only a few seconds since the fuse panel is easy to access. Page 13 of 31
Knowledge Review Which of the following is a primary element of good maintainability? A. The system is designed to run a long time before it breaks down B. The system is designed so that it performs consistently, yielding uniform results when operated under specified conditions C. The system is designed so that it's quick and easy to locate a failure, isolate the cause, and make the repair Correct! Good maintainability is designed in from the beginning to allow problems to be quickly diagnosed and easily repaired. Page 14 of 31
Supportability Defined Supportability is the ability of a system's design and planned logistics resources to meet requirements for operations and readiness at an affordable cost throughout the system's life. Reliability and maintainability have a significant impact on supportability. The term "supportability" can therefore refer to RMS as a whole. Unlike reliability and maintainability, supportability is also impacted by planned activities and resources (such as fuel) that are necessary whether the system fails or not. It also includes all resources (e.g., personnel, equipment, technical data, etc.) that contribute to overall support cost. The plans for these activities and resources directly impact the ability of the system to affordably meet requirements for operations and readiness. Supportability must be designed into a system from the very beginning. To make supportability affordable, we need to identify support parameters at the start of the design process, state those parameters in operational terms, and include them in system specifications. D Page 15 of 31 A chart shows the relationship between Reliability, Maintainability and Supportability (RMS). Reliability and Maintainability both impact Supportability. Supportability is also impacted by planned activities and resources, such as personnel and equipment, that are necessary whether the system fails or not.
Up Front and Early Supportability directly affects not only operational readiness, but also operations and support costs. In general, over 70% of a system's life cycle costs are "Operations and Support" and "Disposition" costs, which are primarily incurred after deployment has begun. Most of those costs are fixed based on program decisions made around the time of acquisition decision event 2B (ADE-2B), during the Obtain Phase of the ALF. Supportability considerations therefore have to be addressed up front and early. D Select the D-link to read a detailed explanation of the graphic Page 16 of 31 The four life cycle cost categories, Research and Development, Investment, Operations and Support, and Disposition, are shown in chronological order over the ALF. The cost categories are shown as four bell curves of different sizes arranged on the X axis of an XY graph. The Y axis is labeled Money and the X axis is labeled Time. The different-size bell curves notionally represent the different amounts of money spent in each category over time. The bell curves partially overlap, showing that part of the spending in each cost category occurs at the same time as spending in adjacent cost categories. Research and Development (R&D) costs are incurred first, roughly from ADE-0 to ADE-2B. The next bell curve represents Investment costs and is about twice the size as R&D costs, showing that more costs are incurred during this phase. It runs roughly from ADE-1 to the middle of the Produce/Deploy/Support/Dispose (P/D/S/D) Phase. The next bell curve, Operations and Support costs, is about twice the size as Investment costs. It begins around ADE-3 and ends just before the end of the P/D/S/D Phase. The last, smallest bell curve is Disposition costs. It is slightly smaller than R&D costs, and begins and ends at the very end of the P/D/S/D Phase.
A Balanced Design During design and development, supportability requirements must compete with other requirements to achieve a balanced system that best meets the user's requirements. The logistician must influence system design for supportability and consider the entire infrastructure needed to sustain the system once it is deployed. In other words, system design must take into account that the system will require logistics support: upkeep, repair, trained operators, supplies, support equipment, technical data, shipping, storage and handling, etc. Page 17 of 31
Reliability, Maintainability and Supportability (RMS) Reliable, maintainable, and supportable systems are achieved through a disciplined systems engineering approach employing the best design, manufacturing, and support practices. The two goals of RMS are: Higher operational effectiveness Lower ownership costs The more reliable the system, the less it costs to operate and maintain it, and the less of a logistics footprint is imposed on operational units. In order to achieve RMS, program managers, engineers, and logisticians must place emphasis on: Understanding user requirements Managing RMS contributors Preventing design deficiencies Developing robust systems Page 18 of 31
Achieving RMS How does a program office select the correct parameters to achieve RMS? A system that meets performance parameters, but not RMS objectives, will be very costly to operate and have limited effectiveness. To ensure cost-effective operation, we need to take the following actions during the appropriate acquisition phase: Evaluate the system's design using estimated or measured RMS characteristics Allocate RMS objectives through the system's design hierarchy Translate RMS objectives into quantifiable contract terms that are traceable to operational requirements Verify contract requirement achievement using engineering analyses and test and evaluation Page 19 of 31
Evaluating RMS Because RMS is so important, it must be evaluated throughout the design and development process. Supportability analysis is used as part of the systems engineering process to influence design, as well as to determine the most cost-effective way to support the system throughout its life. Here are some tools available to evaluate supportability: Failure modes and effects criticality analysis (FMECA) examines each failure in order to determine and classify its impact on the entire system Reliability centered maintenance (RCM) uses a scheduled maintenance approach to identify failures before they degrade system effectiveness Test, analyze, fix and test (TAFT) detects and eliminates design weaknesses in a simulated operational environment using a systematic, iterative process Page 20 of 31 An analytical tool, conducted as part of the systems engineering process, to determine how to most cost-effectively support a system over its entire life cycle. Supportability analysis provides the basis for related design requirements that may be included in specifications. It refines readiness requirements, plans the system's support structure, and compares existing support structures to new requirements.
Knowledge Review Customs and Border Protection (CBP) has embarked on the acquisition of unmanned aerial vehicles (UAVs) with infrared equipment to detect illegal firearms being transported into the United States. What action should the PM take to ensure that the UAVs will meet user requirements for reliability, maintainability, and supportability (RMS)? A. Translate RMS objectives into quantifiable contract terms that are traceable to operational requirements B. Validate any proposed scope, schedule, or budget changes during the course of the program C. Verify contract requirement achievement using an earned value management system Correct! Translating RMS objectives into contract terms is an important part of ensuring that RMS requirements will be achieved. Page 21 of 31
Availability Availability is a measure of the probability that the system will be in a working condition when it is needed. Specifically, it is the probability that an item will be in an operable state when a mission starts at a random point in time. The user is most concerned about this parameter because it reflects the readiness of the system. Reliability and maintainability have a direct impact on a system's availability. In general, the more reliable and maintainable an item, the greater its availability. Simply stated, availability is expressed as a ratio of: There are different types of availability. One of the most common is operational availability. Operational availability refers to a system in its actual support environment, taking into account both corrective and preventive maintenance. Page 22 of 31
Operational Availability In operational availability, uptime includes both operating time and standby time. It is measured in mean time between maintenance (MTBM). Downtime has four contributing factors: Corrective maintenance (the time it takes to make the repairs) Preventive maintenance (routine maintenance) Logistics delays (parts availability) Administrative delays (personnel or paperwork delays) The more reliable a system is, the longer the period of up time; the more maintainable a system is, the shorter the period of downtime. Let's look at an example. Page 23 of 31
Operational Availability Example Customs and Border Protection uses high speed patrol boats in offshore coastal waters to combat maritime smuggling. Uptime or MTBM for the boat is 1,000 hours. Downtime, a combination of corrective and preventive maintenance plus logistics and administrative delays, is 100 hours. What is the operational availability for the boat? Page 24 of 31
Decision Matrix When a system is designed, engineers and program managers must make hundreds of decisions, making trade-offs among multiple factors simultaneously, including factors such as reliability, maintainability, supportability, and availability. One way to methodically approach this is to use a decision matrix. In a decision matrix, the importance of each factor is numerically weighted. This produces quantitative results, which can often help to justify the final decision. Here is the decision matrix process: 1. Identify the items to be compared. 2. Establish performance factor evaluation criteria (e.g., MTBF, MTTR, delivery time, cost). 3. Assign weight to each criteria based on its relative importance (total to be 1.0). 4. Establish a quantitative rating scheme (e.g., scale from 1 to 5, where the higher number is better). 5. Rate and rank factors for each item using the established rating scheme. 6. Multiply the rating for each item by the assigned weight for each criteria. 7. Add the totals for each item. The highest score determines the best value. Let's look at an example that uses MTTR and MTBF as factors in the decision. Page 25 of 31
Decision Matrix Example TSA is purchasing new intra-airport vehicles for transporting security equipment from the warehouse to the airport, and a decision must be made between three sub-assemblies: Sub A1, Sub B2, and Sub C3. These subassemblies are the items to be compared (step 1). The performance and logistics integrated product team (PL IPT) decided to base the decision on four factors: reliability (MTBF), maintainability (MTTR), schedule, and cost. These are the evaluation criteria (step 2). Systems engineers performed tests on a number of sub-assemblies, and the results are displayed below. Sub A1: MTBF = 160 hours MTTR = 3 hours Delivery 10 weeks Cost $10K Sub A2: MTBF = 175 hours MTTR = 2 hours Delivery 11 weeks Cost $14K Sub A3: MTBF = 250 hours MTTR = 1 hour Delivery 14 weeks Cost $16K Which subassembly should TSA use? Before you make that decision, we need to assign a weight to each evaluation criteria. Page 26 of 31
Decision Matrix Example (continued) The performance and logistics IPT decided that reliability is the most important factor, and the other three factors are of equal importance to achieving mission success. They assigned the following weights to reflect their relative importance (step 3): Factor - Weight MTBF.4 MTTR.2 Delivery.2 Cost.2, the IPT established a rating scheme, based on a scale from 1 to 3, with 3 being the best and 1 being the worst (step 4). Then they independently rated and ranked each performance and cost factor as shown below (step 5). RANKINGS MTBF MTTR Delivery Cost Sub A1 1 1 3 3 Sub B2 2 2 2 2 Sub C3 3 3 1 1 they combined the two tables in a matrix and performed the calculations, multiplying each factor rating by the factor weight (step 6). Let's take a look. Page 27 of 31
Decision Matrix Example (continued) Combining the rating scores with the importance weighting factors, they got the following decision matrix. MTBF 0.4 MTTR 0.2 Delivery 0.2 Cost 0.2 Total Sub A1 1 x.4 =.4 1 x.2 =.2 3 x.2 =.6 3 x.2 =.6 1.8 Sub B2 2 x.4 =.8 2 x.2 =.4 2 x.2 =.4 2 x.2 =.4 2.0 Sub C3 3 x.4 = 1.2 3 x.2 =.6 1 x.2 =.2 1 x.2 =.2 2.2 WINNER You can see that the quantitative winner is Sub-Assembly C3, which has the highest score. Although this alternative has the longest delivery time and the highest costs, it offers the greatest reliability. In this case, the scoring criteria placed greater relative importance on the system's MTBF. Now the decision maker is equipped to make an informed decision that takes into consideration cost, schedule, and technical performance. Page 28 of 31
Knowledge Review Molly is examining the Border Patrol's new Unmanned Aerial Vehicle program. She has received test data from the contractor on various engines, and the time is approaching to make a decision on which engine to pick. She decides to use a decision matrix to help structure her thought process. Molly identifies four engines to be considered. She determines that she is going to use the following criteria to evaluate the engines: size, weight, cost, and reliability (MTBF). What is the next step she should take? A. Rank the performance of each engine against the four criteria B. Assign a weight to each criteria based on its relative importance C. Multiply the rating for each item by the assigned weight for each criteria D. Multiply the totals for each item Correct! The next step in the decision matrix process, after evaluation criteria are determined, is to assign weights to each criteria. Page 29 of 31
Summary In this lesson, you learned that a supportable system is reliable and maintainable. These qualities are a result of the design process, so engineering efforts must address them very early in order to create a system that will be supportable in the field. Reliability indicates how long a system will work before it fails. It is measured in mean time between failure (MTBF), the number of hours a component or sub-system will operate on average before a failure occurs. MTBF is calculated by dividing the total number of hours the system is operated by the total number of failures experienced during that time. Maintainability indicates the ability of a system to be kept in, or returned to, working condition by skilled personnel. It is measured in mean time to repair (MTTR), the average number of hours that are required to repair a component or sub-system. MTTR is calculated by dividing the total elapsed time for maintenance by the total number of repairs completed during that time period. Supportability is the ability of a system's design and planned logistics resources to provide for operations and readiness at an affordable cost throughout the system's life. Reliability and maintainability both impact supportability, but supportability also includes activities and resources (such as fuel) that are necessary whether the system fails or not. Reliability, maintainability and supportability goals must be established early in the ALF, and should emphasize both higher operational effectiveness and lower life cycle costs. Availability indicates the probability that a system will be ready and able to operate when needed. It is expressed as the ratio of system up time over the sum of system up time plus down time. Page 30 of 31
Summary (continued) You also learned how to use a decision matrix as a simple, but powerful, tool for aiding the decisionmaking process. A decision matrix is a tool to make system design decisions, making trade-offs among multiple factors simultaneously. It combines decision factors with a determination of the relative importance of each. The resultant calculation provides the decision maker with a quantitative basis for a decision. This technique is often used for making design decisions, but it can also be used for other purposes. You may print the Designing a Supportable System lesson or save it for future reference. Page 31 of 31