Intermediate Systems Acquisition Course. Lesson 2.1 Analysis of Alternatives. Analysis of Alternatives (AoA)

Transcription

1 Analysis of Alternatives (AoA) The Analysis of Alternatives (AoA) is an important element in the acquisition process. It provides a systematic approach to determine the optimal way to fill a gap in mission capability. It also helps senior leadership evaluate the desirability and affordability of an acquisition program. In this lesson you will learn what an AoA is and how it is used in systems acquisition. You may print the Analysis of Alternatives lesson or save it for future reference. Page 1 of 36

2 Objectives Upon completion of this lesson, you should be able to: Differentiate between Analysis of Alternatives (AoA) versus Alternatives Analysis Identify the purpose, composition, organization, and activities of the AoA study team Recognize how key AoA components and processes aid decision makers in the systems acquisition process Recognize how mission tasks (MTs), measures of effectiveness (MOEs), and measures of performance (MOPs) relate to one another within the context of the AoA Recall techniques and methodologies used in cost and effectiveness analysis Page 2 of 36

3 What is an AoA? The AoA is one of the key documents produced during the Analyze/Select Phase of the Acquisition Lifecycle Framework (ALF), at or before acquisition decision event 2A (ADE-2A). It evaluates the cost, effectiveness, and risk of potential materiel alternatives to meet a specific capability gap/need. After rigorous analysis, the AoA culminates in a recommendation to the Acquisition Decision Authority (ADA) for the most cost-effective solution. An AoA does not contain non-materiel solutions; these would be handled apart from the system acquisition process, following the recommendations of the Capability Analysis Report (CAR). If the need requires a non-materiel solution, a Non-Materiel Change Recommendation (NMCR) will be written as part of the Joint Requirements Integration and Management System (JRIMS) process. The process depicted below is iterative, containing activities of alternative development, operational concept assessment, effectiveness analysis, cost analysis and estimation, and cost effectiveness and comparison analysis. These activities are performed by the AoA team through a collaborative effort that results in a recommended alternative that is approved at the ADE-2A review. D Select the image to view an enlargement Select the D-link to read a detailed explanation of the graphic Page 3 of 36 A graphic illustrating the relationships between key AoA components. The process of alternative development is iterative. Operational concepts, effectiveness analysis, and cost estimation are all part of iterative studies and analyses that eventually lead to a cost effectiveness analysis. This in turn leads to a recommended alternative.

4 What is an Alternatives Analysis (AA)? An AA is similar to an AoA in purpose, but is narrower in scope. If the users are already certain which approach best meets the need, they may propose conducting an Alternatives Analysis (AA) instead of an AoA. In other words, an AA may be used in place of an AoA when the preferred solution is already narrowed down to a specific material solution. The program manager (PM) would make this proposal to the ADA at ADE-1. In most cases there will not be a PM or Program Management Office (PMO) at the time of the development of the AoA/AA. Instead, a designated mission sponsor or project lead would present the outcome of the AOA/AA to the ADA. Unlike an AoA, which considers a wide range of options, an AA focuses on one specific materiel solution. For example, suppose that Customs and Border Protection (CBP) has to increase border patrols. If many types of solutions will be considered, such as helicopters, satellites, UAVs, airplanes, Jeeps, armored personnel vehicles, cameras, or radar, the situation would call for an AoA. On the other hand, if CBP has already decided to buy more Jeeps, then the predetermined materiel solution to the capability gap is a Jeep, and the situation calls for an AA. An AA is conducted and approved at the Component level. Both an AoA and an AA should contain a solid business case for the preferred alternative, addressing the quantifiable and qualitative benefits, and costs of the solution. While an AoA and an AA are not the same thing, the steps in creating them are the same. In this lesson, you will see the term AoA/AA. This indicates that the content covered applies to both types of documents. The Office of Program Accountability and Risk Management (PARM) website has a template that applies to the AoA/AA. Page 4 of 36 ADE-1 is when DHS decides to allow the program to proceed from the Need Phase to the Analyze/Select Phase.

5 Why conduct AoAs or AAs? Both AoAs and AAs: Provide unbiased, meaningful and consistent information Address needs, capabilities, cost, risk, and operational effectiveness Identify operational capabilities and effects Provide an analytical basis for program direction/redirection Page 5 of 36

6 When should you perform an AoA/AA? The AoA/AA is based on the Mission Need Statement (MNS) and Capability Development Plan (CDP), approved at ADE-1. It influences and is influenced by the evolutionary development of the Concept of Operations (CONOPS) and the Operational Requirements Document (ORD). The AoA is conducted during the Analyze/Select Phase of the ALF. The subsequent AoA report supports the ADE-2A decision to enter the Obtain Phase. D Select the D-link to read a detailed explanation of the graphic Page 6 of 36 Graphic showing where the analysis of alternatives or alternatives analysis fits into the ALF. The MNS and the CDP are approved at ADE-1 and they feed into the CONOPS, AoA or AA, and the ORD. The CONOPS, AoA or AA, and ORD are developed between ADE-1 and ADE-2A during the Analyze/Select Phase of the ALF. Information flows two ways between the AoA or AA and the CONOPS, and also between the AoA or AA and the ORD. ADE-0 is in the Need Phase. ADE-1 is at the end of the Need Phase. ADE-2A is at the end of the Analyze/Select Phase. ADE- 2B and ADE-2C occur during the Obtain Phase. ADE-3 comes at the end of the Obtain Phase. The final phase is Produce/Deploy/Support/Dispose.

7 Who conducts the AoA? Now that you know the purpose and timing of an AoA, let s take a look at who performs the analysis and how they do it. A study team should plan and conduct the AoA. The team will develop the study plan that will identify and schedule all the tasks that need to be done. The study team must accomplish these major activities: Receive direction from decision makers Identify alternatives Develop operational scenarios and detailed operational concepts Explain the employment of alternatives within the context of approved threats and scenarios Refer to the operational concepts provided in the CONOPS Note: The CONOPS is developed by the user, i.e., the Component, concurrently with the AoA Conduct the analysis Report the results To accomplish all these activities, the organizational structure shown on the next screen has proven to be useful and effective. Page 7 of 36

8 Study Team Structure The AoA/AA study team is typically managed by the AoA study team director, and is independent of the Program Management Office (PMO). The study team is built around core working groups as indicated in this example. The size of the individual working groups will depend on the scope of the study. An AoA considering multiple options requires considerably more time and effort than an AA, which looks at just one option. D Page 8 of 36 A diagram showing an organizational chart with two levels. At the top is the AoA study team director. The second level contains: technology and alternatives, effectiveness analysis, threats and scenarios, operational concepts, and cost analysis working groups.

9 Study Team Goals The goals of the study team are to: Plan and execute an unbiased and rigorous analysis Complete the study within resource limitations Consider a representative set of alternatives Investigate critical assumptions and parameters Present a clear and complete picture Let results determine the conclusions Page 9 of 36

10 Study Team Schedule As a general guide, the AoA study team can expect to spend approximately 40% of their time planning the AoA, 40% on the analysis, and finally 20% on packaging and presenting the analysis. For example, if the entire AoA process takes a year, then: Planning: 3-6 months The PM and the AoA study team define the problem, build out the team, develop a plan, and collect information Analysis: 3-6 months The team begins by collecting data and choosing evaluation tools, then analyzes effectiveness of the various alternatives, then estimates costs and analyzes them Packaging/Presenting: 2-3 months The team gathers up the results, publishes reports, and present their findings to decisionmakers Page 10 of 36

11 Study Plan The sponsor and PM prepare the AoA study plan in conjunction with the organization who will lead the analysis. They should consult with the study director, members of the Joint Requirements Council (JRC), and PARM to ensure all applicable issues are addressed. A sample format for the study plan may be found on the PARM web site in the Analysis of Alternatives template. The main sections are: 1. Introduction 2. Ground Rules and Assumptions 3. Alternatives 4. Determination of Effectiveness Measures 5. Methodology The study plan is submitted to the Component Acquisition Executive (CAE) for approval prior to completing the AoA or AA effort. The AoA study plan is incorporated into the AoA. Page 11 of 36

12 Knowledge Review Which of the following is a major activity of the AoA study team? A. Define mission needs and user operational requirements B. Develop operational scenarios and detailed operational concepts C. Develop the program budget and LCCE Correct! The major activities of an AoA study team include: Identify alternatives, develop operational scenarios and detailed operational concepts, conduct the analysis, and report the results. The AoA study team develops operational scenarios and concepts consistent with the Concept of Operations (CONOPS), which is developed by the Component. Page 12 of 36

13 Characteristics of a Quality AoA So far we've seen what an AoA is, why it is used, who prepares it, and when. Now let's see what goes into it. An AoA must provide useful information to the decision making process. To be useful, the information must be: Focused: The analysis must be directed to the needs identified in the MNS Timely: The study results must be available in time to have an impact on meeting the mission needs Accurate: While much of the study is based on estimates of cost and effectiveness, you should strive to make these estimates as accurate as possible Complete: In order for an analysis to be useful, it must be as complete as possible, considering all feasible alternatives and all identifiable costs Traceable: Measures of performance and effectiveness must be traceable back to mission tasks needed to fulfill the mission need - the study must be repeatable Objective: Analyses must be completed without bias or a predisposition toward one particular solution Page 13 of 36

14 Elements of an AoA/AA A well-written AoA/AA will include the following: Issues and Scenarios: Homeland security mission-related problems to be solved Alternative Development: Proposed materiel solutions to those problems Operational Concepts: How alternatives will actually be used in the field Hierarchy Structure from Objectives to Measures: A series of goals, tasks and metrics to evaluate performance Data Gathering: Collecting information on the characteristics of each alternative Effectiveness Analysis: Rigorous examination of the relative feasibility and suitability of each alternative Cost Analysis: Estimation and examination of the relative life cycle costs associated with each alternative Let's take a look at each in detail. Page 14 of 36

15 Issues and Scenarios Issues in the context of an AoA are those problems or opportunities that drive the program and are captured in the MNS. Scenarios and detailed operational concepts describe how potential solutions or alternatives would be used. Together, these important components of AoA: Describe the problem to be solved or a technological opportunity for process improvement Must be consistent with DHS Strategic Guidance and Planning Provide the context for evaluating alternatives Should not be selectively chosen to bias alternative performance Page 15 of 36

16 Developing Alternatives An alternative is a proposed materiel solution to a need identified in a MNS. The solution could involve acquisition of hardware, software, or a combination of both. It might use existing products, or require new development. The complete solution may also have elements of a nonmateriel solution associated with it. This is especially true for large and complex acquisition programs. For example, the acquisition of a new Immigrations and Customs Enforcement (ICE) or Customs and Border Protection (CBP) patrol car may also require changes in training and doctrine. Some common sources of alternatives are: S&T Directorate Brainstorming National Laboratories Other Federal Agencies Contractors and Vendors User or customer input Page 16 of 36

17 Developing Alternatives (continued) In the early stages of analysis, you should include a wide spectrum of alternatives in your AoA. As your analysis progresses, you will narrow down the options to those with the greatest potential. The AoA may not recommend any of the studied solutions if none are viable. If the use of an AA is approved, alternatives may be narrowed to a specific or bounded materiel solution. For example, an AA may be used if the solution is a replacement for an existing ship or vehicle. Note: The AoA/AA is NOT a source selection to buy a particular system. However, some of the alternatives may be potential candidates in a future source selection. Page 17 of 36

18 Detailed Operational Concepts Detailed operational concepts describe how each alternative will be employed in an operational environment, including concepts for maintenance, support, and interoperability with other systems. Operational concepts are usually the responsibility of an operational concepts working group within the AoA study team. Developing operational concepts is a multidisciplinary effort involving operators, engineers, modelers, and logisticians. These operational concepts need continuous scrutiny by the study team and "sanity checks" by the operational community. End users, to include logisticians, contribute to objective analysis by ensuring that operational concepts are realistic. Operational concepts will evolve over the course of the study. They provide critical inputs for cost and effectiveness modeling, and support detailed cost estimating. Although related, the development of operational concepts should not be confused with the development of the CONOPS. The CONOPS is a separate document written by the user (Component) that tells how the recommended solution suggested in the AoA/AA will be operationally used. The CONOPS document and the AoA/AA are written at the same time and support each other through an iterative development process. Various concepts may be addressed as the CONOPS is developed. Page 18 of 36

19 Metrics for an AoA The AoA uses a hierarchy of metrics to assess the relative effectiveness of different alternatives. MTs, MOEs and MOPs help focus the analysis and provide standard, non-system specific measures to allow fair, unbiased comparison of the alternatives. These measures should relate back to the operational shortfalls or capability gaps identified in requirements documents. Select each button below for detailed information. Mission Tasks Measures of Effectiveness Measures of Performance Page 19 of 36 Mission Tasks (MTs) Mission tasks are usually expressed in terms of general actions that need to be performed to correct gaps or to obtain needed capability. For example, improve training is a mission task for the objective of strengthen response and recovery capacity. The effectiveness of each alternative is measured in terms of the degree to which the task is accomplished. Measures of effectiveness (MOEs) MOEs are objective measures of system performance that are closely related to mission accomplishment. They are often qualitative in nature and describe the utility of the capability in operationally meaningful terms. They indicate the degree to which a system performs a task under specific conditions. For example, the MT improve training could have an MOE of reduced response time. Measures of performance (MOPs) MOPs are quantitative measures of a system characteristic. They are chosen to support one or more MTs. For example, number of personnel who participate in interdisciplinary exercises, and reduced injuries among responders are MOPs for the MT, improve training. MOPs will eventually flow down to key performance parameters (KPPs), which will form much of the criteria for testing the system.

20 Metrics for an AoA (continued) Effectiveness analysis involves determining values for the MOEs and MOPs for each alternative. Which MOEs and MOPs you select can affect the objectivity and traceability of the analysis. Remember: To ensure the analysis is objective, the MOEs must be established up front and then applied systematically and methodically to each of the alternatives To ensure the analysis is traceable, there needs to be a clear linkage between the MTs, MOEs, and MOPs The selection of MOEs and MOPs is usually the responsibility of an effectiveness analysis working group within the AoA study team. Page 20 of 36

21 Knowledge Review Which of the following choices best describes measures of effectiveness (MOEs)? A. General actions that need to be performed to correct gaps or to obtain needed capability B. Objective measures of system performance that are closely related to mission accomplishment C. Quantitative measures of the physical performance or physical characteristic Correct! MOEs are objective measures of system performance that are closely related to mission accomplishment. Page 21 of 36

22 AoA: Data Gathering Data gathering, which is primarily the responsibility of the effectiveness and cost analysis working groups, has a large impact on the quality of the analysis. To ensure a timely AoA, identify required data as early as possible. Data gathering is a difficult and time-consuming process that should not be underestimated To ensure an accurate product, you should select data from the most reliable and current sources. Make every effort to assure reasonable accuracy and validity. Limitations in data should be considered in establishing error ranges in results To ensure traceability, it is critical that you identify and document the sources of all data Page 22 of 36

23 AoA: Effectiveness Analysis The effectiveness analysis process, shown below, illustrates the sequence for analyzing each of the MTs and MOEs for every alternative, and documenting findings. Note that this process is iterative, and requires returning to an earlier step to make revisions based on insights you gain as you perform the analysis. The curved arrows represent possible areas where you may revisit the analysis based on new information. D Select the image to view an enlargement Select the D-link to read a detailed explanation of the graphic Page 23 of 36 Graphic depicting the process for developing an analysis of alternatives. After the MNS is developed, determine the issues, then determine the MTs, and finally determine the MOEs and MOPs, that is, assess the ability to satisfy the MTs. Next, determine the analysis method, select models and data, develop a database, perform the analysis, and determine the alternatives. Arrows indicate that the process is iterative; after you develop alternatives you may circle back to determine the MOEs. As you perform the analysis and develop the database, you may revisit the analysis method, models, and data.

24

25 Effectiveness Analysis Process (continued) When the effectiveness analysis is complete, the study team should be able to document the capabilities found at the MT and MOE level for each alternative. Although many MOEs and MOPs will be analyzed, some may not be used in the final decision-making process because they do not discriminate among the alternatives. Select the D-link to read a detailed explanation of the graphic D Page 24 of 36 Graphic showing how some MOEs, but not all, are analyzed. MT-1 and MT-2 lead to four MOEs each: labeled MOE 1-1 through MOE 1-4, and MOE 2-1 through MOE 2-4. The fourth MOE in each case does not affect the outcome, and so it is not included in the overall analysis of Alternatives 1-3.

26 Effectiveness Analysis: Alternative Screening Process Not all of the alternatives considered at the beginning of the analysis will be evaluated at the end. A screening process is necessary to focus our efforts on those alternatives that show the most promise. Only the top three to five most effective and affordable alternatives should be fully examined in the AoA. Screening criteria may include: Technology availability Prohibitive costs Excessive staffing requirements Operational considerations D Page 25 of 36 Graphic displaying six alternatives, with only three being chosen.

27 Effectiveness Analysis: Models and Simulations We've discussed MTs, MOEs, and MOPs, and the methodology for applying these for the purpose of screening alternatives. Now we'll turn our discussion to modeling and simulation (M&S), which is often very useful in the AoA process. Modeling is the process of creating representations of systems, processes, environments and their interactions. Simulation is the process of executing models to investigate system performance. Together, M&S create a representation of a system that enables you to explore how it would work under various conditions. You can use these representations to compare and contrast aspects of alternatives such as performance, reliability, or life cycle cost. It is important to remember that M&S create an abstraction of the real system. The results that are obtained from M&S must be translated by the analyst from this abstraction back to reality. Page 26 of 36

28 Effectiveness Analysis: Models and Simulations (continued) Study teams often use modeling and simulation to deal with the following conditions. Select each button for detailed information. Problem Complexity Schedule Requirements Limited Study Resources Safety Concerns Lack of Acccess Page 27 of 36 Problem Complexity To reduce a complex problem to its critical components, we model the critical aspects to understand their interactions. For example, it is sometimes impossible or impractical to replicate a real world situation; therefore, we model only those aspects of the situation that we understand to be critical to the outcome. Schedule Requirements The analysis portion of the work of the study team needs to be time-boxed to avoid "analysis paralysis." Simulation is one way to obtain good data relatively quickly. Limited Study Resources Modeling and simulation can save precious funds and time.

29 Safety Concerns There are often dangerous situations that users must address. However, we cannot unnecessarily put a human life or valuable property in harm's way. In that case, modeling and simulation allow us to study the situation without the risk of death, injury, or destruction. Lack of Access Often alternatives in AoAs only exist as concepts and are not fully developed. Thus, without any physical system, modeling and simulation is the only way to study the concept.

30 Effectiveness Analysis: AoA Model & Data Accreditation Obviously, the fidelity of the models used in the AoA is critical. To ensure that the models are accurate representations, they have to be accredited. In other words we need to make sure the models apply to our system under study. The effectiveness analysis working group identifies the models to be used and develops an accreditation plan. Then an independent accreditation team provides an assessment based on inputs from the model owners and users, as well as their own expertise. The assessment addresses how the models work by themselves and as a federated system. The accreditation team makes a recommendation to the Component. The Component reviews the recommendation and accredits the models when it accepts them for use in the study. Every aspect of this process must be documented in the accreditation package, which becomes an appendix to the AoA final report. Page 28 of 36

31 Cost Analysis for the AoA/AA Up to this point in the AoA process, we have focused on comparing and contrasting the performance aspects of alternatives. Another important aspect that you must consider is cost. Even if one alternative can out-perform the others, if the cost associated with it is too high, that alternative might not be viable. A cost estimate for an AoA must provide the cost for each alternative; for an AA, naturally, the study team will consider the cost of the chosen alternative. In both cases, the cost estimate is tailored, and shorter, than the official life cycle cost estimate (LCCE) prepared for ADE-2A at the end of the Analyze/Select Phase. Page 29 of 36

32 AoA Cost Analysis Process Remember, cost estimation for an AoA, while not as comprehensive as that for an LCCE, is quite similar. Usually the ground rules and assumptions used within the cost estimate baseline document (CEBD) will carry into the AoA/AA. However, since often the CEBD is being developed at the same time as the AoA/AA, you may look to the cost data in the Capabilities Development Plan (CDP) that was approved at ADE-1. Identify any specific alternative-based ground rules and assumptions within the CDP, and then determine if there are any cost issues that should concern the decision makers. Next, develop a Work Breakdown Structure (WBS) for every alternative. The WBS for a cost estimate for the AoA does not need to be comprehensive, however, the WBS for each alternative must be broken down to the same level of detail. Then determine your analysis approach, including which cost estimating technique(s) to use, e.g., cost estimating relationships, catalogue cost, contractor quotes, analogy, or expert opinion. Once you have selected your cost estimating techniques and determined your analysis approach, you can continue the cost analysis process by considering which models or simulations to use and begin to develop your cost database. As you gain information, you may have to revisit previous steps. Page 30 of 36 The cost analysis working group will work with the effectiveness analysis working group to determine the alternatives to be evaluated, the appropriate operations and maintenance concepts for each alternative, and any appropriate assumptions and constraints related to each alternative. The main differences between the cost analysis and the effectiveness analysis are the metrics used for assessment. The cost analysis will use metrics dealing with money, budgeting, and costs whereas the effectiveness analysis will focus on metrics dealing with meeting certain mission parameters. When taken together both the cost and effectiveness analyses should identify a best value space defined by cost parameters, and certain mission accomplishment metrics that are calculated over the entire life cycle of the system.

33 Examples of typical cost issues: Cost estimating resources Technology freeze for alternatives Schedule Cost objectives Cost thresholds Affordability Review and oversight Wash costs * * Wash costs are the same cost elements for each alternative, and do not need to be included as part of the AoA/AA. A WBS is created for each alternative, and it is broken down to the same level for each. The WBS should: Define the total system to be developed or produced Display the total system as a product-oriented family tree composed of hardware, software, services, data, and facilities Relate the elements of work to each other and to the end product Life cycle cost WBS templates may be found on the Cost Analysis Division (CAD) website, under templates.

34 Document the Estimates Finally, the cost analysis consists of simply adding together the sub-element cost estimates and documenting the estimates in a chart, like the one shown below. Be sure to provide supporting documentation and perform a sufficiency review to ensure that you have an acceptable product for the cost community. The estimate must be documented sufficiently for replication by a qualified, independent cost analyst. And remember, the cost estimate must account for each phase of the life cycle. For each cost element, include: Alt 1 Alt 2 Alt 3 R&D Invest O&S Total The cost factors used, and the sources of factors used Fiscal year spread (base year and then year $M) for each alternative Identify cost by appropriation type Cost element contents Estimating method Details of estimating method application For each phase of the life cycle, include: Ground rules and assumptions General methodology RDT&E, Investment, O&S, and Disposal data Page 31 of 36

35 Comparing Alternatives Using Cost and Effectiveness The next step is to combine the effectiveness analysis and the cost analysis into a single costeffectiveness analysis. Plotting cost against effectiveness for critical MOEs provides analysts and decision makers a convenient method for comparing alternatives in order to select the alternative that is either low cost acceptable, best value, or meets all the mission needs at or above threshold levels but below an objective maximum level. This graph shows the analysis for one measure of effectiveness. Each of the alternatives has a range of uncertainty or area of risk around it, represented by the yellow ovals. The stakeholders have provided the level of expectations, represented by the threshold line. This example reflects that you have some flexibility in making a decision, as long as you can provide supporting documentation. Select each alternative to read a description of what it represents. D Select the D-link to read a detailed explanation of the graphic Page 32 of 36 Cost-effectiveness comparison chart with a Y axis, Effectiveness; an X axis, Cost; four plotted alternatives labeled one through four, and a threshold line extending outward from the Y axis. Each of the alternatives has a range of uncertainty or areas of risk around it. The first alternative is well below the capability threshold. The second alternative is very close to the threshold line, although it is shown below the threshold. The third alternative is above the threshold; however, the range of uncertainty or areas of risk around it show that its capabilities could actually be below the threshold, and it costs more than the second alternative. Finally, the fourth alternative is well above the threshold and to the far right, representing highest cost and effectiveness. Alternative 1 Not a good possibility; far below threshold

36 Alternative 2 Possibility; below threshold Depends on decision maker constraints Risks involved Schedule could impact decision Flexibility may be required May need more information Alternative 3 Possibility; meets threshold Has some risk More funding may provide extra capabilities Schedule could impact decision High overall effectiveness Alternative 4 Possibility; exceeds threshold Very low performance risk High cost Schedule could impact decision High overall cost-effectiveness vs. Alt 3

37 Comparing Alternatives Using Cost and Effectiveness Here is another example, using color codes rather than a graph to indicate thresholds. The decision maker must look at the remaining alternatives and weigh them against considerations such as cost, effectiveness, politics, schedule, risk, and flexibility. Finally, how do the alternatives rate against the predetermined mission tasks? D Select the D-link to read a detailed explanation of the graphic Page 33 of 36 An MOE summary chart presenting the findings of a cost and effectiveness comparison. On the far left are rows labeled Alt 1 through Alt 4. Columns are titled MOE 1-1 through MOE 1-3 which fall under MT 1, detect vehicles; MOE 2-1 through MOE 2-3 which fall under MT 2, supportability; and MOE 3-1 through MOE 3-3 which fall under MT 3, interoperability. Columns MOE 1-1 through MOE 1-3 and MOE 2-1 through MOE 2-3 are labeled critical, and columns MOE 3-1 through MOE 3-3 are labeled non-critical. Each of the blocks below the MOEs and to the right of the alternatives are shown with numbers reflecting how it meets each of the MOEs. Each of the blocks is measured and a color code is placed in the box with the numbers. Red, yellow, and green are used to represent below threshold, meets threshold, and above threshold, respectively.

38 Knowledge Review Given the data in the chart, and a budget of $300M, which is the best alternative? A. Alternative 1 B. Alternative 2 C. Alternative 3 D. Alternative 4 Select the D-link to read a detailed explanation of the graphic Correct! Alternative 3 provides the decision maker with the most capability items above the threshold, with none even partly below the threshold, and at a reasonable cost. D Page 34 of 36 Reprise of the MOE Summary chart with capability items for each MOE 1-1 through 3-3 of each alternative populated. Capability items are below, above, or meet the threshold. Alternative 1:.95 (above), 1400 (meets), 500 (meets),.60 (above),.85 (above), 150 (meets),.87 (above),.57 (below), 1.00 (above). Decision cost for Alt 1 is $144.5 million. Alternative 2:.75 (below), 1600 (above), 700 (above),.52 (meets),.50 (meets), 300 (above),.85 (above),.75 (meets), and.87 (meets). Decision cost for Alt 2 is $157.6 million. Alternative 3:.87 (meets), 1675 (above), 750 (above),.50 (meets),.87 (above), 275 (meets),.83 (meets),.97 (above), and.98 (above). Decision cost for Alt 3 is $195.7 million. Alternative 4:.90 (above), 1550 (meets), 725 (above),.38 (below),.83 (above), 200 (meets),.55 (below),.95 (above), and.75 (below). Decision cost for Alt 4 is $321.2 million.

39 Supporting the Decision The ultimate goal of the AoA/AA is to identify the most promising approach to meet the user's need within practical boundaries of cost and performance. Within this decision space, the AoA trades off cost and performance to find the optimal, balanced solution that will deliver capability to the user on schedule and within acceptable levels of risk. There needs to be a tight coupling between the MNS, the CONOPS, the cost estimates, and the analysis performed to evaluate the various alternatives. The final step is to document the results and preferred solution alternative(s) for the ADA. A template for the final report can be found on the PARM website. D Page 35 of 36 Graphic depicting the iterative nature of developing the AoA. We start with asking the question, "What do we need?" which is answered in the MNS. We then ask two questions. "How will it behave in the field?" which is answered in the CONOPS, and "How much will it cost over its life cycle?" which is answered in the life cycle cost estimate. Finally, we answer the question, "What is our best option?" in the AoA/AA.

40 Summary In this lesson you learned that an Analysis of Alternatives (AoA) is a high-level comparison, from a cost and performance perspective, of selected solutions for fulfilling a capability gap or need. It is conducted during the Analyze/Select Phase of the Acquisition Lifecycle Framework (ALF) by an independent team, which is composed of user and program management representatives from multiple disciplines. The AoA addresses both operational effectiveness and life cycle cost of each alternative. Effectiveness analysis involves assessing how well each option meets the needs described in the Mission Needs Statement (MNS). A hierarchy of mission tasks (MTs), measures of effectiveness (MOEs), and measures of performance (MOPs) are identified up front to help evaluate each alternative. Cost analysis involves projecting the cost to develop, procure, operate and maintain each alternative over the life of the system. Various cost estimating methods may be used, including cost-estimating relationships, catalog costs, contractor quotes, analogies, and expert opinion. Ultimately, a cost-effectiveness comparison must be made to select the optimum approach to take forward into the Obtain Phase. Additional guidance on AoAs can be found on the Office of Program Accountability and Risk Management (PARM) website. You may print the Analysis of Alternatives lesson or save it for future reference. Page 36 of 36