Risk Module: Risk Management, Fault Trees and Failure Mode Effects Analysis Exploration Systems Engineering, version 1.0

Transcription

1 Risk Module: Risk Management, Fault Trees and Failure Mode Effects Analysis Exploration Systems Engineering, version 1.0 Exploration Systems Engineering: Risk Module

2 Module Purpose: Risk To understand risk, risk management, fault tree analysis and failure mode effects analysis in the context of project development Acknowledge that risks are inevitable and recognize that through systematic management and analytic techniques they can be reduced Review three techniques that are used to discover, assess, rank and mitigate risk - risk management, fault tree analysis and failure mode effects analysis Exploration Systems Engineering: Risk Module 2

3 What are Risks and Risk Management? Risks are potential events that have negative impacts on safety or project technical performance, cost or schedule Risks are an inevitable fact of life risks can be reduced but never eliminated Risk Management comprises purposeful thought to the sources, magnitude, and mitigation of risk, and actions directed toward its balanced reduction The same tools and perspectives that are used to discover, manage and reduce risks can be used to discover, manage and increase project opportunities - opportunity management Exploration Systems Engineering: Risk Module 3

4 What is Risk Management? Risk management is a continuous and iterative decision making technique designed to improve the probability of success. It is a proactive approach that: Seeks or identifies risks Assesses the likelihood and impact of these risks Develops mitigation options for all identified risks Identifies the most significant risks and chooses which mitigation options to implement Tracks progress to confirm that cumulative project risk is indeed declining Communicates and documents the project risk status Repeats this process throughout the project life Exploration Systems Engineering: Risk Module

5 Risk Management Considers the Entire Development and Operations Life of a Project Risk Type Technical Performance Risk Cost Risk Programmatic Risk Schedule Risk Liability Risk Regulatory Risk Operational Risk Examples Failure to meet a spacecraft technical requirement or specification during verification Failure to stay within a cost cap for the project Failure to secure long-term political support Failure to meet a critical launch window Spacecraft deorbits prematurely causing damage over the debris footprint Failure to secure proper approvals for launch of nuclear materials Failure of spacecraft during mission Safety Risk Supportability Risk Hazardous material release while fueling during ground operations Failure to resupply sufficient material to support human presence as planned Exploration Systems Engineering: Risk Module 5

6 Every NASA Space Flight Project Begins with a Plan for Risk Management This plan reflects the project s risk management philosophy: Priority (criticality to long-term strategic plans) National significance Mission lifetime (primary baseline mission) Estimated project life cycle cost Launch constraints In-flight maintenance feasibility Alternative research opportunities or re-flight opportunities The risk management philosophy is reflected in a number of ways: Whether single point failures are allowed Whether the system is monitored continuously during operations How much slack is in the development schedule How technical resource margins (i.e., mass, power, MIPS, etc.) are allocated throughout the development Exploration Systems Engineering: Risk Module 6

7 Other Factors to Consider in Assessing Risk (but not limited to) Complexity of management and technical interfaces Design and test margins Mission criticality Availability and allocation of resources such as mass, power, volume, data volume, data rates, and computing resources Scheduling and manpower limitations Ability to adjust to cost and funding profile constraints Mission operations Data handling, i.e., acquisition, archiving, distribution and analysis Launch system characteristics Available facilities Exploration Systems Engineering: Risk Module 7

8 Risk Identification Risks are identified by the development team, peer reviews, lessons from past projects and expert review Lessons from past projects are captured via trigger questions, or questions that challenge a development strategy or design solution The project risk status and top ten risk list are reviewed periodically - usually monthly - and at the project milestone reviews Exploration Systems Engineering: Risk Module 8

9 Example Risk Trigger Questions Have requirements been implemented such that a small change in requirements has the potential to cause large cost, performance or schedule system ramifications? Do designs or requirements push the current state-of-the-art? Has the concept for operating, maintaining, decommissioning or disposal of the system been adequately defined to ensure the identification of all requirements? Has an independent cost estimate (ICE) been performed? Is the schedule adequate to handle the level of requirements or objectives changes that are occurring or are likely to occur? Have the necessary facilities for environmental test been identified and availability problems been resolved? Exploration Systems Engineering: Risk Module 9

10 More Considerations for Risk Discovery While each space project has its unique risks, a list of the underlying sources of risks would include the following: Technical complexity - many design constraints or many dependent operational sequences having to occur in the right sequence and at the right time Organizational complexity - many independent organizations having to perform with limited coordination Inadequate margins or reserves Inadequate implementation plans Unrealistic schedules Total and year-by-year budgets mismatched to the actual implementation risks Over-optimistic designs pressured by mission expectations Limited engineering analysis and understanding due to inadequate engineering tools and models Limited understanding of the mission s space environments Inadequately trained or inexperienced project personnel Inadequate processes or inadequate adherence to proven processes Exploration Systems Engineering: Risk Module 10

11 Pause and Learn Opportunity Engage the class in identifying risks for a familiar project. What kinds of risks are identified? What is the basis for their search for risks? After the class has thought for a while, the instructor could present some trigger questions which may help discover new risks and show the value of the trigger questions. Exploration Systems Engineering: Risk Module

12 Cartoon: Dilbert Identifies Risks United Features Syndicate, Inc. Exploration Systems Engineering: Risk Module 12

13 The Benefits of Preparing for the Unexpected Background:" On January 21, 200 (Sol 18), Spirit abruptly ceased communicating with mission control. The next day the rover radioed a 7.8 bit/s beep, confirming that it had received a transmission from Earth but indicating that the spacecraft believed it was in a fault mode." Mars Spirit Rover Flash Memory Problem The thing that strikes me most about all this is how critical it was to have that INIT_CRIPPLED command in the system. It s not the kind of command that you d ever expect to use under normal conditions on Mars. But back during the earliest days of the project Glenn realized that someday we might need the flexibility to deal with a broken flash file system, and he put INIT_CRIPPLED in the system and left it there. And when the anomaly hit, it saved the mission. From Roving Mars by Steve Squires, Hyperion 2005 Be prepared for the low probability event with a huge consequence. Exploration Systems Engineering: Risk Module 13

14 After Identification Risks are Assessed Risks are assessed by characterizing the probability that a project will experience an undesired event and the consequences, impact or severity of the undesired event, were it to occur Risks can be compared on iso-curves consisting of a likelihood measure and a consequence measure Since the assessment of the likelihood and consequence of a risk is both subjective and has significant uncertainty the characterization of risk either qualitative (low medium or high) or semi-quantitative (risk are captured on a 5x5 matrix) 1.0 Likelihood (Probability) 0.0 Low Risk Medium Risk High Risk Severity of Consequence Exploration Systems Engineering: Risk Module 1

15 An Example of Some Semi-Quantitative Definitions to Enable a Project to Compare and Rank Risks Scale Probability of Occurrence Measure Near certain to occur (80-100%). Highly likely to occur (60-80%). Likely to occur (0-60%). Unlikely to occur (20-0%). Not likely; Improbable (0-20%). Impact of Consequences Class Technical Schedule Cost Class I Catastrophic (Scale 5) Class II Critical (Scale ) Class III Moderate (Scale 3) Class IV Negligible (Scale 2) A condition that may cause death or permanently disabling injury, facility destruction on the ground, or loss of crew, major systems, or vehicle during the mission A condition that may cause severe injury or occupational illness, or major property damage to facilities, systems, equipment, or flight hardware A condition that may cause minor injury or occupational illness, or minor property damage to facilities, systems, equipment, or flight hardware A condition that could cause the need for minor first aid treatment but would not adversely affect personal safety or health; damage to facilities, equipment, or flight hardware more than normal wear and tear level launch window to be missed schedule slippage causing launch date to be missed internal schedule slip that does not impact launch date internal schedule slip that does not impact internal development milestones cost overrun > 50 % of planned cost cost overrun 15 % to 50 % of planned cost cost overrun 2 % to 15 % of planned cost cost overrun < 2 % of planned cost Exploration Systems Engineering: Risk Module 15

16 A 5x5 Risk Matrix Provides a Quick Visual Comparison of All Project Risks High risks mission success jeopardized - immediate action required Medium risk review regularly contingent action if does not improve Low risk watch and review periodically Exploration Systems Engineering: Risk Module 16

17 Top Risks and their Trends are Periodically Reviewed for the SOFIA Project SOFIA Risk Matrix Likelihood Rank & Trend Risk ID DFRC-3 DFRC-12 DFRC-07 DFRC-2 DFRC-01 DFRC-11 Appr oach R M W A W R Risk Title Landing Gear Door System Failure Sched Integration problems structure vs.. avionics Cost growth for engine components Quality Control Resources insufficient Avionics software behind schedule Payload Capacity & Volume Trade-offs design issues High Med Low 1 Criticality L x C Trend CONSEQUENCES Decreasing (Improving) Increasing (Worsening) Unchanged New Since Last Period Approach M - Mitigate W - Watch A - Accept R - Research 7 8 DFRC-0 DFRC-02 R R Limited Flight Envelope, due to technical issues More flight testing may be required for Soft V&V Exploration Systems Engineering: Risk Module 17

18 Top Risks and their Trends are Periodically Reviewed for the Constellation SE&I SE&I Top Risk List L I K E L I H O O D!! CONSEQUENCE Legend Decreasing (Improving)! Increasing (Worsening)! Unchanged! Top Directorate Risk (TDR)! Top Program Risk (TPR) Top Project Risk ( TProjR ) 1, 2 R a n k T r e n d N N!! N!!! Title! Ares I/Orion Ascent Aeroacoustic Environments! Structural loads on CEV and LSAM during TLI! Requirements Maturation! Program Visibility for Closing the Architecture! (SRR) Abort Site Sea State Limits Launch Availability! Software Development and Assurance! CxP Lifecycle cost!!106 - Tailoring of Human - Rating requirements Owning Team FP_SIG FP_SIG SE&I - PRIMO SE&I - AT&A SE&I_SO A CSI_SIG SE&I_SO A SE&I_PT I_HR L I K E Consequence S A F E P E R F S C H E D C O S T Exploration Systems Engineering: Risk Module 18

19 The Status of the Most Significant Risks and Their Mitigation Options are Reviewed Periodically Title of risk Description or Root cause Possible categorizations System or subsystem Cause category (technology, programmatic, cost, schedule, etc.) Resources affected (budget, schedule slack, technical margins, etc.) Owner Assessment of Implementation risk or Mission risk Likelihood - estimate of the probability of the risk event Consequences - estimate of the performance, cost, safety and schedule effects Mitigation Description, including costs of mitigation options Mitigation option leverage or reduction in the assessed risk Current mitigation activities Current trends in risk significance - likelihood and impact Significant milestones Opening and closing of the window of occurrence Decision points for mitigation implementation effectiveness Exploration Systems Engineering: Risk Module 19

20 Exploration Systems Engineering: Risk Module Part 2 of Risk Module: Fault Tree Analysis Event Tree Analysis

21 Fault Tree Analysis Supports Design Decisions and Failure Investigations Fault Tree Analysis - FTA - uses a top-down symbolic logic model and estimates of failure probabilities of initiators to estimate the occurrence (failure) of the pre-determined, undesirable, top event An initiator is a credible undesirable event that is a contributing cause to top event failure Cut sets are groups of initiators, when taken together, cause top event failure Path sets are groups of initiators that if none occur the top event does not fail FTA is both a design and a diagnostic tool As a design tool FTA is used to compare alternative design solutions and the resulting TOP event probability As a diagnostic tool FTA is used to investigate scenarios that may have led to the TOP event failure - leading to an estimate of the most likely cut sets Exploration Systems Engineering: Risk Module 21

22 Fault Tree Analysis Fault tree analysis is a graphical representation of the combination of faults that will result in the occurrence of some (undesired) top event. In the construction of a fault tree, successive subordinate failure events are identified and logically linked to the top event. The linked events form a tree structure connected by symbols called gates. Exploration Systems Engineering: Risk Module 22

23 Refer to NASA Reference Publication 1358: System Engineering Toolbox for Design-Oriented Engineers Section 3.6: Fault Tree Analysis (Handout) Particular points: And/Or Gates explanation Example Fault Tree (Fig 3-20) Exploration Systems Engineering: Risk Module

24 Event Trees Event trees can be viewed as a special case of fault trees, where the branches are all ORs weighted by their probabilities. Event trees are generated both in the success and failure domains. This technique explores system responses to an initiating challenge and enables assessment of the probability of an unfavorable or favorable outcome. The system challenge may be a failure or fault, an undesirable event, or a normal system operating command. In constructing the event tree, one traces each path to eventual success or failure. This technique is typically performed in phase C but may also be performed in phase B. See NASA Reference Publication 1358: System Engineering Toolbox for Design-Oriented Engineers section 3.8 for additional discussion. Exploration Systems Engineering: Risk Module 2

25 Will the Stage Make it from Hangman s Hill to Placer Gulch? Station Probability of no horses 1, 2, Placer Gulch event tree example from a Safety & Mission Assurance training course by Pat Clemons of Sverdrup. Exploration Systems Engineering: Risk Module 25

26 Fault Tree Analysis of the Placer Gulch Stage Exploration Systems Engineering: Risk Module 26

27 Exploration Systems Engineering: Risk Module Part 3 of Risk Module: Failure Mode Effects Analysis

28 Failure Mode Effects Analysis Objective To ensure all failure modes have been identified and evaluated Technique Select a method to rank project failure modes Identify failure modes including all single point failure modes Analyze failure modes and their mission effect Determine those failure modes that might benefit from corrective action, e.g., Alternative designs Redundancy Increased reliability Determine which, if any, corrective actions will be implemented Exploration Systems Engineering: Risk Module 28

29 Failure Mode Effects Analysis FMEA is a design tool for identifying risk in the system or mission design, with the intent of mitigating those risks with design changes. The FMEA risk mitigation: 1. Recognizes and evaluates the potential failure of a system and its effects; 2. Identifies actions which could eliminate or reduce the chance of a potential failure occurring. FMEA is initiated in Phase B (Preliminary Design) and used to support design decisions in Phase C (Final Design). Exploration Systems Engineering: Risk Module 29

30 Failure Mode and Effects Analysis S C O e l D Item Potential Potential v a c Current e R Potential Causes/ t P Responsibility u s e N Failure Effects of Mechanisms(s) Controls Recommended & Target Function Mode Failure Failure r Prevention/Detection c Action(s) Completion Date Actions Results Actions Taken S e v O c D e t R P N What are the functions or requirements? What can go wrong? - No Function - Partially Degraded Function - Intermittent Function - Unintended Function What are the Effects? How bad is it? How often does it happen What are? the Cause(s)? How can this be prevented and detected? How good is this method at detecting it? What can be done? - Design changes - Process changes - Special controls - Changes to standards, procedures, or guides Who is going to do it and when? What did they do and what are the outcomes Exploration Systems Engineering: Risk Module 30

31 Module Summary: Risk Risk is inevitable, so risks can be reduced but not eliminated. Risk management is a proactive systematic approach to assessing risks, generating alternatives and reducing cumulative project risk. Fault Tree Analysis is both a design and a diagnostic tool that estimates failure probabilities of initiators to estimate the failure of the pre-determined, undesirable, top event. Failure Mode Effects Analysis is a design tool for identifying risk in the system design, with the intent of mitigating those risks with design changes. Exploration Systems Engineering: Risk Module 31

32 Exploration Systems Engineering: Risk Module Backup Slides for Risk Module

33 Uncertainties that Plague Projects Mission Objectives Technical Factors Internal Factors Uncertainties Will the baseline system satisfy the needs & objectives? Are they the best ones? Can baseline technology achieve the objectives? Can the specified technology be attained? Are all the requirements known? Can the plan and strategy meet the objectives? Offsets Thorough study Analyses Cost & schedule credibility Technology development plan Paper studies Design reviews Establish performance margins Engineering model test and prototyping Test & evaluation Resources Manpower skills Time Facilities Program strategy Budget allocations Contingency planning External Factors Will outside influences jeopardize the project? Contingency Robust design Exploration Systems Engineering: Risk Module 33

34 Project Risk Categories Typical Technical Risk Sources Typical Programmatic Risk Sources Typical Supportability Risk Sources Typical Cost Risk Sources Typical Schedule Risk Sources Physical properties Material properties Radiation properties Testing/Modeling Integration/Interface Software Design Safety Requirement changes Fault detection Operating environment Proven/Unproven technology System complexity Unique/Special Resources COTS performance Material availability Personnel availability Personnel skills Safety Security Environmental impact Communication problems Labor strikes Requirement changes Stakeholder advocacy Contractor stability Funding continuity and profile Regulatory changes Reliability and maintainability Training Operations and support Manpower considerations Facility considerations Interoperability considerations System safety Technical data Sensitivity to technical risk Sensitivity to programmatic risk Sensitivity to supportability risk Sensitivity to schedule risk Labor rates Estimating error Sensitivity to technical risk Sensitivity to programmatic risk Sensitivity to supportability risk Sensitivity to cost risk Degree of currency Number of critical path items Estimating error Embedded training Exploration Systems Engineering: Risk Module 3