Mathematical Techniques to Improve the Utility of a Hazard Risk Matrix

Transcription

1 Severity Catastrophic Critical Marginal Negligible Probability 2 Frequent Probable Occasional Remote Improbable B C D E Mathematical Techniques to Improve the Utility of a Hazard Risk Matrix Don Swallom U.S. rmy viation and Missile Command Redstone rsenal, labama

2 ttributes of a well-designed risk assessment matrix Severity Severity scale covers full range of possible outcomes $2k $20k $200k $2M $20M $200M $2B $20B Frequency B C D E F G H I J >00 >0 > >0. >0.0 >0.00 >0.000 > > Injury, no lost work day Lost Work Day Permanent partial disability Low SSWG/Principal for Safety Fatality Medium - 0 Fatalities Serious - PEO 00 Fatalities High - CE Proposed DOD Matrix,000 Fatalities Prohibitive SECDEF 0,000 Fatalities

3 ttributes of a well-designed risk assessment matrix B C D E F G H I Severity Frequency J >00 >0 > >0. >0.0 >0.00 >0.000 > > $2k $20k $200k $2M $20M $200M $2B $20B Injury, no Permanent Lost Work 0 00,000 0,000 Probability lost work partial Fatality Day calibrated with reference Fatalities Fatalities to an Fatalities exposure Fatalities day disability interval (accidents per,000 troops per year, accidents per 00,000 FH, accidents per,000,000 missile firings, etc.) Low SSWG/Principal for Safety Medium - Serious - PEO High - CE Proposed DOD Matrix Prohibitive SECDEF

4 ttributes of a well-designed risk assessment matrix x0 B x0 C x0 D x0 E x0 F x0 G x0 H x0 I x0 J Severity Frequency >00 >0 > >0. >0.0 >0.00 >0.000 > > $2k $20k $200k $2M $20M $200M $2B $20B Injury, no lost work day x0 x0 x0 x0 x0 x0 x0 Lost Work Day Permanent partial disability Low SSWG/Principal for Safety Fatality Medium - 0 Fatalities Serious - PEO 00 Fatalities High - CE,000 Fatalities Equally-proportioned, logarithmic scales Prohibitive SECDEF (, 0, 00, 000 ) Proposed DOD Matrix 0,000 Fatalities

5 ttributes of a well-designed risk assessment matrix y y = f(x) probability = f(severity) Severity Frequency B C D E F G H I J >00 >0 > >0. >0.0 >0.00 >0.000 > > $2k $20k $200k $2M $20M $200M $2B $20B Injury, no lost work day Lost Work Day Permanent partial disability Low SSWG/Principal for Safety Fatality Medium - 0 Fatalities Serious - PEO 00 Fatalities High - CE,000 Fatalities Prohibitive SECDEF 0,000 Fatalities Cartesian Orientation Increase up and to the right x

6 ttributes of a well-designed risk assessment matrix B C D E F G H I J Severity Frequency >00 >0 > >0. >0.0 >0.00 >0.000 > > $2k $20k $200k $2M $20M $200M $2B $20B Injury, no lost work day Lost Work Day Permanent partial disability Low SSWG/Principal for Safety Fatality Medium - 0 Fatalities 00 Fatalities,000 Fatalities Prohibitive SECDEF Risk levels assigned to High cells - CEconsistent with contours of equal risk (iso-risk Serious - PEO contours) 0,000 Fatalities

7 ttributes of a well-designed risk assessment matrix B C D E F G H I J Severity Frequency >00 >0 > >0. >0.0 >0.00 Medium - Sufficient probability or frequency categories so highest >0.000 Low SSWG/Principal for Safety severity level can be assessed at the level of risk if the > probability or frequency of occurrence is low enough > $2k $20k $200k $2M $20M $200M $2B $20B Injury, no lost work day Lost Work Day Permanent partial disability Fatality 0 Fatalities 5E Serious - PEO 00 Fatalities High - CE,000 Fatalities Prohibitive SECDEF 0,000 Fatalities Medium -

8 ttributes of a well-designed risk assessment matrix Severity Frequency >00 B >0 C > D >0. E >0.0 F >0.00 G >0.000 Increasing 2 5 $2k $20k $200k $2M $20M Injury, no lost work day Lost Work Day Permanent partial disability Low SSWG/Principal for Safety Fatality Medium - 0 Fatalities Prohibitive SECDEF 5E Serious - PEO High - CE 6 7 Easily tailored with reporting of risk consistent with other systems within the family of systems.

9 Mother of ll Risk ssessment Matrices (Spaceship Earth) Frequency (Mishaps per 00,000 Hrs (. years)) B C D E F G H I J K L M N $2K E-6 E-7 E-8 E-9 E-0 E- Hazard Severity $20K $200K $2M $20M $200M $2B $20B $200B $2T $20T $200T $2Q 0 Fatal K Fatal 00K Fatal 0M Fatal B Fatal Fatal 00 Fatal 0K Fatal M Fatal 00M Fatal Prohibitive SECDEF High Serious Medium Low de minimis Earth encounter with an asteroid 9

10 Specific Individual Item Fleet or Inventory U.S. rmy System Safety Risk Matrix (MIL-STD-882D) RISK cceptance Levels per DODI , 8 Dec 08 Risk ssessment Levels & Definitions per Tables - thru -IV of MIL-STD 882D, 0 Feb 00 Likely to occur often in the life of an item, with a probability of occurrence greater than 0 - in that life. Continuously experienced Will occur several times in the life of an Will occur frequently item, with a probability of occurrence less than 0 - but greater than 0-2 in that life Likely to occur some time in the life of an Will occur several times item, with a probability of occurrence less than 0-2 but greater than 0 - in that life Unlikely but possible to occur in the life Unlikely, but can of an item, with a probability of reasonably be expected occurrence less than 0 - but greater to occur than 0-6 in that life So unlikely, it can be assumed Unlikely to occur, but occurrence may not be experienced, with possible a probability of occurrence less than 0-6 in that life B C D E HZRD SEVERITY Catastrophic Critical Marginal Negligible Could result in death, permanent total disability, loss exceeding $M, or irreversible severe environmental damage that violates law or regulation. Could result in permanent partial disability, injuries or occupational illness that may result in hospitalization of at least three personnel, loss exceeding $200K but less than $M, or reversible environmental damage causing a violation of law or regulation. Could result in injury or occupational illness resulting in one or more lost work days(s), loss exceeding $0K but less than $200K, or mitigatible environmental damage without violation of law or regulation where restoration activities can be accomplished. Could result in injury or illness not resulting in a lost work day, loss exceeding $2K but less than $0K, or minimal environmental damage not violating law 2 HIGH E B HIGH E C HIGH E D SERIOUS PEO E MEDIUM 2 HIGH E 2B HIGH E 2C SERIOUS PEO 2D MEDIUM 2E MEDIUM SERIOUS PEO B SERIOUS PEO C MEDIUM D MEDIUM E MEDIUM MEDIUM B MEDIUM C LOW D LOW E LOW 0

11 Specific Individual Item Fleet or Inventory U.S. rmy System Safety Risk Matrix (MIL-STD-882D) RISK cceptance Levels per DODI , 8 Dec 08 Risk ssessment Levels & Definitions per Tables - thru -IV of MIL-STD 882D, 0 Feb 00 Likely to occur often in the life of an item, with a probability of occurrence greater than 0 - in that life. 0 - Continuously experienced Will occur several times in the life of an Will occur frequently item, with x0 a probability of occurrence less than 0 - but greater than 0-2 in that life 0-2 Likely to occur some time in the life of an Will occur several times item, with x0 a probability of occurrence less than 0-2 but greater than 0 - in that life 0 - Unlikely but possible to occur in the life Unlikely, but can of x,000 an item, with a probability of reasonably be expected occurrence less than 0 - but greater to occur than 0-6 in that life So unlikely, it can be assumed 0-6 Unlikely to occur, but occurrence may not be experienced, with possible a probability of occurrence less than 0-6 in that life B C D E HZRD SEVERITY Catastrophic Critical Marginal Negligible Could result in death, permanent total disability, loss exceeding $M, or irreversible severe environmental damage that violates law or regulation. Could result in permanent partial disability, injuries or occupational illness that may result in hospitalization of at least three personnel, loss exceeding $200K Could result in injury or occupational illness resulting in one or more lost work days(s), loss exceeding $0K but less than $200K, or mitigatible Could result in injury or illness not resulting in a lost work day, loss exceeding $2K but less than $0K, or minimal environmental damage not but less than $M, or reversible environmental damage causing a environmental damage without violation of law or regulation where violating law violation of law or restoration $M regulation. $200K activities can be $0K $2K accomplished. 2 HIGH E B HIGH E C HIGH E D SERIOUS PEO E MEDIUM x5 x20 x5 2 HIGH E 2B HIGH E 2C SERIOUS PEO 2D MEDIUM 2E MEDIUM SERIOUS PEO B SERIOUS PEO C MEDIUM D MEDIUM E MEDIUM MEDIUM B MEDIUM C LOW D LOW E LOW

12 Specific Individual Item Fleet or Inventory U.S. rmy System Safety Risk Matrix (MIL-STD-882E) RISK cceptance Levels per DODI , 8 Dec 08 Risk ssessment Levels & Definitions per Tables - thru -IV of MIL-STD 882D, 0 Feb 00 Likely to occur often in the life of an item. No Numbers Will occur several times in the life of an item. Continuously experienced. Will occur frequently Likely to occur some time in the life of an Will occur several times item. Unlikely but possible to occur in the life of an item. So unlikely, it can be assumed occurrence may not be experienced in the life of an item. Unlikely, but can reasonably be expected to occur Unlikely to occur, but possible B C D E HZRD SEVERITY Catastrophic Critical Marginal Negligible Could result in one or more of the following: death, permanent total disability, irreversible significant environmental, impact, or monetary loss exceeding $0M. Could result in one or more of the following: permanent partial disability, injuries or occupational illness that may result in hospitalization of at least three personnel, reversible significant environmental impact, or monetary loss exceeding $M but less than $0M. Could result in one or more of the following: injury or occupational illness resulting in 0 or more lost work days, reversible moderate environmental impact, monetary loss exceeding $00K but less than $M. Could result in one or more of the following: injury or illness resulting in less than 0 lost work days, minimal environmental impact, or monetary loss less than $00K. 2 HIGH E B HIGH E C HIGH E D SERIOUS PEO E MEDIUM x0 x0 x0 $0M $M $200K $M $00K $0K $0K $2K 2 HIGH E 2B HIGH E 2C SERIOUS PEO 2D MEDIUM 2E MEDIUM HIGH E B SERIOUS PEO C MEDIUM D MEDIUM E LOW SERIOUS PEO B MEDIUM C MEDIUM D LOW E LOW 2

13 Specific Individual Item Fleet or Inventory U.S. rmy System Safety Risk Matrix (MIL-STD-882D) RISK cceptance Levels per DODI , 8 Dec 08 Risk ssessment Levels & Definitions per Tables - thru -IV of MIL-STD 882D, 0 Feb 00 Likely to occur often in the life of an item. Will occur several times in the life of an item. Continuously experienced Will occur frequently Likely to occur some time in the life of an Will occur several times item. Unlikely but possible to occur in the life of an item. So unlikely, it can be assumed occurrence may not be experienced in the life of an item. Unlikely, but can reasonably be expected to occur Unlikely to occur, but possible B C D E HZRD SEVERITY Catastrophic Critical Marginal Negligible Could result in one or more of the following: death, permanent total disability, irreversible significant environmental, impact, or monetary loss exceeding $0M. Could result in one or more of the following: permanent partial disability, injuries or occupational illness that may result in hospitalization of at least three personnel, reversible significant environmental impact, or monetary loss exceeding $M but less than $0M. Could result in one or more of the following: injury or occupational illness resulting in 0 or more lost work days, reversible moderate environmental impact, monetary loss exceeding $00K but less than $M. Could result in one or more of the following: injury or illness resulting in less than 0 lost work days, minimal environmental impact, or monetary loss less than $00K. 2 HIGH E B HIGH E C HIGH E D SERIOUS PEO E MEDIUM 2 HIGH E 2B HIGH E 2C SERIOUS PEO 2D MEDIUM 2E MEDIUM SERIOUS PEO B SERIOUS PEO C MEDIUM D MEDIUM E MEDIUM MEDIUM B MEDIUM C LOW D LOW E LOW

14 Understanding Probability Probability: number expressing the likelihood that a specific event will occur, expressed as the ratio of the number of actual occurrences to the number of possible occurrences. - The merican Heritage Dictionary of the English Language, Fourth Edition

15 Understanding Probability Math Definition: Repeat a random experiment n number of times. If a specific outcome has occurred f times in these n trials, the number f is the frequency of the outcome. The ratio f/n is the relative frequency of the outcome. relative frequency is usually very unstable for small values of n, but it tends to stabilize about some number p as n increases. The number p is the probability of the outcome. p = f / n for very large values of n 5

16 Understanding Probability Simple example: Probability of rolling a with one die. Roll # - 5, f/n = 0/ = 0 Roll #2-2, f/n = 0/2 = 0 Roll # -, f/n = / =. Roll # -, f/n = / =.25 Roll #,000: 6 s, f/n = 6/000 =.6 Rolls approach infinity f/n =

17 Value Rolling Dice Roll a single die 0 times. The expected value of each roll is.5. What you actually get is somewhat different. 6 USMCOM Safety Data 20 Jan Risk Value verage for 6 trials verage for 0 Trials Trial 7

18 Understanding Probability Hazard: Helicopter strikes wire; results in Class mishap Probability:.06E-06 occurrences per flight hour Flight Hr, no mishap, rate = 0,000 Flight Hrs, no mishap, rate = 0 76,82 Flight Hrs, mishap, rate = 5.676E-06 /flt hr 27,59 Flight Hrs, 2 mishaps, rate = 7.285E-06 /flt hr 700,62 Flt Hrs, mishaps, rate =.28E-06 /flt hr 0,000,000 Flt Hrs, 6 mishaps, rate =.600E-06 /flt hr,000,000,000 Hrs, 07 mishaps, rate =.07E-06 /flt hr Flight hours approach infinity, rate =.06E-06 /flt hr 8

19 NVIR System Safety Risk Matrix

20 NVIR System Safety Risk Matrix FREQUENT () = or > 00/00K flt hrs PROBBLE (B) 0-99/00K flt hrs OCCSIONL (C).0-9.9/00K flt hrs REMOTE (D) /00K flt hrs IMPROBBLE (E) = or < 0./00K flt hrs

21 Occurrences per 00,000 Flt Hrs Flight Hours per Occurrence Occurrences per Flight Hour Occurrences per Year Years per Occurrence Occurrence per Fleet Life Occurrences Fleet Life per per 00,000 Occurrence Flt Hrs Utilization: 20 hours/yr Fleet Exposure Hours:,766,00 hours ircraft Life: 20 years Fleet Hours per Year: 88,20 hours US rmy PEO viation Matrix Frequent Probable Occasional Remote Improbable Very Improbable ssumptions,000 0,000 00,000,000,000 0,000,000 Fleet Size: 68 aircraft ircraft Exposure Hours:,800 hours Utilization: 0-20 hours/yr 0-0 Fleet -5 Exposure 0Hours: -6,766, hours 0 ircraft Life: 20 years Fleet Hours per Year: 88,20 hours Fleet Life Very Frequent,766 Probable 77Occasional 7.7 Remote.77Improbable 0.77Improbable Catastrophic B C D E Flight Hours per Occurrence Occurrences per Flight Hour $M Fleet Life Medium Critical 2 2 2B 2C PEO 2D Occurrences per Year $200K Years per Occurrence Marginal Occurrence per Fleet Life $20K Fleet Life per Occurrence,000 0,000 00,000,000,000 0,000,000 High CS/E 2E B C D E Negligible B C D E F Catastrophic B C D E F $M Low F F F, Medium Critical 2 2 2B 2C PEO 2D $200K Marginal $20K High CS/E 2E B C D E Negligible Low 2F F 0R 0R

22 pplying Probability Classifications to a military helicopter Fleet Size = 68 aircraft Utilization = 20 hours/year Life = 20 years/aircraft ircraft Life = 20 x 20 =,800 hours Fleet Exposure Hours = 68 x 20 x 20 =,776,00 hours Fleet Hours per Year = 68 x 20 = 88,20 hours 22

23 ssumptions Fleet Size: 68 aircraft ircraft Exposure Hours:,800 hours US rmy US rmy PEO Utilization: viation PEO 20 hours/yr viation Expanded Fleet Exposure Matrix Hours:,766,00 Matrix hours Occurrences per 00,000 Flt Hrs ircraft Life: 20 years Fleet Hours per Year: 88,20 hours Very Frequent Probable Occasional Remote Improbable Very Frequent Probable Occasional Remote Improbable Improbable Improbable Occurrences per 00,000 Flt Hrs: Flight Hours per Occurrence Occurrences per Flight Hour Fleet Life Flight Hours per Occurrence:,000 0,000 00,000,000,000 0,000,000 Occurrences per Year Years per Occurrence,000 0,000 00,000,000,000 0,000, Occurrences per Flight Hour: Occurrence per Fleet Life, Fleet Life per Occurrence ,20 flt hrs /year x 0 Occ/00,000 flt hours = 8.8 Occ/year Catastrophic B C D E $M Occurrences per Year: CS/E Medium Critical 2 2 2B 2C PEO 2D $200K Years per Occurrence: Marginal B C D E $20K High,776,00 Negligible flt hrs / fleet life x B 0 Occ/00,000 C flt Dhours = E 77 Occ/life F 2E Low F 2F F 0R Occurrences per Fleet Life:, Fleet Life per Occurrence:

24 ssumptions Fleet Size: 68 aircraft ircraft Exposure Hours:,800 hours US rmy US rmy PEO Utilization: viation PEO 20 hours/yr viation Expanded Fleet Exposure Matrix Hours:,766,00 Matrix hours Occurrences per 00,000 Flt Hrs ircraft Life: 20 years Fleet Hours per Year: 88,20 hours Very Frequent Probable Occasional Remote Improbable Very Frequent Probable Occasional Remote Improbable Improbable Improbable Occurrences per 00,000 Flt Hrs: Flight Hours per Occurrence Occurrences per Flight Hour Fleet Life Flight Hours per Occurrence:,000 0,000 00,000,000,000 0,000,000 Occurrences per Year Years per Occurrence Occurrences per Flight Hour: Occurrence per Fleet Life,000 0,000 00,000,000,000 0,000, , Fleet Life per Occurrence Catastrophic B C D E $M Occurrences per Year: CS/E Medium Critical 2 2 2B 2C PEO 2D $200K Years per Occurrence: Marginal B C D E $20K High Numbers Negligible greater then B are easier C to D understand E and F apply 2E Low F 2F F 0R Occurrences per Fleet Life:, Fleet Life per Occurrence:

25 US rmy PEO viation Expanded Matrix Input Calculated Occurrences per 00,000 Flt Hrs Flight Hours per Occurrence Occurrences per Flight Hour Occurrences per Year Years per Occurrence Occurrence per Fleet Life Fleet Life per Occurrence ssumptions Fleet Size: 68 aircraft ircraft Exposure Hours:,800 hours Utilization: 20 hours/yr Fleet Exposure Hours:,766,00 hours ircraft Life: 20 years Fleet Hours per Year: 88,20 hours Frequent Probable Occasional Remote Improbable Very Improbable ,000 0,000 00,000,000,000 0,000, Fleet Life , Catastrophic B C D E $M Medium Critical 2 2 2B 2C PEO 2D $200K Marginal $20K High CS/E 2E B C D E Negligible Low F 2F F 0R

26 US rmy PEO viation Expanded Matrix Occurrences per 00,000 Flt Hrs Flight Hours per Occurrence Occurrences per Flight Hour Occurrences per Year Years per Occurrence Occurrence per Fleet Life Fleet Life per Occurrence ssumptions Fleet Size: 68 aircraft ircraft Exposure Hours:,800 hours Utilization: 20 hours/yr Fleet Exposure Hours:,766,00 hours ircraft Life: 20 years Fleet Hours per Year: 88,20 hours Frequent Probable Occasional Remote Improbable Very Improbable ,000 0,000 00,000,000,000 0,000, Fleet Life , Catastrophic B C D E $M Medium Critical 2 2 2B 2C PEO 2D $200K Marginal $20K High CS/E 2E B C D E Negligible Low F 2F F 0R

27 Consequences of Risk cceptance Occurrences per 00,000 Flt Hrs Occurrences per Flight Hour Occurrences per Year Years per Occurrence Occurrence per Fleet Life Fleet Life per Occurrence ssumptions Fleet Size: 68 aircraft ircraft Exposure Hours:,800 hours Utilization: 20 hours/yr Fleet Exposure Hours:,766,00 hours ircraft Life: 20 years Fleet Hours per Year: 88,20 hours Very Improbable Consequences of Frequent Risk cceptance: Probable Occasional Remote Improbable On the order of 2 to 7 Class accidents due to this hazard over Flight Hours per Occurrence,000 0,000 00,000,000,000 0,000,000 the life cycle of the aircraft Fleet Life , Catastrophic B C D E $M Medium Critical 2 2 2B 2C PEO 2D $200K Marginal $20K High CS/E 7-2 2E B C D E Negligible Low F 2F F 0R

28 Mishap Risk & Mishap Loss Mishap Risk over Time results in Mishap Loss Operate the system Risk (Predicted Loss) ($/Flt Hour) (Fatalities/Flt Hour) Time 6 (Flight Hours) Mishap Rate (ctual Loss) ($/Flt Hour) (Fatalities/Flt Hour) 28

29 Mishap History Based on this relationship between mishap risk and mishap loss, we can plot mishap histories on a risk matrix as follows: Total Cost from Class mishaps Severity = Total Number of Class mishaps $6,67,08 = = $ 6,0,08 59 Probability = = Total Number of Class mishaps Total Hours Flown 59 =,588,597.7 mishaps / 00,000 Flt Hrs 29

30 Mishap History Mishaps per Class No Total Cost Cost/Mishap 00,000 Flt Hrs 59 $6,67,08 $6,0,08.7 B 9 $8,85,2 $8, C 25 $7,,206 $69, D 2 $970,8 $8, Total 55 $98,609,5 0

31 Severity ($0 Y ) 8 The numbers plot on a chart like this $,000,000 $200,000 $20,000 Class $,000/Flt Hr Class B $00/Flt Hr Class C $0/Flt Hr Class D Mishaps US rmy viation Mishaps - OH-58D B C D E $,000/Flt Hr Mishaps per Class No Total Cost Cost/Mishap 00,000 Flt Hrs F 59 $6,67,08 $6,0,08.7 $0.0/Flt Hr B $00/Flt 9 Hr $8,85,2 $8, $0/Flt Hr C 25 $7,,206 $69, D 2 $970,8 $8, Total 55 $98,609,5 $0.0/Flt Hr $0.0/Flt Hr US rmy viation ccidents Oct 99 to 0 Sep 200 $/Flt Hr Source: US rmy $2,000 Risk Managment Information System $0.0/Flt Hr January Frequency Probability (ccidents(0 per X /Flight 00,000 Hour) Flight Hours $/Flt Hr Total effect (sum) of all hazards to date

32 Severity ($0 Y ) 8 7 Mishaps US rmy viation Mishaps - OH-58D B C D E $,000/Flt Hr $,000/Flt Hr $00/Flt Hr $0/Flt Hr $00/Flt Hr $0/Flt Hr $/Flt Hr F $,000/Flt Hr 6 2 $,000,000 $/Flt Hr $0.0/Flt Hr $200,000 $0./Flt Hr 5 $00/Flt Hr $0.0/Flt Hr $0.0/Flt Hr $20,000 $0/Flt Hr US rmy viation ccidents Oct 99 to 0 Sep 200 $/Flt Hr Source: US rmy $2,000 Risk Managment Information System $0.0/Flt Hr January Frequency Probability (ccidents(0 per X /Flight 00,000 Hour) Flight Hours

33 Severity ($0 Y ) Mishaps US rmy viation Mishaps - OH-58D B C D E $,000/Flt Hr $00/Flt Hr $0/Flt Hr F $/Flt Hr 6 2 $,000,000 $200,000 $,000/Flt Hr $ Series Series $0.0/Flt Series2 Hr Series 5 $00/Flt Hr $0.0/Flt Hr $0.0/Flt Hr $20,000 $0/Flt Hr US rmy viation ccidents Oct 99 to 0 Sep 200 $/Flt Hr Source: US rmy $2,000 Risk Managment B $.87 Information System $0.0/Flt Hr January C $ D$ Frequency Probability (ccidents(0 per X /Flight 00,000 Hour) Flight Hours

34 Severity ($0 Y ) 8 US rmy viation Mishaps US rmy viation Mishaps - Big Fleets B C D E $,000/Flt Hr $00/Flt Hr $0/Flt Hr F 7 $/Flt Hr $,000/Flt Hr 6 2 $,000,000 $0.0/Flt Hr $200,000 5 $00/Flt Hr $0.0/Flt Hr $0.0/Flt Hr $20,000 $0/Flt Hr US rmy viation ccidents Oct 99 to 0 Sep 200 $/Flt Hr Source: US rmy $2,000 Risk Managment Information System $0.0/Flt Hr January Frequency Probability (ccidents(0 per X /Flight 00,000 Hour) Flight Hours

35 Severity ($0 Y ) US rmy viation Mishaps 8 B C D E F ccident Hazards The risk Oct 99 of these - 0 Sep 2009 Source: US rmy Risk Management hazards Inf ormation sum System to give this result Total ccidents 6 5 $,000,000 2 $200,000 Fatality Loss of Situational wareness Brownout-Whiteout Poor utorotation Characteristics Single Engine ircraft Wirestrike $20,000 $2,000 Other Hazards - Human Factors Other Hazards - Material Failure Other Hazards - Environmental Numbers Frequency (ccidents /00,000 Flight Hours)

36 Matrix Relative Risk Values (Clemens) 2 00,000,000 0,000,000,000,000 00,000 0,000,000 0,000,000,000,000 00,000 0,000,000 00,000,000 00,000 0,000, ,000 0,000,

37 Helo Hazard Distribution

38 Helo Matrix Relative Values (Clemens) 5 x 00,000 = 500,000 x 0,000 = 0, x,000 = 65,000 2 x 0,000 = 0,000 6 x,000 = 6,000 2 x 00 = 200 x 0,000 = 0,000 7 x,000 = 7,000 5 x 00 = 500 x 0 = 0 2 x 00 = 200 x 0 = 0

39 Helo Matrix Relative Values (Clemens) 500,000 0,000 65, ,000 6, ,000 7,

40 Clemens Helicopter,000, , , , , ,000 00,000 00, ,000 2 B C D E F 00,

41 Helicopter B Clemens,000, , , , , ,000 00,000 00, ,000 2 B C D E F 00,

42 Helicopter C Clemens,000, , , , , ,000 00,000 00, ,000 2 B C D E F 00,

43 Side by Side Relative Risk by RC Clemens Clemens Clemens,000,000 Helicopter 900, , , , ,000 00,000 00, ,000 2 B C D E F 00,000 0 Helicopter B,000, , , ,000 Helicopter C,000, , , , , , , ,000 00,000 00,000 00,000 00, , ,000 B C D E F 2 00,000 0 B C D E F 00,000 0

44 Risk Pie Chart by RC

45 Risk Pie Chart by RC 65%

46 Risk Pie Chart by RC

47 Risk Pie Chart by RC

48 .6E-0 Hazard Risk Profile.6E-05.6E-06.6E-07.6E

49 .6E-0 Hazard Risk Profile.6E-05.6E-06.6E-07.6E E-05 Sum 5 x.6e-06 =.58E-05 x.6e-07 =.E x.6e-08 = 2.06E E-05 Sum x.6e-06 =.26E-05 6 x.6e-07 =.90E-06 2 x.6e-08 = 6.2E E-05 Sum x.6e-05 =.6E-05 7 x.6e-06 = 2.2E-05 5 x.6e-07 =.58E-06 x.6e-08 =.26E E-06 Sum 2 x.6e-06 = 6.2E-06 x.6e-07 =.6E-07

50 Severity ($0 Y ) Hazard Risk Profile 8 B C D E F 7 6 $,000,000 2 Fatality Helicopter $200,000 5 $20,000 $2, Frequency (ccidents /00,000 Flight Hours) 7 5 2

51 Severity ($0 Y ) Comparing Hazard Profile to ccident History 8 B C D E F $,000,000 2 Fatality 9 Helicopter B ccident History Helicopter $200,000 Numbers 5 25 $20,000 2 $2, Frequency (ccidents /00,000 Flight Hours) 7 5 2

52 Severity ($0 Y ) Comparing Hazard Profile to ccident History 8 B C D E F Helicopter B ccident History 6 $,000,000 2 Fatality 9 Helicopter Helicopter B $200,000 Numbers 5 25 $20,000 2 $2, Frequency (ccidents /00,000 Flight Hours)

53 Severity ($0 Y ) Comparing Hazard Profile to ccident History 8 B C D E F Helicopter B ccident History 6 $,000,000 2 Fatality 9 Helicopter Helicopter B Helicopter C $200,000 Numbers 5 25 $20,000 2 $2, Frequency (ccidents /00,000 Flight Hours)

54 Summary ttributes of a well-designed risk assessment matrix Severity scale covers full range of possible outcomes Probability calibrated with reference to an exposure interval Equally proportioned, logarithmic scales (, 0, 00, 000 ) Cartesian Orientation Increase up and to the right Risk levels assigned to cells consistent with contours of equal risk (iso-risk contours) B C D E F G H I Severity Frequency J >00 >0 > >0. >0.0 >0.00 >0.000 > > $2k $20k $200k $2M $20M $200M $2B $20B Injury, no lost work day Lost Work Day Permanent partial disability Low SSWG/Principal for Safety Fatality Medium - 0 Fatalities Serious - PEO 00 Fatalities High - CE,000 Fatalities Prohibitive SECDEF 0,000 Fatalities

55 Summary ttributes of a well-designed risk assessment matrix Sufficient probability or frequency categories so highest severity level can be assessed at the level of risk if the probability or frequency of occurrence is low enough risk assessment code for hazards whose risk has been eliminated Easily tailored with reporting of risk consistent with other systems within the family of systems. B C D E F G H I Severity Frequency J >00 >0 > >0. >0.0 >0.00 >0.000 > > $2k $20k $200k $2M $20M $200M $2B $20B Injury, no lost work day Lost Work Day Permanent partial disability Low SSWG/Principal for Safety Fatality Medium - 0 Fatalities Serious - PEO 00 Fatalities High - CE,000 Fatalities Prohibitive SECDEF 0,000 Fatalities

56 Value Math Definition: Understanding Probability Repeat a random experiment n number of times. If a specific outcome has occurred f times in these n trials, the number f is the frequency of the outcome. The ratio f/n is the relative frequency of the outcome. relative frequency is usually very unstable for small values of n, but it tends to stabilize about some number p as n increases. The number p is the probability of the outcome. p = f / n for very large values of n Rolling Dice Roll a single die 0 times. The expected value of each roll is.5. What you actually get is somewhat different. USMCOM Safety Data 20 Jan Trial.5 Risk Value verage for 6 trials verage for 0 Trials Summary Understanding Probability 0 2 Understanding Probability Simple example: Probability of rolling a with one die. Roll # - 5, f/n = 0 Roll #2-2, f/n = 0 Roll # -, f/n = / =. Roll # -, f/n = / =.25 Roll #,000: 6 s, f/n = 6/000 =.6 Rolls approach infinity f/n = Understanding Probability Hazard: H-6 strikes wire results in Class mishap Probability:.06E-06 occurrences per flight hour Flight Hr, no mishap, rate = 0,000 Flight Hrs, no mishap, rate = 0 76,82 Flight Hrs, mishap, rate = 5.676E-06 /flt hr 27,59 Flight Hrs, 2 mishaps, rate = 7.285E-06 /flt hr 700,62 Flt Hrs, mishaps, rate =.28E-06 /flt hr 0,000,000 Flt Hrs, 6 mishaps, rate =.600E-06 /flt hr,000,000,000 Hrs, 07 mishaps, rate =.07E-06 /flt hr Flight hours approach infinity, rate =.06E-06 /flt hr 56

57 pplying Probability Classifications to a military helicopter Fleet Size = 68 aircraft Utilization = 20 hours/year Life = 20 years/aircraft ircraft Life = 20 x 20 =,800 hours Fleet Exposure Hours = 68 x 20 x 20 =,776,00 hours Summary Expanded Matrix ssumptions Fleet Size: 68 aircraft ircraft Exposure Hours:,800 hours Utilization: 20 hours/yr Fleet Exposure Hours:,766,00 hours ircraft Life: 20 years Fleet Hours per Year: 88,20 hours Frequent Frequent Probable Probable Occasional Occasional Remote Remote Improbable Improbable Very Very Improbable Improbable Occurrences per 00,000 Flt Hrs Occurrences Flight Hours per Occurrence 00,000 Flt Hrs:, ,000 00,000,000, ,000, Occurrences per Flight Hour Fleet Life Flight Occurrences Hours per per Occurrence: Year, , , ,000, ,000, Years per Occurrence Occurrences per per Fleet Flight Life Hour:, Fleet Life per Occurrence ,20 flt hrs /year x 0 Occ/00,000 flt hours = 8.8 Occ/year Catastrophic B C D E F $M Occurrences per Year: CS/E Medium Critical 2 2 2B 2C PEO 2D $200K Years per Occurrence: Marginal B C D E $20K High,776,00 Numbers Negligible flt hrs greater / fleet life then x B 0 Occ/00,000 are easier C to flt D understand hours = E77 and Occ/life F apply Occurrences per Fleet Life:, E Low 2F F 0R Fleet Hours per Year = 68 x 20 = 88,20 hours Fleet Life per Occurrence: Occurrences per 00,000 Flt Hrs Flight Hours per Occurrence Occurrences per Flight Hour Occurrences per Year Years per Occurrence Occurrence per Fleet Life Fleet Life per Occurrence ssumptions Fleet Size: 68 aircraft ircraft Exposure Hours:,800 hours Utilization: 20 hours/yr Fleet Exposure Hours:,766,00 hours ircraft Life: 20 years Fleet Hours per Year: 88,20 hours Frequent Probable Occasional Remote Improbable Very Improbable ,000 0,000 00,000,000,000 0,000, Fleet Life , Catastrophic B C D E $M Medium Critical 2 2 2B 2C PEO 2D $200K Marginal $20K High CS/E 2E B C D E Negligible Low F 2F F 0R 57

58 Severity ($0 Y ) Severity ($0 Y ) Severity ($0 Y ) Probability = H-6 Mishap History Based on this relationship between mishap risk and mishap loss, we can plot mishap histories on a risk matrix as follows: Total Cost from Class mishaps Severity = Total Number of Class mishaps $,05,079,886 = = $ 5,72,85 8 = Total Number of Class mishaps 8 2,5,860 Total Hours Flown =.529 mishaps / 00,000 Flt Hrs Summary $,000,000 $200,000 ccidents on a Matrix $,000/Flt Hr $00/Flt Hr $20,000 $0/Flt Hr US rmy viation Mishaps - OH-58D B C D E $,000/Flt Hr $,000/Flt Hr $00/Flt Hr $0.0/Flt Hr $00/Flt Hr $0/Flt Hr $/Flt Hr $0/Flt Hr US rmy viation ccidents Oct 99 to 0 Sep 200 $/Flt Hr Source: US rmy $2,000 Risk Managment Information System $0.0/Flt Hr January Frequency Probability (ccidents(0 per X /Flight 00,000 Hour) Flight Hours $/Flt Hr $0.0/Flt Hr $0./Flt Hr $0.0/Flt Hr F 8 7 $,000/Flt Hr US rmy viation Mishaps - Big Fleets B C D E $,000/Flt Hr $00/Flt Hr $0/Flt Hr $/Flt Hr C/MH-7 H-6 U/E/MH US rmy viation Mishaps B C D E F The risk OH-58D of ccident these Hazards Oct 99-0 Sep 2009 events Source: sum US to rmy Risk Management Information System give this result $,000,000 $200,000 $00/Flt Hr $0.0/Flt Hr $0.0/Flt Hr $0.0/Flt Hr UH- OH-58D 6 5 $,000,000 2 $200,000 Fatality OH-58D Total ccidents Loss of Situational wareness Brownout-Whiteout Poor utorotation Characteristics Single Engine ircraft $20,000 $0/Flt Hr US rmy viation ccidents Oct 99 to 0 Sep 200 $/Flt Hr Source: US rmy $2,000 Risk Managment Information System $0.0/Flt Hr January Frequency Probability (ccidents(0 per X /Flight 00,000 Hour) Flight Hours OH-58/C C/RC-2-8 Wirestrike $20,000 Other Hazards - Human Factors Other Hazards - Material Failure Other Hazards - Environmental $2,000 Numbers Frequency (ccidents /00,000 Flight Hours) 58

59 Matrix Relative Risk Values (Risk Units)(Clemens) Summary Relative Risk Values Helo Matrix Relative Values (Risk (Clemens) Units) 00,000,000 0,000,000,000,000 00,000 0,000, ,000 0,000 65, ,000,000,000,000 00,000 0,000, ,000 6, ,000,000 00,000 0,000, ,000 7, ,000 0,000, Helicopter,000, , , , , ,000 00,000 00, ,000 2 B C D E F 00,

60 Severity ($0 Y ) Severity ($0 Y ) Summary Hazard Risk Profile Helo Probability Values for Each Severity Level Helo Probability Values for Each Severity Level.6E-0.6E-05.6E-06.6E-07.6E E-05.6E-0 Sum.6E-05.6E-06 5 x.6e-06 =.58E-05.6E-07 x.6e-07 =.E-06.6E x.6e-08 = 2.06E E-05 Sum x.6e-06 =.26E-05 6 x.6e-07 =.90E-06 2 x.6e-08 = 6.2E E-05 Sum x.6e-05 =.6E-05 7 x.6e-06 = 2.2E-05 5 x.6e-07 =.58E-06 x.6e-08 =.26E E-06 Sum 2 x.6e-06 = 6.2E-06 x.6e-07 =.6E-07 Comparing Hazards to ccident History Comparing Hazards to ccident History 8 B C D E F 8 B C D E F Helicopter B ccident History 6 5 $,000,000 2 $200,000 Fatality Helicopter 6 5 $,000,000 2 $200,000 Fatality 25 9 Helicopter Helicopter B Helicopter C Numbers $20,000 $20,000 2 $2, Frequency (ccidents /00,000 Flight Hours) $2, Frequency (ccidents /00,000 Flight Hours) 60

61 Conclusions High degree of precision? No Gets hazards to appropriate cell of the matrix Confidence that overall assessment reality Very useful for programs with: Reasonably good accident data for analysis well-designed matrix Just one of many tools for managing system safety risk 6

62 Don Swallom Safety Engineer MCOM Safety Office (256) Questions? 62