Statistical Approaches to Estimation of Nominal HEPs using Simulation Data

Transcription

1 PSA 2017 International Topical Meeting on Probabilistic Safety Assessment and Analysis Statistical Approaches to Estimation of Nominal HEPs using Simulation Data Yochan Kim 김요찬金耀燦 Risk and Environmental Safety Research Division Korea Atomic Energy Research Institute

2 Introduction Human reliability analyses (HRAs) Produce estimates of human error probabilities (HEPs) for significant tasks in complex socio-technical systems Various HRA methods use nominal HEPs The probability of a given human error when the effects of performance shaping factors (PSFs) have not been considered [Swain and Guttmann, 1983] HEP = f (HEPs PSFs) HEP = Nominal HEP * PSF multipliers SPAR-H HEART PSF type PSF level Multiplier Stress/ Extreme 5 Stressors High 2 Nominal 1 Complexity Highly complex 5 Moderately complex 2 Nominal 1 PSF Multiplier Unfamiliarity 17 Time availability 11 Ability to detect and perceive 10 Operator experience 4.8 MCR environment 8 High workload 6 2/17

3 The necessity of human reliability data collection Because a wide range of quantitative estimates in the previous HRA methods are not supported by solid empirical bases Human error probability (nominal HEP) Recovery factor Dependency PSF effects on HEPs Data collection efforts CAHR [Sträter, 1996] CORE [Kirwan et al., 1997] SACADA [Chang et al., 2014] OPERA [Jung et al., 2016] 3/17

4 Traditional HEP Estimation from Empirical Data Average HEP (human error probability) formula HHH i = M i /N i = M i /(M i + S i ), HEP i : the probability of the th i type of UA occurrence M i : the number of erroneous behaviours (i.e., UAs-unsafe acts) S i : the number of successful acts N i : the number of total UA opportunities Bayesian update based on M i and N i In some cases, the average HEPs are considered as nominal HEPs or compared with existing nominal HEPs Example Nominal HEPs generated from CORE-DATA [Basra and Kirwan, 1998] HEP list of GRS (Gesellschaft für Anlagen-und Reaktorsicherheit) [Preischl and Hellmich, 2013] Basic HEP estimates of soft controls in digitalized control room [Jang et al., 2013] 4/17

5 Example HEP list of GRS Basic HEP estimates of soft controls Nominal HEPs generated from CORE-DATA 5/17

6 Nominal HEP by Traditional Approach The traditional approach views nominal HEPs as the average HEPs under various contexts. Relatively easy to collect the relevant data Simple to mathematically treat the data for calculating an HEP Inaccurate when the PSFs during data acquisition does not reflect typical situations of the systems When an HEP is obtained from experiments with novice students, not qualified operators,,, If someone collects a reliability data from regular training programs that comprise only simple responses of operators,,, Difficult to exactly measure how much the estimates are skewed from the ideal values Difficult to know which factors were determinant in the HEP estimates 6/17

7 Alternative Nominal HEP Estimation The nominal HEP can be seen as the HEP where all plantspecific or situation-specific PSF effects were not entailed Define nominal states with PSF values E.g.) SPAR-H method Attempt to identify effects of possible PSFs Predictive analytics in statistics/ data mining Regression techniques Bayesian network learning Neural network modeling Classification tree generation Derive an HEP in nominal states <Nominal states defined in SPAR-H> PSF Example Stress /Stressors PSF Level Extreme 5 High 2 Nominal ( Low) 1 Complexity Highly complex 5 Moderately complex 2 Nominal ( Easy) 1 Experience /Training Low 3 Nominal( Medium) 1 High 0.5 Multiplier Example HEPs during operations of soft controls [Kim et al., 2015] Diagnostic HEPs for computer-based control room [Kim et al., 2016] 7/17

8 Example HEPs during operations of mouse controls Logistic regression Automated variable selection Practice Level System Understa -nding X3 Diagnostic HEPs for computer-based control room Comparison between HEPs in different PSFs HEP estimates Data from Domestic full-scope simulator HAMMLAB (HAlden human-machine LABoratory) experiments Procedure Screen Descrip -tion X8 Switch Error Basic HEP = (a) a network for screen switch error PSFs Multiplier HSI [Good-> Poor] 1.03 Procedure [Good-> Poor] 2.50 Stress level [Moderately high -> Extremely high] 2.15 Experience [Skilled->Not-skilled] /17

9 HEP in Nominal States Pros More concretely define the meaning of nominal HEP with PSF levels Beneficial to identify the relationships between nominal HEPs and PSFs Cons Difficult to gather human reliability data regarding diverse PSF levels Difficult to extract all quantitative relations between the PSFs and HEPs Estimated HEP can be distorted by effects of unconsidered PSFs 9/17

10 Case Study Objective: Compare two HEPs predicted by different estimation approaches Null Hypothesis: Average HEPs and HEPs in nominal state are similar Alternative Hypothesis: there are significant differences between Average HEPs and HEPs in nominal state Data source: OPERA database obtained by the HuREX (Human Reliability data Extraction) framework [Kim et al., 2017] 223 training records Emergency (18) + abnormal situations (205) Unsafe act types: 4 cognitive activities Information gathering and reporting Response planning and instruction Situation interpreting Execution 2 error mode EOO (Error of omission) EOC (Error of commission) 10/17

11 HuREX Framework Overview information Reference plant name/type Operating mode Crew information and training information Environmental issues Overall crew characteristics including communication and leadership Response information Related procedure and instruction Task success and failure Task type Component/system type to be controlled Unsafe act information Time pressure Human-machine interface attributes Communication quality Task complexity Task familiarity Procedure quality Recovery information 11/17

12 Case Study Average HEP Average HEP formula HHH i = M i /N i = M i /(M i + S i ) Upper-bound zero failure assumption: human error =1 Cognitive Activity Error opportunity Error mode: EOO* Human Error # HEP Error mode: EOC* Human Error # Information gathering and reporting E E-03 Response planning and instruction E E-03 for information gathering E E-03 for procedure following E E-03 for manipulation E E-02 for ex-mcr control E E-03 Situation interpreting E E-01 Execution E E-03 HEP *EOO (Error of Omission); EOC (Error of Commission) 12/17

13 Case Study HEP in Nominal State Logistic regression with automated variable selection Dependent variable: error occurrence for each cognitive activity and error mode Independent variables: 26 variables in OPERA database Estimation of conditional probabilities under levels of independent variables If we define the nominal state as the positive or typical level in every PSF, Cognitive Activity Error mode: EOO* Error mode: EOC* Nominal State Nominal HEP Nominal State Nominal HEP Information gathering and reporting E-04 Confusing statement = TRUE 1.61E-03 Response planning and instruction for information gathering Simulation mode = 1.96E-05 EMERGENCY Simulation mode = for procedure following Continuous action step 1.05E-04 EMERGENCY for manipulation (discrete) = FALSE Contingency action part 8.68E-04 Training experience = = FALSE 5.68E-04 for manipulation (continuous) TRUE 9.83E-05 Description of object = for ex-mcr control Multiple constraint = TRUE FALSE 8.43E-04 Situation interpreting - - Execution Number of manipulation =1 1.52E-03 Simulation mode = EMERGENCY Time pressure = INSIGNIFICANT Confusing statement = False *EOO (Error of Omission); EOC (Error of Commission) 1.25E E-03 13/17

14 Case Study Comparison Cognitive Activity Error mode: EOO* Error mode: EOC* Average HEP HEP in nominal state Average HEP HEP in nominal state Information gathering and reporting 7.22E E E E-03 Response planning and instruction 1.27E E E-04 for information gathering 3.47E E E-03 - for procedure following 7.38E E E-03 - for manipulation 4.82E E-04; 9.83E E-02 - for ex-mcr control 1.72E E E-03 - Situation interpreting 3.33E E E-01 Execution 9.80E E E E-03 The Average HEP is similar with the HEP in nominal state. The Average HEP is over 5 times higher than the HEP in nominal state. The Average HEP is over 20 times higher than the HEP in nominal state. 14/17

15 Discussion (1) The average HEPs were significantly different with the HEPs in nominal states. The estimates in this case study are not convincing yet due to sample size or uncertainty issues. However, it is obvious that use of estimates from data should be carefully interpreted. HEP HEP Even sample size Even sample size Large sample size Small sample size NEGATIVE LEVEL POSITIVE LEVEL (eg.) (Poor HMIs) (Good HMIs) AVERAGE NEGATIVE LEVEL (Poor HMIs) POSITIVE LEVEL (Good HMIs) AVERAGE Which one could be nominal HEP? 15/17

16 Discussion (2) Considerations Clear definition of Nominal HEP is important. If possible, both the average HEPs and HEPs in nominal state should be investigated. If not, information of all relevant PSFs should be annotated during data collection and the information should be considered during nominal HEP determination. It should be shown that the estimates for nominal HEPs are conservative. To do so, it should be reviewed that too positive states of any PSF were included or not. Comparison between results from multiple data sources is desirable. Plant experience data is valuable. 16/17

17 Thank you for your attention.