Audit report 1 Background 1.1 Analysis and assessment of risk - key elements for safe operations

Transcription

1 Audit report Report Title Report following audit of the implementation, follow-up and use of risk assessments in operations and in connection with minor modifications 1 Background 1.1 Analysis and assessment of risk - key elements for safe operations Analyses and/or assessment of risk are key elements in carrying out various activities in a safe and prudent manner. This applies both to the risks associated with performing various activities and the risks associated with the changes that an activity may entail. However, the analysis and assessment of risk only have real value when they are used to provide input, recommendations and guidance to those who decide, plan and implement activities. This entails that the analysis should include all relevant conditions (at the right level), they must be available at the right time, they must be understandable and provide appropriate decision support to those involved/affected at different levels. Investigations following events, results and findings in various audits and our general knowledge of the petroleum activities, provide, in our opinion, clear indications that there is room for improvement in various analyses and risk assessments. Improving various analyses and risk assessments alone will not prevent accidents and undesirable incidents from occurring, but we believe there may be a potential to reduce the risk of accidents by reconsidering and possibly revising or adjusting some of today's practices. Questions that the PSA have asked are; what are the problems/issues or challenges associated with the current analyses and risk assessments? Is it that the methods used are not suitable or good enough, is the degree of involvement the main challenge, is the follow-up or communication of the results not good enough, or are other factors crucial in relation to not intercepting, analysing and dealing with matters of significance for risk and/or that those involved were not made aware of important factors? To gain a better basis for understanding where the challenges or problems are and/or where the potential for improvement is greatest, the PSA decided to conduct a survey in the form of a questionnaire. The aim was to utilise such a survey to have an impression of how those who are involved in or affected by different types of analysis or assessments perceive the challenges, and form a basis of information for any improvements, locally within individual enterprises or generally in the industry. This report contains a brief summary of the completed survey, and our assessments of the information that emerged through the survey and the analysis carried out on the results of the survey. We have chosen to highlight some of the factors/findings which we consider to be particularly important for further review.

2 2 Implementation 2.1 Identification of some issues related to analysis and assessment of risk In 2009, the PSA conducted an audit to determine how different types of risk analysis or assessment are implemented and used. The activity was conducted in the form of a questionnaire sent out to a number of job categories at selected facilities. The survey was conducted during the period from April to July. The audit activity was limited to examining how different types of analyses and risk assessments are used in connection with minor modifications, and in connection with regular/operational activities. The reason for this delimitation was based on our belief that it is precisely in the intersection between: the activities which are handled as separate projects and those which are not, and what is considered and treated as ordinary/routine activities and those which require special attention, that there are sometimes large variations in the industry in terms of how different types of analyses and assessments are conducted and used. The questions posed in the survey dealt mainly with issues related to: how the selected job categories on a number of facilities or installations are involved in various types of analyses and assessments, how the assumptions and results from different analyses are communicated and understood, and whether or not various types of analyses are perceived as important contributions to safety. Three scenarios were addressed in the survey. For each scenario there was a set of questions related to the implementation and use of analyses in the situation as described in the scenario. The intention of using scenarios was a desire to gain better insight into how the analyses and assessments are used and understood in specific situations, rather than having answers on a basis where the respondents were free to base their answers on very different situations and viewpoints. The first scenario dealt with the assessments or analyses performed when a problem is detected in the process facility or during the implementation of a drilling/well operation. The second scenario dealt with the assessments or analyses performed when other than those involved in the daily operation are planning a minor job/work package, while the third scenario referred to the assessments or analyses performed as part of the preparations for or implementation of, a minor job/work package. Finally, some questions of more general nature relating to the assessment and analysis of risk were asked. Initially, the survey placed emphasis on the fact that all answers would remain anonymous. Still, it was up to the individual to decide whether he/she wanted to participate or not. The design of the questionnaire and the implementation of the survey were conducted in cooperation with Preventor AS, Safetec Nordic AS and Studio Apertura (NTNU Social Research). Henceforth, these companies are collectively referred to as the Consultants in this report. The compilation of the results and the analysis and assessment of the responses was also carried out by the Consultants. 2.2 A non-typical audit methodology

3 The use of surveys as a method to gather information is used by the PSA in several contexts such as the project Trends in risk level in the Norwegian petroleum activities, RNNP. The method used here is different from our traditional supervision methods, perhaps particularly in the sense that it is not possible for the individual facility or company to provide supplemental information, and that there is no verification of the assertions and responses which emerge. The results of the survey will therefore not be used by us, on an independent basis, for assessing compliance with applicable rules and requirements. In many cases, it is also difficult to relate the subjects and issues raised in the survey directly to specific regulatory requirements. Relevant regulatory requirements for this audit are requirements in the Framework Regulations and the Management Regulations, which deal with requirements for analysis and the basis of decisions, and requirements relating to enabling individuals to perform a safe and prudent job. At the same time, it is largely up to the individual responsible player how to meet this requirement. This means that players can choose different solutions, which in turn makes it difficult to compare responses across companies or across the participating facilities. The responses, the analysis and assessment of the responses, must be primarily considered as a basis for further work. 3 Results and assessments 3.1 The compilation and analysis of the responses are presented in a separate report Results from the survey are presented in the report Risk assessments - implementation, follow-up and use in operations, maintenance and modifications, Report No , prepared by Preventor AS, hereinafter referred to as the Preventor report. The Preventer report contains a representation of the responses, analysis of data, discussion of results and a summary. The Preventor report and the questionnaire are attached to this report. The analysis and results in the Preventor report are based on the total sample included in the survey. The results are presented mainly in the form of an average for each question. Additionally, a frequency distribution is given, illustrating the variation in the responses to each question. The Preventor report also contains a discussion and summary where the Consultants have sought to highlight factors of a more general character. Moreover, they have indicated potential problems and challenges. A total of 385 positions were sent questionnaires, distributed over 23 facilities, with seven operators. Of these, 231 people elected to respond, giving a response ratio of approx. 60%. 3.2 The PSA assessments of the survey This chapter presents our assessments of the information collected through this survey. We support the results and conclusions drawn up in the Preventor report, and therefore generally refer to it, but we have chosen to highlight some factors and challenges that we see as important, and we have posed some questions related to these factors Risk assessments when problems / errors occur It emerges from the survey that the operational management is always involved in this type of discussions/decisions, while we see more varying degrees of involvement for other job categories. This is,

4 in our opinion, to be expected, as there normally would not be time and resources for a high degree of involvement in this phase. When it comes to the individual's perception of whether one should have been more involved in these types of discussions, we see that the vast majority are satisfied with their involvement. However, a relatively large number of respondents believe that others than those involved in the daily operations should have been more involved than is the case today and we could therefore question the role pattern in such contexts. A large number of respondents believe that the expertise of those involved in these types of discussions is adequate, and that the assessments made are satisfactory. Moreover, a large percentage says that information about relevant safety functions is available and actively used. It appears that both informal and more formal discussions between operational personnel and professionals or support functions outside of the operative unit is the most used form of communication/consultation in these types of situations. At the same time, it appears that a relatively large percentage believe that more formal analyses are carried out, even if they are not themselves involved in these types of analyses / assessments. We note that relatively many people have a perception that someone else is performing these more formal assessments. Approximately 75% responded that there are guidelines and requirements in governing documents which govern whether or not to perform a formal risk analysis / assessment, while around 22% replied that they did not know. The fact that a large percentage do not know can possibly be both natural and reasonable, as there apparently were no individuals from the categories that include leading operational personnel in this don t know group. However, we believe there is reason to examine this aspect. It appears that a relatively large percentage believe that in this type of situation a verification is made in relation to the current overall risk analysis (QRA / TRA / TRABA), barrier analysis, risk maps, etc. A relatively large proportion of people also answered don t know or to a lesser degree to the question of whether results, assumptions and constraints inherent in this type of analysis are assessed when such situations arise. Contractor employees and the categories area operator / process engineer / operations engineer / production engineer / maintenance technician constituted the largest proportion of the don t know group. There may therefore be reason to look more closely at how these factors, particularly preconditions and assumptions, are attended and made visible. It also emerges several places, including in a large number of free text comments that overall risk analyses and the like have little value, are difficult to understand, etc. This is not necessarily worrisome, but there may be reason to question the usefulness of this type of analysis in an operational context in terms of suitability and the availability of the information that these analyses represent Planning of minor repairs / modifications Based on our general knowledge of the industry, there is great variation in relation to the role of (or the formal requirements for) the HSE Manager/HSE Coordinator in this type of planning/preparation. We note that about 40% of the HSE Manager/HSE Coordinator category state that they are seldom involved

5 in this type of planning / preparation, especially in relation to the role the HSE Manager/HSE Coordinator plays for many organizations in relation to safeguarding and following up the TRA/QRA. A relatively large proportion of the different job categories (16% - 60%) state that they would like a greater degree of involvement in the analyses performed by anyone other than those involved in daily operations. In particular, the HVO/VO (safety delegates) category wants a greater degree of involvement. Moreover, about 25% said that the operational organization should be more involved in this type of planning/preparation. In our opinion, it should therefore be assessed if the degree of involvement is correct. About 80% respond that these types of analyses/assessments are relevant to their role and job performance. A relatively large percentage also state that it is difficult to understand which information in the analyses (results, assumptions and expectations) affects them and their own job performance, including the consequences of deviating from the assumptions, etc. Furthermore, a relatively large percentage (about 35%) say the results, presumptions and assumptions from this type of analysis are not been made known to them. Contractor employees disagreed more with the statement that one was always made aware of the results and recommendations of the risk analyses as compared with operator employees. It also appears that over 40% completely or partially disagree that the contractors involved in these kinds of jobs have sufficient knowledge of risk analyses/assessments applicable to the facility. We can therefore question whether and how the information in the analyses and assessments can be made more accessible to those involved in the planning and activity Risk assessments prior to the execution of the work There are different opinions within the different job categories of how often other risk analyses (such as TRA, HAZID and HAZOP) are used as a basis for planning safe job performance for minor modifications. Over half of the respondents reported that SJAs are used in these types of situations. Furthermore, 85% stated that they always or usually regarded an SJA as a useful tool for improving safety. There were also several who claimed (in free text) that the SJA is used too much, and, for that reason, could lead to weakening its intended function as an important tool. An important observation is that as many as one of ten states that operations are started without implementing the defined risk-reducing measures. Regardless of what causes so many people to answer this, this should be looked into. 37% replied that they completely or partially disagree that major accident potential was assessed in the SJA. Moreover, only about 20% reported that previously conducted analyses were taken into account in performing the SJA. There were also several comments which may indicate that the major accident potential is not a key part of the SJA. Additionally, approximately 31% stated that they disagreed with the statement that the results of risk assessments (for example: risk factors, assumptions, measures) are always attached to the work permit. We ask ourselves how reviews of the major accident potential are performed, in light of the conditions noted above. If this is not registered and handled in an SJA, it seems

6 somewhat unclear how matters of importance to the major accident potential are considered and safeguarded for this type of activities. In particular, we question how those performing the job will be made to understand and handle matters of importance to the major accident potential. In an open question about factors which could improve the usefulness of the SJA, a number of conditions were pointed out with respect to the use of the SJA and the quality of an SJA. It is our assessment that the apparent conditions appear to provide a basis for seeing if there are areas for improvement for the use and quality of the SJA and the associated risk assessments. It is further our opinion that through this survey, a possible hole or gap has been identified between the more formal analyses and SJAs in relation to safeguarding matters of importance to the major accident potential. 4 PSA's preliminary summary of the audit activity For the majority of the questions, the results are not clear, and the usefulness of the survey, for the individual players and the individual facility or facilities, lies in an assessment and interpretation of both average results and especially the spread in the individual responses. Moreover, the responses must be seen in the context of current internal requirements, guidelines and processes which the individual players and facilities have established for the implementation and use of risk analyses in situations similar to those mentioned in the survey. It is our opinion that the survey confirms that there is room to improve the current planning, implementation and use of various types of analyses and assessments. Here we will specifically highlight that a large percentage of those surveyed express that information (results, assumptions and conditions) from these types of analyses is difficult to understand, poorly adapted to the individual's competence and needs in relation to carrying out their own work in a safe and prudent manner. Based on the answers to various questions, we are somewhat uncertain as to how major accident risk is sufficiently controlled. It also emerges that there is limited knowledge regarding different analysis methods and their applications and limitations. Moreover, it appears that there are uncertainties related to the roles, responsibilities and ownership in connection with implementation and follow-up of important circumstances. Based on the total sample responses in the survey, it is, in our opinion, difficult to point out obvious weaknesses or deficiencies in the way different types of risk analyses are performed and used in situations similar to those mentioned in the survey. From what the PSA can see, there is still basis for two important questions in this regard: Where are the larger or smaller challenges in relation to the subjects affected by the audit activity: o In the procedures, instructions, guidelines or processes? o The current analysis methods? o The competence and training? o The roles and responsibilities? o Or other factors? Is there reason to be satisfied with the situation as it emerges from the survey or does the result contain conditions that warrant further work?

7 5 Participants from the PSA Odd Thomassen Vidar Kristensen Hanne Etterlid Einar Ravnås Eivind Sande 6 References Preventor - Report ; Risk assessments - implementation, follow-up and use in operations, maintenance and modifications, dated 28 January 2010