A Guide to Develop Safety Performance Indicators (Draft no.1 22/5/2016)

Transcription

1 A Guide to Develop Safety Performance Indicators (Draft no.1 22/5/2016) Yu Pak Kuen Monitoring and measuring performance has always been part of safety management systems. However, such systems frequently overlook process safety issues because it is difficult to know what to measure. Many organizations including Government Departments use negative indicators such as accident rates and scores of safety audits as the primary measure of the OSH performance. As a result of the failure of outcome indicators on their own to provide an adequate indication of how OSH is managed in an organization, safety and health professionals and some organizations is now identified a need for additional measures of safety performance. This guide is written with the aim to assist organizations to generate additional measures of OSH. The safety performance indicators (SPIs) will focus on how successfully an organization is managing and performing in relation to OSH. These indicators are known as process measures of performance.

2 1 Introduction 1. This guide is intended for senior managers and safety professionals within organizations that wish to develop safety performance indicators (SPIs) to provide assurance that major hazard risks leading the occurrence of serious accidents/incidents are under control. A small number of carefully chosen indicators can monitor the status of safety management systems and provide an early warning should controls deteriorate dangerously. 2. Many organizations rely heavily on failure data to monitor safety performance. The consequence of this approach is that improvements or changes are only determined after something has gone wrong. Effective management of major hazards requires a proactive approach to risk management, so information to confirm critical systems are operating as intended is essential. Switching the emphasis in favour of SPIs to confirm that risk controls continue to operate is an important step forward in the management of major hazard risks. 3. The method of setting SPIs outlined in this guide requires those involved in managing process safety risks to ask some fundamental questions about process control systems, such as: What are gone wrong in the process that causes the occurrence of major accidents/incidents? What existing controls are in place to prevent these major accidents/incidents? What does each control deliver in terms of a safety outcome? How do we know they continue to operate as intended? 4. Organizations who have adopted SPIs under contracts and have fatal accidents or serious incidents need to submit report to demonstrate that they have: increased assurance on risk management and protected their reputation; demonstrated the suitability of the risk control systems of the processes that caused the accidents/incidents; made better use of SPIs information collected for improving their safety management systems. 5. SPIs can be used to highlight a "soft spot" in the organization, thereby triggering off further analysis to find out the underlying causations that is causing the low indicator, and then ultimately to lead to a solution to the problem.

3 2 How Safety Performance Indicators(SPIs) fit within safety management systems (SMS) measuring activities 1. Safety Management Systems (SMS) have been defined by Gallagher as a combination of the planning and review, the management organisational arrangements, the consultative arrangements, and the specific program elements that work together in an integrated way to improve health and safety performance (Gallagher, 2000:1). 2. The SMS in Hong Kong includes 14 elements that will address different needs: Top level system needs including safety policy and safety organization, safety committee; Day-to-day operation including safety rules and regulations, process control, health assurance programme; Longer-term safety of the workplace including risk assessment, emergency, PPE program Personnel-related systems including safety training, safety promotion, subcontractor management Effectiveness of the SMS including safety inspection and accident/incident investigation as measuring activities. 3. Safety audit is the most commonly used means for organization to check the performance of SMS elements against their performance standards. Under the law, safety audit should look at both implementation and functionality of the system are efficiency, effectiveness and reliability. Thus the outcomes of safety audits are very important safety performance indicators to monitor the organizations safety management systems are in place. However, past experiences and outcomes reflect that safety audit is not too effective in evaluating the contractor s process safety performance. Using safety audit alone for measuring the safety performance particularly after the occurrence of fatal accident of contractors is reactive and insufficient to reflect the ability of contractor in preventing the recurrence of the accident. A better method should be developed.

4 4. The use of process safety performance indicators fits between the formal, quarterly safety audits and more frequent weekly workplace inspection and daily safety observation programmes. It is important that a safety audit programme is designed to address legal and contractual issues when compared to the information gained from SPIs. Deficiencies uncovered by a safety audit may highlight the need for a new performance indicator and vice versa. Therefore, the main reason for measuring process safety performance by SPIs is to provide ongoing assurance that the organizations process risks are being adequately controlled. SPIs is a complimentary activity to give more frequent or different information on system performance. Safety inspections Accident/incident investigation Measuring Safety observations Safety performance indicators Safety audits 3. Design of safety performance indicators 1. Most systems and procedures will deteriorate over time, and safety system failures discovered following a serious incident frequently surprise senior managers, who believed that the controls were functioning properly. 2. Thorough investigations are conducted following breaches of safety in the workplace. These safety breaches may or may not result in work-related fatalities or serious incidents. In 1997, James Reason pointed out that accidents can be avoided with an approach of defense-in-depth. He illustrated this by the Swiss

5 Cheese Model, where the hole in the cheese showed failure in the barrier. If the hazards transit through the holes in the barriers, it can convert to either a serious incident or a failure in the system. Therefore, to prevent the re-occurrence of accidents/incidents, the organizations must be able to investigate the accidents/incidents thoroughly to identify and evaluate the causes of the failures of the function barriers in the process control systems and choice the right preventive defense strategies, develop safety performance indicators to measure the effectiveness of the implementation of these defense strategies. 3. However, accidents/incidents occur is not based on linear progression of failures but may occur due to complex interactions in a complex system. The investigations conducted following accidents/incidents will find that management factor and job factor played predominant roles as the precursor to the failure of systems that resulted in the accidents/incidents occurring. However, the success of this approach to measuring safety performance will be the ability of an organisation to go beyond an exclusive focus on indicators of employee behaviour, or even management behavior (human factor), to encompass indicators which address the total workplace environment. These will range from generic indicators of the role, status and effectiveness of hazard reduction programs to more specific indicators of control procedures in place, action taken to meet agreed and/or required standards, the safety consciousness, morale and well-being of employees and the extent of workplace participation in safety issues. 4. While auditing usually confines to looking for the efficient, effectives and reliable of safety management systems rather than outcomes. For example, safety performance indicators, such as failure rate of the limit switches for the landing gates of passenger hoists of a building, reached the lowest tolerable limit. It indicates there is a soft spot in the maintenance process. Further analysis reveals that preventive maintenance needs to be improved and the internal personnel for the company cannot identify the changes necessary to improve. Therefore, some way of measuring or assessing outcomes will be required. In other words, in order to monitor safety management, measurement of its performance will be required. This, in turn, will require the development of safety performance criteria or indicators. Like auditing approaches, SPI involves a focus on the system which gives rise to accidents rather than the accidents themselves. 5. Many measurement tools or indicators are available but none of them standing alone can satisfy all the needs. Examples of such tools and indicators include: Measurement of consequence (Outcome or lagging indicators focus on the measurement of loss, such as lost time injury frequency rates, workers compensation costs or fatality incidence rates) Measurement of cause (management, job and human factors) Measurement of control (process control programme)

6 Each indicator is devised to assess only a certain aspect or component of the safety performance of an organisation. Each of the measurement on its own cannot reflect the big picture of the safety scenario. Using any indicator in isolation must therefore be handled with care to avoid misrepresentation or misunderstanding. It is important that organisations develop and use a balanced mix of both outcome indicators and SPIs to effectively measure OSH performance. The notion of using SPIs to improve safety performance has evolved from the model of process control in quality management. In this model, process steps leading to sub-standard outputs are defined and examined to identify factors that have caused this sub-standard output. By addressing these factors in a timely manner, the standard of the output can be improved. When this process is applied consistently over time it leads to a cycle of continuous improvement. 6. Example of using quality management model on hazardous materials management and working at height

7 a) Inputs (Measures of hazard burden) Input SPIs are measures of what actions or initiatives have been undertaken in the workplace to improve OSH and can provide useful information on participation, leadership and communication. Although they are seen as good indicators of commitment and effort, they are not indicators of the effectiveness of the activities. In practical terms, organisations will need to define those activities in their safety management system that need to be promoted and reinforced. A focus on these activities can be used to visibly drive the safety culture in the workplace. SPIs can be developed for these activities. b) Processes (Monitoring Key Risks) Process SPIs are measures that are used to monitor the major risks in an organisation. These can be developed by identification of the key contributors to the outcomes of concern and developing measures to monitor behaviours and practices. In developing SPIs of this type, organisations should focus on all core risks and ensure that measures are in place to provide an indication that risk control practices are being followed.

8 c) Outputs (Measure of success) Output indicators are used to measure outputs in terms of the achievement of objectives, and on the progress towards the achievement of higher level OSH goals and targets. d) Outcomes (Measures of failures) Reactive monitoring of adverse outcomes resulting in injuries, ill health, loss and accidents with the potential to cause injuries, ill health or loss. 4 Organisational arrangements to implement SPIs 1. Set up an implementation team It will usually be a safety professional within an organisation who will responsible for the work and steer it through to implementation. However, it is more appropriate to form a team to manage the introduction of SPIs. This has the benefit of drawing in people from a range of business operations particularly those involved in the accidents/incidents, providing the opportunity for pooling ideas, especially from employees who have direct knowledge of how control system (barriers) deteriorate or become ineffective. Safety committee may also be helpful to oversee the implementation programme and to check the indicators match current business priorities. 2. Develop a risk profile for the organisation and/or identify OSH outcomes of concern Setting the risk profile is about selecting the right indicators to provide just enough information about the adequacy of process safety controls. SPIs set for the whole organizational will, by their nature, tend to be more generic, whereas those set at plant or site level will be more focused on key activities or processes and give more direct feedback on the functioning of those activities. It is not necessary to measure every aspect or safety element of a safety management system. Since this is a programme required to be conducted after the occurrence of fatal accidents or serious incidents, the authorities will, under contracts, require the involved organization to conduct SPIs at organizational level, site level or both levels. The required SPIs will confine to the contribution causes of the mishaps and related process safety management system. Focusing on a

9 few critical risk control systems (barriers) will provide a sufficient overview of performance. Problems highlighted in one risk control system should trigger a more widespread review. SPIs set at plant or site level provide managers with routine information to show that specific processes or activities are operating as intended e.g. plant design, plant change, planned inspection and maintenance within that sphere of operations. Indicators at this level provide very specific performance information on the activities selected. SPIs at site level provide an overview of critical systems operating across the whole site e.g. managing contractors, emergency arrangements, staff competence etc. SPIs set at organization level will be based on corporate goals and objectives (a top-down approach), but importantly, should also feature information fed up from site level. For complex sites such as multi-site, the SPIs measurement system can be based on a hierarchical approach with very focused site level SPIs feeding up to more generic site level organization level indicators. 3. Review current arrangements for managing OSH to identify areas for improvement To help decide what are gone wrong and how, it is useful to consider the underlying causes of an accident/incident. This is the primary failure mechanism that gives rise to an accident/incident and can usually be categorized by factors (management factor, job factor and human factor) that challenge the integrity of plant or equipment. An assessment of all the factors should help establish the scope of the process measurement system and ensure focus on critical issues. Once the outcomes of occurrence of accidents/incidents in the workplace have been established, and the type and nature of workplace hazards identified, an appraisal should be undertaken to examine if and how those risks are currently being managed. 4. Identify the risk control systems and decide on the outcome The risk control systems in place to prevent major accidents/incidents can be obtained by studying the accidents/incidents investigation report and conduct cause analysis on the factors (Management, Job and/or Human factors). For each identified factor, select the sub-factors (Defects) that causes the accidents/incidents. There may be more than one sub-factors identified in the initial selection. Choose the most critical defect or defects that believe to be the root cause(s) of the accidents/incident. Identify its sub system and the various components of the critical defect; Identify this system s location within the organization s safety management system;

10 Identify the system s relationships with other systems; Compare the effectiveness of the current system with the way the system ought to function in terms of satisfy the organization s information needs To other systems and repeats the first four stages of the analysis. For example, the accident investigation report of an accident disclosed that the critical defect was found in the accident investigation element of the safety management system (defect - poor safety and health management). The components of this system include accident reporting system, accident investigation system, distribution system, trend analyses system etc. The sub systems include compliance procedures, investigation procedures, information requirements, follow-up procedures and dissemination procedures. The cause analysis is conducted on accident investigation system of the involved organization is illustrated below.

11 Decide what success looks like for each control system (barrier) of this or these sub-factors by asking the following questions. What control systems are in place to control the risks? Why this process needed such a risk control system? What is the desired safety outcome? Whether or not the outcome is being achieved? Did an investigation to see why the system failed being conducted? 5. Identify the most important parts of the risk control systems It is not necessary to measure every part of a risk control system. Identify the critical component(s) and sub-systems of each risk control system, (i.e. those actions or processes which must function correctly to deliver the outcomes). Consideration should be concentrated to the following parts: Which activities, or operations must be undertaken correctly on each and every occasion? Which aspects of the system are liable to deterioration over time? Which activities are undertaken most frequently? Examples of common weakness in accidents/incidents system are: Reporting: report of near-misses. Information: site is not preserved and data are not gathered promptly. Investigation: analysis for root causes. Follow-up: recommendation often not tracked Once the critical controls to be monitored are determined, set a SPI against each one to show that system is operating as intended. 6. Define key OSH outcomes that need to be achieved within set timeframes The indicators themselves may take a number of different forms. First, they may measure a percentage increase or decrease in, for example safety audits completed with a perfect (or near perfect) score. Second, indicators may measure an absolute level or amount, for example, the hours of training per employee with an OSH content. Third, they may record steps in the implementation of an agreed programme by a specified date, for example, completion of phase 1 of the hazard reduction program by. The practical viability of a performance measurement system will often be determined simply by its relevance to the individual circumstances of the workplace.

12 SPIs can be set at 100% of actions must be completed on schedule. Alternatively, the organization may accept a degree of slippage before it is flagged up for attention of senior management or highlighted to the management team, in which case the tolerance should be set below 100%. 7. Establish SPIs data collection and reporting system To assist in the task of developing suitable SPIs, a worksheet comprising a matrix combining the categories of SPIs and the Quality Model for Process Improvement should be established. The aim of SPIs system is to indicate where process control systems have deteriorated or are not delivering the intended outcome. It is best to co-ordinate the performance data through an assigned person who will be responsible for collecting all the information, compiling reports for the management team and raising the alarm if there are any deviations from set tolerances. Any deviations from set tolerances or targets and important trends. Graphs or charts are the best way to present. Alternatively, other systems such as traffic lights, or signal numbers can be used to highlight the status of compliance. The senior management team should regularly receive SPIs information, and will need to make decisions on corrective action. 8. Periodic review The scope of the full set of SPIs needs to be reviewed to ensure they still reflect the main process risks. SPIs may need to be changed when: Introduction of new, high-risk processes; Improvement programmes; Alteration in plant design; Accidents/incidents occur. 5 Using SPIs to drive OHS activities 1. An organization when requested by the authority to demonstrate its ability to control hazards that causes the accidents/incident, will usually asked to show its process of safety management system can turn uncontrolled hazards to controlled risks. There are many variations of SMS in use, but all have the following principles: commitment and policy planning implementation measurement and evaluation, and review and improvement.

13 2. How to measure safety management system

14