Railway Safety Act Review Secretariat

Transcription

1 , Railway Safety Act Review Secretariat Performance Measurement in Railway Safety Final Report July 13, 2007 Page 1

2 Executive Summary QGI Consulting Ltd. was retained by the Railway Safety Act (RSA) Review Panel Secretariat to conduct research and analysis on the issue of safety performance measurement in the Canadian railway industry to support the ongoing review of Canada s Railway Safety Act. The principal objectives of this study are: Identification and analysis of current safety performance measurement activity in the Canadian railway sector; Provision of commentary and assessment on the sufficiency and adequacy of existing activities; Identification of gaps or deficiencies within existing safety measurement practices; and Identification of possible improvements resulting from improved or new measures In conducting our research QGI has relied on the review of publicly available literature and direct consultation with the principal stakeholders in the Canadian rail sector including the major railway companies, Transport Canada, the Transportation Safety Board, short lines and industry associations. Stakeholder interviews were conducted by telephone using a structured questionnaire to solicit information on current measurement activities and views on a number of key issues pertaining to measurement. The requirements of Canadian railways for safety reporting and safety performance measurement are in large part determined by regulation. Federally regulated railways are subject to the Railway Safety Act, the Railway Safety Management System Regulations, and the Transportation Safety Board Regulations that identify the specific railway occurrence reporting requirements and safety performance measurements for these railways. Railway safety oversight in Canada is the purview of Transport Canada s Rail Safety Directorate with the exception of investigations into railway occurrences, which are the responsibility of the Transportation Safety Board. Best Practices In order to provide a context for comparison to Canadian activities QGI has conducted research into best practices in safety performance measurement. Given the relative paucity of rail industry specific literature in this regard our research has focused on the issue with respect to health and safety management generally. Safety performance measurement is recognized to be but one element, albeit an important one, of an overall safety management system or process. The principal value of performance measurement in this area is to provide feedback that operational risks are being controlled and that management activities and compliance are in place to direct continual improvement in safety behaviour. Best practices theory identifies performance measurement in all three principal aspects of a safety management system as key to ensuring overall effectiveness. The three core areas of measurement are: measurement of risk, Page 1

3 measurement of management activities (e.g. compliance monitoring), and the measurement of outcomes (accidents and injuries). While safety performance measurement has traditionally focused on measuring outcomes and these continue to be the most prevalent form of measurement in the railway sector and other industries, best practices theory sees these measures as the least important of the three as they are reactive as opposed to proactive in nature. Effective measurement of health and safety performance cannot rely on any single measure of performance. Effectively gauging the safety performance of an organization or industry requires an appropriately selected basket of measures that reflect the breadth of an organization s operations and all aspects of the safety management process or system. Current Measurement Activity in Canada In Canada safety performance measures for the rail industry are developed, produced, and managed by three distinct stakeholder groups: by individual railway companies as required by regulation and as part of their internal safety management processes; by the Transportation Safety Board of Canada specifically with respect to railway safety occurrences in Canada; and by Transport Canada with respect to their own regulatory oversight activities Railway performance measurements are driven by regulation (Railway Safety Management System Regulations) and by the need to support their internal safety management processes. Regulation driven measures are limited in nature and focus on total reportable train accidents and personal injuries. Some measurement activity is also directed by the requirements of the SMS Regulations to establish annual safety targets and report railway performance against such targets on an annual basis. The regulations do not define the specific areas of a company s operations to be included in such targets but are rather left to the discretion of the individual railways. In addition to regulation based measures the railways, in varying degree, produce safety performance measures pertaining to track infrastructure, equipment (freight cars and locomotives) and people. The information provided to QGI by the railways, while giving insight into their measurement practices, does not enable us to comment on the extent, frequency, or adequacy of these activities. Furthermore it is unclear as to how and to what degree these performance measures are used by the railways in assessing the effectiveness of management practices and to guide the development and revision of safety policies and programs. Federally regulated railways in Canada report all railway accidents, incidents, and injuries on an individual occurrence basis to the Transportation Safety Board in accordance with the reportable accident and Page 2

4 incident criteria defined in the Transportation Safety Board Regulations. Based on the railway occurrence data it collects the TSB produces monthly and annual railway occurrence statistics reports that are published on its website and available to the public. TSB measures focus on railway accidents and incidents including absolute counts, frequency, causal factors, and differing types of occurrences. TSB also reports on fatalities and injuries related to federally regulated railway operations. The Rail Safety Directorate of Transport Canada, as the organization responsible for railway safety oversight in Canada, focuses its activities on monitoring of compliance to rules, regulations, and standards through the use of inspections and audits. More specifically Transport Canada produces measures of railway non-compliance to rules, regulations and standards based on inspection results, and measures of Transport Canada personnel activities. In addition to its own measures Transport Canada has access to the occurrence data collected by the Transportation Safety Board. Jurisdictional and Industry Comparisons For purposes of comparison to Canadian activities QGI has researched the type of safety performance measurement activities being done in other railway jurisdictions and other industries. In doing so we have reviewed safety performance reports and statistics published by government oversight agencies, individual companies, and industry associations. Principal areas of research include: the aviation industry; the petroleum refining and natural gas production industries, the marine industry, oil well drilling and contracting, and pipelines. For comparison of safety performance measurements within the railway industry specifically we have reviewed publications for the United States, the United Kingdom, and Australia. Generally speaking we conclude that the principal means of assessing safety performance in these areas is currently focused on the measurement of outcomes as the principal indicators of safety performance much as the Canadian railway industry is. Current measures or performance indicators focus on accidents, incidents, personal injuries, and fatalities measured in terms of both absolute number of occurrences and the frequency of occurrences normalized against industry specific measures of activity or workload. The measures used in these industries and jurisdictions differentiate themselves primarily by the breadth of the analysis related to safety occurrences or the number of dimensions or views that are provided with respect to occurrences. The Rail Safety and Standards Board (RSSB) in the United Kingdom produces the most extensive and sophisticated measurement activities of all the jurisdictions reviewed. Whereas other jurisdictions limit themselves largely to statistical reporting the RSSB differentiates itself significantly from the others by virtue of the level of analysis that it conducts with respect to safety performance. Notable aspects of RSSB measures include benchmarking of performance against other industries and across individual rail Page 3

5 companies, measurement of underlying risk across various aspects of railway operations, and linkage of safety performance measures to industry initiatives designed to improve industry performance in the long term. While the RSSB work is significantly more comprehensive than the statistical measures produced by the other rail jurisdictions and industries reviewed we would note that with the exception of measuring near miss information none of the measures can be considered measures of up stream activities or leading indicators of performance. While comprehensive their analysis is very much focused on outcomes or lagging indicators. Sufficiency and Adequacy of Existing Measures Determination of the sufficiency or adequacy of existing safety performance measurements is a subjective rather than objective assessment as it presumes the existence of a standard against which to measure the current activity. While best practices theory can provide a standard our discussion relies primarily on the views of the stakeholders consulted through the course of our research. Stakeholder views in this area were solicited on the following key issues: The sufficiency and adequacy of current measurement activities to effectively pursue the objective of the safest railway industry possible. Where gaps exist in current measurement practices. The perceived benefits of improved or additional measures. The perceived barriers to effective measurement today and that may impact changes or improvements in this area in the future. Generally speaking, with few exceptions, stakeholders believe that inadequacies exist in current safety measurement practices and that more can and should be done in this area. The views of the stakeholders do however diverge when the discussion turns to how existing measurement activity can be improved and by whom any additional measures should be developed. Many stakeholders believe that the safety performance statistics (measures) published by the Transportation Safety Board are flawed and do not provide an accurate representation of the safety performance of the Canadian railway industry. Key issues raised by stakeholders include the TSB railway occurrence reporting guidelines, the lack of consistency and rigor in data collection, the lack of a severity index, and the need to re-visit the workload measures used to normalize the data to produce frequency statistics. We would add to this the fact that TSB measures only reflect the performance of federally regulated railways excluding all provincially regulated companies and as such do not reflect the performance of the industry as a whole. Page 4

6 There is general consensus that Transport Canada, as the body responsible for railway safety oversight in Canada, is doing little that is meaningful in the area of performance measurement and should in fact do more. While there is not necessarily consensus among stakeholders with respect to the specific data that should be collected by Transport Canada or the measures that should flow from that data there does appear to be general agreement that Transport Canada should take on a more proactive role with significantly broader scope in the areas of safety performance measurement and reporting. During the course of consultations with both the railways and Transport Canada representatives of these organizations have said that Transport Canada should use the Railway Safety Act to develop appropriate regulations to enable the collection of safety performance data from the railways. The ability to collect such data in a systematic manner would provide the necessary inputs to allow Transport Canada to develop metrics that would both improve transparency of railway safety performance and provide Transport with the required knowledge to identify high risk areas to assist it in the planning and execution of its regulatory oversight activities. Most railways believe that, while they are continuously striving to improve their safety management practices including more comprehensive measurement activities, they are currently doing a sufficient level of safety performance measurement to allow them to effectively manage their safety programs. The industry is supportive of the co-regulatory framework in place in Canada that places the principal burden on individual companies to manage the safety of their operations and support the implementation of the Railway Safety Management System Regulations as an effective mechanism for ensuring such compliance. Transport Canada on the other hand believes that there does not exist sufficient transparency with respect to railway performance measurement to allow regulators to determine the safety of railway operations. There is general agreement among all stakeholders that the measurement of risk within rail operations is an area that requires additional development. Improvements, Barriers, and Benefits In formulating our analysis and discussion in this area, and selecting the specific areas identified for improvement, we have attempted to balance the views of stakeholders with our mandate for this research initiative to focus on issues that pertain to safety performance measurement. Key areas identified for improvement are summarized below: Page 5

7 Issue: TSB Performance Measures Accidents and Incidents Changes: Benefits: Barriers: Clarification of reporting guidelines and criteria for accidents and incidents Improvement in underlying data collection, management processes, and analytical capabilities Development of measures that more accurately reflect accident severity, railway switching, trespassing, and suicides Improved consistency and rigor associated with railway accident reporting Enhanced scope of measurement yielding a more comprehensive view of overall industry safety performance Organizational willingness and effectiveness within the TSB Resources technology, human, financial Issue: Safety Performance Measurement Activities of Transport Canada Changes: Development of appropriate and required regulations to enable Transport Canada to collect safety performance data from the railways to support more comprehensive performance measures. Improved data collection, management, and analytical capabilities (technology, process) Benefits: Improved visibility for regulators of railway safety performance Enabler for risk based business planning within Transport Canada Support for fact based development of regulations, rules, and standards Streamlined and consistent safety performance data collection processes and improved organizational effectiveness Barriers: Lack of current regulation and organizational willingness of Transport Canada to establish necessary regulations to support data collection. Resources technology, human, financial Issue: Measurement of Railway Risk Changes: Development of enhanced measurements targeted at quantifying risk Development of measures to gauge risk based management activities within railways Benefits: Improved visibility for regulators regarding railway risk management activities Enhanced capability of regulator to apply risk measures for prioritization of regulatory oversight activities Barriers: Absence of regulations to compel railways to submit the required data and measures to Transport Canada Resources technology, human, financial Page 6

8 The recommended areas for improvement in safety performance measurement in the Canadian railway sector encompass both improvements to existing measures and processes as well as the development of additional measurements and measurement capabilities. Based on QGI s review of current activities and the roles and capabilities of individual stakeholders we conclude that the safety performance measurement activities of the railways, Transport Canada, and the Transportation Safety Board should be improved and enhanced in order to improve the knowledge of all stakeholders regarding safety performance of the industry and to contribute to improved performance in the future. We would recommend the following initiatives to achieve such improvements. 1. Improvement and enhancement of existing safety performance measures dealing with railway safety occurrences produced by the TSB and the coordination of accountability for the future production of these measures with the activities of Transport Canada. Key issues that should be addressed in this regard include: review and potential enhancement of accident and incident reporting criteria, inclusion of provincially regulated railways within measurement activities; review of measures normalization factors (ie. train miles); and development of additional measures dealing with railway switching activities and trespasser incidents including the incidence of suicides. 2. Enhancement of the capabilities and involvement of Transport Canada in safety performance measurement activities. In pursuing this initiative consideration should be given to: clarifying the roles of Transport Canada and the TSB with respect to data collection and measurement production; identifying the required data reporting requirements to support measures production and developing the necessary regulations to establish railway data submission requirements; and completion of a thorough review of Transport Canada capabilities and needs to effectively carry out the proposed measurement activities. 3. Development of new measures to better identify the overall level of risk associated with railway operations in total and to facilitate the identification of high-risk operations to guide the safety management activities of the railways and the regulator. 4. Transport Canada taking a leadership role in a collaborative process involving the railways and all other appropriate federal and provincial stakeholders to develop new and improved measures and the required supporting processes and interfaces. Page 7