Leading Indicators: INDUSTRY TRENDS AND PREDICTING TOMORROW. Zach Hansmann, M.S., CSP, CHMM The Hansmann Group

Transcription

1 Leading Indicators: INDUSTRY TRENDS AND PREDICTING TOMORROW The Hansmann Group

2 Outline Lagging Indicators Weather Accident Theory Leading Indicators Safety, Phenomena, Indicators, and Metrics Predictive Validity Software Health and Dieting

3 How do you evaluate safety performance? Lagging indicators are measurements that relate to loss events or developments in the past Image source: Scott Adams (1998). Dilbert United Feature Syndicate, Inc.

4 Lagging Indicators Total recordable incident rate (TRIR) Occupational Safety and Health Administration (OSHA) TRIR = Number of OSHA Recordable Incidents x 200,000 Total Number of Hours Worked (100 employees working 2,000 h per year) and dividing this value by the total work hours actually worked in the year 2 x 200, x 200,000 Ex. A: 120,000 = 3.33 Ex. B: 2,160,000 = 3.33 This formula is used to calculate many incident rates

5 BLS Data: 2015 Numbers from Michigan

6 BLS Data: 2015 Numbers from Michigan (1) Incidence rates represent the number of injuries and illnesses per 10,000 full-time workers and were calculated as: (N / EH) X 20,000,000 where, N = number of injuries and illnesses, EH = total hours worked by all employees during the calendar year, 20,000,000 = base for 10,000 full-time equivalent workers (working 40 hours per week, 50 weeks per year). So, = 140.6, which equals 1.406

7 BLS Data: 2015 Numbers from Michigan

8 BLS Data: 2015 Numbers from Michigan

9 BLS Data: 2015 Numbers from Michigan

10 BLS Data: 2015 Numbers from Michigan

11 Experience Modification Rating (EMR) Think of EMR as: EMR = Actual Claims Expected Claims It s Actually: A = Actually Primary Losses B = Ballast Value C = Expected Losses D = Expected Primary Losses E = Actual Excess Losses (T-A) F = Expected Excess Losses (C-D) T = Actual Incurred Losses W = Weighting Value EMR = A + B + (E x W) + (F x (1 W)) D + B + F

12 Experience Modification Rating (EMR) Definition: The ratio of actual losses to expected losses over a rolling three-year period average Definition: A measure of worker s compensation loss experience using a baseline of 1.00 Calculating agencies use claim data from three years ending one year prior. A 2017 policy will consider 2013, 2014, and 2015 So what s a good EMR? Excellent: Good: Average to At Risk: At Risk to Adverse:

13 Experience Modification Rating (EMR) Example: Two subcontractors with very different worker s comp history Direct Labor Cost = $10,000,000 Manual Insurance Rate = $13.67 (per $100 dollars of payroll) Initial Premium = $13.67 x $10,000,00 = $1,367, $100 Company A (Modifier of 1.25) pays a modified premium of: $1,367, x 1.25 = $1,708, Company B (Modifier of 0.75) pays a modified premium of: $1,367, x 0.75 = $1,025,250.00

14 Other Lagging Indicators Fatality or other accidents Severity rate Number of lost days experienced as compared to the number of incidents experienced Lost workday rate Chemical releases OSHA citations (number of citations and type) Workers' compensation loss ratios Number of liability claims associated with worksite injuries

15 Lagging Indicator Challenges A growing number of safety professionals question the value of lagging indicators and argue that lagging indicators do not provide enough information or insight to effectively avoid future accidents (Grabowski et al., 2007). An unbalanced focus on lagging after-the-fact based measures may convey an unintended message that safety prevention is less important (Mengolinim and Debarberis, 2008). Organizations that diligently report and investigate accidents are disadvantaged in comparison to careless organizations that do not always report accident occurrences (Votano, 2014). EMR poses some problems in its application because its formulae are quite complex and different versions of calculation exist in practice. Such data may be insufficiently sensitive, of dubious accuracy, purely retrospective, or ignore risk exposure. (Mohamed, 2003).

16 But what about the weather? Image source: The Weather Channel, National Weather Service

17 Leading Indicators Defined Leading Indicators are: Proactive, preventive, and predictive measures that monitor and provide current information about the effective performance, activities and processes of an EHS management system that drive the identification and elimination or control of risks in the workplace that can lead to incidents and injuries.

18 Leading Indicators Defined Leading Indicators (Continued): Leading indicators measure the events leading up to injuries and fatalities, and also provide information about the current state of an organization s safety management system. Specifically, leading indicators are designed to give advanced warning of potential problems so that preventive actions can be taken. Additionally, they help to reveal weaknesses in an organization s procedures or employee behavior before they have a chance to cause real harm.

19 H.W. Heinrich Industrial Accident Prevention: A Scientific Approach 1 Major Injury 29 Minor Injury 300 Non-Injury Incident

20 H.W. Heinrich Industrial Accident Prevention: A Scientific Approach Electrician receives fatal shock Electrician falls from ladder due to jerk reaction from shock Electrician receives minor burn from electric shock Electrician receives minor shock while connecting light fixture Electrician connects light fixture with circuit energized 1 Fatal 30 Lost Time Injuries 300 Recordable Injuries 30,000 Near Miss/First Aid 300,000 Hazards Unsafe Acts/At-Risk Behaviors Source: Floyd et al. (2000). How can we better learn from electrical accidents.

21 H.W. Heinrich Industrial Accident Prevention: A Scientific Approach

22 2-Factor Model from Reason (1990): Human Error Image source: Hallowell, M. R. (2008). A formal model of construction safety and health risk management

23 Systems Model of Construction Accident Causation Source: Mitropoulos (2005). Systems model of construction accident causation.

24 Results with Leading Indicators A Fortune 150 manufacturer reduced its lost work day rate by 97% within one year A Fortune 150 energy company reduced its incident rate by 67% within 18 months A top 20 ENR contractor achieved significant safety improvements including 90% of worksites experiencing no losttime incidents Percent of worker observations that were safe (A) Number of positive reinforcements provided per 200,000 h (B) A top 5 public U.S. university saved over $20 million in insurance fees across four years of building projects Source: Hinze (2013). Leading indicators of construction safety performance.

25 Characteristics of Good Leading Indicators Leading Indicators should be: Actionable Achievable Meaningful Transparent Easy to Communicate Valid Useful Timely

26 Typologies of Indicators Leading versus lagging indicators Outcome versus activity based indicators Input versus output indicators Process versus personnel indicators Positive versus negative indicators Technical versus human factors indicators

27 Passive vs. Active Leading Indicators Examples of passive leading indicators include the following: Number or percent of management personnel with 10-h (or 30-h) OSHA outreach cards. Number or percent of field employees with 10-h (or 30-h) OSHA outreach cards. Number or percent of subcontractors selected, in part, on the basis of satisfying specific safety criterion prior to being awarded the subcontract. Requirement that each subcontractor submit a site-specific safety program that must be approved prior to the performance of any work by that subcontractor. Requirement that the CEO of each subcontractor provide a letter indicating the subcontractor s commitment to construction worker safety. Source: Hinze et al. (2013). Leading indicators of construction safety performance

28 Passive vs. Active Leading Indicators Examples of active leading indicators include the following: Percent of jobsite toolbox meetings attended by jobsite supervisors/managers. Percent of jobsite pre-task planning meetings attended by jobsite supervisors/managers. Percent of negative test results on random drug tests. Number of close calls reported per 200,000 h of worker exposure. Percent of safety compliance on jobsite safety audits (inspections). Aggressive owner promotion of jobsite safety. Worker observation records are evaluated to determine the need for jobsite changes in the job safety program. Source: Hinze et al. (2013). Leading indicators of construction safety performance

29 Indicators vs. Metrics? Phenomena What is the indicator a measure of Indicator What is being measured Metrics How something is measured % of Meetings Attended Management Involvement Engagement Source: Reiman, et al. (2010). Indicators of safety culture selection and utilization of leading safety performance indicators.

30 Leading Indicators: Type One Operations-Based Leading Indicators: Indicators that are relevant to the functioning of an organization s infrastructure (e.g. machinery, operations); potentially site-specific

31 Operations-Based Leading Indicator Categories Compliance Risk assessment Preventive and corrective actions Equipment and preventive maintenance Prevention through design Training Management of change process

32 Leading Indicators: Type Two Systems-Based Leading Indicators: Indicators that relate more to the management of an EHS system; can be rolled up from a facility level to a region/business unit or corporate level

33 Systems-Based Leading Indicator Categories Hazard identification and recognition Leading indicator component evaluation Learning system Permit-to-work system Safety perception survey Communication of safety Recognition, disciplinary and reinforcement system Hazard analysis EHS system component evaluation Risk assessment Preventive and corrective actions

34 Leading Indicators: Type Three Behavior-Based Leading Indicators Indicators that measure the: behavior or actions of individuals or groups in the workplace; people-to-people interactions related to supervision and management; useful at site-specific level through management level

35 Behavior-Based Leading Indicator Categories Leadership engagement Employee engagement and participation At-risk behaviors and safe behaviors Area observations and walk-arounds Off-the-job safety

36 Combined Leading Indicator Metrics Risk assessment (S,O) Identification of the tasks, hazards, and risks of a job prior to work, and the implementation of protective measures to ensure work is done safely. Number of assessments conducted per plan Percent of assessments completed per plan Ratio between the levels of risk identified (high, medium, low) Scoring the steps of an operation on severity, exposure, and probability Number of assessments communicated Number of risks mitigated or controlled Number of assessments validated by EHS manager Percent of assessments reevaluated and revalidated Percent of routine tasks identified Percent of tasks identified Percent of risk assessments completed per schedule/plan Number of assessments to evaluate potential severity

37 Combined Leading Indicator Metrics Hazard identification/recognition (S) Evaluations and assessments (not necessarily audits) through management and employee observations to identify potential hazards. Number of near miss reports Number of unsafe observations (conditions or behaviors) Number of safe observations (conditions or behaviors) Number of unsafe observations per inspection Number of unsafe observations reported per employee per time period Number and percent of previously unknown or uncategorized hazards discovered Inspection count (collection of observations) Ratio of safe to unsafe observations Weighted percent safe observations (using risk matrix) Frequency of 100% safe Number of checklists filled out

38 Predictive Validity of Safety Leading Indicators: Risk Mitigation Measures with Predictive Power of TRIR and SR Near miss reporting Safety observation Stop work authority (SWA) Client audit Job safety analysis (JSA) development Job safety analysis (JSA) engagement Contractor PM engagement and walkthroughs Contractor safety rep engagement and walkthroughs Client safety walkthroughs Contractor safety audits Subcontractor safety audits Project orientation sessions Client participation in project orientation sessions Corrective action items Risk driven action items Project risk assessment Risk driven project risk management plan Daily tasks orientation Client participation in daily tasks orientation Contractor post-job evaluation score Source: Hallowell (2016). Predictive validity of safety leading indicators.

39 Predictive Validity of Safety Leading Indicators: Risk Mitigation Measures with Predictive Power of TRIR and SR Metric variables were calculated as rates using typical OSHA rate formulas. A few examples include: Near miss reporting Number of near misses generated in the year by 200,000 workerhours exposure and dividing by the total work hours worked in the year Safety observation Number of safety observations conducted in the year by 200,000 worker-hours exposure and dividing by the total work hours worked in the year Corrective action items Number of action items addressing near misses, incidents, safety observations, SWA and audits in the year by 200,000 worker-hours exposure and dividing by the total work hours worked in the year Source: Hallowell (2016). Predictive validity of safety leading indicators.

40 Predictive Validity of Safety Leading Indicators: Risk Mitigation Measures with Predictive Power of TRIR and SR Work-in-progress risk management Near miss reporting Job safety analysis (JSA) development Job safety analysis (JSA) engagement Contractor safety audits Workforce engagements and monitoring Contractor PM engagement and walkthroughs Contractor safety rep engagement and walkthroughs Client safety walkthroughs Non-routine safety actions Stop work authority Subcontractor safety audits Corrective action items Source: Hallowell (2016). Predictive validity of safety leading indicators.

41 Management Support and Accountability Leading Indicator Metrics % of goals/objectives incorporating safety % of jobs preplanned Average # of corrective actions per submission (incidents, near misses, observation, inspections) Lagging Indicator Metrics % of projects that work without incidents Documented meetings, metrics used compared to plan (+/-) Preplan verified and onsite Participation in safety meetings, budgets for safety, safety metrics communicated Source: Underwriter Laboratories (2013). Using Leading and Lagging Safety Indicators to Manage Workplace Health and Safety Risk

42 Employee Participation and Involvement Leading Indicators Metrics % of employees involved in safety decision making process % of workforce submitting safe and/or at risk behaviors weekly Lagging Indicators Metrics # of work method changes Average time to implement suggestions and/or corrective action Tracking % increase (or decrease) in the # of submissions being submitted by the workforce Source: Underwriter Laboratories (2013). Using Leading and Lagging Safety Indicators to Manage Workplace Health and Safety Risk

43 Performance Management Systems/Safety Related Leading Indicator Metrics Lagging Indicator Metrics % of performance reviews measuring success in achieving results # inspections compared to individual objective # of safety meetings conducted compared to individual objective # of one-on-one contacts % of losses tied to projects and individual objectives Near misses Incidence rate (frequency and severity) Loss costs %-age of overall rating related to safety performance/metrics Project profitability Source: Underwriter Laboratories (2013). Using Leading and Lagging Safety Indicators to Manage Workplace Health and Safety Risk

44 Moving Forward: Safety Culture and Safety Climate? Shared employee perceptions of how safety management is being operationalized in the workplace, at a particular moment in time (Cooper and Phillips 2004). The term safety culture first made its appearance in the 1987 OECD Nuclear Agency report on the 1986 Chernobyl disaster. Benefits of monitoring safety culture include (Votano, 2014): By measuring different safety climate dimensions, construction organizations have a mechanism to identify problematic areas and optimize investment on safety-related improvements. It is a valuable tool to identify trends in safety performance and to establish both internal and external benchmarks. It is economical and easy to administer. Safety climate survey involves all employees in the process, which helps identify key issues throughout the organization

45 Metrics of Safety Climate or Culture Top management considers safety as equally important as production and profits Top management acts decisively when a safety concern or problem is raised or has occurred Top management evaluates (gives rewards/discipline) employees safety performance Site safety personnel have sufficient power and authority Top management requires each manager/department to improve/maintain safety performance My supervisor or line manager follows safety procedures in every situation, e.g., during deadline, behind schedule, planning stage My supervisor or line manager is committed and shows interest toward safety My supervisor or line manager considers my safety performance My construction project site is a safe place to work Supportive working relationships exist in the project when it comes to safety Source: Votano (2014). Client Safety roles in small and medium construction projects in Australia

46 Measurable Indicators of Safety Climate or Culture The company encourages and acts upon feedback from employees on safety issues The company frequently holds safety campaigns or safety awareness programs Safety rules and procedures are enforced in the project I have received enough training to perform my job safely My safety training provides sufficient knowledge to identify potential safety risks and hazards Safety is one of my priorities when I do my job I am involved to improve safety performance in the project Company s safety policy, information, and issues are available to everyone involved The safety rules and procedures in the project/company are practical, realistic, and appropriate It is easy to access safety rules, procedures, and information when required Source: Votano (2014). Client Safety roles in small and medium construction projects in Australia

47 Leading Indicator Programs Predictive Solutions EHS Insight UL Dashboard System Simple Observation Based System

48 Predictive Solutions

49 Predictive Solutions

50 EHS Insight Dashboard

51 EHS Insight Reporting

52 EHS Insight Risk Register

53 Source: Underwriter Laboratories (2013). Using Leading and Lagging Safety Indicators to Manage Workplace Health and Safety Risk

54 Source: Underwriter Laboratories (2013). Using Leading and Lagging Safety Indicators to Manage Workplace Health and Safety Risk

55 Simple Observation Based System Observations are made on the site: Personal Protective Equipment (PPE) Actions of the workers Site conditions and housekeeping Acceptable and unacceptable observations are recorded and tracked Trends are identified, both in individual compliance areas and overall site performance

56 Simple Observation Based System Observations include situations where a worker either does not have the appropriate PPE on or is not correctly wearing it. Examples include: Hardhats worn over baseball caps Safety glasses worn on hardhats or around the neck Worn, damaged or dirty high-visibility vests

57 Simple Observation Based System Actions include anything the worker does that is not within the expectations of the company (applicable regulations, health and safety plan, etc). Examples include Working too far away from a fall protection attachment, creating a swing fall hazard Carrying materials up or down a ladder instead of using a rope or other lifting method

58 Simple Observation Based System Condition observations are physical working areas, tools, and other systems that are under contractor control: Examples of unacceptable observations include Missing or insufficient rails, cables or covers for openings and holes Incorrectly positioned or constructed ladders and scaffold systems Damaged cords, plugs and ground fault circuit interrupter (GFCI) devices

59 Simple Observation Based System

60 Simple Observation Based System

61 Simple Observations System Trend Lines Over Time 5 Weeks

62 Simple Observations System Trend Lines Over Time 5 Weeks

63 Challenges with Leading Indicators Garbage in, Garbage out Time consuming Revisiting Heinrich Subjectivity and variability between observers Weighting of observations risk vs. severity Managing the measure rather than managing safety (Hopkins, 2009) Image Source: Silver (2013). Making the case for predictive analytics in workplace safety (Predictive Solutions)

64 Challenges with Leading Indicators The indicator can be valid indicator of the phenomenon, but the phenomenon does not bear a relationship to safety. The indicator can be a valid indicator of the underlying phenomenon, but the effect of the phenomenon on system safety is clouded by the effect of other relevant phenomena.. The indicator as such can be a valid indicator of the underlying phenomenon, but the operationalization (metric) of the indicator is such that the measure has a high degree of error (calibration error, hesitancy in reporting, optimizing the score instead of the attending the phenomenon etc). The phenomenon in question cannot be accurate measured by one indicator, rather multiple indicators are needed. Source: Reiman, et al. (2010). Indicators of safety culture selection and utilization of leading safety performance indicators.

65 To Recap: Lagging Indicators: What has already happened? Incident rates Severity ratio Workers comp payments or loss ratios Leading Indicators: What may happen in the future? Operations-based Systems-based Behavior-based Safety Climate or Culture: How is safety perceived? Specialized leading indicators Evaluate safety performance via attitudes and opinion of the workers Many systems are available, but they may not be necessary Challenges with leading indicators can make it difficult to start

66 Have You Ever Dieted Before? Say you want to monitor and improve your health What would the Lagging Metric be? Weight Number of health related incidents Prescription drug costs What would a Leading Metric be? Calories consumed Minutes of exercise Percent of doctor visits completed What would a healthy culture look like? Support groups for exercise (Crossfit) Walking group routines Weight watchers

67 At the End of the Day... What Does a State-of-the-Art Indicator System Look Like? Source: Reiman, et al. (2010). Indicators of safety culture selection and utilization of leading safety performance indicators.

68 Key Takeaways for Leading Indicators from the Campbell Institute Look at what is already being measured; could it be a leading indicator? Make sure indicators communicate meaningful and actionable information Integrate leading indicators into the overall safety management system Just get started; don t spend too much time deliberating Obtain leadership support

69 The Campbell Institute and the National Safety Council For these documents visit the Campbell Institute at:

70 References Church, R. (2011). Experience Modification Rating: An accurate measure of safety? Aon Risk Solutions. Floyd, H.L., Eastwood, K., and Liggett, D.P. (2000). How can we better learn from electrical accidents. Institute of Electrical and Electronics Engineers, Paper No. PCIC Hallowell, M. R. (2008). A formal model of construction safety and health risk management (Doctoral dissertation). Heinrich, H. W. (1941). Industrial Accident Prevention. A Scientific Approach. Industrial Accident Prevention. A Scientific Approach., (Second Edition). Hinze, J., Thurman, S., & Wehle, A. (2013). Leading indicators of construction safety performance. Safety Science, 51(1), Hopkins, A. (2009). Thinking about process safety indicators. Safety Science, 47, Mitropoulos, P., Abdelhamid, T. S., & Howell, G. A. (2005). Systems model of construction accident causation. Journal of Construction Engineering and Management, 131(7), Reiman, T., Pietikainen, E. (2010). Indicators of safety culture selection and utilization of leading safety performance indicators. Swedish Radiation Safety Authority. Salas, R., & Hallowell, M. (2016). Predictive Validity of Safety Leading Indicators: Empirical Assessment in the Oil and Gas Sector. Journal of Construction Engineering and Management, 142(10), Silver, N., Beane, B., Trebek, A. (2013). Making the case for predictive analytics in workplace safety (Predictive Solutions white paper) Underwriter Laboratories (2013) Using Leading and Lagging Safety Indicators to Manage Workplace Health and Safety Risk Votano, S., & Sunindijo, R. Y. (2014). Client safety roles in small and medium construction projects in Australia. Journal of Construction Engineering and Management, 140(9),

71 Thank You! Principal, The Hansmann Group Tel. (517) Web:

72 About the Speaker Zach Hansmann is the Principal at The Hansmann Group and the Contractor Safety Manager at Michigan State University. Zach maintains the Certified Safety Professional (CSP) and Certified Hazardous Materials Manager (CHMM) designations, is on the Board of Directors and the Construction Safety Division of the Michigan Safety Conference, and is an authorized OSHA outreach instructor. He holds degrees from Michigan State University in Human Biology (B.S.) and Construction Management (M.S.), and is currently a Ph.D. student in Planning, Design, and Construction, studying the owner s involvement in construction safety.