March Workplace Safety and Insurance Board Workwell Program 2011 Value for Money Audit

Transcription

1 March 2012 Workplace Safety and Insurance Board Workwell Program 2011 Value for Money Audit

2 Table of contents Workwell program current profile....3 Project Objectives and Scope Basis of Value for Money Audit Opinion Bill 160 Alignment Findings...9 Summary Opinion Audit Opinion Findings Auditors Report.20 Recommendation for Improvement & Management Response Considerations for Program Redesign..23 Audit Opinion Supporting Analysis Bill 160 Alignment Detailed Findings Value for Money Opinion Detailed Findings Interjurisdictional Research Internal Consultations/Documentation Review Appendix WSIB - Workwell Program Value for Money Audit

3 Workwell program current profile Workwell was established as a pilot program in December 1987 as a policy designed to promote health and safety in Ontario workplaces by providing additional assessments to be levied on employers who are not maintaining safe and healthy working environments. The Board s current Operational Policy states that the Workwell program issues an additional premium charge to employers who have not taken sufficient precautions to prevent accidents in the workplace. Workwell operates independently of the Workplace Safety & Insurance Board s (WSIB) experience rating programs and its additional premium charges are issued in addition to any premium or premium rate adjustments that may be realized through experience rating. Employers are considered for the Workwell program based on mature WSIB data (e.g., injury costs, frequency information) input into the Firm Selection Model and Workwell Selection criteria. Referrals also come from Ministry of Labour orders, complaints from workers and other sources. Firms are notified that they have been identified for an audit through a WSIB letter and the phase 1 audit is scheduled by the Workwell Evaluator. A score of 75% is required to pass the audit. Firms that do not pass phase 1 have approximately six (6) months to improve their health and safety system prior to the phase 2 audit. Firms that do not pass phase 2 are subject to a surcharge of up to $500,000 based upon their audit score and annual premium amount. Employers may choose to use internal resources, external consultants or the Health and Safety Associations to assist in their Workwell preparation and implementation efforts. 3 WSIB - Workwell Program Value for Money Audit Workwell program overview Workwell Evaluators closed 964 cases (i.e., Audits or Risk Management Plans) for the 12 month period ended December 2011 There are presently 22 Workwell Evaluators covering the province Workwell Evaluators perform Core Health and Safety Audits, Small Business Health and Safety Audits and Risk Management Plans (RMP) The 2011 (fully staffed) program budget was $4.028 million The 2011 surcharge levied was approximately $1.1 million Approximately 80% of firms do not pass the phase 1 audit Phase 2 audits are successfully completed by approximately 80% of firms

4 The changing landscape of Ontario s health and safety system The health and safety system in the Province of Ontario is currently in a state of transition as a result of the Tony Dean Report, introduction of Bill 160 and the forthcoming WSIB Funding Review and recommendations. Bill 160 actioned recommendations from the Tony Dean Report and enabled the Ministry of Labour to appoint the province s first Chief Prevention Officer, effective October Based on Ministry of Labour communications (MoL Backgrounder Ontario s First Chief Prevention Officer To Spearhead Improvements To Worker Health And Safety, December 16, 2011) the Chief Prevention Officer will be responsible for: Establishing a provincial occupational health and safety strategy; Promoting the alignment of prevention activities across all workplace health and safety system partners; Advising on proposed changes for the funding and delivery of prevention services; and Working with Ontario's Health and Safety Associations (HSAs) to establish effective delivery of prevention programs and services. Presently the Workplace Safety and Insurance Board (WSIB) provides funding to the Ministry of Labour ($95 million) to administer the Occupational Health and Safety Act and to the HSAs ($91 million including mine rescue) to provide services and resources to the province s employers. The WSIB Funding Review has been gathering information and input from stakeholders across the province over the past year to assess specific issues related to WSIB s financial situation. As part of the review employer incentives were evaluated through the following analysis: Is the present design and operation of these programs appropriate? What alternatives exist to promote increased safety in the workplace, fairness in insurance costs to employers, and incentives to employ injured workers? The Funding Review report and recommendations have not yet been made public. 4 WSIB - Workwell Program Value for Money Audit

5 Implications to Workwell from the changing landscape The essence of Bill 160 is that WSIB s previous prevention mandate will be transferred to the Ministry of Labour under the Chief Prevention Officer. WSIB is currently working with the Chief Prevention Officer to identify the existing prevention focused responsibilities within WSIB that could be allocated to the newly created organization. At the same time, the Chief Prevention Officer is beginning to develop the elements of a provincial occupational health and safety strategy that ultimately, amongst many other things, will define the scope and delivery of existing and new prevention programs. Independent of the strategy developed by the Chief Prevention Officer, WSIB needs to look at Workwell and assess whether the program in its current form has a role to play within the scope of WSIB s future operations, or whether there are modifications to the current program that would better align to emerging strategic directions. The Value for Money Audit (VfMA) is an annual regulatory requirement to assess the efficiency and effectiveness of a WSIB program delivered under the Act. In 2011 the Workwell program was selected for the VfMA. Stakeholders were interested in considering where Workwell should be positioned in the health and safety system given the shift of prevention activities. The findings from this project will help to inform future decisions on the scope, nature and service delivery mechanism for Workwell. 5 WSIB - Workwell Program Value for Money Audit

6 Project objectives and scope To meet legislative requirements, the Workplace Safety and Insurance Board is required to undertake an annual Value for Money Audit of at least one program delivered under the Act. Following a formal Request for Proposals and vendor selection process, Deloitte was engaged to conduct the VfMA. The project objectives and scope as referenced in the Request for Proposal and confirmed with management through input from the VfMA Steering Committee are outlined below. Objectives Scope Provide an opinion as to whether there is value for money in the outcomes generated by the Workwell program and recommend improvements that could be made. Include the three key elements of effectiveness, efficiency and cost of the Workwell program in the value for money opinion. Identify program improvements/opportunities that will allow efficient and effective alignment of the Workwell program with the new prevention mandate (Bill 160) as it develops. Identify opportunities for efficient and effective alignment of the Workwell program within the WSIB s Insurance System. Assist WSIB management in the formulation of a response to the recommendations of the Audit which are to be included in the final report. The cost, efficiency and effectiveness of the administration of the Workwell program will be assessed including roles and responsibilities, administrative controls, monitoring and reporting processes, use of information technology and training and development activities. The audit will assess the Workwell program from the Firm Selection Model process through to the completion of a Workwell case. Data will be provided by WSIB to assist in the evaluation of the program s effectiveness. Workwell selected firms from the 2006 through to 2010 cohort years will be assessed. 6 WSIB - Workwell Program Value for Money Audit

7 Basis of Value for Money Audit Opinion The value for money audit opinion for the Workwell program is based on a pre-determined guiding principle in combination with program specific criteria developed for each of the core value for money components systems, resources and outcomes. Opinion Guiding Principle The scope of the audit will focus on the efficiency, effectiveness and cost of Workwell and the program s alignment in the WSIB s Insurance System. Audit Criteria Systems Resources Statutory Definition The adequacy of management systems (including performance standards and measurement), controls and practices, including those intended to control and safeguard assets, and to ensure due regard to cost, efficiency and effectiveness. The extent to which resources have been managed in conducting relevant activities with due regard to cost and efficiency. Workwell Program Key Audit Criteria Attributes Roles, responsibilities, accountabilities, and performance expectations are defined, understood, and consistently executed by management and staff. Appropriate administrative controls and safeguards are in place to mitigate program risks. A tracking and reporting process is in place to monitor program performance; and, performance metrics are monitored and acted upon. The information technology systems in place to support the program have the desired functionality to deliver and monitor the program. Policies and processes supporting the program are documented, communicated, and understood by management and staff. Program resources are managed efficiently to meet program demands. Policies and processes are executed in a timely manner. Identified challenges and issues are addressed in a timely fashion and in accordance with the established escalation and resolution process. Adequate training and development programs are in place for management and staff to ensure succession planning and knowledge retention. Outcomes The extent to which programs, operations or activities of an entity have been effective. 7 WSIB - Workwell Program Value for Money Audit The program has clearly defined objectives that are aligned with organizational strategies, and there are measures in place that demonstrate satisfactory progress towards those objectives. The program contributes to a sustainable (evidenced by WSIB reporting metrics in the years subsequent to a Workwell interaction) reduction in injuries and illnesses in the Province's workplaces. The program meets the needs of workplace parties in the Province of Ontario.

8 Rating categories Using input from documentation review, focus groups, interviews and data analysis, we applied the audit framework to determine the extent to which the Workwell program is achieving Value for Money for the WSIB. Bill 160 Rating Categories Audit Criteria No Improvements Required Systems Alignment Some Improvements Required Significant Improvements Required Resources Critical Outcomes Please note that for reporting purposes, the alignment of Workwell to the changes proposed under Bill 160 has been considered as separate and distinct to the statutory Value for Money Audit criteria, the assessment of which forms the basis of the audit opinion. However, the assessment of the alignment question is presented first in our report since the answer shapes the context for the Value for Money Audit findings and the future opportunities for consideration in developing and implementing health and safety prevention programs. 8 WSIB - Workwell Program Value for Money Audit

9 Bill 160 alignment findings WSIB s new vision aspires the organization to be the leading workplace compensation board. To accomplish this vision the Board s mandate is: To promote health and safety in the workplace Facilitate return to work Provide compensation where required. Bill 160 s intent is to promote, educate and foster commitment to health and safety for employees and employers within the Province s workplaces (Section 4.1 (2)). Specifically the objectives of the Prevention mandate are: To promote occupational health and safety and to promote the prevention of workplace injuries and occupational diseases To promote public awareness of occupational health and safety To educate employers, workers and other persons about occupational health and safety To foster a commitment to occupational health and safety among employers, workers and others The implication of Bill 160 is that WSIB s previous prevention mandate was transferred to the Ministry of Labour under the Chief Prevention Officer effective April 1, While our review indicates that there are a number of facets to the Workwell program including education, creating awareness, monitoring compliance and enforcement activities, all of which can be construed as promoting health and safety in the workplace, we conclude that the program is fundamentally an injury prevention program. Further, as we identify later in our report, there are a number of elements within Workwell that require significant improvement to make the program, or its elements, a more effective contributor to the Province s Health and Safety system. 9 WSIB - Workwell Program Value for Money Audit

10 Bill 160 alignment findings (cont d) Therefore, based on our review of the Workwell program and an assessment of the implications of Bill 160, we conclude that the Workwell program, in its current form, no longer aligns to the WSIB s new strategic direction. There is an opportunity to perform the education, implementation and monitoring activities for employer s health and safety systems on a more integrated basis across the system partners. WSIB needs to work with the Chief Prevention Officer (CPO) to support the CPO in determining the elements of the current Workwell program that could be retained in a new Prevention system, defining the nature and scope of the future offering, and identifying the system partner(s) best qualified to deliver the activities within that offering. 10 WSIB - Workwell Program Value for Money Audit

11 Summary Opinion The scope of the audit assessed the cost, efficiency and effectiveness of the Workwell program from the Firm Selection Model process through to the completion of a Workwell case. We present the following summary conclusions: Systems: Review of current program management systems reveals some controls and practices are in place to safeguard assets with due regard to cost, efficiency and effectiveness. There are opportunities to: Clarify the roles and responsibilities of Workwell Evaluators, particularly with reference to the program mandate (e.g., compliance vs. consultative, incentive vs. punitive). Improve the reporting on the Health & Safety System -Target & Performance Dashboard to include outcome indicators. Improve the use of technology that support Workwell Evaluators. Resources: Program resources are being managed to conduct relevant activities with due regard to cost and efficiency. There is some room for improvement to: Focus attention on the Firm Selection Model and the need to use leading indicators in the model. Perform the Workwell intervention in a more timely manner after the incident which led to the firm being identified for audit. Train and deploy resources based on industry expertise. (cont d..) 11 WSIB - Workwell Program Value for Money Audit

12 Summary Opinion (cont d) Outcomes: An analysis of LTI outcomes has not provided evidence that the program has a unique impact on declines in LTI rates since evidence is mixed and there are many other environmental factors that impact LTI. Although LTI rates do decline after Workwell interventions, there is limited quantitative evidence that declines are due to the audit and the on-going trending of firms undergoing intervention is no different than overall declines in LTI across the Province. There is evidence that some workplace interventions (i.e., within Workwell and considering other programs such as Safety Groups and SCIP) may have more impact than others notwithstanding opportunities to improve clarity around the mandate and execution of the program. Key improvement opportunities include: Clarify the mandate of the program and align the appropriate program elements and activities to the WSIB and health and safety system partners according to the new strategic direction advanced by the introduction of Bill 160. Track and report metrics to evidence Workwell s ongoing success (i.e., fewer injuries and illnesses, reduced risk of injury). Overall: While elements of the Workwell program do provide some value (e.g., notifications to employers and Risk Management Plans) the Workwell program as a whole has not clearly delivered value for money for the Workplace Safety & Insurance Board. 12 WSIB - Workwell Program Value for Money Audit

13 Audit Opinion findings - Systems The following table shows our summary audit findings for the Systems component of the Value for Money Audit Framework as it relates to the Workwell program. Audit Criteria Key Audit Criteria Attributes Observations Rating Systems Roles, responsibilities, accountabilities, and performance expectations are defined, understood, and consistently executed by management and staff. Workwell Evaluator Job description notes that they are to be providing collaborative risk-based consultative services to customers. There is confusion internally and externally regarding their role. The Workload Objective to ensure a minimum of sixty (60) evaluations are completed on a rolling twelve (12) month basis is not being consistently met. Some Improvement Required Resources Appropriate administrative controls and safeguards are in place to mitigate program risks. Efforts to align the Workwell Evaluator Evaluation Process and ISO 19011:2002 standards are partially completed. Further effort is required to ensure the standard is accurately applied to the Workwell process. The Workwell Findings Committee has not met regularly during Concerns were noted by stakeholders that the Workwell Evaluators may not have sufficient and appropriate background to understand the health and safety issues for all of the types of businesses encountered by Workwell. The Workwell program System Collaboration Branch Survey of Employers Undergoing First Workwell Audit only looked at Phase 1 of the process. Some Improvement Required Outcomes A tracking and reporting process is in place to monitor program performance, and performance metrics are monitored and acted upon. The Health & Safety System -Target & Performance Dashboard report is point in time and does not include metrics to demonstrate sustainability of improved performance for employers involved in the Workwell program. Financial metrics for surcharges generated by Workwell audits are not tracked and reported. Some Improvement Required LEGEND No Improvements Required Some Improvements Required Significant Improvements Required Critical 13 WSIB - Workwell Program Value for Money Audit

14 Audit Opinion findings Systems (cont d) The following table shows our summary audit findings for the Systems component of the Value for Money Audit Framework as it relates to the Workwell program. Audit Criteria Key Audit Criteria Attributes Observations Rating Systems The information technology systems in place to support the program have the desired functionality to deliver and monitor the program. There is opportunity for efficiency improvement by enabling the use of electronic forms/documents by Workwell Evaluators to eliminate duplication of input efforts. Instances of poor data quality and integrity issues were noted during the analysis. Some Improvement Required Resources Policies and processes supporting the interpretation of the guiding legislation are documented, communicated to, and understood by management and staff. There is a lack of clarity amongst staff (and consequently customers) regarding the Workwell mandate (i.e., is it compliance or instructional and guidance driven?). Workwell program policy indicates that Employers that fail the initial evaluation are required to work with a health and safety program provider. However, this is not the current practice. Significant Improvement Required Outcomes LEGEND No Improvements Required Some Improvements Required Significant Improvements Required Critical 14 WSIB - Workwell Program Value for Money Audit

15 Audit Opinion findings - Resources The following table shows our summary audit findings for the Resources component of the Value for Money Audit Framework as it relates to the Workwell program. Audit Criteria Key Audit Criteria Attributes Observations Rating Systems Resources Program resources are managed efficiently to meet program demands. The program s actual spend is presently operating at approximately 71% of the year to date budget with only 65% of the budgeted FTE. There is opportunity for additional management support for field operations. The Firm Selection Model may result in numerous large, multi-site operations being selected due to use of the WSIB account number being the primary employer data driver. Locations that caused the incidents may not be selected for review and sites that demonstrate good performance may be reviewed. Subsequent to implementing the 2008 Risk Management and Monitoring Plan Procedure the number of Risk Management Plans has been reduced and represents only 8% of completed cases for the past two (2) years. Through the data analysis, in some instances RMPs demonstrated larger reductions in LTI rates than a full Workwell audit. Significant Improvement Required Outcomes Policies and processes are executed in a timely manner. The use of lagging indicators in the selection models results in a considerable period of elapsed time (potentially greater than 2 years) prior to being identified for Workwell intervention. A selected firm may not be audited for an additional 12 months after being notified that they have met the Workwell criteria. The Firm Selection Model s reliance on a one year negative performance spike may limit the ability of Workwell to interact with firms which experience many years of consistently poor or below average performance. Significant Improvement Required LEGEND No Improvements Required Some Improvements Required Significant Improvements Required Critical 15 WSIB - Workwell Program Value for Money Audit

16 Audit Opinion findings Resources (cont d) The following table shows our summary audit findings for the Resources component of the Value for Money Audit Framework as it relates to the Workwell program. Audit Criteria Key Audit Criteria Attributes Observations Rating Systems Identified challenges and issues are addressed in a timely fashion and in accordance with the established escalation and resolution process. There is lack of an integrated prevention approach and mechanisms between Workwell and other programs in Ontario s health and safety system to foster sustainability (e.g., transition for Workwell audited employer into a sustaining type program such as SCIP or Safety Group; Health and Safety Associations function to provide ongoing support to Workwell audited firms). Despite Workwell appeals not being frequently successful (i.e., results are overturned) analysis of the reason(s) for the appeal being launched is not performed. Significant Improvement Required Resources Adequate training and development programs are in place for management and staff to ensure succession planning and knowledge retention. A formal mechanism to identify the Workwell Evaluator subject matter expert (SME) areas of expertise and to continue to develop and retain the Evaluator s experiences and capabilities is not in place. Formal succession planning processes are not in place. Some Improvement Required Outcomes LEGEND No Improvements Required Some Improvements Required Significant Improvements Required Critical 16 WSIB - Workwell Program Value for Money Audit

17 Audit Opinion findings - Outcomes The following table shows our summary audit findings for the Outcomes component of the Value for Money Audit Framework as it relates to the Workwell program. Audit Criteria Key Audit Criteria Attributes Observations Rating Systems The program has clearly defined objectives that are aligned with organizational strategies, and there are measures in place that demonstrate satisfactory progress towards those objectives. Metrics to evidence the program s ongoing success by internal or external stakeholders (i.e., fewer injuries and illnesses, reduced risk of injury) are not tracked and reported at a level to enable conclusions to be made of the program effectiveness. The current Workwell prevention mandate does not specifically align with the updated WSIB mandate, vision statements and strategic direction. Critical Resources Outcomes The program contributes to a sustainable reduction in injuries and illnesses in the Province s workplaces. Data analytics did not show consistent evidence that Workwell alone has helped to reduce LTI on a sustained basis. All LTI rates, based on WSIB supplied data, show sustained declines independent of an intervention. Notifications combined with a Risk Management Plan (RMP) are generally the most successful Workwell intervention to show evidence of decreasing a selected firm s LTI rate. Consistent criteria to define the application of one Workwell interaction (e.g., Risk Management Plan or Core Audit) versus another depending on the characteristics of the client are not in place. Workwell does not analyze the effectiveness of its various methods of intervention (e.g., Core Audit vs. RMP, vs. Notification only). Critical LEGEND No Improvements Required Some Improvements Required Significant Improvements Required Critical 17 WSIB - Workwell Program Value for Money Audit

18 Audit Opinion findings Outcomes (cont d) The following table shows our summary audit findings for the Outcomes component of the Value for Money Audit Framework as it relates to the Workwell program. Audit Criteria Key Audit Criteria Attributes Observations Rating Systems Resources The program meets the needs of the workplace parties in the Province of Ontario. There are opportunities for overall program improvements to enhance Workwell s operational effectiveness and efficiency. There is some confusion in the employer community with respect to the role of Workwell Evaluators and Ministry of Labour Inspectors and their application and interpretation of regulatory and legislative requirements. Both are often viewed as compliance and enforcement focused and the prevention intent is lost during the intervention. An integrated approach for prevention and health and safety programs is not in place for the Province s workplaces. Our audit indicates that, notwithstanding confusion as to the existing mandate of the Workwell program, Workwell in its current form no longer aligns to WSIB s mandate. Significant Improvement Required Outcomes LEGEND No Improvements Required Some Improvements Required Significant Improvements Required Critical 18 WSIB - Workwell Program Value for Money Audit

19 Summary Audit Findings The table below shows our summary audit findings of each component as it relates to the Workwell program. Audit Criteria Systems Observations There is confusion within the Workwell team and in the employer community about the nature of the program i.e., is it compliance or consultation and collaboration? Program volumes are not being met. Health and Safety dashboard reporting does not track and report trending to enable monitoring of the sustainability of health and safety efforts once a Workwell audit has been performed. The Workwell Case Information System has opportunities for enhanced functionality through the use of electronic forms which would improve the efficiency of Workwell Evaluators. Data analytics on an ongoing basis is a challenge due to WSIB s use of multiple systems which prevents integrated data reporting. Rating Some Improvement Required Resources Outcomes The level of deployed program resources may not be commensurate with the program design and objectives. There is opportunity to evaluate the structure of management support provided to field operations. Employers noted concerns about the background of Workwell Evaluators and whether there is sufficient experience to effectively assess the wide variety of businesses in the province. Resources engaged in large, multi-site audits may not target or visit the worst offender locations within the organization. There is opportunity to use predictive analytics to select firms for Workwell audits. Metrics to evidence Workwell s ongoing success (i.e., fewer injuries and illnesses, reduced risk of injury) are not tracked and reported. Data analytics did not show consistent evidence that Workwell alone has helped to reduce LTI on a sustained basis. All LTI rates, based on WSIB supplied data, show sustained declines independent of an intervention. Consistent criteria to define the application of the type of Workwell intervention are not in place. The Workwell Core Health and Safety Audit tool is very comprehensive. However, there are opportunities for improvements to the design and execution of the tool. The province does not have an integrated approach between Workwell and other prevention and health and safety programs. Significant Improvement Required Critical 19 WSIB - Workwell Program Value for Money Audit

20 Auditors Report To the Directors, Workplace Safety & Insurance Board: We have conducted a Value for Money Audit of the Workwell program at the Workplace Safety and Insurance Board (WSIB) of Ontario in accordance with WSIB s request for proposals No. RFP # SVB. The objective of the audit, as stated in the request document, was to: Provide an opinion as to whether there is value for money in the outcomes generated by the Workwell program and recommend improvements that could be made Identify improvements/opportunities that will allow efficient and effective alignment of the Workwell program with the new prevention mandate (Bill 160) as it develops (please see page 5 of this report for a full description of the project s objectives and scope). We conducted our audit in accordance with the value for money audit standards recommended by the Canadian Comprehensive Auditing Foundation and accordingly included such tests and other procedures as we considered necessary in the circumstances. Specific criteria encompassed by this audit are identified on page 6 of this report. 20 WSIB - Workwell Program Value for Money Audit

21 Recommendation for Improvement & Management Response 21

22 Recommendation for improvement Based on our analysis of the Workwell program audit findings and conclusions and the alignment of the program with the WSIB mandate we offer our recommendation and the respective Management Response. No. Recommendation 1 Workwell program alignment with WSIB mandate As a result of Bill 160, the Workwell program no longer aligns with the WSIB's future mandate and operations. The WSIB should support the Chief Prevention Officer in determining which elements of the current program could be retained and/or refined in the new system, defining the nature and scope of the elements, and identifying the system partner(s) best qualified to deliver the activities within those elements. WSIB agrees with the recommendation. Management Response WSIB will support the Chief Prevention Officer (CPO) in identifying the elements of Workwell that should be retained and/or refined in the new Prevention system. WSIB will assist the CPO in defining the nature, scope and delivery of those elements by the end of Q4, WSIB - Workwell Program Value for Money Audit

23 Considerations for Program Redesign 23

24 Considerations for program redesign In the pages that follow, we present a series of considerations for the redesign of the Workwell program as part of the broader development by the CPO of a new prevention system within the Province. Theme Program development opportunity Program Strategy and Communications Health and safety system partners - System partners should collaborate their efforts to increase awareness and standardization related to the programs, procedures, processes, tools and methodologies which are available to support an organization s health and safety initiatives. Where possible, the use of existing standards such as CSA Z-1000 and ISO should be considered in developing and delivering the programs. Roles and responsibilities should be developed to ensure that firms continue to experience interaction with system partners once the intervention is complete. Ongoing involvement with other programs offered by system partners upon exiting the intervention process may help to sustain health and safety improvements in the organization and reduce the possibility of future incidents or non-compliance. Communications To reduce the likelihood of confusion in the employer and worker community the new program should clearly identify its nature and intent in the official communications and publications. For example, mixing terminology such as incentive, compliance, consultative and audit creates differing perceptions for stakeholders related to outcomes and type of involvement which may be experienced while interacting with the program. 24 WSIB - Workwell Program Value for Money Audit

25 Considerations for program redesign Theme Program development opportunity Role clarity Based on the new program s direction there should be clearly defined roles and responsibilities for the management and staff job descriptions. It is important to delineate more clearly between education versus consultative versus audit versus enforcement roles. The roles and responsibilities should be communicated appropriately to all stakeholders (e.g., workplace parties, WSIB, Health and Safety Associations, and Ministry of Labour). Role Clarity and Expertise Specialist roles and Subject Matter Experts (SME) - Specialist type roles based on industry and sector specific organizations (e.g., to perform interventions at clients such as hospitals and healthcare institutions) should be developed as a program resource. The staff in these roles should be trained regarding the specific types of risks within these organizations and have an understanding of the terminology and health and safety methodologies and practices which should be in place. In addition, program staff that are considered to be subject matter experts (but not necessarily acting in a specialist role) in their industry or area of expertise should be identified and recognized as a resource within the team. Ongoing training needs for the SME should be identified and formalized through the performance management process in order that their knowledge level remains current and relevant. The listing of subject matter experts should be shared across the program team to ensure they may be consulted as required during the interaction with the employer/firm. Active participation and involvement between subject matter experts and the employer will build credibility, enable knowledge transfer and retention of expertise within the program. 25 WSIB - Workwell Program Value for Money Audit

26 Considerations for program redesign Theme Program development opportunity Program Resources Program resources - The level of resources, both financial and human, which are committed to the program, should be commensurate with the program design and objectives. Program outputs and outcomes should be monitored to ensure optimal program performance. Program organization structure - The program organization structure should be designed to ensure that appropriate levels of administrative control, oversight and field level support are in place. For example, a Field Supervisor type role with responsibility to review and approve completed interventions, communicate with employers for customer service issues and provide program introduction seminars to the employer community would be beneficial as a link between field operations and head office functions. Staff performance objectives - The workload target quantity of interventions to be performed by staff should consider the overall program goals and objectives in addition to throughput volumes. A series of factors should be considered to plan the potential effort required to complete an intervention such as: -employer history and risk profile(s); -number of employees; -number of facilities/locations; -square footage; -number and type of shifts; -nature of workplace hazards. Based on the anticipated and actual efforts to complete a case a scalable metric should be developed to record and track the types of interventions to be completed by program staff. 26 WSIB - Workwell Program Value for Money Audit

27 Considerations for program redesign Theme Program development opportunity Targeting and Intervention Use of predictive analysis in Firm Selection Model The system partners should collaboratively identify and establish a methodology and criteria to enable more timely workplace interventions with firms experiencing poor health and safety records. The use of current performance metrics and predictive analysis should be considered in the Firm Selection Model to ensure that issues within firms are quickly and proactively addressed to reduce risk and potential cost to the employee, employer, WSIB and system partners. The Firm Selection Model should be made available to the public through the partner s websites. This would increase program transparency and enhance employers understanding of why they have been selected for intervention. Intervention types and data analytics A series of workplace interventions should be developed to ensure the widest possible coverage of Ontario s workplaces based on results output from the Firm Selection Model and other targeting methods. Workplace interventions by type should be tracked and analyzed. This analysis will also be of use to determine which type(s) are most effective in order to inform future resource deployment. Consistent criteria should be created to identify the type of workplace interaction which is appropriate for the employer s circumstances, risk profile and health and safety system which may be in use. The use of current injury data (i.e., subsequent to the intervention) should also be considered to determine which interactions are most effective at sustaining improved health and safety performance. Factors to consider may include injury profile, firm size and injury results by full-time equivalent (FTE). Targeted interventions - A detailed analysis of multi-site operations identified during the selection process should be completed to ensure that the location(s) and incidents responsible for poor performance are targeted and reviewed during the intervention. This analysis will enable program resources to be deployed to areas of highest risk within selected organizations. 27 WSIB - Workwell Program Value for Money Audit

28 Considerations for program redesign Theme Program development opportunity Outcomes and Reporting Dashboard reporting The program should develop and implement reports and dashboards to enable tracking and ongoing performance for key metrics such as lost time injuries (LTI) and high impact claims for firms which have been selected for intervention. Use of these types of analytics will assist health and safety system partners in identifying those firms which are chronic repeaters in the selection process in a more timely manner. In addition, positive outcomes for firms that have embraced health and safety as a result of interaction with the program may be identified. Where possible, this trending data should be shared with other partners in the health and safety system to ensure that appropriate and targeted interventions are executed. Continuous program improvement The program should develop and implement a process to obtain feedback from stakeholder groups (e.g., employers, workers, system partners) to assess the program s effectiveness and delivery compared to its goals and objectives. Feedback obtained through the process (e.g., program experience surveys completed by firms which have participated in an intervention) should be evaluated by program management and staff to identify solutions in order to address noted concerns. 28 WSIB - Workwell Program Value for Money Audit

29 Considerations for program redesign Theme Program development opportunity Data collecting and sharing - As part of the program design consideration should be given to ensure that data, results and other program information can be shared and reported across the system partners as required. The program should consider an inventory of the relevant data, classification of data, ranking (i.e., who has access to the data) and how the planned use of data aligns with management s decision making. Information Technology Systems 29 WSIB - Workwell Program Value for Money Audit

30 Bill 160 Alignment Detailed Findings Supporting Analysis 30

31 Bill 160 alignment supporting analysis and findings Workwell alignment in the new health and safety system in Ontario Characteristics of Workwell* in in its current form New WSIB Mandate Bill 160 Prevention Mandate Promote health and safety in the workplace Facilitate RTW, recovery and LMR of workers who sustain a work related injury or occupational disease Provide compensation and other benefits where RTW cannot be achieved To promote occupational health and safety and to promote the prevention of workplace injuries and occupational diseases To promote public awareness of occupational health and safety To educate employers, workers and other persons about occupational health and safety To foster a commitment to occupational health and safety among employers, workers and others Targets worst performers Attempts to interpret/provide guidance on regulatory compliance Punitive (fines) versus incentive (achievement) Prescriptive (one way) versus flexible (many ways to meet intent) Short term (one time) versus long term (developmental) focus *Source: Workwell Evaluator Evaluation Process & ISO 19011: WSIB - Workwell Program Value for Money Audit Lack of alignment Partial alignment Strong alignment

32 Value for Money Opinion Detailed Findings Supporting Analysis 32

33 Audit Opinion supporting analysis and findings Rating: Some Improvement Required Audit Criteria Attribute: Roles, responsibilities, accountabilities, and performance expectations are defined, understood, and consistently executed by management and staff Systems Resources Outcomes Roles and Responsibilities Accountabilities Defined Understood Consistently Executed Some confusion around execution Performance Expectations Variations noted Selected Workwell Performance Objectives Organizational Excellence Service Excellence Minimum of sixty (60) evaluations are completed on a rolling twelve (12) month basis Complete assigned evaluations within 6-8 months of the first audit Deliver presentations and workshops as assigned Surcharges and credits to be posted to PeopleSoft A/R within 2 weeks of evaluation completion Introductory phone call to customer within 15 days of notification letter mailing Send evaluation letter, audit or RMP report to customer within 10 days of completing the audit or RMP Job descriptions noting the Job Summary, Major Duties and Responsibilities and Job Requirements are in place for the Workwell Evaluator and Manager roles. Input from the stakeholder sessions noted that there is some confusion internally and externally regarding the extent of Workwell Evaluator s consultative services and compliance type activities (Role Clarity and Expertise). The Workwell objective is to complete 60 cases per evaluator over a 12 month cycle. Workwell Evaluator Closure Statistics for the period December 1, 2009 to December 1, 2011 indicate that the 23 Workwell Evaluators completed 1,748 cases (Program Resources): 38 completed cases per 12 months is the average per Workwell Evaluator; 3 (13%) Workwell Evaluators completed more than 60 cases; 12 (52%) Workwell Evaluators completed between 30 and 59 cases; and 8 (35%) Workwell Evaluators completed less than 30 cases. The Workwell team meets monthly, records meeting minutes and assigns action items for follow-up. 33 WSIB - Workwell Program Value for Money Audit

34 Audit Opinion supporting analysis and findings Rating: Some Improvement Required Audit Criteria Systems Resources Outcomes Attribute: Appropriate administrative controls and safeguards are in place to mitigate Program risks A Workwell Evaluator Evaluation Process and ISO 19011:2002 flow chart has been developed by the team during The flow chart overlays the ISO methodology of Plan, Do, Check, Act on to the Workwell Evaluation Process. However, the application of the methodology does not align with actual Workwell practices (Program Resources). A Workwell Evaluators Findings Committee is in place. The purpose of the Committee is to establish a consistent response to issues which have been identified during execution of Workwell audits. However, the former Chair is no longer with the Workwell program and consequently the Committee has not met regularly during 2011 (Program Resources). The Workwell Evaluators Findings Committee is not specifically identified on the Workwell Evaluator Evaluation Process and ISO 19011:2002 document as a control for monitoring, reviewing or improving the audit program. Workwell assigns cases based on postal codes noted in the FSA List Employer and worker/labour stakeholders noted concerns that the Workwell Evaluators may not have sufficient and appropriate background to understand the nature of all the business types which they are to review (e.g., healthcare institutions, large construction sites). Workwell Evaluators have the opportunity to request the assistance of colleagues to provide background and context (e.g., due to their expertise within an industry/sector) during an audit. There is opportunity to increase the level of auditor calibration and use of industry/sector experienced resources in order to address employer concerns that Evaluators may not have appropriate background to understand their business (Role Clarity and Expertise). A System Collaboration Branch Survey of Employers Undergoing First Workwell Audit was completed January 4, 2011 but did not obtain feedback on the entire Workwell customer experience. There may be further opportunity to address customer concerns within the program execution (e.g., implementation timelines for phase 2; pass/fail requirements for audit elements) by conducting a survey of the entire Workwell timeframe (Outcomes and Reporting). Operating budgets are prepared and monitored monthly. Workwell surcharge/revenue tracking and reporting is not consistently performed. 34 WSIB - Workwell Program Value for Money Audit

35 Audit Opinion supporting analysis and findings Rating: Some Improvement Required Audit Criteria Systems Resources Outcomes Attribute: A tracking and reporting process is in place to monitor Program performance, and performance metrics are monitored and acted upon Tracking & reporting systems with select performance metrics tracked Workwell Case Information System (WCIS) Health & Safety System Target and Performance Dashboard (quarterly) Number of open cases per Workwell Evaluator Aging of open cases Upcoming case load counts, scheduling and timing Case approval listing for Managers (Request Assistant for an evaluation; surcharge > $100,000) Performance reporting Injury Profile LTI Count High Impact Claims LTI Count System Partner Cohort Injury Rates Performance System Partner Cohort Engagement Health and Safety Association Injury Rates Performance Workwell Managers run periodic reports from the Workwell Case Information System (WCIS) to monitor Evaluator case loads and upcoming schedules. Depending on circumstances within the team, cases may be re-assigned to other Evaluators. Management review and approval for case specific requests are also routed through WCIS. WSIB prepares quarterly dashboards which include LTI rates for Workwell audited firms. The dashboard report is a point in time value and does not include metrics to demonstrate sustainability of improved performance over time for firms which have been through Workwell. The LTI rate reported in the dashboard is based on those firms which interacted with Workwell during that quarter. This approach does not provide an assessment of the potential impact of having been through the Workwell evaluation (Outcomes and Reporting). Trend type data analysis is not regularly performed by program management for internal review or to be shared at an operational level with the Workwell Evaluators. 35 WSIB - Workwell Program Value for Money Audit

36 Audit Opinion supporting analysis and findings Rating: Some Improvement Required Audit Criteria Systems Resources Outcomes Attribute: The information technology systems in place to support the Program have the desired functionality to deliver and monitor the Program Workwell Case Information System Selected current system functionality Web based solution is available 24/7 Use of pre-determined alerts Core Health and Safety Audit, Small Business Audit, Agriculture Audit and Risk Management Plan templates Secure access with User ID and Password Schedule manager Case management options by role Editable templates for Audit Cover Letter, Audit Results Automated calculation of audit results Workwell Case Information System Opportunities to enhance functionality Use of electronic forms which can be uploaded into WCIS Built in spell checking within the system Improved formatting capabilities for Audit reports Surcharge tracking to enable management monitoring and reporting Evaluators consider the Workwell Case Information System (WCIS) to be a positive application for performing their duties. There is opportunity for efficiency improvement by enabling the use of electronic forms/documents to eliminate duplication of Workwell Evaluator efforts in manually reentering audit results into WCIS. Presently, Workwell Evaluators manually complete the audit using paper based forms which they then input into WCIS (Information Technology Systems). The Workwell Evaluators noted that they have identified additional improvements to the WCIS functionality which have not been implemented due to resource constraints. During the information request and data selection process numerous instances of poor data quality and integrity issues were noted by Workwell management. Systems controls and reconciliations for data between Workwell and injury history do not appear to be in place. This is further evidenced by the lack of ongoing reporting of injury history related to Workwell firms due to challenges in combining and consolidating results from multiple systems (Information Technology Systems). 36 WSIB - Workwell Program Value for Money Audit

37 Audit Opinion supporting analysis and findings Rating: Significant Improvement Required Audit Criteria Attribute: Policies and processes supporting the Program are documented, communicated and understood by management and staff Systems Resources Outcomes Workwell Program Policy and Guidelines Policy Guidelines Governing Law and Policy The Workwell program issues an additional premium charge to employers who have not taken sufficient precautions to prevent accidents in the workplace. Workwell operates independently of the WSIB s experience rating programs and its additional premium charges are issued in addition to any premium or premium rate adjustments that may be realized through experience rating. Workwell identifies employers with particularly poor accident records and/or high accident costs compared to their rate group and/or rate groups, or who have a history of non-compliance with the Occupational Health and Safety Act and encourages them to improve their prevention programs. Workwell requires these employers to participate in a workplace health and safety evaluation. Employers that fail the evaluation are given a specified time frame to make improvements. If improvement does not occur within the given time the WSIB issues an additional premium charge. s.82 (1) (4) Workplace Safety and Insurance Act During the stakeholder sessions comments were noted that there is lack of clarity amongst staff (and consequently customers) regarding the Workwell mandate (i.e., is it compliance or instructional and guidance driven) (Program Strategy and Communications). Workwell is included on the WSIB website as an Incentive program. The Workwell Program Policy indicates that Employers that fail the initial evaluation are required to work with a health and safety program provider (e.g., a Health and Safety Association) to improve their compliance with the evaluation criteria. Failure to do so results in the immediate levying of the additional premium charge, based on the initial evaluation score. This policy provision has never been put in place. The inclusion of reference to a secondary additional premium charge within the Workwell Program Policy further increases the perception that the program is punitive in its intent (Program Strategy and Communications). Through the stakeholder discussions it was noted that the program s operational policies and procedures are understood by Workwell management and staff Workwell Program 37 WSIB - Workwell Program Value for Money Audit

38 Audit Opinion supporting analysis and findings Rating: Significant Improvement Required Audit Criteria Systems Resources Outcomes Attribute: Program resources are managed efficiently to meet Program demands Number of Workwell Cases Closed Average Cost of Closed Cases Average annual value for Workwell surcharges assessed over the past three (3) years is $978,000 and average program cost is $2.7million. The program s actual spend ($2.9 million) is presently operating at approximately 71% of the year to date budget with only 65% (22) of the 35 budgeted FTE. Management indicated that some staff have left the program due to uncertainty and have not been replaced. Field Supervisor positions are not in place (Program Resources). Stakeholder sessions noted that there is perception in the worker/labour and Health and Safety Association groups that there are insufficient Workwell resources to address the number of firms requiring improvements to their safety practices. Large, multi-site audits impact the ability for Evaluators to meet the 60 completed cases expectation. There is opportunity for more targeted interventions in these instances (Targeting and Intervention). Risk Management Plans represent approximately 8% of completed cases for the past two (2) years. Performing additional Risk Management Plans may be a more effective use Workwell Evaluator resources and increase the coverage of selected employers. (Targeting and Intervention) Source: Evaluator s Activity Summary 38 WSIB - Workwell Program Value for Money Audit

39 Audit Opinion supporting analysis and findings Rating: Significant Improvement Required Audit Criteria Systems Resources Outcomes Attribute: Policies and processes are executed in a timely manner The use of lagging indicators in the Firm Selection Model and Workwell Selection criteria results in a considerable period of elapsed time (potentially greater than 2 years) prior to being identified for Workwell. (Targeting and Intervention) Through discussion with WSIB management it was noted that there are ongoing initiatives to identify and possibly use predictive analysis as a tool to assist in the selection processes for other WSIB compliance type audits. There is opportunity to consider use of predictive metrics in the Firm Selection Model and more targeted interventions within the Workwell program. The Firm Selection Model and Workwell Selection criteria are not published on the WSIB website. Through discussion with employers concern was expressed that firms are being held accountable to standards of which they do not have awareness (and which may also change annually) until such time as they are identified for a Workwell audit. The Workwell process may result in a selected firm not being audited for more than 12 months after being notified that they have met the Workwell criteria. This could be up to three (3) years of elapsed time after the event(s) which resulted in Workwell selection. 39 WSIB - Workwell Program Value for Money Audit

40 Audit Opinion supporting analysis and findings Rating: Significant Improvement Required Audit Criteria Systems Resources Outcomes Attribute: Identified challenges and issues are addressed in a timely fashion and in accordance with the established escalation and resolution process There is lack of an integrated prevention approach and mechanisms between Workwell and other programs in Ontario s health and safety system to foster sustainability (e.g., transition for Workwell audited employer into a sustaining program such as SCIP or Safety Group). Workwell related fee for service options are presented to employers on the Health and Safety Associations websites which also receive funding directly from the WSIB (Program Strategy and Communications). There are ongoing efforts to increase the touchpoints in the system through sharing Firm Selection Model results with the Ministry of Labour and Health and Safety Associations. There is a lack of consistency within standards, methodologies and competencies across the system which results in frustration and reduced engagement of all available resources in the employer community. A Workwell appeals process is in place through the WSIB for employers who are not in agreement with the audit result. Appeals are not frequently successful and the results overturned. Analysis of the reason(s) for the appeal being launched is not performed. (Outcomes and Reporting) 40 WSIB - Workwell Program Value for Money Audit

41 Audit Opinion supporting analysis and findings Rating: Some Improvement Required Audit Criteria Attribute: Adequate training and development programs are in place for management and staff to ensure succession planning and knowledge retention Systems Resources Outcomes Overview of Workwell Evaluator Orientation/Training Program Selected Weekly Objectives Week 1 Week 2 Week 3 Week 4 Week 5 Weeks 6-8 Weeks Obtain an initial understanding of the role and purpose of the Workwell Program with OH&S system Understand the structure and content of the Workwell Core Audit Through case study apply the audit content to the OH&S program of a suitable firm Assigned to two different senior Evaluator s for audit shadowing opportunities Conduct research into assigned firms using CICS/RESET and other databases Attend Collision Free and Skid school driver safety training Shadow a minimum of 2 audits per week with experienced Evaluator Conduct audit of at least 8 of their own files while being shadowed by experienced Evaluator or Manager Team Training Activities for 2010 and 2011 Working at Heights Machine Guarding Insights into Influencing Workshop Note Taking and Interviewing Techniques Facilitation WHMIS: Our Right to Know An Orientation/Training Program is used to onboard new hires to the team. Workwell Evaluators completed CSA Z1000 Essentials Course and OHSAS training in The Workwell team has approximately two (2) professional development days annually Through discussion with employers and Health and Safety Associations concerns were expressed about a lack of experience and background by Workwell Evaluators in some industry/sectors. The stakeholders indicated that the lack of background in some instances resulted in misinterpretation or misunderstanding of certain systems or processes in the firm by the Workwell Evaluator. There are members of the Workwell team who are considered to be subject matter experts in some industries/sectors due to their previous experience. A formal mechanism to identify the areas of expertise and to continue to develop and retain the Evaluator s experiences and capabilities is not in place. (Role Clarity and Expertise) Formal succession planning processes are not in place for Workwell management positions. (Role Clarity and Expertise) 41 WSIB - Workwell Program Value for Money Audit

42 Audit Opinion supporting analysis and findings Audit Criteria Rating: Critical Attribute: The Program has clearly defined objectives that are aligned with organizational strategies, and there are measures in place that demonstrate satisfactory progress towards those objectives Systems Resources Outcomes The Workwell Policy and information published on the WSIB website indicate that Workwell s mandate is prevention and the evaluations help workplaces identify weaknesses in workplace health and safety programs and practices. Metrics to demonstrate evidence of the program s ongoing success (i.e., fewer injuries and illnesses, reduced risk of injury) to internal or external stakeholders are not tracked and reported at a level to enable conclusions to be made of the program effectiveness. As noted earlier, LTI is the key Workwell metric reported on the WSIB dashboards (see below for example of dashboard reporting). (Outcomes and Reporting) Some employers noted that Workwell will temporarily raise awareness of health and safety within an organization but in many cases is not sustained over the long term. Based on review of the WSIB Strategic Plan, Corporate Measures reports, Workwell Policy documents and discussions with stakeholders the current Workwell prevention mandate does not demonstrate alignment with the updated WSIB mandate, vision statements and strategic direction YTD System Partner Cohort Engagement 2011 System Parter Cohort Injury Rates Performance System Partner # of Firms in Cohort WORKWELL 1,260 89% 32% WORKWELL SCIP SAFETY GROUP VU OHCOW MOL 985 3, ,000 87% 97% 100% 100% 22% 11% 0% 100% 100% 100% SCIP SAFETY GROUP VU OHCOW MOL IHSA 1,575 96% 63% IHSA PSHSA 1,002 56% 74% PSHSA WSN % 26% WSN WSPS 2,245 39% 43% WSPS WSIB - Workwell Program Value for Money Audit % Engaged % High Engaged System Partner LTI Rate 2011 LTI Rate 2010 LTI Rate Outcome Total Rate 2011 Total Rate 2010 Total Rate Outcome

43 Audit Opinion supporting analysis and findings Rating: Critical Audit Criteria Systems Attribute: The Program contributes to a sustainable reduction in injuries and illnesses in the Province s workplaces Workwell s effect on reducing injuries and illnesses is not conclusive after analysis of Workwell data. Overall societal trends, economic conditions, and the impact of other Health and Safety organizations/programs such as Experience Rating influence LTI rate decreased presented in addition to Workwell. Analysis suggests that factors external to Workwell have a greater influence than Workwell on LTI rate decreases. The table below exhibits that companies improve LTI rate with or without Workwell intervention. Resources LEGEND Incident Year Year of WW Interaction Average LTI Rate / Cohort Year Average LTI Rate 2006 Cohort Average LTI Rate 2007 Cohort Average LTI Rate 2008 Cohort Average LTI rate 2009 Cohort Average LTI Rate 2010 Cohort Outcomes LTI rates spike in incident years, however, firms revert to their pre-incident year downward trend with or without Workwell interaction (see appendix for trend summaries for each cohort). The spike in LTI rates implies that poor performance years are discrete events as opposed to consistent trends. Firms appear to action these events themselves. LTI rate was decreased even though Workwell interaction did not occur. The amount of decrease is similar to years were Workwell interaction occurred in the year directly following poor performance. 43 WSIB - Workwell Program Value for Money Audit

44 Audit Opinion supporting analysis and findings Audit Criteria Systems Resources Rating: Critical Attribute: The Program contributes to a sustainable reduction in injuries and illnesses in the Province s workplaces Firms experience decreased LTI rates due to a combination of factors such as: Societal Trends - Schedule 1 acts as a proxy for the influence of overall societal trends on LTI rate decreases. Own Motivation / Other Influencers - The No Interaction Met Criteria cohort serves as a proxy for how firms with poor performance behave after the year of poor performance due to influences outside of Workwell. Workwell Influence It is the incremental decrease in LTI rate that firms experience over and above the LTI rate decrease experienced by Schedule 1 and No Interaction,- Met criteria that approximate Workwell s influence. Therefore, even though in 2010 the cohort Had Workwell interaction experienced a 58% total decrease in LTI rate, it is reasonable to estimate that: 29% of that total 58% decrease was due to the firm s own motivation / external factors. 24% of that total decrease was due to overall societal trends. LTI Rate Decrease Outcomes LTI RATE 16% 20% 5% 26% 37% Total LTI rate decrease 5% 33% 42% 29% Invalid 29% Data 26% 24% 2006 Cohort 2007 Cohort 2008 Cohort 2009 Cohort 2010 Cohort 58% Of total LTI rate decrease potentially attributable to WW Of total LTI rate decrease potentially attributable to firm s own motivation Of total LTI rate decrease potentially attributable to overall societal trend 44 WSIB - Workwell Program Value for Money Audit

45 Audit Opinion supporting analysis and findings Audit Criteria Rating: Critical Attribute: The Program contributes to a sustainable reduction in injuries and illnesses in the Province s workplaces Systems Year-Year Comparison Due to a change in Workwell selection criteria in 2009, more firms with performance that was not as poor as 2006/07 cohorts were selected for Workwell. As this cohort s performance was not as bad there was less room for improvement, and therefore the total decrease in LTI rate was smaller. Additionally, the incremental influence of Workwell decreased while the influence of firms own motivation/other influencers decreased. Workwell Impact Short Term Resources WW Interaction 2006 Cohort 2007 Cohort 2008 Cohort 2009 Cohort Outcomes Schedule 1 37% 29% 26% % Decrease in LTI Rate: No WW interaction Met Criteria 63% 62% 68% Incident Year cohort Had WW interaction 79% 82% used 63% Incremental % decrease in LTI rate Incremental LTI rate Improvement over Schedule 1 Incremental LTI rate Improvement with WW Interaction data 26% 33% invalid 42% 16% 20% 5% 42% 5% 45 WSIB - Workwell Program Value for Money Audit

46 Audit Opinion supporting analysis and findings Rating: Critical Attribute: The Program contributes to a sustainable reduction in injuries and illnesses in the Province s workplaces Workwell firms have similar LTI rate decreases from as schedule 1 firms, indicating that they follow the same trend in LTI rate as schedule 1. Workwell does not seem to have a sustainable effect on LTI rate decreases because firms appear to return to their prior LTI level after their year of poor performance and do not consistently perform better than their Schedule 1 peers in the long term. Firms that are Notified Only receive no further interaction in their cohort year beyond a notification letter. However, all firms that have any other interaction with Workwell, (RMP / Audit) receive a notification letter first, in addition to their RMP/ Audit. The percentage decrease that firms experience from an RMP or an Audit may be influenced by the fact that they first receive a letter of notification in addition to their Workwell intervention. RMP was the most effective Workwell interaction overall in terms of decreasing LTI rate. Notification Only was the second most effective Workwell interaction in terms of decreasing LTI rate. % Decrease in LTI Rate Cohort Year WW Interaction 2006 Cohort 2007 Cohort Schedule 1 42% 42% No WW interaction Met Criteria 2008 Cohort 2009 Cohort 2010 Cohort 42% 42% Invalid 52% 59% data 38% 43% Had WW Interaction 59% 50% 35% 42% Most effective at decreasing LTI Rate Incident year Second most effective at decreased LTI Rate: Incident year RMP Notified Only 2007 RMP Notified Only 2008 RMP (same % decrease as Notified only ) 2009 First Audit Pass Workwell Impact Long Term Most Successful type of Workwell Interaction Notified Only (same % decrease as RMP) RMP/ Notified Only (both equal decrease ) 2010 Notified only First Audit Pass 46 WSIB - Workwell Program Value for Money Audit

47 Audit Opinion supporting analysis and findings Rating: Significant Improvement Required Audit Criteria Systems Resources Outcomes Attribute: The Program meets the needs or the workplace parties in the Province of Ontario There is a split of opinion between worker/labour groups (favour Workwell) and employer groups (generally disfavour Workwell). Workwell s program mandate and objectives are not clear in the employer community (e.g., role differentiation with Ministry of Labour Inspectors). The program is perceived as punitive due to the surcharge resulting from failing an audit. Workwell is the only health and safety system compliance evaluation in the province outside of those conducted for fee by independent firms or consultants. Employers indicated support and value for Safety Groups. Safety Groups LTI rates have outperformed Workwell over the past 3 years based on the metrics reported in the WSIB dashboard. The Workwell Core Health and Safety Audit tool provides a very comprehensive evaluation of a firm s health and safety system. However, there are opportunities for improvement in the design and execution of the tool (Targeting and Intervention): Workwell is a bolt-on approach to health and safety versus a built system. The approach is prescriptive and does not recognize systems in place which may be suitable for the business but do not meet Workwell s definitions. The scoring methodology is based on receiving all of the points or none of the points for each element. There is no opportunity for the recognition of efforts that are underway in the organization. A firm can fail a key section of the audit (e.g., health and safety hazards) yet still receive an overall audit pass. An audit pass is only 75% of the Workwell standard. The document is heavily influenced by references to industrial type descriptions and examples which reduces the program credibility in non-industrial applications. (cont d ) 47 WSIB - Workwell Program Value for Money Audit

48 Audit Opinion supporting analysis and findings Rating: Significant Improvement Required Audit Criteria Systems Resources Attribute: The Program meets the needs or the workplace parties in the Province of Ontario ( cont d) The timeline to implement and train employees for the changes required after Phase 1 is generally insufficient. Many organizations do not have resources to dedicate to developing and implementing the health and safety program in the six (6) month timeframe. There is heavy emphasis on documentation versus observing operations/practices and looking at compliance in action. If a firm does not score points for Documentation then Observation and Interview are not evaluated. The province is presently experiencing a dynamic environment related to workplace health and safety due to the introduction of Bill 160, the WSIB Funding Review and the changing WSIB mandate. Ontario workplaces do not have access to an integrated approach for prevention and health and safety programs. Our audit indicates that, notwithstanding confusion as to the existing mandate of the Workwell program, Workwell in its current form no longer aligns to WSIB s mandate. The current system is a series of initiatives, including Workwell, and organizations which have limited involvement with each other and do not have standardized expectations for the types of health and safety systems which should be in place to reduce risks, safeguard employees and protect the interests of WSIB premium paying employers. Outcomes 48 WSIB - Workwell Program Value for Money Audit

49 Interjurisdictional Research Other workplace health and safety prevention and compliance programs 49

50 Interjurisdictional research A review of the practices that other workers compensation boards have around compliance, prevention and incentives has identified several interesting differences amongst programs. Other programs are voluntary and offer rebates as a reward for completing a series of requirements that award successful firms with a Certificate of Recognition (COR). These programs use industry partners for firm audits, required to gain a COR. There is a high degree of collaboration amongst industry partners, other stakeholders, and COR programs. In line with Workwell s approach British Columbia and Alberta have modified programs for small companies. Alberta A Certificate of Recognition (COR) is a document issued jointly by a Certifying Partner (e.g., ENFORM, Construction Safety Association, Manufacturers Health and Safety Association, and Human Resources and Employment, Workplace Partnerships.) The COR recognizes that an employer s health and safety management system has been evaluated by a certified auditor and has been found to meet the Partnerships' standard. The Partners in Injury Reduction (PIR) is completed in order to attain COR. The program encourages the development of effective workplace health, safety and disability management programs. Achieving & maintaining a valid COR is required for earning refunds from WCB if registered in the WCB PIR Program. Through this voluntary program, the Alberta Workers Compensation Board (WCB) works with Alberta Employment and Immigration, industry partners, safety associations, employers and labour groups to offer WCB premium incentives. Employers who reduce their claim costs below predicted targets and achieve a COR can earn up to 20 per cent off their industry rate. In order to qualify for COR an audit must be carried out by a certified auditor. An audit covers the basic elements of a health and safety system, personal interviews, documentation review, and work place observation. To acquire COR, you must receive 80% overall mark without less than 50% in any one element. SECOR : The Small Employer Certificate of Recognition (SECOR) program provides an option for employers with 10 or less employees to develop a health and safety management system and achieve a Certificate of Recognition (COR) referred to as SECOR (Small Employer Certificate of Recognition). This program is specifically designed to meet the training needs of smaller employers. British Columbia Manitoba Partners in Injury and Disability Prevention Program offers incentives to employers who work with a Certifying Partner to go beyond the legal requirements of the Workers Compensation Act and the Occupational Health and Safety Regulation and take a best practices approach to implementing health, safety, RTW management systems. After passing an audit, employers receive a Certification of Recognition (COR) and become eligible to receive an incentive payment. Annual incentive payments are awarded to eligible employers who have earned a certificate of recognition (COR) in one or more of the following areas: Health and Safety Management = 10% incentive Injury management/ RTW system = an additional 5% incentive Extremely detailed and sophisticated statistics reporting. To obtain COR accreditation, an employer must undergo a safety training program, develop a company safety manual, develop and implement a health and safety program, and pass an independent safety audit. COR certification is administered by the Manitoba Heavy Construction Association Work Safely Program and the Construction Safety Association of Manitoba. The initiative is self-funded by all construction employers in Manitoba. There is no additional cost to employers in other industries. The Manitoba Government and the WCB have joined with partners in labour and the business community in a comprehensive injury prevention program under the banner of SAFE Manitoba. The joint prevention initiative grew out of recommendations from the Minister of Labour's Review Committee on Workplace Safety and Health. 50 WSIB - Workwell Program Value for Money Audit

51 Internal Consultations/ Documentation Review Summary of Internal Consultations Conducted and depiction of the Types of Documents Reviewed as part of the Audit 51

52 Stakeholder consultations The following tables summarizes the type of stakeholder consultations that have been conducted in order to augment the audit opinion findings and conclusions. The themes noted during discussions are a high level summary of views expressed by the stakeholders during the sessions while responding to a series of questions related to the Workwell program and Bill 160. Group Format Themes noted during consultations Executives Individual Interviews (x1) Shift responsibility for performing the Workwell audits to the Health and Safety Associations. This would enable the Associations to expand their footprint in the employer community and integrate, where possible, with their existing programs. Employers would be audited by their respective association which may reduce complaints that Workwell Evaluators do not have background or understanding of the relevant business/industry sector. Steering Committee Meetings (x7) N/A Internal Program Management Individual Interviews (numerous discussions throughout the engagement) Retain Workwell within the WSIB as a risk and compliance type program. The program delivery, communications and execution would be enhanced to improve the overall customer experience. Program Staff (Workwell Evaluator Team Meeting December 5, 2011, Hamilton, ON) Focus Group (x1) Retain Workwell within the WSIB as a risk and compliance type program. The program delivery, communications and execution would be enhanced to improve the overall customer experience. Data Analytics Staff Numerous meetings and follow-up throughout the engagement N/A 52 WSIB - Workwell Program Value for Money Audit

53 Stakeholder consultations (cont d) Group Format Themes noted during consultations Employer and/or Umbrella Groups Focus Group(x1) Leverage the previous WSIB efforts and implement an incentivized accreditation type program including employer self-assessments and sample based testing by certified examiners to provide governance and oversight. Eliminate Workwell and continue with Ministry of Labour inspections to provide enforcement of health and safety requirements within Ontario s workplaces. External Worker/labour Groups Health and Safety Associations Focus Group (x1) Focus Group (x1) Retain Workwell within the WSIB as a risk and compliance type program. The program delivery, communications and execution would be enhanced to improve the overall customer experience. Increase the annual number of employers which undergo Workwell. Retain Workwell within the WSIB as a risk and compliance type program. The program delivery, communications and execution would be enhanced to improve the overall customer experience. Government Individual Interviews (x1) Implement the leading practices from the Workwell program into the new continuum of compliance prevention strategy. The leading practices would form the Chief Prevention Officer s basis for various levels and types of employer interventions between prevention activities and enforcement measures. 53 WSIB - Workwell Program Value for Money Audit

54 Documentation review The following tables shows the types of documents that have been reviewed as part of the Audit in order to augment the audit opinion findings and conclusions. Workwell Role, Responsibilities, Policies and Process Details External and Internal Focus Groups, Research and Analysis Relevant Statistical Data and Information Other Workwell Evaluator Job Description Workwell Manager Job Description 2011 Organization Chart at November 11, 2011 Workwell Evaluation Process ISO19011 October 25, 2011 Policy Workwell Program Workwell Health and Safety Core Audit Tool Workwell Case Information System User Guide 2010 and 2011 Workwell Team Meeting agendas Workwell Large Firm Audit Procedure Workwell Findings Committee Procedure Workwell Evaluation and Risk Management Plan files Workwell Evaluator Orientation/Training Program Survey of Employers Undergoing First Workwell Audit Final Report for Pre and Post Audit Surveys January 4, 2011 Firm Selection Model Presentations for Table Differentiating the Roles of the Workwell Evaluator and MOL Inspector Understanding Workwell Core Fall 2011 (presentation to selected employers) Health and Safety Associations Consulting Services Jurisdictional Scan for Workwell Equivalents Workwell Budgets and Actuals Analysis for Listing of Workwell Appeals for Employer Audit Services 2012 Audit Plan Fact Sheet Workwell Evaluator s Activity Summary for Data analytics: Firms Meeting WW Criteria, Firms Notified Only, Firms 1st Audit Pass, Firms 2nd Audit Pass, Firms 2nd Audit Fail, Firms RMP for (as available) 2010 IHSA, PSHSA, WSN, WSPS High Engaged Cohort 2010 MOL Engaged Cohort Derived FTE, LTI count, NLTI count, LTI Rate, NLTI Rate, Traumatic Fatality Count, Lower Back Claim Count, Shoulder Claim Count, Fracture Claim Count, % of Claims off Benefits at 3 months, % of Claims off Benefits at 6 months, % of Claims off Benefits at 12 months for injury years Bill 160 Chief Prevention Officer roles and responsibilities Workwell Presentation to Chief Prevention Officer Tony Dean Report WSIB Funding Review Background material 54 WSIB - Workwell Program Value for Money Audit

55 Appendix 55 WSIB - Workwell Program Value for Money Audit

56 2006 Cohort 56 WSIB - Workwell Program Value for Money Audit

57 2006 Cohort Results Summary By Cohort Cohort Performance: LTI Rate RMP Audit 2006 No Interaction (Met Criteria) 2006 Notified only Schedule 1 - All Firms % change: Interaction incident yr st Audit Pass 73% 2nd Audit Fail 73% 2nd Audit Pass 79% RMP 84% Notified only 82% Average - WW interaction 79% No Interaction 63% Schedule 1 37% 57 2nd Audit Pass 11% 1st Audit Pass 3% Firms per WW Cohort 2nd Audit Fail 2% RMP 13% Notified only 18% Met Criteria - No Interaction 53% WSIB - Workwell Program Value for Money Audit Firms undergoing RMP showed the highest percentage of decrease in LTI Rate, however it cannot be determined what proportion of the decrease is due to the notification versus the actual RMP itself Compared to Schedule 1 firms, which reflect the overall societal trend to be more safety conscious, WW appears to have an impact on decreasing LTI rates in the year directly proceeding poor performance. However, due to the fact that firms with poor performance improve back to the overall societal trend in the same fashion that WW firms do, it cannot be determined that WW improves performance more than firms would on their own. This is particularly so when considering that WW firms have larger room for improvement as their performance is worse than those with no interaction.

58 2007 Cohort 58 WSIB - Workwell Program Value for Money Audit

59 2007 Cohort Results Summary Cohort Performance: LTI Rate RMP Audit 2007 Notified only 2007 No Interaction (Met Criteria) Schedule 1 - All Firms % change: Interaction incident yr st Audit Pass 80% 2nd Audit Fail 47% 2nd Audit Pass 58% RMP 94% Notified only 88% Average - WW interaction 82% No Interaction 62% Schedule 1 29% Firms per WW Cohort Notified only 52% Met Criteria - No Interaction 35% 1st Audit Pass 1% 2nd Audit Pass 4% 2nd Audit RMP Fail 7% 1% RMP firms showed the highest percentage decrease in LTI Rate, followed by Notified Only firms. RMP firms show the most sustainable results of those with WW interaction 52% of firms were notified only, marking an increase in the use of this method compared to 2006 Overall, 2007 Cohort results are less sustained than 2006, with LTI rates increasing again in WSIB - Workwell Program Value for Money Audit

60 2008 Cohort 60 WSIB - Workwell Program Value for Money Audit

61 2008 Cohort Results Summary Cohort Performance : LTI Rate Audit 2008 Notified only Schedule 1 - All Firms 2008 RMP Interaction 1st Audit Pass 2nd Audit Fail 2nd Audit Pass % change: incident yr % 76% 54% RMP 93% Notified 93% only Average - WW 86% interaction No Interaction - Schedule 1 29% 2nd Audit Fail 11% RMP 10% 2nd Audit Pass 2% 1st Audit Pass 3% Firms per WW Cohort Notified only 74% WW targeted 2007 cohort firms in the 2008 cohort year due to the legal issues regarding the selection criteria. As such, a greater number of small firms were included and no firms that met the criteria were left without WW interaction. RMP and Notified Only were the Cohorts with the greatest percentage decrease in LTI Rates. However, as with the 2007 cohort, results for the notified only cohort were not as sustainable as RMP and Audit, rising again in WSIB - Workwell Program Value for Money Audit

62 2009 Cohort 62 WSIB - Workwell Program Value for Money Audit

63 2009 Cohort Results Summary Cohort Performance: LTI Rate Audit 2009 Notified only 2009 No Interaction (Met Criteria) Schedule 1 - All Firms 2009 RMP Interaction 1st Audit Pass 2nd Audit Fail 2nd Audit Pass % change: incident yr % 51% 63% RMP 68% Notified only Average - WW interaction No Interaction 68% 63% 68% Schedule 1 26% Firms per WW Cohort 1st Audit Pass 11% 2nd Audit Pass 23% Notified only 49% 2nd Audit Fail 3% RMP 2% Met Criteria - No Interaction 12% The 2009 Cohort experienced the greatest decrease in LTI Rate from firms which passed the first WW audit. The 2009 Cohort also had 92% of firms with over 20 FTE. 49% of firms were notified only, however, there is no increase in LTI rates following the notification year as with other cohorts. This more sustainable result could be due to the fact that only 14% of small firms were notified only in 2009 compared 30-50% in the cohorts. This may indicate that notifications have the most sustainable effect for larger companies. 63 WSIB - Workwell Program Value for Money Audit

64 2010 Cohort 64 WSIB - Workwell Program Value for Money Audit

65 2010 Cohort Results Summary Cohort Performance: LTI Rate Audit 2010 RMP 2010 Notified only 2010 No Interaction (Met Criteria) Schedule 1 - All Firms Interaction 1st Audit Pass 2nd Audit Fail 2nd Audit Pass % change: incident yr % 54% 51% RMP 62% Notified only Average - WW interaction No Interaction 67% 58% 53% Schedule 1 24% Firms per WW Cohort No Interaction (Met Criteria) 65% 1st Audit Pass 7% 2nd Audit Fail 1% 2nd Audit Pass 12% Similar to the 2009 cohort, the 2010 cohort is composed mostly of firms with over 20 FTE, however, there is a great deal of variability in LTI rates. Compared to previous cohorts, a greater degree of variability in LTI rates, particularly for firms that underwent RMP and Audit occurred in Notified only 12% WSIB - Workwell Program Value for Money Audit RMP 3%

66 66 WSIB - Workwell Program Value for Money Audit