ComplianceWire. 2012/2013 Benchmarking Study

Transcription

1 ComplianceWire 2012/2013 Benchmarking Study

2 ComplianceWire : Life Science 2012/2013 Benchmarking Study Trends and Best Practices Based on the Usage of 250+ Companies Summary Measuring Training to Raise Global Quality Standards In this report, UL EduNeering has identified the 2013 compliance and business performance trends among organizations within the Life Science industry. We rely on several sources for this information. First, we leverage the results of our annual Quality and Compliance Learning Trends survey, conducted with clients in December Second, we leverage ongoing feedback from ComplianceWire deployments and related learning management projects conducted during Finally, we look at specific annual usage data within ComplianceWire, in which more than 22 million training completions were captured from more than 250 Life Science organizations, ranging from global enterprises to small entrepreneurial companies. These statistics will help shape the focus of Life Science companies around the globe. This benchmarking study, Learning Trends and Best Practices, has been conducted since 2008, and we believe these findings can help shape our clients best practices for the coming year. The trends uncovered in this report deliver insight into the state of compliance training and qualifications programs: The highest priority for clients going forward is to effectively leverage training data to support performance and compliance metrics; Increased globalization and expanding supplier networks have put more pressure on quality and compliance teams to implement consistency and standardization into their learning and compliance programs; Taking a risk-based approach to training and qualification remains a key goal for 2013, a trend we first noted in Page 2

3 Priority Training Topics: Expectations for 2013 To identify key compliance and performance improvement training topics, we conducted our annual survey to Life Science clients around the globe, focusing on their experiences in 2012, and forecasting their priorities for key topics Collecting training data to support the organization s quality or compliance metrics Improving SOP/policy training management 2012/2013 TOP TRAINING TOPICS Measuring training effectiveness to ensure that employees are applying knowledge/skills on the job Conducting more role-based training and credentialing programs Responding to inspections from global regulatory agencies Adding risk-based approaches to training programs Improving third party compliance through training programs Conducting more synchronous web activities and reducing live events to reduce costs 0% Priority #1: Training Data as a Source for Business Metrics Many of our clients employ compliance and performance metrics, ranging from basic key performance indicators (KPIs) to using more advanced business intelligence data gathering and assessment programs. Our survey results reveal that these clients are indeed leveraging training records as part of the data collection process for their measurement programs. In fact, 67% of respondents cited collecting training data to support the organization's quality or compliance metrics as a top priority for Employees in Compliance (based on completions) Audit/Inspection Findings Employee Performance Reviews METRICS SUPPORTED BY TRAINING DATA CAPA Initiations Complaint Management Priority Training Topics: Expectations for Page 3 Priority #1: Training Data as a Source for Business Metrics...3 Priority #2: Improving SOP Management...4 Priority #3: Training Effectiveness... 4 Global Audit & Inspection Readiness... 6 Addressing Global Regulatory Convergence... 6 Top Training-Related Audit Observation: Inadequate or Lack of Documentation... 7 Sharp Rise in Global Usage...8 Most Popular Life Science Courses... 9 Expanding Usage: Leadership & Development Conclusion Adverse Events Product Defects (Non-Conformance) Supplier Performance (Scorecard) 0% Page 3

4 We drilled down to learn how clients were leveraging that training data. Employees in Compliance, an actual training metric that ComplianceWire provides through several reports, including the ComplianceWire Dashboard, was the most commonly cited metric. This finding was not surprising, as it is highlighted as a best practice to employ during each program deployment. But a more important finding, and a best practice, is the increased use of ComplianceWire for measuring and managing KPIs such as audit inspection findings, CAPA activity, and customer complaint management. The ComplianceWire Dashboard provides a real-time snapshot of your organization s compliance status based on ComplianceWire completions. You define the dashboard s at-risk user groups or organization nodes. Armed with this data, senior management can root out high-risk compliance areas and drill down to view performance indicators. Best Practice: Leverage training records as part of the data collection process for your measurement programs. For example, ComplianceWire has been used to distribute and track CAPA Alerts, in which training assignments are aligned with CAPA initiations. This enables the Quality organization to conduct a full CAPA education loop, from the time the CAPA is first reported to the time the correction is made, with targeted training on that CAPA delivered to the proper roles within the organization. This aligns with FDA CDRH reports that have stated that organizations that properly monitor and capture field usage feedback are able to improve product quality and the processes that help build a quality culture within an organization. Another example is targeting training as a corrective measure in response to low Quality KPIs, audit findings, field service reports or customer complaints. Training that addresses the root cause is aligned to those specific roles or performance areas. Pre-testing and post-testing can be employed to measure learning effectiveness and develop a program for refresher training and relevant information updates in a just in time manner. Priority #2: Improving Sop Management The second highest cited priority was Improving SOP Training, which consistently ranks at the top of our survey. ComplianceWire s SOP Management Suite is the most widely used feature, having recorded approximately 15 million control document training completions (SOPs, corporate policies, etc.) in Priority #3: Training Effectiveness Best Practice: Add risk-based approaches to your training programs. Training effectiveness was the third most cited priority in the survey. Quality and learning professionals know that measuring training effectiveness to ensure that employees are applying knowledge/skills on the job is a key goal. They have also told us that training effectiveness is a long-term process that includes observation and other simulation exercises to measure any learner behavior change or productivity improvement. These results can be reflected in audit reports, performance reviews, production output, equipment uptime, complaint management data, etc. A best practice that we have noted is one that aligns with a key finding of this study: the need to add risk-based approaches to training programs. Risk Analysis helps identify which activities are high-risk, typically related to patient or product safety, product quality or compliance with regulatory or company requirements, and thereby warrant effective training activities. Page 4

5 COMPLEXITY HIGH LOW SOP TRAINING EXAMPLE OF A RISK-BASED APPROACH SOP Review with Quiz SOP Read and Understand SOP Review SOP Review with Supervisor Walk-Through SOP Review with Quiz SOP Review, Read and Understand SEVERITY SOP Mentoring with Annual Refresher SOP Review with Annual Refresher SOP Review with Quiz This diagram shows a simplified learning approach aligned to SOPs. The least complex procedures, especially those with a low probability of failure, or low severity if not followed properly, generally require a simple read and understand of that SOP. As the complexity of that SOP increases, and/or the severity related to not following that procedure (as it relates to safety, quality or compliance identified in your risk assessment), the type of training ratchets up proportionately. This approach and best practice aligns with the increased use of assessments and on-the-job training tools within our SOP Management Suite. These tools, Quiz Creator, Exam Creator, Forms and SmartForms are used to assess learners understanding of critical topics, as well as capture first-hand observations that supervisors and training managers are conducting within operations and production environments. The SmartForms tool is especially valuable here, as it facilitates the translation of high-risk activities into training activities by automating the workflow and capturing learner s pre-existing knowledge for placement into high-risk groups. Rules are added to form questions so that a specific response adds the learner to a high risk user group, or removes the learner from a user group. HIGH Metrics allow an entire organization to achieve a common understanding of what outcomes are most valued by leadership. Compliance metrics should focus on effectiveness and measure that the right things are being done correctly. Too often the focus of metrics is on efficiency which often is measured in financial terms. Meaningful Performance Metrics for Compliance Roger Jansak, Director of Quality, Abbott Medical Optics Best Practice: Increase the level of training proportionally to the increasing complexity of the SOP. For 2013, our product strategy team will be introducing new reporting tools that enable clients to extract training records from ComplianceWire as reliable data sources for business metrics. Your company s training data will be available in exportable formats for integration with sophisticated business intelligence tools to enhance your risk-based learning programs and to demonstrate the direct impact of training on safety, quality and compliance. In addition, in 2013 we will expand the ComplianceWire assessment toolkit to support your need to demonstrate learning effectiveness and role-based qualifications. COMPLETIONS COMPARISON, Forms +23% YOY Exam Creator +58% YOY K 200K 300K 400K 500K 600K 0 10K 20K 30K 40K Page 5

6 Global Audit & Inspection Readiness The second goal of our annual study is to report on our clients state of audit readiness and how ComplianceWire was used to support audit requests for training and qualification data. As in prior years, the Internal Audit Team was cited most often (84%) as the organization performing the audit, with slight changes in the inspection frequency by agencies such as the FDA and ISO. Internal Audit Team WHO PERFORMED THE AUDITS IN 2012? FDA Client or Sponsor ISO EMA OSHA 0% Best Practice: Focus on root cause analysis, design quality into the product and optimize effectiveness of corrective actions. Addressing Global Regulatory Convergence Life Science executives have told us that understanding the maze of global regulatory requirements has become an obstacle to growth.* Affirming that statement is a 2012 Axendia report of 125 med-tech executives, called Walking the Global Tightrope. The study revealed that the #1 issue worrying these executives was the increasing complexity and cost of global regulatory requirements, and the most cited threat was changes to the regulatory environment, for both developed and emerging markets. We add to this information the year-long educational program delivered by the FDA s CDRH (Center for Devices and Radiological Health) team, called The Case for Quality, and the successful FDA/EMA shared inspection program for Pharmaceutical and API companies. Both initiatives implore the Life Science industry to focus on good company quality practices that lead to quality outcomes that go beyond compliance. They also encourage companies to view compliance as one part of achieving overall quality, rather than the ultimate goal. Focusing on root cause analysis, designing quality into the product, and optimizing effectiveness of corrective actions are the new benchmarks to success. To dig deeper into how our clients are experiencing both this philosophical shift and the increased global regulatory pressures for consistency and standardization, we asked if 2012 audits were more complex and time consuming than in previous years. *Walking the Global Tightrope, 2012, Axendia, Inc. Page 6

7 AUDIT COMPLEXITY INCREASE IN 2012 Clients were asked: In 2012, were audits of your organization by regulatory agency officials more complex and more time consuming than similar audits conducted in the past two years? Disgree Agree Strongly Agree 22.2% Undecided 20.4% 14.8% 42.6% Agree Undecided/NA Disagree With an almost 50% of respondents agreeing with the increased complexity of agency expectations (and 42.6% unsure at this time), we recommend that companies ready themselves for increased scrutiny in 2013 and beyond. We also cite a best practice of developing consistent and standardized training programs across all internal facilities and for critical external supply chain partners. Best Practice: Develop consistent and standardized training programs across all internal facilities and critical external supply chain partners. Top Training-Related Audit Observation: Inadequate or Lack of Documentation In our survey, 33% of respondents indicated that No training related findings were reported, down from 56% in the prior year. This is the first time we ve seen this number go backward over the past few years. We attribute this troubling trend to the aforementioned strengthening of regulatory audits (clients have reported increases in length and intensity in 2012) and the mantra if it isn t documented, it didn t happen. TRAINING-RELATED AUDIT OBSERVATIONS Inadequate or Lack of Documentation Failure to Follow SOPs or Other Documentation No Training-Related Findings Were Reported Inadequate Employee Training Inadequate Employee Qualifications (Lack of CV Available) Inadequate Change Management Methodology Failure to Investigate Quality Problems Supplier Not Adequately Trained Lack of Adverse Event or Medical Device Reporting Process 0% The increase in training findings correlates with the 260% increase in audit findings for inadequate or lack of documentation and sizeable jump in other documentation-related Page 7

8 issues such as employee qualifications (lack of Curricula Vitae), change management methodology and adverse event data. To support these findings, we performed an analysis of recent 483s focused NUMBER OF 483s ISSUED BY THE US FDA around documentation. FDA Center Total 483s Listed 483s that Specified Procedures or Documentation Percentage of Total 483s A best practice to address these findings is to employ ComplianceWire for full role-based training and credentialing across the enterprise. We recommend Devices Drugs Bioresearch Monitoring Biologics % 50% 46% 6% an annual review of your program, or after a major event such as an acquisition or significant new product launch to assure that FDA Good Documentation procedures are employed, that CVs are current and available within ComplianceWire, and that appropriate SOPs and content are aligned to assure that each employee is qualified to perform the job. ComplianceWire Usage: Sharp Rise in Global Usage Best Practice: Address FDA findings by employing current role-based training and credentialing across your enterprise. In 2012 there was a 46% increase in non-english users in ComplianceWire. As administrators know, ComplianceWire recognizes the browser language setting and automatically displays the system in their preferred language. That s prompted many clients to add languages to accommodate employees and contractors around the world. ComplianceWire currently supports 34 languages. TOP NON-ENGLISH LANGUAGES USED IN COMPLIANCEWIRE IN 2012 Spanish (Latin America) German French (European) Portuguese (Brazil) Chinese (Simplified/Mandarin) Italian Dutch Japanese 0 2.5K 5K 7.5K 10K 12.5K 15K 17.5K Page 8

9 Our Most Popular Life Science Courses UL s 1000 course library of mastery-based e-learning courses (CBTs) is widely used by both the US FDA (contributing author and reviewer of curricula) and our clients. The following is a list of the most popular titles in 2012, based on the number of companies who selected the course and course completions. TOP 20 LIFE SCIENCE COURSES TAKEN IN 2012 PHDV79 A Step-by-Step Approach to Process Validation ISPE02 Approach to Computerized Systems Validation and Compliance PHA39 Packaging and Labeling of Finished Pharmaceuticals PHA38 Introduction to GMPs PHDV77 Key Concepts of Process Validation PHA35 Change Control PHA36 Good Clinical Practices (GCPs) for New Product Investigations FDA31 Part 11: Electronic Records; Electronic Signatures PHDV73 Orientation to GMP Compliance PHDV65 Principles of Good Documentation PHDV63 Understanding GMPs for Facilities and Equipment PHDV74 Handling an FDA Inspection PHA51 Writing Validation Protocols PHDV62 Good Laboratory Practices (GLPs) PHDV75 Essentials of an Effective Calibration Program PHDV69 Principles of Auditing DEV50 A Guide to ISO The Quality Management System for Medical Devices PHDV88 Implementing an Equipment Qualification Program PHA50 Resolving Out Of Specification Test Results GCP01 GCP/ICH Obligations of Sponsors, Monitors, and Investigators Page 9

10 Expanding ComplianceWire Usage: Leadership & Development While most of our clients focus their use of ComplianceWire for quality and compliance programs, we have seen a significant expansion of usage across the enterprise. Our survey addressed this expansion, by asking which learning programs were expected to be introduced via ComplianceWire in TRAINING PROGRAMS EXPECTED TO BE INTRODUCED IN 2013 Leadership Development for Managers Health & Safety Training Sales and Field Service Training IT Training Clinical Research Training (clinical teams, sites, etc.) Desktop Computer Skills (MS Office, etc.) Product Training for External Parties Pharmacovigilance (physician training programs) 0% Our solutions have expanded along with the need: Leadership Development: UL has formed a strategic partnership with CrossKnowledge, a global leader in professional development and leadership training content; Health & Safety Training: UL Workplace Health and Safety, a sister company, is a leader in safety and incident management programs; Sales and Field Service Training: ComplianceWire is integrated with several CRM systems to automate the training of personnel, assuring they have the most current knowledge about the products and procedures for sale or service; Clinical Research: Now entering our second year, our partnership with Duke Clinical Research Institute delivers a unique educational solution for global clinical researchers and associates. Page 10

11 Conclusion: When we gain insight into the learning trends that impact our 250+ Life Science clients, we are in a stronger position to provide quality and compliance best practices that can move learning programs forward. This year s study provided several key takeaways, but in 2013 we believe that more clients will rely on training data to help managers make better decisions about how to best leverage resources to meet their stated goals. We believe that training measurement has grown in usage in response to increased global regulatory scrutiny and complexity, which is also reflected in our survey results. For this reason, UL will be exploring best practices for audit preparation throughout 2013, and plan on sharing other training and qualification best practices. Clearly, the need to standardize role-based training across multiple countries will be a priority for clients, given the significant increase in non-english usage of ComplianceWire in As part of UL, we now have become part of an organization with 50% of our 11,000 employees located outside the US. This will enable us to better support your need for global harmonization and implementation of learning programs to meet complex regulatory requirements. Finally, UL will help clients focus on well-designed performance metrics so that senior managers can better evaluate the performance of their business processes, and also make better strategic decisions that can improve the culture of quality and compliance within their organizations in 2013 and beyond. Thanks to client feedback, our focus will be on the areas of audit readiness and training effectiveness, as these are the high priorities identified for Ellen Leinfuss, SVP Life Science Page 11

12 About UL EduNeering UL EduNeering is a business line within UL Life & Health s Business Unit. UL is a global independent safety science company offering expertise across five key strategic businesses: Life & Health, Product Safety, Environment, Verification Services and Enterprise Services. UL EduNeering develops technology-driven solutions to help organizations mitigate risks, improve business performance and establish qualification and training programs through a proprietary, cloud-based platform, ComplianceWire. For more than 30 years, UL has served corporate and government customers in the Life Science, Health Care, Energy and Industrial sectors. Our global quality and compliance management approach integrates ComplianceWire, training content and advisory services, enabling clients to align learning strategies with their quality and compliance objectives. Since 1999, under a unique partnership with the FDA s Office of Regulatory Affairs (ORA), UL has provided the online training, documentation tracking and 21 CFR Part 11-validated platform for ORA-U, the FDA s virtual university. Additionally, UL maintains exclusive partnerships with leading regulatory and industry trade organizations, including AdvaMed, the Drug Information Association, the Personal Care Products Council and the Duke Clinical Research Institute. 202 Carnegie Center Suite 301 Princeton, NJ UL and the UL logo are trademarks of UL LLC uleduneering.com WP/13/101713/LS