LIFE SCIENCE BENCHMARKING STUDY. Trends and Best Practices Based on Our Work with the US FDA and Over 250 Companies

Transcription

1 LIFE SCIENCE 01 BENCHMARKING STUDY Trends and Best Practices Based on Our Work with the US FDA and Over 0 Companies

2 SUMMARY In this report, UL EduNeering has identified the 01/01 compliance and business performance trends among organizations within the Life Science industry. We rely on several sources for this information. First, we leverage the results of our annual Quality and Compliance Learning Trends survey, conducted with clients in December 01. Second, we incorporate ongoing feedback from ComplianceWire deployments and related learning management projects. Third, we look at specific annual usage data within ComplianceWire, in which more than 7 million training completions were captured from our clients, which range from global enterprises to small entrepreneurial companies. Finally, we layer in regulatory trends and guidance, much of which is learned during our quarterly partnership meetings with the US FDA. This benchmarking study, Learning Trends and Best Practices, has been conducted since 008, and we believe these findings can help shape our clients best practices for the coming year. The trends uncovered in this report deliver insight into the state of compliance training and qualification programs. Many companies have advanced along a quality-focused learning maturity model, in which the emphasis on check the box compliance has shifted to gathering data that demonstrates an impact on learners behavior, and thereby, quality and performance metrics; The most-often cited learning priority for 01 is Measuring Training Effectiveness, which provides an opportunity for training organizations to demonstrate relevance to their business partners; The impact of global regulators has resulted in increased numbers of inspections, the drive to improve audit readiness, and the need to align training inputs to measurable outputs.

3 TRAINING EFFECTIVENESS, METRICS AND CONTINUOUS IMPROVEMENT: MAKING THE CONNECTION In our 01 survey, Measuring Training Effectiveness and Collecting Training Data to Support the Organization s Quality or Compliance Metrics were the two most cited learning priorities for Most Cited Learning Priorities Change (+/-) Measuring training effectiveness Collecting training data to support the organization s quality or compliance metrics Responding to inspections from global regulatory agencies Priority 01 Priority Improving SOP and policy management 6-6 Adding risk-based approaches to training programs The top two priorities of our clients coincide with the push from both FDA s two key centers for pharmaceutical and medical device oversight, Center for Devices and Radiological Health (CDRH) and Center for Drug Evaluation and Research (CDER). Officials from CDER and CDRH announced programs to review an organization s quality metrics as a direct determinant of risk and need for inspection. QA teams have long known that training programs can directly influence quality metrics, but based on our survey results, QA teams are seeking more formal methods to document training effectiveness following high-risk training activities. Based on our conversations with QA executives, the only real barrier seems to be the time and resources required to map out, execute and document these activities.

4 WHAT DO LEARNING PRIORITY TRENDS TELL US? In reviewing the survey data for the past three years, it s interesting to note the priority shift from SOP management to learning effectiveness as an example of how companies are moving from check the box compliance to measuring individual knowledge. Measuring training effectiveness Learning Priorities Collecting training data to support the organization s quality or compliance metrics Responding to inspections from global regulatory agencies Improving SOP and policy management We believe that many clients are advancing along the Quality-Based Learning Maturity Model, which we developed by studying the progression of our long-term clients as they began using their learning management system to achieve quality and compliance objectives. The Quality-Focused Learning Maturity Model 1 TRAINING RECORD MANAGEMENT SOP ROLE-BASED MANAGEMENT TRAINING EFFECTIVENESS CTQ METRICS/CULTURE BUSINESS OUTCOME LMS Audit Readiness Validated System Defined Curricula Job-Based Qualifications OJT Checklists Learner Assessments OOS, CAPA, Complaints Continuous Improvement Client Satisfaction Production Output 1 Goals: Many companies use an LMS to meet core compliance and quality regulations, and move records from paper to electronic record form to gain efficiencies and reduce administrative effort. Goals: ComplianceWire subscribers in year 1 have the ability to align SOPs to role-based curricula, thus being able to build qualifications around job roles, satisfy global regulatory concerns and provide visibility for both production-line managers and senior executives. Goals: Clients that reach this stage have met their record management and SOP training management needs. They are now seeking to measure training impact and identify knowledge gaps within the role-based training programs they have developed. It s at this stage that many clients formalize a training effectiveness cycle building measurable and continuous On-the-Job Training programs that include assessments, mentoring programs, and other activities that are aligned with high-risk operations. In fact, in our benchmarking survey, we asked how clients are measuring training effectiveness, and 61 indicated that quality metrics would demonstrate that training is working. In 01, during research about training effectiveness, one QA Director shed some light into this correlation: If the production team is hitting our quality metrics, then training must be effective. What we have seen is that QA teams can document the effectiveness directly into an employee s transcript by observing that the individual performed the function correctly using a checklist developed in our Forms tool. Currently, only 1 of our clients are using this approach, but many QA executives agree that this observable assessment can lead to a reduction in product errors. Method Used Regular Review of Quality Metrics Quizzes (via QuizCreator tool) No Formal Measurement System in Place Documenting On-site Observations via Forms tool 1 of Respondents 61

5 Goals: At this stage, clients are now building curricula that align with continuous improvement metrics, with the goal to demonstrate the value of training on performance and behavior. For QA teams that have reached this stage, providing quality metrics and showing positive trends in quality can help shape the organization s quality culture, the company behaviors or attributes that can lead to positive, predictable quality metrics. For example, at a recent PDA Quality Metrics Conference, QA executives identified Communication and Transparency as having the most positive impact on quality culture. To reach this stage, QA teams are often working in crossfunctional teams to align training to key performance indicators within manufacturing and operational areas, and then capture that data and share with all employees, in addition to senior management. In our benchmarking survey, we asked what training activities have been aligned with Critical-to-Quality (CTQ) initiatives, and not surprisingly, CAPA Initiations were most cited. Training Activities Aligned with CTQ Initiatives CAPA Initiations Audit/Inspection Findings Adverse Events Customer Complaints OOS (or Nonconforming product) Right First Time (Defects) Goals: Increasingly, companies are interested in leveraging training data to demonstrate value of quality management to business growth: reduced manufacturing cycle time, overall equipment effectiveness, and other business and financial metrics. This requires an organization to fully embrace a culture of quality versus compliance. The State of Global Audit & Inspection Readiness In our annual study, we also ask about our clients state of audit readiness and how ComplianceWire provides training and qualification data during audits and inspections. As found over the past few years, the Internal Audit Team was cited most often (87) as the organization conducting the audit. And as we ve noted, aligning training to audit findings and comparing those findings three or six months after the audit is considered a best practice. Just as the industry is experiencing more regulatory investigations, 77 of respondents indicated that their company had experienced an FDA inspection, a 0 jump over 01. This increase in inspections corresponds with FDA s continued efforts to align with global agencies on pilot programs to share audit information. Our clients have also told us that FDA is staying longer, digging deeper and questioning learning approaches that rely on read and understand SOPs. AUDIT BODY US FDA ISO OSHA

6 CONTENT THAT BUILDS A CULTURE OF QUALITY In our discussions with QA executives, many shared the need to provide industry context within training programs to production workers. During a recent PDA Quality Metrics conference, one QA executive summed up the need to engage employees: Employees can t be held accountable if they don t have the knowledge. That s why many QA teams assign UL EduNeering s standard, -minute online courses as part of a GMP orientation program, and also as part of an ongoing development program. When a QA team delivers best practices and regulatory content as part of the training program, the program is elevated above the specific role procedure for the employee: 1) Courses demonstrate to production workers that management has invested in their professional growth, improving both engagement and morale; ) Courses encourage production workers to understand the consequences of their behavior and to think critically, from both a patient safety and regulatory perspective, thus promoting a culture of quality. With 700 standard regulatory courses in our library, we report annually on those with the most usage. This data provides guidance to our clients as they develop their training matrices. UL EduNeering Course Usage: RANK COURSE TITLE RANK COURSE TITLE RANK COURSE TITLE 1 Handling an FDA Inspection 1 Orientation to GMP Compliance 1 Hazard Communication Introduction to GMPs Principles of Good Documentation Change Control Orientation to GMP Compliance GMP Principles of SOPs GxPs Part 11: Electronic Records Electronic Signatures DEA Compliance ICH Q7A: Introduction and Quality Management Understanding GMPs for Facilities and Equipment Meeting GMP Training Requirements Principles of Good Auditing Principles of Good Documentation Handling an FDA Inspection Introduction to the Quality System Regulation (QSR) Part 11: Electronic Records Electronic Signatures Design Control Regulations for Medical Device Manufacturers Quality Systems Inspection Technique (QSIT) Bloodborne Pathogens General Industry Slips, Trips, and Falls Personal Protective Equipment Fire Extinguishers Electrical Safety Hearing Conversation Ergonomics: Body Mechanics and Fitness Access to Employee Exposure and Medical Records Lockout/Tagout Affected 6

7 CONCLUSION In 01, we will continue to see clients advance into measurement stages of quality-based and compliance-based learning. Only through measurement activities can QA teams focus on improving the learning experience, while demonstrating to senior management that these activities are having positive outcomes for the organization s quality culture and the business in general. In 01, we will focus on these activities to help our Life Science clients progress along the maturity model: Continue to explore the elements that should make up a best practice multi-modal training effectiveness program, such as a baseline knowledge assessment or SOP review, followed by mentoring activities, videos, post-training assessment, etc. Enhance our assessment (QuizCreator, ExamCreator), OJT (Forms), risk management (SmartForms, SmartCurricula), and reporting tools so that data can be easily analyzed and used when reporting trends related to continuous improvement programs. Develop new content and blended learning programs such as Risk Management/ISO 1971, the culture of quality, and supply chain management to address industry and regulatory issues. Facilitate transparency and knowledge exchange sharing with our key partners including AdvaMed and US FDA, targeting training related to quality programs, such as CDRH s Case for Quality initiative, to share knowledge and expectations around quality data gathering. 7

8 ABOUT UL EDUNEERING UL EduNeering is a business line within UL Life & Health s Business Unit. UL is a premier global independent safety science company that has championed progress for 10 years. Its more than 10,000 professionals are guided by the UL mission to promote safe working and living environments for all people. UL EduNeering develops technology-driven solutions to help organizations mitigate risks, improve business performance and establish qualification and training programs through a proprietary, cloud-based platform, ComplianceWire. For more than 0 years, UL has served corporate and government customers in the Life Science, Health Care, Energy and Industrial sectors. Our global quality and compliance management approach integrates ComplianceWire, training content and advisory services, enabling clients to align learning strategies with their quality and compliance objectives. Since 1999, under a unique partnership with the FDA s Office of Regulatory Affairs (ORA), UL has provided the online training, documentation tracking and 1 CFR Part 11-validated platform for ORA-U, the FDA s virtual university. Additionally, UL maintains exclusive partnerships with leading regulatory and industry trade organizations, including AdvaMed, the Drug Information Association, the Personal Care Products Council and the Duke Clinical Research Institute. 0 Carnegie Center Suite 01 Princeton, NJ UL and the UL logo are trademarks of UL LLC 01. uleduneering.com WP/1/0901/LS