Global Trends in Audits: Demonstrating Training Effectiveness

Transcription

1 Global Trends in Audits: Demonstrating Training Effectiveness UL and the UL logo are trademarks of UL LLC 2013.

2 Agenda Goals for this Discussion Our Panel Industry Audit Trends Our Benchmarking Responses Panel Discussion 2

3 Goals for this Discussion Trends Surrounding Training-Related Audit Observations How Audit Findings Can Help You Identify Training Priorities How to Best Demonstrate Training Effectiveness to Auditors Shifting Training from a Lagging Indicator to a Proactive Indicator

4 Our Panel Ann Piscetelli Training Manager, Johnson & Johnson Quality Systems Sharon Brower Manager, GMP Training, Management Review and Supplier Quality Roche Diane Ramey VP Medicare Consulting and Account Services Solid Benefit Guidance LLC

5 Audit Findings Drive Quality Priorities May 2013 Survey to QA Managers within Life Science Companies: Are you currently using any of the following metrics to monitor production quality and GMPs? CAPA Initiations 77.78% Production Output Goals 68.89% Product Error Rates 68.89% Employees in Compliance 75.56% Employee Performance Ratings 64.44% Audit Findings 93.33%

6 Audits/Inspections Were More Complex and Time Consuming in 2012 Nearly 43% of respondents agreed with statement: Audits are more complex and time-consuming in 2012 than the year before. Expect and Plan for Increased Scrutiny in

7 DOJ Expectations: GMP Focus Deputy Attorney General Speech During 2013 CBI Pharmaceutical Compliance Congress: What is the DOJ looking for? I urge you to also focus on people. People are at the heart of what you do, and it is the failures of people often the combined failures of a number of people which result in noncompliance. Therefore, in our investigations, we are looking at people to determine responsibility. And for this same reason, we urge you to look at people.

8 DOJ Expectations: GMP Focus Deputy Attorney General Speech During 2013 CBI Pharmaceutical Compliance Congress: Are our people and policies working in harmony? Put another way, do our policies acknowledge how real people work and what they are capable of? Quality assurance processes that rely on unrealistic expectations are doomed to fail.

9 CMS Expectations of MAOs From the CMS Medicare Managed Care Manual (Compliance Program Guidelines): Sponsors must audit their operational areas and those of their first tier entities. [Sponsors must] examine the performance of the compliance program, including a review of training, reporting mechanisms 42 C.F.R (b)(4)(vi)(F), (b)(4)(vi)(F)

10 Global Audit and Inspection Readiness Who Conducted the Audit or Investigation? 10% Less Audits from FDA in % More Audits from ISO in 2012

11 Training Related Audit Findings Inadequate or Lack of Documentation most cited training-related finding in 2012

12 Q: What Are The Best Practices You ve Used Audit Findings to Adjust Your Training Programs?

13 Training Effectiveness Role-based Approach Manages the Issues of Over Training and Under Training

14 Q: How Have You Demonstrated Training Effectiveness to Auditors or Inspectors?

15 Metrics a Top Priority Collecting Training Data to Support Metrics was the most cited priority for 2013

16 Training Data and Metrics Which Metrics Were Supported by Training Data?

17 Training Data and Metrics Example: Audit and Inspection Metrics The audit process is a periodic snapshot of performance by an independent inspector. Functional area self-audits offer an excellent opportunity for the most proactive measurement. Source: Meaningful Performance Metrics for Compliance, Roger Jancsak, Abbott Inc. Published in Achieving Quality and Compliance Excellence in Pharmaceuticals, A Master Class GMP Guide, 2012

18 Training Effectiveness Continuous Learning Programs Forms Used to assess baseline knowledge Core content delivered Refresher Courses Post-Assessment to measure impact of learning Continuous Learning to Measure Impact on KPIs

19 Metrics: Key Examples Audit and Inspection: Audit trends and root cause analysis can impact operational performance. CAPA: Distribute and track CAPA alerts, aligning training assignments to CAPA initiations. Proactive Indicators: Use training as a proactive tool for improving a specific business function or operation.

20 Q: How Can an Organization Shift the Perception of Training from a Lagging Indicator to a Proactive Indicator?

21 Additional Questions?

22 THANK YOU