January 10, 2017 Frank Pokrop BD Becton Dickinson and Co.

Transcription

1 January 10, 2017 Frank Pokrop BD Becton Dickinson and Co.

2 AGENDA 1. Welcome 2. Introductions 3. As We Begin 4. Goals for Today s Talk 5. Why ASQ is Important 6. What is Auditing and Basics? 7. When is an Audit Effective? 8. Measuring Audit Criteria 9. How Many Types of Audits are There? 10. The Science 11. The Art 12. The Value of an Independent Eye, or How to Use Your Auditors 13. Risk 14. Becoming an auditor 15. The Value of a Corporate Audit Function

3 W h a t i s Yo u r V i e w o f A u d i t i n g?

4 3. As We Begin: Thank You! Today s Presentation ASQ Test Prep? About me Q&A At Any Time About You A Few Questions

5

6 S AN D I E G O ASQ - Auditing Science, Art and More 4. Goals for Today s Talk: a. Education, hopefully b. Frank discussion c. Identify creative and constructive uses of audits and auditor assignments d. See the role that risk now plays in our lives: i. Cost and product impact ii. iii. Patient and user Selection of audit sites and target

7 S AN D I E G O ASQ - Auditing Science, Art and More 4. Goals for Today s Talk: e. Understanding: i. VUCA (Volatility, Uncertainty, Complexity, and Ambiguity) ii. Risk Based Thinking f. Risk-Based Thinking i. New Topic, or, ISO 9001: 2015 ISO 13485: 2016?

8 S AN D I E G O ASQ - Auditing Science, Art and More 4. Goals for Today s Talk: g) The value of Corporate auditing or just independence?

9 S AN D I E G O A S Q - Auditing Science, Art and More 5. Why ASQ is Important a. Personal and professional growth b. Value to you, your family and profession c. Learning and insights into many things, processes, companies, issues, etc.

10 S AN D I E G O A S Q - Auditing Science, Art and More 6. What is Auditing, and basics? "Internal auditing is an independent, objective assurance and consulting activity designed to add value and improve an organization's operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes."

11 6. What is Auditing? a. Many, many good definitions b. Walks like a duck. c. Determines state of compliance with rules, regulations, standards, industry-wide tests. An expected level of performance d. ASQ / ISO defined elements formal, announced, sponsor, etc. e. Formal comparison against internal expectations, external requirements and customer needs f. Non-standard, special request type of examination

12 S AN D I E G O A S Q - Auditing Science, Art and More The Goal Of An Audit: - To Collect Objective Evidence - To Permit An Informed Judgment About The Status Of The Systems or Product Being Audited

13 Remember that Audits Have Formal Parts, Formal Requirements and a Formal Process ISO 19011:2011 Guidelines for auditing management systems (see handouts) Phases of an audit: 1. Audit preparation 2. Audit performance 3. Audit reporting 4. Audit follow-up and closure

14 A u d i t P r o g r a m / A u d i t L i f e c y c l e F l o w C h a r t Develop and approve audit schedule Measure audit effectiveness Develop resource plan Write discrete audit plan Perform audit: report to functional mgmt. Functional mgmt. takes ownership of CAPA response Enter nonconformities into CAPA system

15 Audit Findings: (6.4.7) Audit evidence should be evaluated against the audit criteria in order to determine audit findings. Audit findings can indicate conformity or nonconformity with audit criteria. When specified by the audit plan, individual audit findings should include conformity and good practices along with their supporting evidence, opportunities for improvement, and any recommendations to the auditee. Nonconformities and their supporting audit evidence should be recorded. Nonconformities may be graded. They should be reviewed with the auditee in order to obtain acknowledgement that the audit evidence is accurate, and that the nonconformities are understood. Every attempt should be made to resolve any diverging opinions concerning the audit evidence or findings, and unresolved points should be recorded.

16 Audit Findings: A well-written audit finding provides the auditee with enough information to act on and to correct or prevent a reoccurrence.

17 The Audit Report: 1. Cover page or logistics header: contextual information that characterizes the report. 2. Introduction and background: information regarding the purpose and scope that was contained in either the audit plan or notification letter. 3. Results and summary: the overall conclusions and recommendations. 4. Detailed results: a listing of all findings. 5. Appendix: supplemental information that might explain or clarify the information contained in the report.

18 Writing an audit report + ranking a. Clear, consistent, spell-checked, formatted, quantitative, qualitative, respectful, thankful, reviewed (hint: independent eye) b. Lists the site, location, dates, auditors, type of audit, scope and other audit formalities c. Show the conclusions, ranking, number of observations by different severities, due date for responses and corrective actions

19 Writing an audit report + ranking d. Cites the as-found condition including qualitative and quantitative observations and supporting data e. Compares the finding against one or more standards, rules or regulations f. Ranks the severity

20 Risk might be used in (see handouts): Audit observations Audit reports Site conclusions Scheduling Priority

21 QS ASSESSMENT SCORECARD (Risk) High LOW Criteria Quality System: Change X Influence & Deployment X Harmonization Status (BQSR Matrix) X Product: Risk Management (Clinical Significance) X Changes to Product X Regulatory Status: Recall Classification Functional Product Issues Not Addressed: (Production Processes and Design Controls) Organizational Effectiveness: Organizational Support X Quality Organization X Capabilities: Manufacturing (N/A) Engineering (N/A) Supplier Management (N/A) Business Influences: Growth X New Products X Relocating X Expansion X Low Profit X Improvement Process: Quality Organization Capability X Improvement Since Last Audit (N/A) Compliance Culture X

22

23 7. When is an Audit Effective? a. When the president gets the report i. Who decides who sees the report? b. When change takes place? c. When compliance is increased? d. When enforcement takes place? e. When recalls decrease? f. Role of the customer? g. Remember to measure every aspect of an audit

24 When is an Audit Effective? a. When the president gets the report i. Who decides who sees the report? b. When change takes place? c. When compliance is increased? d. When enforcement takes place? e. When recalls decrease? f. Role of the customer? g. Remember to measure every aspect of an audit - Dynamic views. - Changes with leadership - Changes with time - Cost measures - Compliance status - Audit frequency changes - Company goals - Insurance and liability - Specific observations are resolved, or..

25 8. Remember to measure every aspect of an audit see handouts Audits / year Observations: number by site, by product, by category, by severity, type, trends, time to close, number- corporate, year-to-year comparisons Sites audited / Sites not audited Frequency Re-scheduled or cancelled Risk rankings by.

26

27 S AN D IEGO A SQ

28 9. How Many Types of Audits are There? Internal External Compliance Enforcement (FDA, DEA) Complaints R&D Project System Bottom-up Top-down System Product Process Management Agency Approval Agency Approval spec limits Directed Requested Lifecycle Utility Product returns Gov t document Country Process Deviation Aviation failure IT Installation and Repair Field Service Training Vendor Service Third-party contractor Raw material Laboratory?

29 10. The Science (consistent, scientific approach) Phases of Auditing Planning and Preparing for the audit Execution of the audit plan Reporting the audit results Close out of corrective actions Scheduling: Site Product Division Corporate The Visible Part of Auditing: Opening Meeting Collection of Information Record and Grade Nonconformances Evaluation of Number and Significance of Nonconformances Assessment of Compliance to Requirements Preparation of Findings Closing Meeting Review

30 10. The Science as an Auditor: a. Scientific background b. Detailed c. Inquisitive nature d. Consistent approach e. Understands technology, organizations, documentation, standards and how the product works f. Preparation is the key to an effective audit

31 11. The Art: a. Question: how to be effective in every audit? b. Answers: i. Preparation + team input + consistency ii. iii. iv. Not using the same approach too often Team and individual review before starting an audit Alternating duties and roles (leader, auditor, site) v. Changing the focus vi. Changing the makeup of the auditing team i. Guest auditor (development role) ii. Guest expert

32 11. The Art questions: a. What hasn t been audited at this site? b. Changing your starting points c. Reviewing external data d. Watching the practices and processes e. Developing new or additional questions for a specific audit f. Value in the writing the other audit report

33 12. The Value of an Independent Eye, or How to Use Your Auditors:

34 12. The Value of an Independent Eye, or How to Use Your Auditors: a. Standards numerous examples b. System: a. Hardware/electrical - reviewing engineering runs b. Paper-based system examinations c. Benchmarking evaluating weaknesses d. Site audit preparation

35 The Growth of Standards audit = 757 standards audit checklist = 12 standards risk = 2, 525 standards quality audit = 108 standards compliance = 3,533 standards compliance audit = 14 standards

36 12. The Value of an Independent Eye, or How to Use Your Auditors: a. Aviation examples b. Medical device examples c. Examples of cars d. Your examples

37 13. Risk: a. See handouts for examples b. ISO 9001/ ISO (RBT) i. Risk in every activity ii. Risk of not performing an activity correctly must be known (documented too?) iii. Risk in various audit activities: site, plant corporate??? What else? iv. How have you documented this?

38 14. Becoming an auditor Curious Disciplined Understands science Structured Can shift between qualitative and quantitative topics Some industry experience Can build rapport but not friendship Familiar with auditing tools and can shift gears Schedule

39 15. The Value of a Corporate Audit Function From the FDA s Case for Quality report. Relevance & value: - Significant industry input - Moderated by McKinsey - Qualitative and quantitative Firms with an independent corporate auditing function have a better compliance profile than those companies that don t

40 S AN D IEGO ASQ T h a n k Yo u! Frank Pokrop CareFusion, Inc. frank.pokrop@bd.com (858)