Laboratory Internal Auditor Course

Transcription

1 Laboratory Internal Auditor Course Daren C. Valentine IT Director, A2LA Ph: #1 Rule of Auditing If you cannot express the non-conformance in the words of the management system standard, the appropriate technical specification or test method, or the company s own policies and procedures, THEN YOU DO NOT HAVE A NON-CONFORMITY 1

2 What Is An Audit, Really? 1. Main purpose: to verify compliance 2. Focus: to find evidence of compliance 3. The goal should never be to keep digging until nonconformities are found 4. Auditors should not take pleasure in finding fault! 4 A2LA 2010 QMS in Lab Operations Contact Sampling Reporting Feedback PROCESS CONTROL Quality Control (QC) Preparation Test, Pre - Calibration. Data Validation Post Sales Inspection, Cert. Sales CONTROL OF DATA Quality Assurance (QA) QUALITY MANAGEMENT SYSTEMS (QMS) Includes all functions with emphasis on those that influence the safety of persons and/or the (quality) validity of technical decisions or results All business operations functions All management and administrative functions All finance functions All HR functions All purchasing and contract review functions All marketing and communications functions All maintenance functions Why perform Internal Audits? Internal audits are the most comprehensive method of determining if the management system is working for us: Is it supporting the work of the people in the laboratory? Does it help provide a safe environment? Does it help us produce only technically valid results? Internal audits tell us of our management system is: Implemented; Effective, and Allows for continual improvement. 2

3 Continual Improvement (17025) Clause Governing Reference When the deviation Today Tomorrow?? occurs Formal Action Internal Audit Cycle Planning Implement the Audit Program Based on previous results, frequency, scope, resources, etc. Established by top management and the quality team Agreed by all participants, auditees, auditors Plan for each individual audit occurrence Determine resources, scope, objectives for each audit Each individual audit agreed by all participants, auditees, and auditors. Conduct each Audit Document review, checklists, timings for each audit agreed Onsite - opening meeting, interviews, findings, report, closing meeting. Close out and follow up Auditee action corrective/preventive action Formal close out of findings Follow-up as required, including any further audits System Readiness for Audit Documented system Structured system Implemented system 3

4 Documented System Quality policy document Processes / procedures SOPs / instructions Specifications Standards Codes of practice Regulations Records Structured System Policy WHY Level 1 Procedures WHO WHAT WHEN WHERE Level 2 Work Instructions HOW Level 3 Proof RECORDS Level 4 Implemented System Do people have access to the policies and procedures that govern their work? Do people maintain records that demonstrate their use of the policies and procedures that govern their work? Records indicate the level of implementation of Documents, Processes and Procedures Records provide evidence that a system is being used 4

5 Internal Audit Program Approved by top management and contains: Resource allocations Scope and objectives of audit activities Timelines Audit Area / Period Q Q Q Q Admin Sally George Engines Karen Polymers Bruce Sample Reception / dispatch Jill Developing the Audit Plan See the example on page 14 of Chapter 2. Note the list of requirements on page 15 of Chapter 2 when formally presenting the plan during an opening meeting. Audit Approaches Periodic Department/Section Audits Audit of the whole management system on a department by department schedule Pros More manageable Cons Can miss interfaces between departments 15 A2LA

6 Audit Approaches Horizontal Audits Based upon the periodic audit of a particular system procedure in the management system (e.g. records; calibration; training; test methods) Pros Discover differences in application throughout organization Cons Doesn t address all of the combined inputs which produce the final product or test data. Audit Approaches Test Process/Vertical Audits Selection of a particular test or calibration method, and auditing all of the inputs, operations and activities required to produce the data output Pros Examines the interfaces between all inputs and procedures Cons Misses inconsistent application of procedures between different activities Best Audit Approach Test Report Sample Disposition Data Acquisition Test Methods Equipment Calibration Review of New Work Sample Receipt 6

7 Depth of Audits Full audit of a procedure Partial audit Phased audits Non-conforming work evaluations Root cause investigations Corrective action monitoring Follow-up audits 19 A2LA 2010 Opening Meeting 1. Introductions and thanks 2. Audit scope and objectives 3. Audit plan 4. Audit methods and procedures 5. Confidentiality 6. Respond to questions 7. Depart for the tour of the facility Audit Schedule / Plan 1. Agreement on the planned approach 2. Respond to necessary adjustments to the plan 3. Confirm arrangements for logistics and resources 4. Confirm arrangements and timings for subsequent meetings 5. Confirm arrangements for the end of the audit 7

8 Audit Methods 1. Clearly explain investigation activities 2. Be open about the process and emphasise its transparency Confidentiality 1. Confirm audit team s responsibilities 2. Confirm auditing authority s responsibilities Respond to Questions Be prepared to handle questions from the auditees focus on the benefits of the approach and the transparency of the activities. 8

9 Four Human Relation Aspects Attitudes towards auditing Communication skills Listening skills Reducing tension 25 A2LA 2010 Audit Attitudes Audits are fact-finding missions, not fault-finding safaris focus on verifying compliance are not meant as fodder for auditors should not give the auditor joy when writing deficiencies are meant to improve the quality of the work in your laboratory are just another tool to improve the laboratory s management system 26 Audit Perspectives We have talked about the #1 Rule of Auditing, BUT What do you want to see? What do you think you see? Or what s actually there? 27 A2LA

10 Communication Skills Keep it simple: clear, brief, direct, focused Three ways of receiving information: Seeing, hearing, experiencing Spend most of your time listening Stay in the adult mode Equal, not parent/child or teacher/student Adults: thinking, assessing, proactive, logical, mature Parents/teachers: bossy, authoritarian, instructing, nurturing Children/students: emotional, reactive, impulsive, rebellious Express comments and questions in a positive way It s not just what we say, but how we say it And please use the person s name! Listening Skills Why don t we listen? Get bored and we re ready to say something ourselves We re thinking about our reply while they re talking We re jumping to conclusions or making assumptions Slow down! We think 5 times faster than we talk If we don t listen, we will miss: Important statements The significance of what was said The interactions between the people present during the audit And we will convey complete disinterest to the person Listening Skills A good listener Keeps the focus on the speaker Is willing to see things from another perspective Refrains from finishing someone else s sentence Remains calm if attacked, rather than becoming defensive Is accepting of the person, if not their actions Uses active listening skills the concept of being a mirror Attending skills Shows we are prepared to listen Is being sensitive to the speaker s feelings Is asking relevant questions and using minimal encouragers Is mirroring the mode and pace of the other person s speech 10

11 Listening Skills Body Language Relaxed, alert, smiling Lean forward slightly Face squarely and at eye level Open posture Remember the message Eye Contact Expresses interest Establishes trust Focus softly and shift gaze Reducing the Tension Effects of tension Defensiveness, reluctance, aggression Poor performance and mistakes by interviewee Reduce by: Being human Putting people at ease Projecting the right image quiet confidence Challenging the issue, not the person Avoiding absolutes and attacking statements Asking Key Questions What is happening? Why is it happening? Where is it happening? Why is it happening there? When was it done? Why was it done then? Who did it? Why was it done by that person? How was it done? Why was it done that way? 11

12 Direct Questions Require definite answers Establishes something factual Useful for clarifying detail Brings discussion back on track Often results in a yes or no, like: Do you calibrate these instruments daily? Not effective in having the auditee explain Hypothetical Questions Poses the unusual What do you do when you find that the instrument is out of calibration? What does the response indicate? Has a contingency been planned for? Encourages the auditee to explain further Clarifying Questions Prevents misunderstanding Do I understand that you will perform daily instrument calibrations only when the daily intermediate checks are unacceptable? Enables the person to explain even further Repetitive question to further clarify AVOID leading questions 12

13 Other Questions Answer your own questions Avoids questioning the obvious Reassures the person while you examine more Get confirmation Same question, different perspective Same question, different person Even silence can be useful Gives person time to think Places person under pressure to respond But generates tension (so use with care) Limitations of Questions Can t always take answers at face value Answers may not be the truth May be intentionally misleading Coached responses But remember: Auditors are in the evidence business Information or Evidence? Information is what you hear! Evidence is what you see!! So we need to go see 39 A2LA

14 Inspect Facility & Equipment Evaluate design layout of laboratory Proper environmental conditions and monitoring? Sufficient lighting? Temperature? Effective separation between areas? Appropriate security? Good housekeeping? Suitable equipment, range, precision? Status of equipment, maintenance, calibration? Examine Documents Documents available for use? Implemented? Is there proper document control for Internal documents? External documents? Document changes properly made? Draft procedures in use? Obsolete documents removed? Verify that testing and calibration follows written procedures Examine Records Quality and technical records in place? Records allow traceability and repetition? Procedures being followed for archiving? Confidentiality maintained? Audit trail? Records include identity of all personnel? Contemporaneous records? Complete? Error corrections properly made? 14

15 Observe Activities Witness demonstrations of procedures Confirms what we heard from our questions Confirms that analysts are following procedures Determines status of equipment and setup Enables us to judge technical skills Establishes the level and depth of training Gains information on supervision Show and Tell Questions Initiated by the auditor Goes along the spokes Show and Tell Lead by the auditee Goes around the rim Responsibilities for Conduct The auditor is solely responsible for the success of the audit. If a team, it is the team leader. Chair meetings (all of them) Declare findings Sign the report Official link between team and auditees. 15

16 Sampling Discuss the process and pick some records along the way. Best when the process is simple and records are few. Start with random selection of records and discuss the process based on the records. Best when process is complex and records are many. Facing Conflict Remain calm - create distance and space Tell someone (lead auditor) or senior auditee Tell someone else (quality manager) Pause the audit until the situation is resolved Recording Nonconformities 1. No Secrets 2. Disclose/Communicate 3. Vet 16

17 Daily Debreifs 1. Daily to prevent surprises later 2. List of observations - not findings 3. Keep auditee current with progress of audit When Something Seems Wrong Is it really wrong? Do the analysts know its wrong? What is their explanation? Is it an isolated event, or a symptom of a deeper problem? Why didn't the management system detect it? What lapse in the management system allowed this to happen? Audit Findings Two required elements of every finding: A statement of the objective evidence, and A reference to the requirement 17

18 Evidence Supports Findings Sources of audit evidence: Inspecting facilities and equipment Examining documents Examining records Observation of activities What is an audit finding? Definition: result of the evaluation of the collected audit evidence against audit criteria Findings can be good or not so good Can indicate conformity, nonconformity or opportunities for improvement Writing Deficiencies Clear Concise Non-confrontational Non-blaming Non-personal 54 A2LA

19 Writing a Deficiency Structure of a deficiency statement The Requirement Description of the deficiency The objective evidence Writing Deficiencies Directly related to the management system standard, laboratory policies and procedures, technical procedures or customer requirements, etc. And based on a clear statement of objective evidence recorded during the audit 56 A2LA 2010 Writing the Report Solely the responsibility of the auditor (or lead auditor). Only the auditor (or the lead auditor) MUST sign the report. Only the auditor (or the lead auditor) declares the nonconformances. Good lead auditors get consensus. 19

20 Contents of the Report The report must: Be a factual description of the audit activities it covers, and Provide a fair and accurate picture of the quality system audited. Structure of the Report The report contains: Cover page Audit Summary Activity Summary Findings Cover Page The cover page contains: Report number Dates Scope and objectives Reference documents Audited documents Team names Auditee names Signature of lead auditor (and.) Page numbers 20

21 Audit Summary Page The summary page contains: Report number Auditor overall comments Auditor conclusions Auditor recommendations Date of next planned activity Page numbers Activity Summary Page The activity summary page contains: Report number List of observations (areas/activities) List of associated references List of deviations and other finding notation numbers Auditor activity comments Page numbers Findings Page(s) Each finding page contains: Finding (NCR) log number Indication of type of finding Identification of process owner, date and reference Description of the finding Space for root cause analysis, corrective action, and follow up 21

22 Team Debriefing After the conclusion of the audit activity, gather the team together to discuss how it went and look for areas to improve. Determining the need for Root Cause Analysis Three Questions: 1. Does this condition (or might it) present unacceptable risk of recurrence to the organization (or its people or visitors)? 2. Has this condition prevented or might it prevent the organization from producing technically valid results or making technically valid decisions? 3. Does the condition represent a true departure from the Quality System? Answers: Yes/No? Yes/No? Yes/No? Determining the need for Root Cause Analysis Results of the Three Questions: 1. ANY yes answers means a full root cause analysis is REQUIRED 2. If ALL answers are no, then root cause is not required and only remediation (correction or prevention) is needed. 22

23 Root Cause Process 1.Direct Cause. Ask the first Why and identify the Direct Cause. 2.Root Cause. Ask three or four more Why s back along the chain of events and get to a condition or circumstance whose resolution that will permanently resolve the problem (and is still within your scope of responsibility). Find the hole in the System and fill it. Use the indicators from the ISO/IEC Principle of Capacity to help. 3.Document the results of the analysis as part of the overall Corrective or Preventive Action. Example 1 Why was I late for work this morning? PROBLEM: Why didn't I get to work on time? DIRECT CAUSE CONTRIBUTING CAUSE: Car wouldn't start CONTRIBUTING CAUSE: Battery was dead. CONTRIBUTING CAUSE: Dome light stayed on all night. Why? Why? Why? ROOT CAUSE: Kids played in car, left door ajar. Example 2 Received ticket for safety violation. Car exhaust too loud. Muffler knocked loose from tail pipe. Daughter hit pothole. Potholes in road. Winters damaged roads. Govt won't approve extra money for better roads. Govt doesn't have extra money. Govt spent money on other legal issues. Too many lawyers in politics. Solution? Drive car in Sweden where there are fewer lawyers. 23

24 The Chain of Causes 1. Direct Cause: The cause that directly resulted in an event. (The first cause in the chain.) This is the answer to your first question (your problem statement). 2. Contributing Cause: A cause that contributed to an event but, by itself, might not have caused the event (the causes before the direct cause) or are back in the chain of events between the Direct Cause and the Root Cause. For a very simple problem there may not be any contributing causes. 3. Root Cause: The fundamental reason for an event, which if corrected, would prevent recurrence or first-time occurrence. (The first cause in the chain). Root Causes that are outside the ability of an organization to address it are not useful for anything. Stick to the ones we can actually fix. Root Cause Process 1. Determine the Direct Cause. 2. Determine the Root Cause. 3. Document the results Identify the Direct Cause 1. Direct Cause. Ask the first Why and identify the Direct Cause. OR 2. Visualise it: Take a mental picture of the actual occurrence (incident, deviation, etc.). You are looking at the Direct Cause in that picture. 24

25 Root Cause Process 1. Determine the Direct Cause. 2. Determine the Root Cause. 3. Document the results Identify the missing pieces Knowledge/skill of the people involved Environment surrounding the work: Physical Psychological Equipment Quality control / assurance of the work Suitability of the procedures involved Root Cause Listing Possible root causes include the following categories. Personnel Factors Environmental Factors Quality Factors Procedural Factors Organisational Factors 25

26 Personnel Factors Physical capacity Intellectual capacity Physical or physiological stress Emotional or psychological stress Individual skill Individual knowledge Care and attention. Environmental Factors Physical plant and facilities Environmental conditions Tools and equipment Materials and supplies Maintenance Wear and tear Quality Factors Quality control Quality assurance Quality system 26

27 Procedural Factors Standard procedures Specifications Implementation Selection of support Organisational Factors Leadership Communications Motivation Root Cause Process 1. Determine the Direct Cause. 2. Determine the Root Cause. 3. Document the results 27

28 Record Steps Taken Steps taken to eliminate root cause Follow up on implementation Verification audit Subsequent actions What if Problem Recurs? Correct implementation? Original root cause determination incorrect? Back to the drawing board New root cause New corrective action What if Problem Recurs? Correct implementation? Original root cause determination incorrect? Back to the drawing board New root cause New corrective action 28

29 Summary Internal Audits are focused on determining compliance A good auditor collects objective evidence and evaluates that evidence against stated requirements A well written deficiency contains two elements: A statement of the objective evidence A relationship to the requirements Audit findings are followed by: Remedial action Root cause analysis Corrective action If corrective action fails to prevent recurrence, then its back to the drawing board! 29