Presentation Handouts

Transcription

1 Presentation Handouts (9108-QE) Competency Assessment: Finding the Good, the Bad and the Ugly October 6, :30 AM - 12:00 PM

2 Event Faculty List Event Title: 9108-QE: Competency Assessment: Finding the Good, the Bad and the Ugly Event Date: Saturday, October 6, 2012 Event Time: 10:30 AM to 12:00 PM Director L. Jeanne Wall, MEd, MT(ASCP)SBB Consultant Wall Quality Management Disclosures: No Moderator Anne Chenoweth, MBA, MT(ASCP)CM, CQA(ASQ) Technical Specialist AABB Disclosures: No Speaker Judith Sullivan, MS, MT(ASCP)SBB, CQA(ASQ) Quality Source Consultant Blood Systems, Inc. Disclosures: No Speaker Regina Castor, BS, MT(ASCP)SBB Blood Bank Technical Specialist Immucor Disclosures: No

3 Competency Assessment: Examples Regina Castor BS MT(ASCP) SBB cm Objectives Provide examples and/or strategies for assessing competence for the following groups: Clinical Lab Staff Donor Center Staff Nursing Staff Physicians CLIA 88 Requirements Direct Observation Routine Patient Test Performance Performance of Instrument Maintenance/Function Checks If 10 procedures x 10 staff members x 6 elements of CLIA 1

4 Direct Observation Questions that come to mind? Who does the observation? Manager Senior Tech Peer Secret Shopper What / How much do we choose to observe? Routine/Critical tasks Pre analytical, Analytical, Post Analytical Tasks that are new, changed How often do we observe it? Look at regulations How do we document it? Can vary, but it must be done Initial Training Direct Observation The observation must be documented Create a checklist from the procedure Don t forget this is a good time to critique procedures Creating a checklist = an internal audit! 2

5 Donor Center Audit Checklist: *PPE *Bag Inspected *Bag hung properly *Tubing inserted *Hemostat applied *Sample port remains below donor arm etc. An Example of Direct Observation Competency Assessment Procedure for Issuing Blood: 3

6 Could include What if questions to satisfy Problem solving requirement Performance of Instrument Maintenance and Instrument checks Review steps in the procedures to assure you are saying what you do, and doing what you say Make your procedure into a checklist Remember this is a good opportunity to audit procedures for following manufacturer s directions Examples of Instrument Maintenance/Function Checks Calibration of Scales Testing Temperature High/Lows, Alarm activation checks Serofuge Calibrations Daily, Weekly, Monthly Blood Bank automation checks Sterile docking devices Maintenance on plasma thawers Cooler Validations Hgb level detection devices Decontamination 4

7 Use QA schedules to determine who and what to observe CLIA 88 Requirements Monitoring the recording and reporting of test results Most Transfusion Services and Donor Centers do some sort of record review Define how this review is done in your Competency P&Ps Example of Documenting Monitoring and Recording results PROCEDURE STATES: Daily test and QC results are reviewed. Any reporting of results or test performance that is found to be in noncompliance with procedures will be documented on a non conformance form. Non conformance forms for each tech are reviewed prior to determining annual competency. The inspector should be able to follow the documentation trail to see how the process works 5

8 All Routine Tasks 2011 Blood Bank Competency Assessment Summary 1. Direct Observation of routine patient test performance Tech 1 Tech 2 Tech3 Tech4 Tech5 Tech 6 Tech 7 CLIA Header ABO/Rh and antibody screen 11/1/ /1/ /1/ /1/2011 Sep 11 12/6/ /23/2011 Other Accession # of test reviewed 2. Monitoring, recording and reporting of test results CLIA Header Tech 1 Tech 2 Tech3 Tech4 Tech5 Tech 6 Tech 7 ABO/Rh 0223IH9 0223IH IH ;IH IH13 12/6/ ;IH30 Antibody Screen 0223IH9 0223IH IH ;IH IH13 12/6/ ;IH30 Compatibility 0223IH9 0223IH IH ;IH IH13 12/6/ ;IH30 Antigen Typing 2/26/11 6/30/11 1/18/11 2/25/11 7/23/11NA NA Antibody ID 0105IH IH IH6 0801;IH21 NA NA TRXN 0317;IH IH24 Courtesy of Lea Tolzmann, Winter Haven Hospital, Winter Haven, FL CLIA 88 Requirements Review of intermediate test results, QC records, Proficiency testing results, preventive maintenance records Again, most are doing this but don t document it well from a competency standpoint 6

9 Proficiency Testing Be aware that simply rotating Lab Proficiency surveys (such as CAP or API) among staff members and checking the results against the expected results in and of itself does not satisfy all 6 of the CLIA requirements Proficiency Testing If you are observing Proficiency sample testing as a direct observation event, are you treating it like routine patient samples? Proficiency samples must be treated like routine samples Are we there yet????? 7

10 Assessment of test performance through testing previously analyzed specimens, internal blind testing samples, or external proficiency testing samples Internally prepared samples Can be time consuming to prepare Less Cost Variable results depending on storage and use Unique patient samples?? Can you get enough sample for multiple techs? How stable will the samples be? Document results create worksheet? Use downtime recording procedures to audit process Assessment of test performance through testing previously analyzed specimens, internal blind testing samples, or external proficiency testing samples Externally Prepared Samples Available from some Blood Suppliers as a value added service Purchased tech competency products (as opposed to Lab Proficiency products) Scalable CEUs offered Can be used for multiple techs Proficiency products more flexibility on how used Uses of Competency Samples Assign prior to releasing to task Initial competency assessment Assign if no proficiency survey has been assigned for a given task in a given year Assure that all techs have had a challenge sample on all routine procedures per year Assign as part of corrective action For failure on Proficiency tests, failure on annual competencies, Root Cause corrective action 8

11 Uses of Competency Samples Assign based on new methodologies New automation, new potentiator for tube method, etc. Can be ordered as needed CLIA 88 Requirements Assessment of Problem Solving Skills Oral interviews Skills Labs Written Case Studies These may be included in some purchased competency or proficiency products Include infrequently occurring events» Txn Rxns» Adverse Donor reactions» Massive Transfusion Scenarios» Disaster Drills 9

12 Tracer Audits Effective at looking at a variety of documents, staff Can be Retrospective Following a unit being transfused back to time of collection, reagents used, QC, equipment used, results recording, etc. Can be forwarded looking Ex: following equipment from purchase to discontinuation Crossmatch sample collection through transfusion Donor collection through processing/labelling Tracer audits can easily incorporate several elements of CLIA by looking at a variety of records Transfusion Audit Physician Blood Order Consent Proper Utilization? Proper labeling Turn around Time (order to receipt) Phlebotomy Was the sample processed correctly Equipment use Reagent QC/Receipt Blood Bank Results reporting Nursing Time out process (NSPG #1) Proper use of filter Transfusion protocol Transfusion Reaction reporting (if indicated) 10

13 Transfusion Audit Example of facility that uses Microsoft Access to document audits Nursing Competency Record Review Donor deferral Transfusion documentation Transfusion Reactions Direct observations Collecting blood Transfusing units Use of blood warmers Transporting Blood Learning Labs/Simulator Nursing Competency Example of Annual Module assigned through Hospital s Learning Management System Staff required to review and complete quiz 11

14 Physician Competency Ordering Practices (TJC requirement) Is blood ordered at appropriate time Is the order in compliance with Transfusion policy Use of emergent blood Physician Order Are all required elements documented Dated/timed Signature Reason for transfusion Type and Cross vs. Transfuse Special requirements for transfusion, how well are those documented Physician Competency Consent Is it complete Dated and timed before transfusion occurred Are all elements being documented Risks Benefits Alternatives What about refusal process 12

15 Example of Documentation Example of Documentation Conclusions Make checklists from the procedures Audits can incorporate multiple CLIA requirements Include all staff that relate to transfusion medicine, not just the testing personnel Tracer audits are valuable tools Document, Document, Document 13

16 Regina Castor

17 Competency Assessment: Finding the Good, The Bad, and the Ugly Judy Sullivan, MS, MT(ASCP)SBB AABB Annual Meeting October 2012 Objectives Discuss requirements related to competency assessment Describe what assessors, inspectors, and surveyors look for as evidence of compliance Identify methods to meet the requirements for competency assessment What Is Competence? A cluster of related abilities, commitments, knowledge, and skills that enable a person to act effectively in a job or situation. Competence indicates sufficiency of knowledge and skills that enable someone to act in a wide variety of situations

18 Once Competent, Always Competent, Right? Not Necessarily! Dedicated Staff Complacency SOP drift Rotating Staff Loss of familiarity Infrequent activities What Drives Competency Assessment for Laboratory Staff It s the right thing to do!!!! It s required by regulation! CLIA Competency Assessment Key Requirement (b)(8)(9) & 1451(b)(8)(9) Technical Consultant/Supervisor Responsibilities Evaluating the competency of all testing personnel and assuring that the staff maintain their competency to perform test procedures and report test results promptly, accurately and proficiently 2

19 CLIA Competency Assessment Competency for all tests must include: Direct observations of routine patient test performance, including patient preparation, if applicable, specimen handling, processing and testing CLIA Competency Assessment Monitoring the recording and reporting of test results Review of intermediate test results or worksheets, quality control records, proficiency testing results, and preventive maintenance records 8 CLIA Competency Assessment Direct observation of performance of instrument maintenance and function checks Assessment of test performance through testing previously analyzed specimens, internal blind testing samples or external proficiency testing samples Assessment of problem solving skills 9 3

20 Frequency At least semiannually during the first year the individual tests patient specimens At least annually thereafter unless test methodology or instrumentation changes Prior to reporting patient test results, the individual's performance must be reevaluated to include the use of the new test methodology or instrumentation CAP GEN Competency Assessment The competency of each person to perform his/her assigned duties is assessed CLIA elements cited NOTE: The competency of each person to perform the duties assigned must be assessed following training before the person performs patient testing. 11 CAP GEN Competency Corrective Action If an employee fails to demonstrate satisfactory performance on the competency assessment, the laboratory has a plan of corrective action to retrain and reassess the employee's competency. Evidence of Compliance: Records of corrective action to include evidence of retraining and reassessment of competency 12 4

21 What Drives Competency Assessment for Other Staff It s the right thing to do!!!! It s required by Standards! AABB STD Training The blood bank or transfusion service shall have a process for identifying training needs and shall provide training for personnel performing critical tasks. Not Just Testing Personnel! AABB STD Competence Evaluations of competence shall be performed before independent performance of assigned activities and at specified intervals. 5

22 Training Competency What Do Assessors / Inspectors Look For? First Things First Is there a Policy, Process or Procedure Addressing Training and Competency? Laboratory General policies CLIA elements incorporated Blood Bank SOPs specific for the testing performed 18 6

23 The Next Steps How is competency determined? Who can assess competency? Is it defined? No self-assessments allowed!!! What constitutes an assessment? 3 P s, Tools, Checklists, Guidance What tests are being evaluated? Must include routine tests every year Can t pick and choose How is it documented? Does practice match SOPs? 19 Beware the Regulations! Testing Personnel All routine tests All CLIA elements MUST be used for evaluation Other Personnel Facility-specified 20 Considerations New employees Competency assessment separate from training Assessed twice in the first year Incumbents Annual assessment Documentation for staff that work on all shifts Tests being evaluated Is there any distinction made for testing that may be provided on day shift vs after hours Special testing 21 7

24 Competent or Not? Statement of competency If not competent, what was done? Does practice match SOP? I Don t Have Time for This!! 23 Let s Go Back to Basics Direct observation of performance Direct observation of instrument maintenance/function checks Monitoring recording and reporting of test results Review of worksheets, QC records, PT results, PM records Testing of previously analyzed specimens Assessment of problem solving skills 24 8

25 Direct Observation Checklist SOP in checklist form Performed at conclusion of training on that SOP Future use: Semi-annual competency assessment Annual competency assessment Instrument Maintenance Direct observation of performance of instrument maintenance and function checks Which instruments? Who does it? Staff to QC/PM Matrix Blood Bank Staff Reagent QC Provue QC KB QC Special Antigen QC Provue PM Cellwasher PM Daytime FT X X X X X Daytime FT X X X X X Daytime FT X X X X X Evening FT X X X Evening FT X X X Evening FT X X X Evening FT X X X Nights FT X X X Nights FT X X X Nights FT X X X PT X X PT X X PRN X X PRN X X 9

26 Who Is Watching? Someone who Has been trained Can determine competent vs non-competent behavior Is motivated to do it right It Doesn t Always Have to Be the Supervisor!! Just Remember Those who assess competency must also have their competency assessed IF they perform critical tasks! And That Includes the Supervisor!! Duh! I m watching Of course they are going to do it right!! Explain to me 10

27 Let s Go Back to Basics Direct observation of performance Direct observation of instrument maintenance/function checks Monitoring recording and reporting of test results Review of worksheets, QC records, PT results, PM records Testing of previously analyzed specimens Assessment of problem solving skills 31 Aren t You Doing This Already?! Include as part of the competency policy Include as part of direct observation Recording and reporting of test results QC records Preventive maintenance records Worksheets Document! 32 Let s Go Back to Basics Direct observation of performance Direct observation of instrument maintenance/function checks Monitoring recording and reporting of test results Review of worksheets, QC records, PT results, PM records Testing of previously analyzed specimens Assessment of problem solving skills 33 11

28 Previously Analyzed Specimens Patient samples Proficiency testing samples Must be rotated among all individuals performing testing Document as part of competency assessment 34 Danger, Danger, Will Robinson!!! DO NOT share PT samples with other staff until AFTER the results have been received from the PT provider Let s Go Back to Basics Direct observation of performance Direct observation of instrument maintenance/function checks Monitoring recording and reporting of test results Review of worksheets, QC records, PT results, PM records Testing of previously analyzed specimens Assessment of problem solving skills 36 12

29 Assessment of Problem Solving Skills Written test or quiz Case Studies Scenarios: What would you do if? Staff narrative Unsuccessful Result Process for Remediation Actions to take Removing employee from testing until competency is demonstrated Documentation Recurrence Re-assessment Determination Completion at end of each assessment Documentation Trackable and Traceable Checklist not sufficient Each assessment requirement must be documented Direct observation checklists Title and date of record review Title, date, sample ID if using PT Graded test/quiz Assessor name(s) and dates Employee name 13

30 Documentation Determination of competency Signature/date of supervisor Signature/date of individual performing competency assessment if different from the supervisor Signature/date of employee Include a statement of competency by the supervisor and the employee Example Competency Statements (employee sign/date) certify that I am fully trained and competent to perform the roles listed above (assessor sign/date) attest that the staff member listed above is fully trained and competent to perform the roles listed above 41 I m Done! Right? NO!! 14

31 New Procedures Changes in test methods or instrumentation Prior to reporting patient test results, the individual s performance must be reevaluated to include the use of the new test methodology or instrumentation Revised Procedures Determine type of Training Read SOP and sign Perform in service and read & sign Full training and competency assessed Trainer requirements/trainee requirements/do and Evaluation Words of Wisdom In a hierarchy, every employee tends to rise to his level of incompetence. Work is accomplished by those employees who have not yet reached their level of incompetence. ~ Laurence J Peter 15

32 THANK YOU!! Feel Free to Contact Me 47 16