What is? An Overview of Operational Processes

Transcription

1 Friday April 7, 2017 LD4 What is? An Overview of Operational Processes John T. Daly, MD, FCAP Chief Medical Officer, COLA DESCRIPTION: This session focuses on other important topics for the laboratory director regarding lab operations and regulatory compliance, including instrument verification and validation, calibration and calibration verification, random and systemic errors, split sample testing and instrument maintenance. OBJECTIVES: At the end of the session, participants will be able to: Discuss components of laboratory test validation and verification Differentiate between calibration and calibration verification Outline steps which may be taken in troubleshooting problems with an instrument or laboratory test Summarize important steps in instrument maintenance for quality laboratory practice COLA Resources, Inc. and COLA do not endorse, directly or indirectly, the presentations given at this conference or the products or services provided by the exhibiting vendors. Presentations are intended to be free of bias. The use of any particular product is for demonstration purposes only, and does not imply an endorsement of the product by the presenter or the sponsors of the symposium CRI

2 John Daly, MD COLA Chief Medical Officer 1 Verification and Validation Calibration and Calibration Verification Random vs. Systematic Error Split Sample Testing Maintenance Waived Testing 2 1

3 At the end of this session the participant should be able to Discuss components of laboratory test validation and verification Understand difference between calibration and calibration verification Understand steps which may be taken in troubleshooting problems with an instrument or laboratory test Discuss important steps in maintenance for quality laboratory practice 3 4 2

4 Manufacturer s performance specifications must be checked (verified) for each new unmodified moderate FDA approved tests before reporting patient results Applies to new test instruments, tests added to existing instruments, change in manufacturer of test kit Accuracy Precision Reportable range Reference range 5 Accuracy and Precision 6 3

5 For determination of accuracy and precision samples with known values used May be calibrators, quality control materials, previously tested PT samples, previously tested patient samples Actual number may vary depending on test, test system and specimen availability Laboratory personnel must do testing for verification but instrument vendor should be available for education, advice and assistance 7 Rotating the verification samples among testing personnel assists with fulfilling training requirements Verification must be complete before patient reporting and TC/LD must have signed off Document all data, instrument tapes, work sheets and Laboratory Director review and then retain for life of instrument + two years 8 4

6 For accuracy determination, ideally 20 different data points should be analyzed with each sample run in duplicate Bare minimum would be to test at least five samples running a minimum of 5 repetitions each on different days by different personnel. These samples could be QC materials For analysis of precision repeat a given sample at least 5 times before analyzing results 9 Reportable range Highest and lowest values the manufacturer claims accurate results can be produced by the system Samples with known values tested to verify claim. Potentially performed in conjunction with accuracy determination Laboratory can only report values that fall between levels they have verified 10 5

7 11 Reference range/intervals or normal values Manufacturer may suggest reference ranges May also used published reference ranges Over time verify these are appropriate for practice 12 6

8 20 samples can safely be used Individuals selected using specific criteria including exclusion criteria (drugs, recent surgery) and partitioning criteria (age, gender, race) No more than 2 of 20 samples fall outside suggested reference interval If 3 or more outside the limits more data should be collected 13 Twenty samples not needed since precision, reportable range and reference range do not apply Two positive and two negative samples running at least two repetitions of each sample is generally adequate Results from analytes with potential patient safety issues present a different issue Obtain guidance from manufacturer, references etc. 14 7

9 Validation studies must be performed All FDA approved methods modified by laboratory Non-FDA approved or cleared methods Test methods developed by the laboratory in-house Validation requires establishment of Accuracy Precision Reportable range Reference range and in addition Analytic sensitivity Analytic specificity including evaluation of potential interfering substances 15 Lowest level at which a test method can detect the analyte in specimen being tested Assays where significant include TSH Troponin D-dimer CRP Microalbumin hcg 16 8

10 New Test Ability of test to be substance-specific measuring the desired analyte without detecting interfering substances and to be negative in the absence of the analyte To rule out interfering substances, the specimens being examined can be spiked with known interfering substances to assist in specificity determination 17 Gold Standard Test/Known Material Positive Negative Positive True Positive (TP) False Positive (FP) Negative False Negative (FN) True Negative (TN) 18 9

11 NEW TEST NEW TEST Gold Standard Test Positive Negative Positive True Positive 9 False Negative 1 Negative False Positive 0 True Negative 10 TP = TP + FN 9 = = Gold Standard Test Positive Negative Positive True Positive 9 False Negative 1 Negative False Positive 0 True Negative 10 TN = TN + FP 10 = =

12 21 What is calibration? When is calibration required? What are calibration materials? 22 11

13 Calibration is setting the instrument to give an accurate result Technically it is the process of testing and adjusting the instrument or test system readout to establish a correlation between the instrument s measurement of the substance being tested and the actual concentration of the substance 23 Solutions that contain a known amount of the substance (analyte) being measured The test system provides instructions for performing calibration as well as the timing, number, type and concentration of calibration material to use 24 12

14 25 Laboratory is responsible for performing calibration at frequency specified by the manufacturer of the test system Calibration is also required when calibration verification does not give acceptable results 26 13

15 Not required for procedures that do not involve an instrument Not required for manual microbiology procedures General rule: Tests using visual interpretation rather than instrument measurements do not require calibration

16 Testing at least three samples of known concentration that span the reportable range of analyte Samples should test minimum, mid-point and maximum value of reportable range Serves as a check on the test system s calibration to verify it is still valid 29 Must be performed according to manufacturer Must be performed at minimum every six months With complete change of reagents to new lot After major instrument maintenance Quality control does not perform as expected 30 15

17 Requirement for calibration verification is met if the test system s calibration procedure includes three or more levels of calibration material and: Includes a low, mid-point and high value Is performed at least once every six months 31 If one or more calibration verification levels are unacceptable may need to recalibrate Necessary to first troubleshoot the instrument 32 16

18 33 QC results questionable QC drifts, trends, failure Calibration verification failure PT failures Patient results questionable (delta failures, complaints) 34 17

19 Pre-Analytic Think Process Have reagents, controls, calibrators been stored in accordance with manufacturer s directions Were controls/calibrators left to evaporate before use Are all materials in date Has maintenance been timely (daily, weekly, monthly, quarterly) Have environmental conditions been monitored and in range (temperature, humidity) Any recalls on materials in test system 35 Analytic Is employee competent to perform function Have control vials/results been transposed If calibrators lyophilized were they properly reconstituted. Has the instrument been serviced recently and were controls/calibration verification run after service 36 18

20 If issue is with QC, run calibration verification If issue is with calibration verification check preceding items in pre-analytic and analytic process before recalibrating Remember most instrument operator manuals have trouble shooting guides

21 39 95% of results will fall within +/- 2SD 5% chance control will be out of range even when test system is functioning properly This is an example of a type of random error Possible causes of random error also include: Bubble in sample Poor technique Well, it s just random! 40 20

22 Error related to test accuracy Also referred to as bias Constant deviation from true result occurring in a pattern Deviation may be consistently above and below true value Cause of systematic errors must be identified and corrected before reporting patient results

23 Analysis of same specimen by primary instrument and a second analyzer utilizing comparable methodology May be used for comparison of two analyzers in your laboratory May be sample performed at outside laboratory (e.g. hospital, reference laboratory) and your laboratory 43 To perform 6 month comparisons between similar analyzers in your laboratory To verify the accuracy of non-regulated analytes for which you do not perform proficiency testing where it serves as substitute for proficiency testing To perform comparisons between satellite laboratories in a group practice or health system 44 22

24 Only patient samples are acceptable for split sample analysis For instrument comparisons and PT - must be performed on five samples twice a year Laboratory Director responsible for making determination of what is acceptable comparison Results must be retained for two years

25 Must be labeled as to Content Storage requirements Expiration dates Many times when controls/reagents are first opened the expiration date changes and must be documented on the vial 47 All reagents, standards, controls and patient specimens must be stored per manufacturer or procedure Reagents in a kit cannot be interchanged with another kit of different lot number or manufacturer All of above must be discarded at expiration date 48 24

26 Some instruments are affected by humidity and if so, humidity must be monitored Temperatures of temperature dependent equipment e.g. refrigerators, freezers, incubators, water baths and dry baths must be monitored & documented 49 Operator Manual for each instrument will contain maintenance to be done and at what specific interval Manual may also contain a formatted recording form which simplifies documentation When function checks exceed manufacturer limits there must be documented corrective action 50 25

27 51 Tests Designated by FDA as Methods simple and accurate Have easy to follow procedures Insignificant risk of erroneous result Over past 25 years number of analytes and test systems have dramatically increased 128 analytes listed in FDA data base January

28 Follow all of the manufacturer s instructions Ensure employee training and competency Perform quality control per manufacturer Incorporate into Laboratory Quality Management Plan Pre-analytic, analytic and post-analytic system evaluation Organize all documents and retain for 2 years Manufacturer s instructions Lot numbers of each kit used and dates in service All logs and patient results Report problems with test system to manufacturer Package insert comes with every test kit Insert gives information and directions for use Inserts get updated: use the latest version If test uses a meter or instrument to read the results refer to instrument operator s manual in conjunction with package inserts accompanying the reagents 27

29 Information in package insert organized into sections Insert may be a booklet or large sheet Test complexity found in package insert Analyte(s) detected Test matrix Statement if test can be both waived and moderate complexity Common with HCG kits designed for urine and serum Storage and stability Temperature when used Materials supplied Step by step procedure Sensitivity, specificity and interfering substances Additional follow up tests QC usage and timing Internal, external and electronic 56 28

30 Lack of training and competency documentation Failure to perform required QC Lack of current package insert Use of test beyond expiration date Improper reporting of patient results Failure to follow handling instructions ITEM Test kit Directions Reagents Right Order QC Sample Timing Control QUESTION Correct test kit used? Were they followed exactly? Correct, in date reagents used? Reagents/sample added in correct order? Sample added? Correct sample for kit? Was timing accurate Correct for kit and in-date? 29

31 30

32 In past year waived molecular testing introduced Today waived molecular tests include: Influenza A and B and RSV Respiratory Film Array detects 11 viral and 3 bacterial pathogens Strep A Advantageous for increased sensitivity compared with antigen testing Multiple companies are working to expand the menu of waived infectious diseases and major growth area will be STDs Ready? Set? Test? is both a booklet and an on-line one hour course. The booklet is available in both English and Spanish and can be printed for off line use or can be ordered at the site below. The on-line course is excellent orientation for new employees who are not experience Laboratorians. ursedetails/coursedetailsform.aspx?tabid=96&cours eid=