A Verification Study for Implementing the Revised CLSI Breakpoints Jean B. Patel, PhD, D(ABMM) Deputy Director, Office of Antimicrobial Resistance National Center for Emerging and Zoonotic Infectious Disease The findings and conclusions in this presentation have not been formally disseminated by the Centers for Disease Control and Prevention and should not be construed to represent any agency determination or policy Summary Why a study is needed Study design Data analysis Breakpoint Differences Cephalosporin Breakpoints for Enterobacteriaceae CLSI & FDA ceftriaxone breakpoints revised in 2010
Why Implement the CLSI Revised Cephalosporin Breakpoints? The cephalosporin breakpoints are: Based upon new PK/PD data Can be used to guide treatment regardless of underlying resistance mechanism(s) Avoids using a limited phenotypic test to guide treatment decisions False-positives associated with multiple cephalosporinases Does not detect all cephalosporinases Breakpoint Differences Carbapenem Breakpoints for Enterobacteriaceae Why Implement the CLSI Revised Carbapenem Breakpoints? The carbapenem breakpoints are: Based upon new PK/PD data Can be used to guide treatment regardless of underlying resistance mechanism(s) More likely to identify carbapenemase-producing isolates as nonsusceptible (treatment & infection control implications) Avoids using a subjective and sometimes inaccurate phenotypic test to guide treatment decisions
Poll Question: What are your plans? Our lab plans to implement the revised cephalosporin breakpoints this year Yes No Our lab plans to implement the revised carbapenem breakpoints this year Yes No Glossary AST Device A commercial antimicrobial susceptibility testing method Examples: MicroScan, BD Phoenix, Vitek2, Sensititre, and Etest Reference AST method A standardized, non proprietary antimicrobial susceptibility test method Examples: CLSI frozen broth microdilution, disk diffusion Glossary Federal laboratory regulation The regulation: CLIA The enforcers: CAP and Joint Commission FDA The federal agency which regulates antimicrobials & the AST devices Information is in the product label or package insert
Glossary CLSI A not for profit laboratory standards organization Describes AST reference methods & AST interpretive criteria AST interpretive criteria Criteria for categorizing a bacterial isolate as susceptible, intermediate, or resistant to an antimicrobial Also known as breakpoints Glossary Verification The documentation of either commercial or laboratory developed tests to determine or confirm test performance characteristics before the test system is used for patient testing Validation The documentation that a test which has already been verified is repeatedly giving the expected results as the test is performed over a period of time. Why are there Different Breakpoints for the Same Drug? CLSI FDA Different types of organization Different processes for setting breakpoints
Updating FDA Breakpoints Ceftriaxone breakpoints revised Which Breakpoints Can My Laboratory Use? Lab regulatory agencies Joint Commission & CAP allow labs to use either CLSI or FDA breakpoints Lab regulatory agencies also require that commercial antimicrobial susceptibility testing (AST) devices Either meet FDA performance standards or Be verified for acceptable performance by the lab using the test The Issue AST device manufacturers are required by law to use FDA breakpoints Panels can be updated with new drug dilutions for reporting lower MICs if the manufacturer has previous clearance for those drug dilutions Panels cannot be updated to provide new interpretations based upon non FDA breakpoints
Consult with stakeholders If using disk diffusion you can implement revised breakpoints without a verification If using a study device, a verification study is needed IDSA notified members of revised CLSI breakpoints http://www.idsociety.org/antimicrobial_susceptibility_testing/ See Appendix A Validation Suggestion Why is a Study Needed? To meet CLIA Requirement 493.1253(b)(2): Prior to reporting patient test results, the laboratory is responsible for establishing the performance specifications for each modified FDA-cleared or approved test system, each test system not subject to FDA clearance or approval, and each test system for which the manufacturer does not provide performance specifications. The establishment of method performance specifications should provide evidence that the accuracy, precision, analytical sensitivity, and analytical specificity of the procedure is adequate to meet the clients' needs as determined by the laboratory director and clinical consultant.
The Purpose of the Study To evaluate whether an AST device meets the FDA required category agreement FDA requirements for AST devices 90% essential agreement 90% category agreement This requirement has already been met for all dilutions reported by an AST device http://www.fda.gov/medicaldevices/deviceregulationa ndguidance/guidancedocuments/ucm080564.htm What is Essential Agreement? MIC results from a reference method and a device method are within a single doubling dilution Example: Klebsiella pneumoniae meropenem MIC (µg/ml) Reference broth microdilution 1 Device 1 0.5 Device 2 4 Device 3 2 No essential agreement What is Category Agreement? Category agreement for Susceptible, Intermediate, and Resistant Example: Zone diameter vs. MIC
Category Agreement Antimicrobial with S, I, R breakpoints of 1/2/4 Test Method MIC (µg/ml) 0.5 1 2 4 Reference MIC (µg/ml) 0.5 3 1 15 9 2 2 4 1 Example: MIC vs. MIC If essential agreement has already been established, then why does category agreement need to be evaluated? Answer: It is possible to have essential agreement without category agreement Reference MICs Category Agreement Without Essential Agreement Hypothetical experiment: 30 isolates of Enterobacteriaceae were tested for meropenem susceptibility by reference MIC method and an AST device. New S/I/R breakpoints: 1/2/4 0.125 0.125 0.25 0.5 1 2 4 0.25 6 10 0.5 4 8 1 2 2 4 Test MICs Essential agreement = 66% Category agreement = 100%
Essential Agreement Without Category Agreement Hypothetical example: 30 isolates of Enterobacteriaceae were tested for meropenem susceptibility by reference MIC method and an AST device. New breakpoints: 1/2/4 Test MICs 0.5 1 2 4 Essential agreement = 100% Category agreement = 63% Reference MICs 0.5 3 1 15 9 2 2 4 1 Category Errors Interpretation for Reference Method Interpretation for Test Method Type of Error Susceptible Intermediate Minor error Intermediate Susceptible Minor error Intermediate Resistant Minor error Resistant Intermediate Minor error Susceptible Resistant Major error Resistant Susceptible Very major error Acceptable Results No standards for a study like this Laboratories must establish this before starting the study Suggestion: Category agreement 90% No more than 1 major error No very major errors
Calculation Category agreement = # of isolates with the same interpretive result as the reference method # of isolates tested X100 A Study to Implement the Revised CLSI Cephalosporin and Carbapenem Breakpoints Availability of Concentrations Needed to Implement revised CLSI Enterobacteriaceae Breakpoints on Commercial Panels Antimicrobial Agent MicroScan Phoenix Sensititre Vitek2 Etest Aztreonam x 1 x x x x Cefazolin (early 2014) 2 (late 2012) x (Fall 2013) - Cefotaxime (early 2014) (late 2012) x x x Ceftriaxone x x x x x Ceftazidime x x x x x Doripenem x (late 2012) x x x Ertapenem (early 2013) x x x x Imipenem x x x x x Meropenem x x x x x 1 x = Available on one or more routine panels/cards as of 4/1/12 2 Projected availability NOTE: although low concentrations may be available, laboratories must perform verification if using new CLSI (vs. FDA) breakpoints with these systems 30
Possible Isolate Selection Revised Isolates Breakpoints to Test Carbapenems 5 carbapenemase (+) 5 Modified Hodge Test (-) and elevated carbapenem MICs 20 other Enterobacteriaceae Cephalosporins 5 ESBL (+) and Aztreonam 5 ESBL (-) and ESBL screen positive 20 other Enterobacteriaceae (preferably with MICs within S range using old BPs) Labs may consider testing more isolates Reference Method for the Verification Study CLSI reference broth microdilution frozen CLSI agar dilution CLSI disk diffusion The most accessible method Procedures Test all isolate by the test AST method & the reference AST method Test isolates by both methods on the same day Prepare each inoculum from the same subculture plate If possible, use the same inoculum suspension (e.g. 0.5 McFarland equivalent) Use methods that you will use routinely Don t forget about quality control
Example: Data Collection Isolate ID Test AST Microscan Panel MIC (µg/ml) Reference AST Disk Diffusion Zone Diameter (mm) Isolate ID Test AST Microscan Panel MIC (µg/ml) Reference AST Disk Diffusion Zone Diameter (mm) BVS1 0.5 24 BVS2 0.5 23 BVS3 1 19 BVS4 0.5 25 BVS5 2 20 BVS6 1 24 BVS7 4 18 BVS8 0.5 23 BVS9 4 17 BVS10 4 19 BVS11 0.5 24 BVS12 0.5 23 BVS13 0.5 25 BVS14 1 23 BVS15 1 25 BVS16 1 23 BVS17 0.5 24 BVS18 2 21 BVS19 0.5 24 BVS20 0.5 25 BVS21 1 24 BVS22 2 19 BVS23 0.5 24 BVS24 2 24 BVS25 1 24 BVS26 8 18 BVS27 1 24 BVS28 0.5 25 BVS29 2 21 BVS30 0.25 24 Example: Scattergram 16 8 1 MIC (ug/ml) 4 1 1 1 2 1 1 2 1 1 1 2 4 1 0.5 3 5 4 0.25 1 16 17 18 19 20 21 22 23 24 25 26 27 Zone diameter (mm) Example: Identify Errors 16 8 1 4 1 1 1 2 1 1 2 1 1 1 2 4 1 0.5 3 5 4 0.25 1 16 17 18 19 20 21 22 23 24 25 26 27 Minor errors = 2, Major errors = 1, Very major errors = 0 Category agreement = 27 30 = 0.90 or 90%
Documentation The study is needed to fulfill CLIA regulation If it is not documented, you didn t do it Documentation includes: Study procedure Results: data table and/or scattergram Calculations Conclusion statement Verification study results must be approved by laboratory director What if the Results are Unacceptable? 1-2 isolates with discrepant results could indicated random error >2 isolates with discrepant results may indicate systematic error caused by: Technical problems with the methods used in the study Discordance between test method and reference method Unacceptable Results cont d Assessing error type Random errors unlikely to occur upon repeat testing Systemic technical errors will not repeat if the technical problem(s) is addressed Errors caused by a discordance between methods will repeat
Unacceptable Results cont d Review the procedures looking for technical errors Repeat testing can be done, but avoid repeating only isolates with discrepant results Decide on a systematic approach to repeat testing (e.g., repeat 1 to 2 isolates with nondiscrepant results for every one isolate with discrepant results) Unacceptable Results cont d Calculate category agreement and error rates A) Using original data B) Using repeat data If the repeat study results meet the acceptability requirement, then Document the original results Document the repeat results Describe likely source of error Unacceptable Results cont d Discordance between methods cannot be fixed by troubleshooting Consult manufacturer
The AST Manufacturer Cannot: Provide software or alter devices to provide non-fda breakpoints Can: Provide general instructions for how to customize the device expert system to apply the revised breakpoints Alternatively, labs can implement the revised breakpoint in their laboratory information Applying New Breakpoints to Etest Applying revised breakpoints to Etest results is a modification of a FDA-cleared device A verification study is needed Qs & As Q: What if some of the carbapenemase-producing isolates test carbapenem-susceptible or the ESBL-producing isolates test cephalosporinsusceptible A: The purpose of this study is to determine if the category agreement between the test and reference AST methods are within acceptable limits regardless of underlying resistance mechanisms in the challenge isolates.
Qs & As Q: If we aren t scoring results by resistance mechanism, why do I need to test isolates with different resistance mechanisms? A: Using isolates with different resistance mechanisms allows for a more robust challenge of your AST method. Sources for Isolates CDC is sending isolates to: College of American Pathologists ATCC Isolates that you may have on hand Clinical isolates ATCC QC isolates Isolates from previous proficiency testing challenges Acknowledgements Janet Hindler, Jim Jorgensen, Barb Zimmer Kamile Rasheed, David Lonsway, Brandi Limbago IDSA Antimicrobial Resistance Working Group CLSI The Joint Commission CAP FDA
Thank You jpatel1@cdc.gov