Everything You Need to Consider When Considering Automation of Microbiology

Transcription

1 Everything You Need to Consider When Considering Automation of Microbiology Neil Anderson Assistant Professor of Pathology and Immunology Washington University School of Medicine Assistant Medical Director of Clinical Microbiology Barnes Jewish Hospital, Saint Louis MO Nathan A Ledeboer Professor and Vice Chair Department of Pathology and Laboratory Medicine Medical College of Wisconsin Medical Director, Microbiology and Molecular Pathology Wisconsin Diagnostic Laboratories and Froedtert Health Medical Director, Preanalytics and Reference Services Wisconsin Diagnostic Laboratories Milwaukee, WI

2 The Problem of Microbiology

3 Current Culture Workflow Blood drawn in ER, ICU, hospital floors Culture positive Pathogen group Pathogen ID Pathogen resistance Bottle culture Gram stain Samples plated for sub-culturing Resistance testing t=0 t=8h t=24-36h t=48-72h

4 The Phases of Automation Start with PreAnalytics Why we Automate The Instruments The Process The Data Potential Pitfalls What is to Come

5 The Challenge of Microbiology

6 Standardization on Transport Device Liquid Microbiology Standardized specimen transporters Easier to stock clinic rooms/ordering easier Easier to educate our clinical staff on the appropriate device Ensure specimen integrity Boric Acid Tube URINE C&S Vial STOOL ESwab (Copan) skin and soft tissue swabs, throat, GBS. BETTER PATHOGEN RECOVERY, MORE HOMOGENEOUS SPECIMEN Quality Data drove implementation and administrative buy in

7 Flocked swab doesn t have the internal part Sample remains close to the surface of the swab Easy release of the sample into liquid transport medium Liquid transport medium works as a broth of microorganisms

8 Gram Stains Fontana C, et al. BMC Research Notes, 2009.

9 Eswab and Gram Stain Genital specimens (N=80) collected in Eswab, two smears were prepared from each specimen, one manually, by directly streaking the flocked swab of the ESwab sample on a glass slide, the other smear was prepared with 10ul of sample on the WASP. 72/80 specimens were concordant for both manual and WASP smear preparations. Epithelial cells and leucocytes were detectable in ESwab samples with proper staining morphology in both Gram stain preparations. THE EIGHT DISCREPANCIES WERE NEGATIVE IN THE MANUAL PREPARED SMEAR WHILE 8 GRAM NEGATIVE RODS WERE ONLY PRESENT IN THE WASP PREPARED SMEARS. An agreement of % was found and was dependent on the type of bacteria. J. Steenbergen*, M. Stalpaert. Comparison of manual and WASP automation Gram smear preparation with specimens collected in Eswab. ECCMID 2013

10 RECOVERY COMPARISON OF COMMONLY ENCOUNTERED MICROORGANISMS IN A CLINICAL SETTING Grover Smith and Tammye L. Jackson. ASM 2012

11 Organism recovery from different swab systems Evaluation of bacterial recovery and viability from three different swab transport systems. Tan, Thean; Ng, Lily; Sim, Diana; Cheng, Yvonne; Min, Melissa Pathology. 46(3): , April 2014.

12 CFU CFU Spiked Study (Anaerobes) 1. Suspension made in saline 2. Diluted to ~10 5 CFU/mL in eswab 3. Plated by WASP using 1 µl loop 4 C 25 C Study demonstrates recovery of anaerobes at 4 o C to 48 h and recovery of anaerobes at 25 o C to 24 hours.

13 Wound culture Higher total CFU/mL recovered w/ eswab Better recovery of CoNS, Enterococcus spp. Statistically similar recovery for others Saegeman et al., Eur J Clin Microbiolo Infect Dis (2011)

14 WHY WE AUTOMATE

15 Why Automate? Potential answer to shrinking workforce Need to staff when plates are to be read, not just 9-5 Answer to ergonomic realities Quality of life issues/cost to organization Labs are consolidating can do more potentially with less but perhaps larger Better quality product consistent plating Pressure for decreased TAT from receipt to results Pressure to be open 24/7 Can be adapted to various size labs

16 Why Automate? continued. Pre-analytical processing of specimens reduces time to incubation increased quality, consistency in plating Digital Microbiology imaging analysis to aid the CLS Useful for training/documentation Quality Assurance Remote locations less skilled CLS Decrease costs through more efficient use of resources

17 Need for lab automation Driven by declining skilled labor and cost pressures in today s economic climate Macro trends are driving the need for automation The biggest driver of automation is the lack of qualified microbiologists and med techs Key market trends Lack of experienced technologists, supervisors, pathology and microbiology PhDs Decreased financial incentive for in-patient testing and increased incentive for shorter LOS Number of med tech programs US % Med tech enrollment US annual 8,296-65% 2,871 Increasing volume and lab consolidation pressures Pricing and reimbursement pressure Need for sample traceability/ chain of custody Lab professionals eligible for retirement US 2010, percent Need for better coordination between the lab physician pharmacist 60% 40% Eligible Not eligible

18 Trends to Automation? The Industry is Changing Specimens increasing on average 10-15% per year Laboratory consolidation Reimbursement Workforce Less students choose Medical Technology: reduction of 30-50% Pay for technologists is substandard Quality Physicians are demanding more services, in less time Traceability

19 Manual Processing Microbiology too complex to automate Specimen Diversity Collection Device Diversity Diversity of Techniques Diversity of Media The human element Technologists are faster than machines Humans are capable of thinking, machines are not Humans are flexible Automation considered too Expensive Small volumes Only the large labs can automate

20 And Don t Forget:

21 Manual Processing Microbiology too complex to automate Specimen Diversity Collection Device Diversity Diversity of Techniques Diversity of Media The human element Technologists are faster than machines Humans are capable of thinking, machines are not Humans are flexible Automation considered too Expensive Small volumes Only the large labs can automate

22 Advances in Identification

23 The Future of Mass Spectrometry Continued migration to mass spectrometry for microbial ID based on performance and cost Automation will simplify the set-up and further drive down costs Continued expansion of applications

24 Time course of the numbers of total isolates misidentified using phenotypic identification (PID*), isolates confirmed by a second PID* and isolates confirmed by molecular identification (ID**) over 11 years of routine identification in our clinical laboratory. Seng P et al. J. Clin. Microbiol. 2013;51:

25 Time course of the numbers of isolates of 128 rare species, 48 of which were identified using phenotypic identification (PID), and 75 of which were identified using molecular identification (ID). Seng P et al. J. Clin. Microbiol. 2013;51:

26 Available Automation Dr. Anderson

27 Laboratory Automation Systems Specimen inoculation/processing unit Incubation system High-resolution digital imaging system +/- track system for moving plates Workstations Available Models: WASPLab BD Kiestra TLA

28 WASPLab Media Stackers Smart Incubators Reading Stations Inoculation and Streaking Slide courtesy Carey-Ann Burnham Doern and Holfelder. Manual of Clinical Microbiology 11 th ed. Chapter 5.

29 BD Kiestra TLA Inoculation and Streaking -Fully Automatic (FA) -Semiautomatic (SA) -First Inoculation mode Culture Media Inoculation and Streaking Slide courtesy Carey-Ann Burnham

30 BD Kiestra TLA Reading Stations Smart Incubators Culture Media Slide courtesy Carey-Ann Burnham

31 System Comparison Instrument (Manufacturer) WASP (Walk-away specimen processor) (Copan) BD Kiestra InoqulA (BD) Specimen Type Liquid specimens Automated decapping and recapping Liquid specimens (fully automatic) Other specimens (semi-automatic, first inoculation) Inoculation Technique Inoculation loop (1, 10, 30 µl) Pipette (10 µl to 250 µl) Adapted from Rhoads et al J Pathol Inform. DOI: Croxatto et al Clin Microbes Infect. 22:

32 System Comparison Instrument (Manufacturer) WASP (Walk-away specimen processor) (Copan) BD Kiestra InoqulA (BD) Streaking Technique Loop and spreaders, variety of patterns available Magnetic bead, variety of patterns available None Additional consumables required Pipette tip, bead Adapted from Rhoads et al J Pathol Inform. DOI: Croxatto et al Clin Microbes Infect. 22:

33 System Comparison Instrument (Manufacturer) WASP (Walk-away specimen processor) (Copan) BD Kiestra InoqulA (BD) Inoculation Throughput Up to 130/hr Up to 235/hr Plate loading Up to 360/hr Up to 600/hr Adapted from Rhoads et al J Pathol Inform. DOI: Croxatto et al Clin Microbes Infect. 22:

34 The Nuts and Bolts of Automating a Microbiology Laboratory BD Kiestra Style Dr. Anderson

35 Why TLA? Barnes Jewish Hospital is a 1300 bed tertiary care academic hospital Introduced the BD Kiestra TLA system in 2016 Decision to automate driven by: Consolidation of microbiology services (at least four additional hospitals added to workload) Decreasing amount of skilled staff Opportunity to design new laboratory space

36 Implementation Plan Should include the following: Instrument selection Space design Training Superusers? Who gets trained on what part? How will they get trained? Staffing How will you adjust to the change in workflow? Technical support Weekly conference calls? On site support? Data storage Server requirements met? How long will you store images? Verification approach Stepwise? All at once?

37 Implementation Timeline 10/19/16 MRSA screening go-live 5/30/17 VRE screening go-live 5/12/18 Blood culture go-live 1/27/16 Urine culture go-live 3/10/17 Throat culture go-live 2/12/18 GBS screening go-live

38 Verification of Kiestra Cultures Variables: Incubation Length Specimen Volume Media Type Inoculation Method Streak Pattern Incubation Environment Core Components: Precision Accuracy Considerations: Barcoding/labeling Carryover Specimen Delivery Image Quality Specimen Tracking LIS Data Transfer

39 Steps of a Kiestra Go-Live Optimization Verification Acceptable Develop Procedures and QA/QC Not Acceptable

40 Optimization Example: Throat Culture Variables we kept constant Streak pattern (four quadrant) Media used (BAP) Incubation conditions (35 CO2) Variables we explored Specimen volume Incubation time

41 Optimization of Other Culture Types Urine Varied reading time, streak pattern Blood Varied reading time, Gram stain parameters Volume? versus Environment?

42 Optimization Data Varied Volume and Incubation Time All Results Matched Conclusions: Cultures can be read at 18 hours. Sample volumes of 20 ul can be used Tested Cultures with Different Findings

43 Steps of a Kiestra Go-Live Optimization Verification Acceptable Develop Procedures and QA/QC Not Acceptable

44 Verification Example: Throat Cultures All studies were performed only after optimal parameters were decided upon Accuracy 100 prospective samples Plated 50uL onto BAP Incubated at 35 C in 10% CO 2 Compared manual methods to Kiestra

45 Throat Verification Accuracy Data Manual Kiestra 18 hrs 48hrs 48 hrs Positive percent agreement: 4/4 (100%) Negative percent agreement: 96/96 (100%) Read barcode? 100/100 Barcode labels correct? 100/100 Selected correct media? 100/100 Morphology as expected? 100/100 Correct streaking pattern? 100/100

46 Throat Verification: Precision 20 mock specimens prepared 5 S. pyogenes, 5 S. agalactiae, 5 S. dysgalactiae, and 5 A. heamolyticus Prepared 0.5 McFarland, plunged Eswab for 10 seconds, then placed Eswab into transport media Prepared culture replicates by Kiestra and traditional methodology in parallel

47 48 hrs 18 hrs Precision Example: Streptococcus pyogenes Manual Kiestra 24 hrs 48 hrs Agreement with Reference Method for all Studies: 20/20 (100%)

48 Steps of a Kiestra Go-Live Optimization Verification Acceptable Develop Procedures and QA/QC Not Acceptable

49 Procedures and Training The obvious things to address How to load the instrument How to access the software The not so obvious things to address How to manage work ques How will TLA change culture workup

50 Work Que Management Changes Many technologists managed work que by plate stacking Kiestra work que management must be done electronically This needs to be written into procedures and trained!

51 Culture Workup Changes: Urine Technologists must now assess clinical significance at 1000 colonies rather than ,000 CFU/mL x 10uL= 1000 colonies 100,000 CFU/mL x 1uL= 100 colonies Below Threshold Urine Threshold Estimation Tool Written into procedure and available at all benches

52 Culture Workup Changes: Urine Kiestra liquid handler is only capable of pipetting a minimum volume of 10 ul Manual (1 µl) BD InoqulA/TLA (10 µl) More confluent growth on Kiestra, yet more isolated colonies

53 QA/QC QA/QC metrics not yet well defined and vary from user to user Most important form of QA/QC is tracking of errors and issues Mislabels, plate contamination, uninoculated specimens, routing to wrong incubator, etc. Identification of error patterns allows for the implementation of specific QA measures

54 Example of QA Drop Cam Review Triggered by multiple errors in specimen inoculation Specimen drop never applied to plate Drop Cam takes picture of specimen when applied to the plate Instituted a daily review No longer necessary with new software upgrades

55 Example of QA- Positivity Rate Monitoring Important to do when culture parameters changed significantly

56 The Nuts and Bolts of Automating a Microbiology Laboratory WASP Lab Style Dr. Ledeboer

57 Sample Mix 30% Distribution of Sample types 70% 87% 13% Sample Not for culture Culture Samples for WASP automation Culture Samples remain manual Specimen types for automation # samples/ d Urine 264 Wounds 40 Nosocomial - MRSA 25 Group B Screen 24 Throat 21 Sputum 17 Feces 15 Group A Screen 15 Gynecology 13 Nosocomial - VRE 1 Eyes 1 Ears 1 Nose 1

58 Hourly Workload Workload/ hour/ day Sat Sun Mon Tue Wed Thu Fri

59 Laboratory Process Current Samples arrive at Lab Samples are sorted Samples are Acessioned/ label printed Samples are rebagged with req and label Samples transported to Microbiology Samples arrive in Samples Sorted by sample type Samples Logged in / labelled/ QC check Media Labelled Organize samples and media by Accession # Inoculate plates Organize by accession # Urine Samples removed from Incubator 24hour Read Growth Y ID made from N Perform subcultures/ ID/ Sens/ PP Incubate Sample N Y Full 24 hr Inc? Y Report Results N Routine Prelim/ OLD, Resp/Repro, Aer/Ana Bench, Feces, Blood Samples removed from Incubator 24 or 48+ hour Read Growth? Y Subcultures/ ID/ Sens/ PP Biochem tests/ Wet Re- Incubate Sample N Report Results or Reincubate for indicated time Verify ID/Sens MRSA/ VRE / Grp B Samples removed from Incubator 24 hour Read Growth? Y Identify / classify colonies N Report results

60 Laboratory Process - Post Automation Samples arrive at Lab Samples are sorted Samples are Acessioned/ labeled Samples transported to Microbiology Reincubate for required time Samples arrive in Can go on Y Samples Loaded onto automation Automated Plate read ID No growth Growth No Grow th N Grow th Report Results Process Manually Plate reading with 4 Interpretation workbenches Routine Prelim/ OLD, Resp/Repro, Aer/Ana Bench, Feces, Blood Growth? N Y Subcultures/ ID/ Sens/ PP Biochem tests/ Wet Re- Incubate Sample as required Verify ID/Sens Urine / MRSA/ VRE / Grp B Growth? N Y Identify / classify colonies Urine Perform Biochemical Report Results

61 Impact on Staffing # In scope staff scheduling Target FTE Current/ weekday 22 Future/weekday Current staffing mix Processing Routine Technical Special Technical Estimated Labor Savings/ Redeployment 7 FTE 2 0 Day shift E shift N shift

62 Impact on productivity Productivity Index = #samples / #FTEs worked Productivity for hours worked # FTE/d Productivity Index Current FTE Future FTE Productivity - Increased by 51%

63 Return On Investment Solution A 2 processors, 2 O2 and 2CO2 Solution B 1 processor, 1O2, 1CO2

64 Construction of the Physical Plant Plan way ahead for any construction requirements This is a long-term capital investment, make sure your layout will be sufficient for the next 10 years or more Avoid selecting a layout that will work, but is not optimal as changing layout in the future will only increase costs Plan a construction budget with a 20% contingency for changes in scope of project Confirm that your floors can accommodate the weight of instruments Are your doors and elevators adequate?

65 Sizing Your Automation Consider if you want to build in capacity or if you will expand as you need additional capacity How will this impact ROI Will future budget $ be available? Will your vendor assure you that new components will be reverse-compatible? Will you add additional functionality in the future?

66 Connectivity Engage your LIS vendor before you decide to purchase an automation system You will need an interface, what are your options? Will an existing interface suffice? We had an interface with WASP, but would not work with WASPLab as WASPLab needed bidirectional interface Is there a middleware solution? A new interface (generated by both the lab and LIS vendor) will take 6 months or longer to build

67 LIS Troubles We approached our LIS vendor and were immediately told that Cerner Millennium would not accommodate the type of interface the we needed. Follow-up conversations stated we would need to spend up to $50,000 to have the interface built and would need to upgrade our protocol version, another $100,000 Took 3 months just to get a meeting with LIS vendor Compared to MALDI, which we connected via middleware which took less than 1 day to connect!

68 More LIS Take a long-term approach Can you use your automation software as a middleware Will need connectivity to susceptibility testing systems, MALDI, other ID systems Consider the future, can you add rapid ID results, next-generation sequencing results? Consider if you will be changing LIS vendors at any time in the future, what impact will that have?

69 Does Automation Ultimately Work and Benefit Patient Care Dr. Anderson

70 A Traditional Workflow Problem: Time Out How significant of a problem is this? Take All Plates Out in AM We followed >200 blood cultures to find out AM Time spent at inappropriate temperature and atmosphere PM Return All Plates in PM

71 Results: Time Out Day 1 n=232 Day 2 n=232 Day 3 n=147 Day 4 n=35 Plate age (range) 1h51min- 25h37min 26h29m- 50h2m 51h5min- 75h17min 78h22m- 96h50m Cumulative time outside incubator (average) Cumulative time outside incubator (range) 26m 2h9m 5h48m 9h58m 2m-2h1m 52m-7h20m 3h3m-11h57m 6h22m-18h27m Plates as young as 26 hours may have spent as much as 7 hours outside of the incubator Does this matter?

72 Time Out Affect on Growth 26 hours old Neisseria gonorrhea Left out 7 hours before read (max) Left out 2 hours before read (average) Incubated entire time

73 Does Automation Enhance Recovery? Automation workflows dramatically decrease the amount of time cultures spend outside of incubator Could this increase organism recovery? We examined specific organism recovery data (Lainhart, JCM, 2018) Urine cultures One year prior to automation and one year after automation LainhartW,BurnhamC-AD Enhanced recovery of fastidious organisms from urine culture in the setting of total laboratory automation.jclinmicrobiol56:e

74 Recovery of Multiple Organisms Enhance LainhartW,BurnhamC-AD Enhanced recovery of fastidious organisms from urine culture in the setting of total laboratory automation.jclinmicrobiol56:e

75 Enhanced Recovery Through Automation Creates a potential for better patient care Has other surprising benefits! Recovery of Neisseria gonorrhea isolates is becoming more rare in the era of molecular testing At the same time, N. gonorrhea resistance is becoming more common Enhanced recovery has allowed us to assess resistance in our patient population

76 Does Automation Affect Turnaround Time? What do you think? Let s vote!!! Strauss and Bourbeau, 2015, JCM Urine cultures, two month period pre and post TLA (Kiestra) Only statistically significant difference in TAT was for negatives (39 vs. 40hrs) Theparee, Das, and Thompson, 2018, JCM Urine cultures pre- and post- TLA (Kiestra) and MALDI introduction Significant differences in TAT for preliminary negative interps, organism ID, and AST

77 Barnes Jewish TAT Experience We performed a detailed motion capture study Examined 215 urine samples pre-tla nad 203 urine samples post-tla Cultures processed 24/7, though only read on day shifts Quantified Time associated with processing steps TAT to different culture updates Yarbrough M., Lainhart W, McMullen A., Anderson N., and Burnham C. D. Data in submission

78 Processing Steps

79 TAT Study Results Time metric Pre-TLA Post-TLA Mood's Median Test Median IQR Median IQR p-value Pre-inoculation time (min) <0.001 Total processing time (min) < Time to Preliminary Report (hrs) Time to Identification (hrs) Time to Susceptibility (hrs) Time to Final Report (hrs) Positive cultures (hrs) Negative cultures (hrs) TLA total laboratory automation; IQR interquartile range (25 th percentile - 75 th percentile); min minutes; hrs - hours Yarbrough M., Lainhart W, McMullen A., Anderson N., and Burnham C. D. Data in submission

80 TAT Study Summary Data suggests only minor TAT benefit Approximately 4.5hr decrease in negative result TAT Did TLA help our lab??? Number of hospitals served Mean # urine specimens/day Median # urine specimens/day Pre-TLA Post-TLA Yarbrough M., Lainhart W, McMullen A., Anderson N., and Burnham C. D. Data in submission Changes made without additional staffing. Likely would have been impossible without TLA!

81 Will TLA Improve TAT? Data is currently conflicting Likely confounded by different laboratory specific factors Bottom Line: TLA will NOT improve TAT s for all labs TLA MAY improve TAT s for some labs if implemented in specific ways

82 Pitfalls Dr. Anderson

83 Workflow Difference in TAT experiences likely explained by different workflows TLA more amenable to continuous flow processes rather than batched processes Implementation of TLA without addressing workflow will simply magnify inefficiencies

84 Work Flow Analysis

85 Work Flow Analysis Are we loading samples efficiently???

86 Work Flow Analysis How does this bulk of specimen processing affect workflow the next day????

87 Culture Imaging Times Much work not ready to be reviewed during reading shift. Delayed one day. Reading Shift

88 Culture Imaging Times Reading on second shift decreased TAT for nearly half of all cultures Reading Shift Proposed 2 nd Reading Shift

89 Other Pitfalls- Instrument Downs Good morning. Sorry to bug you but the Kiestra has been alarming since midnight and the InoquIA is not working. We haven t been processing specimens for the last 8 hours. Any advice would be greatly appreciated. (content edited for general audiences) Next Steps?

90 Instrument Downs What NOT to do: Stop automation use altogether Often only part of the system goes down and it usually can be bypassed Taking cultures completely off of automation line will potentially affect workflow for the entire life of the culture! What to do: Find out exactly what part of the instrument is affected Find out if it can be repaired and/or bypassed

91 Instrument Downs Have a Downtime Procedure

92 Other Pitfalls New technology Need to develop new expertise! Unique training needed for day to day work and troubleshooting Space considerations Will the instrument fit? Current models only have limited modularity General maintenance Who does this? What warrants re-verification? How long will the instrument need to be down?

93 Other Pitfalls Change management for staff LIS (data storage and integration) Impact on safety Costs ROI (return on investment) may not be clear initially Consider ALL costs (remodels, LIS considerations, change management, etc.)

94 What is to Come Dr. Ledeboer

95 How can we use these images for automation Software analysis - Image differentials Time = 24 hours Time = 0 hours Differential

96 The Algorithm

97 How it Works

98 Result Definitions MP/AP = Manual Positive, Automation Positive (TP) MN/AN = Manual Negative, Automation Negative (TN) MN/AP = Manual Negative, Automation Positive (FP) MP/AN = Manual Positive, Automation Negative (FN)

99 Results Performance of WASPLab TM digital imaging compared to manual reading Clinical test site No. of specimens tested Results (no.) a Performance (% [95% CI]) b MP/AP MN/AN MN/AP MP/AN Sensitivity Specificity Prevalence ( (95-100) 96) 2.1% ( (89-100) 90) 1.6% ( (91-100) 93) 7.3% ( (90-100) 91) 4.6% Total ( (90-100) 91) 2.4% a MP/AP, manual Pos automation Pos; MN/AN, manual Neg/automation Neg; MN/AP, manual Neg/automation pos; MP/AN, manual pos/automation Neg. b CI, confidence interval. Faron et al. 2016

100 Can we use this software for VRE screening? 3 sites Specimens (n=104,730) Rectal Eswabs TM Media (n=2) Colorex VRE (BioMed Diagnostics) Oxoid VRE (Thermo Fisher Scientific) Reference method Manual reading Discrepant analysis Images reviewed by supervisor Compare Automated

101 Representative Images Negative Positive Break through growth Colorex VRE Oxoid VRE Faron et al. 2016

102 Results Performance of WASPLab TM digital imaging of VRE plates compared to manual reading Clinical test site No. of specimens tested Results (no.) a Performance (% [95% CI]) b MP/AP MN/AN MN/AP MP/AN Sensitivity Specificity 1 11,438 1,474 9, (99-100) 91.6 (91-92) 2 75,518 2,822 64,535 8, (99-100) 88.8 (88-89) 3 17,774 2,107 14,315 1, (99-100) 91.4 (91-92) Total 104,730 6,403 87,979 10, (99-100) 89.5 (89-90) PPV c (%) NPV c (%) Prevalence % % % % a MP/AP, manual Pos automation Pos; MN/AN, manual Neg/automation Neg; MN/AP, manual Neg/automation pos; MP/AN, manual Pos/automation Neg. b CI, confidence interval. c PPV, Positive Predictive Value; NPV. Negative Predictive Value Faron et al. 2016

103 Discrepant analysis Discrepant analysis of Manual Negative/Automation Positive Plates Discrepant Category MN/AP a Automation Positive 2 nd Manual Positive Residual Matrix/Yeast Borderline Colors Total number of plates 10, ,234 1,616 Colorex VRE Oxiod VRE a Manual Negative/Automation Positive Faron et al. 2016

104 Comparison of agars Comparison of 2 Chromogenic Agars for the detection of VRE using automated scoring Results (no.) a Chromogenic media No. of specimen s tested Performance (% [95% CI]) b MP/AP MN/AN MN/AP MP/AN Sensitivity Specificity Colorex VRE 86,956 4,296 73,664 8, (99-100) 89.1 (89-89) Oxoid VRE 17,774 2,107 14,315 1, (99-100) 91.4 (91-92) a MP/AP, manual Pos/automation Pos; MN/AN, manual Neg/automation Neg; MN/AP, manual Neg/automation Pos; MP/AN, manual Pos/automation Neg. b CI, confidence interval. c PPV, Positive Predictive Value; NPV. Negative Predictive Value Faron et al. 2016

105 Applying Algorithms to GAS Evaluated 250 throat swabs submitted from single center Specimens tested by: PCR, BAP, Colorex Strep A Compared results of manual read to automated read; compared BAP to chromogenic agar Dien Bard J, et al. ECCMID 2018.

106 What about GBS? 254 vaginal/rectal swabs All swabs were initially incubated in LIM for 18-24h at degrees C Compared ChromID GBS to Carrot Broth Equivalent performance Compared WASPLab segregartion software to CLS read Have subsequently increase n to >4000 specimens enrolled Multi-Center Study comparing with CDC method and PCR currently enrolling Visual Exam. SSW Negative Positive Negative Positive 0 89 Pham and VanHorn, ASM Abstract. 2018

107 Incorporating into the laboratory Negative Specimens Batch viewing 40 images/page Batch report Non-negative Specimens Still requires Technologist View on HD monitor Positive vs Matrix or Yeast Standard of care

108 Manual Processing Technologist Labor is $40.00/hour (w/benefits) Automated Processing $6.40 in labor/negative specimen 9.6 min/negative specimen a $1.33 in labor/negative specimen ~2 min/negative specimen Cost of negative workup for the study (n = 87,979) $563, in labor $117, in labor Savings = $445, a. Shadel et al. Surveillance for vancomycin-resistant enterococci: type, rates, costs, and implications.

109 Can it Quantitate?

110 Yes, IT CAN!!

111 Blood Plate Reading

112 False Positive Example SW POS, human NSG T18H T0 BLOOD MAC CONKEY CNA

113 Can Computers Quantitate and Identify Organisms?

114 Can we use this software to Analyze Urine Using Non-Chromogenic Plates? 3 sites Specimens (n=13,465) Urines (Plated Blood, MacConkey, CNA) Algorithm results POS >10 colonies on any plate Neg 10 colonies in all 3 agars Reference method Manual reading Site specific procedures for results Discrepant analysis Images reviewed by supervisor Compare

115 How well does it work? Performance of WASPLab TM digital imaging software compared to manual reading of BAP, MAC and CNA No. of specimens tested Results (no.) a Performance (% [95% CI]) b MP/AP MN/AN MN/AP MP/AN PPA c NPA c Site (98-99) 50.0 (48-52) Site (99-99) 87.2 (86-88) Site (93-96) 75.1 (73-77) Total (97-98) 74.0 (73-75) a MP/AP, manual Pos automation Pos; MN/AN, manual Neg/automation Neg; MN/AP, manual Neg/automation pos; MP/AN, manual pos/automation Neg. b CI, confidence interval. c PPA, Positive Percent Agreement; NPA, Negative Percent Agreement

116 Urines are not all 1s and 0s Consideration of manual negatives based on rules for interpretation MCW Automation Manual No Growth NFW a NSG b Positive Negative Positive Rules ~ 92% of all MN/AP specimens Total 5201 a No Further Workup: contains > 3 pathogens on the plate b No Significant Growth: Consistent with normal skin and urethra flora LAB results: POS: Positive 10 CFU, Catheter any growth, Urinary clinic any growth NG: No Growth NSG: No Significant Growth - 10 CFU but consistent with Normal skin flora NFW: No Further Workup - 10 CFU, but >3 pathogens (fecal contamination) NEG

117 Summary of 41 manual positive, automation negative specimens with lab report 6 specimen lab report negative 15 specimens (growth) were from catheters <10 cfu 5 specimens >10 colonies called at 48 hours 4 GPR 1 S. anginosus 12 from Urinary Clinic policy similar to catheters 1 unspecified specimen from 16 th street clinic (1 of many out patient facilities) Policy states minimum ID on pathogens less than 100,000 CFU/mL 1 Pregnant patient Growing GBS - reportable Only 1 image at 24 hours had >10 colonies after second review (non-lab report) Evaluation of the 41 manual positive, automation negative specimens by source at MCW Void Catheter Unspecified 12 a,b 17 c,d,e 12 b,f a 3 specimens were negative for growth by laboratory report b 2 specimens were positive after 48 hours c 1 specimen was negative for growth by laboratory report d 1 specimen was positive after 48 hours e Policy states min ID for any growth from Catheter f 2 specimen was negative for growth by laboratory report

118 False Positive Example SW POS, human NSG T18H T0 BLOOD MAC CONKEY CNA

119 Overall Performance based on colony count alone (>10 CFU) and re-evaluation of MP/AN specimens

120 Can AI Identify Organisms, Based on Morphology Timm and Culbreath, ECCMID 2017

121 Summary, Where is the Field and Where are We Going?

122 Questions?