Quality Monitoring Approach for Optimizing Antinuclear Antibody Screening Cutoffs and Testing Work Flow

Transcription

1 ARTICLES Quality Monitoring Approach for Optimizing Antinuclear Antibody Screening Cutoffs and Testing Work Flow Danyel H. Tacker 1 * and Peter L. Perrotta 1 Background: An antinuclear antibody (ANA) testing strategy involving enzyme immunoassay (EIA) screening that reflexed to immunofluorescence assay (IFA) was implemented, monitored, and optimized for clinical utility. Methods: The clinical utility, test performance, and workload implications of various ANA testing strategies were compared during the following study phases: (a) Preimplementation (n = 469) when IFA was used for all ANA screening, (b) Verification (n = 58) when EIA performance was confirmed, (c) Implementation (n = 433) when a reflexive strategy (EIA screen/ifa confirmation) was implemented, and (d) Postimplementation (n = 528) after the reflexive strategy was optimized. Sequential samples were captured in the Preimplementation, Implementation, and Postimplementation phases for clinical performance evaluation. Results: Clinical performance of the EIA screen, per ROC analysis yielded area under the curve (AUC) of in the Implementation phase and increased to Postimplementation (P < 0.01); AUC for IFA similarly increased, from to (P = 0.05). The reflexive testing strategy increased screening sensitivity from 61% Preimplementation (IFA) to 98% (EIA) at Implementation and was maintained after optimization (98%, Postimplementation). Optimization decreased the false-positive rates for both EIA (from 40% to 18%) and IFA (18% to 8%) and was associated with reductions in daily full-time equivalent (by 33%) and IFA slide use (by 50%). Conclusions: Continuous quality monitoring approaches that incorporate sequential data sets can be used to evaluate, deploy, and optimize sensitive EIA-based ANA screening methods that can reduce manual IFA work without sacrificing clinically utility. IMPACT STATEMENT Patients undergoing ANA testing will benefit from more accurate screening results and expanded clinical data, while laboratories performing the testing will benefit from workload alterations enabling automated screening with more efficient and clinically useful manual IFA work. Evidence demonstrates a process for optimizing screening assay cutoffs that simultaneously improve test performance and laboratory efficiency. Knowledge of how to build and cautiously optimize reflexive ANA testing strategies using sequential data sets will be gained from the information presented. 1 Department of Pathology, West Virginia University, Morgantown, WV. *Address correspondence to this author at: West Virginia University Hospital Clinical Laboratory, Box 8009, Morgantown, WV Fax ; dtacker@hsc.wvu.edu. DOI: /jalm American Association for Clinical Chemistry 678 JALM :06 May 2017

2 ANA Testing Algorithm Optimization ARTICLES Fig. 1. The ANA testing algorithm uses EIA screening and reflexes to IFA confirmation only when screening results are equivocal or positive. The diagnosis and classification of connective tissue disease (CTD) 2 is primarily based on clinical findings and patient history. However, measurement of antinuclear antibodies (ANAs) is commonly performed as a screening test for patients who do not meet diagnostic criteria for CTD and have a low likelihood of clinical disease. Despite concerns about ANA test utilization (1, 2), clinical laboratories perform large numbers of ANA screening tests. ANA testing is performed by several methods, the most prominent being indirect immunofluorescence assay (IFA) and immunoassay, whether it be multiplexed immunoassay (MIA) or solid-phase enzyme immunoassay (EIA). IFA using HEp-2 substrate is considered the gold standard method for clinical ANA testing (2 4); it is historically a manual technique with some automation recently available in the US market. MIA and EIA are generally more automatable than IFA and have variable sensitivity for CTD depending on how they are configured (2, 3, 5 8). Staffing challenges and efforts to centralize and consolidate testing within hospital systems have prompted laboratories to examine their ANA testing strategies and work flows. This trend is reflected in recent literature and the updated 2015 American College of Rheumatology (ACR) position statement on ANA testing (4, 8). Our institution implemented EIA-based ANA screens in July of 2015 as a way to focus manual IFA testing on specimens more likely to be positive. Specimens with equivocal or positive ANA screening results found with the sensitive, HEp-2 derived EIA technique were directed to IFA for confirmatory testing (Fig. 1). This algorithm meets clinical needs while accommodating increasing requests for ANA testing without increasing laboratory staffing levels. This report describes our experience launching the 2 Nonstandard abbreviations: CTD, connective tissue disease; ANA, antinuclear antibody; IFA, immunofluorescence assay; EIA, enzyme immunoassay; MIA, multiplexed immunoassay; ACR, American College of Rheumatology; TP, true positive; TN, true negative; FP, false positive; FN, false negative; PPV, positive predictive value; NPV, negative predictive value; AUC, area under the curve; FTE, full-time equivalent. May : JALM 679

3 ARTICLES ANA Testing Algorithm Optimization testing algorithm, monitoring its performance using a continuous quality monitoring approach, and adapting assay cut points to optimize test and algorithm performance. METHODS The ANA Screen Test System (ZEUS Scientific) was selected for EIA-based screening. This kit, cleared by the Food and Drug Administration, uses HEp-2 cell extracts to present antigens in the solid phase, which gives the assay high sensitivity (6). Testing was performed in accordance with manufacturer instructions on automated EIA platform (Dynex Agility) interfaced to the laboratory information system (Sunquest 6.2). Assay data were expressed as unit-less ratios calculated using a runspecific calibration-associated adjustment factor and then interpreted in qualitative terms. Both the ratio and the interpretive result were reported. Manufacturer-defined ratio cutoffs were <0.90 for negative results and 1.10 for positive results, leaving an equivocal range of Any specimen with equivocal or positive ANA screening result reflexed (Fig. 1) to IFA (Kallestad HEp-2 Kit, Bio-Rad), performed according to manufacturer instructions. This assay served as the local predicate for ANA testing since A single minimum dilution (e.g., 1:40) was prepared first to minimize unnecessary titer preparation and maximize the number of specimens that could be tested on a single slide (i.e., 10 patient samples and 2 controls per slide). Fluorescence patterns (e.g., homogeneous, speckled, nucleolar) and intensities (e.g., 0, 1+, 2+) were recorded. All specimens with positive immunofluorescence at the minimal dilution were further titrated to the end point. Highest titer with positive signal, observed pattern, and interpretive comment were reported. Data were collected for 4 phases: (i) Preimplementation before algorithm launch, (ii) Verification data set used during assay validation, (iii) Implementation when patient testing was launched, and (iv) Postimplementation when changes were made to the EIA cutoff and IFA minimum dilution. Data extracted from laboratory middleware (Data Innovations Instrument Manager, version ) and the laboratory information system (SunQuest SMART 6.2) included patient age and sex, ordering physician, ordering location, EIA results, and IFA results as applicable. All data were compiled in spreadsheets using Excel For all phases, equivocal results were considered positive in terms of data handling. The Preimplementation data set included sequential orders for ANA testing placed between 18 May and 24 June During this phase, IFA was the only test used for ANA testing and used a 1:40 minimal dilution. IFA results positive at 1:40 but negative at 1:80 were considered equivocal, and results positive at or above 1:80 were considered positive. The Verification data set contained test results for specimens used to confirm the performance of the EIA screening test and included precision and correlation studies and were conducted in May Specimen remnants (n = 57) were selected from the clinical IFA workload to include a wide range of results. Clinical disease (present or absent) was recorded for each patient before specimen deidentification to help resolve discordant EIA and IFA results. CTD represented by specimens tested in the Verification phase included systemic lupus erythematosus (n = 8), undifferentiated CTD (n = 7), Sjögren disease (n = 6), mixed CTD (n = 2), dermatomyositis (n = 2), CREST syndrome (n = 1), and discoid lupus/systemic lupus erythematosus overlap syndrome (n = 1). The same minimal dilution and resulting criteria were used in Verification as for daily clinical testing, as described above for Preimplementation testing. The Implementation data set included sequential orders for ANA testing placed between 12 August and 20 September During this phase, manufacturer-suggested EIA cutoffs were used, and IFA reflexes used the 1:40 minimal dilution; the lower threshold for IFA positivity 680 JALM :06 May 2017

4 ANA Testing Algorithm Optimization ARTICLES remained 1:80. All EIA results generated were monitored in terms of qualitative and quantitative screen results and correlation between EIA and applicable IFA results. The Postimplementation data set included sequential orders for ANA testing placed between 21 September and 4 November 2015, after ratio cutoffs for the EIA were optimized and the IFA minimal dilution was increased to 1:80; updated positivity threshold for IFA was 1:160. The same data elements were monitored as during the Implementation phase. Medical records were accessed for all orders in the sequential data sets (n = 1430). The Preimplementation data set yielded clinical data for 297 of 469 (63%) sequential orders. The Implementation data set yielded clinical data for 320 of 428 (75%) sequential orders. The Postimplementation data set yielded clinical data for 384 of 528 (73%) sequential orders. In total, 1001 orders (70%) provided clinical information. This study was approved by the West Virginia University Institutional Review Board in a general protocol used for laboratory quality assurance/utilization assessment and improvement projects. Records excluded from clinical assessment either had (a) no follow-up in the medical record (i.e., lost to follow-up), (b) no medical record available (i.e., outreach order placed by an outside provider), or (c) pending referral to the rheumatology division (i.e., unconfirmed suspicion based on clinical presentation and/or results noted by ordering provider or care team) (see Fig. 1 in the Data Supplement that accompanies the online version of this article at issue6). EIA and/or IFA performance was assessed based on clinical diagnosis and specialist notes and graded separately as true positive (TP), true negative (TN), false positive (FP), or false negative (FN). Of 1001 orders included in clinical performance assessment, 155 (15%) came from patients with CTD. Diseases represented were systemic lupus erythematosus (n = 43), sclerotic disease (n = 26), Sjögren syndrome (n = 25), overlap syndrome (n = 22), drug-induced lupus (n = 4), mixed CTD (n = 4), inflammatory myopathy (n = 3), and undifferentiated CTD (n = 28). Clinical performance calculations included (a) sensitivity = TP/(TP + FN); (b) specificity = TN/(TN + FP), (c) positive predictive value (PPV) = TP/(TP + FP), (d) negative predictive value (NPV) = TN/(TN + FN), and accuracy = (TP + TN)/all results. ROC curves were prepared. For EIA, cutoff conditions were assessed in 0.01-U increments between 0.90 and 2.00, in 0.05-U increments from 2 to 5, and in 0.1-U increments between 5 and 6. For IFA, cutoffs were assessed in serial dilution increments (i.e., 1:40, 1:80, 1:160, 1:320, 1:640, 1:1280, 1:2560, 1:5120, 1:10240). Area under the curve (AUC) was calculated for each assay and condition, and these data were used to optimize the EIA and IFA cutoffs. Along with Microsoft Excel, XLSTAT package (Addinsoft) and Minitab 17 were used for data analysis. Pearson χ 2 analysis, Fisher F-test, and Student t-tests were used as appropriate to compare data sets; P values <0.05 were statistically significant. RESULTS Correlation between EIA and IFA results was determined during Verification (Table 1). Correlation data demonstrated higher sensitivity (100% vs 85%) and NPV (100% vs 86%) for the EIA test vs IFA, but lower specificity (70% vs 83%) and PPV (75% vs 82%), respectively. The new reflexive ANA algorithm (Fig. 1) was monitored using cumulative, tabular dashboards that were reviewed on a daily basis (see Table 1 in the online Data Supplement). During Implementation, 279 of 428 (64%) sequential EIA tests reflexed to IFA. Of these, 188 (67% of reflexes) were IFA negative, 25 (9% of reflexes) were IFA equivocal, and 66 (24% of reflexes) were IFA positive at 1:80. May : JALM 681

5 ARTICLES ANA Testing Algorithm Optimization Table 1. Verification phase test performance. EIA IFA Total Positive Negative Positive Negative CTD No CTD Sensitivity 100% (27/27) 85% (23/27) Specificity 70% (21/30) 83% (25/30) PPV 75% (27/36) 82% (23/28) NPV 100% (21/21) 86% (25/29) FN 0% (0/57) 7% (4/57) FP 16% (9/57) 9% (5/57) Implementation data were reviewed with rheumatologists to discuss the implications of high EIA reflex rates and the clinical relevance of low-titer IFA results. To provide a baseline and point of comparison for IFA performance related to the new algorithm, IFA-only Preimplementation data were pulled, and performance parameters were calculated as for the Implementation data. Comparison of Preimplementation and Implementation data showed significant (P <0.001) shifts in performance (Table 2) for IFA. There was a gain in IFA PPV (from 27% to 49%), but decreases in sensitivity (from 61% to 58%), specificity (77% to 73%), NPV (94% to 78%), and accuracy (75% to 69%) were observed. These changes were largely attributable to an increase in the IFA FN rate (from 5% to 13%). IFA FP rate decreased marginally to 18% from 20%. EIA performance (Table 2) in the Implementation phase yielded sensitivity of 98%, specificity of 51%, PPV of 31%, NPV of 99%, accuracy of 60%, FN rate of 0.3%, and FP rate of 40%. This was the first large-scale sequential data set at our site for EIA testing and formed the baseline for further comparisons. ROC curves (Fig. 2) showed that AUC for EIA (Fig. 2A) in Implementation was From associated calculations, the highest achievable sensitivity for EIA was 98%, starting with the default cutoff of Specificity of 90% was achievable at a cutoff of 2.30 with concomitant sensitivity of 51%. ROC analysis for IFA (Fig. 2B) yielded AUC in Implementation and in Preimplementation; this comparison did not reach significance (P = 0.235). The highest achievable sensitivity for IFA was 58%, starting with cutoff titer of 1:40. Specificity of 90% was achievable at a titer of 1:160 with concomitant sensitivity of 38%. Based on review of these data with the rheumatologists and referring to ACR guidelines (1), the testing algorithm was altered as follows: (a) EIA ratios <1.30 were negative, ratios from 1.30 to 1.49 were equivocal, and ratios 1.50 were positive; and (b) a 1:80 minimal dilution would be performed for IFA. The conservative EIA and IFA adjustments, compared to the 90% achievable specificity of cutoff 2.30 for EIA and 1:160 for IFA, were chosen to balance sensitivity and specificity. The reflexing algorithm itself (Fig. 1) was not changed. Optimization decreased the EIA reflex rate by 28% (to 36%, P <0.001), as 189 of 528 sequential orders for EIA screens reflexed to IFA in the Postimplementation phase. Of these, 115 (61%) were IFA negative, 22 (12%) were IFA equivocal, and 52 (28%) were IFA positive at 1:160. As seen in Table 2, the updated testing algorithm did not affect EIA sensitivity (98% for both) or NPV 682 JALM :06 May 2017

6 ANA Testing Algorithm Optimization ARTICLES Table 2. Test performance data in sequential specimens by phase. a Preimplementation (n = 469 orders; 297 had clinical information) Implementation (n = 428 orders; 320 had clinical information) Postimplementation (n = 528 orders; 384 had clinical information) IFA-only testing minimal dilution 1:40 EIA cutoff 0.90 IFA reflex minimal dilution 1:40 EIA cutoff 1.30 IFA reflex minimal dilution 1:80 Positive Negative Positive Negative Positive Negative Positive Negative Positive Negative CTD No CTD Count Sensitivity 61% (22/36) 98% (57/58) 58% (33/57) 98% (60/61) 74% (43/58) Specificity 77% (201/261) 51% (134/262) 73% (94/128) 78% (252/323) 86% (62/72) PPV 27% (22/82) 31% (57/185) 49% (33/67) 46% (60/131) 81% (43/53) NPV 94% (201/215) 99% (134/135) 78% (94/118) 99.6% (252/253) 81% (62/77) Accuracy 75% (223/297) 60% (191/320) 69% (127/185) 81% (312/384) 81% (105/130) FN rate 5% (14/297) 0.3% (1/320) 13% (24/185) 0.3% (1/384) 12% (15/130) FP rate 20% (60/297) 40% (128/320) 18% (34/185) 18% (71/384) 8% (10/130) Specificity of 90% reached Lowest cutoff 1: : :160 Sensitivity 46% 51% 38% 82% 58% Highest achievable performance 1: : :80 Sensitivity 62% 98% 58% 98% 73% Specificity 77% 57% 73% 78% 86% Pearson χ 2 test NA NA <0.001 vs Pre IFA <0.001 vs Imp EIA <0.001 vs Pre IFA<0.001 vs Imp IFA a NA, not applicable; Pre, Preimplementation phase; Imp, Implementation phase; Post, Postimplementation phase. May : JALM 683

7 ARTICLES ANA Testing Algorithm Optimization Fig. 2. ROC curves for EIA (A) and IFA (B). Pre, Preimplementation phase; Imp, Implementation phase; Post, Postimplementation phase. 684 JALM :06 May 2017

8 ANA Testing Algorithm Optimization ARTICLES Table 3. Comparisons of Implementation and Postimplementation data group features. a Preimplementation phase Implementation phase Postimplementation phase P Number of sequential tests NS Phase interval, days NS Age, years ± SD (95% CI) 51.0 ± 18.0 (1.6) 48.4 ± 18.8 (1.8) 51.3 ± 18.3 (1.6) Pre vs Imp, b Pre vs Post, b Imp vs Post, b % Females Pre vs Imp, b Pre vs Post, b Imp vs Post, b % Local inpatient tests Pre vs Imp, c % Local outpatient tests Pre vs Post, c % Regional/outreach tests Imp vs Post, c Median ANA screening result, quantitative (95% CI) NA 1.13 ( ) 0.95 ( ) b % of EIA reflexing to IFA NA <0.001 b Median end point dilution for positive IFA results 1:80 1:160 1:160 Pre vs Imp, c a NS, no statistics; Pre, Preimplementation phase; Imp, Implementation phase; Post, Postimplementation phase. b By Fisher F-test. c By Pearson χ 2 test. Pre vs Post, <0.001 c Imp vs Post, c (99% and >99%, respectively), but did improve specificity (from 51% to 78%), PPV (from 31% to 46%), and accuracy (from 60% to 81%). EIA FN rate did not change but FP rate decreased (from 40% to 18%); the shifts in performance were significant (P <0.001). Also seen in Table 2, the updated testing algorithm showed significant (P <0.001) improvements in performance for IFA, including increases in sensitivity (from 58% to 74%), specificity (from 72% to 86%), PPV (from 49% to 81%), NPV (from 78% to 81%), and accuracy (from 69% to 81%); FN rate changed marginally (from 13% to 12%) and FP rate decreased (from 18% to 8%). Postimplementation IFA clinical performance comparisons to Preimplementation phase data also yielded significant improvement (P <0.001). EIA AUC (Fig. 2A) increased to in the Postimplementation phrase from in the Implementation phase (P <0.001). The highest achievable sensitivity for EIA was again 98%, starting with the default cutoff of Specificity of 90% was achievable at a cutoff of 1.84 with concomitant sensitivity of 82%. IFA AUC (Fig. 2B) similarly increased, from to overall (P = 0.05). However, the change in IFA AUC between Preimplementation and Postimplementation (from to 0.808) was not significant (P = 0.125). The highest achievable sensitivity for IFA was 73% Postimplementation, starting with cutoff titer of 1:80. Specificity of 90% was achievable at a titer of 1:160 with concomitant sensitivity of 58%. There were no significant differences between the demographics representing the Preimplementation, Implementation, and Postimplementation groups (Table 3). Additionally, there were no significant differences between median EIA ratios or median IFA end point dilutions for the Implementation and Postimplementation phases. Preimplementation IFA median end point dilution (1:80) was significantly different than median end point dilutions for May : JALM 685

9 ARTICLES ANA Testing Algorithm Optimization Table 4. Work and reagent comparisons at the IFA bench. Preimplementation Implementation Postimplementation Order count Work days represented Orders/work day No. (%) reflexing to IFA per work day 18 (100) 10 (64) 6 (36) No. IFA needing additional titer, from previous day (% of IFAs) 5 (25) 6 (60) 4 (66) No. wells needed (1 well/screen, 3 wells/titer) 18+15= = =18 Specimens per slide 10 Slides consumed per day No. wells wasted per day a 14+12= =16 2+8=10 Slides wasted Rate of slide waste 65% 53% 50% Hands-on process time per day, h b Total process time per day, h c FTE to complete IFA each day 6/8 = /8 = /8 = 0.5 a Negative IFA screens + noncontributory titers (of 3 prepared). b First slide = 1.5 h h for each additional slide. c First slide =3h+1hforeach additional slide. both the Implementation (1:160, P = 0.005) and Postimplementation (1:160, P <0.001) phases. IFA-driven work flow Preimplementation required 4 IFA slides, 3 h of hands-on time, and 6 h of total testing time [0.75 full-time equivalent (FTE) per day] (Table 4). During the Implementation phase, 3 slides, 2.5 h of hands-on time, and 5 h of total testing time (0.625 FTE) were required each day. Optimization reduced IFA utilization to 2 slides,2hofhands-on time, and4hof total testing time (0.5 FTE) per day Postimplementation. The net 33% decrease in FTE and 50% decrease in IFA slide use over time reflected the decreasing number and percent of samples that reflexed to IFA. DISCUSSION ANA screening methods used in clinical laboratories are typically designed for maximal sensitivity to minimize FN results; however, the performance of ANA tests varies considerably and clinical performance issues with EIA and IFA alike have been reported (2, 3, 7, 9). The consequences of using assays that are too sensitive include high FP rates, overutilization of IFA confirmatory testing, inappropriate diagnoses, and unnecessary specialty referrals. Such concerns provided impetus for the ACR recommendation for parsimonious use of ANA testing and for including ANA serology and subserology among the ACR Choosing Wisely Top 5 List (1, 2). Conversely, the major consequence of using assays that are less sensitive, or inappropriately avoiding testing, is an increase in FN rates, with missed diagnoses and opportunities for intervention. Most EIA-based ANA screening assays are configured to produce nonnegative results equivalent to a 1:40 dilution by IFA, which can yield 40% FP rates in healthy donors and is seen as low-yield and/or dismissible information for rheumatology specialists (2, 3). For this reason, the ACR recommends that minimal dilutions for IFA-based methods start at 1:80 or 1:160 to decrease the FP 686 JALM :06 May 2017

10 ANA Testing Algorithm Optimization ARTICLES results generated in this population to 15% or 5%, respectively (2). Such a change in minimal IFA dilutions, without concomitant adaptations of EIA screening assay cutoffs, can only yield disparate performance data (6). With this in mind, we recognized that any increase to EIA screening cutoffs should be paired with a conservative update to the minimal IFA dilution, thus (a) making the IFA more consistent with ACR guidelines, (b) preventing increases to FN rates while decreasing FP rates, and (c) reinforcing the utility of the IFA test as a confirmatory method. Aware of the shortcomings of EIA-based ANA screening approaches and recognizing the high sensitivity of our chosen screening assay, we launched a reflexive ANA algorithm that incorporates IFA as a confirmatory test (Fig. 1). We used the default settings for EIA at the time of launch to enable clinical assessments of algorithm performance within our population. Verification data (Table 1) suggested that our reflex rate to IFA would be high (Table 2), but the IFA reflex rate of 64% upon Implementation was beyond our expectations. Bernardini et al. (6) suggested that testing sites using EIA for ANA testing should modify assay cutoffs according to local data and demonstrated this with the Zeus kit and selected specimens. Our quality monitoring dashboard (see Table 1 in the online Data Supplement), clinical truth tables (Table 2), and ROC curves (Fig. 2) helped us predict optimal assay cut points over a 5-week Implementation phase, and our rheumatologists agreed the changes should improve test performance in a clinically useful manner. The EIA cut point that we determined as optimal for the Zeus EIA kit (positive at 1.30) was similar to that proposed by Bernardini et al. (6). Citing the ACR evidence-based guidelines suggesting that a 1:80 positive but 1:160 negative IFA result could be considered equivocal (2), a need for the IFA test to yield a lower FP rate than the EIA test, and clinical perspectives that the minimal IFA dilution of 1:40 was suboptimal due to high a Preimplementation FP rate of 20%, the rheumatologists agreed that simultaneously updating both the EIA and IFA methods was the correct approach. The 8% increase in discoverable IFA FN rate (Table 2) between Preimplementation and Implementation was unexpected, but we see this as a strength of the algorithm. We attribute this increase to the consistently high sensitivity of the EIA (98% regardless of cutoff; Table 2), which likely prompted referrals to Rheumatology and/or subserology testing that revealed true clinical disease despite negative IFA. Postimplementation, we noted that the 28% reduction in the EIA reflex rate was mostly attributed to reduced FP screens (Table 2). Optimization improved EIA screening accuracy by 21%, specificity by 27%, and PPV by 15%. All of these improvements to EIA were coupled to improvements in IFA sensitivity (by 14%), specificity (by 13%), and accuracy (by 12%). Accuracy for both tests at the end of the study was 81%, an unexpected and reassuring finding. A major reason for implementing the reflexive algorithm was to reduce manual work at the IFA bench. However, Implementation using manufacturer-recommended EIA cutoffs and maintaining the existing IFA minimal dilution did not initially alter manual IFA work flow (Table 4). However, work flow at the IFA bench Postimplementation yielded clear gains in operator time, materials, and manual workload. Consequently, we are now able to perform IFA testing on fewer days per week, which further minimizes slide waste and ensures that we can manage the relatively large and increasing ANA workload at our site. The first strength of our quality monitoring approach was that we followed hundreds of sequential clinical specimens in each phase, using realtime work flow under normal operating conditions. Published studies tend to report data derived from selected diseased populations and healthy blood donors when addressing clinical test performance May : JALM 687

11 ARTICLES ANA Testing Algorithm Optimization (2, 3, 6, 9). While this approach is important when characterizing EIA performance, sequential testing provides a different view of how work flow, resource needs, and communication with caregivers are applied in daily practice. We plan to incorporate this strategy when validating new serology methods as a result of this experience. The second strength of this approach is that a reflexive testing algorithm was launched that could be maintained despite updates to individual testing parameters. This step provided continuity to the data sets and facilitated comparisons. These standardized data sets were then presented to clinicians who could help us monitor the performance of the testing strategy. A recognized limitation of our study is that we changed both the EIA and IFA cutoffs at the same time. As we discussed, EIA formulations are generally tuned to an IFA dilution of 1:40; this creates a paradigm whereby if the sensitive EIA screen cutoff is increased but the IFA minimal dilution is not, the confirmatory test could be rendered clinically less specific than the screening method and compromise utility. Also, our rheumatologists had requested an increase in the IFA minimal dilution to 1:80 that coincided with our efforts to implement the reflexive algorithm. Thus, we agreed to launch the algorithm with the lower IFA minimal dilution and then update it if the EIA screen showed a low FN rate. Since the EIA FN rate was <1% after implementation (Table 2), we were confident in optimizing the IFA minimal dilution along with the EIA cutoff. After optimization, we again demonstrated a low EIA FN rate and continued to identify similar numbers of FN IFA cases. We also did not perform IFA on specimens with negative EIA screening results upon algorithm implementation. We did, however, use clinical data to determine if the EIA provided FN results, and only found 2 such examples in 684 charts. The literature contains reports of patients who have clinical disease and negative ANA tests by both EIA and IFA (2, 3, 6, 7, 9). Thus, no ANA testing strategy will detect 100% of patients with disease, but considering that our IFA FN rate was 5% before algorithm implementation, and only rose with the described modifications, our algorithm appears to identify more patients with true clinical illness than IFA testing alone. From this experience, we made three conclusions. First, automated EIA-based ANA screening tests present a sensitive means for identifying specimens more likely to need confirmatory testing by IFA. Second, although validating serological assays based on selected specimens is important, testing sequential clinical specimens provides information with respect to testing outcomes and work flow. Third, sequential data should be monitored by both laboratorians and clinical specialists after launching new testing algorithms and when significant changes are made to assay cutoffs. Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. Authors Disclosures or Potential Conflicts of Interest: No authors declared any potential conflicts of interest. Role of Sponsor: No sponsor was declared. Acknowledgments: The authors would like to thank Carole Mahaffey, MT(ASCP), of West Virginia University and the Zeus Clinical Affairs team for assistance with troubleshooting and work flow reviews. 688 JALM :06 May 2017

12 ANA Testing Algorithm Optimization ARTICLES REFERENCES 1. Yazdany J, Schmajuk G, Robbins M, Daikh D, Beall A, Yelin E, et al. Choosing wisely: the American College of Rheumatology's Top 5 list of things physicians and patients should question. Arthritis Care Res. 2013;65: Solomon DH, Kavanaugh AJ, Schur PH. Evidence-based guidelines for the use of immunologic tests: antinuclear antibody testing. Arthritis Rheum 2002;47: Tan EM, Feltkamp TE, Smolen JS, Butcher B, Dawkins R, Fritzler MJ, et al. Range of antinuclear antibodies in healthy individuals. Arthritis Rheum. 1997;40: American College of Rheumatology. American College of Rheumatology Ad Hoc Committee on Immunologic Testing G. Position Statement: methodology of testing for antinuclear antibodies. Portals/0/Files/Methodology%20of%20Testing% 20Antinuclear%20Antibodies%20Position%20Statement. pdf (Accessed February 2015). 5. Barak M, Rozenberg O, Grinberg M, Reginashvili D, Kishinewsky M, Henig C, Froom P. A novel cost effective algorithm for antinuclear antibody (ANA) testing in an outpatient setting. Clin Chem Lab Med 2013;51:e Bernardini S, Infantino M, Bellincampi L, Nuccetelli M, Afeltra A, Lori R, et al. Screening of antinuclear antibodies: comparison between enzyme immunoassay based on nuclear homogenates, purified or recombinant antigens and immunofluorescence assay. Clin Chem Lab Med 2004;42: Kavanaugh A, Tomar R, Reveille J, Solomon DH, Homburger HA. Guidelines for clinical use of the antinuclear antibody test and tests for specific autoantibodies to nuclear antigens. Arch Pathol Lab Med 2000;124: Deng X, Peters B, Ettore MW, Ashworth J, Brunelle LA, Crowson CS, et al. Utility of antinuclear antibody screening by various methods in a clinical laboratory patient cohort. J Appl Lab Med 2016;1: Emlen W, O'Neill L. Clinical significance of antinuclear antibodies: comparison of detection with immunofluorescence and enzyme-linked immunosorbent assays. Arthritis Rheum 1997;40: May : JALM 689