Comparison of 13 Commercially Available Cardiac Troponin Assays in a Multicenter North American Study

Transcription

1 ARTICLES SC Comparison of 13 Commercially Available Cardiac Troponin Assays in a Multicenter North American Study Robert H. Christenson, 1 * Ellis Jacobs, 2,3 Denise Uettwiller-Geiger, 4 Mathew P. Estey, 5,6 Kent Lewandrowski, 7 Thomas I. Koshy, 8 Kenneth Kupfer, 2 Yin Li, 2 and James C. Wesenberg 9,10 Background: We examined the concordance of 13 commercial cardiac troponin (ctn) assays [point-of-care, high-sensitivity (hs), and conventional] using samples distributed across a continuum of results. Methods: ctni (11 assays) and ctnt (2 assays) were measured in 191 samples from 128 volunteers. ctn assays included Abbott (istat, STAT, and hs), Alere (Cardio 3), Beckman (AccuTnI+3), Pathfast (ctni-ii), Ortho (Vitros), Siemens (LOCI, ctni-ultra, Xpand, Stratus CS), and Roche [4th Generation (Gen), hs]. Manufacturer-derived 99th percentile cutoffs were used to classify results as positive or negative. Alternative 99th percentile cutoffs were tested for some assays. Correlation was assessed using Passing Bablok linear regression, bias was examined using Bland Altman difference plots, and concordance/discordance of each method comparison was determined using the McNemar method. Results: Regression slopes ranged from 0.63 to 1.87, y-intercepts from 0.00 to 0.03 ng/ml, and r values from 0.93 to The ctnt methods had a slope of 0.93, y-intercept of 0.02 ng/ml, and r value of For the ctni assays, positive, negative, and overall concordance was 76.2% 100%, 66.0% 100%, and 82.9% 98.4%, respectively. Overall concordance between the 4th Gen ctnt and hstnt assays was 88.9%. A total of 30 of the 78 method comparisons showed significant differences in classification of samples (P <0.001); the istat showed 10, hstnt showed 9, AccuTnI+3 showed 5, Xpand showed 5, and Stratus CS showed 1. Using alternative 99th percentile cutoffs to those listed by manufacturers lowered the method discordance by 6-fold, from 30 to 5 (all involved istat). Conclusions: These data provide insight into characteristics of ctn methods and will assist the healthcare community in setting expectations for relationships among commercial ctn assays. IMPACT STATEMENT This study demonstrates the analytical relationships among 13 commercial ctn assay systems that included 4 point-ofcare assays, 2 high-sensitivity (hs) assays, and 7 central laboratory systems in a multicenter setting. The data can be used to set expectations for the relative differences between ctn assays in terms of classifying negative and positive results. Alternative 99th percentile cutoffs improved agreement between most discordant assays. These data provide insight into relationships between available commercial assays and will assist the laboratory and healthcare community in setting expectations for comparisons among ctn assays. 1 Department of Pathology, University of Maryland School of Medicine, Baltimore, MD; 2 Alere, Waltham, MA; 3 Department of Pathology, Mount Sinai School of Medicine, New York, NY; 4 John T Mather Memorial Hospital, Port Jefferson, NY; 5 DynaLIFE Dx, Edmonton, Canada; 6 Department of Laboratory Medicine and Pathology, University of Alberta, Edmonton, Canada; 7 Department of Pathology, Massachusetts General Hospital, Boston, MA; 8 The Binding Site Inc., San Diego, CA; 9 Alberta Health Service, Red Deer, Alberta, Canada; 10 Red Deer Regional Hospital Centre, Red Deer, Canada JALM :05 March 2017 Copyright 2017 by American Association for Clinical Chemistry.

2 ARTICLES The diagnosis of acute myocardial infarction (MI) 11 has evolved with an increasing focus on biomarkers for diagnosis and risk stratification of patients with signs and symptoms of ischemic heart disease and an increased emphasis on rapid and early assessment of the disease process (1). With over 8 million nontraumatic chest pain visits to emergency departments each year in the US (2), the high mortality associated with overcrowding, and long waiting times in the emergency department, tools are in high demand that assist in providing accurate and timely diagnosis or that help rule out patients with a possible acute MI presentation (3 7). Most patients with acute MI do not have diagnostic electrocardiogram changes upon presentation, so the use of myocardial necrosis biomarkers to aid in the diagnosis and risk stratification of patients presenting with suspicious signs and symptoms is essential (1, 8). Cardiac troponin (ctn) I and cardiac troponin T (ctnt) are troponin isoforms that are unique to the cardiac myocyte, and measurement of either is sensitive and specific for detecting cardiac injury. Over the years, several generations of ctn assays have been developed, cleared by regulatory bodies, and made commercially available for patient care. Modern clinical practice has focused on improved ctn assays for diagnosis of acute MI, as indicated by expert international task force guidance (8) and evidence-based guideline recommendations from professional societies worldwide (1, 9, 10). This guidance recommends use of ctn assays that are capable of quantifying and defining the 99th percentile upper reference limit of a normal control population. More sensitive ctn assays allow for earlier MI detection and identify a higher percentage of emergency department chest pain patients who are at risk for short-term major adverse cardiac events (1, 9, 11, 12). There are a wide variety of laboratory-based and point-of-care (POC) systems that provide ctn measurements in whole blood and/or plasma. Despite the fact that the results from these various testing platform systems may yield similar clinical interpretation for diagnosis, i.e., above or below the 99th percentile of the assay with a rise and or fall of ctn, there are considerable differences in the numerical ctn values between assays. This variability may be due to differences in assay calibration, use of different antibodies, differences in assay design, instrument limitations, multiple detection technologies, and differences in the measurand, i.e., some assays measure ctni and others ctnt. There is a paucity of resources available for directly comparing the commercially available ctn assays. We believe that such a resource would be valuable to practicing laboratorians; therefore, we present here a multicenter study that examined the correlation and concordance of 13 commercial ctn assays using patient samples distributed across a wide range of ctn results. MATERIALS AND METHODS Patient enrollment A total of 191 blood samples with a continuum of ctn results were collected from 128 adult patients enrolled in the emergency department, medical intensive care unit, cardiac intensive care unit, cardiac telemetry, cardiac catheterization lab, and cardiac surgery. The ctn assays used to identify and classify potential subjects at the collection site were the Alere Triage high-sensitivity (hs)-ctni assay (Alere) and Roche Elecsys 4th Generation (Gen) TnT system (Roche Diagnostics). Consecutive samples meeting criteria for the study were collected during work hours. *Address correspondence to this author at: Laboratories of Pathology, University of Maryland Medical Center, 22 South Greene St., Baltimore, MD Fax ; rchristenson@umm.edu. DOI: /jalm American Association for Clinical Chemistry 11 Nonstandard abbreviations: MI, myocardial infarction; ctn, cardiac troponin; POC, point-of-care; hs, high-sensitivity; Gen, generation.... March : JALM 545

3 ARTICLES This study was approved by the Institutional Review Board at the collection site (New York Methodist Hospital, Brooklyn, NY). Study eligibility was confirmed and written informed consent was obtained from all subjects before sample collection. Basic demographic information (age, sex, race) was collected, along with date of collection and the initial ctn concentration. Sample collection The total blood volume collected from each subject was 23 ml into a total of 3 BD Vacutainer tubes; 20 ml blood was collected into 2 10-mL sodium heparin anticoagulated tubes, and 3 ml blood was collected into 1 K 2 EDTA anticoagulated tube. For patients having a change in ctn concentrations within 12 h of the first blood draw, a second sample collection was permitted by the protocol and Institutional Review Board; second specimens were handled independently from those obtained in the first draw. Blood samples were obtained by routine venipuncture and indwelling lines/catheters, in accordance with the collection site's standard procedures. Blood samples were inverted at least 10 times to ensure complete dissolution and mixing of anticoagulant before centrifuging at 3000g for 10 min. Heparinized and EDTA plasma was dispensed within 1 h of collection into 0.5-mL aliquot tubes, which were then frozen and maintained at 70 C. Specimen aliquots were shipped on dry ice to the specimen bank at Alere (San Diego, CA), where they were archived at 70 C until shipment on dry ice to the performing laboratories. Samples were stored at 70 C at performing sites until measurement. ctn assays Table 1 lists the 13 commercial ctn assays included in this study, along with characteristics of the assays. The protocol for this study specified that all measurements were performed in singlicate and in accordance with manufacturers' recommendations. All assay runs that reported results were within laboratory-specified parameters, including Quality Control limits. ctn results were recorded on a Case Report Form and sent to Alere for data analysis by professional statisticians (coauthors K. Kupfer and Y. Li). Sample testing Samples were thawed, 1 set of 9 specimens at a time, by placing them in a water bath at room temperature (21 ± 3 C) for approximately 30 min. The thawed samples were mixed by gentle inversion and then centrifuged to remove particulate matter. Samples were tested within 1hofthawing. Data analysis Data analysis was performed in SAS version 9.3 and Analyse-it version A sample size of >180 was determined to be sufficient for providing significant statistical power for the method comparisons. With the sample size and data distribution assembled here, there is 96% power to detect a difference between a Pearson correlation of 0.93 and 0.88 using a 2-sided test with a significance level of 0.05, or to achieve a 95% CI of when measuring a total agreement of The sample size chosen was sufficient to detect differences in correlation coefficients and to estimate total agreement about the clinical decision point. Assay correlation and assessment of bias The Passing Bablok method was performed for all linear regression analyses that included data above the limit of detection for the assays. Spearman rank correlation coefficient and Pearson r values were determined for all assay comparisons. For bias, Bland Altman difference plots were constructed in 2 ways for each method pair. The first was for the full set data, including points below the assays' limit of detection (see Fig. 1 in the Data Supplement that accompanies the JALM :05 March 2017

4 ARTICLES Table 1. Characteristics of the 13 ctn assays included in this study. ctn assay Analytical System system, manufacturer, address Date approved a,b (anticoagulant) ctn epitopes targeted CV at 99th percentile, % 99th percentile, package insert Alternative value for 99th percentile Limit of detection Upper limit of linearity, ng/ml Performing laboratory Abbott i-stat CTn I i-stat Handheld, Abbott Point of Care Inc., Princeton, NJ 2003 (Heparin) C c : 41 49, (13, 14) Red Deer Regional Hospital Centre, Red D: 28 39, Deer, Alberta, Canada Alere Triage, Cardio3 ctni Alere Triage MeterPro, Alere (EDTA) C: d Alere, San Diego, CA D: 83 93, Abbott ARCHITECT STAT TnI Abbott ARCHITECT, High-Sensitivity TnI ARCHITECT i2000sr, Abbott Diagnostics, Abbott Park, IL ARCHITECT i2000sr, Abbott Diagnostics Beckman AccuTnI+3 Access 2, Beckman Coulter, Brea, CA Xpand CTn I Flex Dimension Xpand Plus, Siemens Healthcare, Newark, DE Stratus CS Acute Care ctni TestPak Stratus CS, Siemens Healthcare TnI-Ultra ADVIA Centaur XP, Siemens Healthcare LOCI CTn I, Dimension Vista 1500, Siemens Healthcare Ortho Vitros TnI ES VITROS 3600 Immunodiagnostic System, Ortho Clinical Diagnostics, Raritan, NJ Pathfast ctni-ii Pathfast, Mitsubishi Chemical Medience Co, Tokyo, Japan 4th Gen TnT Elecsys e411, Roche Diagnostics, Indianapolis, IN hstnt Elecsys e411, Roche Diagnostics 2007 (Heparin) C: 87 91, DynaLIFE, Edmonton, Alberta, Canada D: (EDTA) C: DynaLIFE D: (Heparin) C: e (16) JT Mather Memorial Hospital, Port D: Jefferson, NY 2001 (Heparin) C: f Red Deer Regional Hospital Centre D: (Heparin) C: (14) University of Maryland, School of Medicine, D: Baltimore, MD 2005 (Heparin) C: 41 49, Massachusetts General Hospital, Boston, MA D: (Heparin) C: Red Deer Regional Hospital Centre D: (Heparin) C: 24 40, JT Mather Memorial Hospital D: (EDTA) C: University of Maryland School of Medicine D: , (Heparin) C: Massachusetts General Hospital D: (Heparin) C: (15) Massachusetts General Hospital D: a Date that each assay was approved by the U.S. Food and Drug Administration (FDA) or the Conformité Européene (CE) (European Conformity). b Assay dates with represent products are not FDA cleared, and date indicates CE mark approval. The third through eighth columns were excerpted from the IFCC table at c C, capture antibody; D, detection antibody. d Entry of indicates that the package insert value was used in the alternative 99th percentile analysis (see Materials and Methods). e Recommended Beckman Access 2 Troponin Decision point in European Instructions for Use. f The value of 0.05 ng/ml was used because it is 0.01 ng/ml above the method's detection limit.... March : JALM 547

5 ARTICLES online version of this article at org/content/vol1/issue5). The second analysis focused on ctn values near the respective 99th percentile cutoffs by including samples with values from 0.01 ng/ml below the package insert 99th percentile cutoffs to 0.5 ng/ml (see Fig. 2 in the online Data Supplement). Method comparisons having a bias of 25% and a scatter of 50% were identified. The rationale for use of this bias and scatter criteria was 2-fold. First, these method comparisons demonstrated the top 16.7% of overall agreement. Second, the criteria were convenient because, at a cutoff of 0.04 ng/ml, bias of 0.01 ng/ml is 25% and scatter is 0.02 ng/ml (i.e., 50%). Concordance and manufacturers' 99th percentile cutoff Concordance was assessed by classifying each assay result as either positive or negative according to the manufacturer's labeled 99th percentile cutoffs, regardless of the assays' 10% CV concentration. Results at or below the 99th percentile were deemed negative. The statistical significance of discordance between all assay pairs was evaluated by the McNemar analysis, without a correction for continuity. Because of the multiple comparisons performed, there was type I error inflation. For a total of 78 comparisons, the significance level for each comparison was 0.05/78 = Thus, McNemar P values <0.001 were deemed to show significant discordance. Concordance using alternative 99th percentile cutoffs Alternative cutoffs were applied to discordant methods in a separate analysis to evaluate if the variable interpretation between assays was primarily a function of the manufacturer prescribed 99th percentile. Table 1 lists the alternative cutoffs applied, which were selected as follows. A previously reported lower cutoff of 0.04 ng/ml was used for the Abbott istat (13, 14) and the Siemens Stratus CS assays (14). For the Dimension Xpand, a lower cutoff of 0.05 ng/ml was used because this concentration is 0.01 ng/ml above the limit of detection for the method. Higher alternative cutoffs than the manufacturer specified were used for the Roche hstnt assay at ng/ml (15) and for the Beckman Access 2 at 0.04 ng/ml, based on the assay's 10% CV (16). The alternative cutoff concordance analyses were performed with the McNemar method using a significance level of P < RESULTS A total of 191 samples were collected for the study. Demographics of the 128 subjects who were enrolled in the study are listed in Table 1 in the online Data Supplement. There were 65 subjects who had 1 specimen collected for the study; 63 subjects had 2 specimens collected. There were a total of 2470 ctn measurements performed for this study, of which 19 (0.78%) did not yield a valid ctn result for 1 or more methods. Table 2 in the online Data Supplement shows all Passing Bablok regression data, including the 95% confidence limits for slope and y-intercept, as well as the nonparametric Spearman rank correlation coefficient. Table 2 summarizes the Passing Bablock linear regression results for each ctn assay comparison. For the ctni assay comparisons, slopes ranged from 0.59 to 1.87; y-intercepts were low, and overall ctni correlation was good, as indicated by Pearson r values ranging from 0.83 to As expected, regression slopes for comparisons between ctnt (y axis) and ctni assays (x axis) were substantially lower. Correlation data having Pearson r values <0.80, all of which involved either the Roche 4th Gen ctnt or hstnt methods, are highlighted in red in Table 2. However, correlation between the 2 ctnt methods was good, as indicated by a slope of 0.94, 0 y-intercept of 0.02 ng/ml, and Pearson r of JALM :05 March 2017

6 ARTICLES Table 2. Correlation statistics from Passing Bablok linear regression analysis of paired method comparisons. a Y2 X Alere Cardio3 Abbott ARCHITECT Abbott ARCHITECT hs Abbott i-stat Beckman Access 2 Mitsubishi PathFast Ortho Vitros 3600 Roche 4th Gen TnT Roche hstnt Siemens Centaur Abbott ARCHITECT 1.60 Order of Slope 0.03 regression parameters: y-intercept 0.92 Pearson r 0.85 Pearson r 2 Abbott ARCHITECT hs Abbott i-stat Beckman Access Mitsubishi PathFast Ortho Vitros Roche 4th Gen TnT Roche hstnt Siemens Xpand Siemens Stratus Continued on page March : JALM 549

7 ARTICLES Table 2. Correlation statistics from Passing Bablok linear regression analysis of paired method comparisons. a (Continued from page 549) Y2 X Alere Cardio3 Abbott ARCHITECT Abbott ARCHITECT hs Abbott i-stat Beckman Access 2 Mitsubishi PathFast Ortho Vitros 3600 Roche 4th Gen TnT Roche hstnt Siemens Centaur Siemens Xpand Siemens Stratus Siemens Centaur Siemens Xpand Siemens Stratus Siemens VISTA a Assays are ctni unless otherwise indicated. The descending order of regression results is slope, y-intercept, Pearson coefficient (r), and Pearson r 2. Pearson r values <0.8 are in red. The full complement of linear regression variables, including 95% CIs for slope and y-intercepts are listed in Table 2 in the online Data Supplement JALM :05 March 2017

8 ARTICLES All 78 Bland Altman plots of the data, including 95% CIs, are displayed in Fig. 1 in the online Data Supplement and for the concentrations near the 99th percentile cutoff for the methods in Fig. 2 in the online Data Supplement. Table 3 lists the Bland Altman proportional mean bias and SD data for all method pairs at concentrations near the 99th percentile cutoff. These data show that proportional mean bias using this limited region ranged from a low of 0.4% to a high of 80.3%; the SD of the proportional bias demonstrated scatter between 33.6% and 118.5%. The difference plots for method comparisons having both 25% mean proportional bias and 50% SD of scatter in proportional bias represented the best 16.7% of the total comparison and are displayed in red type in Table 3. Table 3A in the online Data Supplement shows concordance (agreement) data for the method comparisons. Here, the sample results were classified as positive or negative according to the respective manufacturers' listed 99th percentile cutoffs. There are 158 comparisons in this approach because the negative and positive agreements are calculated with each of the 13 assays as the reference method. The classification of samples as method 1 or method 2, and their designation of positive or negative was solely on the basis of the manufacturers' cutoffs. Comparisons were relative, and there was no gold standard. It is important to note when interpreting these data that the purpose of this effort was limited to examining agreement of the methods with regard to their respective cutoffs. As such, this analysis examined the extent of discordance between the methods and did not to determine which method correctly classified patients clinically. As an example, Table 3B in the online Data Supplement shows the determination of positive, negative, and overall agreement when method 1 is the Roche hstnt assay and method 2 is Roche 4th Gen assay. The hstnt assay was positive for all 4th Gen ctnt positive results, thus yielding 100% for positive agreement (i.e., no 4th Gen ctnt samples were negative when the matching hstnt result was positive) where the Roche Table 3. Proportional (%) bias and CV (%) from Bland Altman analysis of method comparison in the area of the 99th percentile cutoff. a Siemens Stratus CS Siemens Xpand Siemens Centaur Roche hstnt Roche 4th Gen TnT Ortho Vitros 3600 Mitsubishi PathFast Beckman Access 2 Abbott i-stat Abbott ARCHITECT hs Abbott ARCHITECT Alere Cardio3 Y2 X Abbott ARCHITECT 64.7% (56.2) Abbott ARCHITECT hs 44.2% (44.8) 37.1% (37.8) Abbott i-stat 4.3% (90.3) 82.4% (58.4) 55.9% (74.4) Beckman Access % (53.2) 28.2% (34.0) 10.2% (33.6) 57.7% (69.6) Mitsubishi PathFast 35.0% (63.0) 22.3% (74.0) 2.2% (71.4) 45.6% (87.0) 5.8% (67.1) Ortho Vitros % (72.8) 35.1% (41.3) 1.0% (52.2) 45.9% (63.2) 7.4% (50.0) 7.1% (83.0) Roche 4th Gen TnT 37.7% (107.9) 37.8% (97.5) 9.8% (103.6) 18.6% (100.7) 13.4% (99.3) 13.1% (118.5) 22.3% (102.0) Roche hstnt 24.8% (106.0) 35.3% (77.9) 0.4% (89.0) 51.8% (90.5) 6.7% (81.1) 23.0% (105.9) 4.8% (76.0) 31.4% (46.6) Siemens Centaur 50.4% (66.7) 25.0% (42.7) 12.6% (39.3) 65.6% (59.9) 1.8% (38.3) 9.6% (72.7) 20.4% (47.0) 21.6% (102.7) 12.2% (84.7) Siemens Xpand 44.1% (41.2) 20.5% (48.4) 2.5% (38.7) 41.6% (69.1) 1.9% (37.2) 5.3% (64.4) 6.6% (62.4) 9.2% (110.1) 19.3% (98.4) 5.3% (58.7) Siemens Stratus 38.3% (50.9) 25.2% (43.0) 1.0% (45.9) 49.3% (73.4) 2.7% (41.8) 0.5% (67.6) 2.2% (54.2) 15.8% (111.0) 20.7% (94.0) 5.6% (57.3) 11.9% (35.4) Siemens VISTA 80.3% (33.6) 12.7% (58.8) 37.0% (44.0) 69.6% (67.5) 32.3% (44.9) 43.8% (68.0) 31.1% (64.4) 29.3% (111.7) 42.0% (100.7) 23.5% (60.3) 39.5% (33.8) 46.8% (42.6) a See Bland Altman difference plots for each comparison in Figure 2 in the online Data Supplement. Method pairs displayed in red have proportional mean bias 25% and SD ( 50%).... March : JALM 551

9 ARTICLES Fig. 1. Heat map format showing agreement of classification of sample results as either equal to or below the specified 99th for the assay (green) or greater than the 99th percentile for the assay (red). There were 3 regions: region 1, in which all samples (except 1) were negative for 8 or more methods; region 2, in which all samples (except 2) were positive (or missing data) for 6 10 methods; and region 3, in which all samples (except 1) had positive results for 11 or more methods. Continued on page 553 4th Gen assay is the reference method. On the other hand, the negative agreement was 72.0%; of interest, 28% of negative results for the 4th Gen ctnt assay did not have a matching hstnt negative result. Overall, 88.9% of the 4th Gen and hstnt results agreed [i.e., concordant results/(concordant + discordant) results]. In the entire data set, 72.5% 98.4% of the positive and negative results were concordant; agreement among positive samples ranged from 68.4% to 100%, and agreement of the negative sample results ranged from 56.1% to 100% (Table 3A in the online Data Supplement). Restricting this analysis to ctni assays, the range of total, positive, and negative agreement was 82.9% 98.4%, 74.4% 100%, and 66.0% 100%, respectively. Fig. 1 displays all study data arranged as a heat map in which each sample measurement is classified as either negative, i.e., equal to or less than the respective package insert's stated 99th percentile value (green cells), or positive, i.e., greater than the respective assay's package insert 99th percentile value (red cells). Here, the ctn assays JALM :05 March 2017

10 ARTICLES Fig. 1. Continued. Continued on page 554 are arranged in alphabetical order with results displayed from the lowest to the highest according to Abbott ARCHITECT ctni values, with corresponding results for the sample from the other 12 ctn assays included in the same data row. For convenience, the heat map is divided into 3 regions: region 1, in which all samples (except 1) were negative for 8 or more methods (n = 74 samples); region 2, in which all samples (except 2) were positive (or missing data) for 6 10 methods (n = 19 samples); and region 3, in which all samples (except 1) had positive results for 11 or more methods (n = 98 samples). Of the measurements in region 1, there were discordant results for 54 (5.7% of region 1 total) samples. Two assays had 10 or more discordant measurements; the Roche hstnt assay had 20 (37.0% of total noncordant results), and the Beckman Access 2 assay had 10 (18.5% of... March : JALM 553

11 ARTICLES Fig. 1. Continued. Continued on page 555 discordant results). In region 3, there were 13 discordant negative measurements (1.0% of region 3 total). No assay had more than 5 discordant results in region 3. In region 2, 163 (68.2%) of the results were positive, and the remaining 76 results (31.8%) were negative. As seen in Fig. 1, in region 2, most of the negative measurements were with the istat (n = 19; 25.0% of negatives), Siemens Stratus CS (n = 16; 21.1% of negatives), and Siemens Xpand (n = 14; 18.4% of negatives). It is noteworthy that for unknown reasons, the Abbott istat and Ortho ctni assays had undetectable results for the same samples that were significantly positive by the other 11 assays. The reason for this observation was not clear but is likely specimen related. Concordance data with McNemar analysis for all 78 of the paired ctni method comparisons are presented in Table 4 in the online Data Supplement JALM :05 March 2017

12 ARTICLES Fig. 1. Continued. Table 4 displays the 30 (39%) total comparisons that were significantly biased in the distribution of nonconcordant results in the McNemar analysis (P <0.001). The Abbott istat method showed a significant difference, with 10 of its 12 (83.3%) paired comparisons; the Roche hstnt assay showed a significant difference with 9 (75.0%) of its paired comparisons; both the Beckman Access 2 and Siemens Xpand methods showed a significant difference with 5 other methods (41.7%); and the Abbott ARCHITECT showed a significant difference with the Siemens Stratus CS (8.5%). Note that all of the discordant paired comparisons for both the Abbott istat and Siemens Xpand were negative by these methods and positive by the other methods. On the other hand, the significantly different discordant comparisons for the Roche hstnt and Beckman Access 2 assays were positive by these methods and negative by the comparison methods. Reanalysis of the concordance data with alternative cutoffs listed in Table 1 was performed for the method comparisons; the results of this analysis are... March : JALM 555

13 ARTICLES Table 4. McNemar analysis table. a Paired comparison Number of samples PP b PN NP NN McNemar P value Abbott i-stat (X) vs Beckman Access 2 (Y) <0.001 Abbott i-stat (X) vs Mitsubishi PathFast (Y) <0.001 Abbott i-stat (X) vs Ortho Vitros 3600 (Y) <0.001 Abbott i-stat (X) vs Roche 4th Gen TnT (Y) <0.001 Abbott i-stat (X) vs Roche hstnt (Y) <0.001 Abbott i-stat (X) vs Siemens Centaur (Y) <0.001 Abbott i-stat (X) vs Siemens VISTA (Y) <0.001 Abbott i-stat (X) vs Alere Cardio 3 (Y) <0.001 Abbott i-stat (X) vs Abbott ARCHITECT (Y) <0.001 Abbott i-stat (X) vs Abbott ARCHITECT hs (Y) <0.001 Roche hstnt (X) vs Siemens Centaur (Y) <0.001 Roche hstnt (X) vs Siemens Xpand (Y) <0.001 Roche hstnt (X) vs Siemens Stratus CS (Y) <0.001 Roche hstnt (X) vs Siemens VISTA (Y) <0.001 Roche hstnt (X) vs Roche 4th Gen TnT (Y) <0.001 Roche hstnt (X) vs Mitsubishi PathFast (Y) <0.001 Roche hstnt (X) vs Ortho Vitros 3600 (Y) <0.001 Roche hstnt (X) vs Alere Cardio 3 (Y) <0.001 Roche hstnt (X) vs Abbott ARCHITECT hs (Y) <0.001 Beckman Access 2 (X) vs Ortho Vitros 3600 (Y) <0.001 Beckman Access 2 (X) vs Siemens Xpand (Y) <0.001 Beckman Access 2 (X) vs Siemens Stratus CS (Y) <0.001 Beckman Access 2 (X) vs Siemens VISTA (Y) <0.001 Beckman Access 2 (X) vs Alere Cardio 3 (Y) <0.001 Siemens Xpand (X) vs Abbott ARCHITECT (Y) <0.001 Siemens Xpand (X) vs Abbott ARCHITECT hs (Y) <0.001 Siemens Xpand (X) vs Mitsubishi PathFast (Y) <0.001 Siemens Xpand (X) vs Siemens Centaur (Y) <0.001 Siemens Xpand (X) vs Siemens VISTA (Y) <0.001 Abbott ARCHITECT (X) vs Siemens Stratus CS (Y) <0.001 a Package insert 99th percentile cutoffs are displayed in Table 1. b PP, X value positive, Y value positive; PN, X value positive, Y value negative; NP, X value negative, Y value positive; NN, X value negative, Y value negative. listed in Table 4. All previously discordant paired comparisons showing significant bias (McNemar P <0.001) were no longer significant except for 5 paired comparisons with the Abbott istat; these specific comparisons are displayed in Table 5 in red type. DISCUSSION Here we present a descriptive study of 13 commercial ctn assays performed in accordance with manufacturers' recommendations. The purpose of this study was to provide a transparent aid for setting expectations when evaluating and comparing various ctn assays. Although comparison studies have been performed previously (16 23), this was a multi- Center, North American study that used a common cohort of samples designed to span the range of ctn measurements. There were several POC assays included in this study, and as has been reported previously (17, 23), the data show substantial variability JALM :05 March 2017

14 ARTICLES Table 5. Comparison data when alternative 99th percentile cutoffs (Table 1) for several of the methods were used. a Paired comparison b Number of samples PP c PN NP NN McNemar P value Abbott i-stat (X) vs Beckman Access 2 (Y) Abbott i-stat (X) vs Mitsubishi PathFast (Y) Abbott i-stat (X) vs Ortho Vitros 3600 (Y) Abbott i-stat (X) vs Roche 4th Gen TnT (Y) Abbott i-stat (X) vs Roche hstnt (Y) c <0.001 Abbott i-stat (X) vs Siemens Centaur (Y) <0.001 Abbott i-stat (X) vs Siemens VISTA (Y) <0.001 Abbott i-stat (X) vs Alere Cardio 3 (Y) Abbott i-stat (X) vs Abbott ARCHITECT (Y) <0.001 Abbott i-stat (X) vs Abbott ARCHITECT hs (Y) <0.001 Roche hstnt (X) vs Siemens Centaur (Y) Roche hstnt (X) vs Siemens Xpand (Y) d Roche hstnt (X) vs Siemens Stratus CS (Y) Roche hstnt (X) vs Siemens VISTA (Y) Roche hstnt (X) vs Roche 4th Gen TnT (Y) Roche hstnt (X) vs Mitsubishi PathFast (Y) Roche hstnt (X) vs Ortho Vitros 3600 (Y) Roche hstnt (X) vs Alere Cardio 3 (Y) Roche hstnt (X) vs Abbott ARCHITECT hs (Y) Beckman Access 2 (X) vs Ortho Vitros 3600 (Y) Beckman Access 2 (X) vs Siemens Xpand (Y) e Beckman Access 2 (X) vs Siemens Stratus CS (Y) Beckman Access 2 (X) vs Siemens VISTA (Y) Beckman Access 2 (X) vs Alere Cardio 3 (Y) Siemens Xpand (X) vs Abbott ARCHITECT (Y) Siemens Xpand(X) vs Abbott ARCHITECT hs (Y) Siemens Xpand(X) vs Mitsubishi PathFast (Y) Siemens Xpand(X) vs Siemens Centaur (Y) Siemens Xpand (X) vs Siemens VISTA (Y) Abbott ARCHITECT (X) vs Siemens Stratus CS (Y) a Significance was determined with the McNemar Analysis. Adjusted 99th percentile cutoffs are listed in Table 1. Bold red type indicates a significant difference between the methods. b The first character represents (X) and second character represents (Y). c PP, X value positive, Y value positive; PN, X value positive, Y value negative; NP, X value negative, Y value positive; NN, X value negative, Y value negative. among these assays. Although measurements in the current study were performed by laboratory personnel using plasma samples rather than a whole blood matrix, some of the POC assays show no significant difference from the central laboratory-based assays in concordance in this study or previously (17, 23). Evidence indicates that there is indeed substantial variability among POC assays, and these assays should be considered individually, rather than as simply a homogeneous class of assays, as was suggested in recent guidelines that state [clinical] sensitivity [of point-of-care troponin assays] is substantially below that of central laboratory methods (1). An important aim of this study was to provide data for the comparison and evaluation of ctn assays and,... March : JALM 557

15 ARTICLES in that way would, help assist practitioners and stakeholders in the field. One question that will likely be relevant involves the expectations that users may have when switching their current central laboratory assays to the next generation of hs-ctn assays. As an example of switching from Roche's 4th Gen assay to the Roche hstnt test, a report by Saenger et al. (19) demonstrated in a multicenter study that the regression equation was as follows: hstnt = th Gen ctnt ng/l; r = In the current study, Table 2 shows that the relationship observed was quite close: hstnt = th Gen ng/ml; r = A different example may involve expectations for implementing Abbott's hstni for users of the Abbott ARCHITECT ctni assay. Rezvanpour et al. (20) compared these assays in over 7000 measurements performed at 3 Canadian sites. For the 5509 points in the ng/l range, the overall relationship was: hstni = 1.12 ctni + ( 5 ng/l); r = In the current study, Table 2 (and Table 2 in the online Data Supplement) shows that for the full cohort studied here, the relationship was somewhat different: hstni = 0.92 ctni ng/ml; r = Further, information from the Rezvanpour et al. (20) contribution showed slopes for linear regression over the entire range of measurements that differed by 0.26, from 1.18 (site 1) to 1.44 (site 3). Of interest, the difference between the lowest slope from the full cohort at their site 1 of 1.18 (20) was also 0.26 different from the slope of 0.92 from the Abbott hstni vs TnI comparison in the current study. Thus, many interesting examinations and comparisons are possible using data from this study and available literature. The Passing Bablok regression analyses showed slopes and y-intercept values that indicate strong linear relationships between the ctni assays examined here. The correlation coefficients (r values) for the comparisons were high, indicating that the rank order and direction of the measurementpairs was consistent. Further, the r 2 values indicate that the comparison data were close to the fitted regression line. As expected, the ctni and ctnt comparisons had lower slopes and values for r and r 2 ; this may be, at least in part, because these assays target distinctly different measurands. However, the 2 ctnt assays, which are both manufactured by the same vendor and use the same antibodies for capture and detection, demonstrated good correlation. Bland Altman difference plots showed bias that varied substantially among the methods. Also, the scatter (SD) between method comparisons varied and was substantial for a number of comparisons examined. Approximately 15% of the ctn comparisons demonstrated 25% bias and 50% scatter. These criteria may be interpreted as a convenient starting point for assessing method agreement and potential for harmonization. The bias and scatter data are transparently displayed in Table 2 in the online Data Supplement so that readers can examine other criteria according to their interest and practice needs. It must be noted that this study was not designed to address the issue of harmonization or standardization of ctni assays, although others have examined this possibility in similar investigations (21, 22). Tate et al. (21) examined 16 ctni assays for measurement equivalence and their standardization capability. Their study concluded that the methods demonstrated a significantly higher degree of measurement equivalence after mathematical recalibration, indicating that measurement harmonization or standardization would be effective at reducing interassay bias (21). Also, Clerico et al. (22) recently conducted a pilot study that included patient samples, external quality assessment controls, and plasma pools from MI patients and healthy subjects. These samples were run on the ARCHITECT High hstni, Centaur ctni, ST AIA-Pack ctni Third Generation (Tosoh Bioscience), and Beckman Access 2. After mathematical recalibration of the assays, the authors found that the between-method ctni variability decreased significantly. The authors suggested that, based on these results, they were able to evaluate between-method variability and produce reliable recalibration and harmonization of results (22) JALM :05 March 2017

16 ARTICLES The 99th percentile cutoff values from the respective manufacturers' package inserts were used to classify samples as either negative or positive, i.e., less than or equal to or greater than the 99th percentile. The Abbott i-stat and Siemens Xpand ctni assays demonstrated a high rate of negative disconcordant results, and the Roche hstnt and Beckman Access 2 ctni assays demonstrated a high rate of positive discordant results. Inspection of region 2 of the heat map (Fig. 1) suggests that these differences may to be related to the diagnostic sensitivity of the respective assays, differences in populations used to establish the 99th percentile, and/or other factors. Inspection of the epitope targets for the various ctni assays (Table 1) show that the amino acid range of is a common epitope target for virtually all ctni assays examined here. Although the assays may be targeting similar measurands, the significant differences may reflect variation in the populations used for determining the assays' 99th percentile cutoffs for classifying samples as positive or negative. To investigate this issue, we used alternative 99th percentile cutoffs for the Abbott istat, Siemens Xpand, and Stratus CS assays to lower feasible values, and the Roche hstnt assay and the Beckman Access 2 assays were increased to higher feasible values. This exercise corrected much of the discordance between methods, reducing the overall number of discordant comparisons by about 83%. However, results from the Abbott istat assay were still discordant when compared to results from 5 other methods. The Abbott istat assay, therefore, appears to have greater variability compared to any of the other methods included; this variability was not corrected by simply changing the 99th percentile cutoff. These data further highlight the variability and heterogeneity among POC devices, some of which showed the same characteristics as laboratory quality assays. These data support the use of a common set of normal reference samples for establishing the 99th percentile for the assays, which may be effective in improving, at least in part, the significant discordance observed in about 40% of comparisons presented here. Further to this point, methodological differences in the 99th percentile and their impact on patient classification have been documented. For example, a recent study by Ungerer et al. (16) included measurements with the Abbott ctni, Beckman Access 2 TnI, and Roche hstnt assays in a common set of 2004 samples. This study found that 50% to 70% of the subjects having ctn values above the 99th percentile cutoffs were unique to individual assays, and that only 4 out of 20 individuals above the cutoff for all 3 assays (16). Use of a common sample bank may be of assistance in determining the diagnostic concordance and discordance among ctn assays. It is noteworthy that sample banks have been reported previously, including the aforementioned 2004-person (16). Also, this point was examined in a cohort of 524 presumably healthy adults assembled by Apple et al. (14), which also found variations in a number of previously established 99th percentile values. Another set of samples consists of 875 presumably healthy males and females and was assembled through the American Association for Clinical Chemistry. This sample bank is unique in that all samples were tested for creatinine, hemoglobin A 1c, and amino-terminal prob-type natriuretic peptide to assess health; the number of males and females recruited is in compliance with the IFCC Task Force on Clinical Applications of Cardiac Bio-Markers (24); and, perhaps most importantly, this sample bank can be purchased by any manufacturer or individual for their use, whereas to our knowledge there is no other properly banked cohort available for general use. For the Abbott istat and Ortho ctni assays, a number of measurements were undetectable (negative), whereas the other assays produced significantly positive results. The reason for this observation was not clear, but is likely specimen-related. There are several limitations to this study. One is that the samples were run in singlicate, and... March : JALM 559

17 ARTICLES therefore discordant results could not be confirmed or refuted as outliers. However, the participating sites were blinded to other results, and the study was designed to reflect what practitioners would experience in real-life performance of the respective assays. We used the Passing Bablok regression technique because it assumes error in both the X and Y methods. With use of the Passing Bablok technique, however, one must be cautioned not to simply square the correlation coefficient to calculate the coefficient of determination because it is calculated from the residuals of the fit. Generally, statisticians do not calculate the coefficient of determination for a Passing Bablok fit because the fit is nonparametric and the coefficient of determination is sensitive to outliers. Also, because there are no clinical data for the specimens analyzed in this study, the data cannot be used to reflect clinical comparison of the assays. In addition, the only sample types used with each assay in this study are listed in Table 1. Therefore, different sample types may yield different results and comparisons may be different. Also, some ctn results are reported to 2 decimal places and others to 3. Therefore, rounding may have had an effect on the data. Initial and then later samples were collected from some individual patients, which may have had an impact on the study. Finally, the assays included here had different characteristics regarding imprecision at low concentrations, which may also have an effect. However, in the Bland Altman plots in Table 3 in the online Data Supplement, we compensated for this by only including data exceeding the detection limit for each assay. In conclusion, this study demonstrates the analytical relationship between 13 ctn assay systems, 4 POC assays, 2 hs assays, and 8 central laboratory systems. This study provides transparent awareness of the relative differences expected for ctn assays in terms of classification of results as either negative or positive relative to the manufacturer specified 99th percentile cutoff of a reference population. Adjusting the 99th percentile cutoffs for the assays greatly improved concordance between the assays for all systems and revealed potential issues with patient selection for determining the 99th percentile of an assay. Although correlation between the ctn assays included was good, correlation between ctni and ctnt measurements varied substantially. There was both bias and scatter among the methods, as demonstrated by difference plots. In large health systems where standardization to a single platform is not possible, these data may provide insights and expectations of relationships for various assays. Author Contributions: All authors confirmed they have contributed to the intellectual content of this paper and have met the following 4 requirements: (a) significant contributions to the conception and design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for intellectual content; (c) final approval of the published article; and (d) agreement to be accountable for all aspects of the article thus ensuring that questions related to the accuracy or integrity of any part of the article are appropriately investigated and resolved. Authors Disclosures or Potential Conflicts of Interest: Upon manuscript submission, all authors completed the author disclosure form. Employment or Leadership: E. Jacobs, T.I. Koshy, K. Kupfer, Y. Li, Alere. Consultant or Advisory Role: R.H. Christenson, Roche Diagnostics, Seimens Healthcare Diagnostics, and Becton Dickinson; D. Uettwiller-Geiger, K. Lewandrowski, T.I. Koshy, Alere. Stock Ownership: T.I. Koshy, Alere. Honoraria: R.H. Christenson, Roche Diagnostics; T.I. Koshy, Alere. Research Funding: R.H. Christenson, T.I. Koshy, J.C. Wesenberg, Alere. Expert Testimony: None declared. Patents: None declared. Other Remuneration: R.H. Christenson, Roche Diagnostics and Siemens Healthcare Diagnostics. Role of Sponsor: The funding organizations played no role in the design of study, choice of enrolled patients, review and interpretation of data, or preparation or approval of manuscript. Acknowledgments: We thank Alere for financial support of this study. The authors also gratefully acknowledge Dr. Robert Birkahn and Paris Datillo, New York Methodist Hospital, for the enrollment of subjects and specimen collection and handling for this study JALM :05 March 2017

18 ARTICLES REFERENCES 1. Amsterdam EA, Wenger NK, Brindis RG, Casey DE Jr Ganiats TG, Holmes DR Jr AHA/ACC Guideline for the management of patients with non-st-elevation acute coronary syndromes: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;130: Storrow AB, Gibler WB. Chest pain centers: diagnosis of acute coronary syndromes. Ann Emerg Med 2000;35: Vermeulen MJ, Stukel TA. Association between waiting times and short term mortality and hospital admission after departure from emergency department: population based cohort study from Ontario, Canada. BMJ 2011; 342:d Sprivulis PC, Da Silva JA, Jacobs IG, Frazer AR, Jelinek GA. The association between hospital overcrowding and mortality among patients admitted via Western Australian emergency departments. Med J Aust 2006; 184: Diercks DB, Roe MT, Chen AY, Peacock WF, Kirk JD, Pollack CV Jr. Prolonged emergency department stays of non-st-segment elevation myocardial infarction patients are associated with worse adherence to the American College of Cardiology/American Heart Association guidelines for management and increased adverse events. Ann Emerg Med 2007;50: Pines JM, Pollack CV Jr, Diercks DB, Chang AM, Shofer FS, Hollander JE. The association between emergency department crowding and adverse cardiovascular outcomes in patients with chest pain. Acad Emerg Med 2009;16: Pines JM, Hollander JE. Emergency department crowding is associated with poor care for patients with severe pain. Ann Emerg Med 2008;51: Thygesen K, Alpert JS, Jaffe AS, Simoons ML, Chaitman BR, White HD, et al. Third universal definition of myocardial infarction. J Am Coll Cardiol 2012;60: Roffi M, Patrono C, Collet JP, Mueller C, Valgimigli M, Andreotti F, et al ESC guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation. Eur Heart J 2016;37: Morrow DA, Cannon CP, Jesse RL, Newby LK, Ravkilde J, Storrow AB, et al. National Academy of Clinical Biochemistry Laboratory Medicine Practice Guidelines: clinical characteristics and utilization of biochemical markers in acute coronary syndromes. Circulation 2007; 115:e Storrow AB, Christenson RH, Nowak RM, Diercks DB, Singer AJ, Wu AH, et al. Diagnostic performance of cardiac troponin I for early rule-in and rule-out of acute myocardial infarction: results of a prospective multicenter trial. Clin Biochem 2015;48: Mueller C, Giannitsis E, Christ M, Ordóñez-Llanos J, defilippi C, McCord J, et al. Multicenter evaluation of a 0-Hour/1-Hour algorithm in the diagnosis of myocardial infarction with high-sensitivity cardiac troponin T. Ann Emerg Med 2016;68: Schneider HG, Ablitt P, Taylor J. Improved sensitivity of point of care troponin I values using reporting to below the 99th percentile of normals. Clin Biochem 2013;46: Apple FS, Ler R, Murakami MM. Determination of 19 cardiac troponin I and T assay 99th percentile values from a common presumably healthy population. Clin Chem 2012;58: Gore MO, Seliger SL, Defilippi CR, Nambi V, Christenson RH, Hashim IA, et al. Age- and sex-dependent upper reference limits for the high-sensitivity cardiac troponin T assay. J Am Coll Cardiol 2014;63: Ungerer JP, Tate JR, Pretorius CJ. Discordance with 3 cardiac troponin I and T assays: implications for the 99th percentile cutoff. Clin Chem 2016;62: Palamalai V, Murakami MM, Apple FS. Diagnostic performance of four point of care cardiac troponin assays to rule in and rule out acute myocardial infarction. Clin Biochemistry 2013;46: Apple FS, Sandoval Y, Jaffe AS, Ordonez-Llanos J, for the IFCC Task Force on Clinical Applications of Cardiac Bio-Markers. Cardiac troponin assays: guide to understanding analytical characteristics and their Impact on clinical care. Clin Chem 2017;63: Saenger AK, Beyrau R, Braun S, Cooray R, Dolci A, Freidank H, Giannitsis E. Multicenter analytical evaluation of a high-sensitivity troponin T assay. Clin Chim Acta 2011;412: Rezvanpour A, Shortt C, Clark L, Worster A, Kavsak PA. Analytical factors to consider when assessing a highsensitivity cardiac troponin I assay compared to a contemporary assay in clinical studies. Clin Chim Acta 2014;429: Tate JR, Bunk DM, Christenson RH, Barth JH, Katrukha A, Noble JE, et al. Evaluation of standardization capability of current cardiac troponin I assays by a correlation study: results of an IFCC pilot project. Clin Chem Lab Med 2015; 53: Clerico A, Ripoli A, Masotti S, Prontera C, Storti S, Fortunato A, et al. Pilot study on harmonization of cardiac troponin I immunoassays using patients and quality control plasma samples. Clin Chim Acta 2016;456: Peacock WF, Diercks D, Birkhahn R, Singer AJ, Hollander JE, Nowak R, et al. Can a point-of-care troponin I assay be as good as a central laboratory assay? A MIDAS Investigation. Ann Lab Med 2016;36: Apple FS, Jaffe AS, Collinson P, Mockel M, Ordonez-Llanos J, Lindahl B, et al. IFCC educational materials on selected analytical and clinical applications of high-sensitivity cardiac troponin assays. Clin Biochem 2015 :2015;48: March : JALM 561