Determination of Optimal Replicate Number for Validation of Imprecision Using Fluorescence Cell-Based Assays: Proposed Practical Method

Transcription

1 Cytometry Part B (Clinical Cytometry) 84B: (2013) Original Article Determination of Optimal Replicate Number for Validation of Imprecision Using Fluorescence Cell-Based Assays: Proposed Practical Method Bruce H. Davis, 1 * Christine E. McLaren, 2 Anthony J. Carcio, 1 Linda Wong, 1 Benjamin D. Hedley, 3 Mike Keeney, 3 Adam Curtis, 1 and Naomi B. Culp 1 1 Trillium Diagnostics, LLC, Bangor, Maine Department of Epidemiology, University of California Irvine, Irvine, California London Laboratory Services Group, London Health Sciences Centre, London, Ontario N6A 5W9, Canada Background: Assay validation includes determination of inherent imprecision across the reportable range. However, specific practical guidelines for determinations of precision for cell-based fluorescence assays performed on flow cytometers are currently lacking. Methods: Replicates of 10 or 20 measurements were obtained for flow cytometric assays developed for clinical in vitro diagnostic use, including neutrophil CD64 expression for infection/sepsis detection, fetal red cell enumeration for fetomaternal hemorrhage detection, human equilibrative nucleoside transporter 1 quantitation in leukocytes for possible correlation with drug responsiveness, and CD341 hematopoietic stem cell enumeration of apheresis products, using up to three different instrument platforms for each assay. For each assay, the mean, 95% confidence intervals (95% CIs) of the mean, standard deviation, and coefficient of variation (CV) of sequential replicates were determined. Results: For all assays and most instrument platforms, <5 replicates were found adequate to validate assay imprecision levels below the 5 10% CV for repeatability claimed by the manufacturers of these assays. Results plotted as a novel parameter derived from the 95% CI and the cumulative mean for replicates, termed variance factor (VF), provide a data-driven means for determining optimal replicate numbers. Conclusions: The novel VF can provide information to guide the practical selection of optimal replicate numbers for validation of imprecision in flow cytometric assays. The optimal number of replicates was assay and instrument platform dependent. Our findings indicate that three to four replicates are sufficient for most flow cytometric assays and instrument combinations, rather than the higher numbers suggested by CLSI guidelines for soluble analytes. VC 2013 International Clinical Cytometry Society Key words: quality assessment; flow cytometry; IVD clearance; regulatory science; assay validation; sepsis test; fetomaternal hemorrhage test; stem cell assay; hent1 assay; laboratory-developed test How to cite this article: Davis BH, McLaren CE, Carcio AJ, Wong L, Hedley BD, Keeney M, Curtis A, and Culp NB. Determination of Optimal Replicate Number for Validation of Imprecision Using Fluorescence Cell-Based Assays: Proposed Practical Method. Cytometry Part B 2013; 84B: One standard parameter for assay validation is the determination of the inherent assay imprecision across the full reportable range following the manufacturer s instructions for use. This assessment should be performed by laboratories during the validation of any laboratory-developed test (LDT) or during the introduction of a new commercial assay into clinical use, as well as by in vitro diagnostic (IVD) device manufacturers during both assay development and clinical validation Additional Supporting Information may be found in the online version of this article. *Correspondence to: Bruce H. Davis, Trillium Diagnostics, LLC, 4 Union Street, Bangor, ME 04401, USA. brucedavis@trilliumdx.com Received 23 December 2012; Revised 25 May 2013; Accepted 10 July 2013 Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: /cyto.b VC 2013 International Clinical Cytometry Society

2 330 DAVIS ET AL. studies supporting European Union CE mark certification or United States Food and Drug Administration (FDA) clearance. Imprecision or error of repeatability is determined for both intra-assay and interassay conditions, by considering replicate measurements and the variance of these measurements. The critical issue in the protocol development of any imprecision evaluation is the number of replicates that are sufficient to determine the level of imprecision for a particular assay. CLSI (formerly NCCLS) document EP5-A2 is one document cited by FDA to suggest 20 as the optimal sample size for performance validation studies. Yet EP5-A2, intended for soluble, not cell-based, analyte measurements, makes only one reference to measuring a sample 20 times to determine imprecision, accompanied by the comment that this may not be the most appropriate way to evaluate precision (1). Although measurement of 20 replicates certainly might provide a robust and thorough method to access reproducibility, which is not too onerous a task for a fully automated chemistry assay, this is both laborious and impractical for cell-based assays. Additionally, unnecessary validation studies are inefficient and costly in both labor and reagent resources. Seemingly, there is a need for a practical approach to optimizing precision studies in the context of each assay s performance specifications. CLSI guidance document EP9-A2 describes procedures for determining the relative bias between two methods, and it identifies factors to be considered when designing and analyzing a method-comparison experiment using split patient samples and further contains recommendations for manufacturers evaluation of bias and statement format for bias claims (2). This document provides for a comparative evaluation of different methods for measurement of the same analyte and puts forth expert recommendation for duplicate determinations with both methods on 40 samples with >50% being outside the reference range. However, this guidance is not intended for use with new assays that is designed to measure a new biomarker or analyte of disease activity. Thus, when a predicate method does not exist to compare method performance, many regulatory scientists default to the need to perform repeatability studies in order to demonstrate imprecision of the assay. No such guidance for cell-based fluorescence assays currently exists. Some flow cytometry assays require minimum specimen volumes for testing patient specimens, such as bone marrow aspirates or cerebrospinal fluid, where the maximal specimen volume that can be obtained either for practical or patient safety reasons is limited, thus making 20 replicates either impossible or improbable for many clinical specimens. Additionally, flow cytometric cell-based assays or particle-based assays are inherently different than most clinical chemistry assays in the way in which assay results are derived. Chemistry or soluble analyte assays are typically a single or at best an average of duplicate measurements, such as fluorescence, absorbance, or turbidity properties. Typical clinical flow cytometric assays are derived from as few as several thousand measurements to over 100,000 individual cellular simultaneous measurements of multiple parameters (light scatter and multiple wavelengths ranges of fluorescence), which are then transformed into the assay result of percent positive or some transformation (mean, median, and ratio) of thousands of individual fluorescence measurements. It would seem logical that a single assay result derived from thousands of multiparametric samplings might require fewer replicates to assess imprecision or repeatability compared with an assay device yielding a result following a single physical measurement. We advocate that regulatory science should be based upon actual data, rather than a theoretic statistical assessment of what might resemble some ideal truth. Further, we believe that regulatory science should allow for analysis of assay performance fit for purpose. In the case of imprecision, the assessment of any assay performance should be done in light of the developer s or manufacturer s intent. Accordingly, we present a novel, but simple procedure whereby laboratories might efficiently determine the adequate number of replicates by which to validate performance of fluorescent cell-based assays performed on flow cytometric instrumentation taking into account the stated imprecision specification of the manufacturer. The determination of the number of the adequate replicates was found to be straightforward for four clinical flow cytometric assays using the manufacturer s claim for imprecision as an additional guidance. Two of the assays, CD34 stem cell counting and fetal red cell counting, are in routine worldwide clinical IVD use. The other two assays tested, neutrophil CD64 assay for infection/sepsis and a human equilibrative nucleoside transporter 1 (hent1) expression assay for acute myeloid leukemia, are in early clinical adoption (Leuko64, Trillium Diagnostics, Bangor, ME; CE IVD marked, but not FDA cleared) or development (hent1- Quant, Trillium Diagnostics). The method presented below demonstrates a practical approach guided by actual data and manufacturer or developer claims (assay performance specifications) that indicates as few as three to four replicates are sufficiently robust to validate claims regarding assay imprecision or reproducibility for most quantitative clinical flow cytometric assays. MATERIALS AND METHODS All studies were performed on discarded or residual human blood, thawed apheresis packs, or bone marrow samples according to institutional and Helsinki guidelines regarding human ethics. The results of this study were neither entered into the medical record nor used in the care of any individual. The assays employed were either commercially developed IVD assays or, in the case of CD34 enumeration, a LDT based on the Stem-Kit TM (Beckman Coulter, Miami, FL) IVD assay. All assays were performed in the manner prescribed in the developer s or manufacturer s instructions for use. The four assays employed were as follows: (1) CD34 enumeration assay using the same clones and

3 DETERMINATION OF PRECISION FOR CELL-BASED FLUORESCENCE ASSAYS 331 FlowCount TM beads based on the Stem-Kit TM design, a three-color (CD45 FITC, CD34 PE, and viability dye, 7- AAD) assay for human progenitor stem cell enumeration (absolute count of CD341 cells per blood volume, cells per microliter) as previously reported (3); (2) FMH Quik- Quant TM (Trillium Diagnostics, LLC), a two-color [antihemoglobin F (HbF) and viability dye propidium iodide] assay for detection of fetomaternal hemorrhage [percent fetal red blood cells (RBCs) in blood] (4,5); (3) Leuko64 TM (Trillium Diagnostics, LLC), a three-color (CD64, CD163, and dual-labeled, FITC and Starfire Red, calibration beads) assay for measurement of neutrophil CD64 expression (PMN CD64 index) as an indicator of infection/sepsis (6,7); and (4) hent1-quant TM (Trillium Diagnostics, LLC), a four-color (anti-hent1, CD45, CD64, and myeloperoxidase) assay intended to quantitate the expression of hent1 on acute leukemia blast cells (hent1 Index) as part of a companion diagnostic assay under development. This assay is comprised of CD45, CD64, antimyeloperoxidase, and anti-hent1 monoclonal antibodies with hent1 expression reported as a ratiometric index between acute myeloid leukemia blast cells and lymphocytes (blast cell AML index). A similar index is derived from the monocyte to lymphocyte relative binding of anti-hent1 binding. The commercial assays were performed exactly to the manufacturer s directions and run on multiple instrument platforms when possible including: Canto II TM (BD Biosciences, San Jose, CA), FACScan TM (BD Biosciences), FC-500 TM (Beckman Coulter, Miami, FL), and Navios TM (Beckman Coulter). The CD34 acquisition was performed on a single specimen with 11 replicates, sequentially analyzed on the FC- 500 followed by the Navios platform. Post acquisition analysis was done on data acquired on the Navios platform using KALUZA software (Beckman Coulter). All other assays were performed as 10 or 20 intra-assay replicates and data analysis was performed using either the assay-specific software (Leuko64) or Winlist TM (Verity Software House, Topsham, ME). All assays were completed within the recommended period of time stipulated by the manufacturer. Statistical Analysis To determine the acceptable replicate number, n, to assess repeatability for each assay, as each additional replicate of a single specimen was measured, we made sequential determinations of the mean, x, standard deviation, s, and standard error of the mean, p sffiffi, and constructed 95% confidence intervals (95% CIs) of the n mean, ðx2t 0:975;n21 p sffiffi ; x1t n 0:975;n21 psffiffiþ. The ð12aþ100% n CIs were based on the t-distribution with significance level, a , and n 2 1 degrees of freedom. Typically, a single CI is constructed based on n independent random samples from a population. In this case, however, as a single assay result is derived from as many as 5,000 to 100,000 individual fluorescence measurements, we considered that the replicate observations to be independent, and that each sequential mean was calculated from a sample of replicates. For example, for the first two measurements of a specimen, the mean and 95% CI of the mean were constructed with n 5 2, and then this was repeated for the first three measurements with n 5 3, and so forth through the total replicate measurements for each assay. To provide a visual representation of the process, for each assay we constructed scatter plots with the number of replicates on the horizontal axis versus the corresponding mean and endpoints of the 95% CI on the vertical axis (Figs. 1,2,3,4). The intent of this study was to examine the use of an assay repetitively performed on a single specimen as guidance to determine the appropriate number of replicates in designing assay validation, such as for a LDT, clinical trial, or IVD regulatory approval-related study. In order to be a broadly applicable method across various methods with a diversity of analyte units and ranges, we reasoned that comparison of the width of the 95% CI and the cumulative mean could be achieved in a manner similar to the derivation of a coefficient of variation, CV 5s=x; it can describe the relationship of the sample standard deviation and mean. Let E be the observed maximum error of the estimated mean of replicates, where E 5 t 0:975;n21 p sffiffi, then the width of the full CI is n 2E and a factor we termed the variance factor (VF) for replicates can be calculated as 2E=x, which is also equivalent to 2 t 0:975;n21 pffiffi CV. A proof of this relationship is n given as Supporting Information. For each assay, scatter plots were constructed with the number of replicates shown on the horizontal axis and corresponding sequential values of this term on the vertical axis (Fig. 5). Data analyses were conducted using SAS 9.3 (SAS, SAS Institute, Gary, NC). RESULTS The Leuko64 TM assay is designed to measure the upregulation of CD64 expression on the surface of circulating blood neutrophils (PMN) with the clinical application being the detection of infection or sepsis (6 10). Measurements use a patented approach to molecular quantitation on cells using an integrated assay with calibration fluorescence beads, spectrally matched labeled antibodies, and data analysis using lot specific software that reports the CD64 expression as a CD64 index whereby an immunologically defined cell population (PMNs, monocytes, and lymphocytes) is ratioed to the internal bead populations (6). The reportable analyte for this assay is the PMN CD64 index and the lot specific software of this assay uses the monocyte CD64 index as an internal assay positive control parameter. Hence, these two parameters were examined as determinates of what the appropriate number of replicates is required to validate the manufacturer s claim of an imprecision or repeatability CV of <5% for intra-assay studies. As expected, the 95% CI decreases with increasing number of replicates for the Leuko64 assay (Fig. 1), FMH Quik- Quant assay (Fig. 2), hent1-quant assay (results not shown), and CD34 stem cell assay (result not shown).

4 332 DAVIS ET AL. FIG. 1. Cumulative means for neutrophil (PMN) CD64 index (A C) with 95% confidence intervals, along with those of the control parameter of monocytes CD64 index (D F) from Leuko64 assay (Trillium Diagnostics). Replicates were assayed on Canto II (A, D), FACScan (B, E, replicate #9 lost), and Navios (C, F) instruments. The 95% confidence interval decreases mostly within the first two to four replicates or cumulative run number. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.] Interestingly, replicate numbers show little correlation with the interassay CV (Fig. 3). However, the narrowing of the variance factor (VF 5 95% CI / sequential mean, see Supporting Information for formula derivation) occurs primarily over the first three replicates, such that repetitions beyond n 5 3 yield little further improvement in this parameter of imprecision (Fig. 4). This is to be expected based on the fact the square root of n for FIG. 2. Cumulative means of fetal RBC percent (% fetal RBC) and median fluorescence intensity (MFI) with 95% confidence intervals from 20 replicates of a single sample assayed on Canto, FACScan, and Navios instruments using the FMH QuikQuant assay (Trillium Diagnostics). The 95% confidence interval decreases mostly within the first two to four replicates or cumulative run number. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

5 DETERMINATION OF PRECISION FOR CELL-BASED FLUORESCENCE ASSAYS 333 FIG. 3. Interassay coefficient of variation (CV) plot of four different fluorescence cell-based assays and using four different instrument platforms. The association between the repetition numbers of a particular assay (horizon axis) with the CV of replicates provides little information other than differences between assays and platform combination. The optimal number of replicates required to most efficiently assess imprecision cannot be determined based upon CV. n 5 1, 2, 3,... decreases quickly for values below 5 but much more slowly for n > 5. Interestingly, some differences were seen between the instrument models of flow cytometers with the Canto II showing a larger 95% CI and VF compared with the FACScan and Navios (Figs. 1 and 4). The FMH QuikQuant assay is designed to identify fetal RBCs by the level of HbF employing a FITC-labeled anti- FIG. 4. Variance factor plot of four different fluorescence cell-based assays and using four different instrument platforms. The association between the repetition number of a particular assay (horizon axis) with the incremental variance factor (vertical axis, VF 5 95% confidence interval width divided by the assay cumulative result mean) provides an assessment of the appropriate sample size for the desired level of precision of a particular assay. Within a given assay, differences between instrument platforms can be identified, as above where the performance of the Canto II is noted to differ from other platforms for the Leuko64 and FMH QuikQuant assays, which may translate into a higher number of replicates to be performed on that platform to achieve performance equal to other instrument platforms.

6 334 DAVIS ET AL. FIG. 5. Variance factor versus inter-replicate CV plots of CD341 stem cell and FMH QuikQuant assays run on different flow cytometer models. This view of the replicate data provides an easy to read method of determining optimum number of replicates in order to confirm the imprecision across the reportable range. The stem cell count assay on both instruments (A, B) would be optimally evaluated with four or more replicates. The FMH assay would be optimally assessed with three or more replicates on the FACScan (C), but five or more replicates would be required on the Canto II instrument (D) to achieve similar imprecision. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.] HbF monoclonal antibody, being a further improvement of a previously developed method by the addition of propidium iodide as a means to more reliably identify and exclude nucleated cells from the red cell analysis (4,5,11,12). The reportable analyte is the percentage of fetal RBC and clinically used as a means for measuring the amount of fetomaternal hemorrhage. The sample analyzed was in the middle of the expected reportable range of 0 5%. Although a form of rare event counting, the assay is an example of percent positive enumeration by flow cytometry. Again the calculated VF decreases with increasing number of replicates (Fig. 2). However, the VF reduces mostly over the first three replicates for the FACScan and Navios instruments, such that more than four repetitions yield little further improvement in this parameter of imprecision (Fig. 4). As was the case with Leuko64 assay, the Canto II instrument did not achieve a comparable plateau in the VF until after five repetitions and values were larger than those of the other two instrument platforms. Of note is that subsequent technical assessment of the Canto II instrument demonstrated a failing PMT for the FITC channel, the relevant parameter for the analyte determination, which in part explained the relatively inferior performance of that instrument model. The hent1-quant assay is currently being developed for the purpose of measuring cellular expression of hent1 in acute leukemia blast cells. The clinical intent of such a test is to serve as a companion diagnostic assay for new forms of therapy that bypass the hent1 mechanism of drug uptake, which is presumed to be one mechanism of drug resistance (13,14). The reportable parameter for this assay is a ratiometric index of hent1 antibody binding of defined leukocyte population to lymphocytes. As was the case with ZAP-70 assays (15), ratiometric indices to report cell expression provided a more reproducible and consistent manner to report a protein expression in leukemic cells when standards for measurement are not available and the expression is ubiquitous, but differentially expressed by

7 DETERMINATION OF PRECISION FOR CELL-BASED FLUORESCENCE ASSAYS 335 FIG. 6. Variance factor versus inter-replicate CV plots for Leuko64 assay for infection/sepsis with reportable analyte of neutrophil CD64 index (A C) and positive control parameter of monocyte CD64 index (D F) run on three different flow cytometer models. This view of the replicate data provides an easy to read method of determining optimum number of replicates in order to confirm the imprecision across the reportable range. The assay would be optimally assessed with three or more replicates on the FACScan and Navios, but four or more replicates would be required on the Canto II instrument to achieve similar imprecision. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.] different cell types. The assay uses the internal monocyte and neutrophil populations as controls, which were measured in the replicate study done on a normal blood sample. Again the VF for both the monocyte and neutrophil hent1 index values decreases with increasing number of replicates (Fig. 4), not seen when plotting replicate number compared with interassay CV (Fig. 3). The decrease in the VF occurs primarily over the FIG. 7. Variance factor versus inter-replicate CV plots for hent1-quant assay for quantitative measurement of the nucleotide and drug-transported hent1 in neoplastic and normal blood or bone marrow cells with reportable analyte of AML blast cell hent1 index (C, F) and positive control parameters of monocyte (B, E) and neutrophil (A, D) hent1 index run on two different flow cytometer models. This view of the replicate data provides an easy to read method of determining optimum number of replicates in order to confirm the imprecision across the reportable range. The assay would be optimally assessed for imprecision with three or more replicates on both FC500 and Navios platforms. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

8 336 DAVIS ET AL. first two replicates for the derived hent1 index for the intended reportable analyte of AML blast cell and both internal assay control populations of monocytes and neutrophils. The data indicate that beyond three repetitions there is little incremental improvement in the VF parameter of imprecision. The documentation for the IVD assay for enumeration of CD34 hematopoietic stem cells reports that regulatory clearance studies demonstrated an imprecision of repeatability studies typically between 5 and 10% CV for all but very low levels (16,17). The thawed apheresis pack used in this study had a CD341 stem cell level of 45 cells per microliter. The calculated sequential replicate CV and the VF parameter for 10 replicates are shown in Figures 3 and 4, respectively. As with the other assays, the VF decreases with increasing number of replicates. As demonstrated in that Figure 4, if an imprecision for the assay is claimed with a CV of less than 10%, this would be achieved and validated with as few as two replicates on both the FC-500 and Navios platforms and increased precision is not observed beyond three replicates. Figures 5 7 display plots of the novel VF parameter compared to sequential replication CV for all assays and all platforms used in this study. This plot of data analysis allows one to readily observe the optimal replicate number that achieves a desired level of imprecision for a given assay on a given platform. The data clearly demonstrate that at least for the instruments and assays selected for this study, different numbers of replicates may be required to achieve a certain level of imprecision among instrument platforms. Parenthetically, the later discovery that the instrument showing higher VF values, a Canto II, was not optimally performing during the study; this may indicate this parameter to have a further value in instrument QC control. By the plots shown in Figures 5,6,7, one can also visually accept the optimal replicate number for a given assay either by selecting a replicate number below a threshold VF, such as O.2, or at the point that the sequential replicate CV and VF form a cluster of indistinguishable points and improved performance cannot be measured. DISCUSSION The purpose of this study was to derive a data-driven approach for determining the optimal number of replicates necessary to evaluate or validate a cell-based fluorescence assay, such as those in the field of clinical cytometry. Although there is accepted guidance for soluble analyte or chemistry measurements in such documents as the CLSI EP5-A2 (1), such soluble analyte measurements are not comparable to cell-based measurements. The unique aspects of cell-based measurements in matrix, variable autofluorescence between cellular populations, ph environments, etc. are detailed in the accompanying ICSH/ICCS guideline (18). Additionally, chemistry assays typically are based upon single measurements of absorption, fluorescence, or turbidity; this is in contrast to the thousands of individual cell by cell measurements made by typical clinical flow cytometric assays, which are translated by data analysis into a single measurement. These inherent differences logically should have some influence on assay imprecision. Our findings indicate that for the four assays selected for analysis, replicates of less than five replicates would be sufficient to achieve detection of imprecision below that specified by the manufacturer or developer. Alternative approaches to sample size planning, as that proposed by Kelley (19), could be used to obtain an accurate parameter estimate of the CV of assay measurements of independent samples by achieving a sufficiently narrow CI. This approach is termed the accuracy in parameter estimation approach. Functions are available in the R statistical programming language as implemented in the Methods for the Behavioral, Educational, and Social Sciences (20 22). However, these methods have not yet been extended to apply to repeated measurements of the same assay. Further, our findings shown in Figure 3 seemingly indicate that using the sequential replicate CV would not be informative for the intent of determining the correct sample size for imprecision validation. Measurement variation in cellular-based assays, as with any clinical tests, is not only a function of analytical precision of the assay itself but also needs to be judged in the light of clinical and biologic variability or so-called clinically relevant imprecision. It is impractical to set a standard of performance for an assay well above the desired or necessary sensitivity from a clinical perspective. For example, if a particular analyte has a normal diurnal variation of 15%, applying a clinical assay with a level of imprecision lower than 10% CV to samples collected na ıve as to the time of collection has little practical value. Similarly, applying regulatory standards for assay validation appropriate for an assay with a <3% CV to an assay claimed to have an imprecision of <10% CV is improper regulatory science. Thus, having a practical approach to defining the appropriate size for replicate studies for formal validation studies where specimens throughout the reportable range should be examined for the level of imprecision. The plots shown in Figures 5,6,7 provide a graphic evaluation of our novel VF, derived from the 95% CI and the cumulative mean, compared with the incremental CV of interassay replicates derived from a single repeatability study. We selected a sample size of N 5 20 for three of the assays and N 5 10 for assays where specimen volume is typically limited. For all cases of assay type and instrument platform we found that a replicate size of less than five would be sufficient to demonstrate imprecision well below that claimed by the manufacturers. Interestingly, there were assay and instrument platform combinations that required more replicates to achieve a similar 95% CI or VF than other instrument platforms with fewer replicates. Thus, suggesting the importance of viewing a cell-based fluorescence assay as a total system of reagents, instruments, calibrators/standards, and software, rather than assuming the components of the system are interchangeable. Our findings indicate that the

9 DETERMINATION OF PRECISION FOR CELL-BASED FLUORESCENCE ASSAYS 337 plot of our VF parameter versus incremental replicate CV provides an easy visual means of establishing an appropriate number of replicates, also demonstrating that correlating the level of the interassay CV with replicate number is not informative with cell-based fluorescence assays. Further studies are required to confirm utility of this approach for all cell-based assays, as well as possible utility in instrument quality control. ACKNOWLEDGMENTS All authors, except Dr. McLaren, report potential conflict of interest by virtue of being employed by or serving as consultant to manufacturer(s) of the assays used in this study. The authors acknowledge the expert assistance of Wen-Pin Chen, M.S., of the Chao Family Comprehensive Cancer Center, University of California, Irvine, in the performance of statistical analyses. LITERATURE CITED 1. NCCLS. Evaluation of Precision Performance of Quantitative Measurement Methods: Approved Guideline, 2nd ed. NCCLS Document EP5-A2 (ISBN ). Wayne, PA: NCCLS; p NCCLS. Method Comparison and Bias Estimation Using Patient Samples: Approved Guideline, 2nd ed. NCCLS Document EP9-A2 (ISBN ). Wayne, PA: NCCLS; Section Sutherland DR, Keeney M, Gratama JW. Enumeration of CD341 hematopoietic stem and progenitor cells. In: Robinson JP, Darzynkiewicz Z, Dean PN, Dressler LG, Rabinovitch PS, Stewart CS, Tanke HJ, Wheeless LL, editors. Current Protocols in Cytometry. New York: Wiley; pp Fong EA, Finlayson J, Robins F, Davies J, Joseph J, Rossi E, Grey DE. Evaluation of a new rapid anti-hbf FITC assay, Trillium QuikQuant, for detection and quantitation of fetomaternal haemorrhage. Int J Lab Hematol 2013;35: Pastoret C, Priol JL, Fest T, Roussel M. Evaluation of FMH Quik- Quant for the detection and quantification of fetomaternal hemorrhage. Cytometry Part B 2013;84B: Davis BH. Improved diagnostic approaches to infection/sepsis detection. Expert Rev Mol Diagn 2005;5: Hoffmann JJ. Neutrophil CD64: A diagnostic marker for infection and sepsis. Clin Chem Lab Med 2009;47: Bhandari V, Wang C, Rinder C, Rinder H. Hematologic profile of sepsis in neonates: Neutrophil CD64 as a diagnostic marker. Pediatrics 2008;121: Icardi M, EricksonY, Kilborn S, Stewart B, Grief B, Scharnweber G. CD64 index provides simple and predictive testing for detection and monitoring of sepsis and bacterial infection in hospital patients. J Clin Microbiol 2009;47: Gibot S, Bene MC, Noel R, Massin F, Guy J, Cravoisy A, Barraud D, Bittencourt M, Quenot J-P, Bollaert P-E, et al. Combination biomarkers to diagnose sepsis in the critically ill patient. Am J Respir Crit Care Med 2012;186: de Wit H, Nabbe KC, Kooren JA, Adriaansen HJ, Roelandse-Koop EA, Schuitemaker JH, Hoffmann JJ. Reference values of fetal erythrocytes in maternal blood during pregnancy established using flow cytometry. Am J Clin Pathol 2011;136: Wong L, Hunsberger BC, Bruce Bagwell C, Davis BH. Automated quantitation of fetomaternal hemorrhage by flow cytometry for HbF-containing fetal red blood cells using probability state modeling. Int J Lab Hematol doi: /ijlh Hubeek I, Stam RW, Peters GJ, Broekhuizen R, Meijerink JP, van Wering ER, Gibson BE, Creutzig U, Zwaan CM, Cloos J, et al. The human equilibrative nucleoside transporter 1 mediates in vitro cytarabine sensitivity in childhood acute myeloid leukaemia. Br J Cancer 2005;93: Galmarini CM, Thomas X, Calvo F, Rousselot P, El Jafaari A, Cros E, Dumontet C. Potential mechanisms of resistance to cytarabine in AML patients. Leuk Res 2002;26: Shults KE, Miller DH, Davis BH, Flye L, Hobbs LH, Stelzer GT. A standardized ZAP-70 assay-lessons learned in the trenches. Cytometry Part B 2006;70B: Keeney M, Brown W, Gratama J, Papa S, Lanza F, Sutherland DR; European Working Group on Clinical Cell Analysis. Immunophenotypic characterization of CD34(pos) cells. J Biol Regul Homeost Agents 2003;17: Beckman Coulter Stem-Kit TM product information sheet, product #IM Davis BH, Dasgupta A, Kussick S, Han JY, Estrellado A; on behalf of the ICSH/ICCS working group. Validation of Cell-based Fluorescence Assays: Practice Guidelines from the ICSH and ICCS Part II Preanalytical Issues. Cytometry B 2013; 84B: in press. DOI: / cyto.b Kelley K. Sample size planning for the coefficient of variation from the accuracy in parameter estimation approach. Behav Res Methods 2007;39: Kelley K. Confidence intervals for standardized effect sizes: Theory, application, and implementation. J Stat Software 2007;20: Kelley K. Methods for the Behavioral, Educational, and Social Sciences (MBESS) [Computer software and manual]. Available at: Accessed 3 May Kelley K. Methods for the behavioral, educational, and social sciences: An R package. Behav Res Methods 2007;39: