January 12, 2015 U.S. Department of Health and Human Services Food and Drug Administration Center for Devices and Radiological Health Office of In Vitro Diagnostics and Radiological Health Division of Immunology and Hematology Hematology Branch RE: (1) [FDA- 2014- D- 1351]Flow Cytometric Devices: Draft Guidance for Industry and Food and Drug Administration Staff To whom it may concern: We are writing on behalf of the International Clinical Cytometry Society (ICCS) to comment on the above- referenced draft guidance. ICCS is a non- profit professional organization dedicated to promoting education and advancements world- wide in the application of flow cytometry to clinical needs (http://www.cytometry.org/web/index.php). It is the largest organization of its kind and its membership is comprised of practicing physicians, scientists, medical technologists, and laboratory personnel focused on promoting the highest standards of clinical practice in this field. Although the majority of ICCS members are not the specified target audience, we do consider ourselves to be crucial stakeholders affected by this guidance. Furthermore, should the recently published Draft Guidance for Industry, Food and Drug Administration Staff, and Clinical Laboratories: Framework for Regulatory Oversight of Laboratory Developed Tests (LDTs) eventually be adopted, ICCS members would become de facto manufacturers of cytometric devices. We thank you for the opportunity to contribute to this important document and look forward to ongoing engagement with the FDA. Our overarching concerns around this guidance are essentially all related to its clarity and timeliness, particularly as regards the distinction between qualitative (i.e. leukemia/lymphoma testing) and quasi- quantitative (i.e. CD34 enumeration)assays. An excellent summary of this distinction and the many fundamental ways in which these types of assays differ may be found in Wood et al. s 2013 Validation of cell- based fluorescence assays: practice guidelines from the ICSH and ICCS - part V - assay performance criteria. (2) Given that the guidance does not include this reference, any of the other related and co- published ICSH/ICCS fluorescence assay validation references (3-8), or indeed any reference more recent than 2007, it is not surprising that the guidance content is somewhat dated and does not always reflect current thinking or practice regarding flow cytometry. The guidance also ignores many technological advances over the last decade. We have included a number of more recent references at the end of this letter, and urge you to consider them when drafting the next version of this guidance.
Insofar as guidance specifics are concerned, we summarize here a number of issues. We start with what we consider to be the most important omissions from the current draft and subsequently address statements that we find to be inaccurate, incomplete, or unnecessarily prescriptive. Note that we use line numbers to reference specific points of the draft guidance, as the section labeling format is not applied uniformly across the document, and does not correspond to the Table of Contents. As indicated earlier in this letter, we consider the distinction between qualitative and quasi- quantitative assays to be of paramount importance (9), and strongly encourage emphasis of this concept in the guidance. Separate sections of the guidance (or possibly even separate guidance documents) addressing the two types of assays would be particularly helpful, as certain concepts such as linearity, reference ranges, and reference intervals are not relevant to qualitative assays. Similarly, the guidance regarding qualitative assays needs to acknowledge that limit of detection (LOD) and limit of blank (LOB) for qualitative assays such a leukemia/lymphoma immunophenotyping is closely linked to the phenotype of a specific patient s aberrant/malignant population (as well as to the number of simultaneously- assessed parameters and the reagents used), and the patient- to- patient variability is such that LOD/LOB is very nearly patient/sample- specific. Consequently, the cut- off attainable for one patient may not be appropriate for another patient. This issue becomes especially relevant in the case of minimal residual disease (MRD). The section entitled Enumeration of Rare Events (lines 626 649), cites suggested target detection levels from the 2006 Bethesda International Consensus Recommendations on the Immunophenotypic Analysis of Hematolymphoid Neoplasia by Flow Cytometry (10) as follows: B cell, 0.1%; T cell, 1%; Myelomonocytic, 0.5%; and Plasma cell, 0.1%. The guidance appears to suggest that these levels might be informative for minimal residual disease. These targets, however, were not intended to be used in the MRD context but instead reflected the experience of the authors when surveyed regarding typical lineage- specific limits of detection when using their usual reagent combinations and their own, laboratory- developed methods for routine diagnosis and monitoring of patients with hematolymphoid neoplasms. We do not recommend use of these ballpark figures as benchmarks for minimal residual disease assays, and instead encourage the use of more rigorous and clinically- relevant, disease- specific decision points as documented in the extensive medical literature on this topic. (11-18) Other statements that we find to be inaccurate, incomplete, or unnecessarily prescriptive include but are not limited to the following: 1. Specimen and Data Analysis, line 480 The use of paired isotype matched control antibodies is suggested here and elsewhere (lines 536 and 895) in the guidance. A large body of literature describes the limitations and failings of isotype controls (19,20) and we do not recommend their use. This section also ignores Fluorescence- Minus- One controls for gating (21) as well as newer computational data analysis tools such as probability state modeling (22,23).
2. Analytical Specificity: Reagents, lines 430-454 This section suggests the use of Western blots to evaluate monoclonal antibodies without noting that Western blots present antigens in their denatured format. Only linear epitopes can therefore be assessed using this technique; epitopes that are presented conformationally on the cell surface will not be recognized by Western blot (24). This section also states that specificity should be demonstrated by showing that the binding of each antibody in the multi- color reagent cocktail (single parameter analysis) is equal to that of the individual antibody- conjugates when used to stain the antigen expressing cells at the same concentration as in the final cocktail. From a practical perspective this requirement may not be possible to meet, as some antigens cannot be reliably detected by themselves. There is no discussion regarding reagents for defining signal to noise ratio, a basic but important aspect of many flow cytometric assays, nor is there any discussion of reagent optimization (25). Non- antibody reagents for flow cytometry assays are not discussed. 3. Detection Sensitivity, lines 511 623, Figure 1 The procedure described in this lengthy and prescriptive section is not referenced, and is not widely used by the flow cytometry community. 4. Linear Range, lines 651-662 We suggest inclusion of guidance for demonstration of linearity for assays measuring antigen density (8,26). 5. Repeatability & Reproducibility, lines 664 736 These sections state that repeatability and reproducibility should be demonstrated using clinical specimens that are prepared 20 times and analyzed in batch fashion and further indicate that this testing be carried out on blood, bone marrow, lymph nodes, or other specimen types. Not only is this suggestion not even remotely feasible given the constraints of patient sample size (27,28), it is unnecessary: recent publications demonstrate that 3 to 6 replicates over three analytical runs are adequate to demonstrate repeatability and reproducibility (2,8,29). Similarly, patient samples that span the reportable range from low, medium and high are essentially irrelevant in the context of qualitative assays for leukemia/lymphoma. We suggest that the statement Where specimen availability and stability permit, you should include at least 20 days and at least 2 replicates per day be deleted, as there are no instances in which specimen availability and stability would permit this testing, and currently no clinical setting to which it would apply. In addition to the above specific issues, we would like to make several additional points regarding revision of this draft guidance. First, recent advances in flow cytometry technology should be discussed, including cytometers with greater than 4- color capacity, cytometers with digital (rather than analog) signal processing, cytometers and assays that do not require compensation, new fluorophors, and new blood collection tubes. Note also that Clinical Laboratory Standards Institutes (CLSI) has adopted the term measurand to replace analyte (30); we encourage the use of current
terminology. Reference ranges in general are similarly no longer considered to be relevant by the clinical laboratory community (31). In conclusion, we would like to thank the FDA for the opportunity to provide constructive criticism on this draft guidance. We encourage you to take advantage of the wealth of experience and expertise within the clinical flow cytometry community in general, and particularly within the International Clinical Cytometry Society. Please do not hesitate to contact any of us for clarification or assistance regarding this letter. We look forward to working together with the FDA on future versions of this guidance. Best regards, The ICCS Advocacy Bruce Davis, brucedavis@trilliumdx.com Jeannine Holden, jeannineholden@gmail.com Jerry Hussong, jerry.hussong@aruplab.com Virginia Litwin, virginia.litwin@covance.com Teri Oldaker, toldaker@cox.net Angela Salazar, SalazarA@Sysmex.com Elizabeth Stone, estone@csilaboratories.com Paul K Wallace, paul.wallace@roswellpark.org
References ICCS Advocacy 1. FDA- 2014- D- 1351-0002 FD. 2014. Flow Cytometric Devices Draft Guidance for Industry and Food and Drug Administration Staff. http://www.regulations.gov/ -!documentdetail;d=fda- 2014- D- 1351-0002. 2. Wood B, Jevremovic D, Bene MC, Yan M, Jacobs P, Litwin V, Group IIW. Validation of cell- based fluorescence assays: practice guidelines from the ICSH and ICCS - part V - assay performance criteria. Cytometry B Clin Cytom 2013;84:315-23. 3. Barnett D, Louzao R, Gambell P, De J, Oldaker T, Hanson CA, Group IIW. Validation of cell- based fluorescence assays: practice guidelines from the ICSH and ICCS - part IV - postanalytic considerations. Cytometry B Clin Cytom 2013;84:309-14. 4. Bene MC, Marti GE. ICSH/ICCS practice guidelines special issue. Cytometry B Clin Cytom 2013;84:279-80. 5. Davis BH, Dasgupta A, Kussick S, Han JY, Estrellado A, Group IIW. Validation of cell- based fluorescence assays: practice guidelines from the ICSH and ICCS - part II - preanalytical issues. Cytometry B Clin Cytom 2013;84:286-90. 6. Davis BH, McLaren CE, Carcio AJ, Wong L, Hedley BD, Keeney M, Curtis A, Culp NB. Determination of optimal replicate number for validation of imprecision using fluorescence cell- based assays: proposed practical method. Cytometry B Clin Cytom 2013;84:329-37. 7. Davis BH, Wood B, Oldaker T, Barnett D. Validation of cell- based fluorescence assays: practice guidelines from the ICSH and ICCS - part I - rationale and aims. Cytometry B Clin Cytom 2013;84:282-5. 8. Tanqri S, Vall H, Kaplan D, Hoffman B, Purvis N, Porwit A, Hunsberger B, Shankey TV, Group IIW. Validation of cell- based fluorescence assays: practice guidelines from the ICSH and ICCS - part III - analytical issues. Cytometry B Clin Cytom 2013;84:291-308. 9. Lee JW, Devanarayan V, Barrett YC, Weiner R, Allinson J, Fountain S, Keller S, Weinryb I, Green M, Duan L and others. Fit- for- purpose method development and validation for successful biomarker measurement. Pharm Res 2006;23:312-28. 10. Wood BL, Arroz M, Barnett D, DiGiuseppe J, Greig B, Kussick SJ, Oldaker T, Shenkin M, Stone E, Wallace P. 2006 Bethesda International Consensus recommendations on the immunophenotypic analysis of hematolymphoid neoplasia by flow cytometry: optimal reagents and reporting for the flow cytometric diagnosis of hematopoietic neoplasia. Cytometry B Clin Cytom 2007;72 Suppl 1:S14-22. 11. Paiva B, Vidriales M- B, Cervero J, Mateo G, Perez JJ, Montalban MA, Sureda A, Montejano L, Gutierrez NC, De Coca AG and others. Multiparameter flow cytometric remission is the most relevant prognostic factor for multiple myeloma patients who undergo autologous stem cell transplantation. Blood 2008;112:4017-4023. 12. Rawstron AC, Child JA, de Tute RM, Davies FE, Gregory WM, Bell SE, Szubert AJ, Navarro- Coy N, Drayson MT, Feyler S and others. Minimal residual disease assessed by multiparameter flow cytometry in multiple myeloma: impact on
outcome in the Medical Research Council Myeloma IX Study. Journal of Clinical Oncology 2013;31:2540-2547. 13. Roussel M, Lauwers- Cances V, Robillard N, Hulin C, Leleu X, Benboubker L, Marit G, Moreau P, Pegourie B, Caillot D and others. Front- line transplantation program with lenalidomide, bortezomib, and dexamethasone combination as induction and consolidation followed by lenalidomide maintenance in patients with multiple myeloma: a phase II study by the Intergroupe Francophone du Myélome. Journal of Clinical Oncology 2014;32:2712-2717. 14. Jaso JM, Wang SA, Jorgensen JL, Lin P. Multi- color flow cytometric immunophenotyping for detection of minimal residual disease in AML: past, present and future. Bone Marrow Transplant 2014;49:1129-38. 15. Paiva B, Chandia M, Puig N, Vidriales MB, Perez JJ, Lopez- Corral L, Ocio EM, Garcia- Sanz R, Gutierrez NC, Jimenez- Ubieto A and others. The prognostic value of multiparameter flow cytometry minimal residual disease assessment in relapse multiple myeloma. Haematologica 2014. 16. Rawstron AC, Bottcher S, Letestu R, Villamor N, Fazi C, Kartsios H, de Tute RM, Shingles J, Ritgen M, Moreno C and others. Improving efficiency and sensitivity: European Research Initiative in CLL (ERIC) update on the international harmonised approach for flow cytometric residual disease monitoring in CLL. Leukemia 2013;27:142-9. 17. Coustan- Smith E, Campana D. Immunologic minimal residual disease detection in acute lymphoblastic leukemia: a comparative approach to molecular testing. Best Pract Res Clin Haematol 2010;23:347-58. 18. Bar M, Wood BL, Radich JP, Doney KC, Woolfrey AE, Delaney C, Appelbaum FR, Gooley TA. Impact of minimal residual disease, detected by flow cytometry, on outcome of myeloablative hematopoietic cell transplantation for acute lymphoblastic leukemia. Leuk Res Treatment 2014;2014:421723. 19. Stetler- Stevenson M, Ahmad E, Barnett D, Braylan RC, DiGiuseppe JA, Marti GE, Menozzi D, Oldaker TA, Orfao de Matos A, Rabellino E and others. Clinical Flow Cytometric Analysis of Neoplastic Hematolymphoid Cells; Approved Guideline- Second Edition. CLSI document H43- A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2007. 20. Stelzer GT, Marti G, Hurley A, McCoy P, Jr., Lovett EJ, Schwartz A. U.S.- Canadian Consensus recommendations on the immunophenotypic analysis of hematologic neoplasia by flow cytometry: standardization and validation of laboratory procedures. Cytometry 1997;30:214-30. 21. Roederer M. Compensation in flow cytometry. Curr Protoc Cytom 2002;Chapter 1:Unit 1 14. 22. Herbert DJ, Miller DT, Bruce Bagwell C. Automated analysis of flow cytometric data for CD34+ stem cell enumeration using a probability state model. Cytometry B Clin Cytom 2012;82:313-8. 23. Pedreira CE, Costa ES, Lecrevisse Q, van Dongen JJ, Orfao A, EuroFlow C. Overview of clinical flow cytometry data analysis: recent advances and future challenges. Trends Biotechnol 2013;31:415-25. 24. Gallagher S, Winston SE, Fuller SA, Hurrell JG. Immunoblotting and immunodetection. Curr Protoc Cell Biol 2011;Chapter 6:Unit6 2.
25. Tario Jr. JD, Wallace PK. Reagents and Cell Staining for Immunophenotyping by Flow Cytometry. In: McManus LM, Mitchell RN, editors. Pathobiology of Human Disease. San Diego: Elsevier; 2014. p 3678-3701. 26. Maher KJ, Klimas NG, Hurwitz B, Schiff R, Fletcher MA. Quantitative fluorescence measures for determination of intracellular perforin content. Clin Diagn Lab Immunol 2002;9:1248-52. 27. Litwin V, Green C. 2013. The role of biomarkers in clinical trials and the fit- for- purpose method validation approach. http://www.fda.gov/downloads/medicaldevices/newsevents/workshopsconf erences/ucm345757.pdf. 28. Oldaker T. 2013. LDTs in Flow Cytometry: ICSH/ICCS guidelines for validation of fluorescent cell- based diagnostic testing. http://www.fda.gov/downloads/medicaldevices/newsevents/workshopsconf erences/ucm345758.pdf. 29. O'Hara DM, Xu Y, Liang Z, Reddy MP, Wu DY, Litwin V. Recommendations for the validation of flow cytometric testing during drug development: II assays. J Immunol Methods 2011;363:120-34. 30. QC, Westgard. 2009. Glossary of QC terms. http://www.westgard.com/glossary.htm. 31. Horowitz GL, Altaie S, Boyd JC, Ceriotti F, Garg U, Horn P, Pesce A, Sine HE, Zakowski J. Defining, establishing, and verifying reference intervals in the clinical laboratory; approved guideline third edition. CLSI document EP28- A3C. Wayne, PA; 2010.