September 12, Division of Dockets Management (HFA-305) Food and Drug Administration 5630 Fishers Lane Room 1061 Rockville, MD 20852

Transcription

1 September 12, 2016 Division of Dockets Management (HFA-305) Food and Drug Administration 5630 Fishers Lane Room 1061 Rockville, MD RE: Docket No. FDA-2016-D-0971: Draft Guidance for Industry and Food and Drug Administration Staff; Infectious Disease Next Generation Sequencing Based Diagnostic Devices: Microbial Identification and Detection of Antimicrobial Resistance and Virulence Markers Dear Sir/Madam: On behalf of AdvaMedDx, a Division of the Advanced Medical Technology Association (AdvaMed), we respectfully submit these comments in response to the FDA s Draft Guidance for Industry and Food and Drug Administration (FDA) Staff: Infectious Disease Next Generation Sequencing (NGS) Based Diagnostic Devices: Microbial Identification and Detection of Antimicrobial Resistance and Virulence Markers (hereinafter ID Guidance ). AdvaMedDx member companies produce advanced, in vitro diagnostic tests that facilitate evidence-based medicine, improve quality of patient care, enable early detection of disease and reduce overall health care costs. Functioning as an association within AdvaMed, AdvaMedDx is the only multi-faceted, policy organization that deals exclusively with issues facing in vitro diagnostic companies in the United States and abroad. Our membership includes manufacturers engaged in the development of innovative diagnostic technologies supporting the advancement of the public health and personalized medicine from next generation sequencing to complex multi-analyte technologies to a breadth of advanced molecular diagnostics helping deliver cutting-edge care to patients worldwide. GENERAL COMMENTS AdvaMedDx appreciates the efforts of FDA to share its thinking regarding its regulation of NGS-based diagnostics. In addition to issuing the ID Guidance, FDA recently published in the Federal Register two related Draft Guidances for Industry and FDA Staff: 1) Use of Standards in FDA Regulatory Oversight of NGS-Based In Vitro Diagnostic (IVDs) Used for Diagnosing Germline Diseases (July 8, 2016) and 2) Use of Public Human Genetic Variant Databases to Support Clinical Validity for NGS-Based In Vitro Diagnostics (July 8, 2016).

2 Division of Dockets Management (HFA-305) September 12, 2016 Page 2 of 14 We applaud the tremendous efforts by FDA to develop these guidances and extensive communications and outreach for public feedback. Through a series of public workshops, FDA has proposed and received feedback on a number of innovative regulatory concepts such as better leveraging of standards to support analytical validity, use of curated databases for establishing the clinical validity of a test, application of a representative variant approach to support validation, improved transparency of information to patients and healthcare professionals, and the use of regulatory-grade genomic sequences as an alternative comparator for clinical evaluation. We appreciate FDA efforts to support flexible, risk-based regulatory approaches that help promote patient access to diagnostic technologies. FDA has incorporated a number of useful concepts into the ID Guidance. Furthermore, AdvaMedDx believes that this proposed guidance should be reviewed and further harmonized in the broader context of FDA regulation of NGS-based diagnostics to ensure a more consistent and cohesive approach in all three guidances. We generally encourage consistency in scientific and regulatory recommendations across these guidances where at all possible. More specifically, we believe FDA should promote this goal of consistency by incorporating the following concepts into the ID Guidance: Set forth FDA s process and criteria when contemplating recognition of standards developed by the scientific community or standard development organizations that incorporate analytical validation requirements for infectious disease NGSbased tests. Permit use of a declaration of conformity with FDA-recognized standards by developers of infectious disease NGS-based tests in premarket submissions. Consider special controls and/or conditions of exemption that could be developed to provide a reasonable assurance of safety and effectiveness for infectious disease NGS-based tests without necessarily the need for a 510(k), where appropriate. AdvaMedDx appreciates the opportunity to provide our comments, which are intended to support diagnostics innovation and foster advancement of personalized medicine. Our specific comments follow with accompanying recommendations to assist FDA. Please do not hesitate to contact me at or jwolszon@advamed.org if you have any questions. Respectfully submitted, /s/ Jamie K. Wolszon, JD Associate Vice President Technology and Regulatory Affairs

3 AdvaMedDx Comments on Infectious Disease Next Generation Sequencing Based Diagnostic Devices: Microbial Identification and Detection of Antimicrobial Resistance and Virulence Markers; Draft Guidance for Industry and Food and Drug Administration Staff Additions are indicated in underline. Removals are indicated in strike out. Edit # 1. lereneg Clarify whether the ID Guidance would apply to profiling applications that consist of characterizing microbial populations through identification and determination of relative proportions, and if so, whether different performance analyses might apply. It is unclear whether these applications are currently included within the scope of the ID Guidance. 2. lereneg Add Discussion of Limit of Quantitation ( LOQ ) Sequencing has the potential of delivering quantitative results, but LoQ is not mentioned in the ID Guidance. 3. General 4. lereneg Define regulatory grade. Add Center for Biologics Evaluation and Research ( CBER ) to the ID Guidance. In the alternative, clarify that NGS HIV tests are excluded from the scope of the ID Guidance. We understand and appreciate the need for databases to be populated with regulatory-grade target sequences and, given their importance, would appreciate clarity in how the FDA defines regulatory grade. Although the Appendix provides some details on the constitution of a regulatory-grade database, we recommend that this term be clearly defined and articulated in the ID Guidance. Per the inter-center agreement between CBER and CDRH, CBER has lead responsibility for all HIV tests and for tests intended for use to screen the blood supply or tissue and cellular products. While the ID Guidance explicitly excludes (lines ) tests intended for use to screen the blood supply or tissue and cellular products from its scope, it does not exclude HIV tests, and in fact mentions HIV tests as an example. Yet, the ID Guidance makes no reference to CBER. September 12, 2016 Page 3 of 14 Docket No. FDA-2016-D-0971

4 Revise as follows: In contrast to human sequencing diagnostics, infectious disease sequencing diagnostics generally require rapid and actionable results, sometimes within hours, as delayed or incorrect initial diagnoses can result in fatalities. Revise as follows: Furthermore, the broad range of specimen types (e.g., urine, blood, cerebrospinal fluid (CSF), stool, sputum, etc.) and the large diversity of the infectious disease agents that can be present in the sample do not allow straightforward pre-analytical, biochemical, or bioinformatics processes. Revise as follows: The opportunity for repeat testing is expected to may be limited due to a frequently small specimen quantity (e.g., Cerebrospinal fluid (CSF)) and the necessity to make a prompt and timely infectious disease treatment decision for the patient. Human sequencing diagnostics also generally require rapid and actionable results. Variation in pre-analytical processing steps is not unique to assays for infectious agents. Moreover, the bioinformatics preanalytical processes are the same, i.e., searching against known databases for nucleic acid sequences. Small specimen quantity is not unique to ID NGS tests. Moreover, while the opportunity for repeat testing may be small for direct tests, for other tests (e.g., gastrointestinal pathogens), the specimen may be abundant or be cultured (e.g., blood). September 12, 2016 Page 4 of 14 Docket No. FDA-2016-D-0971

5 Add footnote to: The inclusion of certain targets (e.g., Hepatitis B, Hepatitis C, HPV and HIV*) could elevate the classification of the device to Class III, and FDA encourages you to contact the Agency for additional guidance. The footnote would state: CBER has lead responsibility for HIV tests. For questions about HIV NGS tests, please contact CBER. As discussed above, CBER has lead responsibility per the intercenter agreement for HIV tests, yet CBER is not mentioned in the ID Guidance. We recommend that CBER be included in the ID Guidance. However, in the alternative, we would remove the reference to HIV in this example In the alternative, if CBER does not opt to be a coauthor of the ID Guidance, revise as follows: The inclusion of certain targets (e.g., Hepatitis B, Hepatitis C, and HPV and HIV) Revise as follows: In addition, FDA recommends that sponsors contact the Agency prior to undertaking any clinical or analytical validation studies to discuss whether additional recommendations are available due to new advancements in this fast moving field. We recommend that the sponsor utilize the Pre-Submission process outlined in the guidance for Industry and Food and Drug Administration Staff: Requests for Feedback on Medical Device Submissions: The Pre-Submission Program and Meetings with Food and Drug Administration Staff. The existing FDA Guidance for Industry and Food and Drug Administration Staff: Requests for Feedback on Medical Device Submissions: The Pre-Submission Program and Meetings with Food and Drug Administration Staff is utilized heavily by manufacturers, yet this guidance is not referenced in this section of the ID Guidance. We recommend adding wording that points manufacturers to this guidance in this section of the ID Guidance. September 12, 2016 Page 5 of 14 Docket No. FDA-2016-D-0971

6 Revise as follows: An Infectious Disease NGS Dx device differs from traditional diagnostic devices in that it may be targeted to detect specific organisms or markers and could simultaneously detect multiple organisms present in a sample during a single run. Re-title Section IIB. to Databases as an Alternative Comparator Method for Clinical Evaluation and revise the ensuing paragraph to: FDA, in collaboration with various federal agencies, has developed the database entitled FDA-ARGOS: FDA database for Regulatory Grade microbial Sequences; BioProject To promote a least burdensome regulatory approach for devices that incorporate Infectious Disease NGS Dx technology, FDA proposes the use of an alternative comparator method for clinical evaluation that relies heavily on public databases populated with regulatorygrade target sequences. Such databases may be public or proprietary and supply a set of validated regulatorygrade microbial genomic sequence entries that are near complete high-quality draft genomes with metadata requirements. For more information on proprietary databases, see Sections VI(D) and VIII. FDA, in collaboration with various federal agencies, has developed a public database entitled FDA-ARGOS: FDA database for Regulatory Grade microbial Sequences; BioProject This database supplies a set of validated regulatory-grade microbial genomic sequence entries which is available at the National Center for Biotechnology Information s (NCBI) website This sentence applies equally to any multiplex device. Please distinguish within the ID Guidance what applies to NGS and multiplex devices versus what is unique to ID NGS devices. As written currently, this section focuses on the FDA-ARGOS database and its use as an alternative comparator method to clinical evaluation. However, as the FDA clarifies in Section VIII, genomic sequence depositions in the public domain or in proprietary databases can be evaluated and qualified for regulatory purposes. In short, both public and proprietary databases that use regulatory-grade genomic sequences may be used as an alternative comparator for clinical evaluation. As such, we recommend revising this section to account for this, as its focal point should be that regulatory-grade databases, whether public or proprietary, may be used as an alternative comparator method for clinical evaluation. September 12, 2016 Page 6 of 14 Docket No. FDA-2016-D-0971

7 ( (update with FDA web portal link)). Regulatory-grade microbial sequences are near complete high quality draft genomes with metadata requirements. For more information see Section VIII, Appendix Clarify the statement: This database supplies a set of validated regulatory-grade microbial genomic sequence entries which is available at the National Center for Biotechnology Information s (NCBI) website ( (update with FDA web portal link)). Clarify the statement: Regulatory-grade microbial sequences are near complete high-quality draft genomes with metadata requirements. Revise as follows: Targeted Infectious Disease NGS sequencing: Targeted sequencing requires a priori knowledge of the target sequence; thus, its scope is limited to specific targets. For the purposes of this document, targeted sequencing refers to preferential amplification of defined regions that target a specific organism(s) or marker(s) selected for analysis a priori by any laboratory or bioinformatics method (e.g., amplicon sequencing, whole genome sequencing or a k-mer signature database) based on the diagnostic device s intended use. The ID Guidance does not address how the FDA-ARGOS database information was validated. Please include a reference or detailed information on the FDA-ARGOS database validation criteria and process. It is unclear what constituted near complete high-quality draft genomes for the FDA-ARGOS database. It also is unclear what constituted the metadata requirements. Whole genome sequencing (WGS) could be added in "targeted sequencing approaches" for the purposes of the ID Guidance. We are aware of WGS-based tests that use a priori knowledge of sequence targets and database-driven algorithms to identify defined infectious agent(s) or marker(s) of interest. September 12, 2016 Page 7 of 14 Docket No. FDA-2016-D-0971

8 Section IV Address discovery of emerging or novel pathogens or other research applications in a manner consistent with human sequencing. Clarify the following statement: We note that device performance should be established prior to using this alternative comparator. Re-title Section IV. to Risk Management. Revise as follows: The guidance document entitled Factors to Consider When Making Benefit-Risk Determinations in Medical Device Premarket Approvals and De Novo Classifications provides information on FDA benefit-risk determinations. Premarket submissions should include a discussion of the potential benefits and risks associated with the device that is being assessed, the hazards and consequences associated with those risks, analytical strengths and Currently, the ID Guidance explicitly states that it does not cover discovery of emerging or novel pathogens or other research applications. However, emerging and novel pathogens are akin to germline or somatic variants of unknown significance. The expectations for inclusion of these targets should be clarified and should be similar to those for NGS for human sequences. It is unclear what the FDA means by this statement. Based on the discussion throughout the guidance, the FDA-ARGOS database seems to be most useful as an alternative comparator method in the clinical evaluation of Infectious Disease NGS Dx devices. Therefore, it appears that device performance in this context may refer to the analytical performance of the device, but this needs to be clarified. Benefit-Risk Analysis is terminology that, through guidance, FDA has applied to PMA and De Novo submissions. Given that the focus of this Draft Guidance is 510(k)s, PMA terminology such as Benefit-Risk Analysis should avoid being used, and this section should simply be re-titled to Risk Management. Given the focus on 510(k) products in this ID Guidance, the wording should be modified to reflect the activities that take place when a risk analysis is conducted following the principles outlined in the FDA-recognized consensus standard ISO Medical Devices Application of Risk Management to Medical Devices. See also comment above. September 12, 2016 Page 8 of 14 Docket No. FDA-2016-D-0971

9 weaknesses of the technology, and the mitigations that have been put in place to reduce the risks as much as possible. ISO Medical Devices Application of Risk Management to Medical Devices (FDA recognition number 5-40) may be a useful resource when conducting such a risk analysis. clinical information that is available demonstrating device effectiveness Revise as follows: You should describe in detail the methodology used by your device. You should describe, at a minimum, the following elements, as applicable to the device: Design inputs and outputs with a risk analysis and traceability matrix Revise as follows: When applicable, you should include descriptions of design control specifications that address or mitigate risks associated with Infectious Disease NGS Dx devices. For example, design controls may be needed: For manufacturing process procedures that may affect quality. Design inputs and outputs are not typically included in a 510(k), and we do not believe that information is warranted here. See, e.g., Federal Food, Drug, and Cosmetic Act (FDCA) 513((i)(D)(i). FDA can assess this information as part of a postmarket inspection of the company s compliance with the QSR. 21 C.F.R (c) (design input); 21 C.F.R (d) (design output). Information related to manufacturing process procedures is not typically included in a 510(k), and we do not believe that information is warranted here. See, e.g., FDCA 513((i)(D)(i). FDA can assess this information as part of a post-market inspection of the company s compliance with the QSR Revise as follows: A description of how you intend to use your risk assessment to mitigate risks through implementation of any necessary controls over ancillary reagents should be addressed using your risk assessment. These may include, where applicable: It is unclear how an infectious disease NGS device would have an ancillary reagent that is lot-number controlled. A company s complaint-handling protocol or information related to Corrective and Preventive Action (CAPA) should not be September 12, 2016 Page 9 of 14 Docket No. FDA-2016-D-0971

10 Identification of reagent lots that will allow appropriate performance of your device. Complaint handling protocols. required as part of a 510(k) submission. See, e.g., FDCA 513((i)(D)(i). FDA can assess this information as part of a postmarket inspection of the company s compliance with the QSR. 21 C.F.R ; Corrective and preventive actions Revise as follows: This includes any positive and negative controls intended for use with the assay as well as the appropriate internal external controls recommended, but not necessarily provided, for the assay. Revise as follows: You should submit your minimum requirements and cutoffs used as quality control for your material, including, for example, but not limited to the following factors Agarose gel or equivalent technologies that separate DNA fragments based on size A specific quantification of DNA versus RNA. Positive and negative controls are external controls, so the use of external is redundant in this sentence. Furthermore, internal controls are valid and actually control with each patient specimen (i.e., more frequently than once daily). The ensuing discussion includes, appropriately, internal controls as well as external controls. s provide a list of quality control metrics that must be included in infectious disease NGS submissions. However, other than sample amount and sample volume, the remaining QC metrics listed are very likely too low to be measured (for example, viral RNA yield from a plasma specimen). In another context, the target(s) to be sequenced is/are not likely to be distinguishable from other nucleic acids present in the sample prior to enrichment, so the value obtained from the QC metrics would be meaningless. For example, a total nucleic acid extraction from a whole blood sample will likely be >99% human DNA with very good yield (>500 ng), but little to none of this is likely to be microbial, so the values are meaningless for a microbiome assay. Nevertheless, sequencing this sample will still potentially yield a valid microbial profile. We, therefore, recommend that these factors be provided as examples rather than mandatory for inclusion in submissions. September 12, 2016 Page 10 of 14 Docket No. FDA-2016-D-0971

11 Capillary systems with special gels are widely used. Depending on the target, the specific quantification of DNA versus RNA may be needed Clarify the statement: The sequences of the multiple DNA\RNA fragments that comprise the signal outputs of the reaction should be stored in a suitable format that allows subsequent bioinformatics analysis. There are a number of data formats applied to Infectious Disease NGS; however, the most common are the text-based formats FASTA (stores the biological sequence format used to search National Center for Biotechnology Information BLAST database) and FASTQ (stores both the biological sequence and its corresponding quality scores). Revise as follows: (9) Sample to Result Turn-Around Instrument Run Time (TAT) The sample-to-result turn-around instrument run time should be provided for the Infectious Disease NGS Dx device. Please include data demonstrating turn-around times for locked-down Infectious Disease NGS Dx for both: (1) laboratory workflow from clinical sample to sequence, and (2) the subsequent computational analysis of Infectious Disease NGS data from sequence to actionable clinical result. FASTQ and FASTA files are not signal outputs from the sequencers but already processed results (by sequencer provider base-calling programs embedded with the sequencer). If FDA wants signal outputs stored, then FASTQ and FASTA files are not the right formats for that storage. FDA should clarify what data it wants stored, and modify the types of files accordingly. In line 1003, FDA is asking for laboratory workflow from clinical sample to sequence. Since the test is not cleared yet, there is no way for the manufacturer to know how the test will actually be implemented for pre- and post-analytical factors (i.e., timing of results reporting). The only thing the manufacturer can control is sample processing and instrument run time, and not how specimens flow through the laboratory (especially if the test is run on a batched system where laboratories wait to accumulate a minimum number of specimens before running the test). Likewise, before the test is marketed, there is no way of knowing how long, on average, it takes laboratories to report out results. September 12, 2016 Page 11 of 14 Docket No. FDA-2016-D-0971

12 Clarify the statement: Limit of Detection (LoD) provides a measure of assay analytical sensitivity for a particular target, and is defined as the lowest concentration of a target that can be sequenced reliably and distinguished from negative specimens with consistent detection in 95% of the specimen replicates. Revise as follows: In brief, the LoD determination can be performed with a pool of different claimed targets in claimed specimen matrix. This pooled approach is also applicable for inclusivity and reproducibility studies. Since all NGS technologies have a limited capacity within which all DNA molecules are competing to be processed, the pooled approach should be carefully designed to avoid the bias possibly introduced by a too high level of multiplexing. Revise as follows: Please note that in the majority of instances only one site may be located outside of the United States. If additional sites are desired, please contact the Agency. It is unclear what FDA means by can be sequenced reliably in this statement. It may mean coverage at a minimum of 20X over 95% of the core genome at Phred like quality score 8 => Q30 or a lower level of coverage if adequate justification is provided, as is indicated in lines , but clarification on this point is warranted. Added to help avoid potential for bias. FDA should consider a manufacturer request to include more than one site outside of the United States, particularly if an infectious agent is at higher prevalence outside of the United States, or if the study timing coincides with disease seasonality (e.g., flu studies) Establish sample requirements more consistent with prior Agency requests for NGS or PCR diagnostic products or provide sound statistical rationale for requiring 1500 prospective samples for the evaluation of negative percent agreement. The ID Guidance requests 1500 prospective samples for the evaluation of negative percent agreement, which is inconsistent with precedent; FDA required 500 samples to demonstrate accuracy for NGS diagnostic devices in previous related submissions and 755 samples were used in a prospective clinical study to evaluate the performance of recently cleared PCR assay intended as an aid for infection control in detecting September 12, 2016 Page 12 of 14 Docket No. FDA-2016-D-0971

13 and differentiating genetic markers of resistance. While the Agency provides statistical rationale to support the need for at least 50 positive specimens for the evaluation of positive percent agreement (lines ), FDA fails to provide a similar statistical rationale for why this number of samples is needed for infectious disease NGS diagnostic devices. We urge FDA to establish sample requirements more consistent with previous expectations. 30. Section VI. E. (1) and (2) Allow regulatory-grade genomic target sequence databases to be used as alternative comparators for both negative percent agreement and positive percent agreement evaluations. The ID Guidance indicates that the use of regulatory-grade databases as an alternative comparator method in clinical evaluations is only suitable for negative percent agreement and not positive percent agreement evaluations. We disagree with this approach and believe that regulatory-grade databases are suitable for use as alternative comparator methods in both negative percent agreement and positive percent agreement evaluations. The choice of a comparator method for sequencing applications can be very challenging, particularly for rare variants. There are few FDA-cleared or approved technologies from which to choose, and their technology is often inferior to that of the submitted device. Use of a regulatory-grade database as an alternative comparator alleviates these challenges associated with the use of traditional comparator methods Revise as follows: Furthermore, for Class II devices, modification or an update of a library and bioinformatics pipeline would require premarket review by the FDA if the modification has the potential to significantly affect the safety or effectiveness of the device, as determined by the manufacturer according to 21 C.F.R In their current form, lines indicate that all library and bioinformatics pipeline updates or modifications should be communicated to the FDA prior to their use and implementation. For Class II devices (which are the main focus of this guidance), this is contrary to the principles provided in the Federal Food, Drug, and Cosmetic Act and the implementing regulations. In September 12, 2016 Page 13 of 14 Docket No. FDA-2016-D-0971

14 should be communicated to FDA prior to use and implementation particular, the regulations state that, for Class II devices, changes or modifications to a device that could significantly affect its safety or effectiveness would require premarket notification. 21 C.F.R Therefore, it should not be necessary to submit all library and bioinformatics pipelines updates or modifications to the FDA for premarket review but only those that could significantly affect the device safety or effectiveness. For example, a manufacturer may determine that a modification to a single line of code in a bioinformatics pipeline does not influence the device s safety or effectiveness. Such a modification should, therefore, not be submitted to the FDA for premarket review or as a notification. The text should be revised accordingly. Additionally, we encourage the FDA to consider a more flexible approach to modifications of NGS tests to encourage innovation while protecting public health Clarify the statement: Extracted gdna should be of high quality and purity, and at sufficient concentration to achieve a suitable yield to assure adequate depth and breadth of genomic coverage for the type of sequencing method employed. We agree with this statement, but the statement is very subjective. We recommend that FDA add criteria, for example, of what constitutes high quality, what constitutes high purity, and what is adequate depth and breadth of genomic coverage that was used to establish the regulatory-grade database FDA- ARGOS. September 12, 2016 Page 14 of 14 Docket No. FDA-2016-D-0971