with drmid Dx for Illumina NGS systems

Transcription

1 Performance Characteristics BRCA MASTR Dx with drmid Dx for Illumina NGS systems Manufacturer Multiplicom N.V. Galileïlaan Niel Belgium Revision date: July 27, 2017 Page 1 of 8

2 Table of Contents 1 WORKFLOW PERFORMANCE CHARACTERISTICS OF BRCA MASTR DX AND MID DX FOR ILLUMINA MISEQ ON GERMLINE DNA EXTRACTED FROM BLOOD STUDY SETUP ON TARGET READ COUNTS UNIFORMITY OF AMPLIFICATION ACCURACY, SENSITIVITY & SPECIFICITY PRECISION PERFORMANCE CHARACTERISTICS OF BRCA MASTR DX AND MID DX FOR ILLUMINA MISEQ ON FRESH FROZEN TISSUE DNA STUDY SETUP ON TARGET READ COUNTS UNIFORMITY OF AMPLIFICATION ACCURACY, SENSITIVITY & SPECIFICITY LIST OF ABBREVIATIONS DISCLAIMER... 8 List of Figures Figure 1: graphic representation of the MASTR workflow... 3 List of Tables Table 1. Performance characteristics BRCA MASTR Dx and MID Dx for Illumina MiSeq on germline DNA extracted from blood... 5 Table 2. Performance characteristics BRCA MASTR Dx and MID Dx for Illumina MiSeq on FFT DNA... 7 Revision date: July 27, 2017 Page 2 of 8

3 1 WORKFLOW Figure 1: graphic representation of the MASTR workflow Revision date: July 27, 2017 Page 3 of 8

4 2 PERFORMANCE CHARACTERISTICS OF BRCA MASTR DX AND MID DX FOR ILLUMINA MISEQ ON GERMLINE DNA EXTRACTED FROM BLOOD 2.1 Study setup The combined performance characteristics of the BRCA MASTR Dx and the MID Dx for Illumina MiSeq kits were assessed in three independent medical genetic laboratories. In total 146 independent DNA samples were amplified, sequenced and analyzed; as described in the corresponding IFUs. Each laboratory extracted, amplified and sequenced DNA samples selected amongst clinical samples. DNA was extracted by either Phenol/chloroform extraction method, NaCl extraction method, Maxwell 16 Instrument MX3030 (Promega), QuickGene DNA Whole blood kit L (FUJIFILM Life Science), Qiagen Autopure LS or chemagic STAR DNA Blood Kits (Perkin Elmer). A reference method, such as Sanger sequencing and/or 454 sequencing using BRCA MASTR Dx, was performed for these samples. Each laboratory also processed and sequenced 4 proficiency DNA samples, provided by Multiplicom. Thermal cyclers used by the laboratories: Biorad, Biometra (Westburg), Applied Biosystems Fragment analyzers used: Qiaxcell (Qiagen) and Applied Biosystems. A total of five Illumina MiSeq runs was performed; three runs with the Illumina MiSeq Reagent Kit v2 (44 samples each) and two runs with the Illumina MiSeq Reagent Nano Kit v2 (15 samples each). This resulted in the generation of 3,573,788 sequenced bases, of which 2,924,088 bases had been determined by one of the reference methods. Software for data analysis included the SeqNext module v4.1.2 of the Sequence Pilot software (JSI medical systems, Kippenheim, Germany), the Sophia DDM application with pipeline ID ILL1MR1G3 (Sophia Genetics, Ecublens, Switzerland) and internally with the MASTR Reporter by uploading the existing FASTQ files to MASTR Reporter v On target read counts The total number of filter passed reads per sample was mapped to the human genome and the on target reads counted. The percentage of on-target, or amplicon-derived, reads are represented in Table Uniformity of amplification For each sample the number of read pairs per amplicon were counted and normalized to the average read counts per amplicon. Combining all normalized data showed that 99.7 % of the total read pairs fall within 20 % of the mean number of read pair counts (Table 1). 2.4 Accuracy, sensitivity & specificity These parameters are calculated using the following definitions: True positive (TP): variant bases present both in the NGS and the reference dataset True negative (TN): non variant bases present both in the NGS and the reference dataset False positive (FP): variant bases present in the NGS dataset, absent in the reference dataset False negative (FN): variant bases present in the reference dataset, absent in the NGS dataset Accuracy = (TP + TN) / (TP + TN + FP + FN) Sensitivity = TP / (TP + FP) Specificity = TN / (TN + FN) Only those regions with sufficient coverage were taken into account for the calculation, thus 2,918,750 bases instead of 2,924,088 bases. The observed values and their 95 % confidence intervals for these 3 parameters analyzed with Sequence Pilot, Sophia DDM and MASTR Reporter are represented in Table 1. Revision date: July 27, 2017 Page 4 of 8

5 2.5 Precision Based on the NGS dataset from the two DNA samples tested twice in each laboratory that participated in the validation studies, the inter-run accuracy was calculated as (95 % CI [ %]) and the inter-laboratory accuracy was determined (95 % CI [ %]) with Sequence Pilot, Sophia DDM (Table 1) and MASTR Reporter. Table 1. Performance characteristics BRCA MASTR Dx and MID Dx for Illumina MiSeq on germline DNA extracted from blood Parameter (SeqNext) On target read counts 98.4 % (range: 94.7 % %) Uniformity amplification Sensitivity* Specificity* Accuracy* Reproducibility* of % [ 99.8 %] % [ %, ] % [ %, ] [ %] (Sophia DDM ) [ 99.8 %] [ %] [ %] [ %] (MASTR Reporter) [ %] [ %] [ %] [ %] * Excluding pure changes in length of homopolymer stretches of 10bp or longer. ** MASTR Rep: excluding homopolymer stretches of 10 bp or longer. A sensitivity of (95 % CI [ %]) was achieved in the performance evaluation study described in table 1. However, one could also apply following additional procedures for attaining high sensitivity for BRCA MASTR Dx with MID Dx for Illumina MiSeq : Detailed analysis of amplicon sizes: o Perform, for each set of BRCA MASTR Dx-derived amplicons ( Plex ), the procedure described in section 10.5 Quality control of Universal PCR by fluorescent labeling and fragment analysis of the IFU. o PCR products from samples that contain insertions or deletions of one or more bases will result in a size change of the amplicon(s) harboring such variants. o Detailed comparison of the amplicon profiles to a reference profile without insertion/deletion variants, will allow identifying the presence of insertion/deletion variants in specific amplicons of the sample analyzed. o Detailed analysis of the sequence reads of amplicons with modified lengths may in a limited number of cases allow confirming variants that were missed by standard variant calling. Reduced stringent filter settings for the sequence read data analysis using the SeqNext module of SeqPilot (JSI medical systems): o Review the variants in the distinct and the other table that are close to the threshold of 15 % in one or both directions. Revision date: July 27, 2017 Page 5 of 8

6 o Technical validation of variants by experienced customers may be obtained by assessing the supportive evidence in the sequence reads as visualized by the SeqNext software. In case of doubt, it is highly recommended to confirm variants with a second test 3 PERFORMANCE CHARACTERISTICS OF BRCA MASTR DX AND MID DX FOR ILLUMINA MISEQ ON FRESH FROZEN TISSUE DNA 3.1 Study setup The combined performance characteristics of the BRCA MASTR Dx and the MID Dx for Illumina MiSeq kits on fresh frozen tissue (FFT) were assessed in one medical genetic laboratory and at Multiplicom. DNA extracted from in total 10 ovarian and 20 breast cancer tissues, and their matched adjacent normal tissues was amplified, sequenced and analyzed; as described in the corresponding IFUs. The DNA of all tissue samples was extracted using Nucleon extraction kit (Gen- Probe). Since no reference data was available for the samples, the sequencing results obtained on the normal tissues were used as reference data for the data obtained on the matched cancer tissues. Thermal cyclers used by the laboratories: Applied Biosystems Fragment analyzers used: Qiaxcell (Qiagen) and Applied Biosystems. A total of three Illumina MiSeq runs was performed; two runs with the Illumina MiSeq Reagent Kit v3 and one with Illumina MiSeq Reagent Kit v2. Software for data analysis included the SeqNext module v4.1.2 of the Sequence Pilot software (JSI medical systems, Kippenheim, Germany) and the Sophia DDM application with pipeline ID ILL1MR1S4 (Sophia Genetics, Ecublens, Switzerland). 3.2 On target read counts The total number of filter passed reads per sample was mapped to the human genome and the on target reads counted. The percentage of on-target, or amplicon-derived, reads are represented in Table Uniformity of amplification For each sample the number of read pairs per amplicon were counted and normalized. Combining all normalized data showed that % of all amplicons are covered with more than 20 % of the mean number of read pair counts (Table 2). 3.4 Accuracy, sensitivity & specificity These parameters are calculated using the following definitions: True positive (TP): variants present in both cancer and normal tissue NGS datasets True negative (TN): non variant present in both cancer and normal tissue NGS datasets False positive (FP): variant bases present in the cancer NGS dataset, absent in the normal tissue NGS dataset, except when Sanger sequencing confirmed this observation False negative (FN): variant bases present in the normal tissue NGS dataset, absent in the cancer NGS dataset, except when Sanger sequencing confirmed this observation Accuracy = (TP + TN) / (TP + TN + FP + FN) Sensitivity = TP / (TP + FP) Specificity = TN / (TN + FN) Revision date: July 27, 2017 Page 6 of 8

7 No regions were excluded due to insufficient coverage. The observed values and their 95 % confidence intervals for these three parameters analyzed with Sequence Pilot and Sophia DDM are represented in Table 2. Table 2. Performance characteristics BRCA MASTR Dx and MID Dx for Illumina MiSeq on FFT DNA Parameter (SeqNext) (Sophia DDM ) On target read counts 97.8 % (range: 91.8 % %) Uniformity of amplification % Sensitivity* Specificity* Accuracy* [ 99.3 %] % [ %] % [ %] * Excluding pure changes in length of homopolymer stretches of 10bp or longer. [ 99.3 %] % [ %] % [ %] 4 LIST OF ABBREVIATIONS CE: DNA: FFT: FN: FP: IFU: IVD: MASTR Dx: drmid: NGS: PCR: Plex: TN: TP: The CE symbol certifies that a product complies with the European standards. Deoxyribonucleic acid Fresh Frozen Tissue False Negative False Positive Instructions For Use For In Vitro Diagnostic Use MultiPlex Amplification of Specific Target for Resequencing for Diagnostics dual read Molecular Identifier Next-Generation Sequencing Polymerase Chain Reaction Set of MASTR Dx-derived amplicons True Negative True Positive Revision date: July 27, 2017 Page 7 of 8

8 5 DISCLAIMER The purchase of this product enables the purchaser to use it for the amplification of nucleic acid sequences for human genes. Purchaser has to take into account that information obtained from amplicons generated using this product may not be used in procedures that are protected by valid claims owned and/or controlled by a third party, unless prior written approval of such party has been obtained. IVD product performance claims apply only when combining MASTR Dx and MID Dx and when both CE-Mark kits are used according to the specific CE-IVD Instructions For Use. PR NUMBER PR ENE Printed in Belgium, September 2017 Revision date: July 27, 2017 Page 8 of 8