Using Molecular Kits or Laboratory Developed Tests (Home Brew), Emphasis on Validation Molecular Diagnosis Challenges & Solutions Behzad Poopak, DCLS PhD Tehran Medical Branch- Islamic Azad University bpoopak@yahoo.com 1391
Molecular Diagnosis Clinical molecular diagnostics testing: - Oncology - HLA typing - Hematology - Parentage - Inherited diseases - Forensics - Infectious diseases Test types Laboratory-Developed molecular Tests (LDTs) Lab-modified FDA-approved molecular tests FDA-approved molecular tests (exception- Micro) co)
Common Platforms in Molecular l Diagnostics i Assays Conventional PCR Realtime PCR Sequencing FISH Microarray
Important Topics in Molecular Diagnostics Verification & Validation Proficiency Testing Quality Control A Timely & Accurate Result
Principle Requirements of ISO 15189:2007 5.5.1 If in-house procedures are used, they shall be appropriately validated for their intended use and fully documented. The methods and procedures selected for use shall be evaluated and found to give satisfactory results before being used for medical examinations. A review of procedures by the laboratory director or designated person shall be undertaken initially and at defined intervals.
The process of implementing a molecular genetic test for diagnostic use Validation determines that we are doing the correct test; verification confirms that we are doing the test correctly.
Validation The steps involved in test validation are meant to answer one main question Is a given test ready to be implemented in the clinical laboratory? In order to answer this question, the clinical utility (ie, usefulness) of the test must be established, along with its analytic performance characteristics and limitations.
Assay Validation / Evaluation WHO definition: The process of proving that a procedure works as expected and consistently achieves the expected result. CLIA (42 CFR 493.1253) and CAP (GEN 42020-42163) require that laboratories validate the performance of tests before reporting patient results
Requirements For Test Validation CLIA requires that each lab establish or verify the performance specifications of moderate- and highcomplexity test systems that are introduced for clinical use. For an FDA-approved test system without any modifications, the system has been validated d by the manufacturer; therefore, the laboratory need only verify (ie, confirm) these performance specifications. In contrast, for a modified FDA-approved test or an LDT, the laboratory must establish the test s performance specifications, including accuracy, precision, reportable range, and reference range.
Principle requirements of fiso17025 17025:2005320053 The validation shall be as extensive as is necessary to meet the needs of the given application or field of application. The laboratory shall record: - The results obtained, - The procedure used for the validation, & - A statement as to whether the method is fit for the intended use.
Some molecular l assays have been commercially produced d and have gone through an FDA 510(k) clearance or PMA (premarket approval) process. These assays are referred to as in vitro diagnostics or IVDs. In contrast, most new molecular diagnostic assays are developed within individual clinical laboratories and are referred to as laboratory-developed tests or LDTs. Many of these clinically important LDTs are unlikely to ever become FDA cleared or approved because low test volumes make it economically unfeasible to go through h the current FDA 510(k) or PMA approval process.
Verifying Performance Specifications Applies to unmodified, approved test systems Laboratory must Demonstrate measurement response it can is obtain shown to performance be valid. specifications comparable to the manufacturer before are directly proportional to the concentration reporting patient results of the analyte in the sample. Accuracy Precision Reportable range of test results Verify appropriate reference intervals (normal values) for the laboratory s population Reportable range: The range of values over which the relationship between the instrument, kit, or system s Linearity: The ability of the test to return values that
Recommended Approach to Test Validation Validation work has been artificially divided into 3 steps. Step 1: Planning Phase to Define the Requirements of the Test Step 2: Generate Validation Data Step 3: Implement the Test Each successive step should be addressed in sequence and carefully documented
Pre-Validation Considerations for the Design of Laboratory Developed Tests Stringent design/analysis of primers and probes Quality and quantity of extracted nucleic acid Appropriate platform for the test Commutable calibrators and controls Optimization of amplification and detection
Modified FDA-approved tests or non-fda cleared tests (e.g., Laboratory Developed Test-LDT) Validating Performance Specifications Applies to Laboratory-developed tests Approved tests that have been modified by the laboratory Before reporting patient results, the laboratory must establish performance characteristics for Accuracy Precision Reference and Reportable ranges Analytical sensitivity (LOD) (LOQ) Analytic specificity Interfering substances Diagnostic (Clinical) Validity
Validation of a Modified FDA Test Alteration of any process or analyte in an approved test that may affect performance Changes in: type or concentration of specimen used storage specifications extraction method amplification parameters or platforms manufacturer s cut off using the test for another purpose (quantitative vs. qualitative)
Clinical (Diagnostic) Validation Address the clinical significance and utility of the test. Diagnose a disease or disease state Confirm the results of another laboratory test, or clinical diagnosis Monitor and assess disease progression, prognosis or resolution Comparing the test to a gold standard Another test (culture, EIA, chemistry test, FISH) Clinical outcome / endpoint (response to therapy) Use Positive and Negative Predictive values Use ethnic variation and geographic distribution Cite references to clinical studies in the literature
Validation & Type of Specimens Were validation studies with an adequate number and representative (reasonable) distribution of samples performed for each type of specimen expected for the assay (e.g., blood, fresh/frozen tissue, paraffin embedded tissue, prenatal specimens? Verification and Validation studies should be performed on all types of specimens that the test is used for.
Analytical Sensitivity Can Be Done By Control material of known concentration or copy number (calibrators / standards) Dilutions of analyte (microorganism, gene) of known quantity Quantified amount of RNA or DNA
Validation of Laboratory Developed Tests, A. Assay Optimization
Validation of Laboratory Developed Tests, B. Analytic Validation 1. Accuracy a. Analyze a known sample (either of known concentration or result, or both) and compare the result with that t obtained by a reference ( gold-standard ) method Include specimens from the indicated patient population p (specimens used for validation should reflect anticipated population with regards to sex, age, race etc.) Choose appropriate data analysis techniques Choose appropriate reference methods Choose appropriate comparative methods
Validation of Laboratory Developed Tests, B. Analytic Validation 1. Accuracy b. If disease is uncommon, it may be necessary to obtain commercially available reference materials, if they are available. Otherwise, controls may need to be obtained from another laboratory. For sequencing assays, however, the number of possible mutations essentially precludes the use of reference materials.
Validation of Laboratory Developed Tests, B. Analytic Validation 1. Accuracy c. Compare results between new method and reference method or method already established in the lab d. The appropriate number of specimens depends on many factors including, but not limited to: Complexity of the assay Relevance of targets/alleles in the intended use population Data analysis schemes Established accuracy of reference methods
Validation of Laboratory Developed Tests, B. Analytic Validation 2. Precision (getting the same results with repetition of the assay) Closeness of agreement between independent (repeated) results of measurements obtained under stipulated conditions. Describes the degree of dispersion of the values of repeated measurements form each other.
Validation of Laboratory Developed Tests, B. Analytic Validation Elements of precision: Repeatability closeness of agreement between repeated tests of the same sample under the same operating conditions. Performed under the same conditions with multiple replicates, various sample concentrations (QUANTITATIVE), and various patterns of variants (QUALITATIVE). This is referred to as intra- assay variation. Reproducibility closeness of agreement between the results of measurements when operating conditions are varied. All potential sources of variability should be considered. This is referred to as inter-assay variation. Typically, the same material is analyzed by the same protocol on different days, by different operators and with different equipment.
Validation of Laboratory Developed Tests, B. Analytic Validation Sources of variability: Instrument Laboratory Operator (most common source of variability) Sample concentration Sample source Reagent lot (second most common source of variability) Run Time of the day Laboratory environment
Validation of Laboratory Developed Tests, B. Analytic Validation The laboratory should consider each source of variability that can occur during testing. a. Within run: Run one sample several times in one run b. Between runs: Run one or more samples in several different runs over several days c. Determine the mean, standard deviation and coefficient of variation (standard deviation mean) d. Precision studies should be carried out for the entire assay, as applicable, from extraction and amplification to detection
Examples of Precision Evaluation 1. FOR QUANTITATIVE ASSAYS: Take at least 3 sample concentrations (plasma pools or control samples) that cover the clinically important decision levels (e.g. for HCV quantitative assays a low control of~100 IU/mL, a medium of~5000 IU/mL, and a high of ~100,000 IU/mL). Depending on the assay the low concentration replicates can be two to four times the established Level of Detection (LoD), high concentrations should be close to the 99th percentile of tested concentrations/titers.
Thank you, any question?