ESCMID Online Lecture Library. by author

Transcription

1 Molecular diagnostics: when to use (commercial vs in house) Marijke Raymaekers (Belgium) Kate Templeton (UK)

2 Are you mainly using home-brew or commercial assays?

3 In house diagnostics home brew assays Commercial diagnostics kitified assays Commercial diagnostics blackbox assays

4 Are home-brew real-time PCR assays to be trusted as much, less or more than commercial assays? Some considerations

5 CE-IVD, FDA mark: assurance of quality of commercial assay? Check data of kit-insert! Limited validation? Request data or validation from company CE-IVD new concept, stricter regulation? CE-IVD/FDA mark no longer applicable after adaptations (different sample volume, extraction, PCR instrument, )

6 Evaluation of in-house assays (NAT) - Neisseria gonorrhoeae - ccpb target: 18 / 24 false negatives

7 Evaluation of in-house assays (NAT) - Neisseria gonorrhoeae - N. gonorrhoeae ccpb target (absent in strains lacking plasmids)

8 Blind testing of panels can show quality Panels performed by home brew tests and commercial test How many panels? Include all genotypes. Comparison by independent party Role for QC providers like QCMD?

9 Training Reduced for commercial assays? In general necessary: contamination prevention. Need for ongoing education

10 Training Training plan: documents, theoretical, under supervision, test for analyses with subjective interpretation (reference, EQC,..) authorization according to criteria competence declaration or corrective action (additional training), no analysis of patient samples before competency declaration

11 Commercial kit is not available

12 Full control over home-brew assay Not always information on target gene for commercial kits? Limitations to the test (sensitivity, target sequences, ) Black box testing without curves. Commercial assays can also fail. Troubleshooting is more difficult: rely on the good-will of manufacturer (proof default) Detection of seasonal variants? Easy adaptation of home-brew assays Quality control on production by manufacturer? Home brew: entrance control for new lot critical reagents

13 Case commercial kit false negative result for RSV B PCR RSV B positive patient sample with PCR method 1, negative with PCR method 2 during validation of new extraction method Analysis of patient sample in other lab (3rd method): confirmed sample is positive Method 2 does not detect a variant of RSV A complaint was formulated to the company Method 2 was stopped in the lab

14 Problems with Hepatitis C PCR (quantitative assay) Characteristics of standard curve not within specifications Slope: -3.6 m -3.1 Efficiency 90% Case: home brew assay Coefficient of correlation 0.99 r high background fluorescent signal Cq value for inhibition and amplification control high

15 Problems with Hepatitis C PCR Step 1: Troubleshooting: Case: home brew assay Problem with amplification reagents? Change probe, primers, mastermix to new lot: problem remains Problem with reverse transcriptase step no 3 Cq shift after dilution of cdna Change lot of reverse transcriptase: characteristics of standard curve are within specifications after use of new lot superscript Problem is solved!

16 Case: home brew assay Problems with Hepatitis C PCR Step 2: Conclusion: problem with lot of reverse transcriptase: repeat EQC panel repeat patient samples

17 problems with Hepatitis C PCR Step 2: Problem solved? Case: home brew assay after repeat analysis of patient samples on 2 different instruments: difference of 5 Cq values for each patient result Run time lower on instrument 2 Temperature check on instrument: problem with temperature profile of instrument 2

18 Beginner level PCR Inappropriate use of commercial assays No experience in developing home brew assays: expertise takes time and costs money Trust in commercial assays: blind boxes!?

19

20 Commercial assays are more expensive Some laboratories include less controls: risk Reluctance for repeat analysis of samples? Cost of validation commercial vs home brew assays

21 Standardization? Efficient testing Same instrument (extraction, real-time PCR, ) Same procedures for real-time PCR More easily for home-brew assays (some commercial companies have assays that can be analyzed simultaneously)

22 Standardization? Variation between assays

23 Standardization?

24 Standardization? Eleven (19.3%) data sets were generated using conventional in-house assays, 11 (19.3%) data sets using commercial real-time PCR assays, and 35 (61.4%) data sets using in-house real-time PCR assays.

25 Standardization? The results of this first international EQA demonstrate encouraging analytical sensitivity for the detection of HHV-6-DNA in human plasma, although we observed extensive inter laboratory variation of quantitative HHV-6 DNA results. Standardization needs to be improved to allow further elucidation of the clinical significance of HHV-6 loads

26 Standardization? Calibrators and standard curve used?

27 Standardization?

28 Standardization?

29 Both assays need to be validated equally Commercial assays should be better validated and quality controlled than home-brew assays This can also be achieved with home brew assays also, but means investment for laboratory. Does the lab has acces to the amount of patient samples? Rare diseases? Variants?

30 How should validation and implementation of such assays be carried out?

31 Examination procedure Following aspects should be taken into account when implementing a new diagnostic test: Scientific evidence found in peer reviewed articles, guidelines or expert opinions Target patient population (which might influence subsequent test selection criteria) Sample type (ease of collection, transport conditions, minimal volume, ) Required turn-around time Practical consequence for the laboratory Technical test performance should meet clinical needs

32 Verification/Validation in 4 steps 1. Postulation of aims: description, method, purpose 2. criteria: expected clinical and technical performance PLAN of validation: up front 3. results: includes raw data 4. conclusion: fit for purpose or not RESULTS of validation

33 VERIFICATION FDA/CE-IVD Peer review multicentre publications VALIDATION Home brew Adapted FDA/CE-IVD Adapted peer review multicentre publications

34 The greatest concern A false negative critical qpcr resulting from: Faulty extraction Amplification inhibition Poor diagnostic accuracy Suboptimal routine QA/QC

35 PCR controls

36 Group discussion Pathogens commercial / home-brew assays HIV viral load CT/NG HCV viral load HBV viral load Respiratory pathogens (viral, bacterial) CNS (viral/bacterial) Legionella Bacterial gastro enteritis Norovirus

37 Conclusion The reliability of an assay is not dependent on whether it is home brew or not The reliability is dependent on how well the test is implemented and validated

38 Quality in general Quality of patient result is the main concern: impact on patient care KIS-principal (Keep It Simple) Laboratory has to prove quality IS is very broad Need for translation of ISO15189 to routine practice in guidelines

39 ISO15189-Particular requirements for quality and competence: Specifies the quality management system requirements particular to medical laboratories. Includes: provision of advice to users of the laboratory service the collection of patient samples the interpretation of test results acceptable turnaround times how testing is to be provided in a medical emergency the lab's role in the education and training of health care staff Updated in 2012

40 IS is very broad Interpretation is auditor dependent No suggestions concerning the implementation in daily practice Many aspects are topic of debate Need for translation of ISO15189 to routine practice in guidelines

41 National guidelines

42 MDx guidelines Conform with national regulation Quite a few guidelines out there: National: Belgium, The Netherlands, United Kingdom, Germany Professional groups: QCMD, CLSI Peer-reviewed publication, consensus statements No EU wide guidance

43 Current existing guidelines: Reflections and proposals to assure quality in molecular diagnostics. Consensus interpretation on chapter 5 of ISO15189, Acta Clin Belg (1): Raymaekers M, Bakkus M, Boone E, de Rijke B, El Housni H, Descheemaeker P, De Schouwer P, Franke S, Hillen F, Nollet F, Soetens O, Vankeerberghen A; on behalf of MolecularDiagnostics.be working group. ( Health Protection Agency. (2013). Guidance on the Development and Validation of Diagnostic Tests that Depend on Nucleic Acid Amplification and Detection. UK Standards for Microbiology Investigations. P 4 Issue 1.

44 Current existing guidelines: Nukleinsäure-Amplifikationstechniken (NAT), Mikrobiologischinfektiologische Qualitaetsstandards (MiQ, 1/2011, 3. Auflage) Public Health England. (2013). Good Laboratory Practice when Performing Molecular Amplification Assays. UK Standards for Microbiology Investigations. Q 4 Issue 4.4 Quality in the molecular microbiology laboratory, Methods Mol Biol, 2013;943: Wallace PS, MacKay WG. European medical laboratory accreditation. Present situation and steps to harmonisation, Clin Chem Lab Med Jul;50(7): Huisman W.