BEIPH Final Report. QCMD 2009 Hepatitis C virus (HCVRNA09) EQA Programme

Transcription

1 QUALITY CONTROL for MOLECULAR DIAGNOSTICS Block 4, Kelvin Campus, West of Scotland Science Park, Glasgow, G SP Scotland Tel: +44 () Fax: +44 () info@qcmd.org BEIPH Final Report QCMD 9 Hepatitis C virus (HCVRNA9) EQA Programme William G MacKay on behalf of QCMD and its Scientific Advisory Board June 9 Not to be reproduced or quoted without permission of QCMD. Any queries about this report should be addressed to the QCMD Neutral Office. The QCMD programme is organised in collaboration with the European Society for Clinical Virology and the European Society for Clinical Microbiology & Infectious Diseases. Registered in Scotland Reg No: SC9746 Registered Office: 7 Castle Street, Edinburgh EH AH

2 . Programme aims The primary aims of this External Quality Assessment Programme were: ) To assess the proficiency of laboratories in the detection of HCV RNA. ) To assess the proficiency of laboratories in the quantification of HCV RNA.. Programme details Table : Programme details HCVRNA9 Date of panel distribution //9 Number of participants Number of countries Number of respondents (%) Number of datasets submitted Number of qualitative datasets submitted 8 (4%) Number of qualitative and quantitative datasets submitted 5 (76%). Panel composition This EQA panel for the detection of HCV consisted of eight samples containing various concentrations of HCV and one sample negative for HCV. Samples were obtained as a result of a screening programme at the South African blood bank, and through well-characterised clinical sources. Negative plasma was obtained through the Groningen blood bank (The Netherlands) and was also used to dilute the samples according to specifications. Table : Panel composition Sample Sample Sample * Sample conc. Sample content matrix IU/ml status HCV9- HCV Negative Plasma Plasma Negative HCV9- HCV Type Plasma 9 Strong Positive HCV9- HCV Type a Plasma 794 Strong Positive HCV9-4 HCV Type a Plasma 4645 Strong Positive HCV9-5 HCV Type Plasma 947 Strong Positive HCV9-6 HCV Type Basematrix 5 47 Strong Positive HCV9-7 HCV Type Plasma 4656 Strong Positive HCV9-8 HCV Type 5a Plasma 4 Strong Positive HCV9-9 HCV Type Plasma 54 Positive Key to Table Sample: QCMD panel sample codes for the samples distributed to participants. Sample content: viral or microbial content of the panel samples. Sample matrix: material used as a matrix in preparation of the panel samples. Sample conc.: Consensus values calculated from all of the data returned by participants, once outliers had been removed. The values are not technology specific and should not be used by participants for method comparison or as a target for individual laboratory assessment. Sample status: The sample status assigned to each panel sample consisting of 'Strong positive', 'Positive', 'Weak positive' or 'Negative'. Please see Appendix A for more information. * Plasma: human plasma negative for HCV RNA. Basematrix 5: a processed defibrinated human plasma product (SeraCare). QCMD 9 Hepatitis C virus (HCVRNA9) EQA Programme PAGE of 9

3 4. Programme results 4a. Qualitative analysis of the EQA data The number (percentage) of correct qualitative results are presented in Table. Qualitative data were returned by participants as 'positive', 'negative' or 'not determined'. Not determined results were counted as incorrect for all panel samples (positive or negative). QCMD organises datasets according to commercial and in-house technology groups, which are Conventional PCR, Real time PCR, NASBA, SDA, TMA and bdna. Where datasets were reported as other for a technology or kit method this was reviewed by the QCMD Neutral Office and assigned to an appropriate group where possible. Table : Number of correct qualitative results per panel member and technology type Sample Sample Sample conc. content IU/ml d Total Conventional Real time datasets Commercial a Commercial c In-house n= n=9 n=8 n=4 n= n % n % n % n % n % HCV9-5 HCV Type HCV9-7 HCV Type HCV9- HCV Type HCV9-9 HCV Type HCV9-4 HCV Type a HCV9- HCV Type a HCV9-8 HCV Type 5a HCV9-6* HCV Type HCV9- HCV Negative Plasma PCR TMA g Key to Table Sample: QCMD panel sample codes for the samples distributed to participants. Sample content: viral or microbial content of the panel samples. Sample conc.: Consensus values calculated from all of the data returned by participants, once outliers had been removed. The values are not technology specific and should not be used by participants for method comparison or as a target for individual laboratory assessment. Total datasets: number and percentage of datasets reporting the correct qualitative result for each panel sample. A breakdown of the results for all datasets is also provided based on technology type. * The matrix of panel sample HCV9-6 was Basematrix 5. a: Roche Amplicor HCV (Manual) (n=), Roche Amplicor HCV Monitor (n=), Roche COBAS Amplicor HCV Monitor (n=4), Roche COBAS Ampliscreen HCV (Blood Screen) (n=). c: Abbot Molecular RealTime HCV RNA Assay (n=6), QIAGEN (Details not provided) (n=), QIAGEN artus HCV PCR Kit (RG) (n=), Roche COBAS AmpliPrep/COBAS TaqMan HCV Test (n=). d: Details not presented. g: Chiron Procleix Ultrio (n=). QCMD 9 Hepatitis C virus (HCVRNA9) EQA Programme PAGE of 9

4 4b. Qualitative performance scores Table 4: Qualitative performance scores per technology type Sample Sample Status Key to Table 4 Sample: QCMD panel sample codes for the samples distributed to participants. Sample status: the sample status assigned to each panel sample. Please see Appendix A for more information. Total. All technologies: number of datasets awarded each score ( to ). A breakdown of the results for all datasets is also provided based on technology type. These data are presented graphically in Figure. * The matrix of panel sample HCV9-6 was Basematrix 5. a: Roche Amplicor HCV (Manual) (n=), Roche Amplicor HCV Monitor (n=), Roche COBAS Amplicor HCV Monitor (n=4), Roche COBAS Ampliscreen HCV (Blood Screen) (n=). c: Abbot Molecular RealTime HCV RNA Assay (n=6), QIAGEN (Details not provided) (n=), QIAGEN artus HCV PCR Kit (RG) (n=), Roche COBAS AmpliPrep/COBAS TaqMan HCV Test (n=). g: Chiron Procleix Ultrio (n=). Total All technologies n= Conventional Commercial a Real time In-house d HCV9-5 Strong Positive HCV9-7 Strong Positive HCV9- Strong Positive HCV9-9 Positive HCV9-4 Strong Positive HCV9- Strong Positive HCV9-8 Strong Positive HCV9-6* Strong Positive HCV9- Negative PCR Commercial c n=9 n=8 n=4 TMA g n= Figure : Percentage of qualitative performance scores per technology type % a c d g a c d g a c d g a c d g a c d g a c d g a c d g a c d g a c d g HCV9-5 HCV9-7 HCV9- HCV9-9 HCV9-4 HCV9- HCV9-8 HCV9-6 HCV9- Technology group per panel sample a: Conventional commercial PCR, c: Real time commercial PCR, d: Real time in-house PCR, g: TMA. QCMD 9 Hepatitis C virus (HCVRNA9) EQA Programme PAGE 4 of 9

5 4c. Quantitative analysis of the EQA data and performance scores Consensus concentration score The consensus concentration score relates to scoring on the basis of a consensus mean for each positive panel sample. Table 5: Quantitative scores by technology type in comparison to the consensus concentration Sample Consensus Log virus Total All technologies concentration n=5 Mean SD LOD/NR LOD/NR LOD/NR LOD/NR HCV HCV HCV HCV HCV HCV HCV HCV9-6* Key to Table 5 Sample: QCMD panel sample codes for the samples distributed to participants. Consensus Log virus concentration: the mean quantitative value and standard deviation value for each panel sample expressed in log units and calculated once outlying values had been removed. Total. All technologies: number of datasets awarded each score ( to ). LOD/NR refers to datasets where a limit of detection value was reported or no value was reported (these data were excluded from the scoring). SD refers to the Standard Deviation. A breakdown of the results for all datasets is also provided based on technology type. These data are presented graphically in Figure. * The matrix of panel sample HCV9-6 was Basematrix 5. a: Roche Amplicor HCV Monitor (n=), Roche COBAS Amplicor HCV Monitor (n=). d: Details not presented. Conventional Commercial a n=4 PCR Commercial c Real time In-house d n=8 n= c: Abbot Molecular RealTime HCV RNA Assay (n=6), QIAGEN (Details not provided) (n=), QIAGEN artus HCV PCR Kit (RG) (n=), Roche COBAS AmpliPrep/COBAS TaqMan HCV Test (n=). Figure : Percentage of quantitative performance scores in comparison to the consensus concentration % LOD/NR a c d a c d a c d a c d a c d a c d a c d a c d HCV9-5 HCV9-7 HCV9- HCV9-9 HCV9-4 HCV9- HCV9-8 HCV9-6 Technology group per panel sample a: Conventional commercial PCR, c: Real time commercial PCR, d: Real time in-house PCR. QCMD 9 Hepatitis C virus (HCVRNA9) EQA Programme PAGE 5 of 9

6 Technology consensus score The technology consensus score relates to scoring on the basis of a technology group consensus mean per positive panel sample. Table 6: Quantitative scores by technology type in comparison to the technology consensus concentration Sample Tech. Consensus Real time Commercial c Log virus concentration n=8 Mean SD LOD/NR HCV HCV HCV HCV HCV HCV HCV HCV9-6* Key to Table 6 Sample: QCMD panel sample codes for the samples distributed to participants. Tech. Consensus Log virus concentration: the mean quantitative value and standard deviation value for each panel sample expressed in log units and calculated once outlying values had been removed. The number of datasets awarded each score ( to ) is then presented. LOD/NR refers to datasets where a limit of detection value was reported or no value was reported (these data were excluded from the scoring). SD refers to the Standard Deviation. These data are presented graphically in Figure. * The matrix of panel sample HCV9-6 was Basematrix 5. c: Abbot Molecular RealTime HCV RNA Assay (n=6), QIAGEN (Details not provided) (n=), QIAGEN artus HCV PCR Kit (RG) (n=), Roche COBAS AmpliPrep/COBAS TaqMan HCV Test (n=). Figure : Percentage of overall quantitative performance scores in comparison to the technology consensus concentration % LOD/NR c c c c c c c c HCV9-5 HCV9-7 HCV9- HCV9-9 HCV9-4 HCV9- HCV9-8 HCV9-6 Technology group per panel sample a: Conventional commercial PCR, c: Real time commercial PCR, d: Real time in-house PCR, h: bdna. QCMD 9 Hepatitis C virus (HCVRNA9) EQA Programme PAGE 6 of 9

7 Acknowledgements Data analysis and report generation were performed by Silvia Ciferri of the QCMD Neutral Office. QCMD 9. The QCMD EQA programme samples, associated reports and data generated during this programme are intended for External Quality Assessment (EQA) and Proficiency Testing (PT) purposes only. QCMD operates according to a strict Code of Practice which is in line with ISO guide 4- and associated standards. Data reported in QCMD programmes is representative of a laboratory s standard diagnostic testing protocols irrespective of the technology they use. The data provided in the reports are based on technical information provided by the individual laboratories as part of the assessment process, as such it does not constituent a formal technology method comparison. All text and images produced by QCMD are the property of QCMD unless otherwise stated. The reproduction and use of these materials is not permitted without the express written consent of QCMD. The use of the information provided in QCMD reports for commercial purposes is strictly prohibited. QCMD 9 Hepatitis C virus (HCVRNA9) EQA Programme PAGE 7 of 9

8 Appendix A Assigning the sample status QCMD uses a colour-coded scheme for scoring based on the classification of results in relation to expected or consensus results. Each panel sample is assigned a status in the EQA programme. The statuses for panel samples containing the target are 'Strong Positive', 'Positive' and 'Weak positive'. Panel samples negative for the target are assigned a 'Negative' status. The sample status is defined based on performance in the EQA programme and the expertise of the QCMD Scientific Advisory Board. 'Strong Positive': Defined as an EQA panel sample containing amounts of target agent at levels that are considered detectable by most laboratories using currently available methods. This sample produces an unequivocal positive result. 'Positive': Defined as an EQA panel sample containing amounts of target agent at levels considered detectable using the majority of the current methods available within routine clinical laboratories. This sample should produce an acceptable level of positive results within the peer group. 'Weak Positive': Defined as an EQA panel sample containing amounts of target agent at a level that is known to be problematic for current laboratory methods in routine clinical use. 'Negative': Defined as an EQA panel sample with a common matrix to other test samples but containing no target agent and producing an unequivocal negative result. Scoring system for qualitative EQA data The scores awarded for qualitative EQA data were based on the sample status (see Section ). The scoring system is represented in the following table, where is 'highly satisfactory' and is 'highly unsatisfactory'. Colour has been included as an extra visual aid. Scoring system based on the assigned sample status Sample status Participant's result Negative Not determined Positive Strong Positive Positive Weak Positive Negative QCMD 9 Hepatitis C virus (HCVRNA9) EQA Programme PAGE 8 of 9

9 Scoring system for quantitative EQA data In order to compare the participants results within specific technologies or kit methods, where sufficient datasets were reported (5 or more) methods or kits were assigned to a technology group e.g. Real time, bdna or NASBA etc. Where datasets were reported as 'other' for a technology or kit method this was reviewed by the QCMD Neutral Office and assigned to an appropriate group where possible. The HCV negative panel sample (HCV9-) was not included in these analyses. The normal distribution was estimated from the log of the datasets submitted by participants for each panel sample. Scores were assigned based on the distance from the mean value for each panel sample. The scoring system used for quantitative EQA data ranged from (highly satisfactory) to (highly unsatisfactory). A quantitative mean value was calculated for each panel sample using two methods. These were:. Consensus concentration - the mean of the participants' results once outliers had been removed.. Technology consensus concentration - the mean of the participants' results per technology group once outliers had been removed. Scores were awarded based on the distance from the calculated mean value for each panel sample. Zero points was awarded if the quantitative value returned was within one standard deviation from the mean. One point was awarded if the quantitative value was between one and two standard deviations, two points if the value was within two and three standard deviations and three points for quantitative values more than three standard deviations from the mean. ±SD -SD +SD -SD +SD Each coloured division represents one standard deviation (SD) from the mean, so that zero points were awarded for quantitative values that were within one standard deviation and three points for quantitative values that were more than three standard deviations from the mean. QCMD 9 Hepatitis C virus (HCVRNA9) EQA Programme PAGE 9 of 9