BEIPH Final Report. EQA Programme 2011 Hepatitis B Virus (HBVDNA11A) William G Mackay on behalf of QCMD and its Scientific Council July 2011

Transcription

1 BEIPH Final Report EQA Programme 2011 Hepatitis B Virus (HBVDNA11A) William G Mackay on behalf of QCMD and its Scientific Council July 2011 Not to be reproduced or quoted without permission of QCMD. Any queries about this report should be addressed to the QCMD Neutral Office. Unit 5, Technology Terrace, Todd Campus, West of Scotland Science Park, Glasgow, G20 0XA, Scotland Tel: +44 (0) , Fax: +44 (0) , info@qcmd.org, Web:

2 Contents 1. Programme aims Programme details Panel composition Programme results Qualitative performance on the core proficiency samples Qualitative analysis of the EQA data for all panel samples Qualitative performance scores for all panel samples Quantitative performance on results returned in IU/ml... 6 Acknowledgements... 6 Appendix A of 7

3 1. Programme aims The primary aim of this External Quality Assessment Programme was to assess the proficiency of laboratories in the detection and quantitation of Hepatitis B virus (HBV) DNA. Participants are encouraged to read the QCMD Participants' Manual, which can be downloaded from the QCMD website. Any queries about this report should be addressed to the QCMD Neutral Office (neutraloffice@qcmd.org). 2. Programme details Table 1: Programme details HBVDNA11A Date of panel distribution 04/04/2011 Number of respondents 19 (86%) Number of participants 22 Number of datasets submitted 20 Number of countries 1 Number of qualitative datasets submitted 11 Number of quantitative datasets submitted 20 Quantitative datasets returned in IU/ml Number of datasets submitted 18 (90.0%) Number of quantitative datasets submitted 8 (44.4%) Number of qualitative and quantitative datasets submitted 10 (55.6%) Quantitative datasets returned in Copies/ml Number of datasets submitted 2 (10.0%) Number of quantitative datasets submitted 1 (50.0%) Number of qualitative and quantitative datasets submitted 1 (50.0%) Three participants did not return results. One withdrew officially, citing 'assay under development'. 3. Panel composition Table 2: Panel composition Sample Sample content Sample * matrix Sample conc. Log 10 IU/ml Log 10 Copies/ml Sample status Sample type HBV11-04 HBV Type A Plasma Frequently detected Core HBV11-05 HBV Type A Plasma Frequently detected Core HBV11-08 HBV Type A Plasma Frequently detected Core HBV11-02 HBV Type A Plasma Frequently detected HBV11-07 HBV Type D Plasma Frequently detected Core HBV11-06 HBV Type D Plasma Frequently detected Core HBV11-03 HBV Type D Plasma Frequently detected HBV11-01 HBV Negative Plasma Negative Core Key to Table 2 Sample: QCMD panel sample codes for the samples distributed to participants. Sample content: viral content of the panel samples. Sample matrix: material used as a matrix in preparation of the panel samples. Sample conc.: consensus values calculated from all of the data returned by participants in the full EQA programme, once outliers had been removed. The values are not technology specific and should not be used by participants for method comparison or as a target for individual laboratory assessment. Sample status: the sample status assigned to each panel sample. Please see Appendix A for more information. Sample type: panel samples classified as core proficiency samples. *Human plasma. 3 of 7

4 Percentage of datasets 2011 Hepatitis B Virus 4. Programme results 4.1. Qualitative performance on the core proficiency samples Figure 1 shows the performance of BEIPH participants on the core proficiency samples. In this round of the EQA 90.9% of datasets returned to QCMD by BEIPH participants reported all core proficiency samples correctly. Figure 1: Performance on the core proficiency samples for BEIPH participants 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% 90.9% 9.1% 0.0% 0.0% 0.0% 0.0% 0.0% 6/6 5/6 4/6 3/6 2/6 1/6 0/6 Number of core samples correct 4.2. Qualitative analysis of the EQA data for all panel samples The number (percentage) of correct qualitative results are presented in Table 3. Qualitative data were returned by participants as 'positive', 'negative' or 'not determined'. Not determined results were counted as incorrect for all panel samples (positive or negative). QCMD organises datasets according to commercial and in-house technology groups, which are Conventional PCR, Real time PCR, NASBA, SDA, TMA and bdna. Where datasets were reported as other for a technology or kit method this was reviewed by the QCMD Neutral Office and assigned to an appropriate group where possible. Table 3: Number of correct qualitative results per panel member and technology type Sample Sample Sample conc. content Log 10 IU/ml Total datasets Commercial c In-house d n=11 n=9 n=2 n % n % n % HBV11-04 HBV Type A HBV11-05 HBV Type A HBV11-08 HBV Type A HBV11-02 HBV Type A HBV11-07 HBV Type D HBV11-06 HBV Type D HBV11-03 HBV Type D HBV11-01 HBV Negative PCR Real time Key to Table 3 Sample: QCMD panel sample codes for the samples distributed to participants. Sample content: viral content of the panel samples. Sample conc.: consensus values calculated from all of the data returned by participants in the full EQA programme in IU/ml, once outliers had been removed. The values are not technology specific and should not be used by participants for method comparison or as a target for individual laboratory assessment. Total datasets: number and percentage of datasets reporting the correct qualitative result for each panel sample. A breakdown of the results for all datasets is also provided based on technology type. c: Abbott RealTime HBV test (n=3), QIAGEN artus HBV PCR Kit (RG) (n=2), Roche COBAS AmpliPrep/COBAS TaqMan HBV Test (n=2) & Roche COBAS TaqMan HBV/ High Pure System (n=2). d: Details not presented. Additional information on false positivity One not determined result was reported on panel sample HBV of 7

5 4.3. Qualitative performance scores for all panel samples Table 4: Qualitative performance scores per technology type Sample Sample status Total PCR All technologies Real time Commercial c In-house d n=11 n=9 n= HBV11-04 Frequently detected HBV11-05 Frequently detected HBV11-08 Frequently detected HBV11-02 Frequently detected HBV11-07 Frequently detected HBV11-06 Frequently detected HBV11-03 Frequently detected HBV11-01 Negative Key to Table 4 Sample: QCMD panel sample codes for the samples distributed to participants. Sample status: the sample status assigned to each panel sample. Please see Appendix A for more information. Total. All technologies: number of datasets awarded each score (0 to 3). A breakdown of the results for all datasets is also provided based on technology type. These data are presented graphically in Figure 2. c: Abbott RealTime HBV test (n=3), QIAGEN artus HBV PCR Kit (RG) (n=2), Roche COBAS AmpliPrep/COBAS TaqMan HBV Test (n=2) & Roche COBAS TaqMan HBV/ High Pure System (n=2). d: Details not presented. Figure 2: Percentage of qualitative performance scores per technology type % c d c d c d c d c d c d c d c d HBV11-04 HBV11-05 HBV11-08 HBV11-02 HBV11-07 HBV11-06 HBV11-03 HBV11-01 Technology group per panel sample c: Real time commercial PCR, d: Real time in-house PCR. 5 of 7

6 4.4. Quantitative performance on results returned in IU/ml Of the 20 quantitative datasets returned to QCMD 18 (90%) were returned in International Units (IU)/ml. Quantitative analysis of the EQA data for all panel samples IU/ml Table 5: Consensus concentration values for BEIPH participants Consensus log 10 virus concentration All participants n=202 BEIPH participants n=18 Sample Mean SD Mean SD HBV HBV HBV HBV HBV HBV HBV Key to Table 5 Sample: QCMD panel sample codes for the samples distributed to participants. Consensus Log10 virus concentration: the mean quantitative value and standard deviation value for each panel sample expressed in log10 units and calculated once outlying values had been removed. SD refers to the Standard Deviation. All participants: values calculated from IU/ml results submitted by all participants in the QCMD 2011 Hepatitis B Virus DNA EQA Programme A. BEIPH participants: values calculated from IU/ml results submitted by BEIPH participants only. Acknowledgements Data analysis and report generation were performed by the QCMD Neutral Office. QCMD The QCMD EQA programme samples, associated reports and data generated during this programme are intended for External Quality Assessment (EQA) and Proficiency Testing (PT) purposes only. QCMD operates according to a strict Code of Practice which is in line with ISO/IEC and associated standards. Data reported in QCMD programmes is representative of a laboratory s standard diagnostic testing protocols irrespective of the technology they use. The data provided in the reports are based on technical information provided by the individual laboratories as part of the assessment process, as such it does not constitute a formal technology method comparison. All text and images produced by QCMD are the property of QCMD unless otherwise stated. The reproduction and use of these materials is not permitted without the express written consent of QCMD. The use of the information provided in QCMD reports for commercial purposes is strictly prohibited. 6 of 7

7 Appendix A Assigning the sample status Sample status is assigned by peer-group consensus, based on the qualitative results returned by all participants in the full EQA programme. It is not a measure of the 'strength' of a positive sample nor is it technology-dependent, and is used solely for the scoring of the EQA data. The rationale for the sample status is: Frequently detected: More than 95% of datasets recorded the correct positive result. Detected: Between 65% and 95% of datasets recorded the correct positive result. Infrequently detected: Less than 65% of datasets recorded the correct positive result. Negative: A panel sample that does not contain the target and produces an unequivocal negative result. Scoring system for qualitative EQA data The scores awarded for qualitative EQA data were based on the sample status. The scoring system is represented in the following table, where 0 is 'highly satisfactory' and 3 is 'highly unsatisfactory'. Colour has been included as an extra visual aid. Scoring system based on the assigned sample status Sample status Participant's result Negative Not determined Positive Frequently detected Detected Infrequently detected Negative Calculation of the consensus and technology concentrations In order to compare participants results within specific technologies or kit methods, where sufficient datasets were reported (5 or more) methods or kits were assigned to a technology group e.g. Real time PCR, bdna or NASBA etc. Where datasets were reported as 'other' for a technology or kit method this was reviewed by the QCMD Neutral Office and assigned to an appropriate group where possible. The negative panel samples were not included in these analyses. The parameters of the normal distribution were estimated from the log of the positive datasets submitted by participants for each panel sample. Scores were assigned based on the distance from the mean value for each panel sample. The scoring system used for quantitative EQA data ranged from 0 (highly satisfactory) to 3 (highly unsatisfactory). An assigned value was calculated for each panel sample using two methods. These were: 1. Consensus concentration - the mean of the participants' results once outliers had been removed. 2. Technology consensus concentration - the mean of the participants' results per technology group once outliers had been removed. The standard deviation was calculated as the square root of the mean square for error from the ANOVA table where the response was the log concentration of participants results with outliers removed. The factor was technology group. Outliers were defined as values with a standardised residual with modulus greater than three. Although outliers were removed for the calculation of the assigned values they were included in the data analysis. Further information about the QCMD method of analysing EQA data can be found in the following peer-reviewed publication: Staines HJ, Garcia-Fernandez L, Pogothata R, Wallace PS, MacKay WG, van Loon AM. Monitoring performance of nucleic acid-based diagnostic measurement system users by EQA. Accred Qual Assur. 2009; 14: of 7