Quality Control in Flow Dr David Westerman Head of Haematopathology Peter MacCallum Cancer Centre
Aims Quality Assurance Quality Control Literature In house competencies
SHOT DATA 1996-2009 Ref: SHOT Annual Report 2009 Summary
Quality Assurance ls the planned and systematic activities implemented within the quality system that can be demonstrated to provide confidence that a product or service will fulfill requirements for quality Ultimately the customer defines quality but Accuracy and consistency pivotal
Components of Quality Assurance Quality Control Policies/procedures/records Proficiency testing (External Quality Assessment) Staff training/competency Inspections/Audits Quality Improvement Preanalytical, analytical & post analytical components
Policies and Procedures Policies what to do Processes how it happens Procedures how to do it - (SOPs) Records
Quality Control A set of procedures performed by the laboratory staff for the continuous and immediate monitoring of laboratory work in order to decide whether the results are reliable enough to be released. 1. Operational Techniques/activities in sync with ISO standards 2. Internal QC
Internal: Flow Cytometry QC Instrument QC Antibody QC Reagent QC Procedural/method control/ commercial controls Specimen integrity verifications Patient (internal) QC
QC cont. Daily setup & performance measurements -laser power, alignment, voltages, detector function, optical filters & background noise - beads alignment, reference, comp. - Levy Jennings plots
Assess robust CV and electonic noise Euroflow standardization of flow cytometer instrument settings Leuk 2012; 26: 1986-2010
Euroflow standardization of flow cytometer instrument settings Leuk 2012; 26: 1986-2010
Euroflow standardization of flow cytometer instrument settings Leuk 2012; 26: 1986-2010
Longitudinal Display Biological Control (Levy Jennings plot) CD3 CD3/CD4 CD3/CD8
Longitudinal PMTV monitoring
Longitudinal MFI monitoring 42000 40000 38000 36000 34000 32000 30000 28000 26000 24000 22000 Target Range (Lot-31519) - PE Bright Beads 55000 50000 45000 40000 35000 30000 Target Range (Lot--31519) - PerCP-Cy5.5 Bright Beads PerCP-Cy5.5 Record Date PE Bright Count Record Date 21000 19000 17000 15000 13000 11000 9000 Target Range (Lot-31519) - V450 Bright Beads 48000 43000 38000 33000 28000 Target Range (Lot-31519) - APC Bright Beads Record Date V450 Bright Count Record Date APC Bright Count Target threshold established by Calculating Mean +/- 0.15
QC cont. Reagent QC -titering, lot to lot, - determine equivalence &stability - cocktailing Weekly tasks - optical alignment, linearity Less Frequent tasks Compare instruments - biannually
Post major instrument service Re-establishing Target Ranges Confirm Cytometer Performance passes Acquire 5 data set throughout the day, across 5 consecutive days. 3SD or 15% +/-Median Re-standardisation and re-compensation if significant change in performance
Limits Set Criterion - acceptability/ tolerance/ out of range values Detection methods Trends important Consistency checks at an individual ptlevel lymphosum, normal populations
Standardisation btninstruments Optimise panel settings on Primary Instrument Run beads to acquire MFI values on Primary with optimise PMTV settings Run same bead lot on Secondary Instrument Adjust PMTV to obtain same MFI values from Primary Instrument Run Normal and Abnormal samples to verify acceptable MFI and Clinical standardisation. 6 monthly Equivalence Check
Target determination The bright peak of the CS&T beads becomes target value Median fluorescent intensity (MFI) of each fluorochrome Figure 2: Target determination of the B2 tube using CS&T beads to obtain the bright MFI of each channel.
Application of target values Figure 3: Application of target values to the B2 tube of the cell therapies instrument using CS&T beads PMT adjustment to meet target MFI values.
Results Figure 4: Comparison between pathology FACSCantoand Cell Therapies FACSCantoof CD4 positive lymphocytes post optimisation and compensation. Path MFI CD4+ = 1.70 CBCT MFI CD4+ = 1.67 MFI log difference 0.03
Pre-analytical Anticoagulant Transportation Time to performing test - validation
Analytical variables Instrument setup, startup & maintenance Daily, weekly, monthly, servicing, etc Storage Abs (dark; 2-8 degrees) Expiry, dates Ab titering Lysing agents Lot to lot testing Validation of cocktails, duration
Stain lyse wash Euroflow standardization of flow cytometer instrument settings Leuk 2012; 26: 1986-2010
Records Quality Control Non-conformances Corrective Action Equipment Records Training/Competency Validation Reports and records Proficiency testing If you didn t document, it didn t happen
NPAAC Supervision Document 2007 -FINAL Standard 1 Staffing and supervision S1.1 There shall be sufficient professional and support staff with adequate training and experience to supervise and conduct the work of the laboratory. S1.2 The designated person in charge, under whose direction and control the accredited pathology laboratory operates must: determine the range of tests provided, their methods and procedures approve and be responsible for operational practices and staffing of the laboratory (including staff training) ensure regular review of the laboratory s quality systems, proficiency testing data, laboratory reports and discussion of all aspects of the laboratory s performance with the scientific staff ensure appropriate consultation on medical
Staff Training & Competencies Instrumentation understanding -courses Know your Panels Understand your reagents (& their achilles) & controls Understand acquisition methods & data files Training data analysis & interpretation Competency testing, documentation Continuing education -teamwork Open transparent non-conformance reporting
Ongoing Training and Competency Assessment Internal Quality Assurance Program Run Bi-annually Commercial control blind in-house ficolled prepared sample (frozen) All relevant staff must participate Assessment Setup (process control) and Gating (software analysis ) component Compliance to SOP Interpretation, grading and quantification Diagnosis
PMac In house CLL MRD testing 14 LMD files 6 operators Varying levels MRD -5 btn0.1-1%; rest < 0.1% Total WBC s CV 6% % MRD CV 35% overall 0.1-1% CV 13% 1 new operator
Requirements for Medical Path Services S7.2 The Laboratory must be enrolled, participate and perform to an acceptable standard in external proficiency testing programs that cover all test methods performed where such programs are available. C7.2(i) Where External Quality Assessment programs do not exist for a test method, the validity of the test results must be demonstrated by methods such as inter-laboratory comparisons or the analysis of reference material. C7.2(ii) All staff performing Medical Pathology Services must participate in External Quality Assessment programs in accordance with Laboratory policy and according to their responsibilities. The policy must include frequency of participation.
External Quality Assurance RCPA QA -oncology - immunology - CD34 testing -PNH British NEQAS - MRD testing Other -CAP
External Quality Assessment Large numbers of participants robust data generated to: Facilitates direct inter-site comparison Facilitates performance assessment Facilitates detailed methodological analysis Required for regulation/ certification Educational & can influence practice
NIH initiative for COG B-ALL harmonisation Hub spoke : 1-7 6BD: 2BC Standardised setup and analysis Varying analysis software COG panel, 6 colour Discordance defined 0.5 log difference Wet and dry lab components, 3 phases
Keeney M, Wood BL et al. Cytompart B 2017
Keeney M, Wood BL et al. Cytom part B 2017 cont. Phase 2 with 3 rounds LMD -26% discordant rate cf NEQAS B-ALL 10-15% discordance rate Phase 3, education 9% discordance; 4/7 labs perfect concordance Learnings: Shipping/ sample artefacts-unfamiliarity paed inexperience Haematogones- differentiation Inadequate boolean gating for differentiation pops Phenotypic shift Education imp.
Flanders et al. Blood 2013 Heterogeneity in Myeloma MRD testing Survey 30 US institutions, 26 responded No. events acquired 100,000-4 x 10 6 6/11 btn 100-500,000 no. plasma cells for positivity 20-50 Definition abnormal varied Sensitivity 0.0005%-0.02%
How can we achieve Standardisation? Standardisation underpins the accuracy & requires - Teaching and Training - Good Laboratory Practice - Good Internal Quality Control - External Quality Control - Appropriate Instrument Validation - National/International Guidelines - International Reference Preparation
Summary Quality Assurance (QA) controls pre-analytic, analytic and post analytic activities Quality Control (QC) monitors and maintains assay specifications (precision and accuracy) over time. Customer expectations for quality through continuous quality improvement Teaching, education & research underpin improved practice
References: Flanders A, Stetler-Stevenson M, Landgren O. Minimal residual disease testing in multiple myeloma by flow cytometry: major heterogeneity. Blood. 2013;122:1088-1089. Wood B, Jevremovic D, Bene MC, et al. Validation of cell-based fluorescence assays: practice guidelines from the ICSH and ICCS, part V: assay performance criteria. Cytometry B Clin Cytom. 2013;84:315-323. Landgren O, GormleyN, Turley D, et al. Flow cytometry detection of minimal residual disease in multiple myeloma: lessons learned at FDA-NCI roundtable symposium. Am J Hematol. 2014;89:1159-1160.
Uniform procedures SOP s procedures transport, anticoagulant, processing/ time, reagent validation & evaluation Titered antibodies Precocktailed(ideally) Fixative containing lysing agents (slide) Instrument optimisation, alignment and QA uniform configuration, track PMT voltages, Reporting integration with morphology Training/ competencies/ record keeping Panel design; backbones in panels
Quality Measures: Optical alignment eg. Microsphere beads; fluidics, laser, & PMT voltages Compensation eg. Spectral overlap Carryover Assay Validation Accuracy dilution testing x 5 each dilution; - CV <30% at LLOD Precision & Reproducibility triplicate testing, aiming CV<10% Analytical sensitivity Analytic specificity: = TN/TN+ FP (TN = true negative; FP = false positive) Stability, viability External & Internal Quality Assessment Proficiency programs