Standardization of Laboratory Tests - How to do it

Transcription

1 Standardization of Laboratory Tests - How to do it Greg Miller, PhD Virginia Commonwealth University Medical Center Richmond, Virginia, USA greg.miller@vcuhealth.org

2 Outline What is standardization Why is it important How is it accomplished How are challenges being addressed

3 What is standardization Equivalent results, within clinically meaningful limits, among different measurement procedures for the same laboratory test

4 Terminology Standardization: achieving equivalent results by having calibration traceable to a JCTLM listed reference system component Harmonization: achieving equivalent results among different measurement procedures Usually implies there is no JCTLM listed reference measurement procedure or certified reference material

5 Why is it important Patients will get the correct treatment Many medical decisions are informed by laboratory results Many clinical guidelines use a fixed laboratory test value for treatment decisions

6 How to achieve equivalent results 1. Calibration of all measurement procedures is traceable to a common reference system ISO 17511:2003 (under revision) 2. All measurement procedures measure the same quantity (the same molecular form) Analytical selectivity for the measurand

7 ISO Standards 17511:2003, Calibration Traceability 18153:2003, Traceability for Enzymes 15193:2009, Reference Measurement Procedures 15194:2009, Certified Reference Materials 15195:2003, Reference Measurement Laboratories

8 Joint Committee for Traceability in Laboratory Medicine Lists reference materials, reference measurement procedures and reference laboratories that conform to the ISO Standards

9 TRACEABILITY Panel of Patient Samples Primary Reference Material (pure substance) Pure Substance Calibrator Secondary Reference Material (matrix) Manufacturer s Product Calibrator Procedures for identity and mass balance Reference Measurement Procedure (e.g. Gravimetry) Reference Measurement Procedure (e.g. IDMS) Manufacturer s Internal Procedures Medical Laboratory Procedure Patient s Sample Patient s Result

10 Measurands for which reference procedures exist or can be developed Everything else we measure in the clinical laboratory

11 What happens when there is no reference measurement procedure

12 TRACEABILITY Traceability is established to a reference material Secondary Reference Material (matrix) Manufacturer s Product Calibrator Patient s Sample Manufacturer s Internal Procedures Medical Laboratory Procedure Patient s Result

13 TRACEABILITY Value assignment Commutability Secondary Reference Material (matrix) Manufacturer s Product Calibrator Patient s Sample Manufacturer s Internal Procedures Medical Laboratory Procedure Patient s Result

14 What happens when there is both: no reference measurement procedure no reference material

15 TRACEABILITY TRACEABILITY TRACEABILITY TRACEABILITY Traceability is established to to a material selected for a measurement procedure No coordination among producers (IVD or LDT) Manufacturer s Working Calibrator (master lot) Manufacturer s Product Calibrator Patient s Sample Manufacturer s Internal Procedure Medical Laboratory Procedure?????????? Patient s Result

16 The commutability challenge

17 TRACEAB ILITY Primary Reference Material (pure substance) Pure Substance Calibrator Secondary Reference Material (matrix) Manufacturer s Product Calibrator Patient s Sample Procedures for identity and mass balance Reference Measurement Procedure (e.g. Gravimetry) A non-commutable calibrator breaks the traceability chain Key step for commutability Reference Measurement Procedure (e.g. IDMS) Manufacturer s Internal Procedures Medical Laboratory Procedure Patient s Result

18 TRACEAB ILITY Primary Reference Material (pure substance) Pure Substance Calibrator Secondary Reference Material (matrix) Key step for commutability Manufacturer s Product Calibrator Patient s Sample Procedures for identity and mass balance Even though manufacturers show traceability, the Reference process fails to provide equivalent Measurement results for patient Procedure (e.g. Gravimetry) samples among different measurement procedures Reference Measurement Procedure (e.g. IDMS) Manufacturer s Internal Procedures Medical Laboratory Procedure Patient s Result

19 Commutability (Commutable) Property of a reference material demonstrated by the closeness of agreement between the relation among results for a reference material obtained from two measurement procedures and the relation among results for clinical samples from the same two measurement procedures (Rephrased from VIM 3: 2008)

20 Measurement Procedure 2 Commutable: same relationship for clinical samples and reference materials Clinical Samples Reference Materials Measurement Procedure 1

21 Measurement Procedure 2 Non-commutable: different relationship for clinical samples and reference materials Clinical Samples Reference Materials Measurement Procedure 1

22 Measurement Procedure 2 Calibration with non-commutable materials Clinical Samples RM as Calibrator Measurement Procedure 1 causes patient sample results to be different

23 Guidelines are available from CLSI: EP30-A Characterization and qualification of commutable reference materials for laboratory medicine (2010 as C53-A) EP14-A3 Evaluation of commutability of processed samples (2014)

24 Measurement Procedure 2 Result 50 Deming Regression Line 40 95% Prediction Interval Commutable 30 Commutable 20 Non-commutable 10 0 Native Clinical Samples Reference Materials Measurement Procedure 1 Result Adapted from CLSI EP30-A (used with permission)

25 Measurement Procedure 2 Result 50 Deming Regression Line 40 95% Prediction Interval Native Clinical Samples Reference Materials Measurement Procedure 1 Result Criteria for commutability near decision thresholds not resolved Adapted from CLSI EP30-A (used with permission)

26 Measurement Procedure 2 Result % Prediction Interval Deming Regression Line Native Clinical Samples Reference Materials Measurement Procedure 1 Result More scatter in the relationship should not make the RMs commutable Modified from CLSI EP30-A (used with permission)

27 IFCC Working Group on Commutability (established March 2013) Operating procedures for the formal assessment of commutability Criteria for commutability taking into account the intended use of a reference material

28 Criteria for commutability 1. Criteria based on statistical distribution of results for patient samples do not relate to the intended use of a RM o Criteria change among measurement procedures with different performance characteristics Method D Method B Method C Method A

29 Criteria for commutability 2. Recommend fixed criteria based on the intended use of the reference material Method D Method B Method C Method A

30 Intended use refers to the allowable total error (TE a ) for a medical decision Fraction of the uncertainty required for a RM s use in a calibration traceability hierarchy Fraction of the uncertainty required for assessment of performance using EQA

31 Statistical models 1. Assess the closeness of agreement for the difference in bias for RM compared to clinical samples between measurement procedures 2. Assess harmonization effectiveness of a RM used for calibration traceability by a group of measurement procedures

32 Difference in bias model 1. Estimate the bias between 2 measurement procedures for the patient samples and for the reference material(s) 2. Estimate the random error components including sample specific effects 3. Calculate the difference in bias for reference material(s) vs. patient samples 4. Estimate the uncertainty of the difference in bias 5. Commutable if the difference in bias plus uncertainty are within a criterion that is suitable for the intended use of the reference material

33 Difference in bias vs. fixed criteria Bias ( y- x) CS RM Bias for CS Crit eria Crit eria Mean Concent ration ( x+ y) / 2

34 Difference in bias vs. fixed criteria Bias ( y- x) Mean Concent ration ( x+ y) / 2 Commutable: bias plus uncertainty is within criteria CS RM Bias for CS Crit eria Crit eria

35 Difference in bias vs. fixed criteria Non-commutable: bias plus uncertainty exceeds criteria Bias ( y- x) CS RM Bias for CS Crit eria Crit eria Mean Concent ration ( x+ y) / 2

36 Difference in bias vs. fixed criteria Indeterminate: bias plus uncertainty overlaps criteria Bias ( y- x) CS RM Bias for CS Crit eria Crit eria Mean Concent ration ( x+ y) / 2

37 Harmonization effectiveness model 1. Estimate the inter-measurement procedure CV for each clinical sample s results 2. Calculate an overall pooled inter-measurement procedure CV and its uncertainty for all clinical samples 3. Compare the pooled CV plus uncertainty to a fixed criterion that is suitable for the intended use of the reference material 4. Use the RM for calibration traceability and repeat steps 1-2-3

38 Harmonization effectiveness Current state Inter-Measurement Procedure CV (%) 20% 10% 5% Criterion 0%

39 Harmonization effectiveness Current state Inter-Measurement Procedure CV (%) 20% RM 1 as calibrator Not Commutable 10% 5% Criterion 0%

40 Harmonization effectiveness Current state Inter-Measurement Procedure CV (%) 20% RM 1 as calibrator Not Commutable 10% RM 2 as calibrator Indeterminate 5% Criterion 0%

41 Harmonization effectiveness Current state Inter-Measurement Procedure CV (%) 20% RM 1 as calibrator Not Commutable 10% RM 2 as calibrator RM 3 as calibrator Indeterminate Commutable 5% Criterion 0%

42 Other commutability topics 1. Qualify measurement procedures for inclusion 2. Individual samples vs. pools 3. Replacement of a RM with a new lot or batch 4. Correction to the assigned value of a noncommutable RM

43 TRACEABILITY TRACEABILITY TRACEABILITY TRACEABILITY The harmonization challenge Manufacturer s Working Calibrator (master lot) Manufacturer s Product Calibrator Patient s Sample Manufacturer s Internal Procedure Medical Laboratory Procedure?????????? Patient s Result

44 ISO TC 212 WG2 Revision of the traceability standard is expected to include traceability to a harmonization protocol as one type of calibration traceability hierarchy

45 A new standard is in development to support JCTLM listing of a harmonization protocol. ISO 20089: Requirements for international harmonization protocols intended to establish metrological traceability of values assigned to product (end user) calibrators and patient samples

46 TRACEABILITY Harmoniza on Reference Material Assign value Calibrate Harmoniza on Protocol Manufacturer s Working Calibrator (master lot) Manufacturer s Product Calibrator Assign value Calibrate Assign value Manufacturer s Selected Procedure Manufacturer s Standing Procedure Pa ent s Sample Calibrate Assign value Medical Laboratory Procedure Pa ent s Result

47 Critical components for harmonization 1. Calibration of all measurement procedures is traceable to a common reference protocol 2. All measurement procedures measure the same quantity (the same molecular form) 3. Traceability can be sustained over time and location

48 Example 1: harmonization protocol Clinical samples Process for value assignment Reserve set of clinical samples for sustainability End user calibrator End user calibrator End End user user calibrator product calibrator Equivalent results for clinical samples Assign the value of each end-user product calibrator to recover the values for clinical samples End user End user calibrator calibrator End Medical user lab measurement calibrator procedure Supported by producer (mfr or LDT) master lot(s) with appropriate target values EQA for surveillance

49 Example 2: harmonization protocol Clinical samples Process for value assignment Specification to recreate the clinical samples End user calibrator End user calibrator End End user user calibrator product calibrator Equivalent results for clinical samples Assign the value of each end-user product calibrator to recover the values for clinical samples End user End user calibrator calibrator End Clinical user lab measurement calibrator procedure Supported by producer (mfr or LDT) master lot(s) with appropriate target values EQA for surveillance

50 The prioritization and coordination challenge

51 International Forum organized by AACC in October, participants from 12 countries Representing 62 organizations & manufacturers

52 The Roadmap Develop an infrastructure to coordinate harmonization activities world wide: 1. Prioritize measurands by medical importance 2. Coordinate the work of different organizations 3. Develop technical processes to achieve harmonization when there is no reference measurement procedure or reference material 4. Promote surveillance of the success of harmonization

53

54 International Consortium for Harmonization of Clinical Laboratory Results Strategic Partners Group Council Organizational Member Harmonization Oversight Group Harmonization Implementation Groups Special Working Groups

55 ICHCLR: Council members

56

57

58

59 Perfect is the Enemy of Good