Equivalency Testing: The roles of assay precision, truth, bias, tolerance, risk and control strategy in setting equivalence margins

Transcription

1 Equivalency Testing: The roles of assay precision, truth, bias, tolerance, risk and control strategy in setting equivalence margins Kim E. Zerba, Sutan Wu 1 and Gary McGeorge Bristol-Myers Squibb, New Brunswick NJ 1 inventiv Health Clinical, Lake Mary FL IFPAC 23 January

2 Outline Control strategy: Assessing switching from a reference to alternative potency assay method during pharmaceutical manufacturing What s the right question? Setting equivalence margins Method precision Tolerance Interval Bias Patient Risk 2

3 Control Strategy: Switching from Reference to Alternative Potency Method What s the right question? Are they different? Paired t-test Are they equivalent? Schuirmann s two one-sided tests (TOST) or 3

4 Control Strategy: Switching from Reference to Alternative Potency Method What s the right question? We do not expect there to be no differences between the reference and alternative methods Small, statistically significant differences from 0 can be detected with a paired t-test which may not be meaningful in the context of risk to patients Schuirmann s TOST assesses method equivalence using a threshold that can be predefined based on method precision and patient risk A meaningful average method difference range representing an equivalence margin ( + X%) is preferred over the traditional zero-difference approach when assessing the suitability of switching from the reference method to the alternative method 4

5 Equivalency Testing Setting Equivalence Margins Critical and difficult part of control strategy in pharmaceutical manufacturing when switching from reference method to new potency assay method or technology Must take into account Precision of each assay Potential for bias Potential risk to patients 5

6 Control Strategy: Switching from Reference to Alternative Potency Method Each method must be accurate on average in estimating potency Each method has it s own precision Precision = reproducibility + repeatability How does precision impact the potential for a potency difference between a reference method and true but unknown batch potency and an alternative method? Answer requires knowledge about relationship of precision to how the methods are applied in actual manufacturing 6

7 Role of Method Precision in Differences from True Potency: A Simplistic Example Reference Method with All Tablets Identical Average Difference Batch 1 Tablet 1 True Potency All tablets All Batches True Potency 0 Batch Average Difference from True Potency Batch 2 Tablet 1 Reference method differs on average from true potency for each batch Precision-related batch-specific factors Reference method accurate in estimating potency on average among batches Same principles apply to alternative method with different precision-related batch-specific factors 7

8 Role of Method Precision in Differences between Methods: A Simplistic Example- One Batch with All Tablets Identical Reference Method Tablet 1 True Potency All tablets Batch average difference Alternate Method Tablet 1 Alternative Method differs from Reference Method for each Batch Precision-related Method- and Batchspecific factors True Potency All tablets Batch average difference Alt - Ref difference batch average 8

9 Reference Alternative Method Simulations: 10,000 Batches with 1:1 Tablet Potency Measurements All Tablets Identical Percent Mean = Reference - Alternative Method Batch Mean Difference Each method accurate in estimating true potency on average among batches Average overall batch mean method difference near zero and no systematic bias expected Individual batches can have non-zero mean method differences due to precision of each method 9

10 Setting Equivalence Margins: The Role of the Tolerance Interval Mean = A tolerance interval can be used to set an initial equivalence margin Percent Example: 95/95 Tolerance Interval 95% Confidence that 95% of batch mean differences between alternative and reference method are contained in this interval Reference - Alternative Method Batch Mean Difference Represents the minimum equivalence margin that can be achieved based on the precision of each method 10

11 Potential Bias The Role of the Tolerance Interval Percent Mean = /95 TI Reference - Alternative Method Batch Mean Difference When observed average method difference for a batch is outside the tolerance interval, suggests the potential for chance or alternative method bias When observed average method difference for a batch is inside the tolerance interval, any bias assessment needs to be examined in the context of multiple batches where average method difference is expected to be near zero for many batches if there is no bias 11

12 Using initial equivalence margin from precision-related tolerance interval: Potential for patient risk due to errors in batch release Probability of Batch Acceptance (%) REG 2 REG 2 REG 1 REG 1 Internal 100 True Potency Ref Alt Neither method without risk Risk highest at internal limits Risk drops off rapidly with distance from internal limits Is risk for each method at regulatory limits meaningful for patients? Is risk difference between methods at regulatory limits meaningful for patients? * Probability curves in figure based on method-specific precision and assumption of normally distributed potency estimates 12

13 Summary Setting equivalence margins to assess switching from a reference to new potency assay method or technology during pharmaceutical manufacturing should include consideration of Precision and how methods are applied in practice Tolerance Interval to set an initial minimal margin that can be achieved based on precision Potential for bias Potential for meaningful risk to patients due errors in batch release 13