Phase Appropriate Validation Design for Potency Assays

Transcription

1 Phase Appropriate Validation Design for Potency Assays from IND Enabling Studies through Method Validation for Licensure Ruojia Li Bristol-Myers Squibb CASSS CMC Strategy Forum January 29, 2018

2 Outline General overview of potency assay validation strategy Determination of validation acceptance criteria and sample size Case studies 2

3 Analytical Method Lifecycle Method validation study Confirm suitable method performance Method design, development and understanding Method performance verification Method selection & optimization Define method performance target Develop method with desirable performance Routine monitoring of the method performance 3

4 Potency Method Validation Requirements Required validation parameters for potency methods Accuracy, Precision, Linearity, Range, Specificity Other relevant assessments Method robustness Can be evaluated during assay development Stability indicating properties Required for licensure studies (FDA 2015 guidance for industry: Analytical Procedures and Methods Validation for Drugs and Biologics) Data for meaningful assessment may not be available for IND enabling studies System suitability criteria Integral part of potency methods Phase-appropriate approach 4

5 Validation Objective and Approach Validation objective Demonstrate suitable method performance for intended use Elements for a meaningful validation approach Simple and efficient study design Meaningful acceptance criteria Appropriate data analysis Standardized approach 5

6 Potency Method Validation Study Design Considerations Efficient assessments of required validation parameters Combined study design can often be used to assess accuracy, precision, linearity and range simultaneously Sufficient sample size Independent of mechanism of action Replication strategy Validation experiment can be based on single replicate Reportable results based on multiple-replicates may be used during routine testing (σσ rrrrrrrrrrrrrrrrrrrr = σσ ssssssssssss / nn, where n is the number of replicates) Co-validation Pros: Very common; data from multiple laboratories Cons: Logistics; Parameter estimation may be challenging Technology transfer in combination with validation in representative QC lab can have advantages 6

7 Example: Validation Study Design and Data Cell-based bioassay, mid stage (Phase II) Nominal Potency 50% 75% 100% 125% 150% Run Observed % Recovery Mean % Recovery Overall SD (single replicates) Replication Strategy The reportable value for routine sample testing is defined as average of n=3 independent replicates, the predicted SD for reportable results is 4.5/ 33 =2.6 High accuracy and precision result in a powerful assay (strong capability of differentiating true changes) 4.5 Validation study is based on single replicates One dataset (total sample size N=30) covers accuracy, precision, linearity and range Replicates at each nominal potency level are from independent assay runs 7

8 Example: Validation Outputs Summary of assay accuracy, precision, linearity and range Nominal Potency (%) Accuracy (Mean % Recovery) Intermediate Precision (Overall SD of N=30 %Recovery Results) Predicted SD of Reportable Results (Mean of 3 Replicates) Linearity Mean Determiined Relative Potency (%) Linear Regression Plot y = 1.00x R² = Target Relative Potency (%) Validated Range: % 8

9 Validation Acceptance Criteria Considerations Not too narrow Suitable method should be expected to pass the validation criteria The uncertainty of calculated results associated with sample size should always be taken into consideration Not too wide Passing the validation criteria should provide sufficient confidence that the method is suitable for intended purpose scientifically / statistically meaningful validation acceptance criteria 9

10 Strategy for Setting Meaningful Validation Acceptance Criteria Anticipated specification limits Define assay performance target Develop assay with desirable performance (suitable for intended purpose). Initial proposal of replication strategy Define validation design based on assay performance knowledge ( not too narrow ) Define validation criteria based on expected specification ( not too wide ) Final validation acceptance criteria and study sample size 10

11 Step 1: Setting Assay Performance Target Step 1 Statistical Assessment Define assay performance that will ensure acceptable method capability Required additional inputs for method capability: Expected overall product mean potency σ pppppppppppppp, variability due to manufacturing process. (OOS calculation uses the total variability from process and method σσ tttttttttt = σσ pppppppppppppp = +σσ aaaaaaaaaa = σσ pppppppppppppp + σσ ssssssssssss rrrrrrrrrrrrrrrrrr /nn ) 11

12 Example Contour Plot: Assay Performance Criteria (Specification: %) X-axis: Assay accuracy (Recovery: %) Y-axis: Assay precision (reportable results) (SD: 0-200) Contour lines: Specified OOS rate Area below the contour line: assay performance with expected OOS rate less than the specified value Assumption for this plot: Product mean potency=100%, variability from manufacturing process is negligible 12

13 Example: Contour Plot Details (Specification Limit: %) X-axis: Assay accuracy (Recovery: %) Y-axis: Assay precision (reportable results) (SD: 0-20) Contour lines: Specified OOS rate Area below the contour line: assay performance with expected OOS rate less than the specified value Assumption for this plot: Product mean potency=100%, variability from manufacturing process is negligible 13

14 Example Contour Plot: Assay Performance Criteria (Specification: %) X-axis: Assay accuracy (Recovery: %) Y-axis: Assay precision (reportable results) (SD: 0-20) Contour lines: Specified OOS rate Area below the contour line: assay performance with expected OOS rate less than the specified value Assumption for this plot: Product mean potency=100%, variability from manufacturing process is negligible 14

15 Example Contour Plot: Assay Performance Criteria (Specification: %) X-axis: Assay accuracy (Recovery: %) Y-axis: Assay precision (reportable results) (SD: 0-20) Contour lines: Specified OOS rate Area below the contour line: assay performance with expected OOS rate less than the specified value Assumption for this plot: Product mean potency=100%, variability from manufacturing process is negligible 15

16 Numerical Examples OOS Rate vs. Assay Performance Corresponding SD for Assay performance Upper bound of OOS rate single replicates Accuracy Precision (SD (Reportable results (Recovery) for reportable) based on 3-replicates) % % % % % % 0.0% 0.0% 0.0% % % 0.0% 0.0% 0.0% % % 0.0% 0.0% 0.0% % % 0.0% 0.0% 0.1% % % 0.0% 0.1% 2.4% % % 0.0% 0.0% 0.0% % % 0.0% 0.0% 0.0% % % 0.0% 0.0% 0.1% % % 0.0% 0.0% 0.5% % % 0.0% 0.2% 4.4% % % 0.0% 0.0% 0.0% % % 0.0% 0.0% 0.0% % % 0.0% 0.0% 0.2% % % 0.0% 0.0% 1.2% % % 0.0% 0.4% 6.8% % % 0.0% 0.0% 0.0% % % 0.0% 0.0% 0.0% % % 0.0% 0.0% 1.5% % % 0.0% 0.0% 4.2% % % 0.0% 1.0% 12.4% 16

17 Step 1 Summary: Setting Assay Performance Target There is a direct correlation between acceptance criteria for specification vs. assay performance requirements The required assay performance can be defined based on acceptable method capability resulting in acceptable OOS rate during routine testing (e.g., 1%) Desirable assay precision may be achieved using reportable value based on multiple replicates 17

18 Step 2: Develop Assay with Desirable Performance Step 2 Optimize assay parameters. Define proper data analysis model and system suitability criteria. Propose replication strategy 18

19 Step 3: Define Validation Design Based on Assay Performance Knowledge ( not too narrow ) Step 3 Statistical Assessment Calculate probability of passing given validation acceptance criteria with expected assay performance 19

20 Example: Probability of Passing Validation Criteria with Given Assay Performance and Validation Study Design Expected performance (single replicate) Accuracy (mean recovery) Precision (SD) 93% 13 Validation study design (based on single replicates) Accuracy criteria (mean recovery per level) Precision criteria (SD) % % % % % % % % 20 Probability of passing N validation 30 16% 60 29% 30 18% 60 30% 30 63% 60 88% 30 71% 60 92% 30 86% 60 95% 30 96% % 30 89% 60 95% % % % % % % The validation study design with acceptable passing probability (green boxes) will be considered for next step 20

21 Example: Impact of Validation Sample Size N (Validation Criteria: Accuracy %, precision 20) Area below the line: assay performance that will pass the validation criteria with 95% probability with specified validation sample size N X-axis: assay accuracy (80-120% recovery) Y-axis: assay precision SD (0-20) 21

22 Step 3 Summary: Define Validation Design Based on Assay Performance Knowledge ( not too narrow ) Probability of passing validation acceptance criteria can be calculated based on expected assay performance and validation sample size N Select validation study design based on acceptable probability of passing the validation with expected performance (e.g., 95%) Increasing the validation sample size N may help achieving the acceptable probability of passing 22

23 Step 4: Define Validation Criteria Based on Expected Specification ( not too wide ) Step 4 Statistical Assessment Calculate worst case probability of failing anticipated specification Ensure suitability for intended purpose 23

24 Example: Probability of Not Passing Anticipated Specification at Limits Expected performance (single replicate) Accuracy (mean recovery) Precision (SD) 93% 13 Validation study design (based on single replicates) Accuracy criteria (mean recovery per level) Precision criteria (SD) % % % % % % 20 N Probability of passing validation Anticipated specification limits Worst case probability of not passing specification at limits 60 95% % 0.0% 30 96% 0.5% % 0.5% % % 0.2% % % 1.5% % 1.5% % % 4.2% % % 8.3% Assumptions for this example: 1. Product mean potency=100%, variability from manufacturing process is negligible. 2. Reportable value is the mean of 3 replicates

25 Step 4 Summary: Define Validation Criteria Based on Expected Specification ( not too wide ) Probability of not passing anticipated specification at limits will inform selection of final validation acceptance criteria In case desirable validation acceptance criteria ( not too narrow and not too wide ) can not be achieved, consider further assay development and / or re-assessment of replication strategy. 25

26 Case Study Goal Determine meaningful validation acceptance criteria for an early-phase study Background information Anticipated specification limits % Overall product mean potency 100% Potency variability component (SD) due to manufacturing process Acceptable OOS rate during routine testing 1% Acceptable probability of passing validation acceptance criteria σσ pppppppppppppp =3 95% 26

27 Step 1: Setting Assay Performance Target Set assay performance target based on OOS assessment using overall SD: Target Assay Performance σσ tttttttttt = 22 σσ pppppppppppppp 22 + σσ aaaaaaaaaa Note: Tighter assay performance requirement compared to the case when σσ pppppppppppppp is negligible 27

28 Step 2: Develop Assay with Desirable Performance Actual performance of the developed assay (single replicate) Accuracy: 108% Precision: SD=12 Proposed replication strategy for reportable value: Mean of 3 replicates SD for reportable values =12/ 33 = 7 Overall SD (process + method): =7.5 Estimated OOS rate: 0.0% 28

29 Step 3: Define Validation Design Based on Assay Performance Knowledge ( not too narrow ) Expected performance (single replicate) Accuracy (mean recovery) Precision (SD) 108% 12 Validation study design (based on single replicates) Probability of passing validation Accuracy criteria (mean recovery per level) Precision criteria (SD) N % % % % % % 60 99% % % % % % % % % % % % % % 30 97% % % % % % % 29

30 Step 4: Define Validation Criteria Based on Expected Specification ( not too wide ) Expected performance (single replicate) Accuracy (mean recovery) Precision (SD) 108% 12 Validation study design (based on single replicates) Accuracy criteria (mean recovery per level) Precision criteria (SD) Probability of passing validation Anticipated specification limits Worst case probability of not passing specification at limits N % % % % 0.1% % % 0.6% % 0.6% % % 2.1% % % 2.1% % % 0.3% % 0.3% % % 1.8% % % 4.5% 30 97% 1.5% % % 1.5% % % 4.7% % % 8.7% 30

31 Summary of Case Study Define assay performance target based on OOS assessment using anticipated specification and process knowledge Based on assay performance knowledge, identify selections of validation criteria that will allow the assay to pass ( not too narrow ) Based on specification, define final validation criteria that ensure suitable performance ( not too wide ) The final validation criteria are not too narrow and not too wide. 31

32 Summary of Presentation A scientific/statistically-driven approach has been defined to provide a practical solution for determining meaningful validation acceptance criteria for potency methods Define assay performance target based on OOS assessment Develop assay with desirable performance. Gain good knowledge of assay performance. Based on assay performance knowledge, identify validation criteria that are not too narrow Based on specification, define final validation criteria that ensure suitable performance ( not too wide ) The concept can be applied across methodologies beyond potency assays 32

33 Acknowledgement Bioassay Center of Excellence Jeff Glenn Cassie Liu Weiguo Cai Marcel Zocher 33