Setting the Bar and Establishing In-Study Cut Points for Immunogenicity Testing. Ron Bowsher, Ph.D. 16-May-2016

Transcription

1 Setting the Bar and Establishing In-Study Cut Points for Immunogenicity Testing Ron Bowsher, Ph.D. 16-May-2016

2 B2S Consulting Team: o Rocco Brunelle, M.S. (Statistics) o Kim Krug, M.S. (Statistics) o Paula Santa, B.S. (Project Management) o Mary Seger, M.S. (Statistics + Medical Writer) o Wendell Smith, Ph.D. (Statistics) Viswanath Devanarayan, Ph.D., Abbvie 2016 NBC 2

3 Produce and characterize custom reagents & conjugates Surrogate PAB/MAB generation, purification & characterization Assay development, optimization & troubleshooting Custom reagents for immunoaffinity LC-MS/MS Reagent storage, LCM & distribution NAB assay design & optimization Balanced study design Data transformation & ANOVA models Statistical-based outlier removal Screening, confirm., & Titer CPs NAB CPs PK/TK/PD Data Analysis Evaluate clinical immunogenicity Data mining for ADA Assay troubleshooting 2016 NBC 3

4 Consultative support / Bioanalytical + Statistical data analysis ~ 100 CP analyses in 2015 & cumulative experience is several hundred over past decade Use a systematic statistical approach for CP estimation: o SAS and R-based computations o Robust method consistent with Shankar recommendations o For SCP the usual data transformation is Log (S/N) o Employ ANOVA model / fixed & random effects o Iterative removal of analytical & biological outliers o Monitor normality and skewness Much Increased frequency of requests for In-study CPs In-study & Study Specific CP are synonymous terms 2016 NBC 4

5 Validation ADA CPs In-Study ADA CPs o Executed proactively o Commercially-obtained NHS, unlimited numbers and vol. o Balanced stat. design o Ample sample volume to test multiple times +/- drug o Only a few biological outliers o After outlier removal, acceptable normality o SCP set at 5% FPER o Often reactive in response to a study finding or commercial sera produce dissimilar results o Limited to predose sera from study subjects/patients o Limitations in sample # & vol can constrain CP study design o Often implement an incomplete balanced design & fewer runs o Can show higher incidence of biological outliers & right tail o SCP set at 5% FPER, but design constraints may result in a less robust CP estimate 2016 NBC 5

6 Prospective (proactive) Retrospective (reactive) Often prompted by a change in study pop. or disease-state Compliance with regulatory recommendations Unanticipated finding Usually appropriateness of established CPs from validation is questionable In-Study CP Application 2016 NBC 6

7 Proactive Planned in advance to conduct multiple CP determinations for different species & study populations Anticipated lag between assay validation & sample analysis Desire to maximize reliability of CP estimates (lower risk) Reactive Postpone multiple CP determinations and only perform them when needed Desire to reduce development time and lower cost higher risk as CP estimates may not be well-suited for different study populations 2016 NBC 7

8 CP assessment is a planned activity Progression into Phase II/III clinical investigation Prior experience with disease-state samples 2016 FDA draft guidance Line 196 Because samples from different target pop and disease states may have components that can cause the bkg. signal from the assay to vary, different CPs may be needed for discrete pops FDA draft guidance Line 1076 Samples from different pop. can have different bkg. activity in ADA assays. Thus, it s necessary to confirm that the CP determined during assay development is suitable for the pop. being studied. A sufficient number of samples from the target pop. should be used, and justification for the number used should be provided. Higher probability for a complete balanced CP study design 2016 NBC 8

9 CPs determined in validation with commercial samples are judged to be unsuitable for In-testing testing of samples Examples: o Noticeable difference in bkg. responses of study samples in comparison to results from samples used for CP estimation o Differences found across groups of animals from different colonies o Appreciable lag time between validation & sample analysis o Commercial samples are not representative of study samples o Progression into Phase II/III studies / impact of disease-state o Change in a critical reagent (e.g., new detection reagent) CP design can be impacted by sample availability = incomplete designs, confounded factors and less informative NBC 9

10 During Validation = Use commercially-obtained samples or subset of study predose samples for a preliminary investigation of disease-state(s) impact After validation = compare results with CP validation. If FPER is < 2% or > 12%, consider performing an In-study CP (If yes, then Next Slide) Statistical significance vs. Practical significance Final observed Positivity In-study will not necessarily equal 5%. Often 10% due to statistics & pre-existing ADA 2016 NBC 10

11 Analyze assay responses in terms of Log(S/N) Define an optional ANOVA model for robust CP estimation Identify & remove Analytic & Biologic Outliers Compute the CP: o Obtain parametric & nonparametric CP estimates o Assess normality & skewness o Evaluate the CP estimate 2016 NBC 11

12 50 50 *Devanarayan V., Update strategy on CPs in Immunogenicity testing, Training Seminar, 7-April NBC 12

13 2016 NBC 13

14 Setting the Bar and Establishing In-Study Cut Points for Immunogenicity Testing

15 Biological Outliers A SAMPLE whose measured values consistently deviate from the overall mean of all test samples. o Biological outliers often display appreciable %INH which likely signifies pre-existing ADA Analytical Outliers A SAMPLE RESULT (measured value) that deviates from the mean of the test sample NBC 15

16 The distribution of ANOVA conditional residual values from each model is evaluated to identify analytic stat outliers, defined as values > 75 th quartile x [interquartile range] or < 25 th quartile x [inter-quartile range]. Outliers identified by this criterion were removed and the model was re-fit until no outlying value was detected. Inter-quartile (IQR) is the difference between the 75 th percentile and the 25 th percentile NBC 16

17 The distribution of individual sample best linear unbiased predictor (BLUP) values was then examined to identify samples as biologic statistical outliers by applying the same criterion used for conditional residuals. All values for individual samples identified as biologic outliers were removed and the statistical analysis was repeated until no further outliers were present. Normality of conditional residual values and sample BLUP values were evaluated by the Shapiro- Wilk test and consideration of the skewness coefficient as a relative measure of symmetry NBC 17

18 B2S Diagnostic Plot %INH Confirmatory CP, 0.1% FPER True Positives? Confirmatory CP, 1% FPER Screening CP False Positive 2016 NBC 18

19 Need for Iterative Removal 2016 NBC 19

20 2016 NBC 20

21 Screening CPs: Parametric 2.55 Nonparametric NBC 21

22 2016 NBC 22

23 2016 NBC 23

24 2016 NBC 24

25 Screening CPs: Parametric 1.04 Nonparametric NBC 25

26 Today, we are seeing increased need for application of the use of study-specific CPs for detection and characterization of ADA. In-study CPs can provide improved classification of results for study samples Use of SSCPs requires prospective planning for optimal implementation 2016 NBC 26

27 2016 NBC 27

28 Ron Bowsher, Ph.D (office) (cell) NBC 28

29 Plate Order Analyst 1 Analyst 2 Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Group A Group B Group C Group C Group A Group B Assay 1 Assay 4 Assay 7 Assay 10 Assay 13 Assay 16 Group B Group C Group A Group A Group B Group C Assay 2 Assay 5 Assay 8 Assay 11 Assay 14 Assay 17 Group C Group A Group B Group B Group C Group A Assay 3 Assay 6 Assay 9 Assay 12 Assay 15 Assay 18 Each sample is tested once (duplicate wells) in each run by both analysts. Each run consists of 3 ELISA plates Group A: Samples 1-17 Group B: Samples Group C: Samples Samples 306 Total Values Balanced Latin Square Design NAb assays often have fewer samples and fewer assay values. Need to have a large enough sample size for the analysis and cut point estimates. If possible use the typical ADA study design good design for the analysis 2016 NBC

30 *Key Assumption is a unimodal normal distribution 1. Define the data Transformation (usually log) 2. Define the Model (Fixed and Random Effects) 3. Fit the model to the data & examine the individual sample residuals & lot predicted means 4. Identify Outliers statistically (not visually), remove iteratively and refit the model until all outliers are identified and removed 2016 NBC 30

31 ANOVA Use this model to identify analytical and biological outliers. Fixed Effects: Group Analyst Plate Order Disease-state Random Effects: Sample(Group) Run(Analyst) Assay Number Residual 2016 NBC