Statistical approaches for comparability assessment

Transcription

1 Statistical approaches for comparability assessment A regulatory statistician s views and reflections Andreas Brandt Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 1

2 Disclaimer The views expressed in this presentation are the presenter s personal views and not necessarily the views of BfArM or EMA Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 2

3 Overview Currently applied statistical methods Considerations on role of quality experts and statisticians EMA Reflection paper on statistical methodology for the comparative assessment of quality attributes in drug development Summary Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 3

4 Personal experiences Started to be involved in similarity assessment for analytical biosimilarity less than 1 year ago Lesson learned: quality data are different Involved in several procedures not in two of them the same statistical methods for similarity assessment was used Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 4

5 Reference range based approaches Reference range is established based on reference batches Min-Max range Tolerance interval Mean +/- x * standard deviation Similarity is decided based on coverage of test batches by reference range All test batches included X% of test batches included Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 5

6 Example reference test Min-max 90%-95% tolerance interval Mean +/- 2* SD Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 6

7 Equivalence-testing based approaches Choose parameter(s) that describe(s) the distribution of QA of interest Similarity decision is based on similarity ( equivalence ) of parameters for test and reference distribution Similar parameters similar distributions Can be decided based on a statistical test ( equivalence test ) Recommended using the mean by FDA for QAs in Tier 1 Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 7

8 Example: Normal distribution mean SD Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 8

9 Example: Normal distribution reference test Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 9

10 What is the correct statistical approach? There are no right or wrong statistical methods Statistical methods itself do not define what is similarity Statistical methods are tools for decision-making on true similarity Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 10

11 Key question: What is true similarity? If the underlying truth was known (i.e. infinitely many representative batches could be sampled): when can test and reference be considered as similar? Is high overlap of true test distribution with specification limits of reference sufficient? Is it required that distributions of QAs are similar? Question needs to be answered (primarily) by quality experts, not statisticians Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 11

12 Underlying truth Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 12

13 Similar? Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 13

14 Underlying truth vs. sampled data Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 14

15 Data available to statistician Possible decision criteria based on available data: Min-max 90%-95% tolerance interval Mean +/- 2* SD 95% confidence interval for difference in means: [-1.5, 0.4] Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 15

16 What is the statistician s task? Underlying truth is unknown Statistician s task: find fair criteria to decide whether true similarity is fulfilled based on limited number of samples Properties of statistical methods (roughly speaking) Type 1 error : Probability to declare products to be similar that are not Power : Probability to declare products to be similar that are similar Properties of statistical methods should be known Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 16

17 Published for public consultation in March 2017 Reflections on the statistical framework for comparability assessment Does not provide a final solution Aim: facilitate discussions and develop a common language and understanding Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 17

18 Understand the data What is the unit of observation? Batch, tablet, vial? Dependencies between observations need to be taken into account Knowledge of sources of variability required for standardization Between-batch variability: e.g. batch age Within-batch variability Within-sample variability: e.g. storage conditions Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 18

19 Within-batch and between-batch variability Blue curves: infinitely many samples drawn from the single batches within batch variability does not represent true variability Red curve: single samples drawn from infinitely many batches Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 19

20 Source of variability young batches old batches Consistent production process? Influence of batch age on QA? Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 20

21 Similar? reference test young batches old batches Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 21

22 Understand data generating process: sampling Are the batches used for comparability exercise representative? Ideally: Random sampling from all/a large number of batches Random sampling often not possible Consistency assumption required Knowledge on sources of variability needed Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 22

23 Assumptions for inferential analysis Descriptive statistics: only statements about sampled data (e.g. min-max range) Inferential statistics: making conclusions on underlying truth based on sampled data Assumptions for inferential analysis Consistency: consistent production process, or sources of variability known Representative sampling Often: Distribution assumption (e.g. normal distribution) Inferential statistical concepts Equivalence testing Tolerance intervals Assumption that sample mean +/- 2* sample SD covers ~ 95% of reference distribution Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 23

24 Limitations of reference range based approaches Statistical properties (type 1 error, power) poorly understood Probability for conclusion of similarity increases with uncertainty Probability that reference range covers test batches is larger for few test batches Reference ranges defined based on statistical intervals to quantify uncertainty of location are wider when uncertainty is large Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 24

25 Example: Tolerance interval %-95% tolerance interval using all reference data 90%-95% tolerance interval using 2/3 of reference data Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 25

26 Equivalence testing Framework for appropriate inferential equivalence testing described in reflection paper Choose characteristic(s) to be compared (e.g. mean) Find metric to describe distance of characteristics (e.g. difference in means) Define equivalence limits based on maximal acceptable difference Justification of equivalence limits required Knowledge about the association between quality characteristics and clinical outcome needed Some arbitrariness acceptable? Well-understood approach from statistical side but practical? Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 26

27 Summary Several statistical approaches are used for similarity assessment Key questions to be addressed For quality experts: What is true similarity? For statisticians: How can fair decisions on true similarity be made based on limited data? Reflection paper on statistical methodology for similarity assessment Understand raw data and data generating process Understand the assumptions Be aware of limitations Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 27

28 the curtain closed and all the issues open Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 28

29 Acknowledgements EMA Biostatistics Working Party Thomas Lang (Vice-Chair) Norbert Benda David Brown Christian Gartner Robert James Hemmings Armin Koch Anja Schiel (Chair) Steven Teerenstra Ferran Torres Jörg Zinserling Cecilia Hedlund + Observers BfArM Ann-Kristin Leuchs Astrid Schäfer Brigitte Brake Ute Fischer Katrin Buss Cornelia Lipperheide Birgit Schmauser Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 29

30 Thank you very much for your attention! Contact Federal Institute for Drugs and Medical Devices Division Research, Unit Biostatistics and Special Pharmacokinetics Kurt-Georg-Kiesinger-Allee 3 D Bonn Contact person Dr. Andreas Brandt andreas.brandt@bfarm.de Tel. +49 (0) Andreas Brandt Statistical approaches for comparability assessment: A regulatory statistician s views and reflections CMC Strategy Forum Europe 2017 Page 30