Statistics in Validation. Tara Scherder CSO Supply, Arlenda, Inc

Transcription

1 Statistics in Validation 05 Arlenda Tara Scherder CSO Supply, Arlenda, Inc IVT Validation Week Philadelphia, PA Oct 7,05

2 Agenda Evolution of Validation 0 FDA Guidance Why Use Statistics Stage Process Design Stage Process Performance Qualification Stage 3 Continued Process Verification 05 Arlenda

3 Early Development Process Development Technical Transfer Supply Validation Assay Development Days Gone By. Assay Development Early Development Process Development Technical Transfer Validation Supply Limited Designed Experiments; Process Factors, Product Specifications Even less Designed Experiments 3 replicate batches Testing for Release; limited trending(apr); isolated investigation 05 Arlenda

4 Evolving. ( ) International Conference on Harmonisation (ICH) Q8, Q9, Q0 a systematic, modern risk and science based approach to pharmaceutical manufacturing and development across the product lifecycle Q8 Pharmaceutical Development Describes science and risk-based approaches for pharmaceutical product and manufacturing process development Q9 Quality Risk Management Describes systematic processes for the assessment, control, communication and review of quality risks Q0 Pharmaceutical Quality System Describes key systems that facilitate establishment and maintenance of a state of control for process performance and product quality 05 Arlenda 4

5 Evolving. Assay Development Early Development Process Development Technical Transfer Validation Supply Quality Target Product Profile Multivariate Designed Experiments; Design Space Quality By Design Control Strategy 05 Arlenda Some Designed Experiments 3 replicate batches 5 Testing for Release; limited trending(apr); isolated investigation

6 0 FDA Guidance Process Validation Stage Process Design Stage Process Performance Qualification Stage 3 Continued Process Verification the commercial process is defined based on knowledge gained through development and scale-up activities 05 Arlenda the process design is evaluated and assessed to determine if the process is capable of reproducible commercial manufacture SCIENTIFIC EVIDENCE ACROSS LIFECYCLE Ongoing assurance is gained during routine production that the process remains in a state of control

7 Why Use Statistics? Benefits the Patient and the Business 05 Arlenda

8 Lifecycle Process Validation...lifecycle approach to process validation that employs risk based decision making throughout that lifecycle. Severity X Occurrence X Detection / Control 05 Arlenda

9 Statistics is a Risk Management Tool Occurrence What factors affects likelihood, and how? How often will something occur? How can you be sure? 05 Arlenda

10 Stage Process Design Stage User Requirements Initial Quality Target Product Profile (QTPP) Commercial manufacturing platform knowledge Refined QTPP Criticality Analysis Control Strategy Data analyses Design Space Acceptance criteria for CQAs Development Report PPQ Protocol 05 Arlenda

11 Control Strategy Control Strategy A planned set of controls, derived from current product and process understanding, that assures process performance and product quality. The controls can include parameters and attributes related to drug substance and drug product materials and components, facility and equipment operating conditions, in-process controls, finished product specifications, and the associated methods and frequency of monitoring and control. (ICH Q0) 05 Arlenda

12 From Risk Analysis to Control Strategy Unit Op Critical Process Parameters Critical Material Attributes Unit Op CPP CMA Experiments Unit Op 3 CPP CMA Control Strategy 05 Arlenda

13 Space Exploration Design Space Multidimensional space demonstrated to provide assurance of quality Normal Operating Space Operating region where process is being run Regulatory Flexibility Working within the design space is not considered a change Unexplored Space Operational region where we have no information Design Space Normal Operating Space Knowledge Space Explored Space 05 Arlenda

14 Sources of Information First Principles Process Observations (historical data) Experimentation One Factor At a Time (OFAT) Multifactor Design of Experiments (DOE) 05 Arlenda

15 What is Design of Experiments (DOE)? DOE is a structured, organized method for determining the relationship between factors (X s) affecting a process and the output of that process (Y). (ICH Q8) Input Materials (X - attributes) Input Process Parameters (X - process factors) Process*, Process Step, or Unit Operation Other Factors Product or Intermediate (Y - attributes) *e.g. Manufacturing process, analytical process, chemical reaction, etc. 05 Arlenda

16 Maximizing Information - Hidden Replication One Factor at a Time Factorial Design Temp. Time Pressure 05 Arlenda Factors varied simultaneously, yet evaluated independently

17 Limitations of OFAT Experiments Ignores the possibility of interactions between factors Often does not find the optimum Fails to maximize the value of each run (provides less information/run) 05 Arlenda

18 Overview of DOE: DOE and Phases of Development Screening Experiments: Many Factors; Product/Process Not Well Understood Identify Important Factors Characterization Experiments / Optimization Experiments: Few Key Variables Build Predictive Model Scale-up Experiments / Process Robustness Experiments: Few Key Variables Robustness to Noise Factors Operating Ranges Screening Designs Factorial Designs with Interactions Response Surface Designs 05 Arlenda Response Surface Designs Robust Parameter Designs

19 Term Pareto Chart of Effects Pareto Chart of the Standardized Effects (response is Coarse, Alpha = 0.05) 3.8 A C Factor A B C Name Roll Force Roll Speed Roll Gap B AC AB BC ABC Standardized Effect Arlenda

20 Mean Main Effects and Interaction Plots % Coarse Particles Main Effects Plot for Coarse Data Means Interaction Plot for Coarse Data Means 4 3 Roll Force Roll Speed Roll Force Roll Force Point Type.0 Corner 7.5 Center 9.0 Corner Roll Gap Roll Speed Roll Speed Point Type Corner Center 0 Corner 3 0 Roll Gap Arlenda

21 A v erage Granulation Porosity (% ) Factorial, Response Surface, Predictive Design Space Factor B : F lo w R a te 3 Roll Pressure Granulation Porosity Roll Gap Design-Expert Software Water Remaining Design Points Water Remaining 0 9 X = A: Back Pressure X = B: Flow Rate Roll Speed A: Back Pressure Probability of meeting all sustained released dissolution Bayesian Predictive Modeling 05 Arlenda Factor

22 Experimental Planning Step Input Variables. Planning Budget Dependent Variable Research. Factor ID Brainstorm Control Noise 3. Interaction Hypothesize Pick 4. Design Choice Budget Interaction Run Combination Randomize Effect Size Blocking/Center Point Full/Fractional/Response Resolution Replication 5. Run Training Experimental Log. Analyze Plot the data Compute Effects 7. Confirm Confounding Optimal Case Build the model 8. Recommend Settings SOPs Follow-up Experiments 05 Arlenda

23 Design Choice Budget Effect Size determination Replication possibilities Type of design Randomize Order of experiments Materials for the experiment Based on physical restrictions Interactions Resolution Type of design Effect Size Replication Ensure statistical significance Run Combination Limited Experimentation Factor Level changes Blocking/Center Points Blocking groups similar experimental units within an experiment Center Points allow the study of curvature and provides replication runs Screening/Refining/Optimizing Screening Many Factors Refining - Full factorial or Fractional Factorial Fewer Factors + Interactions Optimizing -Response Surface Vital Few Factors continuum Resolution Budget Replication Type of Design Replication Independent test of an experimental run Estimation of experimental error Allows calculation of Standard Deviation 05 Arlenda

24 Summary DOE for Process Design design a process suitable for routine commercial manufacturing that can consistently deliver a product that meets its quality attributes. The designed experiment gives a mathematical model relating the input factors and responses, relative to the noise in the process No more experiments where you can t draw conclusions Maximize information per experiment, which translates to minimizing cost, and increased precision The statistical significance of the results is known, so there is more confidence in the results Can assess both independent and interactive effects 05 Arlenda

25 Stage PPQ Goal of PPQ Sampling and Acceptance Criteria in PPQ Intra and Inter Batch Analysis Number of PPQ batches Assessing Need for Heightened Testing in Stage 3A 05 Arlenda

26 Stage PPQ Stage Quality Target Product Profile Criticality Analysis Report Process Design Control strategy Data analyses Acceptance criteria for CQAs 05 Arlenda Confirmation of Control Strategy Assessment of Control, Capability, Reproducibility Stage 3a CPV plan Recommendations for associated risk based sampling plans Adjustments to the CA Report and CPV plan Implemented Control Strategies Manufacturing Procedures Updated Process Risk assessments as required

27 Claims of Process and Product Quality In Control: Consistent, stable, predictable Control charts, graphs, variance components Reproducible: Robust to typical sources of variability Graphs, variance components Capable: Able to meet specification Statistical intervals, such as tolerance interval, Capability Analysis such as Cpk/Ppk, graphs 05 Arlenda

28 Population and Sampling 05 Arlenda

29 Example Capability Statements We are 95% confident that the mean assay will be between 97.5 and 00. (confidence interval) We are 95% confident that 95% of assay values in this batch will be between 97.5 and 00. (tolerance interval) There is 95% probability that 9% of batches will pass potency specification (Bayesian tolerance interval) We are 90% confident that 95% of samples in this batch will pass the uniformity dosage unit claim (ASTM 709/80) We are 95% confident that there are less than % bottles with label defects in this batch (binomial confidence interval) 05 Arlenda

30 Elements of a Statistical Interval Measurement that must meet criteria Mean, individual results, % non-conforming Confidence or Probability (Bayesian) Related to the risk you are willing to take the statistical statement is not correct Coverage of a population (e.g. lot) For individual and % non-conforming, what percent of the population must meet some criteria 05 Arlenda

31 Example Interval: Tolerance Interval Statement about expected range of individual values.we are 95% confident that 95% of assay values in this batch will be between 97.5 and 00. How do you choose confidence and coverage values? Estimate ± Margin of Error; X ks What influences k, and hence the size of the interval? Where do you obtain estimates of S? How does this influence sample size? 05 Arlenda

32 Percent Percent Effect of Confidence Level Tolerance Interval Plot for concentration 95% Tolerance Interv al A t Least 95% of Population C ov ered Statistics N 0 Mean StDev 77.9 Normal Normal Nonparametric Normal Probability Plot Lower Upper Nonparametric Lower Upper Normality Test AD 0.0 P-Value Tolerance Interval Plot for concentration 80% Tolerance Interv al A t Least 95% of Population C ov ered Statistics N 0 Mean StDev 77.9 Normal Normal Nonparametric Normal Probability Plot Lower Upper Nonparametric Lower Upper Normality Test AD 0.0 P-Value Arlenda

33 Percent Percent Effect of Population Coverage Tolerance Interval Plot for concentration 95% Tolerance Interv al A t Least 95% of Population C ov ered Statistics N 0 Mean StDev 77.9 Normal Normal Nonparametric Normal Probability Plot Lower Upper Nonparametric Lower Upper Normality Test AD 0.0 P-Value Tolerance Interval Plot for concentration 95% Tolerance Interv al A t Least 99% of Population C ov ered Statistics N 0 Mean StDev 77.9 Normal Normal Nonparametric Normal Probability Plot Lower Upper Nonparametric Lower Upper Normality Test AD 0.0 P-Value Arlenda

34 Effect of Sample Size 05 Arlenda

35 Tolerance Interval Procedure. Determine confidence / coverage requirement. In design of PPQ sampling plan, use estimate of standard deviation, along with confidence and coverage to determine the sample size that will result in a projected interval that falls within specification limits 3. Select representative sample size n from lot. Sample size considerations: adequately assesses sources of variability (B/M/E, measurement error, across time etc...) Projected interval will fall within specification 4. Graph data; check for non-random features. 5. Acceptance Criteria: If the interval mean ± k sigma is within the spec limits, the lot readily passes. If not, consider root cause, risks to determine appropriate action (documentation of known special root cause, heightened test and plans for Stage 3A, leverage other statistical methods, PPQ failure). 05 Arlenda

36 Homogeneous materials Do you need to estimate uncertainty if the material is homogenous? Where are the sources of variability? What are the key questions? How can you be sure? What methods/options can be used? 05 Arlenda

37 Graph! Graph! Graph! Always! Always! Always! 05 Arlenda

38 Bayesian Modeling Directly compute the probability of OOS, or the interval associated with some probability for various sample sizes Establish Prediction or Tolerance Intervals from Posterior Distribution Uncertainty in parameter estimates is incorporated Can result in smaller sample sizes needed to accurately predict performance Easily model multiple components Multiple Equipment (process and measurement) Batch to Batch, Location to Location, Unit to Unit Measurement Error 05 Arlenda

39 Intra and Inter batch sampling The number of samples should be adequate to provide sufficient statistical confidence of quality both within a batch and between batches Intra-batch Multiple samples taken within a single batch used to assess control and capability of that batch Can evaluate using graphs, statistical intervals, variance components, control charts if there is a time element Typically greater than commercial sampling Inter-batch Results of multiple batches are compared to assess reproducibility Can evaluated using graphs, variance components, ANOVA PPQ data can be too limited to make strong Inter-batch assessment. Evaluation will continue in Stage 3A 05 Arlenda

40 Sampling Plans 05 Arlenda

41 Variance components Separate amount of variability contributed by each source Batch Batch to Batch Location Location Location Location Location to Location Replicate Replicate Replicate Replicate Replicate Replicate Replicate Replicate Sampling and Measurement Batch Location Location Location Location Replicate Replicate Replicate Replicate Replicate Replicate Replicate Replicate Examples: Bottle filling using multiple fill nozzles and torque heads API material separated into multiple drums Vessel separated into top, middle and bottom Tablets across multiple time locations and two press sides 05 Arlenda

42 Variance Component Example Nested ANOVA: Particle Size vs batch, container Analysis of Variance for D0 Source DF SS MS F P Batch container Error Total Variance Components % of Source Var Comp. Total StDev Batch container Error Total Arlenda

43 fill weight fillwtadj Variance Component Example Nested ANOVA: Fill Weight versus Batch, Location, needle Analysis of Variance for fillwtadj Source DF SS MS Batch Location needle Total Variance Components % of Source Var Comp. Total StDev Batch -0.00* Location needle Total Individual Value Plot of fill weight Multi-Vari Chart for fill wt by needle - Batch 4 Panel variable: Batch Location needle Location 3 4 Arlenda Batch

44 Assessing Need for Heightened Testing in Stage 3A conc 90 Individual Value Plot of conc location batch T M B T M B T M 3 B T M 4 B During PPQ, intra-batch samples were drawn from top, middle, and bottom of vessel to show that concentration was consistent throughout.???? 05 Arlenda

45 Number of PPQ Batches Statistically based; combine inter and intra batch estimates. Can result in smaller intra batch sample size Risk based; based on process knowledge and performance Should include typical sources of variability Equipment trains Raw materials Operational Measurement (be wary of measurement confounding with process) Minimum is 3 in EU Annex 5 05 Arlenda

46 Stage Summary The process design is evaluated and assessed to determine if the process is capable of reproducible commercial manufacture Statistics is a tool to enable: A risk based approach to PPQ Conclusions that incorporate uncertainty from sampling Focus on key sources of variability Confirmation that control strategy maintains product quality Integration of criticality, performance and risk 05 Arlenda

47 Stage 3 CPV Goal of CPV The Nature of Pharmaceutical Manufacturing Data State of Control Some statistical considerations Risk Based Approach to CPV Risk based approach to responding to statistical signals Risk based approach to monitoring frequency Risk based approach to choice of attributes and parameters It s About the Patient 05 Arlenda

48 Continued Process Verification 0 FDA Guidance on Process Validation Ongoing assurance is gained during routine production that the process remains in a state of control Manufacturers should: Understand the sources of variation Detect the presence and degree of variation Understand the impact of variation on the process and ultimately on product attributes Control the variation in a manner commensurate with the risk it represents to the process and product 05 Arlenda

49 Stage 3 CPV Stage 3A Stage 3a CPV plan Recommendations for associated risk based rationales and sampling plans Pre-approval adjustments to the CA Report and CPV plan Implemented Control Strategies Manufacturing Procedures Updated Process Risk assessments as required 05 Arlenda State of control Stage 3B plan Sampling Variables Static control limits Process Capability - will be used to assess risk

50 Design of a CPV program WHAT Attributes and Parameters WHEN How Often WHO Statistician? Process SME? HOW Charts State of Control 05 Arlenda

51 Control Charts the VOICE of the PROCESS Individual Value Red signal indicate patterns that are unexpected by random chance likely due to a special cause I Chart of hours UCL= _ X= LCL= Observation - time order Arlenda

52 Common vs. Special Cause Variation Common Cause Stable, In Control Variation is predictable Variation is due to random or chance causes Special Cause Unstable, Out of Control Variation is not predictable; not expected by random chance Variation is due to non-random or assignable causes of variation (i.e. a signal that the process has changed ) May observe shifts, trends or out of limit points The essence of statistical control is predictability. Dr. Donald J. Wheeler 05 Arlenda

53 Understanding Process and Data Strictly speaking, observations in a control chart should be Independent (successive observations are not related to each other) From a single distribution (single mean and variance) Neither are likely true in pharmaceutical manufacture.. Interpret Variation Accordingly! 05 Arlenda

54 Understanding process and data What are some process factors that affect the output? Do they affect the process randomly (are results independent of each other)? How does that affect the control chart? Our interpretation and response to common cause and special cause variation must recognize this data structure 05 Arlenda Process Established Short term Operator Adjustment Raw Material Change Different Machine Long term

55 Individual Value State of Control I Chart of Content Uniformity UCL=0.09 _ X=99.37 LCL= Expect to see shift signals in a process having this Observation behavior Some special cause variation is expected. That is the state of control Short term limits (used in default Shewhart charts) will reflect within group variability, and will therefore typically be more narrow than long term limits

56 Mindset Change This is not real-time SPC Don t let fear of signals trigger manipulation of data and charts. You may forfeit learning. There is no requirement to initiate an investigation for statistical signals...not all signals are created equally. Magnitude of reaction depends on the severity of the signal () If red dots are always the enemy, consider changing the business process. 05 Arlenda () Alex Viehmann, FDA/CDER/OPQ, ISPE PV Statistician Forum April 05

57 All signals are not created the same Consider the severity Capability Magnitude of excursion Historical behavior Process and Measurement knowledge 05 Arlenda

58 Individual Value Individual Value Risk Based Approach to Responding to Statistical Signals Individual Value Individual Value I Chart of potency I Chart of potency UCL= UCL= _ X= _ X= LCL= LCL= Observation Observation I Chart of potency historical I Chart of potency historical 3500 USL USL UCL= UCL= Arlenda Observation _ X=400 LCL=95 LSL _ X=38 LCL=749 LSL Red 0 is 9 8 the 37 4new black 8 9 Observation

59 Sample StDev MR of Subgroup Mean Sample StDev Subgroup Mean Sample Mean Subgroup Charts for In Process Controls Tests performed with unequal sample sizes Xbar-S Chart of hardness lot 37 lot _ UCL=.57 X=.37 LCL=.08 I-MR-R/S 47 (Between/Within) Chart of hardness UCL=0.734 _ S=0.53 LCL= Within subgroup variability < between subgroup variability UCL=.57 _ X=.37 LCL=0.49 UCL=.00 MR=0.3 LCL= lot UCL=0.734 _ S=0.53 LCL=0.390 Tests performed with unequal sample sizes

60 Normality and Transformation leptokurtophobia - an irrational fear of using non-normal data in your analysis To transform or not to transform? How much does non-normality matter? Is it mathematically appropriate? Does it affect the likelihood of our ultimate goal? To learn bout the sources of variability? 05 Arlenda

61 Effect of Non-normality on 3-sigma limits Donald Wheeler, Quality Digest, 0 Nov Arlenda

62 Primary Question When Data Aren t Normal Whenever you fit a model to your data you are assuming that those data are homogeneous. If they are not homogeneous, all of your statistics, all of your models, and all of your predictions are going to be wrong () Transform only when underlying distribution is normal (physical, chemical, etc. ) Observed distribution could be happenstance, not underlying. Transforming is over fitting current data Limited data in distribution tails to model accurately, so any transform could be inaccurate Negatively affect ease of interpretation Consider risk of non-normality () Donald Wheeler, Quality Digest, 30 July 0 05 Arlenda

63 Transformation to Normal Questions to Ask Is this distribution expected in the future? Or is it happenstance? Which chart is easier to interpret for magnitude of changes? What other information is forfeited? 05 Arlenda

64 Transformation of Trend 05 Arlenda

65 Risk Based Approach to What to Monitor Stage 3A: Establish State of Control Critical Quality Attributes Consider Process Parameters that vary, and can influence a quality attribute. Document justification to not monitor. Establish tentative control limits until expected sources of variability have been incorporated 05 Arlenda

66 Individual Value Individual Value Individual Value Establishing Limits Distribution Distribution of variable Variable A reflecting A, with additional all initial sources sources of variability of variability, µ=0, σ= µ=.5, µ=9, σ=.5 LCL LCL UCL UCL mu0 mu mu mu mu9 mu9 UCL=5.38 UCL=5.38 UCL=5.38 UCL=.88 Final limits (n=90) Early limits (n=30) 05 Arlenda Observation _ X=0.3 _ X=0.3 X=0.07 LCL=5.08 LCL=3.5 LCL=5.08 Beware of establishing permanent limits before all sources of variability are captured.

67 Risk Based Approach to What to Monitor Stage 3B: Ongoing Monitoring Critical Quality Attributes Process Parameters How much does it vary? The control strategy may limit the variability to a range that does not influence the critical quality attribute What attribute does it influence? What is the severity of that attribute? The performance of that attribute? These are re-evaluated on an ongoing basis. If the state of control changes, adjustments to the CPV plan can be made based on a change in risk 05 Arlenda

68 Risk Based Approach to Monitoring Frequency Ideally, it is desirable to evaluate performance after a few results, after campaign, etc.; however, this is not always practical or necessary The longer the time between evaluation, the more difficult to uncover sources of variability What is the performance of the quality attribute? How capable is it to meet specification? What is the state of control? Don t be constrained by a poorly designed business process, such as complex reporting and approvals This is re-evaluated on an ongoing basis. If the state of control changes, adjustments to the CPV plan can be made based on a change in risk 05 Arlenda

69 Process Capability Indices, Cpk and Ppk LSL USL 3 Cpk Ppk USL x x LSL Min or Arlenda C pk uses a short term estimate of σ, based on the average moving range of the sample P pk uses a long term estimate of σ (typical calculation)

70 Cpk vs. Ppk Short Term vs. Long Term Individual Value Individual Value I Chart of Content Uniformity UCL=0.8 _ X= LCL= Observation I Chart of Acceptance Value UCL= _ X= Observation LCL= Arlenda

71 Cpk vs. Ppk Which should you use? Ppk accounts for shifts in the mean that naturally occur over time. Cpk reflects what the capability could be without the shifts Indeed, you would like the two to be equal; however, if your process is highly capable, it does not serve the interest of the business or patient to identify and eliminate the root cause of every shift in mean. Prioritization and action must be risk based The difference between the two metrics can provide insight into your process, that is, the influence from sources of variability, such as raw materials, campaigns, equipment, analytical campaigns. 05 Arlenda

72 Process Capability Notes Process will often be out of statistical control because of nonindependence If data are non-normal, evaluate practical effect on interpretation use of capability (is risk of failure more or less than what is expected from reported capability index) Be wary of combining within and between batch variance components Other statistical methods can provide likelihood of failure, e.g., Bayesian methods. 05 Arlenda

73 Individual Value It s about the Data! The Process! The Patient! Process Capability of Concentration Process Data LSL * Target * USL 3000 Sample Mean 89.3 Sample N 0 StDev (Within) StDev (O v erall) USL Within Overall Potential (Within) C apability Cp * C PL * C PU.08 C pk.08 O v erall C apability Pp * PPL * PPU 0.9 Ppk 0.9 C pm * Cpk/Ppk are borderline for enhanced monitoring. Should you use Cpk? And the data aren t normal. Maybe if the data were transformed O bserv ed Performance PPM < LSL * PPM > USL 0.00 PPM Total 0.00 Exp. Within Performance PPM < LSL * PPM > USL PPM Total Exp. O v erall Performance PPM < LSL * PPM > USL PPM Total I Chart of Concentration UCL=939 It s about the data And the data is about the process _ X=89 And the process is about the patient/consumer Observation LCL= Arlenda

74 CPV Summary Ongoing assurance is gained during routine production that the process remains in a state of control Holistic picture of what, when, who and how Nature of the data and the State of Control Risk Based Approach to CPV Mindset Change Red is the New Black It s about the data, the data is about the process, and the process is about the patient 05 Arlenda

75