Common Issues in Qualification and Validation of Analytical Procedures

Transcription

1 Common Issues in Qualification and Validation of Analytical Procedures Alexey Khrenov, PhD OTAT/CBER/FDA CMC Strategy Forum January 29, Washington, DC

2 Disclaimer These comments are an informal communication and represent my own best judgment. These comments do not bind or obligate FDA. 2

3 Purpose of Qualification/Validation Qualification/validation is the process of demonstrating that the analytical procedure is suitable for its intended use. What does suitable for its intended use mean for regulatory reviewers? Can we (and you) trust your past and future data? 3

4 Validation vs. Qualification Qualification Confirm that the analytical procedure (as performed) is suitable for the specified purpose, and the results are reliable and reproducible. Analytical procedures must be qualified, but specifics of qualification are at the company s discretion based on scientific judgement. Qualification reports are not expected, but the company may be requested to provide data to prove that the methods are sufficiently qualified. Validation Can be performed after gaining sufficient knowledge of the analytical procedure. Formal exercise to determine the ability of the method to meet the acceptance criteria pre-defined in the validation protocol for each of the test parameters. Validated analytical procedure must be described in sufficient details for it to be performed in FDA laboratory, and produce results comparable to those obtained by the company. Validation reports must be included in the BLA/NDA. 4

5 Validation vs. Qualification (contd.) Qualified analytical procedures should always be used in the regulated environment (IND/BLA stage). Coordination of the development of analytical methods, manufacturing process, and product is required Analytical procedure may need to be re-qualified or re-validated if the acceptance criteria or matrix changes. If the analytical procedure is changed, bridging study needs to be performed. Do NOT change the manufacturing process and analytical procedure at the same time! Commercial process must be supported by fully validated analytical procedures, i.e., the analytical procedures must be validated BEFORE the commencement of the PPQ campaign. Analytical procedures used in clinical trials must be validated. 5

6 Risks of Inadequate Validation Validation failures result in additional development work and delay regulatory submission. Data excluded from review of license application The later the procedure is validated, the greater the risk. Risk mitigation strategy Define and qualify the performance characteristics of the analytical procedure that are critical to its intended use from the start. Develop system suitability procedures and criteria. Monitor and trend performance of analytical procedure. Keep gaining knowledge about capabilities of analytical procedures. Validate the procedure early in product development. 6

7 Analytical procedure lifecycle Development Initial assessment of the suitability of analytical procedure Optimization of analytical procedure details Qualification Assessment of the performance characteristics of analytical procedure Assessment of the capability of analytical procedure to support specification Gaining knowledge for setting final specification acceptance criteria and design of validation protocol Performance characteristics are inadequate to control the product and the manufacturing process. Validation Confirmation that the performance characteristics of analytical procedure are established correctly. Confirmation that analytical procedure is capable to support specification acceptance criteria. Failure to meet validation acceptance criteria Changes in specification Changes in the process or product Modification of the analytical procedure 7

8 Considerations for Setting Specifications Risk assessment Process capability Clinical safety /efficacy Analytical capability Acceptance criteria for different specification parameters may be limited by different factors. Acceptance criteria are frequently adjusted when more data become available. Changes in the manufacturing process/ analytical procedures require re-assessment of the acceptance criteria. Sound statistical approaches should be used where possible. Data to be included in the justification of specification should be carefully selected based on their relevance to the commercial manufacturing process. 8

9 Full Validation Validation characteristics Accuracy Precision Repeatability Intermediate Precision Specificity Detection Limit Quantitation Limit Linearity Range Robustness Additionally, for crossvalidation Comparability 9

10 Validation Acceptance Criteria Should be based on intended use, knowledge of procedure performance, and capability to confirm the suitability for its intended use. Should be specific for the analytical procedure. Should be specific for the validation parameter. 10

11 The Big Problem We must do validation because FDA wants validation report Leads to Poor design of the validation study Lack of scientific assessment of data Disconnect between validation and intended use of the method 11

12 Acceptance criteria issues Acceptance criteria (AC) are too wide Risk: Analytical procedure may be found unsuitable and data excluded from review. Example 1 Actual variation of peak position was 1/200 of AC AC for peak position Actual RSD% for peak area was 1/25 of AC Anything will PASS! Review of validation is based on the intended use of the analytical procedure. Meeting an irrelevant AC does not confirm suitability. 12

13 Acceptance criteria issues (contd.) Same AC are used for multiple validation parameters and methods Example 2 1/6 of the specification range was used as the range where the results should fall for Precision validation. The same value was then used to calculate the AC for standard deviation (SD) when validating the Accuracy, Repeatability, and Intermediate Precision for multiple methods. As a result, an acceptable SD equal to 1/6 of specification range (± 3 SD covered the whole specification range) was set as the AC for multiple methods and parameters. As the approach was defined in the SOP, it resulted in a Form 483 Observation during the facility inspection. 13

14 Typical issues: Linearity Lack of visual inspection and qualitative assessment of the linearity data. Risk: Analytical procedure may be found unsuitable and data excluded from review. Example 3 No evidence that visual inspection was performed of a plot. Data show a loss of sensitivity at the lower limit of range. Procedure was re-validated. Linearity should also be evaluated by visual inspection of a plot of signals as a function of analyte concentration or content. A plot of the data should be included. (ICH Q2) 14

15 Typical issues: Range Validated range does not cover 80% - 120% of specification range. Risk: Re-validation of the procedure may be required, part of data may be excluded from review. Example 4 During validation, the matrix containing the analyte was spiked with analyte solution. The analyte in the matrix was not taken into account, and the low part of the range was not validated as intended. Supplemental validation was performed. 15

16 Example 5 Typical issues: Range (contd.) During validation, the dilution scheme of the standard solution used for validation was incorrect, and an incorrect range was validated. Supplemental validation was performed. Example 6 After change of specification acceptance criteria, the validated range became insufficient to support specification. The new range was not revalidated before the submission of the license application. The analytical procedure was re-validated during application review. 16

17 Typical issues: Specificity Specificity is validated by showing lack of response to unrelated or irrelevant substances. Risk: Procedure may be found not specific enough to distinguish between the target analyte and related substances potentially present. Assay may be deemed unsuitable and data may be excluded from review. 17

18 Typical issues: Accuracy Accuracy was not validated over the entire range of the analytical procedure. Risk: Results at the edges of the range may not be accurate, leading to false OOS results. Accuracy is validated using the number of repeats different from the routine procedure (for the methods using replicates). Risk: Accuracy of the assay in general is overestimated. 18

19 Typical issues: Accuracy (Contd.) Accuracy is calculated incorrectly. Risk: Validation exercise may fail acceptance criteria. Example 7 For the validation of accuracy, different volumes of the standard solution were spiked into the matrix with fixed concentration of analyte to obtain fixed final volume. When the expected amounts of analyte were calculated, the changes in the amount of analyte coming from the variable volumes of the matrix were not taken into account. As a result, accuracy was underestimated. Results were re-calculated and the validation report was revised. 19

20 Typical issues: Precision For multi-step procedures variability only for final step is assessed. Risk: Precision of the procedure may be overestimated. Example 8 For the validation of a procedure involving complex sample preparation, including chromatographic separation of sampled fractions, followed by separate analysis of each fraction, only the data regarding the precision of final analysis were included. Data were generated outside of the validation study, and only provided in response to an Information Request. 20

21 Typical issues: Precision (contd.) Validation study design and AC do not allow for identification and control of source of variability, and establishment of suitability for intended use. Risk: Unsuitable procedure may be validated. Example 9 Intermediate Precision validation was designed to have two data sets: (Analyst 1, Day 1, Instrument 1) and (Analyst 2, Day 2, Instrument 2). The single AC for SD was set at 5.8, applicable both to each set and combined set. Specification limit was set at 15. SDs of 0.6 and 0.8 was obtained for each set, and an SD of 3 for the combined set. The difference of means between the sets was ~5. OOS/OOT results were observed in routine testing and attributed to the variability of the test. The procedure was found to be unsuitable for its purpose due to low Intermediate Precision. Company performed investigation, and found the root cause for the high variability. The procedure was modified and re-validated. 21

22 Typical issues: Robustness Parameters used to validate robustness are not relevant or set of parameters is not complete. Risk: Robustness of the procedure is not established and adequate control are not established to control for potential causes of variability. Example 10 During the validation of robustness, only the effect of the concentration of reagents was assessed. The method is known to be sensitive to temperature and time, etc. The procedure was re-validated. 22

23 Typical issues: Robustness Example 11 During the validation of robustness for test method using a commercially available kit, only 2 lots of the kit were used, and it was concluded that it is not a factor affecting assay robustness. The procedure was re-validated. 23

24 Typical issues: Comparability Acceptance criteria and statistical analysis are inappropriate to establish comparability. Risk: New procedure may produce significantly different results and may not be able to support specification. Example 12 The company proposed to use a new method to test an existing specification parameter. The company also proposed to maintain the existing acceptance criteria. Comparability was established based on the mean difference of 5.8% from testing 6 samples by both methods, meeting the acceptance criterion of < 6%. During review, the paired t- test was applied to the data, which produced a p-value of less than The supplement was withdrawn. 24

25 Special Considerations for Reference Material Based Assays Manufacturers are recommended to develop in-house reference material(s) early in the development cycle to ensure reliability of test results. Assay should be validated using the same reference material as used in routine testing. In case of a change in the reference material, calibration of the old material against the new one must be performed. Commercially available critical reagents should be qualified using the in-house material. Use of reference materials provided with commercially available kits is discouraged because their quality cannot be assured. 25

26 Example 13 Data Integrity Issues Bridging study was performed for two methods, and the data were submitted. The company stated that similar profile was obtained. Upon review, it was found that the data for the new method were not fitted correctly in Excel. 26

27 Data Integrity Issues (contd.) Example 14 A new procedure was submitted for approval along with the validation summary. The summary stated that validation was successful with one minor deviation. The summary included raw data for all parameters except Intermediate Precision. The original validation report was requested from the company. During review of report, it was found that: Accuracy validation failed, supposedly because a wrong concentration of the stock solution was used. The data were then recalculated using the correct concentration to obtain acceptable results. Intermediate Precision validation failed significantly. Acceptance criteria were met by using incorrect datasets in the calculation of statistical parameters. The analytical procedure was deemed unsuitable for its intended purpose. 27

28 Thank you!