Recommendations for the development and validation of confirmatory anti-drug antibody assays

Transcription

1 Themed Issue: Antibody Drug Conjugates For reprint orders, please contact Recommendations for the development and validation of confirmatory anti-drug antibody assays Identification and characterization of anti-drug antibodies is a critical component of biopharmaceutical drug development. The tiered approach for immunogenicity testing consists of screening, confirmatory, and characterization assays. Herein, we provide recommendations for confirmatory assays by expanding upon published guidance and present common practices across the industry. The authors recommend scientific approaches for development and validation of confirmatory assays using competition methods in ligand-binding assays, along with statistical formulae for routine use and validation. The paper will assist in understanding the confirmatory assay, and carefully implementing validation criteria a priori, as well as during sample analysis. These approaches represent the authors current knowledge and practices, with the aim that more uniform practices will be applied across the industry. Efficacy of biotherapeutics has been successfully demonstrated in many indications because of their excellent targeting ability. However, anti-drug antibodies (ADA) can impact drug exposure by affecting the ability of the drug to reach the intended target, alter the PK profile, and potentially mediate serious adverse effects. Based on the information gained through the onset, magnitude and incidence of drug-specific antibody formation, and the subsequent impact on efficacy, investigators may better anticipate antibody-mediated events, and effectively manage patient safety [1]. Identification and characterization of ADA is a critical component of biopharmaceutical drug development [2]. The currently recommended threetiered approach for immunogenicity testing consists of screening, confirmatory, and characterization assays [3,4]. It has been recommended that screening assays should be intentionally designed to deliver a demonstrable false-positive rate (screen positive and confirm negative) of approximately 5%, to decrease the assignment of false negatives [5,6]. Some publications and guidance documents provide insights into the development and validation of the screening assay [3 6]; however, limited guidance regarding the development and validation of confirmatory assays is available [6 8]. In addition, recent debates regarding orthogonality of competition confirmatory assays and false-positive rates [9 12] have fostered further discussions. Thus, it is evident that industry consensus is still evolving, which supports the need for further alignment on ADA confirmation practices. A variety of methods can be adopted for the confirmatory assay such as immunoprecipitation, immunodepletion [13], or most commonly, specific inhibition demonstrated in the presence of an excess of therapeutic. Competition is the general term applied when exogenous drug is added to a sample that contains ADA to the drug of interest. The exogenous drug competes for binding to ADA with the labeled drug(s) used to capture (direct assay), or to capture and detect (bridging assay). The exogenous drug is added, and assay signal inhibition is evaluated by evaluating a sample in the presence versus the absence of excess amounts of competing drug. Several strategies for confirming a positive response using the competition method have been discussed in the literature, ranging from an arbitrary assignment of a Darshana Jani*,1, Robin Marsden 2, Alvydas Mikulskis 3, Carol Gleason 4, Thomas Klem 5, Corinna Krinos Fiorotti 1, Heather Myler 4, Lin Yang 6 & Michele Fiscella 6 1 Pfizer Inc., One Burtt Road, Andover, MA 01810, USA 2 La Jolla Pharmaceutical Company, Telesis Court, 6th Floor, San Diego, CA 92121, USA 3 Biogen Idec, Inc., 125 Broadway, Boston, MA 02142, USA 4 Bristol-Myers Squibb, Route 206 & Province Line Road, Princeton, NJ 08540, USA 5 Shire, 300 Shire Way, Lexington, MA 02421, USA 6 Covance, Inc., 3635 Concorde Parkway, Suite 100 Chantilly, VA 20151, USA *Author for correspondence: darshana.jani@pfizer.com part of /BIO Future Science Ltd Bioanalysis (2015) 7(13), ISSN

2 Jani, Marsden, Mikulskis et al. Key terms Anti-drug antibody: Antibody response specific to drug of interest. Confirmatory assay: Assay used to confirm specificity of antibody response, typically via the addition of exogenous drug. Ligand-binding assay: Plate-based assay utilizing binding between antibody and respective ligand to generate signal. minimum of 50% inhibition, to widely accepted and applied complex statistical methods using the variation of percent inhibitions from a therapeutic-naïve population or a therapeutic-naïve population spiked with low levels of ADA [6,7,9]. The development and validation strategies as well as the terminology presented in this paper focus on the competition approach in ligand-binding assays because it is the most widely used [7]. Other methods may be used if scientifically justified. The selection of an appropriate method of confirmatory analysis is dependent on a variety of factors including the expected in vivo concentrations of a therapeutic, drug tolerance of an assay, the likelihood of specific versus nonspecific interference inherent to the biological system [7,14], and known attributes of a therapeutic. The purpose of this publication is to provide an overview of development and validation of confirmatory assays along with the recommendations based on the authors collective scientific and technical experience on some key aspects of ADA confirmation. Three case studies demonstrating immunogenicity results are presented to support conclusions. General considerations for confirmatory assays The goal of the confirmatory assay is to confirm that therapeutic-specific antibodies are present, and reduce or eliminate the chance that something other than ADA is generating a false-positive result. In the following section, we review some of the standard approaches for assay selection and format, effect of matrix proteins, reagent selection, acid dissociation and other relevant parameters typically considered for the confirmatory assay. The scientist(s) should carefully consider all key parameters before starting assay development experiments. Assay selection An advantage to using a confirmatory assay founded on the screening assay is the greater likelihood of similar relative sensitivities between the two methods [8]. However, certain factors should be considered for this methodology, for example, the amount of therapeutic required for effective competition in samples containing high titers of pentavalent IgM may be significantly greater than that needed for samples comprised primarily of bivalent immunoglobulins [9,15]. Another method demonstrating the specificity of binding in the screening assay is based on the use of immunodepletion via the removal of immunoglobulins by affinity purification using protein A-, G- or L-labeled beads. A potential complication of using this method is denaturation of antibodies or therapeutics in the sample as incubations with extreme ph or salt and detergent concentrations may be required. Yet another method to consider is the use of secondary reagents to confirm results using surface plasmon resonance. This method can be used not only to screen and to confirm the results, but can also provide information about antibody isotype [16,17]. The assay platform (e.g., ELISA and Electrochemiluminescence ECL) often impacts the choice of assay format. For example, using a shared platform for the screening and confirmatory assays allows verification of the false-positive rate. Competition assays When designing the confirmatory assay, the format selected will dictate the means by which confirmation will be demonstrated. The bridging assay takes advantage of the selectivity inherent to antibodies, providing the capability to detect all isotypes, and avoiding the need for species-specific detection reagents. Table 1 demonstrates the relationship between the amount of positive control (PC) ADA present and the degree of inhibition possible in a bridging assay format. Three samples prepared by independently spiking PC ADA at high concentrations in matrix were serially diluted to just below the level of detection for the assay. Samples were diluted into buffer with and without the required concentration of exogenous drug (4 μg/ml final concentration). Signal was measured for each condition, and percent inhibition calculated. In Figure 1, confirmation of positive samples is easy to demonstrate in samples with greater amounts of antibody, where there is a larger magnitude of inhibition possible. In the case where there are smaller amounts of antibody present and therefore reduced potential for signal inhibition, confirmation of a positive can be more difficult. In addition, in the bridging assay format, there is a further challenge because of very low background values. The small magnitude of inhibition contributes to significant variation, as evident in the increasing percent coefficient of variation with the decreasing levels of ADA in Table 1. Some laboratories prefer to investigate inhibition in the naive population, as well as the naïve population spiked with either a series of ADA concentrations or a low level ADA spike, to better assess the degree of magnitude required for 1620 Bioanalysis (2015) 7(13)

3 Recommendations for the development & validation of confirmatory anti-drug antibody assays White Paper Table 1. Relationship of antibody concentration and percent inhibition. PAb conc. (ng/ml) No excess drug added Excess drug added %Inh Replicate mean RLU (n = 3) Replicate %CV Replicate mean RLU (n = 3) Replicate %CV Mean replicate (n = 2 wells) RLUs for antibody spiked into pooled serum at the concentration noted in the absence and the presence of excess drug at a final concentration of 4 μg/ml. %Inh was calculated using the following equation: %Inh = 100 ([mean RLU without drug - mean RLU with drug )/RLU without drug ]). Approximate sensitivity. Inh: Inhibition; RLU: Relative luminescence units. confirmation of a putative positive with low levels of ADA present as previously noted [5]. As there is no guidance on the amount of antibody used to target the correct level of reduction in signal needed to confirm a positive result, and because of the broad spectrum of antibody responses to a therapeutic, it is important to take care in implementing this method. An increased likelihood of assignment of a false negative status is possible if the estimated percent inhibition required for confirmation is too high. Therefore, one should determine the amount of antibody required to generate a reproducible reduction in signal near the level of assay sensitivity, or identify appropriate ranges of inhibition corresponding to a variety of antibody concentrations expected due to the heterogeneous nature of antibody response. Acid dissociation For both direct and bridging assays, an acid dissociation step may be used (ideally complemented by other methods) [15] when high drug concentrations are present in the samples. The acid dissociation steps that have been described generally fall into two major formats: the antibodies are captured from the sample prior to any acid dissociation, or the complexes are dissociated from ADA in a single acid dissociation step. The acid dissociation steps may pose a challenge in designing the confirmatory step for competition assays, as the acidification frees the therapeutic from the antibody. It is recommended that in a competitive assay format, the exogenous drug is added to the sample subsequent to neutralization and prior to the capture step, so that the exogenous drug has the opportunity to bind to the antibody, in order to demonstrate the required reduction in signal. If the intent is to detect low-affinity antibodies, other platforms such as surface plasmon resonance may be more appropriate [15]. Matrix proteins Confirmation must be able to define a sample as truly positive in representative matrix. Assay specificity is vigorously tested during screening assay development, however, this should also be evaluated during confirmation step to eliminate reporting of false-positive results [18]. The nature and binding properties of assay reagents also play a vital role in assay performance. Matrix proteins including, but not limited to, endogenous targets, polyreactive antibodies, soluble receptor, shed targets, binding proteins and rheumatoid factor, should be considered as potential interfering compounds that can produce an additive or subtractive effect, depending on their potential for binding to the PC antibody or the therapeutic. For example, angiopoietin (the drug target) competed with the binding of ADAs to the AMG386 fusion protein (drug), which yielded false-positive results. The interference was eliminated by incubating the samples with a high affinity monoclonal antibody to the epitope on angiopoietin that competed with the AMG386 binding to the ADA [19]. It is recommended that the effect of potentially interfering compounds be examined at progressively increasing concentrations including, and in excess of, the expected physiological concentrations prior to the addition of exogenous drug, to ensure reproducibility of the selected assay system. A careful evaluation of a large number of target samples is beneficial to predict performance of assay during validation and sample testing. It is expected that if significant matrix interference is observed, it should be mitigated at the screening tier stage by sample dilution, or modification of assay parameters. If interference persists and affects the confirmation tier, the assay format and reagents should be critically examined and changed. The readers are encouraged to refer to several industry papers to overcome matrix interferences in protein based assays [20,21]

4 Jani, Marsden, Mikulskis et al. Total inhibition (%) Antibody concentration (ng/ml) %Inh Linear (%Inh) 100 ng/ml 50 ng/ml 25 ng/ml Figure 1. Relationship between antibody concentration and reduction in signal via competition. Inh: Inhibition. Drug tolerance during development While acknowledging that state-of-the-art methods have the potential for much greater sensitivity, it has been suggested that the confirmatory assay be able to confirm at least 250 ng/ml of ADA in the presence of the drug concentration expected at the sample collection points, and not just in its absence [22]. However, one needs to keep in mind the limitations of this concept, as the PCs used to establish drug tolerance likely will not be representative of antibodies in the samples, as discussed elsewhere [17,23]. Reagent selection The availability of quality and consistently performing reagents is a crucial part of assay selection. The characteristics and properties of the drug should be understood as much as possible to ensure reliable assay results. Because of the relationship in binding between the drug and the PCs antibody, lot-tolot variation also needs to be addressed for the PC selected. In the case of significant lot-to-lot variation, the addition of excess drug may result in varying degrees of inhibition for a competition assay. Determination of the amount of drug required to inhibit signal to the extent required is largely based upon the immunoreactivity of the selected PCs, which may vary widely from source to source, and from antibody to antibody. With this caveat in mind, attempts should be made to use controls more representative of the samples [1,24]. Levels of inhibition should be examined at progressively increasing concentrations of exogenous drug. The results from each these assessments will help determine what constitutes a required excess of drug. Once this concentration has been determined, the method selected will dictate the manner and the step at which the excess drug is added to the system. Method development Optimization of assay conditions Once the assay has been proven feasible, it is recommended that optimization experiments be executed using a multifactorial approach if only a few factors need to be optimized. A formal design of experiment (DOE) is recommended if many factors need to be optimized [25,26]. DOE should be able to address the impact of changes for interdependent factors (e.g., incubation temperature, time and reagent concentration); however, well-planned experiments are necessary to guide the confirmatory conditions that will indicate the potential limit of detection, but also reproducibility on a day-to-day basis, regardless of the analyst or equipment used in the experiment. Interested researchers are directed to consult other publications for application of DOE to ADA assays specifically, and bioanalysis in general [27,28]. Experiments with naïve target matrix and a low concentration of ADA should be run on the entire assay plate to identify potential hot spots or positional effects in the assay. For example, a 96-well plate may be subject to edge effects, whereas samples from an assay that are read sequentially (e.g., Biacore ) may be subject to drift in results between samples at the start of the cycle and at the end of the cycle, which need to be addressed prior to sample analysis. Assay controls For confirmatory assays, US FDA s current Draft Assay Development for Immunogenicity Testing of Therapeutic Proteins Guidance recommends demonstrating that signal for the high, medium and low positive control (HPC, MPC, LPC, respectively) signals can be effectively reduced beyond the assigned confirmatory cut point (CCP) threshold in the presence of a competing drug. The PC may be titered to select appropriate control concentrations. The final HPC, MPC and LPC are identified during validation for sample analysis. There is no alignment in the industry on the use of MPC for screening or confirmatory assays. However, it is more common that only HPC and LPC are used. Pre-validation preparation Once the final conditions have been determined during development, a final set of experiments are suggested to verify that the confirmatory assay is suitable for validation or reagent qualifications. The minimum prerequisites for validation are a draft method, single-use vials of controls and critical reagents, dedicated lots of critical materials (e.g., plates, chips and disk), trained analyst(s), validation protocol or validation plan and calibrated instruments with CFR Title 21, Part 11 validated software Bioanalysis (2015) 7(13)

5 Recommendations for the development & validation of confirmatory anti-drug antibody assays White Paper Validation of confirmatory assay Confirmatory assay validation is typically performed in parallel with the validation of a screening assay, as previously discussed in various forums [2,6,7]. These are general recommendations that should be applicable to a majority of the ADA confirmatory assay formats. The screening assay controls can be adapted for use in the confirmatory assay to ensure that assay performance meets both screening and confirmatory assay acceptance criteria. Assay controls must be prepared and tested in an assay in the same way as the test samples. Similar to screening assay validation, the following parameters are recommended for confirmatory assay validation: CCP; Assay selectivity and/or matrix interference; Assay specificity (generally not evaluated, but may be assessed in some special cases, e.g., multidomain proteins); Inter- and intra-assay precision of assay endpoints; Robustness. CCP determination The CCP is typically established at the beginning of assay validation. It is essential that the cut point be determined in the laboratory in which sample testing will be performed unless appropriate assay robustness/ ruggedness involving inter-laboratory assay validation assessments are tested. Identifying a CCP is an important step to report a subject ADA positive during clinical study. It is important to select a statistically justified threshold to determine the degree of signal change considering the variability of the target population to build confidence in data reporting. As the ADA methods are quasi-quantitative, the use of a threshold becomes necessary to discriminate between ADA positive and negative samples. The following sections provide guidance on population selection and proposed methods when working on several disease indications. Selection of populations Samples used in cut point evaluations (both screening and confirmatory) should be drug naïve and representative of the intended target disease population. However, it may be challenging to obtain sufficient numbers of relevant samples for some rare diseases in which healthy-volunteer samples may be used, especially for Phase 1 drug development programs. The guidance for selecting samples for CCP determination has been briefly described in prior publications [6,9,29] and will be further reviewed in the case study section. Key term Cut point: Value at or above which a sample is considered positive. When working with samples from several populations, it is recommended that the percent inhibition values be evaluated to determine if it is appropriate to combine responses across populations to produce an overall CCP. Use of descriptive statistics (mean, variance or SD, and ranges) and graphical techniques such as histograms and box plots can provide sufficient justification for combining responses. Figure 2 shows a histogram overlaying the frequency of percent inhibition values from samples from four different oncology disease states (n = 10/disease state); included is a table of summary statistics for each disease state. The table in Figure 2 shows that the means and SDs for the percent inhibitions from the four different disease states are similar based on the range of means ( ), SDs ( ), with overlapping minimum and maximum values across the four indications. The histogram shows the degree of overlap in the four sets of percent inhibitions, and shows that normality can be achieved for the combined distribution, an assumption that can be further justified using standard tests for normality available in most commercial software packages. Justification of the normality assumption is needed for subsequent calculation of a combined cut point. If the means and/or variances are found to be markedly different, a separate CCP for each indication will be required. With small sample sizes (e.g., n = 10/indication), graphical displays are helpful in showing the differences. If additional justification for combining samples is needed, analysis of variance tests can be utilized to confirm visual differences. It is equally important to ensure that the selected samples do not contain pre-existing antibodies and are negative for ADA, as even the drug-naïve samples may produce a positive result in the screening assay. In the past, it was recommended that if there is a low incidence of ADA-specific positives in the drugnaïve population, they may be removed as biological outliers from the CCP calculation [7]. The number of biological outliers removed should be carefully monitored as removal of too many samples will result in a poor approximation of the CCP [30]. Since 50 samples are recommended for cut point determination, additional samples (e.g., 10 samples, for n = 60) should be assayed to account for loss of those identified as biological outliers, so that at least 50 samples are utilized for the CCP determination. Further discussion of handling of outliers is provided in the next section. In the extreme situation where a large (e.g., approximately >30%) of the population

6 Jani, Marsden, Mikulskis et al. 8 6 n = 10 donors/population Disease #1 Disease #2 Disease #3 Disease #4 Frequency Percent inhibition: 100 (RLU buffer RLU spike )/RLU buffer Figure 2. Degree of overlap among the four indications with a unimodal distribution across indications, and a distribution that is roughly normal in shape. RLU: Relative luminescence units. is positive, alternative confirmatory approaches may be needed. Sample size for CCP determination It has been recommended that at least 50 samples be utilized to set the CCP [5,6,15]; however, this recommendation will be revisited in the case study section. For some disease states like oncology conditions, it may be impractical to obtain a minimum of 50 samples; in these cases, the data for various indications could be compared with the data from a larger population where a CCP was determined (e.g., normal sera), to see if the CCP would be appropriate for the targeted population. If not, a preliminary CCP can be calculated from the available data for the disease-state population, and can be re-assessed when actual patient pretreatment samples are available from an ongoing clinical trial. Wakshull and Coleman [9] put forth the argument that, while the screening samples should be assayed over multiple days, it is not necessary to repeat the analysis of the drug-spiked samples on more than one occasion for determination of the CCP. The argument suggests that by processing samples both in the absence and presence of drug on the same plate, the sample results are normalized for individual differences arising from both underlying biology as well as intrinsic experimental artifacts, leaving one with relatively smaller well-to-well variation. However, it is common occurrence that the percent inhibition results vary between analysts, days, different runs, etc. and it is recommended that samples from target population be assayed over multiple plates, days, and by at least two analysts, which allows for attribution of sources of variability due to analysts, experimental days and/or samples. Readers are encouraged to use Table 1 in Shankar et al. [6] for an example of a balanced experimental design. Statistical considerations for CCP determination Calculation of the CCP generally uses the percent inhibition for the samples processed with and without excess exogenous drug in wells on the same plate: Responsewithoutdrug Responsewithdrug %Inhibition = 100 # Responsewithoutdrug Equation 1 Prior to calculation of the cut point, the data should be reviewed for the presence of outliers and normality 1624 Bioanalysis (2015) 7(13)

7 Recommendations for the development & validation of confirmatory anti-drug antibody assays White Paper of the resulting distribution. If the sample sizes are large enough (e.g., n 50), standard tests for normality available in most software packages can be utilized. For small sample sizes, inspection of the data distribution using a histogram can provide graphical assurance that normality is a viable assumption. The recommended method for assessment of outliers is to use the graphical tool known as a box plot [30]. Analysis of variance models can also be used to identify outliers through examination of the residuals from the model. Figure 3 provides a flow chart for the process of determining the CCP. As a risk-based approach is commonly accepted industry-wide, the practice of targeting an approximate 5% screening false-positive rate is commonly implemented. The samples that are truly positive and reactive to the drug should be reported in the confirmatory assay. Based on the immunogenicity risk assessment [2], a false-positive rate of 1% is typically used for the purpose of establishing a CCP; there are cases where a 0.1% false-positive rate is desired. Once outliers have been removed, the CCP is calculated as: CCP = Mean%Inhibition + zsd%inhibition (whenusing naivedrug population) Equation 2 In the equation above, Mean %Inhibition is the mean of the percent inhibitions; z is the z-score from the standard normal distribution corresponding to the upper 1 or 0.1% tail area under the normal curve targeting the 1% false-positive rate (z = 2.33), or upper 0.1% false-positive rate (z = 3.09) and SD %Inhibition is the estimate of the standard deviation for the distribution of percent inhibition values. If specialized statistical tools are not available, the following EXCEL formula can be used to calculate the upper 99th percentile on the standard normal distribution: Normlnv(0.99% Mean %Inhibition, SD %Inhibition) Equation 3 In the case that the population is not normally distributed, log conversion of the raw signal response ratio can be used in an effort to improve normality of the distribution: The transform is log10ratio a Responsewith drug Responsewithout drug Equation 4 Once transformed, the mean and SD are determined in the same manner as that described above and the k CCP is calculated from the antilog of the Mean log10ratio minus the desired SD log10ratio : ^ CCP = Meanlog10ratio-zSDlog10ratio Equation 5 In cases when the distribution of percent inhibition values in drug-naïve samples may not be reliable for data modeling, either due to pre-existing drug-specific interactions in drug-naïve samples, or due to persistent analyst-dependent differences in confirmatory assay results, it is recommended to use the response from baseline samples to correct for these variance components to provide accurate assessment of ADA responses in patient samples [32]. As a final comment, we suggest that the scientist keeps in mind that determining the CCP is a difficult statistical task when based on a small subset of a patient population. Therefore, there may be a need to re-evaluate the cut points once additional data become available from larger clinical trials. Intra- & inter-assay precision of assay outputs PCs are typically used in assessing the intra- and interassay precision. Assessment of precision using the pooled negative control does not need to be tested in the confirmatory assay. To determine confirmatory inter-assay precision, it is recommended that the percent inhibition for at least three sets of PCs (at HPC and LPC concentration) should be analyzed in different locations on a plate, on at least six plates, run independently by at least two analysts on multiple days. Loading precision samples in random plate locations captures the variability that is likely to be observed during in-study sample testing. Each control sample within a set (both competed and non-competed controls) should be analyzed in the same number of wells as test samples (typically duplicate). Accurate assessment of intra-assay precision using six or more sets of controls per plate is recommended [5,6,28]. Fewer than the minimum three sets on each plate may be used but will require more plate runs to assess the intra-assay precision. Calculating precision estimates of percent inhibition The method outlined in DeSilva et al. [28] is recommended for precision assessments for both screening and confirmatory control samples. The formulae can be easily utilized as these calculations are based on a simple one-way analysis of variance (ANOVA) model for partitioning variance due to inter- and intraassay variation. The resulting variance estimates are h

8 Jani, Marsden, Mikulskis et al. Calculation of %Inhibition 100 ([Response w/o drug Response w/ drug ]/Response w/o drug ) If appropriate, remove biological outliers Remove outliers (box and whiskers plot) Graphically assess distribution Not normally distributed Normally distributed Use appropriate transformation to normality Calculate CCP as mean + z SD where z = 2.33 (99%) Figure 3. Flow chart for confirmatory cut point determination. CCP: Confirmatory cut point. expressed as %CV: 100 SQRT (variance)/mean. If other sources of variation are of interest, (e.g., analysts, days, and plates), the ANOVA model can be expanded to include these additional variables. It is recommended to seek assistance of a statistician for fitting such higher order models [31]. Assay selectivity & matrix interference Selectivity and/or matrix interference in a confirmatory assay should be assessed by evaluating individual matrix samples from the target population. It is recommended to test a minimum of 10 individual matrix samples spiked at LPC/HPC concentrations. Samples are diluted to the minimum required dilution in assay diluent prepared with or without excess exogenous drug. The mean percent inhibition of the samples should be CCP and the percent CV should be within the variability of the assay. No further analysis of selectivity is warranted if the CCP had been determined using samples spiked with the PC at the sensitivity level. A procedure should be drafted to disqualify the samples prior to initiating the study sample testing if potentially interfering compounds are identified during validation. Assay specificity In general, specificity analysis performed to identify potential interference due to recognition and binding of related or similar analytes during the screening assay development is sufficient, and no further validation experiments are recommended for confirmatory assay. Robustness Robustness is typically evaluated by comparing the impact of small but deliberate changes to the PCs related to a primary variable typical of the confirmatory assay; for example, pre-incubation time with excess drug. Other parameters like reagent preparation and plate readers are selected as a part of screening assay robustness, and not necessarily repeated for the confirmatory assay. In-study cut point considerations Depending on the assay format and drug concentration in the samples, if an approximate targeted 5% falsepositive rate during screening is not obtained [33] or if the applicability of the validation CCP to the subject population is in question, subset of the pretreatment 1626 Bioanalysis (2015) 7(13)

9 Recommendations for the development & validation of confirmatory anti-drug antibody assays White Paper samples from subjects participating in the study can be used to assess an in-study CCP. In smaller studies with fewer subjects (e.g., 100), all pretreatment samples may be chosen for evaluation of the in-study CCP. In larger studies with 100 subjects, pretreatment samples should be chosen randomly [26]. In both small and large studies, it is recommended that the in-study CCP evaluation results be decoupled from the study results to avoid potential complications such as having a screening-negative sample that confirms positive during the in-study CCP evaluation. Decoupling of results can be achieved by physically aliquoting the selected in-study CCP evaluation samples in a blinded fashion and generating a new sample set that will be used solely for in-study CCP evaluations, and are not traceable to the source study samples. From this point on, the evaluation of cut point may proceed as described earlier. Analytical interpretation of sample results If the sample result from the screening assay is equal to or above the cut point, regardless of the presence of circulating drug, the sample is assigned a putative positive status, and must be examined in the confirmatory step. If the percent inhibition of a putative positive sample is greater than or equal to the CCP, the status of the sample is positive. If the sample is a putative positive with a percent inhibition less than the CCP, the sample is considered a false-positive and assigned an ADA negative status. The recommendation provided here covers most situations; however, appropriate scientific judgment is advised on a case-by-case basis. Case studies Assignment of a priori ADA assay criteria facilitates data interpretation, allowing verification of the intended output. This can be exemplified by targeting the 5% false-positive rate (screen positive and confirm negative) for screening assays. Selecting a 95% CI for the assignment of cut point effectively translates to a lowered value for the screening cut point, resulting in an elevated number of potentially positive specimens, which are then filtered through the confirmatory step, which identifies the smaller subset of truly positive specimens. Three case studies are described below (Tables 2 & 3) which show the comparison of prestudy versus in-study validation cut points and impact on data. These case studies are from different laboratories, and therefore present multiple approaches to the calculation of CCP. Samples were tested in a tiered approach consisting of initial screening followed by confirmation by competition. Determination of the screening and CCP were performed using methods consistent with current industry practices. Case study 1: study-relevant assay cut points using in-study predose samples The first case study reports the incidence of ADA against a monoclonal antibody drug (drug B) administered to patients as a monotherapy. This case study describes the use of in-study pretreatment patient samples to obtain the cut points, which are more relevant to the study population than commercially purchased sera. In short, during assay validation, screen cut point factor (SCPF) and CCP were determined using 50 commercially procured samples, with SCPF determined using a 95% CI and CCP determined using a 99% CI [6,33]. The samples were analyzed in duplicate, and each of the two analysts repeated each assay, for a total of four assays. When the prestudy SCPF of 1.75 and CCP of 61.5% inhibition were used to evaluate 656 clinical samples, the false-positive incidence was 0.6% (Table 2). Hence, 183 in-study pretreatment samples were used to re-calculate SCPF and CCP for the assay. For the in-study assessments, SCPF of 1.29 and CCP of 46.9% were obtained based on results of each sample. Based on the in-study cut points, a 4.4% false-positive rate was observed. The disease-state samples for prestudy validation cut point determinations were obtained commercially without any known stratification criteria, and those patients could have been at different stages of disease progression and/or treated with different drugs. The patients enrolled in the clinical trial were stratified according to the clinical protocol, represented a more homogeneous population, and were thus more appropriate for the calculation of cut point. Since the target percentage of false positives was obtained, the use of predose samples for the calculation of in-study cut points was deemed acceptable. To note, in some instances, only the SCPF may change while the CCP may not, which may result in the expected false-positive rate. Case studies 2: a 5% false-positive rate supported use of assay prestudy cut points The second case study describes the correlation between the prestudy validation statistical design and the instudy results of an immunomodulatory Fc-fusion protein (drug Z) with low ADA incidence (4%) and titers ( 20). The prestudy validation cut point was generated using standard methods as described [6]: a bridging assay format, using a matrix of individual commercially procured disease state serum (n = 50), and using a 95% CI for the screening cut point (1.29) and a 99% CI for the CCP. Samples with a screening RLU mean NC RLU 1.29 were deemed putative positive. Samples with a CCP percent inhibition 30 were confirmed positive, and reported as being seropositive in the final results. When used to evaluate clinical samples, these CPs yielded results that were consistent with the intentions

10 Jani, Marsden, Mikulskis et al. Table 2. Analysis of clinical results to evaluate cut point accuracy. Validation SCPF Validation CCP (%) In-study In-study Clinical False positives : screen (+) and confirm (-) NS DS NS DS SCPF CCP (%) samples Validation SCPF and In study SCPF and (n) CCP (%) CCP (%) / Biotherapeutic No. of validation SCP samples Case study 1 Drug B (monoclonal Ab) >50 NA 1.29 ND 30 ND ND ND 2 Drug Z (FC-fusion protein 1) SCP, SCPF, CCP, as the % signal suppression at or greater than which the sample is considered positive; validation SCPF or CCP, determined prestudy, using commercially procured samples. False positives were calculated using DS cut points. Ab: Antibody; CCP: Confirmatory cut point; DS: Serum from disease individuals; NA: Not applicable; ND: Not determined; NS: Serum from normal individuals; SCP: Screening cut point; SCPF: Screening cut point factor. of the statistical design and recommended guidance (FDA). Out of 382 samples, 36 samples (9%) screened positive and 16 samples (4%) confirmed positive, yielding a false-positive screen rate (screen positive, confirm negative) of 5% (Table 2), while eliminating false-positive reporting in the final results as intended via confirmatory tier analysis. Case study 3: re-examination of cut points following change of drug product lots The third case study describes the correlation between the prestudy validation statistical design and the instudy results of an immunomodulatory protein with an endogenous analog and ADA incidence of approximately 15% (titers ranging from 5 to 625). The study describes a data set with a high degree of prestudy and in-study cut point comparability using naive donor or pretreatment patient samples when switching between drug product (DP) lots. The prestudy validation cut point was generated using standard methods: a bridging assay format, a matrix comprised of individual commercially procured disease state sera (n = 50), a 95% CI for the screening CP and a 99.9% CI for the CCP. In this case study, the prestudy validation cut point and in-study cut point, examined at two different times due to a change in DP lot, were determined to be similar as shown in Table 3. Endogenous protein 2 has a prestudy SCPF of 1.22 versus in-study SCPF of 1.07 using DP lot 1. The prestudy validation cut point factor and CCP of 27% were used to support sample analysis until the DP lot change. The new lot of DP had a SCPF of These changes in CP are likely to have negligible impact on study results due to the high degree of comparability between stage-specific CP assessments. During initial in-study CP evaluation (DP lot 1), the CCP was not examined due to the high degree of comparability between the validation and in-study SCPF. However, when the DP lot change was introduced, CP and CCP were assessed where the reassessed CCP was shown to be comparable to the prestudy validation CP (Table 3, 27 prestudy and 28% in-study). Prestudy validation cut points were established using 94 combined normal and disease-state specific donor samples each run four times (2 days, two analysts) with and without excess drug. In-study cut points were established using the maximum available number of disease-state specific clinical pretreatment samples (46), each run once over 2 weeks of production (two analysts) with and without excess drug to define instudy SCPF and CCP. Due to the high degree of similarity (visual and not statistical) between DS1 and DS2 CP and CCP, a single CP and CCP were determined based upon the combined populations. This strategy has subsequently been applied for further testing Bioanalysis (2015) 7(13)

11 Recommendations for the development & validation of confirmatory anti-drug antibody assays White Paper Table 3. Prestudy cut points and in-study cut points with two drug product lots. Endogenous protein 2 Life cycle CP Subjects (n) Validation SCPF (DP lot 1) Validation CCP (DP lot 1) 27% 32 In-study CCP (DP Lot 1) NA NA In-study SCPF (DP Lot 2) In-study CCP (DP Lot 2) 28% 46 Cut point factor was calculated from 94 individual serum samples, originated from naive (n = 30), and disease state population, DS1 (n = 32) and DS2 (n = 32). Data from DS1 is described as an example in this table. CCP = percentage of signal suppression at or greater than which the sample is considered positive; CCP: Confirmatory cut point; CP: Cut point; DP: Drug product; DS: Serum from disease individuals; NA: Not applicable; SCP: Screening cut point (negative control x SCPF); SCPF: Screening cut point factor; Three hundred and four samples (304) were assessed using an ECL bridging ADA. Using the prestudy SCPF of 1.22, 47 out of 304 (15%) samples screened positive. Out of these 47 screen positive samples, 34 samples (11%) confirmed positive and 13 samples (4%) confirmed negative using the 27% prestudy CCP. While this roughly meets the intended 5% screen positive and confirm negative rate, the mean percent inhibition of the 13 samples that confirmed negative was 15%, notably greater than the percent inhibition observed with the NC which was -2%. These patient data will be monitored and evaluated in the context of the immunogenicity time course and will be described further, if necessary, in the clinical and integrated immunogenicity report. In all three case studies examined, the percentage of false positives approximated the target 5%, representing the expected difference between the screening assay and CCPs. Reassessment of the cut points with in-study samples was required to meet the acceptance criteria in the first case study, and data from a larger number of samples were used resulting in the intended 5% rate. Use of normal serum samples for cut point determination is an option for early development or other situations when only a small number of DS samples are available. Case study three provided insight into the need to assess comparability of DP in terms of the confirmatory assay. It should be kept in mind that the CCP calculated using purchased samples or based on a normal population may not reflect the true targeted 5% false positive due to shifts in responses, or more critically, increased variability because of interpatient differences. If it is observed that the false-positive rate is well below 5% for a patient population, the screen and confirmatory cut points should be reassessed using predose samples from an ongoing clinical trial. Overall, it is important to note that correct assay validation and cut point determination give confidence that the assay is accurately detecting antibodies in the study samples. However, it should be noted that different assays may detect different antibodies in samples, some assays may preferentially detect low affinity antibodies and each assay format needs to be carefully evaluated considering program needs [4,6,33]. Conclusion While several publications provide recommendations for the development and validation of the immunogenicity screening assays, clear and detailed recommendations or guidance documents for confirmatory assays are limited. This paper aims to help address this gap, and presents three case studies with examples of the same assay format used for screening and confirmation by competition. The confirmatory assay is part of the tiered approach of immunogenicity evaluation [3]. It is defined by the identification of the true antibody positive samples and accurate identification of false-positive samples derived from the screening assay. It is recommended that the established confirmatory assay is fully validated in a manner similar to, and if possible, in parallel with, the screening assay, including assessment of inter-assay and intra-assay precision, assay selectivity and/or matrix interference and confirmatory assay specificity. While commercially procured samples may be used to establish the validation assay cut points for the intended patient population, the accuracy of those determinations can only be assessed after testing clinical samples. In early clinical studies, typically only limited clinical data are available, which may not be sufficient for accurate estimation of the false-positive rate; therefore, diminishing the value of this preset criterion. It has been recommended that for patient populations with high incidences of pre-existing antibodies where it is difficult to obtain a suitable number of true negative samples, a healthy donor population may be used to set the cut point [5]. Use of samples from healthy subjects in those specific situations and also during early clinical trials could greatly simplify the assay validation process, with CP reassessment being performed with in-study samples as they become available. These very uncertainties faced during early clinical trials provide additional justification for the establishment of more conservative CPs to reduce the possibility of false negatives

12 Jani, Marsden, Mikulskis et al. In-study validation, including the monitoring of assay performance and study results is an essential component required to ensure that an assay is performing as intended. It is important to keep in mind that false positives can be further evaluated with different assays if deemed necessary. Conversely, false negatives in the screening assay would not be identified in subsequent tiers, which may lead to inaccurate drug safety assessments and may compromise pharmacovigilance efforts. Discussions on orthogonality of screening/confirmation assays are outside the scope of this manuscript, and with the first case study we showed that the false positive rate of the assay can be corrected by using samples from a more representative patient population to establish the cut points. In the case of combination therapy, or multiple domain therapeutics, it is recommended to ensure that the CCP is evaluated in a manner appropriate for each therapeutic or modality, including assignment of separate and unique confirmatory cut points, if required. The evolution of combination therapies and new biological entities will require additional and complex evaluation, and is outside the scope of this paper. In a clinical setting, the use of the same assay format for screening and confirmation, and the use of a 5% false-positive rate for screening and 1% for confirmation are useful to avoid false-negative results. Hence, we support the 5% false-positive rate as a reasonable target for the screening assay, and 1% for the confirmatory assay unless clear regulatory guidelines are presented to change this currently accepted approach. However, it does not overcome intrinsic limitations of the assay format. The criteria applied to each assay parameter need to be developed with scientific rationale to produce data that will hold up to the level of rigor demanded for each program. The recommendations presented in this paper are examples of industry best practice; as it is often acknowledged, other approaches that are scientifically valid and objective could be considered to develop and validate robust confirmatory assays. Future perspective Immunogenicity evaluation of biotherapeutics is a critical component for assessment of the clinical impact of ADA on safety and efficacy. Along with development References 1 Shankar G, Arkin S, Cocea L et al. Assessment and reporting of the clinical immunogenicity of therapeutic proteins and peptides harmonized terminology and tactical recommendations. AAPS J. 16(4), (2014). 2 Koren E, Smith HW, Shores E et al. Recommendations on risk-based strategies for detection and characterization of antibodies against biotechnology products. J. Immunol. Methods 333, 1 9 (2008). of new classes of biotherapeutics such as antibody drug conjugates, multidomain therapeutics, bispecific antibodies, and fusion proteins, new and improved ultra-sensitive technologies have also become available, adding an increasing level of complexity to immunogenicity testing. The confirmatory assay will remain an integral aspect of understanding and interpreting immunogenicity results, however, new algorithms and methods will need to be evaluated accordingly. Acknowledgements The authors wish to thank S Kirshner from the US FDA, B Silva and R Pillutla from Bristol Meyers Squibb for their critical review of the manuscript and H Gilbert from Genentech/Roche for his contribution to the statistical section. They thank M Barbosa for her valuable feedback in generating the manuscript. The authors also thank the members of The Therapeutic Protein Immunogenicity Focus Group (TPIFG) and Ligand Binding Assay Focus Group (LBAFG) of the BIOTEC Section, American Association of Pharmaceutical Scientists (AAPS) organization. Disclaimer The contents of this article reflect the personal opinions of the authors and may not represent the official perspectives of their affiliated organizations. Financial & competing interests disclosure D Jani is an employee of Pfizer Inc. R Marsden is an employee of La Jolla Pharmaceutical Company. A Mikulskis is an employee of Biogen Idec Inc. C Gleason is an employee of Bristol-Myers Squibb. C Krinos-Fiorotti is an employee of Pfizer Inc. H Myler is an employee of Bristol-Myers Squibb. L Yang is an employee of Covance Inc. M Fiscella is an employee of Covance Inc. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. No writing assistance was utilized in the production of this manuscript. Informed consent disclosure The authors state that they have obtained verbal and written informed consent from the patient/patients for the inclusion of their medical and treatment history within this case report. 3 FDA Draft guidance for industry. Assay development for immunogenicity testing of therapeutic proteins (2009). 4 Guideline on immunogenicity assessment of biotechnologyderived therapeutic proteins, EMEA/CHMP/ BMWP/14327/2006 (2007). 5 Mire-Sluis AR, Barrett YC, Devanarayan V et al. Recommendations for the design and optimization of 1630 Bioanalysis (2015) 7(13)