Immune assay harmonization and external quality assurance: Results of the ongoing Proficiency Panel Program of the Cancer Vaccine Consortium

Transcription

1 Immune assay harmonization and external quality assurance: Results of the ongoing Proficiency Panel Program of the Cancer Vaccine Consortium Sylvia Janetzki MD Sylvia Janetzki, M.D. ZellNet Consulting, Inc.

2 Introduction CVC (1) Immunotherapy community-wide id membership organization Run under Sabin Vaccine Institute umbrella ( ) Now program of the Cancer Research Institute (CRI) Vision To cure and prevent cancer through the advancement of cancer vaccines and immunotherapeutics.

3 Cancer Vaccine Consortium - History Immuno- Assessment Satellite Symposium CVC Founded 1 st Elispot Panel 2 nd Elispot Panel FDA Workshop Colloquium and Annual Meeting Potency Clinical Trials Testing Working Group Workshop 3 rd Elispot, 1 st ICS, CFSE, and Multimer Commercialization Strategy Meeting Phase 3 Clinical Trials Workshop

4 Proficiency Panel - Program Objectives To offer an external assay validation program To enhance assay harmonization Goal: Establish immune assays as standard monitoring tools in clinical trials and clinical routine settings

5 Reminder: Reasons for hesitation ti measuring immune endpoints 1. Lack or very limited it knowledge about correlation with clinical outcome 2. Lack of confidence in data obtained from bioassays Validity of data

6 1. Correlates of protection: there are examples Dudley: Science 2002; Melanoma (repopulation with TILs tumor regression) Morgan: Science 2006; Melanoma (transfer of genetically modified lymphocytes cancer regression) Todryk: PLos 2008, Correlation of memory T cell response and protection against Malaria Forrest: Clin Vaccine Immunology 2008; Correlation vaccination cellular immune response (Elispot) protection against Influenza

7 Challenges Lack of consistent correlation (countless publications) Limited knowledge about Tcell mediated efficacy (Merck STEP trial) Rather qualitative than quantitative monitoring (Appay) Polyfunctional effector cells (Darrah, Roederer) Migration of effector cells to tissue (DeVries/tetramers) Maximum recovery of relevant immunological data Reliable immune monitoring techniques

8 2. Confidence in data obtained from bio-assays Negative example: Lack of acknowledgment of immune monitoring data from Sipuleucel-T trial (phase 3; Small 2006) despite significant ifi correlation with vaccination status t Reason: lack of bioassay validation (proliferation assay) Do YOU believe immune monitoring data published? Always? Data need to be reliable, repeatable, plausible =VALID

9 Validity of Data GLP: Ensure that the results of a study involving human subjects are Reliable Repeatable Auditable Recognizable GLP includes training, SOP, Reagents and instruments, audits and external quality assurance. = proficiency panel testing

10 Program Objectives CVC Proficiency Panel To offer an external assay validation program To enhance assay harmonization Goal: Establish immune assays as standard monitoring tools in clinical trials and clinical routine settings Establish immune assays as a platform for biomarker discovery and development

11 Assay Harmonization To allow comparability of results across labs Identification of crucial protocol variables that influence assay outcome Formulation of acceptable and feasible recommendations to the field Enhancement of their implementation and confirmation of their usefulness Application of harmonized assays in clinical i l trials The CVC Proficiency panel program

12 The CVC Proficiency Panel Program 2005: 1st Elispot panel (36 labs, 10 countries) 2006: 2nd Elispot panel (29 labs, 7 countries) 2007: 3rd Elispot panel (35 labs) 1st Multimer panel (29 labs) 10 countries 1st ICS panel (28 labs) 1st CFSE panel (21 labs) Central lab: PBMC (pretested), antigen, shipping Scientific panel leaders CVC office for central administration Independent statistician

13 Panelists Total: 63 entities (some with multiple labs participating) 1. Academia/hospitals/non-profit: Biotech/biopharma/services: Big Pharma: 4 4. Government / Armed forces: 3 (5. Supplies/central lab: 4) Cancer: 46/59 = 78% Infectious diseases: 11/59 = 19% (HIV, TB, other) Other: 2/59 = 3%

14 Elispot panel: Protocol 4 pre-tested PBMC from same lot (3 runs, 2 protocols) Pre-tested CEF and CMV peptide pool from same lot PBS/DMSO from same lot Same plate layout (6 replicates; 24 medium alone) Same number of cells and amount of reagents IFN- Elispot, no pre-stimulation Lab-specific protocol, materials and equipment (based on premise that each participating lab has a SOP) (based on premise that each participating lab has a SOP) No PHA or PMA/Iono (confluence and counting issues)

15 Response Definition None: < 10 spots/200,000 PBMC Low: spots/200,000 PBMC Medium: spots/200,000 PBMC High: >100 spots/200, PBMC Response detection (yes/no): 10 spots/200,000 PBMC 3x background (PBMC alone)

16 EPP-1 F2 all participants (medium strong response CEF) 4/36 labs outliers = missed to detect > 50% of responses 17/36 labs missed to detect low responder against CMV 13/36 labs missed to detect low responder against CEF

17 Challenges to detect crucial protocol variables Study design with open protocol choices Wide range in protocol and reagent choices (e.g. 7 ab sources, 10 enzyme sources, 7 readers, cell resting times between 2 and d20h hours, unique serum choices for each lab etc.) Dispersion of protocol choices small sample sizes No adequately powered statistical analysis possible

18 EPP2 Panel participation available for all CVC members with sufficient experience No outliers 4 labs missed to detect the low responder 47% in EPP1 14% in EPP2 (+ 5 marginal) F2

19 Different protocols same results D1 Lab 1 Lab 2 Lab 3 Plate PVDF MCE PVDF Antibodies Mabtech Mabtech Own Kit Development HRP/AEC AP/BCIP HRP/Novared Serum Human AB FCS FBS DNase No Yes Yes Resting No Yes/ON Yes/ON Cell counting Hemacyto Hemacyto Automated/Guava Reader Zeiss AID CTL

20 Similar protocols different results D3 Lab 4 Lab 5 Plate MCE MCE Antibodies Mabtech Mabtech Development HRP/AEC HRP/AEC Serum Human AB AIM-V DNase No No Resting No No Cell counting Hemacytometer Hemacytometer Reader Zeiss Zeiss

21 Influence of automated cell counter use 1000 s per well Spot count No outlier 2 labs missed weak responder 1 D1-CMV D1-CEF D1-med D2-CMV D2-CEF D2-med D3-CMV D3-CEF D3-med D4-CMV D4-CEF D4-med Donor & Reagent Apoptosis! p< Mean cell viability Guava users Non-Guava users Donor Donor Donor Donor

22 Advantage of overnight resting before plating Mean spo ot count per well hours rest 20 hours rest 0 D1- CMV D1- CEF D2- CMV D2- CEF D3- CMV D3- CEF D4- CMV D4- CEF Donor and Reagent Background reactivity: 0-5 spots

23 Plate evaluation is operator- and SOP-dependent Counts Lab X Lab Y Lab Z Own Central Own Central Own Central Mean SD Median Minimumi Maximum Background counts (PBMC alone)

24 Initial Elispot Harmonization Guidelines A. Establish lab Elispot SOP for: A1. Counting method for apoptotic cells in order to determine adequate cell dilution for plating A2. Overnight resting of cells prior to plating B. Use only pretested serum with optimal signal:noise ratio C. Establish SOP for plate reading, including: C1. Human auditing during reading process C2. Adequate adjustment for technical artifacts D. Only let well trained personnel conduct assay Janetzki et al., CII Mar;57(3):303-15

25 EPP 3 (35): new PBMC, 13 labs new (Recommendation to implement harmonization guidelines) 3/22 repeaters missed response = 14% No outliers 4/13 new labs missed response = 31%

26 EPP3: Validation Phase 20 labs said that they did comply with all Harmonization guidelines (57%). 9 of these labs were top performers (9/9 =100%) 15 labs said they did not comply with all Harmonization guidelines (43%). 5 of those labs missed (a) response(s) (=33%). 13 labs introduced changes to their protocol in response to the harmonization guidelines (37%). - 7 added overnight resting - 2 changed medium/serum

27 Overnight rest of cells EPP1 EPP3 Ttl# Total of flb labs Yes 17 (47%) 29 (83%) No 19 (53%) 6 (17%) >8hrs 10 (59%) 22 (76%) <8 hrs 7 (41%) 7 (24%)

28 Automated cell counter use Apoptosis assessment EPP1 EPP3 Total lab# Yes 7 (19%) 14 (40%) No 29 (81%) 21 (60%) Yes- but missed Response(s) 2 (29%) 0 (0%)

29 Confirmation of guidelines 7/13 responses missed due to serum choice (high background reactivity) 4/13 responses missed due to plate reading strategies Next round (EPP4): Tackle of serum issue Integration of guidelines

30 Serum Task Goal: to identify at least 1 serum-free medium that performs at least equally well as lab s medium/serum Who: Top performers What: Own medium/serum, 3 serum free media (ExVivo, AIM-V, CTL) How: EPP pretested PBMC and antigen Same setup for all 4 plates Same protocol Same experiment

31 CVC Multimer panel 1 Scientific leaders: Pedor Romero, LICR/Lausanne & Cedrik Britten, Leiden 27 labs from 7 countries PBMCs from 5 pre-tested t donors with various reactivity it against HLA-A2-restricted - Influenza-M (GILGFVFTL) - Melan A/Mart (ELAGIGILTV) Standard d multimers (antigens) were provided: d Tetramers from Beckman Coulter Pentamers from ProImmune Labs used their own staining protocol

32 MPP1 - Outcome Three clear recommendations could be deduced (Britten, Romero; manuscript in preparation) Next steps: Integration of new recommendations Discussion about background control Systematic examination of protocol variables identified which might have influence on final results

33 Panelist received: ICS Panel 1: Rude awakening Scientific leader: Holden Maecker & Maria Jaimes, BD 1) PBCMs 3 donors with different CEF and CMVpp65 reactivity 2) Lyoplate containing antigens for stimulation of cells (triplicates) 3) Lyoplate containing staining reagents: a) CD4 FITC / IFNg+IL-2 PE / CD8 PerCP-Cy5.5 / CD3 APC b) Comp Control beads and instructions for acquisition and analysis using those beads 4) EDTA, FACS Lysing solution, FACS Permeabilizing Solution2 Highly standardized protocol and reagents Note: protocol and reagents validated at BD and via NIH ICS panel

34 Gold standard ICS panel 1 Example1 Example2 CD4/LP13

35 Next Steps - ICS The 1 st ICS panel has shown: it is not a matter of using someone's validated protocol and reagents - own validation, training per own SOP, experience and own standards are key! Standardized protocol Own protocol First recommendations: Compensation and gating g strategies Background control issue Experience? SOPs?

36 Example: CD8 - all CFSE panel Scientific leader: Jean Boyer, Dan Schullery, UPenn LP7(negative) LP19 LP27 p (Relative to DMSO) labs: 3 donors against CMV pp65 and CEF Protocol recommendation, but only few strict requirements

37 Challenges for CFSE panel Huge variability Very few acceptable responses observed How much do we know about the assay and its applicability in immune monitoring of cancer vaccines? Diverse sera sources (serum known to be of huge influence in CFSE assay NIAID panel) Vague gating strategies Discontinued due to questionable position in cancer field

38 Summary Proficiency panel program has shown to be of high value for external quality assurance and for assay harmonization First harmonization guidelines for Elispot First recommendations for Multimer and ICS CFSE with questionable current position in cancer vaccine monitoring Further systematic examination of crucial protocol variables ongoing Open for collaboration

39 Acknowledgement The Cancer Vaccine Consortium, its Executive Council, Executive office, and umbrella organization. The Steering Committee of the Assay Working Group. All supporters and suppliers (BD, Beckman Coulter, CTL, NIH/ Division i i of AIDS, Proimmune, Seracare). Our participants.