Recommendations for an. appropriate com par ator. definition Justifying a control arm in paediatric clinical trials. Deliverable number D4.9.

Transcription

1 Deliverable number D4.9 Recommendations for an appropriate comparator definition Justifying a control arm in paediatric clinical trials GRiP Global Research in Paediatrics Network of Excellence HEALTH-F Lead Beneficiary SickKids Author(s) Lauren E Kelly, Elin Haf Davies, Martin Offringa Revision date 4 February 2016 Diss emination Level Public PU Start date 01/01/2011 Duration 78 months Project Coordinator Dr. Carlo GIAQUINTO Azienda Ospedaliera di Padova (AOPD) Refere nce W P WP4 New methods for c linical studi es in paediatrics Refere nce Activity Ta sk 4.06 De velop guidance for what defines an appropriate com par ator This project has received funding from the European Union s Seventh Framework Programme for research, technological development and demonstration under grant agreement n

2 Index Index... 1 List of authors... 3 Abstract... 4 Receivers of the document Background, rationale and objectives Methods Literature and Regulator Guidance Review Review of Paediatric Investigation Plans Review of Marketing Authorisations Creation of Tool to Select Justifiable Comparators in Paediatric Clinical Trials Results Regulatory Guidance and Literature Review Review of Paediatric Investigation Plans Review of Marketing Authorizations Creation of a Tool to Justify Comparator Selection in Paediatric Clinical Trials Discussion and limitations Setting the research agenda Document history References Appendices Appendix I. Current controversy regarding comparator choice in clinical trials Appendix II. Defining standard of care in paediatrics Appendix III: Case Report Form Part A Appendix IV: AMSTAR: Assess Methodological Quality of Systematic Reviews Appendix V: Case Report Form: Part B Appendix VI: Summary of EMA Comparator Evaluation Appendix VII: Mann and Djulbegovic philosophy and justification framework Appendix VIII: Full details of 2013 paediatric MAs...40 GRiP-D4.09-Recommendations for an appropriate comparator definition 2

3 List of authors Name Institution Writers Lauren E Kelly: lauren.kelly@sickkids.ca Martin Offringa: martin.offringa@sickkids.ca Elin Haf Davies: elin@elinhafdavies.co.uk Hospital for Sick Children, Toronto Hospital for Sick Children, Toronto PENTA Collaborators Oscar Della Pasqua Henneke van der Lee Winnie Chan Sven van Dijkman Paolo Tomasi Agnes Saint-Raymond Mariagrazia Felisi Adriana Ceci Lucia Ruggieri UCL AMC Hospital for Sick Children, Toronto Universiteit Leiden European Medicines Agency-EMA European Medicines Agency-EMA CVBF-TEDDY CVBF-TEDDY CVBF-TEDDY GRiP-D4.09-Recommendations for an appropriate comparator definition 3

4 Abstract Background Children need drugs with evidence of safety and efficacy specifically demonstrated in the paediatric age range for which the drug is intended. In clinical research aimed to establish safety and efficacy of a particular drug treatment, determining the appropriate comparator for that intervention is critical because estimates of relative effectiveness are obtained. An inappropriate comparator may lead to both over- and underestimation of the true treatment effects, both beneficial and adverse, as has been shown in empirical research into comparator bias. A transparent and valid justification of comparator selection will improve the quality of the design, the conduct, and the applicability of paediatric clinical trials. Clarity on the appropriateness of comparator use in paediatric drug trials will also ultimately allow for good-quality meta-analyses, clinical guideline development and implementation, and improve health outcomes for children. Aim To develop an evidence-based tool to justify comparator arm selection in paediatric clinical trials. Methods A random selection of Paediatric Investigation Plans (PIPs) adopted by the Paediatric Committee of the European Medicines Agency (EMA) in 2013 was reviewed. Inclusion criteria were pediatric clinical trial protocols evaluating an intervention. Preclinical studies, single arm studies, pharmacokinetic studies, dose finding studies and studies where patents served as their own control were excluded. Descriptive information about the clinical trials in these PIPs was collected. Documents regarding discussions on the comparator selection and use between the submitting sponsor and the regulator were evaluated and paediatric trial specific challenges were identified. Evidence provided in justification of the comparator selection was reported. Similarly, paediatric studies included in the European Paediatric Assessment Reports (EPARs) of drugs centrally authorised for paediatric use in 2013 were analysed. Using the available literature and empirical evidence gathered a list of considerations for the justification of comparator was formulated into a tool for justifiable the selection of a comparator arm in paediatric clinical trials. Results Paediatric Investigation Plans from 2013 were randomly selected and the included trials reviewed (N=173). There were 117 trials which did meet our inclusion criteria. Prior to study approval, 29% (16/56) of trials had issues identified by regulators with the selection and/or use of a comparator. The majority of issues identified (56%; 9/16) were paediatric specific; the remaining issues were applicable to both adult and paediatric trials. Paediatric specific issues included: 1) the comparator was licensed for paediatric use in all recruiting countries; 2) ethical and safety concerns regarding placebo use; 3) insufficient evidence to support the comparator dosing regimen; 4) insufficient age-specific pharmacokinetic data on a paediatric formulation of the comparator; 6) insufficient effectiveness and safety data and (7) a lack of an age-appropriate formulation. There were 22 trials including paediatric patients submitted in support of the 2013 marketing authorizations (EPARs). Half of trials (11) were controlled, including 8 active controlled (36%), 1 placebo controlled trial (14%), 1 historical control, 1 comparison to no treatment. GRiP-D4.09-Recommendations for an appropriate comparator definition 4

5 With reference to EPARs, all the active comparators had been granted a MA in at least one European country. Placebo, no treatment and historical controls were justified given a lack of age-appropriate active controls or via compliance with relevant Guidelines. Conclusion Justification of comparator selection must consider/address the complexities of comparator selection in the paediatric population. A tool to assist clinical trialists in providing transparent justification for the choice of comparator in pediatric clinical trials is generated. Based on empirical evidence and published literature, this tool requires validation by regulators, industry and academic trialists. GRiP-D4.09-Recommendations for an appropriate comparator definition 5

6 Receivers of the document User group GRIP Beneficiaries (CVBF-TEDDY, AOPD, PENTA and the EMA) Identified need(s) Be informed about the development of an evidence- and consensus based tool for the choice of comparators in paediatric clinical trials European Commission Be informed about the development of an evidence- and consensus based tool for the choice of comparators in paediatric clinical trials Clinical investigators, researchers Industry Journal editors Awareness and endorsement of a tool that can be used when selecting and justifying the choice of comparator in the design of a paediatric clinical trial Awareness and endorsement of a tool that can be used when selecting and justifying the choice of comparator in the design of a paediatric clinical trial Awareness, endorsement and implementation of tool GRiP-D4.09-Recommendations for an appropriate comparator definition 6

7 Description of the Tool 1. Background, rationale and objectives Well-designed clinical trials are necessary to generate evidence for improved health outcomes and inform policy change. According to the WHO, clinical trials should enhance the well-being, treatment, prevention and diagnosis of ill health including children [1]. Regulators, including the European Medicines Agency (EMA) and the Food and Drug Administration (FDA) have legislated approval process updates which require new drugs to be tested for safety and efficacy in children or apply (with justification) for a waiver. Children represent a unique and vulnerable population that requires special considerations when designing a clinical trial. When designing a clinical trial comparator arm, or control arm selection is a critical step in clinical trial design as it will become the basis for estimation of relative intervention efficacy and safety. An inappropriate selection may only over- or under-estimate treatment effect size, it can affect ethical review and patient recruitment and [2]. Comparator selection in paediatric clinical trials presents an additional layer of complexities. Children are a diverse population and require special considerations when selecting a comparator including age appropriate formulation availability, developmental pharmacology, the extrapolation of pharmacokinetic and pharmacodynamics data, as well as the vast use of off label and unlicensed medications. The EMA and FDA have issued guidance on the choice of comparator in adult clinical trials. This guidance, while applicable to some extent, does fully address specific issues of selecting a comparator for paediatric trials. There are several additional ethical considerations regarding the use of placebo, especially as consent for paediatric clinical trials are obtained through a proxy, usually the parents. It has been recommended that if placebos are to be used in a paediatric trial, certain safety requirements must be in place: appropriate rescue and escape protocols must be available to manage children whose clinical status worsens by a predefined level [1]. Also, placebos as well as active comparators need to be available in acceptable paediatric formulations, for which compliance is expected to be high and deterred by taste and texture. EMA recommendations regarding clinical trials in the paediatric population state that placebo use is justifiable for scientific reasons such as, a lack of standard treatment, variable placebo effect (pain, hay fever), or a high frequency of adverse drug reactions seen with standard of care [1]. This creates difficulty in choosing comparators in paediatric trials as often the standard of care is variable between centres, even in the same city, and many drugs are used off-label without the availability of age-group relevant pharmacokinetic/dynamic data. If such data are available, drugs that do have an approved labelling for use in children can still be chosen as comparators presuming this choice is based on scientific evidence and the best medical judgment [3]. The active or placebo formulation for the comparator treatment should include minimal excipients, be easy to swallow if being administered orally, and heat/humidity stable [1]. The route of administration and bioavailability in children also must be considered [2]. Currently, while regulators and research funders require explanation of trial design, including choice of comparator, there is a lack in guidance on what evidence needs to be provided in order to be justifiable. Determining whether a comparator is justified Prior to the selection of a comparator the investigators must be in a state of genuine uncertainty over which treatment will be better: the (new) intervention or the comparator. This state of equipoise is a requirement for the ethical recruitment of patients randomised GRiP-D4.09-Recommendations for an appropriate comparator definition 7

8 into a prospective clinical trial. This condition must be met by all trials, including those with placebo controls. For placebo controls it is essential that the placebo effect is variable or unknown in order to be justified as a choice of comparator. Furthermore, both placebo and active comparators need to have available paediatric friendly forms and formulations. Outcome measures must also be chosen that are paediatric and disease specific. The selection and timing of outcome measure collection needs to consider the comparator s pharmacokinetics. Justification entails that sufficient evidence has been provided to support the use of comparator both from a scientific and ethical perspective. The objective is to identify: 1. What are sources of contention between sponsor/industry s proposed comparator and what is accepted by regulators? 2. How are these paediatric challenges resolved in the MAs submitted for paediatric indications and in the paediatric investigation plans? 3. What evidence is currently used in justifying comparator selection in clinical trials? In this document we present the development of a GRiP tool to aid in selection of a comparator arm in paediatric clinical trials. This tool is intended for use by academic and industry clinical investigators, clinical research associates, and regulators to support evidence-based transparent selection of a comparator arm(s) in paediatric clinical trials. 2. Methods Literature and Regulator Guidance Review, finished June 2014 EMA [4] and ICH guidelines [5] were reviewed to identify recommendations outlined for choosing appropriate comparators when conducting drug trials. Works by Mann and Djulbegovic Choosing a control intervention for a randomised clinical trial [6] and Comparator bias: why comparisons must address genuine uncertainties [2] were reviewed to gain a better understanding of the philosophy supporting the choice of appropriate comparators and biases associated. While these guidance documents are paediatric specific, many considerations may be applied in both adults and children. Guidelines were summarised and used to develop a protocol for evaluating the use of appropriate comparators in clinical trials. To define the appropriateness of comparators, this rationale that underpins comparator choice must be more clearly defined Review of Paediatric Investigation Plans (PIPs), finished December 2014 The PIP review process The role of the EMA is to review study protocols, submitted as PIPs, and ensure appropriate data is collected to support paediatric marketing authorizations. These PIPs are typically created as follows: a pharmaceutical industry proposes a development program, including main a plan for paediatric trials (PIP) and submits it to EMA for evaluation process on D0 (day 0). The EMA staff reviews and comments, then submits the draft PIP assessment to two members of the PDCO for further review. It is then presented and discussed at the PDCO at D30 and at D60 a request for modification is submitted to the company for changes. The clock stops for the company to address the request. At D61 procedure restarts and an opinion is agreed on D120. PIPs were reviewed from the time of first submission (sponsor proposal, D0) to the time of approval by the PDCO. GRiP-D4.09-Recommendations for an appropriate comparator definition 8

9 Study selection Paediatric investigation plans (PIPs) adopted by the EMA in 2013 were reviewed. Empirical data from industry sponsored PIPs was collected from the EMA internal paediatric records database (PedRA). The sponsor is defined as the applicant company submitting the trial protocol(s) to the EMA for evaluation. A standard set of data were obtained on all trials included in a random sample of PIP applications once adopted by the Paediatric Committee (PDCO). PIPs were sequentially numbered and a 40% sample was randomly selected using an online random sequence generator. Exclusion criteria for the present study were preclinical studies, dose-finding studies, studies using patient baseline values as control, single arm studies, and studies with pharmacokinetic analysis as a primary endpoint. Data extraction For all trials included in the selected adopted PIPs, data was collected on (1) therapeutic area, (2) indication, (3) study design, (4) main objectives, (5) study population and subset definition, (6) the number of participants by subset (age, sex, severity or disease stage), (7) dosing regimen for study drug including route of administration, (8) location of recruitment, (9) the comparator, and (10) dosing regimen for comparator including route of administration. In collaboration with stakeholders from the FDA and EMA, empirical evidence will be gathered on the MAs submitted for paediatric indications as well as PIPs. PIP data was collected by SickKids at EMA in September MA data was collected remotely. Data collected on each paediatric clinical trial was recorded in the case report forms provided in Appendix III and if appropriate Appendix V. Reviews conducted or cited in justification of comparator choice was assessed for quality using AMSTAR standards (Appendix IV) Review of Marketing Authorisations (MA), finished April 2015 Committee for Medicinal Products for Human Use (CHMP) approved MAs were reviewed and details regarding trials submitted in support of paediatric licensing in Europe were evaluated. Study selection All CHMP-approved paediatric MAs have been identified. Trials which have been reviewed refer to MAs approved in Medicines approved at non-centralized level were included in this study. Only controlled trials were selected for the analysis. Data Extraction For all trials the following data were collected: a) Study Code, b) Type of study, c) n. of paediatric patients, d) n. of total patients, e) randomization, f) blinding, g) if controlled or, h) type of control, i) competitor details, j) age of experimental population To assess if active comparators were justified we considered the following parameters: 1. MA of the comparator in at least one EU country (any indication) 2. PK data (in children or in adults) 3. PD data (in children or in adults) GRiP-D4.09-Recommendations for an appropriate comparator definition 9

10 4. Indication, dose, age group and formulation in which the comparator is used (authorized/ authorized) 5. Systematic Review availability In case of the placebo-controlled trial, the existing current/standard therapy has been considered Creation of Tool to Select Justifiable Comparators in Paediatric Clinical Trials, finished May 2015 Upon completion of analyses of empirical evidence gathered, data was analysed to characterize how well the current guidance from EMA, ICH and the FDA is being followed for the selection of comparators in paediatric clinical trials. This analysis was used to identify gaps in the current guidance, identify paediatric specific challenges and was used to formulate a tool to guide researchers in the selection and justification of appropriate comparators during the study design process. Analyses examined findings using both quantitative and qualitative methodologies. Quantitative analyses: using data collected, estimated the prevalence of well-justified comparators, as determined by 1) regulator guidelines (Appendix VI) and 2) the Mann and Djulbegovic justification framework (Appendix VII). Qualitative analyses: the quality of reasoning behind the comparator selection processes was assessed for each PIP and MA using the data gathered in CRF 1 & 2 (Appendix III, V). Furthermore, the quality of the evidence used to support and justify the comparator choice was assessed using the AMSTAR checklist (Appendix IV). In order to evaluate the current process and evidence presented for selecting comparator arms in paediatric clinical trials all documentation between coordinators at the EMA, the sponsor and the PDCO discussions on comparator selection was analysed. Issues or challenges were identified in the selection (and/or use) of the comparator following the submission of a PIP. These challenges were defined as points which required revisions or further justification by the sponsor prior to the final PIP adoption by the PDCO. These included discussions and recommendations made to the PDCO by the Formulation Working Group (FWG) on the formulation of the comparator. Qualitative discourse thematic analysis using documented meeting minutes from the PDCO/FWG meetings as well as hard-copies of the written communication and responses between sponsor and the PDCO was applied. Anonymized illustrative examples of each unique challenge was provided. The study results were analysed using a mixed-methods approach including a quantitative assessment and a thematic analysis performed by two independent scientists (LEK, EHD). Issues were determined to be paediatric challenges if they would be considered challenges in the same trial involving adult patients. For trials with unique active comparators, evidence supporting the compounds selection and dose were evaluated. Evidence included; (1) a marketing authorization issued for any indication of this drug in at least one European country; (2) whether the comparator was being used for the authorized population (disease severity, dose, formulation, age); (3) pharmacokinetic (PK) and pharmacodynamics (PD) data in children; (4) PK/PD data presented in other populations; (5) a published systematic review and (6) international/national clinical practice guidelines. GRiP-D4.09-Recommendations for an appropriate comparator definition 10

11 The challenges identified from available literature guidance and the empirical assessment of current PIP decision making were formulated into a list of considerations for justifying the selection of a comparator arm in paediatric clinical trials. 3. Results 3.1. Regulatory Guidance and Literature Review A review of EMA and FDA publications revealed that guidelines are developed by the International Harmonization Conference (ICH), ICH is a framework for the development of guidelines with participation of regulators and industry. ICH has published four Efficacy guidelines that provide recommendations on choosing appropriate comparators, including ICH E8 General Considerations for clinical trials, ICH E9 Statistical Principles for Clinical trials, ICH E10 Guidance for Choice of Control Groups and ICH E11 Notes for Guidance on Clinical Investigation of Medicinal Products in the Paediatric Population. EMA regulators have since developed a reflection paper titled, The need for active control in therapeutic areas where use of placebo is deemed ethical and one or more established medicines are available, that provides MA applicants with guidance in deciding between the use of an active and/or placebo control, when both comparators are possible [4]. In order to adequately measure new treatment for children with appropriate comparators, their conclusion is that EMA, 2010 and ICH E 8-11 guidelines be used in conjunction with each other. A summary of EMA/ICH guidance is provided in Appendix VI. Howard Mann and Benjamin Djulbegovic s research on comparators led to the development of the concept of comparator bias that will be introduced by inappropriate choice and use of a comparator in a clinical study [2, 6]. Comparator bias may be seen when effective treatment is withheld from the patient, or when a weak, or low dose active comparator is used which does accurately reflect treatment impact. Djulbegovic proposed a method to reduce comparator bias, which includes the provision of systematic reviews assessing comparator choice groups alongside clinical trial protocols. He further provides factors to be considered when conducting a clinical intervention trial and methods to assist the justification and choice of appropriate comparators [2]. For the purposes of this GRiP Task, methodologies outlined by Djulbegovic et al. were extracted to provide an (additional) justification framework to the protocol developed to empirically evaluate the choice of comparators in paediatric clinical trials. Further details regarding the Mann/Djulbegovic framework is provided in Appendix VII. Considerations assessing if placebo is justified include (Appendices I, VI, VII): 1. Is there evidence that an established treatment reduces mortality, lessens disease progression, and/or prevents long term disability? (If yes, must choose active comparator) 2. Is there no standard of care in the study domain? 3. Is the commonly used therapy of questionable efficacy? 4. Is the commonly used therapy associated with a questionable risk/benefit ratio, or a high rate of adverse effects? 5. Is rescue treatment available? If so, are early escape (withdrawal) endpoints defined a priori? GRiP-D4.09-Recommendations for an appropriate comparator definition 11

12 6. Are parents and patients adequately informed of the long terms risks associated with potentially avoiding an active treatment (however questionable that treatment is)? Considerations for the justification of an Active comparator include (Appendices I, VI, VII): 1. Is there a current standard of care for the treatment of the disease of interest? if, placebo use may be justified 2. Is the standard of care in the study jurisdiction based on: a. Marketing authorization from the FDA/EMA? b. Systematic reviews/meta-analysis? c. Expert consensus/clinical practice guidelines? 3. Is the standard of care recommended for the age group being studied? 4. Is there pharmacokinetic and pharmacodynamic data available for the use of this treatment in children? Has the dose used/recommended been properly established? 5. If paediatric specific pharmacokinetic data is available, can inferences be made from adult populations taking into account: a. Disease aetiology and natural history b. Developmental considerations (ontogeny) 6. Is there established assay sensitivity? 7. Can the medication be given by a child friendly route of administration? 8. Are non-inferiority (or equivalence) margins defined a priori and based on: a. Historical evidence? b. Clinical judgement? c. Conservative (< 10 % absolute risk difference)? 3.2. Review of Paediatric Investigation Plans EMA s PDCO adopted a total of 97 scientific opinions on PIPs in The 40 randomly selected PIPs included 174 individual trial proposals, which were further evaluated. Of these, we excluded 117 trials including preclinical studies (n=64), dose-finding studies (n=5), studies using patient baseline values as control (n=5), single arm studies (n=24) and studies with pharmacokinetic analysis as a primary endpoint (n=19) (Figure 1). GRiP-D4.09-Recommendations for an appropriate comparator definition 12

13 Figure 1. Selection of trials for analysis Included were 57 clinical trials with active comparators (n=19), standard of care (n=6), placebo in combination with the standard of care (n=12), and 15 placebo-only controlled trials. There were four with historical controls and one trial comparing to no treatment (Figure 1). All trials were industry sponsored. Fourteen therapeutic areas were represented, the breakdown of which can be found in Figure 2. Using qualitative analysis, we identified 12 separate general methodological issues/concerns with comparator selection and paediatric specific issues (table 1). Prior to PIP adoption by the PDCO, 28% (16/57) of trials evaluated had at least one issue/concern related to the comparator selection. The themes of these challenges included the selection of the comparator arm (including issues with defining the standard of care ) (n=6) and the study design including the practical use of a comparator (n=12). Three trials had issues with both the selection of, and the use of the comparator. All but in one case were sponsors and regulators able to reach an agreement on the choice of comparator resulting in PIP approval. The case GRiP-D4.09-Recommendations for an appropriate comparator definition 13

14 which was resolved surrounded an inappropriate (unsupported) dose extrapolation from adult studies. Immunology - Rheumatology - Transplantation Haematology Pain Dermatology Vaccines Neonatology - Paediatric Intensive Care Gastroenterology - Hepatology Pneumology - Allergology Cardiovascular Diseases Neurology Infectious Diseases Endocrinology - Gynaecology - Metabolism Other Oncology Figure 2. Proportion of PIP trials in 2013 broken down by therapeutic area Challenges linked to the choice of an active comparator were most common in the area of infectious disease, where all trials (5/5) evaluated had issues. Challenges were reported in any cardiovascular, gastroenterology-hepatology, immunology-rheumatology transplantation or vaccine trials studied in this cohort. There were 3 infectious disease trials with challenges regarding both the selection of, and the use of the comparator. Examples of paediatric issues with the selection of comparator included: 1) The comparator was licensed for paediatric use in all countries for which recruitment is planned; 2) Concerns regarding ethical placebo use in the absence of an approved treatment; 3) Safety concerns regarding placebo use in moderate asthma: 4) Variability in comparator dosing and lacking evidence to support the current dosing regimen; 5) Lacking age-specific pharmacokinetic data on a new paediatric formulation of the comparator; 6) Adult extrapolation of active comparator dose supported; 7) No efficacy/effectiveness data has been obtained in the population of interest. The dosage form or formulation of comparator was an issue in 17% (n =14) of trials. Formulation concerns included tablet size, acceptability and excipient use. However, these concerns were specific to the comparator and were often expressed for both the intervention and control arms. GRiP-D4.09-Recommendations for an appropriate comparator definition 14

15 Table 1. Specific, anonymized examples of concerns regarding comparator selection and/or comparator use in paediatric clinical trial protocols. Comparator concern at Paediatric the time of submission issue (Y/N) Example Solution identified at the time of PDCO adoption 1. Comparator is licensed for paediatric use * 2. Standard of care treatment variability between recruitment centres * 3. Concerns regarding synergistic effects with the standard of care with a primary endpoint of decrease in standard treatment use 4. Concerns regarding ethical placebo use in children * 5. Unblinded data assessment 6. Concerns regarding the risks associated with placebo injection * 7. Concern regarding the duration of placebo (on top of standard of care) exposure in an oncology trial 8. Safety concerns regarding placebo use for moderate asthma Y Y N Y N N N Y Linezolid is licensed for use in children in the USA but in Europe Regional differences in microbial susceptibility Placebo + Morphine vs. Drug Y + Morphine No alternative drugs are approved, no standard of care is available to be used as an active comparator Due to an unequal randomization ratio the analysis can be blinded Intraglandular injection may involve sedation or general anaesthesia Planned phase 3 placebo-controlled phase of 26 weeks A previous placebocontrolled clinical trial was terminated due to increased adverse One comparator may be suitable in all trial locations 1. Multiple active controls to be applied 2. Agreement between recruitment sites on one active comparator Primary endpoints could be modified to accommodate comparator use. In this case from decrease in morphine required to the percentage of responders based on a pharmacodynamic endpoint (defined as a 30% reduction in age-appropriate pain score) Independent working group of clinicians, ethicists, and patient organizations assembled to discuss risk/benefit of this comparator Blind outcome assessors and use objective outcome measures used where possible Staggered approach, 2:1 placebo approved with no placebo use in children less than 6 years Agreement that placebo duration will be shortened to 16 weeks Deferral of paediatric studies until adult Phase III data has been collected and evaluated. Studies will events and high patient include strict stopping rules. drop-out rates in placebo Comparator will be placebo group on top of standard of care GRiP-D4.09-Recommendations for an appropriate comparator definition 15

16 Comparator concern at Paediatric the time of submission issue (Y/N) Example Solution identified at the time of PDCO adoption 9. Comparator has undefined optimal dosing strategy 10. Age-specific pharmacokinetic data lacking for comparator formulation 11. Effectiveness data on the standard of care has been established 12. Dose of comparator proposed is supported based on adult extrapolation Y Y Y Y Standard of care varies across recruitment centres with no defined optimal dosing strategy Trial planned to use a new paediatric formulation of the comparator Lack of agreement of standard of care for wound healing Subcutaneous comparator dose for Myelodysplastic syndromes of 100mg/m 2 will result in an unacceptable volume of 4mls (given 1m 2 ) Comparator dosing regimen adapted from several recruitment site (local) formularies and accompanying literature agreed between sponsor and regulator Comparative pharmacokinetics study with sparse sampling between new formulation and licensed tablet planned Three arm parallel study evaluating the effectiveness of standard of care and new intervention vs. placebo Negative opinion following 120day review Anonymized examples of challenging issues surrounding comparator selection are summarized along with the proposed solution in Table 1. There were 19 trials with 11 unique (different) active comparators. Evidence provided in justification for active comparator selection is summarized in Table 2. All active comparators had received a marketing authorization in at least one European country. None of the systematic reviews cited had published an optimal dose of comparator to be used in the population of interest. GRiP-D4.09-Recommendations for an appropriate comparator definition 16

17 Table 2. Justification provided for active comparator selection in pediatric investigation plans. Justification for Active Comparator Selection in PIPs MA for comparator in at least one EU country (any indication) # unique active % comparators % Pharmacokinetics presented in children 6 55% Pharmacodynamics presented in children 3 27% Pharmacokinetics presented in other populations 7 64% Pharmacodynamics presented in other populations 6 55% Other population(s) Adults 4 36% Alternative pediatric age group 4 36% The comparator used in used in the authorized indication 10 91% Disease aetiology similar in other population 8 73% Comparator used in the authorized dose 9 82% Comparator used in authorized formulation 10 91% Comparator used in authorized age group 7 64% Systematic review presented 6 55% Of the systematic reviews presented (n=6) Systematic review published Review identify an optimal dose Dose match what was used in the trial Was the review sponsored by the same company funding the trial 5 83% None None None Was the systematic review completed independently 6 100% Are there clinical practice guidelines for managing this disease 9 82% Do these guidelines list the active comparator as first line in the *population of interest 6 55% * specific for age, disease and severity of condition GRiP-D4.09-Recommendations for an appropriate comparator definition 17

18 3.3 Review of Marketing Authorizations In 2013 EMA approved 20 medicinal products for children corresponding to 4 vaccines and 12 different active substances. Most of authorized medicines were antineoplastic/immunomodulating agents (7/20, Figure 3). 5 out of the 12 active substances were generic or biosimilar drugs, so the studies supporting the authorization were those already submitted at the time of MA of their originators. Moreover, for one active substance just literature data were reported in the application. Serie1; ATC R Respiratory system; 1; 5% Serie1; ATC B Blood and blood forming organs; 3; 15% Serie1; ATC A Alimentary tract and metabolism; 3; 15% Serie1; ATC H Systemic hormonal preparations, excl. Sex hormones and insulins; 1; 5% Serie1; ATC L Antineoplastic and immunomodulating agents; 7; 35% ATC L Antineoplastic and immunomodulating agents ATC J Antiinfectives for systemic use ATC H Systemic hormonal preparations, excl. Sex hormones and insulins ATC A Alimentary tract and metabolism ATC B Blood and blood forming organs ATC R Respiratory system Serie1; ATC J Antiinfectives for systemic use; 5; 25% Figure 3. Classification by ATC of authorised paediatric medicines in 2013 EPARs. There were 22 trials including paediatric patients submitted in support of the 2013 marketing authorizations (table 3). Only 6 trials were exclusively paediatric. Half of trials (11) were controlled, including 8 active controlled ones, 1 placebo controlled trial, 1 historical control, 1 comparison to no treatment. One among active-controlled trials has a three-arm design, so two comparators are used. As detailed below, also the placebo-controlled trial employs three study arm. Table 3. Details of studies with paediatric patients included in the EPARs (year: 2013) TRADENAME - ACTIVE SUBSTANCE Study Code BPLG- 003 Type of study Phase II/IIIa, efficacy, safety, PK/PD N. of paediatric patients/ total patients 37/37 Randomi zation yes Blindnes s Controlle d study Type of control Competitor details yes active Geropin singleblinded SOMATROPI Phase III, BPLG- N efficacy 004 BIOPARTNER and safety 178/178 yes no yes active Geropin - somatropin Phase III, LGefficacy SHCL004 and safety 60/60 yes no yes active Eutropin Phase II, SHCL002 efficacy 41/41 yes no yes active Eutropin and safety GRiP-D4.09-Recommendations for an appropriate comparator definition 18 Age of experimenta l population prepubertal children prepubertal children (>3 and <12 yrs) prepubertal children prepubertal children

19 TRADENAME - ACTIVE SUBSTANCE VONCENTO - human coagulation factor VIII / von Willebrand factor NOVOEIGH - Turoctocog alfa RELVAR ELLIPTA - fluticasone furoate/vilante rol DEFITELIO - defibrotide GRASTOFIL - filgrastim Study Code CSLCTBI O CSLCT- BIO CSLCT- BIO CSLCT- BIO CSLCT- BIO NN NN NN NN NN HZA HZA HZA applicabl e doubleblind available Type of study Phase II, PK, safety, efficacy Phase II/III, PK, efficacy, effectiven ess Phase II/III, efficacy and safety Phase II/III, efficacy, safety and tolerability Phase II, PK Phase I,PK Phase III, PK, safety and efficacy Phase III, PK, safety and efficacy Phase III, efficacy and safety Phase I,PK Phase IIIa, efficacy and safety Phase IIIa, efficacy and safety Phase IIIa, efficacy and safety pooled efficacy analysis Phase III, efficacy and safety Phase III, efficacy and safety Phase II, efficacy and safety dose accuracy studies N. of paediatric patients/ total patients Randomi zation Blindnes s Controlle d study 3/81 no no no 3/22 no no no 1/23 no no no 2/30 no no no 1/16 no no no Type of control controlled controlled controlled controlled controlled Competitor details applicable applicable applicable applicable applicable Age of experimenta l population applicable yrs 3 year-old child adolescents (16-17 yrs) adolescent (17 yrs) 1/23 no no yes active Advate > 12 yrs available/1 50 no no no 63/63 no no no available/1 87 no no no 2/7 no no no 82/609 yes 23/586 yes 281/2019 yes applicable applicable doubleblind doubleblind applicable controlled controlled applicable applicable > 12 yrs < 12 yrs controlled applicable > 12 yrs controlled applicable yrs yes placebo applicable 12 yrs yes yes applicable active active controlled 58/144 no no yes historical 356/356 yes no yes available/1 51 available no treatment yes no yes active available available no applicable Fluticasone furoate, fluticasone propionate Fluticasone Furoate applicable historical control no treatment different doses of defibrotide applicable 12 yrs 12 yrs applicable 16 yrs > 1 month-18 yrs 18 yrs available The 8 studies with an active comparator have been further analysed: Studies were conducted both in EU and extra-european countries. Only 1 study seems be included neither in the European database of clinical trials, nor in the global one. GRiP-D4.09-Recommendations for an appropriate comparator definition 19

20 Half of the studies (4/8) are Phase III trials, 2 are phase II trials, while 1 is phase II/III and 1 is a phase I trial. Active-controlled trials included a number of paediatric patients ranging from 23 to 281, involving all paediatric age subgroups, except neonates. Details on active comparators and placebo in EPARs (MAs) All the active comparators received a marketing authorization, with the following specification: Eutropin Inj (studies LGSHCL004 and SHCL002) was approved in Europe, but in South Korea where the study was conducted. The administration of Eutropin in the trials reflected the indication and dosage of the marketing authorisation [ Most of the comparators were in the form of injectable formulations: in the case of Geropin and Eutropin, the IMP was a prolonged-release formulation of the same active subtance (recombinant human growth hormone), with the new formulation allowing the reduction of medicine administration frequency. No one of the comparators was studied in a PIP. A waiver was granted for Advate, on the ground that it did represent a significant benefit for Immune Tolerance Induction (ITI) in patients with haemophilia A (congenital Factor VIII deficiency) who have developed inhibitors to Factor VIII, treatment and prophylaxis of bleeding in patients with haemophilia A (congenital factor VIII deficiency). With reference to the use of comparator in the approved formulation and dosage, we ed that fluticasone furoate was approved in Europe in the form of nasal spray, while as comparator in studies HZA and HZA a formulation for oral inhalation was used. In addition, in this trial fluticasone furoate has been administered at a dosage of 200 mcg/day while the recommended dosage of the approved nasal spray is 110 mcg/day A three arm trial involving adolescent patients has been conducted for Relvar Ellipta (HZA106827) that results both placebo-controlled and active controlled trial to demonstrate the inequality of the treatment of fluticasone/vilanterol against fluticasone furoate alone and placebo. The study adopted trade label salbutamol/albuterol inhalation aerosol as rescue treatment to treat asthma symptoms throughout the study. Even if the study design is consistent with the Guideline on the clinical investigation of medicinal products for the treatment of asthma, recommending three arm studies (study drug placebo active comparator (standard of care) we ed that fluticasone furoate was currently authorised for the treatment of asthma as monocomponent. The adopted drug formulation is considered acceptable for the study population, considering that dry powder inhaler are recommended [CHMP/EWP/2922/01 Rev.1, the new version of the guideline has been agreed and that will come into effect in May 2016]Errore. Il segnalibro non è definito. in children aged > 4 years (with exceptions for children aged 4-6 years). Full details on active comparators are available in Annex VIII. GRiP-D4.09-Recommendations for an appropriate comparator definition 20

21 Table 4. Justification provided for active comparator selection in studies leading to marketing authorisation Justification for Active Comparator No of active comparators responding % to criteria* MA for comparator in at least one EU country (any indication) 8 80 PK presented in children 5 50 PD presented in children 4 40 PK presented in adults 8 80 PD presented in adults 4 40 The comparator used in the authorized indication 7 70 Comparator used in the authorized dose 7 70 Comparator used in authorized formulation 7 70 Comparator used in authorized age group Systematic review presented Creation of a tool to select justifiable comparator arms in paediatric clinical trials A tool was created to assist clinical researchers in providing sufficient evidence for justification in their choice of comparator arm. This tool was created using all available evidence including the supporting literature and based on the evaluation of PIPs and MAs and the supporting literature. This flowchart is meant to provoke thought and consideration regarding comparator justification in paediatric clinical trials. All considerations should be yes in order for either active or placebo comparator arm. This tool was presented to the PDCO during a meeting on October 7 th The tool was assessed for clarity and completeness amongst the 24 members of the PDCO and 15 alternative members. Several modifications to the proposed list of comparator justification criteria were requested and the tool was presented in both the form of a flow diagram and a checklist. The preferred format was a flow diagram (in comparison with a checklist) and numerical bullets were removed as they appeared to insinuate a hierarchy of importance of the items. All items were deemed equally important and no deletions were requested. Considerations including Is the disease adequately defined? and has efficacy been demonstrated? were added. For the flow diagram the term control arm was preferred to comparator arm. One clarification was made to the term study (termination) endpoint with regards to offering rescue treatment and when rescue treatment would be offered to patients. GRiP-D4.09-Recommendations for an appropriate comparator definition 21

22 Figure 3. A tool for justifying a control arm in paediatric clinical trials 4. Discussion and limitations Comparator selection was a point of discussion and debate between the sponsor/industry and regulators in 28% of our selection of PIPs discussed for adoption by the PDCO in It is important to consider the source of this controversy, and whether it be variability in end goal, ethical concerns, formulation concerns, or scientific discourse with study design. Below we will review the various different types of concerns including how they relate to the scientific literature and current regulatory guidance on justifiable comparator selection. All unique active comparators evaluated had obtained a marketing authorization in at least one European country for any indication. This, however, did account for use in alternative age groups or disease categories. Types of challenges specific to paediatrics Infectious disease trials appear to present a particularly challenging disease area for comparator selection in our sample. This may result from variability in disease aetiology and prevalence in multiple jurisdictions. Selection of an appropriate comparator is essential for producing high impact, reproducible clinical trials in order to inform change and improve health outcomes in children. Currently, there is a knowledge gap in regards to how to justify comparator selection in paediatric clinical trials. Comparator selection must consider the general scientific principles reducing comparator selection bias, the location of the trial and include paediatric appropriate outcome measures. If a trial is to be completed in multiple locations with varied standard of care practices or differences in GRiP-D4.09-Recommendations for an appropriate comparator definition 22

23 disease susceptibility it may be appropriate, while ideal for data analysis, to select multiple comparator arms to account for these jurisdictional differences. Furthermore, variability in marketing authorization may prevent the use of the same comparator in all countries for which the trial plans to recruit. This work also identified examples where unlicensed medicines were deemed as acceptable active comparators given previously obtained high-quality evidence of safety and effectiveness. Outcome measures should be objective and outcome assessors should be blinded where possible, regardless of whether there is an active or placebo comparator. As up to 50% of medication prescribed to children is off-label [11], PK and PD data are rarely available which may explain why there was little paediatric PK/PD data presented in the population of interested. To circumvent this lack of evidence, data was presented in adults and other paediatric age groups which were deemed appropriate extrapolations by the PDCO. Further generation of age appropriate PK/PD data in children is an essential for comparator justification and in improving medicines for children [11, 12]. Data from MAs underline that the main problem in children research is the scarcity of well conduct controlled trials and of evidence-based motivations (no systematic reviews were identified in our search) to support the choice of the comparator. Moreover, in the MA scientific discussion some crucial information such as the treatment currently available in the countries in which the trial is conducted or justification for comparator dosages are available. However, when accepted by the Regulatory Authority as part of the approval dossier, the comparator demonstrates to be in line with the main regulatory requirements. Scientific literature and guidance While there are often scientific advantages to using placebo, there is occasionally a question of ethical concern, particularly as a factor which influences parents attitudes towards enrolling their children in clinical trials [13]. Ethical considerations for placebo use in children have been previously summarized [14]. Placebo trials are often necessary to show effectiveness and are generally associated with adverse effects. According to the International Committee on Harmonization the use of placebo is ethically acceptable when there is no serious harm of withholding effective treatment [15]. In this context serious harm is defined as risk of death or irreversible morbidity. This statement applies to use in paediatrics and should further consider impact on development. When justifying the choice of a placebo control in paediatrics, one must also consider what treatments are currently available in the jurisdiction where the trial is recruiting (as standards of care may vary). When there are proven effective therapies, use of placebo can be considered if both the new treatment and placebo are given as an add-on to the standard-of care treatment(s), as was seen more frequently than placebo use alone in 2013 PIPs. This report suggests that modifying study design, including randomization ratios, can improve methodological justification for placebo use by regulators. Initiating an independent working group to discuss ethical concerns can shed light on these considerations and should include clinicians, ethicists and parent/patient groups. In summary, the selection of an appropriate comparator is essential for producing high impact, reproducible clinical trials in order to inform change and improve health outcomes in children. Comparator selection must consider the general scientific principles reducing comparator selection bias, the location of the trial and include paediatric appropriate outcome measures which account for the synergistic potential between treatment and control. Supporting clinical researchers, industry, financial sponsors and regulators in selecting the GRiP-D4.09-Recommendations for an appropriate comparator definition 23

24 right comparator is critical to facilitate the design of informative paediatric clinical trials leading to optimal health outcomes for children. Limitations Although the pediatric investigation plans that were evaluated covered a broad range of disease areas the snapshot time frame of 2013 didn t allow for an evaluation of justification criteria changes over time. Evaluating the impact and uptake of guidance on comparator selection will be an important component of determining transparent comparator selection. Similarly, the small sample of paediatric marketing authorizations did allow for insight on the trends in comparator selection. To this aim, an analysis of comparator choice in a wider time span would be advisable in the future. In conclusion, this study identified challenges and compromises between regulators and sponsors regarding the selection of a comparator arm in paediatric trials. Justification criteria presented to the EMA in support of comparator selection were evaluated. An evidence-based tool was generated and reviewed which provides justification criteria for the selection of a comparator arm in paediatric clinical trials. Transparent justification for the selection of a comparator arm improves transparency and trial reproducibility leading to reduced research waste and increases the impact of the results. GRiP-D4.09-Recommendations for an appropriate comparator definition 24

25 Setting the research agenda When selecting a comparator, it is important that trialists provide evidence for justification as to why this comparator ethical and scientifically sound choice within the context of the trial. Clarity and transparency will enhance the trial quality and impact. The first step in selecting a comparator should be to synthesize all ongoing and previously reported trials in the population of interest. Involving patients in the comparator decision has the potential to create more meaningful trials [16]. The following activities are recommended to follow the development of this work to date: This tool to select justifiable comparators in paediatric clinical trials should be further validated by regulators and members of the EMA PDCO as a tool to assess the evidence provided by submitting sponsors in support of a comparator selection. What is the utility and benefit of using this tool when reviewing paediatric clinical trials? This tool also requires evaluation by paediatric clinical trialists in industry and academia to assess the utility of this tool in facilitating transparent evidence-based comparator justification. Patient or parent/care giver consultations were rarely reported prior to comparator selection in the PIPs assessed. What is the utility and benefit of using this tool when designing paediatric clinical trials? Clarification regarding defining the standard of care is desperately needed in paediatrics, as currently most local institutions have their own independent process for selecting treatment options, leading to variability across sites even within the same country. In order to support evidence-based comparator selection a tool and guidance document are needed. How much evidence, and what kind of evidence justifies a standard of care? Only slightly more than half of comparator selection was supported by systematic literature reviews which is concerning. Literature reviews ensure that all previously available evidence has been considered and are a critical step in planning randomized controlled trials [17]. The challenge may be unawareness of the importance of a priori evidence synthesis. What can be done to implement best research practice? GRiP-D4.09-Recommendations for an appropriate comparator definition 25

26 Document history Date Author Changes 17-May-2013 Offringa Original version 28 May 2013 Offringa Edited Methods section 29-May-2013 Chan, Offringa Reviewed and edited document 14-June-2013 Chan, Offringa Updated document, developed protocol 5-July March Chan, Offringa, Davies Chan, Offringa, Davies Reviewed and edited document Finalization of methods 26- June-2014 Kelly, Offringa Updated document Added controversy surrounding comparator use and the choice of placebos - Citations list extended Added discussion surrounding definition of standard of care Updated CRF 1 and CRF2 Protocol for review (methods) amended to include placebo-controlled trials as well active comparators. 09-Sept-2014 Tomasi, Saint- Raymond Reviewed and edited document 17-Sept-2014 WP 4 executives Protocol approved by Paola Baiardi & work package 4 executives 21-Sept-2014 Kelly, Edited methods and initiated PIP investigation Davis, Tomasi 04-Nov Kelly, Davies Updated document Address comments on earlier versions 04-Dec-2014 Kelly PIP empirical work added 21-January-2015 Kelly, Felisi, Ceci 15-March-2015 Kelly MA empirical work added 14-May 2015 Kelly, Davis, Tool first drafted Data sharing agreement (SickKids-CVBF-Teddy) signed and methods for MA review approved 02-Sept Sept Jan-2016 Kelly, Saint- Raymond, Offringa Kelly, Offringa Ceci, Ruggieri, Kelly Teleconference to draft tool for selection justifiable comparators in paediatric clinical trials, tool added into deliverable Final data added, formatting changes and closure of deliverable for September 30 th deadline. MA data analysis expanded and final submission to GRiP Executive board. GRiP-D4.09-Recommendations for an appropriate comparator definition 26

27 References 1. European Medicines Agency. Ethical considerations for clinical trials on medicinal products conducted with the paediatric population 2008; Available from: ftp://ftp.cordis.europa.eu/pub/fp7/docs/ethical-considerations-paediatrics_en.pdf 2. Mann H, Djulbegovic B. Comparator bias: why comparisons must address genuine uncertainties. Journal of the Royal Society of Medicine Jan; 106(1): American Academy of Paediatrics Paediatric Drugs Committee. Off-Label Use of Drugs in Children. Paediatrics March 1, 2014; 133(3): European Medicines Agency. Reflection paper on the need for active control in therapeutic areas where the use of placebo is deemed ethical and one or more established medicines are available European Medicines Agency. ICH Topic E10 Choice of Control Group in Clinical Trials 2001; Available from: /09/WC pdf 6. Mann H, Djulbegovic B. Choosing a control intervention for a randomised clinical trial. BMC medical research methodology Apr 22; 3:7 7. Temple RJ. Special study designs: early escape, enrichment, studies in nonresponders. Communications in Statistics - Theory and Methods /01/01;23(2): Pearson SD. Placebo-controlled trials, ethics, and the goals of comparative effectiveness research: comment on "lack of head-to-head trials and fair control arms". Archives of internal medicine Feb 13;172(3): Tramer MR, Reynolds DJ, Moore RA, McQuay HJ. When placebo controlled trials are essential and equivalence trials are inadequate. Bmj Sep 26;317(7162): Glass KC. Ethical obligations and the use of placebo controls. Canadian journal of psychiatry Revue canadienne de psychiatrie Jul;53(7): t Jong GW, Klassen TP, MacLeod SM. A landmark report on improving medicines for children. JAMA pediatrics Mar;169(3): Rose AC, Van't Hoff W, Beresford MW, Tansey SP. NIHR Medicines for Children Research Network: improving children's health through clinical research. Expert review of clinical pharmacology Sep;6(5): Buscariollo DL, Davidson MA, Black M, Russell WE, Rothman RL, Moore DJ. Factors that influence parental attitudes toward enrollment in type 1 diabetes trials. PloS one. 2012;7(8):e Díaz RAG. The ethical use of placebo in pediatric research. Journal of Clinical Research & Bioethics ICH Harmonised Tripartite Guideline: guideline for good clinical practice. Journal of postgraduate medicine Apr-Jun;47(2): Goodare H, Lockwood S. Involving patients in clinical research. Improves the quality of research. Bmj Sep 18;319(7212): Sutton AJ, Cooper NJ, Jones DR. Evidence synthesis as the key to more coherent and efficient research. BMC medical research methodology. 2009;9: Taubes G. Use of placebo controls in clinical trials disputed. Science Jan 6;267(5194):25-6. GRiP-D4.09-Recommendations for an appropriate comparator definition 27

28 19. Temple R, Ellenberg SS. Placebo-controlled trials and active-control trials in the evaluation of new treatments. Part 1: ethical and scientific issues. Annals of internal medicine Sep 19;133(6): World Medical Association declaration of Helsinki. Recommendations guiding physicians in biomedical research involving human subjeclinical trials. JAMA : the journal of the American Medical Association Mar 19;277(11): Ellenberg SS, Temple R. Placebo-controlled trials and active-control trials in the evaluation of new treatments. Part 2: practical issues and specific cases. Annals of internal medicine Sep 19;133(6): Otto MW, Nierenberg AA. Assay sensitivity, failed clinical trials, and the conduct of science. Psychotherapy and psychosomatics Sep-Oct;71(5): Stang A, Hense HW, Jockel KH, Turner EH, Tramer MR. Is it always unethical to use a placebo in a clinical trial? PLoS medicine Mar;2(3):e Baron JA, Sandler RS, Bresalier RS, Lanas A, Morton DG, Riddell R, et al. Cardiovascular events associated with rofecoxib: final analysis of the APPROVe trial. The Lancet. 2008;372(9651): Lasagna L. Placebos and controlled trials under attack. European journal of clinical pharmacology Jul;15(6): Hill AB. Medical ethics and controlled trials. British medical journal Apr 20;1(5337): Rothman KJ, Michels KB. The continuing unethical use of placebo controls. The New England journal of medicine Aug 11;331(6): Glass KC, Waring D. The physician/investigator's obligation to patients participating in research: the case of placebo controlled trials. The Journal of law, medicine & ethics: a journal of the American Society of Law, Medicine & Ethics Fall;33(3): Glass KC, Freedman B. Legal liability for injury to research subjects. Clinical and investigative medicine Medecine clinique et experimentale Apr;14(2): Wilkinson D, Karim SS, Coovadia HM. Short course antiretroviral regimens to reduce maternal transmission of HIV. Bmj Feb 20; 318(7182): Benatar SR, Singer PA. A new look at international research ethics. Bmj Sep 30; 3 21(7264): Shea BJ, Bouter LM, Peterson J, Boers M, Andersson N, Ortiz Z, et al. External validation of a measurement tool to assess systematic reviews (AMSTAR). PloS one. 2007;2(12):e Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC medical research methodology. 2007; 7: 10. GRiP-D4.09-Recommendations for an appropriate comparator definition 28

29 Appendices Appendix I. Current controversy regarding comparator choice in clinical trials There are several schools of thought surrounding the choice of comparators in clinical trials. The use of placebo controls is one of the most controversial issues faced by researchers today [18]. The arguments consider both an ethical and scientific perspective and include positions both for and against the use of placebo in trials addressing similar research questions. Mann and Djulbegovic suggest that the choice of comparators must address genuine uncertainty and that both clinicians and patients must be genuinely uncertain of which treatment is superior [2]. This principle of equipoise is generally agreed upon for the choice of a comparator for both active and placebo-controlled trials. Furthermore, there is universal agreement that in cases where therapy has been shown to improve survival or decrease serious morbidity the use of no treatment or placebo alone is unethical [19]. Controversy surrounding the use of placebo alone stems from a statement made in the Declaration of Helsinki which reads In any medical study, every patient including those of a control group, if any- should be assured of the best proven diagnostic and therapeutic method [20]. Arguments in support of placebo controlled trials There are those who argue that trials comparing a treatment for symptom relief with placebo are ethical so long as the patients are adequately informed of the risks of delaying treatment [19]. Authors also argue that there are many instances where active control trials are unable to produce meaningful scientific findings raising separate ethical concerns [21]. These concerns stem from equivalence trials lacking assay sensitivity and the term non-inferior leading to an erroneous conclusion of efficacy and overestimating treatment effects [19]. Assay sensitivity is the ability of a trial to differentiate between an effective and ineffective treatment and is an essential component of internal clinical trial validity [22]. An example of misleading conclusions in the absence of placebo is seen in the Vioxx Gastrointestinal Outcomes Research (VIGOR) trial where rofecoxib (Vioxx) was compared with an active comparator (Naproxen). The trial concluded that there was a fivefold difference in the incidence of myocardial infarction(mi) but failed to differentiate between an increased risk for MI with rofecoxib and a decreased risk for MI with Naproxen, thus rofecoxib continued to be prescribed [23]. Four years following the VIGOR trial, Merck announced its withdrawal of rofecoxib due to increased cardiovascular risks after tens of millions of patients received the drug. The rofecoxib withdrawal was based on the results of a placebo-controlled trial [24]. The use of placebo is also justified by some when the condition of interest shows a high rate of spontaneous improvement [25]. In conditions such as depression or nausea, spontaneous improvement can occur, and the inherent variability of the condition studied can result in problems establishing assay sensitivity against an active comparator. In contrast to active comparators, placebos are generally associated with adverse drug reactions. Due to a usually smaller effect size seen between two active treatments a larger sample size is required to reach significance. In placebo controlled trials the anticipated effect size is larger and can be supported by fewer patients. Therefore, if a treatment is unsafe, fewer patients would be exposed in an adequately powered trial using placebo than with that active control. Manufacturers also argue that comparing their new treatment to an old one is a disadvantage as older treatments had only to show superiority over placebo [23]. For practitioners it is important that more than one effective therapy be GRiP-D4.09-Recommendations for an appropriate comparator definition 29

30 available for each condition as patients often respond differently to various treatment regimens. Authors also argue that alternative study designs can limit the length of exposure to placebo, use placebo as add-on therapy, and can include early escape protocols. Early escape protocols include a well-defined treatment failure endpoint where patients benefiting from their assigned treatment can change therapy [7]. Active controlled trials also receive scrutiny as the standard of care is often variable depending on jurisdiction, based on experience rather than evidence, and the most relevant active comparator changes over time [8]. The ethical acceptability of placebos is dependent on the population of interest and the overall research outcomes [9]. There is, however, no disagreement that placebo use is justified in conditions where there is no established effective therapy or the current local standard of care offers a questionable risk-benefit ratio to the patients [5, 10, 21]. Arguments against the use of placebo or no treatment A pioneer of modern clinical trials, Sir A Bradford Hill suggested that when an orthodox treatment of proved or accepted value is available, the doctor will wish to know whether a new treatment is more, or less effective than the old, that it is more effective than hing [26]. This argument is made against the use of no treatment as a comparator. Kathleen Cranley Glass supports this statement with regards to placebo use in the absence of standard of care and argues that the scientific validity argument often made in support of placebo-controlled trials is an insufficient requirement for unethical trials. She also states that even if placebos were required for methodological rigor, this would make such trials ethical [10]. The Canadian Medical Association s Code of Ethics lists Consider first the well-being of the patient as a rudimentary responsibility for health care providers and many argue that placebo is commonly in the patients best interest [10]. Along similar lines the American Medical Association s Code of Ethics mirrors that of the Canadians, stating that the physician-patient relationship exists and that professional judgment and skill must be executed in the best interest of the patient [10]. Arguments against placebo use include that hiding behind informed consent passes the ethical burden directly to the patients [27]. Patients are rarely as informed as the physicians recruiting them into the study, despite the investigators best efforts. It is commonly presented that patients are more likely to choose to participate in a trial with two active treatment arms. Rothman and Michels also suggest that patients may be more inclined to participate in any trial, regardless of comparator. Their desire to be in a clinical trial may persuade them to agree to placebo controlled trials without adequate comprehension of the risks of delaying treatment [27]. Ethical counterarguments to the use of placebo also include that patients ought to face unnecessary pain or disease on account of a trial and that in some cases withholding effective treatment can lead to serious long term consequences [10]. Cases of harm have been documented by Glass and Waring resulting from failure to treat patients in placebo-controlled studies [28]. Questions regarding the legal liability for harm done to a research participant by withholding treatment have also been raised [29]. In regards to the alternative placebocontrolled study designs including withdrawal clauses, often patient inconvenience or harm experienced before their removal from the trial is ignored. In summary, those who argue against the use of placebo alone feel as though clinical trials are an excuse to practice substandard medicine [10]. GRiP-D4.09-Recommendations for an appropriate comparator definition 30

31 Appendix II. Defining standard of care in paediatrics Further clarification is required regarding the definition of standard of care. If the standard of care is to be used as a comparator this decision must be justified. The importance of a clear definition for standard of care is highlighted by many trials in the developing world where standard practices often differ between regions. In 1997, attention was brought to researchers testing an intervention to prevent HIV transmission during pregnancy [30]. These trials were using a placebo comparator even though an effective treatment (azidothymidine, AZT) was considered to be standard of care in developed nations. Researchers argued that the developing countries in which the trials were taking place could afford the expensive AZT regimen that standard of care in this region was no treatment at all, and that trial participants were worse off receiving placebo. Access to medical care, the structure of the health care system as well as biologic considerations has the potential to impact the designation of a treatment as standard of care. It is important that the designated standard of care consider the political, economic and social conditions in the region which the study is recruiting patients [31]. Accounting for local factors will ensure that the research study question and overall findings are relevant to the local population The American Academy of Paediatrics supports that sound scientific evidence, solely label indication, should be used to determine the standard of care from which clinicians make their treatment decisions [3]. For the purpose of the current investigation, standard of care will be evaluated using the following criteria: a. Commonly used therapy in jurisdiction of trial (evidence based) b. Available Meta-analysis / Cochrane systematic review c. Available Evidence from at least one RCT published in a peer reviewed journal d. Follows Clinical Practice Guidelines (National/other) recommendations e. Available Monographs/Marketing authorization f. Drug has been prescribed in children for many years (experience based) Appendix III: Case Report Form Part A The Case Report Form A will be used in analysis of all selected MAs and PIPs. Any systematic reviews cited in justification of comparator choice will be assessed for quality using the AMSTAR standards found in Appendix 2. Further clarification regarding the definitions used in the case report forms are listed below: Alternative use: use in any population for which marketing approval has been obtained Assay Sensitivity: the ability of the outcome measures used in the trial to differentiate between the treatment and control arm Authorized: an indication for which marketing approval has been obtained in that patient population (age-group and disease severity specific) Equipoise: after considering all of the relevant existing evidence, clinicians and patients are genuinely uncertain regarding which treatment is more favourable Other populations: any population other that the current study population which could include adults, animal studies or studies in children with different disease states Standard of care : a commonly used treatment based on evidence which will be assessed on a case by case basis using the criteria specified in Section 5.4 GRiP-D4.09-Recommendations for an appropriate comparator definition 31

32 Study population: current population of interest. This population should consist of only individuals meeting the inclusion criteria set a priori. Inclusion criteria regarding age rages, disease severity. This population must meet any of the exclusion criteria. Case Report Form A Identifier: Country of Approval: Date of Approval: 1. Study Characteristics 1 Type of Comparator(s): 2 Study Sponsor: 3 Type of Trial (cross-over, factorial, three arm, withdrawal design, dose-response): 4a. Trial Design: 1 = superiority, 2 = equivalence, 3 = non-inferiority 4b. Trial Design: Length of treatment 5 Disease of Interest: 6 Total number of participants consented: 7 Age of Participants: 8 Inclusion criteria (disease severity, diagnosis): 9 Exclusion criteria: 10 Intervention Details (drug, dose, route, duration): 11 Control Details (drug/placebo, dose, route, duration): 12 Primary Outcome Measure: 13 Timing of Outcome Assessment: 14 Has assay-sensitivity been established? Evidence for assay-sensitivity: 15 Measurement Instrument(s): 2. Regulator Guideline Compliance 16 Was the study prospectively registered in a trial database? Trial database: 17 Are the findings of the study published? Please Cite 3. Comparators 18 Has there been a MA granted for the comparator? 19 Is the comparator used according to the authorized indication? 20 Is PK/PD data referenced for study population of interest? 21 Are inferences made from PK/PD data in other populations? 22 Is disease aetiology similar in study population compared to other population? 23 Is the comparator used according to authorized dose? 24 Is the comparator used according to authorized formulation? 25 Was a systematic review completed for the comparator in the population to be studied? 26 Is the systematic review valid with respect to methodology (AMSTAR score/11) 27 Are the findings of the systematic review published? Please Cite GRiP-D4.09-Recommendations for an appropriate comparator definition 32

33 28 Did the findings of the systematic review identify an optimal effective dose in the study population of interest? 29 Is the dose identified in the systematic review the dose used in the clinical trial? 30 Was the systematic review completed or sponsored by the same company funding the project? 31 Was the systematic review completed independently or by a third party? Regarding alternative use: 32 When an alternative use (dose, route or application) was applied to that approved in the MA, was a systematic review completed for the alternative use? 33 Is the systematic review regarding alternate use valid with respect to methodology (AMSTAR score / 11) 34 Are the findings of the systematic review on alternate use published? Please Cite 35 Was the systematic review completed or sponsored by the same company funding the project? 36 Was the systematic review completed independently or by a third party? 37 Are other clinical trials (ongoing/completed) that have used this comparator in the study population? 38 If no systematic review exists, are there any randomized controlled trials that evaluate the comparator? 39 Are the findings of the trial published? Please Cite 40 Was the randomized controlled trial completed by the same company funding the project? 41 Was the randomized controlled trial completed independently or by a third party? 4. Justification Framework 42 Were the investigators in equipoise (valid uncertainty) at the time of the study design? If no, please discuss 43 Was a placebo used as a comparator? 44 Was the decision for placebo use justified 45 Explain justification for placebo 46 Was an active control arm used? 47 Was the decision for active control use justified? Explain justification for active control 48 Were both a placebo and active control feasible? If yes, complete Case Report Form part B to evaluate the process of comparator choice 49 Was the standard of care used as a comparator? 50 Was the decision justified? Explain the justification of the standard of care: 51 What was the standard of care in the jurisdiction(s) where the trial was conducted? 52 Is there national/international guidelines that recommend standard of care /use of comparator? 53 Were national/international guidelines followed in terms of dose/regimen? GRiP-D4.09-Recommendations for an appropriate comparator definition 33

34 GriP Global Research in Pediatrics Appendix IV: AMSTAR: Assess Methodological Quality of Systematic Reviews [32, 33] GriP D4.09-Recommendations for an appropriate comparator definition

35 32 GRiP-D4.09-Recommendations for an appropriate comparator definition 35