Adaptive Design for Medical Device Development A guide to accelerate clinical development and enhance portfolio value
Executive Summary In May 2015, the FDA released a draft guidance document regarding the use of adaptive designs for medical device trials, including premarket approval, 510(k), de novo, humanitarian device exemption and investigational device exemption submissions. The adoption of adaptive design for the development of a medical device offers a number of valuable benefits. These include improved device development efficiency, increased odds of pivotal trial success, shorter time-to-market, and greater decisionmaking opportunities to optimise investment in a portfolio of products. This white paper provides insight into the value of adaptive designs for medical device development, requirements for implementation, and case studies of device adaptive design trials conducted by ICON. It has been written to help senior R&D leaders understand the critical role of adaptive design in medical device development and appreciate the value that the approach can generate. An Introduction to Adaptive Design In the FDA s draft guidance Adaptive Designs for Medical Device Clinical Studies, the FDA defines adaptive design as a clinical trial design that allows for prospectively planned modifications based on accumulating study data without undermining the trial s integrity and validity. The FDA issued the document to encourage the use of adaptive designs, when appropriate, to improve efficiency and increase the likelihood of study success when conducted in a pre-specified, thoughtful, scientifically valid manner. From a commercial point of view, adaptive designs offer significant value not just for the efficiency and success of a single trial, but also the performance of a large portfolio. In short, adaptive design can: Sample size reassessment designs can also mitigate the costs and delays associated with overpowered trials and unnecessary recruitment. Improving the Chances of Success Adaptive designs can increase pivotal trial success by allowing preplanned sample size reassessment, which can transform an underpowered study likely to fail into a wellpowered study that has a higher chance to succeed. Adaptive designs may also be used to improve the chances of success of a device more likely to be effective only in a subgroup of patients. By using an adaptive enrichment design, study inclusion/exclusion criteria may be modified in a preplanned way by using unblinded data at one or more interim looks. Reducing Time to Market Adaptive designs can improve time to market by shortening planned development time by reducing white space. Adaptive designs offer the possibility to combine, in a prospectively planned manner, a late phase pilot study into a pivotal study to shorten the overall development process. Growing Portfolio Value Choosing which development candidate to back when there is a large portfolio of products competing for a fixed level of investment can be a difficult and complex process. The adoption of an adaptive design strategy can augment opportunities and data for the critical decision-making required to deliver an optimised pipeline of products. Expanded Portfolio Value from Adaptive Designs 1. Enhance development efficiency by reducing costs and trial duration 2. Maximise the probability of success of an individual development programme 3. Reduce time to market by removing white space and delays 4. Grow portfolio value and the productivity of fixed resources by optimising investment decision-making Enhancing Development Efficiency Adaptive design trials enhance development efficiency by allowing preplanned early stopping of a clinical trial for effectiveness, which leads to shorter development time. Alternatively, in the case of an ineffective device, a preplanned early stop for futility can preserve millions of dollars that may otherwise have been spent on the trial. Portfolio Value Portfolio Using Only Conventional Designs Shorter Time to Market Stronger Clinical Evidence More Products Developed and Commercialised Higher Early Product Valuations Portfolio that Incorporates Adaptive Designs
Relevant Types of Adaptive Designs When is an Adaptive Design Appropriate? The FDA, in its draft guidance on adaptive design, highlighted a concept called anticipated regret to weigh the appropriateness of adaptive design for a trial. In this approach, one anticipates study outcomes that could lead to failure and asks what one may have regretted in the planning phase that led to this outcome. For example, in many cases there is a certain level of uncertainty regarding the assumptions driving study sample size, such as the anticipated level of treatment effect or the variability of the primary endpoint. In these cases an adaptive design with planned sample size reassessment would increase the probability of study success. A final assessment of whether an adaptive design is appropriate for a particular trial and the business objectives of a sponsor requires expertise in the statistical, medical and operational aspects of adaptive device trials. Those knowledgeable about adaptive designs will assess factors such as the duration of endpoints and recruitment. Studies with shorter endpoints and longer recruitment times, for example, would be more suitable for adaptive designs than studies with very fast recruitment and short endpoints. Types of Adaptive Designs The choice of an adaptive strategy can be achieved in a stepwise fashion and requires input from statistical, medical, clinical development, and regulatory strategy experts familiar with adaptive design. Many potential adaptations are possible, all based on predetermined thresholds. Adaptations can take many forms, ranging from blinded sample size reassessment to group sequential designs or changing study hypotheses. The most valuable types of medical device adaptive designs are: Group Sequential Design: Interim analyses allow the study to stop early for either success or futility. If the device is performing better than expected, the study can stop early, which saves resources and allows the device to get to market faster. Similarly, if the device is performing worse than expected, the study can stop early allowing resources to be allocated to more promising projects. Preset threshold allows early termination for efficacy or futility. Fail Early, Fail Fast, Succeed Faster Changing Study Hypothesis: Studies can be planned to investigate both superiority and non-inferiority. A study can be designed as a superiority trial that allows investigating non-inferiority in the event that the superiority hypothesis fails. This allows for the largest chance of study success while also providing the opportunity to make the strongest claim possible. Non-inferiority Trial Superiority Trial If the preset threshold is met, the study continues, allowing collection of superiority data in a single trial. Early trial data that demonstrates non-inferiority can be used for immediate regulatory filings. Faster to Market with Potentially Stronger Claims Changing the Randomisation Ratio: A study can be designed with an adaptive randomisation plan that allows changing the randomisation ratio between treatment arms during the course of the study based on treatments outcome (e.g., from a ratio of 1:1 to 2:1). By altering the allocation, study efficiency can be improved. Furthermore, the study may experience better investigator and patient enrolment knowing that the allocation can be altered to enhance patient benefit and protection. Many potential adaptations are possible, all based on predetermined thresholds. 1:1 1:1 2:1 Preset threshold allows allocation ratio to be adjusted based on treatment outcome. Improved Study Efficiency and Enrolment
Sample Size Reassessment: Assumptions surrounding the sample size calculation (e.g., effectiveness rates or standard deviations) can be examined during the course of the study and then adjustments can be made to right-size enrolment. This allows underpowered trials to be rescued, saving sponsors the time and money associated with running another properly powered trial. N = 100 Implementation Requirements and Challenges An adaptive design trial does require upfront investment of time in order to perform the simulations necessary to identify the risks that may arise over the course of a trial and develop a design that optimally addresses those risks. This investment is offset by adaptations that, for example, compress study duration through combined pilot and pivotal studies, adjust the sample size to preserve statistical significance, or prevent an overpowered design from continuing to recruit unnecessary subjects. N = 80 N = 85 Study design assumptions can be checked and, subsequently, the study correctly sized. Curtailed Recruitment / Risk of Inconclusive Result Proper implementation of adaptive design also requires a specialised team of statisticians, clinical operations experts, and technologies to maintain trial validity and integrity, especially during interim analyses. The team must be knowledgeable in device-relevant adaptive trials, as the same strategies and operational processes that are appropriate for pharmaceuticals do not necessarily translate to benefits in device development. Adaptive Enrichment: In cases where the device may work better in a particular subgroup when compared to the total patient population, a study can be designed to adjust the inclusion/exclusion criteria to limit enrolment to the appropriate subgroup. This allows for the study to demonstrate a success claim in a responder group and prevent failure or underperformance due to the dilution effect of irrelevant patient subpopulations. Full Patient Population The FDA draft guidance defines two key requirements that govern the use of adaptive design trials to ensure the clinical study produces valid scientific evidence: (1) control of the chance of erroneous conclusions (positive and negative) (2) minimisation of operational bias In adaptive designs, controlling false positive conclusions can be a statistical challenge. With preplanning and statisticians versed in adaptive design, however, these types of error can be well controlled. Relevant Population Study inclusion/exclusion criteria can be adjusted to limit enrolment to the relevant patient subgroup for the device Demonstrate Success in a Responder Group, Prevent Dilution Effect from Irrelevant Subpopulations Operational bias is defined as the bias that arises because some or all participants (investigators, patients, caregivers) in the study have access to study results by treatment group and this information has the potential to influence the on-going operations of the study. Operational bias can be a significant threat to the scientific integrity of a clinical study and cannot be overcome by statistical adjustments to account for its presence. It requires robust, pre-specified and well-documented procedures and firewalls to protect access and control dissemination of unblinded information and results. Data monitoring committees (DMCs) will generally be the appropriate entity to implement all pre-specified study adaptations decision rules. The DMC should be appropriately constructed to assure that its members possess the necessary expertise and experience for an adaptive study design.
Technology Infrastructure Specialised infrastructure is required for design, analysis, and execution of adaptive design trials. The design, simulation, and analysis of adaptive trials require statistical software validated for the specific chosen design. While academic statistical packages may be available for certain designs, commercial software packages validated on previous trials have been licensed by regulatory agencies worldwide to assess submitted designs. The execution of interim analyses and adaptations is not advisable through conventional trial management infrastructure, which may fail to protect the integrity of the trial. An appropriate platform should be validated for harmonisation of data monitoring committee access, project management, data management, and site monitoring requirements that are unique to adaptive designs. It must incorporate electronic data capture, randomisation and inventory management, data-driven monitoring, and realtime data cleaning. Outsourcing the design and execution of adaptive design clinical trials to an experienced CRO is one way to immediately derive the benefits of adaptive designs without having to invest in infrastructure. ICON can provide both a validated platform for execution of adaptive trials as well as statistical software, called ADDPLAN, for the design and analysis of adaptive trials. ADDPLAN is fully validated, regulatory compliant software that is currently licensed by more than 50 pharmaceutical and medical device companies, as well as by the FDA, EMA, and Japan s PMDA. Risk-Based Monitoring The CDRH, in its draft guidance on adaptive designs for medical device clinical studies, has advised device companies to have a risk-based monitoring plan in place which focuses on specific aspects of adaptive studies that are of particular importance and may not be present in traditional (non-adaptive) trial designs. ICON s ICONIK integrated informatics platform enables riskbased monitoring, utilising an advanced approach called Patient Centric Monitoring. Patient Centric Monitoring is designed to prevent, detect, mitigate and learn from risks and errors in the conduct of clinical trials, whilst maintaining inspection readiness in line with regulatory guidance. ADDPLAN is fully validated, regulatory compliant software currently licensed by 50+ pharmaceutical and medical device companies, the FDA, EMA, and Japan s PMDA
Case Studies The following case studies are examples of trials designed and conducted by ICON for medical device manufacturers applying an adaptive design. Significant benefits were obtained in following these strategies. Case Study #1 Mitigating Development Risks through Adaptive Design Background A U.S.-based manufacturer developing an electrostimulation device for chronic pain was challenged with designing a trial that supported registration of its device as early as possible. A conventional trial strategy would be to have a multi-centre, randomised, double-blind, sham-controlled trial with a fixed sample size. The analysis of this type of trial would be conducted following the completion of all subjects in the trial. Adaptive Design Strategy The sponsor had multiple aims, including: 1. Ensure that the sham device was working properly during the initial pilot phase 2. Offer the possibility to end the trial early if the device is performing well and well compared to the sham 3. Allow the trial the ability to right-size to have an adequate number of subjects enrolled to make proper conclusions To accommodate these aims, a group sequential adaptive design trial was constructed that had two interim analyses. For the first interim analysis, which represents a pilot for the device, the sham success rate was examined. If this rate were to be too high, the study would be terminated. For the second interim analysis, stopping rules were created to allow for early termination if the device performed above expectations (success) or below (futility). Additionally, at the second interim analysis, if the study were to continue, the assumptions regarding the sample size were re-examined to ensure that additional subjects are enrolled to offer 90% power upon study completion. Outcome ICON advised the client to terminate the study early. This saved the manufacturer $1.9 million and five months of development time, which would be reinvested to accelerate the development of an improved device. At the second interim analysis, after enrolling only half of the total planned number of patients, ICON s proprietary adaptive design software platform, called ADDPLAN, identified a response rate much lower than the prospectively planned threshold. The manufacturer deemed the medical device ineffective and, with the financial resources and time saved due to early termination of the study, accelerated development of an improved treatment regime and delivery methodology that would be more efficient to prove in a future clinical study. The expertise of ICON s team in adaptive design protected the client s development capital by failing fast, which minimised patient exposure and allowed diversion of resources to engineer a better product. Case Study 1 Decision Tree Treatment Effect > Threshold END EARLY FOR SUCCESS Study Power > 90% Sham Response < Threshold Treatment Effect Within Set Range STUDY PROCEEDS TO COMPLETION INTERM ANALYSIS 2 Enrolment Expanded Up to a Preset Limit INTERM ANALYSIS 1 STUDY PROCEEDS TO COMPLETION TERMINATE Study Power < 90% Sham Response > Threshold TERMINATE FOR FUTILITY Treatment Effect < Threshold OUTCOME The study was terminated for futility five months early and after enrolling only half of the planned patient population. This saved $1.9 million, which was reinvested to develop a better product.
Case Study #2 Accelerating Time to Market and Eliminating $5 Million in Development Costs through Adaptive Design Background An orthopaedics manufacturer developing a disc replacement device as an alternative to the widespread practice of fusion surgery in patients with degenerative disc disease was conducting a trial to test non-inferiority of its device. Promising initial data spurred plans to conduct another trial to prove superiority, a rare claim in the space. However, designing a new study and restarting engagement with new sites and patients proved costly for the manufacturer. Adaptive Design Strategy ICON s Medical Device & Diagnostics Research group proposed a midstream change to the on-going noninferiority trial to enable evaluation of superiority claims. ICON supported negotiations with the FDA to allow collection of superiority data in the study, as well as designed and executed an adaptive design that would provide a patient-sparing, efficient approach to address both claims in a single trial. The study utilised a composite endpoint of pain, neurological function, safety, and subsequent surgery. Outcome The manufacturer successfully demonstrated the non-inferiority claim and the superiority claim in the trial. ICON s adaptive design trial reduced the manufacturer s clinical development expenses by more than $5 million and accelerated time-to-market with a stronger value proposition. ICON s team provided all of the statistical, regulatory, and operational expertise and technologies to design and execute a successful adaptive trial. Case Study 2 Decision Tree LL Confidence Interval > Threshold 1 ONLY NON-INFERIORITY CLAIM INITIAL ANALYSIS ADDITIONAL SUPERIORITY CLAIM LL Confidence Interval > Threshold 2 OUTCOME Both non-inferiority and superiority claims were demonstrated in the same trial, accelerating entrance to the market with a stronger claim and saving the more than $5 million in expenses that a separate superiority trial would otherwise require. Sources 1 Draft Guidance for Industry and Food and Drug Administration Staff: Adaptive Designs for Medical Device Clinical Studies, May 18, 2015
ICON plc Corporate Headquarters South County Business Park Leopardstown, Dublin 18 Ireland T: +353 1 291 2000 F: +353 1 291 2700 devices@iconplc.com ICONplc.com/devices