The Truth About Forecasting Clinical Trial Supplies

Transcription

1 Next Generation Technologies ebook Series Part II The Truth About Forecasting Clinical Trial Supplies Trial Blazers

2 In Part I of the Next-Generation Technologies ebook series, we discussed the new class of IWR/IVRS/IRT support systems as they compare to legacy systems: uncovering the benefits as well as the limitations and challenges. We established that the nextgeneration is solving the painful problems of the current state of IVRS cutting trial setup times and adapting better to changes that can occur during the process of a trial. Click here for Part I: Next-Generation IWR/IVRS. Clinical trial supply forecasting and simulation is the focus for Part II of the Next- Generation Technologies ebook series. Clinical Trial [Supply] Simulation is the practice of modeling all important aspects of protocol design and supply chain, adding assumptions where needed and then producing an approximation of kit demand over time (in the simple case) or a daily global snapshot for each day (in a robust case). This ebook series uses concepts which will be key to understand. Some of them are technical in nature, but all result in dramatic real-world impacts and therefore should be laid out clearly. Please click on the highlighted words throughout the paper to link to the descriptions in the Important Concept section located at the back of this ebook. Please note: this ebook will sometimes use the three terms IWR, IVRS, IRT interchangeably to describe a type of system in a general sense, unless a specific interface is being discussed. 2

3 Clinical Trial Supply Simulation Not all Clinical Trial Simulators are created equal. Some are built specifically for trial startup logistics, some focus on enrollment and others focus on the possible results of adaptive or dynamic randomizations. However, none of these focus on clinical supply. The simulator discussed in this ebook takes clinical supply factors as input and generates a forecast. Hence, it s a Clinical Trial Supply Simulator. INPUT (Trial Structure and Supply Chain Details) OUTPUT (When & Where the Supply is Being Used) IVR SIMULATOR The Business Value of Simulation In a clinical trial, significant business value can be derived from the proper management of supply. In order to realize this, the complexities and challenges in clinical trials and how to overcome them must first be understood. Complexities and Challenges There is a multitude of complexities in modern clinical trials. An IVRS is built to support these complexities. Clinical Supply Simulators are built to model the IVRS. However, the scope of what may be modeled in a clinical trial is daunting. For example: 3

4 1. When a site requires a shipment and not all of the drug is available, do we ship anyway (partial shipments)? 2. A trial is dispensing different drugs in three different regions due to availability of different dose strengths, and in one country the comparator will be sourced locally. Complexities such as these are not trivial to model. Simulation cannot generate a valid forecast if these complexities are modeled incorrectly. The validity of the forecast is directly proportional to business value. The higher the validity, the higher the business value. The first level of complexity and challenge that must be overcome is trial design and logistical plan. This must be clear and well understood. Without this, little else can be realized. The second level is to understand the IVRS resupply algorithm and its control parameters. This can be particularly challenging, especially when working with multiple vendors who use different algorithms. Nevertheless, a basic understanding of the IVRS resupply algorithms and control parameters is critical for a simulator to generate a valid forecast. The third and final level of complexity and challenge is dealing with simulation tools themselves. When representing all the various levels of complexity that can occur, it is easy to produce an overwhelming amount of available functionality. This can be overcome by collaborating with simulation vendors to improve and simplify the use of their tools. Read more about our benchmarking survey by clicking here. Power and Value Despite challenges, the power of clinical supply simulation is such that it can provide value at a scale and speed unavailable in any other facet of the clinical trial business. Global pharmaceutical companies (large and small) are under tremendous pressure to operate more efficiently, even as clinical trial designs and the demands of regulatory agencies continue to increase. Operational efficiency can be realized via Clinical Supply Simulation. 4

5 Two sources of inefficiencies in trials are stockouts and unnecessary overage. Stockouts result from trials that are run with inventory levels that are too low and can yield delays and additional costs. To prevent stockouts, excess drug is produced. However, this results in unnecessary overage: excess drug that is produced and never shipped. Clinical Supply Simulation can determine the necessary BEFORE SIMULATION: Unnecessary Overage overage for a trial and thereby help a small biopharma company run a set Intuition (from experience) is used to plan how much drug is needed of trials on a budget or save a large biopharma company tens of millions of at each site - to cover predictable & unpredictable demand dollars in drug that is produced and never even shipped. The concept of necessary overage is very trial specific. In our experience, we have seen trials that require 0% overage, trials that require 1500% overage and everything in between. Each trial has the potential for both savings and risk reduction. The bottom line is that clinical supply simulation can save hundreds of thousands or even millions of dollars in a single trial and reduce the risk of stockouts. In some cases, it enables otherwise failing trials to run. This significant cost savings are possible for the low cost of a simulation tool and a couple weeks of modeling. The Technical Details of Simulation AFTER SIMULATION: Necessary Overage Monte Carlo simulation output reveals the amount of drug needed at each site - to cover predictable & unpredictable demand We now understand that while intuition is powerful and sometimes accurate for planning supply in simple trials, it is not sufficient to address complex trials. Simple math operations (+, -, *, /), on paper, in Excel, or in a purpose-built web application only produce a baseline supply consumption. Simulation produces a forecast. 5

6 Types of Simulation Simple simulation (a.k.a. statistically-driven simulation ) employs statistical formulae to approximate trial activity and does not model all the important elements of a protocol. This type of simulation (if designed correctly) can produce a more accurate forecast. Robust simulation is capable of modeling each element of the study by representing all the driving forces of a trial as objects, on each day in the future. This method is more readily understood and issues can thereby be identified and resolved quickly. INPUT STUDY DESIGN including but not limited to: Kits (dispensing units) Treatment Groups Visit Schedule, including - Durations and windows - Drop rates - Phased visits or skips - Screening, randomization (and re-randomization) Plus any further complexities that drive the progress of patients through the study. All culminating in the dispensing matrix for the trial: What kit(s) should be dispensed to which subjects by visit, by Treatment Group? What titration options may be in place, and what probabilities? What rules based on prior dosing or other classification? OUTPUT Shows when and where your supply is being used. Reduces risk of stockout with smarter use of supply. See supply problems before they happen. SUPPLY CHAIN including but not limited to: Depots Sites Shipping lines and lead times Patient enrollment (estimate) Drug lot, label and expiry information (if available) Country and site start dates Intended IVRS resupply algorithm and settings 6

7 In both simple and robust simulation, the simulator may be executed multiple times while varying the randomness in each run or vision of the future. This is called Monte Carlo simulation. Monte Carlo is especially powerful since it can vary random events like randomization, patient arrival and titration, which forces the supply plan to account for many different possibilities just like reality. Since robust simulation models all the important elements of a trial, it is a more powerful method for forecasting and monitoring clinical trials. Click here to check out our blog Taking the art out of The Art of Clinical Drug Supply Forecasting. IVRS in Simulation Since IVRS tends to be at the center of most modern clinical trials whenever clinical supply is a concern, it naturally plays a large role in simulation. The chosen IVRS typically moves clinical supply from depot to depot, from depot to site, and dispenses drug to patients at the site. In some cases it facilitates complex titration rules all the while taking into account expiry, country labeling and other characteristics that are stipulated by trial design, drug packaging, local regulatory authorities, or some combination thereof. It would be fair to say that IVRS is the driving or originating system for all actions that influence clinical supply during the normal course of a trial. A good simulator then, must act like there is an IVRS in the middle of it. In fact, most major IVRS functionality that drives drug supply must be at least approximated by the simulation, or else the simulation will not faithfully represent what actually happens during the trial which would defeat the very purpose of simulation in the first place! If IVRS functionality is the core or inside of a simulation model, a host of other capabilities form the outside layer. This Starts with the ability to introduce patients and drug into the trial via projected enrollment rates and packaged lots respectively. Once simulated sites, patients, drug, etc., are introduced, time can be moved forward, future days begin counting and these outside elements simulated patients, kits, shipments, etc. start interacting with the inside element: the IVRS. In the end of a single simulation run, one of the possible outcomes of the trial should be evident. With robust simulators, it is even possible to examine the global state of the trial and all elements in it at any future date in the study. 7

8 Parameter Driven Systems If IVRS is the core of a modern simulator, then the resupply algorithm is: the epicenter. The resupply algorithm ultimately determines how much value IVRS can bring to a sponsor in clinical supply savings. But the resupply algorithm itself is only half of the solution. The second half of the solution is the parameters and settings that control the algorithm. The best algorithm will always supply a large safety stock threshold to every site, if that is what the parameters tell it to do. From a simulation point of view, there are two aspects to the resupply algorithm that need to be addressed: 1. Is the simulator capable of matching the resupply algorithm that the sponsors chosen IVRS is using? 2. Can the simulator shed light on what settings and parameters ought to be used to drive the IVRS resupply algorithm? In order to address the first consideration, the simulation tool vendor must be aware of the resupply algorithms that exist at IVRS companies. Obtaining technical details on the various IVRS resupply algorithms can be a challenge because this topic is complex and too few people possess the knowledge. Furthermore, since IVRS companies can invent new algorithms, it can be nearly impossible to state at any given time that a simulator contains approximations of all IVRS resupply algorithms. A solution is for the simulator to contain bits and pieces of different solutions, which may be included or disabled as needed in order to cobble together an approximate match for the IVRS. In order to address the second consideration, the simulator can be employed to derive values for the control parameters that drive the resupply algorithm. The important parameters to understand are the buffers. The floor and ceiling buffers exist in order to supply unpredictable demand and thereby reduce or eliminate risk. Clinical supply simulators can automatically determine the necessary buffer stock values by country, site, or over time. This capability is the only scientific way to decide what site and depot buffers should be, and when. The alternative is to employ full time staff to continuously analyze and tune the site buffers globally which is costly, error prone and produces waste. Establishing appropriate buffers for the resupply algorithm is a requirement for running a clinical trial in order to maximize business value. In summary, the IVRS resupply algorithm and its properly configured control parameters drive supply efficiency during a clinical 8

9 trial. If the parameters are poorly configured, then risk and waste is introduced. A simulator is a convenient and inexpensive way to pilot resupply algorithms and their control parameters. Why not figure out if a resupply algorithm and a set of parameters is any good before shipping drugs globally? Parameters and Pluggable Code in Simulation In retrospect, it may seem obvious that throughout this ebook, we have been focused on parameter driven systems. The idea of putting the system directly in the hands of the sponsors or business experts, the difficulties in approximating IVRS and IVRS resupply algorithms, even the section on challenges which describes the daunting task of learning simulation tools, all imply that the tools in question must be parameter driven. A custom-programmed system for each trial would have none of these challenges although the programmers responsible for the system would certainly have to do some work! But neither would it be available as a shrinkwrapped commercial off-the-shelf product, which can be used by ordinary people to set up and simulate models of clinical trials. A subtle point emerges here: in order to put the power to model complex clinical trials into the hands of the sponsor or business experts, then the system must be parameter driven. Trials must be able to be modeled without specifications and programmers on hand. And so, most modern clinical trial supply simulators are in fact parameter driven. Parameter driven systems are by nature quite complex. They are also prone to complexities because they must stand ready to model any trial regardless of its complexity. A robust clinical trial modeling system, such as a simulator, is usually ready to model a wide variety of clinical trial and supply chain features which, on any given trial, the sponsor will not need. So a natural challenge is inherent there and techniques are currently being developed to address this. With that challenge aside, parameter driven clinical supply simulators also face the same challenge that is described previously about parameter driven systems: no matter how complex they are, they cannot be built to handle all possible cases. As a careful reader of the concepts section might assume, the solution for a fully robust parameter driven system to be able to handle unexpected features or events, is the use of custom pluggable code. Custom plugins can take a variety of forms, but in their simplest incarnation, are simply scripts, usually written in the same programming language as the product is written in; this can be inserted into the simulation then picked up and run at run-time (i.e. during the simulation itself). 9

10 With the more advanced clinical trial simulators, these plugins are able to make use of and extend or even modify some of the existing functionality of the tool itself. Ideally, these can be presented directly in an editor so that none of the usual build and deployment tasks are necessary to literally customize a running simulation product, on the fly. Make no mistake, custom plugins certainly are programming code but they are typically very small snippets of programming code which can make use of powerful system code already present in the simulation tool, and they can be carefully developed in an isolated area, either by sponsor personnel or more likely by vendor staff. With custom plugins to simulation tools, one can achieve nearly anything imaginable from simple things like sending arbitrary shipments between depots, or modifying IVRS settings on the fly based on arbitrary milestones, to modifying or even literally replacing a randomization, or dispensing or resupply algorithm with something more representative of what will actually happen in the trial. Custom plugins are not required in the majority of cases for clinical supply simulation, but they can be immensely handy when something odd pops up. Conclusion In this ebook, we have discussed the types of clinical trial simulations, methods of forecasting and simulation modeling. Further, we learned that despite the many complexities and challenges associated throughout the process, the power of clinical trial supply simulation is such that it can provide value at a scale and speed unavailable in any other facet of the clinical trial business. Part III of this ebook series will reveal the relationship between simulation and IWR/IVRS recognizing similarities, modeling structures and data processing as described; the 360 concept. 10

11 Important Concepts Concept 1: Clinical Supply Forecasting / Simulation Clinical trials involve a variety of unknowns. Where in the world will subjects enroll and at what rate? What treatment group will they randomize to? What titration path will they take within a treatment group? What about dropout rates? What unplanned visits will occur? What about expiring drug? Despite these unknowns, a clinical supply plan must be created to properly supply and run a clinical trial. In the simplest case, it is usually possible to calculate baseline drug consumption for a clinical trial by multiplying the number of intended patients in each group by the number and types of intended doses for that group. However, this ignores the complexities of how much overage to place at sites and depots, in order for the trial to run. Since drugs cannot be instantly delivered to sites where patients enroll, it usually has to be pre-positioned. This may drive the necessary or required overage for a trial that is, the amount drug that you will definitely not use in the trial but you need anyway. There are approximately four different methods that people use to calculate overage for a given clinical trial. In order of effectiveness, they are: Simulation Excel Intuition Basic math (on paper or in a simple web application; actually does not produce overage) ORDER OF EFFECTIVENESS SIMULATION EXCEL INTUITION BASIC MATH ON PAPER OR IN A SIMPLE WEB APPLICATION Although all four methods are still in use. The method employed depends on the sponsor, complexity of the trial in question and tools at hand for the person creating the supply plan. Since the middle of the last decade, simulation has emerged as the leading and most accurate method of forecasting clinical supply requirements. 11

12 Concept 2: IWR / IVRS / IRT In clinical trials, IVRS is an imperfect name for a very specific type of technology. This phone-based technology (literally: Interactive Voice Response System ) was originally built to support patient randomization over the phone, but has evolved to include management of clinical supply (from depots to sites to dispensing to subjects). Other common functions of IVRS include screening / registration (e.g. generation of the patient ID), changing a subject status (e.g. screen failed), changing a kit status (e.g. lost or damaged), break blind (e.g. subject emergency), and advanced dispensing (e.g. titration or investigator choice). All along, clinical supply algorithms have been improving and becoming more complex, with most serious IVRS now including complex predictive algorithms for just-in-time delivery of future patient dosing. In summary IVRS has evolved from a phone based randomization systems into an advanced clinical supply management dispensing system. Accordingly, the primary driver for companies to use IVRS is the significant savings in drug supply, relative to non-ivrs trials, and therefore most IVRS companies spend a good deal of effort attempting to do this well. As IVRS became more advanced, migration to the web began. Either as an option included with IVRS or as a separately built system, this was dubbed IWR for Interactive Web Response. Yet another common acronym is IRT for Interactive Response Technology or Interactive Randomization Technology. This ebook commonly uses the terms IVRS and IWR/IVRS, either of which should be taken to mean all systems that fall into that category regardless of their presentation on web, phone or both. Concept 3: Resupply Algorithms Significant supply savings is a common driver for companies that adopt IVRS for multi-center trials. The non-ivrs method of providing clinical supply to patients is to label entire courses of medication with the patient randomization number in advance and then ship all of that drug to sites in blocks (usually four-patient blocks at a minimum) in advance of seeing any patients at all. This method is incredibly wasteful compared to the IVRS paradigm of using patient interchangeable supply, which is shipped to the sites just in time for patient consumption. 12

13 The IVRS vs. non-ivrs debate being long concluded, the next challenge was for IVRS to be able to supply trials efficiently. For example, if the IVRS knows about all subjects in the world and knows what treatment groups that they are in and what their upcoming visit schedule will require for dispensing, then it should be possible to pre-position patient kits (dispensing units) for each patient at each site. Add to this a small buffer for breakage or loss; in theory it is possible to have an efficient site supply/resupply algorithm that will perform considerably better than a simple buffer or floor/ceiling algorithm. However, most trials have at least some element of unpredictability. Many are centrally or dynamically randomized, which means that the treatment required by a patient is not known until moments before it is required (at the randomization event). Others have elements of titration, variable dosing and variable visit schedules. Some IVRS address the unpredictability via predict-all-options (which is a worst case scenario) or via probability based resupply. The most advanced IVRS resupply algorithms mitigate this uncertainty by adding buffers or floor/ceiling values to the predictive calculations for known future demand. All of these methods of figuring out how much to send to sites in advance are controlled by parameters: from the very simple (buffer, floor/ceiling) to the very complex (look-ahead time windows, trigger windows, and patient counting rules). Before we move onto the next concept, keep in mind: IVRS resupply algorithms are complex and controlled by a sets of parameters IVRS resupply algorithms are poorly understood IVRS resupply algorithms is where significant supply savings is realized 13

14 Concept 4: Parameter Driven Systems A Parameter Driven system is a system that can be configured by an administrator to perform all of the required functions via user interface (UI) controls (e.g. switches, knobs, lists, and dropdown menus). By contrast, a traditional IVRS system needs to have a specification written according to the protocol design. The specification is then given to developers to program the system. The custom built system is then given to testers to validate the system. Since each clinical trial is different and because fully featured parameter driven systems are very difficult to create, most IVRS systems are traditional programmed systems. Some more modern IVRS systems rely upon validated libraries of functions and therefore are only partially programmed partially parameter driven. The next-generation IWR/IVRS are fully parameter driven for example, all aspects of the system may be configured by a nonprogrammer resource, using features of the system itself in order to complete the setup of a clinical trial. The primary limitation of parameter driven systems is that it is not practical to build parameters for every possible trial design. The parameters will satisfy 80% of most protocols. The last 20% is best addressed by extending IVRS via pluggable custom code. Concept 5: Pluggable Code Pluggable custom code is one modern solution to the limitations of parameter driven systems. Code plugins allow small pieces of programmed functionality to be inserted into a system to perform very specific functions of a study, which could not be realized via parameters. Study-specific functions, such as sending a specific shipment based on an arbitrary trigger or performing a logical operation on data which derives some other operation (such as a dosing or stratification decision), are examples of custom code plugins. In 14

15 addition to enhancing existing functions, a custom code plugin can also add novel functions like trial adaptation or cold chain temp tales. The important point with plugins is that the amount of specification, programming and testing is significantly minimized because they are small isolated bits of code. 15