Risk-Based Monitoring Update Volume V

Transcription

1 18Jan2016 Introduction TransCelerate s Risk Based Monitoring Initiative in the 2nd half of 2015 continued to be focused on deliverables that facilitate implementation being mindful of both qualitative and quantitative feedback to better understand how well RBM is working. The purpose of RBM Update Volume V is to share information on the 8 quantitative metrics as well as highlight some of the key points from the recently completed work from the team. Quantitative Performance Measures As published in RBM Update Volume IV, TransCelerate utilizes a core group of 8 metrics focused on quality, efficiency, and cycle time. Quarterly, a blinded third party has collected metrics from Member Companies where RBM has been implemented. No differentiation or consideration has been made as to the extent to which a company may have implemented the TransCelerate methodology. The intent of the quantitative metrics is to determine: the impact and effectiveness of the proposed RBM methodology on managing quality and risks associated with the conduct of clinical trials if the RBM methodology works from the standpoint of operational impact on an organization, clinical sites and investigators As of September 30th, 7 TransCelerate member companies voluntarily reported metrics on 59 RBM Trials to a blinded third party. Not all of the member companies have provided metrics but through survey it is known that presently, at least ten companies have implemented RBM methods in at least 125 trials in order to evaluate the methodology and that all eighteen member companies who participate in the RBM team have stated that they either implemented or intend on implementing RBM. Each of the eight metrics are described below. When reading the information below, please note that a quarter refers to the length of time a trial has been operating as opposed to a calendar based quarter. This allows measures to be comparative for a given time period e.g. enrollment within the same cohort of Trials. 1

2 1. Average number of major / critical audit findings per audited site (Quality) Results have been positive. Reducing the average number of major findings related to quality assurance activities is the expected outcome as there is improved focus on what matters. For the purposes of these early trials, audit practices have been reported to be largely unchanged and audit findings have not been restricted to monitoring findings. Details can be seen in Figure 1. Figure 1: Illustrating the results of audits conducted within RBM Trials within a specific quarter 1 One or more audits may have been assessed by a company within the reported RBM trial for any given quarter 2. Percentage per site of unreported, confirmed SAEs compared to total SAEs (Quality) Results have been positive. Metric rationale is based on the enhanced focus on safety event reporting. The expected outcome is that application of RBM will result in a decrease in unreported, confirmed SAEs. Details can be seen in Figure 2. 2

3 Figure 2: Illustrating the results of unreported, confirmed SAEs as compared to total SAEs 3. Significant Protocol Deviations rate per treated subject - Total number of deviations / total number of subjects for the protocol (Quality) Results have been positive. Overall the expected outcome is that Protocol Deviations will decrease. What is typically observed is an initial rise. Further assessment has led to the conclusion that the initial increase is in part due to better detection and better reporting possibly due to a shift in SDR which is more focused on process. Details can be seen in Figure 3. 3

4 Figure 3: Illustrating the results of Significant Protocol Deviations rate per treated subject 4. Median number of days from patient visit to ecrf data entry (Cycle Time) Results are positive. The expected outcome is either no change or reduction in the time it takes for sites to enter information. The timeliness of data entry is a key enabler of central monitoring activities. Guidance for companies has been to measure this metric using first data entered or the majority of data entered. This metric is also tracked as a risk indicator through more regular central review and as such we would expect that over time this metric would track positively. Details can be seen in Figure 4. 4

5 Figure 4: Illustrating the results of the days from patient visit to ecrf data entry 5. Median number of days from query open to close (Cycle Time) Results have been positive. This metric is a surrogate for unintended consequences of reduced site visits. The expectation for this measure is the result is not changed against baseline. Recommendation has been to exclude auto-generated queries and to use the response by the site as the close date. Details can be seen in Figure 5. Figure 5: Illustrating the results of Query Open to Close 5

6 6. Median number of days from significant / major issue open to close (Cycle Time) Results have been positive. This metrics is a surrogate for earlier identification and resolution of issues. The expectation is that the time it takes to resolve significant issues will decrease. Reportedly this metric has been difficult to measure. What is considered significant is left to the company to define. Details can be seen in Figure 6. Figure 6 Illustration of the results of days required to close significant / major issues *results not reported for 4 trials 7. Average Monitoring (all types) cost per site (Efficiency) Results tend to show that early in the life of a trial, monitoring costs may be best expected to be neutral and the shift to better financial outcomes tends to start becoming apparent between 6 and 9 months. These measures are taken on a per trial basis and may not reflect what a sponsor may see when implementing across their portfolio. Broader monitoring types and how they are implemented may affect cost within new structure. This metric includes pass through and labor costs. Details can be seen in Figure 7. 6

7 Figure 7 Illustration of the results of Average Monitoring Cost 8. Average interval between on-site monitoring visits per site (Efficiency) Results have been mixed. Indicator of whether more work is being done remotely. It is expected the trend will be that the interval between on-site visits will increase. For any particular site RBM may increase or decrease the number of visits to a site. Details can be seen in Figure 8. Figure 8 Illustration of the results of average interval between on-site monitoring visits. 7

8 RBM Publications and New Posted Materials In the 4th quarter of 2015, the TransCelerate RBM team has published two manuscripts in DIA s Therapeutic Innovation and Regulatory Science (TIRS) journal (available in January paper version) and submitted a third manuscript to DIA (publication pending). The team has also made available on the TransCelerate website a risk indicator library as well as a 2nd paper supporting technology needs for RBM. Below is a description of what is available now (all are available at the TransCelerate web site, under the RBM Initiative Assets). Defining a Central Monitoring Capability: Sharing the Experience of TransCelerate BioPharma s Approach, Part 2 This article is a follow-up to the paper that was published in September of Member companies that have implemented Central Monitoring have done so differently and this article expands on what has been learned through those various applications. In addition, the work on this paper also involved the development of a risk indicator library and corresponding guidance on the use of those tools. Risk Indicator Library Member companies provided a list of risk indicators (RIs) that they were using through a blinded third-party survey, the results of which were then compiled into the library that is available on the TransCelerate RBM Assets web page. The team does intend on reviewing this library in the 2nd half of 2016 in order to provide a more experienced picture of the relative value of various RIs. It is already anticipated that viewing various indicators in different combination may be of more predictive value. Detecting Data Quality Issues in Clinical Trials: Current Practices and Recommendations This article is one of potentially a series of articles on data quality. The focus for this article was assessing the current landscape for the detection of data quality issues, irrespective of origin or motive. The methods involved both a survey of member companies as well as literature search across industries from 1985 to Although a multifaceted approach using a combination of site monitoring, central monitoring and statistical monitoring is recommended, more work is needed to better evaluate the best application of statistical monitoring. The early work on this paper led to the second paper which was submitted in December and involved creating, embedding fraudulent data, and statistical analyses of datasets that would mimic small and large Trials at different time points of data collection (publication pending later in 2016). 8

9 Technology Considerations Part 2 This paper presents additional functionality details for an integrated RBM technology solution with more details than what was provided in the initial paper, Technology Considerations to Enable the Risk-Based Monitoring Methodology, published in TIRS in September The intended audience for this paper are those sponsor representatives that may need to select and implement a system to support RBM, and technology vendors as they develop solutions that support risk based approaches to monitoring. There is no one size fits all approach and readers of this paper should keep in mind that there may be many different innovative technical solutions that would still meet the needs of RBM. 9