Adopting Site Quality Management to Optimize Risk-Based Monitoring

Transcription

1 WHITE PAPER Adopting Site Quality Management to Optimize Risk-Based Monitoring In today s pressure-packed environment, the quest for improved data quality at a lower cost is of paramount importance to all clinical organizations. A risk-based model for monitoring has become a growing and crucial component in this search. This white paper will break down the issues related to the implementation and tracking of a risk-based monitoring program.

2 Introduction As it is now well documented, clinical R&D organizations have been actively embarking on a paradigm shift in their approach to ensuring quality in the conduct of clinical trials. In particular, a more targeted, risk-based approach to site monitoring and data cleaning is being actively pursued by organizations both large and small. The primary goals of this trial conduct and corresponding data, and the maintenance or even reduction of clinical trial timelines. following factors: Lack of full electronic data capture (EDC) adoption, which severely hampers an organization s ability to proactively assess quality across all sites in a clinical trial. Concern over the potential of increased regulatory scrutiny, especially in the absence of clear guidance or endorsement for risk-based approaches from the FDA or EMA. Potential for actual decreased trial and data quality, due to less intensive on-site monitoring during trial conduct. on risk-based quality management in clinical trials. These documents not only provide the industry guidance on how to demonstrates this: FDA encourages greater reliance on centralized monitoring practices than has been the case historically, with correspondingly less emphasis on on-site monitoring. The following will help you understand the challenges faced when implementing risk-based monitoring and how developing a site quality management program especially one leveraging real-time site quality analytics can simplify the adoption of this strategy. 1 AUG-2011 FDA Draft Guidance: Guidance for Industry: Oversight of Clinical Investigations A Risk-Based Approach to Monitoring Medidata Solutions Worldwide 2

3 3 WHITE PAPER: ADOPTING SITE QUALITY MANAGEMENT TO OPTIMIZE RISK-BASED MONITORING The Role of Data Quality and Metrics in the Adoption of Risk-Based Monitoring one question in this regard: to what extent is patient electronic case report form (ecrf) data actually corrected during trials, as a result of all of the intense scrutiny traditionally applied to that data by study teams in the form of 100 percent this, Medidata computed the following two measures using the Medidata Insights metrics warehouse, comprising over 2,500 studies from 65 contributing sponsor organizations: Total Data Correction Rate: The total percentage of ecrf data found to have one or more updates ( corrections ) following initial submission of the ecrf form. Post-Capture Data Correction Rate: The total percentage of ecrf data found to have one or more updates following The surprisingly low industry median for these two measures is illustrated in Figure 1. The total data correction rate comes in at just 4.3 percent, while the post-capture data correction rate is only 2.7 percent. Putting this in context, reporting, analysis and submission before any site monitor, data manager or other sponsor representative has had a chance to scrutinize the data! FIGURE 1: ecrf DATA CORRECTION RATES 5.0% 4.32% 4.0% 3.0% 2.68% 2.0% 1.0% 0.0% Total Post-Capture Source: Medidata Insights metrics warehouse It is relevant to note that these metrics currently account for any updates to data values in the EDC audit trail, which includes natural patient event-based updates, such as adverse event (AE) resolution/outcome information that do not analysis of AE and concomitant medication forms on various studies reveals a very high correction rate on these percent overall.

4 clinical trial costs and site monitoring in particular is one of the two largest cost drivers at 30 percent or more. If nothing approach to ensuring clinical data quality. FIGURE 2: CLINICAL TRIAL COST BREAKDOWN* IVRS and Drug Distribution Project and Clinical Leadership Data Management and Statistics Other Site Payments Monitoring *Estimated for a large, global clinical trial Source: Medidata internal analysis The Impact of Regulatory Guidance on Site Management site monitoring and centralized monitoring. On-site monitoring is the current de-facto practice, so the introduction of paradigm. But what exactly is centralized monitoring or more to the point what should it be? The FDA draft guidance outlines the primary goals of centralized monitoring that should guide a solution. Among them are the following: Target on-site monitoring by identifying higher risk clinical sites (e.g., sites with data anomalies or a higher frequency of errors, protocol violations or dropouts relative to other sites) Augment on-site monitoring by performing monitoring activities that can only be accomplished using centralized processes (e.g., statistical analyses to identify data trends not easily detected by on-site monitoring) Conduct aggregate statistical analyses of study data to identify sites that are outliers relative to others and to evaluate individual subject data for plausibility and completeness Conduct analyses of site characteristics, performance metrics (e.g., high screen failure rates, high frequency of eligibility violations, and delays in reporting data), and clinical data to identify trial sites with characteristics correlated with poor performance or noncompliance 2 Ibid Medidata Solutions Worldwide 4

5 5 WHITE PAPER: ADOPTING SITE QUALITY MANAGEMENT TO OPTIMIZE RISK-BASED MONITORING and other electronic systems. Together, these can help sponsors identify sites with emerging quality-related risks, relative sites with emerging risks as proactively as possible to remediate any issues, thereby ensuring optimal quality across the trial. Any implementation of centralized monitoring should therefore include an effective signal detection system that emphasizes earliest possible detection of emerging quality-related risks at one or more trial sites. share similar key risk categories including the following: Patient safety monitoring and reporting, for example AEs and serious AEs (SAEs), Patient data quality, and Protocol compliance, including eligibility criteria, and visit and dosing schedules. Key Metrics to Consider for Adopting Centralized Monitoring As such, the industry can and should identify a relatively small set of measures that serve as effective surrogates for the quality of trial conduct by sites in each of these categories. As an example, one might consider the following measures related to the quality of patient ecrf data capture at each site, all of which should be readily computable using available EDC-based information: Data correction rate (percent of data corrected after initial entry) Rate of ecrf queries from clinical data management Rate of auto queries (i.e., queries resulting from programmed data checks within the EDC system) Cycle time from patient visit to entry of ecrf data by the site Cycle time for sites to respond to queries from the sponsor team All of the above measures relate to processes that may have a real impact on the quality of ecrf data. One approach might be to compute all of these measures in parallel to help detect sites with emerging issues. This may have some degree of success; however, it will more likely lead to a proliferation of signal-detection noise that will actually detract same site process and then choose the one or two that enable the most proactive and effective assessment of site quality around that process. With respect to data quality, for example, it is fair to say that ecrf query rates (e.g., auto queries, data management queries, site monitor queries, etc.) and data correction rates should all be good indicators of the level of quality with which sites are capturing patient data into the sponsor s EDC system. Insights metrics warehouse, which reveals a very strong correlation between each of these query rates and data issues. However, one of them auto query rate is clearly the most effective when viewed from the perspective of early signal detection. This is because, unlike manually generated queries and subsequent data corrections, auto queries are generated and observable immediately upon entry of the ecrf data at the site. Therefore, both the numerator and denominator comprising an auto query rate are always current, whereas the numerator in the other rates always lags behind the denominator by weeks or months (and yes, even years of delay in query generation have been observed!).

6 effective among the obvious choices. Note that the Visit-to-eCRF Entry cycle time metric while it does overlap with auto query rate as a surrogate for data quality also measures an additional aspect of site behavior that is worth observing. In particular, lack of timely attention to capture of patient data in the sponsor s EDC system and slowness in responding to sponsor queries may portend broader issues with site s attentiveness to and engagement with the trial overall. Additionally, various sponsor reviews of incoming EDC data begin losing their value the longer the delay in getting access to that data. So these cycle times provide an important complement and addition to the auto query rate. Understanding a Typical Site Quality Management Scenario It is indeed important to assess all potential site quality measures in the context of not only how effectively, but how proactively, they are able to support signal detection. Getting to a relatively small, focused set of standard quality measures is critical to a successful centralized monitoring implementation. From a tool perspective, organizations and study teams should be looking for a centralized dashboard in which these quality measures are pre-computed across all sites and that provides for each site a measure-by-measure and overall assessment of risk-level with respect to quality. risk levels requiring follow-up. Figure 3 depicts what the tool might display for a given site and measure. In the conceptual example, the AE rate for the study overall is 8.7 AEs per subject-year, and for site 105 it is 5.6, which puts that site at an elevated (yellow) risk level for potentially under-reporting AEs. FIGURE 3: EXAMPLE SITE AND STUDY AE RATE SCORECARD Site 105 AE Rate Study AE Rate Note that this would not mean conclusively that the site is under-reporting AEs only that it is at an elevated risk of such. The study team would be responsible for deciding what, if anything, is the appropriate course of action to address this observation. It could be as simple as having the site monitor take some extra time at the next site visit to scrutinize the site s AE reporting process and perhaps to review AE reporting requirements with the site staff. It could also include increasing the amount of SDV targeted for patients enrolled at that site particularly for the AE forms. Or it could be a combination of the above. Either way, with a site quality management program, sponsors could better anticipate such risks and identify the root causes more quickly. Implementing a Site Quality Management Program Using a standard set of measures, this type of centralized dashboard could be deployed very quickly for each new study a critical requirement given the importance of early issue detection during trial conduct. Organizations should also plan to employ a dashboard tool that is turnkey: not only easy to deploy for each study but also with site quality metrics automatically assessed and presented in a near real-time fashion. Such a centralized monitoring tool is not only ideal but quite feasible, and will help your organization revolutionize the way total quality is managed in clinical research. based monitoring by leveraging the site quality management concepts in this paper, please contact your Medidata representative. Medidata Solutions Worldwide 6

7 About Medidata Solutions Worldwide Medidata Solutions is a leading global provider of cloud-based clinical development solutions that enhance the efficiency of customers clinical trials. Medidata s advanced platform lower the total cost of clinical development by optimizing clinical trials from concept to conclusion: from study and protocol design, trial planning and budgeting, site negotiation, clinical portal, trial management, randomization and trial supply management, clinical data capture and management, safety events capture, medical coding to business analytics. Our customers include biopharmaceutical, medical device and diagnostic companies, academic and government institutions, CROs and other research organizations, encompassing 20 of the top 25 global pharmaceutical companies as well as research organizations of all sizes. mdsol.com info@mdsol.com Optimizing Clinical Trials: Concept to Conclusion Trademarks are the property of their respective owners. Copyright Medidata Solutions, Inc /12 BUSINESS ANALYTICS STUDY & PROTOCOL DESIGN TRIAL MANAGEMENT, PLANNING & BUDGETING SITE NEGOTIATION RANDOMIZATION & TRIAL SUPPLY MANAGEMENT EDC/CDM MONITORING AE/ SAE CAPTURE CODING CLINICAL PORTAL