For all the talk about how risk-based

Transcription

1 October 2017 A CenterWatch Feature Article Reprint Volume 24, Issue 10 Tracking adoption of risk assessment and RBM Growing utilization but wide variation in approaches and impact Tracking the adoption of RBM Percentage of companies reporting use of risk-based monitoring approaches 52% 75% By Karyn Korieth 27% For all the talk about how risk-based monitoring (RBM) can improve efficiencies in clinical trial processes, the actual level of execution varies widely among sponsor companies, the industry lacks a uniform approach to the methodology and some organizations have not yet pursued alternatives to traditional monitoring strategies. Most sponsor companies have indicated they intend to move away from the long-established monitoring approach that relies on frequent on-site visits and 100% source data verification (SDV) to ensure patient safety and data quality. Yet in practice, the challenges of implementing a targeted or risk-based monitoring strategy have proven to be harder than anticipated. A new Metrics Champion Consortium (MCC) survey reported a 37% increase in use of RBM supported by centralized data analytics across programs or on a pilot basis from 2013 to Yet only half of respondents used RBM in pivotal trials and a majority (62%) continue to use traditional 100% SDV. RBM programs at many sponsor companies have not advanced beyond the pilot phase or have been introduced across the organization without standardized processes or established success measures. In a 2017 survey of more than 300 clinical research leaders about risk, the Avoca Group found that about a third of sponsor respondents acknowledged a lack of good understanding about best practices for risk-based approaches. Moving forward, however, more studies are expected to use RBM methodologies as companies are required to incorporate the revised International Conference on Harmonization (ICH) Good Clinical Practice (GCP) E6 R2 guidelines into their processes and practices. The new guidelines, which were finalized in November 2016, expect companies to implement a riskbased approach to quality management and clinical trial execution that includes monitoring practices. The regulatory document also supports the use of RBM and centralized monitoring processes. To show compliance with the revised guidance, companies will be asked to document and defend the rationale for the chosen monitoring strategy. Kristin Mauri, global head of Risk-Based Monitoring at Bioclinica, said many companies have struggled, however, to understand the new regulations or have failed to acknowledge that current practices don t fully comply with ICH E6 R2. Due to the Source: Metrics Champion Consortium 2016 extent and significance of the changes required, companies that have not begun to assess the impact of the revised guidelines on their organizations are at risk of falling behind. Mauri believes it may take a large company having a widely publicized inspection failure to drive more rapid adoption of risk-based approaches that are compliant with the new regulations. Many of us in the industry expected a much faster adoption curve than we have seen, said Mauri. Many companies are now just starting to do an impact analysis on the revised guidelines and are looking to change their approach. It s happening, but it s slow. I expect that as we move into 2018, the number of RBM studies will increase significantly. Adoption rates slowly rising The Clinical Trials Transformation Initiative (CTTI), a public-private partnership formed in 2009 by the FDA and Duke University to drive adoption of quality by design (QbD) principles in clinical trials, made promoting RBM as part of a broader overall quality framework a key initiative and helped increase industry awareness around the approach. CTTI raised con- The CenterWatch Monthly (ISSN ). Volume 24, Issue CenterWatch centerwatch.com

2 cerns, which have been supported by a growing body of retrospective data analyses and data findings, that 100% SDV has a negligible impact on data quality and diverts attention and resources from more critical clinical trial activities. Published studies have shown that current SDV monitoring practices cost companies an average of one-third of the entire phase III pivotal trial budget. While a few forward-thinking sponsor companies and CROs have experimented with alternatives to traditional on-site monitoring and SDV approaches for several years, primarily as a way to reduce monitoring costs, adoption rates began to rise after 2013 when the FDA and EMA each released regulatory documents that endorsed RBM as an important part of an effective quality risk-management system for clinical trials. As an indication of the impact of this guidance, MCC, which has tracked organizational approaches to RBM since 2013 and established important benchmarks for the industry, found the top reasons organizations implement RBM programs has shifted from reducing monitoring costs (78%) in 2013 to improving quality oversight (83%) and data quality (85%) in Overall, in its 2015 update of risk-based management practices, MCC found the proportion of organizations that have either used or piloted RBM with central analytic monitoring increased from 52% to 71% between 2013 and In a 2016 CenterWatch survey of investigative sites, conducted in collaboration with the Association of Clinical Research Professionals (ACRP), respondents reported that the number of studies requiring RBM nearly doubled from 2014 to 2015, from four to seven, and more than half expect the number of RBM studies at their sites will continue to rise. CROs also report an increase in the number of clients accepting RBM in proposals and asking about RBM strategies and capabilities when they initially engage CRO partners. Top electronic data sources reviewed during centralized and remote monitoring review 82% EDC 76% CTMS 59% 59% 56% Central lab data Interactive response technologies etmf 47% Central imaging Source: Metrics Champion Consortium, 2016 This increase is not exponential but rather a slight, steady up trend in acceptance of RBM, said Nicole Stansbury, executive director, Adaptive and Intelligent Monitoring at PPD. This signals that RBM is no longer just for the early adopters, but for those who have been more skeptical as well. We are at the tail end of the early adopter phase. The actual pace of adoption has been difficult to assess, however, since each company has a different definition of what RBM means. Some companies have aggressively pursued innovative RBM approaches that are part of overall quality risk-management programs while others are more cautiously experimenting with components of the model, such as looking at alternative monitoring strategies or how central data could be used in different ways on a pilot basis. Xavier Flinois, president of Parexel Informatics, said that most models and approaches begin with the identification of risk and data criticality, but there are vast differences in the percent of critical data identified, the amount of SDV and even the amount of source data review (SDR). He said there are also differences in the types of statistical tests used to detect variance in data and the methods by which centralized monitoring is conducted. Most sponsors and CROs have piloted RBM; however, the extent varies greatly across the industry. Even within companies that promote RBM, different levels of execution exist and there is an acceptance that the approach must be applied thoughtfully given the particular population size, critical data, site selection and other risks that vary widely in individual studies, said Flinois. In general, there is no-one-size-fits-all approach advocated for RBM but rather a balance of traditional, adaptive and centralized monitoring techniques based on each study s individual need. Most advanced RBM models Members of TransCelerate Biopharma s consortium, which include some of industry s largest R&D research organizations, appear to be the furthest along in implementing disciplined, standardized RBM programs across their companies. TransCelerate established a model framework for RBM in 2013 that shifts dependence away from on-site monitors and instead emphasizes centralized or off-site activities that focus on critical processes and data that have the most potential to impact human subject protection and the reliability of trial results. The model incorporates QbD principles and employs risk assessment tools and processes to drive monitoring strategies. The consortium developed a library of more than 140 risk indicators that organizations could use in central or off-site monitoring models to detect poten- centerwatch.com 2

3 tial issues for further investigation or mitigation; MCC has also developed a more consolidated list of key risk indicators. TransCelerate companies are in various stages of implementing methodologies outlined in its RBM approach and members have participated in pilot programs aimed at testing the efficacy of certain elements of RBM. Boehringer Ingelheim, which helped lead TransCelerate s RBM initiative, adopted a risk-based approach to clinical trial monitoring, which included reduced SDV and centralized monitoring. It has become part of a broader risk-based quality management program, implemented at the company last year. With the approach, risk assessment tools are used early in the clinical trial design and protocol development phase, which allows teams to incorporate QbD into the final protocol, and risk assessment processes are repeated at various points throughout the program. Cross-functional teams, which include medical staff, data managers, statisticians, operations personnel and others involved in the conduct of the clinical trial, are included in risk assessment and quality reviews. Risk levels for trials are based on the program, trial, investigational product and sites. Monitoring plans are then developed once the risks and thresholds for action are identified and can later be adjusted as needed. Throughout the conduct of the trial, central monitors analyze technical data in near real-time from various electronic sources, such as EDC or IVRS systems, to uncover outliers and trends. Central monitors also evaluate and flag other potential risk factors that can t be easily seen in the data, such as protocol amendments or the addition of new countries to a trial design. Importantly, the program recently implemented risk managers to support teams throughout the conduct of a clinical trial with risk management and mitigation activities. Adoption of RBM by geographic region Percent of companies report using risk-based site monitoring approaches 37% 56% Western Europe 32% 38% 54% Eastern Europe Our program started with the understanding that monitoring is one part of holistic quality management. We also wanted to find, based on data and the analysis of the data, predictors and indicators that would help us proactively manage quality in our trials, said Lilija Kircheis, M.D., Ph.D., head of Global Clinical Operations at Boehringer Ingelheim. It was clear when we started our pilot trials in 2015 that with our risk-based monitoring or riskbased quality approach, the goal was not to reduce cost. Our goal was and remains to reduce and manage the risks. Eli Lilly had made various attempts at RBM over the years before implementing its current approach in 2013 based on TransCelerate s RBM methodology, which Lilly had a key role in developing. Jacqueline Gough, advisor, Clinical Risk Management at Lilly, said the new methodology differs from previous industry attempts at RBM because it s grounded in methodological risk-assessment that drives activities downstream. The program uses TransCelerate s Risk Assessment and Categorization Tool (RACT) and its accompanying processes, which were built on approved FDA methodologies, to understand how teams can have conversations about risk and demonstrate that they not only understand the risks but are addressing them appropriately. Lilly also uses central monitoring as one component of its approach, which can lead to triggered monitoring activities based on emerging risk or issues identified at an individual site. The program has been used in 74% North America Latin America % 32% 27% Asia 41% Source: Metrics Champion Consortium more than 100 trials to date and will include almost all of Lilly s late-stage portfolio trials moving forward. The reason to implement RBM is that it is a better way to run a trial. It allows us to focus more specifically on the data, processes that are most impactful to patient safety, data integrity and to address the greatest risks, said Gough. While the scope of early RBM models often focused on reducing SDV, MCC survey data suggests that data analytics being used to support RBM and RBM programs have grown in sophistication. In addition, the majority of organizations (69%) conduct risk assessments prior to study conduct, although fewer are repeating assessments during study conduct. Challenges to implementation Many challenges have slowed the implementation of RBM and centralized monitoring processes across the industry. Shifting to an RBM strategy requires significant change management, since the model touches many different departments and processes within a company, and requires a change in both practice and mindset. Operational staff need to think about risk prevention instead of addressing current problems, for example, and monitors must be retrained to interpret data and resolve problems rather than verify data in case report forms. Many companies, uncomfortable with moving away from 100% SDV processes, centerwatch.com 3

4 are waiting for evidence new drug applications that used RBM approaches will pass regulatory scrutiny. In addition, the industry has yet to determine which metrics should be used to measure the success of RBM approaches. Some companies that have been working on their RBM initiatives for a few years have collected evidence to validate their model, from both a quality and cost reduction standpoint, and TransCelerate has conducted surveys on core quantitative performance metrics among its members that show generally positive results. Company-specific metrics, however, have not been widely published or shared in the industry. Technology challenges have also affected the adoption of risk management and RBM strategies. Comprehensive, integrated data from multiple sources is needed to facilitate RBM. Yet systems are not typically integrated and most clinical trial technologies, such as EDC, lack the capabilities needed to tag and run analytics around critical data. Early RBM adopters usually built data aggregation and analytics in-house to support risk-based processes and ensure timely access to integrated data. While technology companies have begun to develop capabilities to integrate data from different sources for RBM and apply sophisticated algorithms to detect risk, the technology solutions tend to lag behind company needs and are expensive to implement. The 2017 Avoca Group survey found that only 34% of sponsors were satisfied with the level of innovation and technology in the areas of RBM and riskbased quality management. Technology providers in RBM are developing parts and pieces, said PPD s Stansbury. Not all of the systems connect, even within a single technology provider, and user interfaces may not align with CRO roles and process, reducing efficiencies and delivery according to the CRO strategy and process. In addition, there are costs to obtain multiple separate pieces Expectations on the number of studies involving risk-based monitoring Compared to 2015, do you expect the number of studies that will require RBM at your investigative site 2016 to... 41% Remain the same 1% Decrease 58% Increase Among those expecting an increase in the number of studies requiring RBM from 2015 to 2016, an average increase of 33% in the number of studies requiring RBM is expected. Source: CenterWatch ACRP survey 2016; n=200, excludes I don t know and link components across different technology vendors and CRO systems. Sponsors and CROs also need to develop RBM and centralized monitoring processes that don t transfer risk and administrative burden to investigative sites. During the transition to remote monitoring, some organizations lacked access to needed information and asked sites to scan or fax documents to off-site locations. As these models evolve, an effective program shouldn t add burden to investigative sites. But sites most likely will need to interact with both a central monitor and a clinical research associate (CRA), which some investigators worry could result in duplication of effort, and sponsors and CROs need to consider how the role of CRAs and their relationships with sites will change. Ideally, the RBM model should not require any additional work at the site level, said Parexel s Flinois. However, we know from site and investigator feedback that many have experienced a decrease in support and guidance from CRAs in trials where RBM was employed. Early RBM studies decreased SDV and extended visit schedules without making compensatory adjustments to ensure that new practices were completed with training and support provided by the CRA in a cohesive environment regardless of whether interactions occur on-site or off. In next generation RBM studies, sites should be given this type of support and receive feedback from the CRO or sponsor project team about possible emerging risks as well as potential solutions based on signals surfaced through RBM technological solutions. New approaches required Companies will need to address challenges and implement a system to manage quality throughout the lifecycle of a trial using risk-based approaches to comply with the revised ICH E6 R2 guidelines. Importantly, the new guideline recognizes that not all risks are of equal importance from a compliance and quality perspective and that companies can increase the overall quality of their data and provide better oversight of patient safety by defining and focusing attention on critical processes and data. The new guidelines consolidate the thinking that we have seen from multiple regulatory agencies around implementation of quality management processes and applying risk-based approaches to trial execution, said Lilly s Gough. What this guideline clarifies is the need for processes to set clearly defined, data-based, quantitative thresholds to the identified risks and also the continuous learning cycles that will inform how those thresholds are set centerwatch.com 4

5 and updated throughout the execution of a clinical trial. Many industry leaders have stated that the ICH GCP guidelines were revised just as advances in technology have begun to reach a point where they could support the processes and data needed to support a more wide-spread adoption of risk-based approaches to quality management and monitoring. We have two things converging, said Linda Sullivan, co-founder and president of MCC. The ability to bring data together from different systems and report on it will open the door to gaining the insights needed to do RBM and remote monitoring. Using central analytics will be powerful once you finally connect those pieces. We are in a transition period now. The ability exists. The challenging part is that it s a combination of people, technology, culture and resistance to change, which is why it s not a simple thing. Looking ahead While RBM models have advanced at a few large companies and surveys show that overall adoption rates have risen, even with the release of new ICH E6 R2 guidance, it will be several more years until RBM becomes a standard part of operations within sponsor organizations. Changing the approach and the methodology will ultimately lead to more effective patient safety oversight and higher data quality when we are not focusing on the minutia and are really focusing on the areas that have the largest impact, said Bioclinica s Mauri. Yet it s an industry that is not progressive in its mindset. It is not going to change easily. Moving people away from the way they have always done things, whether it s effective or not, is extremely difficult. Karyn Korieth has been covering the clinical trials industry for CenterWatch since Her 30-year journalism career includes work in local news, the healthcare industry and national magazines. Karyn holds a Master of Science degree from the Columbia University Graduate School of Journalism. karyn.korieth@centerwatch.com. centerwatch.com 5