RISK-BASED MONITORING CURRENT PERSPECTIVES ON YOUR PEER-R EVIEWED GUIDE TO GLOBAL CLINICAL T RIAL S M A NAGEMENT. appliedclinicaltrialsonline.

Transcription

1 E-BOOK YOUR PEER-R EVIEWED GUIDE TO GLOBAL CLINICAL T RIAL S M A NAGEMENT appliedclinicaltrialsonline.com CURRENT PERSPECTIVES ON RISK-BASED MONITORING CO-SPONSORED WITH:

2 Introduction The development of this ebook came on the heels of Applied Clinical Trials last successful ebook, titled Risk-Based Monitoring in Clinical Trials. This ebook, Current Perspectives on Risk-Based Monitoring, was developed in conjunction with SAS, and is based largely on two projects: a custom research survey conducted by a third-party research firm using an in-depth phone interview of 20 executives, culled from the Applied Clinical Trials audience database, regarding risk-based monitoring usage and perceptions. The second project was a December 2013 webcast titled Myths, Models and Momentum: Risk-Bssed Monitoring, which featured three presentations from industry experts. Andrew Schafer, President, Industry Standard Research, presented results on its survey and covered the high-level issues regarding risk-based monitoring. Mark Hronec, Director at PwC, presented on risk-based monitoring from a CRO and sponsor perspective, delving more into the models. Laurie Rose, Principal Industry Consultant, Health & Life Sciences at SAS, then discussed using more integrated technologies that improve data quality with less human intervention to gather and report the data information needed for a risk-based monitoring program. Schafer also wrote a peer-reviewed article earlier this year that went into his research findings more in-depth, which touched on the concerns and challenges executives have with risk-based monitoring. But in the end, he noted in the article, Survey results reinforced that traditional clinical monitoring will not go away. It can t. Its scope is a critical component to help ensure squeaky clean data and utmost patient safety. Yet, according to study results, if sponsors and service providers are to relieve stubborn industry-wide financial pressures and meet growing industry demands, clinical monitoring must begin to morph into more of a hybrid model at least in the later phases of clinical trials. As one respondent aptly put it, There is a large area of opportunity to improve how clinical trials are conducted. Hopefully the combination of new technologies and alternative models can help to improve monitoring and data quality. And this ebook will address those concerns, how companies go about applying scores and rating signals in clinical trials, and how technology is being used now, and how it is envisioned to be used more effectively in the future. TABLE OF CONTENTS 3 Why Risk-Based Monitoring? Costs, patient quality, data quality factor into monitoring decisions. E-BOOK E-BO YOUR PEER-R EV IEW ED GUIDE TO GLOBA L CLINICA L T R I A L S M A NAGEMENT appliedclinicaltrialsonline.com 6 Risks, Signals, and Scoring What determines risk and how are risk levels determined? 8 Technology Use to Achieve Risk-Based Monitoring Where is the technology for RBM exactly and what should organizations be looking for? COVER DESIGN: DAN WARD CURRENT PERSPECTIVES ON RISK-BASED MONITORING CO-SPONSORED WITH: O U R M I S S I O N Applied Clinical Trials is the authoritative, peer-reviewed resource and thought leader for the global community that designs, initiates, manages, conducts, and monitors clinical trials. Industry professionals learn effective and efficient solutions to strategic and tactical challenges within the tightly regulated, highly competitive pharma ceutical environment. 2 APPLIED CLINICAL TRIALS appliedclinicaltrialsonline.com

3 Why Risk-Based Monitoring? Costs, patient safety, and data quality factor into monitoring decisions. Biopharmaceutical firms and CROs are feeling the pressure to conduct what has been touted as more cost effective, risk-based monitoring of clinical trials. But there are several roadblocks to using this technique: Complexity of managing international sites. Monitoring multiple international sites is complicated. According to the NIH, 261,000 clinical studies were registered in August 2013; 44% were non-us only. Over the average 7-12 year period of a clinical trial, that means millions spent on travel and site visits using traditional monitoring practices. Resistance to business process change. Regulatory authorities are pushing risk-based monitoring, but companies are resisting the necessary business process changes to make that happen. Global regulatory authorities are suggesting the move from 100% source data verification (SDV) to a risk-based approach to monitoring, but there is not enough evidence of success with this new method to drive business process change. Variability according to clinical trial type. Monitoring trials varies according to therapeutic area, site selection, and trial stage. The bigger and more complex the trial, the more likely the failure rate. Whether these roadblocks are myths or fact, they remain perceptions uncovered in a group of in-depth telephone interviews of 20 executives representing both CROs and sponsors, conducted by SAS with Applied Clinical Trials, in October Characteristics of Companies On-Site Monitoring Trials Larger companies with larger budgets Larger companies with rigid SOPs Early phase trials Small trials Limited sites Therapeutic area/high level of expertise Trial plan established with CRO Source: SAS, Applied Clinical Trials Risk-Based Monitoring, October 2013 There are three central objectives in clinical trials that pertain to risk-based monitoring: ensuring patient safety throughout the trial, obtaining the highest quality data, and from a cost perspective, conducting the trial at the lowest cost without compromising patient safety and data quality. But what is risk-based monitoring? In Figure 2, which Mark Hronec, Director of PriceWaterhouseCoopers, shared during a December 2013 webinar sponsored by SAS and Applied Clinical Trials titled Myths, Models and Momentum: Risk- Based Monitoring, the four types of risk-based monitoring strategies are described. The first is around reduced monitoring, where organizations conduct less than 100% source document verification (SDV). Targeted monitoring is based on particular triggers that could happen during the study that would trigger a site visit. Centralized monitoring is using algorithms or predictive ana- appliedclinicaltrialsonline.com APPLIED CLINICAL TRIALS 3

4 lytics to determine the level and type of monitoring that occurs in a trial. And the final piece is remote monitoring, where monitoring is achieved either off site and/or potentially in a different country. Taken together and implemented holistically on a study or in a company, the determination of which sites would require an increase in monitoring resources or potentially decreases is based on the risk-based approach. By focusing resources to the sites in need, the risk-based approach then is increasing the patient safety and data quality coming from those sites, rather than having those resources focused on sites that do not necessarily need the extra attention, explained Hronec. Laurie Rose, Principal Industry Consultant Health & Life Sciences at SAS, agreed during the webinar, and stated that another misconception is that RBM means moving completely away from onsite monitoring. But in reality, at least for the near future, it s more a matter of optimizing the activities of monitoring where only essential onsite visits are required, and then putting technology in place for the more advantageous, analytical monitoring of data. According to the SAS research, respondents believe that 30% of clinical trial costs are attributed to on-site monitoring. Reducing these costs, as noted above, is a key driver for the interest in risk-based monitoring. However, behind the cost drivers, there are challenges around the adoption of risk-based strategies. While 80% of the survey partcipants responded that they do take a triggered or targeted approach to risk-based monitoring, 80% also said that they desired 100% SDV in 100% of trial participant records. Further, 55% said that they use risk-based monitoring in their organizations and 80% said centralized monitoring is attractive but slow on the uptake in their organization. These discussions, while not statistically high, are indicative to the level of understanding of risk-based monitoring in the clinical trials industry. Basically, if your company is currently applying triggers or targets for site visit Do You Use Risk-Based Monitoring in Your Organization? Source: SAS, Applied Clinical Trials Risk-Based Monitoring, October 2013 Figure 1. Four Strategies of Risk-Based Monitoring Source: PriceWaterhouseCoopers (PwC), August 2013 Figure 2. Risk-based monitoring Reduced monitoring Centralized monitoring Targeted monitoring Remote monitoring No Yes Data-driven oversight is tailored to the specific study risk areas, and the monitoring approaches are continuously updated via robust data analysis. Monitoring at less than 100% SDV depending on study and site risk Monitoring based on triggers and predetermined assessments across sites Algorithm-based study risk assessment following customized parameters for each study Automated and nonautomated intervention based on statistical data review Transfer of monitoring activities to low-cost countries, reducing the need for high-cost resources requirements, your company is doing risk-based monitoring, as per the PwC descriptive table. And while these respondents also stated a 100% SDV preference, that may not be the key to quality data. Rose said, Another area very common to traditional organizations where they have concerns is that if you move away from 100% source data verification, then you might risk patient safety and data integrity. But, in fact, there s al- 4 APPLIED CLINICAL TRIALS appliedclinicaltrialsonline.com

5 ready evidence from studies by Clinical Trial Transformation Initiative, other organizations are showing significant impact by reducing SDV. She also notes that there are improvements in data quality and in capturing patient safety signals using EDC and through leveraging central statistical monitoring. From its survey at Industry Standards Research, Andrew Schafer, President, reported in the webcast that there are factors hindering risk-based monitoring. However, they are Upfront costs, however, will also lead to a cost reduction over time, by having better upfront identification of risk and trial performance expectations. not related to the models, but rather related to the industry and sponsor-based reasons. These reasons include a lack of strong regulatory guidance; models being too new and unproven; general corporate inertia; being risk averse; and averse to trying new techniques. A Note on Regulation The SAS survey also found the legion of respondents strictly adhering to the final FDA Guidance Oversight of Clinical Investigations A Risk-Based Approach to Monitoring issued in August However, they also noted they find the guidance lacking in detail or insight into cutting edge technologies. Rose noted during the webinar that RBM is not required by regulators. What is required is the monitoring of the progress and conduct of the trial. Rose said, The risk-based approach is being suggested as a way to become more efficient in clinical trial operations. And like in other areas, regulators are encouraging innovation with technology and analytics to help improve these processes. Behind the Costs That being said, cost was found to be the major driver for conducting risk-based monitoring studies, according to ISR s research. In the webinar panel discussion regarding costs, Rose said, A lot of costs [are] from the clinical research associates, whether it s travel or training. And sometimes with high turnover rates, retraining or having to re-recruit and retrain. So with a more central remote modeling, there are some upfront investments; and there are going to be business process changes within the organization. There are very likely going to be some software investments so that they are better improving the data capture at the site. Rose said these upfront costs, however, will also lead to a cost reduction over time, by having better upfront identification of risk and trial performance expectations. That can reduce a lot of the costs in the operations along the way. Things like faster recognition of patient safety, and protocol compliance, and data integrity issues, all of those will allow for some optimization of the trial. Hronec mentioned, many companies are doing pilots and still trying to figure risk-based monitoring out. Which includes exactly where they will get cost savings to then build that into the organization. He said, Right now it s in the early stages; but over the next several years, we ll be able to see those savings throughout the studies. Schafer concluded by saying that the momentum of riskbased monitoring is at a fairly good clip within the organizations they surveyed. Comparing to other studies that ISR has conducted, the momentum with risk-based monitoring is higher and more favorable than epro technology adoption, or even adaptive trial design. Schafer said, Risk-based monitoring isn t going anywhere and seems to be going to increase in momentum within the sponsor organizations. As revealed by the SAS research, companies are at different stages of development for risk-based monitoring. Companies realize they need to have more efficiently executed trials and risk-based monitoring can help them get there. With streamlined clinical trial operations, a more patient-centric approach is enabled. Patient data contains enormous amounts of information. We have to change the way patients are being treated within the system so we can design protocols that will actually fit the patient, said an executive director of a CRO. When that happens, clinical trials will be truly adaptive. They will go beyond collecting data to meet a specific endpoint. Instead, studies will have quality inherent in their design, their execution will be optimized, and they will drive the development of successful new therapies. The risks are simply too great for not doing riskbased monitoring. appliedclinicaltrialsonline.com APPLIED CLINICAL TRIALS 5

6 Risks, Signals, and Scoring What determines risk, and how are risk levels determined? Whichever risk-based monitoring type a company uses for data-driven oversight, a set of metrics, signals and levels of scoring risk is necessary to determine the monitoring plan. According to the SAS custom research project, many metrics and signals were used to identify, track, and understand the impact of trial characteristics. The most significant of these were identified as queries being issued at particular sites, EDC discrepancies, patient risk, and regulatory risk. Then the respondents identified measures or signals that would be used during an on-going trial to adjust or respond to the performance of a risk-based monitoring plan. These included work level readjustment, missed data points, data quality, data quantity, and number of queries. Across the board, the SAS research respondents determined what would trigger a sitemonitoring visit. These included: sites reporting a problematic issue with the trial; a centrally identified problematic trend in reporting; complexity of the trial; delayed recruitment; adverse events; case report forms needing to be monitored/not being filed for new enrollees; lack of communication by site; increases in enrollment; and on-site monitor reporting need for increased future monitoring. One director at a CRO noted, We don t have a formal process to initiate a site visit. When the monitor is seeing issues and questions coming from a site and we start seeing a gap in training or issues that occur repeatedly. Site Monitoring Schedules Source: SAS, Applied Clinical Trials Risk-Based Monitoring, October 2013 Figure 1. Dynamic site-monitoring Static site-monitoring The SAS research also concluded that when serious things go wrong with a trial, such as an adverse event or an incident report is filed, it is easy for the respondents to identify issues; however, unanticipated and less obvious situations can happen throughout the trial and put it at risk. These events should be identified as realtime as possible so that changes can be made as quickly as possible to help respondents stay on top of these issues. Mark Hronec, Director of PriceWaterhouse- Coopers, noted during a December 2013 webinar sponsored by SAS and Applied Clinical Trials titled Myths, Models and Momentum: Risk-Based Monitoring, this exact situation. He termed them traditional and non-traditional risk factors. 6 APPLIED CLINICAL TRIALS appliedclinicaltrialsonline.com

7 Along the traditional were those previously mentioned recruitment rates, protocol deviations, serious adverse events, the drug safety profile or the site location or experience at the site. However, those non-traditional risks that could be identified and leveraged using real-time big data predictors included average patient visit window deviation, the number of visits the site has per day, the number of inclusion or exclusion criteria, and the length of the informed consent form, among others. Hronec also said that the traditional and non-traditional risk factors are further framed under three categories protocol risk, study site risk, and dynamic risk. For example, protocol risks include the therapeutic area, drug safety profiles, etc. But also aspects around the number of inclusion/exclusion criteria, the number of procedural visits, etc., can be made to complete this profile of protocol risk. For the sites, the number of universities per square mile near the site, the physician density, etc. can be examined. And for the dynamic risks, the number of visits per day can be a potential risk indicator. Looking at these non-traditional factors, the potential relationships can help inform the riskiness of the site, the protocol, and then the overall study. Many questions in the webinar were centered on riskdeterminations. Laurie Rose, Principal Industry Consultant, Health & Life Sciences at SAS, recommended using historical data, information on past site performance; country and regional information and statistics; sites that perform better with recruitment, enrollment, and actual study startup. [With those data, you are] using a predictive model rather than just summarizing and putting them all together. Using a predictive model can help determine a better set of expectations of what might come in the future. Rose also noted other predictive data would include information within the same therapeutic area or other similar compounds from past studies and using that understanding in future studies. Where data is available, the analysis will be helpful in preparing for the next study. Who actually determines risk assessment in the organization? According to the experts on the webinar, risk should be determined in a cross-departmental way that includes the physician, clinical operations lead, a statistician, data manager, etc. Hronec said, It depends on whether or not the organization is doing this as a point-by-point solution, in which case you would mostly have that centralized within the study team. But if the organization wanted to take this on holistically, then you really look at how do we categorize the risk levels of allergy studies versus oncology studies? And with those types of conversations, those need to be escalated within the organization and include drug safety representatives as well. Another point that Rose brought up is risk re-use in studies. Risks will not be the same for every trial, but a number Do You Use Triggers and Risks to Determine a Site Monitoring Visit? No Yes Source: SAS, Applied Clinical Trials Risk-Based Monitoring, October 2013 Figure 2. Of the Following Benefits from RBM Technology, Which is the Most Important? 0.9% Source: SAS, Applied Clinical Trials webcast, December Figure % 25% 25% (during study) Early signaling of site compliance to the protocol, GCP compliance (pre-study start) Identify parts of the protocol according to their risk profile and implement mitigation strategies (during study) Early signaling of study risks and trends, potential protocol adjustments can be identified (pre-study start) Identify sites with quality risks and implement mitigation strategies (during study) Early signaling of under-reporting 29.6% of studies will have similar rules and certain set thresholds. There might be some of those that are common and can be used each time, but there will be others that will be specific to the protocol or to the therapeutic area that the medical personnel need to have influence on how those might change from study to study. The upfront understanding and measuring of risk to suitably determine the monitoring plan, as well as early discussion with regulatory authorities about the monitoring plan, can impact timelines. Andrew Schafer, President of Industry Standard Research, noted in the webinar, If you have not really planned ahead, it can elongate study startup and, in fact, the entire study timeline. He continued, From our respondents, most people believe that that upfront time is worth it to conduct the risk-based monitoring study; but you just have to account for it. appliedclinicaltrialsonline.com APPLIED CLINICAL TRIALS 7

8 Analytics See how leading firms are optimizing risk-based monitoring. If risk-based monitoring is the silver bullet to more quickly developing safe and effective products at lower costs, how are life science organizations approaching this critical need? Learn about the barriers and rewards to risk-based monitoring, get suggestions for moving forward, and see what the ideal system looks like. Read the report SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are trademarks of their respective companies SAS Institute Inc. All rights reserved. S119211US.0114

9 Technology Used to Achieve Risk-Based Monitoring Where is the technology for RBM and how do companies get to next stage predictive analytics? A SAS custom research project determined that organizations currently using riskbased monitoring are receiving benefits, and are focused on at least meeting or exceeding the benefits of on-site visits. These organizations intend to continue implementing innovative, technologically savvy, riskbased monitoring plans that will allow them to cut costs, improve quality, and meet regulatory requirements while putting patient safety first. The respondents also said they hope to achieve the following benefits in that exercise: identifying what is key in data collection/integration; the ability to focus resources on highest-need sites; reduction in site fatigue; empowering adaptive studies; better patient outcomes; and increasing understanding and integration of global data. As was observed in the previous articles in this ebook, upfront technology costs will be necessary, and the need to pull data into predictive tools is key to the risk-based monitoring profile. But what technologies are companies currently using to achieve risk-based monitoring? According to the SAS research, many were using proprietary applications, EDC platforms, CTMS, and IVRS; and many sponsor organizations were looking to rely on their CRO systems for these needs. And from the custom research, it appears that CROs are ahead of biopharma companies in using technology, while smaller biopharma companies lack the resources to keep up. We do remote monitoring via EDC, Are You Considering a Centralized Monitoring Strategy? Source: SAS, Applied Clinical Trials Risk-Based Monitoring, October 2013 Figure 1. said the data operations leader of one CRO. Another firm reported it mixes hard copy and electronic reports. We have regular reporting from the CRO, after a dosing schedule and interim reports and final reports. We use both electronic and hard copy to evauate and review, said the CEO of a biopharmaceutical company. We establish the site in advance, and I review the sites and audit the sites in person, responded the CEO of a biopharmaceutical company. Then we do the study. It is fixed once we establish this. In a separate survey by Applied No Yes 8 APPLIED CLINICAL TRIALS appliedclinicaltrialsonline.com

10 Clinical Trials, conducted in October 2013, 36% of the respondents said they are looking to purchase technology to support risk-based monitoring within the next two years. Centralized Monitoring Centralized monitoring is one riskbased monitoring strategy that is benefitted by a healthy dose of technology investment. The majority of respondents in the SAS custom research project said that while they want to increase centralized monitoring, they don t have strategic plans in place to take them there. Instead, they are still relying on static schedules for on-site monitoring. But using technology for centralized, risk-based monitoring and predictive analytics, as defined by regulatory guidelines, is well documented in the industry. While the respondents of the SAS research project noted a lack of specificity in the FDA Guidance, the documents does state: FDA encourages greater use of centralized monitoring Study Data practices, where appropriate, than has been the case historically, with correspondingly less emphasis on on-site monitoring. Again, not specific what types of electronic systems should be used, however, the FDA recommends that sponsors who plan to incorporate centralized monitoring, should ensure that the processes and expectations for site record keeping, data entry, and reporting are well-defined and ensure timely access to clinical trial data and supporting documentation. While firms would like to move toward technologicallydriven, centralized risk-based monitoring; they are only taking baby steps. Risk-based is the only way, said the clinical development group leader of a CRO. And the chief scientific officer of a CRO added: Everybody is going to go risk-based with less frequent monitoring. Due to technology, the information is available in most systems. What Technology for RBM? The key for risk-based monitoring, from an information Source: SAS, Applied Clinical Trials webcast, December com/monitoring Figure 2. Everybody is going to go risk-based with less frequent monitoring. Due to technology, the information is available in most systems. Risk Identification and Assessment Risk Identification & Assessment Risk Business Rules Risk / Predictive Modeling Assess & Score Monitor & Report management perspective, is to bring together both clinical data from past studies, as well as the operational data, especially site performance, to use in assessing risk for new studies, said Laurie Rose Principal Industry Consultant, Health & Life Sciences at SAS, during a webinar sponsored by SAS and Applied Clinical Trials titled Myths, Models and Momentum: Risk-Based Monitoring. Those historical data sources enable views of the past studies that can be analyzed to help set a baseline for what to expect in similar new studies. Once the new study is established, incoming data is also prepared for views and risk analysis by country or therapeutic area, site, investigator, CRA, or any other preferred hierarchy. But where is the technology for RBM exactly and what should organizations be looking for? Rose said that one myth is that there s really no proven RBM technology. While she conceded it may not be a complete myth, she did note a number of pilots where results are being published, including a joint project in the UK by the Medical Research Council, Department of Health, and the Medicines in Healthcare Regulatory Agency, published in In addition, the TransCelerate BioPharma consortium of 19 pharmaceutical companies that have been sharing and presenting best practices on a number of initiatives, including Risk-Based Monitoring. Said Rose, TransCelerate as a whole has been sharing methodology and analytical appliedclinicaltrialsonline.com APPLIED CLINICAL TRIALS 9

11 During the presentation and the Q&A, the inherent challenge of disparate data sources was noted, as well as preparing the data for advanced analytics. techniques that leverage technology to innovate these processes. The TransCelerate methodology does not suggest specific technologies or solutions that are offered by technology vendors in its current paper. It does list a Systems Requirements and Preferred System Attributes, under categories including Planning, Data Capture, Data Aggregation, Analytics, Reporting and Visualization, and Miscellaneous. But for Analytics, TransCelerate says that those preferred system attributes include the following abilities: apply analytics to aggregated data (virtually or actual) in order to identify outliers, trends apply thresholds relative to outliers; prefer dynamic threshold definition indicate potential quality risk automated reporting Rose described the components that SAS believes will take shape in risk-based monitoring technology decisions and strategies in the near future. The technology approach starts with information management. It is the critical first step that pulls together essential data sources and provides a view across sites, investigators, countries, monitors, and patients. And then using analytics to assess the risk elements of the trial and managing those risks, those will make up core components for moving to a more riskbased approach and having the ability to mitigate risks that are discovered through this more central approach to monitoring. It is in the analytical exercise that additional factors from historical data would be leveraged--from what did happen to look more into the future for what is the potential of something happening. Said Rose, A particular area that it s going to be useful in is with adverse events or serious adverse events where a predictive model is going to spot a trend where adverse events might only be showing up in a couple of sites, not all sites. But in those unique sites that have some outliers, there might be similar characteristics of the types of patients; and you re just not going to find that in the traditional monitoring where you re looking at that data one site at a time. But when you have the ability to look all the way across, use more predictive models to look at trends, then you can start seeing things like safety signals with patients much sooner. Data Aggregation in Practice The processes that need to take place after the data sources are identified are filtering and aggregating to determine which data elements make sense to use in assessing the risk, and automating the data collection and the refresh process, explained Rose. Depending on the stage of the trial or even the type of the study, that refresh and update of the data might vary from near-real time updates to daily, weekly, or monthly summaries. The data can also be refreshed in a way that can support cumulative analysis, as well as just a snapshot as both of those may help reflect different types of risk. During the webinar presentation and the Q&A, the inherent challenge of disparate data sources was noted, as well as preparing the data for advanced analytics, which drives the risk assessment and the scoring process. The audience wondered how industry could manage the aggregation of data when multiple providers are being utilized in outsourced functional service provider models. Rose admitted that it is a challenge, but that it would be true with any data aggregation issues with different providers and different systems, and it was not specific to risk-based monitoring technologies. However, Rose advised that a plan for that aggregation of that data be in place, especially on the provider s side. For a sponsor who might have multiple providers, from a pure data integration and from a technological perspective, there are information management systems available that can gather data from any type of a data source and bring that together. I know from a clinical data perspective, it would be very helpful to have CDISC standard formatted data, which most of the EDC systems are using. Not minimizing the challenge, Rose said how organizations are going to bring the data together and what data they will use to run that monitoring process from a more centralized approach, is something that needs to be determined when making their risk management plans. She noted, Innovative technology will enable improved processes for collecting and analyzing data centrally, across an entire study, so that both central risk monitoring groups and on-site monitors will have improved insight into clinical studies. 10 APPLIED CLINICAL TRIALS appliedclinicaltrialsonline.com