Welcome to the training on the TransCelerate approach to Risk-Based Monitoring. This course will take you through four modules of information to

Transcription

1 Welcome to the training on the TransCelerate approach to Risk-Based Monitoring. This course will take you through four modules of information to introduce you to the concepts behind risk-based monitoring, the TransCelerate Approach, some hands on exercises and some information to provide to sites as well on tips for transitioning projects and studies into the new model. 1

2 2

3 At the conclusion of this workshop, attendees will be able to Module 1 - Introduction to Risk-Based Monitoring (RBM) Methodology 1. Explain the rationale for Risk-Based Monitoring (RBM) 2. Describe TransCelerate s founding principles and key assumptions 3. Describe the Risk-Based Monitoring (RBM) Methodology as compared to traditional monitoring methods Module 2 - Risk Assessment 1. Discuss methods for identifying risk for planning purposes 2. Identify Critical Data/Processes for Risk-Based Monitoring (RBM) application Module 3 Risk Management 1. Define and utilize Risk Indicators and Thresholds in decision-making 2. Discuss implementation of risk mitigation plans 3. Identify site-level risks 4. Describe how to conduct monitoring activities in the RBM model including central, off-site and on-site monitoring 5. Describe appropriate responses to potential issues and risks throughout the study Module 4 - RBM and Change Management 1. Discuss considerations with implementation within an organization and at sites 2. Describe metrics used to measure the impact of the proposed methodology 3. Q & A 3

4 As previously stated, each module will have some key objectives. At the conclusion of this module, you will be able to describe the RBM Model as compared to traditional monitoring methods, explain the rationale for RBM, and describe TransCelerate s key assumptions and concepts. This course was designed to be very interactive and as we move through the materials we will ask you to challenge yourself and to share your knowledge. Many of you already probably know quite a bit about RBM and some of the questions we will be asking are designed to help you determine if there are areas in which you still have questions. 4

5 What is driving the adoption and move towards risk-based monitoring in the clinical trials industry? 5

6 Note to speaker: Possible solutions/answers are on the next slide. Wait until participants generate/provide some of their own ideas before displaying next slide. Suggestion for activity is to have each person write down their three ideas and then share with the group or write up to three ideas on individual sticky notes and they are all passed forward for speaker to read. Note: Depending on delivery method, the question on the slide can be removed at member company discretion 6

7 The endorsement of a change in monitoring by key regulatory authorities serves as a rationale for looking at monitoring in a different way. These regulatory authorities are communicating that monitoring can and should be designed and customized to meet the specific needs of the program and study. From the sponsor perspective, there may be multiple and varied reasons for changing our approach to monitoring. The rationale for change may vary depending upon everything from the size of the sponsor to the type of study being conducted. Changing our monitoring approach potentially provides benefits such as improved risk mitigation, adapting monitoring to the needs of the trial or site, and more effective use of current state technology. Some challenges facing the industry, such as protocol complexity, the focus on costbenefit ratio, and the need to have smarter use of resources, are also serving to help drive a shift in our monitoring philosophy. A key benefit is the ability to focus limited resources on high quality high impact activities. 7

8 Note: These speaker notes can be amended as needed As you can see based on the timeline on the screen, there have been a number of actions taken that are directed towards modifying our monitoring approach. The 1988 FDA Guidance on Monitoring of Clinical Investigations stressed personal contact between the monitor and investigator. This was withdrawn by FDA in 2010 as evidence grew for the need of a shift in our approach to monitoring. The 1996 ICH E6 (GCP Guideline) provided flexibility in how trials are monitored; centralized monitoring alone appropriate only in exceptional circumstances. In 1998 there was an FDA Guidance issued in which the agency suggested more flexibility in what s considered acceptable monitoring and provided data standards for studies with minimal on-site monitoring. In 2007 at the IFPAT meetings (GMP Manufacturers), Janet Woodcock, Chief Medical Officer of CDER/FDA talked about the intersection of Process Analytical Control (PAT) in GMP and Quality by Design (QbD) in clinical development. Additionally, Helen Winkle (Director, Office of Pharmaceutical Science, CDER/FDA) gave a presentation on QbD in September In 2009 CTTI was formed with the mission to identify practices that through broad adoption will increase quality and efficiency of clinical trials. 120 members from FDA, academia, industry, government, and patients/investigators participating. One of the earliest projects was to identify current monitoring practices and apply Quality by Design (QbD) principles to clinical trials. Between 2009 and 2010 we saw more FDA Warning Letters to Sponsors with findings of failure to adequately monitor clinical investigators. This finding includes improper selection of investigators who subsequently fail to meet GCP requirements, failure of monitors to find protocol compliance issues, and/or failure of sponsors to promptly take actions to correct deficiencies when identified through monitoring. In 2011 two draft documents were issued from the FDA and the EMA, and can be considered a clear 8

9 indication that we are being encouraged to modify our practices. In 2013, the FDA and EMA each finalized their guidance documents. 8

10 Some of the documents mentioned on the timeline include the Clinical Trials Transformation Initiative, the FDA Guidance for Industry, A Risk-Based Approach to Monitoring, and the EMA Reflection Paper on Risk Based Quality Management in Clinical Trials. Note: These speaker notes can be amended as needed These three documents provide a framework for some of the concepts that are driving the industry to change. We will not be spending a great deal of time discussing these documents, but it is important to understand that there is a regulatory drive to adapt our practices according to risk and move away from the idea of one size fits all. The CTTI project focused on gathering data, confirming the current industry approaches to monitoring, and verifying that our primary focus should shift from post-hoc inspection to incorporation of quality into the scientific and operational design of a trial. CTTI stated there is not one single approach that is appropriate or necessary in all circumstances, and that the monitoring approach for a given clinical trial should be tailored to the needs of the trial and may combine several methods of monitoring. Furthermore, the CTTI participants agreed that the quality of the protocol is likely an important determinant of the quality of monitoring. The FDA Guidance was intended to assist sponsors in developing risk-based monitoring strategies and plans tailored to the specific human subject protection and data integrity risks of the trial. It included a focus on critical study parameters, encouraged the use of a combination of monitoring activities and promoted greater reliance on centralized monitoring practices, where appropriate. 9

11 The EMA Reflections Paper focused on Risk Based Quality Management through assessment of the use of risk identification and control. The key points were to develop a plan at the start of a program, adapt protocol by protocol, build on experience gained with each study and build on technical, regulatory, and other advances. 9

12 10

13 This quote from the EMA paper stresses that applying risk based quality concepts to clinical trials can help us maximize the use of resources and result in better, more informed decisions and conduct of our trials. Additionally, the EMA provide comments about the current Environment and Planning and Conduct of Studies. With regard to the Environment, the EMA mentioned: Environment High cost of Clinical Development and resource limitations Fragmentation of roles, often without clear distribution of tasks (piecemeal implementation of technology) Risk aversion (public and private) Stifling of innovation by restrictive business practices, preconceived ideas, incorrect perceptions Resistance to implement and accept new approaches and new technology Over-interpretation or misunderstanding of regulatory environment With regard to the Planning and Conduct of studies, the EMA commented Planning and Conduct of Studies Poor design of studies, complexity of protocol, complexity of processes Failure to identify priorities (loss of focus on important objectives and critical processes) Poor risk identification and mitigation (corrective rather than preventive) Lack of proportionality (one size fits all) 11

14 4/23/2015 Answer: D 12

15 4/23/2015 Answer: D 13

16 So, you may be wondering what exactly is TransCelerate? Why did TransCelerate develop a RBM methodology? And what are the core concepts of that methodology? 14

17 TransCelerate BioPharma Inc. is an independent non-profit organization focused on accelerating the development of new medicines. TransCelerate was founded with a mission to identify and solve common drug development challenges with the end goals of improving the quality of clinical studies and bringing new medicines to patients faster. TransCelerate was launched on September 19, At that time, the organization chose to focus on five initiatives related to clinical trials designed to increase efficiency, reduce costs and enhance patient safety. On this slide you see the 19 companies that are currently part of TransCelerate. Take a moment to look at the vision, mission statement and core values.

18 TransCelerate is well aware that it is part of a very wide healthcare eco-system. It is one thing to develop and design solutions that will benefit the all stakeholders, it is another thing to get them adopted more broadly by the industry as a whole. Therefore it is critical for TransCelerate to engage with, and secure feedback from a variety of partners in the industry and regulatory bodies. Some examples: TransCelerate has engaged with BIO - BIO is the world's largest trade association representing biotechnology companies, academic institutions, state biotechnology centers and related organizations across the United States and in more than 30 other nations to ensure they understand what TransCelerate is trying to accomplish and are aware of TransCelerate s proposed solutions for the industry. Secondly, as many of TransCelerate s solutions are intended to improve efficiency at the study-level, ensuring that Investigator Sites and Clinical Research Organizations are aware of TransCelerate is important. To that end TransCelerate recently finalized an agreement with the Society for Clinical Research Sites (SCRS) to establish dedicated Site Advocacy Groups, specifically to participate on TransCelerate workstream. SCRS is a fast growing global organization created by the investigator community to advocate for the investigator community. SCRS provides TransCelerate direct access to top investigative sites in the US, Europe, Latin 16

19 America and soon China. Through this newly formalized partnership, TransCelerate's working teams will have direct access to these top sites and investigators to gather timely input/feedback on various workstreams outputs. TransCelerate also engages with CROs through their trade organizations, ACRO and JCROA in Japan. Finally, TransCelerate recognizes the importance of open lines of communication & dialogue with regulatory bodies / health authorities around the world to ensure their input and guidance is obtained on proposed solutions and that their feedback is taken into account prior to any releases of deliverables. 16

20 TransCelerate BioPharma Inc. developed a methodology that shifts monitoring processes from an excessive concentration on Source Data Verification (to be discussed in further detail later) to comprehensive risk-driven monitoring. The TransCelerate RBM team started from an understanding that by building quality and risk management approaches into the scientific design and operational conduct of clinical trials, risks can be mitigated and issues can be detected early or prevented entirely. Additionally, the TransCelerate partners recognize that although current on-site monitoring practices do provide a level of control, advances in risk-based approaches and technology provide an opportunity for a more holistic and proactive approach. This philosophical shift in monitoring processes employs Centralized and Off-site mechanisms to holistically monitor important study parameters and uses adaptive Onsite Monitoring to further support site processes, subject safety, and data quality. Through modernization, including use of technology enablers, efficiencies can be gained without impacting subject safety by implementing quality risk management approaches to clinical trial oversight. 17

21 There are 9 assumptions underlying the TransCelerate monitoring methodology promoted in the May 2013 position paper. 1. Central and off-site monitoring form the foundation of monitoring, complemented by targeted, risk-based on-site monitoring activities. 2. Monitoring activities should be responsive to issues/risks identified and increased in a targeted, temporary manner. The goal should always be to return to the baseline level of monitoring. Root cause analysis is critical to prevent identified issues from recurring. 3. Monitoring methodology and activities must be tailored to the technology available for the program, study, and/or site. 4. Since central/off-site monitoring is the foundation of sponsor oversight, timely data entry and query resolution are critical. Sponsors should establish expectations in site contracts. 5. The Integrated Quality Risk Management Plan (IQRMP) is a living document and should be revised/amended throughout the study as needed in response to changes in the study, identified risks, etc. 6. It is acceptable to define risk-based monitoring standards in associated SOPs rather than within each study-specific IQRMP. 7. The defined methodology is applicable to all phases of studies, types of studies, and stages of clinical trials. 8. Communication plans should be tailored or customized in whatever way is necessary to maximize effectiveness. 9. Risk assessment should be completed prior to protocol and Case Report Form (CRF) finalization as a means to address and minimize risks before the trial starts. Then, monitoring strategies can be used to oversee and manage risks that cannot be prevented via protocol/crf design strategies. 18

22 TransCelerate s RBM methodology embraces the concept of building Quality by Design (QbD) into clinical trials starting with protocol development and extending across all aspects of a trial. A well-written protocol and CRF are important facets which may impact the success of the RBM methodology. Quality refers to the ability to effectively and efficiently answer the intended question about benefits and risks while assuring subject safety. Decisions are supported by quality data which is not considered error-free data, but rather fit-for-purpose data that sufficiently supports conclusions equivalent to those derived from error-free data. QbD includes a focus on identifying key risks to subject safety, data quality, and GCP/regulatory compliance. QbD also involves the application of an efficient monitoring approach to rapidly detect and correct issues while the study is ongoing and developing a quality risk management plan that focuses on factors that are at high risk for generating errors. It is also critical to apply monitoring strategies that are tailored to risks, permit timely oversight (through central/off-site monitoring and use of technology), and are focused on Critical Processes and Critical Data. On-site interventions should be targeted in nature. In summary, QbD provides a basis for implementation of RBM and is a fundamental principal of the TransCelerate methodology and tool application as we will discuss in later modules. 19

23 As you can see Building QbD is the first step in the TransCelerate approach. TransCelerate developed a standard approach for RBM that can be adopted for any type, phase, and stage of trial. The TransCelerate RBM methodology improves efficiency by changing the focus to Central or Off-site Monitoring activities that are intended to identify potential issues sooner than a monitoring strategy that relies primarily on site monitoring visits. TransCelerate s RBM methodology uses quality risk management as a foundation in ensuring subject safety and data quality through the implementation of the following: (1) building QbD into trials (2) early and ongoing risk assessment, (3) a focus on Critical Processes and Critical Data, (4) use of Risk Indicators which assess critical Data and other study variables (in many cases by comparing across program / protocol / country / site) and Thresholds, defined as the level, point, or value associated with a Risk Indicator that will trigger an action. Details surrounding the Risk Indicators and Thresholds will be documented in the various study plans, which fall under the Integrated Quality and Risk Management Plan, or IQRMP, (which we will discuss in detail in a later module) and (5) adjustment of monitoring activities based on the issues and risks identified throughout the study. By monitoring available data Off-site or Centrally, On-site Monitoring can be targeted to activities which cannot be assessed remotely. Additionally, TransCelerate has adopted the term Source Data Review (SDR) which describes review of source data for protocol compliance, quality of documentation, as well as site processes in contrast to transcription checking, referred to as Source Data Verification (SDV). The TransCelerate RBM methodology is aligned with the QbD paradigm, and with the monitoring and study oversight expectations of health authorities; when RBM methods are used, applicable ethical standards, subject rights, laws and regulations are expected to be followed. Further, TransCelerate s methodology is being developed concurrent with the transition to risk-based inspection processes by health authorities. 20

24 As noted, we will be asking a lot of questions and pushing you to challenge yourself and test your knowledge. Let s go ahead and begin with a true/false question. ANSWER: False RBM is not fixed, it is an adaptive approach to clinical trial monitoring that directs monitoring focus and activities to the evolving areas of greatest need which have the most potential to impact subject safety and data quality. Each clinical trial requires its own customized monitoring approach to ensure that risks are minimized and sponsor responsibilities are satisfied. It is based on awareness that different trials, different data points/processes, and different sites represent differing risks to the product s development. Note to speaker: The answers are listed on each slide following the true/false question and may be hidden if you choose not to use. 21

25 As noted, we will be asking a lot of questions and pushing you to challenge yourself and test your knowledge. Let s go ahead and begin with a true/false question. ANSWER: False RBM is not fixed, it is an adaptive approach to clinical trial monitoring that directs monitoring focus and activities to the evolving areas of greatest need which have the most potential to impact subject safety and data quality. Each clinical trial requires its own customized monitoring approach to ensure that risks are minimized and sponsor responsibilities are satisfied. It is based on awareness that different trials, different data points/processes, and different sites represent differing risks. 22

26 Risk-Based Monitoring, or RBM, is not a new concept and there has been a significant amount of discussion around the topic for the past several years. Many sponsors and CROs have adapted to a type of RBM based on recent regulatory guidance documents circulated in the US and in Europe. In this module we will compare the RBM model to traditional monitoring methods in order to better understand the intent of RBM. 23

27 Note to speaker: The intent of the question is to ensure the audience comes to a general consensus on what traditional monitoring means as everyone may have a slightly different interpretation and this should lead to some discussion. Some possible answers and examples you can provide are: Monitoring On-Site, monitoring on a set schedule, source verifying all data on site. The next slide will further define On-Site Monitoring which should be considered general traditional monitoring practice in the industry (although some companies may have taken a different approach, monitoring every 4-6 weeks, verifying 100% source documents on site has been very typical). Note: Depending on delivery method, the question on the slide can be removed at member company discretion 24

28 We generally think of traditional site monitoring as an approach consisting of monitoring all data, in person, on site. Traditional on-site monitoring can be defined as an in person evaluation carried out by sponsor or CRO representatives at the location where the study is being conducted. The visits are generally conducted based on a set visit window schedule such as every four to six weeks and all data is source verified 100% regardless of the type of study, safety risks, phase of the study, stage of the study, or experience of the individuals conducting the study. On-site monitoring is conducted to identify missing data in source records and data entry errors in case report forms, assess compliance with protocol and investigational product accountability, and to evaluate investigator supervision. Remember that we were monitoring before there was technology, so there was little choice in the approach we could take to review the data other than reviewing it on site. Reference: FDA Guidance for Industry: Oversight of Clinical Investigations A Risk-based Approach to Monitoring, August

29 The term monitoring is used very loosely in the industry and has come to refer to a number of different activities conducted by sponsor and CRO personnel in clinical trials. We have seen it in the context of an individual that goes to the site to review data, or when data is checked by data management. Medical monitors and/or pharmacovigilance groups review safety data and this may be referred to as monitoring. And finally, there may be Quality Control monitoring by Sponsor and CRO internal processes and systems. However, for the purposes of our RBM discussion. the focus is on updating and revising traditional site monitoring approaches. Note to speaker: The next slide will ask the audience to tell us how they might define traditional site monitoring. 26

30 So, if RBM is an approach that updates our concept of monitoring from the traditional approach, how are they different? While traditional monitoring is conducted in a one size fits all schedule and approach (every site is visited every 4-6 weeks), RBM customizes the monitoring approach to each individual trial based on risk assessment to identify potential issues. RBM makes use of all available technology to allow sponsors/cros to supervise study conduct without having to be at the site location. RBM involves many different functions and roles of the sponsor/cro, not just Clinical Research Associates or Monitors. It includes a recognition that monitoring is a crossfunctional responsibility. And finally, as opposed to depending primarily on activities conducted at the site (onsite monitoring), RBM relies more heavily on central and off-site monitoring activities. Detailed descriptions of central, off-site and on-site monitoring will be covered later in greater detail. 27

31 The concept of comprehensive monitoring within the TransCelerate RBM methodology involves 4 key ideas: The baseline monitoring activities, such as sampling strategy and monitoring frequency, are defined by the program and study-specific risks Central and/or off-site monitoring activities provide a foundation to ensure timely and adequate monitoring On-site monitoring activities are triggered by Risk Indicators and Thresholds Monitoring involves everyone, not just Clinical Research Associates/Monitors Central Monitoring Activities: Application of analytics to compare Risk Indicator data and information (across studies, between investigative sites, etc.) which may include: Protocol deviation rates Data entry and query resolution metrics Adverse event trends or outliers Subject discontinuation trends 28

32 Unusual data trends or patterns Error rates in Critical Data/Processes Offsite Monitoring Activities Confirm timeliness and quality of data entry Review query resolution Review CRF to check protocol compliance Confirm site s completion of previously identified actions Assess site s recruitment and enrollment Monitor for changes in site staff Monitor delegation of responsibilities Conduct training Onsite Monitoring Activities Informed Consent Review Eligibility verification Subject safety review Investigational Product Accountability Essential Documents Review (if appropriate) Face-to-face training and discussions with site staff 28

33 So, let s see what you ve learned so far. [Read slide] ANSWER: All statements are true definitions, potential uses, and/or appropriate applications of centralized monitoring. Note to speaker: Answer key is on the next slide if you would like to show it 29

34 So, let s see what you ve learned so far. [Read slide] ANSWER: All statements are true definitions, potential uses, and/or appropriate applications of centralized monitoring. 30

35 In summary, the RBM Model varies from traditional monitoring methods through using a combination of On-site, Off-site, and Central Monitoring and risk assessment to identify critical study points and plan an individualized approach for monitoring based on the risks of the study. The rationale for RBM is based on changes in the industry driven by protocols, technology, and resources TransCelerate s key assumptions and concepts include a focus on a proactive approach to monitoring through quality protocols and case report forms, and shifts monitoring processes from an excessive concentration on SDV to comprehensive risk-driven monitoring based on risk assessment, mitigation and management. 31

36 Module 2- Risk Assessment. In module 2 we will be focusing on how to identify and quantify risk and will address one of the key measurement tools, the RACT, in detail. 32

37 In this module, we will be taking a more hands-on approach to the application of concepts, tools, and materials described in the TransCelerate Risk-Based Monitoring (RBM) methodology. By the end of this module, learners will be able to identify protocol risks. Learners will be able to identify Critical Data/Processes and use the Risk Assessment Categorization Tool to perform risk assessment. 33

38 In this section we will be defining risk and the general steps in the cycle of risk assessment. The output of all risk assessment activities leads into the overall plan for monitoring the study. 34

39 What is a risk? Risk is defined in the Quality Risk Management (Q9) topic from the International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) as the combination of the probability of harm occurring and the severity of that harm. We can think of a project or protocol risk as something that can go differently than planned and thus, jeopardizes our ability to achieve a goal or objective. There are four key questions that can be used when working with risks: 1. What might go wrong? 2. What is the likelihood it will go wrong? 3. What are the consequences? 4. How easy it is to detect? The answers to these questions help us identify risks and ensure we plan for those that are most likely to occur and have the greatest impact. 35

40 An understanding of risks and their management is fundamental to the RBM methodology. So early identification is a core principle of the process and allows us to plan for managing risks that arise during the study. The risk assessment process includes identifying potential risks, evaluating their likelihood, their impact on the program and/or clinical trial, one s ability to detect and how risks can be either accepted and managed or prevented altogether. 36

41 This process flow will serve as our roadmap as we learn about risk management activities in the RBM methodology. First, we assess our risks. Then, we move into further evaluating the protocol to answer three key questions: 1. Now that we know what our risks are, what helps us to detect them? Risk indicators are established to allow us to detect potential problems in the study. 2. At what point in the study will we know there is a potential problem requiring action? Establishing Thresholds for our Risk Indicators helps us to establish expectations and sets a trigger point for action. 3. How will we respond when a Threshold is reached and/or a potential problem is found? Actions are designed to appropriately respond to data and information that indicates a possible problem. Let s look at the topic of the RACT first and see how they can help us to detect risks to our study. 37

42 Monitoring activities and resources are focused on Critical Data and Processes in the application of the RBM methodology. This section will cover Critical Data/Processes and their relationship to risk assessment. 38

43 Critical Data includes data that support primary and key secondary objectives and data that is critical to subject safety (e.g. serious adverse events, events leading to discontinuation of treatment). Critical Data includes data that will be used to make decisions about the product s safety and efficacy profile. In identifying Critical Data, team members should think about specific end point data that will be used to satisfy the study s primary objectives for efficacy and/or safety. Critical Processes include processes that underpin data quality such as blinding, referring events for adjudication, and controlling inter-rater variability. Critical Processes also underpin subject safety and ethical/gcp compliance such as seeking appropriate medical consultation or scheduling extra visits or procedures in the event of significant clinical or laboratory findings. For example, in a psoriasis study, a Critical Process may be the measures taken to ensure consistency in clinical evaluations such as requiring site evaluators to meet certain qualification requirements. Critical Processes can include how exams are conducted, how lab specimens are processed and handled if they must be stored at exact temperatures, reconstitution or preparation of Investigational Product (IP), etc. Note: Depending on delivery method, the question on the slide can be removed at member company discretion 39

44 Let s work through an example together to understand how to identify Critical Data and Processes. 40

45 Continuing with this same protocol, let s look now at the primary and secondary objectives and identify any Critical Data and/or Processes we can find within this information. Note to Speaker: Ask a participant to read the objectives or read them aloud to the participants. Facilitate the completion of the assignment (in orange/yellow text). Answer key is provided below and on the next slide. 41

46 Note to Speaker: This activity will be completed through review of a sample protocol synopsis. Instructions for the activity are provided separately in the participant workbook, recognizing that some organizations may prefer to use an internal protocol synopsis to make the activity more applicable to the specific work environment. 42

47 Critical Data includes data that support primary and key secondary objectives and data that is critical to subject safety (e.g. serious adverse events, events leading to discontinuation of treatment). Critical Data includes data that will be used to make decisions about the product s safety and efficacy profile. In identifying Critical Data, team members should think about specific end point data that will be used to satisfy the study s primary objectives for efficacy and/or safety. Critical Processes include processes that underpin data quality such as blinding, referring events for adjudication, and controlling inter-rater variability. Critical Processes also underpin subject safety and ethical/gcp compliance such as seeking appropriate medical consultation or scheduling extra visits or procedures in the event of significant clinical or laboratory findings. For example, in a psoriasis study, a Critical Process may be the measures taken to ensure consistency in clinical evaluations such as requiring site evaluators to meet certain qualification requirements. Critical Processes can include how exams are conducted, how lab specimens are processed and handled if they must be stored at exact temperatures, reconstitution or preparation of Investigational Product (IP), etc. 43

48 When we identify Program-level risks we need to consider some of these questions (read slide) 44

49 As we further identify Trial-level risks we need to consider (read slide) 45

50 The evaluation of risks often starts with considering the significance or rank of the risk. Determining the rank of one risk is often relative to other identified study risks. Categories of monitoring activity risk might, for example, be ranked as high in red, medium in yellow, or low in green, based on discussions using agreed upon risk categorizations (i.e., completing the RACT or a similar tool). 46

51 Finally, it is always preferable to prevent risks from occurring if at all possible. Once a significant risk is identified, team members should decide if the risk can be eliminated by modifying the protocol or case report form(s). However, we must accept that it is not possible to eliminate every risk in our studies. We can, however, plan our risk management activities to detect, measure, and act upon our anticipated risks in a timely manner to lessen their impact. Monitoring strategies can also be adapted to ensure oversight of what is not prevented via protocol or CRF design. These aspects of risk management are the topic of Module 3 of this course and will be reviewed in more detail there. Note: Depending on delivery method, the question on the slide can be removed at member company discretion 47

52 Documentation of Critical Data & Processes Team alignment on what matters Focused Risk Assessment Consistent Approach to Assessment Each category has a series of questions Provides examples of high, medium, and low risks Output Determines the Overall Risk Level (high, medium, or low) for monitoring activities Provides direction on where the most attention is needed for monitoring and mitigation 48

53 This slide shows the types of categories that are listed in the RACT. Some categories may be specific to program level risks and others may be applicable to both program and protocol risks. All of these categories need to be considered when defining risks. 49

54 Often it is believed that everything is critical, but after the dashboard becomes active, engine has done its check of systems, only critical lights come on. 50

55 When is the RACT completed? During study planning and before functional risk mitigation plans are finalized (such as the Monitoring Plan, Data Plan, Safety Plan, and/or other plans contained within the Integrated Quality Risk Management Plan (IQRMP)). Who is involved in completing the RACT? This should be a cross-functional group involving various roles and team members. Some participants may include representation from the following groups: Data Management, Monitoring, Medical Monitoring, Study Management, Quality Assurance, Statistics, Regulatory and Safety/Pharmacovigilance. The exact participants may vary depending upon companyspecific infrastructure and processes. Reminder to Trainer: update the slide prior to the training session in order to properly communicate your company s process/role & responsibilities. 51

56 52

57 53

58 54

59 In this module, we have discussed the importance of methods to proactively identify and assess risks in the RBM methodology. Participants have learned about Critical Data and Processes and practice their identification as part of risk assessment. Finally, we ve reviewed the purpose and use of the RACT from the RBM Toolkit and seen how its use can facilitate a thorough and systematic risk assessment process. 55

60 Module 3- Risk Management. Module 3 will further address risk management and how to define critical Risk Indicators and Thresholds in decision making. 56

61 By the end of this module, learners will be able to define Risk Indicators and Thresholds for a particular trial and demonstrate their use in decision-making through a hands-on activity built around a mock protocol synopsis. Learners will be able to discuss the implementation of risk mitigation plans and describe how monitoring activities are conducted in the RBM model, including the concept of a baseline monitoring approach, and modifications to that approach. Finally, learners will be able to describe how potential risks and issues can be managed throughout the study by applying appropriate response plans and actions. 57

62 Let s start by reviewing the concepts of Risk Indicators and Thresholds. These concepts add substantial value as personnel make data driven decisions utilizing information obtained through monitoring of a clinical trial. 58

63 The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, known as ICH, defines risk management in Topic Q9 as [read slide]. Assessment of risk is accomplished in the RBM methodology through identification of Critical Data and Processes and completion of the Risk Assessment Categorization Tool (RACT). Controlling, communicating, and reviewing risks will be the topic of this module as we talk about Risk Indicators, Thresholds, action plans and risk mitigation. 59

64 This process flow will serve as our roadmap as we learn about risk management activities in the RBM methodology. As we just mentioned, in the RBM methodology, first we identify our risks through the process of risk assessment using the RACT tool we reviewed in module 2. Let s look at the topic of Risk Indicators first and see how they can help us to detect risks to our study. 1. Now that we know what our risks are, what helps us to detect that risk? Risk indicators are established to allow us to detect potential problems or risks in the study. 60

65 Risk Indicators can be thought of as variables defined through risk assessment that need to be reviewed regularly and may indicate a potential issue. Risk Indicators are variables assessed by comparison across a program, study, country, and/or site. All Risk Indicators are considered to have underlying influence on the quality of a study. 61

66 TransCelerate created a collection of Risk Indicators, provided in the position paper, which are intended to be monitored in an ongoing fashion either centrally or through off-site activities. This table may serve as a valuable resource to facilitate discussions within the team in the identification of Risk Indicators. Here you see 8 TransCelerate categories and an example for each potential Risk Indicator. There is no requirement that every Risk Indicator in the TransCelerate paper has to be implemented for every clinical trial. These are suggested Risk Indicators that can be applied where appropriate to the risks and Critical Variables for a particular protocol. Note to Speaker: For each example on the right side, highlight the potential risk that this Indicator would allow us to detect. For example the first one is outliers/trends in number of adverse events per subject visit/site. This Risk Indicator would allow you to detect sites that have a low number of AEs (compared to other sites); potential under-reporting would result in a risk to the overall safety profile of the product as well as an increase inspection risk. Alternatively, a site with a comparatively high number of AEs could indicate a risk to subject safety at that site or the site is reporting each individual symptom instead of the underlying event. Data Quality: for data queries - # of outstanding queries, # or queries > 30 days, variability of data blood pressure values are all the same across visits (e.g. 120/80 for 90% of values). Although BP may not be a critical data, it raises concerns of bigger issues at site. 62

67 1. The process of identifying Risk Indicators includes program and protocol review of the Critical Data and Processes for the associated risks identified in the RACT. 2. Then, team members identify the best monitoring approach(es) (central, off-site, onsite monitoring or combination) to detect a risk or signal of a potential problem. Where and how could you collect data or information that could indicate a risk? If technology permits continual monitoring, Risk Indicators are best monitored through either central or off-site monitoring. This allows for more timely identification of potential problems and issue resolution. Not every risk can be detected through central or off-site monitoring (e.g. specific processes such as informed consents or investigational product accountability); if information is not available through off-site or central monitoring, it may require on-site review at specific intervals or time points. 3. Based on the monitoring approach, specify the relevant data element or information that will best identify the Risk Indicator, e.g. number of subjects enrolled since last monitoring visit indicating under/over enrollment. 63

68 Returning to our roadmap from earlier in the module, we have now completed identification of the Risk Indicators. Now that we know how to detect risks, we are ready to ask the second question what range or value do we expect from the Risk Indicators? Setting Thresholds allows us to evaluate data and information to determine when there is a potential problem that requires action. 64

69 Take a minute to review the definition of a Threshold (from the TransCelerate position paper). Thresholds are set in order to clearly understand when the data/information from our Risk Indicators is inconsistent with the data we expect. 65

70 Different levels of Thresholds may be set for a specific Risk Indicator. This practice can direct the action taken to the level of perceived risk. The TransCelerate position paper recommends that a Risk Indicator system or tool be used to help display information that will help track the indicators, such as a color coded alert system on a dashboard. The system used should be able to provide the right level of information to the functional team member performing a monitoring activity. For example, a person with study-level responsibilities should be able to assess risk across all sites for a given protocol. The Risk Indicator system or tool used should be able to display information such that it allows comparison of subject-level data across a site, allows comparison of a site against its peer sites within a country or within a protocol, allows comparison of one country to another, allows comparison of data across protocols, and facilitates the detection of problems that require further investigation. Shown here is an example which involves three different levels (high, medium, and low) and shows how they can be conceptualized in a color coded alert system. A Risk Indicator in the high range reflects a warning and would require more immediate attention than an item coded as Medium or Low. The medium range provides more of an awareness of a potential problem. A Risk Indicator in the low range may be considered more acceptable in terms of risk and possibly require no action. 66

71 Returning to our roadmap from earlier in the module, we have now completed discussion of Thresholds. Now that we know how when action must be taken, we are ready to ask the final question what should the response be? 67

72 The Companion Guide to Risk Indicators from the TransCelerate Position Paper can be used to facilitate consistency among team members in applying Risk Indicators and Thresholds for decision-making and responding to risks. In the right hand column, there are examples of possible actions that could align to each threshold. For a given trial, the monitoring plan would have specific and consistent recommendations. Notice how the left column on the screen illustrates the application of three different levels of Thresholds to the specific Risk Indicator of Adverse Event collection /reporting. We will examine the right hand column a bit later when we review setting actions for Thresholds. 68

73 SLIDE 74 When a given Threshold is reached, a decision needs to be made regarding the appropriate action to take. The timing of our actions in response to Thresholds also often depends upon the Threshold level. Therefore, more immediate action may be necessary for a High Threshold level in comparison to a Medium level. At the Low or green level, often no additional actions beyond ongoing comprehensive monitoring will be needed. At the Medium or yellow Threshold level, the choice of action, depending on the issue, may simply be to continue Central or Off-site Monitoring for potential trends. Examples of additional actions for a Medium Threshold level include: Assess other types of data remotely for example, if a site s AE rate is lower then expected, one could review EDC off-site to identify any changes in subject concomitant medications. These could represent potential AEs. Contact the site to gather additional information for example, discuss how the site collects and assesses potential AEs and confirm the site has qualified individuals involved in AE management In response to a High or Red Threshold level, many of the same actions as identified above may be taken but in a more immediate timeframe. Additionally, the following actions may be taken: Visit the site to assess documentation that is only available on-site and cannot be made available remotely through electronic means for example, this may be necessary to review subject medical records to confirm there are no unreported or missed AEs. This is also an opportunity to have a discussion with the site about the trend and to determine the root cause that needs to be rectified. Collect site documentation for example, one could ask to review a copy of the site s SOP for AE management if one exists 69

74 In some situations, it may be useful for an action plan to make use of a decision tree. This would guide team members through the various steps of investigation when a particular Threshold is reached. A sample of this approach is provided here. Note to Speaker: Walk through the decision tree with participants. Squares/rectangles represent action plans and diamonds represent decision points. 70

75 Our objective in this section is to describe risk mitigation plans and how they are implemented throughout the conduct of the clinical trial. 71

76 Risk Mitigation is an element of risk management and is simply defined as a plan which assigns responsibility and defines the actions taken to prevent or decrease the probability of a risk becoming an issue. In the RBM methodology, completion of the RACT identifies those risks which require mitigation and the functional plans associated with each risk. The responsible functional plans are then incorporated into the Integrated Quality Risk Management Plan or IQRMP The IQRMP and its associated risk mitigation plans define the actions that each function will take to proactively identify, assess, and manage risk throughout the life of the clinical trial. 72

77 Here the IQRMP is graphically depicted. Notice that the functional plans are all aligned around the risk assessment, Critical data, and Risk Indicators and their Thresholds. It should be noted that these key elements may not necessarily be separate physical plans but instead some may be combined. The procedures and activities described within the IQRMP should not duplicate instructions contained in Standard Operating Procedures. The IQRMP should describe the trial-specific actions/processes that will be implemented to address identified risks and focus on Critical Data/Processes. The IQRMP is not intended to duplicate the content of existing functional plans; these are linked or referenced within the IQRMP and accountability for each plan remains with the relevant function. However some risks that are identified in functional plans may need to be added to the IQRMP for comprehensive monitoring. The overall accountability for the development and maintenance of the IQRMP should be assigned to a centralized function such as project or program management to ensure that the key elements are aligned across all functional areas. 73

78 Note to Speaker: Use this slide content as guide for facilitating a discussion about how a Data Management Plan represents risk mitigation. Traditionally, Data Management Plans describe the procedures and responsible personnel involved in data collection, data review, and data query and resolution. So, how do these plans fit into our concept of risk mitigation? Data collection: Timely data collection and data entry is a critical factor in successfully implementing the RBM methodology to allow for real-time review of potential risks. So, data management teams may mitigate this risk by setting Risk Indicators, Thresholds, and action plans related to timely data collection/entry by sites. Data review: Expectations may be set for timely and ongoing (preferably continuous) data review cycles to ensure risks are detected in a timely manner. Also, the use of programmed edit checks for Critical Data or to assess Risk Indicators can be a component of risk mitigation. Data query and resolution: Reviews may occur to determine if certain data points have a higher than average query rate. If so, the risk may be mitigated by the following: 1. Process to validate the query is firing correctly 2. Determine impact of the query is it resulting in delays in ability to review Critical Data? 3. Determine extent of query problem all sites across the study or only certain ones? 4. Review ecrf completion instructions to determine if appropriate guidance was given to sites. Risk mitigation can also be managed through other data systems such as entering data from a lab 74

79 directly into an IVRS system to avoid transcription error or setting lab alerts for values that meet a specified criteria. 74

80 Note to Speaker: Use this slide content as guide for facilitating a discussion about how components of the MP represent risk mitigation. The trial-specific Monitoring Plan is a plan within the IQRMP, typically created by the clinical monitoring or clinical operations group. The MP should guide Monitors beginning after site activation until close-out. What do we mean by baseline monitoring approach? This is an aspect of the MP which defines the level of monitoring across sites and the processes for monitoring (i.e., central, off-site, and/or on-site) that reflect the appropriate level of monitoring in response to the identified risks (Critical Data, Critical Processes, and the RACT). How does detailing the monitoring activities conducted centrally, off-site, and on-site reflect risk mitigation? It assigns responsibility (who will do what) and defines the process (how will they do it) by which we will detect risks, measure their impact, and take actions to minimize their impact. Finally, how and why would changes in our baseline monitoring approach or activities be required? For example, if the Overall Risk Level of the study decreases at a certain stage, such as moving from active enrollment/treatment stage to subject follow-up stage, the extent of activities such as Source Data Review (SDR) and/or Source Data Verification (SDV) may change. The monitoring approach and/or activities may also 75

81 change if Thresholds are reached in various Risk Indicators or if other compliance/quality issues are identified at the site or study level. 75

82 Note to Speaker: Use this slide content as guide for facilitating a discussion about how the Safety Plan relates to risk mitigation. Safety or pharmacovigilance plans describe how pharmacovigilance/drug safety functional areas will manage safety risks related to a product. These activities involve ongoing review of emerging safety data from all studies being conducted for the product. Additionally, drug safety personnel ensure that investigators, regulatory authorities, and potentially ethics committees are updated as required when new safety information becomes available. So, how are safety plans critical for mitigating risks that arise from safety risks related to the product? Safety plans should establish the processes and assign responsibility for ongoing review of all safety data coming in from clinical trials being conducted. Reports of serious adverse events (SAEs), for example, should receive expedited review by safety personnel as they represent a higher level of risk to subject safety. Communication may need to be expedited for certain adverse events based upon their rate of occurrence. The incidence and severity of adverse events is evaluated against the existing product information allowing the identification of an increase in the expected rate that requires further investigation. For example, in a dry eye study, if ocular hypertension is occurring in 40% of study subjects but was only seen at a rate of 20% in previous studies, the safety plan would detail how this safety signal will be further evaluated. 76

83 For significant safety data, the safety plan may possibly establish a Threshold at which the study would need to be stopped. This may become part of the charter for a Data Safety Monitoring Committee where appropriate. 76

84 77

85 78

86 79

87 80

88 Now let s drill down a little farther and take a look at site level risk assessment 81

89 At this stage a lot of work has been done to identify risks and mitigation plans at the program and protocol level. Before we dig into the actual conduct of a trial and the management of the risks let s take a brief look at site level risks. Potential risks may be identified at any point either prior to or during the conduct of the study. Risks can be defined as issues based on their evaluation. During the site evaluation, the process a site follows for issue management is assessed and expected to be followed during study execution. Documentation of issue management follows the established site process. 82

90 Some considerations or questions that may be considered when identifying site level risks are: (Note to speaker: review questions on slide) 83

91 Whether or not a risk is really a risk depends on a number of factors. When considering site level risks think in terms of the people involved and the infrastructure in place. The likelihood and impact of site posing risks may vary depending on what they are asked to do. These are some questions that may be asked when evaluating site-level risks; note that they are already evaluated in our current site selection process, so there is no real process change at this point. Furthermore, risks that may be present at a country or region level may also need to be considered. For example, standard of care practices should be evaluated at the site level. When do we assess risk site feasibility? Who assesses site level risk? Other roles may be involved as needed, depending on the potential risks identified. For example, if you have questions about the patient population or the qualification of a potential investigator, you may contact your Local Medical Monitor or Disease Area Heads may be the source of valuable information from the clinical perspective and their advices may help in the decision making process. Can you think of other risk indicators that may be present at the site level? 84

92 A key component of managing risk is the documentation. If it isn t written, it was not done. Documentation expectations for sites are described in.. Specific risks and related mitigation plans can be found in When risks occur and require mitigation or resolution, this can be documented in 85

93 Risk Mitigation is an element of risk management and is simply defined as a plan which assigns responsibility and defines the actions taken to prevent or decrease the probability of a risk becoming an issue. Not all risks require mitigation. Lets talk about planning for risk mitigation. When risks require mitigation: When you do decide to mitigate a risk, what other site-level risk mitigation factors should you consider? 1. You would want to review your SOPs for compliance. 2. Is there a guidance document on specific actions related to the risk you are addressing? 3. Who are the people (what roles) that should be involved at your site? Then you will need to propose/develop the best strategy to suit the risk. For example the plan may include: Additional Training for your staff - this could be via a Teleconference with study team especially before you enroll your first subject study procedures or inclusion / exclusion critieria are complicated or new to your staff. If risk levels are high for your site and mitigation strategies have been unsuccessful, you may wish to reconsider if this study is right for your site The SMP brings crucial information for SMNs; among them, it describes the monitoring activities conducted centrally, off-site, and on-site. How does it reflect risk mitigation? It assigns responsibility (who will do what) and defines the process (how will they do it) by which we will detect risks, measure their impact, and take actions to minimize their impact. Each site level risk is unique, as it is related to the site s particular reality; therefore, there is no one size fits all strategy to mitigate them. But by keeping this concept in mind and understanding the tools and resources you have available, you will be able to propose adequate mitigation plans to meet the specific needs of your site. This slide brings some suggestions on how to apply this concept for the development and documentation of adequate mitigation strategies. 86

94 Our next objective in this section is to describe on-site, off-site, and central monitoring activities used in study oversight. The appropriate use of each of these monitoring approaches is fundamental to the TransCelerate RBM methodology. Before detailing each of these, let s review some of the key assumptions underlying the application of the RBM methodology as detailed in the position paper. 87

95 The concept of comprehensive monitoring within the TransCelerate RBM methodology involves 4 key ideas: The baseline monitoring activities, such as sampling strategy and monitoring frequency, are defined by the program and study-specific risks Central and/or off-site monitoring activities provide a foundation to ensure timely and adequate monitoring On-site monitoring activities are triggered by Risk Indicators and Thresholds Monitoring involves everyone, not just Clinical Research Associates/Monitors Central Monitoring Activities: Comparisons of Risk Indicator data and information (across studies, between investigative sites, etc.) may include: Protocol deviation rates Data entry and query resolution metrics Adverse event trends or outliers Subject discontinuation trends 88

96 Unusual data trends or patterns Error rates in Critical Data/Processes Offsite Monitoring Activities Confirm timeliness and quality of data entry Review query resolution Review CRF to check protocol compliance Confirm site s completion of previously identified actions Assess site s recruitment and enrollment Monitor for changes in site staff Monitor delegation of responsibilities Conduct training Onsite Monitoring Activities Informed Consent Review Eligibility verification Subject safety review Investigational Product Accountability Essential Documents Review (if appropriate) Face-to-face training and discussions with site staff 88

97 As technology has evolved we have been enabled to conduct less on-site monitoring and focus more on centralized and off-site monitoring techniques. Central monitoring involves a review of centralized data not just reviewing data from a central location. Off-site monitoring (sometimes called remote monitoring) is an evaluation carried out by sponsor personnel or representatives at a location other than the investigative site. Both of these techniques may be used to check that data is consistent and complete, identify unusual distribution of data, identify higher risk sites to target additional monitoring, and to ensure routine review of data is completed in real time. In other words, we can do a significant amount to be proactive in addressing issues before ever going on site and identifying study risk factors and potential indications of risk. Ensuring that data is consistent and complete and identification of unusual distributions of data can be realized through analytics and visualization of data across the study, across regions, across a site and across a patient. Also, in order to be able to ensure that data is consistent and completed, there will need to be emphasis on the ability to integrate data from disparate sources. Central monitoring may be carried out by the same individual that would conduct on site monitoring such as a Clinical Research Associate or Clinical Monitor, or by other functional roles such as a data manager or statistician. 89

98 According to data from the Clinical Trials Transformation Initiative, even though many sponsors have access to centralized data, 33% or fewer sponsors use centralized data monitoring to guide, target, or replace site visits. So this leads us to considering further changes to our monitoring approach to make better use of technology and our resources. additional NOTE: Source Data Review and Source Document Verification remain on site activities. Source Documents remain on site. TransCelerate RBM does not involve mass copying/faxing source documents for remote SDV 89

99 These are examples and not an all inclusive list. Activities performed during Off-site Monitoring may be identified depending on the risk assessment for the study. Key message: everything that CAN be done remotely SHOULD be done remotely since it can be done in a more timely manner and issues can be identified sooner. Information obtained from other monitoring activities (on-site and/or Central) should be utilized to ensure that off-site reviews and follow-up are focused on identified risks and issues. 90

100 Read through each example of what off-site monitoring is not. 91

101 These are examples and not an all inclusive list. Activities performed during on-site monitoring may consist of a mix of the ones listed on the slide depending on the risk assessment for the study. Some activities may be required at every visit, and others may be required more infrequently. Information obtained from Off-site Monitoring review and Central monitoring should be utilized to ensure that necessary on-site follow-up is focused on identified risks and issues. Consent Forms - Consent forms may not require 100% review but the size of sample should be based on risk and on the types of issues identified off-site or on-site. For example, if an unauthorized person has obtained consent, the consent review sample may be increased. Investigational Product Accountability and reconciliation activities performed on-site include verifying protection of the blind, ensuring that correct subject assignments have been made against treatment assignments, ensuring product use dates are suitable and that IP logs are up-to-date. The RBM methodology does not recommend verification of pill counts for subject-level accountability. Essential Documents On-site review should be based on issues and risks identified for the site. Unless part of issue and risk management, there should not be a requirement to perform a detailed on-site regulatory file review nor on-site reconciliation with the TMF. Review of Essential Documents can occur remotely. Periodically, the Monitor can conduct a cursory evaluation of the site file for general appearance while on-site to ensure there are no obvious issues. Site Interactions: Face-to-face training and discussions with site staff will occur as needed to support changes in staff, issue resolution, and Risk Indicators. SDR and SDV these represent two distinct on-site monitoring activities in the TransCelerate RBM 92

102 model and will be described in more detail on the next two slides. 92

103 Commonly known as transcription checking, SDV is the process by which data within the site s original source documentation are compared to data within the CRF (or other data collection systems) (and vice versa). SDV involves 1:1 verification. It s purpose is to confirm that the data were transcribed accurately (i.e. data from source matches data in the CRF or other system and vice versa). SDV is performed on Critical Data only and decisions about the amount of SDV are based upon the risk assessment process and outcomes. 93

104 SDR mainly involves reviewing key pieces of source documents for areas where there is no associated CRF data field to ensure they: Are compliant with good documentation practice (ALCOA criteria) Clearly documents clinical data Confirm compliance to protocol and show PI oversight provides a means of reviewing the processes by which the data were collected and documented - including protocol compliance, PK collection, ICF process, investigator involvement and delegation to qualified staff, etc. This can provide additional assurance of data integrity. is sometimes performed by reviewing source documents and other records in their entirety like reading a book that tells the story of the subject s participation in the study. review amount depends on several factors including sampling approach, criticality of the data point and the adaptive approach during the course of a trial that may increase or decrease review levels depending on site performance Let s consider a few examples of SDR. Let s assume that the CRF only includes the lab test name and the result (e.g. blood glucose value of 105 mg/dl). Whereas SDV is verifying that the blood glucose value shown on the local lab report is accurately entered into the CRF (i.e. value of 105 in the CRF matches the value in the source and vice versa), SDR might involve reviewing the source to check that the patient was fasting for the glucose test (if fasting was required per protocol). Let s think about another example involving weight measurement. Whereas SDV is verifying the source weight measurement is accurately entered into the CRF (i.e. CRF matches the source and vice versa), SDR might involve checking how the weight was obtained (e.g. with shoes on or off, using the same scale across visits) to determine if the protocol requirements for obtaining the weight were followed. 94

105 TransCelerate draws the distinction between SDV and SDR to allow for prioritization of the high value task of compliance checking and the low value task of checking for transcription errors. Transcription errors identified by SDV are typically infrequent, insignificant and do not lead to study data being unusable as supported by the SDV manuscript published by TransCelerate. Issues with compliance i.e.. protocol violations are one of the reasons for study data not being used in the final efficacy analysis. If there is inadequate investigator involvement increasing SDV will not fix this. If Investigational Product is not stored properly SDV will not be effective in determining if the product was fit for use. Instead, issues of these type will be more readily identified through SDR. So, the two activities address different risks in study conduct and can help sponsors get answers to different questions about site performance. Therefore, different levels of SDV and SDR can be used according to the study-specific needs to address risks. 95

106 4/23/2015 TransCelerate has carried out a retrospective analysis, including 1168 phase I-IV biopharmaceutical studies across 53 sponsors. The relative contribution of SDV to the total amount of data corrections observed across all submitted ecrf data was assessed. Only 1% of the total ecrf data corrections were attributable to SDV. The second key focus of the analysis was the importance of SDV in detecting subject events (AEs and SAEs) that were not initially recorded in the ecrf. Any AEs or SAEs that were recorded within 1 day or within 7 days following on-site monitoring activity were considered to have a temporal association with the SDV. For AEs, the overall industry median values were 7.5% reported within 1 day from the on-site monitoring and 11.8% within 7 days. For SAEs, the overall industry median values are low: 1.7% reported within 1 day and 3.6% within 7 days. The conclusions of this analysis were that overall percentages of SDV can be safely reduced or eliminated based on the % of ecrf data corrections attributable. The impact of on-site monitoring and the use of SDV versus SDR should consider the 96

107 type of data being monitored visit data versus subject event data. SDV should be set at a level to ensure that minimum transcription expectations for data accuracy have been met (visit data). Evidence supports review of source using SDR to detect missing events (event data). The retrospective analysis described was analysed together with a literature review and audit data review. All three together overwhelmingly support shifting monitoring resources to focus on SDR (to monitor the protocol, not just the data), and to employ central monitoring (to trigger the need for SDV and / or SDR to assess low event reporting at sites). A manuscript Challenging the Value of SDV will soon be available (due for publication in November) if you wish to read the full review. Overview presentation is on the TransCelerate site. 96

108 This slide shows the further breakdown of the data that was corrected. 96.3% of the data was never changed. Of the 3.7% of the data that was changed, approximately 1/3 was a result of system checks (validations), one third was a result of other queries (such as generated from listing review and other cleaning methods) and 1/3 was due to SDV. 97

109 Note to Speaker- a formal answer key also appears on the next slide ANSWER: True The methodology improves efficiency through the application of continuous monitoring activities to measure, evaluate, and assess the proactively identified risks for each study (and potentially each site). This shifts the focus to proactively mitigating risk and early detection of issues. 98

110 ANSWER: True The methodology improves efficiency through the application of continuous monitoring activities to measure, evaluate, and assess the proactively identified risks for each study (and potentially each site). This shifts the focus to proactively mitigating risk and early detection of issues. 99

111 What RBM brings to the table is realization of overlapping responsibilities with the addition of central monitoring(timely, regular, across whole study, utilizing risk indicators and thresholds upfront, flexible based on ongoing findings to target action etc) 100

112 4/23/2015 A cross-functional initiative that will vary among sponsors. 101

113 Our final topic for this module is to summarize our responsibility to appropriately respond to potential issues and risks identified or encountered throughout the study. 102

114 A four step process is commonly used in site and study management for responding to issues or risks occurring in clinical trials. Let s briefly review these steps. 1. First, potential issues may be identified through central monitoring by reviewing the risk indicators. The next slide will discuss this process in more detail. 2. The second step involves what is often called root cause analysis, which means thoroughly investigating the issue/risk to clearly identify the underlying cause of the problem. While not specifically covered in the TransCelerate position paper, the concept of root cause analysis is supported by regulatory authorities and can be considered part of quality risk management. Also, many of the actions we design for responding to Thresholds help to move us toward understanding the root cause(s). 3. The third step requires that appropriate solution(s) to the identified root causes be applied at the site, study, country, and/or program level. These solutions could include both immediate corrective actions and more long-term preventive actions. 4. Finally, we should ensure that we evaluate the effectiveness of the solutions in preventing recurrence of the problems This can often be accomplished through ongoing central and/or off-site monitoring. 103

115 Let s take a closer look at the identification of issues. On a predefined frequency, the first step is that the central monitor will review the risk indicators through the generation of reports or a visualization tool. As previously mentioned, technology may differ between sponsors. If a risk indicator exceeds a threshold, the central monitor may generate additional reports to dig deeper to assess the trends and outliers. They may run a listing or assess other data. This will provide information to determine if the data represents a true outlier. Issues will be escalated to the appropriate study role, e.g. study manager, site manager. The issue will be escalated as appropriate depending on the severity. If a risks indicator indicates a severe issue, it will be escalated to the appropriately role immediately (e.g. safety issue). The next few slides will provide an example of this process. 104

116 Let s take another look at the previous group challenge involving IP with a known association for elevated liver enzymes and its Risk Indicator (as shown on the slide). Let s use this scenario to illustrate how one would respond to issues/risks identified for this Critical Process. Note to Speaker: Ask a participant to read the study description reminder and Risk Indicator (or read these aloud to the participants). Additional information and questions for the participants are provided on the next slide. 105

117 So, we previously determined that we would monitoring through a central approach, using reports provided by the central laboratory vendor. We have analyzed the central lab data as follows [read bullet points] Note to Speaker: Facilitate the discussion of potential issues/risks, guiding participants to determine if the data indicates only site-level risks, only study-level risks, or both. Focus only on identifying the risks at this point, not on root cause(s) and/or solutions. We will approach those after we ve identified the issues. Answers appear below and on the next slide. High overall average rate of missing required re-tests 6.7 missed out of 10.3 incidence 6 sites have higher than average missing required re-tests 106

118 Depending on the team structure, the central monitor may escalate to the study team to determine the underlying cause. So, two potential issues or risks include a high overall rate for the study of missing required re-tests. On average, each site has 10.3 ALT/AST results that would require retesting. On average, 6.7 missed re-tests occur per site. So, this can be thought of as a study-level issue/risk. Then, we have a potential site-level issue/risk with the 6 outlier sites that have a higher than average number of missing required re-tests. Our next step is to think about and identify possible root cause(s) of the issues we ve identified. Note to Speaker: Facilitate discussion to identify the possible root causes of the potential issues/risks identified on the slide. Participants should clearly identify the potential root causes in order to identify possible solutions. Answers are provided below and detailed in the speaker notes of the next slide. Unclear expectations Reporting issues Subject compliance Training gap 107

119 Some possible root causes for the issues we ve identified are provided here. Let s look at them more closely. Perhaps the protocol or other communication with sites has not provided clear expectations for the required re-testing. Maybe the protocol doesn t specify how quickly the re-test should be performed after the elevated ALT/AST results are received. There could be reporting issues with the central lab vendor. For example, maybe it is unclear form the lab report that the ALT/AST results are elevated (if, for example, the normal ranges aren t printed on the lab report). Perhaps sites are struggling with getting subjects to comply with the re-testing requirements. Returning to the clinic for re-testing may be extremely difficult for subjects, for example due to work schedules, if the subject population is physically disabled, or if transportation to the clinic is problematic. Finally, there may truly be a training gap in which investigative site personnel were not trained properly on the re-testing requirements for the ALT/AST elevations. Note to Speaker: After discussing the possible root causes, facilitate discussion to identify possible solutions to address the potential issues/risks. Answers are provided below and detailed in the speaker notes of the next slide. 108

120 Some possible solutions are provided here. To address the potential lack of clarity in protocol expectations, a re-testing decision tree tool could be developed and provided to all sites. Review and training on the use of this tool could take place remotely through teleconference. To address the potential laboratory reporting root cause, a re-test flag could be incorporated into the laboratory report that would clearly identify for investigators those ALT/AST results that require re-testing. To address the potential subject compliance factor, perhaps a subject travel stipend could be incorporated to off-set transportation costs. This would, of course, need to be reviewed and approved by the institutional review board/ethics committee and incorporated through an informed consent revision. Finally, to address potential training gaps, training materials could be developed to be presented remotely to sites through teleconference. 109

121 The final step in our process flow is to evaluate the effectiveness of the solutions we implement. For our group challenge example, this would involve continuing to monitor the Risk Indicator for the study overall with a goal of reducing the average number of missing required re-tests/site. We would also hope to see a reduction in the average number of missing required re-tests for the 6 outlier sites. For the 6 outlier sites, the central monitor may look across all of the risk indicators for those sites to see if there are any thresholds for those sites that have been exceeded. The information would be escalated to the site monitor for further assessment and follow-up with the site. 110

122 Now let s take a look at how off-site monitoring participates to evaluate a potential issue and determine what action is needed. An off site assessment is conducted by Site Monitors. The first step is the gathering of available information from the central monitor or other sources (e.g. enrollment, a major protocol amendment, new study personnel). The next step is to confirm whether the potential issue is real. Once determined, the site monitor will document any intervention and then escalate as needed to appropriate study and line management personnel. Let s take a look at an example of off-site monitoring activity 111

123 Read the slide 112

124 Read the slide. Ask the participants to identify possible causes of the higher SAE rate? 113

125 One cause might be there is a safety issue. The site may not be dispensing drug correctly or it could be that they are reporting all of the symptoms of an adverse event separately and not the underlying cause of the event. The next step is to determine the potential root cause, what would you do to investigate? Allow participants time to respond. 114

126 To investigate further you could run a listing of AEs for the site to take a closer (possibly already mentioned). As you look further you see that there are many symptoms of a URI reported on two dates August 6 and September 13. You hypothesis is that it is a training issue and decide you should call the site. 115

127 READ the slide So the monitor concludes it was a training issue and provided the appropriate training. However (go to next slide) 116

128 Some possible root causes for the issues we ve identified are provided here. Let s look at them more closely. Note to Speaker: After discussing the possible root causes, facilitate discussion to identify possible solutions to address the potential issues/risks. Answers are provided below and detailed in the speaker notes of the next slide. 117

129 READ the slide 118

130 Before we identify what actions to take let s take a minute to discuss when on-site visit should be performed. Mandatory on-site monitoring visits are determined in the monitoring. On-site visits are conducted by Site Monitors/CRAs. Additional on-sites are determined by the identification of potential issues through central/off-site monitoring. They should be performed to confirm if an actual issue occurred and identify the root cause if the activity cannot be performed off-site. The monitor should perform the appropriate intervention (e.g. SDR/SDV of source documents) and communicate, escalate and document as required. 119

131 Now let s return to our scenario You determine that an on-site visit is needed to review the source and identify and possible unreported events. This cannot be performed off-site. It is an urgent situation because unreported SAEs could impact the safety profile of the compound. At the site you review the source documents for unreported AEs/SAEs. You provide training to the coordinator and refer her to appropriate GCP training. You schedule time with the investigator to discuss his oversight of the study including his involvement with the subjects, the requirements for reporting of SAEs and his responsibility to insure appropriate staff training. You escalate to the study manager, medical monitor and your line manager to determine additional action to be taken. 120

132 The final step in our process flow is to evaluate the effectiveness of the solutions we implement. 121

133 It is really important to identify the appropriate action to take. As a site monitor working on an RBM protocol, you will first need to review the monitoring plan to determine if you need to go to the site or if the potential issue can be resolved off-site, e.g. with training. Examples of other needs that cannot be adequately managed offsite include: increasing the level of SDR or SDV, scheduling more visits to the Pharmacy Department, organizing a training that focuses on a specific topic, etc. 122

134 The Integrated Quality Risk Management Plan (IQRMP), recommended in the TransCelerate RBM methodology, includes these two potential elements that may be used to communicate, track, and monitor issues and risks throughout the study. Note to Speaker these are provided as optional for discussion. These may or may not be used by your project teams and/or organizations. 123

135 In summarizing this module, we have covered several key practices and concepts related to the TransCelerate RBM methodology. We reviewed the use of Risk Indicators and Thresholds for decision-making in clinical trials. We discussed the implementation of risk mitigation plans for minimizing the impact of potential risks in our trials. We described the monitoring activities conducted in the RBM model including clarification of SDV and SDR activities, establishing a baseline monitoring approach, and modifying monitoring approaches and activities. Finally, we reviewed how potential risks and issues can be managed throughout the study by applying appropriate response plans and actions. 124

136 Module 4 will focus on the application and considerations of RBM plan implementation. In this section we will address a practical approach to implementation and management, as well as how to transition projects, protocols and sites into the RBM model. 125

137 In this module, we will be discussing various aspects related to the implementation of the RBM methodology. By the end of this module, learners will be able to provide investigative sites with the information they need to know about the RBM model and its impact on their activities. Finally, participants will be able to discuss challenges with RBM implementation across different cultures and systems and the metrics available to help measure the impact of RBM. 126

138 In this final module, let s discuss the possible challenges that may be met when implementing Risk-Based Monitoring. 127

139 The challenges encountered in implementing the RBM methodology exist in three general areas: Global or Regional challenges encountered when working with global teams and/or investigative sites or challenges that exist due to differences in specific regions or countries Challenges arising within our own companies or organizations which may be related to Individual personnel who are resistant to change Implementation of the RBM model across different functional areas within the company Challenges encountered due to specific characteristics of investigative sites or institutions Let s look at each of these individually. 128

140 Whenever we encounter a change in the industry or with an internal process, there may be challenges and possibly resistance. Challenges with any change can be based in the culture of an organization/company or from cultural differences in the areas where we conduct our trials. Other challenges can arise from perceived limitations of the systems we use. Individuals may defend the current process or raise an objection to the new methodology by saying it will require new systems or technology. Example 1: Different country cultures pose a significant challenge for implementing RBM such as sites finding face-to-face discussions more respectful. Example 2: Study team agrees with RBM concepts/principles but do not think it can be implemented or do not want it to be applied to their study. You may hear statements like That s not the way we do things here. We ve always done it this way. The system won t work that way. What if this leads to an inspection finding? The best way to meet challenges with any new idea or methodology is to be prepared and to understand the reasons or motivation behind the challenges. Some of the reasons for resistance are stated on the slide - it may be that some individuals understand the upcoming change but don t agree with it; there may be some fear of the unknown; or team members may even be concerned about the change generating more work for everyone. 129

141 In the global arena, we may face challenges as to whether the regulatory authority(ies) and ethics committees in the areas where we work are in support of the concept of applying risk-based approaches to monitoring. This uncertainty could result in global team members and/or investigative sites resisting the implementation of RBM. There are language challenges that must be managed. The TransCelerate RBM methodology includes specific terminology, principles, and concepts that will need to be translated correctly into local languages to ensure full comprehension and success. Challenges may arise due to a variety of cultural differences (e.g. roles and responsibilities or different staffing/personnel models). 130

142 Do our SOPs and work processes need to be revised? There may be challenges if they are complex or easily adaptable. Do we have sufficient resources internally? Will job descriptions need to be revised? Clarification in roles and responsibilities is critical. Does our culture support change? What resources are available for change management? Are our technology and systems (including those of outside vendors like Data Management, Interactive Voice Response Systems (IVRS), Interactive Response Technology (IRT), electronic patient reported outcomes (epro), central labs, etc.) capable of enabling RBM? Do all of our strategic partners view RBM the same? What are the partner s RBM capabilities and experience? 131

143 The investigators and institutions with whom we work may have various limitations that can pose challenges to RBM implementation. For example, Institutional SOPs may be complex and/or not readily changed. Therefore, it might be difficult for them to implement internal quality assurance or issues management (root cause analysis, corrective and preventive action plans) policies and procedures. There may be problems obtaining sufficient personnel to meet expectations for timely data entry, query responsiveness, quality assurance, etc. As mentioned previously, technological resources must be available and personnel must be comfortable with and trained on its use. Contracts and/or budgets may need to be revised in the RBM methodology such as to include data entry expectations, identification of how payments are triggered if not using Source Data Verification any longer, sites may request compensation for internal quality assurance, etc. This process can lead to time delays and/or budgetary challenges. Note to Speaker: use the question at the bottom of the slide to solicit other ideas from participants in regards to investigator/institutional challenges. 132

144 4/23/2015 Early identification of issues, trends and outliers On time data entry will allow central monitoring and this should help alleviate query backlog it should help identify issues early and prevent repeated errors. Patient safety and data integrity maintained Patient safety and data integrity must be maintained but with the addition of central monitoring the timeliness may actually improve. Site ownership of data quality Sites have expressed concern that they are being abandoned really they are being embraced. We are applying additional monitoring techniques to ensure patient safety and data integrity. Some sites have referred to RBM as remote-based monitoring. They recall times when they were asked to fax in CRFs this is not that. Sites have always been responsible for data quality, this doesn't change with RBM. Sites should utilize an internal quality control mechanism to ensure that they are inspection ready at any time. Timely CRF data entry will enable sponsors to monitor data holistically. A variety of monitoring techniques We will utilize technology digitization of the data to make monitoring in a variety of ways possible. The need for on-site monitoring is not going away we are utilizing time at the sites to focus on the processes and adherence to the protocol and ICH/GCP. A combination of on-site, off-site and central techniques will be applied to ensure 133

145 maximum coverage of monitoring. 133

146 4/23/2015 During Q3 & 4, 2014, TransCelerate partnered with the Society of Clinical Research Sites (SCRS) organization, to conduct Site Advocacy Group (SAG) sessions to have open dialogue, exchange ideas, and work together to gain information from the site perspective that will contribute to continued development of RBM within our industry. These slides were presented to the SAG participants live at the SCRS Summit in October 2014 to confirm what we heard during the SAG sessions. The icons (fact or opinion) indicate the TransCelerate response to what was heard from sites on their perspectives and perceptions of RBM. What was heard: (read each item: then explain that this is true, e.g. fact OR this was a site opinion that TransCelerate explained is not the reality). One of the site s question was: Can sites still use their own Risk Assessment methods, or do they need to abandon and use the TransCelerate RACT? Sites were encouraged to continue utilizing their own methods as the TransCelerate RACT is focused on protocol risks from a sponsor perspective. Additionally, sites want RBM training at PI meetings or at site initiation meetings, just minutes, to help them understand the RBM process. 134

147 4/23/2015 Sites felt that RBM reduces on-site monitoring time, so they think overall quality will decrease; they think sponsors will conduct less monitoring. Some sites lean on CRAs for their QC checks, but they were reminded that both sponsors and sites hold joint responsibility for regulatory compliance and subject safety/data integrity. Feedback was received that sponsors perform Risk Assessments in silos and don t share information w/sites before or during study. Sites shared that when dealing w/cros, it s very hard to get direct answers from sponsor. They want single or dual communication w/sponsors. Regarding technology: sites are strained dealing with multiple providers and multiple portals, they want a single study portal that can access all the different systems. TransCelerate s Shared Investigator Platform (SIP) initiative could help to address some of these concerns. 135

148 As sponsors shift to a RBM model, there are definitely some potential benefits for investigative sites. First of all, subject safety is maintained and possibly enhanced through RBM because key safety data are being reviewed in a more real-time manner. Secondly, problems are solved before they recur in the study through early identification of issues and near real-time data query resolution. RBM should prevent the problem of data queries being issued months or years after the data was generated. With less frequent, regular on-site monitoring visits, site personnel may be freed up to focus on their core job functions. Investigative sites are inspection-ready as a result of the enhanced expectations for the completion of corrective and preventive actions when issues arise. Finally, sites may see increased quality as RBM expectations promote increased Investigator involvement and internal quality assurance activities. 136

149 Note to speaker: Answer key is on the next slide if you would like to show it and provided below. Answers are B and D. 137

150 Answers are B and D. Sites have always been responsible for data quality, this doesn't change with RBM. Sites should utilize an internal quality control mechanism to ensure that they are inspection ready at any time. The use of techniques such analytics will better enable the detection of issues before they become systemic A is not correct because there is generally an expectation of some level of on-site monitoring for most trials. For example, at the beginning of the study, an important early activity is to ensure Critical Processes are being followed and subjects are being properly enrolled. This typically requires on-site monitoring. C is not correct because a routine of scheduled daily teleconferences to discuss issues is not aligned with the RBM concept of monitoring resources and activities aligning with risks. 138

151 Now that we have concluded our comprehensive discussion of the implementation of the Risk-Based Monitoring methodology, we d like to give you the opportunity to reflect on specific challenges that might affect your role in the organization. Try to identify any possible tools, systems, or strategies that currently exist or could be developed to manage challenges. Note to Speaker: This provides an opportunity for each participant to spend a few minutes thinking about how their individual role may be affected or challenged by RBM implementation. Participants should be encouraged to identify any possible tools, systems, or strategies that currently exist or could be developed to manage these challenges. A suggested list of solutions appears on the next slide. 139

152 A suggested list of solutions appears on this slide. Of course, each organization will define and develop its own tools, systems, and strategies to streamline and facilitate the implementation of the RBM model. Develop or revise existing tools or worksheets to adapt to RBM methodology for example, add Risk Indicators/Thresholds on existing tools; redesign SDV tracker for Clinical Research Associate use Establish a communication plan or methodology for sharing monitoring activity across functional areas for example, a calendar to display frequency of reports pulled/reviewed across various functional plans or areas Enhance existing dashboard or reporting systems to provide color-coded visuals for Thresholds Consider developing RBM-specific training for sites in regards to developing their own internal quality assurance functions/activities Develop Advanced Electronic Data Capture (EDC) training to cover topics such as trending review and generating reports and listings Provide RBM-specific training for Clinical Research Associates at study kick-off meetings with study-specific scenarios to practice application of monitoring plan content, Risk Indicators and Thresholds, and responding to issues 140

153 Technology enables comprehensive measuring of the monitoring process and oversight activities. Metrics can be developed to measure the effectiveness and efficiency of the new RBM process. 141

154 So what are metrics? BusinessDictionary.com defines metrics [read definition]. Another way to think about a metric is that it is any type of measurement used to evaluate some quantifiable component of a company's performance. A valuable metric should allow the company to make better business decisions and allow comparison, for example, to the metric obtained from a different approach or from an industry benchmark. Reference: 142

155 The TransCelerate position paper on the RBM methodology defines three dimensions that should be evaluated in respect to the effectiveness of implementing the RBM model: quality, timeliness/cycle time (of data collection and issue resolution), and efficiency of trial operations. It is critical to know ahead of time whether we plan to compare our RBM metrics to historical data or simultaneously to an ongoing control program or study to measure efficiency, quality and cycle time changes. These dimensions should not be evaluated in isolation or used individually to draw conclusions. They should be used collectively to make business decisions about RBM. Sponsors may choose to use and evaluate metrics on an ongoing basis during the study or after the closure of a study for a higher level of evaluation at the program level. 143

156 Can you identify at least one quality metric to evaluate the effectiveness of the RBM methodology on clinical trial operations? Let s start by just looking at the Quality dimension. Facilitate discussion among participants to identify metrics within the Quality dimension. In the Quality dimension, the TransCelerate position paper suggests these possible metrics: Number and classification of major/critical audit/inspections findings per audited site Rationale: Improved focus on what matters should benefit results of QA process Number of significant protocol deviations per site Rationale: Indicates impact on safety or the ability to evaluate subjects as well as reflects quality of protocol design Number of unreported, confirmed SAEs as discovered through any method Rationale: Ensures focus on safety Improvements or reductions in these measurements, as compared to a study that was conducted through a non-rbm model, would indicate the RBM methodology positively impacted the level of Quality in the trial. 144

157 Let s see what possible metrics you can identify next in the Timeliness/Cycle Time dimension. In the Timeliness/Cycle Time dimension, the TransCelerate position paper suggests these possible metrics: Average number of days from data entry to initial monitoring Rationale: Surrogate for issue identification Median number of days from visit to CRF data entry Rationale: Impacts on ability to centrally monitor Median number of days from query open to close Rationale: Surrogate for unintended consequences of reduced site visits Median number of days from issue open to close Rationale: Indicates the effectiveness of issues management system Improvements or reductions in these measurements, as compared to a study that was conducted through a non-rbm model, would indicate the RBM methodology improved the Timeliness/Cycle Time dimension. In other words, the RBM model allowed us to reduce the cycle times for data entry, monitoring, and query resolution. 145

158 Finally, can you identify at least one metric within the Efficiency dimension to evaluate the impact of the RBM methodology? In the Efficiency dimension, the TransCelerate position paper suggests these possible metrics: Average monitoring (all types) cost per site Average interval between On-site Monitoring visits per site No one metric can be considered in isolation to assess impact of the RBM methodology. A holistic analysis of all metrics must be performed to gauge level of success. 146

159 Note to speaker: read summary on the slide 147

160 Key ideas in the TransCelerate RBM methodology include: Applying an efficient monitoring approach to rapidly detect and correct issues while the study is ongoing Focusing on errors that represent risks to subject safety, critical data, data integrity, and/or regulatory compliance. Reinforce that investigators are responsible for their site s data quality and are expected to partner with the Sponsor to address, resolve, and prevent issues. 148

161 4/23/

162 150

163 Any questions? 151

164 152