Use of IRB Quality Metrics for Performance Assurance of the Ethics Committee

Transcription

1 Use of IRB Quality Metrics for Performance Assurance of the Ethics Committee Pornpimon Adams FTM-EC Secretary Faculty of Tropical Medicine, Mahidol University

2 Background Ethics Committee Faculty of Tropical Medicine (FTM-EC) Registered with the Federal-wide Assurance (FWA) of the US-Office for Human Research Protections (OHRP) since 22 Recognized by SIDCER/FERCAP since 28 Composed of 2 panels, one for clinical studies and one for non-clinical studies Functions and practices comply with international standards and industry guidance

3 Conclusion: Delays in the approval or non-approval of studies were mainly due to unclear research methodology. The FTM-EC has employed several mechanisms in its operations, including transparency in the review process, building good relationships via open communication with investigators

4 Objectives To assess the performance of FTM s 2 Ethics Committee panels, as part of our commitment to assure quality and effectiveness, using IRB Quality Metrics (IRB-QM) To determine factors associated with time-toapproval over 6 Days

5 Assessment Tool IRB Quality Metrics (IRB-QM) comprise 3 domains: 1. Structure IRB components IRB qualification 2. Review Process meeting pattern workloads timeliness 3. Outcome Decision Issues for revisions

6 Source of Data for Analysis IRB meeting reports Notification letters to investigators IRB activities January 21 to June 213

7 The results were analyzed and presented by: Year Type of Panel Clinical Studies Non-Clinical Studies Type of Review Full Board Review Expedited Review Type of Study New non-exempt study Continuing & amended study

8 Results: IRB-QM - Structure & Activities Metrics Clinical studies (Panel I) Non-clinical studies (Panel II) IRB Components Numbers of IRB Members by Personal Qualifications TOTAL Physicians 6 2 Non-physicians 6 4 Lay persons 3 3 Non-affiliated to FTM 3 4 Numbers of Staff Working on Pre-review Process Scientific staff 1 1 Administrative Staff 2 1 Number of Primary Reviewers per Study (21-213) Full Board review Average (Min-Max) 2 (2-3) 2 (2-2) Expedited review Average (Min-Max) 2 (2-2) Use of Alternate Reviewers and Consultants (21-213) Number of studies Average (Min-Max) 3 1 Number of reviewers Average (Min-Max) 2 (1-3) 2 (1-3)

9 Results IRB Structure (for reviewing 1-11 studies per year) 25 members in both panels 3 staff perform pre-review 5 lay persons in both panels 2 primary reviewers per study 2 external reviewers/consultants needed per month

10 Results : IRB-QM - FTM-EC Workload Clinical studies (Panel I) Non-clinical studies (Panel II) Metrics N=46 N=3 N=44 N=13 N=67 N=66 N=65 N=34 Numbers of New Non-Exempt Studies Total 19 (41%) 12 (4%) 21 (48%) 1 (77%) 59 (88%) 6 (91%) 58 (89%) 3 (88%) - Full Board review 19 (1%) 12 (1%) 21 (1%) 1 (1%) 44 (75%) 38 (63%) 24 (41%) 18 (6%) - Expedited review Numbers of Re-submitted Studies Total 1 (3%) - Full Board review - 1 (1%) (4%) 2 (1%) 3 (23%) 3 (1%) 15 (25%) 22 (37%) 34 (59%) 2 (3%) (1%) 12 (4%) - Expedited review Numbers of Continuing/Amended Studies (Note: A study may be amended more than once) Total 27 (59%) 17 (57%) 21 (48%) 8 (2%) 6 (9%) 5 (8%) 4 (12%) - Full Board review (11%) (18%) (19%) (12%) (83%) (2%) - Expedited review (89%) (82%) (81%) (88%) (17%) (8%) (1%) -

11 Results IRB Workload & Activities: On average, 11 members (Panel 1) and 7 members (Panel 2) attend at least 3-hour meetings On average, study protocols are reviewed each month: 7 new non-exempt studies (5 for Full Board review; 2 for expedited review) 2 continuing & amended studies 1 study reporting deviations 1 study reporting SAEs Average review time for a study (prereview + meeting time) = 2 hrs As part of the quality assurance review of approved studies, both panels conduct annual study-site monitoring visits of purposively selected studies with high risk

12 Results : IRB-QM - Review Decision Metrics Clinical studies (Panel I) Non-clinical studies (Panel II) New Non-exempt Studies N=19 N=12 N=21 N=1 N=59 N=6 N=58 N=3 Numbers of Studies Approved (With/Without Revisions) - Full Board review 18 (95%) 9 (75%) 17 (8%) 4 (4%) 4 (68%) 33 (55%) 19 (33%) 16 (53%) - Expedited review (25%) 21 ((35%) 34 (59%) 12 (4%) Numbers of Studies Deferred/Not Approved - Full Board review (8%) (1%) (4%) (2%) (3%) - Expedited review Numbers of Studies Terminated or Pending a - Terminated 1 (5%) 2 (17%) 2 (1%) - 4 (7%) 5 (8%%) 3 (5%) - Pending at data cut-off (2%) (7%) Amended Continuing Studies N=27 N=17 N=21 N= N=8 N=6 N=5 N=4 Numbers of Studies Deferred/Not Approved - Full Board review - 1 (6%) 1 (5%) - Expedited review (5%) (17%)

13 Results : IRB-QM - Issues for revision Metrics Clinical studies (Panel I) Non-clinical studies (Panel II) New Non-exempt Studies N=19 N=12 N=21 N=1 N=59 N=6 N=58 N=3 Numbers of New Non-exempt Studies with Issues Requested for Revisions (Note: A study might have more than one issue (as shown on IRB s initial reviewing forms & meeting consensus) Objectives 5 (26%) Research Methodology 18 (95%) Informed Consent (Document & Process) 18 (95%) 4 (33%) 9 (75%) 9 (75%) 3 (14%) 2 (95%) 18 (86%) 4 (4%) 1 (1%) 1 (1%) 18 (31%) 48 (81%) 28 (47%) 1 (17%) 32 (53%) 3 (5%) 12 (21%) 51 (88%) 27 (47%) 1 (3%) 21 (7%) 15 (5%) Case Record Forms 1 (53%) 4 (33%) 4 (19%) 2 (2%) 31 (53%) 15 (25%) 19 (33%) 7 (23%) Research Facilities 7 (37%) 1 (1%) 22 (37%) Budgeting 1 (5%) 1 (5%) 3 (5%) 1 (2%) 1 (2%) Compensation 5 (26%) 4 (33%) 7 (33%) 6 (6%) 11 (19%) 2 (3%) 18 (31%) 7 (23%)

14 Results IRB Review Outcomes 95% of studies were approved 3% of studies were deferred; 2% were not approved Issues for revision: clinical studies 75-9% for research methodology & informed consent (research methodology includes sample size, inclusion, exclusion criteria, specimen collection, risk-benefit) 2-4% for objective, CRFs & compensation Issues for revision: non-clinical studies 7-9% for research methodology 4-5% for informed consent 2-4% for objective, CRFs & compensation

15 Results : IRB-QM - Timeliness of Review Process (1) Metrics Clinical studies (Panel I) Non-clinical studies (Panel II) New Non-exempt Studies a N=17 N=11 N=21 N=1 N=57 N=57 N=58 N=3 Days From Protocol Submission to Investigator s Notification - Full Board review - Expedited review Mdn (Min-Max) Mdn (Min-Max) 27 (12-55) 35 (17-56) 28 (8-65) 28 (2-33) 41 (17-155) 19 (4-86) Days From Investigator s Notification to Investigator s Revision/Clarification b - Full Board review Mdn (Min-Max) 7 (11-269) 4 (31-9) 68 (19-175) 52 (25-114) - Expedited review Mdn (Min-Max) 2 (2-231) 13 (-18) 4 (7-63) 19 (5-35) 45 (1-273) 22 (2-58) 3 (22-56) 17 (7-36) 59 (-142) 15 (1-195) 31 (22-7) 22 (7-28) 26 (1-98) 13 (1-55) Total Days From Submission of Protocol to Final Approval - Full Board review Average (SD) 112(68) 76(32) 98(47) 63 (39) 83(52) 83(43) 8(42) 59(28) - Expedited review Mdn (Min-Max) 15 (35-288) 71 (21-135) 82 (31-25) 58 (21-139) 68 (31-262) 81 (8-238) 77 (27-168) 55 (22-13) Average (SD) 35(27) 34(21) 46(49) 32(22) Mdn (Min-Max) 29 (4-14) 32 (7-226) 26 (7-83)

16 Results : IRB-QM - Timeliness of Review Process (2) Metrics Clinical studies (Panel I) Non-clinical studies (Panel II) Numbers of Times Revision Required per New Study N=19 N=12 N=21 N=1 N=59 N=6 N=58 N=3 None n (%) 6 (1%) 12 (2%) 12 (21%) 9 (3%) 1 time n (%) 12 (63%) 9 (75%) 14 (66%) 9 (9%) 49 (83%) 37 (62%) 39 (67%) 2 (67%) 2 times n (%) 2 (11%) 3 (25%) 5 (24%) 1 (1%) 3 (5%) 11 (18%) 7 (12%) 1 (3%) >2 times n (%) 5 (26%) 2 (1%) 1 (2%) Amended Continuing Study d N=27 N=17 N=21 N= N=8 N=6 N=5 N=4 Total Days From Submission of Amended Continuing Study to Final Approval - Full Board Average (SD) 35(33) 45(21) 35(17) - 13(-) 33(18) 72(-) - review Mdn (Min-Max) (16-73) (21-57) (21-57) (13-) (17-63) (72-) - Expedited review Average (SD) 24(26) 23(12) 17(1) - 1(5) 8(-) 6(4) 8(2) Mdn (Min-Max) 17 (3-128) 22 (9-42) 13 (5-47) - 8 (4-19) 8 (8-) 6 (1-11) 8 (6-1)

17 Results IRB Timeliness of Review Process Average time for all studies from submission to investigator notification = 32 days, from notification to clarification/revision = 47 days from last revision to approval = 8 days Average time from submission to approval Clinical studies = 8 days Non-clinical studies = 7 days Trend of decrease over the past 3.5 years (21-213) Clinical studies: 112 to 63 days Non-clinical studies: 83 to 59 days Average number of requests for revision Clinical studies: 1 time = 9%, 2 times = 1% Non-clinical studies: No revision = 15%. 1 time = 75%., 2 times = 1%

18 Results Criteria for determining factors associated with time from submission to approval Target: Time from Submission to Approval = 6 Days (3 days from Protocol Submission to First Notification + 3 Days for Protocol Revision to Final Approval) Factors associated with Total Time from Submission to Approval > 6 Days: Type of Study Common issues notified to investigator for revision:

19 Results : Factors associated with Time to Approval > 6 Days All Protocols Factors Percent of studies with total time >6 Days Prevalence Ratio (95%CI) Type of Study - Clinical Study 45/62 (73%) 1.8 ( ) - Non-Clinical Study 84/27 (41%) 1 Issues Notified to Investigator by FTM-EC Objectives Yes 37/57 (65%) 1.5 ( ) No 92/212 (43%) 1 Research Methodology Yes 117/29 (56%) 2.8 ( ) No 12/6 (2%) 1 Informed Consent Yes 16/155 (68%) 3.9 (2.3-5.) No 23/114 (2%) 1 Case Record Forms Yes 6/92 (65%) 1.7 ( ) No 69/177 (39%) 1 Research Facilities Yes 19/3 (63%) 1.4 (1.-1.9) No 11/129 (46%) 1 Budgeting Yes 6/7 (86%) 1.8 ( ) No 123/2 62 (47%) 1 Compensation Yes 39/6 (65%) 1.5 ( ) No 9/29 (43%) 1

20 Results 44% of non-clinical studies, and 76% of clinical studies took > 6 days for approval The 2 main issues for which revision was requested, leading to approval time > 6 days were informed consent process and research methodology Other issues were objective, CRF, research facilities, budgeting, and compensation

21 Conclusions & Implications IRB-Quality Metrics can be used for assessing workload, to plan IRB structure and review processes determining efficiency of IRB in terms of its outcomes and working/monitoring activities identifying factors associated with delay in time to approval serving as quality assurance of its functionality serving as baseline information for annual work plan

22 Conclusions & Implications Work plan based on IRB-QM, : Aim to reduce submission-to-approval time to 45 days Implement pre-review clinic to assist investigators with protocol preparation before submission Arrange protocol-writing workshop for research community, focusing on issues for which FTM-EC commonly requests revisions Arrange annual refresher courses on GCP and human-subject protection, focusing on the informed consent process Arrange an annual meeting of IRB members for self-assessment; review roles and functions, SOPs, work practices, and institutional guidelines in response to IRB- QM

23 Thank you for your attention.