Timeliness of TMF Document Submission: from Finalized to Filed

Transcription

1 Timeliness of TMF Document Submission: from Finalized to Filed Vittoria Sparacio, MSc CBI s etmf Forum Hyatt at de Bellevue Philadelphia 18 November 2014

2 Finally my turn to say... I know I am standing between you and the drinks so... 2 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

3 Agenda Understand why timeliness of document submission really matters Walk through common hurdles experienced around document submission Possible best practices to decrease cycle times for document submission and release AUTHOR S DISCLAIMER: THIS PRESENTATION REFLECTS THE VIEWS OF THE AUTHOR AND SHOULD NOT BE CONSTRUED TO REPRESENT NOVARTIS VIEWS. CONFIDENTIALITY NOTICE: THIS PRESENTATION SLIDES IS FOR THE SOLE USE OF THE INTENDED RECIPIENTS PARTICIPATING AT THE EVENT ETMF FORUM FURTHER COPYING OR DISTRIBUTION WITHOUT EXPRESS CONSENT OF THE AUTHOR IS PROHIBITED. 3 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

4 Why Timeliness? Hmm... 4 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

5 Why Timeliness Really Matters Everyone recognizes that timeliness matters. Timely TMF/eTMF filing enables: Visibility and oversight Action-ability Contemporaneousness An out of date TMF gives no confidence of an organization s ability to manage and oversee trial conduct 5 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

6 Can etmf Resolve Our Problems? An etmf system per se does not resolve TMF management inadequacies. However, It helps highlight the gaps and make them transparent in real time It emphasizes the need to ensure robust measures in place to prevent, identify and remediate gaps. In other words, an etmf system enables collecting, combining and analyzing key information to quantify the degree of TMF compliance (timeliness, quality, completeness). Quantification provides the insight for actions and decisions to be taken, risk management and ultimately for better process improvements planning. 6 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

7 How etmf Changed Filing One of the hurdles to timely filing is represented by the fact that with the introduction of etmf the approach to filing has shifted to a more complex task that takes time upfront. Many organization have not adjusted their resource algorithms accordingly neither the expectations around the procedural steps for completing the filing. paper TMF electronic TMF Collect, copy, generate doc & print Collect, copy, generate doc Place into desired binder section Scan, upload to desired destination - X Enter metadata, send for QC - Perform QC, rectify, release 7 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

8 Common Hurdles to Timely Filing What makes it so hard to ensure consistently timely TMF filing? Variability based on document types Unclear roles and responsibilities ( not my job ) Multiple contributors / stakeholders Lengthiness of validated filing method Bad resource algorithms (does not account for doc peaks, different process steps) Culture (filing is still an after the fact act) Comfort of more frequently used tools (outlook, sharepoint, share drive, C: drive, CTMS, EDC, etc.) Not-a-priority (only time critical when audit or inspection is notified) Limited visibility on benefits (why should I do that?) 8 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

9 Map Out the Process Mapping out the etmf process steps helps determine parameters for metrics to develop a metric strategy plan. Collecting, Filing Release Using, Cleaning Archive What do you want to measure? 9 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

10 Plan for a Metrics Strategy Quality, timeliness and completeness parameters should be defined in the metrics strategy plan to develop appropriate targets and expected ranges for success / failure for internal and external stakeholders (e.g. CROs). For example: What needs doing to file in the TMF: End-to-end from document creation to finalization and QC release End-to-end from document collection to upload and QC release Partial from document collection to upload Partial from document creation to finalization Who is responsible for filing steps: Trial Manager, CTA, CRA TMF Specialist, Document Manager Trial Lead, Statistician, Medical Writer 10 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

11 Plan for a Metrics Strategy, 2 When in the process is a document considered on file in the TMF: Programmatically by workflow approval Release upon metadata and image verification (QC) Upon upload, metadata assignment and send for QC Upon upload only Upon cyclic content verification (periodic review/reconciliation) How is a document filed in the TMF: Original Electronic Original Paper Copy Electronic Copy Paper What is the acceptable and what requires re-work: Quality of image Quality of metadata 11 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

12 Metrics.. 12 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

13 Metrics Timeliness There are many categories of metrics that can be used. The TMF Reference Model Metrics and Reporting Sub-team has identified 6. Type Administration Completeness Quality Timeliness Use Volume Definition Indication of whether TMFs have defined ownership and planning measures in place Extent to which a TMF/eTMF contains all documents that are expected at the current point in the study (usually based on last milestone date), or, for completed trials, at the end of the trial. Measure of whether document content, metadata, and indexing are complete and accurate Indication of whether documents are available when expected or needed, and of how long documents take to finalize Measure of how frequently an electronic TMF system is accessed Measures of the types, numbers and sizes of documents in a TMF/eTMF 13 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

14 Metrics Timeliness When are documents due? Most documents can be tied to a milestone With milestone data (generally coming from CTMS), an etmf can forecast and track due dates for documents Assumptions can be made for documents not tied to specific timeline based on the general ICH-GCP standard Compliance with protocol and etmf management plan Is a document finalized by its due date? Timely availability of documents Is a document collected and uploaded in a timely manner so that it can be QC / released in a timely manner? Is a document quality checked and finalized in a timely manner after receipt? Did it require re-work to finalize? (also a quality metric!) 14 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

15 Metrics Quality Accuracy and completeness of document content For authored documents, part of review process For scanned / uploaded documents, part of collection process Accuracy and completeness of document image For scanned documents, part of QC process For authored documents, part of review / doc control process Accuracy of document metadata Assignment to correct document type, trial, country, site Correctness of other metadata collected, e.g. document date, page count Effectiveness of QC process How many tickets to rectify document metadata? How many tickets to delete duplicates? How many tickets for poor image? 15 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

16 Metrics Completeness Completeness Compare list of missing documents against expected Expedite upcoming and overdue documents Identify excessively slow workflow processes Discover systematic quality defect patterns 16 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

17 Drive Behavior via Increased Visibility Metrics are only useful if they: Drive desired behaviors Help improve decision making It is essential to ensure transparency: Insight to allow stakeholders to evaluate status Routine distribution of metrics reports or, better, self service Defined escalation processes for reacting to issues and trends uncovered in key performance indicator reports 17 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

18 Metrics Caveats Results will hardly be 100% accurate and pure Setting metrics with clear assumptions still adds value Exclude invalid data (assumptions) that would undermine confidence in overall metrics, or would derail purpose (people defensive) Non allocated countries Non selected sites For data derived from assumption use best case scenario 18 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

19 Best practices to decrease cycle times Develop own metric strategy to decrease cycle times for document filing. Consider Tie to milestone (event date in CTMS system plus defined processing time) Enforce by procedure timelines (defined in SOP) Monitor KPIs regularly (associate frequency with audience, consider self-service) Implement dedicated specialist roles Limit user driven metadata (from outlook to etmf; scan to etmf, workflows, metadata fill-in) Enforce in annual objectives 19 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

20 New Opportunities Over time, as more data is collected across multiple trials, trends can be analyzed, exploding the added value of the etmf. Organizations can determine whether problems are unique to one particular site or if there is a common problem across multiple trials run in that site. For example, archiving process is too lengthy for numerous trials, what steps can be taken to shorten this activity and improve cycle time? A specific CRO document quality is consistently poor, what steps can be taken to ensure improvement The etmf can help with business decisions by gathering an array of quality, performance, and operational metrics that are both internal and external and across multiple sites and studies, CROs and internal personnel. 20 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

21 From obtaining to using the data Focus shifted from obtaining the information... get the documents in the system get them in quickly ( real time) get them consistently filed..to using the information maintain oversight and monitor progress in real time anticipate and plan for upcoming activities track completeness take corrective actions optimize resource use reveal patterns/trends for CROs, countries, functional areas CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

22 Warning! Actions always have effects that are unanticipated or unintended. (Law of unintended consequences) The golden rule triangle may skew! Use balance - pushing too hard in one dimension may cause damage to another dimension - True story: etmf Archiving issue 22 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

23 Setting the expectations.. 23 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

24 General SLA A generic service level agreement (SLA) can be defined as part of the quality system. This includes setting the expectations over fundamental process requirements into the appropriate TMF management SOPs. Purpose, scope and applicability Ownership, roles and responsibilities Regulatory references High level process requirements and timing References to work instructions where the procedure is expanded to more granular level Standard tools expected to be used (e.g. standard table of content, specifications, transmittal sheets, forms, workflows) Training requirements 24 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

25 Study specific SLA A study specific SLA is defined in an etmf management plan which puts more meat around nuances around etmf management that are planned for the individual study. Tie the etmf management plan to CRO task order. The etmf management plan addresses questions such as the following: Who owns the etmf oversight for this study? Who are the key stakeholders, their roles and responsibilities? Does the TOC need customization (e.g. not applicable)? What will be the planned documentation flow? What will be the upload timelines? What will be the QC timelines? What will be the periodic content review cycles frequency? Will there be any filing convention (e.g. central IRBs, satellite sites)? Does the etmf need unblinded document access? Does a CRO need access? What are the escalation paths? 25 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

26 What matters to everyone The right information In the right format To the right user At the right time! 26 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted

27 Q & A Thank You! For more information and references TMF Reference Model Metrics and Reporting Sub-team 27 CBI's etmf Forum Sparacio 18 Nov 2014 Timeliness of TMF Doc Submissions from Finalized to Filed Restricted