Data Integrity Prioritization and Gap Analysis to Prove Data Integrity

Transcription

1 Data Integrity Prioritization and Gap Analysis to Prove Data Integrity Oct 9 th 2018

2 Speakers Alex Brindle Principal Consultant with DPS. Alex holds a B.Sc. in Chemistry, a M.Sc. in Analytical Science and a Ph.D. in Process Engineering. Alex s experience covers strategic consulting to project implementation for Drug Development, Manufacturing Operations, Project Design and Quality Initiatives. Formally spearheaded numerous remediation projects for various clients. Designed the project strategy for mass data integrity, gap analysis and remediation.

3 Approaches to Data Integrity Data Integrity is critical in a number of components of the data hierarchy and is critical all the way from regulatory submission to batch records Data is currently held in electronic or paper format (or electronic scans of paper format) Simple data integrity audits will provide assurance of proficient data integrity in an organization Data Integrity Gap Analysis and remediation will be typically be required when there are major issues in data integrity

4 Data Integrity Hierarchy Compliance with Regulatory Agency Submission Compliance with Product Licensing Requirements Regulatory Agency Requirements Compliance with ICH Annual Product Reviews Compliance with ISO Requirements Compliance with Controlled Substance Handling Quality System Compliance Change Control Corporate Quality System OOT / OOS CAPA Annual Product Review Validation Site procedures Quality Systems Training Back-up systems Site/Function Oversight Systems, Local Regulations Environmental Regulations & Permits Local human resources EH&S Requirements Role of Quality Unit Management Reviews Waste Management Manufacturing Functions Raw Material and Finished Product Quarantine and Release Water System Testing Compliance with Facility Design Personnel Gowning Qualification Training Compliance with Quality System Building Automation Systems Segregation of Areas Environmental Monitoring Personnel Monitoring Analytical methods development, validation, transfer Product performance (including BA/BE) R&D Functions Product Quality, including Stability Process development, validation and transfer Regulatory Submissions and data supporting regulatory decisions (approval, shelflife extension, etc.) Other Functions Quality Control method development validation and transfer Quality metrics Equipment/Systems Level Batch Records Forms Checklists SOP's Probe Data Sensor Data Device Data Data from Controllers Access control Validation Data Calibration Data MES Data Historical Data Data from Displays Instrument Data Various IT

5 This Presentation Focus on older Legacy Systems Focus on Paper (or scanned) Data Integrity (older systems) Focus on Large Scale Data Integrity Focus on Project Management

6 Methodology - Overview Scoping & Risk Prioritization Pilot Phase Gap Analysis Prioritized Gap Analysis Remediation Since most gaps in data integrity are formally unknown (though there is usually organizational gut feeling of where issues are) it is critical to prioritize the important issues Gap Analysis of an entire companies data integrity is an enormous undertaking which would take several years to fully complete Prioritizing risk/importance to divide data into smaller projects allows for manageable data integrity gap analysis projects Since even bite size data integrity gap analysis projects are large data can be collected on risk Additionally this also provides likely projections of other data integrity in other areas, their severity and volume Finally, decisions should be made as to remediate data integrity gaps, document explanation of gap or do nothing

7 Methodology - Overview Scoping & Risk Prioritization Pilot Phase Gap Analysis Prioritized Gap Analysis Remediation 0-5 years. Highly dependent on nature of gaps months. Execution of as many Portfolio of Gap Analyses as required. 3 weeks. Execute Pilot Phase Gap Analysis according to plan. Develop workflows templates for first Gaps. 1 week. High level scoping & prioritization. Development of draft execution plan and scope, schedule and resource needs.

8 Scoping & Risk Prioritization Scoping & Risk Prioritization Pilot Phase Gap Analysis Prioritized Gap Analysis Remediation Scoping identifies what functions/systems will be incorporated into project Perform Risk Assessment to quantify risk and to prioritize gap analysis activities Determine risk levels that will drive immediate, mid-term or no gap analyses and depth of gap analyses Develop execution plan for pilot phase and Prioritized Gap Analysis

9 Scoping & Risk Prioritization Tools Used Scoping & Risk Prioritization Pilot Phase Gap Analysis Prioritized Gap Analysis Remediation Work Breakdown Structure Model Completion Schedule A Schedule Punch List Document Tracker Flight Path

10 Scoping & Risk Prioritization - Overview Detailed scope definition Define strategy Execution plan Resource requirements & loading Resource skills definition Define Scope of Data Integrity Project Discussions with Sponsors / Steering Committee Summary of the execution plan Approach to execution Develop WBS for each stream Develop detailed Flow Chart for each stream Assign durations to each stream task Quantify number of products / potential gaps per product Quantification of the WBS tasks to estimate the resource requirements, Resource skill definitions Identify required skill from WBS

11 Products x 100 Scope Definition - Example Scoping & Risk Prioritization Pilot Phase Gap Analysis Prioritized Gap Analysis Remediation In this example scope was defined as legacy test methods The amount of products in scope (and associated test methods) defines amount of effort What data or potential gaps that are being looked for are important in definition as this defines gap analysis template and effort Methods x 500 Appropriateness and status of Qualification & Validation Suitability of method Compliance of method Submission/annual update alignment with current method Alignment between R&D and QC What data issues do we have? What methods cause problems?

12 Workflows Concept and Decision Routing Document Assembly The project is designed to be executed in work streams with specific groups of tasks Gap Analysis This will promote efficiency as sub teams or work centers can focus on repeat tasks and mass production NO Gaps? YES Decision points are inserted at workflows at appropriate points to allow document routing Completed Product Packet Remediation Planning or Further Action

13 Detailed Work Flow Example Site Obtains Current Approved TM from Document Management System 1. Obtain methods from document control (electronic or paper) 2. Multiple sites maybe providing different documents. Detailed workflows created for all workflow streams for each of the work packages No Compendial Method? 1. Simple assessment form documenting determination with perform verification Yes Explanations of expected work products, outputs and decision points are captured in the workflows Compare TM to Industry Standard per Established Criteria for TM Type 1. Completion of a pre-issued check lists perform and verify 2. Team leads/ Technical specialists assess TM 3. Team lead/ Technical specialists issues summary report ( Report is reviewed by Quality) Compare TM to Pharmacopeial Method 1. Completion of a pre-issued check lists perform and verify 2. Team leads/ Technical specialists assess TM 3. Team lead/ Technical specialists issues summary report ( Report is reviewed by Quality) The workflows describe how the teams will operate in the most efficient way to channel work efficiently into subsequent work centers TM Current w/ Industry Standards 1. Team leads/ Technical specialists assess TM 2. Team lead/ Technical specialists issues summary report ( Report is reviewed by Quality) Yes Document Analysis 1. Assemble documentation packet to be included with the product packets No Is the Method Used the Same as the Pharmacopeial Method? (Documented in step above) No Equivalency Documented/Justified 1. Team leads/ Technical specialists assess TM 2. Team lead/ Technical specialists issues summary report ( Report is reviewed by Quality) No Document Remediation Findings & Requirements 1. High level assessment of business impact and remediation priority. 2. Develop remediation plan by RA and Quality Risk Assessment 1. Subject matter experts conduct risk assessments and document/approve report 2. Review of conclusion by RA and Quality Risk Not Significant Classify for Remediation 1. Document remediation level per Company policies Significant Risk Quality Impact Assessment 1. High level assessment of business impact and remediation priority. 2. Develop remediation plan by RA and Quality Immediate Remediation Program 1. Execute and document immediate remediation's per COmpany policies Complete for Project X Complete for Project X Complete for Project X

14 Work Breakdown Structure Work Breakdown Structures were created for individual packets of work These included estimating at the task and hour level the work needed to complete collection of data, execution of analysis and development of remediation plans Resource functions were added and estimates made of their needs to complete the task The individual work breakdown structure and resource estimates were combined into a model to create a total project package based on repetition of work

15 Total Task/Resource Requirements The Work Breakdown Structure can then be translated into resource (type and quantity) needs

16 Resource Loading

17 Project Organization Main point of contact Steering Committee Project Management Team Project Manager Project Controls Admin Assistant Document Assembly Lead 1 Regulatory Lead 1 Chemical Method Lead 1 Microbiology Method Lead 1 Stability Lead 1 Project Staff and Experts Gap Analysis Phase 1 Remediation Plan Phase 2 ) Field Staff 20 Technical Advisor 1 Field Staff TBD Technical Advisor TBD Field Staff Review 4 Regulatory Affairs 1 Field Staff Review TBD Regulatory Affairs TBD Tech. Field Staff 3 Quality 5 Tech. Field Staff TBD Quality TBD

18 Impact (Financial and/or time schedule) Project Risk Analysis 1 Risk: Scope assumptions incorrect Mitigation: Pilot study Show stopper Risk: Inability to get key resources available quickly Mitigation: Alignment, understanding of need and commitment Major Risk: Aggressive project team ramp up Mitigation: Early commitment, aggressive staffing Moderate Risk: Adjustment in scope or priority from business objectives or FDA Mitigation: Clear direction from management, change control 5 Risk: Infrastructure (offices, IT) Minor Mitigation: Understand stakeholders and gain commitment Unlikely Possible Likely Likelihood Almost certain 6 Risk: Design of forms and outputs that don t meet FDA expectations Mitigation: Required expert input at start 7 Risk: Deadline missed Mitigation: Proactive Project Management

19 A Note About Workflows and Task Definition

20 Pilot Phase Scoping & Risk Prioritization Pilot Phase Gap Analysis Prioritized Gap Analysis Remediation Set up of Gap Analysis Project management Team Team will be responsible for managing project execution, feedback and resource definition/onboarding Development of document/other collation checklists simple checklists/forms developed to direct gathering sources of documentation required to perform gap analysis. Collate more information on breadth of targeted search. Collate material within scope of gap analysis from different sources and locations Execute Pilot Phase Gap Analysis team performs gap analyses on items in scope for collated material using checklists/forms Finalize Pilot Phase Gap Analysis collate information and data for reporting and to support projections of other gaps outside of Pilot Phase Gap Analysis scope

21 Example Work Flow Optimization Pilot Phase can be used to re-assess estimate from planning phase Analysis of what works, efficiency and resource skills helps improve process

22 Example Typical Gaps Method Validation Typical Validation Gaps (1) Linearity: R than y intercept evaluation Range not broad enough (80% -120%) not enough test (5 points) not performed for Assay not performed for all impurities Range: Not studied Not on appropriate sample Not performed for all impurities Linearity Intermediate Precision and Accuracy not appropriate Accuracy: Not studied Not enough samples - triplicates Not performed for all impurities Not appropriate sample No acceptance criteria Intermediate Precision: Not enough samples (n < 12) Not appropriate sample sample not pooled - expired Concentration not appropriate Not different days Not different equipment Not enough analysts Reproducibility (Transferred Methods): Not studied Not appropriate Specificity: Diode Array Detection was not used Extraneous peaks not evaluated Not performed Peak purity not studied No purity or threshold values PQL/PDL: Not determined PQL solution not appropriate Not enough injections Typical Validation Gaps (2) LOQ/LOD: Not determined Not determined for all impurities Not appropriate No threshold values Robustness: Not performed Not appropriate Not performed on appropriate samples Variations causes significant changes Stability Indication: Diode Array Detection was not used Not on degraded samples Expiration dates not provided Not performed Not appropriate (ICH guidelines) No force degradation of samples Performed on expired samples No purity or threshold values Relative Response Factor: Not studied Solution Reagent Stability: Not studied Not enough data Test results do not match with test method System Suitability: Not performed No bracketing standard drift criteria in the STM (N=5 or 6) Not appropriate Reference Standard: Method uses a different reference No indication of reference standard used USP Reference Standard not used Not evaluated

23 Example Gap Analysis Proforma Example of proforma gap analysis sheet developed for specific type of gaps Important to finalize and completely agree on gap analysis template by end of pilot phase to avoid significant rework in main gap analysis phase

24 Gap Analysis Phase Scoping & Risk Prioritization Pilot Phase Gap Analysis Prioritized Gap Analysis Remediation Prioritized Gap Analysis phase start multiple full gap analyses on items as high priority during Risk Analysis. These Prioritized Gap Analysis Projects can be run in parallel, staggered or sequentially Project teams designed in production line system with lessons learned from Pilot Phase (90% repetitive work, 10% unique work) Collate material within scope of gap analysis from different sources and locations Execute Prioritized Gap Analysis team performs gap analyses on items in scope for collated material using checklists/forms

25 Working Streams Regulatory Assessment Stream Chemistry Stream Microbiology/ Sterility Stream Stability Stream Regulatory Assessment Test Method Packet Document Assembly Chemistry Test Method Packet Document Assembly Microbiology/ Sterility Test Method Packet Document Assembly Stability Test Method Packet Document Assembly Regulatory Gap Analysis Checklist Chemistry Test Method Gap Analysis Checklist Microbiology/ Sterility Test Method Gap Analysis Checklist Stability Test Method Gap Analysis Checklist Checklist completed. No gaps Completed Product Packet Checklist incomplete. Gaps Remediation plan for any stream with gaps

26 Example Partial Final List of Gaps

27 Remediation Phase Scoping & Risk Prioritization Pilot Phase Gap Analysis Prioritized Gap Analysis Remediation Gaps identified in Gap Analysis Projects systematically sorted to understand metrics around quantity, types of gaps and remediation strategies Remediation Phase will first risk prioritize gaps Develop project execution plan for remediation Decisions made on type of remediation for different types of gaps full gap remediation, position/white paper justifying gap as no risk or reduced risk or no action Execute remediation on gaps and/or produce position/white papers justifying gaps (again vast majority of work is repetitive)

28 Remediation Do Nothing

29 Remediation Find the Data This may seem obvious (and why didn t you find it in Gap Analysis Phase??!!) In legacy situations there can be many reasons why data in hiding: Years old data Products moving sites Sites being bought, sold, decommissioned or built Knowledge retiring or leaving organization Mergers and acquisitions Often in Gap Analysis phase there is significant effort to find data but not as much organizational will as when gaps are actually found, noted and the cost to close those gaps calculated Interesting how much more data can be found when there is significant financial impact

30 Remediation - Position Papers Position Papers (or any name chosen) can address significant categories of gaps across multiple areas Supported by science, data, best practices they can be used to justify swathes of gaps and mitigate missing data

31 Remediation - White Papers White Papers (or whatever you want to call them) address smaller data integrity issues and can be used to justify missing data Justify small gaps with science, historical data and best practices Can be made efficient and a lot of repeating many elements from former documents Probably better course of action than Do Nothing

32 Remediation Rework or Improvement In certain situations a gap cannot be explained or justified In this situation many segments of work must be repeated (or work similar to repeating the work) Ideally use it as opportunity to improve something Important to execute in a timely manner in parallel to document working streams which are justifying gaps

33 Conclusions A project to identify Data Integrity Gaps and Remediate them can be organized efficiently Effective planning of workflows and tasks can have huge impact on resource need, schedule and budget Need to agree on templates and outputs (included what documents look like) during planning or pilot phase or it will be a lot of rework Large percentage of work for paper legacy data integrity gap analysis (and remediation) can be rote in which case possible to organize team like a production line with special teams to handle new, unique or difficult data integrity challenges