Welcome! MDIC Case for Quality Metrics Workshop

Transcription

1 Welcome! MDIC Case for Quality Metrics Workshop

2 Today s Agenda Time Discussion Leader Description 11:30 12:00 Marla Phillips and Stephanie Christopher 12:00 12:45 Sean Boyd and Cisco Vicenty Lunch, Welcome, and MDIC Overview FDA Case for Quality Vision and Strategic Priorities for :45 2:00 Marla Phillips Effective Metrics Implementation 2:00 2:15 All Break 2:15 3:30 Monica Wilkins Successful Management Review Process and Incorporation of Enterprise-Wide Feedback Loops 3:30 4:00 Cisco Vicenty Maturity Model Discussion and Path Forward

3 Our Mission. 3

4 Making a Difference Over 150 Pharmaceutical Professionals, Medical Device Professionals and FDA Officials are formally engaged with Xavier Health Conferences FDA/Xavier PharmaLink Conference FDA/Xavier MedCon Conference Xavier Combination Products Summit Xavier Cybersecurity Summit Xavier Device Leaders Forum Initiatives The Xavier Center for Artificial Intelligence Xavier Supply by Design Initiative MDIC Case for Quality Metrics Initiative Xavier/PwC Pharma Metrics Initiative Xavier/Schellingpoint Cultural Initiative Development Xavier Medical Device Leaders Certificate Xavier 3 rd Thursday Lunch and Learn Series On-Demand Device Training Toolbox 4

5 Discussion Leaders Captain Sean Boyd Marla Phillips Cisco Vicenty Monica Wilkins

6 6 Introduction to the MDIC Case for Quality Stephanie Christopher Program Director, Case for Quality Medical Device Innovation Consortium

7 7 What is MDIC? MDIC is a 501(c)(3) non-profit organization and is the first-ever public-private partnership created with the sole objective of advancing regulatory science of medical devices for patient benefit Align Resources WORKING COOPERATIVELY to re-engineer pre-competitive technology innovation Accelerate Progress Achieve Results REDUCING TIME and resources needed for new technology development, assessment, and review HELPING PATIENTS gain access to new medical technologies sooner MDIC HIGHLIGHTS 59 participating member organizations Leading resource on issues important to the Medtech innovation ecosystem 6 Projects have been initiated Congressional testimony on modernizing clinical trials Over $4.4m funding from grants and contracts for Program initiatives.

8 A Brief History of MDIC and the Case for Quality CDRH hatches the Case for Quality MDIC is founded 2015 FDA BAA Contract Four solid deliverables 2017 Fitting the pieces together

9 9 MDIC Case for Quality Vision: MDIC and its partners are collaborating to identify methodology that will raise the overall level of quality of medical devices in the industry through Case for Quality Forums. These ongoing discussions offer greater continuity, and focus, informing industry design and manufacturing practices as well as agency regulatory models. Focus Areas: An ongoing Case for Quality Forum that encourages stakeholders to participate in discussion around opportunities for improving device quality within the industry A Maturity Model program focused on developing a means for using an independent assessment of quality maturity at a device manufacturer. A Quality Outcomes program aimed at creating a standard, independent, and reliable source of information on device quality, using registries and patient reported data Provide research and stakeholder input to CDRH to inform the development of voluntary quality program and pilot proposal to be announced in December 2017

10 10 How Are FDA and MDIC Making This A Reality In 2017? FDA Actions Modify review requirements Manufacturing reviews 30-Day Notices, Site Changes, PMA Original manufacturing module Tgt. July 2017 Design reviews 510(k), PMA original design module Tgt. December 2017 Voluntary program pilot implementation plan Define operating parameters Establish engagement rules Conduct public meeting August 2017 Announce program pilot and solicit participants December 2017 MDIC Support Maturity Team Establish assessment program Assessment criteria development Monitoring strategy Data collection and metrics Analytics Team Expand development of data sources Identify outcome comparison methods Identify integration of maturity assessment metrics and output Program pilot operations Support defining operating parameters Information sharing expectations Support development of engagement rules Coordination of additional stakeholders

11 11 Resources MDIC Case for Quality: White Papers, dashboards and other resources at:

12 12 Join us! Questions? Want to get involved? Contact Stephanie Christopher

13 FDA Case for Quality Vision and Strategic Priorities for

14 Overview CDRH Vision, Case for Quality, and Our Strategic Priority Case for Quality Roadmap What does it mean? How are we doing this?

15 CDRH Vision Patients have access to high quality, safe, effective medical devices, of public health importance, first in the world. Case for Quality promotes that vision through a shared FDA-industry goal to provide patient access to high-quality medical devices.

16 2017 CDRH Strategic Priority Promote a Culture of Quality and Organizational Excellence GOAL: Strengthen product and manufacturing quality within the medical device ecosystem By December 31, 2017, propose a voluntary program to recognize independent evaluation of product and manufacturing quality. obacco/cdrh/cdrhvisionandmission/default.htm

17 Where is the Case for Quality Going? FDA develops and announces a voluntary program pilot that recognizes independent assessment of manufacturing and product quality Voluntary program pilot starts Evaluate results collected and potential KPIs Begin collecting outcome metrics for comparative product quality Enhance program recognition Program improvement and enhancement Share outcome indicators Expand program to provide resources for new innovators Establish mechanism for firms struggling with compliance Operationalize program Expand program options and tools Improve premarket/post market decisions Leverage real world data for regulatory decisions

18 What Does This Mean? FDA Regulatory Paradigm Shift What does a focus on quality and organizational excellence mean for FDA and innovation? Increased manufacturing and product confidence Faster time to markets, better information to drive regulatory decisions, improved resource allocation A focus on what is most important to patients Remove participants from the agency work plan for routine inspections Waive pre-approval inspections where appropriate Engagement and meetings on issue resolution Modifying submission requirements and faster FDA response Accelerated approval path Competitive market around product excellence

19 Goal Statement Develop a program which leverages CMMI as the standard maturity model by which medical device organizations may measure their capability to produce high quality devices. FDA will adjust their engagement activities and submission requirements as a recognition of this independent assessment of quality maturity. Reduced defects / rework Reduced costs Accelerated time to market Increased Customer Satisfaction A culture of quality - across the organization.

20 CMMI QMM Process Area lens Optimizing Quantitatively Managed Continuous Improvement Data-driven Decision making Defined Useful Process Infrastructure Managed Plan your work; Work your plan Initial Work happens

21 SCAMPI Level & FDA Modifications Pilots demonstrated to FDA that the model can work: Agreement that this is moving in a positive direction The appraisals yield granular, beneficial discussions Starting with SCAMPI C Discovery Approach is where we can begin FDA wants to enable the Program to utilize evidentiary levels (SCAMPI A)

22 FDA CfQ 2017 Focus Focused Project Activity Manufacturing Module Cross-center FDA teams establish program centered around full manufacturing with support from external MDIC teams Develop third party assessment evaluating organizational touch points Reduce submission requirements for manufacturing activities Benefits Focused direction Tangible objectives Measurable Data and metrics easier to define Adaptable Allows for the development of program that can be easily adapted for additional areas Submission reduction can be easily monetized FDA resource reutilization can be quantified Data to begin feeding outcomes can start to be collected

23 How Are We Making This A Reality In 2017? FDA Activities Modify review requirements Manufacturing reviews 30-Day Notices, Site Changes, PMA Original manufacturing module Design reviews 510(k), PMA original design module Voluntary program pilot implementation plan Define operating parameters Establish engagement rules Conduct public meeting August 2017 Announce program pilot and solicit participants December 2017 Next Steps Create a program which leverages the CMMI framework to drive continuous improvement Determine how we collect, analyze, and share the assessment results Determine how do we evaluate the effectiveness of the program Define milestones for how quickly we need to scale Establish how the FDA program shifts bring value to participants as increasing maturity levels are achieved

24 Questions to think about High manufacturing quality 1. How do you define high manufacturing quality? 2. What behaviors show a focus on increasing: Control Speed Visibility 3. How is that measured or assured?

25 Today s Agenda Time Discussion Leader Description 11:30 12:00 Marla Phillips and Stephanie Christopher 12:00 12:45 Sean Boyd and Cisco Vicenty Lunch, Welcome, and MDIC Overview FDA Case for Quality Vision and Strategic Priorities for :45 2:00 Marla Phillips Effective Metrics Implementation 2:00 2:15 All Break 2:15 3:30 Monica Wilkins Successful Management Review Process and Incorporation of Enterprise-Wide Feedback Loops 3:30 4:00 Cisco Vicenty Maturity Model Discussion and Path Forward

26 Effective Metrics Implementation

27 Purpose and Outcome Identify Meaningful Metrics for Industry To provide a system of metrics across the Total Product Lifecycle that: 1. Informs internal company decisions and triggers action 2. Shifts the Right-First-Time mentality closer to the initial days of development

28 Metrics Team Members (1 of 5) First Last Company Paul Andreassi Fisher & Paykel Healthcare Karen Archdeacon FDA Pat Baird Baxter Healthcare Kathy Bardwell Steris Anupam Bedi AtriCure Pankit Bhalodia PwC Kankshit Bheda PwC Steve Binion BD Robin Blankenbaker W.L. Gore & Associates Rafael Bonilla ScottCare Gina Brackett FDA Kate Cadorette Steris

29 Metrics Team Members (2 of 5) First Last Company Patrick Caines Baxter Healthcare Tony Carr Boston Scientific Kara Carter Abbott Vascular Division Vizma Carver Carver Global Health Ryan Eavey Stryker Joanna Engelke Boston Scientific Tom Haueter Clinical Innovations Chris Hoag Stryker Jeff Ireland Medtronic

30 Metrics Team Members (3 of 5) First Last Company Frank Johnston BD Greg Jones BSI Bryan Knecht AtriCure Jonathan Lee PwC Bill MacFarland FDA Kristin McNamara FDA Rhonda Mecl FDA Brian Motter J&J MD&D Ravi Nabar Philips

31 Metrics Team Members (4 of 5) First Last Company Steven Niedelman King & Spalding LLP Scott Nichols FDA Pete Palermo CR Bard Luann Pendy Medtronic Marla Phillips Xavier University Greg Pierce Engisystems Susan Rolih Meridian Bioscience, Inc. Barbara Ruff Zimmer Biomet Joe Sapiente Medtronic Gin Schulz CR Bard Benjamin Smith Biomerieux

32 Metrics Team Members (5 of 5) First Last Company Isabel Tejero FDA Shelley Turcotte DePuy Synthes Sam Venugopal PwC Marta Villarraga Exponent Monica Wilkins Abbott 46 Members

33 Let s follow the process used by the Metrics Team

34 Methodology Divided into Teams by Phase of Production 11 Assessed Critical Systems

35 11 Critical Systems We identified 11 Critical Systems that would help us ensure we did not have any gaps in our thinking and that we assessed each system for predictive measures: 1. CAPA 2. Change Control 3. Complaint Handling 4. Customer- Related/VOC 5. Design Controls 6. Distribution 7. Management Controls 8. Post-Launch Surveillance 9. Production and Process Controls 10. Servicing 11. Supplier Controls 35

36 Methodology Divided into Teams by Phase of Production 11 Assessed Critical Systems 97 Identified Key Activities

37 Pre-Production Key Activity Example Critical System: Supplier Controls Exchange of Quality Analytics with Suppliers through a partnership of true continuous improvement. Manufacturer receives and reviews in-process data from key suppliers. Engagement beyond the scorecard report.

38 Production Key Activity Example Critical System: Change Controls A quality design should require minimal changes. Changes should be enhancements or due to technological advancement. Design team should identify issues during testing and remediate before design transfer.

39 Post-Production Key Activity Example Critical System: Complaint Handling Use of a mobile app to guide Sales team in providing complete complaint information. Ensures the right questions are asked such that complete information is received the first time the complaint is submitted. Increases the completeness of evaluation and investigation, speeds complaint evaluation and investigation and enables a quicker awareness of emerging trends in product performance.

40 Methodology Divided into Teams by Phase of Production 11 Assessed Critical Systems 97 Identified Key Activities 208 Determined How to Measure Activities Not every idea is a good fit for the intended purpose 124 Assessed Using Cause and Effect Matrix Critical Criteria

41 How to Measure our 3 Examples Pre-Production: Evidence that customer input (including that of internal customers) is actively solicited, assessed and documented at all stages of project and as design evolves Production: No product failures due to inadequate design evaluations or knowledge. Low percentage of NC, OOS, Complaint reports, and no recalls due to inadequate understanding of product design Post-Production: CAPA effectiveness across all systems 41

42 C&E Method How it can be used Requirement Descriptor: Importance Rating >>> of the Critical Requirement versus the Vision statement Measure Evidence that customer input (including that of internal customers) is actively solicited, assessed and documented at all stages of project and as design evolves. Post launch VOC information considered at start of new projects Evidence that design projects capture voice of targeted, diverse group of stakeholders Evidence of analysis of issues, root causes and CAPAs across related projects with evidence of actions brought forward to new project where trends are observed. Evidence that manufacturability requirements are included in design inputs Linkage to Critical System Patient Safety Design Robustness Process Reliability Quality System Robustness Failure Costs Correlation to Critical Requirement? 0 = Highly Improbable; 3 = Slightly Improbable; 6 = Slightly Probable; 9 = Highly Probable Total VOC VOC VOC CAPA PPC

43 17 Measures Across TPLC 2 Enterprise-Wide Continual Improvement 3 Post- Production 4 8 Pre-Production Transfer Production R&D Continual Improvement & Risk Mgmt. Production Continual Improvement & Risk Mgmt.

44 Implementation in Your Organization This process can be used to identify ways to measure previously untracked areas of quality and/or risk Assess which critical requirements the metric is correlated to, in order to be sure it has the potential to be effective Be sure to assess the usefulness of the metric over time Is it a flat-line result over time? Are any decisions or actions ever taken as a result of tracking? Has the metric demonstrated acceptable improvement and steady-state? Is unacceptable quality and/or risk experienced even though this metric is consistently acceptable?

45 Methodology Divided into Teams by Phase of Production 11 Assessed Critical Systems 97 Identified Key Activities 208 Determined How to Measure Activities Not every idea is a good fit for the intended purpose 124 Assessed Using Cause and Effect Matrix Critical Criteria Converted Measures into Metrics

46 Pre-Production: Design Robustness Goal of the Pre-Production Metric: to drive the Right- First-Time (RFT) mindset in the research and development phase such that post-design transfer changes due to inadequate product/process development are not needed. total # of changes (product and process across projects) total # of projects

47 Pre-Production: Design Robustness total # of changes (product and process across projects) total # of projects Only include changes required due to inadequate product or process development (harmonize definition across organization) Calculated with or without the denominator Each organization must determine what constitutes a project Ideal to measure the changes required during transfer, then production, then at a predetermined frequency on-market

48 Pre-Production: Design Robustness Planned versus Unplanned Changes Remember that the goal is to make your organization better, not to make the metric better. Planned Enhancements and New Features already considered during the design phase but not implemented prior to launch. Examples: Realignment of Production Process Steps for efficiency (Lean Manufacturing), Adding Product Feature (Software features to product) if said feature does not change the product performance. Unplanned Corrections implemented due to unanticipated VOC feedback, complaints, or any other unforeseen required changes. Examples: Quality issue driven by Complaints, Returns, CAPA, Production yields, Nonconformance reports, etc.

49 Production: Right First Time Goals of this Metric: To gather nonconformance information during production operations related to inadequate product/process development. To inform senior leaders of the nonconformance rate in production related to inadequate development, such as to drive improvement in the overall research and development process for future products. To drive a Right-First-Time mindset throughout the organization such that failures occurring during production due to inadequate product/process are reduced or eliminated. # of units mfg. Right-First-Time within or across lots # of units started

50 Production: Right First Time # of units mfg. Right-First-Time within or across lots # of units started Include all non-conformances initially Organization then must triage the root causes Include planned rework and set-up scrap to minimize the waste Track and trend on a rolling basis Must ensure terms are defined consistently across products and sites

51 Post-Production Metrics Goals of the Post-Production Metrics: To analyze key post-market surveillance data to eliminate, reduce, and prevent future on-market failures. This assessment feeds into an overall Quality Management System (QMS) performance assessment and holistic QMS scorecard. To triage root causes of the on-market indicators that are related to inadequate product/process development. To drive a Right-First-Time mindset in product and process development such that failures occurring during post-production related to inadequate product/process development are reduced or eliminated for future products.

52 Post-Production Metrics 1. Service Records: Records per product / total # of units in service for the period (unplanned service only) 2. Installation failures: # of installation failures/ total # of installations for the period 3. Complaints: Complaints for the product per period / units sold per period 4. MDRs: MDRs for the product / units sold Include MDRs, MDVs and AERs 5. Recalls (rate): # of units recalled / # of units sold measure of impact Include recalls, field corrective action and safety alerts 6. Recalls (total): # of recalls (for the period) - measure of absolute number Include recalls, field corrective action and safety alerts

53 Post-Production Metrics Step 1: Ensure your organization is calculating and tracking each of the post-production metrics. Identify trends, and have predetermined action limits. Step 2: Advanced Calculations are used to incorporate product risk profile information into the calculation of each indicator. Step 3: Comparative analyses can be conducted through mechanisms such as dashboards, score cards, heat map tools, etc. Can incorporate pre-production and production metrics with post-production This can give senior leaders an enterprise-wide view of the risk to product quality

54 To Inform Decisions and Trigger Action

55 Step 1. Severity Classifications Risk Analysis of Complaints Severity Classification Severity Weighted Value Catastrophic 50 Potential for Death Severity Definition Critical 30 Potential for Serious Injury Marginal 19 Potential for Non-Serious Injury Negligible 1 Minor Customer Annoyance, Cosmetic Issue, No injury to patient

56 Step 2. Risk of Each Complaint 1 st Quarter Product A Risk Profile Score: 1000 Complaints for period 1 Critical Complaint/1000 Complaints X 30 X 100 = 3 3 Marginal Complaints/1000 Complaints X 19 X 100 = Minor Complaints/1000 Complaints X 1 X 100 = 99.6 Total Risk Profile Score = nd Quarter: Product A Risk Profile Score: 1000 Complaints for period 2 Critical Complaints/1000 Complaints X 30 X 100= 6 4 Marginal Complaints/1000 Complaints X 19 X 100 = Minor Complaints/1000 Complaints X 1 X 100 = 99.5 Total Risk Profile Score = 113.1

57 Step 3. Rate of Occurrence Quarter Number of Complaints Number of units released Complaint Rate Total Multiplier Reported rate 1 st 1,000 10, , nd 1,000 15, , rd , , th 2,500 10, ,

58 Step 4. Side-by-Side Comparison 1 st Quarter Product A 2 nd Quarter Product A 3 rd Quarter Product A 4 th Quarter Product A Complaint Rate Complaint Risk Profile Score

59

60 Spider Chart

61 Heat Map - TPLC Risk Analysis Y-axis = Internal Risk Score Internal includes pre-production and production metric total risk score Each Point = Risk Across TPLC X-axis = External Risk Score External includes the total post-production risk score of appropriate indicators

62 Trigger Action! Enterprise-Wide Continual Improvement Post- Production Pre-Production Transfer Production R&D Continual Improvement & Risk Mgmt. Production Continual Improvement & Risk Mgmt.

63 Today s Agenda Time Discussion Leader Description 11:30 12:00 Marla Phillips and Stephanie Christopher 12:00 12:45 Sean Boyd and Cisco Vicenty Lunch, Welcome, and MDIC Overview FDA Case for Quality Vision and Strategic Priorities for :45 2:00 Marla Phillips Effective Metrics Implementation 2:00 2:15 All Break 2:15 3:30 Monica Wilkins Successful Management Review Process and Incorporation of Enterprise-Wide Feedback Loops 3:30 4:00 Cisco Vicenty Maturity Model Discussion and Path Forward

64 Successful Management Review Process and Incorporation of Enterprise- Wide Feedback Loops

65 Today s Agenda Time Discussion Leader Description 11:30 12:00 Marla Phillips and Stephanie Christopher 12:00 12:45 Sean Boyd and Cisco Vicenty Lunch, Welcome, and MDIC Overview FDA Case for Quality Vision and Strategic Priorities for :45 2:00 Marla Phillips Effective Metrics Implementation 2:00 2:15 All Break 2:15 3:30 Monica Wilkins Successful Management Review Process and Incorporation of Enterprise-Wide Feedback Loops 3:30 4:00 Cisco Vicenty Maturity Model Discussion and Path Forward

66 Maturity Model Discussion and Path Forward

67 Questions to think about High manufacturing quality 1. How do you define high manufacturing quality? 2. What behaviors show a focus on increasing: Control Speed Visibility 3. How is that measured or assured?

68