Critical Components of the Complaint Management Lifecycle Dick Roy Product Surveillance Director GE Healthcare
About your speaker 25+ years in Medical Device Industry 18 years in Post Market Management dealing with Class II and Class III products Expertise in Complaint Handling, MDR and other reporting requirements, Quality Systems and Audit * The contents of this presentation are based on the experience and views of the presenter and do not necessarily reflect those of GE Healthcare. 2
Agenda Overview of the Complaint Handling Process Establishment of a Robust Complaint Handling System Additional Topics 3
Establishing a Robust Complaint Handling System Complaint Handling Findings Regulatory and Competitive Pressures Education and Training for the Team Scalable features of a High Volume Complaint System 4
Regulatory Requirements for Complaint Handling Complaint Handling requirements are documented in 21CFR820.198, available at: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfc fr/cfrsearch.cfm?fr=820.198 These include requirements for complaint files, procedures, investigations, communications with complaint Formally Designated Units, and evaluation for MDR reporting. 5
Where do Manufacturers encounter difficulties? FDA publishes 483 findings by fiscal year. FY2014 findings are available at: http://www.fda.gov/iceci/inspections/ucm424098.ht m The site* groups 4010 findings originating from 972 form 483s * There are 483 findings that are not listed on this site. 6
Where do Manufacturers encounter difficulties? Most frequent 483 findings by Quality System citation involving postmarket include: 7
Where do Manufacturers encounter difficulties? What should be in a complaint procedure? For example, your firm s complaint handling procedure document does not Adequately designate a formal unit to review and evaluate complaints Require complaints to be evaluated to determine whether a complaint represents an event that is required to be reported to FDA under 21 CFR 803 Require complaint files to be maintained for all received complaints. During the inspection, your firm s general manager, indicated that the complaints deemed insignificant or invalid upon initial review are not documented as complaints WL 10/10/12 8
Where do Manufacturers encounter difficulties? What should be in a complaint procedure? Specifically, your complaint handling procedure, does not: a. ensure that records of complaint investigations, when necessary, include the required information. For example, your firm failed to document the following elements in 10 complaints associated with your devices: i. adequate details or nature of the complaint including the setting in which the malfunction occurred; ii. iii. resultant patient adverse effects; and whether further medical intervention was require enquired. b. define timeliness of complaint investigations until after a subject device is returned to your firm. WL 2/9/15 9
Where do Manufacturers encounter difficulties? What should be in a complaint record? your firm's complaint form does not contain a field for an evaluation for MDR reportable events. WL 7//15 Information recorded on complaint records, was inaccurate, in that the complaint records concluded no corrective action was necessary when, in fact, a Corrective and Preventive Action (CAPA) had been initiated WL 1/29/14 For example, your Customer Complaint Handling and Vigilance procedure, fails to include the requirement for review and evaluation of complaints to determine the necessity of an investigation, or a record to include the rational for not conducting an investigation. WL 12/19/14 10
Where do Manufacturers encounter difficulties? Do complaint records comply with procedures or regulation? Specifically, 10 of the 11 non-complaint records reviewed by the FDA investigator meet the definition of a complaint. WL 2/21/12 Twenty-eight of twenty-eight Customer Service Reports were not adequately completed to include: name of the device, date the complaint was received, or customer address and phone number. WL 12/22/10 Your firm has not analyzed quality data from customer returns, including product return codes, complaint codes, and action codes in the Return Good Authorization (RGA) log to identify existing or potential causes of nonconforming product and other quality problems WL 8/23/12 11
Where do Manufacturers encounter difficulties? Do complaint records comply with procedures or regulation? Your firm failed to complete separate complaint records for complaints involving multiple lots or multiple patients. As a result, your complaint data is inaccurate and incomplete for analyzing trends.. WL 6/4/14 For example: 72% (18 of 25) complaint records including complaints, reviewed during our inspection attribute the root cause of the complaints to wear and tear during service calls, and no investigations were conducted. WL 8/8/14 12
Where do Manufacturers encounter difficulties? When must a manufacturer investigate? Any complaint involving the possible failure of a device, labeling, or packaging to meet any of its specifications shall be reviewed, evaluated, and investigated, unless such investigation has already been performed for a similar complaint and another investigation is not necessary. 820.198c Any complaint that represents an event which must be reported to FDA under part 803 of this chapter shall be promptly reviewed, evaluated, and investigated by a designated individual(s) and shall be maintained in a separate portion of the complaint files or otherwise clearly identified 820.198d 13
Where do Manufacturers encounter difficulties? Are investigations adequate? a. Complaint Investigation Reports (CIR) for Customer Service Reports do not include investigations into the cause of non-conformities related to leaking vests and masques. b. CSR outlined a complaint that reported a ''patient had a very bad reaction to the product." There was no investigation. The CSR indicates that the action or investigation required was a ''refund.'' d. No investigation was conducted for the following complaints that reported missing device components: WL 12/22/102 14
Where do Manufacturers encounter difficulties? What can comprise an investigation? The procedure, Complaint Handling, does not require a review of Device History Records (DHR) for disposable items unless the event is reportable under the Medical Device Reporting regulations, even when the device is returned or the lot number is known. Specifically, complaints received for showed that lot numbers were provided, but DHRs were not reviewed prior to the complaints being closed. WL 1/29/14 The complaint record for complaint further identifies several individuals who were interviewed as part of the complaint investigation; however, there is no record documenting the details of these interviews. 3. Further, your firm subsequently received the suspect unit and conducted testing identifying the pressure relieve valve did not function as intended. There are no records of the dates the testing was conducted or any documentation of corrective actions taken as a result of the complaint and subsequent investigation. WL 12/8/14 15
Where do Manufacturers encounter difficulties? Consider having a disinterested person show where each clause in the Regulation exists in your procedures Be sure to use definitions as they exist in Regulation. If you have one procedure that combines requirements from many countries, be able to show clearly that each requirement is met. 16
Regulatory and Competitive Pressures FDA makes MDRs available on MAUDE 17
MAUDE Search Tools MAUDE data files may be downloaded from: http://www.fda.gov/medicaldevices/device RegulationandGuidance/PostmarketRequir ements/reportingadverseevents/ucm1278 91.htm. These files include more data than is available in the online search tools 18
MAUDE Search Tools Who uses MAUDE? How do you use the data? Who else uses MAUDE? Is MAUDE ever incorrect? Remember, MAUDE data has a number of limitations delineated on http://www.accessdata.fda.gov/scripts/cdrh/ cfdocs/cfmaude/textsearch.cfm 19
MAUDE Search Tools Searching MAUDE will identify User and Voluntary Reports that you may not have otherwise received. Be expansive User and Voluntary Reports may render your company s name in many different ways. Consider whether your company wants to echo User and Voluntary reports with a corresponding Manufacturer report. Consider reviewing MAUDE content for high-priority issues to confirm reports were captured accurately. 20
The Complaint Team: Education and Training Formally Designated Units should understand the product, the product s use, the therapy or diagnosis rendered, the patient and the clinician. Those preparing records should be able to explain the event, including impact to the patient before, during and after the event occurs. 21
The Complaint Team: Education and Training How do you do this? How long does it take to train a new person? What should be included in the training? How much of the training is desk training? How much is hands-on? 22
Overview of the Complaint Handling Process What are we trying to accomplish? Steps in the complaint process Reporting, trending and analysis of metrics 23
Overview of the Complaint Handling Process What are we trying to accomplish? Understand product performance throughout the product s lifecycle Accurately represent product information to internal audiences, regulators, and others 24
Overview of the Complaint Handling Process Receive Complaint Customer Response Regulatory Reporting Review and Update Design / Risk Analysis Documents Complaint Investigation Coding Trend Analysis Corrective Action Returned Product Lab Analysis Verify Effectiveness Model for the post market process complaint handling is central to understanding performance 25
The Complaint Handling Process: Scalable Features Receive Complaint Customer Response Regulatory Reporting Review and Update Design / Risk Analysis Documents Complaint Investigation Coding Trend Analysis Corrective Action Returned Product Lab Analysis Verify Effectiveness What is the product? How many complaints come in each day? 26
The Complaint Handling Process: Scalable Features Can you build system tools to prepare records for evaluation? For edit checks prior to closure? To identify similar complaints or prior complaints for the same device or devices with the same production lot number? 27
The Complaint Handling Process: Scalable Features Which steps require lower- or higher-level skills? Are there many regulatory reports? Should report decisions and preparation take place in-line or with a separate group? Can some of these be outsourced or take place offshore? 28
The Complaint Handling Process: Possible Structures Partitioning complaint receipt can permit focus on preparing records for detailed review. Combining Investigation, Coding and Regulatory Reporting may be effective if a company files large numbers of MDRs. Close proximity to Lab Analysis will help with accurately tying analysis results to customer descriptions. 29
Complaint Capture and Processing Complaint Capture depends on accurate information from complaint sources. Manufacturers may receive information from field support staff, service staff and repair depots, clinicians, and other sources including: Marketing, Focus Groups Customer Service Newspapers, Technical Publications, Social Media Lawsuits Clinical Studies /Post Market Studies / Marketing Studies 30
Complaint Capture and Processing All employees must be trained on how to recognize a complaint and what to do with that complaint. Training can differ based on the employee s role and proximity to clinical use. Training to identify a customer complaint vs an inquiry, order or scheduled maintenance Consider using examples of what should or should not be forwarded. 31
Complaint Capture and Processing Tools can be set up to promote compliance: What information is required by the complaint group? Design smartforms or software tools to walk through a field event using workflow and language familiar to the field staff. Set decision points so that the field staff must enter only the required information Interfaces eliminate transcription errors, provide for data accuracy Consider systems to submit incomplete reports after a certain time interval. 32
Complaint Capture and Processing Raw information from the field may require merging with internal information, including: Prior complaints for that individual medical device Patient implant information Returned product analysis or field service for that device 33
Complaint Evaluation All complaints must be evaluated. Evaluation includes assessing whether the complaint is valid, an investigation is required or regulatory reporting is necessary. 34
Complaint Evaluation Investigations are necessary for: (c) Any complaint involving the possible failure of a device, labeling, or packaging to meet any of its specifications shall be reviewed, evaluated, and investigated, unless such investigation has already been performed for a similar complaint and another investigation is not necessary. 21CFR820.198c (d) Any complaint that represents an event which must be reported to FDA under part 803 of this chapter shall be promptly reviewed, evaluated, and investigated by a designated individual(s) and shall be maintained in a separate portion of the complaint files or otherwise clearly identified. 21CFR820.198d 35
Complaint Evaluation What comprises an investigation? 36
Complaint Evaluation / Root Cause Analysis How do I investigate a complaint if the product is not available or not returned? Refer to information that may be obtained from: Manufacturing records for that device Prior complaints for that device Similar devices that may be within your control, e.g. reserve samples 37
Complaint Evaluation / Root Cause Analysis A structured analysis process can help with the investigation. Quality tools such as Five Whys, Is/Is Not, and other elements can help. Investigation depth can be tailored to the type of field event. Defining the appropriate depth can help with application of resources. 38
Reporting, Trending and Metrics Metrics fall into two categories: Process Performance, including Incoming complaint volume, including volume by geography Time to milestones in the complaint process, including initial assessment, coding, regulatory reporting, customer response and closure Complaint WIP and Aging to include median age and age of oldest complaint Accuracy as shown by assessments of complaint records Product performance 39
Reporting, Trending and Metrics Metrics fall into two categories: Process Performance Product performance, including Product reliability Failure rates Run charts of as-reported and as-investigated Remember to use alert and action limits Signals should be traceable to some action Pareto charts of leading failure modes 40
Reporting, Trending and Metrics Trending has three objectives: Is it good enough? Is it getting better or getting worse? See design predictions, look for gradual rise or spikes Identify predominant causes of nonconformance What is causing this? Review Pareto charts and FMEA, determine if it is an early issue or a wearout problem Provide triggers for further action Do I need to do something? Run rules, Weibull predictions 41
Reporting, Trending and Metrics Have a Structured Process for Trending Define Who looks at the data Define how Frequently you will look at data Define your Criteria for initiating action Count of similar observations Rate of occurrence Risk analysis Define the next Activity you will undertake Define how you will Document the investigation Review progress at cross-functional meeting 42
Links to Risk Management ISO14971 provides guidance on applying risk to medical devices. Risk may be used for determining risk/benefit or setting triggers for taking action. New failure mode? Unanticipated changes in rates of occurrence or severity? ISO14971 requires manufacturers to collect and review information about medical devices in the production and post-production phase (ISO14971:2012) 43
CAPA All complaints require some assessment for Investigation; however, systemic or high severity issues may have to be elevated to a CAPA. CAPAs will have a rigorous process for determining root cause, action plans, dates, and effectiveness checks. CAPA review boards will monitor CAPAs as they work through the process and confirm the adequacy of actions. 44
Documentation Documentation should stand on its own, including procedures, complaint records and CAPA records. An SOP documenting the overall procedure can be valuable during inspections, when training new staff, and for other audiences outside the complaint department. Complaint records should be laid out to show a clear path from receipt to resolution. CAPAs / trend reports can frame the issue, bound the affected population, track investigation, identify actions to take and verify effectiveness. 45
Process Controls Variation can be minimized using tools: Decision Charts MDR decision trees MDR list of common situations System screen layout to promote a workflow System error checks, whether in-line or before record is closed Pivot tables: Compare specific codes to MDR decisions Compare as-reported codes to failure mode. Text Analytics to identify records with known keywords or to identify patterns 46
Expectations (a) Each manufacturer shall maintain complaint files. Each manufacturer shall establish and maintain procedures for receiving, reviewing, and evaluating complaints by a formally designated unit. Such procedures shall ensure that: (1) All complaints are processed in a uniform and timely manner What is timely? I know it when I see it Justice Potter Stewart, Jacobellis v. Ohio 47
Additional Topics How do other companies do this? How do I know I have the right number of staff? When things go wrong Electronic MDRs The future 48
How do Other Companies do this? Compliance Alliance has surveyed Medical Device manufacturers on several occasions. These surveys focus on post-market activities. 103 responses were received from companies with <100 to >50,000 employees <5 to >25 FTEs directly involved with complaints <100 to >5,000 complaints per month Survey results are available at http://www.compliance-alliance.com/free-documents-andwebsites/complaint-handling/ 49
Compliance Alliance Survey Results 50
Compliance Alliance Survey Results 51
Compliance Alliance Survey Results 52
Compliance Alliance Survey Results 53
Compliance Alliance Survey Results 54
Compliance Alliance Survey Results What kind of efficiencies have you introduced in the last 12 months (partial list) Developed tools to assist with trending and escalation to CAPA; Established a single standardized procedure for complaints Trending is performed based upon normalized data for both disposable and repairable equipment Complaint file quality improvement Utilization of SAS for Pareto and trending Combining Complaints Handling with Customer Service - one single entry point f or all complaints into the system Standardized and Harmonize the complaint handling department across multiple business units. 55
Staffing Questions How many people do I need? Consider capacity studies for staffing Ask managers to document how many staff are required to do current tasks. Subdivide tasks to a meaningful level (probably <10 segments); understand that tasks will vary with the event. Know how many tasks will require reporting, follow up and other paramaters. 56
Staffing Questions What qualifications are required? Sufficient to perform the work product reporting specialist was making decisions about MDR reportability for the... The training record for this particular employee showed that this person only had a high school diploma with some additional in-house training. WL June 1, 2009 Qualifications can match the work to be performed Firms ARE NOT required to have every MDR report reviewed by a person qualified to make a medical judgment and/or a person with a medical degree or training. Compliance Program Guidance Manual 7382.845 57
Staffing Questions How do I train the staff? Training must extent beyond procedure compliance; consider training on product, patient, disease process, therapy provided, and relationships with field, product analysis and engineering activities. Don t forget training on regulations. Testing may provide objective evidence that the specialist is qualified to perform the role. 58
When Things go Wrong Manufacturers may encounter situations in which a product issue expands widely. This may result in field actions with attendant increases in complaint volume sometimes two or three times baseline levels. Staffing may have to be increased quickly and work processes may be re-examined to ensure compliance needs are met for this higher volume. 59
When Things go Wrong Handle similar events consistently Develop standard language for events and MDRs within an advisory and events documenting field assessments Explore where automation can be used, but ensure a specialist will document investigation and reporting decisions and event closure. Assign data requests to a small number of staff. Retain query criteria and output files. 60
When Things go Wrong Re-evaluate your workflow Segment work to permit training for temporary staff Assign training and monitoring activities to technical leaders in the group. Implement stringent quality checks for work completed by temporary staff Assess metrics to ensure that work does not accumulate mid-process and perform reconciliations to ensure work does not get lost. 61
When Things go Wrong Assign trend or similar numbers to fields within an advisory. Run queries periodically to confirm similar coding, reporting decisions and language where appropriate. Generate and keep pdfs of all MDRs submitted. This will be easier if you are using emdr. You will receive many requests from regulators, legal and others for sets of MDRs. Similarly, generate and retain pdfs of all correspondence with FDA and other regulators. 62
When Things go Wrong Create queries with information that has been sent to FDA, including record number, MDR number and submission date. Review periodically for consistency. Review data on MAUDE periodically to ensure: Information in MAUDE records is accurate compared to submitted information You have captured all user reports. 63
When Things go Wrong Develop position paper on MAUDE analysis. Remember that MAUDE data cannot be used to compare data among manufacturers due to differences in Language used Reporting Criteria Reporting timelines FDA MAUDE update schedules Summary vs individual reports and loading supplemental reports in MAUDE 64
Electronic MDRs Electronic Manufacturer MDR reporting will be mandatory in August, 2015. Information is available at: http://www.fda.gov/medicaldevices/deviceregulationandguid ance/guidancedocuments/ucm175805.htm http://www.fda.gov/medicaldevices/deviceregulationandguid ance/postmarketrequirements/reportingadverseevents/emd R%E2%80%93ElectronicMedicalDeviceReporting/ucm201932 7.htm Consider applying for low volume solution as a backup even if you are using the HL-7 system 65
Observations on working with FDA Focus on the customer: What does FDA want? Enough detail so that they don t have to follow up with questions Enough information so that they can fulfill their mission to protect the public health Lessons from Inspections Review the Process as the inspection begins Make your Procedures simple and understandable Make your Investigation and Reporting Rationale clear Success means handing files to investigators for review and getting them back with no follow-up questions. 66
The Future http://www.fda.gov/downloads /MedicalDevices/Safety/CDR HPostmarketSurveillance/UC M348845.pdf FDA and others are working to improve the data and methods used to monitor product performance. This assessment proposes actions focused in four areas: Implementing Unique Device Identifiers (UDI) Registry development Modernizing Adverse Event Reporting and Analysis Develop new methods for evidence generation and synthesis 67
The Future Proposed Mission Statement: The National Medical Device Postmarket Surveillance System (MDS) supports optimal patient care by leveraging the experiences of patients to inform decisions about medical device safety, effectiveness, and quality in order to promote the public health. Goals: Coordinate medical device postmarket activities to ensure there is a harmonized http://www.fda.gov/downloads national approach focused on improving /AboutFDA/CentersOffices/Off evidence and reducing burden. iceofmedicalproductsandtoba Build and facilitate access to data partners cco/cdrh/cdrhreports/uc using the emerging national electronic M435112.pdf information infrastructure. 68
Any Questions? 69