1 Data Integrity in Pharma QC Labs What You Need to Know Richard Mutkoski Agilent Technologies 1
2 The current regulatory environment in historical context Myths and critical thinking skills Auditing your software vendors Evaluating your systems How software vendors are (or should be) responding 2 Data integrity problems in pharmaceutical quality control laboratories are driving more regulatory action than ever before. It is obvious that something has changed to drive all this activity. While there is plenty of information available, much of it seems to confuse or frustrate rather than clarify or help. This article will provide clarity to (hopefully) dispel common myths by looking at facts based on a study of available resources and direct interactions with US Food and Drug Administration (FDA) staff and their consultants. It will discuss how: - to put the current enforcement environment into historical context - to apply critical thinking skills to various myths regarding data integrity - to evaluate current laboratory software and associated processes against these new expectations - vendors are redesigning laboratory software to help respond to these new realities
3 Data integrity defined In the context of laboratory data integrity within a GMP environment, this can be defined as: generating, transforming, maintaining and assuring the accuracy, completeness and consistency of data over its entire life cycle in compliance with applicable regulations. - R.D. McDowall, Ph.D. Scientific Computing, September 2013 3 In a recent article in Scientific Computing (September 2013), Bob McDowall defined data integrity as: generating, transforming, maintaining, assuring the accuracy, completeness, and consistency of data over its entire life cycle in compliance with the applicable regulations.
4 Myth: New data integrity regulations Reality: 1998: 21 CFR Part 11 released 2003: Scope and Application 2010: Data integrity inspection focus announcement (FDA hiring and training) 2013-present: increase in visible enforcement activity More reality: More inspectors are trained Data integrity is now a primary inspection point FDA is now naming (vs. redacting) products and systems While it is a common myth that the applicable regulations are new, 21 CFR Part 11 Electronic Records; Electronic Signatures was first released in 1997. In 2003, after the pharmaceutical industry spent a few years struggling with the regulation, and after discussions between industry and the FDA, the FDA released their Scope and Application guidance document, clarifying some of the requirements in Part 11. This guidance also included a specific discussion regarding the FDA s selective enforcement strategy based on what they were finding during their inspections. In 2010, the FDA announced their data integrity inspection focus. At that time, there were very few people within the Agency that were qualified to understand data integrity aspects of computerized systems. The result was a real conundrum; it was difficult to have a focus on data integrity inspections without enough skilled people to do the inspections. Consequently, the Agency started a significant training exercise in data integrity, data integrity inspections, and regulations for personnel that included chemistry experts, manufacturing experts, and people with other GxP expertise. They also did some very strategic hiring of experts in fraud investigation, including people from the US Federal Bureau of Investigation. Thus, beginning in 2013, data integrity has been a primary inspection point, and there has been a visible increase in data integrity enforcement activity, not only in India or China, but across all geographies. In addition, starting in 2014, as a result of these inspections, the FDA often includes names of hardware and software products and systems in their warning letters and related public information documents in a less than subtle message to the hardware and software manufacturers that the Agency has begun expecting them (the hardware and software manufacturers) to provide their customers help and assistance with data integrity and compliance concerns.
5 Myth: If you are using xyz software, the FDA will shut you down Reality: Capability violation > 130 mph 5 Another myth commonly heard is that if a pharmaceutical company is using a particular software system, the FDA will shut them down. The question often takes the form of, Systems throughout manufacturing organizations and laboratories may have potential weaknesses that could be considered a data integrity risk. If that weakness has not been exploited, does the FDA have grounds for delivering an observation in a 483 warning letter? The very clear answer is no. Capability does not equal violation (this is a sometimes contentious point worth further discussion). The analogy is that, while the speed limit throughout much of the USA is 65 miles an hour, the car I drive has the ability to exceed 130 miles an hour. However, officers must observe a speeding infraction, via direct witness or electronic means, in order for them to issue a citation for violating the speed limit. My car s ability to exceed 65 miles per hour does not justify a citation. Similarly, in a pharmaceutical company, if potential data integrity weaknesses have not been exploited for data manipulation or deletion of data (or other fraudulent activities), then there is no basis for the agency to issue a citation, or a 483 warning letter. One should expect a detailed conversation with the inspector, as the focus will turn to procedural controls to ensure that the known weaknesses are not exploited.
6 Myth: This software is Part 11 compliant Reality: This is a logical impossibility Example: 11.10(j) written policies regarding electronic signatures Software does not comply with regulations Software is, in fact, inert Firms use software to support their compliance with regulations Software contains technical controls to support compliance with applicable regulations 6 Let s consider the statement: This software is Part 11 compliant. There are a few problems with this statement. First of all it is a logical impossibility. There are components of Part 11 that are not meant to be satisfied by technical controls within a computerized system. For example, CFR Part 11.10(j) requires policies for the use of electronic signatures. This is a requirement that a chromatography data system (for example) is not going to satisfy. It is an element of the regulation, but it is not something that is expected to be a technical control. Software, in fact, does not comply with regulations. Software itself is inert; software contains the technical controls to support compliance with the applicable regulations. In addition to technical controls, procedural controls must also be in place. A discussion about procedural controls versus technical controls is seen quite often in FDA Warning Letters, particularly when there are exploitations of gaps in a system s ability to support technical controls required by various regulations.
7 The world has changed Procedural controls vs. technical controls Procedural controls require: SOPs must be established People must be trained on SOPs SOPs must be followed Records must exist Adherence to SOPs must be confirmed Technical controls must: Exist Be activated 7 A standard operating procedure (SOP), used as a procedural control, can substitute for technical controls as long as: - procedural controls/sops themselves exist, - people are trained on those SOPs, - the SOPs are followed, and - adherence to the SOPs is confirmed by quality oversight and/or compliance auditing activities. Very often, however, even if SOPs exist, they are not followed, and adherence isn t properly verified. Consequently, the FDA will demand system remediation in order to prevent a recurrence of the behavior. Audit trails within computerized systems are one example of technical controls. The software must have the ability to generate audit trails that contain all the components the regulations require, and those controls must be enabled. Audit trail records automatically (21 CFR Part 11.10(e)) show that that system is working and it s doing its job properly, and can be consulted in an audit or an inspection without any extra work by the humans.
8 Myth: The vendor provided Certificates of software validation/21 CFR Part 11 Readiness Reality: A Certificate of Software Validation is of little value, given that FDA expects the software to be validated for its intended use by the user in the environment where it will be used. More reality: While a vendor may provide detailed information about their product s ability to support compliance, that information and software solution is still based on the vendor s interpretation of the regulations. 8 Many software vendors provide a Certificate of Software Validation, or they may issue a certificate that is titled along the lines of 21 CFR Part 11 Readiness Claims. The reality is that such a certificate that you might receive from your software vendor is of extremely limited value, given the fact that the FDA expects the software to be validated for its intended use by the users in the environment where it will be used. While vendors should engage customers in order to build and design systems according to customer s needs, and usually spend considerable time testing that software before they deliver it, the development and testing work does not (and cannot) substitute for the customer declaring their intended use and then validating their system according to that intended use. Another related, interesting point is that the US FDA does not have legal jurisdiction over (for example) a vendor s laboratory informatics software organization. Therefore, (unless the software is registered as a Medical Device with the FDA) any documentation that they might provide cannot be recognized by the FDA because of that lack of enforcement authority. In addition, vendors may be able to provide detailed information about their products abilities relative to Part 11, however, that information and the software solution that they provide is based on the vendor s interpretation of the regulations. This interpretation may or may not agree with various pharmaceutical manufacturers, or the FDA itself. The vendor s interpretation cannot substitute for an audit to determine the software s functional ability to satisfy regulatory concerns.
9 Your responsibilities Audit your vendors Assess your systems for compliance support Validate your system 9 Data integrity compliance responsibilities include vendor audits, a computer system assessment, and software validation.
10 Auditing your vendor: Apples and Oranges* (Jacques Mourrain, Ph.D.) The problems with many auditors ( fourfold dilemma ): Disparity Regulations are few. Guidelines are many. Interpretation is the problem. Partiality Audit reports are partial (by this I mean incomplete); splitters vs lumpers Variability Auditors are humans too. Some auditors obsess over disaster recovery while others are blinded by Part 11. Legibility While the auditor may communicate the issues found, can they communicate the risks to patient safety, product quality, or data integrity? 10 When auditing vendors, there is a fourfold dilemma, or a set of problems that are actually focused on the auditors themselves (See Mourrain, J., Therapeutic Innovation and Regulatory Science, Volume 40 (#2), pp. 177-183.). The first dilemma is disparity. Regulations are few. The Part 11 regulation, for example, is a grand total of three pages, not counting the information in the Preamble section of the 1997 Federal Register entry. Guidelines are many, and interpretations of the guidelines are even more plentiful. The disparity of interpretation is the problem in many auditing situations. For example, a customer during an audit might see a regulation a certain way, whereas the vendor may look at the regulation differently. Partiality is also an issue with auditors. Audit reports are partial, meaning incomplete. There are certain auditors that feel they need to have a large number of audit findings. As a result, they will have separate findings for every problem in every SOP, and they will list all of them separately, creating the potential misconception that the audit report is good simply based on volume. Other auditors may take examples and put them into a very few large observations in order to support their case for a major finding in an audit. It is possible that neither of these types of reports will tell the whole story of what was learned during the audit, and it is guaranteed not to uncover all issues in the target of the audit. Another auditor issue is variability. Auditors, to no one s surprise, are human. Some auditors will obsess over certain subjects such as disaster recovery, while others are consumed with Part 11. Legibility (not in the handwriting sense) is next (but perhaps not last). The auditor may
be able to communicate the issues that they found during an audit, however, they are often challenged to communicate the so what? ; the gap in the computerized system and its impact on patients, to the product, and to data integrity. 10
11 Auditing your vendor: Apples and Oranges* (Jacques Mourrain, Ph.D.) The model (not a checklist): Procedures: coverage, maintenance, reviews, current Training and personnel: evidence of training, independence of QA Infrastructure: operation, maintenance, disaster planning, security (often irrelevant unless the vendor is housing your GxP data) Software development: SDLC, deliverables, reviews Testing: throughout SDLC, complete, robust, traceability Quality management systems: configuration management, change control, problem tracking, anomaly analysis, CAPA Scoring models in these six areas support defensible individual and comparative evaluation (scoring) of GxP software and service vendors as well as inform risk-driven validation strategies. *Therapeutic Innovation & Regulatory Science April 2006 vol. 40 no. 2 177-183 11 What is the solution to these audit dilemmas? The solution is a model, rather than a checklist, the components of which include: - procedures, - training of personnel, - software development activities, - testing activities, - quality management systems, and - infrastructure. The latter is often irrelevant unless the vendor takes custody of the GxP records in, for example, a cloud-based application. Using a model approach, scoring can transform a fairly subjective process into an objective measurement system that supports a defensible individual vendor audit, as well as comparative evaluations between multiple software or service vendors. The point of all of this activity is that the vendor audit can contribute to a risk-based validation strategy (a la FDA s General Principles of Software Validation, Section 6.3): the better job a vendor is doing, (theoretically) the less work required during software validation.
12 Assessing your software s support for compliance Should consider all regional regulations where your company does (or may want to do) business Should be based on evidence (vs. vendor responses) Areas for review: Validation Accurate copies, secure retention and retrieval of records Authorized access to systems, functions, and data Electronic audit trails Operational and device checks Date integrity, date and time accuracy Controls for open systems Electronic signatures Identification codes and passwords System development and support 12 Assessing a software s compliance support ability requires attention to all of the regional regulations where a regulated company does business or may intend to do business. Some regulated companies, if they are solely doing business within the US, or solely within Europe, may choose to pay attention only to CFR Part 11, or only Annex 11 requirements. 21 CFR Part 11 and Annex 11 share commonalities. However, any software evaluation for compliance should be based on evidence rather than hearsay (that shiny Part 11 Certificate). Agilent, like many other vendors, receives numerous customer checklists, and provides straightforward responses with product answers. However, it is important that the evaluation of the responses be based on evidence, rather than being strictly limited to what the vendor has said the system will do. Areas for review should include data integrity issues, access controls, audit trails, device checks, etc., as per applicable regulations. This review is valuable during a software assessment process because observed gaps indicate where procedural controls or customization may be required. Feedback to the vendor regarding any gaps observed is therefore important so that the procedural controls or custom solutions do not become permanent.
13 Myth: I bought the vendor s IQ/OQ package Reality: Buying the vendor s validation package is generally insufficient to establish validation for intended use. - Monica Cahilly, Workshop on Data Integrity and Industry Practice June 2015, Peking University, Beijing 13 Another common software compliance support myth is, I bought the vendor s IQ/OQ package implying that, therefore my system is validated. However, buying the vendor s validation package (traditionally limited to a basic software IQ and a core, generic OQ) is generally insufficient to establish validation of the system for its intended use. Validation responsibility cannot be abdicated to vendors, but they are (or should be) a reliable source of information and assistance for your validation effort.
14 Myth: Any system change requires full revalidation Reality: Whenever software is changed, a validation analysis should be conducted not just for validation of the individual change, but also to determine the extent and impact of that change on the entire software system. - General Principles, Section 4.7 14 A related validation myth that is, Any system change requires full revalidation. The reality is that whenever software is changed, the software change needs to be evaluated to determine the extent and impact of the change on the entire software system, and the risk of that change to patient safety, (drug) product quality, or data integrity. Very often, companies will limit their change validation to the change only, or they will go to the other extreme and will needlessly repeat the entire validation. The right answer is somewhere in between.
15 Agilent s Goal Agilent will not ship software that introduces or maintains regulatory exposure for our customers. - Loren Smith Remember, you too are (or may become) a patient! 15 Agilent s goal is to avoid shipping software that introduces or maintains regulatory exposure for our customers, by incorporating regulators priorities into our system designs. The US FDA is clearly pushing for more technical controls, prioritizing technical controls over procedural controls, and prioritizing prevention controls over detection controls. 21 CFR Part 11.10(a) talks about systems needing to have the ability to detect invalid or altered records. That is the only part of the regulation that actually talks about detecting records that may have been manipulated through nefarious means. The remainder of the controls are preventive. So yes, detection controls are important, but prevention controls are better. Prioritizing online records over hard copy printouts has also been emphasized. The FDA cannot always trust the paper that s being handed to them.
16 How Agilent is responding: system design Prioritize technical controls over procedural controls Prioritize prevention controls over detection controls Prioritize on-line records over hard-copy printouts 16 Another example of system design is the use of an online audit trail review, as illustrated in Figure 1 (Slide 17). There are a lot of systems that may allow the generation of audit trail reports in printed form, but Agilent s new CDS version has a built in tool that allows a user to review electronic audit trail entries. These audit trail entries are organized by type, and an online review can be performed, and electronic signatures incorporated.
17 Quote of the day Mess with the science, and it s no longer science. - Peter Baker, USFDA Workshop on Data Integrity and Industry Practice June 2015, Peking University, Beijing 17 Pharmaceutical manufacturers need to pay attention to system design, implementation, and validation. They also need to pay attention to a corporate culture that may drive or tolerate fraudulent behavior.
18 The current regulatory environment in historical context Myths and critical thinking skills Auditing your software vendors Evaluating your systems How software vendors are (or should be) responding 18
19 Questions