33rd SQA Annual Meeting, National Harbor, MA, USA Session C // Presentation C 2 28 March 2017 // 1:30 3:00 PM // V1.0

Transcription

1 Computerized Systems Using Artificial Intelligence (AI) Components in a Regulated Environment: Implications on and Challenges of Establishing a Validated State René Kasan NNIT Switzerland AG Zürich, Switzerland Manu Reddy Bayer US, Whippany NJ 33rd SQA Annual Meeting, National Harbor, MA, USA Session C // Presentation C 2 28 March 2017 // 1:30 3:00 PM // V1.0

2 Disclaimer Views and opinions expressed in the following powerpoint slides and presentation are those of the individual presenter and should not be attributed to any organisation with which the presenter is employed or affiliated. These powerpoint slides are the intellectual property of the individual presenters and are protected under the copyright laws of the United States of America and other countries. Used by permission. All rights reserved. Slide 1

3 Agenda Objectives What s In It For Me? Compendious View New Information Age? (incl. AI & Pharma Use Cases) Validation Discussion of Stumbling Points and Options Selected Consolidated Insights Conclusions Slide 2

4 Objective Understand Validation Challenges when Using AI Components Foster and Provoke Thought Processes (What can I reckon with soon, how is it different and how to deal with) Possible options and Potential Solutions Learn Tease & Think Solve & Share Slide 3

5 What s In It For Me? I am. QA & IT Professional QA & Non-IT Professional CONFIRMATION Experienced CONTRIBUTION IT RELATED PERSPECTIVE THOUGHT PROVOKING AI PERSPECTIVE Newbie AI PERSPECTIVE THOUGHT PROVOKING SENSITIVITY THOUGHT PROVOKING Slide 4

6 Compendious View The New Information Age Digitalization I ll be back! Artificial Intelligence Support Automation Slide 5

7 Compendious View The New Information Age I release the manufactured batch of drug substances automatically dependend on what you have taught me OMG.They have taken over! I exchange the post approval site extension submissions for the drug substance automatically and get approval via the agency s block chain I autonomously consolidate for all ISO IDMP data points based on the input of the various transactional systems thus creating a new electronic record I make decision support suggestions, which are then approved by a human being for instance on how your clinical study reports shall be written based on the available data Slide 6

8 Compendious View Artificial Intelligence (AI) & Machine Learning (ML) What is AI (Simplified)? A broader concept of machines being able to carry out tasks in a way that we would consider smart Neuronal networks What is ML? The capacity of a computer to learn from experience Permutating algorithms and knowledge bases built up by interactions but within limits of set parameters Slide 7

9 Compendious View Artificial Intelligence & Machine Learning Why AI? Ability to process huge & complex data High degree of data connections from various sources (big data) Usage of AI most likely unique to the scoped, targeted area of application Speed and reliability Total Cost of Ownership & Enabling human brains to focus on areas needed Knowledge generation & transfer & protection Slide 8

10 How AI: Compendious View Artificial Intelligence & Machine Learning Given Process Steps in a Value Chain Regulated Activity Conducted by Human Being Regulated Activity Conducted by Machine Regulated Activity Jointly Conducted High Impact vs. Low Impact Activities Slide 9 Flow of Business Activities

11 Compendious View Selected Use Cases in Regulated ( GxP ) Areas Area: Application Type: Mode: Use Case: Drug Discovery New Drug Hypotheses On premise installation (potentially cloud) Decision making or decision support Depends on the variety of databases and journal publications // ability to correlate, assimilate and connect all the data Slide 10

12 Compendious View Selected Use Cases in Regulated ( GxP ) Areas Area: Application Type: Clinical Automated Report Writing Mode: On premise installation (potentially cloud) Decision making or decision support Use Case: Depends on architecture design // decision support through full automation of record creation Slide 11

13 Compendious View Selected Use Cases in Regulated ( GxP ) Areas Area: Application Type: Mode: Use Case: Data Management Medical records mining On premise installation (potentially cloud) Decision making or decision support Autonomous drafting of medical reports Slide 12

14 Compendious View Selected Use Cases in Regulated ( GxP ) Areas Area: Application Type: Regulatory Affairs // Reg Operations Data mining in text based knowledge Mode: On premise installation (potentially cloud) Decision making or decision support Use Case: Data mining for ISO IDMP data points in MS Word/ PDF CMC documentation (module 3) Slide 13

15 Compendious View Selected Use Cases in Regulated ( GxP ) Areas Area: Application Type: Regulatory Affairs Programming language add on for middleware using AI for data translation Mode: On premise installation (potentially cloud) Decision making or decision support Use Case: Creation of middleware data pipes consolidating various contradictory sources to dedicated data points (e.g., ISO IDMP) Slide 14

16 Validation Discussion of Stumbling Points & Options Overview Specification Electronic Records Design Controls Migration & Archiving Robot Traceability Audit Trail Environment Qualification Production Controls Slide 15

17 Validation Discussion of Stumbling Points & Options Specification Challenge Iconized Requirements Approach Powerful Enough for I O Specifications? Impacted CSV Area User Requirements, Functional/Configuration Specifications Options More Intuitive I O Specification via Scenario Based Documentation Including Ranges for AI Components (FS, CS) Slide 16

18 Validation Discussion of Stumbling Points & Options Design Controls Challenge Output Control of AI Components? AI Knowledge Still In Line with Specifications? Impacted CSV Area Functional Specifications, Software/System Design Specifications Options Redundant & Independent AI Component Architecture Output Comparison to Reduce Likelihood of Unwanted Results Slide 17

19 Validation Discussion of Stumbling Points & Options Traceability Challenge How Valid is my Traceability Information Pointing to Test Evidence that is Created Using AI Components? Impacted CSV Area Traceability Matrices Operations SOPs Options Work with Learn& Freeze AI Components Provoking: How do we Handle the Human Brain in the TM? Slide 18

20 Validation Discussion of Stumbling Points & Options Environment Qualification Challenge How can I sufficiently Qualify a Sequence of Controlled Environments Containing Self Permuting AI Components? Impacted CSV Area Controlled Environment Qualification/Deployment VAL, PROD, (TRAIN) Options Check with Technology Provider on Knowledge Transfer Options Conduct Standard Training Interaction Sequence Before Use Slide 19

21 Validation Discussion of Stumbling Points & Options Production Controls Challenge Level of Confidence During Runtime in Production that AI Supported System Runs Within Specifications Impacted CSV Area Operations SOPs Monitoring, Training Business Process Design Controls Options Business Monitoring Controls of I O // (Re ) Training of Human and Machine // 4 Eye Machine Human Principle Slide 20

22 Validation Discussion of Stumbling Points & Options Audit Trail Challenge AI Component Changes Controlled Record and Must be Captured as Actor Impacted CSV Area User Requirements Audit Trail Software/System Design Specification Options Design Specification on Unique & Allowed AI Actors in System AI in Non Learning Mode: Capture of Frozen Knowledge Data Slide 21

23 Validation Discussion of Stumbling Points & Options Migration and Archiving Challenge How to Long Term Archive AI Supported Systems? How to Migrate AI Knowledge Upon System Paradigm Shift? Impacted CSV Area Operations SOPs Archiving Data Migration Plan, (Machine) Training Plan Options Archive Entire System and Data Including Operating Environment Check with Technology Provider // Manual Training Approach Slide 22

24 Validation Discussion of Stumbling Points & Options Electronic Record Challenge How to Treat the Knowledge of the Learning AI Component? Impacted CSV Area System Impact Analysis, User Requirements, Functional Risk Analysis Options Yes: Audit Trail and Record Retention Dilemma No: Provoking We do not Store the Human Brain Either Slide 23

25 Selected Consolidated Insights Analyze AI Candidate Area Consider Scope and Evidence Creation Consider & Ensure Data Lineage and Traceability Leverage Application Layer Monitoring Control Think of Exit Options and Implications Clarify Environment Qualification Options & AI Knowledge Transfer Contemplate Controlling AI Components like Human Beings Slide 24

26 Conclusions Objectives: Learn Tease Think Solve Share Flow: Stage Discuss Advise Take Away CSV Traditional: CSV New: Human & Computer Learning Computer Take Away Similar Controls for Learning Computer and Human Being Take Away The Journey has Started It will Come Slide 25

27 Thank You Questions? Slide 26

28 Contacts René Kasan Subject Matter Expert Regulatory Compliance Manu Reddy Sr. GxP IT Auditor GxP IT Audit Management NNIT Switzerland AG Bändliweg 20 CH-8048 Zürich Switzerland Bayer US LLC 100 Bayer Blvd, P.O.Box 915 Whippany, NJ U.S.A (direct) (mobile) (direct) (mobile) Slide 27