E550I Bayer Case Study: Manufacturing Excellence in the Biotech Industry. David Kavanaugh, PCS Engineer Berkeley, CA

Transcription

1 E550I Bayer Case Study: Manufacturing Excellence in the Biotech Industry David Kavanaugh, PCS Engineer Berkeley, CA ABB Automation and Power World Presentation 2012

2 David Kavanaugh Process Control System Engineer for Bayer HealthCare, Berkeley CA PCS since 1978, when huff and puff was still King Oil, Gas, Chemical, Food, Textile, Nuclear, Pulp/Paper, Power, Pharmaceutical and others I don t remember First ABB (Taylor MOD 300) exposure 1989 at co-gen plant in Rocklin, CA ABB Resident Engineer at Bayer 1995 Bayer PCS Engineer since 2001 Current duties: Data and Process Visualization and ABB Support backup

3 Bayer, Berkeley CA

4 Kogenate: a Factor VIII blood clotting agent for persons with hemophilia, a blood clotting disorder

5 Bayer and ABB: Twenty-four years of Partnership First MOD 300 System: Building Two MOD 300 Systems: Buildings 52 & 5A Two MOD 300 Systems: Buildings 57 & MOD 300: Building B55 converted to Process Portal B B60 converted to Process Portal B New PPB system for new Building Buildings 49 and 5A converted to PPB xA Site Wide Historian New 800xA system for new Building B60 and B81 evolving to 800xA and Batch Enterprise Connectivity Solution for Manufacturing Execution Systems (Electronic Bach Records) M Controllers and Increased Batch Implementation Evolved B55 Process Portal B system to 800xA 5.1 with PG2 graphics

6 EFFICIENCY & FLEXIBILITY is now a Strategic Requirement for Plant Automation

7 EFFICIENCY & FLEXIBILITY is now a Strategic Requirement for Plant Automation Systems Yesterday Low level of Automation (no closed loop, advanced control) Poor standardization Automated sequences were hard coded (not flexible) No, or difficult documentation of programming code. No, or difficult, interconnection between different manufacturers Hard or impossible to automatically get data from a system Systems Today Process analyzers and advanced control implemented Use of OPC and standard 3rd party drivers Electronic Batch records with recipes selected through MES/ECS systems Self-documenting tools and auditing Systems integrated in a site wide Network among different Plants/ Utilities/ Buildings and with IT (MES Electronic BPR, integrated Paging, Remote Maintenance, Reporting,...) Data automatically transmitted, web services

8 How Did We Get There

9 Process Control System Lifecycle Options Risks Rewards Maintain Lack of New Parts No Capital Expenditures Higher Maintenance Costs Familiarity Unable to meet strategic functionalities No new training required Rip & Replace Production down time New Features (Vertical*) Large capital expenditure Longevity Increase training Lower maintenance costs Evolve Increased time to completion New features (Phased*) Higher costs over time Flexibility Schedule Less time pressure *ARC Strategies: The Control System Migration Survival Guide, March 2010, Larry O'Brien & Dave Woll

10 We Chose to Evolve

11 How Will We Evolve? Changes are occurring through concurrent multiple projects: MES, DCS upgrades, batch implementation and others Learn the new skills required for new technologies. Add new functionalities to our systems (connections to other systems, remote desktop, advanced control, etc.) Hardware and software changes, standardize across systems Partner with internal and external suppliers

12 Automation Projects New site-wide DCS platform to integrate existing DCS systems. Upgrade to latest 800xA version will allow the integration. Integration to MES for Electronic Batch Record following the ISA S95 standard Will add new small scalable controllers for on-skid installation (800M Controllers) while utilizing existing controllers/logic for remainder of plant Flexible integration connector (ECS) will also allow future integration with other Control systems (BAS, SCADA) and IT systems (cmms, SAP, LMS,..) Open standard automation interfaces (OPC, Fieldbus) to be less dependent on a single vendor and to be ready for PAT integration

13 Automation Projects Introduction of ISA S88 Batch in API manufacturing will prepare the basis for Berkeley to became a multiple-products site, through flexible control-recipes management (new products) Object-based development (reusable library objects) will allow streamline implementation and validation and will simplify system documentation (Bayer Standard Libraries) Paging functions and equipment efficiency (OEE) will occur through Asset Optimization New Historian to connect all PCS systems (web services reporting, data analysis tools, )

14 Site Automation Network Infrastructure Corporate IT Network Local DCS Systems Batch and Continuous Controls Historical data collection, Remote Desktop, Alarms, Paging Central Services Electronic Batch Records Historical data collection, Backups, Antivirus Web Reporting IT firewall Other Local Systems BAS Building Automation Systems ABB DCS ECS BPR Connector Labs Supervision REES SCADA Site DCS Historian Utilities Supervision SCADA ABB DCS ABB DCS Site BAS Historian Site REES Historian DMZ PCS firewall WFI Supervision SCADA Packaging Supervision SCADA DCS web services ABB DCS Particle Monitoring System Backups services ABB DCS Antivirus services Fermentors Fermworks SCADA Electrical Distribution Monitoring ABB DCS

15 Lessons Learned Through Evolution Be open to changes in the schedule (have alternative next steps ) Partner with all stake holders (suppliers, internal suppliers, and especially your end users) Keep an Open Mind, be receptive to new ideas and methods The systems are becoming more of an IT based system, requiring IT skills

16 Schedule Problems

17 Challenges to Meeting Schedule Production schedule requires flexibility to work around last minute changes, stay informed PCS up time is different, might require a phased approach Training of operators in new HMI use Un-expected changes to design Staffing and training

18 Partnering Issues

19 Partnering Identify all stake holders Solicit input with several review cycles throughout the entire process Leverage the experience your partners have (what a graphic should look like, how systems can connect, what type of computers, etc.)

20 New Ideas

21 Incorporating New Ideas New systems = New ways Leverage partners experience Design for Openness (flexibility, connectivity) Solicit input/ideas from others (Trade Shows, trade magazines, other industries) Implementation through Lab Development systems

22 PCS Meets IT

23 A New IT System/Infrastructure We now use standard IT hardware/software Interconnection between the PCS and IT worlds must follow IT rules We need the same services (anti-virus, OS patches, ABB patches, Third party patches, performance monitoring, etc.) We talk the same talk : TCP/IP, firewall, switches, routes, permissions, etc.)

24 The Challenge of Using Your IT Department Challenges: The IT department knows Windows: Yes they do, but they may not know process control and the special requirements need for it. The IT department can provide on-call coverage: When we investigated the on-call procedure we found that the best response time available was 4 hours. The IT department has structure: Yes, but do they report up through the same chain of command for escalation of issues? Costs: Original proposal for full time support was $175/month per server. 200 servers x $175/mon. = $420,000/year Solutions: Work on the IT departments training: and the Engineering department s training of IT technology. Supplement with Engineering on-call: engineers love to help. Service Level Agreements: specifically outline responsibilities and escalation of issues Purchase warranties from vendors: for on-site support, have well defined SLAs

25 Mitigating the Challenges Align the evolution schedule with Production Planning to work within their production schedule and implement new equipment while concurrently running old equipment, continued production and live training Investigate ways to run old and new systems in parallel Involve end users from the beginning i.e. use operators to design and build or approve graphics Work with ABB or System Integrator to have a defined method on how to implement required changes with little or no impact to schedule Have back-ups for essential personnel and invest in preimplementation training of critical personnel

26 Berkeley Site Automation Strategy, an Integrated Vision ERP SAP IT Corporate Network MES LIMS ebpr cmms EDMS ABB/Werum AUTOMATION Plant Automation Network Historian Historian Historian Historian Historian SCADA Rees Centron Wireless DCS ABB 800xA - Batch 800M AC460 Rockwell PLC (PAT integration Profibus) SCADA GE ifix Rockwell PLC Siemens PLC BAS Siemens Apogee SCADA Siemens WinPM Siemens PLC QC GBD CMF API FPM PACK UTIL HVAC ELE EQUIPMENT

27 Benefits Flexible integration connectors in place for integration with future Control and IT systems (cmms, INtools, LMS,..) Open automation interfaces (OPC, Fieldbus) and Standards (ISA S88, ISA S95) allow us to be prepared for future expansion with connections to any type of system The new platforms/infrastructure is the basis for other services such as a site- Historian database, web services for plant access and reporting, data analysis tools, etc Use of standard IT equipment and standards decreased time for installation and allows for sharing of knowledge with local IT experts including Corporate Allowed the introduction of Web based access to production data reducing costs and capital investment Remote engineering access to the Production systems reducing time to implement new control strategies and time to support automation problems

28 What s Next? Continued evolution of the PCS systems for flexibility and connectivity Virtualization of PCS systems New Site Wide Historian for all PCS systems New Operator interfaces/technology (tablets, PDAs/Smartphones, cellular,?) Remote connectivity/access through Web based applications (Cloud Computing, WIKI type search capabilities) Tighter integration between PCS and IT groups

29 Questions?

30 backup slides

31 Engineering IT Puzzling Systems Array Distributed of Technologies in Layers (ISA-95 Standard) Enterprise Layer TOPAS Instrument WMS PLC TD DCS cmms ERP CRM AM? SAP LIMS MES Layer PP MM ebpr SRM EDMS BAS PAT Automation Layer SPE SCADA

32 B60: Completion of 800xA +Batch +800M Plant w ide History Consolidation W 2KS DNS #1 W 2KS DNS #2 W 2KS PPB 1.2 Client & Config Srvr (common to all systems) (32) PPB 1.2 Clients (37) clients total EH 3.3/2 Node 1600 W 2KS EH 3.3/2 Node 1700 W 2KS RTDS #1A w/ PPB 1.2 Client Node 1800 W 2KS Mod Connect 2.3/0 RTDS #2A w/ PPB 1.2 Client Node 1900 W 2KS Mod Connect 2.3/0 RTDS #1B w/ PPB 1.2 Client Node 2800 W 2KS Mod Connect 2.3/0 RTDS #2B w/ PPB 1.2 Client Node 2900 W 2KS Mod Connect 2.3/0 MOD OPC Server 1 Node 600 W 2KS MOD OPC Server 2 Node 700 W 2KS AdvaBuild - ES p2 Node 100 W 2KS w/ MED & TCL Bldr Redundant Ethernet TCP/IP Multibus DDE (300) (DCN 14.6/7) Plant A 460 Controller Node 0B00 (3) 1:1 Rdnt CS01 CS02 CS03 CS04 CS05 Plant A 460 Controller Node 0C00 (3) 1:1 Rdnt Plant B 460 Controller Node 0D00 (3) 1:1 Rdnt Plant B 460 Controller Node 0E00 (3) 1:1 Rdnt Utilities 460 Controller Node 0F00 (3) 1:1 Rdnt TRIO LAN TRIO LAN TRIO LAN TRIO LAN TRIO LAN TRIO LAN TRIO LAN TRIO LAN TRIO LAN TRIO LAN New KG-PF Skids TRIO LAN TRIO LAN TRIO LAN TRIO LAN TRIO LAN

33 Electronic Batch Records Information Sharing - Pre-built solution: Enterprise Connectivity Solution (ECS) - Stand-alone : batches run regardless of connection to the MES system ABB Smart Client - Allows for safe access to DCS information - Large Bang for the Buck - Low system resource overhead - Easy to configure

34 Flexible, Scalable, Controls: 800M 800M uses current supported programming standards Easily integrated into the 800xA system Allows for easy model type programming Hardware costs were comparable Skid Integrators were eager to partner and learn new hardware.

35 Cutter founded in 1897 (Fresno, CA) - Black Leg vaccine for cattle Cutter moved to Berkeley, CA in Small pox Vaccination at 1906 earth quake - Allergens developed 1927 (Poison Oaks and Ivy) - Tetanus (Horse Serum), Major producer of penicillin, Plasma fractionation acc. to Cohn during WW 2 - Mosquito repellents, snake bite kits, 1950 s Plasma Fractionation plant built 1967 Konyne: Hemophilia B, 1969 Koate: Hemophilia A, 1974 Cutter absorbed by Miles Laboratories, 1978 Prolastin: Congenital Emphysema, 1987 Kogenate: 1993 Miles Laboratories became Bayer in 1994 Bayer Berkeley History

36 Efficient Operations: Automation, Flexibility, Integration AUTOMATION Berkeley Site Today Berkeley Site Yesterday Low level of Automation (most operations are executed manually) Low flexibility (our automated sequences were mostly hardcoded ) Local, stand alone, isolated Systems in each Plant /Building not allowing for data exchange and external services New platforms will allow automation of manual processes SCADA supervision FLEXIBILITY Pre-built and pre-validated softwarew libraries Batch control: flexible use of multiple recipes Scalable controllers in process areas SCADA supervision in Utilities and INTEGRATION Packaging on top of standard PLC All PCS systems integrated in a sitewide Network among different Plants & Buildings Integration between PCS and IT (MES Electronic BPR) Integrated Paging of Alarms Remote System Maintenance Data mining, System reports, etc

37 1. Why Evolve A need to change our systems while maintaining production, reliability, increased efficiency, access and accountability 2. How Partner with PCS vendors for system flexibility/expandability Partner with Production to minimize conflict with product delivery Co-ordinate access and data flow with Corporate IT Develop a robust and secure network infrastructure 3. What A system that meets all user s data requirements, Production, Data Miners, Engineers, for robustness and accessibility Secure, complete access of information to all users Traceability of code changes and information flow

38 MES-DCS Implementation: Basic Connectivity MES BF 1 BF 2 Master BPR BF 3 (part recipe) BF 4 batch_id recipe_id start unit_id EBR relevant data batch_id recipe_id start unit_id EBR relevant data (end of Unit procedure) DCS BATCH Unit procedure 3 Start OP 1 OP 2 End OP 3 Control Recipe Automatic run per Unit Execution on DCS level Unit procedure 4 Start OP 1 OP 2 OP 3 End