Enterprise Removable Media Manager

Transcription

1 Enterprise Removable Media Manager A Service Offering for Open System Tape Management Ulf Troppens, ermm Development, IBM Mainz troppens@de.ibm.com 2005 IBM Corporation

2 Disclaimer 2 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

3 Agenda ermm Overview ermm Details ermm Offline Demo (Screenshots) ermm & TSM ermm Roadmap Conclusion 3 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

4 Agenda ermm Overview ermm Details ermm Offline Demo (Screenshots) ermm & TSM ermm Roadmap Conclusion 4 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

5 Customer Problems and Requirements today Problems: No centralized access control, administration and reporting Media management has to be done by SAN every application SCSI/FC SCSI/FC SCSI/FC prop. Interface Difficult (or nearly impossible) to share libraries, drives and scratch pools, especially in heterogeneous Media Changer Streaming Library Mgmt. environments Devices Devices (Drives) Streaming Media Changer Multitude of Media Changer and Removable Media Hardware Devices (Drives) Devices Library Management Interfaces Libraries with SCSI Media Changer Removable Media Hardware (SCSI, IBM 3494, STK ACSLS/LS, ) (e.g. IBM 3584) Libraries with integrated Adding storage or upgrading storage Management (e.g. IBM 3494) technology requires changes to every Requirements: application Improved manageability to cope with rapidly growing storage Reduced TCO Efficient usage of tape storage capacity Increased utilization of tape drives Integrated support for vaulting 5 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

6 Customer Scenarios E D T Scenario I E D T TSM Server E D T Scenario II Legato Networker and HiBack Server SAN Scenario III TSM on z/os TSM LAN-free clients for open systems SAN STK ACSLS Server STK SN 6000 STK Silos Customer wants: consolidated media management for Open Systems first, later with z/os too centralized administration and control to get rid of Gresham EDT Statically partitioned IBM 3584 Customer wants: enhanced utilization of drives to create drive groups and cartridge pools that span multiple libraries centralized media management administration and control for Networker and HiBack applications enhanced monitoring and reporting IBM Tape Hardware Customer wants: consolidated media management for heterogenous environment Integrated support for vaulting policy-based cartridge lifecycle management 6 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

7 Case Study: Backup Consolidation on zlinux (1/2) Business need The customer wants to consolidate its backup infrastructure with TSM on zlinux. Business value The customer builds a backup infrastructure which is very flexible and which scales very well. The advanced management functions of zlinux/vm allow to provision new TSM Servers on demand. Use case (planned new service) At the customer a business unit can request to establish a new security zone including a dedicated VPN to protect certain applications and its confidential data. The infrastructure team automatically creates a new zlinux Instance with a new TSM server which is only visible in the new VPN. The concept of having a dedicated TSM server for each security zone simplifies the mechanisms to secure confidential backup data. 7 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

8 Case Study: Backup Consolidation on zlinux (2/2) ermm TSM 1 TSM 2 TSM n TSM n+1 TSM n+2 TSM n+3 TSM n+m z/vm 1 z/vm 2 z zfcp Ports for Tape, 8 zfcp Ports for disk Fibre Channel SAN IBM 3584 Tape Library 65 x IBM 3592 Tape Drive 2320 x 3592 JA Cartridge IBM DS ,7 TB RAID 5 net Disk cache for TSM Similar setup available in collocated data center Both data centers connected via DWDM (Gigabit Ethernet + Fibre Channel) Fail-over of zlinux instances between both data centers planned for later New TSM servers added on demand ermm s advanced tape management capabilities help to master the complexity of this frequently changing setup. 8 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

9 Value of the Enterprise Removable Media Manager Improve the flexibility of the removable media infrastructure Results Improved Application Availability Create cross library drive and scratch pools Optimized Storage Resource Utilization Share drives between heterogeneous applications Provide a cross application scratch pool Define policies that priories concurrent mount requests Define policies to move cartridges to vaults Enhanced Storage Personnel Productivity Create a single point of control, administration, reporting, security and auditing for removable media resources Move, add or change removable media resources (drives, cartridges, libraries) without requiring modifications to applications like TSM 9 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

10 Tape Configuration without ermm SAN IBM 3584 IBM 3494 StorageTek 10 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

11 Enterprise Removable Media Manager Enterprise Removable Media Manager SAN IBM 3584 IBM 3494 StorageTek 11 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

12 Enterprise Removable Media Manager Enterprise Removable Media Manager SAN Drive Pool Pools of capacity Segmented based on business need IBM 3584 LTO Drives Networker Cartridges Common Scratch Pool IBM 3494 STK Drives TSM Cartridges StorageTek Cartridge Pool Access for owning application only Scratch Pool For all applications Segmented based on business needs 12 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

13 Enterprise Removable Media Manager Policies Drive and cartridge allocation based on priorities Cartridge movement Cartridge life-cycle management Enterprise Removable Media Manager LTO Drives Networker Cartridges SAN TSM Cartridges New Efficiencies Cross application drive sharing Cross application STK Drivesscratch pool Mount request queuing Centralized reporting Audit trails Integrated vaulting IBM 3584 Common Scratch Pool IBM 3494 StorageTek 13 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

14 Dynamic grouping and pooling of drives and cartridges DriveGroup 1 DriveGroup 2 Cartridge Group 1 Cartridge Group 2 Library 1 Library 2 DriveGroups: Used to aggregate drives Drives can be added and removed dynamically Used to implement access permission model and preferential usage policy May span multiple libraries CartridgesGroups: Used to aggregate cartridges Cartridges can be added and removed dynamically Used to implement access permission model and preferential usage policy May span multiple libraries 14 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

15 Access Roles and Preferential Usage Policy for DriveGroups and CartridgeGroups DriveGroup 1 DriveGroup App 1 App 2 Cartridge Cartridge Group 1 Group Library 100 Access Roles: Applications are subject to access control Two level access permission model Administrative applications may access every Drive Standard applications may only access Drives within DriveGroups which have been assigned to them by an administrative application Preferential Usage Policy: DriveGroup-Application relations have priority attribute Drives have priority attribute too Priority is used by ermm to find drive for a mount operation 15 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

16 ermm Features Dynamic library and drive sharing across heterogeneous application boundaries and heterogeneous operating systems Dynamic drive and cartridge pooling Mount request queuing Policy-based drive and cartridge allocation Policy-based media life cycle management (*) Integrated off-site media management and tracking (Vaulting) (*) Centralized access control, administration and reporting Advanced reporting and auditing (*) (*) Planned for future releases 16 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

17 Top Customer Problems/Requirements Inefficient capacity usage because tape storage is statically partitioned by application Inefficient drive utilization because its difficult to share drives (physical mapping App->Lib. Partition) Media management has to be done by every application No centralized access control, administration and reporting No integrated support for Vaulting Adding storage or upgrading storage technology requires changes to every application App 1 App 2 SAN App 3 Provide centralized access control, administration and reporting accessible through open standards based API (IEEE1244) and SSG Console WUI Provide integrated support for manually operated libraries and vaults together with policy-based cartridge life cycle management (planned for future release). Eliminate changes when storage is added and minimize impact of technology upgrades. App 1 ERMM Solution Provide access to and sharing of a common scratch pool and dynamically resizable cartridge groups with access control. Provide dynamic sharing of tape drives including mount request queuing + additional utilization enhancements through dynamic grouping and pooling of drives Provide centralized mainframe-class media management accessible through open standards based API (IEEE1244) App 2 SAN App 3 Media Management and Sharing 17 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

18 Agenda ermm Overview ermm Details ermm Offline Demo (Screenshots) ermm & TSM ermm Roadmap Conclusion 18 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

19 ermm Architecture DM DM DM DM Media Manager Admin Console SAN Library Manager ermm IBM 3584 IBM 3494 STK ACSLS 19 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

20 ermm Architecture Controls drives on behalf of the media manager DM DM DM DM Coordinates access to drives and cartridges Central repository Log of all activities Media Manager Command line interface and GUI Admin Console SAN Library Manager ermm Controls libraries on behalf of the media manager Encapsulates library hardware IBM 3584 IBM 3584 StorageTek 20 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

21 ermm Control Flow (1.) TSM Server wants to mount a scratch tape (2.) TSM ELM sends request to Media Manager via TCP DM DM DM DM Media Manager Admin Console SAN Library Manager ermm IBM 3584 IBM 3584 StorageTek 21 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

22 ermm Control Flow (3.) Media Manager selects scratch cartridge and idle drive (4.) Media Manager forwards request via TCP DM DM DM DM Media Manager Admin Console SAN Library Manager ermm (5.) Library Manager moves cartridge to drive IBM 3584 IBM 3584 StorageTek (6.) Drive Manager loads cartridge into drive 22 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

23 ermm Control Flow (8.) Response to TSM Server via TCP and ELM (7.) Response to Media Manager via TCP DM DM DM DM Media Manager Admin Console SAN Library Manager ermm IBM 3584 IBM 3584 StorageTek 23 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

24 ermm Control Flow (9.) Direct access to drive DM DM DM DM SAN Media Manager Library Manager Admin Console ermm IBM 3584 IBM 3584 StorageTek 24 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

25 Agenda ermm Overview ermm Details ermm Offline Demo (Screenshots) ermm & TSM ermm Roadmap Conclusion 25 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

26 ermm Offline Demo (Screenshots) I. Setup II. Drive and cartridge sharing (between EMC/Legato Networker and TSM) III. Reports IV. Advanced Reporting 26 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

27 Setup (1) managed libraries 27 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

28 Setup (2) managed drives 28 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

29 Setup (3) managed cartridges 29 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

30 Setup (4) authorized applications 30 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

31 Setup (5) drive groups 31 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

32 Setup (6) drive groups (authorized applications) 32 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

33 Setup (7) cartridge group 33 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

34 Setup (8) cartridge group (authorized application) 34 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

35 Sharing (1) common scratch pool 35 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

36 Sharing (2) Networker begins using one drive 1. Allocate scratch volume 2. Label volume 3. Start backup 36 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

37 Sharing (2) TSM wants two drives 3. MOVE DATA needs two drives 4. TSM is waiting for second drive 37 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

38 Sharing (3) ermm synchronizes drive access 5. TSM is blocked 38 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

39 Sharing (4) Networker is done and TSM continues 6. Networker is ready and unmounts cartridge 7. TSM gets second drive 39 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

40 Reports (1) drive LTO263 was shared Drive Sharing between TSM and Networker 40 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

41 Reports (2) cartridge groups have changed 41 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

42 Reports (3) drive usage statistics 42 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

43 Advanced Reporting The following screen shots show prototyped features, which may become part of ermm in a future release. 43 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

44 Advanced Reporting (1a) Audit trails Step 1: Select filter in GUI Which Cartridge was mounted in DRIVE1 in January 2004? 44 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

45 Advanced Reporting (1b) Audit trails Step 2: Download report as CSV file 45 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

46 Advanced Reporting (1c) Audit trails Step 3: Import CSV file into Excel 46 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

47 Advanced Reporting (2a) Error logs Central error log file for all drive errors 47 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

48 Advanced Reporting (2b) Error logs Details for a specific error 48 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

49 Advanced Reporting (2c) Error logs Error statistics by day 49 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

50 Advanced Reporting (3a) Drive Utilization Total number of drives in use 50 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

51 Advanced Reporting (3b) Drive Utilization Utilization of a single drive 51 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

52 Advanced Reporting (4) Performance Total throughput by day 52 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

53 Agenda ermm Overview ermm Details ermm Offline Demo (Screenshots) ermm & TSM ermm Roadmap Conclusion 53 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

54 Value add in a homogeneous TSM environment (1/6) TSM provides its own tape management capabilities (e.g. TSM Library Sharing) ermm is designed to support heterogeneous applications Why do I need ermm in a homogeneous TSM environment? 54 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

55 Value add in a homogeneous TSM environment (2/6) Flexible provisioning of drives and cartridges sharing of TSM drives with other applications like EMC Legato NetWorker and OS tools like mksysb and tar simplified setup of TSM LAN-free environments no need to define tape paths inside TSM all tape paths are automatically generated inside ermm central check whether drive is used by TSM Server or Storage Agent flexible adding and removing of drives to TSM storage pools all changes are handled inside ermm updates of the TSM servers or other applications are not required enabling of cross library scratch pools enabling of cross library drive pools automatic labeling of scratch cartridges (can be disabled in config file) 55 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

56 Value add in a homogeneous TSM environment (3/6) Advanced reporting and auditing ermm provides statistics on how many drives were used when by which application (future). ermm provides statistics on how many data was written during a mount. This data can be aggregated to provide throughput reports for drives, drives groups, applications and a whole library. ermm provides statistics on how often a cartridge was mounted and how many data was written/read for the whole cartridge life cycle (future). TSM looses its data, when a cartridge is returned to the TSM scratch pool. The TSM server stores tape alerts in the TSM activity log. In addition to that, ermm stores all log sense pages in the ermm database. This additional data can be very useful for error analysis. 56 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

57 Value add in a homogeneous TSM environment (4/6) Enhanced access control to library resources In an outsourcing environment ermm can share the drives across several TSM servers whilst it strictly separates the cartridges of each TSM server. ermm can also provide a separate scratch pool for each TSM server thus cartridges are not exchanged between the TSM servers. Additional operational benefits dismount of cartridge when TSM server goes down physical erase of cartridges before return to scratch pool (future) improved cartridge life-cycle management (e.g. threshold for wornout tapes) (future) 57 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

58 Value add in a homogeneous TSM environment (5/6) Customer Environment The customer backs-up a large application server using TDP for Oracle. This backup job starts four sessions and thus uses four drives. Customer Issue When TSM starts a backup of the TSM database and all drives are in use, then TSM dismounts one volume and uses it for the TSM database backup. As a result the backup of the large application server fails, even when it was running for several hours and even when this backup job was nearly complete. ermm Solution Configure TSM with two external libraries which use ermm managed resources. Use one library for the backup of the application server and the other for the backup of the TSM database. Inside ermm map these two libraries to the same resources (DriveGroup, CartridgeGroup). As a result ermm will queue the backup of the TSM database. Now the TSM administrator can decide which job to cancel. 58 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

59 Value add in a homogeneous TSM environment (6/6) TSM provides its own tape management capabilities (e.g. TSM Library Sharing) ermm is designed to support heterogeneous applications ermm s advanced tape management capabilities provides value add even in a homogeneous TSM environment! 59 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

60 Agenda ermm Overview ermm Details ermm Offline Demo (Screenshots) ermm & TSM ermm Roadmap Conclusion 60 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

61 Supported platforms (ermm 1.2) ermm Media Manager SLES 8 for zseries GA AIX 5.2/5.3 GA SLES 8 for Intel RPQ Applications Tivoli Storage Manager GA Native OS commands (e.g. tar, mksysb) GA EMC Legato Networker RPQ Veritas NetBackup future iseries BRMS future zseries DFSMSrmm future Other backup applications future Tape Libraries IBM 3584 Open Systems GA IBM 3494 Open Systems GA StorageTek ACSLS GA StorageTek HSC-LS RPQ Other tape libraries future 61 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

62 Implemented features (ermm 1.2) Dynamic/heterogeneous drive sharing Dynamic/heterogeneous drive and cartridge pooling Mount request queuing Tape library virtualization Policy-based drive and cartridge allocation Centralized access control, administration and reporting Policy-based cartridge lifecycle management Offsite-media management and tracking (vaulting) Advanced reporting and auditing GA GA GA GA GA GA future future future 62 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

63 Agenda ermm Overview ermm Details ermm Offline Demo (Screenshots) ermm & TSM ermm Roadmap Conclusion 63 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

64 Summary The goal of ermm is to dramatically reduce the Total Cost of Ownership for data stored on tape by improving its access and its management across heterogeneous application, server, and tape library boundaries. ermm is an enterprise tape library virtualization solution which scales very well and provides very high availability and reliability. ermm provides mainframe class, centralized management functions like tape cartridge pooling, tape drive pooling, tape drive sharing, mount request queuing, policy based cartridge lifecycle management, offsite media management, centralized access control and auditing, and advanced reporting functions. ermm s intelligent sharing capabilities increase tape drive, tape cartridge and therefore tape library utilization whilst it reduces the time spent for tape library administration. 64 Enterprise Removable Media Manager (ermm) 2005 IBM Corporation

65 ermm Resources Internet ermm Business Development Detlef Krause Mail: Mobile: Enterprise Removable Media Manager (ermm) 2005 IBM Corporation