Accelerate with IBM Storage: IBM DS8880: Hybrid cloud integration with Transparent Cloud Tiering

Transcription

1 : IBM DS8880: Hybrid cloud integration with Transparent Cloud Tiering Craig Gordon Consulting I/T Specialist IBM Washington Systems Center

2 Webinars The Free IBM Storage Technical Webinar Series Continues in Washington Systems Center Storage experts cover a variety of technical topics. Audience: Clients who have or are considering acquiring IBM Storage solutions. Business Partners and IBMers are also welcome. To automatically receive announcements of upcoming webinars, Clients, Business Partners and IBMers are welcome to send an request to accelerate-join@hursley.ibm.com. Located in the Blog: Also, check out the WSC YouTube Channel here: Webinars: January 9 DS8880 Easy Tier January 17 Start 2018 Fast! What's New for Spectrum Scale V5 and ESS February 8 - VersaStack - Solutions For Fast Deployments February 16 - TS7700 R4.1 Phase 2 GUI with Live Demo February 22 - DS8880 Transparent Cloud Tiering Live Demo March 1 - Spectrum Scale/ESS Application Case Study Register Here: March 7 - Spectrum Storage Management, Control, Insights, Foundation; what s the difference? Register Here: March 15 - IBM FlashSystem A9000/R and SVC Configuration Best Practices Register Here:

3 Webinar Agenda What is the Problem? Why is Cloud Storage the Answer? Tell me more about the Benefits Give me more Details How do I Get Started? Demonstrate Transparent Cloud Tiering in Action 3

4 ILM Still Utilizes a 1970 s Architecture CPU Cycles Online Immediate Access DSS/HSM Half Data Movers 16K Blocksize DFSMS Reorgs, Audits, Backups Continual Recycle Offline Delayed Access Serial Access to Tape 4

5 Transparent Cloud Tiering Client Value TCT for DS8000 and DFSMShsm saves z/os CPU utilization by eliminating constraints that are tied to original tape methodologies Direct data movement from DS8000 to cloud object storage without data going through the host Transparency via full integration with DFSMShsm for migrate/recall of z/os datasets Migrate with Tape Reduce CPU Utilization 16K Blocksizes Dual data movement Recycle Serial Access to Tape Co-location HSM inventory (Eliminates OCDS) Migrate with TCT & Cloud Storage IBM Cloud Object Storage IBM TS7700

6 Cloud Simplicity and Differentiation Each object represents a dataset instead of a tape volume Allows for parallelism for migrate and recall (eliminates serial access to tape) Storage tiers are not new Cloud is a new storage tier (MIGRATC) Not meant to replace ML2 but additive Data does not have to go through ML1 or ML2 to go to MIGRATC IF TAPE Select a tape (partial, full, scratch?) Allocate a drive Invoke DSS DSS Reads data and passes to HSM HSM reblocks the data into 16K blocks 16K blocks are written over the Channel SYNCH data on tape Tape flushes buffers and stops streaming Handle EOV, Spanning, FBID RECYCLE processing Continuously rewrites older data to new tapes 12

7 Transparent Cloud Tiering for DS8000 Server-less direct data transfer from DS8870/DS8880 to cloud storage DFSMS No additional appliances in data path Optimized for DFSMShsm - Saving System z CPU utilization Software Using Existing DS8000 Infrastructure Microcode Upgrade Only Available for DS8870 (7.5SP5 via RPQ), DS8880 R8.2.3 (June GA) No Additional Hardware Required Uses existing Ethernet ports in DS8870 and DS8880 CECs (Power8 Servers) Supports Openstack Swift / Amazon S3 Object Store connectivity Auditing / Security Transparent Cloud Tiering Ethernet Ports are Outbound Ports Only No method to access DS8k CECs Support of System z Audit Logging Architected with System z security (RACF, Top Secret) IBM Cloud Object Storage IBM TS7700 7

8 DS8880 Object Offload to TS7700 Take advantage of TCT for DS8880 and DFSMShsm with TS7700 as an object store IBM DS8880 Transparent Cloud Tiering IBM TS7700 Data stored on TS7700 as objects, not tape volumes Embeds GRID data movement engine within DS8000 to move data Supports 2x2 GRID for redundant data GRID Data Movement Engine Ethernet GRID Links GRID Data Movement Engine Note: Current TCT/object configuration cannot be with existing GRID configurations

9 How much in CPU savings? MIPS / Data Set Size Transparent Cloud Tiering Pre Catalog Eligibility Serialize CDS I/O Move HSM / DSS Handshake Blocking I/O Post Tape Synch Scratch Catalog CDS I/O * MIPS / Data Set Size DFSMS I/O Small Medium Large Extra Large PreProcessing Data Movement Post Processing Small Medium Large Extra Large PreProcessing Data Movement Post Processing * Values in graphs are not from actual tests, but rather just represent a relative comparison 9

10 CPU Efficiency Estimator IBM has created a tool to estimate CPU savings HSM writes various statistics to SMF record specified by SETSYS SMF(smfid) Recommended smfid is 240 FSR records are written to smfid+1 (241) FSRCPU records CPU time Fields include dataset size and amount of data written With a few days worth of SMF data, the estimator can determine: 1. Size of datasets to target for greatest cost savings 2. Estimated amount of CPU cycles saved by using Transparent Cloud Tiering 10

11 TCT CPU Saving Estimator Example Savings Output Estimated Savings based on amount of CPU used versus estimated usage with TCT View number of tracks migrated in relation to savings View client s dataset size based on real SMF data 1 1

12 Delivery Plan The Cloud Storage Solution will be delivered iteratively Intent is to ship enhancements every six to nine months Some support will be z/os only, some DS8K only, and some will be a combination Most z/os support will be shipped via SPEs Some support will be delivered on Release boundaries 12

13 Updates delivered to date 4Q2016 Initial RPQ only solution for DS8870 APAR OA51622 (zos 2.1) 2Q2017 Support on DS8880 family with R8.2.3 APARs OA51622 (zos 2.1) and OA50677 (zos 2.2) SWIFT API connection to the cloud Simplex volumes only 3Q2017 DS8880 family with R8.3 APAR OA52384 (zos 2.1/2.2) OA53538 (zos 2.3) Metro Mirror and Hyperswap volumes now eligible Add IBM Cloud Object Storage as a new cloud type Add Amazon S3 API 13

14 Example Setup 14

15 Setup on DS8880 Plug in Ethernet cables into both CEC s free Ethernet ports Two empty ports per card today Use DSCLI to import and/or export your certificates if you plan to use TLS Use DSCLI or GUI to create a userid to authenticate to the DS8880 HMC from zos Use DSCLI to configure TCPIP on Ethernet cards setnetworkport [-ipaddr IP_address] [-subnet IP_mask] [-gateway IP_address] Port_ID This will automatically set up the firewall outgoing ports only Use DSCLI to configure DS8k to the Cloud Storage / TS7700 mkcloudserver -type cloud_type [ ssl tls_version] -account account_name -user user_name -pw user_password -endpoint location_address port port_num cloud_name

16 Add DS8880 Self-signed certificate to RACF SITE ring RACDCERT SITE ADD(S13FCM01.CERTIF.PEM) WITHLABEL('DS8000 FCM01 SELF SIGNED SITE') TRUST PERMIT IRR.DIGTCERT.LISTRING CLASS(FACILITY) ID(DFHSM) ACCESS(READ) 16

17 Configure Cloud in SMS Same cloud_name specified in ISMF panels, defining the DS8000 HMC as the endpoint DS8000 userid for authenticating to GUI or DSCLI

18 Configure Cloud in SMS (continued) Uniform Resource Identifier authentication endpoint HTTPs authentication port information Specifies the name of the key store to be used. The value can be one of the following: - A SAF keyring name, in the form of userid/keyring - A PKCS #11 token, in the form of *TOKEN*/token_name

19 Configure Cloud in SMS Uniform Resource Identifier authentication endpoint

20 DFSMShsm MIGRATE DATASET(dsname) CLOUD(cloud) HSM invokes DSS to migrate data sets to the Cloud HSM inventory manages the Cloud, Container and Object prefix Transparent to applications and end users No Recycle Recall works just as it does today Audit support VOLUME and STORAGEGROUP keywords also supported As today, volser will be changed to MIGRAT ISPF will display MIGRATC, as opposed to MIGRAT1 or MIGRAT2 20

21 DFSMShsm Migration As today, DFSMShsm will automatically Recall a data set to Primary Storage when it is referenced RECALL, HRECALL, ARCHRCAL all support recalling from the Cloud. There are no parameter changes, as all information is stored within the HSM control data sets Common Recall Queue is supported Fast Subsequent Migration Remigrated data sets are just reconnected to existing migration objects if the source data set was not updated No additional data movement 21

22 DFSMShsm: DB2 Image Copy Offload Step 1: Create PiT DB2 Images Copies using FlashCopy Step 2: Wait for background FlashCopy to complete DB2 Source Objects DB2 Image Copies FlashCopy DFSMS Cloud Tier IBM TS7700 Step 3: MIGRATE STORAGEGROUP(DB2IMGC) CLOUD(MYCLOUD) DB2 Offline PiT Image Copies with the Data never going through the host 22

23 DFSMShsm: DB2 Transparent Archiving DB2 V11 Transparent Archiving of Temporal Data DB2 Active Table Storage Tiers High End DB2 automatically moves deleted rows to an archive table Increases efficiency and reduces size of base table Archive Table migrated to cloud storage DB2 Archive Table Migrate / Recall IBM Cloud Object Storage Recalled for Queries from the Archive table Thinly Provisioned Nearline Serverless Data Movement 23

24 Live Demonstration 24

25 Summary: Transparent Cloud Tiering for DS8000 Server-less direct data transfer from DS8870/DS8880 to cloud storage Optimized for DFSMShsm - Saving System z CPU utilization DFSMS Software Using Existing DS8000 Infrastructure Current ethernet connections limited to 1Gbps Enhancements planned Solutions implemented today should be limited to applications that can sustain delays in recall time Ideal time to install and configure storage management policies to prepare for production rollout Auditing / Security Transparent Cloud Tiering Ethernet Ports are Outbound Ports Only No method to access DS8k CECs Support of System z Audit Logging Architected with System z security (RACF, Top Secret) IBM Cloud Object Storage IBM TS

26 Thank You.

27 Please Note: IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice and at IBM s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. 27