Copyright 2018 Dell Inc.

Transcription

1 1

2 Dell EMC Unity: Performance Analysis and Troubleshooting Qin Tao Performance Engineering Midrange & Entry Solutions

3 The Dell EMC Midrange Family SC5020 SC5020F SC7020F SC9000 Dell EMC Unity 600/650F Dell EMC Unity 500/550F Dell EMC Unity 400/450F SC7020 SCv3000/3020 SC Series Dell EMC Unity 300/350F Dell EMC Unity COMMON TOOLS FOR MANAGEMENT, MOBILITY & PROTECTION POWERPATH RECOVERPOINT AVAMAR NETWORKER 3

4 Our customer commitment Because our combination is very complementary we are fully committed to, supporting and investing in both Dell EMC Unity and Dell EMC SC. Michael Dell us/company/our commitment/letter to customers and partners.htm 4

5 Industry s #1 Midrange Portfolio SC SERIES DELL EMC UNITY INTELLIGENT EFFICIENCY POST-PROCESS DATA REDUCTION INLINE EFFICIENCY INLINE DATA REDUCTION FEDERATED DATA MOBILITY ACROSS MULTIPLE SYSTEMS UNIFIED UNIFIED FILE & BLOCK DATA BEST ECONOMICS LOW ENTRY PRICE - LOWEST $ / GB INTEGRATED HYBRID CLOUD UNIFIED CLOUD TIERING 5

6 Agenda Overview System resource balance and performance Tools Introduction to Unity performance tools Scenarios Fast track to become a Unity performance expert 1. Noisy Neighbor detection and management 2. New Fibre Channel environment deployment 6

7 Overview It s all about BALANCE! Balance the utilization of available system resources Unity Designed to automatically balance workload Storage Processors, Ports, Tiers Management Performance tools Health check Capacity planning Troubleshooting and resolution 7

8 Unity Performance Tools Unisphere Performance Dashboard GUI performance report Historical (up to 90 days) Real-time Unisphere Command CLI CLI performance report Historical or real-time Different output formats for customized reports CloudIQ Cloud-native application Centralized view across your storage systems Proactive monitoring and risk reduction All at No Cost! Live Optics Online software Collect, visualize and share the performance data Assist IT infrastructure planning and collaboration 8

9 Unity Performance Tools: Unisphere Performance Dashboard GUI presentation of historical and real-time performance metrics Real-time: 5 seconds sample interval Historical: longer sample interval for older data Sampling interval Retention period 1 minute 3 days 5 minutes 14 days 1 hour 28 days 4 hours 90 days Performance details at both system and object level 9

10 Unity Performance Tools: Unisphere Command CLI Allows collection and post-processing of performance data in a customized way Real-time Historical Allows short sample interval (5 seconds) for more granular data uemcli -d <IP> -u <user> -p <pwd> /metrics/metric -availability real-time show Lists all available metrics, ~750 in total Longer sample period for older data uemcli -d <IP> -u <user> -p <pwd> /metrics/metric -availability historical show Lists all available metrics, ~90 in total 10

11 Unity Performance Tools: No cost cloud-native storage analytics application Easy to get started Connect Secure Remote Services(ESRS) -> enable CloudIQ -> login to cloudiq.dellemc.com Analyze and identify storage issues faster, simpler and accurately Centralized Monitoring 11 Predictive Analytics Proactive Health Score

12 Unity Performance Tools: Free, lightweight, remote and agentless software Stream configuration and performance data to an online analytics engine to generate a report Collect, visualize and share workload characteristics Get valuable insight and stay in control of buying process Agnostic on vendor, hardware and platform 12

13 Scenarios

14 Scenario 1 Noisy Neighbor Detection and Management Background Storage admin Mike manages multiple Unity systems for an IT company The Unity systems are connected with CloudIQ 14

15 Scenario 1 Noisy Neighbor Detection and Management Performance issue! Mike meets Julie from the testing group, who tells him that her VM is kind of slow that morning Mike checks the CloudIQ Overview page and identifies the Unity and LUNs with Performance Anomalies 15

16 Scenario 1 Noisy Neighbor Detection and Management Narrow-down the Issue Mike launches Unity management directly from CloudIQ Mike starts Unisphere Performance Dashboard to get more details 16

17 Scenario 1 Noisy Neighbor Detection and Management Troubleshooting 17

18 Scenario 1 Noisy Neighbor Detection and Management Troubleshooting Pool: Application Dev Pool: DB Dev Pool: Test Team CG1 CG2 LUN0 LUN1 LUN2 CG1 CG2 18

19 Scenario 1 Noisy Neighbor Detection and Management Troubleshooting Noisy Neighbor impact! 19

20 Scenario 1 Noisy Neighbor Detection and Management Approach Solutions Mike finds the hosts which access the 2 DB Dev LUNs Mike calls DB Dev team manager and takes notes: IO burst is triggered by a DB scanning task IO intensive, but not latency sensitive DB team is expanding with more projects Bursty workloads can recur in future 20

21 Scenario 1 Noisy Neighbor Detection and Management Problem Solving Mike s short term solution Quality of Service (QoS) on the Noisy Neighbor LUNs 21

22 Scenario 1 Noisy Neighbor Detection and Management Problem Solved! Host I/O Limit kicks in DB Dev LUNs IOPS drop to defined value No impact to the Test LUN Host I/O Limit kicks in Response time drops 22

23 Scenario 1 Noisy Neighbor Detection and Management Future Planning? Need more CPU resource for growing demands Mike s long term solution - rebalancing data within other Unity systems, upgrading to a higher platform, or buying a new one? 23

24 Scenario 1 Noisy Neighbor Detection and Management Future Planning Mike collects Live Optics report for future planning 24

25 Scenario 1 Noisy Neighbor Detection and Management Summary STAGES PROCESS TOOLS Narrow-down Isolate issue to Unity and specific LUNs CloudIQ Troubleshooting Solution Examine storage configurations Check performance metrics Rule out common bottlenecks Set host I/O Limit Unisphere Management & Performance Dashboard Unisphere Management Future Planning Rebalance data, upgrade Unity or new purchase Live Optics 25

26 Scenario 2 New Fibre Channel Environment Deployment Background Storage admin Mike manages a small data center New deployment on a Unity 650F with Fibre Channel environment 26

27 Scenario 2 New Fibre Channel Environment Deployment Infrastructure SAS port 1 connected Unity 650F Follow Best Practices to configured multiple FE ports and BE SAS port FC switch(another switch to be added for redundancy) 5 ESXi Hosts 27

28 Scenario 2 New Fibre Channel Environment Deployment Sanity Test 1 pool with 36 flash drives Avoid being drive bound in test 40 x 200GB Thin LUNs Avoid the limit from host FC HBA per-lun Queue Depth 4-corner performance tests Read IOPS with small IOs (8KB) Write IOPS with small IOs (8KB) Read MB/s with large IOs (256KB) Write MB/s with large IOs (256KB) Test tool- Vdbench Batch workload 28

29 Scenario 2 New Fibre Channel Environment Deployment Low Performance? Read% Data Warehouse 8KB Read IOPS: 131,672 8KB Write IOPS: 85,700 CRM/ ERP Block Storage Consolidation 256KB Read MB/s: 3, KB Write MB/s: 2,872 IO Size Both reads and writes have ~3000 MB/s bandwidth Each FC port can deliver ~1500 MB/s based on Unity Best Practices paper 29

30 Scenario 2 New Fibre Channel Environment Deployment Troubleshooting Check on hosts Host configurations Overall ESXi host health condition Similar performance on each VM client? FC HBA connectivity and speed Test configuration Vdbench configuration file 30

31 Scenario 2 New Fibre Channel Environment Deployment Troubleshooting Check on array/unity Low bandwidth High latency 31

32 Scenario 2 New Fibre Channel Environment Deployment Troubleshooting The CPU is not saturated, how about cores? The reported CPU utilization is an average utilization of all cores 32

33 Scenario 2 New Fibre Channel Environment Deployment Troubleshooting Only 2 Unity ports are active! Bottleneck identified!! 33

34 Scenario 2 New Fibre Channel Environment Deployment Troubleshooting System is healthy All Fibre Channel ports are connected 34

35 Scenario 2 New Fibre Channel Environment Deployment Troubleshooting All FC initiators are only connected to 2 Unity ports! 35

36 Scenario 2 New Fibre Channel Environment Deployment Root Cause and Problem Solving Run FC switch CLI zoneshow Only 2 Unity ports are configured in the zoning 36

37 Scenario 2 New Fibre Channel Environment Deployment Improved Performance Read% Data Warehouse 8KB Read IOPS: 131, ,151 8KB Write IOPS: 85, , KB Read CRM/ ERP Block Storage Consolidation MB/s: 3,142 11, KB Write MB/s: 2,872 5,763 IO Size Much higher performance! 37

38 Scenario 2 New Fibre Channel Environment Deployment Understand the Performance IOs are distributed on all ports 38

39 Scenario 2 New Fibre Channel Environment Deployment Understand the Performance Higher CPU utilization with higher performance Balanced core utilizations 39

40 Scenario 2 New Fibre Channel Environment Deployment Summary STAGES PROCESS TOOLS/REFERENCES Setup Environment Follow Unity Best Practices Unity Best Practices Paper Sanity Test Hosts Run 4-corner tests Unity Switch Vdbench Host Config Best Practices All Components Troubleshooting Examine storage configurations Check performance metrics Rule out common bottlenecks Unisphere Performance Dashboard Unisphere Cmd CLI Switch Cmd CLI Solution Correct FC zoning Switch Cmd CLI 40

41 It is all about BALANCE! 41

42 Other Midrange & Entry Solutions Activities Technical Breakout sessions Session Title Day Time Day Time Dell EMC Unity: Update & Architectural Overview Mon 8:30 AM Wed 8:30 AM Dell EMC Unity: Performance Best Practices Tue 8:30 AM Thu 10:00 AM Dell EMC Unity: Performance Analysis & Troubleshooting Mon 3:00 PM Wed 1:30 PM Dell EMC Unity: NAS & File Capabilities Tue 12:00 PM Thu 8:30 AM Dell EMC Unity: Disaster Recovery & Business Continuity Mon 12:00 PM Wed 3:00 PM Dell EMC Unity: Virtualization & Application Integration Tue 1:30 PM Thu 1:00 PM Dell EMC Unity: Built-In Hybrid Cloud & Software Defined Capabilities Mon 1:30 PM Wed 12:00 PM Dell EMC SC Series: Update & Architectural Overview Mon 12:00 PM Wed 1:30 PM Dell EMC SC Series: Performance Best Practices Tue 12:00 PM Thu 10:00 AM Dell EMC SC Series: Performance Analysis & Troubleshooting Tue 8:30 AM Thu 11:30 AM Dell EMC SC Series: Multi-Array Federation Mon 3:00 PM Wed 8:30 AM Dell EMC SC Series: Disaster Recovery & Business Continuity Tue 1:30 PM Thu 1:00 PM Dell EMC SC Series: Virtualization & Application Integration Mon 8:30 AM Wed 3:00 PM Dell EMC Unity & SC Series: Migration from VNX and PS Tue 3:00 PM Thu 11:30 AM Dell EMC Entry Level Storage Update Mon 8:30 AM Tue 3:00 PM Plus: Dell EMC Unity & SC Series Hands-on-Labs running all day, everyday. Solutions Expo Hall Booth #325: 8 Kiosks and Equipment on display. 42

43 Extend the value of Dell EMC Unity with best-in-class services ProSupport Plus ProDeploy Plus Intelligent Data Mobility Optimize for Storage Up to 28% fewer service requests* Over 1,600 certifications held by Dell EMC Deployment engineers Over 1 exabyte of data migrated annually 97% overall customer satisfaction *:Based on a March 2018 internal analysis of service requests from August 2017 to March 2018 for Dell EMC Storage and Data Protection products comparing service requests for products with ProSupport Plus for Enterprise vs. products without it. Actual results may vary. 43

44 Want to Win An R2D2 App Enabled droid? while at Dell Technologies World 1 Winner will be chosen daily Monday Thursday April 30-May 3 All winners will be notified through Twitter Direct Message NO PURCHASE NECESSARY. Ends 05/03/2018. To enter and for Official Rules, visit the Dell EMC Booth #325. All prizes will be shipped to the winners address of choice post show. 44

45