NetApp Flexgroup Volumes in ONTAP. August 2017 SL10312 Version 1.0

Transcription

1 August 217 SL1312 Version 1.

2 TABLE OF CONTENTS 1 Introduction What is a NetApp FlexGroup? The Value of NetApp FlexGroup Lab Objectives Prerequisites Lab Environment Lab Activities Create and Share a FlexVol Volume Create and Share a FlexGroup Volume Examine FlexVol and FlexGroup Volume Layout NFS Test - High File Count and Small Files NFS Test - Low File Count and Large Files Viewing FlexGroup Performance with OnCommand Performance Manager FlexGroup File Layout Resizing a Flexgroup Using Volume Move to Re-distribute Volumes Across a Cluster Snapshot Restore Expanding a Flexgroup by Adding Member Volumes Using XCP to Migrate from FlexVol to FlexGroup References Version History NetApp, Inc. All rights reserved. NetApp Proprietary

3 1 Introduction This lab introduces you to NetApp FlexGroup volumes and illustrates the capacity, performance, and ease of management benefits that FlexGroups offer for environments facing the following storage challenges. Very high file count. Large capacity footprint. Metadata intensive. This lab also explains how FlexGroup volumes compare favorably to FlexVol volumes, and how to best migrate data onto them. 1.1 What is a NetApp FlexGroup? A NetApp FlexGroup volume is a single namespace made up of multiple member volumes that are managed and act like a single FlexVol volume. Files in a FlexGroup are allocated to individual member volumes, and are not striped across volumes or nodes. With FlexGroup volumes, a storage administrator can easily provision a massive single namespace in a matter of seconds. FlexGroup uses constituent member volumes that work in collaboration to dynamically balance load and space allocation evenly across all members. There is no required maintenance or management overhead with a FlexGroup volume. You simply create the FlexGroup volume and share it with your NAS clients. This design allows workloads to take advantage of more CPU threads per node, as well as being able to leverage more nodes per cluster for a single file system. The main performance gains occur during the initial ingest of files, but they also continue later because workloads are more evenly spread across the precious cluster resources. 1.2 The Value of NetApp FlexGroup NetApp FlexGroup volumes solve multiple problems that cannot be addressed by a single FlexVol. A NetApp FlexGroup blends the following concepts: 3 Capacity NetApp FlexGroup volumes provide >1TB in capacity by using multiple FlexVol volumes in concordance. Architecturally, there are no limits to how large a FlexGroup volume can grow (so long as you do not exceed the maximum number of FlexVols supported in a cluster). Additionally, the design of the NetApp FlexGroup allows for a nearly unlimited number files in a single namespace, allowing for greater flexibility with your ONTAP storage systems. Scalability With NetApp FlexGroup, a single namespace can live on multiple aggregates on a node or multiple nodes in a cluster. This provides a true scale-out file system, with the ability to add capacity, compute and network resources non-disruptively to large file systems. See TR-4571 (listed in the References section) for further details. Performance NetApp FlexGroup does not just provide immense capacity by way of multiple constituent member volumes, it also provides vastly improved performance for high metadata workloads by allowing file ingest to occur across multiple volume affinities. By performing these operations in parallel, a NetApp FlexGroup can provide 2-6x the performance for certain NAS workloads. Simplicity of Management Prior to NetApp FlexGroup, it was possible to simulate the volume concurrency for high metadata workloads by using FlexVol volumes and junction paths. However, this design was limited to the architectural constraints of the FlexVol (1TB, and 2 billion files per volume). Creating multiple volumes and connecting them with junction paths created a file system that was defined and controlled by the storage administrator rather than the application owners. Additionally, managing multiple FlexVol volumes required more overhead to maintain the export policies, rules, CIFS shares, etc. NetApp FlexGroup brings the benefits of a multi-flexvol volume architecture, while removing the management headaches. A FlexGroup is intended to be managed like a single FlexVol volume. 217 NetApp, Inc. All rights reserved. NetApp Proprietary

4 1.3 Lab Objectives The lab includes the following activities: Steps to create a NetApp FlexGroup volume. Test NFS and SMB file creation on FlexGroup and FlexVol. View the effects of small and large files on FlexGroup volumes. Compare performance (wall clock) of migrating data to FlexVol versus FlexGroup. View FlexGroup volumes in OnCommand Unified Manager. Compare throughput, latency, IOPs and time to completion for FlexGroup vs FlexVol volumes in OnCommand Performance Manager. Run commands to review file system layout in FlexGroup. Test snapshot creation and restores. Manage FlexGroup volumes (growing, adding members, enabling storage efficiencies). Migrate data to NetApp FlexGroup using XCP for NFS. 1.4 Prerequisites This lab assumes that you have a general working knowledge of NAS (both NFS and SMB), how to manage FlexVols in ONTAP using the command line and GUI, and how to run step by step commands. The lab also assumes that you are generally familiar with the concepts of clustered Data ONTAP NetApp, Inc. All rights reserved. NetApp Proprietary

5 2 Lab Environment The following diagram illustrates the environment for this lab. Figure 2-1: All of the servers and storage controllers presented in this lab are virtual devices, and the networks that interconnect them are exclusive to your lab session. While we encourage you to follow the demonstration steps outlined in this lab guide, you are free to deviate from this guide and experiment with other ONTAP features that interest you. While the virtual storage controllers (vsims) used in this lab offer nearly all of the same functionality as physical storage controllers, they are not capable of providing the same performance as a physical controller, which is why these labs are not suitable for performance testing. The Lab Host Credentials table provides a list of the servers and storage controller nodes in the lab, along with their IP address. Table 1: Lab Host Credentials Hostname Description IP Address(es) Username Password dc1 Active Directory Server Demo \Administrator Netapp1! jumphost Windows 212R2 Remote Access host Demo \Administrator Netapp1! rhel1 RHEL 6.8 x root Netapp1! ocum OnCommand Unified Manager admin Netapp1! opm OnCommand Performance Manager admin Netapp1! cluster1 ONTAP 9.1 cluster admin Netapp1! The Preinstalled NetApp Software table lists the NetApp software that is pre-installed on the various hosts in this lab NetApp, Inc. All rights reserved. NetApp Proprietary

6 Table 2: Preinstalled NetApp Software Hostname 6 Description jumphost Data ONTAP DSM v4.1 for Windows MPIO, Windows Unified Host Utility Kit v7.., NetApp PowerShell Toolkit v4.3. ocum OnCommand Unified Manager 7.1 opm OnCommand Performance Manager 7.1, pre-integrated with OCUM 217 NetApp, Inc. All rights reserved. NetApp Proprietary

7 3 Lab Activities An understanding of the following terminology will aid you in completing this lab. A FlexVol volume is provisioned from the available storage in an aggregate. FlexVol volumes are flexible and can be increased or decreased in size dynamically without impacting or disrupting the environment. A single aggregate can contain many FlexVol volumes. A FlexVol volume is not tied to any specific set of disks in the aggregate and is striped across all the disks in the aggregate. A FlexGroup volume is a single namespace made up of multiple constituent member volumes that is managed and acts like a FlexVol volume to storage administrators. Files in a FlexGroup volume are allocated to individual member volumes and are not striped across volumes or nodes. A Constituent/Member volumeare interchangeable terms when referring to FlexGroup volumes. These are the underlying FlexVol volumes that make up a FlexGroup volume and provide the capacity and performance gains seen only with a FlexGroup volume. A Namespace is a single, unified location where storage that lives on multiple nodes in a cluster can be accessed. Ingest Heuristics is an algorithm by which a FlexGroup volume determines how best to allocate newly ingested files across member volumes. 3.1 Create and Share a FlexVol Volume In this exercise you will create a FlexVol volume from OnCommand System Manager. This will provide you with a frame of reference for comparison in the next exercise when you see how to create a FlexGroup. 1. On Jumphost, launch Chrome from the taskbar. Tip: If you prefer you may use Firefox instead, but the instructions and screenshots in this lab guide utilize Chrome. 1 Figure 3-1: The browser's home page is set to automatically open tabs for OnCommand System Manager (for the lab's ONTAP cluster, cluster1) and for OnCommand Unified Manager. 2. Select the tab for OnCommand System Manager (if it is not already selected). 3. Log in with the following credentials: 7 Username: admin Password: Netapp1! 217 NetApp, Inc. All rights reserved. NetApp Proprietary

8 2 3 Figure 3-2: System Manager displays the Dashboard page. 4. Click the SVMs tab. 4 Figure 3-3: 5. Click the entry for the Demo SVM NetApp, Inc. All rights reserved. NetApp Proprietary

9 5 Figure 3-4: 6. Click the Volumes sub-tab. 7. In the Volumes pane, click the Volumes sub-heading (which may already be selected by default). This navigates you to the pane for managing FlexVols. 8. Click Create Figure 3-5: The Create Volume window opens. 9. Select the General tab if it is not already selected. 1. Populate the fields on this tab as follows: NetApp, Inc. All rights reserved. NetApp Proprietary

10 Name: flexvol Aggregate: aggr1_node1 Storage Type: NAS Total Size: 1 GB Space Reserve (optional): Thin Provisioned 11. Click Create Figure 3-6: The Create Volume windows closes, and focus returns to the Volumes pane. 12. Click the Namespace sub-tab. 13. In the Namespace list, select the entry for the flexvol storage object. 14. Observe that the junction path for this storage object is /flexvol, and that it is set to the default export policy. This means the FlexVol is exported to NFS through /flexvol and uses the export policy default for its access NetApp, Inc. All rights reserved. NetApp Proprietary

11 Figure 3-7: Examine the SVM's export policies. Click the SVM Settings sub-tab. In the left pane of the SVM Settings view, under the Policies section, click Export Policies. In the policy list in the right pane, select the entry for default. The rules for the default export policy are set to allow wide open access to all NFS clients. 217 NetApp, Inc. All rights reserved. NetApp Proprietary

12 Figure 3-8: Next create a CIFS share for this FlexVol volume. 19. Click the Shares sub-tab. 2. Click Create Share Figure 3-9: The Create Share window opens NetApp, Inc. All rights reserved. NetApp Proprietary

13 21. Populate the fields in this window as follows: Folder To Share: /flexvol Share Name: flexvol 22. Click Create Figure 3-1: The Create Share window closes and focus returns to the Shares view for the DEMO SVM in System Manager. 23. A new CIFS share named flexvol now appears in the share list NetApp, Inc. All rights reserved. NetApp Proprietary

14 23 Figure 3-11: 3.2 Create and Share a FlexGroup Volume In this activity you create a FlexGroup volume from OnCommand System Manager. As you will see, most of the operational procedure is very similar to that for creating a FlexVol. 1. You should already have System Manager open from the preceding exercise. If not, launch Chrome from the taskbar, and log in with username admin, and password NetApp1!. 1 Figure 3-12: The browser's home page is set to automatically open tabs for OnCommand System Manager (for the lab's ONTAP cluster, cluster1), and for OnCommand Unified Manager. 2. Click the SVMs tab NetApp, Inc. All rights reserved. NetApp Proprietary

15 2 Figure 3-13: 3. Click the entry for the Demo SVM. 3 Figure 3-14: 4. Click the Volumes sub-tab. 5. In the Volumes pane, click the FlexGroups sub-heading, which navigates you to the pane for managing FlexGroups. 6. Click Create NetApp, Inc. All rights reserved. NetApp Proprietary

16 5 4 6 Figure 3-15: The browser displays the Create FlexGroup page. 7. Populate the fields on this page as follows: Name: flexgroup Aggregates: Select aggregates Aggregate Selection List: Check aggr1_node1 Check aggr1_node2 Space Reserve (optional): Thin Provisioned Size: 1 TB Note: Observe that you cannot select which protocols to enable, as these are automatically derived from the protocols that are enabled on the SVM. 8. Click Create NetApp, Inc. All rights reserved. NetApp Proprietary

17 Figure 3-16: The browser displays the Summary page showing the details of the FlexGroup you just created. Note that the FlexGroups supports both the CIFS and NFS protocols 9. Acknowledge the SUCCESS notification by closing the green notification pop-up. 1. Click Create Share to create a CIFS share for this FlexGroup NetApp, Inc. All rights reserved. NetApp Proprietary

18 9 1 Figure 3-17: The browser displays the Shares page for the DEMO SVM. 11. Click Create Share NetApp, Inc. All rights reserved. NetApp Proprietary

19 11 The Create Share window opens. Figure 3-18: 12. Populate the fields in this window as follows: Folder To Share: /flexgroup. Share Name: flexgroup. 13. Click Create NetApp, Inc. All rights reserved. NetApp Proprietary

20 12 13 Figure 3-19: The Create Share window closes and focus returns to the Shares view for the DEMO SVM in System Manager. 14. A new CIFS share named flexgroup now appears in the share list NetApp, Inc. All rights reserved. NetApp Proprietary

21 14 Figure 3-2: 15. Click the Namespace sub-tab. 16. In the Namespace list, select the entry for the flexgroup storage object. 17. View the General properties at the bottom of the page. Note the junction path is /flexgroup, and the export policy is set to default. That means the FlexGroup volume is exported to NFS through / flexgroup, and uses the default export policy for its access, which as you recall from the previous exercise has rules that allow access to everyone NetApp, Inc. All rights reserved. NetApp Proprietary

22 Figure 3-21: 18. On the taskbar of Jumphost, launch Windows Explorer. 18 Figure 3-22: Windows Explorer opens. 19. In Windows Explorer menu bar, enter the UNC path \\flexgroup, and hit Enter. Note: The Storage Virtual Machine you are using for this lab is named DEMO, but the CIFS server name for this SVM is FLEXGROUP. Since under Windows UNC names are not casesensitive, you are using the lower case form flexgroup here NetApp, Inc. All rights reserved. NetApp Proprietary

23 19 Figure 3-23: This CIFS server currently offers two shares, flexgroup, and flexvol. 2. Double-click the flexgroup share. 2 Figure 3-24: 21. Right-click inside the flexgroup share. 22. In the context menu navigate to New > Text Document NetApp, Inc. All rights reserved. NetApp Proprietary

24 21 22 Figure 3-25: 23. Accept the default name for the new text file. If you received no errors then you created the file successfully. Feel free to edit the file and save the new contents to verify that SMB/CIFS access to the NetApp FlexGroup volume is working properly NetApp, Inc. All rights reserved. NetApp Proprietary

25 23 Figure 3-26: 3.3 Examine FlexVol and FlexGroup Volume Layout In this activity you use the ONTAP CLI to examine how a FlexVol volume and a FlexGroup volume are laid out differently on the storage. 1. On the taskbar of Jumphost, launch PuTTY. 1 Figure 3-27: The PuTTY Configuration window opens. 2. If the right pane does not contain the header Basic options for your PuTTY session, then in the left pane select the Session category. 3. In the right pane, in the Saved Sessions list, double-click cluster NetApp, Inc. All rights reserved. NetApp Proprietary

26 2 3 Figure 3-28: A PuTTY teminal session window opens. 4. Log in with the username admin, and password Netapp1!. login as: admin Using keyword-interactive authentication. Password: cluster1::> 5. Display the namespace junction paths for the flexvol and flexgroup volumes. cluster1::> volume show -vserver DEMO -volume flex* -fields junction-path vserver volume junction-path DEMO flexgroup /flexgroup DEMO flexvol /flexvol 2 entries were displayed. cluster1::> Both volumes are alike in that they each have a single (albeit different) junction point in the namespace. 6. Display the layout for the flexvol and flexgroup volumes. cluster1::> volume show -vserver DEMO -volume flex* vserver volume aggregate aggr-list DEMO flexgroup aggr1_node1,aggr1_node2 DEMO flexvol aggr1_node1 aggr1_node1 2 entries were displayed. -fields nodes,aggregate,aggr-list nodes cluster1-1,cluster1-2 cluster1-1 cluster1::> NetApp, Inc. All rights reserved. NetApp Proprietary

27 The flexvol volume only spans one aggregate on a single node. The flexgroup volume spans multiple nodes and multiple aggregates. This allows a FlexGroup to leverage all of these resources, potentially improving overall performance for workloads that utilize the FlexGroup. Tip: Observe the - character reported as the Aggregate value for the flexgroup volume. This is an indication that this volume has constituent members (i.e., that it is a FlexGroup). Figure 3-29: 7. Display a more detailed view of the constituents that comprise this FlexGroup. cluster1::> volume show -vserver DEMO -is-constituent true Vserver Volume Aggregate State Type Size Available Used% DEMO flexgroup 1 aggr1_node1 online RW 1.25TB 1.19TB 5% DEMO flexgroup 2 aggr1_node2 online RW 1.25TB 1.19TB 5% DEMO flexgroup 3 aggr1_node1 online RW 1.25TB 1.19TB 5% DEMO flexgroup 4 aggr1_node2 online RW 1.25TB 1.19TB 5% DEMO flexgroup 5 aggr1_node1 online RW 1.25TB 1.19TB 5% DEMO flexgroup 6 aggr1_node2 online RW 1.25TB 1.19TB 5% DEMO flexgroup 7 aggr1_node1 online RW 1.25TB 1.19TB 5% DEMO flexgroup 8 aggr1_node2 online RW 1.25TB 1.19TB 5% 8 entries were displayed. cluster1::> FlexGroup volumes consist of multiple FlexVol member volumes that work together to create a multithreaded, large bucket container. This FlexGroup is comprised of 8 independent FlexVol member volumes. Even though it has multiple member volumes, a FlexGroup only has one junction into the NetApp, Inc. All rights reserved. NetApp Proprietary

28 namespace. This is how a single FlexGroup volume can outperform a single FlexVol volume, through concurrency of ingest operations that are spread out across the member volumes. Note that each member volume in this example is 1.25 TB in size. This is because ONTAP will divide the total specified size of the FlexGroup (1 TB in those cases) amongst the FlexVol member volumes. The available size in each member is 1.19 TB because there is 5% snap reserve. 8. Create a snaphot on each of the flexvol and flexgroup volumes. cluster1::> snapshot create -vserver DEMO -volume flexvol -snapshot base cluster1::> snapshot create -vserver DEMO -volume flexgroup -snapshot base cluster1::> The procedure to create a snapshot is the same for both FlexVol and FlexGroup volumes. 9. View the snapshots for these volumes. cluster1::> snapshot show -vserver DEMO -volume flex* ---Blocks--Vserver Volume Snapshot Size Total% Used% DEMO flexgroup hourly _ MB % 31% base 8KB % 15% flexvol hourly _25 84KB % 14% base 268KB % 34% 4 entries were displayed. cluster1::> 1. Elevate the cluster shell to diag mode, and display the snapshots again. cluster1::> set diag Warning: These diagnostic commands are for use by NetApp personnel only. Do you want to continue? {y n}: y cluster1::*> snapshot show -vserver DEMO -volume flex* ---Blocks--Vserver Volume Snapshot Size Total% Used% DEMO flexgroup hourly _ MB % 31% base 84KB % 15% flexgroup 1 hourly _25 268KB % 32% base 76KB % 12% flexgroup 2 hourly _25 276KB % 32% base 76KB % 12% flexgroup 3 hourly _25 268KB % 33% base 76KB % 12% flexgroup 4 hourly _25 292KB % 34% base 76KB % 12% flexgroup 5 hourly _25 28KB % 33% base 76KB % 12% flexgroup 6 hourly _25 264KB % 32% base 76KB % 12% flexgroup 7 hourly _25 88KB % 15% base 272KB % 35% flexgroup 8 hourly _25 244KB % 31% base 76KB % 12% flexvol hourly _25 84KB % 14% base 268KB % 34% 2 entries were displayed. cluster1::*> NetApp, Inc. All rights reserved. NetApp Proprietary

29 Tip: You may see zero or more hourly, daily, and weekly snapshots listed beside just the base snapshot depending on how long your lab has been up and running. Only the base snapshots are important for this exercise There are entries here for the flexvol and flexgroup volumes that you saw generated by the unprivileged command in the preceding step. However, there are now also entries for each of the member volumes ( flexvol 1, flexvol 2, etc.) that together comprise the FlexVol volume. Each member volume has a list of snapshots that match those listed for the overall FlexVol ( flexvol in this example). The member snapshots are all created at the same time during a FlexGroup volume snapshot operation, which ensures consistency across the container. The metrics for each member snapshot listing reflect the consumption on just that member, whereas the metrics for the snapshots directly under the overall FlexGroup entry (for example, flexgroup ) reflect a summary for that snapshot across all the members. 11. Change the cluster1 login privilege level back to the default admin level. cluster1::*> set admin cluster1::> Leave your PuTTY session to cluster1 open, as you will need it again later. 3.4 NFS Test - High File Count and Small Files In this activity you run a python script that populates your FlexGroup volume with a directory tree containing 5, small files, so you can see how and why FlexGroups offer superior performance for this workload. 1. On the taskbar on Jumphost, right click on the PuTTY icon. 2. Select PuTTY from the context menu NetApp, Inc. All rights reserved. NetApp Proprietary

30 2 1 Figure 3-3: 3. In the Saved Sessions list, double click on rhel NetApp, Inc. All rights reserved. NetApp Proprietary

31 3 Figure 3-31: 4. Log in with username root, and the password Netapp1!. login as: root password: Last login: Mon Aug 7 17:59:2 217 [root@rhel1 ~]# 5. Mount the flexgroup volume from the flexgroup SVM as /mnt/flexgroup. [root@rhel1 ~]# mount flexgroup:/flexgroup /mnt/flexgroup [root@rhel1 ~]# 6. Review the mount details. [root@rhel1 ~]# mount grep /flexgroup flexgroup:/flexgroup on /mnt/flexgroup type nfs (rw,addr= ) [root@rhel1 ~]# The volume is NFS-mounted from the address , which is a LIF IP for the flexgroup SVM. 7. Create a test file in the mounted filesystem. [root@rhel1 total 12 drwxr-xr-x drwxr-xr-x. -rwxr-xr-x drwxrwxrwx [root@rhel1 [root@rhel1 total 12 drwxr-xr-x drwxr-xr-x. 31 ~]# ls -al /mnt/flexgroup 2 root root 496 Aug 3 21:1. 5 root root 496 Apr 21 13: root bin Aug 3 21:1 New Text Document.txt 12 root root 496 Aug snapshot ~]# touch /mnt/flexgroup/fg_file1 ~]# ls -al /mnt/flexgroup 2 root root 496 Aug root root 496 Apr 21 13: NetApp, Inc. All rights reserved. NetApp Proprietary

32 -rw-r--r-1 root root Aug -rwxr-xr-x 1 root bin Aug drwxrwxrwx 12 root root 496 Aug [root@rhel1 ~]# fg_file1 3 21:1 New Text Document.txt 7 17:5.snapshot In these next steps you will run a script that creates 5, directories and 5, files on the mounted FlexVol. While this script is running you can monitor performance metrics on the cluster. 8. Launch the file creation script, and leave the script running. Important: Complete steps 9-12 while this script is running. Once started, the script takes about 5 minutes to complete. [root@rhel1 ~]# python /scripts/file_create.py /mnt/flexgroup Starting overall work: :8: In your PuTTY session to cluster1, display periodic performance statistics for the cluster. You will need to elevate your privilege level to advanced before you can run this command. Tip: The following command generates wide output, so you may want to maximize your PuTTY session window before you issue the command. cluster1::> set advanced Warning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel. Do you want to continue? {y n}: y cluster1::*> statistics show-periodic cluster1: cluster.cluster: 8/7/217 17:11:3 cpu cpu total fcache total total cluster cluster cluster disk disk pkts pkts avg busy ops nfs-ops cifs-ops ops recv sent busy recv sent read write recv sent % 72% MB 2.66MB % 1.6MB 1.59MB 9.4KB 697KB % 76% MB 4.27MB % 2.54MB 2.51MB 727KB 75KB % 74% MB 2.53MB % 1.52MB 1.5MB 2.76MB 8.35MB % 73% MB 4.6MB % 2.44MB 2.41MB 48.1KB 739KB % 76% MB 2.86MB % 1.71MB 1.69MB 12.3KB 15.4KB % 73% MB 3.95MB % 2.37MB 2.35MB 756KB 923KB % 75% MB 3.62MB % 2.1MB 1.95MB 3.56MB 8.87MB % 72% MB 3.2MB % 1.94MB 1.95MB 87.6KB 472KB cluster1: cluster.cluster: 8/7/217 17:11:25 cpu cpu total fcache total total cluster cluster cluster disk disk pkts pkts avg busy ops nfs-ops cifs-ops ops recv sent busy recv sent read write recv sent Minimums: 64% 72% MB 2.53MB % 1.52MB 1.5MB 12.3KB 15.4KB Averages for 8 samples: 68% 73% MB 3.37MB % 2.1MB 1.99MB 123KB 2.59MB Maximums: 71% 76% MB 4.27MB % 2.54MB 2.51MB 3.56MB 8.87MB data data data busy recv sent % 1.9MB 1.7MB 1% 1.78MB 1.75MB % 1.6MB 1.4MB 1% 1.68MB 1.65MB % 1.18MB 1.17MB 1% 1.62MB 1.6MB 1% 1.67MB 1.67MB % 1.9MB 1.7MB data data data busy recv sent % 1.6MB 1.4MB % 1.4MB 1.38MB 1% 1.78MB 1.75MB cluster1::*> Let a few iterations of the output appear, and examine the values reported in the total ops column. This column reflects the total number of IOPs processed by the cluster during the associated time interval ( NetApp, Inc. All rights reserved. NetApp Proprietary

33 seconds by default). The number of IOPs reported in the nfs-ops column is the same as the total ops column since this is an NFS-only workload. 1. Press Ctrl-c to terminate the statistics command. (If you are on a Mac you might need to use Cmd-c instead.) 11. View the real-time distribution of files across members of the FlexGroup. This command is only available at the diag privilege level, so you will need to elevate privileges first. cluster1::*> set diag Warning: These diagnostic commands are for use by NetApp personnel only. Do you want to continue? {y n}: y cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 7964 msec * next refresh message due in 979 msec (last to member x8d56a2c) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L 1525 % % 12% R 1519 % % 12% L 1539 % % 12% R 1546 % % 12% L 1527 % % 12% R 1543 % % 12% L 1549 % % 12% R 1531 % % 12% Probabilities D-Ingest D- [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 7 msec * next refresh message due in 7 msec (last to member x8d56a2b) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R 1525 % % 12% L 1519 % % 12% R 1539 % % 12% L 1546 % % 12% R 1527 % % 12% L 1543 % % 12% R 1549 % % 12% L 1531 % % 12% Probabilities D-Ingest D- [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + cluster1::*> The output includes sections for each cluster node (cluster1-1 and cluster1-2), and within those sections there are entries for each member volume NetApp, Inc. All rights reserved. NetApp Proprietary

34 The D-Ingest and D-Alloc columns represent ingest and allocation statistics for directories, while the F-Ingest and F-Alloc columns represent ingest and allocation statistics for files. Observe that directories and files are being created across all the member volumes. The Urg, Targ, and Probablilities columns represent Urgency, Tolerance, and Probablity. These are heuristics statistics that ONTAP uses to decide on which member volume it should place an ingested file or directory. Note: For an in-depth explanation of these metrics and the ingestion heuristics in general, please refer to the NetApp FlexGroup: A Technical Overview document listed in the References section of this lab guide. For the pupose of this exercise, it's only important to note that the Urg, Targ and Probabilities displayed for each member volume are relatively even, meaning that the ingest heuristics are maintaining an even distribution of local files and folders across the members. 12. Run the command a few times to see the changes in file and directory ingest and allocation, as well as any changes in target or urgency. Tip: You can use the up-arrow key to recall previous commands. cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 2375 msec * next refresh message due in 467 msec (last to member x8d56a2e) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L % % 12% R % % 13% L % % 12% R % % 12% L % % 12% R % % 12% L % % 11% R % % 12% Probabilities D-Ingest D- [1% 1% %] + + [1% 1% 86% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% 88% %] + + [1% 1% %] + + Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 4339 msec * next refresh message due in msec (last to member x8d56a29) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R % % 12% L % % 13% R % % 12% L % % 12% R % % 12% L % % 12% Probabilities D-Ingest D- [1% 1% %] + + [1% 1% 86% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] + + [1% 1% %] NetApp, Inc. All rights reserved. NetApp Proprietary

35 R L % % 11% [1% 1% 88% %] + + % % 12% [1% 1% %] + + cluster1::*> Note how the target and urgency percentages all stay relatively close in value across different command invocations. That means the FlexGroup is ingesting data very evenly, and is not seeing a large discrepancy in file or folder allocation for this workload. 13. Return the PuTTY session for cluster1 back to the admin privilege level. cluster1::*> set admin cluster1::> As you wait for the script running on rhel1 to finish, please review the following material. The best way to get a feel for how a FlexGroup typically outperforms a FlexVol for this type of workload is to have you run a similar test against a FlexVol volume. Unfortunately, that's not practical in this lab for the following reason. This lab uses simulated rather than physical controllers (i.e., vsims rather than FAS/AFF hardware). While the vsims are great for demonstrating functionality, they are much more limited when it comes to demonstrating performance. Creating 5, files might seem like a large workload, but when running on a simulator it is actually not large enough to significantly differentiate the results between a FlexGroup and a FlexVol. Increasing the number of files the script creates would increase that differentiation, but at the cost of a significantly longer run time and, more importantly, a much longer inlab wait time for you. Repeating such a test twice, once for each of a FlexGroup and a FlexVol, would simply take too long for this lab's alloted time. Still, this lab includs a live exercise of running this workload against a FlexGroup on a vsim so you can see how to run such a performance test and view the ingestion statistics. To give you a better feel for what kind of performance benefits FlexGroups can offer, the following graphs show you the results of a more realistic workload that was run using physical storage controller hardware. In this test of a 2-node A7 cluster, a GIT compile of a gcc library on a FlexGroup volume showed better overall performance vs. a FlexVol volume. Observe that as the workload increases, the FlexGroup offered 6x the throughput vs. the FlexVol NetApp, Inc. All rights reserved. NetApp Proprietary

36 Figure 3-32: These A7 controllers were able to provide 2.5x times the IOPs from the FlexGroup vs. the FlexVol with the same workload. Figure 3-33: NetApp, Inc. All rights reserved. NetApp Proprietary

37 A NetApp FlexGroup volume was able to complete these jobs in approximately 1/3 of the total time of a FlexVol volume. (In this test, FlexGroup 3 is all local to a single node, while FlexGroup 1 is spread over 2 nodes.) Figure 3-34: When more nodes are added to the cluster, performance can scale. The following graph shows the same test run on a 4-nodeAFF88 cluster. (In this test, FlexGroup3 is again all local to a single node, whereas FlexGroup 1 is hosted across 2 nodes. ) NetApp, Inc. All rights reserved. NetApp Proprietary

38 Figure 3-35: In benchmark tests, a NetApp FlexGroup can reach upwards of 2, IOPS for a single container before hitting a significant latency spike. (This particular test was run on a FAS88 with SAS drives.) Figure 3-36: 14. Switch to the PuTTY session to rhel1. Once the script is finished running in your lab, note the execution time reported by your script. Important: There are no commands to run in this step, just examine the output shown in these two command output sections and in your PuTTY session to rhel1 in your lab so you can compare them. ### Example execution time in this lab NetApp, Inc. All rights reserved. NetApp Proprietary

39 ~]# python /scripts/file_create.py /mnt/flexgroup Starting overall work: :8: End overall work: :12: total time: ~]# The script required seconds to complete in this example. Compare the execution time from your lab with the results of the same script run on a CentOS client VM against a FlexGroup hosted on an AFF84. ### Example execution time for an AFF84 # python /scripts/file-create.py /flexgroup/ Starting overall work: :51: End overall work: :53: total time: The same script tool just 98.9 seconds on an AFF88. Leave your PuTTY sessions running for use in later exercises. 3.5 NFS Test - Low File Count and Large Files In this activity you run a script that utlizes the linux dd command to create a small number of large files on your FlexGroup volume. 1. In your PuTTY session to rhel1, launch the script to generate the files. This script creates 8 1MB files on /mnt/flexgroup, and takes approximately 1-2 minutes to complete. While it runs you will examine performance metrics in ONTAP, so continue with this exercise without delay after launching the script. Attention: This script only takes a minute or so to complete, and you need to complete steps 2-3 while it is still running! [root@rhel1 ~]# cd /scripts [root@rhel1 scripts]# time./dd_create_fg.sh 8 Starting clients... Writing Switch to your PuTTY session for cluster1, and view the real-time distribution of files across the members of the FlexGroup volume. You will need to elevate to the diag privilege level to run this command. cluster1::> set diag Warning: These diagnostic commands are for use by NetApp personnel only. Do you want to continue? {y n}: y cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 921 msec * next refresh message due in 1 msec (last to member x8d56a28) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L % % % R 364 % % 22% L % % 18% R % % % + 39 Probabilities D-Ingest D- [1% 1% 99% %] + + [1% 1% 77% %] + + [1% 1% 81% %] + + [1% 1% 1% %] NetApp, Inc. All rights reserved. NetApp Proprietary

40 L R L R % % 6% [1% 1% 93% %] + + % % 3% [1% 1% 96% %] + + % % 18% [1% 1% 81% %] + + % % 3% [1% 1% 69% %] + + Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 814 msec * next refresh message due in 14 msec (last to member x8d56a27) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R % % % L 364 % % 22% R % % 18% L % % % R % % 6% L % % 3% R % % 18% L % % 3% 3+ Probabilities D-Ingest D- [1% 1% 99% %] + + [1% 1% 77% %] + + [1% 1% 81% %] + + [1% 1% 1% %] + + [1% 1% 93% %] + + [1% 1% 96% %] + + [1% 1% 81% %] + + [1% 1% 69% %] + + cluster1::*> The allocation statistics in this exercise for a large files workload should look different from those you saw in the preceding exercise for a small files workload. Instead of the Targ and Probabilities values being even, you should now see a higher variation in values across member volumes. When a member shows a Targ value of % it means it is less likely that this member volume will be the target for the next file placement. This pattern occurs during this workload because as larger files get written to the member volumes and fill their free space, other members that are not as utilized become more likely landing spots for additional new files. 3. Repeat the command for the file distribution statistics several times, and note how the percentages change over time. This is because the ingest algorithms adjust as new files are written and space is utilized. cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 858 msec * next refresh message due in 5 msec (last to member x8d56a28) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L 1624 % % 9% R % % % 3 15 L 9659 % % 1% R 5815 % % 26% L % % 1% R % % 1% 4 Probabilities D-Ingest [1% 1% D- 9% %] + + [1% 1% 1% %] + + [1% 1% 89% %] + + [1% 1% 73% %] + + [1% 1% 89% %] + + [1% 1% 89% %] NetApp, Inc. All rights reserved. NetApp Proprietary

41 L R % % 26% [1% 1% 73% %] + + % % 5% [1% 1% 94% %] Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 579 msec * next refresh message due in msec (last to member x8d56a2d) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R % % 7% L % % % R 9659 % % 12% L 5815 % % 27% R % % 12% L 131 % % 9% R 5863 % % 27% L % % 4% 1 1 Probabilities D-Ingest [1% 1% D- 92% %] + + [1% 1% 1% %] + + [1% 1% %] + + [1% 1% 72% %] + + [1% 1% %] + + [1% 1% 9% %] + + [1% 1% 72% %] + + [1% 1% 95% %] + + cluster1::*> 4. Switch back to rhel1, and wait for the script to finish. ### Example output from the script execution for this exercise. [root@rhel1 scripts]# time./dd_create_fg.sh 8 Starting clients... Writing... Reading... Collecting results... Done real 1m.253s user m.181s sys m.16s [root@rhel1 scripts]# 5. Switch back to cluster1, and issue the command to display the file distribution statistics one more time. cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 1358 msec * next refresh message due in 373 msec (last to member x8d56a2a) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L % % 12% R % % % L % % 12% R 7959 % % 29% L % % 8% R % % 12% L % % 24% Probabilities D-Ingest D- [1% 1% ] + + [1% 1% 99% 99%] + + [1% 1% ] + + [1% 1% 7% 7%] + + [1% 1% 91% 91%] + + [1% 1% ] + + [1% 1% 75% 75%] NetApp, Inc. All rights reserved. NetApp Proprietary

42 8 155 R % % % [1% 1% 1% 1%] Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 626 msec * next refresh message due in 26 msec (last to member x8d56a27) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R % % 12% L % % % R % % 12% L 7959 % % 29% R % % 8% L % % 12% R % % 24% L % % % + + Probabilities D-Ingest D- [1% 1% ] + + [1% 1% 99% 99%] + + [1% 1% ] + + [1% 1% 7% 7%] + + [1% 1% 91% 91%] + + [1% 1% ] + + [1% 1% 75% 75%] + + [1% 1% 1% 1%] + + cluster1::*> The file and directory ingest and alloc fields are for all the members, because the script has ended and there are not any new directories or files being placed. Since the percentages are so different from member to member, it is clear that the member volumes did not see even new file allocation, which makes sense given that the most recent workload was designed to produce uneven file allocation. If new directory and file placement were to resume, can you tell which member volume would be most likely to receive new file placement? (Hint: A lower Targ means less likely, a higher Targ means more likely. If you sum up the Targ values across members it should equal approximately 1%, meaning the member with the largest Targ value will likely be the next target. ) 6. Return the cluster command prompt back to the admin privilege level. cluster1::*> set admin cluster1::> Leave your PuTTY sessions open for use in later exercises. 3.6 Viewing FlexGroup Performance with OnCommand Performance Manager In this activity you view performance metrics for a FlexGroup volume and its member volumes using OnCommand Performance Manager. This activity cannot explore too deeply into specific FlexGroup performance scenarios because it uses simulated ONTAP controllers running on a shared virtualization infrastructure. You would need real dedicated FAS/AFF hardware for that. However, this exercise does give you a flavor of the kinds of performance data that OnCommand Performance Manager tracks for you, and how you can view that data. 1. In your web browser on the Jumphost (recall that this lab guide uses Chrome), select the tab for OnCommand Unified Manager. 2. Log in with username admin, and password Netapp1! NetApp, Inc. All rights reserved. NetApp Proprietary

43 1 2 2 Figure 3-37: The browser displays the Manage Clusters page. 3. In the left pane, under the list of Managed Clusters, click cluster1. Note: OnCommand Unified works best with a large windows, so you should maximize your browser window NetApp, Inc. All rights reserved. NetApp Proprietary

44 3 Figure 3-38: 4. Click the Menu icon at the top left of the page. 5. Select the Performance entry from the sub-menu NetApp, Inc. All rights reserved. NetApp Proprietary

45 4 5 Figure 3-39: The browser displays a performance summary for the one cluster in your environment. 6. On the menu bar at the top of the OnCommand Unified Manager window, select Storage 7. In the sub-menu, select SVMs NetApp, Inc. All rights reserved. NetApp Proprietary

46 6 7 Figure 3-4: 8. Click DEMO NetApp, Inc. All rights reserved. NetApp Proprietary

47 8 Figure 3-41: The browser displays the SVM:DEMO Explorer view. 9. In the volume list in the left pane, find the entry for the flexgroup volume, and click Add. 1. At the top of the right pane, click the Time Range dropdown NetApp, Inc. All rights reserved. NetApp Proprietary

48 1 9 Figure 3-42: A calender sub-window opens. 11. In the right side of the calendar sub-window, click Last Hour. 12. Click Apply Range NetApp, Inc. All rights reserved. NetApp Proprietary

49 11 12 Figure 3-43: The calender sub-window closes, and the right pane displays a series of performance charts. 13. OnCommand Performance Manager is a historical rather than a real-time monitoring tool. You may have to wait several minutes after the workload scripts complete before you are able to see any metrics for the wokloads in the graphs. Use the Refresh button (in the upper right corner of the browser window, not shown) to periodically refresh the screen until you see non-idle metrics. It is possible that you completed the last two exercises in close proximity to each other, so the graphs may show some overlapping results for the two workload scripts. Tip: You can click the Zoom View button for a given graph to see a larger view of it. When you are done examining the zoomed graph, in the left pane click flexgroup to view the full set of graphs again NetApp, Inc. All rights reserved. NetApp Proprietary

50 13 Figure 3-44: The graph waveforms are triangular, which makes sense for a short duration workload running on an otherwise idle system. In the small file test the IOPs are relatively high while the MBps throughput is relatively low. This is reasonable because the workload creates lots of small non-zero files. There is data being written to disk, but it does not require a lot of throughput to write that small amount of data. The large file test is essentially a streaming workload with a much smaller number of files, meaning greater throughput and latency, but that does not translate to nearly as many overall IOPs as in the small file test, which is why the IOPs graph for the large file test looks smaller. 14. Notice that the left pane also includes entries for each of the flexgroup member volumes. Click Add for each of the member volumes in the order of their member names NetApp, Inc. All rights reserved. NetApp Proprietary

51 14 14 Figure 3-45: 15. Notice the general evenness of the graphs for each FlexGroup member volume. This means that the ingest of the files and folders is taking place evenly across all member volumes and nodes in the cluster. The latency curve is predictable across the volumes. Also take note that the flexgroup volume graph line is well above the individual member volume lines; this is because the FlexGroup totals are an aggregation of the total of all the member volumes. This shows how a FlexGroup is able to scale performance by adding new members NetApp, Inc. All rights reserved. NetApp Proprietary

52 15 Figure 3-46: 3.7 FlexGroup File Layout In this activity you use the ONTAP volume explore diagnostic command to examine metadata in the WAFL filesystem. This command can be destructive if used improperly, so in a production environment you should only use this command with the assistance of NetApp technical support personal. However, the command is safe to run in this lab. Additionally, this activity provides a general idea of where specific files and directories might live within a FlexGroup, as well as how files are distributed for different workload types. The two workload scripts you ran in an earlier activity populated your flexgroup volume with directories and files. The following description of those directory structures will help you understand the command output you will view in this exercise. Many small files workload: Starting at the root of flexgroup, the script created 1 directories named topdir_ to topdir_9, each containing 5 subdirectories named subdir_ to subdir_4999, which each in turn contain 1 files named file1 to file1. Few large files workload: Starting at the root of flexgroup, the script created 8 directories named c to c7, each of which contains a.log and.out file prefixed by the containing directory name (e.g the c6 directory contains c6.log and c6.out), plus 8 large files named F1.dat to F8.dat. 1. In your PuTTY session to cluster1, elevate to the diag privilege level. cluster1::> set diag Warning: These diagnostic commands are for use by NetApp personnel only. Do you want to continue? {y n}: y NetApp, Inc. All rights reserved. NetApp Proprietary

53 cluster1::*> 2. Display a list of the directories and files located in the root of the flexgroup volume. The flexgroup.64 notation indicates you are querying inode 64 in the flexgroup volume. In WAFL, inode 64 always corresponds to the volume's root directory. cluster1::*> volume explore -format dir flexgroup.64 directory data from block @ at location 148@635523b entry : inum 64, generation , name "." entry 1: inum 64, generation , name ".." entry 2: inum flexgroup , generation , type 1, name "New Text Document.txt" (8.3 "NEWTEX~1.TXT") entry 3: inum 97, generation , name "fg_file1" (8.3 "FG_FILE1") entry 4: inum flexgroup 7.96, generation , type 2, name "topdir_" (8.3 "TOPDIR_") entry 5: inum flexgroup 4.96, generation 46985, type 2, name "topdir_1" (8.3 "TOPDIR_1") entry 6: inum flexgroup 7.97, generation , type 2, name "topdir_2" (8.3 "TOPDIR_2") entry 7: inum flexgroup 4.97, generation 46986, type 2, name "topdir_3" (8.3 "TOPDIR_3") entry 8: inum flexgroup 6.96, generation 46986, type 2, name "topdir_4" (8.3 "TOPDIR_4") entry 9: inum flexgroup 8.96, generation 46987, type 2, name "topdir_5" (8.3 "TOPDIR_5") entry 1: inum flexgroup 3.96, generation , type 2, name "topdir_6" (8.3 "TOPDIR_6") entry 11: inum flexgroup 2.96, generation 46989, type 2, name "topdir_7" (8.3 "TOPDIR_7") entry 12: inum 6581, generation , name "topdir_9" (8.3 "TOPDIR_9") entry 13: inum flexgroup 5.96, generation , type 2, name "topdir_8" (8.3 "TOPDIR_8") entry 14: inum flexgroup , generation , type 2, name "c6" (8.3 "C6") entry 15: inum flexgroup , generation , type 2, name "c1" (8.3 "C1") entry 16: inum flexgroup , generation , type 2, name "c7" (8.3 "C7") entry 17: inum flexgroup 8.973, generation , type 2, name "c5" (8.3 "C5") entry 18: inum 832, generation , name "c" (8.3 "C") entry 19: inum flexgroup , generation , type 2, name "c4" (8.3 "C4") entry 2: inum flexgroup , generation , type 2, name "c2" (8.3 "C2") entry 21: inum flexgroup , generation 4651, type 2, name "c3" (8.3 "C3") cluster1::*> The output shows an entry for each directory/file in the specified volume and directory (in this case, the root directory of the flexgroup volume). The value following the name parameter represents the entry's file or directory name. The value following the inum parameter represents the entry's inode number. If an inum is local to the volume then the inum value is purely numeric. If an inum is remote, meaning for a FlexGroup that it is stored in a member volume, then the inum value is prefixed by the member volume name and a dot character. Observe that many of these subdirectories are alocated to different remote member volumes. 3. Find an entry for a topdir_ subdirectory that has a remote inum (which in this exercise is an inum that contains the substring flexgroup ). Display a list of the files and directories in this remote sub-directory. To do that, repeat the preceding command, but replace the flexgroup.64 argument with the inum value for your chosen remote entry. In this example we've chosen the entry for the topdir_5 directory, which has an inum value of flexgroup cluster1::*> volume explore -format dir flexgroup 8.96 directory data from block @ at location 155@587744b entry : inum 96, generation 46987, name "." entry 1: inum flexgroup 1.64, generation , type 2, name ".." entry 2: inum 588, generation 46989, name "subdir_" (8.3 "SUBDIR_") entry 3: inum flexgroup , generation , type 2, name "subdir_1" (8.3 "SUBDIR_1") entry 4: inum flexgroup , generation , type 2, name "subdir_2" (8.3 "SUBDIR_2") entry 5: inum flexgroup , generation 46116, type 2, name "subdir_3" (8.3 "SUBDIR_3") entry 6: inum flexgroup , generation , type 2, name "subdir_4" (8.3 "SUBDIR_4") NetApp, Inc. All rights reserved. NetApp Proprietary

54 .. directory data from block at location entry : inum flexgroup , generation , type 2, name "subdir_4983" (8.3 "KQF21~") entry 1: inum 864, generation , name "subdir_4984" (8.3 "PE2~") entry 2: inum flexgroup , generation , type 2, name "subdir_4985" (8.3 "DHR11~") entry 3: inum flexgroup , generation , type 2, name "subdir_4986" (8.3 "KA41~") entry 4: inum flexgroup , generation , type 2, name "subdir_4987" (8.3 "HR1~") entry 5: inum flexgroup , generation , type 2, name "subdir_4988" (8.3 "8LQ2~") entry 6: inum flexgroup , generation , type 2, name "subdir_4989" (8.3 "BC92~") entry 7: inum flexgroup , generation , type 2, name "subdir_499" (8.3 "WOC2~") entry 8: inum flexgroup , generation , type 2, name "subdir_4991" (8.3 "L2E2~") entry 9: inum flexgroup , generation , type 2, name "subdir_4992" (8.3 "MW1~") entry 1: inum flexgroup , generation , type 2, name "subdir_4993" (8.3 "HWD2~") entry 11: inum flexgroup 2.368, generation , type 2, name "subdir_4994" (8.3 "V31~") entry 12: inum 8651, generation , name "subdir_4995" (8.3 "BPE2~") entry 13: inum flexgroup , generation , type 2, name "subdir_4996" (8.3 "3572~") entry 14: inum flexgroup , generation , type 2, name "subdir_4997" (8.3 "3SA1~") entry 15: inum flexgroup , generation , type 2, name "subdir_4998" (8.3 "AIH2~") entry 16: inum flexgroup , generation , type 2, name "subdir_4999" (8.3 "DPW1~") cluster1::*> This command generates quite a lot of lines of output; remember that each topdir_ directory contains 5 subdirectories! Once again, the entries for these subdirectories (e.g., subdir_...) are spread across different remote member volumes. 4. Pick one of the last remote subdirectories in the command output, and use that subdirectory's inum value to list its contents. This example uses the value from entry 15 for the preceding command example, inum flexgroup , which is itself allocated to member volume flexgroup_4. cluster1::*> volume explore -format dir flexgroup directory data from block @ at location 151@49183b entry : inum 82474, generation , name "." entry 1: inum flexgroup 8.96, generation 46987, type 2, name ".." entry 2: inum 82475, generation , name "file1" (8.3 "FILE1") entry 3: inum 82476, generation , name "file2" (8.3 "FILE2") entry 4: inum 82477, generation , name "file3" (8.3 "FILE3") entry 5: inum 82478, generation , name "file4" (8.3 "FILE4") entry 6: inum 82479, generation , name "file5" (8.3 "FILE5") entry 7: inum 8248, generation , name "file6" (8.3 "FILE6") entry 8: inum 82481, generation , name "file7" (8.3 "FILE7") entry 9: inum 82482, generation , name "file8" (8.3 "FILE8") entry 1: inum 82484, generation , name "file9" (8.3 "FILE9") entry 11: inum 82485, generation , name "file1" (8.3 "FILE1") cluster1::*> Recall that the workload script only created two levels of directories, and then placed files in the lowest subdirectories. Observe that the listed files (recall that the script only created 2 levels of directories and then files) are all local to the member volume, so in this case they are also hosted on member volume 4. This is because a FlexGroup favors using the local member rather than a remote member for workloads with many small files in order to improve performance. The ingest algorithm produces different results for workloads with larger files, as you will now see as you examine the files produced by the large files workload script NetApp, Inc. All rights reserved. NetApp Proprietary

55 5. Re-run the command to list the files and directories in the flexgroup volume's root directory. cluster1::*> volume explore -format dir flexgroup.64 directory data from block at location entry : inum 64, generation , name "." entry 1: inum 64, generation , name ".." entry 2: inum flexgroup , generation , type 1, name "New Text Document.txt" (8.3 "NEWTEX~1.TXT") entry 3: inum 97, generation , name "fg_file1" (8.3 "FG_FILE1") entry 4: inum flexgroup 7.96, generation , type 2, name "topdir_" (8.3 "TOPDIR_") entry 5: inum flexgroup 4.96, generation 46985, type 2, name "topdir_1" (8.3 "TOPDIR_1") entry 6: inum flexgroup 7.97, generation , type 2, name "topdir_2" (8.3 "TOPDIR_2") entry 7: inum flexgroup 4.97, generation 46986, type 2, name "topdir_3" (8.3 "TOPDIR_3") entry 8: inum flexgroup 6.96, generation 46986, type 2, name "topdir_4" (8.3 "TOPDIR_4") entry 9: inum flexgroup 8.96, generation 46987, type 2, name "topdir_5" (8.3 "TOPDIR_5") entry 1: inum flexgroup 3.96, generation , type 2, name "topdir_6" (8.3 "TOPDIR_6") entry 11: inum flexgroup 2.96, generation 46989, type 2, name "topdir_7" (8.3 "TOPDIR_7") entry 12: inum 6581, generation , name "topdir_9" (8.3 "TOPDIR_9") entry 13: inum flexgroup 5.96, generation , type 2, name "topdir_8" (8.3 "TOPDIR_8") entry 14: inum flexgroup , generation , type 2, name "c6" (8.3 "C6") entry 15: inum flexgroup , generation , type 2, name "c1" (8.3 "C1") entry 16: inum flexgroup , generation , type 2, name "c7" (8.3 "C7") entry 17: inum flexgroup 8.973, generation , type 2, name "c5" (8.3 "C5") entry 18: inum 832, generation , name "c" (8.3 "C") entry 19: inum flexgroup , generation , type 2, name "c4" (8.3 "C4") entry 2: inum flexgroup , generation , type 2, name "c2" (8.3 "C2") entry 21: inum flexgroup , generation 4651, type 2, name "c3" (8.3 "C3") cluster1::*> Pick one of the directories named cn that is hosted on a remote member volume, and note its inum value for use in the next step. This example will use the directory c5 with inum flexgroup List the files and directories in your selected cn directory. cluster1::*> volume explore -format dir flexgroup directory data from block @ at location 155@467958b entry : inum 973, generation , name "." entry 1: inum flexgroup 1.64, generation , type 2, name ".." entry 2: inum 975, generation , name "c5.log" (8.3 "C5.LOG") entry 3: inum flexgroup , generation , type 1, name "c5.out" (8.3 "C5.OUT") entry 4: inum flexgroup , generation , type 1, name "F1.dat" (8.3 "F1.DAT") entry 5: inum 7686, generation , name "F2.dat" (8.3 "F2.DAT") entry 6: inum 76861, generation , name "F3.dat" (8.3 "F3.DAT") entry 7: inum 76862, generation , name "F4.dat" (8.3 "F4.DAT") entry 8: inum flexgroup , generation , type 1, name "F5.dat" (8.3 "F5.DAT") entry 9: inum 76863, generation , name "F6.dat" (8.3 "F6.DAT") entry 1: inum 76864, generation , name "F7.dat" (8.3 "F7.DAT") entry 11: inum 76865, generation , name "F8.dat" (8.3 "F8.DAT") cluster1::*> This time you should see a blend of local and remote files. The files are not *all* local to the FlexGroup member that owns the directory (flexgroup 8 in this example). This is because the portion of the ingest algorithm that co-locates many small files locally is partially dependent upon the time interval between the creation of each file. The large files workload script created the files serially, and since large files take longer to create than small ones, there was a greater delta between each file's creation time. The files created by the script are all 5 MB each; creating even larger files would have resulted in even greater deltas, and therefore an even higher degree of remote placement NetApp, Inc. All rights reserved. NetApp Proprietary

56 Attention: The 5MB files used in this exercise may still result in all local placement for some directories, but this will not be the case for all of the directories. If the directory you chose has all local files, try picking a different directory to examine; you will eventually find one that has some files with remote placement. 7. Return the cluster login session to the admin privilege level. cluster1::*> set admin cluster1::> Leave your PuTTY sessions open for use in later exercises. 3.8 Resizing a Flexgroup There are two basic strategies available for increasing the size of an existing FlexGroup: Growing the existing member volumes. Adding new member volumes. In most cases, the preferred approach is to grow the sizes of the member volumes, as this helps maintain the I/O balance across the member volumes. You can grow a FlexGroup's member volumes using OnCommand System Manager, or the ONTAP CLI. The first portion of this exercise will show how to do this using ONTAP CLI commands. 1. In your PuTTY session to cluster1, display the current size of your FlexGroup member volumes. cluster1::> volume show -vserver DEMO -is-constituent true -fields size vserver volume size DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB 8 entries were displayed. cluster1::> 2. Add 1TB to the size of the FlexGroup volume flexgroup. cluster1::> volume size -vserver DEMO -volume flexgroup -new-size +1t [Job 49] Job is queued: Modify flexgroup. [Job 49] Steps completed: 8 of 9. vol size: Volume "DEMO:flexgroup" size set to 11t. cluster1::> 3. Display the size of flexgroup's member volumes. cluster1::> volume show -vserver DEMO -is-constituent true -fields size vserver volume size DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB DEMO flexgroup TB 8 entries were displayed. cluster1::> NetApp, Inc. All rights reserved. NetApp Proprietary

57 4. Now try to reduce the size of the member volumes. cluster1::> volume size -vserver DEMO -volume flexgroup -new-size -1t vol size: Error setting size of volume "DEMO:flexgroup". The specified size is not valid because decreasing the size of a FlexGroup is not supported. Current size: 11TB ( B). Requested size: 1TB ( B). cluster1::> Note: ONTAP does not currently support decreasing the size of a FlexGroup. You can also use System Manager to increase the size of a FlexGroup. 5. Switch to your Chrome browser, and select the tab for System Manager. 6. Log in with the following credentials. Username: admin Password: Netapp1! Figure 3-47: System Manager displays the Dashboard page. 7. On the command bar inside System Manager, select the SVMs tab NetApp, Inc. All rights reserved. NetApp Proprietary

58 7 Figure 3-48: System Manager displays the SVMs page. 8. In the SVM list, click DEMO. 8 Figure 3-49: System Manager displays the Overview page for the DEMO SVM. 9. Click the Volumes sub-tab NetApp, Inc. All rights reserved. NetApp Proprietary

59 9 Figure 3-5: 1. In the Volumes pane, click the FlexGroups sub-heading, which navigates you to the pane for managing FlexGroups. 11. In the FlexGroups list, select the entry for flexgroup Figure 3-51: 12. Click the Actions dropdown. 13. Select Resize from the dropdown menu NetApp, Inc. All rights reserved. NetApp Proprietary

60 12 13 Figure 3-52: System Manager displays the Resize FlexGroup page. 14. Set the Size field to 128 TB. 15. Click Resize NetApp, Inc. All rights reserved. NetApp Proprietary

61 15 14 Figure 3-53: System Manager processes the resize request and returns to the FlexGroups view. 16. Acknowledge the succesfull resize operation by clicking the X in the green box NetApp, Inc. All rights reserved. NetApp Proprietary

62 16 Figure 3-54: 17. In the FlexGroups list, again select the entry for flexgroup. 18. Click the Actions dropdown. 19. Select Resize from the dropdown menu Figure 3-55: System Manager displays the Resize FlexGroup page. 2. Set the Size field to 11 TB. 21. Click Resize NetApp, Inc. All rights reserved. NetApp Proprietary

63 21 2 Figure 3-56: System Manager displays an error message indicating that decreasing the size of a FlexGroup is not supported. 22. Acknowledge the error by clicking Ok NetApp, Inc. All rights reserved. NetApp Proprietary

64 22 Figure 3-57: The error window closes, and System Manager returns the the FlexGroups view. Figure 3-58: NetApp, Inc. All rights reserved. NetApp Proprietary

65 3.9 Using Volume Move to Re-distribute Volumes Across a Cluster When you add new nodes to a cluster, you may be able to improve the performance of your existing FlexGroup by moving some member volumes to the new nodes, which may be preferable to adding more member volumes to the FlexGroup. A member volume move can currently only be done using the ONTAP CLI. In this activity you will swap the locations for the member volumes flexgroup 1 and flexgroup_2. To demonstrate that this procedure is non-disruptive, you will re-use the script from the high file count/small files exercise to generate I/O to the FlexGroup while the member moves are in progress. 1. In your PuTTY session to cluster1, display a listing of your current FlexGroup member volumes along with their current aggregate location. cluster1::> volume show -vserver DEMO -is-constituent true -fields volume,aggregate vserver volume aggregate DEMO flexgroup 1 aggr1_node1 DEMO flexgroup 2 aggr1_node2 DEMO flexgroup 3 aggr1_node1 DEMO flexgroup 4 aggr1_node2 DEMO flexgroup 5 aggr1_node1 DEMO flexgroup 6 aggr1_node2 DEMO flexgroup 7 aggr1_node1 DEMO flexgroup 8 aggr1_node2 8 entries were displayed. cluster1::> Note which aggregates are hosting the flexgroup 1 and flexgroup 2 member volumes. In this example they are hosted on aggr1_node1 and aggr1_node2, respectivly, but these aggregate values may be different in your lab. This exercise requires two PuTTY sessions to cluster, so you will now create a second session to cluster1. 2. Right-click inside the PuTTY session window title bar. 3. Select Duplicate Session from the context menu NetApp, Inc. All rights reserved. NetApp Proprietary

66 2 3 Figure 3-59: 4. Login with user name admin, and password Netapp1!. 5. Elevate the cluster shell login to the advanced privilege level. cluster1::> set advanced Warning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel. Do you want to continue? {y n}: y cluster1::*> 6. Display real-time NFS performance metrics. cluster1::*> statistics show-periodic cluster1: cluster.cluster: 8/9/217 18:43:34 cpu cpu total fcache total total data data data cluster cluster cluster disk disk pkts pkts avg busy ops nfs-ops cifs-ops ops recv sent busy recv sent busy recv sent read write recv sent % 4% 41.KB 4.5KB % B B % 41.KB 4.2KB 76.1KB 415KB % 4% 55.3KB 55.2KB % 246B 16B % 55.1KB 54.9KB 19.3KB 384KB % 8% 45.4KB 45.KB % B B % 45.3KB 44.9KB 31.7KB 24.7KB % 4% 35.9KB 35.5KB % 245B 159B % 35.6KB 35.2KB 28.1KB 24.6KB % 17% 7.7KB 72.2KB % B B % 7.7KB 72.1KB 21.2KB 14.2KB NetApp, Inc. All rights reserved. NetApp Proprietary

67 . The cluster is not serving any IOPs because there is currently no workload currently against it. 7. In the PuTTY session for rhel1, create a working directory on /mnt/flexgroup for use by the workload generator, and then launch the workload generator. [root@rhel1 ~]# mkdir /mnt/flexgroup/volmove [root@rhel1 ~]# python /scripts/file_create.py /mnt/flexgroup/volmove Starting overall work: :4: Switch back to the PuTTY session to cluster1 that is running the statistics and verify that the output is showing NFS IOPs. show-periodic command,... cluster1: cluster.cluster: 8/9/217 18:44:22 cpu cpu total fcache total total data data data cluster cluster cluster disk disk pkts pkts avg busy ops nfs-ops cifs-ops ops recv sent busy recv sent busy recv sent read write recv sent % 4% 43.4KB 43.2KB % B B % 43.4KB 43.1KB 133KB 74.KB % 14% KB 48.2KB % 2.5KB 1.72KB % 46.9KB 46.4KB 35.2KB 24.7KB % 1% 39.2KB 39.KB % B B % 39.2KB 38.9KB 14.KB 373KB % 62% MB 1.29MB % 53KB 512KB % 819KB 88KB 1.25MB 1.14MB % 81% MB 4.49MB 1% 1.9MB 1.85MB % 2.67MB 2.64MB 35.1KB 21.1KB % 85% MB 6.2MB 2% 3.4MB 2.93MB % 3.29MB 3.26MB 115KB 7.2KB % 85% MB 4.29MB 1% 2.1MB 1.94MB % 2.18MB 2.16MB 267KB 1.9MB Switch to your original PuTTY session for cluster1, and launch the volume move operations, remembering that you want to adjust the command arguments to swap the aggregate assignments for these two member volumes. In this example, flexgroup 1 started on aggr1_node1, so the following command shows it moving to aggr1_node2. cluster1::> vol move start -vserver DEMO -volume flexgroup 1 -destination-aggregate aggr1_node2 [Job 495] Job is queued: Move "flexgroup 1" in Vserver "DEMO" to aggregate "aggr1_node2". Use the "volume move show -vserver DEMO -volume flexgroup 1" command to view the status of this operation. cluster1::> vol move start -vserver DEMO -volume flexgroup 2 -destination-aggregate aggr1_node1 [Job 496] Job is queued: Move "flexgroup 2" in Vserver "DEMO" to aggregate "aggr1_node1". Use the "volume move show -vserver DEMO -volume flexgroup 2" command to view the status of this operation. cluster1::> 1. Periodically display the status of the volume move operations until they complete, which should only take a minute or two. cluster1::> vol move show -vserver DEMO Vserver Volume State Move Phase Percent-Complete DEMO flexgroup 1 done completed 1% DEMO flexgroup 2 healthy replicating 6% 2 entries were displayed. 67 Time-To-Complete Wed Aug 9 18:46: NetApp, Inc. All rights reserved. NetApp Proprietary

68 cluster1::> vol move show -vserver DEMO Vserver Volume State Move Phase DEMO flexgroup 1 done completed DEMO flexgroup 2 done completed 2 entries were displayed. Percent-Complete Time-To-Complete % - 1% - cluster1::> 11. Switch to your PuTTY session to cluster1 that is running the statistics show-periodic command, and hit Ctrl-c to halt the command. Note that the NFS IOPs never dropped to zero while the volume moves were running, meaning there was no I/O disruption during that time. When you are finished examining the output, exit this PuTTY session.... cluster1: cluster.cluster: 8/9/217 18:44:22 cpu cpu total fcache total total data data data cluster cluster cluster disk disk pkts pkts avg busy ops nfs-ops cifs-ops ops recv sent busy recv sent busy recv sent read write recv sent % 62% MB 1.29MB % 53KB 512KB % 819KB 88KB 1.25MB 1.14MB % 81% MB 4.49MB 1% 1.9MB 1.85MB % 2.67MB 2.64MB 35.1KB 21.1KB % 85% MB 6.2MB 2% 3.4MB 2.93MB % 3.29MB 3.26MB 115KB 7.2KB % 85% MB 4.29MB 1% 2.1MB 1.94MB % 2.18MB 2.16MB 267KB 1.9MB % 82% MB 3.93MB 1% 2.2MB 1.94MB % 2.1MB 1.99MB 1.64MB 8.24MB % 71% MB 3.64MB 1% 2.4MB 1.97MB % 1.71MB 1.67MB 1.37MB 5.22MB % 69% MB 3.23MB 1% 1.36MB 1.32MB % 1.89MB 1.91MB 22.7KB 16.2KB % 67% MB 3.93MB 1% 2.7MB 2.1MB % 1.97MB 1.92MB 126KB 54KB % 69% MB 2.79MB 1% 1.21MB 1.18MB % 1.6MB 1.61MB 17KB 598KB % 69% MB 3.3MB 1% 1.85MB 1.8MB % 1.54MB 1.5MB 3.48MB 9.5MB % 69% MB 3.4MB 1% 1.35MB 1.32MB % 1.72MB 1.73MB 31.1KB 24.9KB % 72% MB 2.66MB 1% 1.35MB 1.31MB % 1.36MB 1.35MB 92.5KB 52KB % 72% MB 4.7MB 1% 2.3MB 1.99MB % 2.13MB 2.8MB 39.3KB 436KB % 69% MB 3.21MB 1% 1.28MB 1.25MB % 1.95MB 1.96MB 4.73MB 1.2MB % 72% MB 3.71MB 1% 2.7MB 2.2MB % 1.75MB 1.7MB 15.6KB 15.6KB cluster1::*> exit 12. In your PuTTY session to rhel1, hit Ctrl-c to kill the workload script. Tip: There is no new command to enter in this step other than issuing Ctrl-c against the command you started in step 7. [root@rhel1 ~]# python /scripts/file_create.py /mnt/flexgroup/volmove Starting overall work: :4: ^CTraceback (most recent call last):keyboardinterrupt... return self.poll() File "/usr/lib64/python2.6/multiprocessing/forking.py", line 17, in poll pid, sts = os.waitpid(self.pid, flag) KeyboardInterrupt [root@rhel1 ~]# NetApp, Inc. All rights reserved. NetApp Proprietary

69 13. In your remaining PuTTY session to cluster1, display a listing of your current FlexGroup member volumes along with their current aggregate location. cluster1::> volume show -vserver DEMO -is-constituent true -fields volume,aggregate vserver volume aggregate DEMO flexgroup 1 aggr1_node2 DEMO flexgroup 2 aggr1_node1 DEMO flexgroup 3 aggr1_node1 DEMO flexgroup 4 aggr1_node2 DEMO flexgroup 5 aggr1_node1 DEMO flexgroup 6 aggr1_node2 DEMO flexgroup 7 aggr1_node1 DEMO flexgroup 8 aggr1_node2 8 entries were displayed. cluster1::> Note that the flexgroup 1 and flexgroup 2 volumes have swapped to their opposing aggregates. Leave your remaining PuTTY sessions open for use in later exercises. 3.1 Snapshot Restore In this activity you perform a snapshot restore on the FlexGroup volume in order to revert its contents to the state prior to when you ran the file generation workload scripts. 1. Switch to your PuTTY session to cluster1. 2. Make sure that your cluster shell is running at the admin privilege level. cluster1::> set admin cluster1::> 3. Display a list of the available snapshots on the FlexGroup volume flexgroup. cluster1::> snapshot list -vserver DEMO -volume flexgroup ---Blocks--Vserver Volume Snapshot Size Total% Used% DEMO flexgroup base 3.73MB % % hourly _ MB % % hourly _ MB % % hourly _ MB % 1% hourly _ MB % % 5 entries were displayed. cluster1::> 4. Attempt to restore the flexgroup volume to the base snapshot. As you may recall, you created this snapshot in exercise 3.3, before you ran any workload scripts. cluster1::> snapshot restore -vserver DEMO -volume flexgroup -snapshot base Error: command failed: This command is not supported for FlexGroups or FlexGroup constituents. cluster1::> The attempt fails because you are not allowed to initiate snapshot restore operations on FlexGroups from the admin (or even the advanced) privilege level. 5. Elevate your cluster shell to the diag privilege level and re-attempt the snapshot restore. Acknowledge the warnings NetApp, Inc. All rights reserved. NetApp Proprietary

70 Tip: You may not see all the warnings listed here in your lab. For example, the 'Snapshot copy base is not the most recent copy' warning only appears if you have newer snapshots than base on your FlexGroup. cluster1::> set diag Warning: These diagnostic commands are for use by NetApp personnel only. Do you want to continue? {y n}: y cluster1::*> snapshot restore -vserver DEMO -volume flexgroup -snapshot base Warning: Snapshot copy "base" is not the most recent copy. Promoting this Snapshot copy will delete all copies made after it. Do you want to continue? {y n}: y Warning: Quota rules currently enforced on volume flexgroup may change during this operation. If the currently enforced quota rules are different from those in Snapshot copy base, you may have to resize or reinitialize quotas on this volume after this operation. Do you want to continue? {y n}: y Warning: Export policies currently enforced on the qtrees of volume "flexgroup" will not change during this operation. If the currently enforced export policies are different from those in Snapshot copy "base", reassign the export policies of the qtrees on this volume after this operation. Do you want to continue? {y n}: y cluster1::*> 6. List the snapshots for all the member volumes of flexgroup. cluster1::*> snapshot list -vserver DEMO -volume flexgroup* ---Blocks--Vserver Volume Snapshot Size Total% Used% DEMO flexgroup base 924KB % 14% flexgroup 1 base 12KB % 14% flexgroup 2 base 84KB % 11% flexgroup 3 base 84KB % 11% flexgroup 4 base 84KB % 11% flexgroup 5 base 14KB % 13% flexgroup 6 base 84KB % 11% flexgroup 7 base 28KB % 32% flexgroup 8 base 84KB % 11% 9 entries were displayed. cluster1::*> Any snapshots on your member volumes that were more recent than base are now gone, which is normal behavior for a snapshot restore operation. 7. Set your cluster shell privilege level back to admin. cluster1::*> set admin cluster1::> 8. Switch to your PuTTY session to rhel1, and verify that the files your workload script invocations created on /mnt/flexgroup are now gone. [root@rhel1 total 12 drwxr-xr-x drwxr-xr-x. -rwxr-xr-x 7 ~]# ls -al /mnt/flexgroup 2 root root 496 Aug 3 21:1. 5 root root 496 Apr 21 13: root bin Aug 3 21:1 New Text Document.txt 217 NetApp, Inc. All rights reserved. NetApp Proprietary

71 drwxrwxrwx 3 root root 496 Aug [root@rhel1 ~]# snapshot Leave your remaining PuTTY sessions open for use in later exercises Expanding a Flexgroup by Adding Member Volumes You can also expand the capacity and performance of a FlexGroup volume by adding new member volumes. However, there are potential performance implications that you need to bear in mind before proceeding down this path. When ingesting files, a FlexGroup favors placing new files on less-empty members. Since new member volumes will by definition be empty, the ingest algorithm will prioritize the new member volumes during file placement. This will not balance out until all member volumes approach a simliar fullness level. In other words, depending on your workload you may be trading one performance bottleneck for another, at least until such time as the member volumes equalize. In this activity you add new members to a FlexGroup volume, and examine how that affects file ingestment. 1. In your PuTTY session to cluster1, set the cluster shell to the diag privilege level. cluster1::> set diag Warning: These diagnostic commands are for use by NetApp personnel only. Do you want to continue? {y n}: y cluster1::*> 2. Display the current file distribution probabilities for the flexgroup FlexGroup. cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 289 msec * next refresh message due in msec (last to member x8d56a27) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R 147 % % 12% L % % 12% L % % 12% R % % 12% L 1474 % % 12% R % % 12% L % % 12% R % % 12% + + Probabilities D-Ingest D- [1% 1% ] + + [1% 1% ] + + [1% 1% ] + + [1% 1% ] + + [1% 1% ] + + [1% 1% ] + + [1% 1% ] + + [1% 1% ] + + Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 118 msec * next refresh message due in msec (last to member x8d56a2b) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L 147 % % 12% R % % 12% Probabilities D-Ingest D- [1% 1% ] + + [1% 1% ] NetApp, Inc. All rights reserved. NetApp Proprietary

72 R L R L R L % % 12% [1% 1% ] + + % % 12% [1% 1% ] + + % % 12% [1% 1% ] + + % % 12% [1% 1% ] + + % % 12% [1% 1% ] + + % % 12% [1% 1% ] + + cluster1::*> Observe that the target and probablility percentages for the member volume are all fairly even. 3. Add 1 new member volume to the FlexGroup. cluster1::*> volume expand -volume flexgroup -aggr-list aggr1_node1 -aggr-list-multiplier 1 Warning: The following number of constituents of size 16TB will be added to FlexGroup "flexgroup": 1. Expanding the FlexGroup will cause the state of all Snapshot copies to be set to "partial". Partial Snapshot copies cannot be restored. Do you want to continue? {y n}: y [Job 497] Job is queued: Volume Expand Job. [Job 497] Job succeeded: Successful cluster1::*> The --aggr-list parameter in this command specifies the list of aggregates on which to create the new members, whereas the aggr-list-multiplier parameter specifies the number of times to iterate over that list. In this example the combination of only a single specified aggregate and a multiplier of 1 results in the creation of only a single new member volume. 4. Display the file distribution probabilities for the FlexGroup again. cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 175 msec * next refresh message due in 758 msec (last to member x8d56a27) * Ref count is 1 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R 1472 % % % L % % % L % % % R % % % L 1474 % % % R % % % L % % % R % % % L 734 % % 99% + + Probabilities D-Ingest D- [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% 1% 1%] + + [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% % %] + + Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 626 msec * next refresh message due in msec (last to member x8d56a2d) * Ref count is NetApp, Inc. All rights reserved. NetApp Proprietary

73 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L 1472 % % % R % % % R % % % L % % % R 1474 % % % L % % % R % % % L % % % R 734 % % 99% + + Probabilities D-Ingest D- [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% 1% 1%] + + [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% 99% 99%] + + [1% 1% % %] + + cluster1::*> There is now a new member volume in the list (the entry with the Idx value of 9). Note that the Targ percentage for this member is very high, while the Targ for the other members are all very low. As you saw earlier, a high Targ value translates into a high probability for new file placement. 5. Switch to your PuTTY session to rhel1, and create two subdirectories in the mounted FlexGroup that you will use for testing in this exercise. [root@rhel1 ~]# ls /mnt/flexgroup New Text Document.txt [root@rhel1 ~]# mkdir /mnt/flexgroup/run1 [root@rhel1 ~]# mkdir /mnt/flexgroup/run2 [root@rhel1 ~]# ls /mnt/flexgroup New Text Document.txt run1 run2 [root@rhel1 ~]# 6. Run the file_create_expand.py script against the run1 directory. [root@rhel1 ~]# python /scripts/file_create_expand.py /mnt/flexgroup/run1 Starting overall work: :47: Switch to your PuTTY session to cluster1 and re-run the command to display the file distribution probablities. Repeat the command periodically until you start to see a change in the distribution of the files. cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 5996 msec * next refresh message due in 1 msec (last to member x8d56a27) * Ref count is 11 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R % % % L % % 1% L % % 1% R % % 1% L 1474 % % 1% R % % 1% L % % 1% Probabilities D-Ingest D [1% 1% 1% 1%] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] NetApp, Inc. All rights reserved. NetApp Proprietary

74 8 155 R L % % 1% [1% 1% % %] + + % % 9% [1% 1% % %] Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 5322 msec * next refresh message due in msec (last to member x8d56a28) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L % % 3% R % % 4% R % % 4% L % % 4% R 1474 % % 4% L % % 4% R % % 4% L % % 4% R % % 62% Probabilities D-Ingest D- [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] + + [1% 1% % %] cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 9648 msec * next refresh message due in 637 msec (last to member x8d56a27) * Ref count is 1 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R % % 12% L 2422 % % % L % % 13% R % % 12% L % % 13% R % % 12% L % % 13% R % % 12% L % % 9% Probabilities D-Ingest [1% 1% D- % %] + + [1% 1% 1% %] [1% 1% % %] [1% 1% % %] + + [1% 1% % %] [1% 1% % %] + + [1% 1% % %] [1% 1% % %] + + [1% 1% 75% %] Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 943 msec * next refresh message due in msec (last to member x8d56a2d) * Ref count is 8 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L % % 12% R 2422 % % % Probabilities D-Ingest [1% 1% D- % %] + + [1% 1% 1% %] NetApp, Inc. All rights reserved. NetApp Proprietary

75 R L R L R L R % % 13% [1% 1% % %] % % 12% [1% 1% % %] + + % % 13% [1% 1% % %] % % 12% [1% 1% % %] + + % % 13% [1% 1% % %] % % 12% [1% 1% % %] + + % % 9% [1% 1% 75% %] cluster1::*> Over time, the Targ percentages for the member volumes begin to converge, meaning that file placement rates converge as well. 8. In your PuTTY session to rhel1, wait for the script to complete and note the reported total execution time in your lab. In this example the script took almost 12 seconds to complete. ### Full output of script execution [root@rhel1 ~]# python /scripts/file_create_expand.py /mnt/flexgroup/run1 Starting overall work: :47: End overall work: :49: total time: [root@rhel1 ~]# 9. In your PuTTY session to cluster1, add 8 more member volumes to the FlexGroup. cluster1::*> volume expand -volume flexgroup -aggr-list aggr1_node1,aggr1_node2 -aggr-list-multiplier 4 Warning: The following number of constituents of size 16TB will be added to FlexGroup "flexgroup": 8. Do you want to continue? {y n}: y [Job 498] Job is queued: Volume Expand Job. [Job 498] Steps completed: 1 of 1. [Job 498] Job succeeded: Successful cluster1::*> In this example, there are 2 listed aggregates, so a multiplier value of 4 results in the creation of 8 new member volumes. 1. Display the files distribution probabilities once again. cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 8472 msec * next refresh message due in msec (last to member x8d56a37) * Ref count is 18 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R % % 1% L 2442 % % % L 179 % % 2% R 1915 % % 1% L % % 2% R % % 1% L % % 2% Probabilities D-Ingest [1% 1% 98% D- 98%] + + [1% 1% 1% 1%] + + [1% 1% 97% 97%] + + [1% 1% 98% 98%] + + [1% 1% 97% 97%] + + [1% 1% 98% 98%] + + [1% 1% 97% 97%] NetApp, Inc. All rights reserved. NetApp Proprietary

76 R L L + 16 R L R L R L R % % 1% [1% 1% 98% 98%] + + % % % [1% 1% 99% 99%] + + % % 11% [1% 1% 88% 88%] + + % % 11% [1% 1% 88% 88%] + + % % 11% [1% 1% 88% 88%] + + % % 11% [1% 1% 88% 88%] + + % % 11% [1% 1% 88% 88%] + + % % 11% [1% 1% 88% 88%] + + % % 11% [1% 1% 88% 88%] + + % % 11% [1% 1% 88% 88%] + + Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 569 msec * next refresh message due in 57 msec (last to member x8d56a29) * Ref count is 16 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L % % 1% R 2442 % % % R 179 % % 2% L 1915 % % 1% R % % 2% L % % 1% R % % 2% L 1914 % % 1% R 1975 % % % R 7352 % % 11% L 7352 % % 11% R 7352 % % 11% L 7352 % % 11% R 7347 % % 11% L 7347 % % 11% R 735 % % 11% L 735 % % 11% + + Probabilities D-Ingest [1% 1% 98% D- 98%] + + [1% 1% 1% 1%] + + [1% 1% 97% 97%] + + [1% 1% 98% 98%] + + [1% 1% 97% 97%] + + [1% 1% 98% 98%] + + [1% 1% 97% 97%] + + [1% 1% 98% 98%] + + [1% 1% 99% 99%] + + [1% 1% 88% 88%] + + [1% 1% 88% 88%] + + [1% 1% 88% 88%] + + [1% 1% 88% 88%] + + [1% 1% 88% 88%] + + [1% 1% 88% 88%] + + [1% 1% 88% 88%] + + [1% 1% 88% 88%] + + cluster1::*> There are now 8 new member volumes in the list (Idx values 1-17). The Targ percentages for the new member volumes are the same, and are all notably higher than for the older member volumes, but the difference between new and old is not as great as in the preceding test. This is because 8/17 of the volumes are new here, rather than 1/9 of the volumes being new as you saw in the previous test. In other words, the more new member volumes you add at once, the more they will share the load NetApp, Inc. All rights reserved. NetApp Proprietary

77 11. Switch to your PuTTY session to rhel1, and run the file_create_expand.py script against the run2 directory. ~]# python /scripts/file_create_expand.py /mnt/flexgroup/run2 Starting overall work: :53: Switch to your PuTTY session to cluster1 and re-run the command to display the file distribution probablities. Repeat the command periodically until you start to see a change in the distribution of the Targ values. cluster1::*> node run * flexgroup show flexgroup 2 entries were acted on. Node: cluster1-1 FlexGroup x8d56a26 * next snapshot cleanup due in 6118 msec * next refresh message due in 12 msec (last to member x8d56a35) * Ref count is 18 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc R % % 6% L 2442 % % 6% L % % % R % % 6% L % % % R % % 6% L % % % R % % 6% L % % % L 7495 % % 8% R 7455 % % 8% L 7512 % % 8% R 742 % % 8% L 752 % % 8% R 7424 % % 8% L 7511 % % 8% R 7436 % % 8% Probabilities D-Ingest D- [1% 1% % %] + [1% 1% % %] + + [1% 1% 6% %] + 2+ [1% 1% % %] + [1% 1% 6% %] + 2+ [1% 1% % %] + [1% 1% 6% %] + 2+ [1% 1% % %] + [1% 1% 1% %] + [1% 1% % %] + 9+ [1% 1% % %] [1% 1% % %] [1% 1% % %] + 5+ [1% 1% % %] [1% 1% % %] + 5+ [1% 1% % %] [1% 1% % %] Node: cluster1-2 FlexGroup x8d56a26 * next snapshot cleanup due in 996 msec * next refresh message due in msec (last to member x8d56a2d) * Ref count is 16 Idx Member L Used Avail Urgc Targ Alloc F-Ingest F-Alloc L % % 1% R 2442 % % % R 1796 % % 2% L % % 1% R % % 2% Probabilities D-Ingest [1% 1% D- % %] + [1% 1% 1% %] + + [1% 1% % %] + 2+ [1% 1% % %] + [1% 1% % %] NetApp, Inc. All rights reserved. NetApp Proprietary

78 L R L R R L R L R L R L % % 1% [1% 1% % %] + % % 2% [1% 1% % %] + 2+ % % 1% [1% 1% % %] + % % % [1% 1% % %] + % % 11% [1% 1% % %] + 9+ % % 1% [1% 1% % %] % % 11% [1% 1% % %] % % 11% [1% 1% % %] + 5+ % % 11% [1% 1% % %] % % 11% [1% 1% % %] + 5+ % % 11% [1% 1% % %] % % 1% [1% 1% % %] cluster1::*> Once again observe that over time the Targ percentages start to converge. 13. In your PuTTY session to rhel1, wait for the script to complete, and note the total execution time. [root@rhel1 ~]# python /scripts/file_create_expand.py /mnt/flexgroup/run2 Starting overall work: :53: End overall work: :54: total time: [root@rhel1 ~]# In this test the script took approximately 97 seconds to run, which is significantly less that the approximately 12 seconds it took to run when there was only 1 new member volume. 14. In your PuTTY session to cluster1, return the cluster shell to that admin privilege level. cluster1::*> set admin cluster1::> Leave your remaining PuTTY sessions open for use in later exercises Using XCP to Migrate from FlexVol to FlexGroup One of the challenges of using FlexGroup volumes is deciding how to best migrate your data to them. ONTAP does not currently support in-place FlexVol to FlexGroup migration, so you must instead use a copy-based migration strategy. For NFSv3 traffic, NetApp provides a tool named XCP that can move data at a high rate, and it works well for moving data from a FlexVol volume to a NetApp FlexGroup volume. XCP can be a much more efficient means for copying files than alternative solutions like rsync (or even using traditional file copy commands in parallel). XCP is a multi-threaded application that is designed to perform many copy-related actions in parallel, including leveraging multiple TCP/IP connections and CPU cores. It will copy as much data over as many threads as your network and CPUs can handle. When coupled with NetApp FlexGroup volumes, throughput gets even better because you are parallelizing the storage portion of the workload too. This lab guide only illustrates how to use XCP to copy a directory tree to a FlexGroup, but you should know that XCP is a feature-rich tool that is also able to verify destination trees, synchronize differential incremental updates between the source and destination trees after the initial copy, resume interrupted copy operations, and scan and report the contents of file systems to help estimate the scope of proposed migrations. Unfortunately, this lab is not able to examine all of these features, but consult the References section of this document for more information. The following activity shows how to use XCP to migrate a large amount of data via NFSv3 from a FlexVol source to a FlexGroup destination in a small amount of time. The rhel1 client has XCP pre-installed NetApp, Inc. All rights reserved. NetApp Proprietary

79 1. In your PuTTY session to cluster1, use the CLI to create a new FlexGroup named XCP_FG. cluster1::> volume create -volume XCP_FG -aggr-list aggr1_node1,aggr1_node2 -aggr-list-multiplier 4 -junction-path /XCP_FG -size 1t -space-guarantee none Warning: A FlexGroup "XCP_FG" will be created with the following number of constituents of size 1.25TB: 8. Do you want to continue? {y n}: y [Job 52] Job is queued: Create XCF_FG. [Job 52] Job succeeded: Successful cluster1::> 2. In your PuTTY session to rhel1, mount the new flexgroup as /mnt/xcp_fg. [root@rhel1 ~]# mount flexgroup:/xcp_fg /mnt/xcp_fg [root@rhel1 ~]# mount grep XCP_FG flexgroup:/xcp_fg on /mnt/xcp_fg type nfs (rw,addr= ) [root@rhel1 ~]# 3. Use XCP to copy the files under rhel1's /XCP_sourecfiles directory to the mounted XCP_FG FlexGroup. [root@rhel1 ~]# xcp copy -parallel 1 localhost:/xcp_sourcefiles/ flexgroup:/xcp_fg/ XCP 1.3-8ea9ec; (c) 217 NetApp, Inc.; Licensed to NetApp FlexGroup Lab [NetApp FlexGroup Lab] until Sat Oct 1 18:9: 216 xcp: WARNING: CPU count is only 1! xcp: WARNING: No index name has been specified, creating one with name: autoname_copy_ _ xcp: mount 'localhost:/xcp_sourcefiles': WARNING: This NFS server only supports 1second timestamp granularity. This may cause sync to fail because changes will often be undetectable. 19,821 scanned, 6,497 copied, 7.9 MiB in (1.58 MiB/s), 3.61 MiB out (738 KiB/s), 5s 26,881 scanned, 13,481 copied, 14.1 MiB in (1.24 MiB/s), 8.23 MiB out (943 KiB/s), 1s 32,46 scanned, 18,324 copied, 4,396 indexed, 18.6 MiB in (89 KiB/s), 11.7 MiB out (626 KiB/ s), 16s 36,21 scanned, 22,657 copied, 4,396 indexed, 22.3 MiB in (746 KiB/s), 14.6 MiB out (581 KiB/ s), 21s 4,196 scanned, 25,933 copied, 14,436 indexed, 25.5 MiB in (66 KiB/s), 17.4 MiB out (58 KiB/s), 26s 43,781 scanned, 31,369 copied, 19,367 indexed, 3.1 MiB in (892 KiB/s), 21.4 MiB out (782 KiB/s), 31s 47,716 scanned, 35,378 copied, 24,476 indexed, 33.7 MiB in (711 KiB/s), 24.4 MiB out (586 KiB/s), 36s 51,41 scanned, 39,98 copied, 24,476 indexed, 36.9 MiB in (618 KiB/s), 26.9 MiB out (482 KiB/s), 42s 57,781 scanned, 45,48 copied, 29,472 indexed, 42.6 MiB in (1.13 MiB/s), 31.4 MiB out (91 KiB/s), 47s 64,346 scanned, 49,73 copied, 34,521 indexed, 46.9 MiB in (856 KiB/s), 34.4 MiB out (623 KiB/s), 52s 68,686 scanned, 53,581 copied, 39,465 indexed, 5.5 MiB in (738 KiB/s), 37.4 MiB out (594 KiB/s), 57s 73,911 scanned, 57,944 copied, 44,463 indexed, 54.6 MiB in (84 KiB/s), 4.5 MiB out (645 KiB/s), 1m2s 77,446 scanned, 61,371 copied, 44,463 indexed, 57.7 MiB in (553 KiB/s), 42.8 MiB out (413 KiB/s), 1m8s 81,486 scanned, 67,687 copied, 44,463 indexed, 62.9 MiB in (1.2 MiB/s), 47.1 MiB out (865 KiB/s), 1m13s 84,921 scanned, 71,327 copied, 44,463 indexed, 66. MiB in (639 KiB/s), 49.5 MiB out (479 KiB/s), 1m18s 87,231 scanned, 73,49 copied, 44,463 indexed, 68. MiB in (369 KiB/s), 51. MiB out (279 KiB/s), 1m23s 9,196 scanned, 79,241 copied, 54,427 indexed, 72.8 MiB in (939 KiB/s), 55.9 MiB out (961 KiB/s), 1m29s 94,446 scanned, 82,386 copied, 64,474 indexed, 75.9 MiB in (528 KiB/s), 58.2 MiB out (412 KiB/s), 1m34s 99,561 scanned, 87,279 copied, 74,68 indexed, 8.4 MiB in (935 KiB/s), 62.2 MiB out (797 KiB/s), 1m39s 12,851 scanned, 91,35 copied, 79,643 indexed, 84. MiB in (575 KiB/s), 65.2 MiB out (55 KiB/s), 1m46s 11,21 scanned, 98,154 copied, 84,62 indexed, 9.2 MiB in (1.23 MiB/s), 7.1 MiB out (978 KiB/s), 1m51s 114,976 scanned, 12,381 copied, 84,62 indexed, 94.1 MiB in (741 KiB/s), 72.9 MiB out (537 KiB/s), 1m56s 119,231 scanned, 16,842 copied, 94,722 indexed, 98.1 MiB in (88 KiB/s), 76.5 MiB out (717 KiB/s), 2m1s 124,376 scanned, 111,65 copied, 99,727 indexed, 12 MiB in (884 KiB/s), 79.9 MiB out (71 KiB/s), 2m6s NetApp, Inc. All rights reserved. NetApp Proprietary

80 129,811 scanned, 117,416 copied, 99,727 indexed, 18 MiB in (61 KiB/s), 83.8 MiB out (469 KiB/s), 2m15s 133,946 scanned, 124,422 copied, 19,72 indexed, 113 MiB in (1.17 MiB/s), 89.2 MiB out (1.7 MiB/s), 2m2s 133,946 scanned, 128,435 copied, 119,773 indexed, 117 MiB in (632 KiB/s), 92.8 MiB out (734 KiB/s), 2m25s 133,946 scanned, 133,627 copied, 124,724 indexed, 12 MiB in (779 KiB/s), 96.9 MiB out (823 KiB/s), 2m3s 133,946 scanned, 133,945 copied, 133,946 indexed, 121 MiB in (817 KiB/s), 97.7 MiB out (661 KiB/s), 2m31s. ~]# The XCP copy session in this example took approximately 2.5 minutes to complete. In comparision, a test using rsync in this same lab took nearly 13 minutes copy this same data to the FlexGroup; more than 5 times as long! Remember, XCP's scalable architecture allows it to perform better as you give it more resources. With larger workloads, bigger network pipes, and more clients, the speed at which XCP can copy files increases dramatically NetApp, Inc. All rights reserved. NetApp Proprietary

81 4 References The following references were used in writing this lab guide. FlexGroup podcast: FlexGroup blogs: FlexGroup technical FAQ for ONTAP 9.2: NetApp FlexGroup Technical Overview NetApp FlexGroup Best Practices and Implementation Guide Top Best Practices for NetApp FlexGroup Volumes Insight 216 sessions on Field Portal : Introduction to FlexGroups : FlexGroups Technical Primer 217 NetApp, Inc. All rights reserved. NetApp Proprietary

82 5 Version History 82 Version Date Document Version History 1. Aug 217 Initial Release 217 NetApp, Inc. All rights reserved. NetApp Proprietary

83 Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exact product and feature versions described in this document are supported for your specific environment. The NetApp IMT defines the product components and versions that can be used to construct configurations that are supported by NetApp. Specific results depend on each customer's installation in accordance with published specifications. NetApp provides no representations or warranties regarding the accuracy, reliability, or serviceability of any information or recommendations provided in this publication, or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information in this document is distributed AS IS, and the use of this information or the implementation of any recommendations or techniques herein is a customer s responsibility and depends on the customer s ability to evaluate and integrate them into the customer s operational environment. This document and the information contained herein may be used solely in connection with the NetApp products discussed in this document. Go further, faster 217NetApp, Inc. All rights reserved. No portions of this document may be reproduced without prior written consent of NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo, Data ONTAP, ONTAP, OnCommand, SANtricity, FlexPod, SnapCenter, and SolidFire are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such.