Using EMC ControlCenter StorageScope to Support Storage Chargeback

Transcription

1 Engineering White Paper Using EMC ControlCenter StorageScope to Support Storage Chargeback Abstract This white paper defines problems and needs for storage chargeback systems and solutions available through EMC ControlCenter applications, including EMC ControlCenter StorageScope. Published 5/26/2003

2 Copyright 2003 EMC Corporation. All rights reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. Part Number H899 Using EMC ControlCenter StorageScope to Support Storage Chargeback 2

3 Table of Contents Abstract...4 Objectives...4 Storage Chargeback Overview...5 Server Consolidation, Data Center Compression, and Cost Justification: IT Trends Motivating Chargeback Systems... 5 Challenges for Business Units... 5 Challenges for Data Center Operations... 5 Uses of Storage Chargeback... 6 Recovering Costs from Storage Consumers... 6 Reducing Costs and Increasing Storage Efficiency by Motivating Behaviors... 6 Assisting Users in Making Rational Decisions on Storage Requirements... 7 Demonstrating Business Value to Storage Users and Senior Management... 7 StorageScope Overview...7 EMC ControlCenter... 7 EMC ControlCenter StorageScope... 7 EMC ControlCenter StorageScope API... 8 Supporting Chargeback Using StorageScope...8 Asset Costs and Rate Determination: Identifying Assets Using the ControlCenter Common Infrastructure... 8 Management Costs and Rate Determination: Clarifying Correlations between Staff Members, Systems, and Costs... 9 Storage Utilization Statistics: Charging for Actual Use... 9 Getting Daily and Weekly Aggregate Statistics of Capacity Utilization for Applications... 9 Showing Customers What They Are Using Exporting Storage Utilization Statistics to Other Applications Conclusion...12 Using EMC ControlCenter StorageScope to Support Storage Chargeback 3

4 Abstract The following trends in storage technology have increased the need for centralized storage chargeback reporting: Networked storage Both network-attached storage (NAS) and storage area network (SAN) implementations have allowed companies to reduce the cost of storage while increasing information accessibility. Server consolidation and data center floor space compression have increased the manageability and value of storage resources, but they have also made it difficult to track storage costs at the individual application or business organization level. Cost pressure Companies are increasingly being called on to monitor and improve resource utilization to reduce costs and justify computing resources in terms of business value. Fast application cycle times In order to be competitive, companies require a flexible IT infrastructure that can launch a new application quickly and accommodate growth. How storage consumers used storage in the past affects their needs for new and upgraded applications. For many companies, chargeback for storage resources offers the cost accountability and assessments of value necessary to address these trends. EMC ControlCenter StorageScope can be used to generate storage chargeback statistics for hosts and NAS file servers. This white paper shows how to use EMC ControlCenter StorageScope and other applications to help set storage chargeback rates, organize storage resources for convenient chargeback reporting, and generate storage utilization reports in a variety of output formats. Objectives The objectives of this white paper are to: Outline the common business challenges that motivate the use of storage chargeback systems Describe common types and purposes of storage chargeback systems Explain how ControlCenter applications including StorageScope, can provide asset management and management cost information for chargeback rate setting Show how StorageScope reports and other output formats can provide storage utilization statistics for a chargeback system Describe tools and best practices for the most effective creation of chargeback information and statistics Using EMC ControlCenter StorageScope to Support Storage Chargeback 4

5 Storage Chargeback Overview While chargeback is not a new idea, it has become an increasingly important solution as data storage and access resources become consolidated and as cost justification has become necessary for IT operations. This overview discusses the trends that have made chargeback both necessary and advantageous to data center administrators and business units alike. Server Consolidation, Data Center Compression, and Cost Justification: IT Trends Motivating Chargeback Systems The trend of server consolidation has meant more control and manageability to data center administrators, gathering far-flung and perhaps even unknown IT resources into accessible centralized systems. Larger storage arrays and a Fibre Channel storage network made server consolidation possible. They also allowed more and more computing resources to take up smaller and smaller physical spaces. The trend of data center compression has meant companies and organizations can grow their computing resources without the costs of additional rent or the difficulties of moving into larger data center quarters. Server consolidation and data center compression, so beneficial for the organizations taking advantage of them, have created the circumstances where centralized storage chargeback can become necessary, possible, and beneficial. Finally, the trend of cost justification has forced IT operations to explain and defend their costs. Storage chargeback can become an essential process in an environment of cost justification. Challenges for Business Units With server consolidation and data center compression, widely different applications now use the same storage systems, and even servers. Yet different applications make different demands of computing systems. Different applications require different storage capacities, levels of performance and uptime, and attention from administrators. Administrators may be geographically distant from application developers and their business units, and data center centralization may mean some parts of the organization have lost local control of IT resources. Now more than ever, business units depend on centralized data center operations to provide essential computing resources for their business tasks. For most organizations, storage requirements are still growing, and the costs of storage management are no less significant to monitor and control. When storage is provided without accountability, business units cannot understand the impact of their storage-related requests and practices. They cannot modify their behaviors in ways that help either their own functions or those of the data center administrators. Business units frequently lack the information they need to understand the costs of their actions so they can assess the net value they are providing the organization. A storage chargeback system can provide business units valuable information they can use when they assess their own technology requirements, ask for computing resources, and ask for the financial resources to pay the bill. When business units have information about their storage utilization, they can articulate their needs more clearly to the data center. Challenges for Data Center Operations Data center operations and the organizations they serve have benefited immensely from centralized control of computing resources. Centralized server and storage resources are much more manageable both as business assets and as technical systems, and they provide business value that dispersed systems cannot. Capacity is cheaper to buy and improved performance easier to obtain. Data is easier to protect and share. But a generation of higher quality computing resources and increased business expectations along with it has increased the demands on data center managers and staff. Very simply, all storage consumers want percent uptime. All storage consumers want unlimited storage capacity on demand, essentially without restrictions. All consumers want their applications to run as quickly as possible, and they want complete safety for their data. Moreover, data center providers of storage services want to offer this level of service. But they are increasingly called upon to justify their Using EMC ControlCenter StorageScope to Support Storage Chargeback 5

6 share of the budget and to reduce costs. Data center administrators can have difficulty discovering and communicating the costs of their assets and labor, the costs of excessive and harmful uses of the storage, and the benefits and value of the services the storage providers offer to the business. A storage chargeback system can help data center operations make visible their internal costs and pressures, allowing business units to adapt to the data center s realities and, conversely, to assist the data center in meeting business units needs. Uses of Storage Chargeback Using chargeback systems, storage providers can optimize and improve their offering to storage consumers, together with a business justification of the offering. Chargeback systems can do this using either of two bases: cost and value. In this white paper, cost means the expense of storage capacity and the staff who manage it. Value is the sum of the operational costs and the opportunity cost of not using the resources elsewhere. Different storage chargeback systems can achieve one or more of the following goals for the storage provider: Recover costs from storage consumers Reduce costs and increase storage efficiency by motivating storage consumer behavior Assist storage consumers in making rational decisions on storage requirements Demonstrate business value to storage consumers and to senior management Cost and value have an important role in these goals. Recovering Costs from Storage Consumers Some organizations use chargeback with the goal to allow the IT department to recover costs. The costs of providing storage are many, but users typically do not understand them. From capital costs like best-ofbreed disk storage to the labor costs of tape mounts, storage systems cost money. In this approach, chargeback systems provide cost accountability to storage consumers and needed funding to data centers. The storage provider can use chargeback data to charge for storage directly or to justify revenue requests in the budget process. Reducing Costs and Increasing Storage Efficiency by Motivating Behaviors A second approach to chargeback has the goal to encourage low-cost behaviors and discourage high-cost behaviors, making storage utilization and management more efficient. In this approach, rigid cost transference to customers may not be necessary if the storage provider has a base operational budget independent of revenues from internal customers. The storage provider charges rates designed to minimize costs and optimize the service provided to all consumers, rather than recover costs directly. The rate structure may include extra charges for premium services, lower charges (or none) for standard services, and even lower rates (or perhaps bonuses and rebates) for low-cost storage alternatives. Some chargeback systems charge higher than cost for premium services or for discouraged behaviors (such as storing MP3 files on high-availability, mirrored, and cached disk arrays). For a chargeback system to encourage and discourage behaviors with success, the system must allow storage consumers to identify how their behaviors can either reduce their costs or increase the value they are getting. Storage users rarely give storage back. Chargeback data can motivate them to request storage at a slower pace, or to recover wasted space. Chargeback can encourage greater utilization of the allocated space. Combined with effective storage resource monitoring and automated provisioning, higher utilization percentages can be achieved. Using EMC ControlCenter StorageScope to Support Storage Chargeback 6

7 Assisting Users in Making Rational Decisions on Storage Requirements A third approach to chargeback has the goal to help storage consumers decide what kind of storage they want and need. All storage consumers want percent uptime, capacity on demand, and complete safety for their data. In this approach to chargeback, consumers understand the costs of what they are asking for. The chargeback system motivates consumers to choose what is really important to them: for example, capacity on demand versus well-tuned performance. Consumers who want it all instant capacity on the fastest arrays will pay the most. Chargeback helps the storage consumer and the storage provider to communicate business needs and technical capabilities through measures of cost and value. Demonstrating Business Value to Storage Users and Senior Management In a fourth approach, chargeback systems encourage use of storage systems according to their value to each internal customer. Rather than presenting storage charges at cost, in this approach the storage provider charges based on the value of the services provided. Such an arrangement may work best when internal storage providers compete with one another and with external storage service providers. Value-based chargeback communicates the importance of the IT resources to the business units, an important justification in cost-cutting times. A switch to a value-based chargeback system requires budgeting systems to provide the business units the funds they need for storage use and requires the business units to justify the value of their projects as well. StorageScope Overview Chargeback systems that strive for the goals described in the previous sections require the same types of information: Asset costs Labor costs Usage statistics EMC ControlCenter applications, including EMC ControlCenter StorageScope, can help you gather the information you need. This section provides an overview of EMC ControlCenter StorageScope, followed by a description of StorageScope tools for supporting chargeback. EMC ControlCenter The EMC ControlCenter family is a suite of applications for discovering, monitoring, controlling, and reporting on storage resources. The ControlCenter shared infrastructure is used by multiple ControlCenter applications, enabling a consistent view across your storage infrastructure. These applications leverage a shared Repository that contains configuration, status, health, utilization, and capacity data for your storage resources. The EMC ControlCenter Console is the gateway interface for storage managers and systems administrators to monitor and control their storage infrastructure from a central point. EMC ControlCenter StorageScope EMC ControlCenter StorageScope is a Storage Resource Management reporting application that enables customers to report on their multivendor storage resources. With StorageScope, storage managers and administrators can easily answer common questions like: How much storage do we have? Who is using it? When will we need more? This information can be flexibly organized by lines of business, application, or any other way a company wants to track and account for its storage assets, improving its ability to plan storage procurement, prevent capacity related outages, and support internal and external chargeback systems. Using EMC ControlCenter StorageScope to Support Storage Chargeback 7

8 StorageScope is a Web-based application that produces historical reports of your storage infrastructure. These reports are based on the information collected by ControlCenter agents, which is stored in the ControlCenter Repository. The reports include data summarized by groups defined in the ControlCenter Console. EMC ControlCenter StorageScope API The EMC ControlCenter StorageScope API provides access to the information in the StorageScope reports, allowing greater integration flexibility while leveraging the ControlCenter infrastructure. The API provides access to this information through both SQL and XML interfaces. StorageScope-generated XML files provide access to structured historical snapshots of the information contained in the ControlCenter Repository. StorageScope-defined database views to the ControlCenter Repository provide access via SQL queries to the most recent data collected by ControlCenter agents. Both interfaces provide array configuration information, SAN configuration information, and NAS device information. Information is accessible for host volume managers, file systems, databases, and backup applications. Details about the relationships among these storage elements are also provided. In summary, EMC ControlCenter StorageScope provides historical reports of storage: Asset information Configuration Allocation Utilization Relationships Supporting Chargeback Using StorageScope Three types of information are vital in a storage chargeback system. EMC ControlCenter StorageScope can help data center administrators capture all three. Asset costs: Storage resources grouped by location and type. Management costs: Storage resources grouped by the department and possibly individual staff members who manage them. Storage utilization statistics: Storage resources grouped by department and application, including the host file systems, network resources, and storage space used. Additional types of cost information can include: Capital depreciation of assets: Quantify capital depreciation as an aspect of asset costs. Future storage costs: Quantify future storage needs and their costs using capacity planning techniques. See Using EMC ControlCenter StorageScope for Capacity Planning for more information on capacity planning. Asset Costs and Rate Determination: Identifying Assets Using the ControlCenter Common Infrastructure Chargeback rates should reflect an accurate assessment of storage costs across all your storage assets. The ControlCenter shared infrastructure provides automated and assisted discovery processes that make asset management simpler, faster, and more flexible. You identify your storage assets using automated and assisted discovery techniques, and then gather them in an asset management group visible in the Console. After you complete discovery of your assets, in StorageScope you print and export reports of your assets, moving the data into other asset management packages or into your traditional custom spreadsheets. Calculations of asset values and capital depreciation will include all the assets ControlCenter can find and manage, including details you may never have known before or gathered in one place. These discovered assets factor into the rate calculations that determine how much you will charge your storage customers. Using EMC ControlCenter StorageScope to Support Storage Chargeback 8

9 StorageScope reports list the arrays, switches, NAS file servers, and hosts discovered by ControlCenter. You can create groups in the ControlCenter Console that organize your resources by geography, floor, data center, asset class, operating cost (expensive versus inexpensive), and accounting treatment (depreciated assets versus nondepreciated assets). Then you generate reports based on the groups. Management Costs and Rate Determination: Clarifying Correlations between Staff Members, Systems, and Costs Chargeback rates should reflect the investment you have made in assembling a capable storage management staff that delivers the services your consumers seek. ControlCenter s flexible user-defined groups make management easier because staff members and teams can view and work with groups containing their assigned storage resources. When setting chargeback rates, use the information about staff assignments to determine the costs for managing particular resources and groups of resources, including hosts and arrays. Storage Utilization Statistics: Charging for Actual Use Utilization statistics are the key factor in assessing charges fairly. Clear and readily available storage utilization statistics promote the simple principle of charging people for what they use. Getting Daily and Weekly Aggregate Statistics of Capacity Utilization for Applications EMC ControlCenter StorageScope shows you and your users what they are using. With the StorageScope reports that appear in your Web browser, you can see daily and weekly statistics on file system and database utilization for groups of storage resources. StorageScope host chargeback reports provide information on the host-accessible capacity what hosts can physically see. These reports reflect the storage that is actually available to the hosts, excluding capacity used for mirroring, business copies, and remote replicas. The following sections describe common types of configurations. A Host Running an Application A host dedicated to an application is the simplest and perhaps most common chargeback scenario. StorageScope reports provide aggregated capacity utilization of the file systems and databases on a host. See the Exchange application in Figure 1. Multiple Hosts Processing Data for a Single Application Large and powerful applications often require multiple dedicated hosts for processing power or redundancy. StorageScope reports provide aggregated capacity utilization for an application s file systems and databases across multiple hosts. See the Lotus application in Figure 1. Using EMC ControlCenter StorageScope to Support Storage Chargeback 9

10 The Exchange application storage on a single host The Lotus application storage aggregated across three hosts Figure 1. Host Chargeback Report for Single-Host and Multiple-Host Applications Multiple Departments and Applications Sharing a Single Host In some cases, multiple applications run on a single host. This is common in branch offices and in environments where server consolidation has taken place. StorageScope reports break down the file systems on the hosts and enable you to charge back to separate departments. In Figure 2, host losaq064 contains multiple file systems for different departments. The storage used by each file system is shown. Figure 2. Host File Systems Report for a Host Shared by Multiple Departments Multiple Departments Sharing a NAS File Server NAS file servers commonly contain multiple file systems with storage for different departments. StorageScope reports break down the file systems on the hosts and enable you to charge back to separate departments. In Figure 3, the storage utilization is shown for each of the file systems on a NAS file server. Figure 3. NAS File Server File Systems Report for a File Server Shared by Multiple Departments Using EMC ControlCenter StorageScope to Support Storage Chargeback 10

11 Showing Customers What They Are Using With StorageScope, members of business units can see the statistics showing what they are using. Because StorageScope is a browser-based Web application, you can grant access to users and managers outside the storage management team, and you can respond to inquiries about utilization statistics by allowing users to view the statistics themselves. In the ControlCenter Console, place application file systems and databases in a group for each application. StorageScope can show you the utilization for each application. In many reports, you can create layouts that show only the information that a particular department needs. Creating layouts is an easy way to create reports that show the applications for individual departments. You can also export these reports and use the exported data to create invoices in your spreadsheets or other application. In Figure 4, a custom layout called Finance Chargeback shows the groups associated with finance department applications. The export capability is shown in the upper-right corner. The storage administrator creates these groups in the ControlCenter Console. They contain the hosts, file systems, and databases that make up the storage for the finance applications. Figure 4. Host Group List Report Showing the Finance Department Application Storage Utilization Figure 5 shows two departments. Both the storage administrators and storage consumers can use this report to drill into their storage utilization by department, application, and host. Figure 5. Host Group List Report Showing the Finance Department Application Storage Utilization Exporting Storage Utilization Statistics to Other Applications You can export storage utilization statistics in comma-delimited and XML formats. Also, you can use an API to query the ControlCenter Repository for storage utilization statistics. Figure 4 shows the export capability to CSV or XML in the upper-right corner of the screen. Using EMC ControlCenter StorageScope to Support Storage Chargeback 11

12 Conclusion Trends in data center consolidation, centralization, and cost justification are creating opportunities to use information more effectively and efficiently for business goals. Storage chargeback can help address many of the challenges that these trends create. Chargeback systems can help reduce costs, increase the quality of IT services, balance costs effectively among departments, and make the business value of IT solutions more readily visible. EMC ControlCenter StorageScope and other ControlCenter applications are powerful tools for supporting storage chargeback. By providing the right information in timely and usable ways, StorageScope makes it easier to determine chargeback rates based on actual costs and to assess charges based on actual utilization. Consolidated storage environments can take advantage of economies of scale, and yet individual business units still pay their share. IT managers and business managers can communicate readily, using the facts available in easy-to-use reports. Storage chargeback is a useful component of an overall strategy for managing networked storage a strategy with many parts. Capacity planning uses actual storage data to anticipate storage demand. Alertsbased monitoring provides notification when storage consumers have an actual storage need. Automated provisioning simplifies the process of meeting the need for storage capacity. Storage chargeback provides the accountability necessary to address expanding storage needs in ways that encompass both the realities of the data center and the values and priorities of the business. Using EMC ControlCenter StorageScope to Support Storage Chargeback 12