Real Life Challenges in Today s Storage World

Transcription

1 Real Life Challenges in Today s Storage World EMC Proven Professional Knowledge Sharing September, 2007 Kiran Ghag Senior Systems Administrator Kiran@kiranghag.com Page 1 of 9

2 Table of Contents Introduction... 3 More is just not enough!... 3 Ageing... 3 Thin provisioning benefits not utilized... 4 Untested backups and DR/BCP plans... 4 Power challenges keeping it cool... 4 Design flaws... 5 Information Lifecycle Management... 5 Getting the right budget... 6 Problems with distributed management... 6 Performance issues... 6 Lack of skilled people... 6 Multi-vendor solutions... 7 Interoperability... 7 Maintaining the software... 8 Storage technology... 8 Sign off...9 Author s Biography... 9 Disclaimer: The views, processes or methodologies published in this compilation are those of the author. They do not necessarily reflect EMC Corporation s views, processes, or methodologies. Page 2 of 9

3 Introduction Storage implementations are growing faster than ever. The storage world is booming as technology continues to evolve. Users continue to view storage as a panacea that is going to solve all their data, performance and protection needs. SAN does work well, but many practical hurdles prevent businesses from achieving optimal benefits. This article lists the prominent storage challenges faced by Storage Architects or Administrators in today s environments. More is just not enough! Memory and speed are two examples. You keep adding bigger and faster disks but still receive at least daily disk requests that make you wonder how you can be out of space! The OS teams views SAN as a large pool. Application designers may tend to oversize storage requirements because it would be on a SAN/NAS array. Moreover, if OS teams are not diligent about returning unused disks to the pool, the cumulative wastage can be significant. Storage administrators need to continuously hunt for unused space and reclaim it to satisfy incoming requests. Even with minimal wastage, available storage space seems to diminish every day. Ageing Ageing is a generic problem that describes the effects of time on documents, policies, work practices, baselines, or performance. Documentation is critical for any setup, irrespective of its size. In the beginning, documentation is passionately maintained. As the setup scales and time passes, documentation tends to lag behind the environment. This may occur for a variety of reasons but ultimately leads to incorrect data. Incorrect data leads to incorrect forecasts and budgeting, and may lead to ineffective, business-impacting decisions. Aligning documentation to the environment at a later date is a time-consuming, non-productive activity. Page 3 of 9

4 Thin provisioning benefits not utilized Internet surveys illustrate that most data centers underutilize their storage. Thin provisioning is one way to address this issue. It shakes up existing concepts where the final physical setup is tightly bound with the performance figures and design. Thin provisioning is most beneficial when space utilization follows a steady trend, but this may not happen everywhere. Some are reluctant to employ thin provisioning because it requires detailed monitoring of the setup and careful planning. Untested backups and DR/BCP plans Your data is only as good as your last backup. But is your backup good? Many shops spend time developing backup and disaster recovery (DR) strategies during the early phases of setup. Once the process is formalized, there are generally no routine tests performed to validate the backup and DR plans. Even with a perfect DR plan, something is bound to break at night, and the application will not failover correctly when the best storage professionals on your team are on holiday! Need we say more? Power challenges keeping it cool Power needs have become increasingly critical to keep things running. The more equipment you have, the more power you need. This requires investment in additional cooling. Surveys show that power requirements for US data centers are forecasted to double in the next five years! One solution is for the array to power up/down disks on demand, but that s not practical for OLTP needs. Thin provisioning can help to reduce power consumption. Server consolidation via virtualization is being chosen as a preferred solution to tackle power and cooling needs. Virtualization combined with thin provisioning can further reduce power consumption and cost. Cooling has evolved from simple air-conditioners to special low consumption cooling techniques. It s worth investigating and investing in them than spending on electricity bills. Page 4 of 9

5 Design flaws A good design leads to achieving goals. Proper design is required at all stages including, but not limited to data-center design, equipment purchase, and policy development. Sizing and capacity planning go hand-in-hand with design. Overlooking details during the design phase can have long-term repercussions. Even with the perfect design, unforeseen situations can have a negative impact. Improper sizing and capacity planning can manifest itself in ways like insufficient storage space, performance issues, and inadequate power. Design flaws usually require rootcause analysis and an extended time to fully investigate and correct to ensure there is no reoccurrence. Information Lifecycle Management ILM is a set of technologies and strategies to achieve efficient storage management. We view ILM and tiered storage as a solution to many of the issues we ve raised but it is not always easy to implement. ILM offers a set of new concepts; data classification being one. Understanding the data lifecycle in an organization, and classifying its value at different stages is a basic requirement for ILM solutions. But, it is not as simple as it seems. Data holds different value in different setups. Information lifecycles tend to change. Without proper classification, ILM cannot be perfectly implemented. Tiered storage supports ILM by providing data storage facilities with different cost and performance factors. It s easy to set up tiered storage by using different models of storage arrays. However, the availability of tiered storage alone does not guarantee a perfect ILM implementation. There can be issues moving data across products. Even before reaching that point, deciding what information to move, where to move it, and when to move it can prove very difficult. Page 5 of 9

6 Getting the right budget Storage is expensive. Disks, arrays, racks, space, power, cooling; everything is costly. Not only is the hardware costly, the software required to work with it is also costly. Most of us are caught between developing plans to align with business requirements and budgeting. Designers like to have the best hardware and software available for the setup. Administrators like to get more disks to respond to queued up requests. The business always wants the best RPO, TRO and TCO. We sometimes find ourselves running behind decision makers asking them to sanction the money required to achieve their goals. Problems with distributed management In larger setups, specialized groups support different functions (OS team, Applications team, Performance team, Storage team, Troubleshooting team etc). Unless there is alignment among these teams, achieving an optimal setup can be difficult. If the setup is large enough to require geographically dispersed teams for a single function, achieving harmony can be difficult. Performance issues Performance issues prevail when discrete teams work on storage. For smaller setups, there can be finer control over the design and implementation. It s often a person or two who manage storage from end-to-end. In bigger shops, this may not be possible. Architects may never talk to the storage professional and vice-versa. Even if they could, the volume and variety of requests may make it difficult to achieve an optimum fit for every request. Lack of skilled people There is a global shortage of skilled storage professionals. As storage needs increase, the need for skilled people is also increasing. OS professionals double as storage personnel in most cases. For other technologies, such as system administration or software development, much can be learned by self-study. You can create a test setup at minimal cost by using an off the shelf computer. Every small company is able to provide a real life work experience. Page 6 of 9

7 Storage is still not a commodity technology. It s not available for experimentation by novices, unless they are in a testing facility. Effective OS administration concepts help with routine storage management. Specialized, storage-oriented work profiles like performance tuning and solutions design demands trained storage resources. The lack of skilled storage professionals affects the quality and timeliness of projects. Many times, people are available but relocation is not always possible due to VISA rules or personal preferences. Projects can be delayed or even cancelled when resources are not available. Multi-vendor solutions Multi-vendor solutions are preferable to avoid vendor-dependence for critical work functions. With this strategy, the setup might not adhere to a single vendor s specific optimizations/guidelines. You may have to apply common best practices or devise multiple sets of guidelines (read maintenance headache). Multiple solutions require multiple resources. As mentioned earlier, locating, hiring and retaining skilled storage professionals is a challenge; finding skilled people who know multiple vendor technologies is even more difficult. Interoperability Interoperability remains a major issue in the storage world. This is highlighted when a homogenous solution fails to work. Unfortunately for customers or end users, it can lead to one vendor blaming another while none assumes responsibility. Customers need seamlessly integrated storage solutions from different vendors. The lack of a heterogeneous storage market reduces buyer s options. As a result, business planning and storage provisioning decisions are at times driven by choice of vendor. Page 7 of 9

8 Maintaining the software Heterogeneous and large systems often struggle with software currency. Every vendor has different support matrices. One vendor s certified configuration may not be supported by another. Apart from driver revisions, there are agents to be installed on hosts. Agents allow management software to probe hosts remotely and assist connectivity, monitoring and management. There are no widely used standards for this communication. SMI-S is catching up; SNMP didn t fully exploit its potential. As a result, multiple agents must be managed depending upon the number of vendors involved. Server teams are already busy with their server patch management; add storage software maintenance and you add to their burden. With a large host count, ensuring that agents are running and providing current information is no easy task. Oh wait; did we say anything about array code yet? Storage technology The widespread usage of specialized storage equipment is growing in recent years. Many market changes and technological advances have happened in the last 1-2 years. There have been merger and acquisition activities; people are still waiting for the benefits of these M&As. EMC is visible in the marketplace as it has a strong line-up of units that can deliver a complete integrated offering. The recent announcement about incorporating RSA technology into Symmetrix is one such welcome step. On one front, things are picking up speed rapidly for infrastructure. 1Gigabit was standard for a long time. Most implementations are now on 2Gigabit but 4Gigabit is catching up. Page 8 of 9

9 For NAS, there are announcements of 10Gigabit chips on the anvil. Disk vendors are pushing the envelope with higher density drives. On the other hand, technology is still evolving at a higher level. ILM, virtualization, thin provisioning etc solutions are yet to be streamlined for mass use. Customers are often confused by this rapid development. Storage infrastructure requires a substantial investment but procurement decisions usually take months. The high TCO of storage equipment requires that purchases need to be in production for at least a few years. Unstable technology makes purchasing decisions difficult. Sign off There is no perfect world and storage is no exception. Storage vendors are doing their best to address the issues mentioned here. Things are surely changing in a positive direction. Customer s needs drive vendors, and vendor solutions are helping customers move closer to storage utopia. This article has listed the major issues that customers can elaborate/track down further based on their setup, while vendors can use this as hints for their future development. This article has tried to touch upon many issues; I hope it has been helpful. May the space be with you! Author s Biography Kiran has been a storage professional since 2004, with hands-on experience on diverse operating and storage platforms. He is a SNIA Certified Architect and EMC Proven Professional specializing in EMC Symmetrix and CLARiiON. Kiran is currently a storage administrator supporting EMC Symmetrix, IBM DS8000 and Hitachi Tagmastore USP, totaling 1.3 Petabytes in a Mission critical set of Data Centers. Page 9 of 9