White Paper Cloudian HyperFile Brings Enterprise NAS Functionality Closer to Object-Based Storage Sponsored by: Cloudian Amita Potnis December 2017 EXECUTIVE SUMMARY As the volume of unstructured data created and used continues to increase rapidly, organizations are faced with challenges around data storage. In response, many new technologies specifically objectbased storage, software-defined storage, all-flash arrays, and cloud-based storage services are experiencing rapid adoption. Yet object-based storage has been limited by the need to rewrite existing file-based applications a prospect that is costly and requires more manpower than is practical for many organizations. New solutions are now available that are designed to address the limitations of object-based storage. In effect, these solutions combine enterprise-class file service capabilities with object-based storage platforms and have extensive features and functionality akin to enterprise network-attached storage (NAS). As a result, these solutions are appropriate for a wide range of file-based applications. Indeed, object-based storage that includes support for cloud and on-premises applications in a single management console is the foundation for next-generation NAS solutions. Cloudian offers both an object storage platform and more recently an enterprise file services product. When deployed together, Cloudian's products are designed to consolidate file and object data into a single repository and can enable organizations to manage environments that span the cloud as well as on-premises. SITUATION OVERVIEW Management of rapidly expanding data sets is a central challenge for IT organizations. Today's digital transformation (DX) era drives business outcomes. Business imperatives translating to outcomes in turn drive information technology (IT) strategies that support management, maintenance, and analytics to gain insights from extensive data sets. The widespread use of mobile devices and social platforms paired with artificial intelligence (AI) and machine learning (ML) projects results in these extensive data sets. According to IDC, by 2025, the digital datasphere will grow to 163ZB, or 163 trillion gigabytes. In comparison, 2016 generated 16ZB of data. The amount of data generated will continue to outpace our ability to store it and will drive a need for large, scalable storage environments with significantly lower management overheads and costs. December 2017, IDC #US43243617
Much of the data growth is driven by unstructured data in the form of files audio, video, images, documents, and so forth. Across various industries, this growth creates multiple challenges for any IT organization, including: Aging infrastructure with performance and scalability limitations Data sprawl across disparate storage solutions Management, maintenance, and data security Existing network-attached storage solutions often struggle to meet these growing data storage requirements because of the limitations of scale. Many customers report that managing seven or more NAS arrays becomes an enormous task because of increased complexity. At the same time, tightening budgets threaten to limit storage purchases, which drives painful trade-offs. And finally, there are new business objectives that must be supported through analysis of existing data sets, a trend that will further accelerate with increased adoption of AI/ML technologies. In some sense, IT organizations face pressure from all directions as though continuously playing the game of tug-of-war while delicately balancing the needs of traditional and next-generation business requirements. To serve the needs of growing data sets, including unstructured data, many new storage technologies such as object-based storage, software-defined storage, all-flash arrays, and cloud-based storage services are seeing rapid adoption in the marketplace. Adoption of object-based storage, however, has been limited somewhat by application support employing object-based storage for unstructured data often means rewriting existing file-based applications, which can be a strenuous task from a financial and manpower standpoint. New solutions promise to remedy this limitation by merging enterprise-class file service capabilities with object-based storage platforms. While products that deliver file services on object storage have been in the market for some time now, the solutions were often limited in features and performance or were too costly for general-purpose applications. New alternatives now deliver richer features and functionality approaching the full capabilities of enterprise NAS making them viable for a much wider range of file-based applications. Furthermore, because these solutions are built on object storage, which is the native storage technology employed by most cloud providers, they offer cloud convergence capabilities that combine multiple environments in a single management framework cloud and on-premises environments in one. With file support, that same framework also includes both file and object storage. The result is the foundation of next-generation scale-out NAS. Some key requirements of next-generation NAS solutions are: Ability to align IT projects. Organizations can choose various file- and object-based storage platforms to serve the needs of unstructured data including infrastructure as a service (IaaS), platform as a service (PaaS), cloud-native applications, and on/off-premises traditional or cloud deployment locations. The next-generation solution will be flexible and lend itself to make on-premises data available to cloud-native applications with integrated management tools. Reduced costs and increased efficiency. Any next-generation NAS solution will support converged data access across platforms such as file- and object-based storage. In essence, data can be written as file and read via S3 or other cloud-based applications or vice versa. 2017 IDC #US43243617 2
This will enable enterprises to use their traditional file-based applications while leveraging lowcost, extremely scalable object stores. Extensive search and insights. Enterprises increasingly need data to be searchable and quickly accessible across storage technologies. A next-generation NAS solution will enable this capability across file- and object-based storage platforms in support of many workloads including ediscovery, artificial intelligence, and machine learning. Risk mitigation. Support for snapshots or write once, read many (WORM) as a feature will protect against accidental deletion of files or in case of unexpected disasters. CLOUDIAN'S FILE AND OBJECT OFFERINGS Object-based storage can address many CIO concerns in terms of reduced costs and risk and business alignment objectives. Cloudian has been a long-standing object storage supplier, offering an object storage platform called Cloudian HyperStore. The firm has recently announced an offering in the enterprise file services space called Cloudian HyperFile. Through these two offerings, Cloudian aims at consolidating file and object data into a single repository. Cloudian HyperStore is an on-premises, scale-out object storage system with a multitenant, multiapplication, distributed storage technology that can scale from 10TB to large-scale environments comprising hundreds of petabytes. Typically deployed on-premises by organizations building a private cloud, HyperStore is also widely used by public cloud service providers to support S3-compatible storage services. The product is available in the form of an appliance or as software that can be deployed on commodity hardware. The high-density appliance supports 840TB in a 4U rack. Capacity can be added seamlessly without downtime, and nodes can be swapped out every four to five years without disruption. This combination allows business to run without any interruptions. Cloudian HyperStore provides a single pool of storage for reduced operational expense, enabling a full-time IT employee to manage installations larger than 10PB single-handedly. In addition to its full S3 API compatibility and petabyte scalability, a key differentiator for Cloudian HyperStore is the company's hybrid cloud data management capability it can tier data to the public cloud by either replicating on-premises data or migrating it. In either case, Cloudian manages the entire data set, on-premises and in the cloud, from a single pane of glass. Multicloud support is included as well, including GCP, Microsoft Azure, Amazon S3, and S3 compatible public clouds. In each of these clouds, data is stored in the platform's native format. For example, data stored to Azure Blob can be directly accessed by Azure applications, without the need to traverse the HyperStore platform. With Cloudian, organizations therefore have a single S3-compatible access point and a single management console that can traverse all major cloud platforms and will store data in that cloud's native format. Deploying Cloudian HyperFile on Cloudian HyperStore Cloudian recognized that the adoption of file services on object storage has been limited largely by the lack of enterprise NAS capabilities such as high availability, snapshots, and WORM. To tackle this issue, Cloudian announced Cloudian HyperFile, a POSIX-compliant, scalable enterprise file services 2017 IDC #US43243617 3
platform. When deployed in conjunction with Cloudian HyperStore, the combined offering will enable customers to capitalize on the cost and scale benefits of object stores for file services. HyperFile includes features that encompass the most important capabilities of enterprise NAS, including high availability (with dual-node controllers operating in active-passive mode), snapshots, WORM, and encryption. Users can create up to 64 namespaces on a single HyperFile controller, and a single namespace can span multiple sites. Cloudian claims that theoretically there is no threshold on the number of sites supported but that typically its customers have deployed three sites. Data access is supported via NFS v3/v4, SMB (CIFS), and FTP interfaces. Active Directory authentication maintains data access across sites while keeping data secure. Because HyperFile employs Cloudian HyperStore as its back-end storage, it benefits from that platform's integrated data protection, which includes erasure coding and data replication features that can deliver up to 18 nines data durability, depending on the configuration. In many use cases, this configurable data durability eliminates the need for a separate backup process and software license. Cloudian HyperFile includes a data migration engine that supports nondisruptive data migration from existing NAS solutions. This simplifies the implementation task by allowing IT managers to select and then migrate either specific shares or an entire NAS device. Migration occurs as a background task, thus eliminating the downtime and disruption that might otherwise occur. Universal Data Access To support an organization's cloud initiatives, next-generation NAS solutions will make on-premises data available to cloud-native applications. HyperFile supports this requirement with the concept of data anywhere file data can be transparently accessible in both cloud and on-premises locations. Within the datacenter, this allows both file-based and object-based applications to access the same data. Users can write file data to on-premises storage and then directly read that data from HyperStore as an object. There is no need to again traverse HyperFile. When data is replicated or migrated to the cloud, the same concept applies. Data written as files can be directly accessed in the cloud, again with no need to traverse HyperFile. That is, a file stored in the datacenter via SMB/NFS can be read in the cloud as an object. Effectively, data access is unified files and/or objects become portable across deployment models. For multicloud deployments, Cloudian supports this universal access on Microsoft Azure, GCP, Amazon S3, and all S3-compatible clouds. Across clouds and on-premises storage, files and objects become portable. Use Cases and Configurations Cloudian HyperFile and HyperStore together serve many use cases including data protection, storage as a service, active archives, video surveillance, collaborative workloads, and global file sharing. Likely vertical markets include media and entertainment, healthcare, engineering, finance, and retail. Because the solution includes WORM for data immutability, it will likely find applications in medical and financial record management. IT managers concerned about malware and ransomware threats also view WORM storage as an added line of defense against these attacks. Cloudian HyperFile is available in two varieties: Basic and Enterprise. Cloudian HyperFile Basic software is included as a standard feature of HyperStore. HyperFile Basic includes essential NAS 2017 IDC #US43243617 4
functionality, including multiprotocol support and high availability. Complete on-premises solutions including storage start at half a cent per gigabyte per month. HyperFile Enterprise is separately licensed by capacity and adds snapshot, WORM, and global namespace capabilities. Cloudian HyperFile Basic and Enterprise are available both as a virtual machine and as an appliance. The 2U height HyperFile appliance includes active/passive controllers and local caching for reduced latency. Various levels of caching are offered, allowing IT managers to configure HyperFile to match their performance needs. All Cloudian products are offered directly from Cloudian sales and from resellers. CHALLENGES/OPPORTUNITIES Cloudian HyperFile offers an additional class of NAS, with scale-out capacity and performance, at substantially less cost than traditional enterprise NAS. It is best suited for large-scale workloads that are not latency sensitive. Recently the market has seen an increased interest in file services and capabilities. File service offerings that work adjacently to existing object stores are a relatively new concept and are gaining traction in the marketplace. One main challenge would be the existing mindset of end users who are used to traditional NAS solutions and may be cautious about a new offering. Such end users typically have existing relationships with traditional storage vendors, making the barrier for entry higher. According to IDC's recent file- and object-based storage adoption survey, 60% of the respondents are considering either retiring or replacing existing file-based storage solutions. The top 3 reasons for organizations to consider retiring or replacing their file-based storage infrastructure are high hardware costs, aging of file-based storage solutions, and concerns about scalability. Cloudian's products leverage commodity hardware, an established object storage offering, and now an enterprise NAS offering that works with its object store. Combined, these products can address the aforementioned concerns well and serve as a long-lasting, future-proof storage solution. CONCLUSION Unstructured data is expected to grow, and IT organizations will continue to seek new solutions that make management, maintenance, and analytics easier. According to IDC's Worldwide File- and Object-Based Storage Forecast, 2017 2021 (IDC #US42280717, September 2017), capacity deployed to support file and object environments will reach 422EB, growing at a CAGR of 30.6%. Cloudian's long-standing experience in object storage has earned HyperStore a position as a Major Player in IDC MarketScape: Worldwide Object-Based Storage 2016 Vendor Assessment (IDC #US41918416, December 2016). Cloudian's new offering, HyperFile, a strategic extension to Cloudian's object storage offering, is poised to help the company serve its customers more efficiently. 2017 IDC #US43243617 5
About IDC International Data Corporation (IDC) is the premier global provider of market intelligence, advisory services, and events for the information technology, telecommunications and consumer technology markets. IDC helps IT professionals, business executives, and the investment community make factbased decisions on technology purchases and business strategy. More than 1,100 IDC analysts provide global, regional, and local expertise on technology and industry opportunities and trends in over 110 countries worldwide. For 50 years, IDC has provided strategic insights to help our clients achieve their key business objectives. IDC is a subsidiary of IDG, the world's leading technology media, research, and events company. Global Headquarters 5 Speen Street Framingham, MA 01701 USA 508.872.8200 Twitter: @IDC idc-community.com www.idc.com Copyright Notice External Publication of IDC Information and Data Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2017 IDC. Reproduction without written permission is completely forbidden.