The State of Kubernetes 2018

Transcription

1 RESEARCH REPORT The State of Kubernetes 2018 Presented by: Heptio

2 Introduction There have been several studies conducted recently on cloud native technologies. These studies have focused more on measuring the momentum of specific technologies, and less on the underlying factors that influence decision makers and affect deployments. Heptio was interested in shedding more light on cloud native obstacles and priorities, so we commissioned Dimensional Research to conduct a study to dig a little deeper. The resulting research effort tapped nearly 400 organizations for their perspective on why cloud native technologies more specifically Kubernetes have gained so much momentum in such a short time. Our goal was to explore various characteristics of the companies that are adopting Kubernetes in greater detail: how they re going about it, who s making the decisions, and what the biggest benefits are. Since it was open sourced in 2014, Kubernetes has become the orchestration platform of choice for containers. Developers find Kubernetes to be a practical framework for application development and management. And it s not just great for development, organizations are running Kubernetes in production with confidence. Operators are digging out from a mountain of tickets and expanding to multiple clouds. Both developers and operators appreciate Kubernetes for its open and extensible design, failure tolerance, ability to scale, and its open and transparent community. For technology leaders, Kubernetes is fostering the transition to cloud native architecture and transforming the way businesses operate. What are you going to learn from this study? For starters, you ll find out the stage where organizations need the most help... and where they go to get it. You ll learn how organizations that dove headlong into Kubernetes are fairing, and discover their two biggest asks from the Kubernetes community. And you ll see which organizations are achieving the greatest velocity and increasing resource utilization. We believe the study results are a compelling case for cloud native and proof that Kubernetes is being used to deliver real business impact. 2 The State of Kubernetes 2018

3 Setting the Stage Before we dig into the key takeaways of this report, it s important to set the stage. The next few pages provide some details on who we talked to and some specifics regarding our findings on Kubernetes and multi-cloud adoption. Demographics We surveyed 378 IT and technology decision makers responsible for cloud native technology deployment and strategy. Care was taken to include key stakeholders in both operations and development. At right is the demographic make-up of our respondents. Almost all the organizations surveyed have a significant developer footprint. About 37% have between 100 and 1,000 software developers, and 23% have more than 1,000 developers. While our sample is slightly overweight in technology companies (26%) and financial services companies (16%), all major industry sectors are represented including: education (9%), manufacturing (8%), and healthcare (6%). Later in this report, we ll look at some key differences in sentiment based on demographic variables such as company size and number of developers. Region United States or Canada (72%) Europe (16%) Asia (4%) Latin America (4%) Australia or New Zealand (2%) Middle East or Africa (2%) Company Size (# of employees) More than 10,000 (37%) 5,000 10,000 (21%) 1,000 5,000 (42%) Job Level Team Manager (42%) Senior Individual Contributoor (29%) Executive (18%) Primary Job Responsibility IT Operations (23%) Architecture (19%) DevOps (18%) Software Development (18%) Infrastructure (17%) Product Management (3%) Site Reliability (2%) Number of Software Developers More than 2,500 (14%) 1,000 2,500 (9%) 500 1,000 (11%) (26%) (24%) Less than 10 (16%) Developer, Admin, or Other Frontline Staff (11%) 3 The State of Kubernetes 2018

4 Kubernetes Adoption One of the first questions we asked respondents is where they are with Kubernetes adoption. 60% are already using Kubernetes for container orchestration. Within a year, the number of Kubernetes adopters in this sample will grow to 65%, while the use of alternative orchestration platforms (Docker Swarm, Mesos, and many others) will fall or remain flat. We also wanted to find out the types of applications Kubernetes is supporting. There s a common misconception that containers are only useful for stateless applications 1. However, as the figure shows, organizations today trust Kubernetes with multiple application types, including data-intensive analytics, machine learning, and IoT that are anything but stateless. Stateless (static) 65% 60% 60% of respondents are using Kubernetes today Stateful (dynamic) 63% Data analytics 53% Machine Learning 31% IoT 27% 0% 10% 20% 30% 40% 50% 60% 70% The State of Kubernetes 2018

5 Kubernetes appears to be helping build people s confidence in open source software as a whole. When we asked whether they were using open source or proprietary technology for important container infrastructure software, we found that respondents were far more likely to use an open source platform (largely Kubernetes) and also strongly preferred open source solutions for monitoring and logging. Solutions further from the core (load balancing, storage, etc.) were slightly more likely to be proprietary. Kubernetes in Production Kubernetes is more than just an experiment. Overall, 30% of the organizations surveyed use Kubernetes in production, and the bigger the organization, the more likely that becomes: Among the forty percent of organizations not yet using Kubernetes, the number one roadblock was lack of internal expertise (54%) followed by no urgency (41%). These organizations need a champion to move the ball forward. Open source Proprietary 53% of organizations with more than 1,000 containers use Kubernetes in production 80% 70% 60% 50% 72% 43% 64% 49% 65% 45% 51% 56% 57% 66% 49% 61% 46% of organizations with more than 1,000 developers use Kubernetes in production 40% 30% 20% 10% Among all organizations using containers in production, 68% are using Kubernetes twice as many as are using Docker Swarm (34%). 0% Platform Monitoring Logging Networking Storage Load Balancer 5 The State of Kubernetes 2018

6 The Multi-Cloud Movement Multi-cloud is more than just an idea; it has become common practice. Two out of three organizations (69%) are already operating in multiple clouds. Of the public cloud providers, AWS dominates today: 69% of respondents picked AWS as a current cloud platform versus 50% for Microsoft Azure and 27% for Google Cloud Platform (GCP). However, the footprints of both Azure and GCP are expected to grow to 53% and 36% of respondents respectively over the next year. The reasons why organizations choose multiple clouds are exactly what you would expect, as illustrated in the figure. A Word to the Wise Kubernetes, correctly implemented, is inherently multi-cloud, allowing you to deploy the same applications in various public clouds or on-premises with new management tools offering a common operating environment. Keep in mind that cloud providers don t have much incentive to enable multi-cloud, so the key to successful multi-cloud Kubernetes is to avoid getting locked in to a single vendor. Understanding where community-provided technology ends and vendor-proprietary distributions begin can be difficult. You can quickly get locked in if you create applications that rely on closed-source capabilities. 8% Other 6% There are no benefits of using multiple clouds 29% Simplicity of operation 35% Improved performance 41% Reduced cost of operation Reasons why organizations choose multiple clouds 47% Negotiating power with providers 54% Greater access to features/ capabilities 6 The State of Kubernetes 2018

7 Central Themes In this section, we explore peoples experiences with Kubernetes a little more deeply. In particular, we wanted to know how variations among the organizations surveyed have affected both their attitudes and results. The most interesting breaks come in comparing the responses of organizations based on their stage of deployment (evaluation, development, production) and the overall scale of their operations. We will tease out these factors as we explain the following key takeaways: The obstacles The opportunities The impacts 7 The State of Kubernetes 2018

8 Top Pain Points The Obstacles Kubernetes is young, and the ecosystem is still taking shape. Different capabilities within Kubernetes and the many associated ecosystem tools are changing rapidly and not all at the same pace. Consequently, organizations are encountering a number of different obstacles while evaluating and running Kubernetes. Lack of internal expertise is a persistent issue, but many of the problems, such as decision-making and team alignment, are exactly what you would expect in organizations going through a major transition in operations. Navigating design and deployment decisions (47%) Aligning internal teams around common technologies (47%) Managing a growing footprint without internal expertise (45%) Ensuring reliability to support mission critical applications (43%) Managing complex policies (42%) 8 The State of Kubernetes 2018

9 What stage of Kubernetes adoption requires the most help? 41% Architecture Design The Importance of Patience When we asked at what stage of Kubernetes adoption they required the most help, 41% of respondents selected architecture design. This makes intuitive sense because there s so much to think about. You have to evaluate both your physical and software infrastructure, how to integrate new technologies, how to operate in new ways, etc. A distant second at just 16% of respondents was deployment, and just 14% chose day-to-day management. This suggests to us that the pain of Kubernetes adoption is most acute early on and that the situation stabilizes fairly quickly. 16% 14% Deployment Day-to-Day Management 9 The State of Kubernetes 2018

10 A few additional challenges emerged as we examined the data further Who ya gonna call? People appear to be relying almost equally on a variety of sources for advice, ranging from orchestration providers to independent consultants to peers and forums. Given that just 4% of respondents deployed Kubernetes without any help, this suggests that organizations may be struggling to find the guidance they need. Who s the boss? When we asked who is driving architectural decisions for Kubernetes, we got a wide range of answers. In many organizations, it may be unclear where the responsibility for Kubernetes should actually reside. As a result there are a lot of seats at the table, and different interests at play. It s no wonder aligning teams was a top pain point. Where do you go for expert advice? Who is driving architectural decisions for your Kubernetes footprint? Infrastructure team 67% IT operations team 54% Orchestration platform provider 55% Industry peers 54% Site reliability engineering Cross-functional group 27% 31% Line of business 20% External consulting firm 10% Online help or forums 53% External independent consulting firm 42% Other 4% 0% 10% 20% 30% 40% 50% 60% 70% 80% 10 The State of Kubernetes 2018

11 SRE-Ops is the new DevOps Another pattern that s clear from the data is that larger organizations are much more likely to depend on Site Reliability Engineering (SRE). The SRE role combines aspects of IT operations and software engineering with the goal of ensuring software reliability and scalability. The bottom line is that we believe SRE- Ops will be critical in environments where DevOps must scale. 31% 38% 48% Cloud native technology adoption changes both an organization s infrastructure and its team structure. of all respondents lean on SRE-Ops to help make architectural decisions of organizations running Kubernetes in production depend on SRE of organizations with more than 1,000 developers depend on SRE 11 The State of Kubernetes 2018

12 The Opportunities The good news is that most respondents appear to have a very clear understanding of what they need to improve their Kubernetes environments. Solutions are already available in the ecosystem in some form for the capabilities shown in the figure. There are a few opportunities in particular that stand out. Mo clouds, mo problems While multi-cloud has been widely embraced, it creates some new challenges. Respondents reported complexity of management (71%) as the biggest challenge. As you might expect, complexity goes up when you deploy Kubernetes in multiple clouds. Any gaps in management tools are sure to become more apparent. The second biggest multi-cloud obstacle at 45% was once again internal alignment. Talkin bout lock-in Organizations in production with open source Kubernetes are more motivated to avoid vendor lock-in (66% vs an average of 60%) and to enable multi-cloud (56% vs an average of 48%). In particular, organizations with more than 1,000 developers are much more likely to recognize the risks of lock-in in terms of its effects on cost, access to technology, and support (80% vs an average of 60%). 12 The State of Kubernetes 2018

13 Are you seeing what I m seeing? One item on almost everyone s wish list is greater visibility into exactly what s happening across their Kubernetes footprint. In particular, organizations that have more than 1,000 developers are much more likely to desire greater visibility into real-time cluster performance (70% vs 58% average). Those same organizations are also more likely to prize access to historical audit logs and forensics (53% v. 45% average). No matter where you are in Kubernetes deployment, it s worth checking out the wide variety of ecosystem solutions and considering what additional capabilities will help you to better run your Kubernetes footprint. What additional capabilities would help you? 58% said 55% said 49% said 49% said 45% said Simplified cluster Ongoing Integrated support backup and recovery conformance testing and troubleshooting Greater visibility into performance Access to historical audit logs and forensics 13 The State of Kubernetes 2018

14 An Open Letter to Organizations Peering Over the Edge If you re currently evaluating Kubernetes or in the midst of development, there are some important lessons you can learn from those who have gone before you. Need for greater real-time visibility Production 65% Development 55% Evalutaion 50% According to 65% of those in production, you are going to need visibility into real-time cluster performance. This was ranked even higher (69%) by those operating greater than 1,000 containers. As you scale the number of containers, you need more clusters. As you operate more clusters across more clouds you need greater visibility and control. Running a close second is the need for access to historical audit logs and forensics. Of those running Kubernetes in production, 65% saw this as a critical capability. Write-in responses to this question focused on the need for standardization/conformance testing and simple backup. 14 The State of Kubernetes 2018

15 The Impacts Is it worth it to work through the obstacles and deploy Kubernetes? Our study suggests that companies using Kubernetes in production are highly likely to realize the big three benefits of shortened software development cycles, more efficient resource utilization, and access to complementary open source technologies. It goes without saying (but we ll say it anyway) that for most organizations increasing the velocity of software development while improving resource utilization is the prime directive. Kubernetes and related technologies are the tools that make that possible. The big three benefits are achieved less often in small deployments. Organizations in production are likely to be of larger scale; 18% are running more than 50 clusters (versus 8% on average and just 4% for organizations in deployment). 40% Primary benefits of Kubernetes 47% Access to complementary technologies 58% 68% Shortened software dev cycles 63% More efficient resource utilization Avg 69% In production 15 The State of Kubernetes 2018

16 Summary and Recommendations Kubernetes is in production, and the entire ecosystem is evolving quickly with new and enhanced tools and solutions being released all the time. While current adopters are working to overcome some obstacles, on the whole respondents to this survey clearly believe that the benefits outweigh any potential challenges. To advance the cause of Kubernetes, the central need is for greater visibility and control. Organizations already want the ability to see across multiple clusters and multiple clouds. This requirement highlights the importance of upstream Kubernetes establishing a platform that is cloud independent so as to avoid lock-in. This ensures organizations have greater access to best-in-class open source technologies and maintain maximum flexibility at a time when the ecosystem is changing so rapidly. If you re considering Kubernetes or in the process of evaluation and deployment, you can benefit from the hard-won wisdom of those who have gone before you. No one said the move to cloud native technologies including Kubernetes would be simple, but it s clearly worth it for organizations that prioritize software development and want to achieve faster development cycles, better resource utilization, and access to best-in-class open source technology. 16 The State of Kubernetes 2018

17 Learn More If you want to pick up the cloud native conversation with a team that holds strong opinions about Kubernetes, we re at your disposal. Visit heptio.com to inquiries@heptio.com