Accelerate Application Success With Operational Efficiency

Transcription

1 A Forrester Consulting Thought Leadership Paper Commissioned By HPE January 2017 Accelerate Application Success With Operational Efficiency

2 Table Of Contents Executive Summary...1 Infrastructure And Operations Must Master Delivery To Achieve Application Success...2 Process, People, And Technology Challenges Create Latency In The I&O App Delivery Pipeline...2 I&O Leverages DevOps To Speed App Delivery...5 Improve Automation To Increase DevOps Delivery Effectiveness... 6 Key Recommendations...8 Appendix A: Methodology...9 Appendix B: Supplemental Material... 9 Appendix C: Demographics/Data...9 Appendix D: Endnotes...11 ABOUT FORRESTER CONSULTING Forrester Consulting provides independent and objective research-based consulting to help leaders succeed in their organizations. Ranging in scope from a short strategy session to custom projects, Forrester s Consulting services connect you directly with research analysts who apply expert insight to your specific business challenges. For more information, visit forrester.com/consulting. 2016, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester, Technographics, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. For additional information, go to [1-10U5VM7] Project Director: Andrew Magarie, Market Impact Consultant Contributing Research: Forrester's Infrastructure And Operations research group

3 1 Executive Summary The ability to deliver high-quality applications quickly is a key differentiator for firms in the age of the customer. Infrastructure and operations (I&O) organizations have been charged with helping developers accelerate application development. I&O teams can do this by eliminating sources of latency in the application delivery pipeline through DevOps practices that emphasize automation and collaboration solutions. DevOps is a set of practices and cultural changes supported by the right tools that creates an automated software delivery pipeline, enabling organizations to win, serve, and retain customers. In September 2016, HPE commissioned Forrester Consulting to evaluate I&O teams application delivery lifecycle challenges and DevOps practices maturity. Then to further explore this trend, Forrester developed a hypothesis that tested the assertion that I&O organizations face unique application life-cycle challenges that slow delivery, and that holistic DevOps practices are needed to solve these challenges. latency, they can also lead to buggy releases that negatively impact customer experience. Twenty-eight percent of organizations said that customers are very frequently the ones finding problems in the production environment. Collaboration between Dev and Ops is a challenge that organizations must overcome. Ops and Dev work together to support applications post-deployment at 5 of surveyed organizations. However, only 3 of teams collaborate through daily stand-up meetings, and only 22% use collaboration technologies. Performance testing and production environment monitoring can improve delivery effectiveness. Performance testing is the No. 1 area with the most potential for effectiveness improvement in DevOps, with production monitoring close behind. Automating these processes and increasing visibility are key to accelerating app delivery and, ultimately, improving the customer experience. To test this hypothesis, Forrester conducted in-depth online surveys with 216 IT decision-makers at organizations that deliver applications at least once a quarter. Forrester found that the majority of these companies with a mandate to deliver software quickly leverage DevOps practices in their infrastructure and operations group to overcome people, process, and technology problems in the I&O app delivery pipeline, but with varying levels of maturity. KEY FINDINGS Forrester s study yielded four key findings: Faster app delivery is becoming an executive priority. Almost half of survey respondents stated their company s DevOps initiatives are being driven by a company executive. Organizations with executives taking responsibility for success are more likely to have more mature DevOps practices. Manual I&O processes slow release speeds and jeopardize quality. Manual review processes in test, release, and production phases can not only create

4 2 Infrastructure And Operations Must Master Delivery To Achieve Application Success Today, companies must differentiate themselves by providing unique and valuable customer experiences, and digital is a cornerstone of those engaging experiences. Businesses increasingly depend on applications to win, serve, and retain customers. A 2015 study conducted by Forrester Consulting on behalf of HP found that 64% of firms believe that their success relies on high-quality applications. 1 This means that businesses must deliver high-quality software quickly in order to support and serve customers. Delivering high-quality software fast is not just the responsibility of developers; many infrastructure and operations teams share a similar business mandate. To see what I&O teams are doing to accelerate application delivery and improve the customer experience, Forrester Consulting surveyed 216 infrastructure and operations decision-makers at companies that deliver software quickly, averaging one release per quarter or more. Our study found that: Businesses demand that I&O teams accelerate application delivery. Seventy percent of survey respondents said that the business requires their I&O teams assist in accelerating application development and delivery, making this the most important business priority for I&O. 2 I&O teams contribute significant resources to software delivery. Over 75% of infrastructure and ops organizations we surveyed said that software delivery is a top priority for the team and that they contribute significant resources to it (see Figure 1). Of these, 33% of respondents say that software delivery is their most important priority. Process, People, And Technology Challenges Create Latency In The I&O App Delivery Pipeline In order to meet the business goal of faster application delivery to win, serve, and retain customers, many organizations have adopted Agile development methodologies. Agile adoption has taken software development teams by storm, but many organizations are still failing to reach their business goals of releasing highquality applications fast. 3 Unfortunately, Agile developers, even those starting with Water-Agile-Fall, find that they are often simply creating shelfware faster because I&O pros are unable to deploy these smaller, targeted releases into production, let alone move them consistently and quickly through the development life cycle. 4 The challenges that I&O teams face in Agile delivery has led them to be perceived as a bottleneck of faster app delivery. The current application development pipeline is fraught with challenges for infrastructure and operations teams that slow down application delivery and lead to inferior application FIGURE 1 Software Delivery Is A Top Priority For Most I&O Groups Today Which of the following statements best expresses your infrastructure and operations group s attitude to software delivery? high Software delivery is our most important priority 33% priority low Software delivery is one of our top priorities, and we contribute significant resources to it We play a role in software delivery, but it is one of many priorities for us We are asked to help with software delivery for certain projects or on an ad hoc basis 2% 45%

5 3 releases. Like developers, I&O teams struggle with ecosystem complexity. However, I&O faces its own, unique challenges based on the scope of their responsibilities across the application life cycle as well as a lack of training and resources. The top factors that prevent I&O teams from moving faster are (see Figure 2): Application complexity challenges. I&O teams face challenges with app complexity that are magnified by the number of tools they deal with across the entire app life cycle and across different applications required to support the life cycle, extending beyond infrastructure tooling to developer tools and more. Our survey data shows this is the No. 1 overall challenge for I&O teams, with 6 indicating it s a top five challenge. Lack of skills. Thirty-nine percent of organizations stated that a skill shortage prevents them from developing applications faster, with 16% selecting it as the No. 1 challenge, indicating it is the most critical challenge. The root cause of this challenge lies in the rapid adoption of Agile methodologies and skills by developers, without a similar mandate for I&O teams. In developer orgs, there has been a huge push to get teams Agile and educated on DevOps practices to increase velocity, meaning these developers have a jump-start on methodologies and best practices. However, on the operations side, that level of education in automation tooling and methodologies is far less common, due to a lack of formal training. This has led to an Agile skills gap on infrastructure and operations teams that is impeding application delivery. Without the appropriate training, it will be difficult for I&O to deliver on FIGURE 2 I&O Teams Face People, Process, And Technology Challenges That Slow App Delivery What are the top factors that prevent you from moving faster (e.g., introduce lag) in tasks associated with the application development life cycle? Rank 1 Rank 2 Rank 3 Rank 4 Rank 5 Complexity of the application and/or technology ecosystem 15% 1 10% 1 Timing dependent on other ongoing project(s) 10% % Staffing resource constraints 15% 13% 12% Waiting for other roles outside of infrastructure and operations to complete tasks 14% 10% 6% Lack of budget 1 16% 4% 5% Lack of skills/expertise 16% 6% 3% 5% Lack of communication Waiting for other roles in infrastructure and operations to complete tasks 14% 6% 5% 6% 5% 5% Lack of supporting tools 3% 5% 6% Inflexible job roles 4% 6% 1 Inconsistent adoption of different tools by different teams 3% 3% 10% 13% 6% Inconsistent system and software used in development, 3% 6% test, and production environments 5%

6 4 the business mandate of accelerated delivery of quality software. Collaboration and communication challenges. In order to use Agile methodologies to deliver high-quality software at a fast cadence, I&O needs to effectively collaborate with developers to automate application delivery and support applications. Ops and Dev work together to support applications post-deployment at 5 of organizations we surveyed (see Figure 3). Further, organizations that have expanding or widespread DevOps practices are 11 percentage points more likely to have both teams supporting applications together. However, survey data shows that collaboration between the two teams is most often done on an ad hoc basis. Only 3 of teams collaborate through daily stand-up meetings, and even fewer use collaboration software to communicate with their developer counterparts (see Figure 4). Inefficient security and production processes. As more and more applications store and utilize sensitive customer data, application security has become of paramount importance for developers and I&O teams alike. The good news is that the majority of organizations in the survey now include automated security testing as part of continuous integration practices (see Figure 5). However, 3 of organizations are still using manual code review processes, which will slow release cycles. In addition, many organizations take weeks or longer to get applications changes approved for production. Over half of organizations surveyed approve changes in change advisory board meetings that take place weekly or less often. 5 These are just two examples of processes that introduce latency in the application development life cycle that slow software releases. Manual review processes in test, release, and production can not only create latency, they can also lead to buggy releases that negatively impact customer experience. One scary statistic to represent this: 2 of organizations said that customers are very frequently the ones finding problems in the production environment, and a further 4 said this happens occasionally! FIGURE 3 Dev And Ops Share Responsibility For Supporting Apps Post-Deployment At 5 Of Organizations Which team is responsible for supporting applications post-deployment? Ops and Dev working together 5 Support is outsourced 2% Ops 32% Dev Source: A commissioned study conducted by Forrester Consulting on behalf of HPE, September 2016 FIGURE 4 Collaboration Between Dev And Ops Is Often On An Ad Hoc Basis How does your IT operations team collaborate with their application development counterparts? Periodic meetings When errors occur in production in ad hoc fashion Postmortem meetings Daily stand-up meetings When errors occur in production through formal dashboards Through collaboration software (e.g., chat) 22% % Twenty-eight percent of organizations said that customers are very frequently the ones finding problems in the production environment, and a further 4 said this happens occasionally! Source: A commissioned study conducted by Forrester Consulting on behalf of HPE, September 2016

7 5 FIGURE 2 FIGURE 5 Almost 80% Of I&O Groups Are Leveraging Automated Security Testing Is On The Rise, But 3 Are Still DevOps Using In Manual Some Form Review Today, Processes But Maturity Varies Please select the statements that best describe your approach to security in DevOps processes. Please select the statement that best describes how DevOps practices are being leveraged in your infrastructure and operations group today. We do automated security testing as part of the continuous integration process Security is evaluated starting as early as the design of the code We evaluate code through automated processes but not until the end of development and functional testing DevOps practices are widespread across the entire organization We are currently expanding use of DevOps across our organization 46% 62% 60% 30% We evaluate code through manual review processes We are piloting DevOps or using with a few small teams 3 2 We do our vulnerability testing once the application is live We re not currently able to evaluate the security of our code We are considering DevOps as an option, 15% but haven t done anything yet 12% We wait until security events happen before we review our code Source: A commissioned 1 study conducted by Forrester Consulting on behalf of HPE, September 2016 I&O Leverages DevOps To Speed App Delivery DevOps practices remove barriers between developers, testers, and operations and fully reap the benefits that Agile promises. DevOps does this by providing a set of practices and cultural changes supported by complementary tools that automates the software delivery pipeline, enabling operations teams and developers to deliver what customers want faster. 6 Automation of manual processes in the app delivery life cycle can eliminate sources of human error and free up resources, which will improve the speed and quality of software delivery, which will in turn improve the customer experience. Automation is a key pillar of DevOps, and many organizations are utilizing tools to at least partially automate release management, configuration, and testing processes. Fully automating these processes in addition to others like workload management, knowledge management, and handoffs between teams and life-cycle steps will allow organizations to optimize their software delivery. The promise of DevOps is higher-quality applications, delivered faster, at lower costs. To better realize this promise, the use of DevOps practices has expanded from application development teams to infrastructure and ops teams that play a critical role in the app delivery pipeline. Many I&O groups are behind their developer counterparts in the adoption of DevOps practices, though adoption is spreading quickly. Survey results show: I&O teams are using DevOps, but maturity varies. According to our study, 7 of I&O groups are currently using DevOps in some form (see Figure 6). However, for most, DevOps is still a work in progress. Only said that DevOps is widespread across the I&O organization. And 2 are just starting to pilot DevOps, indicating that I&O DevOps maturity varies widely. DevOps initiatives are gaining executive support. Almost half of DevOps initiatives are led by company executives, meaning these initiatives are coming straight from the top (see Figure 7). This executive sponsorship and responsibility is essential for companies that are trying to expand their DevOps practices. Organizations with an executive responsible for DevOps initiatives are 11 percentage points more likely to report that they are expanding or already have widespread DevOps practices.

8 6 FIGURE 6 Almost 80% Of I&O Groups Are Leveraging DevOps In Some Form Today, But Maturity Varies Please select the statement that best describes how DevOps practices are being leveraged in your infrastructure and operations group today. DevOps practices are widespread across the entire organization We are currently expanding use of DevOps across our organization We are piloting DevOps or using with a few small teams We are considering DevOps as an option but haven t done anything yet FIGURE 7 Almost Half Of DevOps Initiatives Are Now Led By Company Executives Individual development team(s) 14% Manager of several applications 2 Infrastructure and operations professionals 10% 2 Source: A commissioned study conducted by Forrester Consulting on behalf of HPE, September % Who is primarily responsible for leading DevOps initiatives at your organization? Company executive (e.g., CIO, CEO, etc.) 4 Improve Automation To Increase DevOps Delivery Effectiveness Infrastructure and operations groups can enhance their organizations DevOps practices by focusing on key areas of the delivery pipeline that have the most potential for improvement. Specifically, I&O teams should look to (see Figure 8): Improve performance testing across the delivery life cycle. Survey respondents say performance testing is the No. 1 area with the most potential for effectiveness improvement in DevOps. Enable cross-functional performance monitoring. Both Dev and Ops teams need to be able to monitor the production environment to catch and remediate errors before they are found by customers. To avoid timeconsuming finger-pointing between different groups, use a single monitoring technology to provide as the single source of truth. Today, only 55% of organizations have these capabilities in place. 7 Automate modifying and provisioning environments. By enabling developers to provision their own environments, organizations can improve delivery speed as well as reduce risk and improve quality by standardizing the environment developers are working in. By using consistent and automated provisioning throughout the life cycle, I&O pros ensure that what has been developed and tested is what is delivered to customers. Provide metrics to continuously improve. The operations team has access to infrastructure-related metrics that can improve the delivery process, like response time, customer satisfaction, and capacity (see Figure 9). I&O teams must make sure they are providing these metrics to developers at a daily cadence in order to improve processes. While 56% of organizations said customer experience metrics are helpful in improving DevOps processes, only 52% of organizations can produce this data daily or in real time. Source: A commissioned study conducted by Forrester Consulting on behalf of HPE, September 2016

9 7 FIGURE 8 Focus On Performance Testing, Production Monitoring, And Environment Provisioning To Improve Delivery Effectiveness Which areas of your DevOps practice (provision, release, configure, monitor) has the most potential for effectiveness improvement in delivery? Performance testing across the delivery life cycle Monitoring production environment 16% Provisioning environments 14% Modifying environments Automating workload management Automating handoffs between software delivery life-cycle stage 10% Orchestrating tools across the software delivery life cycle Automating knowledge collection Increasing testing automation Version control of artifacts across software delivery life cycle Automating approvals 4% 4% 3% 3% Automating release readiness criteria 0% FIGURE 9 Key Metrics Can Improve DevOps Practices I&O Must Make Them Available What kind of infrastructure-related metrics do you use to help in improving the DevOps process? How often do you provide these metrics? In real time Daily Response time 63% Capacity (N = 120) 46% 32% Customer satisfaction 56% Response time (N = 136) 40% 36% Capacity 56% Mean-time-to-repair (MTTR) (N = 84) 50% 25% Change success 4 Load (N = 85) 32% 4 Percentage of manual versus automatic processes 40% Percentage of manual versus automatic processes (N = 88) 40% 2 Load 3 Release speed (N = 78) 25% 36% Mean-time-to-repair (MTTR) 3 Change success (N = 89) 2 32% Release speed 36% Number of production errors (N = 57) 23% 33% Number of production errors 26% Customer satisfaction (N = 121) 23% 2

10 8 Key Recommendations DevOps practices can enable IT to improve the customer experience by producing higher-quality software, faster, and at lower costs. However, DevOps adoption varies widely, not only between organizations, but also between different teams responsible for application development and delivery. Nearly half of the survey respondents have not started, or are just starting to pilot DevOps, meaning many organizations are still using waterfall or agile waterfall delivery practices. However, regardless of where your organization is in DevOps, gains in stability, speed, and quality are achievable. Forrester s in-depth surveys with IT executives yielded several important recommendations to jumpstart DevOps for I&O: Find an executive sponsor for DevOps. Organizations with an executive responsible for DevOps initiatives are 11 percentage points more likely to be expanding or have widespread DevOps practices. Take any grass roots DevOps activities and get an executive sponsor to help remove people, process and technology hindrances. Bridge collaboration and communication gaps with developers. Organizations that are expanding or have widespread DevOps practices are 11 percentage points more likely to have both teams supporting applications together. Ad hoc meetings are not sufficient to bridge the gap between Dev and Ops. Sharing goals, such as faster delivery of better quality applications, is a start, but true collaboration means both sides need visibility into the entire application delivery pipeline. Create cross-functional teams to quickly solve customer challenges and share responsibility for published software. Collaboration between developer and operations can reduce lag between steps in the app delivery lifecycle and help ensure that software being released is high quality. Automate key processes plagued by human error. Current processes are still slowing down ability to release quickly. Manual review processes in test, release, and production can not only create latency, they can also lead to buggy releases that negatively impact customer experience. Automation can not only speed delivery, it also eliminates potential sources of human error. Performance testing, monitoring the production environment, and modifying/creating environments are areas of the delivery pipeline that will benefit most from automation.

11 9 Appendix A: Methodology In this study, Forrester conducted an online survey of 216 organizations in the US, UK, France, and Germany that release applications at least once a quarter and have IT budgets over $10 million annually to evaluate infrastructure and operations teams current DevOps maturity and application delivery life-cycle challenges. Survey participants included IT decisionmakers with visibility into the test, release, and deployment stages of the application delivery life cycle. The study began in September 2016 and was completed in December Appendix B: Supplemental Material RELATED FORRESTER RESEARCH To Compete, Keep Up With DevOps Practices, Forrester Research Inc., June 23, 2016 Brief: Good DevOps Requires Collaboration, Automation, And Cultural Change, Forrester Research, Inc., June 21, 2016 DevOps Makes Modern Service Delivery Modern, Forrester Research Inc., December 16, 2015 Agile Experts Focus On Downstream Delivery, Forrester Research, Inc., October 5, 2015 Brief: Water-Agile-Fall Is A Steppingstone To Faster Delivery, Forrester Research, Inc., May 12, 2016 Appendix C: Demographics/Data FIGURE 10 Survey Firmographics In which country are you located? Which of the following best describes the industry to which your company belongs? United States 25% United Kingdom 25% France 24% Germany 25% Approximately what is the size of your annual IT budget in US dollars? $500 million or more $200 million to $499 million $50 million to $199 million $10 million to $49 million % Technology/software Manufacturing and materials Financial services and insurance Retail Business or consumer services Transportation and logistics Chemicals and metals Construction Consumer product manufacturing Telecommunications services Healthcare Government Energy, utilities, and waste management Media and leisure Electronics Education and nonprofits Agriculture, food, and beverage 14% 12% 12% 1 6% 5% 4% 3% 2% 2%

12 10 FIGURE 11 Survey Demographics Which of the following most closely describes your job function in IT? CIO/Office of the CIO/CTO IT infrastructure and operations PMO/project or program office 1 24% 42% Which title best describes your position at your organization? Manager 12% Project manager 6% C-level executive 3 Applications development Enterprise architect Line-of-business management QA/Testing director manager 6% Director 35% Vice president 10% Are you involved in the provisioning, deployment, monitoring, or support of any of the following applications? How often does your team or teams release applications? Mobile apps (B2B or B2C) ecommerce apps (B2B or B2C) Cloud-native apps 76% 75% 74% We average a release a day 15% We release many times per day 5% We average one release per quarter 2 Java web apps 70% Packaged apps Other internal apps IoT apps 55% 4 63% We average one release a week We average one release a month 32%

13 11 Appendix D: Endnotes 1 Source: A commissioned study conducted by Forrester Consulting on behalf of HP, April Source: Agile Experts Focus On Downstream Delivery, Forrester Research, Inc., October 5, Source: Brief: Water-Agile-Fall Is A Steppingstone To Faster Delivery, Forrester Research, Inc., May 12, Source: To Compete, Keep Up With DevOps Practices, Forrester Research Inc., June 23,