How Performance Management Maturity Drives Business Agility and Innovation

Transcription

1 Research Paper How Performance Management Maturity Drives Business Agility and Innovation A Data-Driven Performance Management Maturity Model for Modern IT By Edwin Yuen, Analyst; with Adam DeMattia, Director of Research January 2017 This ESG Paper was commissioned by Riverbed Technologies and is distributed under license from ESG.

2 White Paper: How Performance Management Maturity Drives Business Agility and Innovation 2 Contents Performance Management Market Overview... 3 The Importance of Performance Management Maturity... 4 Improving the Application Development Process... 4 Positive Impacts on Application Problem Resolution... 5 Increased Usage of Cloud Technologies... 6 IT s Standing as a Competitive Differentiator for the Business... 7 Behaviors That Organizations Should Adopt to Become Experts in Performance Management... 7 Wider Breadth of Performance Management Capabilities... 7 More Frequent Evaluation of and Investment in Performance Management... 8 The Bigger Truth... 9 Assess Your Performance Management Maturity Today... 9

3 White Paper: How Performance Management Maturity Drives Business Agility and Innovation 3 Performance Management Market Overview In the modern IT organization, there needs to be exceptional understanding not only of the performance of systems, but also of their impact on applications and end-users. Simple monitoring and alerts for errors related to infrastructure and applications, without greater impact analysis, is not sufficient. The industry is coalescing behind the idea of holistic performance management, defined as the collection of technologies used to monitor and remediate issues throughout the stack: infrastructure, network, application, and end-user experience. Performance management gives the complete view of IT systems and allows IT to respond to issues throughout the entire application architecture. This allows IT to maximize application availability and performance, thus maximizing the value IT delivers to the business. Still, ensuring optimized performance is not easy. As the IT landscape continues to change, with the rapid adoption of cloud-based technologies, as well as the proliferation of third-party application components and microservices, it becomes increasingly difficult for IT to gain a full view of the application environment. IT cannot simply take a monolithic view of an application and its components. Problems can arise in many places across an environment: the network, software code, the devices applications are consumed on, etc. These resources can reside within the traditional data center or in the cloud, where the ability to monitor resources is radically different than IT is used to. The numerous places where issues can arise and the complexities of potential issues drive the need for sophisticated performance management solutions that can monitor and diagnose issues at each layer. Cross-solution integration is also key so that information can be shared across tools, correlated, and presented in the context of the full application architecture, without becoming a burden on IT in terms of cost or complexity. Measuring Performance Management Maturity Riverbed commissioned ESG to conduct a survey to assess the maturity of organizations with respect to performance management and create a data-driven segmentation model to correlate maturity to other organizational behaviors and outcomes. The research consisted of 345 qualified respondents. Participants included IT professionals responsible for or familiar with their organization s networking and performance management solutions (96%) or business decision makers holding a manager title or higher who have application ownership responsibilities (4%). The survey covered a range of company sizes, from midmarket (250 to 999 employees, 27%) to enterprise (1,000 or more employees, 73%). The geographical coverage of the research included North America (57%), Western Europe (29%), and Asia Pacific (14%). The survey also included a wide range of industry verticals, including financial services (19%), manufacturing (12%), retail (10%), and technology (10%). Respondents were segmented through a scoring system based on the questions about performance management. The segmentation questions included the following areas: Business alignment: Vision, strategy, metrics, and measurement Organization: People, leadership, and collaboration Process: Documentation, consistency, and continuous improvement Technology: Tools, capabilities, and integration Each possible response to each question in the areas listed above was assigned a point value. Scoring was designed to give each segmentation criteria roughly equal weight while still ensuring particularly forward-looking behaviors were given a commensurately high maturity score. Respondents could earn a maximum of 40 maturity points and a minimum of 0 maturity points based on responses. Respondents achieving a score in the bottom third of possible scores (0-13 pts.) were

4 White Paper: How Performance Management Maturity Drives Business Agility and Innovation 4 rated as Nascent, respondents scoring in the middle third (14-26 pts.) were rated as Aspiring, and respondents scoring in the top third (27-40 pts.) were rated as Expert. Table 1 displays the specific vectors of maturity on which organizations were assessed, the responses in each area evaluated to be the most mature, and the percentage of Expert, Aspiring, and Nascent respondents who identified with those most-mature states. Table 1. Aspects of Performance Management Maturity on which Respondents Were Evaluated Business Alignment Organization Process Technology Most Mature State Line of business can access performance metrics in a self-service fashion via dashboards or automated reporting tools Routinely analyze network/app/infrastructure/end-user experience/business performance metrics Performance management center of excellence is responsible for performance management Vice president-level position oversees performance management Regular collaboration with line of business about policies, processes, and results Formal performance management policies and processes documented and used by IT and line of business Extensive integration between performance management tools Tools and processes monitor all business-critical applications Percentage of Expert Organizations Percentage of Aspiring Organizations Percentage of Nascent Organizations 38% 17% 1% Range of 88-78% for all metrics Range of 60-42% for all metrics Range of 48-9% for all metrics 66% 4% 0% 22% 7% 1% 84% 18% 3% 84% 25% 8% 56% 8% 2% 78% 36% 14% The Importance of Performance Management Maturity As ESG examined the resulting data, clear correlations between Expert organizations and several application key performance indicators (KPIs) and technology adoption trends emerged. These correlations include: Application Development Meeting application timelines, reducing application issues, and agile development Application Problem Resolution Increased availability, reduced time to resolution, and reduced negative impacts Embracing Transformative Technologies Wider and deeper use of cloud technologies Many of these KPIs have significant IT and line-of-business impacts, which will be discussed in this paper. Improving the Application Development Process As organizations begin to implement DevOps principles and practices, the ability of IT operations to support the growing size and speed of internal development initiatives will become critical. IT will no longer simply supply resources to developers; rather, operations and development staff will work in concert to ensure development excellence.

5 White Paper: How Performance Management Maturity Drives Business Agility and Innovation 5 ESG s research highlights that organizations rated as performance management Experts experience a significantly smoother application development process. Nascent organizations reported twice as many internally-developed applications were released behind schedule (mean of 40%) compared with Expert organizations (mean of 20%). Similarly, Experts experience issues during application deployments at a much lower rate (see Figure 1). Figure 1. Reduced Application Release Failures Frequency with which respondents report that 10% or fewer of internally-developed application releases result in change failures. (Percent of respondents) 74% 38% 40% Three out of four Expert organizations report 10% of app releases or fewer cause issues. Only about two of five Aspiring or Nascent organizations report such a low frequency. Expert (N=31) Aspiring (N=211) Nascent (N=92) Moreover, organizations with Expert performance management maturity also reported higher adoption of agile development practices. One hundred percent of the Expert organizations claimed to have good or extensive adoption of agile development methodologies (compared with 70% of Aspiring organizations and 58% of Nascent organizations). While we cannot claim a causal relationship, it is clear that there is a positive correlation. ESG posits that as application development becomes more dynamic, with more releases and continuous improvements, it becomes more important to be able to monitor whether application changes cause issues for any users. Organizations with a high level of performance management sophistication are well placed to rapidly evolve the way application code is delivered to end-users, enabling faster delivery of more enhanced applications with fewer break/fix events. Positive Impacts on Application Problem Resolution As noted above, Expert performance managers see a noticeable reduction in application development issues. Still, problems can and do occur in the operation of custom-built and packaged applications. However, ESG research shows a positive correlation between increasing maturity and the uptime of applications, time to resolution, and avoidance of negative impacts experienced. In terms of uptime, Expert organizations are more than twice as likely as Aspiring organizations to report that 5% of critical apps, or fewer, have issues on a monthly basis (66% vs. 29%). A similar pattern is observed with Nascent organizations (34%). Fifty-nine percent of Expert organizations also indicated their mean time to resolution as less than 30 minutes, which is 1.8x higher than the percentage of Aspiring organizations (32%) and 2.5x higher than the percentage of Nascent organizations that indicated the same mean time to resolution (see Figure 2).

6 White Paper: How Performance Management Maturity Drives Business Agility and Innovation 6 Figure 2. Frequency of Application Issues and Time to Resolution In a typical month, what percent of your organization s business-critical applications have unplanned outages/significant performance degradations that result in a negative end-user or customer experience? (Percent of respondents) 5% or less More than 5% 34% Expert 66% 71% Aspiring 29% 65% Nascent 34% When a business-critical application suffers from an unplanned outage or significant performance degradation, what is the average time to resolution? (Percent of respondents) 30 minutes or less More than 30 minutes 41% Expert 59% 67% Aspiring 32% 73% Nascent 24% ESG also asked about specific negative outcomes for the application issues that have been experienced. Expert organizations were over twice as likely to report no negative outcomes associated with application issues in the past 12 months as Aspiring organizations, and nearly three times as likely as Nascent organizations. To conclude, Expert performance management organizations had fewer application issues, which were resolved faster and were less likely to cause negative business outcomes. Increased Usage of Cloud Technologies Another behavior that exhibited a correlation to Expert-level maturity is the use of public cloud services. ESG s research found that Expert organizations were using SaaS (78% vs. 65% and 60% of Aspiring and Nascent organizations, respectively), IaaS (59% vs. 59% and 49%), and PaaS (59% vs. 45% and 46%) at least as frequently as their counterparts. Additionally, Expert organizations using the cloud also run a greater percentage of their applications in the cloud as opposed to Aspiring or Nascent organizations. Two-thirds (66%) of Expert organizations using the cloud today report that more than 30% of their applications currently run in the public cloud compared with 40% of Aspiring organizations and 37% of Nascent organizations. Organizations that are best at leveraging performance management tools appear to be most able to embrace transformative application consumption models, likely in part because they are better able to monitor the performance of applications regardless of where they run.

7 White Paper: How Performance Management Maturity Drives Business Agility and Innovation 7 IT s Standing as a Competitive Differentiator for the Business As evidenced by ESG s research data, there are many ways in which being an Expert performance manager is correlated to positive outcomes and behaviors, but what is the impact for IT? Expert performance managers were much more likely to report that their IT group was viewed as a competitive differentiator than their Aspiring or Nascent counterparts (see Figure 3). ESG posits that by providing more stable, resilient, and continuously improved services for end-users, Expert organizations are much more likely to be highly valued by the business and making significant contributions to the business s success. ESG s research also found that Expert organizations are three to five times more likely than Aspiring and Nascent organizations to report a highly cooperative work environment with different roles and functional groups when dealing with application performance problems. This cooperative environment likely also contributes to how the business characterizes IT. Figure 3. LOB Perception of IT Frequency with which respondents believe their line-of-business stakeholders view the IT organization as a competitive differentiatior. (Percent of respondents) 41% Line-of-business stakeholders at Expert organizations view IT as a competitive differentiator at nearly 7x the frequency of Nascent organizations and nearly 4x the frequency of Aspiring organizations. 11% 6% Expert (N=32) Aspiring (N=217) Nascent (N=96) The rare Expert organization is clearly the example for Aspiring and Nascent organizations becoming an essential part of the business and a clear, positive driver of business value is the state which all IT organizations strive to achieve. Behaviors That Organizations Should Adopt to Become Experts in Performance Management Expert organizations are rare based on ESG s research. Just 9% of the 345 respondents participating in the research were rated with that level of maturity. However, there are several behaviors these organizations consistently share, which could be beneficial for Aspiring and Nascent organizations as they seek to improve their standing. Wider Breadth of Performance Management Capabilities While all organizations have some degree of performance management tooling, Expert organizations reported the broadest. For example, the percentage of Expert organizations that reported they are able to monitor applications and infrastructure compares favorably with Nascent organizations regardless of modality: in the data center (91% vs. 46%), in the cloud (59% vs. 14%), or at a remote office (59% vs. 45%). Additionally, Expert organizations were the most likely to report performance management capabilities at each layer of the application delivery architecture (see Figure 4).

8 White Paper: How Performance Management Maturity Drives Business Agility and Innovation 8 Figure 4. Performance Management Capabilities Which of the following capabilities are included within your organization s performance management environment? (Percent of respondents, multiple responses accepted) Database performance monitoring 40% 49% 88% Infrastructure monitoring (SNMP, WMI, etc.) 32% 46% 75% NPM (network performance monitoring) 22% 44% 63% Log file analytics 33% 35% 47% Nascent (N=96) APM (application performance monitoring) SIEM (security incident and event management) 24% 36% 19% 34% 59% 56% Aspiring (N=217) Expert (N=32) End-user experience (EUX) monitoring 18% 34% 63% Unified communications (UC) monitoring 16% 23% 56% More Frequent Evaluation of and Investment in Performance Management In addition to reporting more current performance monitoring capabilities, Expert organizations appear to be giving performance management more priority when looking ahead. Nearly seven out of ten Expert organizations report they evaluate their performance management tools and processes multiple times per year. By contrast, just 25% of Aspiring organizations and 10% of Nascent organizations report this same level of continuous focus. Not only do Expert organizations evaluate their performance management capabilities more often, but these organizations are also more likely than their counterparts to have earmarked financial capital to the cause. As shown in Figure 5, Expert organizations report the highest frequency of planned investment over the next 24 months in each of the areas ESG included in its research. From a performance management product perspective, more than four out of five Expert respondents expect their company to invest in upgrades/enhancements of tools and nearly three out of five expect entirely new tools to be added to the environment.

9 White Paper: How Performance Management Maturity Drives Business Agility and Innovation 9 Figure 5. Performance Management Investments With respect to performance management, which of the following investments do you expect your organization to make over the next 24 months? (Percent of respondents, multiple responses accepted) Upgrades or enhancements of existing tools 32% 50% 84% Improvements in processes and best practices 31% 45% 75% Purchase new tools 24% 41% 59% Nascent (N=96) Increased training 32% 39% 56% Aspiring (N=217) Increased staffing 25% 30% 38% Expert (N=32) Use outside consulting services 19% 26% 28% None of the above 5% 1% 0% The Bigger Truth What ultimately matters to the business is that the applications employees need to do their jobs and that customers need to engage with the company are up, performing, and reliable. One would expect that organizations with the tools, processes, and people in place to effectively manage application performance would be best positioned to ensure desired business outcomes are achieved. This expectation is soundly validated by ESG s quantitative research. Organizations needing to improve the resiliency, performance, or availability of applications would be well served to study the habits and characteristics of performance management Experts and affect change where possible to align with those principles. Assess Your Performance Management Maturity Today To enable greater performance management maturity, you must first understand where you stand today. Riverbed and ESG have made available an interactive online assessment tool based on this research. It will allow you to see where you stand in relation to your peers and help you understand your strengths and weaknesses. Start the journey toward transforming your IT organization into a competitive differentiator today. Launch Assessment Now!

10 About Riverbed Riverbed, at more than $1 billion in annual revenue, is the leader in Application Performance Infrastructure, delivering the most complete platform for the hybrid enterprise to ensure applications perform as expected, data is always available when needed, and performance issues can be proactively detected and resolved before impacting business performance. Riverbed enables hybrid enterprises to transform application performance into a competitive advantage by maximizing employee productivity and leveraging IT to create new forms of operational agility. Riverbed's 27,000+ customers include 97% of the Fortune 100 and 98% of the Forbes Global 100. Learn more at Riverbed and any Riverbed product or service name or logo used herein are trademarks of Riverbed Technology, Inc. All other trademarks used herein belong to their respective owners. All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of The Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at Enterprise Strategy Group is an IT analyst, research, validation, and strategy firm that provides actionable insight and intelligence to the global IT community by The Enterprise contact@esg-global.com Strategy Group, Inc. All Rights Reserved. P