Unifying End-User, Network, and Application Performance Monitoring and Management

Transcription

1 Unifying End-User, Network, and Application Performance Monitoring and Management Ensuring Successful Digital Transformation Through Unified Digital Management An ENTERPRISE MANAGEMENT ASSOCIATES (EMA ) White Paper Written by Stephen D. Hendrick and Prepared for Riverbed June 2018 IT & DATA MANAGEMENT RESEARCH, INDUSTRY ANALYSIS & CONSULTING

2 Table of Contents Introduction...1 Why Unified Digital Management?...1 APM Limitations...1 EUEM Limitations...2 NPM Limitations...2 Characteristics of the Unified Digital Management Platform...3 Objectives of a Unified Digital Management Platform...3 Capabilities of a Unified Digital Management Platform...3 Enterprise Requirements for Unified Digital Management...5 Conclusions, Recommendations, and Next Steps...6

3 Introduction The digital economy is the new economy. It represents more than 25 percent of the world s economy and continues to grow. 1 Web services, cloud services, big data/analytics, smartphones, and social media are not new by today s standard, but are the foundation of the new digital economy. Because digital transformation democratizes digital service delivery, every enterprise must ensure that their applications are operationally performant. Competing in this digital economy requires a digital strategy and the willingness to reengineer how an enterprise creates and delivers value. Digital products and services must be carefully aligned and continuously updated to customer needs and expectations. Differentiation is the primary driver of customer loyalty. For example, in the financial services industry, 29 percent of mobile customers said they would immediately switch to another provider if their current service provider could not address their needs. 2 Addressing customer needs in a digital economy requires a digital strategy that delivers digital products and services, but delivering a portfolio of digital products and services is not enough. Brand loyalty is challenging in a digital economy. Competing services are just a few clicks away. A successful portfolio of digital products and services must continuously address customer needs on a functional level, while delivering a performant digital experience from the perspective of the user experience and application performance. Managing the customer s digital experience is a mission-critical activity. Customers expect digital production experiences to be always available, accessible across devices, and able to meet or exceed their performance expectations. The end-user experience subsumes application performance, but the only way to ensure a satisfactory user experience is to carefully monitor the performance of the application from both the user device and application server perspectives. These same tenets must also be applied to projects in the pipeline to validate that every release is available, accessible, and operationally performant before it reaches production. End-user experience monitoring (EUEM) and application performance management (APM) are two sides of the same coin. EUEM looks at application performance from the perspective of the customer s device and is an effective technique for evaluating business impact. APM looks at the application hosting environment and is effective at evaluating IT performance. Each approach has its strengths, but the only way to establish a complete view of digital performance is through a unified EUEM and APM solution. Network performance monitoring (NPM) approaches are effective at evaluating network performance, also a critically important driver of application performance in the eyes of the end-user. EMA believes that unified EUEM, APM, and NPM solutions represent an evolving new market segment defined as unified digital management (UDM). Why Unified Digital Management? APM, EUEM, and NPM products each address operational performance, but have fundamentally different design points. This similar while different approach is confusing and complicated. Most vendors upgraded their APM and/or EUEM solutions so they each overlap with each other to some extent. While these improvements to APM and EUEM addressed incremental customer demands, APM, EUEM, and NPM do not independently provide a complete view into the operational performance of an application. APM Limitations APM products address operational performance across web-based or mobile environments where it is possible to install agents on browsers and servers. APM is highly effective at evaluating the activities of IT data center systems. This means that APM products provide immense insight into server-side operations, but limited analysis of the user s experience when using SaaS or thick-client applications. The primary focus of APM is on system health, system and application performance, 1 Accenture and Oxford Economics, 2016 World Economic Forum 2 CSI, The Changing Landscape of Digital Banking Page 1

4 scalability, QoS, and efficiency. The challenge this creates is a last mile problem, which is an understanding of application performance from the user s perspective. Browser-based JavaScript injection and synthetic monitoring are techniques that APM vendors use to gain better insight from a user perspective, but JavaScript injection applies to web-based applications and only collects data from the browser without context of the underlying system metrics and processes. Synthetic monitoring doesn t measure the end-user experience; it simply emulates it. APM is an effective solution to operational performance for applications from an IT admin perspective, but is not intended to monitor applications from the end-user perspective. EUEM Limitations EUEM products grew out of the need for a modern approach to understand the digital experience from the perspective of the user. EUEM products typically support a wide range of devices and application environments including thick client, web-based, Rich Internet, Java,.NET, and applications running on any mobile devices. The value of EUEM is that it monitors the user experience across geographies, applications, and devices. While EUEM products are easy to install, don t require configuration, and excel at monitoring the real user experience, these are also limitations of EUEM products. EUEM products can point to performance outside established service-level agreements, but can t pinpoint a root cause. Because they re deployed on the end user s device, EUEM products don t provide the deep insight into the network and application stack that NPM and APM products deliver. Consequently, while EUEM products provide invaluable insight on the digital user experience, this data is most useful when analyzed in combination and context with IT-focused operational performance data. EUEM providers have made efforts to mitigate these shortcomings by including synthetic monitoring, but this technique simply emulates and approximates the true end-user experience. EUEM is an effective solution to operational performance for applications from a user perspective across many types of applications, devices, and communication protocols. However, EUEM products lack the deep insight APM and NPM products provide. EUEM is an important component of a digital management solution and provides a complete solution when used in conjunction with APM and NPM products. NPM Limitations Approaches to network monitoring share many of the same principles that drive APM and EUEM products. These principles include polling-based approaches like the synthetic transaction methods relied on by EUEM, as well as agent-based deep packet monitoring and analysis similar to how many APM products operate. Recent EMA empirical research of NPM users 3 shows that respondents would like to see NPM products add EUEM and APM insights. This suggests that NPM products are challenged to deliver user- and application-centric performance metrics. This also suggests that NPM users are looking for a more complete and comprehensive performance management product in a single product form factor. These results confirm the belief that EUEM, APM, and NPM products each provide important insight into aspects of application performance, but a complete and comprehensive understanding of application performance can only be achieved by using all three products or a unified platform that brings these three capabilities together. 3 Network Management Megatrends 2018: Exploring NetSecOps Convergence, Network Automation, and C loud Networking (April 2018, Shamus McGillicuddy) Page 2

5 Characteristics of the Unified Digital Management Platform Given the limitations in scope of using a EUEM, APM, or NPM product independently, the solution would be to use these products together. However, this raises likely integration and scalability challenges that would be difficult and unreasonable for enterprises to address. Therefore, the best approach would be to select a vendor that provides either a unified EUEM, APM, NPM solution, or UDM platform. Based on customer interviews and primary research into the EUEM, APM, and NPM markets, it is possible to define the objectives and capabilities of a unified digital platform. Objectives of a Unified Digital Management Platform Monitor every application. In a digital economy, there are two broad categories of applications: informational and transactional. Information is the basis for transactions, which means that all application types are important. IT s focus on efficiency results in resource sharing. Monitoring every application protects against the impact of noisy neighbors and enables IT to have a complete understanding of workloads and system utilization. Monitor every transaction. Synthetic and time-based monitoring do provide a way to emulate application performance. However, when applications are operating at scale, generating hundreds or even thousands of transactions per second, synthetic or time-based monitoring is no longer effective. Monitoring every transaction enables outliers to be identified and corner cases resolved. There is no substitute for a complete understanding of application performance. Monitor continuously. Because a digital economy never sleeps, continuous real-time monitoring 24x7x365 is a core requirement. This way, companies can optimize a strategy for resource utilization based on all online and batch processing workloads. Monitor any device. This means providing support for any server, user device, or connected device physical or virtual. Practically, this means prioritizing support for any device OS that is in widespread use or is of strategic importance to a customer. Monitor any environment. Deep data center and cloud service provider (CSPs) support at an infrastructural and network level is critical to ensure a complete view into application performance. This ensures coverage for traditional and SaaS applications. As IoT matures, this list will also include gateways. Monitor/capture all relevant transaction data. Data captured must include device KPIs and metadata for each transaction as it traverses devices from source to target across the network. Capabilities of a Unified Digital Management Platform Provide a complete digital performance management solution. EUEM (user-focused), APM (server-focused), and NPM (network-focused) each provide valuable insight into an application s performance, but must be brought together to provide a complete understanding of how an application is performing. A key aspect of delivering a complete solution is to meet the objectives defined, and doing so at scale. Intelligent data collection. Intelligent data collection begins with real-time continuous data collection of all transactions and transaction KPIs across devices and in the call stack, with minimal custom configuration. This should also include intelligent pruning of data that automatically excludes simplistic methods that have a low probability of driving errors. Page 3

6 Intelligent data management. The high volume and detail of data collected at scale requires the use of techniques specifically designed for high throughput and availability. These responsibilities are divided between the UDM platform and its agents. Device agents are responsible for intelligent data collection and data compression prior to shipment to the platform. The UDM platform is responsible for a clustered approach to data capture, management, and access, as well as key value pair-based storage for efficient access and queries. This clustered approach enables a UDM platform to collect and leverage data at scale. Deliver UDM as a common architectural platform. A UDM platform is not just a simple aggregation or union of APM and EUEM products. It is a ground-up effort to provide extensive APM and EUEM capabilities using a single common architectural model. This approach ensures ease of implementation, use, and management because of its single architecture, consistent user interface and access methods, and declarative policy-driven interface for configuration. Polyglot deployment model. UDM products should be available as dedicated solutions that can be implemented privately either on-premises or through any leading CPS. Alternatively, UDC products should also be available as multitenant SaaS solutions publicly hosted on the leading CSPs. Single scalable license model. UDM licensing should encourage use across enterprises of all sizes and offer economies of scale. Careful attention to pricing is necessary to ensure that small and medium enterprises are incented to become customers, while large enterprises are incented to scale out their use. Deep analytical capabilities. The value that a UDM product delivers is primarily tied to the operational performance insights it delivers. Analytics capabilities begin with providing a wide variety of reporting methods to segment, summarize, and visualize UDM data. More advanced capabilities based on machine learning should include auto baselining, auto thresholding, auto correlation, pattern detection, and alerting. The market for applied advanced analytics is rapidly maturing. For this reason, more analytic techniques involving predictive analytics, optimization, and adaptive systems will come in the coming years. Integration with ITSM and DevOps tool chains. APM, EUEM, and NPM products reside at the intersection of development and operations. APM, EUEM, and NPM products have historically been aligned with operations. While this represents an appropriate starting point, to be consistent with agile methods and DevOps experience, the use of UDM products must shift left. Focusing UDM products on development and continuing their use into production will accelerate application quality and time to market, as well as reduce risk and cost across the entire software development lifecycle. The implications of such a strategy are that UDM tools must be integrated with the leading DevOps and ITSM tools. This integration ensures high levels of process automation across the lifecycle and facilitates the easy adoption of UDM tools by developers and IT operations staff. Page 4

7 Enterprise Requirements for Unified Digital Management The prior discussion of UDM objectives and capabilities was based on a combination of vendor activities and customer requirements. The following recent empirical research by EMA into large enterprises corroborates these UDM objectives and capabilities. EMA performed primary research in the fall of to better understand enterprise perspectives on how to manage their digital experience. The following data is from that study. Figure 1 looks at how enterprises define UDM. The variance in defining UDM is closely related to the alignment of EUEM to business impact and APM to application performance. Complete the following statement: "In our enterprise, Unified Digital Management is " A business concern 20% A technology concern 21% Equally a business and technology concern 59% Because UDM brings EUEM, APM, and NPM together, UDM should be about both business and technology. However, at the time of this survey, there were no UDM products available. The frame of reference for enterprises was APM, EUEM, and NPM, and only those enterprises with experience or understanding all three products would likely have responded as part of the 59 percent that said UDM was both a business and technology concern. The remaining 41 percent that was equally split between defining UDM as either a business or technology concern were likely influenced by their use of either an APM, EUEM, or NPM product. 4 Best Practices for Successful Digital Experience Management (July 2017, Dennis Drogseth) Page 5

8 Figure 2 shows enterprise perspectives on the top drivers of digital application management. While this is not a term introduced yet, it was used in this question as a proxy for the combination of APM, EUEM, and NPM. What are the Top Drivers of Digital Application Management? IT Operational Performance 62% Application Performance Employee Productivity Business Competitiveness Business Process Optimization 40% 37% 34% 34% IT operational performance is the leading priority with 62 percent of respondents. Because the term IT Operational Performance is a more expansive term than just Application Performance, it aligns most closely to the management of the application across its entire lifecycle. Application performance at 40 percent is closely tied with employee productivity (37%), business competitiveness (34%), and business process optimization (34%). Figure 2 emphasizes the value that enterprises put on business impact and application performance. The stronger focus on application and IT management reflects the greater insight that today s tools provide into the application and serverside activities. Conclusions, Recommendations, and Next Steps Since the APM, EUEM, and NPM markets overlap, the roadmap to a unified digital management destination has already been identified. What is not known is which waypoints each vendor will define for this journey, or how long it will take. The positive news is that UDM solutions are now beginning to appear in the market. This means that a UDM solution as defined in this paper is close or already within the grasp of one or more vendors. Keep in mind that the following decision criteria depend upon your enterprise digital application management maturity: How significant is your digital/ebusiness portfolio today, and how will it change over the next two years? UDM objectives and capabilities represent a lofty yardstick as a point of comparison for your enterprise s current strategy. Incremental improvement can be acceptable in many situations. Risk and OpEx investment must be factored into all decision making. Strongly consider a shift-left strategy for your current and future digital application management activities if you are pursuing agile development and DevOps. Page 6

9 Managing the performance of digital assets is important to any enterprise that drives a significant portion of their revenue or brand identity through digital channels. In today s digital economy, this means just about every enterprise. For enterprises invested in digital channels and digital application management, there are three starting points: 1. APM, EUEM, NPM, or UDM products are not used 2. A subset of APM, EUEM, and NPM products is used 3. APM, EUEM, and NPM products are all used For enterprises that do not use any APM, EUEM, or NPM products, consider the risk that not understanding application performance represents to your ebusiness revenue stream. Any APM, EUEM, or NPM solution is considerably better than no solution. That said, your choice of APM, EUEM, or NPM probably rests with whoever is responsible for supporting your application portfolio. If most operational support is outsourced, then a EUEM solution is a great starting point. However, if most of your portfolio is internally developed, then APM and/or NPM will be highly effective at identifying and resolving operational problems. For enterprises that already use a subset of EUEM, APM, and NPM products, the choice of what to do next is more complex. Until now, the incremental approach would be to adopt an additional tool or tools to give you a (more) complete view of application performance. Sourcing other tools from the same vendor that provides your existing tool would likely provide a higher likelihood of an integrated solution. However, the advent of UDM solutions becomes the new yardstick for evaluating what your current vendor can provide, and may suggest a more extensive review of what is available in the market. Finally, for enterprises who use APM, EUEM, and NPM products, your next step is quite simple. Determine the roadmap that your existing vendor(s) is/are on and evaluate that alignment with these UDM objectives and capabilities and the vendors that provide a UDM solution. Page 7

10 About Enterprise Management Associates, Inc. Founded in 1996, Enterprise Management Associates (EMA) is a leading industry analyst firm that provides deep insight across the full spectrum of IT and data management technologies. EMA analysts leverage a unique combination of practical experience, insight into industry best practices, and in-depth knowledge of current and planned vendor solutions to help EMA s clients achieve their goals. Learn more about EMA research, analysis, and consulting services for enterprise line of business users, IT professionals, and IT vendors at or blog.enterprisemanagement.com. You can also follow EMA on Twitter, Facebook, or LinkedIn. This report in whole or in part may not be duplicated, reproduced, stored in a retrieval system or retransmitted without prior written permission of Enterprise Management Associates, Inc. All opinions and estimates herein constitute our judgement as of this date and are subject to change without notice. Product names mentioned herein may be trademarks and/or registered trademarks of their respective companies. EMA and Enterprise Management Associates are trademarks of Enterprise Management Associates, Inc. in the United States and other countries Enterprise Management Associates, Inc. All Rights Reserved. EMA, ENTERPRISE MANAGEMENT ASSOCIATES, and the mobius symbol are registered trademarks or common-law trademarks of Enterprise Management Associates, Inc. Corporate Headquarters: 1995 North 57th Court, Suite 120 Boulder, CO Phone: Fax: