White Paper Application Development, Test & Delivery Hybrid SAP Applications with Modern Digital Architectures Require a New Test Strategy
Table of Contents page Executive Summary...1 From the SAP Monolith to the Hybrid IT Landscape...2 Five Basic Requirements for Testing SAP Processes...2 Three Crucial Testing Principles...5 Conclusion: Customize Your Testing Strategy... 6 Key Questions to Ask...6 Example Scenarios...8 Automated Testing of Hybrid SAP Landscapes...9 Testing Mobile Applications and SAP Fiori On All Devices In Parallel...9 Further Information... 10
In practice, the time and resource spend on SAP testing can be significantly reduced by implementing Micro Focus solutions. Users of other test automation solutions can also benefit by integrate their existing test scripts with Micro Focus s test and quality management solutions, which offer a rich set of additional functionality. Executive Summary Project leaders often say that the most important lessons they ve learned from developing and integrating SAP systems are: they underestimated the effort they needed to invest in testing; and tests take too long to run. There are a number of closely related reasons for this, and they combine to make testing using conventional methods more challenging. Historically, SAP was typically the only system involved. Over the years, system landscapes have become more complex. Today, systems might have some components on the cloud, with other components onpremises, as well as integrations with numerous in-house and external systems. Constant updates and security patches require continual testing, which has become a critical and endlessly recurring expense. What has been missing is the technological support that complex testing requires, as well as knowledgeable staff to develop and run the tests to ensure that everything still works as it should after a routine update. Manual testing simply cannot check existing processes and new functionality in modern systems with sufficient depth and breadth. The increase in digitization requires automated end-to-end tests in hybrid landscapes. The solution is to implement testing environments that map the complexity of the system under test as accurately as possible, in order to provide maximum test coverage. This includes testing of interfaces and web services, as well as integrations between systems, and simulation, or virtualization, of services which are only available in production environments. Service virtualization has led to massive productivity gains in testing environments: Cloud-based systems can be reproduced today within minutes, at the push of a button. In the past, it took hours, days, or even longer to make this happen. A modern testing strategy must include all of these capabilities. In addition, cloud solution providers often update their services quarterly for example, SAP S/4HANA Cloud or even monthly. In multi-cloud environments, every change must be checked to ensure that all interfaces continue to work, and that all processes and devices continue to be secure. In practice, the time and resource spend on SAP testing can be significantly reduced by implementing Micro Focus solutions. Users of other test automation solutions can also benefit by integrating their existing test scripts with Micro Focus s test and quality management solutions, which offer a rich set of additional functionality. www.microfocus.com 1
White Paper Hybrid SAP Applications with Modern Digital Architectures Require a New Test Strategy From the SAP Monolith to the Hybrid IT Landscape A few years ago, things were very different. Core processes were centralized within the SAP system, releases were yearly, and it was possible to plan and optimize for long-term projects. Today, agile projects are the norm, with faster innovation cycles and continuous deployment of changes to the IT landscape. Cloud solutions are regularly updated by providers, and consumers of their services are forced to use the updated version, whether they want to or not. SAP systems no longer exist independently. They are enriched and complemented by other SAP cloudbased systems such as SuccessFactors, Hybris, or Ariba, as well as other providers systems, such as Salesforce or Workday. During their life cycle, business processes often pass through several internal IT systems and go out to external systems, such as payment systems, web stores, logistics service providers, and supplier systems. Hybrid landscapes will become even more common, and will be standard in the future. Today, agile projects are the norm, with faster innovation cycles and continuous deployment of changes to the IT landscape. Cloud solutions are regularly updated by providers, and consumers of their services are forced to use the updated version, whether they want to or not. User interfaces have become massively diversified as well. In the past, SAP systems used the SAP GUI. But today, numerous devices, such as smartphones, tablets, desktops, multifunction devices, each with different operating systems (e.g., ios, Android), platforms (e.g., SAP FIORI, UI5), and browsers (e.g., Internet Explorer, Edge, Chrome, Firefox, Safari), are in use. This increased complexity and heterogeneity has had a direct impact on test management. Previously, SAP consumers might limit themselves to manual functional and integration tests, with some automated load tests. Today, consumers must implement comprehensive test management of the entire system landscape, including SAP systems, non-sap systems, and cloud-based systems. As well as being important to test on a system that accurately resembles the production environment, it is necessary to be transparent to the team, and to management. For example, what was already tested? What did the tester do before the error occurred? Which defects were resolved? Are there any restrictions or show stoppers? Can we push to production? Five Basic Requirements for Testing SAP Processes Here are five basic requirements for a testing solution that apply to all systems in general, and to SAP systems in particular: 2
SAP systems often have several interfaces to other systems sometimes even hundreds of integrations. If you need to test a system where SAP ERP or S/4HANA, a Web store, and Salesforce are all integrated and working with each other, you must be able to represent the data and the dependencies between the different components accurately and consistently. A modern testing environment must be able to support this type of test data. 1. Teams Simply Want to Be Able to Test Project teams often have only a limited amount of time available for function tests. This means that automated testing must be employed to test the functionality, as well as the performance and security. Instead of manually clicking through the user interface, managers are looking to technology to help their teams assess whether or not a process is functioning correctly. Modern test tools are able to record the user s actions while they conduct exploratory testing of new functionality, and can create automated tests based on those actions. These tests can then be run automatically in succession whenever the functionality has to be tested. In addition, each test can be run with different test data values. Virtualizing the test data allows it to be used multiple times, resulting in stable, repeatable and regression-safe testing. Consequently, knowledge of the business process is preserved, time is saved, and quality is increased. 2. Test Environments Must Accurately Represent Reality SAP systems often have several interfaces to other systems sometimes even hundreds of integrations. If you need to test a system where SAP ERP or S/4HANA, a Web store, and Salesforce are all integrated and working with each other, you must be able to represent the data and the dependencies between the different components accurately and consistently. A modern testing environment must be able to support this type of test data. SAP systems usually use BAPI or RFC calls for communication. Modern test solutions are able to capture a request from a consumer to a provider, analyze the response from the provider, and simulate or virtualize the interface between them for future tests. In this case, the provider is no longer needed for testing, and the requests and responses between them are simulated. If cloud-based services are part of the system to be tested, virtualization is often a requirement. If you send several million requests to the service provider for load tests, the provider is likely to block access, or may even bill you per request, which can become very expensive. It is much more efficient and cost-effective to simulate requests and responses to and from the cloud-based service. You must also be able to virtualize web services or other services for tests. For example, credit card payments, credit checks or logistics systems can be simulated, with consistent results returned for customer data records. Virtualization also plays a big role on the client s side, because of the variety of devices, operating systems, language settings, UI, and, browsers. There are so many combinations that must be tested, and it is not realistic to test everything on real devices. Time constraints often mean that tests must be run concurrently, rather than one-by-one. www.microfocus.com 3
White Paper Hybrid SAP Applications with Modern Digital Architectures Require a New Test Strategy 3. Developers Need Detailed Error Descriptions Bugs can be difficult to find. They might only appear in certain configurations, making them hard for developers to duplicate, without a lot of effort and time-consuming research. A major time-saver during the root-cause analysis and troubleshooting phase is the ability to automatically generate a description of a problem. If a tester s error report includes automatically generated screenshots, log files and time stamps, in addition to the tester s own notes, problems and root causes can often be identified and corrected faster. Often, reporting an error to a service provider can result in time-consuming ping-pong between the provider and the consumer who reported the issue. When an automatically generated error report which includes all of the relevant information is submitted, that ping-pong can be avoided, as the provider has everything they need to reproduce and resolve the issue. 4. Project Managers Want More Visibility into Test Results When a change is made to the IT landscape, the management team has to evaluate whether the software is ready for production. A wrong decision can lead to defects and issues which are frustrating for users, as well as expensive to fix. Today, the IT backbone is usually much more than just the SAP system. Solutions involve multiple technologies from a variety of service and system providers. Testing the readiness of an enhancement or a new feature can t be done without crossing system boundaries. And a meaningful measurement of end-to-end quality requires a consistent evaluation of test results across SAP and non-sap systems. Today, the IT backbone is usually much more than just the SAP system. Solutions involve multiple technologies from a variety of service and system providers. Testing the readiness of an enhancement or a new feature can t be done without crossing system boundaries. And a meaningful measurement of end-to-end quality requires a consistent evaluation of test results across SAP and non-sap systems. Ideally, you should be able to map your tests to your business requirements, for example, through SAP Solution Manager or another Application Lifecycle Management (ALM) tool, and then test against those requirements. This becomes even more necessary with S/4HANA, and affords many additional benefits to IT managers. The test solution should offer meaningful reporting, and easy-to-understand visualizations. Decisionmakers need visibility into the system both in terms of breadth (i.e., what has already been tested), and depth (i.e., what errors have been found, and fixed). 5. Agility Requires Readily Available and Stable Test Environments It should be possible to spin up a test system at the push of a button, and quickly conduct short load tests on the system without needing to maintain a huge testing infrastructure. Combining testing using internal systems with external systems from different world regions, and over wireless networks (WLAN), is no longer a dream. Virtualization creates endless possibilities and unprecedented flexibility. Today you can test in the browser and from the cloud, and access virtual test systems. 4
Development and test environments should be flexible and easily configurable. The cost to create a robust test environment can be very low due to virtualization and automated provisioning, and the environment can be accurately reproduced at any time. This is essential to achieving shorter development cycles, continuous delivery, and continuous deployment. You can also save on costs, because you don t need to maintain test environments when they re not being used; they can be generated on-demand in the cloud when required and you pay for what you use. Development and test environments should be flexible and easily configurable. The cost to create a robust test environment can be very low due to virtualization and automated provisioning, and the environment can be accurately reproduced at any time. This is essential to achieving shorter development cycles, continuous delivery, and continuous deployment. Three Crucial Testing Principles End-to-end system testing involves functional testing, performance testing and security testing. Ensure Functionality Functional tests should be automated. Today s solutions allow test scripts to be easily created by recording. The test can then be run automatically on multiple different environments (operating systems, devices, browsers, screen resolutions). As new functionality and updates are tested manually, the tests can be recorded to create automated, documented test cases. The library of automated test cases grows with each manual test. Guarantee Performance Performance testing can be made significantly easier with virtualization: load generators can be hosted by cloud service providers (e.g., AWS and AZURE) for on-demand load testing, and on-premises load generators can be installed and configured for internal applications (e.g., SAP GUI). Load tests can be easily scaled. Simply enter the desired number of simulated users, and the load will be created on the system. It s also possible to dynamically increase (ramp-up) or decrease (ramp-down) virtual users, define whereabouts in the world the load comes from, and select the network (WLAN, Wifi, mobile, etc.) and network conditions (connection quality, carrier, time of day, etc.) for the traffic. As you generate load, you ll need to monitor the backend s behavior. You should set the service levels (maximum response times), visualize, filter and analyze the response times, and create a test report with the results of the test. Ensure Security Safety issues are of great importance in the age of digital systems, particularly as they cross system boundaries. Your testing should include static code analysis to identify potential vulnerabilities in the source code, as well as dynamic code analysis, testing of known hacks, SQL injection, and other methods of attack. www.microfocus.com 5
White Paper Hybrid SAP Applications with Modern Digital Architectures Require a New Test Strategy Conclusion: Customize Your Testing Strategy In hybrid IT landscapes, with on-premises and/or cloud SAP solutions, mobile applications and integrations with third-party cloud solutions, you need to adopt a new approach to testing. An effective test strategy must take into account the various devices that will be used to access the system, and the interfaces to its subsystems and the external systems it accesses. Previous testing strategies of periodic bursts of intensive testing are no longer acceptable. Testing today is an ever-present, continuous activity. The trend towards multi-cloud environments involving multiple vendors is likely to dictate changes to the system landscape from the outside, and will require IT departments to act at very short notice. If you want to be able to continuously check all of the interfaces, and understand how changes affect your business processes, you will need very powerful test solutions. An effective test strategy must take into account the various devices that will be used to access the system, and the interfaces to its subsystems and the external systems it accesses. Previous testing strategies of periodic bursts of intensive testing are no longer acceptable. Testing today is an ever-present, continuous activity. Key Questions to Ask Test automation has been around for more than 20 years, and is used by many organizations. But with so many changes in the IT landscape in general, and the SAP landscape in particular, companies are reaching the limits of their current solutions. Here are some questions that can help you decide whether you should consider upgrading your current testing solutions: Migration Assistance and Automation How easy is it to convert exploratory tests into automated tests? How easy is it to extend your existing tests to the entire system landscape? Can you use your test scripts for additional testing purposes (e.g., for load testing and monitoring)? Testing on Mobile Devices How easily can you test on a variety of real or emulated mobile devices, interfaces, browsers and terminals? How easy is it to connect your physical devices (e.g., via USB interface) for tests? Are you able to lease mobile devices from the cloud and connect on an hourly basis, so that you don t need to maintain a stock of physical devices? Enabling Continuous Delivery by Shortening Test Cycles Can you provision your development and test environments quickly and automatically? To what extent can you run tests in parallel, to reduce the testing cycle time? 6
Test automation has been around for more than 20 years, and is used by many organizations. But with so many changes in the IT landscape in general, and the SAP landscape in particular, companies are reaching the limits of their current solutions. Can you convert manual tests into automated tests, and integrate them into your automated test cycle? Are you able to consistently create tests, run them, and reproduce test results? Can you test customizations of the SAP Cloud Platform during development? Do you use test-driven development and DevOps methodologies? Management of Hybrid Landscapes How broad is your technology coverage? Does your toolset support all of the technologies that you need? Do your tools integrate with a higher-level lifecycle management platform, e.g., ALM Octane or SAP Solution Manager? Are you able to you record data that is sent through interfaces (APIs) and third-party systems, and use it for testing? Are you able to collect data from the GUI and from mobile apps, and use it for testing? Collaboration Are you using traditional, or agile development methodologies, or a hybrid of both? Traditional: Are you able to use the project s requirements for testing? Agile: Are you able to map tests to your user stories, tasks and backlog items? Are your dashboards simple and easy to read? Can your distributed teams see your release and defect dashboards? Evaluation and Analysis How detailed are your problem descriptions? Do they help you in root-cause analysis? Are screenshots, object descriptions, and time-stamped log files included? Are you able to easily duplicate tests on other browsers or platforms? Can you perform and report on higher-level, cross-team tests, based on corporate objectives (e.g., at the Epic level) Enterprise Standardization Are standards consistently applied across the organization, so that teams use similar quality metrics and indicators? Do you have standards for roles, workflows, and reporting? Can your testing framework be used by several teams or departments possibly even from distributed locations? Are testing assets shared or reused across the organization? Is it possible to see how assets are being used by all projects? www.microfocus.com 7
White Paper Hybrid SAP Applications with Modern Digital Architectures Require a New Test Strategy Example Scenarios Automated Testing Factory: Cloud-Based Test Automation for Cloud Solutions SAP SuccessFactors is released on average four times a year. Each new release has to be tested to ensure that it s working correctly in the customer environment. However, the testing window is usually very limited, so it is increasingly important to test efficiently. SAP SuccessFactors is released on average four times a year. Each new release has to be tested to ensure that it s working correctly in the customer environment. For organizations that use SAP, the following observations can be made: Manual testing of cloud-based applications is outdated; The depth and accuracy of the testing depends on the employees, and on how much risk the company is prepared to take; Valuable team members are assigned to repetitive test activities several times a year; Employee motivation is eroded from release to release. PwC worked with their customers to develop automated test scenarios for SuccessFactors: Manual testing is progressively reduced, saving time and lowering costs. Verifiable documentation: The tool records each test, and creates high-quality test documentation at the click of a mouse. Test Case Library: PwC have used their experience and technological know-how to put together a pre-defined set of SuccessFactor test cases, which are regularly maintained. Test on-demand: The tests can be run any time and from anywhere. Figure 1. Increase in test automation over time 8
Real-world testing systems can be made constantly available, and highly scalable, for example, for load tests that need to simulate load from a large number of users around the globe. Automation also reduces the number of people required to set up test environments and create and run tests. Benefits: Increase test quality and efficiency Reduce complexity and increase reusability of test cases and assets Save costs and resources after initial investment Less stress for engineers and management Automated Testing of Hybrid SAP Landscapes Automated testing is a proven technique, and is particularly suitable for hybrid landscapes, including SAP systems, that are accessed from a wide variety of devices. Real-world testing systems can be made constantly available, and highly scalable, for example, for load tests that need to simulate load from a large number of users around the globe. Automation also reduces the number of people required to set up test environments and create and run tests. Testing Mobile Applications and SAP Fiori on All Devices In Parallel For mobile applications, the variety of devices that will be supported must also be taken into account. Practically speaking, device emulators can only give an initial indication of an app s quality. The only way to determine whether an app works correctly on actual devices is to install and test the app on real devices. This is true for native mobile apps as well as to browser/web applications that are optimized for mobile use. But even in the latter case, the application must be tested on different browsers and resolutions, otherwise it s not possible to accurately predict how the application will behave. Because SAP departments don t usually have a large number and variety of current mobile devices available, tests must be run on a combination of local physical mobile devices (e.g., connected by a USB cable to the test platform) and leased devices (Mobile Device Cloud). www.microfocus.com 9
Contact us at: www.microfocus.com Today s mobile and web environment requires a new testing strategy. This strategy includes cloud-based solutions, and enables testers to create and run tests more quickly and easily on different operating systems, browsers, screen resolutions, and on locally hosted, remote, and emulated mobile devices. Exploratory testing is also automated as much as possible, with these automated tests running in parallel, so as not to waste any time when tests have to be carried out on a large number of different end devices, browsers and operating systems. Further Information Application Delivery Management Solutions from Micro Focus Micro Focus Functional Testing Solutions Micro Focus Performance Testing Solutions 162-000150-001 M 05/18 2018 Micro Focus. All rights reserved. Micro Focus and the Micro Focus logo, among others, are trademarks or registered trademarks of Micro Focus or its subsidiaries or affiliated companies in the United Kingdom, United States and other countries. All other marks are the property of their respective owners.