Take Your UAT from the Stone Age to the Digital Age

Take Your UAT from the Stone Age to the Digital Age

TABLE OF CONTENTS Boost your UAT with Panaya Click for Demo 3 Introduction 5 Chapter 1: How UAT Was Handled in the Stone Age 12 Chapter 2: UAT Best Practices and Tips for the Business Professional 15 Chapter 3: Migrating UAT from the Stone Age to the Digital Age 16 Chapter 4: How Panaya can help you boost your UAT cycles 2 Take Your UAT from the Stone Age to the Digital Age

INTRODUCTION Boost your UAT with Panaya Click for Demo In the past few years, following the pressure of digital transformation, business has been forced to support new business models. They now need to quickly introduce new offerings to the market and penetrate new markets. Digital transformation allows business to adhere to ever increasing customer demands. Unfortunately, IT didn t follow and is facing major challenges delivering value at the pace of business. The main bottleneck for this inability is UAT (User Acceptance Testing). User Acceptance Testing, or UAT, refers to testing a software product by the end-user to determine whether it meets the technical and business requirements, and whether it can be accepted or rejected. In simple terms: UAT verifies that the process does what the user expects it to do. In major business projects, failures in UAT can have dramatic impacts on the quality of the project, not to mention introduce serious risks in production. Developers or analysts are not business users. Nor are they mind readers. Even with detailed functional specifications documents, business users still need to make sure that their needs are met. UAT does not replace manual testing conducted by your quality assurance team or automated testing such as unit or integration testing. So why are UATs still managed like it is the Stone Age, even though this is the last quality gate before production? Currently, UAT is stuck with manual processes that belong to the stone age. While this was sufficient for small teams all working in the same location, or in waterfall project management systems where long waits between releases were expected, this is not sufficient for today s multinational, agile, and iterative release cycles. 3 Take Your UAT from the Stone Age to the Digital Age

The main challenges that are usually faced by IT organization to drive changes in UAT practice are the following: Boost your UAT with Panaya Click for Demo How to get business users to use tools, knowing that most test management solutions are designed for professional testers and not business users How to ease collaboration between users when some end-to-end scenarios may involve dozens of key users How to ensure high quality of test evidence without impacting user productivity This ebook will first focus on sharing the poor practices and induced pains. The second part will provide best practices to overcome them. Read on to discover how to take UAT out of the Stone Age. And no, you don t need a time machine! 4 Take Your UAT from the Stone Age to the Digital Age

CHAPTER 1 How UAT Was Handled in the Stone Age Here is how User Acceptance Testing was managed in the Stone Age: STONE AGE UAT STEP 1 KNOWLEDGE GATHERING FOR TEST PLAN CREATION Gather the key acceptance criteria: the things you are going to test. You can t test everything but you do want to make sure the important items are tested. Ideally, you should have the input of multiple teams and departments to make sure nothing gets left out. If something is missing, it s going to be difficult to test it later. A test plan for UAT usually contains the following: List of business processes to test Per business process: the sequence of actions (steps) to be tested at a high level For instance: Purchase to Pay Business process contains the following actions: Create Purchase Requisition Create Purchase Order Release Purchase Order Goods Movement Invoice Verification Per step: some guidelines regard which data to use, the expected result, and the team responsible to test it 5 Take Your UAT from the Stone Age to the Digital Age Throw Out Those Excel Sheets!

Gathering this information requires a lot of collaboration between the integration manager, the different functional leads, and the business process owners. In the Stone Age, a test lead or test coordinator would open an Excel file and start writing the list of business processes to be tested in the project. He or she then sends the Excel file to the module team leaders and they begin to populate the different business process steps. After a few iterations, the test lead finally managed to collect the list of business processes with their steps. The main problem in this approach is TIME! A lot of time is wasted collaborating between all the stakeholders. Reusability is usually very low and this process is begun every time from scratch, project to project. STONE AGE UAT STEP 2 ALLOCATION OF BUSINESS STEPS 300 business processes, 10 business steps in the average per business process, 3 or 4 countries to rollout, this is usually around 10000 steps to allocate to different key users, across functional areas, and across countries. In the Stone Age, with Excel files, it becomes very tricky to allocate the business steps to the right tester, especially when this is done very late in the planning phase. Indeed, the final testers are usually only defined a few days before a UAT cycle starts. This error-prone process takes a lot of time since it requires the different test leads to synch their allocations on a single source of truth. 6 Take Your UAT from the Stone Age to the Digital Age Throw Out Those Excel Sheets!

The main challenges are to have: A single source of truth during the planning phase A real time tracking of the planned workload per team/ tester Enable multiple people to collaborate on the same plan Be able to define an integrated schedule for the UAT: who should test what and when to avoid bottlenecks STONE AGE UAT STEP 3 UAT KICK OFF Once the planning is completed and all steps are properly assigned to the testers with a due date in Excel, these major issues remain: How do I communicate all this to the users? How do I enable them to report the test status? How do I make sure that the different teams and testers will collaborate in order to get through all the business processes? How can I make sure everyone stays on the same page and nothing is lost in translation or collaboration? For weeks, the business representatives from different locations may work together from company headquarters. The test lead may decide that the best way to handle this is to ask all the stakeholders to fly to the same location. How repeatable is that? How much can business users be completely taken out of their daily jobs? 7 Take Your UAT from the Stone Age to the Digital Age Throw Out Those Excel Sheets!

This is not practical for many companies it is expensive to send many people for an extended period of time. There are legal issues such as visa restrictions. Language and communication gaps remain. To send many people to a single location entails expensive flights, hotels, and food costs. Suddenly, they need entertainment in the evenings and what the German worker enjoys may not be the same as what the Chinese plant manager or California sales manager wants. DID YOU KNOW? Up to 50% of a key user s time during a cycle is spent documenting the test executions Many changes also need to be tested on-site. Is the plant equipment working with the machinery? In today s age of SaaS collaboration and cheap communication there must be another way to be in touch without expensive and disruptive flights and hotels. STONE AGE UAT STEP 4 UAT EXECUTION, BEWARE OF BOTTLENECKS! In a short kickoff on day 1, the test lead explains the procedure to the testers, who are now ready to test. For a couple of weeks, the business representatives from different locations will work together from the company headquarters but some things must be tested on-site. The test lead splits the different business processes into multiple printed Excel, Word, or PDF files in order to ease the test results capture and collaboration between the users. 8 Take Your UAT from the Stone Age to the Digital Age Throw Out Those Excel Sheets!

Sharing Excel files are very tedious. Different versions and language localizations can introduce incompatibilities. Communication across languages and cultures are difficult. Does the ERP manager in France understand what the IT Director in New York wrote? Can the Chinese plant manager make sense of the comments of the German CIO? The people responsible for logistics wait politely for the orders to be created, while financial professionals running a few independent tests also are waiting for the big chunk to arrive after all the logistics steps are completed. The test lead bounces from one testing room to the other, in a sometimes futile attempt to know what s going on. Serge in the Paris office took the day off. Because he lacked visibility, he didn t know that he needed to complete his test the day before going on vacation before six other testers in another office could continue their work. And the other testers didn t realize that they were behind. DID YOU KNOW? 20% of tester time is idle due to poor communication between testers This doesn t prevent the following: Users still need to find the next tester in line so the process can continue Some users remain idle for hours until their turn comes to test, more precisely until the predecessors managed to reach out to them! It is difficult to get them engaged in these conditions Defects need to be documented, and the easiest way is to simply send them by email The test coordinator bounces from one testing room to the other, in a sometimes difficult attempt to know what s going on 9 Take Your UAT from the Stone Age to the Digital Age Throw Out Those Excel Sheets!

STONE AGE UAT STEP 5 PREPARING THE DAILY BRIEF OH NO! At the end of the day, it is time to collect the status of the different steps that were executed throughout the day: gather the printed business process to update the main Excel spreadsheet, read emails to check open defects, talk to team leaders... They pray it won t take too much time because the first daily brief with management is first thing in the morning. After a few hours, they finally figure out what s the accurate status progress and the real bottlenecks. DID YOU KNOW? Up to 80% of a test manager s time during the UAT cycle is spent collecting data and creating daily reports. Bad surprise! They discover that they are already behind schedule: many processes are stuck in logistics some blocking defects are preventing the procurement team to go through their steps. Defects were indeed reported to the test manager, but these were lost among his 100 unread emails from today. There is no choice but to reduce the scope and hope that this won t happen again the day after. Meanwhile, it took three hours to generate the daily reports for tomorrow s review. Hopefully nothing goes wrong. By the way, one of the items cut was security testing. As a result, the new security plugins won t be deployed. Remember this three years later after that big data breach. Oh well, it s all in the past, right? 10 Take Your UAT from the Stone Age to the Digital Age Throw Out Those Excel Sheets!

MAJOR PROBLEMS IDENTIFIED BY STONE AGE UAT TESTING Manual processes Lack of visibility Communication and Cultural Gaps Expensive travel and entertainment costs to host a multinational team in one location Scope being reduced along the cycle because a lack of time Users not sure what and when they should test Handoff between multiple users who aren't co-located Bottlenecks Missed deadlines The following sections will provide you some tips and best practices to overcome the following challenges: Painful collaboration both in planning and execution phases Hard to get users engaged Difficult to get a clear picture of the progress and often too late Prepare Distribute Collect Status Update Status Present at Management Meeting 11 Take Your UAT from the Stone Age to the Digital Age Throw Out Those Excel Sheets!

CHAPTER 2 UAT Best Practices and Tips for the Business Professional DEFINE GOOD METRICS Define good metrics so you can track your progress multiple times per day and adjust as soon as necessary. Define goals and KPIs not everything needs to be tested, but understand what the goal of a test is so you don t cut unnecessary tests or miss important tests. For example, the number of steps tested per hour. Don t wait until the end of the day to re-adjust. Each hour counts during a UAT cycle. INCENTIVES Give the testers good incentives without demotivating them. On good testing days, order in pizza or Chinese food and on bad days, remind them that you may have to extend the testing by a week or two. That will certainly motivate any UAT tester! ENGAGEMENT BUILDS ADOPTION Include as many key users as you can to expand buy-in. By making sure that business users are part of the User Acceptance Testing process, they will feel a sense of ownership. Because they are involved in the process, this increases buy-in. 12 Take Your UAT from the Stone Age to the Digital Age Take My UAT into the Digital Age

USER AWARENESS Make your business users aware of the number of steps they block by not doing their assigned testing. It may not seem urgent to them to release an order, but if a tester knows that there are 30 steps after them that have to happen in order to finish the process and he or she is blocking it he may think twice about it. DATA PREPARATION IS CRITICAL So much time wasted in trying to find the right data to be used in a test. Allocate time to generate usable data in your QA environment. Allocate time to properly document it in your test script. CHASE AND REMOVE Chase and remove obsolete steps in your test plan. The business process may have changed. Some steps may not be required anymore. It takes time for the testers to communicate to the test manager that some of the steps are obsolete and for the test manager to readjust the plan. Make sure to review at least the high level steps with some business user representatives to make sure you limit these type of issues. MANAGE TESTERS If you have managed to concentrate all the testers in the same location, group the testers by functional area. There are always questions and training adjustments required during a UAT. You will save precious time by making sure that you have both IT and business representatives, even from different countries, sitting together to go through their testing tasks. Grouping people by region won t necessarily bring the same benefits. Consider video conferencing and collaboration software even before the meeting. 13 Take Your UAT from the Stone Age to the Digital Age Take My UAT into the Digital Age

CLEAR WORKLIST Make sure each tester has a clear worklist! You may have organized your test plan by business process. In the end, the business user needs to know exactly what they have to do NOW! You will save your testers a lot of time by providing a clear list of what is currently needed to be tested. If you don t use a tool, you may need to print or issue reports or Excel spreadsheets per testers per day or halfday for them to be more effective. 14 Take Your UAT from the Stone Age to the Digital Age Take My UAT into the Digital Age

CHAPTER 3 Migrating UAT from the Stone Age to the Digital Age As you can see from the above examples and your own experience, the old way of doing things just doesn t work anymore in today s digital, multicultural, and agile age. You can t afford infrequent updates and expensive travel. Should you go for a traditional test management solution? Are there alternatives? Traditional test management was originally designed for professional testers in software companies, which means: Ease-of-use was not the main design factor It focuses on unit testing by single testers, not on end-to-end business process testing The creation of test evidence is a manual process In order to migrate from the Stone Age to the digital age and improve UAT, consider looking for tools that are focused on the needs of ERP testing. A good UAT tool, such as Panaya, should have: The ability to plan end to end business processes with multi-user collaboration A way for the users to document their test with no effort Live in the cloud in any location, on any device Provide real time monitoring, reporting, with integrated dashboards Document test runs to ease defect resolution and auditing Communicate seamlessly between business users and technical and business users Luckily, Panaya helps move companies to the digital age. 15 Take Your UAT from the Stone Age to the Digital Age Take My UAT into the Digital Age

CHAPTER 4 How Panaya can help you boost your UAT cycles Panaya can boost your UAT, drastically increase your key user productivity, and enable real time reporting - all while providing a user friendly interface for your users. Users will be happy and eager to take part in the testing! By working in the cloud, there are no translation difficulties dispersed teams can get real-time updates, without concerns about localization, file conversion errors, and different software versions. It s also accessible anywhere, on any device, and so progress can be monitored even when on the road or the team member on vacation can take care of the urgent request. Because there s real time monitoring of the test progress, and handoff from A to B is automated, time gets spent testing instead of reporting. As a result, there is no need to cut items from project scope due to deadline pressure. Project scope is also made more relevant. Panaya s data analysis, powered by machine learning and big data, helps the team prioritize tests. With better visibility of what is most likely to break and which objects are impacted, the project risks are reduced. While in the Stone Age, users had to manually document and handoff test runs. With a modern solution like Panaya documentation is automated. This also gives organizations reusable tests that can be stored in their test library. 16 Take Your UAT from the Stone Age to the Digital Age Take My UAT into the Digital Age Get a Free Demo

Business users who are busy with other tasks feel that they don t have time to deal with UAT but reluctantly join in because they know that the consequences of a missioncritical functionality not working is far worse. But, because of the ease of use of Panaya, adoption increases. They aren t forced to participate they urgently volunteer. In addition, communication barriers are broken down. Instead of relying on manual emails between technical and business users, Panaya automatically passes the baton between testers and developers, increasing collaboration and reducing idle time. 17 Take Your UAT from the Stone Age to the Digital Age Take My UAT into the Digital Age Get a Free Demo

www.panaya.com About Panaya Panaya CloudQuality Suite disrupts the risk, time and costs required to deliver all types of SAP and Oracle EBS changes. We are passionate about the need to make your ERP more agile and align it with your business. Powered by big data analytics and aggregating since 2008, Panaya CloudQuality Suite delivers insights that tell you what will break, how to fix it and what to test. It is constantly improving and finding smarter ways to perform everything from day-to-day maintenance to major projects. 1,220 companies in 62 countries, including a third of the Fortune 500, use the Panaya CloudQuality Suite to enhance and maintain their enterprise apps without skipping a beat. 18 Take Your UAT from the Stone Age to the Digital Age