Implementing Test Automation

Transcription

1 Implementing Test Automation Myths Truths Steps to Success page 1 plvision.eu

2 Introduction More and more successful software companies are getting to the point where their existing testing strategy stops working. They still complete all milestones and deliver their software products to market within the planned timeline, but it s becoming obvious that their current testing capabilities have almost reached their limit. The logical option for software vendors struggling to keep their software quality on the highest level is software testing automation. According to Test Automation Market - Global Industry Analysis, Size, Share, Growth, Trends, and Forecast , the global test automation market s worth in 2015 was $12.91 billion, and is expected to hit $85.84 billion by the end of Such impressive figures are a certain indicator of increasing demand for test automation, and a fuller understanding of its benefits. Yet numerous misconceptions about automated testing persist in the modern technological world, many of which derive from lack of fundamental insights into a successful test automation process. Test automation (TA) is a substantial change for any development cycle, with associated specific risks that must be understood and considered. However, unrealistic expectations are sometimes placed on testing automation, with stakeholders hoping that it will dramatically and instantly reduce both the effort and testing cost. In reality, test automation should be a key part of a continuous delivery workflow, and as with any complex process, it must be set up right. Later on we will discuss conditions and recommendations that are vital for establishing successful software testing automation. Test Automation: Why Do You Need It? Many new enterprises and startups establish their own development process, mostly putting little, if any effort into testing automation. This approach works up to a certain point in a business s evolution, until the product suddenly stops being delivered to the end user within acceptable time and with expected quality. With immense competition, software vendors, especially startups, need to work hard to produce high quality software products, while decreasing time to market and boosting customer satisfaction. In many cases test automation is a powerful tool for this, though not the most obvious one. page 2

3 What are the most certain signals that test automation is necessary for your software product development? Table 1. Typical indicators of the need for test automation Challenge Solution Benefit Exceedingly long testing stage, testing team overloaded Frequent release date postponements due to critical issues End-users are unhappy with product quality or delivery time Automated tests can be executed over non-working hours and weekends/holidays, in parallel for multiple platforms or software versions Automated testing simplifies testing effort estimation, and allows detection of critical issues before they impact the project severely Set up correctly, automated testing enables increased product testing coverage, verification of more user scenarios, testing of long-run cases, and execution of other hard-to-emulate-manually scenarios that are vital for your product s quality Reduced testing effort and cost Faster time to market, less costs reserved for risks management Increased customer satisfaction and higher competitiveness of your product Test Automation Setup: Myths and Truths If test automation benefits are so obvious, why isn t everyone using it, and why is it not always delivering expected results? As any other complex process, test automation needs to be implemented correctly, without impacting existing development workflow. Based on our experience, here are the most common myths and truths associated with test automation setup. Table 2. Common myths and truths about test automation implementation Myth Investment for test automation will not pay back Automated testing will disrupt the established development process Test automation requires vast effort and investment Truth Involving expert TA consultants will help you define clear and measurable goals, and decide if automated testing is a reasonable option in your specific case TA consultants can develop an effective test automation strategy for your product, considering the specifics of your development process to augment and enhance it. Initial investment for test automation setup will be fully returned, saving you a great deal of effort and costs in the longer run. page 3

4 Establishing Test Automation: 6 Steps to Success Based on 10 years experience of successfully helping our customers to establish a test automation process, we would like to share our high-level vision of an effective test automation implementation plan. Here are the steps you need to take to safeguard a positive outcome. 1. Create a plan of current process stabilization to guarantee success of ongoing development a. Define responsibilities of test team members b. Define and formalize testing goals c. Define release requirements and development process flow d. Analyze release goals and process, compare with actual testing team activities, specify activities that must be performed to achieve expected quality e. Define a new release plan of test activities to achieve the same (or very slightly changed) product quality within required terms page 4

5 Establishing Test Automation: 6 Steps to Success 2. Define and formalize product requirements and user scenarios. 3. Based on defined requirements, develop a test automation plan a. Define the list of all test activities needed to verify requirements and use scenarios b. Create a list of test activities that can be automated c. Create a list of test activities that must be automated d. Define necessary alterations to the manual test process as the first step towards automation - infrastructure usage rules, test plan development guide, test case development guides, manual test case results storage for comparison with automation test results during implementation e. Define requirements for an Test Automation Framework f. Define required skillset of the test automation team 4. Elaborate test cases automation procedure and define timeframe a. Assign test automation activities to milestones, sprints, releases, etc. b. Design the procedure for automated test cases acceptance for regression testing, as well as release candidates testing procedures. 5. Perform the test automation process according to the plan. 6. Monitor test automation plan execution and react to changes in real time. page 5

6 PLVision s Test Automation Approach: Success Stories At PLVision, we continuously help our customers to achieve ambitious goals, while enabling them to cut their time to market, improve product quality, and lower development costs. In many cases, we use test automation as one of our key tools to help achieve the best results. Having dealt with numerous enterprises on many challenging technological projects, we have gained in-depth expertise in various test automation approaches and best practices. Below are just a few examples of many projects types in which we utilized the power of test automation to provide business value to our customers. Story #1 Company: Start-up that develops hardware and software based solutions. Initial conditions: A typical case of a company growing to a point where the existing testing process no longer delivered satisfactory business results. The number of supported platforms and delivered software versions for various hardware and end-users reached a barely manageable amount. Alterations to the development cycles were attempted as a solution, which didn t work in the expected way. A basic automated testing process was already established when the customer reached out to us, which needed further expert analysis and consultancy for correct implementation. Solution: Initially a fully automated development process was proposed, following all formal steps. After a deep analysis and assessment of the customer s current project state, issues, and risks, it was decided that a stand-alone test automation process would not be an efficient solution in the customer s case. Consultancy and expert aid was provided to effectively introduce elements of test automation into the development cycle, thus providing a flexible solution, well-adjusted to the customer s needs. Business results: The customer s development process effectiveness was greatly improved, preventing critical issues that caused delivery disruption previously. This resulted in significantly reduced expenses for risk management, and cut the product s time to market. page 6

7 Story #2 Company: A leading hardware and software vendor, with established and well defined development processes, a clear vision and requirements for the collaboration process with contractors. Initial conditions: The company planned to launch a new product in a sphere that was completely new to them at that time. Expert consultancy was required in terms of test automation establishment, as well as research and introduction to the unfamiliar industry. Solution: PLVision successfully developed an automated testing process for the customer following a standard workflow. The development procedures were modified and adjusted to make test automation transparent, with easily measurable results. A custom test automation framework was created, specifically designed to interact with the customer s business environment, including third party hardware and software testing tools. Test cases were created for various types and methods of testing. All development goals were successfully met on a long-term time scale. Business results: Our automated testing solution was successfully integrated with the customer s infrastructure, enabling the construction of a CI system with support for the customerspecific test environment and features. Several sub-projects were supported within the one CI system, providing improved visibility of the result for each stage of the process, with associated significant reductions in management overhead, time and costs. page 7

8 Story #3 Company: A leading hardware and software vendor, with wide presence in the Computer Networking domain. Initial conditions: The customer had an established and well defined development process, and a clear vision and requirements for the collaboration process with contractors. The company planned to significantly extend its product line, and required external expert resources to participate in product development and test automation. Solution: PLVision provided the customer with experienced development and test automation teams, which were successfully incorporated into the company s business processes. This involved distributed teams around the globe, and contractors from various external vendors. Introduction of a test automation process was proposed, including development of a test automation framework for product testing automation. Business results: Development of a custom test automation framework increased test case coverage, enabling the customer to meet all milestones on an extremely aggressive timeline, even after an increase in supported platforms, without affecting the product quality. page 8

9 Test automation journey: PLVision s recommendations 1. Use open source tools In many cases, open source tools are a great way to decrease test automation cost and duration, as they provide very flexible, easily expandable solutions. However, open source tools may not be easy to adapt to the specifics of each process, which is why we would recommend that you implement only effective, industry-approved open source tools, backed up with the support of expert consultancy. 2. Develop a solid strategy Investing in developing a solid test automation strategy is crucial, providing a foundation for your entire project and influencing its performance in the short and long term. This will ensure not only transparency of the product development process, but also high quality of your products and faster time to market. Our recommendation is to follow best testing practices to be able to eliminate and mitigate risks effectively. 3. Follow the strategy It seems pretty obvious: however it is easier to lay down a perfectly polished test automation strategy than to actually follow it. Your project s management processes and approaches should be streamlined with the TA plan. Slight deviations are inevitable, but they should be detected early and corrected promptly or, if the need for deviations arises too often, you should revise the strategy itself. Make sure that continuous development and deployment best practices are included into your product s roadmap, and invest in a test automation team who are not only technically skilled, but also aware of the TA strategy and able to advocate for it. Learn about kutest-net, a Python-based test automation platform, providing the core of your product s testing strategy: plvision.eu/kutest-net page 9

10 Summary Test automation is one of the key tools ensuring the best quality for your software product. If release delays, critical issues, and negative feedback from your customers are getting more frequent, these are clear indicators that your project requires professional test automation. If TA is already on your product roadmap but fails to produce the expected results, you should reconsider the TA approach you are using. To secure a fast and profitable outcome, test automation should be implemented wisely, with use of best practices and a customized approach. Fundamental stages, such as requirements analysis, test plan development, and test automation framework development are essential to a successful automated testing process, but their execution should be aligned with the project and product specifics. Reaching out to test automation experts is a great shortcut to getting valuable industry knowledge on test automation and putting it on your product s roadmap. Expert consultants can analyze your current TA strategy and provide quick insights on its ups and downs, or work with you to develop an effective TA strategy from scratch. Experienced test automation engineers already possess in-depth practical knowledge of TA best practices, and have solid experience in using industry-approved open source tools to considerably reduce TA expenses. Start your successful test automation journey today! Learn more at: plvision.eu/software-testing-automation/ page 10

11 About PLVision PLVision is a European provider of software engineering and testing services in the Computer Networking and Internet of Things domains, empowering the world s leading businesses to stay at the forefront of technology since Headquartered in Kraków, Poland, the company has development centers in Lviv and Odesa, Ukraine, with a team of about 140 highly qualified professionals, and representative partners in Germany, USA and Brazil. With any further inquiries about the whitepaper or test automation in general, feel free to contact our Test Automation expert Mykola Chaplahin at mykola.chaplahin@plvision.eu plvision.eu