Simplifying Risk-Based Testing

Transcription

1 Simplifying Risk-Based Testing Ingo Philipp, Product Manager 2016 Tricentis All rights reserved.

2 Everything involves risk. Testing is no exception. Any software tester can tell you that risk and testing are related, but many still do not know what risk-based testing actually is. Risk-based testing is much more than testing based on risk, though that is still the definition heard most often. Let s start by sorting out what business risk means. If someone asks you how risky is it?, they are asking you to quantify the potential of losing something valuable. If it s personal risk we are talking about, that something valuable could be anything from loss of life, to loss of pride, etc. Business risk means losing something valuable from a business perspective. This could include loss of faith of clients, loss of reputation, loss of money, etc.

3 When it comes to testing, that risk potential arises from software errors causing a condition or capability required by a stakeholder, i.e. a requirement, not to be met or possessed. Depending on the complexity of the software, the requirements could range from the tens to the thousands. Any time there is a discrepancy between the intended and the actual behavior of the software, it is the result of a software error. If you are going to diminish risk through testing as early as possible in the development process, you have to first be able to estimate the threat of damage caused by these discrepancies on the level of your requirements. The problem with such a risk-based approach is that it usually takes a professional and a huge amount of time time that Agile teams often don t have when trying to continuously test to keep the DevOps cycle flowing. For many teams in this sticky situation, the answer is to either test everything exhaustively, or test at random. As we all know, neither of these solutions are actually very helpful though. 67% average level of redundancy in enterprise test portfolios 1 40% average risk coverage 1 achieved in comprehensive enterprise test portfolios 1 Source: Customer projects, Tricentis

4 This doesn t mean however, that continuous testing and risk-based testing can t get along. What it does mean, is that we need a fast, lightweight approach to making a smart and reliable estimate about the risks. It is exactly the sort of solution we designed Tosca Testsuite s Risk Optimization feature to provide. The first step then, would be to figure out how to determine the requirements that pose the highest risk. We can do that with Tosca Testsuite by asking a requirement two simple questions. 1. How often is it used? This question determines the frequency of usage of a requirement. How often is the functional and non-functional spectrum of the requirement used? Some requirements may be used in daily business, while others only a few times - say, once per quarter. Some may be used by multiple users, some by a few users. We have to give priority to requirements that are used often and heavily. 2. What is its potential damage? This question determines the potential damage of a requirement. How important is this requirement to the daily processes of the program? If this requirement were to stop working, would it crash the whole program or halt critical business functions? Or would it simply be an inconvenience that could be worked around? 4

5 Then we answer each question with a number between 1 and. Next, Tosca Testsuite s risk coverage optimization does the math to compute the risk contribution of each individual requirement. As a result, Agile teams working hard meet tight deadlines can be selective about which tests cases should be executed first. By implementing this approach to risk-based testing, you can cover the most important product risks as early (and therefore, cheaply) as possible. Securities Trading Capture Order Client Side Validation Check Eligibility Check Suitability Check Availability Market Side Validation 4 4 Damage Class 4 Absolute Weight Rectify Order 2 2 Cancel Order Frequency Class

6 When it comes to life, not to mention testing, risk assessment and survival go hand-in-hand. Risk is not a passive force it needs to be actively mitigated. This means that without a risk-based approach in place, there is simply no frame of reference to measure success in terms of business risk coverage. Ultimately, a risk-based approach enables you to ensure that your testing activities are aligned with your business risk objectives. Because really, testing harder isn t the answer. Testing smarter is. 6

7 Stay in touch Impressum Locations Contact 7