Tips and Tricks for successful Risk Based Testing (RBT) Satyanarayana Murthy Kotta Sep 2013

Transcription

1 Tips and Tricks for successful Risk Based Testing (RBT) Satyanarayana Murthy Kotta Sep 2013

2 Context Setting x This discussion is not around the theory part of what RBT is all about. Focus is on the HOW part for a successful RBT implementation 2

3 Quick brush up on RBT 3

4 Tips & Tricks 4

5 Tip 1 Do not take to RBT just because... the theory sounds really good and fancy Never pursue this until there is a commitment from Senior Management 5

6 Trick 1 Management to first understand the customization of this concept to their context and then drive from front 100% commitment from Senior management with a Sponsor to track fund allocation and progress of the initiative Highlight the benefits and encourage implementation of RBT in public forums like Town halls, Steering committee meetings, Rewards n Recognition (RnR) functions RBT will work best when Top down approach for Senior management Communication Bottoms up approach for Implementation 6

7 Tip 2 Just because there is the word Testing in RBT, do not assume that it does not have any other stake holder involvement This is never a ONE Team show 7

8 Trick 2 Identify the stakeholders from BUSINESS, ARCHITECTURE DEVELOPMENT and TESTING teams Define the RACI matrix for all the RBT activities. All the RBT activities static testing, risk assessment, prioritization should be part of estimation and project plan Ensure to explicitly mention the RBT approach in the Test Strategy document that is signed off by all stakeholders RBT will work best when All stakeholders understand, agree and realize the benefits RBT brings to their teams and the project on the whole. Testing team acts a facilitator in the whole journey 8

9 Tip 3 Do not just get the external consultant/vendor to design a generic framework for your organization Framework designed without internal stakeholders involvement will always FAIL 9

10 Trick 3 Form a TMO office first. Workshop with relevant stakeholders (business, development and testing heads) in finalizing the framework. This will help in easy roll-out across teams Right framework should focus on 3 main aspects of RBT (static testing, risk assessment and execution based on prioritization) Proper flow detailing the tasks/ roles of stakeholders (like business criticality value from a business analyst and failure probability value from the coordinate effort of the development and testing teams) Tools available at each step. RBT will work best With a simple to follow framework. Customized in line with the Risk guidelines existing for the organization 10

11 Tip 4 Do not go for a Big Bang approach to roll out RBT Improper deployment strategy and unstructured change management are the biggest hurdles 11

12 Trick 4 Identify at least 2 projects to pilot the RBT framework. Handhold them in implementation. Refine the framework and tools based on feedback. Help them analyse benefits in the scenarios of with / without RBT in place. Role of change management is critical for successful implementation. Coordinating with the right people through the right forums is essential Let the pilot projects do the talking to the other teams for a successful RBT across the organization Prepare the training plan on tools/process for each role in the project team Identify the right approaches (like identifying champions or portfolio managers) to conduct the trainings. An implementation tracker will be the key to success RBT will work best When asking for prioritization of tasks become a buzzword right from the project team-member level onward 12

13 Tip 5 RBT is a methodology. Does it mean there are no tools to assist this? General feelings: RBT is difficult and takes long time. No simple to use tools Existing tools don t support this 13

14 Trick 5 Develop / Reuse simple to use tools that motivate the team and make their life easy 2 tools for static testing, 1 for risk assessment along with the right framework in place can do wonders Ambiguity checker like a spell checker highlights possible ambiguous words and generates a quick report in less than a minute Testability scorecard - excel tool that verifies the testability risk of a requirement based on values given to parameters (like completeness, consistency, unambiguous, discrete, stable and traceability) Risk Assessment excel tool to auto calculate Risk for chosen values for Business criticality and Failure probability Tools from HP, IBM come with in built features to support RBT. Explore to make effective use of this module RBT will work best By assisting the team with Simple and Effective tools that create a wilful pull to practice RBT than a resistive push 14

15 Tip 6 What cannot be measured cannot be controlled is a basic reality The Notion: RBT is more subjective and less objective with no defined metrics for measurement 15

16 Trick 6 Identify the right set of metrics that makes RBT objective Testability Risk Score: Indicates the testability risk at a requirement level in the early phase of the project - Good indicator of the issues that need upfront clarification. Risk Assessment: Indicates the number of requirements under high, medium and low categories - Good indicator for prioritization. High Severity Defect Removal Effectiveness (HDRE): Indicates the Sev1 and 2 defects in UAT and production against total number of defects. Good indicator of the success/failure of RBT and helps in refining the risk assessment guidelines. Risk-based test execution report: Indicates test progress against the risks associated with requirements. Good indicator of testing focus and residual risk at any point of time that assists business to take informed decision based on quantitative analysis than qualitative analysis. RBT will work best When we realize that this is no different from what we do in day to day life with a mentor in place 16

17 Thank You 17

18 About the Authors : Authors of this point-of-view have worked hand in hand with an Australian Banking client for 1.5 years from the definition to the successful implementation of Risk-based Testing (RBT) Satyanarayana Murthy Kotta is a Principal Consultant at Infosys Technologies. With over 14 years of rich IT testing experience, he has helped companies in Australia, Middle East and Europe to define n implement testing efficiency improvement programs. Satya s key strength is to provide customized practical solutions that fit client needs. Kamalahasan Madaswamy is a Senior Project Manager at Infosys Technologies. With over 15 years of rich IT experience across verticals, he has involved in development, maintenance, production support and testing projects. His vast experience and strong acumen to development life cycle has helped him to play the role of testing consultant to define and implement Risk Based Testing and Metrics Governance program successfully for a leading Australian bank References for this article: Our own working experiences that taught us more lessons to share Glossary RACI Responsible,Accountable,Consulted,Informed 18