Emerging Paradigms in Testing. Refocus your view of testing to drive superior results

Transcription

1 Emerging Paradigms in Testing Refocus your view of testing to drive superior results

2 A good testing experience The quality of the application, rapidity of testing, as well as the ability to easily and confidently update the service once live, were major benefits. The test driven development and continuous integration approach was the most professional I ve seen and provided very high quality and stability. Ian Frew, Leader IT, The Buzz at IAG

3 The case study! Multiple vendors using mixed approaches to development, agile and waterfall - Approximately 60 developers! Mainframe backend, SOA and multiple front-end systems and external 3 rd party system.! Testing covered performance, security (including PCI), disaster recovery, deploy/ rollback, failover/recovery, user acceptance, usability and functional.! 5 full time testers! 6 months development, after last feature developed 1 week to go live. no Severity 1 defects in first 9 months.! After go live could make a code change and deploy in the same day no Severity 1 or Severity 2 defects in 9 months.

4 Testing Principles We believe testing should focus on building quality in rather than "testing it in". We do this by involving everyone in the testing effort, using fast feedback to make decisions. We believe tests are an asset of the product and not the project. Working in this way enables faster delivery into production, provides a clear and consistent view of testing across the organization and helps business optimize value within the software.

5 Before we get into the details! Describe the principle! Present a practice demonstrating the principle in action! Provide a real world example from the case study highlighting the principle in action

6 Principle: Build quality in You can not inspect quality into the product; it is already there. W. Edwards Deming! Remove the cost of fixing the defects by preventing them from occurring.! Find mistakes early and prevent them from becoming prevalent.! Reduce reliance on band aid fixes.

7 Practice: ATDD (Acceptance Test Driven Development) BA works with business to define a feature in a testable, bite sized vertical slice (implementable in 2-3 days) BA and Tester work with business to define how the feature will be tested X test fails Test passes Tester works with developers to implement the tests and developers implement the feature BA and Tester test the feature and how it fits in to the broader system and business process BA and Tester get handover of feature from developers

8 Example: Defects as a measure Preventing Defects Finding Defects Finding Defects Preventing Defects

9 Principle: Fast feedback! Get feedback to the people who need it as soon as possible.! Work with up to date information and make informed decisions.! Regression suites should be run in hours not weeks.

10 Practice: Test Pyramid Scenario More tests Acceptance Integration Increased execution Time Unit

11 Example: Mainframe developers

12 Principle: Involve everyone Testing Testers! Target utilization of testing resources.! Get better tests through diverse input and the team looking after the tests.! Better use of resources equalizing.

13 Practice: Automation Framework Write Test Specification in Text BUSINESS REPORT TEST INTENTION DATA Use Code TECHNICAL REPORT DOMAIN SPECIFIC LANGUAGE (DSL) Write Code HTML SERVICE DATABASE REPORT TEST RUNNER

14 Example: Rating Table

15 Principle: Tests as an asset Code + Tests Product! Reuse tests across projects, don t create new tests all of the time.! Better understanding of test coverage.! Treat the test cases and the application code as the product don t separate them.

16 Practice: Version Control! Automated Tests are code treat them like code! Test code is likely to be the largest code base in your organisation! Version controlling application code is common practice so it should be for test code.

17 Example: Can t rollback tests

18 Principle: Faster delivery into production! Do not have months of testing after development is complete.! Return on investment early start earning revenue as soon as you can.! Testers should stop focusing on preventing a product going live and instead focus on how to get it live.

19 Practice: Build Pipeline

20 Example: Dev to Prod in same day Dev FrontEnd Dev Dev CMS Dev Dev Frontend Dev Dev 3 rd Party Doc Dev 3 rd Party Claim Dev DR Dev Test Security Dev Pre-Int Int Sys SIT UAT Prod Dev Operations Services Exploratory Acceptance Performance Claims Call Center Integration Security Call Center

21 Principle: Clear & consistent view! Simple reporting so you can compare one project with another.! Metrics should not drive behavior.! Open and honest.

22 Practice: Example Report

23 Principle: Optimize business value! Testing is not just insurance.! Target testing to features that are changing not a scatter gun test everything approach.! Help drive new features and functionality.! Allow business to try new ideas in a safe environment providing quick feedback

24 It is not just small web apps! Trainline - Train ticketing system: distributed team, UK and India approximately 200 People. Financial Services integration of over 70 systems (internal and 3 rd party). Distributed team, Australia and India approximately 100 People Data warehousing - consolidation of multiple (approx 30) data warehousing solutions into one. Multiple streams in the program of approx 150 people.

25 It is about the principles!