Agile Testing - Joe Caravella 1

Transcription

1 1

2 From Long Island NY Spent 20 years in USAF Sys Admin Software Development Working in Software Development & Testing since 1993 Hobbies include motorcycles, boating, and water sports 2

3 Waterfall lots of documents and diagrams that become obsolete when coding begins. Plus long development cycles and annual releases Scrum-but follows the basic tenants of Scrum, BUT inconsistent iteration lengths, no retros or reviews, and still doing annual or bi-annual releases Scrum solid understanding and execution of all Scrum principles being led by a certified ScumMaster Kanban workflow queues with WIP limits that encourage pulling story cards through your development process and into production 3

4 My learnings were incremental and continue even today After many retrospectives and tweaks, I ll describe some adjustments I ve made 4

5 Product, Dev, and QA should sit and work together every day Rule of Three for all product discussions (QA, Dev & Product) 5

6 Create a Quality Development Process instead of trying to to test quality into your product Unit testing and continuous integration + governance can really help Get Devs an integrated environment to test their changes before going to QA Create an automation framework that is dev friendly they will use it as well 6

7 The entire development team should create and have buy in to the testing & certification procedures Consider the Agile Testing Pyramid 7

8 The test pyramid is a concept developed by Mike Cohn, described in his book Succeeding with Agile Mike s point was to demonstrate that you should have more lower level unit tests in your testing strategy I added percentage goals, exploratory testing, and the need to always try and push testing lower 8

9 A Mind Map is a diagram used to visually organize information Exhaustive testing is rarely possible so strategies like Pairwise Testing can help identify the combination of test cases that have the highest probability to find defects 9

10 Don t let your regressions grow so large that it takes more than 1 business day to certify a release Testing managers and test leads should add regression pruning to their monthly housekeeping procedures Querying your previous test results could help identify tests that never fail and are at a lower risk of 10

11 Testing & development should be parallel activities Avoid running a testing sprint after a development sprint Acceptance should occur only after QA is done 11

12 Avoid making defect fixes directly in QA envs those change sometimes do not get checked in and the issue will occur in production All changes to QA envs should be applied by a versioned increment of your product (this included DB changes!) QA should initiate builds & deploys only when Continuous Integration is Green (meaning all UTs have passed) 12

13 To avoid surprises, use the same people/team that will deploy to Production for all QA deployments Many times they will identify an issue that won t work in Production 13

14 Create a repeatable release cadence Don t sit on code get value in front of your customers as soon as possible This will also require teams to figure out how to be more efficient with their release certification procedures 14

15 Try to keep production as close to the trunk version of your code as possible Consider using Release Trains & Feature Toggles Release Trains code needs to be checked in by certain day to make it on the train Feature Toggles configuration that controls when features are available to users 15

16 16

17 Create a monthly data capture process so you stay informed about changes in your regression Try to minimize manual regression as much as possible If you can keep your automated regression over 85% you are doing very well 17

18 Create a simple graph of regressions growth and share with teams monthly This helps teams remain sensitive to what s being put in regressions It also helps teams visually see when manual testing is starting to grow too high 18

19 Identify a way to measure the effectiveness of your quality initiatives At a minimum, however, an analysis should be performed on issues that escape the testing phase and becomes customer facing learnings should be shared across all teams Consider creating a company or department KPI that gets updated frequently so teams can react and adjust procedures immediately 19

20 20

21 It s difficult to be agile and produce frequent releases using manual regression tests they just don t scale well When done right, automation helps to create repeatable test execution process with a known timeline 21