AutomatedQA Webinar 1

Transcription

1 AutomatedQA Webinar 1

2 AutomatedQA Webinar 2

3 Today, the tips are not about technology. We won t be covering TestComplete specifically. The focus is on people and communication. 3

4 What is a successful test project? 4

5 A successful test project starts with a clear, realistic set of objectives agreed on from the top down. The project goals are communicated to all project stakeholders. The project is completed on time with the goals achieved as agreed. 5

6 Isn t that the goal of any smart project? 6

7 All projects should work that way but test projects are often treated with less respect. It s up to you to make the difference. Resist the pressure to skip the basics of project preparation. 7

8 Your organization doesn t know why it s testing. Image courtesy of db*photography 8

9 Before you start your testing project, give your decision makers a brief summary that outlines the expected benefits. Focus on the benefit to the organization. 9

10 Benefit: Estimate savings in development and support costs Example: Estimated cost to find and fix bugs found by QA before release: $320/bug Estimated cost to fix and support bugs found by customers: $1,084/bug We expect to save more than $1,000 in development and support costs for each bug found by QA and fixed before release. (Source: Critical Testing Processes by Rex Black, 2004) 10

11 Benefit: Increase user satisfaction. Get a baseline by polling your user base with a service like SurveyMonkey. Example: Would you recommend this application to a coworker? How satisfied are you with the quality of this application? Limit the survey to 3-5 questions to improve response rate. Send the survey to 100+ users. Resend the survey 30 days after the new release. 11

12 Problem: Your organization doesn t know why it s testing. How to avoid: Define the goals Use simple metrics to illustrate benefits Get buy in from decision makers Track the metrics Follow up and share the results 12

13 Your organization hasn t agreed on what kind of problems you re looking for. All tests are not equal Image courtesy of Dave Bleasdale 13

14 Your organization hasn t agreed on what kind of problems you re looking for. Test everything? Not possible. Business processes, business logic, UI standards, performance, load tests, client configurations Prioritize and communicate. 14

15 Create another one page summary that outlines what you can test and what you can t. List categories like business logic, installation/configuration and performance not specific test details. Share it with stake holders and decision makers and agree on it before starting. 15

16 Problem: Your organization hasn t agreed on what kind of problems you re looking for. How to avoid: Don t agree to test everything. Spell out up-front what will be tested. Document what will not be tested. Get sign off from decision makers. 16

17 Raise your hand if someone on your team is building a test framework right now Image courtesy of jonny.hunter 17

18 You re building test tools instead of doing testing. Image courtesy of gb500 18

19 Smart testers are also trying to improve their tools and processes. Encourage this. But not at the expense of actual testing. 19

20 Problem: You re building test tools instead of doing testing. How to avoid: Put the tool work on hold. Schedule time for tools. Define a separate project. Require minimum # of new issues. Do allow tool work later. 20

21 You re testing the wrong things. These are easy to test, but not important to test. Image courtesy of kristin_a 21

22 Testing priorities change Exploratory testing is valuable but needs direction. Scripted testing is great but can turn redundant if unmonitored. Use continuous feedback to stay on course. 22

23 Problem: You re testing the wrong things. How to avoid: Set priorities. Track what s been tested. Track what s changed in builds. Review reported issues vs priorities. Triage daily or at least weekly and adjust. 23

24 Your test team doesn t know how to test. Image courtesy of Kristian D. 24

25 Your test team needs more training: Testing basics How to use the platform How to perform tasks How to verify the business process/logic. How to use the testing tools. 25

26 Problem: Your test team doesn t know how to test. How to avoid: Testing training Test tool training Good requirement docs Subject matter experts + testers Developers + testers 26

27 Development doesn t understand or can t reproduce your problem reports. Image courtesy of Kaibara87 27

28 Development doesn t understand or can t reproduce your problem reports. How to avoid: Report the problem first, not the solution. Include system configuration Include the scripted test Let the developer see the issue (screenshots, video) Use VMs to reproduce Embed testers in development Pairs testing 28

29 Your tested system isn t testable enough. 29

30 Your tested system isn t testable enough. How to avoid, part 1: Include QA early in the cycle Include QA when choosing new components Test new software tools with automated testing tools Developers use same testing tools as QA 30

31 Your tested system isn t testable enough. How to avoid, part 2: Dev build server with automatic smoke test Use VMs for baseline configs Embed developers in QA Pairs testing 31

32 1. Your organization doesn t know why it s testing. 2. Your organization hasn t agreed on what kind of problems it s trying to find. 3. You re building test tools instead of testing. 4. Your team is testing the wrong things. 5. Your team doesn t know how to test. 6. Development doesn t understand your reports. 7. Your tested system isn t testable enough. 32

33 Questions about Test Project Problems? If you have technical questions about TestComplete, please visit 33

34 34