AGILE TEST MANAGEMENT WITH VISUAL STUDIO

Transcription

1 AGILE TEST MANAGEMENT WITH VISUAL STUDIO

2 any companies are implementing an agile methodology, but often still have waterfall based tools. We ve been working on several agile projects, one of which we collaborate with our client using the.net platform with Visual Studio. We use VS Mtogether with TFS (Team Foundation Server) and MTM (Microsoft Test Manager) to seamlessly manage the entire testing life cycle including task assignment, test case design, test execution and results tracking. Although we use many agile methods with our clients, probably the most common is scrum, or some form of it. Scrum, originally from an abbreviation for scrummage, is a method of restarting play in rugby where players pack closely together with their heads down while attempting to gain possession of the ball. Hence the roots of one of the most popular agile development methods with close collaboration. This white paper discusses Visual studio and its components, Microsoft Test Manager and Team Foundation Server, in using them to implement an agile testing methodology with a particular focus on scrum. First we ll describe the scrum roles and process because these terms are used later with VS. Then we ll describe and show how it is implemented using Visual Studio. Scrum is one of several agile methods (including XP, Kanban, Crystal Methods, FDD) that have periodic sprints (iterations) and specific roles in an iterative and incremental framework for managing product development. SCRUM PROCESS OVERVIEW FIGURE : BASIC SCRUM PROCESS

3 SCRUM PROCESS OVERVIEW» In summary, scrum projects consist of a series of "sprints where each sprint is an iteration and with each iteration, the software is delivered as working to the product owner. The definition of working varies amongst organizations but in most cases means that the software feature or function has been completed through testing. Sprints are usually referenced or named as: current sprint, future sprint, past sprint. Sprints are typically -4 weeks in length where actual duration depends on the organization s development environment and team. The main point is to have shorter versus longer iterations as per the agile manifesto.» The Product Owner, who represents the end users and stakeholders, decides what goes into the product backlog as shown in Figure. The product backlog is an ordered list of software requirements that is maintained for a product. Requirements are basically work items, things to do. This could include anything it takes to get the product out the door. It consists of features, bug fixes, non-functional requirements, etc. Product backlog items are prioritized by the Product Owner based on considerations like risk, business value, dependencies, date due, etc.» For each sprint, the Product Owner will move the task from the product backlog to a specific sprint, according to the product requirements, the features that will be designed, built and tested during a sprint. A sprint should also include the work needed to fix defects as decided by the team. Part of this step in the process involves estimating the work involved and assigning tasks out to individuals on the team. A customer or product owner should not change the requirements or work activities during a sprint, but occasionally this does happen. That is one of the features of a short iteration. That is, to focus for a short period without change.» Then each day, there is a daily scrum meeting where the team members put their heads together (not exactly like in rugby but you get the idea), and discuss basically what they will do for the day, what they accomplished the day before and what problems or issues they have. They have the daily scrum meetings until the end of the sprint.» At the end of the sprint, there is a demo of the working product to the product owner, where working and done should be defined in the beginning of the project. Depending on the definitions of working and done, the product may be delivered to the end user as a product increment. There is also usually a retrospective where the whole team discusses what worked, what didn t work, and improvements that could be made. PRODUCT OWNER The person who is responsible for creating and prioritizing the product Backlog and reviewing the working product (with other stakeholders) at the end of the Sprint. SCRUM MASTER Scrum is facilitated by a Scrum Master, who is accountable for delivering the product goals. The Scrum Master can also be a team member that also participates in product development (not just management). The scrum master manages the whole process keeping everyone focused and serves the team to remove blocking issues. TEAM MEMBERS Team members committed to achieving a Sprint Goal and do whatever it takes to achieve the goal. Team members should be multi-disciplinary, but most will gravitate toward their strong suit which is also best for the team. Team members include product managers, developers, QA, etc.; basically whoever is involved and delivering the product. 2

4 PRIMER ON TFS, MTM AND VS Team Foundation Server (commonly abbreviated to TFS) is a Microsoft product which provides source code management, reporting, requirements management, project management, automated builds and testing. In other words, it covers the entire application lifecycle. TFS enables teams work together in different locations via a web portal, meaning that testers, developers and product owners can work and collaborate together on an agile project. WHAT IS MTM? Microsoft Test Manager (MTM) is part of the Microsoft Visual Studio toolset designed for test managers and testers to plan, manage, and execute both manual and automated tests. It used to be a separate component, but since VS 202, it has been merged into Visual Studio. FIGURE 2: UNDERSTANDING HOW TFS, VS AND MTM WORK TOGETHER RELATIONSHIP OF TFS, MTM AND VS Since MTM (Microsoft Test Manager) was merged into Visual Studio Studio 202, all the functions that were provided by MTM also can be done in the VS. Therefore from this point forward, we will combine the MTM and VS together and will not discuss MTM separately. FUNCTION TFS MTM VS Create backlog items Create tasks Design text cases Link cases with tasks Associate automation script Create test plan Execute test cases Check result FIGURE 3: TFS AND VS (MTM) FUNCTIONS 3

5 Now that we ve provided the basic introduction to some basic TFS functions and terms, we ll demonstrate how to implement agile test management in TFS using a scrum background. CREATE BACKLOG ITEMS IN TFS We can use the Scrum template (contains backlog, task, bug ), which is created by Microsoft for convenience, but you can also do your own, or customize the one provided. As in any agile project, we need to manage the Product Backlog. We do this by first creating Backlog items. Backlog items are all the things that we are going to do in the project. By simply clicking + Product Backlog Item button, you can create a backlog item. Go to the Work page as shown in Figure 4 and you can see the item has been created successfully. FIGURE 4: CREATING BACKLOG ITEMS IN TFS For a task that is a new feature to be developed, we usually divide the requirements into different user stories of how that feature would be typically used. For those created tasks for the user story, some of the tasks are development related work while others are testing related work. Only after all of the tasks of the user story have been finished, then this user story is considered done. TO CREATE A PRODUCT BACKLOG Click the + Product Backlog Item button. The Work page will show your newly created Backlog. 4

6 CREATE TASKS IN TFS 2 As mentioned earlier, a key part of scrum is breaking down work into tasks with estimates on amount of work or hours per task. Let s create a task for this backlog item. Go to the Board page under Work Click the + icon to add a task. Fill in the necessary information, such as Assigned To, Activity, Original estimate and Remaining Work. Assigned To means who will be responsible for the work. As shown below in Figure 5 as an example, Angela will responsible for this work. We will show this work in VS later on. In the Activity column we can choose the work type. If we choose Developer, this means this task belongs to a developer. From Figure 5, we can see the task is a testing task. 6 7 Original Estimate is the time that we originally estimate for how long the task will take. We fill the actual time that we have spent for this task in Completed Work column as well as hours in the Remaining Work, if more time is still needed for accomplishing the work. In the Backlog Priority, usually the Product Owner will define the priority of a developers task or it will be discussed in the planning meeting. For testing work, we determine the priority based on workload and time to release. FIGURE 5: CREATING TASKS IN TFS 5

7 CREATE TEST CASES IN TFS Here, we show how to create and manage your test cases in TFS & VS and the relationships and differences between them.» To create a test case in TFS, click Work Items > New > Test Case. Then complete the test case steps and expected results on the right-hand side. TIP: When we design the case, we usually put the actual operation in Action column and in the Expected result column we usually put the expected result according to the product requirements. FIGURE 6: CREATING TEST CASES IN TFS TIP: Only after you update your TFS to TFS 202 Update 3, then you can add steps here. Otherwise, the Click or type here to add a step won t work; and you have to use the following way. Before design a test case in VS (MTM), we need to connect to TFS. Let s see how to do this step by step: 6

8 DESIGNING TEST CASES IN VS To create a test case in Visual Studio, with the assigned user... Connect to Team Foundation Server -> select the correct project (Meetingpoint4 as example). TIP: You should have a TFS account with correct permissions FIGURE 7: CONNECT TO TEAM FOUNDATION SERVER TO DESIGN TEST CASES Once connected to TFS, click My Work under Team Explorer Home. FIGURE 8: MY WORK ON THE PROJECT 7

9 In scrum, there are different kinds of tasks and roles in TFS as we have mentioned before. For example, a developer will see developers tasks. As a tester, we would see the testing tasks. Here we examine a tester s work.» In Progess Work is a task status, which means the tasks are being done by me now. If you have to stop your current work to look at another bug, you can just hit Suspend button. This will move the work to Suspended Work and save your current working environment. After you have finished the other bug, then you can click the resume button and you are back where you left off.» Suspended Work is a task status, which means those tasks are stopped for some reason and cannot be continued.» Available Work Items is also a task status, which contains all the To-Do work that belongs to me. Below is my To-Do work list: FIGURE 9: WORK LIST FOR THE USER LOGGED IN - A TESTER EXAMPLE We drag the task 6997 Perform test to In Progress Work > click New > Test Case FIGURE 0: ADDING A TEST CASE After clicking the Test Case, now you will see the following screenshot as shown in Figure (on next page). However, even though you can fill in the test case name and save it in Visual Studio, you cannot add steps for this case in VS. 8

10 FIGURE : ADDING STEPS TO A TEST CASE This is most likely a defect in Visual Studio. Microsoft provides a workaround clicking Edit with Microsoft Test Manager to edit in MTM. Figure 2 shows how it looks in MTM where you click Insert Step to add a step to a test case. FIGURE 2: ADDING STEPS TO A TEST CASE Usually there are many test cases that can use parts of other test cases. This is where you need to think carefully about test organization and design. When you want to use common parts of test cases, use Insert shared steps. This enables you to share the common actions for different cases. 9

11 USING SHARED STEPS COMMON ACTIONS FOR TEST CASES Click the Organize -> New -> Save and close button after finishing the shared steps as shown in Figure 3. FIGURE 3: CREATING SHARED STEPS FOR TEST CASES 2 Click on the Steps -> Insert shared steps as shown in Figure 4. FIGURE 4: INSERTING SHARED STEPS FOR TEST CASES 0

12 LINK TEST CASES WITH TASKS You should link your test cases to their original tasks such that every test case is connected to a task. By linking all of the cases to corresponding tasks, it makes it convenient to manage the test cases. This can be done in TFS, VS, or MTM as shown in Figure 5. The following screenshot demonstrates linking test cases to tasks in VS. Remember that MTM and TFS work the same. FIGURE 5: LINKING TEST CASES WITH TASKS ASSOCIATE AUTOMATION SCRIPT With agile, we want to automate as much as possible if it makes sense. So if you have fairly stable features and code, we usually start to think about what parts of the testing can be automated. If you have created a CodedUI script for a test case, you can associate these scripts. You can use VS to create automated tests of the user interface (UI) known as coded UI tests. These tests provide functional testing of the UI and validation of UI controls. Automated UI tests enable you to test that the UI is functioning correctly after code changes. Coded UI tests are quicker to run than manual tests, meaning that you can run them more frequently and faster.

13 However, automated testing totally can t take the place of the manual testing. If the requirement always changes (as they often do in agile) or the logic is complex, then you may continually need to make changes to your code to either optimize or maintain. So, you have to look at each feature and function to decide if automation is suitable. Learning to write automated tests in CodedUI takes some time. Here in Figure 6, we show a simple verification log using CodedUI. FIGURE 6: SIMPLE EXAMPLE OF TEST AUTOMATION WITH CODEDUI Please note:» Associating an automation script can ONLY be done in Visual Studio.» You need to rebuild the solution (rebuild solution is a function provided by VS) before associating the automation scripts. FIGURE 7: ASSOCIATING AUTOMATION WITH A TEST CASE 2

14 CREATE TEST PLAN Up to this point, we have created test cases. Many think that test plans are not needed in agile which is true to an extent. You don t need a detailed plan spelling out everything, but it s still good to group test cases together to make them easier to manage into test suites. Here, we show you how to create a test plan to execute the test cases. The name of the test plan is up to you, but it should be easy to understand, reference and find not only for you, but for the whole team. Please note:» Regarding how to organize the test plan, we organize the test plan by Company Products. This way it is convenient for us to see how many User Stories were created for a product as shown in Figure 8.» The ID of a test case is generated automatically, which means that each test case has a unique ID.» Test suite: in software development, a test suite is a collection of test cases that are intended to be used to test a related function of a software program. FIGURE 8: CREATING A TEST PLAN 3

15 EXECUTE TEST CASES The benefit of putting test cases into a test plan is that you can execute them together. To do this, go to the Test tab in MTM. Make sure you are at the correct test plan, right click a test case and start execution. If the test case is not automated, you will see a window like this: TO START TEST CASE Click Start Test as shown in Figure 9. Then as shown in Figure 20, you can fill Pass or Fail for each step. If you find a defect, click the Create Bug icon to add a defect, also shown in Figure 20, then add the steps as shown in Figure 2. FIGURE 9: EXECUTING TEST CASES FIGURE 20: PASS/FAIL FOR EACH STEP 4

16 FIGURE 2: ENTERING DEFECT STEPS Actually, it s more convenient to create the bugs in TFS because in TFS shows an overview of the bugs with the user story or task as shown in Figure 22. FIGURE 22: TFS OVERVIEW CONNECTING BUGS AND TASKS 5

17 If the test case you want to run is associated with an automation script, right click the case, then select Run with options as shown in Figure 23. FIGURE 23: EXECUTING AN AUTOMATED TEST CASE FIGURE 24: CHOOSING THE EXECUTION ENVIRONMENT FOR AUTOMATED TEST CASES You are then able to select on which build and environment you want to use for executing the test case as shown above. FIGURE 25: EXECUTING AN AUTOMATED TEST CASE Here s what it looks like when the test case starts to run. It will invoke the associated CodedUI automation script, and run the test in the script as shown in Figure 25. 6

18 CHECKING RESULTS After test case execution, go to Plan > Results to check the results. You can change the view based on Test Suites, Test Configuration, and Testers separately. Figure 26 shows comprehensive results for all of the sprints. FIGURE 26: EXAMINING TEST CASE EXECUTION RESULTS We can find lots of useful information in Figure 26. Let s examine some of the most important elements. TEST SUITES (TEST PLAN): We could choose a specific sprint to check all of the results for that sprint as shown in Figure 27. FIGURE 27: EXAMINING RESULTS FOR A SPECIFIC SPRINT 7

19 TEST CONFIGURATIONS: If you have different configuration options for the test plan, then you can see various configurations. In this example however, there is only one configuration: FIGURE 28: EXAMINING RESULTS FOR DIFFERENT CONFIGURATIONS CONCLUSION Wow. That was a lot. We started off by providing an overview of agile and the scrum process. After defining some common terms in scrum such as roles and sprints, we showed some typical agile test management tasks using TFS. SOME OF THE TASKS WE DEMONSTRATED INCLUDED:» Create Backlog Items in TFS» Create Tasks in TFS» Creating Test Cases in TFS» Using Shared Steps Common Actions for Test Cases» Link Test Cases with Tasks» Associate Automation Script» Create Test Plan» Execute Test Cases» Checking Results We ve shared some lessons learned along the way via tips and tricks that we ve discovered. We hope that you ve found this brief tutorial helpful. If you have any questions or comments, please let us know via ; services@xbosoft.com. 8

20 ABOUT XBOSOFT XBOSoft is a full-service software and application testing company serving both the US and European markets. With a proven track record working for Fortune 00 companies to small independent developers, XBOSoft delivers a full range of software testing services. XBOSoft s testing services range from Functional testing to Security and Mobile Usability and UX consultation with collaboration tools for secure 24/7 access to project information. CONTACT XBOSOFT WEBSITE: services@xbosoft.com