About the Tutorial. Audience. Prerequisites. Copyright & Disclaimer STLC

Transcription

1 i

2 About the Tutorial Software Testing Lifecycle is a standard procedure divided into different phases, followed by the QA Team to complete all testing activities. This is a brief tutorial that introduces the readers to the various phases of Software Testing Life Cycle. Audience This tutorial has been prepared for beginners to help them understand the software testing lifecycle. It would help all those professionals who would like to understand the Testing Framework in detail along with its types, methods, and levels. Prerequisites There are no specific prerequisites for this tutorial. Any software professional can go through this tutorial to get a bigger picture of how the high-quality software applications and products are designed. However, it would certainly be an advantage if the readers have a basic of understanding of some fundamental testing concepts. Copyright & Disclaimer Copyright 2017 by Tutorials Point (I) Pvt. Ltd. All the content and graphics published in this e-book are the property of Tutorials Point (I) Pvt. Ltd. The user of this e-book is prohibited to reuse, retain, copy, distribute or republish any contents or a part of contents of this e-book in any manner without written consent of the publisher. We strive to update the contents of our website and tutorials as timely and as precisely as possible, however, the contents may contain inaccuracies or errors. Tutorials Point (I) Pvt. Ltd. provides no guarantee regarding the accuracy, timeliness or completeness of our website or its contents including this tutorial. If you discover any errors on our website or in this tutorial, please notify us at contact@tutorialspoint.com i

3 Table of Contents About the Tutorial... i Audience... i Prerequisites... i Copyright & Disclaimer... i Table of Contents... ii 1. STLC OVERVIEW COMPARISON STLC AND SDLC TESTING FUNDAMENTAL PRINCIPLES STLC REQUIREMENT ANALYSIS... 6 Activities Performed for Requirement Analysis STLC ENTRY AND EXIT CRITERIA... 8 Entry Criteria... 8 Exit Criteria STLC ACCEPTANCE CRITERIA STLC TEST PLANNING Aspects of the Test Planning Phase STLC TEST CASE DEVELOPMENT Activities in the Test Case Development Phase Activity Block Diagram STLC TEST ENVIRONMENT SETUP Activities Performed for Test Environment Setup ii

4 10. STLC TEST EXECUTION Activities for Test Execution Activity Block Diagram STLC DEFECT LIFE CYCLE STLC DEFECT CLASSIFICATION What is Priority? Priority Listing What is Severity? Severity Listing STLC TEST CYCLE CLOSURE iii

5 1. STLC Overview STLC STLC stands for Software Testing Life Cycle. STLC is a sequence of different activities performed by the testing team to ensure the quality of the software or the product. STLC is an integral part of Software Development Life Cycle (SDLC). But, STLC deals only with the testing phases. STLC starts as soon as requirements are defined or SRD (Software Requirement Document) is shared by stakeholders. STLC provides a step-by-step process to ensure quality software. In the early stage of STLC, while the software or the product is developing, the tester can analyze and define the scope of testing, entry and exit criteria and also the Test Cases. It helps to reduce the test cycle time along with better quality. As soon as the development phase is over, the testers are ready with test cases and start with execution. This helps to find bugs in the initial phase. STLC Phases STLC has the following different phases but it is not mandatory to follow all phases. Phases are dependent on the nature of the software or the product, time and resources allocated for the testing and the model of SDLC that is to be followed. Requirement Analysis Test Planning Test Cases Designing Test Environment Setup Test Execution Test Closure Defect Reporting There are 6 major phases of STLC: Requirement Analysis When the SRD is ready and shared with the stakeholders, the testing team starts high level analysis concerning the AUT (Application under Test). Test Planning Test Team plans the strategy and approach. Test Case Designing Develop the test cases based on scope and criteria s. Test Environment Setup When integrated environment is ready to validate the product. Test Execution Real-time validation of product and finding bugs. Test Closure Once testing is completed, matrix, reports, results are documented. 1

6 2. Comparison STLC and SDLC STLC In this chapter, we will understand the factors of comparison between STLC and SDLC. Let us consider the following points and thereby, compare STLC and SDLC. STLC is part of SDLC. It can be said that STLC is a subset of the SDLC set. STLC is limited to the testing phase where quality of software or product ensures. SDLC has vast and vital role in complete development of a software or product. However, STLC is a very important phase of SDLC and the final product or the software cannot be released without passing through the STLC process. STLC is also a part of the post-release/ update cycle, the maintenance phase of SDLC where known defects get fixed or a new functionality is added to the software. The following table lists down the factors of comparison between SDLC and STLC based on their phases: Phase SDLC STLC Requirement Gathering Business Analyst gathers requirements. Development team analyzes the requirements. After high level, the development team starts analyzing from the architecture and the design perspective. Testing team reviews and analyzes the SRD document. Identifies the testing requirements - Scope, Verification and Validation key points. Reviews the requirements for logical and functional relationship among various modules. This helps in the identification of gaps at an early stage. Design The architecture of SDLC helps you develop a high-level and low-level design of the software based on the requirements. Business Analyst works on the mocker of UI design. Once the design is completed, it is signed off by the stakeholders. In STLC, either the Test Architect or a Test Lead usually plan the test strategy. Identifies the testing points. Resource allocation and timelines are finalized here. 2

7 Development Development team starts developing the software. Integrate with different systems. Once all integration is done, a ready to test software or product is provided. Testing team writes the test scenarios to validate the quality of the product. Detailed test cases are written for all modules along with expected behaviour. The prerequisites and the entry and exit criteria of a test module are identified here. Environment Set up Testing Development team sets up a test environment with developed product to validate. The actual testing is carried out in this phase. It includes unit testing, integration testing, system testing, defect retesting, regression testing, etc. The Development team fixes the bug reported, if any and sends it back to the tester for retesting. UAT testing performs here after getting sign off from SIT testing. The Test team confirms the environment set up based on the prerequisites. Performs smoke testing to make sure the environment is stable for the product to be tested. System Integration testing starts based on the test cases. Defects reported, if any, get retested and fixed. Regression testing is performed here and the product is signed off once it meets the exit criteria. Deployment/ Product Release Maintenance Once sign-off is received from various testing team, application is deployed in prod environment for real end users It covers the post deployment supports, enhancement and updates, if any. Smoke and sanity testing in production environment is completed here as soon as product is deployed. Test reports and matrix preparation are done by testing team to analyze the product. In this phase, the maintaining of test cases, regression suits and automation scripts take place based on the enhancement and updates. 3

8 3. Testing Fundamental Principles STLC The common objective of testing is finding bugs as early as possible. And, once the bugs are fixed, make sure it is working as expected and not breaking any other functionality. To achieve these goals, seven basic principles are given for software testing: What Testing shows? Testing can show that defects are present but there is no way to prove that there is no defect in the product. Testing phases make sure that the application under test is working based on the given requirement and it helps to reduce the probability of undiscovered defects in the application. But, even if no defects are found, it does not mean that it is absolutely correct. We can assume that AUT is matching with our exit criteria and maintaining the requirements according to SRD. Is Exhaustive Testing possible? 100% coverage or testing of all combinations of inputs and possible combinations are not possible except of trivial cases. Instead of exhaustive testing, risk analysis and priorities are used to define the scope of testing. Here, most of the real time scenarios can consider including most probable negative scenario as well. This will help us track the failure. Early Testing Testing activities should start as soon as possible and be focused on defined objectives in Test Strategy and expected results. Early stage of testing helps to identify Requirement Defect or design level discrepancies. If these types of bugs are captured in initial stage, it helps us save time and is cost-effective too. The answer to why testing should start at an early stage is very simple as soon as the SRD is received, the testers can analyze the requirement from the testing perspective and can notice a requirement discrepancy. Defect Clustering Based on previous product defect analysis, it can be said that most of the defects are identified from small set of modules which are critical for application. These modules can be identified based on complexity, different system interaction or dependency on different other modules. If testers can identify these crucial modules, they can focus more on these modules to identify all possible bugs. In a study, it is found that 8 out of 10 defects are identified from 20% functionality of AUT. Pesticide Paradox What is pesticide paradox if pesticides are frequently used on crops, there comes when the insects develop a certain kind of resistance and gradually the pesticides thus sprayed seem to be ineffective on the insects. The same concept is applicable on testing also. Here, insects are bugs while pesticides are test cases that are used to run again and again. If the same sets of test cases are executed again and again, these test cases become ineffective after certain timeframe and the testers will not be able to identify any new defect. 4

9 To overcome these conditions, test cases should be revised and reviewed time to time and new and different test cases can be added. This will help in identifying new defects. Testing is Context Dependent This principle states that two different type of application can t be tested using same approach until both applications are of same nature. For example, if a tester uses the same approach for Web Based Application and Mobile Application, that is completely wrong and there is high risk of poor quality of product release. Testers should use different approaches, methodologies, techniques and coverage for different types and nature of applications. Absence of Error Fallacy This principle states finding defects and fixing them until the application or system is stable, is time consuming and also eats up on the resources. Even after fixing 99% of the defects, there is a high risk of unstable application. The first essential thing is to verify the stability of the application and the prerequisites of the environment. If these two conditions fulfill, only then we can start with the detailed testing. 5

10 4. STLC Requirement Analysis STLC Requirement Analysis is the first phase of STLC and it starts as soon as the SRD/SRS is shared with the testing team. Let us consider the following points to understand the Requirement Analysis in STLC. The entry criteria of this phase is the provision of SRS (Software Requirement Specification); it is also recommended that the application architecture is handy. In this phase, the QA team analyzes at a higher level what to test and how to test. The QA team follows up with various stakeholders like Business Analyst, System Architecture, Client, Test Manager/Lead in case any query or clarification is required to understand the requirement. Requirements may be functional or non-functional like performance, security, usability, etc. or both functional and non-functional. The exit criteria of this phase is to complete the RTM document, automation feasibility report and a list of questions if applicable to be more specific on the requirements. Activities Performed for Requirement Analysis There are three major activities that are performed by the QA team in this phase. The activities have been described below. Defining the Scope The QA team identifies the scope of testing at high levels and divides into various functional modules. The team also identifies the types of testing required to perform smoke testing, sanity testing, functional testing, regression testing, etc. The QA team analyzes the prerequisites and the environment details where testing is supposed to be performed. The team gathers details about the testing priorities and lays focus on the sequence of modules to be validated. It also identifies the requirement defects if modules are contradicted and functionality is not getting carried over along with other modules. Prepare RTM Requirements tracing is a process of documenting the links between the requirements and the work products developed to implement and verify those requirements. The RTM captures all the requirements at the Requirement Analysis along with their traceability in a single document. All of this is delivered at the conclusion of the life cycle. The Matrix is created at the very beginning of a project as it forms the basis of the project's scope and deliverables that will be produced. The Matrix is bidirectional, as it tracks the requirement forward by examining the output of the deliverables and backward by looking at the business requirement that was specified for a particular feature of the product. 6

11 Automation Analysis In the requirement phase, the QA team analyzes the scope of automation for regression testing. If automation is added in scope, the team decides which tool can be used, what functionalities will be covered as automation, the time-frame and the resource allocation involved for automation development. Once this analysis is completed, the QA team provides the Automation Feasibility Report to different stakeholders to provide signoff. 7

12 5. STLC Entry and Exit Criteria STLC In this chapter, we will see the Entry and Exit Criteria at different levels in STLC. The following points need to be considered to understand the criteria. Ideally, the QA team does not proceed with the next phase until the exit criteria of the current phase meets. The entry criteria should include the completion of exit criteria of the previous phase. In real time, it is not possible to wait for the next phase until the exit criteria is met. Now, the next phase can be initiated if the critical deliverables of the previous phase have been completed. In each phase of STLC, the entry and exit criteria should be defined. Entry Criteria Entry Criteria for STLC phases can be defined as specific conditions; or, all those documents which are required to start a particular phase of STLC should be present before entering any of the STLC phase. Entry criteria is a set of conditions that permits a task to perform, or in absence of any of these conditions, the task cannot be performed. While setting the entry criteria, it is also important to define the time-frame when the entry criteria item is available to start the process. For Instance, to start the Test Cases development phase, the following conditions should be met: The requirement document should be available. Complete understanding of the application flow is required. The Test Plan Document should be ready. Exit Criteria Exit Criteria for STLC phases can be defined as items/documents/actions/tasks that must be completed before concluding the current phase and moving on to the next phase. Exit criteria is a set of expectations; this should be met before concluding the STLC phase. For Instance, to conclude the Test Cases development phase, following expectations should be met: Test Cases should be written and reviewed. Test Data should be identified and ready. Test automation script should be ready if applicable. 8

13 6. STLC Acceptance Criteria STLC Acceptance criteria means the expected behavior of a functionality, module and application as listed in the requirement documents. It is verification stages/checkpoints to determine whether or not the software system has met the requirement specifications. The main purpose of this test is to evaluate the system's compliance with the business requirements and verify if it has met the required criteria. Acceptance Criteria is a set of statements, that mentions clearly about the expected pass/fail result. Acceptance criteria specifies both functional and non-functional requirements. These requirements represent conditions of satisfaction or expected behaviour. There is no partial acceptance; either a criterion is met or it is not met. These criteria defines the boundaries and parameters of a functionality/module and determines whether functionality/module is completed and is working as expected. High Level Acceptance criteria is mentioned at the Test Plan Level. The acceptance criteria is converted to a list of points to be verified or expected results at the test cases level. Acceptance criteria is defined on the basis of the following attributes: Functional Correctness and Completeness Data Integrity Data Conversion Usability Performance Timeliness Confidentiality and Availability Install and Upgrade ability Scalability Documentation 9

14 7. STLC Test Planning STLC A test plan outlines the strategy that will be used to test an application, the resources that will be used, the test environment in which testing will be performed, and the limitations of the testing and the schedule of the testing activities. Typically, the Quality Assurance Team Lead will be responsible for writing a Test Plan. What does a Test Plan Include? A Test Plan includes the following. Introduction to the Test Plan document. Assumptions while testing the application. List of test cases included in testing the application. List of features to be tested. The sort of approach to be used while testing the software. List of deliverables that need to be tested. The resources allocated for testing the application. Any risks involved during the testing process. A schedule of tasks and milestones to be achieved. Important Points for Test Planning The following points need to be considered for Test Planning in STLC. Ideally, the Test Analyst (Lead)/the Manager prepares the Test Strategy/Test Plan Document. Analysis is more focused on application related data/information. It is the first phase of actual testing tasks. This phase answers WHAT is to be tested and WHAT RESOURCES are required to test. The basic entry criteria of this phase is provision of Requirement Documents (updated version of unclear/missing/clarified requirements) along with Requirement Traceability Matrix. If automation is in scope, Automation Feasibility Report should be prepared before entering in this phase. The exit criteria of this phase is completion of Test Strategy/Test Plan Document and Test effort Estimation document. 10

15 Aspects of the Test Planning Phase The main objective of this phase is to prepare a Test Plan/Test Strategy document. It includes three major aspects Scope of Deliverables, Effort estimation and Resource Plan. Scope of Deliverables Following activities need to be performed to conclude over the scope of deliverables: Identify suitable engagement and delivery model. Define test objectives, scope of testing, testing phases and activities. Review business requirement and system requirement to identify test feasibility. Define testing process, type of testing and procedures. Define defect management and change management procedures. Identify testing tools, techniques and best practices. Define Risk Analysis. Define automation solution and identify suitable candidates for automation if applicable. Effort Estimation Estimation is the process of finding an estimate, or approximation, which is a value that can be used for some purpose even if input data may be incomplete, uncertain, or unstable. Estimation determines how much money, effort, resources, and time it will take to build a specific system or product. Estimation is based on: Past Data/Past Experience Available Documents/Knowledge Assumptions Identified Risks The four basic steps in Testing Estimation are: Estimation of the size of the AUT (Application Under Test). Estimation of the effort in person-months or person-hours. Estimation of the schedule in calendar months. Estimation of the project cost in agreed currency. 11

16 Resource Plan Resource plans are the key element in testing phases. These plans are inversely proportional to the time taken by the testing team to complete a particular task. Increasing the number of resources will decrease the number of days of completion for a certain limit after that it will be saturated and increasing the resource will not have much impact and might not lead to a decrease in the completion days. A Resource Requester, usually a project manager, creates resource plans to ask for resources, track efforts and costs. A Resource manager can modify and approve resource plans before the plans are used. The normal workflow for a resource plan is: Planning by Project Manager Request raised by Project Manager Approve/Modify/Reject by Resource Manager Complete Closing the request after sign off by Resource Manager 12

17 8. STLC Test Case Development STLC Once the Test Plan is ready, the QA Team initiates the development of test cases. The main objective of this phase is to prepare test cases for an individual unit. These functional and structural test cases cover the functionality, points of verification and validation mentioned in the Test Plan. The following points need to be considered for Test Case Development in STLC. In this phase, the QA team writes the test case with a stepwise approach. The Test Case is then signed off by a Business Analyst after reviewing or rework on test cases in case modifications are required. Once test cases are ready, the QA team prepares the Test Data based on preconditions. The entry criteria of this phase is that the activities in test planning should be finished and the test plan should be ready. The exit criteria of this phase is that the test cases should be signed off, the test data should be ready and the test scripts prepared if automation is in scope. Test cases should be mapped with the Requirement Traceability Matrix to follow up on coverage of requirements if anything is missed. Activities in the Test Case Development Phase Following are the three activities that are carried out in the Test Case Development phase: Test Scenarios Identification Scenarios ease the testing and evaluation of a complex system. The following strategies help in creating good scenarios: Enumerate possible users, their actions and objectives. Evaluate users with hacker's mindset and list possible scenarios of system abuse. List the system events and how does the system handle such requests. List benefits and create end-to-end tasks to check them. Read about similar systems and their behavior. Studying complaints about competitor's products and their predecessor. Test Cases Writing A test case is a document, which includes test data, preconditions, expected results and post conditions, developed for a particular test scenario in order to verify compliance against a specific requirement. 13

18 Test Case acts as the starting point for test execution. After the a set of input values is applied; the application has a definitive outcome and leaves the system at some end point which is also known as execution post condition. Test Data Preparation Test Data is used to execute the tests on test ware. Test data needs to be precise and exhaustive to uncover the defects. To accomplish these three objectives, it is followed by a stepwise approach as given below: Identify test resources or requirements Identify conditions/functionality to be tested Set priority test conditions Select conditions for testing Determine expected result of processing of test cases Create Test cases Document test conditions Conduct test Verify and correct test cases based on modifications Activity Block Diagram The following diagram shows the different activities that form part of Test Case Development. Use cases and requirements from Client Understand the requirements Define the approach and number of test cases Develop detailed test procedures for scenarios Send query log for clarifications Identify high level scenarios for each test case Perform review on the completed test cases Send estimates and Plan for testing Send the scenarios for review Update review findings Prepare and map Traceability Matrix Deliver to client for signoff 14

19 9. STLC Test Environment Setup STLC Test Environment consists of elements that support test execution with software, hardware and network configured. Test environment configuration must mimic the production environment in order to uncover any environment/configuration related issues. The following points need to be considered in a Test Environment Setup. It is a combination of hardware and software environment on which the tests will be executed. It includes hardware configuration, operating system settings, software configuration, test terminals and other support to perform the test. It is the most crucial aspect of the testing process. Unavailability or faulty environment setup can ruin all the testing efforts. Practically, the QA team cannot start actual work without having the right environment to test. It is and independent activity and can be started along with test case development. Most generically, this activity is performed by developers in technical aspects while requirement conditions can be done by Customers/Business Analysts. Readiness of the test environment can be validated by smoke testing, and performed by the QA team. Ideally, we can say that the Entry Criteria of this phase is the provision of Test Plan, readiness of Smoke Test cases and preparation of test data. The exit criteria of this phase is that the test environment should be ready and smoke testing should be performed successfully with expected results. Activities Performed for Test Environment Setup The following activities are performed in this phase. Design Test Environment Following factors play an important role for the designing of a test environment. Determine if test environment needs archiving in order to take back-ups. Verify the network configuration. Identify the required server operating system, databases and other components. Identify the number of license required by the test team. 15

20 Set Up of the Environment Analyze the environment setup requirements and prepare a list of software and hardware requirements for the setup. Get the official confirmation for setup of the test environment and configure to access the test environment. Smoke Testing Once the environment is set up and the QA team has the access to it, a quick round of smoke testing should be performed for validation of test environment build stability. If the results are as expected, the QA team can move to the next phase else point out the discrepancies and wait for deployment after fixes. 16

21 10. STLC Test Execution STLC Test execution is the process of executing the code and comparing the expected and actual results. Following factors need to be considered for a test execution process: Based on a risk, select a subset of test suite to be executed for this cycle. Assign the test cases in each test suite to testers for execution. Execute tests, report bugs, and capture test status continuously. Resolve blocking issues as they arise. Report status, adjust assignments, and reconsider plans and priorities daily. Report test cycle findings and status. The following points need to be considered for Test Execution. In this phase, the QA team performs actual validation of AUT based on prepared test cases and compares the stepwise result with the expected result. The entry criteria of this phase is completion of the Test Plan and the Test Cases Development phase, the test data should also be ready. The validation of Test Environment setup is always recommended through smoke testing before officially entering the test execution phase. The exit criteria requires the successful validation of all Test Cases; Defects should be closed or deferred; test case execution and defect summary report should be ready. Activities for Test Execution The objective of this phase is real time validation of AUT before moving on to production/release. To sign off from this phase, the QA team performs different types of testing to ensure the quality of product. Along with this defect reporting and retesting is also crucial activity in this phase. Following are the important activities of this phase: System Integration Testing The real validation of product / AUT starts here. System Integration Testing (SIT) is a black box testing technique that evaluates the system's compliance against specified requirements/ test cases prepared. System Integration Testing is usually performed on subset of system. The SIT can be performed with minimum usage of testing tools, verified for the interactions exchanged and the behavior of each data field within individual layer is also investigated. After the integration, there are three main states of data flow: Data state within the integration layer Data state within the database layer Data state within the Application layer 17

22 Note: In SIT testing, the QA team tries to find as many defects as possible to ensure the quality. The main objective here is finding bugs as many as possible. Defect Reporting A software bug arises when the expected result doesn't match with the actual result. It can be an error, flaw, failure, or fault in a computer program. Most bugs arise from mistakes and errors made by developers or architects. While performing SIT testing, the QA team finds these types of defects and these need to be reported to the concerned team members. The members take further action and fix the defects. Another advantage of reporting is it eases the tracking of the status of defects. There are many popular tools like ALM, QC, JIRA, Version One, Bugzilla that support defect reporting and tracking. Defect Reporting is a process of finding defects in the application under test or product by testing or recording feedback from customers and making new versions of the product that fix the defects based on the client s feedback. Defect tracking is also an important process in software engineering as complex and business critical systems have hundreds of defects. One of the most challenging factors is managing, evaluating and prioritizing these defects. The number of defects gets multiplied over a period of time and to effectively manage them, defect tracking system is used to make the job easier. Defect Mapping Once defect is reported and logged, it should be mapped with the concerned failed/blocked test cases and corresponding requirements in Requirement Traceability Matrix. This mapping is done by the Defect Reporter. It helps to make a proper defect report and analyze the impishness in product. Once the test cases and requirements are mapped with the defect, stakeholders can analyze and take a decision on whether to fix/defer the defect based on priority and severity. Re-testing Re-testing is executing a previously failed test against AUT to check whether the problem is resolved. After a defect has been fixed, re-testing is performed to check the scenario under the same environmental conditions. During re-testing, testers look for granular details at the changed area of functionality, whereas regression testing covers all the main functions to ensure that no functionalities are broken due to this change. Regression Testing Once all defects are in closed, deferred or rejected status and none of the test cases are in progress/failed/no run status, it can be said that system integration testing is completely based on test cases and requirement. However, one round of quick testing is required to ensure that none of the functionality is broken due to code changes/ defect fixes. Regression testing is a black box testing technique that consists of re-executing those tests that have had an impact due to code changes. These tests should be executed as often as possible throughout the software development life cycle. 18

23 Types of Regression Tests: Final Regression Tests - A "final regression testing" is performed to validate the build that has not undergone a change for a period of time. This build is deployed or shipped to customers. Regression Tests - A normal regression testing is performed to verify if the build has NOT broken any other parts of the application by the recent code changes for defect fixing or for enhancement. Activity Block Diagram Following block diagram shows the important activities performed in this phase; it also shows the dependency from the previous phases: Sign off test cases Identify test data requirements Perform functional testing Perform Regression Testing Prepare test data and perform smoke testing Record/screenshots test results Record test results and log defects if any Log Defects Retest defect fixed Retest defect fixes Review test Results Deliver the results to stakeholders 19

24 11. STLC Defect Life Cycle STLC Defect Life Cycle, also known as Bug Life Cycle, is the journey of a defect, the cycle which a defect goes through during its lifetime. It varies from organization to organization and also from project to project, as it is governed by the software testing process and also depends upon the tools used. Defect Life Cycle Workflow The following diagram shows the workflow of a Defect Life Cycle. States of a Defect Life Cycle Following are the different states of a Defect Life Cycle. New Potential defect that is raised and yet to be validated. Assigned Assigned against a development team to be addressed. Active The Defect is being addressed by the developer and investigation is under progress. At this stage, there are two possible outcomes Deferred or Rejected. Test / Fixed / Ready for Retest The Defect is fixed and ready for testing. Verified The Defect that is retested and the test has been verified by QA. 20

25 Closed The final state of the defect that can be closed after the QA retesting or can be closed if the defect is duplicate or considered as NOT a defect. Reopened When the defect is NOT fixed, QA reopens/reactivates the defect. Deferred When a defect cannot be addressed in that particular cycle it is deferred to future release. Rejected A defect can be rejected for any of the three reasons duplicate defect, NOT a Defect, Non Reproducible. 21

26 12. STLC Defect Classification STLC Defects are classified from the QA team perspective as Priority and from the development perspective as Severity (complexity of code to fix it). These are two major classifications that play an important role in the timeframe and the amount of work that goes in to fix defects. What is Priority? Priority is defined as the order in which the defects should be resolved. The priority status is usually set by the QA team while raising the defect against the dev team mentioning the timeframe to fix the defect. The Priority status is set based on the requirements of the end users. For example, if the company logo is incorrectly placed in the company's web page then the priority is high but it is of low severity. Priority Listing A Priority can be categorized in the following ways: Low This defect can be fixed after the critical ones are fixed. Medium The defect should be resolved in the subsequent builds. High The defect must be resolved immediately because the defect affects the application to a considerable extent and the relevant modules cannot be used until it is fixed. Urgent The defect must be resolved immediately because the defect affects the application or the product severely and the product cannot be used until it has been fixed. What is Severity? Severity is defined as the impishness of defect on the application and complexity of code to fix it from development perspective. It is related to the development aspect of the product. Severity can be decided based on how bad/crucial is the defect for the system. Severity status can give an idea about the deviation in the functionality due to the defect. Example: For flight operating website, defect in generating the ticket number against reservation is high severity and also high priority. 22

27 Severity Listing Severity can be categorized in the following ways: Critical /Severity 1 Defect impacts most crucial functionality of Application and the QA team cannot continue with the validation of application under test without fixing it. For example, App/Product crash frequently. Major / Severity 2 Defect impacts a functional module; the QA team cannot test that particular module but continue with the validation of other modules. For example, flight reservation is not working. Medium / Severity 3 Defect has issue with single screen or related to a single function, but the system is still functioning. The defect here does not block any functionality. For example, Ticket# is a representation which does not follow proper alpha numeric characters like the first five characters and the last five as numeric. Low / Severity 4 It does not impact the functionality. It may be a cosmetic defect, UI inconsistency for a field or a suggestion to improve the end user experience from the UI side. For example, the background colour of the Submit button does not match with that of the Save button. 23

28 13. STLC Test Cycle Closure STLC A check against the test exit criteria is very essential to claim that the testing is now complete. Before putting an end to the test process, the product quality is measured against the test completion criteria. The entry criteria of this phase is that the execution of the test case is complete, test results are available and the defects report is ready. The criteria for test completion includes the following: Specified coverage has been achieved. No showstoppers or critical defects. There are very few known medium or low-priority defects. These do not affect the usage of the product. The exit criteria of this phase is the provision of test closure reports and preparation of matrices which are later signed off by the client. Let us now discuss the activities involved in the closure of Test Cycle. Test Completion Report Test completion reporting is a process, whereby the test metrics are reported in summarized format to update the stakeholders. This enables them to take an informed decision. Test Completion Report contains the following information. Test Summary Report Identifier Summary Variances Summary Results Evaluation Planned vs Actual Efforts Sign off A good Test Completion Report indicates quality, measures outstanding risks, and identifies the level of a tested software. 24

29 Test Completion Matrix Upon completion of testing, various matrices are collected to prepare the test reports. The criteria for preparing the reports includes the following: Number of Tests Executed Number of Tests Passed Number of Tests Failed Number of Test Failed based on each module Number of Test Defects Raised during the execution cycle Number of Test Defects Accepted Number of Test Defects Rejected Number of Test Defects Deferred Status of Active defects Calculating Quality Index of the Build Test Results While executing a test case, re-testing defects and performing regression test case, Test results articulate should be saved and can be produced along with the test cycle closure documents to support the completion of test execution. Articulates can be screenshots, database query results, recording, log files, etc. 25