White Paper. Driving Efficiency through Test Life Cycle Metrics

Transcription

1 White Paper Driving Efficiency through Test Life Cycle Metrics

2 2 Abstract Issues in products arise because of defects in live production. Testing of the product in the early stages will help to identify and fix the issues for a stable product. Testing is an important phase in the Software Development Life Cycle (SDLC), testing of the product is one of the ways of ensuring business assurance, hence ensuring proper and efficient testing enhances the efficiency through metrics is of paramount importance not only to IT but to also for business continuity and availability of the business application.this is also being recognized increasingly by CTO of leading organizations. This paper addresses some of the ways through which testing metrics can help improve overall product efficiency. Objective The objective of this white paper is to identify the key software testing metrics that could be used in measuring defects during software testing. Introduction Metrics are gaining importance and acceptance as organizations mature and they strive to improve enterprise quality. Process measurement and analysis, utilization of qualitative methods for quality management is very important. We can use the information to define the base line of the metrics to help determine what information can be tracked, who will track the information and at what frequency. There is a need to implement the required processes necessary to effectively track, calculate, manage and interpret the defined metrics. This will help to define the areas of improvement. Importance of software test life cycle metrics Test metrics are key information that the project managers use to understand the current position and to prioritise their activities to reduce the risk of schedule over runs on software releases. Test metrics can be used as an important risk management tool, which helps to measure the status of the performance, project and delivery. The project tracking is done by recording the actual effort and time spent on each task and compares it to the actual estimates. The rate at which the team is able to find a defect and the time taken to fixing the defects determines the efficiency of testing as well as if they can ship the product to the client. Metrics help to better control the software projects and do a Process Capability Baseline to be used for better estimate and to predict the quality of the projects in the future.

3 3 Approach for software test life cycle metrics (A) Identify: The important aspect for metrics is to identify and define the process required to capture data Identifying the right metric Define and Classify the various metrics Identify the data for metrics and process required to capture data

4 4 Test Process Flow Automatetesting wherever required Defect Tracking and Analysis Request for Testing Effort & Schedule Estimation Plan the test strategies Test Plan Preparation Test case Preparation Design Test Cases Test Automation Regression Testing Run the test cases and report Bugs Defect Analysis To confirm no side effect due to bug Project Closure Final Report Project Initiation (B) Plan: - The types of metrics and the process required for capturing the required data Base Metrics Metrics for which data can be captured directly (Cost, efforts, schedule, etc) Derived Metrics - Productivity, Quality etc. Cost Variance Cost Variance Metrics help in arriving at the project s cost in its reference to the planned and budgeted cost. This is an important metric as this helps us at deriving the ROI on the project. The formulae for assessing cost variance are: CV = 100*(AC-BC)/BC, where AC is expanded as Actual cost and BC is Budgeted Cost. Budgeted cost - If the cost variance is Positive, this is an indicator for the project management that the project costs are shooting the planned expenditure. Effort Variance: - Effort variance helps in identifying the effort overrun in the project. The effort variance has a direct relation to the cost variance, as it is an indicator of the actual cost in comparison with the actual cost, which also relates to the allocation of the resources. EV can be measured as: EV= 100*(AE-BE)/BE Where AE is Actual Expenditure and BE is Budgeted Expenditure.

5 5 When the result is negative, this is a good sign as the effort is being well managed within the project. Schedule: This metric valuates the project management s Planning, Scheduling, and Tracking techniques. It indicates that the project performance of planned activities is in accordance to the project schedule. SV can be measured by SV = 100* (AD-ED) ED Where AD is Actual Duration and ED is Estimated Duration When the result happens to be positive, this is a clear indication that the project is behind schedule and negative means we are ahead of schedule. (C) Implement: The method adopted to implement the methodology of Test Life Cycle Metrics Capture and refine data Verify and Analyze the data Communicate frequency to all the stakeholders Reports Defect Classification The defects are in general classified in some of the types as below. Defect Metrics

6 6 An analysis of the metrics helps us to understand the effectiveness of the testing process and test execution. Test progress tracking metrics This metric tracks the cumulative test cases or test points that are planned, attempted, and successful over the test execution period. Defects by injection phase This metric is an indicator of the most problematic phase in the software development Defects by detection phase This metric is an indicator of the most efficient phase of software development in terms of finding and removing bugs.

7 7 Defects by Priority and Severity The defects by priority and severity help to understand the quality of the software, this in turn help to decide on the release of the software. If the defects severity is high on critical priority, the software should not be released or have a strong basis of reasoning for the defect to be waived off. A caveat sign off should also be okay. Defect by Cause This metric helps the development team and test teams to focus on areas for improvement. The graph will show the various elements under which the defects are categorized. As a project manager, he can focus on areas of improvement. (D) Continuous Improvement for Software Testing: The Software life cycle testing means that testing occurs in parallel with the development cycle and is a continuous process. The software testing process should start early in the application life cycle and not

8 8 just in the traditional validation-testing phase after the coding phase has been completed. Testing should be integrated into application development. The organization should adopt continuous improvement and this needs a commitment from the Management to integrate the testing and development teams for close communication with the quality assurance function. Conclusion Developing effective metrics may appear easy at first glance, but many have fallen into common traps that can be avoided. Some of the common pitfalls involve Developing metrics for which you cannot collect accurate or complete data. Developing metrics that measure the right thing, but cause people to act in a way contrary to the best interest of the business to simply "make their numbers." Developing so many metrics that you create excessive overhead and red tape. Developing metrics that are complex and difficult to explain to others. Metrics should be simple. If they require a lot of explanation and definition, then collecting data and translating that data into actions becomes more difficult. Easy-to-understand metrics are easier to sell, and have a stronger impact on the process and the people who us it. References Metric Based Approach for Requirements Gathering and Testing Read Part 1: Guide to Effective Test Reporting. Read 2nd Part: Guide to Metrics Collection in Software Testing (includes Benefits of implementing metrics in software testing)

9 About 3i Infotech 3i Infotech ( provides software products and IT services for the Banking, Financial Services & Insurance (BFSI); Manufacturing, Retail & Distribution; and Government verticals. The Company's quality certifications include ISO 9001:2008 for BPO, ISO/IEC 27001:2005 for Data Center Operations and ISO/IEC :2005 for Data Center Management Services. The Company has embarked on re-appraisal under SEI CMMI ML 5 for some of its software development Centers in India. By using its domain knowledge and through continuous investment in technologies, 3i Infotech enables corporations transform their businesses through its expertise in enterprise-class software solutions, software services including Testing Services, information security consulting, system integration services, IT infrastructure and Disaster Recovery solutions. For more information, contact: marketing@3i-infotech.com or visit Copyright 3i Infotech All rights reserved.