WHITE PAPER. Building potential QA Teams - A no cost Metrics based Approach PRESENTED IN STC 2013 PRESENTED BY. Building Num 14 Building Num 14

Transcription

1 WHITE PAPER ON Building potential QA Teams - A no cost Metrics based Approach PRESENTED IN STC 2013 PRESENTED BY Uppuluri Pranitha B.V.S.S.R.S.SASTRY syamala_p_uppuluri@uhc.com sastry_bvssrs@uhc.com United Health Group United Health Group Building Num 14 Building Num 14 Raheja Mind Space, HitechCity Raheja Mind Space, HitechCity Hyderabad (AP) Hyderabad (AP)

2 Page-1 Abstract To build a robust defect free product is the endeavor of every IT organization. This helps minimize the cost of building and maintaining the products, as well as maximize customer retention. However, despite QA team s best techniques and efforts to build a defect free product, due some invisible factors defect leakages occur. If unnoticed, then they become potential threat to the organization. Building automation tools, training the resources on tools or techniques to minimize defect leakages is cost effective and is also not the only solution to discover these invisible hurdles. There are other inexpensive strategies which can help in identifying the invisible hurdles and extracting true potential of the QA teams. A metrics based approach which we identified is an inexpensive optimal solution, which enlists the factors to be considered and how to build an efficient tracking mechanism. Keywords: QA Team, Metrics, Software Testing. I. INTRODUCTION Software Testing is the Crucial Phase for any Product as it is the essential phase which ensures the quality of the Product and helps in determining whether it meets the desired results. Assessing Product Quality is a important phase as QA team needs to understand requirements (Requirement Analysis), build efficient Test Plan, Assign Resources who are well aware of the Functionality of Requirements specified, Identify Reusable Components if any, Provide Proper Test Scripts which ensures each and every functionality is covered and execute the test scripts which ensures Defectfree product which can be released live into the market. If any product is not meeting Quality Standards, then the product cannot be released into the market. Let us see in detail in this paper what are the myths related to Software Testing, how QA team is organized and what are the efficient ways in which we can build the team. II. AREAS WHICH NEEDS TO BE FOCUSED: Working across various releases and functionalities, QA team need to identify various areas which need improvement. Some of the Common areas which require attention are as follows: Concentrate towards building Efficient Test Metrics Evaluating Complexity of the application Tracking QA Metrics such as Defect Leakage, Non Value Added Defects etc., Evaluation Complexity of the releases per year Tracking of how many builds Requirements freeze time Well Established QA Environment We need to identify the metrics for the above areas so that we can track the effectiveness of the QA team based on that.

3 Page-2 III. Metrics for Building Potential QA Teams: S Let us see in brief the metrics for building Potential QA Teams. a) Establishing Process Dependent QA Team Let us see the metrics for establishing process dependent QA Team as follows: Skill Matrix: A skill matrix is a table that matches personnel, or other resources, with desired skills to provide views of the need for additional development, training or the acquisition of new resources. Skill Matrix Contains the following attributes: o Experience o Soft Skills, Testing Skills, Domain Skills o Process Maturity Identifying Core Group (SME) and Project Group (Resources): Based on the Skill Matrix, divide the team into two groups: Core Group Team and Project Group. Core Group consists of SME members and Project Group consists of Project Resources, Shadow Resources and Loaner Resources. SME members need to be classified as More SME s, Sufficient amount of SME s, No SME in the team, Fresher as an SME. Roles & Responsibilities should be defined separately for Core Group and Project Group by Project Manager. Knowledge Repository: Knowledge Repository need to be updated on regular basis and need to be audited by the Project Leads. Knowledge Repository should be created by the Core Group if it does not exist. Knowledge Repository should be maintained with Application Related Documents and Web-ex Recordings maintained by the Core Group and Project Group Should upload all the documents related to the Project which they work on. b) Establish effective Test management Let us see the metrics for building effective test management as follows: Test Management: Test Management is the activity through which we analyze the requirements. Metrics that can be used to Track Test Management is Requirement Traceability Matrix (RTM). Test Management Metrics also include Test Case Efficiency, Cost of Quality. Test Estimation: Software Test Estimation is the process of Providing Estimations based on previous projects, Requirements, Number of Resources available for testing, Number of test components to be executed, and Complexity of test Cases. Functional Point Analysis (FPA) is one of the metric used to track Test Estimation. Test Lead is the responsible person for doing Test Estimation, according to the proposed model SME should be also involved in this phase. Test Coverage: Test Coverage is the process of meeting the requirements. It can be tracked through peer reviews, Internal Walkthroughs.

4 Page-3 Identifying Re-usable Test Artifacts: Identifying re-usable Test Artifacts is the essential phase where we identify Test Artifacts of previous cycles/releases and essentially re-use them. Identifying Various tools and Optimization Models that fit: Identifying various tools and Optimization Models plays a major role where we prioritize the requirements which help to execute test cases on priority basis. c) Tracking Defect Metrics: Let us see the metrics for tracking Defects as follows: Defect Leakage: Defect leakage is calculated as below Defect leakage = (No of defects found on the current phase/total defects including the previous phase) *100 Defect Removal Efficiency: DRE = Number of defects found during software testing / Number of defects found during software testing + Number of Defects found by User. Defect Density: Defect Density = Number of Known Defects / Size. Size is expressed in terms of Function Points (FP). Non Value Added Defects: Non Value Added Defects are the Defects which have been rejected such as Documentation Change, Environment Cause, Improper understanding of Requirements. d) Evaluating Complexity of the Releases Per Year: Release Cycle: It comprises of various stages planning, developing,testing, alpha/beta and final deployment. Every major version of a software product passes through all these stages before being deployed to the production environment. Deployment Plans: Deployment Plans are done during Estimation Phase, which will be modifying across Test Coverage. Based on the Test Coverage, Release will be decided either as Go/ No Go. Post Shipment Defects: Tracking of Post Shipment Defects needs to be evaluated and need to be tracked which needs to be maintained as a part of Lessons Learnt, so that it helps for the resources not to miss the functionality for the next release. e) How Many Builds Requirements Freeze Time:

5 Page-4 Count of builds per release: Count of builds per release needs to be tracked as it needs to be in limited number per release. If Count of builds per release increases, the amount of regression testing also increases. Requirement Tracking: Requirement Tracking is the mechanism where understanding of the Requirements is evaluated based on Peer Reviews and Walkthroughs. Requirement Freeze time: Requirement Freeze Time needs to be defined prior to Test Execution, if the Requirements need to be modified during Test Execution Change Control should be raised which can be handled during next release or a break fix needs to be issued for the change control. Code Freeze Time: Code Freeze Time needs to be defined during Test Execution Phase. After undergoing Testing across Various Cycles, Code Should be freezed during Freeze Cycle. Any Code Changes to be done during Freeze Cycle should be raised a Change Control Which should be handled during next release or a break fix to be issued. IV. Establishing Process Dependent QA Team Reiterating SME teams Model Let us see in detail the Process Model for Establishing Process Dependent QA Team Reiterating SME Teams. Figure 1 depicts the Process Model in three phases, (i) Present Scenario, (ii) Proposed Process Model, (iii) Post Implementation of Proposed Process Model Present Scenario: Let us see in detail how the Process is organized in the Present Scenario. Team Composition: The Team Composition Comprises of Project Manager who is responsible for the overall success of the team. He is majorly involved in resource planning, addressing project/team issues that hinder the testing efforts. Under Project Manager, there will be Project Lead/Test lead who is responsible for planning, keeping track of various testing actives and tasks. Project leads in collaboration with mangers and other project stakeholders create the test strategies, plans and policies. Under Project Lead there are Senior Test Engineer, Test Engineer and Associate Test Engineer whose responsibilities involve Understanding Requirements, Creating Test Scenario/Test Plan Documents, Executing Test Cases, Identifying Regression Scenarios and Executing them with the current functionality.

6 Page-5 Figure 1: Process Dependent QA team Reiterating SME Teams Architecture Team Skill Matrix: Team Skill Matrix is evaluated by Test Lead & Test Manager based on the Skills the resources perform across various Modules in the Application. In Figure1, We are depicting the scenario considering 15 people in the team where there are 3 SMEs (Subject Matter Experts) who have high proficient knowledge/skill level across various modules, Project Team Comprises of 10 People who have medium proficient knowledge/skill level across respective modules which they work with and 2 Loaner Resources who have low/medium proficient knowledge/skill level for the module they are working with. Loaner Resources are the resources who are working for other applications, but based on the bandwidth of the resource they will be moved to other application for a particular release. Knowledge Repository: Knowledge Repository is the document storage library for any application. In the Knowledge Repository, we do maintain description about various modules, their functionality, integration of modules, integration of current application with other applications etc., But at Present we are facing issues while maintaining Knowledge Repository such as No Proper standard defined for Repository and also for the Folder Structure, No Reusable Documents etc., Let us see how we are resolving these issues in the proposed Process Model. Proposed Process Model: In Proposed Process Model, we have Process Model Architecture Clearly defined along with Process Model in Action. Process Model Architecture:

7 Page-6 In the Process Model Architecture, we have Process Owner who initiates the Process Model, it may be the Director, Delivery Manager, Project Manager. Process Model Committee is identified to put Process Model in Action. The Committee Comprises of Process Owner, Project Leads & Above, Offshore Business Analysts & System Analysts. The Core Committee works closely to implement Process Model in Action in various phases such as Team Analysis, Roles & Responsibilities, Creating/Updating Knowledge Repository and Evaluation Phase respectively. Once the Process Model is established, Core Committee will review the Process Model and Formulate improvement Plans/Road Map. Figure 2: Time Lines, Benefits & Savings of Process Dependent QA team Reiterating SME Teams Architecture Figure 2 Shows the Time Lines, Benefits & Savings of Proposed Process Model. Time taken to initiate and form a Process Model Committee is 1 month. Time taken to Put Process Model into Action for the first time is 1 month. Phases of Process Model: We have various phases in Process Model. They are as follows: (i) Team Analysis Phase: In Team Analysis Phase, Project Manager analyses the team & classifies the team members into Core Team & Extended Team based on their skill matrix. Core Team Consists of SMEs, Project Manager responsibilities in establishing a Core Team are Identify Module SMEs, Cross Train between SMEs & Analyze Contributions made by SMEs such as Creation of Knowledge Repositories, Trainings Conducted etc., Extended Team Consists of Project Team & Loaner Resources, Project Manager responsibilities in establishing an Extended Team are Analyze Skill Set for Project Team & Loaner Resources and establish a plan for them to move

8 Page-7 to next levels. For Team Analysis Phase, time taken for first time is 1 month. (ii) Roles & Responsibilities Phase: In Roles & Responsibilities Phase, Roles & Responsibilities are defined by Process Committee for Core Team & Extended Team. Time taken for this phase for first time is 1 month. (iii) Knowledge Repository Phase: Knowledge Repository Phase is the crucial phase as it ensures that all the documents are created and maintained as per the standards and are updated on regular basis. (iv) Evaluation Phase: In Evaluation Phase, Process Model Committee evaluates Core Team & Extended Team and assign them roles & responsibilities based on their evaluation. During Evaluation Phase, Process Model Committee Formulate Improvement Plan & Road Map. After Implementing Process Model: After Implementing Process Model, SMEs in the team are increased in a huge manner. If in future, Existing SMEs move to other application/organization other resources can take over the responsibility as a part of defined process. V. Benefits of Implementing Process Model: 1) Responsibilities for each resource are clearly defined. 2) Resources follow a process to clarify issues, create documents etc., 3) Strong Documents are available 4) Standard Folder Structure 5) Organized, Reusable Documents/Knowledge Repository 6) Regular evaluation of teams progress 7) Regular feedback to improve team capabilities. VI. CONCLUSION: In this paper, we have presented various metrics and Process Dependent QA team Reiterating SME Teams through which we can track the QA Team and make it Process Dependent instead of People Dependent. VII. REFERENCES Software Engineering Institute, Carnegie Mellon, "CMMI-Development," Version 1.2, accessed February 11, 2010.

9 Page-8 3. The MITRE Institute, September 1, 2007, "MITRE Systems Engineering (SE) Competency Model," Version 1", Section 3.7, p. 45. VIII. AUTHOR S BIOGRAPHY Pranitha Uppuluri has obtained Masters of Commerce and Information Systems from Osmania University, Hyderabad, Andhra Pradesh, India. She has 3 Years of Experience in Software Testing. Her expertise includes web based and data base testing. She has worked in content management, E-Commerce and Health care domains. She is currently working as a Test Engineer in United Health Group, Hyderabad. B V S S R S Sastry has obtained Masters of Technology from JNTU, Hyderabad, Andhra Pradesh, India. He has published 15 International papers together on mobile computing and social networking. His expertise includes web technology and data base testing. He is currently working as Associate Test Engineer in United Health Group, Hyderabad. He has a total of 1.6 Years Experience in Software Testing.