Authors: M. Raveendran M. Paramesh Honeywell Technology Solutions

Transcription

1 Test Process Improvement Authors: M. Raveendran M. Paramesh Honeywell Technology Solutions

2 Abstract In this paper, we have described about the techniques/methods of test process improvements which can be applied in any type of application/product test life cycles. This has been already in use for different product lines and results are notable in our organization. We have also analyzed the effectiveness of these approaches which results in reducing the effort spent for each test cycles. We have covered those methods and best practices in this paper.

3 Summary Test Process Improvement Iterative process Test prioritization model Test tool management Test resource pool management Pros & Cons References

4 Test Process Improvement: Test process management is the process which helps to drive the complete road map of the testing cycle. Test process improvement (TPI) provides you with a framework to streamline and improve your testing process. In testing, the challenge is to find the fastest and most costeffective way to get the best testing results. TPI works by finding strong and weak points in the testing cycle via gap analysis. Here we will explain that what are best practices/techniques we are following to improve the test process and how it will benefit compare to traditional process. This will help us to identify the risks at early stages and it will be an effective way in terms of cost & schedule

5 Iterative process: One of the techniques for test process improvement is the iterative process. This is the customized process of IBM RUP. In the traditional testing life cycle model such as water fall model, Spiral model & etc, there are some possibilities that the high risks/critical issues introduced at the later stage and it will impact the cost, schedule & qualify of the products. One of the reasons for this, the test execution is planned at later stage i.e. once the coding phase is completed. To avoid this, we are following the iteration process with different phases and disciplines which will give risks identification in very early stage and it will help us to avoid cost & schedule slippage. In this iteration process, each phase will involve test execution which will help to identify the risks/issues at early stage of the program. An iterative process is nothing but performing test cycles at different phases of a project iteratively in parallel with development activities. By this method the high risk items are covered at early stage of the projects.

6 Below chart depict the benefits of iterative process how the high risk items are identified and mitigated at early stage of the project life cycle. Following slides will explain briefly about the iteration process. In iteration process, we have following two major flavors; Stand alone Delivery and Express Delivery

7 Stand alone Delivery: If the nature of project is big or having major changes or multiple cycles of testing then we plan for standalone delivery which will cover the entire test process areas. Express Delivery If the nature of project is minimal scope like patch releases then we plan for express delivery which will focus minimal mandatory test process. Iteration Phases: Following are the major four phases in the test iteration process; Inception Elaboration Construction and Transition

8 Phases Vs Discipline

9 Inception Phase: In Inception, the Test Discipline focuses solely on its planning activity, with emphasis on the project as a whole. For Inception, we have a Project focus. This activity identifies the appropriate focus of subsequent test activities for the project, and gains agreement with stakeholders on the corresponding goals that will direct those activities. For subsequent phases and iterations, we have an Iteration focus at that time; this activity identifies the appropriate focus of the subsequent test activities for the iteration, and gains agreement with stakeholders on the corresponding goals that will direct those activities. For Inception: Understand and capture the general expectations for Test Establish preliminary resource requirements Gain stakeholder support Establish a basis for subsequent iteration test planning The Test Plan is base lined in Inception phase and refined with each Iteration

10 Elaboration Phase: The primary objective is to mitigate the key risk items identified by analysis up to the end of this phase. The elaboration phase is where the project starts to take shape. In this phase the problem domain analysis is made and the architecture of the project gets its basic form. For Elaboration, Refine the Test approach Expand/refine Test assets in line with evolving Requirements Assist the project in identifying corrective actions Construction Phase The primary objective is to build the software system. In this phase, the main focus is on the development of components and other features of the system. This is the phase when the bulk of the coding takes place. In larger projects, several construction iterations may be developed in an effort to divide the use cases into manageable segments that produce demonstrable prototypes.

11 For construction, Refine the Test approach Expand/refine Test assets in line with evolving Requirements Assist the project in identifying corrective actions Transition Phase The primary objective is to 'transit' the system from development into production, making it available to and understood by the end user. The activities of this phase include training the end users and maintainers and beta testing the system to validate it against the end users' expectations. The product is also checked against the quality level set in the Inception phase. For transition, Refine the Test approach Expand/refine Test assets in line with evolving Requirements Assist the project in identifying corrective actions If all objectives are met, the Product Release Milestone is reached and the development cycle is finished.

12 Test Prioritization model: One another test process improvement is test prioritization model. This is called Heuristics method. The objective of this model is to run only the risk based/ high priority test cases and reduce the cycle time. The scope of this analysis is dual: Heuristic Prioritization What test case should we run? Heuristic testing What test case should we produce? How to define (Prioritization) this model? Define Priority = { Amount of Changes, Frequency of Use} High priority = HH, HM, MH, MM Medium priority = LH, HL Low priority = ML, LM

13 Group Test Cases by Priority Allocate Test Cases to Phases Alpha = High priority Regression = Medium + Low Priority Optional Regression = Test Cases with no changes + LL Run Test Cases by Priority from Groups Note: This prioritization done by the Technical/Product experts & Product managers. Heuristics for two main test phases: System Testing: The more changes you have in the code that is tested by a certain test case, the more likely it is for that code to have a defect. The more your customer uses the code that is tested by a certain test case, the more likely it is for the customer to find a defect. Regression Testing: The less often you test the code that is tested by a test case, the more likely it is for this code to have a defect. Note: This model may not be applicable for first time testing of new products.

14 Test tool management There are different artifacts (Test plan, Test design/cases and test report) used in testing life cycle. We have to maintain/update all these artifacts on regular basis in order to keep the track of all the activities a tester perform. Test artifacts management in manual method is really difficult as there are chances for manual errors and also people have to use different documents for each item. So, need for a test management tool is very high and given more importance to overcome above said difficulties. So generally people prefer to use any of the test artifact management tools in their testing cycle. These tools help the testers to maintain the artifacts in a common place and there would be single point where we could find all the testing related documents related to any particular project/application. All the modification and revision information can be maintained in the management tool. These tools can also be used for integrating with defect tracking tools. So that any failure/defect can be logged in defect tracking tool and we could provide direct link from test management tool. Better consolidation and efficient tracking is done with the help of these test management tools.

15 Following is the pictorial representation of how each artifact are maintained in manual method. Test Plan Maintained separately in project repository Test Design Test Execution Results update Maintained separately in project repository Maintained separately in project repository Defect/Anomalies Reporting Maintained separately in Defect tracking tool Test artifacts in manual method

16 Following is the pictorial representation of how all the artifacts are maintained in test tool management method. Test Plan Test Design Test Execution Results update All artifacts are maintained in a single Tool (Test Management Tool) Defect/Anomalies Reporting Test artifacts in tool management method

17 Test resource pool management One of the other techniques in test process improvement is that test resource pool management. With this approach available test resources are utilized in best effective way across multiple products/applications. By this way each test resource would get quite amount of knowledge on multiple products, so those resources can be utilized at any point of time in any products. This will help utilize the resources in effective manner with on schedule delivery and cost effectiveness. It also helps to improve the competency of individuals.

18 Pros & Cons: Pros: Improved productivity Increased quality Decreased cost Decreased labor Decreased time to market (to field, to launch) Cons: Few methods may not applicable for first time testing of new development projects. Less availability of single tools to use from requirement phase to test phase for complete traceability. There would be challenge to test thin slice during the early phase of the project itself in iterative method. Reference materials

19 Questions??