Software Testing Practices

Transcription

1 Software Practices Mr. Ganesh B. Regulwar M.E. (Info-Tech) 1 st Year Sipna s College of Engg. & Tech., Amravati (Maharashtra) Mr. Vijay S. Gulhane Assistant Professor, CMPS & IT Dept. Sipna s College of Engg. & Tech Amravati (Maharashtra) Abstract In the simplest terms, Software is a process to ensure the accuracy, completeness and quality of any developed software. Rather than limiting this to a formally technical process used to investigate and measure quality of the developed computer software in terms of correctness, completeness and security, it is imperative to test the software components and the software as a whole for integrity, capability, reliability, efficiency, portability, maintainability, compatibility and usability of the software. This is particularly important in an operating environment where it is planned to be implemented, and testing will need to adhere to the technical requirements as described under ISO standard ISO Effective testing requires the presence of adequate and suitable test data. Where test results are specified, it is presumed that the inputs will be consistent from text execution to execution. In situations where automated testing methodologies are employed, a standardized test bed of data may be required to facilitate the automated assessment of "passed". The best approach is to conduct end to end testing. methodology is applied in two major phases: Functional and Non- Functional. One of the purposes of automated testing is to ease regression testing. Once the system is stable, then automated testing can be started. Preferably, automated tests can be run overnight on software builds, rather than the live version. This will help give a clearer indication to the stability of the build the following morning. Hopefully leading to more time being available for the rest of the testing and development. INTRODUCTION There are many approaches to software testing, but effective testing of complex products is essentially a process of investigation, not merely a matter of creating and following rote procedure. One definition of testing is "the process of questioning a product in order to evaluate it", where the "questions" are things the tester tries to do with the product, and the product answers with its behavior in reaction to the probing of the tester. Although most of the intellectual processes of testing are nearly identical to that of review or inspection, the word testing is connoted to mean the dynamic analysis of the product putting the product through its paces. helps is verifying and Validating if the Software is working as it is intended to be working. Thins involves using Static and Dynamic methodologies to Test the application. Software Fundamentals objectives: 1. is a process of executing a program with the intent of finding an error. 2. A good test case is one that has a high probability of finding an as yet undiscovered error. 3. A successful test is one that uncovers an as yet undiscovered error. should systematically uncover different classes of errors in a minimum amount of time and with a minimum amount of effort. A secondary benefit of testing is that it demonstrates that the software appears to be working as stated in the specifications. The data collected through testing can also provide an indication of the software's reliability and quality. But, testing cannot show the absence of defect -- it can only show that software defects are present. WHAT IS SOFTWARE? Software in Instructions (computer programs) that when executed provide desired function and performance. Software is data structure that enables the programmer to adequately manipulate information. Software is document that describes the operation and use of the programs. SOFTWARE DEVELOPMENT LIFE CYCLE: Sequence of stages from conception to operation of a program is known as Software Life Cycle. It has following important stages which forms software through proper path. a. Software Requirement b. Analysis c. Design d. Coding e. f. Maintenance SOFTWARE TESTING: It is a process used to help identify the correctness, completeness and quality of developed computer software. With that in mind, testing can never completely establish the correctness of computer software. Rather than limiting this to a formally technical process used to investigate and measure quality of the developed computer software in terms of correctness, completeness and security, it is imperative to test the software components and the software as a whole for integrity, capability, reliability, efficiency, 1

2 portability, maintainability, compatibility and usability of the software. Users of an enterprise are called customers, suppliers, etc. WHEN TESTING SHOULD START: early in the life cycle reduces the errors. Test deliverables are associated with every phase of development. The goal of Software Tester is to find bugs, find them as early as possible, and make them sure they are fixed. WHEN TO STOP TESTING: This can be difficult to determine. Many modern software applications are so complex, and run in such as interdependent environment, that complete testing can never be done. "When to stop testing" is one of the most difficult questions to a test engineer. Common factors in deciding when to stop are: Deadlines (release deadlines, testing deadlines.) Test cases completed with certain percentages passed Test budget depleted Coverage of code/functionality/requirements reaches a specified point The rate at which Bugs can be found is too small Beta or Alpha period ends The risk in the project is under acceptable limit. Practically, we feel that the decision of stopping testing is based on the level of the risk acceptable to the management. As testing is a never ending process we can never assume that 100 % testing has been done, we can only minimize the risk of shipping the product to client with X testing done. The risk can be measured by Risk analysis but for small duration / low budget / low resources project, risk can be deduced by simply: - Measuring Test Coverage. Number of test cycles. Number of high priority bugs. SOFTWARE TESTING LIFE CYCLE: The test development life cycle contains the following components: Requirements Use Case Document Test Plan Test Case Test Case execution Report Analysis Bug Analysis Bug Reporting Typical interaction scenario from a user's perspective for system requirements studies or testing. In other words, "an actual or realistic example scenario". A use case describes the use of a system from start to finish. Use cases focus attention on aspects of a system useful to people outside of the system itself. Users of a program are called users or clients. Software Life Cycle Requirements Test Plan Test Case Test Case execution Report Analysis Defect Management SOFTWARE TESTING WHAT IS DEFECT? The software doesn t do something that the product specification says it should do. The software does something that the product specification says it shouldn t do. The software does something that the product specification doesn t mention. The software doesn t do something that the product specification doesn t mention but should do. Difficult to understand, hard to use, slow etc. ATTUTUDE TOWARD TESTING: New programmers are not accustomed to viewing testing as a discovery process. We take pride on what we developed. We defend ourselves it is not my fault! Conflict between tester and developer. Testers try to find faults, developers try to take pride. Hurt feelings and bruised egos have no place in the development process as faults are discovered. TEST METHODS: Black box testing (also called functional testing): Test to the specs (Do not look inside the code) White box testing (also called structural / glass box testing): Test to the code (Do not look inside the specs) BLAC BOX TESTING: 2

3 validity. STATIC TESTING VS DYNAMIC TESTING: Static A form of verification that does not requires execution of software. Ex. Model Checking Dynamic Requires the execution of the software. Ex. Output Validation. Approach to testing where the program is considered as a black-box. based solely on analysis of requirements (specification, user documentation, etc.). Also called functional testing or data-driven or I/Odriven testing. The program test cases are based on specifications. Black-box testing techniques apply to all levels of testing (e.g., unit, component, product, and system). Test planning can begin early in the software process. Black box testing is practiced widely. Conducted for Integration testing, System & Acceptance Test Case Design Methods: o Equivalence Partitioning Method o Boundary value Analysis o Cause Effect Graph o State transition testing o Use case based testing o Error guessing WHITE BOX TESTING: that takes into account internal mechanism of a system or component, types include branch testing, path testing etc Aspects missed out in black box testing are tested Disadvantages: Does not ensure that user s requirements are met Does not establish if the decisions / conditions / paths / statements are insufficient White box testing assumes that the procedural design is known to us. Procedural design can act as driver to test case design. The tester is going to test the chronological order of program. The tester can design test cases which can test internal logic of program. Derive test cases that: 1. Exercise all independent execution paths 2. Exercise all logical decisions on both true and false sides 3. Execute all loops at their boundaries and within operational bounds 4. Exercise internal data structures to ensure Control flow testing: Statement coverage Edge/Branch coverage Condition coverage Multiple condition coverage Path coverage PEOPLE CHALLENGES IN SOFTWEAR TESTING: A real tester must take responsibility for improving the testing process is a Professional discipline requiring trained, skilled people Cultivate a positive team attitude of creative destruction is a highly creative and challenging task Programmers and testers are not adversaries: they join hands in a team effort to add value to a program Testers HUNT for defects Testers pursue defects, not the people behind the defects tries person s qualities like patience, fairness, ambition, credibility, capability can affect a person s mental and emotional health in view of office politics and inter-personal conflicts 3

4 SKILLS REQUIRED IN TESTER: Negative Thinking Negative thinking helps testers derive the negative user scenarios. Reading Skills Testers routinely encounter large quantities of data to read and comprehend. At the requirements review stage, testers have to review hundreds of pages of requirements. Communication and Interpersonal Skills For a tester, both verbal and written communication are crucial e.g. communication of defects to developers, quality level of product Time Management and Effort Prioritization Testers have to juggle a lot of tasks like creating test cases, execution, documenting test results, creating test metrics etc. Tester may be involved in testing more than one module or project at the same time. SOFTWARE VERIFICATION AND VALIDATION: VERIFICATION: Disciplined approach to evaluate whether a software product fulfils the requirements or conditions imposed on them (Are we doing the job right?) Method: walkthrough, formal inspection and review of each software product Also called static testing Done by systematically reading the contents of a software product with the intention of detecting defects Helps in identifying not only presence of defects but also their location A filter applied at various points during the SDLC to purify the product as it progresses through various phases Verification Examples: Requirement Reviews - Developers, Users Design Reviews - Developers Code Walkthroughs - Developers Code Inspections - Developers VALIDATION: Disciplined approach to evaluate whether the final, as-built software product fulfils its specific intended use (Are We Doing the Right Job?) Method: testing each software product at each phase of life cycle using test plans test cases for properly selected types of tests. Also called dynamic testing Done by systematically testing a software product with the intention of finding defects Helps in identifying the presence of defects, not their location Necessary to demonstrate not just that the software is doing what it is supposed to do, but also is not doing what it is not supposed to do Validation Example: Unit - Developers Integrated - Testers System - Testers, Users User Acceptance - Users PRINCIPLE OF V & V: Software Verification and Validation... Is the discipline of verifying that products of each software life cycle phase comply with previous life-cycle phase requirements and establish the proper basis for initiating the next life cycle phase, and of validating that the completed end product complies with established software and system requirements. TECHNIQUES FOR SW VERIFICATION: Audits: An independent assessment of a project To verify whether or not the project is in compliance with appropriate policies, procedures, standards, contractual specifications An audit may include operational aspects of the project Walkthrough: An informal process, initiated by the author of a software product to a colleague for assistance in locating defects and for suggesting improvements Normally not planned Author explains the product Colleague comes out with observations Author provides clarification if required Author notes down relevant points and takes corrective actions Benefits of Walkthroughs: Errors caught at earliest possible time. Greatly improved software quality. Project communication improved. Software easier to maintain. Better project control. Quicker integration of new personnel. Increased programmer expertise. Less disruption with personnel leave. Switch emphasis from individual contemplation to clear, precise communication with others. Review Normally done at the end of a phase of SDLC, when the author/s feel/s that the product is error-free and can go to the next phase To determine the defects early in the lifecycle Usually conducted by a Manager Main purpose is to decide if the product can go the next phase Includes checking if suitable inspections have been done After review, the product is base-lined Reviews seek to ascertain that tolerable levels of quality are being attained. 4

5 The review team is more concerned with design deficiencies and deviations from the conceptual model and Management requirements than it is with the intricate line-by-line details of the implementation The focus of a review is not on discovering technical flaws but on ensuring that the design and development fully and accurately address the needs of the application V TESTING CONCEPT: Life Cycle testing: CONTINUOUS testing throughout the SDLC Goes hand-in-hand with formalized system development process Need to plan the testing activities parallel with the SDLC phases THE V MODEL: Customer Requirements System Specifications High Level Design Low Level Design Coding Integration Unit Acceptance System Levels of : Unit - done by the developer at unit level. Integration - Conducted by the project team integrating the modules System - Conducted by project team or by separate testing team if any Acceptance - Conducted by client either in developer's site or at his site UNIT TESTING: The smallest piece of software that can be tested in isolation It is procedure used to validate that individual unit of source code is working properly. Approaches Black Box White Box Benefit Ensures code meets the requirements Simplifies integration INTEGRATION TESTING: Starts at module level when various modules are integrated with each other to form a sub-system or system More stress is given on interfaces between the modules Focuses on design and construction of the software architecture. Four Basic Approaches To While Integrating Modules Bottom Up Top Down Big Bang Bottom Up Integration : Bottom Up The program is combined and tested from the bottom of the tree to the top 1. Requires a module driver for each module. Put together sub trees and test until whole tree. 2. Very common and effective approach, especially when combined with the Object Oriented Design 3. Each component at the lowest level of the system hierarchy is tested individually first, then next components to be tested. Component Driver: A special code to aid the integration. A routine that calls a particular component and passes a test case to it. Take care of driver s interface with the test component Top Down Integration : Top Down 1. Design, implement and test the top modules using stubs (or dummy modules). 2. Stubs do not perform any real computations or manipulate any real data. 3. Tests are conducted as each component is integrated. 4. Stubs are removed and integration moves downward in the program structure. Stub: Problem: A component being tested may call another that is not yet tested! Solution: Write a special-purpose program to simulate the activity of the missing component. The special-purpose program is called a stub. If the lowest level of components performs the input and output operations, stubs for them may be almost identical to the actual components they replace. Big Bang : A common approach in non-process oriented organizations All modules are integrated at once Many disadvantages and few advantages!! Hard to debug, hard to check interfaces Location of the defect is difficult to find No stubs and drivers are required 5

6 SYSTEM TESTING: System testing is testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. Validates that the system meets its functional and non-functional requirements It is also intended to test up to and beyond the bounds defined in the software/hardware requirements specifications. Final phase of testing before delivery ACCEPTANCE TESTING: Final stage of testing before the system is accepted for operational use Done with data supplied by the client Validates User Needs (Functional) System Performance (Non-Functional) Alpha : Tested at developer site by customer Developer "looks over shoulder" and records errors & usage problems Tests conducted in a controlled environment Beta : Beta testing conducted at one or more customer sites by end user of software Live application environment cannot be controlled by developer Customer records all problems encountered and reports to developer at regular intervals WHAT IS A BUG LIFE CYCLE? CONCLUSIONS Software testing is an art. Most of the testing methods and practices are not very different from 20 years ago. It is nowhere near maturity, although there are many tools and techniques available to use. Good testing also requires a tester's creativity, experience and intuition, together with proper techniques. is more than just debugging. is not only used to locate defects and correct them. It is also used in validation, verification process, and reliability measurement. is expensive. Automation is a good way to cut down cost and time. efficiency and effectiveness is the criteria for coverage-based testing techniques. Complete testing is infeasible. Complexity is the root of the problem. At some point, software testing has to be stopped and product has to be shipped. The stopping time can be decided by the trade-off of time and budget. Or if the reliability estimate of the software product meets requirement. may not be the most effective method to improve software quality. Alternative methods, such as inspection, and clean-room engineering, may be even better. REFERENCES [1] Paul Ammann and Jeff Offutt, Introduction to Software, Cambridge University Press, Cambridge, UK, ISBN , [2] Title: The Practitioner, Author: Erik van Veenendaal, Publisher: UTN Publishers, Publication Date: 2002, ISBN: [3] Title: Introducing Software : A Practical Guide for Getting Started, Author: Louise Tamres, Simon Mills, And Publisher: Pearson Education / Addison Wesley, Publication Date: 2002, ISBN: or [4] Title: Systematic Software, Authors: Rick D. Craig and Stefan P. Jaskiel, Publisher: Artech House, Publication Date: 2002, ISBN: [5] Title: Software, Author: Ron Patton, Publisher: Sams Publishing, Publication Date: 2001, ISBN: [6] Title: IT: An Off-the-Shelf Process, Author: John Watkins, Publisher: Cambridge University Press, Publication Date: 2001, ISBN: X, [7] Title: Effective Methods for Software, Author: William E. Perry, Publisher: John Wiley & Sons Inc, Publication Date: 2000, ISBN: X 2nd Edition; rd Edition) [8] Title: Black Box : Techniques for Functional of Software and Systems, Author: Boris Beizer, Publisher: John Wiley & Sons Inc, Publication Date: 1995, ISBN: [9] Title: The Art of Software, Author: Glenford Myers, Covey Sandler, Tom Badgett, Todd M Thomas, And Publisher: John Wiley & Sons Inc, Publication Date: 2004, ISBN: [10] Title: Software : a craftsman's approach, Author: Paul Jorgensen, Publisher: CRC Press, Publication Date: 2002, ISBN:

7 [11] Basili85] Victor R. Basili, Richard W. Selby, Jr. "Comparing the Effectiveness of Software Strategies", Technical Report, Department of Computer Science, University of Maryland, College Park, [12] Beizer90] Boris Beizer, Software Techniques. Second edition [13] Textbook: Pressman Roger S.: Software Engineering, A Practitioner s Approach TMH. [14]Textbook: Somerville: Software Engineering (Addison- Wesley) [15] Davis A: Principles of Software Development 2010 International Journal of Computer Applications ( ) 7