BASICS OF SOFTWARE TESTING

Transcription

1 BASICS OF SOFTWARE TESTING CONTENTS I. Software Quality, Definition of testing, Role of testing. II. III. IV. Failure, Error, fault, Defect, Bug Terminology Objectives of Testing What is a Test Case? V. When to start and Stop Testing of software. VI. Skills for Software Tester. VII. Quality Assurance, Quality Control, Verification and Validation, V Model

2 I. Software Quality, Definition of testing, Role of testing 1. Software Quality Software quality is the degree of conformance to explicit or implicit requirements and expectations. i. Explicit: clearly defined and documented ii. Implicit: not clearly defined and documented but indirectly suggested iii. Requirements: business/product/software requirements iv. Expectations: mainly end-user expectations 2. Definition of Testing (Question: Explain testing with software quality- 2 Marks each, 4 Marks) Testing can be defined in simple words as Performing Verification and Validation of the Software Product for its correctness and accuracy of working. i. Software testing is as old as the hills in the history of digital computers. ii. The testing of software is an important means of assessing the software to determine its quality. iii. Since testing typically consumes 40% to50% of development efforts, and consumes more effort for systems that require higher levels of reliability, it is a significant part of the software engineering. iv. With the development of fourth generation languages (4GL), which speeds up the implementation process, the proportion of time devoted to testing increased. v. As the amount of maintenance and upgrade of existing systems grow, significant amount of testing will also be needed to verify systems after changes are made. 3. Role of Testing (Question: Explain the role of Testing 6 Marks) i. Software testing is to identify errors in order to reveal and spot it. ii. The extent of software testing consists of implementation of that code in different domain and also to look at the features of the code does the software do what it is should be done and methods respect to the condition. iii. It is proposed to begin testing from the first phase of the software development. iv. It saves time as well as cost. It is a continuous method, which is probably nonstop but has to be stopped anywhere, for the need of time and resources.

3 v. The basic need of the testing is to provide best quality product without taking so much time and money. vi. The test engineer has to pursue some technical way by which he/she can review that all the points of necessity for testing have been covered or not. In SDLC stage there are some most importance things as described below: i. Recognition of Error and Faults Testing step is one step which resolves the errors and faults in the software application. These errors may be in unit level or in system level. After going through so many testing the application will be free of errors that may be disturbing the application. ii. Statistics to Shareholders and Status of Organization Testing stage helps to know the condition of product and work standards. The stakeholders get better data through testing stage about utility value too. iii. Enhancement in Product Standards Testing can help to know the real result and the probable result. It also helps to pick up the standards of the software. With proper testing an application can come out of bugs and build up ideal software for the end-users. iv. Technical Significance Testing segment is significant for technical characteristics of any SDLC, as the software then completed with technically satisfied. v. To Succeed of any Contentious Programmers Ideal testing functions and tools aid to evolve up the product in business and keep programmers away from the other contestant. Going through all stages of testing, the software application will be more bugs free, protected and technically sound. vi. Free from any Risk Whenever going to develop any software, testing is an essential part.

4 When develop software without any testing then it may cause lots of risks to the end users. To free everyone from any risk, it is essential that to go under all testing stages. vii. Enhanced Standards Appropriate tested application provides additional assurance of build up with best software. Moreover, it refines standards of application as incessant and all types of testing stages have prepared a protected and harmless software application that could be worn by the end users. viii. Confirmation and Corroboration One of the major targets of testing stage in SDLC is for confirmation and corroboration. Testing is greatly used in confirmation and corroboration method. Depending on the result we can compare among standards of several software application. ix. Credibility Evaluation Testing stage also insist this important issue. If the software application has gone through all the testing types (like unit testing, regression testing etc.), the application will surely be a reliable one. So, testing evaluate credibility of software application. Testing provides the greatest analytical process to give equipped testing on product ensuing in a credible product. x. Demonstrate Accessibility and Feasibility One of the most significant targets of testing is to demonstrate the product is both accessible and functional. Accessibility testing is where the application is delivering to a select assembly of users and their functioning with the application is noticed. All type of a user's communication with the application, like easiness of applies and whenever users are getting troubles, are preserved and examined. xi. Avoid Fault Immigration

5 In the first stage of SDLC, most of the faults have been found. If the faults can be noticed earlier, then these may be prohibited from immigrating to the following progress stage. If the errors could be discover previously then the saving of software development cost will be vast. xii. Commercial Significance A full tested software application will have excellent business aspects. As all are like to work with reliable and trusted application in commercial. II. Failure, Error, fault, Defect, Bug Terminology (Question: Define Bug and list the terms related to software failure: definition - 1 mark, list 1 mark, explanation 2 marks = 4 marks) Definition: A bug can be defined in simple term as any error or mistake that leads to the failure of the product or software either due to the specification problem or due to communication problem, regarding what is developed and what had to be developed. 1. Bug Terminology: Failure, Error, Fault The various terms related to software failure with respect to the area of application are listed as Defect, Variance, Fault, Failure, Problem, Inconsistency, Error, Feature, Incident, Bug, and Anomaly. i. Problem, error, and bug are probably the most generic terms used. ii. Anomaly, incident, and variance don t sound quite so negative and infer more unintended operation than an all-out failure. iii. Fault, failure, and defect tend to imply a condition that s really severe, maybe even dangerous. It doesn t sound right to call an incorrectly colored icon a fault. These words also tend to imply blame: It s his fault that the software failed. iv. As all the words sound the same they are distinguished based on the severity and the area in which the software failure has occurred. v. When we run a program the error that we get during execution is termed on the basis of runtime error, compile time error, computational error, and assignment error. vi. The error can be removed by debugging, if not resolved leads to a problem and if the problem becomes large leads to software failure. vii. A bug can be defined as the initiation of error or a problem due to which fault, failure, incident or an anomaly occurs.

6 viii. The program to find the factorial of a number given below lists few errors problem and failure in a program. For example 1. #include<stdio.h> 2. void main() 3. { 4. int i, fact, n; 5. printf( enter the number ); 6. scanf( %d,&n); 7. for(i =1 ;i <=n;i++) 8. fact = fact * i; 9. printf ( the factorial of a number is %d, fact); 10. } As in line number 4 the fact is not initialized to 1, so it takes garbage value and gives a wrong output, this is an example of a bug. If fact is initialized to zero (fact = 0) than the output will be zero as anything multiplied by zero will give the output as zero. This is a bug which can be removed by initializing fact = 1 during initializing. As the fact is declared as integer, for the number till 7! will work perfectly. When the number entered is 8, the output is garbage value as the integer limit is from to , so in declaration change the initialization to long int fact; III. Objective of Testing (Question: Explain the objective of Testing- 6 Marks) 1. Finding defects which may get created by the programmer while developing the software. 2. Gaining confidence in and providing information about the level of quality. 3. To prevent defects. 4. To make sure that the end result meets the business and user requirements. 5. To ensure that it satisfies the BRS that is Business Requirement Specification and SRS that is System Requirement Specifications. 6. To gain the confidence of the customers by providing them a quality product.

7 IV. When to start and stop testing of software 1. When to Start Testing (Question: Give the steps when should software testing start in project development- 6 Marks) i. An early start to testing reduces the cost, time to rework and error free software that is delivered to the client. ii. Software Development Life Cycle (SDLC) testing can be started from the Requirements Gathering phase and lasts till the deployment of the software. iii. It also depends on the development model that is being used. For example in Water fall model formal testing is conducted in the Testing phase, but in incremental model, testing is performed at the end of every increment/iteration and at the end the whole application is tested. iv. Testing is done in different forms at every phase of SDLC like during Requirement gathering phase, the analysis and verification of requirements are also considered testing. v. Reviewing the design in the design phase with intent to improve the design is also considered as testing. vi. Testing performed by a developer on completion of the code is also categorized as Unit type of testing. 2. When to Stop Testing (Question: Give the steps when should software testing stop in project development- 6 Marks) i. Unlike when to start testing it is difficult to determine when to stop testing, as testing is a never ending process and no one can say that any software is 100% tested. ii. Following are the aspects which should be considered to stop the testing: Testing Deadlines. Completion of test case execution. Completion of Functional and code coverage to a certain point. Bug rate falls below a certain level and no high priority bugs are identified. Management decision. iii. Testing should be stopped when it meets the completion criteria. iv. Completion criteria should be based on Risks. v. Testing should be stopped when : Test cases completed with certain percentage passed and test coverage is achieved.

8 There are no known critical bugs Coverage of code, functionality, or requirements reaches a specified point; Bug rate falls below a certain level, now testers are not getting any priority 1, 2, or 3 bugs. vi. The risk can be measured by Risk analysis but for small duration / low budget / low resources project, risk can be deduced by simply: - Measuring Test Coverage. Number of test cycles. Number of high priority bugs. V. Skills of a software tester (Question: List and explain skills of software tester- 8 Marks) 1. Analytical and logical thinking i. The major objective of testing is to identify the hidden errors, not simply prove that the software works. ii. For a tester to be effective in his role, he must be able to analyze the given business situation and judge all the possible scenarios. iii. He should have the capacity to identify and tackle unfamiliar problems and should develop a strategy to validate it. iv. Creating situations and validating the application under test, before presenting it to customers, can be done effectively only by a person who has strong analytical skills. ii. A tester should be able to separate the whole into logical parts -- to examine a complex problem, its elements and their relationships. iii. He should be able to develop a logical argument based on relationships between elements and propositions, as well as identify implications, relationships, redundancies and contradictions without leaving any room for inconsistency and ambiguity. iv. He should be consistent in analyzing and solving complex, multi-step problems. 2. The ability to envision business situations i. A tester should be able to envisage real-time business situations through mental mapping, abstracting the idea inferred from the specifications. ii. The real-time business scenarios should crystallize in a tester's mind, and he should think about what the case is rather than what ought to be the case or what he believes the case is.

9 iii. A tester should be able to anticipate complex problems, in addition to visualizing and articulating them. iv. He should be able to do a complete system simulation rapidly and accurately. v. In the present software development environments, it is hard to believe that teams/individuals will get enough time to do a series of conventional brainstorming sessions to finalize the concept mapping. vi. Therefore, it is vital that a tester develop his conceptualization skills through mental mapping. 3. A sense of intellectual curiosity and creativity i. A tester should understand that being an intellectual and being intellectually curious are not the same. ii. A tester should arguably be the latter one -- intellectually curious -- which is all about asking questions and not about having answers. iii. He should believe in the pursuit of knowledge as a value in and of itself. iv. He should love asking questions and should not consider it a blow to his ego if he is wrong about something. v. It is intellectual curiosity that motivates and prompts a tester to identify interesting questions about the software being tested. vi. Thus, a tester should develop the skill to see what everyone else hasn't seen, to think what no one else has thought of and to do what no one else has dared. 4. A "glocal" approach i. Software systems have become extremely complex. ii. Most of the time, the system designed involves multiple stakeholders, and dealing with such systems is not always easy. iii. A tester should be able to deal effectively with business situations marked by complexity and the number of interactions with third-party systems. iv. He should be able to identify how the system under test interacts with other constituents of the system. v. He should also be able to isolate the minutest units of the application under test and do the validation, keeping in mind the behavior of the system as a whole. vi. A tester will stand out among his peers if he is able to detach himself from the system, look at it as an outsider, and present his findings on the relationship between the individual components convincingly in a logical fashion. vii. At the same time, he should keep his eyes and ears open so as not to ignore issues that may be inherent in individual components. viii. Thus, a "glocal" (global + local) approach is essential. 5. Critical thought and rational enquiry i. The quality of life of an individual and the quality of what he produces/delivers depends largely on the quality of his thought process.

10 ii. The thought process of a tester should be undistorted, impartial and without any prejudices. iii. A tester should be able to take charge of the inherent structures and impose intellectual standards upon the software under test. iv. He should be able to raise vital questions precisely and clearly, gather and assess relevant information, interpret it effectively in order to come to well-reasoned conclusions and solutions, and test those conclusions against the given criteria and standards. v. A tester should be open minded, suspend any judgment in the absence of sufficient evidence to support a decision, and always abide by logical reasoning. 6. The ability to apply basic and fundamental knowledge i. Knowledge in the context of testing can be attributed as the fluid mix of experience, values, contextual information and expert insight. ii. Those things provide a framework for evaluating the system under test. One can attain knowledge by so many means, but that knowledge is worthwhile only when it adds value to situations encountered. iii. A smart tester should be able to apply the knowledge attained over years of experience with the domain, process, product, customers, mistakes and successes in his testing. iv. He should be able to make use of fundamental communication, mathematical and software application skills. v. He should also be able to effectively apply the skills he has attained to practical situations. 7. Continue to learn i. Organizations and business environments change rapidly, which means the approaches and processes that work well today will be outdated tomorrow. ii. Therefore, it is imperative that a tester place priority on noticing, adapting and learning from change that is happening around him. iii. That doesn't mean a tester should continually undergo training or certification, rather he should be open to learning from everything in life that comes he across. iv. If he has gained basic and fundamental knowledge, then the rest can be achieved through self-directed learning. v. Learning should be a lifelong habit. 8. Respect for truth and intellectual integrity i. A tester should be able to examine the piece of software under test and the resulting processes, with focus on the given specification, and understand the behavior of the software.

11 ii. Being human, a tester may have severe biases, prejudices and intolerances that prevent him from performing well. iii. He should possess the intellectual integrity to correct those barriers in order to efficiently understand the nature of the software under test. iv. He should also be willing to shrug off the set of practices and character traits that undermine his intellectual integrity. v. And he should be able to exemplify intellectual virtues such as honesty, impartiality and openness to the views of others. 9. Planning, time management skills i. Planning is nothing but writing the story of the future. ii. A tester needs to have a thorough plan and must develop a well-thought test strategy/approach. iii. And that plan must be in place before work begins on any software testing assignment. iv. It should describe the items and features to be tested, the test strategy and levels of testing, pass/fail criteria, suspension/resumption criteria, schedule, etc. v. The plan developed should be monitored continually, and validations should be done through organized system feedback. vi. Sticking to the plan and monitoring the progress in order to ensure timely delivery is key to any software testing assignment's success. 10. Effective communication skills i. A tester must be able to communicate his thoughts and ideas effectively, using a variety of tools and media. ii. He needs to develop and use this skill throughout his career and should learn to communicate effectively to the stakeholders so as to avoid ambiguities and inconsistencies. iii. For example, printed presentations should be concise and to the point and should follow logically. iv. The language should be pragmatic rather than philosophical, and arguments should be supported by facts. v. In the case of oral presentations, the voice, body language and appearance of the presenter are as important as the content and visual aids. vi. A tester should develop his skills to overcome shyness and any fear of speaking. He should also have good listening skills.

12 VIII. Quality Assurance, Quality Control, Verification and Validation, V Model 1. Quality Assurance (Question: Explain the concept of Quality Assurance and Quality control in software testing. - 2 marks + 2 marks =4 marks) i. Quality Assurance: A part of quality management focused on providing confidence that quality requirements will be fulfilled. ii. All the planned and systematic activities implemented within the quality system that can be demonstrated to provide confidence that a product or service will fulfill requirements for quality iii. Quality Assurance is fundamentally focused on planning and documenting those processes to assure quality including things such as quality plans and inspection and test plans. iv. Quality Assurance is a system for evaluating performance, service, of the quality of a product against a system, standard or specified requirement for customers. v. Quality Assurance is a complete system to assure the quality of products or services. It is not only a process, but a complete system including also control. It is a way of management. 2. Quality Control i. A part of quality management focused on fulfilling quality requirements. ii. The operational techniques and activities used to fulfill requirements for quality. iii. Quality Control on the other hand is the physical verification that the product conforms to these planned arrangements by inspection, measurement etc. iv. Quality Control is the process involved within the system to ensure job management, competence and performance during the manufacturing of the product or service to ensure it meets the quality plan as designed. v. Quality Control just measures and determines the quality level of products or services. 3. Precision and Accuracy (Question: Explain with diagram the difference between precision and accuracy. :- diagram 1 mark, explanation 3 marks = 4 marks)

13 i. The Accuracy of a measurement system is the degree of closeness of measurements of a quantity to that quantity's actual (true) value. ii. iii. The Precision of a measurement system, also called reproducibility or repeatability, is the degree to which repeated measurements under unchanged conditions show the same results. Although the two words precision and accuracy can be synonymous in colloquial use, they are deliberately contrasted in the context of the scientific method. Diagram for Accuracy and Precision For example Figure 1: Accuracy and Precision 1. A measurement system can be accurate but not precise, precise but not accurate, neither, or both. 2. If an experiment contains a systematic error, then increasing the sample inputs generally increases precision but does not improve accuracy. 3. The result would be a consistent yet inaccurate string of results from the flawed software. 4. Eliminating the systematic error improves accuracy but does not change precision. 4. Verification and Validation (Question: Explain the concept of verification and validation in software testing. - 2 marks + 2 marks = 4 marks) Verification i. It makes sure that the product is designed to deliver all functionality to the customer.

14 ii. Verification is done at the starting of the development process. It includes reviews and meetings, walkthroughs, inspection, etc. to evaluate documents, plans, code, requirements and specifications. iii. It answers the questions like: Am I building the product right? iv. Am I accessing the data right (in the right place; in the right way). v. It is a Low level activity vi. Performed during development on key art facts, like walkthroughs, reviews and inspections, mentor feedback, training, checklists and standards. vii. Demonstration of consistency, completeness, and correctness of the software at each stage and between each stage of the development life cycle. Validation 1. Determining if the system complies with the requirements and performs functions for which it is intended and meets the organization s goals and user needs. 2. Validation is done at the end of the development process and takes place after verifications are completed. 3. It answers the question like: Am I building the right product? 4. Am I accessing the right data (in terms of the data required to satisfy the requirement). 5. It is a High level activity. 6. Performed after a work product is produced against established criteria ensuring that the product integrates correctly into the environment. 7. Determination of correctness of the final software product by a development project with respect to the user needs and requirements. 5. V Model (Question: Draw the diagram of V Model 4 Marks)

15 Figure 2: The V Model