SOFTWARE Testing Introduction UNIT I. Software Testing. Software Testing. Why it is hard? MCA-303, Software Testing

Transcription

1 SOFTWARE Testing Introduction UNIT I Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jian U1. 1 Software Testing process of executing a program with the intent of finding errors Software Testing Why it is hard? Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.1

2 An Example The program is designed to add two numbers, which you enter. Each number should be one or two digits. The program will echo your entries, then print the sum. Press <Enter> after each number. To start the program type Adder Guidelines 1. Start with an obvious and Simple test 2. Make Some notes about what else needs testing 3. Check the valid cases and see what happens 4. Do some testing on fly 5. Summarize what you know about the program and its problems Question of Interest Can you test this program exhaustively? If so, how many test cases are required? Assuming that each test case can be executed and analyzed in one second, how long would it take for you to run all the tests? Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.2

3 Limitation of Testing Can we test the program completely? Some text Many managers Test coverage analyzer Salesperson Some tester No matter how hard we try, how cleverly we plan, how much time we spend, and how many staff and computer we use, we still cannot do enough testing. We still miss bugs Reasons Theoretical Foundation No general purpose testing or analysis procedure can be used to prove program correctness Impracticality of Testing All Data The domain of possible inputs is too large to test Impracticality of Testing All Paths There are too many possible paths through the program to test Reasons.. No absolute proof of correctness Howden suggests that there are proofs of equivalency unless a formal specification can be shown to be correct and, indeed, reflects exactly the user's expectations, no claims of product correctness can be made. The user interface issues are too complex to completely test The challenge is to achieve maximum coverage through minimum test cases. It is rightly said that testing process is as efficient as the test data. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.3

4 Are Well Designed Test Cases Sufficient Apart from well designed test cases it is important to: Analyze the results properly Establish relation between faults and incidents Report the detected fault effectively Association with a module Steps for regenerating the problem encountered Optionally a probable impact Software Testing Consumes at least half of the labor Process of testing software product Contribute to Delivery of higher quality product More satisfied users Lower maintenance cost More accurate and reliable results Error Fault/Defect Failure Incident Test Test Case TERMINOLOGY Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.4

5 Error People make errors. Typographical error Misreading of a specification Misunderstanding di of functionality of amodule A good Synonym is Mistake. When people make mistakes while coding we call these mistakes bugs Fault/Defect Representation of an error DFD Hierarchy chart Source Code An error may lead to one or more faults Fault of Omissions If certain specifications have not been programmed Fault of Commission If certain program behavior have not been specified Fault/Defect.. Defects generally fall into one of the following three categories Wrong Missing Extra Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.5

6 Failure, Incident Failure A particular fault may cause different failures, depending on how it has been exercised Incident When a failure occurs, it may or may not readily apparent to the user Incident is the symptom associated with a failure that alerts the user to the occurrence of a failure Test, Test Case Test: A test is the act of exercising S/W with test cases Test Case: A test case has an identity and is associated with a program behaviour It has a set of inputs and a list of expected outputs t Test Case Template Test Case ID: Section I (Before Execution) Purpose: Pre Condition: (If Any) Inputs: Expected Outputs: Post Conditions: Written by: Date: Section II (After Execution) Execution History: Result: If Fails, any possible reason(optional): Any Other observation: Any Suggestions: Run By: Date: Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.6

7 S/W Testing Testing is the process of demonstrating that errors are not present The purpose of testing is to show that a program performs its intended functions correctly Testing is the process of establishing confidence that a program does what it is supposed to do Testing is the process of executing a program with the intent of finding errors Testing Life Cycle Testing Life Cycle.. Three phases Putting Bugs IN Testing Phase Finding Bugs Last Three phases are Getting Bugs OUT The fault resolution step is another opportunity for errors. When a fix causes formerly correct software to misbehave, the fix is deficient. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.7

8 Testing Principles 1. Testing shows presence of defects 2. Exhaustive testing is impossible 3. Early testing 4. Defect clustering 5. Pesticide paradox 6. Testing is context depending 7. Absence of errors fallacy h Relative Costs to Fix Errors What can you infer from this graph? Cost to fix an error increases as it is found later and later in the software lifecycle 0 Requirements Design Implementation Testing Maintenance Testing Cost Curve Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.8

9 Testing Process Input domain Input data UseOperational Profile Construct Input Test Plan Test Case Use Execute Program Behaviour Is behavior as expected yes Terminate testing? yes No File Test Report No Use Cause of Error? File pending error Report Debug program Error to be Fixed now? Specifications yes Fix error No Assignment 1 Q1. The following were some of the statements made by people in a product development organization. Identify the fallacies if any in the statements and relate it to the principles discussed 1. The code for this product is generated automatically by a CASE tool- it is therefore defect free. 2. We are certified according to the latest process models-we do not need testing. 3. I have run all the tests that I have been running for the last two releases and I don t need to run any more tests. 4. This automation tool is being used by our competitors- hence we should also use the same tool Assignment 1 Cont.. Q2. Assume that each defect in gathering requirements allowed to go to customers cost $10,000, and that the corresponding costs for design defects and coding defects are $1000 and $100, respectively. Also assume that current statistics indicate that on average ten new defects come from each of the phases. In addition, each phase also lets the defects from the previous phase seep through. What is the total cost of the defects under the current scenario? If you put a quality assurance process to catch 50% of the defects from each phase not to go to the net phase, what are the expected cost saving? Assume the cost of fixing bug in the same phase is $10 only. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.9

10 Venn Diagram to show relationship between the set T of test cases and the sets S and P of specified and implemented behaviors. S P T The goal of tester is to make the region where these sets all intersect as large as possible Venn Diagram.. specified behaviors exist for which no test case are available, the testing is necessarily incomplete. If certain test cases correspond to unspecified behaviors, two possibility arise test case is unwarranted or the specification is deficient. Role of Testing in Software Evolution whole life-cycle process- must be applied at each stage in the software process. The development approach determines the specific intermediate products to be created objectives must be identified for each of the product created Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.10

11 Verification & Validation Traditionally testing used to be an End-of-the-development-cycle activity With Life-Cycle testing model there is a requirement to test deliverables of each phase. The associated activities can be classified as Verification & Validation. Thus, software testing is essentially the verification and validation processes put together. Both the processes are complementary in nature and provide their own set of error filters Verification Verification: Are we building the product right? Static and proactive in nature (QA). Checks if product (result of a particular phase of SDLC) conforms with its specifications. Is static in nature. Involves manual checking, doesn t include execution of code. The target documents are Req. Specs., HLD, DB Design The methods used are Inspection, Walkthrough etc. Validation Validation: Are we building the right product? Dynamic and reactive in nature (QC). Checks the software to insure it meets customer s requirements. Is dynamic in nature. Involves execution of code. The target could be a component, module, a set of integrated modules or system as a whole. The methods used are Black box, white box testing etc. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.11

12 Verification & Validation Req Specs HLD Detailed Design Code Integration Verify Verify Verify Verify & Validate Verify & Validate System Testing UAT V Model The V-model is a software development process which can be presumed to be the extension of the waterfall model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape. The V-Model demonstrates the relationships between each phase of the development life cycle and its associated phase of testing. V Model The V-model deploys a well-structured method where each phase can be implemented by the detailed documentation of the previous phase. Testing activities like test designing start at the beginning i of the project well before coding and therefore Saves a huge amount of the project time and thus helps in cost reduction Assures better validation Assures involvement of people with appropriate skills. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.12

13 V Model V Model The V-Model further promotes the notion of early test preparation. finds faults in baselines and is an effective way of detecting faults early The V-model promotes the idea thatt the dynamic test stages (on the right hand side of the model) use the documentation identified on the left hand side as baselines for testing. V Model: Problems Derivative of Waterfall model => no provision for increments V Model focuses on specific dynamic test stages And has no mention of the effectiveness of static tests such as reviews, inspections, static code analysis and so on. This is a major omission and the V-Model does not support the broader view of testing as a constantly prominent activity throughout the development lifecycle. No mention of important activities like: Planning Post implementation maintenance Testing Quality Control Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.13

14 W Model Objectives In general, the objective is to insure that the product satisfies the user needs. The discovery of defects in a system The assessment of whether or not the system is usable in operational situations. Verification Techniques Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.14

15 Recall There are two basic types of testing Execution-based testing Non-execution-based testing V&V Verification Determine if the workflow was completed correctly Validation Determine if the product as a whole satisfies its requirements Remember The terms verify / review are also used for all non-executionbased testing Case Study log in Project Consider the requirement for log in project as follows Ability to login into the system using the correct user id and password Facility to change the password Ability to reset the password in case the user forgets the password Case Study log in Project cont.. Refined Requirement The purpose of the application is to allow the users to login into the system. The user will be required to enter a user is and password to login into the system Changing the password will be allowed any time In case the user forgets his/her password, there will be a provision of recovering the passwords. The recovery of the password will be dependent on the user specifying their ids correctly. The link to reset the password will be sent to their ids. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.15

16 Case Study log in Project cont.. System design Designing the screen elements Designing the look & feel of the screen Design database Case Study log in Project cont.. Login Screen Login ID Login Id Password Password Log In Reset Forgot Password? Change Password? Case Study log in Project cont.. Change Password Screen New Password New Password Change Reset Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.16

17 Login Table Row no User ID Name Password id Case Study log in Project cont.. Coding Case Study log in Project cont.. Login Screen Change Password screen Reset Password Screen 8h 6h 4h Case Study log in Project cont.. Testing 60 min Defect in change password screen Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.17

18 Scenario 1 Defect found at testing phase Update requirement document 10 min Update Design document 60 min Update code 90 min Retest 30 min Total time to fix defect 190 min Total time to complete change password screen Actual Coding Time 6h First time testing 60min Fix bug 190 Total time 610min Scenario 2 Using Review Technique Defect not found at the time of requirement review Defect found at design review Total time to implement Change password Screen? Reviews A review is any activity in which a work product is distributed / presented to reviewers who examine it and give feedback. In execution-based testing, one defect may mask another so several executions might be required. On the other hand, many different defects may be discovered in a single review. Because of the reuse of domain and programming knowledge, reviewers are likely to have seen the types of error that commonly arise In a peer review, co-workers of a person who created a software work product examine that product to identify defects and correct shortcomings Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.18

19 What Can be Reviewed All interim and final development work products, including: Requirements specifications User interface specifications and designs HLD, LLD Source code Test plans, designs, cases, and procedures Software development plans, including project management plan, configuration management plan, and quality assurance plan Objective of Review Verify whether the work product Correctly satisfies the specifications found in any predecessor work product. Adheres to standards. Suggest improvement opportunities to the author. Gain consensus amongst the project team members. Secure approval from stakeholders. Promote the exchange of techniques and education of the participants. Review: Categories Technical Ensure conformance to specifications. Check if development is in accordance to plans, standards and guidelines. Ensure changes to software elements are implemented properly. Managerial Check if project making progress as per plan. Identify and evaluate alternatives. Maintain global control through adequate resource allocation. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.19

20 Techniques Pass-around/ Informal review Walkthrough Inspection Technical Review Verification Techniques Passaround/ Informal Review Participants & Entry-Criteria Participants The author selects the participants in a passaround. No specific roles are assigned. However, it can be beneficial to ask individual reviewers to examine the work product from specific perspectives like: checking internal cross-references, confirming interface definitions, checking conformance to standards, or identifying grammatical and typographical errors. Entry Criteria The author selected a passaround review approach. The author has stated his objectives for the review. The document has been spell-checked. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.20

21 Tasks & Responsibilities Task Responsible Either distribute a physical or electronic copy of the work product Author to each reviewer, or create a file to which all reviewers have read/write access. Notify reviewers that work product is available and indicate the Author date by which review comments are to be provided. Either hand-write comments directly on the work product, or Reviewers enter comments into the work product file, using a revision marks convention or an annotation feature of the tool being used. If working with an individual copy of the work product, deliver the work product with comments to the author after completing the review. Tasks & Responsibilities Task After the final date of the review period, remove work product from the shared file location and examine added comments. Based on reviewer comments, perform any necessary rework of the work product. Responsible Author Author If a shared file was used to accumulate comments, delete each Author comment from the work product as it is resolved. Leave any comments that require discussion in the work product until agreement is reached on actions to be taken. Deliverables, Verification, Exit-Criteria Deliverables Modified work product Verification No verification of rework is required. The author is responsible for making appropriate decisions on issues and for correctly performing any rework. Exit-Criteria The author has addressed all review issues raised.. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.21

22 Review Techniques Walkthrough Walkthroughs A walkthrough is an informal way of presenting a technical document in a meeting. The author runs the walkthrough: calling the meeting, inviting the reviewers, soliciting comments and ensuring that everyone present understands the work product. After the meeting, the author should follow up with individual attendees who may have had additional information or insights. The walkthrough is preceded by preparation by designers/developers Lists of items Items not understood Items that appear to be incorrect Managing walkthroughs Document driven Person responsible for document walks the reviewers through the document Reviewers interrupt with comments triggered by the presentation Interactive process Not to be used for the evaluation of participants Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.22

23 Participants & Entry Criteria Participants The author selects the participants in a walkthrough. No specific roles are assigned. Entry Criteria The author selects a walkthrough review approach for the product being reviewed. The author has stated his objectives for the review. Tasks & Responsibilities Task Select review participants, obtain their agreement to participate, and schedule a walkthrough meeting. Distribute work product to reviewers prior to the meeting. Describe the work product to the reviewers during the meeting in any appropriate way. Lead discussion on the topics of interest or concerns about the work product. Present comments, possible defects, and improvement suggestions to the author. Based on reviewer comments, perform any necessary rework of the work product. Responsible Author Author Author Reviewers Author Deliverables, Verification, Exit-Criteria Deliverables Modified work product Verification No verification of rework is required. Changes are made at the author s discretion. Exit-Criteria The author has made any appropriate changes in the work product. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.23

24 Verification Techniques Inspections Participants In general, a work product should be reviewed by: The author of any predecessor document or specification Someone who must base their subsequent work on the work product Peers of the author Anyone responsible for a component to which the work product interfaces Attendance by anyone with supervisory authority over the author is by invitation of the author only. Roles - Responsibilities Author: Creator or maintainer of the work product to be inspected. Initiates the inspection process by asking the peer review coordinator to assign a moderator. States his objectives for the inspection. Delivers work product and its specification or predecessor document to moderator. Works with moderator to select inspectors and assign roles. Addresses items on the Issue Log. Reports rework time and defect counts to moderator. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.24

25 Roles - Responsibilities Moderator Plans, schedules, and leads the inspection events. Works with author to select inspectors and assign roles. Assembles inspection package and delivers it to inspectors at least 3 days prior to the inspection meeting. Determines whether preparation is sufficient to hold the meeting. If not, reschedules the meeting Roles - Responsibilities Moderator (Contd.) Facilitates inspection meeting. Corrects any inappropriate behavior. Solicits input from inspectors as reader presents each section of the work product. Records any action items or side issues that arise during the inspection. Leads inspection team in determining the work product appraisal. Serves as verifier or delegates this responsibility to someone else. Delivers completed Inspection Summary Report to the organization s peer review coordinator. Roles - Responsibilities Reader Presents portions of the work product to the inspection team to elicit comments, issues, or questions from inspectors. Recorder Records and classifies issues raised during inspection meeting. Inspector Examines work product prior to the inspection meeting to find defects and prepare for contributing to the meeting. Records preparation time. Participates during the meeting to identify defects, raise issues, and suggest improvements. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.25

26 Roles - Responsibilities Verifier Performs follow-up to determine whether rework has been performed appropriately and correctly. Peer Review Coordinator Custodian of the organization s inspection metrics ti database. Maintains records of inspections conducted and data from the Inspection Summary Report for each inspection. Generates reports on inspection data for management, process improvement team, and peer review process owner. Entry Criteria The author selected inspection as the approach for the product being reviewed. All necessary supporting documentation is available The author has stated his objectives for this inspection. Reviewers are trained in the peer review process. Documents to be inspected are identified with a version number. All pages are numbered and line numbers are displayed. The documents have been spell-checked. Entry Criteria Source code to be inspected is identified with a version number. Listings have line numbers and page numbers. Codecompileswithnoerrorsorwarningmessagesusing the project s standard compiler switches. Errors found using code analyzer tools have been corrected. For a re-inspection, all issues from the previous inspection were resolved. Any additional entry criteria defined for the specific type of work product are also satisfied. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.26

27 Planning: Tasks & Responsibilities Give moderator the work product to be inspected and supporting documents, such as specifications, predecessor documents, or pertinent test documentation: Author Determine whether work product satisfies inspection entry criteria: Moderator Based on the size and complexity of the work product, determine how many inspection meetings will be required. Moderator and Author Planning: Tasks & Responsibilities Select inspectors and assign roles to individuals. Gain agreement from the other inspectors to participate: Moderator and Author Determine whether an overview meeting is required: Author Schedule the inspection, and possibly overview, meetings and distribute a meeting notice: Moderator Distribute the inspection package to the participants at least 3 working days prior to the inspection meeting: Moderator or Author Overview: Tasks & Responsibilities Describe the important features of the work product to the rest of the inspection team. State inspection objectives: Author Evaluate the assumptions, history, and context of the work product: Inspectors Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.27

28 Preparation: Quiz Ask individual inspectors to prepare with specific objectives in mind, such as: checking for interface errors; checking traceability to, and consistency with, predecessor specifications; or checking conformance to standards. Moderator and Author Examine the work product, to understand it, find defects, and raise questions about it. Use the appropriate defect checklist to focus attention on defects commonly found in the type of product being inspected. Use other analysis methods to look for defects as appropriate. Inspectors Log minor defects found, such as typographical errors or style inconsistencies, on the Typo List. Deliver this to the author at or prior to the inspection meeting. Inspectors Inspection Meeting: Process - Quiz Open the Meeting: Introduce the participants (if necessary) and state their roles, state the purpose of the inspection, and direct inspectors to focus their efforts toward finding defects, not solutions. Remind participants to address their comments to the work product under review, not to the author. Moderator Establish Preparedness: Ask each inspector for his preparation time and record the times on the Inspection Summary Report. If preparation is insufficient, reschedule the meeting. Moderator Present Work Product: Describe portions of the work product to the inspection team. Reader Inspection Meeting: Process - Quiz Raise Defects and Issues: Point out concerns, potential defects, questions, or improvement opportunities after the reader presents each section. Inspectors Record Issues: Capture the information on the Issue Log for each issue raised. State aloud what was recorded to make sure it was recorded accurately. Recorder Answer Questions: Respond briefly to any specific questions raised, and contribute to defect detection based on special understanding of the work product. Author Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.28

29 Inspection Meeting: Process - Quiz Make Product Appraisal: After all meetings scheduled for a given inspection are complete, decide on the work product appraisal. If the inspectors disagree, assign the most conservative appraisal offered by any of the inspectors. Inspectors Sign Inspection Summary Report: All participants sign the Inspection Summary Report to indicate their agreement with the inspection outcome. Inspectors Collect Inspection Feedback. Ask the inspectors to evaluate the inspection and suggest improvements, using the Inspection Lessons Learned Questionnaire. Moderator Information to Record for Each Defect Found Origin Development phase in which the defect was introduced Type Missing (something needs to be there but is not) Wrong (something is erroneous or conflicts with something else) Extra (something unnecessary is present) Usability Performance Non-defect issue (question, point of style, suggestion, clarification needed) Severity Major (could cause product failure or cost significantly more to correct in the future) Minor (non-fatal error, cosmetic problem, annoyance, or a workaround is available) Location Page and line or section number where the defect is located Description Concise description of the issue or possible defect Inspected Work Products: Possible Appraisals Appraisal Accepted As Is Accept Conditionally Re-inspect Following Rework Meaning Modifications may be required in the work product, but verification of the modification is not necessary. Defects must be corrected, and the changes must be verified by the individual named on the Inspection Summary Report. A substantial portion of the product must be modified, or there are many changes to make. A second inspection is required after the author has completed rework. Inspection Not Completed A significant fraction of the planned material was not inspected, or the inspection was terminated for some reason. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.29

30 Rework Task Correct defects and typos found, resolve issues raised, and modify work product accordingly. Mark issues list to indicate action taken. Correct any other project documents based on defects identified in the inspected work product. Record any uncorrected defects in the project s defect tracking system. If rework verification is not needed, report the number of major and minor defects found and corrected and the actual rework effort to the moderator. Record the actual rework effort on the Inspection Summary Report. Responsible Author Author Author Author Moderator Follow-Up Task Responsible Confirm that the author has addressed every item on the Issue Log. Determine whether the author made appropriate decisions as to which defects not to correct and which improvement suggestions not to implement. Verifier Examine the modified work product to judge whether the Verifier rework has been performed correctly. Report any findings to the author, so rework can be declared complete, incorrect rework can be redone, or items that were not originally pursued can be addressed. Report the number of major and minor defects found and corrected and the actual rework effort to the moderator. Author Follow-Up Task Responsible Check whether the exit criteria for the inspection and for the peer review process have been satisfied. If so, the inspection is complete. Moderator Check the baselined work product into the project s configuration management system. Author Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.30

31 Measurements The moderator shall collect the data items from each inspection. These data items are used to calculate the process metrics and to monitor and improve the inspection process. The moderator shall record the data items in the appropriate spaces on the Inspection Summary Report and Issue Log and report them to the organization s peer review coordinator. The peer review coordinator shall maintain these metrics in a repository and produce periodic reports of summary data for practitioners and managers Data to be Collected Data Item Effort: Planning Effort: Overview Effort: Preparation Effort: Rework Time: Meeting Definition total labor hours spent by the moderator and author in planning, scheduling meetings, assembling, duplicating, and distributing materials, and any other related tasks total labor hours spent by the participants in an overview meeting, if one was held total labor hours spent by the inspectors and author preparing for the inspection total labor hours the author spent correcting defects in the initial deliverable and making other improvements; include verification time from the follow-up stage duration of the inspection meeting in hours Data to be Collected Data Item Defects Found: Major, Minor Defects Corrected: Major, Minor Size: Planned, Actual Number of Inspectors Definition total number of major and minor defects found by the inspection team; do not include non-defect issues raised, such as questions, requests for clarification, points of style, or items from the Typo Lists total number of major and minor defects corrected during rework total physical lines of code (not including comments and blank lines) or number of document pages that were planned for inspection and that were actually inspected number of active participants in the inspection meeting Inspection Appraisal inspection team s decision about disposition of the inspected work product (accepted as is, accepted conditionally, re-inspect following rework) Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.31

32 Rules to be followed Inspection should be carried out by 3-6 participants All participants can act as inspector The author shall not act as inspection leader The author should not act as reader or recorder Roles may be shared among the team members Individual participant may act in more than one role Deliverables 1. Baselined work product 2. Completed Inspection Summary Report 3. Completed Issue Log 4. CompletedTypoLists 5. Counts of defects found and defects corrected Exit-Criteria 1. All of the author s inspection objectives are satisfied. 2. Issues raised during the inspection are tracked to closure. 3. All major defects are corrected. 4. Uncorrected defects are logged in the project s defect tracking system. 5. The modified work product is checked into the project s configuration management system. 6. If changes were required in earlier project deliverables, those deliverables have been correctly modified, checked into the project s configuration management system, and any necessary regression tests were passed. 7. Moderator has collected and recorded the inspection data. 8. Moderator has delivered the completed Inspection Summary Report and defect counts to the peer review coordinator Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.32

33 Inspection Vs. Walkthroughs Walkthrough Inspection Adv. Circulation of Documents Not Required Yes Adv. Preparation by No Yes reviewers Presentation in the Authors Readers meeting by Inspection Vs. Walkthroughs Walkthrough Inspection Rigor Informal to formal Formal Purpose Judge quality, find defects, Measure/improve quality of training product and process Effectiveness Low to medium Low to very high, depending on training and commitment Which Review Method to Use Work products that fit in one or more of categories discussed, are considered high risk. A product is considered low risk if an undetected error will not significant affect the project s ability to meet its schedule, quality, cost, and feature objectives. Use inspections for high-risk work products, or the high-risk portions of large products, and for major work products that are about to be baselined. Less formal reviews are acceptable for other work products. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.33

34 Identifying High Risk Components Components that use new technology, techniques, or tools Key architectural components Complex logic or algorithms that are difficult to understand but must be accurate and optimized Mission-critical / security-critical components with dangerous failure modes. Components having many exception conditions or failure modes. Components that are intended to be reused Identifying High Risk Components Components that will serve as models or templates for other components Components that affect multiple portions of the product Complex user interfaces Components created by less experienced developers Code modules having high complexity Modules having a history of many defects or changes Verification Techniques Code Review Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.34

35 Code Review A code review is a special kind of inspection in which the team examines a sample of code and fixes any defects in it. In a code review, a defect is a block of code which does not properly implement its requirements, which does not function as the programmer intended, or which is not incorrect but could be improved For example, it could be made more readable or its performance could be improved Good candidates for code review A portion of the software that only one person has the expertise to maintain Code that implements a highly abstract or tricky algorithm An object, library or API that is particularly difficult to work with Code written by someone who is inexperienced or has not written that kind of code before, or written in an unfamiliar language Code which employs a new programming technique An area of the code that will be especially catastrophic if there are defects Checklist for Code Review: Example Data declaration All variables declared? Default values understood? Arrays and strings initialized? Variables with similar il names? Correct initialization? Control flow Each loop terminates? DO/END statements match? Input/output OPEN statements correct? Format specification correct? End-of-file case handled? Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.35

36 Verification Techniques Deskchecks Deskchecks A deskcheck is a simple review in which the author of a work product distributes it to one or more reviewers. The author sends a copy of the work product to selected project team members. The team members read it, and then write up defects and comments to send back to the author. Deskchecks Unlike an inspection, a deskcheck does not produce written logs which can be archived with the document for later reference. Deskchecks can be used as predecessors to inspections. In many cases, having an author of a work product pass his work to a peer for an informal review will significantly reduce the amount of effort involved in the inspection. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.36

37 Verification Techniques Pair Programming Types of Review: Pair Programming Pair programming is a technique in which two programmers work simultaneously at a single computer and continuously review each others work. Although many programmers were introduced to pair programming as a part of Extreme Programming, it is a practice that can be valuable in any development environment. Pair programming improves the organization by ensuring that at least two programmers are able to maintain any piece of the software. Types of Review: Pair Programming In pair programming, two programmers sit at one computer to write code. Generally, one programmer will take control and write code, while the other watches and advises. Some teams have found that pair programming works best for them if the pairs are constantly rotated; this helps diffuse the shared knowledge throughout the organization. Others prefer to pair a more junior person with a more senior for knowledge sharing. The project manager should not try to force pair programming on the team; it helps to introduce the change slowly, and where it will meet the least resistance. It is difficult to implement pair programming in an organization where the programmers do not share the same nine-to-five (or ten-to-six) work schedule. Some people do not work well in pairs, and some pairs do not work well together. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.37

38 Testing Software Requirements User being the dominant party, should take responsibility for requirement phase testing In case of multiple users, responsibility may be assigned to a committee, which may be the same committee that develops the requirements Objective Most cost effective phase to detect a system flaw Primary objectives of testing Determine that the requirements fairly represent what the user needs Determine that the needs have been defined and documented Verify that a cost/benefit study has been performed and that it is reasonable Determine the business problem has been solved Verify that the control requirements have been specified Verify that a reasonable process was followed in developing the business solution Input The most common deliverables from the requirement phase Proposal to management describing the problem, the alternatives, and proposing a solution Cost/benefit study describing the economics of the proposed solution Detailed description of the recommended solution, highlight the recommended method for satisfying those needs List of system assumptions, such as the life of the project, the value of the money, the average skill of the user and so on Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.38

39 Prepare a risk matrix Do procedure Perform a Test Factor Analysis for the Requirement Phase Conduct Requirements Walkthrough Do procedure cont.. For example need to verify Requirements Comply with Methodology (Methodology Test Factor) Functional Specification Defined (correctness test factor) Usability Specifications determined (Ease-of-use Test factor) Maintenance Specifications Determined (Maintainable Test factor) Portability needs Determined (portable Test factor) System Interface Defined (Coupling Test Factor) Performance Criteria Established(Performance Test factor) Check Procedures and output Only output is a report indicating requirements deficiencies Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.39

40 Design Phase Testing Provides the opportunity to test the structure Overview Involves the close working relationship between the user and the system designer System design is an IT responsibility Objective Design phase concerns are described and a testing process is proposed to assess whether those concerns have been adequately addressed by the project team Scoring the design success factor, conducting a design review and inspecting design deliverables Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.40

41 Inputs The common design phase deliverables for testing Input specifications Processing specifications File specifications Output specifications Control specifications System flowcharts Hardware and software requirements Manual operating procedure specifications Data retention policies Concerns to be analyzed Data integrity controls designed Authorization rules designed File integrity controls designed Contingency plan designed Method to achieve service level designed Access procedure defined Design complies with methodology Concerns to be analyzed cont.. Design conforms to requirements Design facilitates use Design is maintainable Design is portable Interface design is complete Design achieve criteria Needs communicated to operations Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.41

42 Output The moderator s certification of the product, releasing the product to the next phase of the process Test Software Design Some important concerns to be analyzed during this phase are: Conformance to requirements Compliance with methodology Design of Data Integrity Controls, Authorization Rules, Contingency Plans Audit Trails, Maintainability Portability User ManualTesting Provides the opportunity to test the documentation Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.42

43 Effective Documentation Improves Usability Lowers customer support Improves reliability Increase Maintainability Improve installability Enhance salability Reduces liability Tester s Objective Checking the technical accuracy of every word Verify that writer describe the product as it is Look out for missing features No authority to demand stylistic changes Benefits You will find more bugs than you expect An important source of real world combination test cases Bug reports arising out of documentation testing are particularly credible Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.43

44 How to do Use the program exactly as the manual says Try every suggestion Check every statement of fact and every obvious inference from the stated facts, instructions and suggestions Project Audits Project Audits( 审计 ) Project Audits In a process oriented approach for software development, two key task are: Process definition Process implementation. The process definition activities deal with identifying & specifying processes, which when followed will give Good Quality & Productivity. The process implementation activities ensure that the defined process are followed in the project. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.44

45 Cont Unless the projects adhere to the standard process one cannot find the Capability of the process. Therefore without knowing the capability of a process one cannot Improve the process. Since the Process are executed by people, there is a possibility that the process may be skipped. Cont The reason for not following the process are: People tend to take shortcuts It maybe do to pressure of deadline Or maybe some process are not clear It may also be due to overconfidence And also people resist changes, because it restrict their freedom Taking shortcut or skipping a process may not always lead to project failure. A project could potentially fail if the processes are not followed. Cont An analogy for this could be Traffic Rules. Just because a rash driver failed to observe traffic rule & reaches home safely does not mean traffic rules is unnecessary. Therefore a active effort is needed to ensure compliance to the defined process. The basic purpose of AUDIT is to ensure compliance to the defined process. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.45

46 Cont Audits are essential part of Quality Assurance KPA of Level 2 It is also a requirement of ISO 9000 Quality System There are many ways of Organizing AUDIT in an Organization. We shall study the Audit Mechanism employed at Infosys. Project Audit (The Challenge) As the projects become larger and more complex, understanding and providing effective validation of the project management processes is a significant challenge for today s information systems organizations. In addition to the awareness and implementation of the project management procedures, recognition of the quality of the standards and practices is critical for continued performance improvement. The Objective of Audit: Insight into the project team s use of the project management standards Identification of the project s Project Management related risks Detail corrective action plan for addressing the risks, incomplete procedures and standards training Awareness of the areas of opportunity for improvement of the Project Management methods and behaviors Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.46

47 Audit Process Auditing is a systematic & independent examination of various activities of project execution. It is intended to determine compliance with the quality system of a organization Its main focus is on implementing the process of an Organization. It is also used to determine the effectiveness of a process & to identify area of Improvement. Cont Audit can be Internal or External External Audit is generally employed for some type of certification Internal audit is conducted by the Organization using people of the Organization. The primary goal is to ensure compliance with Organization s process & help in process improvement. Cont To ensure reasonable degree of compliance with the defined process, audit must be done regularly. The people performing audit could be member of SEPG or person who has maturity & stature to assess the implementation on a project objectively. It could also be performed by other project members. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.47

48 Cont When others perform Audit, they can learn & give advice. Since they become Law Keepers they also respect & appreciate Law. The Audit activity has three components: Planning Auditing Follow-up Planning Before audits are conducted, they must be carefully planned to achieve optimal result. Planning of Audit operates at three level: Strategy High-level plan Detailed schedule. Audit strategy The Audit strategy defines how Audit will be scheduled & planned, so as to monitor compliance & the effectiveness of processes. The strategy at Infosys are: Audit are conducted d monthly During a audit, a sample of the project is selected for audits, but the projects selected are not known before the month of Audit Each month has some focus area for audit, which will be examined in detail during audit. Bharati Vidyapeeth s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.48