THE FUTURE CONTENTS. Software Testing

Transcription

1 THE FUTURE CONTENTS I. Software Quality Assurance: 1. Quality is Free 2. Testing and Quality Assurance in the Workplace 3. Software Testing 4. Quality Assurance 5. Other Names for Software Testing Groups 6. Test Management and Organizational Structures 7. Capability Maturity Model (CMM) 8. ISO II. Your Career as a Software Tester: 1. Your Job as a Software Tester 2. Finding Software Testing Position 3. Gaining Hands-on Experience 4. Internet Links 5. Professional Organizations

2 I. Software Quality Assurance: (Question: Explain the SQA in software testing = 4 marks) 1. Quality is Free 1. The cost of quality into two categories: the costs of conformance and the costs of nonconformance. 2. The costs of conformance are all the costs associated with planning and running tests just one time, to make sure that the software does what it s intended to do. 3. If bugs are found and you must spend time isolating, reporting, and regression testing them to assure that they re fixed, the costs of nonconformance go up. 4. These costs, because they are found before the product is released, are classified as internal failures and fall mostly on the left side of Figure 1. Figure 1 5. If bugs are missed and make it through to the customers, the result will be costly product support calls, possibly fixing, retesting, and releasing the software, and in a worst-case scenario a product recall or lawsuits. 6. The costs to address these external failures fall under the costs of nonconformance and are the ones on the right side. 2. Testing and Quality Assurance in the Workplace 1. Testing, Software Quality Assurance, Software Quality Control, Software Verification and Validation, Software Integration and Test, or one of many others. 2. Frequently these names are used interchangeably or one is chosen over the others because it sounds more official Software Quality Assurance Engineer versus Software Tester, for example.

3 3. It s important to realize, though, that these names have deeper meanings and aren t necessarily plug-in replacements for each other. 4. On one hand there s the philosophy that it s only a name, that what you ultimately do in your job is what counts. 5. On the other hand, your job title or your group s name is what others on the project team see. 6. That label indicates to them how they will work with you and what expectations they will have, what deliverables you will provide to them, and what they will give to you. 7. The following sections define a few of the common software-testgroup names and should help clarify the differences among them. 3. Software Testing 1. The goal of a software tester is to find bugs, find them as early as possible, and make sure they get fixed. Being a software tester and working under this charter has a unique and very important characteristic. 2. You aren t responsible for the quality of the software! This may sound strange, but it s true. 3. You didn t put the bugs in the software, you had your project manager and the programmers review and approve your test plan, you executed your plan to the letter and despite all that effort, the software still had bugs. It s not your fault! 4. Quality Assurance 1. A Software Quality Assurance person s main responsibility is to examine and measure. 2. The current software development process and find ways to improve it with a goal of preventing bugs from ever occurring. 5. Other Names for Software Testing Groups 1. Depending on where you work, your test group may use one of many other names to identify itself. Software Quality Control (SQC) is one that s frequently used. 2. This name stems from the manufacturing industry where QC inspectors sample products taken off the manufacturing line, test them, and, if they fail, have the authority to shut down the line or the entire factory. 3. Few, if any, software test groups have this authority even ones that call themselves Software QC. Software Verification and Validation is also commonly used to describe a software test organization. 4. This name is one that actually works pretty well. Although it s a bit wordy, it states exactly what the test group is responsible for. 5. It s even possible to have two groups, one for verification and one for validation.

4 6. Test Management and Organizational Structures 1. Besides a test group s name and its assumed responsibilities, there s another attribute that greatly affects what it does and how it works with the project team. 2. That attribute is where it fits in the company s overall management structure. 3. A number of organizational structures are possible, each having its own positives and negatives. 4. Some are claimed to be generally better than others, but what s better for one may not necessarily be better for another. 5. If you work for any length of time in software testing, you ll be exposed to many of them. 6. Here are a few common examples. Figure 2 In this organization, the teams responsible for software quality report directly to senior management, independent and on equal reporting levels to the individual projects. The level of authority is often at the quality assurance level, not just the testing level. The independence that this group holds allows them to set standards and guidelines, measure the results, and adopt processes that span multiple projects. Information regarding poor quality (and good quality) goes straight to the top.

5 Figure 3 7. Capability Maturity Model (CMM) (Question: Explain the concept of CMM with diagram explain its levels = 8 marks) 1. The Capability Maturity Model for Software (CMM or SW-CMM) is an industrystandard model for defining and measuring the maturity of a software company s development process and for providing direction on what they can do to improve their software quality. 2. It was developed by the software development community along with the Software Engineering Institute (SEI) and Carnegie Mellon University, under direction of the U.S. Department of Defense. Figure 4

6 3. If you take the entire universe of software companies today, most are at Maturity Level 1, many are at Maturity Level 2, a few are at Maturity Level 3, a handful are at Maturity Level 4, and an elite couple are at Maturity Level 5. Here are descriptions of the five CMM Maturity Levels: Level 1: Initial: The software development processes at this level are ad hoc and often chaotic. The project s success depends on heroes and luck. There are no general practices for planning, monitoring, or controlling the process. It s impossible to predict the time and cost to develop the software. The test process is just as ad hoc as the rest of the process. Level 2: Repeatable: This maturity level is best described as project-level thinking. Basic project management processes are in place to track the cost, schedule, functionality, and quality of the project. Lessons learned from previous similar projects are applied. There s a sense of discipline. Basic software testing practices, such as test plans and test cases, are used. Level 3: Defined: Organizational, not just project specific, thinking comes into play at this level. Common management and engineering activities are standardized and documented. These standards are adapted and approved for use on different projects. The rules aren t thrown out when things get stressful. Test documents and plans are reviewed and approved before testing begins. The test group is independent from the developers. The test results are used to determine when the software is ready. Level 4: Managed: At this maturity level, the organization s process is under statistical control. Product quality is specified quantitatively beforehand (for example, this product won t release until it has fewer than 0.5 defects per 1,000 lines of code) and the software isn t released until that goal is met. Details of the development process and the software s quality are collected over the project s development, and adjustments are made to correct deviations and to keep the project on plan. Level 5: Optimizing: This level is called Optimizing(not optimized ) because it s continually improving from Level 4. New technologies and processes are attempted, the results are measured, and both incremental and revolutionary changes are instituted to achieve even better quality levels. Just when everyone thinks the best has been obtained, the crank is turned one more time, and the next level of improvement is obtained. 8. ISO 9000 (Question: Explain the ISO 9000 standard and give reasons why it works well. = 4 marks) 1. Another popular set of standards related to software quality is the International Organization for Standardization s (ISO) ISO is an international standards organization that sets standards for everything from nuts and bolts to, in the case of ISO 9000, quality management and quality assurance.

7 ISO 9000 works well for the following reasons: 1. It targets the development process, not the product. 2. It s concerned about the way an organization goes about its work, not the results of the work. 3. ISO 9000 dictates only what the process requirements are, not how they are to be achieved. 4. Performing design reviews is a good exercise that a responsible design team should do (which is why it s in ISO 9000), but exactly how the design review is to be organized and run is up to the individual team creating the product. 5. ISO 9000 tells you what to do but not how to do it. Some of the requirements in ISO include 1. Develop detailed quality plans and procedures to control configuration management, product verification and validation (testing), nonconformance (bugs), and corrective actions (fixes). 2. Prepare and receive approval for a software development plan that includes a definition of the project, a list of the project s objectives, a project schedule, a product specification, a description of how the project is organized, a discussion of risks and assumptions, and strategies for controlling it. II. Your Career as a Software Tester: 1. Your Job as a Software Tester (Question: Explain the various jobs of a software tester? = 4 marks) 1. Software test technician:- This is often a true entry-level test position. You would be responsible for setting up test hardware and software configurations, running simple test scripts or test automation, and possibly working with beta sites to isolate and reproduce bugs. Some work can become mundane and repetitive, but being a test technician is a good way to become introduced to software testing. 2. Software tester or software test engineer: - Most companies have several levels of software testers based on experience and expertise. An entry-level tester may perform the duties of technician, working their way up to running more advanced and complex tests. As you progress, you ll write your own test cases and test procedures and might attend design and specification reviews. You ll perform testing and isolate, reproduce, and report the bugs you find. If you have programming abilities, you ll write test automation or testing tools and work closely with the programmers as you perform white-box testing. 3. Software test lead: - A test lead is responsible for the testing of a major portion of a software project or sometimes an entire small project. They often generate the test plan for their areas and oversee the testing performed by other testers. They re frequently involved in collecting metrics for their products and presenting them to management. They usually also perform the duties of a software tester.

8 4. Software test manager: - A test manager oversees the testing for an entire project or multiple projects. The test leads report to them. They work with the project managers and development managers to set schedules, priorities, and goals. They re responsible for providing the appropriate testing resources people, equipment, space, and so on for their projects. They set the tone and strategy for the testing their teams perform. 2. Finding Software Testing Position 1. Use the Internet, a quick search done using several job search engines just before this book went to print found more than 1,000 open software testing positions at companies all around the country. Many of these positions were for entry-level testers. 2. There were jobs testing music software, interactive TV, networking, medical equipment, Web sites, and more. You name it. 3. Look through newspapers and magazines, as with the Internet, most large city newspapers list numerous software testing jobs every weekend in their hightech or computer help wanted ads. Computing and programming magazines are also a good source for job ads. 4. Simply call and ask. Do you like a certain technology or a specific software application, or are you interested in a particular area of computing? Look up the company or companies writing that software, give them a call or send them your résumé. Often there are open software testing jobs that never make it to the help wanted ads. 5. Resourceful testers grab them before anyone knows they re available. 6. Look into internships and co-ops.if you re in college, take a summer or a semester as an intern or co-op at a software company as a software tester. Most provide great positions where you ll actually be contributing to the success of a real product as you gain experience in the field. If you work hard, you might be offered a full-time position after you graduate. 7. Take temporary jobs, many software companies hire software testers under temporary contracts to help run test scripts as a product nears completion. The jobs may last just one month or several months, but you ll gain valuable experience and can take on more difficult jobs each time, learning as you go. Some software testers love to work this way because it gives them the opportunity to work at different companies and try their hand testing very different software. 3. Gaining Hands-on Experience 1. Software testing is just like most other computer topics you can read about it all day, but until you actually try the things you ve read about, it s difficult to comprehend. 2. For this reason, one of the best ways to learn software testing is to try it yourself, on your own computer with your own software.

9 4. Internet Links The Internet has a wealth of material on software testing. Web sites dedicated to software testing and software bugs that will get you started: 1. BugNet ( publicizes bugs found in commercial software and points you to the appropriate fixes. 2. Software Testing Hotlist ( lists dozens of pointers to software testing related Web sites and articles. 3. Software Testing Online Resources ( is the selfproclaimed nexus of Software Testing Online Resources designed to be a first-stop on the Web for software testing researchers and practitioners. 4. QA Forums ( provide ongoing discussions of software testing, automated testing, test management, test tools, and many other topics. 5. The newsgroup comp.software.testingand its FAQ (frequently asked questions) document at lots of ongoing discussions by testers and test managers regarding tools, techniques, and projects. 6. The newsgroup comp.risksdescribes and analyzes recent software failures. 7. The Risks Digest (catless.ncl.ac.uk/risks/) is a forum on risks to the public in computers and related systems. 5. Professional Organizations 1. The American Society for Quality (ASQ) at its software division at the National Quality Forum annually in October (national quality month). They publish journals and articles on quality and administer the Certified Quality Engineer (CQE) and the Certified Software Quality Engineer (CSQE) designation. 2. The Association of Computing Machinery (ACM) at its Special Interest Group on Software Engineering (SIGSOFT) at more than 80,000 members in educational and scientific computing. The software engineering