Chapter 6. Software Quality Management & Estimation

Transcription

1 Chapter 6 Software Quality Management & Estimation

2 What is Quality Management Also called software quality assurance (SQA) s/w quality:- It is defined as the degree to which a system, components, or process meets specified requirements. OR The degree to which a system, components or process meets customer or user needs or expectation. Reduces the amount of rework, which results in lower costs and improved time to market Encompasses A software quality assurance process Specific quality assurance and quality control tasks (including formal technical reviews and a multi-tiered testing strategy) Effective software engineering practices (methods and tools) Control of all software work products and the changes made to them A procedure to ensure compliance with software development standards Measurement and reporting mechanisms 2

3 Quality Defined Defined as a characteristic or attribute of something Refers to measurable characteristics that we can compare to known standards In software it involves such measures as cyclomatic complexity, cohesion, coupling, function points, and source lines of code Heart of quality is variation control A software development organization should strive to minimize the variation between the predicted and the actual values for cost, schedule, and resources They should make sure their testing program covers a known percentage of the software from one release to another One goal is to ensure that the variance in the number of bugs is also minimized from one release to another 3

4 Quality Defined (continued) Two kinds of qualities like Quality of design The characteristic that designers specify for an item This encompasses requirements, specifications, and the design of the system Quality of conformance (i.e., implementation) The degree to which the design specifications are followed during manufacturing This focuses on how well the implementation follows the design and how well the resulting system meets its requirements Quality also can be looked at in terms of user satisfaction User satisfaction = compliant product + good quality + delivery within budget and schedule 4

5 Quality Control It focuses on operational techniques & the activities used to fulfill & verify requirements of quality. Involves a series of inspections, reviews, and tests used throughout the software process Key concept of quality control is that- all work products have defined & measurable specifications Ensures that each work product meets the requirements placed on it Includes a feedback loop to the process that created the work product This is essential in minimizing the errors produced Combines measurement and feedback in order to adjust the process when product specifications are not met Requires all work products to have defined, measurable specifications to which practitioners may compare to the output of each process Feedback loop useful to minimize the defects in products. 5

6 1.Quality Assurance Functions It is set of auditing and reporting functions that assess the effectiveness and completeness of quality control activities Main objective of it is to support management with the data that is required to be informed about product quality. Provides insight into the quality of the products If problem is identified by the data provided through quality assurance, then management responsibility to focus on the problems & they can apply the necessary resources to resolve quality issues 6

7 2. The Cost of Quality Includes all costs incurred in the pursuit of quality or in performing quality-related activities To find out current cost of quality, Is studied to Provide a baseline for the current cost of quality Identify opportunities for reducing the cost of quality Provide a normalized basis of comparison (which is usually dollars) Involves various kinds of quality costs (See next slide) Increases dramatically as the activities progress from Prevention Detection Internal failure External failure 7

8 Kinds of Quality Costs Prevention costs Quality planning, formal technical reviews, test equipment, training etc. Appraisal costs It include in process & inter process Inspections, equipment calibration and maintenance, testing Failure costs subdivided into internal failure costs and external failure costs Internal failure costs Incurred when an error is detected in a product prior to shipment Include rework, repair, and failure mode analysis External failure costs Involves defects found after the product has been shipped Include complaint resolution, product return and replacement, help line support, and warranty work 8

9 Software Quality Assurance Q. What is SQA? Their activities

10 Software Quality Defined Definition: "Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software SQA consists of a means of monitoring the s/w engg. Processes & methods used to ensure quality. SQA is a planned & systematic way to evaluate quality of s/w product standards, processes & procedures. SQA is an umbrella activity that is applied though s/w process. 1. An SQA process. 2. Specific quality assurance & quality control tasks. 3. Effective s/w engg. Practice. 4. Control of all s/w product. 5. A procedure to ensure compliance with s/w development standards. 6. Measurement & reporting mechanism. (More on next slide) 10

11 Software Quality Defined (continued) This definition emphasizes three points Software requirements are the foundation from which quality is measured; lack of conformance to requirements is lack of quality Specified standards define a set of development criteria that guide the manner in which software is engineered; if the criteria are not followed, lack of quality will almost surely result A set of implicit requirements for easy to use & good maintainability. if software ensures to its implicit requirements but fails to achieve implicit requirements, software quality is not good. SQA includes different tasks 1. the s/w engineers who performs technical work 2. The SQA group with responsibilities like record keeping, analysis, quality assurance planning, reporting, maintainability etc. Software engineers apply solid technical methods and measures, conduct formal technical reviews, and perform well-planned software testing 11

12 The SQA Group Serves as the customer's in-house representative Assists the software team in achieving a high-quality product Views the software from the customer's point of view Does the software adequately meet quality factors? Has software development been conducted according to preestablished standards? Have technical disciplines properly performed their roles as part of the SQA activity? Performs a set of activities that address quality assurance planning, oversight, record keeping, analysis, and reporting (See next slide) 12

13 SQA Activities Prepares an SQA plan for a project:- - plan is developed & is reviewed by all the people who are involved in project. Participates in the development of the project's software process description:- - The s/w team selects a work process to be performed. - SQA group reviews the process description for compliance with organizational policy, internal & external s/w standards. Reviews software engineering activities(analysis, design, construction, verification, & management) to verify compliance with the defined software process Audits designated software work products to verify compliance with those defined as part of the software process:- - SQA group also verifies that corrections have been made, & periodically reports the results of its work to the project manager. 13

14 SQA Activities Ensures that deviations in software work and work products are documented and handled according to a documented procedure Records any noncompliance and reports to senior management Configuration management monitoring :- - it is also called change control management. - controlling the changes to the s/w items. - it help to store & retrieve the configurable. Helps to collect and analyze software metrics 14

15 Statistical Software Quality Assurance

16 Concept of statistical SQA Statistical analysis is of error or defects helps to focus & concentrate on probable areas. Statistical quality assurance provides substantial quality improvements The following Process steps are involved:- 1. Collect and categorize information (i.e., causes) about software defects that occur. 2. Try to trace each error & defect. 3. Using the Pareto principle (80% of defects can be traced to 20% of all causes), isolate the 20% 4. Once, origin of causes have been identified, move to correct the problem that have produced the errors & defects.

17 A Sample of Possible Causes for Defects Incomplete or erroneous specifications Misinterpretation of customer communication Intentional deviation from specifications Violation of programming standards Errors in data representation Inconsistent component interface Errors in design logic Incomplete or erroneous testing Inaccurate or incomplete documentation Errors in programming language translation of design Ambiguous or inconsistent human/computer interface 17

18 Six Sigma Popularized by Motorola in the 1980s Is the most widely used strategy for statistical quality assurance Uses data and statistical analysis to measure and improve a company's operational performance Identifies and eliminates defects in manufacturing and servicerelated processes It used to eliminates error or faults or bugs in s/w by evaluating the performance of the s/w process. It minimizes the cost of poor quality. It monitors day-to-day activities of an organization. The "Six Sigma" refers to six standard deviations (3.4 defects per a million occurrences) (More on next slide) 18

19 Three core steps Six Sigma (continued) Define customer requirements, deliverables, and project goals via well-defined methods of customer communication Measure the existing process and its output to determine current quality performance (collect defect metrics) Analyze defect metrics and determine the vital few causes (the 20%) Two additional steps are added for existing processes (and can be done in parallel) Improve the process by eliminating the root causes of defects Control the process to ensure that future work does not reintroduce the causes of defects 19

20 Six Sigma (continued) All of these steps need to be performed so that you can manage the process to accomplish something Manage and improve the work process Control the work process Analyze the work process Measure the work process Define the work process The work to be done 20

21 Concept of DMAIC 1. DMAIC(Define, Measure, Analyze, Improve, Control) - DMAIC checks whether a process is performing correctly. - DMAIC improves the system by improving the process. - It used when a product or process is in existence & is not meeting customer specification. Sr. no. Attributes Functions 1. Define Identify the goal of project & user requirement. 2. Measure Quantify the process to determine the current performance of the s/w. 3. Analyze Examine & determine the causes of the error. 4. Improve Improve the process by eliminating errors. 5. Control Determine the future process performance for prevention of (More on next slide) 21 errors.

22 Concept of DMDAV 1. DMDAV(Define, Measure, Design, Analyze, verify) - It used when product or process is in existence at an organization but is not meeting customer specification or requirements. Sr. no. Attributes Functions 1. Define Specifies the goal of project & user requirements. 2. Measure Evaluate and determine user requirements. 3. Analyze Identify performance goals and determine how process inputs are likely to affect process outputs. 4. Design Work out details, optimize the methods, run simulations if necessary and plan for design verification. 5. Verify The process model will avoid the defects and meets the customer requirements. (More on next slide) 22

23 SQA Plan ISO 9000 quality standards International organization for standardizations established the term ISO It refers to a set of quality management standard. The ISO 9000 standards are maintained by ISO and administered by accreditation and certifications bodies. ISO first published its quality standard in1987, revised them in 1994 and then republished an updated version in 2000 and new standard are referred as ISO 9000:2000 standards. ISO 9000:2000 standard applied to all kind of organization in all kind of areas include manufacturing, processing, printing, electronics, banking, telecommunications, agriculture, government, educations, s/w development, biotechnology and so on. Experience and knowledge from international experts who contributed in ISO technical committee.

24 Benefits ISO standard Customer focus. Leadership. Involvement of people. Process approach. System approach to management. Continuous improvement. Decision making. Mutually beneficial supplier relationships. 24

25 Benefits ISO standard Increase in efficient use of organization resources. Customer satisfaction. People know the exact goal of organization and will be motivated towards it. Full involvement of people who are committed and motivated. Opportunities for creative ideas and innovations. Lower cost. Increase profitability. Improve bottom line performance. Reduce waste and rework. Short cycle time. Results and improved consistent and predictable. Develop an ability to concentrate on key processes. Powerful and useful decision making with informed decisions. 25 Increase in an ability to create value for both organization and supplier.

26 CMMI History CMMI (Capability Maturity Model Integration) is a proven industry framework to improve product quality and development efficiency for both hardware and software. Sponsored by US Department of Defence in cooperation with Carnegie Mellon University and the Software Engineering Institute (SEI) Many companies have been involved in CMMI definition such as Motorola and Ericsson CMMI has been established as a model to improve business results A Capability Maturity Model (CMM) is a reference model of mature practices in a specified discipline, used to improve and appraise a group s capability to perform that discipline. CMM contains a list of key process areas that includes software engineering, project management, and process improvement divided into different levels 26

27 CMMI History CMMI, staged, uses 5 levels to describe the maturity of the organization, CMMI provides a way to focus and manage hardware and software development from product inception through deployment and maintenance. Behavioral changes are needed at both management and staff levels Initially a lot of investment required but, if properly managed, we will be more efficient and productive while turning out products with consistently higher quality. 27

28 CMMI Models within the Framework Models: Systems Engineering + Software Engineering (SE/SW) Systems Engineering + Software Engineering + Integrated Product and Process Development (IPPD) Systems Engineering + Software Engineering + Integrated Product and Process Development + Supplier Sourcing (SS) Software Engineering only The CMMI definition of Systems Engineering - The interdisciplinary approach governing the total technical and managerial effort required to transform a set of customer needs, expectations and constraints into a product solution and to support that solution throughout the product s life. This includes both hardware and software. Slide 28 of 146

29 CMMI Staged Representation - 5 Maturity Levels Level 4 Level 5 Optimizing Process performance continually improved through incremental and innovative technological improvements. Quantitatively Managed Processes are controlled using statistical and other quantitative techniques. Level 3 Defined Level 2 Managed Level 1 Processes are well characterized and understood. Processes, standards, procedures, tools, etc. are defined at the organizational level. Proactive. Processes are planned, documented, performed, monitored, and controlled at the project level. Often reactive. Initial Slide 29 of 146 Processes are unpredictable, poorly controlled, reactive.

30 Maturity Level 0 :- Not performed - one or more specific goals associated with the process area is not satisfied. Maturity Level 1:- Initial or performed - Maturity Level 1 deals with performed processes. - Processes are unpredictable, poorly controlled, reactive. - The process performance may not be stable and may not meet specific objectives such as quality, cost, and schedule, but useful work can be done. Slide 30 of 146

31 Maturity Level 2 :- Managed Maturity Level 2 deals with managed processes. A managed process is a performed process that is also: Planned and executed in accordance with policy Employs skilled people Adequate resources are available Controlled outputs are produced Stakeholders are involved The process is reviewed and evaluated for adherence to requirements Processes are planned, documented, performed, monitored, and controlled at the project level. Often reactive. The managed process comes closer to achieving the specific objectives such as quality, cost, and schedule. Slide 31 of 146

32 Maturity Level 3 Defined at the Organization Level Maturity Level 3 deals with defined processes. A defined process is a managed process that: Well defined, understood, deployed and executed across the entire organization. Proactive. Processes, standards, procedures, tools, etc. are defined at the organizational (Organization X ) level. Project or local tailoring is allowed, however it must be based on the organization s set of standard processes and defined per the organization s tailoring guidelines.

33 Maturity Level 4 :- Quantitatively Managed - There is an organizational responsibility to use statistical & other qualitative methods to control sub-processes. -i.e. collected process & product measurements must be used in process management. Maturity Level 6:- Optimizing - At this highest level, the organization must use the process & product measurements to drive process improvement. Slide 33 of 146

34 Sr. No. CMMI ISO 1. Capability maturity model integration 2. Used for system engg. & s/w development process. International standard organization Not for any specific business or service. Used by any organization. 3. It is a process model ISO is set of standards. 4. Very less or no documentation is Required Registration documentation is Required 5. Audit not required. Repeated audits are done. 6. CMMI is derived from the best practices followed by the leaders in the business. 7. It is designed & developed in s/w Engg. Institute. 8. Mainly focuses on inward part of the business process. 9. Focus on continuous quality improvement. ISO is certification tools ISO founded in 1947 having it s headquarter at Geneva, Switzerland. focuses on outward part Focuses on minimum requirements with indirect continuous 34 improvements.

35 McCall s Quality Factors There are two groups that affect quality. 1. Directly measurable factors like uncovered defects during process of testing. 2. Indirectly measurable factors like usability or maintainability of s/w. It focus on three important aspects of a s/w product. 1. Its operational characteristics. 2. Its ability to undergo change, 3. Its adaptability to new environments. 35

36 McCall's Quality Factors 1. Product operations factors: Correctness - The extent to which a program satisfies its specification and fulfills the customer's mission objectives. Reliability does it do it accurately all the time? The extent to which a program can be expected to perform its intended function with required precision. Efficiency - The amount of computing resources and code required by a program to perform its function. Integrity it is secure Extent to which access to software or data by unauthorized persons can be controlled. Usability it is designed for the user? Effort required to learn, operate, prepare input, and interpret output of a program. 36

37 2. Product Revision:- Maintainability can I fix it? it is nothing but the maintenance of the s/w product. Flexibility can I change it? Effort required to modify an operational program. Testability can I test it? Effort required to test a program to ensure that it performs its intended function. 3. Product Transition:- Portability will I be able to use it on another machine? Effort required to transfer the program from one hardware and/or software system environment to another. Reusability will I be able to reuse some of the s/w it is the ability of s/w products to be reused, in whole on in part, for new applications. Interoperability will I be able to interface it with another system? Effort required to couple one system to another. 37

38 End of Chapter 38