Software Quality Management

Transcription

1 Marco Scotto

2 Course Outline Introduction to Software Quality Software quality frameworks Product-oriented approaches Process-oriented approaches Management topics in Software Quality Testing Measurement methodologies Software inspection 2

3 Logistics Schedule Lecture Wednesday :30 Thursday :30 Lab Thursday 10:30-12:30 Copies of the slides will be available on Teleacademy web site ( 3

4 Bibliography Martin Wieczorek, Dirk Meyerhoff, B. Baltus, Software Quality: State of the Art in Management, Testing, and Tools, Springer- Verlag, 2001 N. E. Fenton, S. H. Pfleeger. Software Metrics: a rigorous and practical approach second edition, International Thomson Computer Press, UK, 1997 Ron. S. Kenett, Emanuel R. Baker, Software Process Quality, Marcel Dekker, Inc.,

5 Exam Oral: 40% Project: 60% 5

6 Introduction (1/3) Software presents both an opportunity and a threat In many applications, software is a critical component Examples: Elevators Airlines Telecommunications Medical devices 6

7 Introduction (2/3) From a survey of the Software Engineering Institute (SEI): 60% of software development organizations are at the lowest level of capability The assessment is based on the Capability Maturity Model (CMM) A framework for achieving process improvement The survey involved over 600 organizations worldwide 7

8 Introduction (3/3) The situation is even worse The survey involved organizations that have started improvement efforts or made a commitment to do so If we add in all the organizations that: have not assessments performed or have no plans to implement process improvement the number of organizations at the lowest level is probably 80% 8

9 Software Quality (1/3) What is the consequence of this low level of process capability in most software companies? The quality of software! How many users are really happy with the quality of their software products? Stories about people losing hours of work because the computer locked up at an inopportune time are very common 9

10 Software Quality (2/3) Definition of quality Degree of excellence, relative nature or kind of character, class or grade of thing as determined by this, general excellence" (The New Oxford Illustrated Dictionary) 10

11 Software Quality (3/3) Software definitions: Intellectual product consisting of information stored on a storage device (ISO/DIS 9000: 2000) Software is "intellectual creation comprising the programs, procedures, rules and any associated documentation pertaining to the operation of a data processing system" A software product is the "complete set of computer programs, procedures and associated documentation and data designated for delivery to a user" (ISO ) 11

12 Why does software have errors? When executing complex tasks, human beings do commit errors This cannot be avoided Experienced programmers commit 1 error in every 10 lines of code (average) About 50% of these errors are corrected during code compilation Further errors are corrected in testing About 15% of the errors are still in the system when delivering it to the customer 12

13 Errors and faults in software And what if the software was (life) critical? How many errors and faults is it acceptable it has? How can we handle errors and faults? What does software quality really mean? 13

14 Definitions Software quality =??? 14

15 Definition of Software Quality Q = Deliverables Expectations 15

16 Different focuses over time (1/4) In the heroic age of computer science : A program was "good" if it was able to run - at least once-, to terminate running after a time - the length of running-time was not really important -, and to produce results similar to those expected Around : micro-efficiency A program was "good" if its use was cheap in a certain hardwaresoftware environment, and it handled the given resources of the computer in an optimal way Beginning with the 80 s: ''macro-efficiency'' A "good" computer program is portable, its use can be learned by common people, and it is highly user-friendly 16

17 Different focuses over time (2/4) Nowadays Correctness, reliability, integrity, interoperability etc. of the whole system is important We usually think of a good system if its documentation is complete and understandable if it interacts with other - computer aided or classic - dataprocessing systems in an optimal way if its maintenance can be done at low costs if it is able to cooperate with new software-hardware tools and so on Complex organizational frame, well-organized teamwork are needed 17

18 Different focuses over time (3/4) Nowadays: Structured systems' analyzing and developing methods and methodologies have been worked out and complex computer-aided tools have been developed The use of embedded intelligence Total Quality Management "to create a system of management procedures that focus on customer satisfaction" 18

19 Different focuses over time (4/4) Nowadays: Software drives airplanes and cars moves our money executes surgical interventions interferes with all aspects of our life! Bad software/ software with errors: Can kill people Can cause financial crash Can lead to worldwide disasters 19

20 Importance of Software Quality (1/5) Several historic disasters attributed to software 1988 shooting down of Airbus 320 by the USS Vincennes Cryptic and misleading output displayed by tracking software 1991 patriot missile failure Inaccurate calculation of time due to computer arithmetic errors London Ambulance Service Computer Aided Dispatch System Several deaths On June 3, 1980, the North American Aerospace Defense Command (NORAD) reported that the U.S. was under missile attack 20

21 Importance of Software Quality (2/5) Several historic disasters attributed to software First operational launch attempt of the space shuttle, whose real-time operating software consists of about 500,000 lines of code, failed Synchronization problem among its flight-control computers 9 hour breakdown of AT&T's long-distance telephone network Caused by an untested code patch 21

22 Importance of Software Quality (3/5) Ariane 5 crash June 4, 1996 Maiden flight of the European Ariane 5 launcher crashed about 40 seconds after takeoff Lost was about half a billion dollars Explosion was the result of a software error Uncaught exception due to floating-point error: conversion from a 64-bit integer to a 16-bit signed integer applied to a larger than expected number Module was re-used without proper testing from Ariane 4 Error was not supposed to happen with Ariane 4 No exception handler 22

23 Importance of Software Quality (4/5) Mars Climate Orbiter - September 23, 1999 Mars Climate Orbiter, disappeared as it began to orbit Mars Cost about $US 125-million Failure due to error in a transfer of information between a team in Colorado and a team in California One team used English units (e.g., inches, feet and pounds) while the other used metric units for a key spacecraft operation 23

24 Importance of Software Quality (5/5) Internet viruses and worms Blaster worm ($US 525 millions) Sobig.F ($US 500 millions 1billions) Exploit well known software vulnerabilities Software developers do not devote enough effort to applying lessons learned about the causes of vulnerabilities Same types of vulnerabilities continue to be seen in newer versions of products that were in earlier versions. Usability problems 24

25 Garvin s quality definitions (1984) (1/5) Transcendent definition Quality is absolute and universally recognizable, despite the fact that it cannot be defined precisely Only experience can teach to recognize quality 25

26 Garvin s quality definitions (1984) (2/5) User-based definition Quality is "fitness for use This definition starts from the assumption that the individual customers have different needs and those goods that best satisfy their needs are considered to have the highest quality 26

27 Garvin s quality definitions (1984) (3/5) Value-based definition Defines quality in relation to costs A quality product provides performance at an acceptable price or conformance at an acceptable cost 27

28 Garvin s quality definitions (1984) (4/5) The product-based definition Regards quality as a precise and measurable variable, as an inherent characteristic of goods 28

29 Garvin s quality definitions (1984) (5/5) Manufacturing-based definition Identifies quality as conformity with specifications It focuses on the supply side 29

30 The elements of software quality (1/5) Product Process Resource Definition Quality attribute Metric 30

31 The elements of software quality (2/5) Product Use cases Design artifacts Code.. Process Duration Costs Process Duration Costs 31

32 The elements of software quality (3/5) Software quality attributes are useful to more precisely define customer s quality requirements Examples Reliability Performance Security Functionality Usability Maintainability Portability Reusability Integrability Testability 32

33 The elements of software quality (4/5) Quality attributes Attributes may have different measures describing them from different perspective Example Complexity is an internal attribute of products There are many complexity measures: to trace the complexity in structure we may consider the cyclomatic complexity of McCabe CC cannot track the algorithmic complexity 33

34 The elements of software quality (5/5) Quality Attributes It easier that a a company collects measure of length and complexity of the code (internal attribute of product) than measure of quality (external attribute) of the product Internal attribute: internal characteristics of product, process, and human resources External attributes: due to external environment 34

35 Measurement Definition The process through which we attach numbers or symbols to the attributes of the real world s elements, with the scope to describe them according to some, well defined rule (Fenton) 35

36 Type of measures Direct: from a direct process of measuring Indirect: from a mathematical equation in the world of symbols Objective: the same each time they are taken (e.g. automated collected by some device) Subjective: manually collected by direct human evaluation Example: temperature of coffee - taste of a coffee 36

37 The quality framework Characteristics Metric Quality attribute Definition Objects Product Process Resource 37

38 Different approaches towards software quality Characteristics Metric Quality attribute Definition Product GQM Process Resource ISO 9126 (Boehm, McCall)... Objects ISO 9001:2000 CMM, SPICE CMMI ISO TSP, PSP P-CMM, Weinberg... Methodologies (RUP, Prince...), Life-cycle support 38

39 What does software quality definition depend on? the personality / value-perception / point of view of the evaluator the type of software production the development life cycle the type of the system / the application domain of the software business policy 39

40 Quality framework, objects and perspectives object perspectives Product Process Resource user user satisfaction based participation based interaction based engineer system based design based application based project manager project based control based allocation based 40

41 Quality as depending on the life cycle maintainability flexibility testability Product revision Product transition Product operation portability reusability interoperability correctness reliability usability efficiency integrity (McCall) 41

42 Quality as depending on the evaluator (1/4) User Software does what I wish it to do Quick response time Ease of use Should be ready as soon as possible Able to follow business changes Able to handle erroneous data Developer Well specified Technically correct Easy to maintain Easy to develop Stable Receives adequate data... 42

43 Quality as depending on the evaluator (2/4) User is interested in: Using the software, its performance, effects of using the software User not interested in the internal aspects of software or how it was developed Functionality, reliability, efficiency, usability, portability 43

44 Quality as depending on the evaluator (3/4) Developer is interested in: Intermediate product quality and final product quality Quality characteristics and metrics might differ according to the phases of the life cycle Maintainability Final product quality characteristics have to be agreed with the user 44

45 Quality as depending on the evaluator (4/4) Manager is interested in: Overall quality Not a specific quality characteristic Quality improvement with management criteria (schedule delay, cost overrun) Will need to assign weights, reflecting business requirements, to the individual characteristics 45

46 Quality according to software application domain Space shuttle or airplane Satellite Detection and alarm Management system Test bench 46 correctness reliability efficiency integrity usability maintainabiy flexibility testability portability reusability interoperabi.

47 Software quality and business policy Rapid development Honoring schedule constraints 100% conformity to the specification Complete coverage during testing... 47

48 Total Quality Management (TQM) (1/2) It represents a style of management aimed at achieving long-term success by linking quality and customer satisfaction The approach include the creation of a culture in which all members of the organization participate in the improvement of processes, products, and services 48

49 Total Quality Management (TQM) (2/2) Despite variations in its implementation, the key elements of a TQM system can be summarized as follows: Customer Focus Process Human side of quality Measurement and analysis 49

50 TQM Key Elements (1/2) Customer focus The objective is to achieve total customer satisfaction Customer focus includes studying customers wants and needs, gathering customer requirements, and measuring and managing customers satisfaction Process The objective is to reduce process variations and to achieve continuous process improvement This element includes both the business process and the product development process Through process improvement, product quality will be enhanced 50

51 TQM Key Elements (2/2) Human side of quality The objective is to create a companywide quality culture Focus areas include leadership, management commitment, total participation, employee empowerment, and other social, psychological, and human factors Measurement and analysis The objective is to drive continuous improvement in all quality parameters by the goal-oriented measurement system 51

52 Total Quality Management Total Quality Management Continuous Improvement Customer Focus Process Improvement Human Side of Quality Metrics, Models, Measurement, and Analysis 52

53 Total Quality Management Various organizational frameworks have been proposed to improve quality Specific examples include: Plan-Do-Check-Act (PDCA) Quality Improvement Paradigm/Experience Factory Organization Capability Maturity Model (SEI-CMM) Lean Enterprise Management 53

54 Plan-Do-Check-Act (PDCA) PDCA is based on a feedback cycle for optimizing a single process or product line Created by W. Edwards Deming in the 50 s It uses techniques such as feedback loops and statistical quality control to experiment with methods for improvement to build predictive models of the product 54

55 PDCA Basics Plan Plan your improvement, also setting clear numerical goals Do Do your improvement plan, recording the different steps Check Check what you have done and how closely you have reached your goals Act Act on the basis of the result to get ready to the next circle of improvement 55

56 The circular way Plan Do Act Check 56

57 Quality Improvement Paradigm/ Experience Factory Organization (1/2) It aims at building a continually improving organization based on its evolving goals and assessment of its status relative to those goals It uses such techniques as Goal-Question- Metric (GQM) 57

58 Quality Improvement Paradigm/ Experience Factory Organization (2/2) The six fundamentals steps are: 1. Characterize the project and its environment 2. Set the goals 3. Choose the appropriate processes 4. Execute the processes 5. Analyze the data 6. Package the experience for reuse 58

59 Capability Maturity Model (CMM) The SEI CMM is a staged process improvement, based on assessment of key process areas until you reach level 5, which represents a continuous process improvement 59

60 Lean Enterprise Management (1/2) It is based on the principle of concentration of production on value-added activities and the elimination or reduction of not-value-added activities The approach has been used to improve factory output The goal is to build software with the minimum necessary set of activities and then to tailor the process to the product s requirements 60

61 Lean Enterprise Management (2/2) The approach uses such concept as: Technology management Human-centered management Decentralized organization Quality management Supplier and customer integration The implementation of Lean Management in Software Engineering is represented by Agile Methodologies 61

62 Summary Software quality is a complex issue The need to be familiar with basic concepts and definitions There is no always good solution There is no solution good for everyone There are many possible approaches It is not easy to know them all It is not easy to choose the right one 62