Quality. And Software Product Management. Autumn 2017 CSM14104 Software Product Management 1

Size: px

Start display at page:

Download "Quality. And Software Product Management. Autumn 2017 CSM14104 Software Product Management 1"

Vincent Thomas
6 years ago
Views:

1 Quality And Software Product Management Autumn 2017 CSM14104 Software Product Management 1

2 Lecture Outline What do we mean by Quality? What are the factors that influence quality? How is quality measured? Autumn 2017 CSM14104 Software Product Management 2

3 What is Quality? Autumn 2017 CSM14104 Software Product Management 3

Quality The proof of the pudding is in the eating "Christmas pudding" by Musical Linguist at the English language Wikipedia. Licensed under CC BY-SA 3.

4 Quality The proof of the pudding is in the eating "Christmas pudding" by Musical Linguist at the English language Wikipedia. Licensed under CC BY-SA 3.0 via Wikimedia Commons Autumn 2017 CSM14104 Software Product Management 4

5 Definitions Definitions from two pioneers of industrial Quality Control and Management P. Crosby [1] : Quality means conformance to requirements J. Juran [2] : 1) Quality consists of those product features which meet the needs of customers and thereby provide product satisfaction 2) Quality consists of freedom from deficiencies [1] Crosby, P. B.: Quality is Free. McGraw-Hill, New York, [2] Juran, J. M.: Juran s Quality Control Handbook, 4th ed. McGraw-Hill, New York, Autumn 2017 CSM14104 Software Product Management 5

6 Definitions Crosby s definition expects that a product has a defined set of requirements that is unambiguous, extensive, objective and that covers the needs of all stakeholders The quality of a product can be determined by comparing its true (measured) attributes to what is the expected level stated in the requirements But it does not take into account the possibility of errors in the requirements A missing or incomplete feature (from user s perspective) does not lower the quality of the prodcut, if it still conforms to the requirements - but the user is unhappy! A software product almost never has a complete and unambiguous definition of requirements Autumn 2017 CSM14104 Software Product Management 6

7 Definitions Juran s definition places the needs and satisfaction of the customer and the user in the center, as in Fitness for use The real needs vs. (documented) requirements The features and characteristics of the product that respond to those needs The second important aspect is freedom from deficiencies 1 +2 = the product has the right features and they work as the customer expects Puts the emphasis on the customers and the users instead of defined requirements Requires some other way to find out the true needs than a binding set of requirements defined at the beginning of a project Changes are to be expected and must be prepared to respond to them along the way Autumn 2017 CSM14104 Software Product Management 7

8 Stakeholders and quality A product or service has many stakeholders to whom quality means different things The end-user (consumer, employer) Operator, in charge of deploying to production and running it Developer, maintainer, contractor Tech support Customer service Sponsor, financer, investor Financer of maintenance and further development Public authorities, regulator and so on!! Autumn 2017 CSM14104 Software Product Management 8

9 Is quality subjective? Different stakeholder groups have different needs and expectation but there can also be variation within a group Professional vs. casual users Enthusiasts and early adopters vs. conservatives, etc. Weinberg, G. [3] : Quality is value to some person Value = what someone is ready to pay to get her need fulfilled A key question is whose needs are taken into account - and whose not A software product manager needs to Understand what quality means to the different stakeholders Balance conflicting views and make trade-offs [3] Weinberg, G. M.: How Software is built. Leanpub, Autumn 2017 CSM14104 Software Product Management 9

10 What s a bug? (a.k.a defect, problem) [Good quality means] The absence of defects that would make software either stop completely or produce unacceptable results. Defects can be traced to requirements, to design, to code, to documentation, or to bad fixes of previous defects. [5] A Bug is an attribute of a software product That decreases its value to a favored stakeholder Or increases its value to a disfavored stakeholder Without a sufficiently large counterveiling benefit. [6] [5] Capers Jones, Applied Software Measurement. McGraw-Hill, [6] C. Kaner & J. Bach, Black Box Software Testing Foundations, Autumn 2017 CSM14104 Software Product Management 10

11 A quality model Standards view Autumn 2017 CSM14104 Software Product Management 11

12 ISO/IEC:n definition of Software Quality ISO/IEC* definition Degree to which a software product satisfies stated and implied needs when used under specified conditions Thus providing value to its stakeholders A very general definition but one that emphasis the needs of the stakeholders Individual or organization having a right, share, claim or interest in a system or in its possession of characteristics that meet their needs and expectations *) ISO/IEC 25010:2011 Systems and software engineering--systems and software Quality Requirements and Evaluation (SQuaRE) --System and software quality models, A copy available form course materials via a link to cs intranet (requires university user id and password) Autumn 2017 CSM14104 Software Product Management 12

13 ISO/IEC series of standards, SQuaRE Software Quality Requirements and Evaluation Development of Standards and Technical Reports for Software Product and System Quality Requirements, Measurement and Evaluation Includes a quality model, defines how quality is measured in principle and provides some examples of possible measures Available as paper copies from the Campus Library Some draft versions available electronically from course web page (only to be used for studying purposes) Autumn 2017 CSM14104 Software Product Management 13

14 ISO/IEC series of standards, SQuaRE Key standards Quality model division ISO/IEC 25010: System and software quality models (ISO/IEC quality model for IT-services is under development) ISO/IEC 25012: Data quality model Quality measurement division ISO/IEC 25020: Measurement reference model and guide ISO/IEC 25021: Quality measure elements ISO/IEC 25022: Quality in use measures ISO/IEC 25023: Measurement of system and software product quality ISO/IEC 25024: Measurement of data quality Autumn 2017 CSM14104 Software Product Management 14

15 ISO/IEC Quality Model Defined set of characteristics, and of relationships between them, which provides a framework for specifying quality requirements and evaluating quality Common vocabulary and a frame of reference for thinking about software system quality Gives a superset of quality characteristics that can be important for software systems and products Covers a wide range of systems not all characteristics are relevant to all systems Every product/system does not have to invent their own model but adapt the generic model Autumn 2017 CSM14104 Software Product Management 15

16 ISO/IEC Quality Model Is divided into two sub-models 1) Quality in use 2) Software product quality ISO/IEC Data Quality Model is a closely related but a separate standard Autumn 2017 CSM14104 Software Product Management 16

17 ISO/IEC Terminology software quality characteristic = category of software quality attributes that bears on software quality may be refined into multiple levels of subcharacteristics and finally into software quality attributes. quality attribute 1 = measurable component of quality stakeholder = individual or organisation having a right, share, claim or interest in a system or in its possession of characteristics that meet their needs and expectations (e.g. software developers, system integrators, acquirers, owners, maintainers, contractors and end users). user = individual or group that benefits from a system during its utilization. direct user = person who interacts with the product 1 attribute: inherent property or characteristic of an entity that can be distinguished quantitatively or qualitatively by human or automated means [ISO/IEC 15939:2002] Autumn 2017 CSM14104 Software Product Management 17

18 Quality in use Is composed of five characteristics (some of which are further subdivided into subcharacteristics) that relate to the outcome of interaction when a product is used in a particular context of use Applicable to the complete human-computer system, including both computer systems in use and software products in use The subcharacteristics are representative of typical concerns for each main characteristic not necessarily an exhaustive list Autumn 2017 CSM14104 Software Product Management 18

19 Quality in use characteristics Effectiveness Efficiency Satisfaction Usefulness Trust Pleasure Comfort Freedom from risk Economic risk mitigation Health and safety risk mitigation Environmental risk mitigation Context coverage Context completeness Flexibility Effectiveness Accuracy and completeness with which users achieve specified goals Efficiency Resources expended in relation to the accuracy and completeness with which users achieve goals Satisfaction Degree to which user needs are satisfied when a product or system is used in a specified context of use Freedom from risk Degree to which a product or system mitigates the potential risk to economic status, human life, health, or the environment Context coverage Degree to which a product or system can be used with effectiveness, efficiency, freedom from risk and satisfaction in both specified contexts of use and in contexts beyond those initially explicitly identified Autumn 2017 CSM14104 Software Product Management 19

20 Software product quality Is composed of eight characteristics (which are further subdivided into subcharacteristics) that relate to static properties of software and dynamic properties of the computer system Applicable to both computer systems and software products Autumn 2017 CSM14104 Software Product Management 20

21 Software product quality characteristics Functional suitability Functional completeness Functional correctness Functional appropriateness Performance efficiency Time behaviour Resource utilization Capacity Compatibility Co-existence Interoperability Usability Appropriateness recognizability Learnability Operability User error protection User interface aesthetics Accessibility Reliability Maturity Availability Fault tolerance Recoverability Security Confidentiality Integrity Non-repudiation Accountability Authenticity Maintainability Modularity Reusability Analysability Modifiability Testability Portability Adaptability Installability Replaceability Autumn 2017 CSM14104 Software Product Management 21

22 ISO/IEC Data Quality Model Data quality is a key component of the quality and usefulness of information derived from that data, and most business processes depend on the quality of data A common prerequisite to all information technology projects is the quality of the data which are exchanged, processed and used between the computer systems and users and among computer systems themselves The data quality model categorizes quality attributes into fifteen characteristics considered by two points of view: inherent and system dependent Autumn 2017 CSM14104 Software Product Management 22

23 Inherent data quality From the inherent point of view, data quality refers to data itself, in particular to: data domain values and possible restrictions (e.g. business rules governing the quality required for the characteristic in a given application); relationships of data values (e.g. consistency); metadata Autumn 2017 CSM14104 Software Product Management 23

24 System dependent data quality System dependent data quality refers to the degree to which data quality is reached and preserved within a computer system when data is used under specified conditions. From this point of view data quality depends on the technological domain in which data are used; it is achieved by the capabilities of computer systems' components such as: hardware devices (e.g. to make data available or to obtain the required precision), computer system software (e.g. backup software to achieve recoverability), and other software (e.g. migration tools to achieve portability). Autumn 2017 CSM14104 Software Product Management 24

25 Characteristic Inherent System dependent Accuracy X Completeness X Consistency X Credibility X Currentness X Accessibility X X Compliance X X Confidentiality X X Efficiency X X Precision X X Traceability X X Understandability X X Availability X Portability X Recoverability X Autumn 2017 CSM14104 Software Product Management 25

26 Using quality models The product quality and quality in use models are useful for specifying requirements, establishing measures, and performing quality evaluations The defined quality characteristics can be used as a checklist for ensuring a comprehensive treatment of quality requirements, thus providing a basis for estimating the consequent effort and activities that will be needed during systems development The characteristics in the quality in use model and product quality model are intended to be used as a set when specifying or evaluating computer system or software product quality Autumn 2017 CSM14104 Software Product Management 26

27 Using quality models It is not practically possible to specify or measure all subcharacteristics for all parts of a large computer system or software product Similarly it is not usually practical to specify or measure quality in use for all possible user-task scenarios The relative importance of quality characteristics will depend on the high-level goals and objectives for the project Therefore the model should be tailored before use as part of the decomposition of requirements to identify those characteristics and subcharacteristics that are most important, and resources allocated between the different types of measure depending on the stakeholder goals and objectives for the product Autumn 2017 CSM14104 Software Product Management 27

28 Influencing factors Autumn 2017 CSM14104 Software Product Management 28

Four views to quality "Gran Canyon USA" by Ot Pi - Own work.

0 via Wikimedia User experience "Blaues Fahrrad mit Achter" by 4028mdk09 - Own

product Internal attributes Developers and manufacturers Autumn 2017 "Disassembled

0 via Wikimedia Commons CSM14104 Software Product Management "Bike mechanic at a

29 Four views to quality "Gran Canyon USA" by Ot Pi - Own work. Licensed under CC BY-SA 3.0 via Wikimedia User experience "Blaues Fahrrad mit Achter" by 4028mdk09 - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons Externally visible and measuable attributes of the product Internal attributes Developers and manufacturers Autumn 2017 "Disassembled Campagnolo Centaur cassette - side view" by Ximeg - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons CSM14104 Software Product Management "Bike mechanic at a local bike shop" by Andrew Dressel - Taken by Andrew Dressel at en.wikipedia. Licensed under CC BY-SA 3.0 via Wikimedia Commons - 29

30 Quality influences The needs of users and the context of use are the starting point Example: getting timetables for public transport with a mobile device User s needs and expectations (quality in use): Effectiveness, Satisfaction The information delivered must be accurate and up to date Efficiency, Satisfaction It must be intuitive and fast to find the timetable for a certain line and a certain stop User s needs and expectations (data quality): Accuracy, Currentness, Availability, Efficiency Autumn 2017 CSM14104 Software Product Management 30

31 Quality influences The external product quality characteristics that influence the user ecxperience (looking at the system as a black-box) Usability for example How many interaction steps does it take to get to the information (taps, selections, button presses)? How quick and clear is the feedback from the UI to user s actions? Performance efficiency (tehokkuus) for example How long does the whole use case take? How long does it take for a new view to render? How fast does the application start? How fast data connection does the application need? Autumn 2017 CSM14104 Software Product Management 31

32 Quality influences Internal product quality enables (or inhibits!) achieving the desired external product quality (looking at the system as a glass-box) Performance efficiency (response time, through put) is largely determined by the design and architecture of the application as well as from the technologies used and the correctness of their use Usability is influenced directly by the UI design and indirectly by the ablity to monitor the usage of various features for logging and statistical inference Autumn 2017 CSM14104 Software Product Management 32

33 Quality influences Autumn 2017 CSM14104 Software Product Management 33

34 Measuring quality Autumn 2017 CSM14104 Software Product Management 34

35 Quality is free P. Crosby It is always cheaper to do the job right the first time Quality is free, but no one is ever going to know it if there isn't some sort of agreed-on system of measurement You can t control what you can t measure Tom DeMarco, Autumn 2017 CSM14104 Software Product Management 35

36 Measurements The quality goals of an organization can be defined by setting target levels for quality indicators A measurement program is set up to define the indicators and needed measurements E.g. measuring customer satisfaction and loyality Opinion polls/surveys Net Promoter Score Number of failure reports/complaints and support requests as a function of time and relative to the number of (active) users We want to assess, estimate and collect statistics about the products, processes and resources for management and development purposes Autumn 2017 CSM14104 Software Product Management 36

37 Measurements Establishing a measurement program can be used for communicating the quality targets and engaging the organization even if the program is not applied in full or only for a limited It is more important to make people aware of the goals and get their commitment rather than to implement the program in full scale N.B.! Using measurements as the basis of promotions/bonuses or in assessing personal performance can have unwanted, counterproductive side-effects There can be many ways to make the metrics look right when in fact the situation is quite different (gaming the metrics) Developers do not like personal measurements and vote with their feet You get what you measure are you sure that really is what you want? Autumn 2017 CSM14104 Software Product Management 37

38 Direct, indirect and surrogate measures Measures can be direct or indirect Direct measures indicate immediately some aspect of the state of the object of measurement (valid by nature). E.g. the temperature of a physical object. f(a ribute) value Indirect measures are obtained as a function of direct measures. E.g. the average monthly temperature of some place. h( f(o 1 ),g(o 2 ), ) value Care must be exercised with software related metrics to declare some metric as Direct! A Surrogate metric is used when we do not know how to directly measure an attribute that we are interested in Using the size of a software module as an indication of how error prone it is Using the number of open bug reports to gauge the product s readiness for release (can be very questionable!) Autumn 2017 CSM14104 Software Product Management 38

39 Onks kymppi paljon vai vähän? Autumn 2017 CSM14104 Software Product Management 39

40 Choosing measures We can t measure everything, and we do not even know how to measure many interesting things How do we then choose what to measure? ISO series gives some ideas The Goal - Question - Metric method is one of the best known and widely used approaches (GQM, Basili & Rombach, 1988). Autumn 2017 CSM14104 Software Product Management 40

41 Goal - Question - Metric According to GQM [7], in order to develop a sensible and effective measurement program, the organization needs to 1. Define the company- and project-level targets or goals 2. Determine the product and process level indicators/information that bear on those goals 3. Plan a framework for collecting and interpreting the data in practice In other words: 1. What do we want to achieve? What are the business and organization level goals that we want to follow? 2. What information do we need about the products and processes to see whether we are reaching the goals or? 3. How do we collect the data in practice and how do we interpret the data? GQM defines a process how to do this [7] Autumn 2017 CSM14104 Software Product Management 41

42 Example (from [7], not really about quality!) Autumn 2017 CSM14104 Software Product Management 42

Software metrics. Jaak Tepandi

Software metrics. Jaak Tepandi Software metrics, Jekaterina Tšukrejeva, Stanislav Vassiljev, Pille Haug Tallinn University of Technology Department of Software Science Moodle: Software Quality (Tarkvara kvaliteet) Alternate download: