Software Measurement. Software Economics 2009

Size: px

Start display at page:

Download "Software Measurement. Software Economics 2009"

Samantha Lynch
5 years ago
Views:

1 Software Measurement Software Economics 2009

2 Anton Litvinenko Co-founder and CTO at Metrics for Software Projects Key competence: software measurement and metrics 8 years of software development at Mobi, MicroLink and Programeter MSc in computer science at Tartu University 2/134

3 Outline Today Next week Function point analysis, introduction of metrics in the organization Third week Measures and metrics, what kind of different metrics exist Usage of metrics Fourth week Presentation of group-work assignments 3/134

4 Agenda 1.What is a software metric? 4/134

5 Are software metrics good or bad? 5/134

6 6/134

7 What is a measure? 7/134

8 What is a measure? Way of associating a number with some attribute of a physical object height meters temperature degrees Celsius 8/134

9 What is measure? One-to-one mapping between physical and formal objects 9/134

10 Relationships and Operations Collections of XBox games: Mark has 4 games Anton has 6 games My collections is larger :) We can combine collections and get even bigger one! =] 10/134

11 Same Stuff Formally Relational System tuple consisting of Set of objects Relations on these objects Binary operations on these objects Examples: Set of all XBox games collections, larger than or same size as, combine with What would be corresponding formal relational system? 11/134

12 We defined a complete transition from real world into formal world 12/134

13 Same Stuff Formally... again Let A be a relational system of physical objects B be a relational system of formal objects (e.g. numbers) m be a measure from A to B then Tuple A, B and m is a scale if Relations from A equivalent to relations from B For each operator in A there is a corresponding operator in B 13/134

14 Why is this important? Software design: 10 modules with complexity range 20 modules with complexity range Which one is less complex? We don't have intuition for such cases 14/134

15 Intelligence Barrier 15/134

16 What Can You Say? 16/134

17 Example: Temperature Facts: Dasha: today is 40ºF, yesterday was 80ºF Anton: today is 4ºC, yesterday was 27ºC Statements: Dasha: Yesterday was warmer than today Anton: Yesterday was warmer than today 17/134

18 Example: Temperature Facts: Dasha: today is 40ºF, yesterday was 80ºF Anton: today is 4ºC, yesterday was 27ºC Statements: Dasha: Yesterday was 2x times warmer Is this a meaningful statement about temperature? 18/134

19 Statement is meaningful when it gives same result on all similar scales 19/134

20 Scales are similar when there is a transformation from one scale to another that retains all defined relations and operations 20/134

21 Nominal Scale Giving names to objects Gender Equality Any naming is similar to any other Numbers on t-shirts of football players Any unique numbering is similar to any other 21/134

22 View from 3000 feet :) Nominal Scales 22/134

23 Ordinal Scale Giving names in particular order More... than... Middle element median Rating of tennis players Similar: any other rating that retains the order 23/134

24 All Ordinal Scales Are Nominal Nominal Ordinal Scales 24/134

25 Interval Scale Assigning numbers so that interval is also meaningful Both median and arithmetic mean Similar reachable via positive linear transformation: t(x) = ax + b Temperature in Celsius scale Similar: Fahrenheit scale 25/134

26 Interval Scales Are Ordinal Nominal Ordinal Interval Scales 26/134

27 Ratio Scale Ratio of two measures is meaningful All statistical measures Similar reachable via positive linear transformation in form of t(x) = ax Length, height,... Similar: Imperial units 27/134

28 Ratio Scales Are Interval Nominal Ordinal Interval Ratio Scales 28/134

29 Absolute Scale Only one way of measuring objects! Similar identity transformation: t(x) = x Counting: My team has 5 members My software is 25 lines of code 29/134

30 Absolute Scales Are Ratio Nominal Ordinal Interval Ratio Absolute Scales 30/134

31 Exercise 1 Suggest a measure and a scale for Mass of physical object Human intelligence Movies Cost of cars Speed of different computers User-friendliness of a software 31/134

32 Exercise 2 Cost is usually a measure with ratio scale Quality is only ordinal (rarely interval) Judgment in terms of value Quality per unit of cost Should we pay 2x for 2x quality? Combining cost measure on a ratio scale with quality measure on ordinal scale, what scale do you get? 32/134

33 This Course: Metric = Measure 33/134

34 Software Metric is a measure of anything directly related to software or its production 34/134

35 Agenda 1.What is a software metric? 2.Examples of software metrics 1.Most famous :) 35/134

36 Can anybody name any software metric? 36/134

37 Lines Of Code (LOC) Product Size /134

38 Lines Of Code 38/134

39 Lines Of Code Summary Accurate, easy to measure How to interpret... Empty lines Comments Several statements on one line Language dependent Doesn't respect complexity and content 39/134

40 McCabe's Cyclomatic Complexity Thomas McCabe, 1976 Complexity of a program Number of linearly independent paths through a function Usually calculated using flow graph V(G) = e n + 2p e num of edges, n num of vertices, p num of unconnected parts of graph 40/134

41 McCabe's Cyclomatic Complexity 41/134

42 McCabe's Cyclomatic Complexity 2: System.out.println(" "); for (Client c : clients) 4-5: System.out.println(c.getId() + " " + c.getfirstname()); e = 7 n = 6 p = 1 V(G) = 3 if (clients.size() == 0) 8: System.out.println("\tNothing"); 10: System.out.println(" "); 42/134

43 Cyclomatic Complexity Summary Automated Maintainability V(G) > 10 Probability of defects rises Testability V(G) is an upper bound for the branch coverage V(G) is a lower bound for the path coverage Each control structure was evaluated both to true and false All possible paths were executed Doesn't respect other types of complexity Data structure, data flow, interfaces 43/134

44 Exercise 3 Calculate LOC Draw a flow graph Calculate McCabe's cyclomatic complexity Code snippet 44/134

45 Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 45/134

46 Object Oriented Metrics Shiyam Chidamber and Chris Kemerer, 1994 Metrics based on firm theoretical basis and experience of professional software developers Measure unique aspects of the object oriented approach 46/134

47 Inheritance Metrics Depth of inheritance tree (DIT) Depth of the class in the inheritance tree Number of children (NOC) Number of immediate descendants NOC: 3 A DIT: 1 B DIT: 2 C NOC: 2 D E F 47/134

48 Complexity Weighted method count (WMC) Sum of McCabe's cyclomatic complexities of all methods Response for a class (RFC) Number of different methods that can be executed when a method on a object is invoked 48/134

49 Complexity Example RFC = 6, WMC = = 4 49/134

50 Coupling Coupling between object classes (CBO) Number of classes coupled to a given class Lack of cohesion in methods (LCOM) Number of methods pairs that are not related to each other through the sharing of some instance variables 50/134

51 Complexity Example CBO = 2, LCOM = 3 0 = 3 51/134

52 Complexity Example LCOM = 2 1 = 1 52/134

53 Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 53/134

54 Object Oriented Design Bad design symptoms: Class design principles Rigidity, fragility, immobility, viscosity Open closed principle, Liskov substitution principle,... Package architecture principles Stable dependencies principle, Stable abstractness principle, 54/134

55 OO Design Quality Metrics Robert Martin, 1994 Measure quality of an object oriented design 55/134

56 Dependencies Between Classes Can we divide dependencies into good and bad? 56/134

57 Dependencies Stable (good) vs unstable (bad) class Stable No need to change = independent Hard to change = many dependents = responsible Unstable Depends on many = dependent Easy to change = no dependents = irresponsible 57/134

58 Class Category Class category group of highly cohesive classes Closed and open to changes together Reused together Same goal Packages in Java, namespaces in C# 58/134

59 Dependency Metrics Afferent Coupling (Ca) number of classes outside the category depending on the classes inside the category Incoming dependencies Efferent Coupling (Ce) number of classes inside the category depending on the classes outside the category Outgoing dependencies 59/134

60 Example - Coupling Package Two Package One Package Three B A D C E Ca(Package One) = 1, Ce(Package One) = 2 60/134

61 Instability (I) Ratio of outgoing dependencies to total number of dependencies I = Ce / (Ca + Ce) Stable I = 0 Ce = 0 Unstable I = 1 Ca = 0, Ce > 0 61/134

62 Should all categories be stable? 62/134

63 How can a stable category be extensible? 63/134

64 Abstractness (A) Degree to which a category is abstract Ratio of abstract classes to the total number of classes in category Completely abstract A = 1 all classes are abstract Completely concrete A = 0 no abstract classes in category 64/134

65 Is there a relationship between Instability and Abstractness? 65/134

66 Main Sequence 66/134

67 Distance From Main Sequence D' = A + I 1 Normalized to range from [0, 1] 67/134

68 Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 4.Developer and team metrics 68/134

69 Developer and Team Metrics Productivity Knowledge How much developers know the software they are working on Expertise How active developers are, how much work is being done What kind of tools and libraries developers use Team healthiness Communication and knowledge sharing 69/134

70 Productivity: Code Churn Metrics Amount of code changed in the software during the period of time Churned LOC number of added, modified and deleted lines of code Churn Count number of changes made to a file Files Churned number of changed files 70/134

71 Exercise 4 Suggest a way of calculating code churn metrics 71/134

72 72/134

73 73/134

74 74/134

75 75/134

76 76/134

77 77/134

78 01/Mar 01/Apr 01/May 78/134

79 Code Churn Metrics Overview of activity and productivity Increase in relative code churn metrics increase in defect density Number of defects per line of code Vulnerable files have higher code churn metrics Vulnerability instance of violation of the security policy 79/134

80 Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 4.Developer and team metrics 1.Productivity 2.Knowledge 80/134

81 Knowledge Metrics Which parts of the software developer is comfortable working with? Better planning Does developer share his knowledge with colleagues? Risk management 81/134

82 82/134

83 83/134

84 84/134

85 85/134

86 86/134

87 Unique: 2 / 5 40% Unique: 1 / 5 20% Shared: 1 / 5 20% Shared: 1 / 5 20% 87/134

88 Example If developer decides to leave all his unique knowledge is lost for the team Unique - 35% Unique - 10% Shared - 10% Shared - 35% 88/134

89 Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 4.Developer and team metrics 5.Project size metrics 89/134

90 How would you measure product size? 90/134

91 Function Points (FP) Size estimation based on functionality Step 1: For each use-case identify: User inputs (information input): User inquiries (no derived data, data retrieval): add new purchase, search, wizard view customer data User outputs (includes derived data, algorithms): monthly purchase breakdown, notification message Files: db table with customer data External Interfaces: integration with supplier warehouse 91/134

92 Function Points Step 2: Assign complexity for each component Step 3: Calculate number of unadjusted points: Count Simple Average Complex User inputs x = User outputs x = User inquiries x = Files x External interfaces x = = Number of unadjusted points (unadjusted-total) = Sum() 92/134

93 Function Points Fi: Step 4: Calculate complexity adjustment values: Sum(Fi) No Influence 0 Incidental 1 Moderate 2 Average 3 Significant 4 Essential 5 Complex processing _ Distributed data processing _ Performance _ Heavily used configuration _ Online data entry _ Distributed transactions _ Installation ease _... 93/134

94 Function Points Final: Calculate total functions points: FP = unadjusted-total x ( x Sum(Fi)) 94/134

95 Function Points Summary Independent of language Possible to apply early in the project No source code required Manual Complexity estimation is subjective No physical meaning 95/134

96 Perfect Hours One hour of ideal engineering How many perfect hours in a work day? Relative measure of effort How many ideal engineering hours required to complete the feature Team specific Applied early Manual and subjective 96/134

97 Points Generalization of a perfect hour Relative measure of effort required to complete the feature Not tied to time Team specific Applied early Manual and subjective 97/134

98 Velocity How much work can a team complete per iteration Completed Points Iterations 98/134

99 Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 4.Developer and team metrics 5.Project size metrics 6.Quality metrics 99/134

100 Quality Metrics What does high quality mean? 100/134

101 Quality Metrics Many different models, checklists McCall's, FRUPS, ISO 9126 Functionality, reliability, usability, portability, Cannot be measured directly derived from other metrics Cannot be counted not an absolute scale :) 101/134

102 102/134

103 Quality Developer's Perspective Comprehensibility Style and cleanness of source code Architecture and design Used technologies and libraries Testability + Existing tests Easiness of automated testing Code coverage with tests 103/134

104 Quality PM's Perspective Predictability Effort required for development, testing,... Delivery planning Additional costs Correctness Satisfies specification Serves customer needs 104/134

105 Quality Customer's Perspective Value for money Supports organizational goals Return on investment Transparency Partner's effort is recognizable Delays and troubles are not hidden 105/134

106 Quality User's Perspective Usability Ease of use Comprehensibility Performance Responsive Critical functionality is quick Functionality Software does the right thing 106/134

107 Example: Defect Removal Efficiency Efficiency of quality assurance procedures How many bugs are delivered to customer DRE = E / (E + D) E errors found before delivery D defects = errors found after delivery What would be an ideal situation? 107/134

108 Course: IDY0204 Software Quality and Standards 108/134

109 Agenda 1.What is a software metric? 2.Examples of software metrics 3.Classification of software metrics 109/134

110 Classification of Software Metrics Subject of measurement 110/134

111 Subject: Development Process Measuring the efficiency of process application On the organizational level strategic purposes On the project level tactical purposes Examples of metrics Length of (development) iteration Number of changes in requirements Number of finished tasks 111/134

112 Subject: Resources Measuring usage of resources and their properties Examples of metrics Developer competency Developer fluctuation Developer productivity and know-how in the project Maturity of the code written by developer 112/134

113 Subject: Product Measuring product attributes Size, complexity, scalability Examples of metrics LOC, commented lines of code, function points McCabe's cyclomatic complexity Code coverage with test Code stability 113/134

114 Classification Overview i t a h W u s a e sm? d re Metric A Process Resources Product 114/134

115 Classification of Software Metrics Lines of Code vs Quality 115/134

116 Direct Metrics Directly measurable Examples of metrics: LOC, function points, McCabe's cyclomatic complexity Number of requirements 116/134

117 Indirect Metrics Not possible to measure directly Derived from other properties Examples of metrics Code quality, code readability Developer productivity, efficiency Reliability 117/134

118 Classification Overview i t a h W u s a e sm? d re Metric A Is it measurable? Process Direct Resources Indirect Product 118/134

119 Classification of Software Metrics (In)dependency on the measurement context 119/134

120 Internal Attributes Measurement context/environment is not relevant Examples of metrics LOC McCabe's cyclomatic complexity Code coverage with tests 120/134

121 External Metrics Measured with respect to environment/context Examples of metrics Software reliability Developer productivity Source code comprehensibility Usability 121/134

122 Classification Overview i t a h W u s a e sm? d re Metric A Is c on Is it measurable? te x td ep e nd en Process Direct Internal Resources Indirect External t? Product 122/134

123 Classification Example i t a h W u s a e sm? d re LOC Is c on Is it measurable? te x td ep e nd en Process Direct Internal Resources Indirect External t? Product 123/134

124 Agenda 1.What is a software metric? 2.Examples of software metrics 3.Classification of software metrics 4.Exercises 124/134

125 Exercise 5 Make up 5 metrics for evaluation of this course For each metric clearly state attribute being represented Example: Number of workshops size of the course 125/134

126 Exercise 6 Devise formulas/algorithms for calculating values of the following metrics: Developer's contribution size Developer's effectiveness Source code maturity 126/134

127 Exercise 7 We have a tight schedule and we asked two testers Alice and Bob to test 5 software components Bob reported that he had time to thoroughly test only 1st component and have found totally 20 bugs Alice reported that she shallowly tested all components and found 5, 6, 4, 2 and 5 bugs correspondingly Whose report is more useful? What decisions can you make? 127/134

128 Corollary Exercise How do you think where is the highest probability of finding undetected bug? Why? In components showing LARGE number of known bugs In components showing SMALL number of known bugs 128/134

129 Corollary Exercise "as the number of detected errors in a piece of software increases the probability of the existing of more undetected errors also increases" Glenford Myers, 1976 Bugs tend to cluster as you find a bug you should stop and write more tests for components where you have found a bug 129/134

130 Software Metrics Recap What is measure? What is software metric? 130/134

131 References G. Ford, Measurement theory for software engineers Wikipedia H. Nestra, Metrics, Software engineering M. Gökmen, Software process and project metrics C. Lange, Metrics in software architecting Lines of code 131/134

132 References II McCabe's cyclomatic complexity S. Chidamber and C. Kemerer, A metrics suite for object oriented des. C. Martin, OO Quality design metrics R. Pressman, Software engineering: a practitioner's approach More 132/134

133 Home Reading David Longstreet Function Point Manual 133/134

134 Thank you for your time and attention! See you next week! 134/134

Software Measurement. Software Economics 2010

Software Measurement. Software Economics 2010 Software Measurement Software Economics 2010 Anton Litvinenko Co-founder and CTO at Metrics tracking kit for software development Key competence: software measurement and metrics 9 years of software development