Software Measurement Software Economics 2010
Anton Litvinenko Co-founder and CTO at Metrics tracking kit for software development Key competence: software measurement and metrics 9 years of software development at Programeter, Mobi, and MicroLink MSc in computer science at Tartu University 2/134
Outline Today Next week (14.Sept.) Measures and metrics, what kind of different metrics exist Function point analysis Third week (21.Sept.) Introduction of metrics in organizations Application of metrics Fourth week (28.Sept.) Presentation of group-work assignments 3/134
Agenda 1.What is a software metric? 4/134
Are software metrics good or bad? 5/134
6/134
What is a measure? 7/134
What is a measure? Way of associating a number with some attribute of a physical object height meters temperature degrees Celsius 8/134
What is measure? One-to-one mapping between physical objects and formal objects (e.g numbers) 9/134
Relationships and Operations Apples: Steve Jobs has 7 apples Steve Ballmer has 4 apples Jobs has more apples Jobs and Ballmer can cooperate and put their apples together to have a larger pile 10/134
Same Stuff Formally Relational System tuple consisting of Set of objects (e.g. collections of apples) Relations on these objects (e.g. more, equal) Binary operations on these objects (e.g. combine, put together) What would be the corresponding formal relational system? 11/134
We defined a complete transition from real world into formal world 12/134
Same Stuff Formally... again Let A be a relational system of physical objects (e.g. apples) B be a relational system of formal objects (e.g. numbers) m be a measure from A to B then Tuple A, B and m is a scale if Relations from A equivalent to relations from B For each operator in A there is a corresponding operator in B 13/134
Why is this important? 14/134
What Can You Say? 15/134
Why is this important? Software design: 10 modules with complexity 20 30 range 20 modules with complexity 10 30 range Which one is less complex? We don't have intuition for such cases 16/134
Intelligence Barrier 17/134
Example: Temperature Facts: Steve: today is 40ºF, yesterday was 80ºF Anton: today is 4ºC, yesterday was 27ºC Statements: Steve: Yesterday was warmer than today Anton: Yesterday was warmer than today 18/134
Example: Temperature Facts: Steve: today is 40ºF, yesterday was 80ºF Anton: today is 4ºC, yesterday was 27ºC Statements: Steve: Yesterday was 2x times warmer Is this a meaningful statement about temperature? 19/134
Statement is meaningful when it gives same result on all similar scales 20/134
Scales are similar when there is a transformation from one scale to another that retains all defined relations and operations 21/134
Nominal Scale Giving names to objects Gender Equality Any naming is similar to any other Numbers on t-shirts of football players Any unique numbering is similar to any other 22/134
View from 3000 feet :) Nominal Scales T-shirt Numbering Gender 23/134
Ordinal Scale Giving names in particular order More... than... Middle element median Rating of tennis players Similar: any other rating that retains the order 24/134
All Ordinal Scales Are Nominal T-shirt Numbering Nominal Gender Ordinal Grading Top 100 25/134
Interval Scale Assigning numbers so that interval is also meaningful Both median and arithmetic mean Similar reachable via positive linear transformation: t(x) = ax + b Temperature in Celsius scale Similar: Fahrenheit scale 26/134
Interval Scales Are Ordinal T-shirt Numbering Grading Nominal Ordinal Gender Top 100 Interval Temperature 27/134
Ratio Scale Ratio of two measures is meaningful All statistical measures Similar reachable via positive linear transformation in form of t(x) = ax Length, height,... Similar: Imperial units 28/134
Ratio Scales Are Interval T-shirt Numbering Nominal Ordinal Grading Temperature Gender Top 100 Interval Ratio Length Height 29/134
Absolute Scale Only one way of measuring objects! Similar identity transformation: t(x) = x Counting: My team has 5 members My software is 25 lines of code 30/134
Absolute Scales Are Ratio T-shirt Numbering Ordinal Grading Temperature Length Nominal Gender Top 100 Interval Ratio Height Absolute Scales Team Size 31/134
Exercise 1 Suggest a measure and a scale for Mass of physical object Human intelligence Movies Cost of cars Speed of different computers User-friendliness of a software 32/134
Exercise 2 Cost is usually a measure with ratio scale Quality is only ordinal (rarely interval) Judgment in terms of value Quality per unit of cost Should we pay 2x for 2x quality? Combining cost measure on a ratio scale with quality measure on ordinal scale, what scale do you get? 33/134
This Course: Metric = Measure 34/134
Software Metric is a measure of anything directly related to software or its production 35/134
Agenda 1.What is a software metric? 2.Examples of software metrics 1.Most famous :) 36/134
Can anybody name any software metric? 37/134
Lines Of Code (LOC) Product Size 12 14 18 38/134
Lines Of Code 39/134
Lines Of Code Summary Accurate, easy to measure How to interpret... Empty lines Comments Several statements on one line Language dependent Doesn't respect complexity and content 40/134
McCabe's Cyclomatic Complexity Thomas McCabe, 1976 Complexity of a program Number of linearly independent paths through a function Usually calculated using flow graph V(G) = e n + 2p e num of edges, n num of vertices, p num of unconnected parts of graph 41/134
McCabe's Cyclomatic Complexity 42/134
McCabe's Cyclomatic Complexity 2: System.out.println("----------"); for (Client c : clients) 4-5: System.out.println(c.getId() + " " + c.getfirstname()); e = 7 n = 6 p = 1 V(G) = 3 if (clients.size() == 0) 8: System.out.println("\tNothing"); 10: System.out.println("----------"); 43/134
Cyclomatic Complexity Summary Automated Maintainability V(G) > 10 Probability of defects rises Testability V(G) is an upper bound for the branch coverage V(G) is a lower bound for the path coverage Each control structure was evaluated both to true and false All possible paths were executed Doesn't respect other types of complexity Data structure, data flow, interfaces 44/134
Exercise 3 Calculate LOC Draw a flow graph Calculate McCabe's cyclomatic complexity Code snippet 45/134
Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 46/134
Object Oriented Metrics Shiyam Chidamber and Chris Kemerer, 1994 Metrics based on firm theoretical basis and experience of professional software developers Measure unique aspects of the object oriented approach 47/134
Inheritance Metrics Depth of inheritance tree (DIT) Depth of the class in the inheritance tree Number of children (NOC) Number of immediate descendants NOC: 3 A DIT: 1 B DIT: 2 C NOC: 2 D E F 48/134
Complexity Weighted method count (WMC) Sum of McCabe's cyclomatic complexities of all methods Response for a class (RFC) Number of public methods in a class and methods directly called by these 49/134
Complexity Example RFC = 6, WMC = 1 + 2 + 1 = 4 50/134
Coupling Coupling between object classes (CBO) Number of classes given class is coupled to Lack of cohesion in methods (LCOM) Number of method pairs that do not share instance variables vs number of methods that share at least one instance variable 51/134
Coupling Example CBO = 2, LCOM = 3 0 = 3 52/134
Coupling Example LCOM = 2 1 = 1 53/134
Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 54/134
Object Oriented Design Bad design symptoms: Class design principles Rigidity, fragility, immobility, viscosity Open closed principle, Liskov substitution principle,... Package architecture principles Stable dependencies principle, Stable abstractness principle, 55/134
OO Design Quality Metrics Robert Martin (aka Uncle Bob), 1994 Measure quality of an object oriented design 56/134
Dependencies Between Classes Can we divide dependencies into good and bad? 57/134
Dependencies Stable (good) vs unstable (bad) class Stable No need to change = independent Hard to change = many dependents = responsible Unstable Depends on many = dependent Easy to change = no dependents = irresponsible 58/134
Class Category Class category group of highly cohesive classes Closed and open to changes together Reused together Same goal Packages in Java, namespaces in C# 59/134
Dependency Metrics Afferent Coupling (Ca) number of classes outside the category depending on the classes inside the category Incoming dependencies Efferent Coupling (Ce) number of classes inside the category depending on the classes outside the category Outgoing dependencies 60/134
Example - Coupling Package Two Package One Package Three B A D C E Ca(Package One) = 1, Ce(Package One) = 2 61/134
Instability (I) Ratio of outgoing dependencies to total number of dependencies I = Ce / (Ca + Ce) Stable I = 0 Ce = 0 Unstable I = 1 Ca = 0, Ce > 0 62/134
Should all categories be stable? 63/134
Why a stable category needs to be extensible? How? 64/134
Abstractness (A) Degree to which a category is abstract Ratio of abstract classes to the total number of classes in category Completely abstract A = 1 all classes are abstract Completely concrete A = 0 no abstract classes in category 65/134
Is there a relationship between Instability and Abstractness? 66/134
Main Sequence 67/134
Distance From Main Sequence D' = A + I 1 Normalized to range from [0, 1] 68/134
Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 4.Developer and team metrics 69/134
Developer and Team Metrics Productivity Knowledge How much developers know the software they are working on Expertise How active developers are, how much work is being done What kind of tools and libraries developers use Team healthiness Communication and knowledge sharing 70/134
Productivity: Code Churn Metrics Amount of code changed in the software during the period of time Churned LOC number of added, modified and deleted lines of code Churn Count number of changes made to a file Files Churned number of changed files 71/134
Exercise 4 Suggest a way of calculating code churn metrics 72/134
73/134
74/134
75/134
76/134
77/134
78/134
01/Mar 01/Apr 01/May 79/134
Code Churn Metrics Overview of activity and productivity Increase in relative code churn metrics increase in defect density Number of defects per line of code Vulnerable files have higher code churn metrics Vulnerability instance of violation of the security policy 80/134
Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 4.Developer and team metrics 1.Productivity 2.Knowledge 81/134
Knowledge Metrics Which parts of the software developer is comfortable working with? Better planning Does developer share her knowledge with colleagues? Risk management 82/134
83/134
84/134
85/134
86/134
87/134
Unique: 2 / 5 40% Unique: 1 / 5 20% Shared: 1 / 5 20% Shared: 1 / 5 20% 88/134
Example If developer decides to leave all his unique knowledge is lost for the team Unique - 35% Unique - 10% Shared - 10% Shared - 35% 89/134
Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 4.Developer and team metrics 5.Project size metrics 90/134
How would you measure product size? 91/134
Perfect Hours One hour of ideal engineering How many perfect hours in a work day? Relative measure of effort How many ideal engineering hours required to complete the feature Team specific Applied early Manual and subjective 92/134
Points Generalization of a perfect hour Relative measure of effort required to complete the feature Not tied to time Team specific Applied early Manual and subjective 93/134
Velocity How much work can a team complete per iteration Completed Points 15 10 5 0 1 2 3 4 5 6 7 8 9 Iterations 94/134
Function Points Will be covered during next workshop 95/134
Agenda 1.What is a software metric? 2.Examples of software metrics 1.LOC and McCabe's cyclomatic complexity 2.Object oriented metrics 3.Object oriented design quality metrics 4.Developer and team metrics 5.Project size metrics 6.Quality metrics 96/134
Quality Metrics What does high quality mean? 97/134
Quality Metrics Many different models, checklists McCall's, FRUPS, ISO 9126 Functionality, reliability, usability, portability, Cannot be measured directly derived from other metrics 98/134
99/134
Quality Developer's Perspective Comprehensibility Style and cleanness of source code Architecture and design Used technologies and libraries Testability + Existing tests Easiness of automated testing Code coverage with tests 100/134
Quality PM's Perspective Predictability Effort required for development, testing,... Delivery planning Additional costs Correctness Satisfies specification Serves customer needs 101/134
Quality Customer's Perspective Value for money Supports organizational goals Return on investment Transparency Partner's effort is recognizable Delays and troubles are not hidden 102/134
Quality User's Perspective Usability Ease of use Comprehensibility Performance Responsive Critical functionality is quick Functionality Software does the right thing 103/134
Example: Defect Detection Percentage Efficiency of quality assurance procedures How many bugs were delivered to customer DDP = E / (E + D) E errors found before delivery D defects = errors found after delivery What would be an ideal situation? 104/134
Example: Time Between Escaped Defects How often new defects are found in delivered versions of the product How would you use this metric? 105/134
Course: IDY0204 Software Quality and Standards 106/134
Agenda 1.What is a software metric? 2.Examples of software metrics 3.Classification of software metrics 107/134
Classification of Software Metrics Subject of measurement 108/134
Subject: Development Process Measuring the efficiency of process application On the organizational level strategic purposes On the project level tactical purposes Examples of metrics Length of (development) iteration Number of changes in requirements Number of finished tasks Defect detection percentage 109/134
Subject: Resources Measuring usage of personnel & resources and their properties Examples of metrics Developer competency Developer fluctuation Developer productivity and know-how in the project Maturity of the code written by developer 110/134
Subject: Product Measuring product attributes Size, complexity, scalability Examples of metrics LOC, commented lines of code, function points McCabe's cyclomatic complexity Code coverage with test Code stability 111/134
Classification Overview is t a h W s a me? d e r u Metric A Process Resources Product 112/134
Classification of Software Metrics Lines of Code vs Quality 113/134
Direct Metrics Directly measurable Examples of metrics: LOC, function points McCabe's cyclomatic complexity Number of requirements 114/134
Indirect Metrics Not possible to measure directly Derived from other properties Examples of metrics Code quality, code readability Developer productivity, efficiency Reliability 115/134
Classification Overview is t a h W s a me? d e r u Metric A Is it measurable? Process Direct Resources Indirect Product 116/134
Classification of Software Metrics (In)dependency on the measurement context 117/134
Internal Attributes Measurement context/environment is not relevant Examples of metrics LOC McCabe's cyclomatic complexity Code coverage with tests 118/134
External Metrics Measured with respect to environment/context Examples of metrics Software reliability Developer productivity Source code comprehensibility Usability 119/134
Classification Overview is t a h W s a me? d e r u Metric A Is c on Is it measurable? tex t de pe nd en Process Direct Internal Resources Indirect External t? Product 120/134
Classification Example is t a h W s a me? d e r u LOC Is c on Is it measurable? tex t de pe nd en Process Direct Internal Resources Indirect External t? Product 121/134
Agenda 1.What is a software metric? 2.Examples of software metrics 3.Classification of software metrics 4.Exercises 122/134
Exercise 5 Make up 5 metrics for evaluation of this course For each metric clearly state attribute being represented Example: Number of workshops size of the course 123/134
Exercise 6 Devise formulas/algorithms for calculating values of the following metrics: Developer's contribution size Developer's effectiveness Source code maturity 124/134
Exercise 7 We have a tight schedule and we asked two testers Alice and Bob to test 5 software components Bob reported that he had time to thoroughly test only 1st component and have found totally 20 bugs Alice reported that she partially tested all components and found 5, 6, 4, 2 and 5 bugs correspondingly Whose report is more useful? What decisions can you make? 125/134
Corollary Exercise How do you think where is the highest probability of finding undetected bug? Why? In components showing LARGE number of known bugs In components showing SMALL number of known bugs 126/134
Corollary Exercise "as the number of detected errors in a piece of software increases the probability of the existing of more undetected errors also increases" Glenford Myers, 1976 Bugs tend to cluster as you find a bug you should stop and write more tests for components where you have found a bug 127/134
Software Metrics Recap What is a measure? What is a software metric? 128/134
References G. Ford, Measurement theory for software engineers Wikipedia http://www3.itu.edu.tr/~gokmen/se-lecture-2.pdf H. Nestra, Metrics, Software engineering 2005 http://www.win.tue.nl/~mchaudro/sa2007/metrics%20architecting%202005.pdf M. Gökmen, Software process and project metrics http://en.wikipedia.org/wiki/software_metrics C. Lange, Metrics in software architecting http://courses.cs.ut.ee/2010/se/uploads/main/measurement-theory.pdf http://courses.cs.ut.ee/2005/tvt/uploads/main/software_engineering_21.pdf Lines of code http://en.wikipedia.org/wiki/source_lines_of_code 129/134
References II McCabe's cyclomatic complexity http://en.wikipedia.org/wiki/cyclomatic_complexity http://www.stsc.hill.af.mil/crosstalk/1994/12/xt94d12b.asp http://www.answers.com/topic/cyclomatic-complexity S. Chidamber and C. Kemerer, A metrics suite for object oriented des. C. Martin, OO Quality design metrics http://bit.ly/2xy21f http://cs.allegheny.edu/wiki/cs290f2004/uploads/123/oodmetrc.pdf R. Pressman, Software engineering: a practitioner's approach http://bit.ly/gcwvm http://www.laynetworks.com/software%20engineering.htm http://www.parlezuml.com/metrics/oo%20design%20principles%20&%20metrics.pd http://www.parlezuml.com/metrics/metrics%20definitions.pdf More 130/134
Please Form Groups 3-4 students Register your group by sending email With names of all members To: anton.litvinenko@programeter.com By 13.Sept.2010 131/134
Home Reading David Longstreet Function Point Manual 132/134
Thank you for your time and attention! See you next week! 133/134