2IS55 Software Evolution. Software metrics (3) Alexander Serebrenik

Transcription

1 2IS55 Software Evolution Software metrics (3) Alexander Serebrenik

2 Administration Assignment 5: Deadline: May students / SET / W&I PAGE 1

3 Sources / SET / W&I PAGE 2

4 Recap: Software metrics So far Metrics scales Size: LOCs, #files, functionality (function points, API) Complexity: Halstead, McCabe, Henry-Kafura OO: Chidamber-Kemerer (WMC, DIT, etc.) LCOM and variants Aggregation Today Package metrics Churn metrics / SET / W&I PAGE 3

5 Package metrics Size: number of classes/interfaces number of classes in the subpackages Dependencies visualization à la fan-in and fan-out Marchesi s UML metrics Martin s D n : abstractness-instability balance or the normalized distance from the main sequence PASTA Do you still remember aggregations of class metrics? / SET / W&I PAGE 4

6 How can we visualize dependencies between packages? In the same way as with classes Can we focus on one package? / SET / W&I PAGE 5

7 How can we visualize dependencies between packages? Package Surface Blueprints [Ducasse et al. 2007] Can be extended to encode system boundaries by means of color Can be extended to incorporate inheritance / SET / W&I PAGE 6

8 Fan-out [Marchesi 1998] [Martin 2000] PK 1 or R: C e : [Martin 1994] [JDepend] [Martin 2000] PAGE 7

9 Fan-in Fan-in similarly to the Fan-out Afferent coupling (Martin) PK 2 (Marchesi) [Hilton 2009] Dark: TDD, light: no-tdd Test-driven development positively affects C a The lower C a - the better. Exception: JUnit vs. Jerico But Jericho is extremely small (2 packages) / SET / W&I PAGE 8

10 More fan-in and fan-out SAP (Herzig) Fan-in similarly to the Fan-out Afferent coupling (Martin) PK 2 (Marchesi) Validation Afferent Efferent [Martin 2000] Correlation post-release defects Class-in Class-out Fan-in Fan-out / SET / W&I PAGE 9 Marchesi Railway simulator Warehouse management Manmonths #Pack avg(pk 1 ) CASE tool

11 Evolution of afferent and efferent coupling Sato, Goldman, Kon 2007 Almost all systems show an increasing trend (Lehman s growing complexity) Project 7 (workflow system) is almost stable but very high! Outsourced development No automated tests Severe maintainability problems / SET / W&I PAGE 10

12 Package metrics: Stability Stability is related to the amount of work required to make a change [Martin, 2000]. Stable packages Do not depend upon classes outside Many dependents Should be extensible via inheritance (abstract) Instable packages Depend upon many classes outside No dependents Should not be extensible via inheritance (concrete) PAGE 11

13 What does balance mean? A good real-life package must be instable enough in order to be easily modified It must be generic enough to be adaptable to evolving requirements, either without or with only minimal modifications Hence: contradictory criteria PAGE 12

14 D n Distance from the main sequence Abstractness = #AbstrClasses/#Classes D n = Abstractness + 1 zone of uselessness Instability 1 [R.Martin 1994] 0 zone of pain main sequence 1 PAGE 13 Instability = C e /(C e +C a )

15 Normalized distance from the main sequence Dark: TDD, light: no-tdd Test-driven development positively affects D n The lower D n - the better. The same exception (Jericho vs. JUnit) [Hilton 2009] / SET / W&I PAGE 14

16 Distribution and evolution Exponential distribution For all benchmark systems studied, here Vuze Peak: many feature requests (average Dn) 28 JBoss PAGE 15

17 PASTA [Hautus 2002] PASTA Package structure analysis tool Metrics Similarly fan-in/fan-out : based on dependencies between packages Go beyond calculating numbers of dependencies Focus on dependencies between the subpackages Some dependencies are worse than others What are the bad dependencies? Cyclic dependencies, layering violations / SET / W&I PAGE 16

18 PASTA [Hautus] Idea: remove bad (cyclecausing) dependencies Weight number of references from one subpackage to another one. Dependencies to be removed are such that The result is acyclic The total weight of the dependencies removed is minimal Minimal effort required to resolve all the cycles Upwards dependencies should be removed / SET / W&I PAGE 17

19 From dependencies to metrics PASTA(P) = Total weight of the dependencies to be removed / total weight of the dependencies No empirical validation of the metrics No studies of the metrics evolution / SET / W&I PAGE 18

20 One metric is good, more metrics are better (?) Recall MI ln( V) 0.23V ( g) 16.2ln( LOC) Halstead McCabe LOC [Kaur, Singh 2011] propose an adaptation MIP CC 0.23ln( S) 16.2ln( NC) Related to PK 1 and instability Related to NOC and NOM Related to nesting, strongly connected components, abstractness and PK 2 / SET / W&I PAGE 19

21 Summary: package metrics Size: number of classes Dependencies à la fan-in and fan-out Marchesi s UML metrics Martin s D n : abstractness-instability balance or the normalized distance from the main sequence PASTA Aggregations of class metrics: reminder Metrics independent: average, sum, Gini/Theil coefficients from Assignment 6 Metrics dependent: Distribution fitting / SET / W&I PAGE 20

22 Measuring change: Churn metrics Why? Past evolution to predict future evolution Code Churn [Lehman, Belady 1985]: Amount of code change taking place within a software unit over time Code Churn metrics [Nagappan, Bell 2005]: Absolute: Churned LOC, Deleted LOC, File Count, Weeks of Churn, Churn Count, Files Churned Relative: / Mathematics and Computer Science PAGE 21

23 Case Study: Windows Server 2003 Analyze Code Churn between WS2003 and WS2003- SP1 to predict defect density in WS2003-SP1 40 million LOC, 2000 binaries Use absolute and relative churn measures Conclusion 1: Absolute measures are no good R 2 < 0.05 Conclusion 2: Relative measures are good! An increase in relative code churn measures is accompanied by an increase in system defect density R / Mathematics and Computer Science PAGE 22

24 Case Study: Windows Server 2003 Construct a statistical model Training set: 2/3 of the Windows Set binaries Check the quality of the prediction Test set: remaining binaries Three models Right: all relative churn metrics are taken into account / Mathematics and Computer Science PAGE 23

25 Open issues To predict bugs from history, but we need a history filled with bugs to do so Ideally, we don t have such a history We would like to learn from previous projects: Can we make predictions without history? How can we leverage knowledge between projects? Are there universal properties? Not just code properties but also properties of the entire software process / Mathematics and Computer Science PAGE 24

26 Metrics of software process How much will it cost us to build the system? How much effort has been spent on building the system? Effort estimation techniques Size-based Complexity-based Functionality-based More advanced techniques are known but go beyond the topics of this class / SET / W&I PAGE 25

27 Size-based effort estimation Estimation models: In: SLOC (estimated) Out: Effort, development time, cost Usually use correction coefficients dependent on Manually determined categories of application domain, problem complexity, technology used, staff training, presence of hardware constraints, use of software tools, reliability requirements Correction coefficients come from tables based on these categories Coefficients were determined by multiple regression Popular (industrial) estimation model: COCOMO / SET / W&I PAGE 26

28 Basic COCOMO E as T ce E effort (manmonths) S size in KLOC T time (months) a, b, c and d correctness coefficients b d / SET / W&I PAGE 27 Information system Embedded system log T a b c d More advanced COCOMO: even more categories log S

29 Advanced COCOMO / SET / W&I PAGE 28

30 Complexity-based effort estimation Do you recall Halstead? Effort: E = V * D V volume, D difficulty E ( N n N2)ln( n1 n2)* * N n 2 2 Potentially problematic: questioned by Fenton and Pfleger in 1997 Time to understand/implement (sec): T = E/18 / SET / W&I PAGE 29

31 Code is not everything Lehman 6: The functional capability < > must be continually enhanced to maintain user satisfaction over system lifetime. How can we measure amount of functionality in the system? [Albrecht 1979] Function points Anno 2012: Different variants: IFPUG, NESMA, Determined based on system description Amount of functionality can be used to assess the development effort and time before the system is built Originally designed for information systems / SET / W&I PAGE 30

32 Functionality and effort What kinds of problems could have influenced validity of this data? No data < 10% US comp. No data / SET / W&I PAGE 31

33 Functionality and effort 104 projects at AT&T from 1986 through 1991 ln( E est ) ln( FP) / SET / W&I PAGE 32

34 Function points Cost per fp What about the costs? $ $ $ $ / SET / W&I PAGE 33

35 How to determine the number of function points? [IFPUG original version] Identify primitive constructs: inputs: web-forms, sensor inputs, mouse-based, outputs: data screens, printed reports and invoices, logical files: table in a relational database interfaces: a shared (with a different application) database inquiries: user inquiry without updating a file, help messages, and selection messages / SET / W&I PAGE 34

36 Software is not only functionality! Non-functional requirement necessitate extra effort Every factor on [0;5] Sum * Result * Unadjusted FP 1994: Windows-based spreadsheets or word processors: / SET / W&I PAGE 35

37 Function points, effort and development time Function points can be used to determine the development time, effort and ultimately costs Productivity tables for different SE activities, development technologies, etc. Compared to COCOMO FP is applicable for systems to be built COCOMO is not COCOMO is easier to automate Popularity: FP: information systems, COCOMO: embedded / SET / W&I PAGE 36

38 But what if the system already exists? We need it, e.g., to estimate maintenance or reengineering costs Approaches: Derive requirements ( reverse engineering ) and calculate FP based on the requirements derived Jones: Backfiring Calculate LLOC (logical LOC, source statements) Divide LLOC by a language-dependent coefficient What is the major theoretical problem with backfiring? / SET / W&I PAGE 37

39 Backfiring in practice What can you say about the precision of backfiring? Best: 10% of the manual counting Worst: +100%! What can further affect the counting? LOC instead of LLOC Generated code, Code and functionality reuse / SET / W&I PAGE 38

40 Function points: Further results and open questions Further results OO-languages Open questions Formal study of correlation between backfiring FP and true FP AOP Evolution of functional size using FP / SET / W&I PAGE 39

41 How does my system compare to industrial practice? ISBSG (International Software Benchmarking Standards Group) 17 countries Release 11: > 5000 projects Per project: FP count, actual effort, development technologies / SET / W&I PAGE 40

42 Alternative ways of measuring the amount of functionality FP: input, output, inquiry, external files, internal files Interface Amount of functionality = size of the API Linux kernel = number of system calls + number of configuration options that can modify their behaviour E.g., open with O_APPEND / SET / W&I PAGE 41

43 Amount of functionality in the Linux kernel / SET / W&I PAGE 42 Israeli, Feitelson Multiple versions and variants Production (blue dashed) Development (red) Current 2.6 (green) System calls: mostly added at the development versions Rate is slowing down from 2003 maturity? Configuration options: superlinear growth change in option format/organization

44 Conclusions Package metrics Directly defined: D n, Marchesi metrics, PASTA Aggregation based Metrics-independent: average, sum, Gini coefficient Metrics-dependent: fitted distributions Churn metrics Effort estimation metrics / SET / W&I PAGE 43