Schedule. Complexity of software systems. McCabe s cyclomatic complexity

Transcription

1 Beyond Lines of Code: Do We Need More Complexity Metrics + An Extensive Comparison of Bug Prediction Approaches Wei Wang Feb. 7 th, 2013 Schedule Background Complexity metrics Comparing complexity metrics Bug prediction Comparing bug prediction Discussion 2 Complexity of software systems Concentrate on complexity of source code Possible metrics for complexity: Source Lines of Code, Line of Code Number of functions McCabe s cyclomatic complexity (max/average) Halstead s length McCabe s cyclomatic complexity if statement Loop = # # + # 3 4

2 Halstead s length Focusing on the number of operators/operands (broadly defined) #Distinct operators/operands:, # Total operators/operands:, Length: = + Volume: = + ; = + Surveying software systems ArchiLinux.org : packages for Linux Level: = Effort: = 5 6 Result Interpreting the result LOC has a performance no inferior than other metrics that are based on syntactic complexity This may suggest that software complexity in other dimensions are not captured Metrics based on sematic info. may offer better result 7 8

3 Predicting (number of) bugs Comparing previous bug prediction techniques on the same set of corpus Combining bug fixing knowledge with other techniques Constructing a benchmark for bug predicting Predicting (number of) bugs Given a release x of a software system s, released at date d, the task is to predict, for each class of x, the number of post release defects, i.e., the number of defects reported from d to six months later Assumption I Technique based on Assumption I Complex changes are more error-prone than simpler changes

4 Metrics Number of revisions Assumptions II Number Number - Number Lines added and removed (sum, max, average) History tells the future: a defect in the past predicts new defects in the future. Code churn (sum, maximum and average) Change set size (maximum and average) Age and weighted age Type of bug Technique based on Assumption II Assumption III Complex code chunks are difficult to develop, maintain, or change, hence error-prone

5 Technique based on Assumption III Metrics Assumption IV Technique based on Assumption IV Entropy of source code, along with other metrics of source code, predicts whether that chunk of source code is error-prone

6 Technique based on Assumption IV Result I Worst performance, but it works best according to Zimmermann[PROMISE 07] Measure the complexity of the variants of a metric over subsequent sample versions. The more distributed over multiple classes the variants of the metric is, the higher the complexity Best performance Result II : combining BF Predictive power of past bugs Combining BF improves all items in adjusted Adding bug-fixes improves results Alternative explanation for this: For a software module that is more frequently used, the chance that we notice bugs in that module grows as well The complaint about lack of desired features can appears as bugs, and then get fixed Best 23 24

7 Cost in data collection and analysis Some metrics requires compilable source code Computational cost of generating data How actionable is the result? Complexity Is syntactical complexity really that bad? Bug Prediction We can first test classes that are predicted to be most error-prone. The cost of integrating heterogeneous data Link bugs with versioning systems Bugzilla may contain issues other than corrective maintenance. You want me to change my coding style to satisfy this clyclomatic metrics? Let us test error-prone classes more frequently Conclusion LOC is a good metric for measuring syntactical complexity in terms of performance and cost Churn of source code works better than other metrics in bug predicting Bug history can be a good predictor for future bugs 27