Structural Quality Measurement

Transcription

1 Software Analytics Structural Quality Measurement and the State t of IT Software Dr. Bill Curtis Chief Scientist, CAST & SVP, CAST Research Labs Director, Consortium for IT Software Quality (CISQ) 1 PSM 2011 Curtis 13JUL11 CAST 2011

2 How Do We Get to Dependable Software National Research Council Software for Dependable Systems As higher levels of assurance are demanded testing d d ti cannot deliver the level of confidence required at a reasonable cost. The cost of preventing all failures will usually be prohibitively expensive, so a dependable system will not offer uniform levels of confidence across all functions. Jackson, D. (2009). Communications of the ACM, 52 (4) 2 PSM 2011 Curtis 13JUL11 CAST 2011 The correctness of the code is rarely the weakest link.

3 Internal Quality Is Often Overlooked Quality The degree to which a product meets its specified requirements Problem Customers struggle to state functional requirements. They do not understand non-functional requirements. a failure to satisfy a non-functional requirement can be critical, even catastrophic non-functional requirements are sometimes difficult to verify. We cannot write a test case to verify a system ss reliability The ability to associate code to non-functional properties can be a powerful weapon in a software engineer s arsenal. 3 PSM 2011 Curtis 13JUL11 CAST 2011

4 What Is An Application Product Catalogue? hosts? threads Resource pools? Connection pools? Error handling? Timeouts? Retail Website? hosts?k threads Order Entry Application? hosts? threads Credit Card Application? hosts? threads GODOT s lament: Ever have an application hang waiting for a response that will never come? Shipping Application? hosts? threads Express Service Application? hosts? threads 4 PSM 2011 Curtis 13JUL11 CAST 2011

5 Multi-Tier, Multi-Language Applications Enterprise Applications Middleware User Interface Tier ASP/JSP/VB/.NET Web Services Application Logic Tier Java, C++, Frameworks Struts MVC, Spring Legacy Applications CICS Monitor (Cobol) Tuxedo Monitor (C) CICS Connector COBOL Batch Shell Scripts Data Management Tier EJB Hibernate Database Files Databases 5 PSM 2011 Curtis 13JUL11 CAST 2011

6 Revising Our Understanding of Defects Study of defects across 1 open source and 2 large NASA applications Observation Percent of cases Fixes mapping to > 2 files 60% Fixes mapping to > 3 files 30-40% Fixes mapping to > 2 components 10-36% Fixes mapping to > 2 subsystems 10-20% Narrow spread of faults Types of defects 80% of faults in 20% of files Requirements 33% Coding 33% Data 14% M. Hamill & K. Goseva-Popstojanova (2009). Common faults in software fault and failure data. IEEE TSE, 35 (4), PSM 2011 Curtis 13JUL11 CAST 2011

7 Application Quality vs. Code Quality Application Quality Application quality also measures how well the individual components work together to make up the overall business system Component Quality Code quality is the measure of individual components for compliance with standards and best practices in the context of a specific language g 7 PSM 2011 Curtis 13JUL11 CAST 2011 Good component quality Good application quality

8 The 4 th Wave in Software Engineering 4 Product What: When: Why: Architecture, Quality characteristics, Reuse 2000 Ensure software is constructed to standards that meet the lifetime demands placed on it 3 Process What: CMM, ITIL, PMBOK, Agile When: Why: Provide a more disciplined environment for professional work incorporating best practices 2 Methods What: Design methods, CASE tools When: Why: Give developers better tools and aids for constructing software systems 1 Languages What: 3 rd & 4 th generation languages, structured programming When: Why: Give developers greater power for expressing their programs 8 PSM 2011 Curtis 13JUL11 CAST 2011

9 Software Physiology 9 PSM 2011 Curtis 13JUL11 CAST 2011

10 Comparing Ancient Languages APL COBOL JAVA 10 PSM 2011 Curtis 13JUL11 CAST 2011

11 How the Data Were Collected N ORM TION ATFO LICAT E PLA APPL ENCE AST A LLIG CA NTEL IN PARS SERS UAGE ANGU LA Oracle PL/SQL Sybase T-SQL SQL Server T-SQL IBM SQL/PSM C, C++, C#, Pro C COBOL CICS Visual Basic VB.Net, ASP.Net Java, J2EE JSP XML HTML Javascript VBScript PHP PowerBuilder Oracle Forms PeopleSoft SAP ABAP, Netweaver Tibco Business Objects Universal Analyzer QUALI ITY METRI ICS ON A CATIO ADATA APPLIC META Security Performance Changeability 11 PSM 2011 Curtis 13JUL11 CAST 2011

12 The Computation of Quality Quality Metrics Subset Quality Indicators Health Factors Application Quality Over 800+ architectural and language e-specific cod de checks SQL Complexity Distribution Class complexity (Inher. depth) Class complexity (Inher. width) Artifacts having recursive calls Method complexity (control flow) Multiple artifacts inserting data on the same SQL table Coupling Distribution File conformity Dead code Structuredness Controled data access Empty code Modularity Encapsulation conformity Inheritance Package naming Class naming Interface naming Package comment Class comment Method comment Package size Class size (methods) Interface size Complexity Architecture Programming Practices Naming Conventions Documentation Size Performance Robustness Security Transferability Changeability Maintainability Immediate Impact Application Quality On-Going Impact 12 PSM 2011 Curtis 13JUL11 CAST 2011

13 Uses of Structural Quality Measures Vendor Managers IT Executives FIN HR CRM ERP Portfolio insight App / Project Managers Deliverables insight Developers Application insight Remedial insight 13 PSM 2011 Curtis 13JUL11 CAST 2011

14 14 PSM 2011 Curtis 13JUL11 CAST 2011 The Study Sample

15 Distribution of Application Size in KLOC 15 PSM 2011 Curtis 13JUL11 CAST 2011

16 Industry Segments & Technologies 16 PSM 2011 Curtis 13JUL11 CAST 2011

17 Top 4 Violations by Language Java EE.NET C C++ ABAP Use of accessors to Private Fields Artifacts with High Fan-Out Unreferenced Methods Unreferenced Fields Declaring Public Class Fields Artifacts with High Fan-Out Classes with a High Lack of Cohesion Artifacts with High Fan-In Artifacts with High Fan-Out Large Functions - too many Lines of Code Functions with SQL statement including Subqueries Artifacts with High Cyclomatic Complexity Data Members that are not Private Artifacts with High Fan-Out Included files including other files Artifacts with High Cyclomatic Complexity Artifacts with a Complex SELECT Clause Artifacts with High Fan-Out Artifacts with High Cyclomatic Complexity Artifacts with High Essential Complexity 17 PSM 2011 Curtis 13JUL11 CAST 2011

18 Distribution of Security Scores 18 PSM 2011 Curtis 13JUL11 CAST 2011

19 Security Distributions by Language urity Sec NET C/C++ COBOL Java EE Oracle 4GL 19 PSM 2011 Curtis 13JUL11 CAST 2011

20 Distribution of Performance Scores 20 PSM 2011 Curtis 13JUL11 CAST 2011

21 Performance Distributions by Language P erforma nce Sco res PSM 2011 Curtis 13JUL11 CAST 2011.NET C/C++ COBOL Java EE Oracle 4GL

22 Distribution of Changeability Scores 22 PSM 2011 Curtis 13JUL11 CAST 2011

23 Changeability Distribution by Industry Changeability Scores Energy & Utilities Financials Insurance IT Consulting Tech n ology Telecom Manufactu uring Government Industry 23 PSM 2011 Curtis 13JUL11 CAST 2011

24 Does Size Matter? ality Index tal Qua Tot 24 PSM 2011 Curtis 13JUL11 CAST 2011 K Lines of Code

25 R 2 TQI with Size for COBOL Apps R 2 =.45 Tota al Quality Ind dex 25 PSM 2011 Curtis 13JUL11 CAST 2011 K Lines of Code

26 % Complex Components by Language 120 Comp plex Ob bjects % of High NET C/C++ COBOL Java EE Oracle 4GL 26 PSM 2011 Curtis 13JUL11 CAST 2011 Technologies

27 GO TO Considered Eternal COBOL Use of GOTO Use of PERFORM... THROUGH THRU Missing WHEN OTHER when using EVALUATE Components with High Cyclomatic Complexity 27 PSM 2011 Curtis 13JUL11 CAST 2011

28 CISQ An Industry Response Co-sponsorship IT Executives CISQ Technical experts 28 PSM 2011 Curtis 13JUL11 CAST 2011