A New Wave in IT Risk Management

Transcription

1 A New Wave in IT Risk Management Dr. Bill Curtis Chief Scientist, CAST & SVP, CAST Research Labs Director, Consortium for IT Software Quality (CISQ) 1 Keynote Curtis CAST 2011

2 Increasing Risk in IT Applications National Research Council Software for Dependable Systems As higher levels of assurance are demanded testing cannot deliver the level of confidence required at a reasonable cost. The ecosto of pe preventing e gall failures ues will usually be prohibitively expensive, so a dependable system will not offer uniform levels of confidence across all functions. The correctness of the code is rarely the weakest link. 2 Keynote Curtis CAST 2011 Jackson, D. (2009). Communications of the ACM, 52 (4)

3 Increasing Complexity of IT Applications Enterprise Applications Middleware User Interface Tier ASP/JSP/VB/.NET Web Services Application Logic Tier Java, C++, Frameworks Struts MVC, Spring Legacy Applications CICS Connector CICS Monitor (Cobol) Tuxedo Monitor (C) COBOL Batch Shell Scripts Data Management Tier EJB Hibernate Files Databases Database Preventing application level defects requires analysis of all the interactions between components of heterogeneous technologies 3 Keynote Curtis CAST 2011

4 Application Quality vs. Component Quality Application Quality Application quality measures how well individual components interact to deliver business functions Component Quality Application quality is driven by structural quality Code quality is the measure of individual components for compliance with standards and best practices in the context t of a specific language Good code quality Good application quality 4 Keynote Curtis CAST 2011

5 Structural Quality Is the Risk Issue Quality The degree to which a product meets its specified requirements Problem Problem Customers struggle to state functional requirements. They do not understand non-functional requirements. a failure to satisfy a non-functional requirement can be critical, even catastrophic non-functional requirements are sometimes difficult to verify. We cannot write a test case to verify a system s reliability The ability to associate code to non-functional properties can be a powerful weapon in a software engineer s arsenal. 5 Keynote Curtis CAST 2011

6 Structural Quality Limits Business Agility Ossified, contorted, complex legacy code Harmfully complex Hard to change Easy to hack into Costly to maintain Speed in responding to the market depends on the ease of modifying business applications Well-engineered, high-quality code Secure and robust Easy to change Cheap to maintain Faster to build 6 Keynote Curtis CAST 2011

7 The 4 th Wave in Software Engineering 4 Product What: When: Architecture, Quality characteristics, Reuse 2005 Why: Ensure software is constructed t to standards d that meet the lifetime demands placed on it 3 Process What: CMM, ITIL, PMBOK, Agile When: Why: Provide a more disciplined environment for professional work incorporating best practices 2 Methods What: Design methods, CASE tools When: Why: Give developers better tools and aids for constructing software systems 1 Languages What: 3 rd &4 th generation languages, structured programming When: Why: Give developers greater power for expressing their programs 7 Keynote Curtis CAST 2011

8 Supplementing CMMI Structural quality engineering supplements CMMI to unlock even more business value from applications CMMI focus process improvement Six Sigma SQE focus product improvement Design for Six Sigma CMMI provides a strong foundation on which to build a rigorous application quality program: Level 2 project quality practices Level 3 standardized quality processes Level 4 statistical quality management Level 5 quality innovations 8 Keynote Curtis CAST 2011

9 CAST & Structural Quality Measurement ANALYZERS APPLICATION KNOWLEDGE BASE AD GOVERNANCE DASHBOARD Oracle PL/SQL Sybase T-SQL SQL Server T-SQL IBM SQL/PSM C, C++, C# Pro C Cobol CICS Visual Basic VB.Net ASP.Net Java, J2EE JSP XML HTML Javascript VBScript PHP PowerBuilder Oracle Forms PeopleSoft SAP ABAP, Netweaver Tibco Business Objects Universal Analyzer for other languages TECHNICA AL METR RICS T APPLIC CATION META ADATA A H APPLICATION HEALTH Immediate Impact Robustness Performance Security On-going Impact Transferability Changeability APPLICATION SIZE Functional Technical Size Weight T TECHNICAL INVENTORY Analysis of all system artifacts M V MANAGEMENT VISIBILITY To assess, monitor, and improve applications, development teams and 3 rd Party delivery teams DRILL-DOWN TO ACTION Overview Portfolio Apps Modules Objects Health factors Submetrics Rules Compliance to remediation 9 Keynote Curtis CAST 2011

10 Uses of Structural Quality Measures Vendor Managers IT Executives FIN HR CRM ERP Portfolio insight App / Project Managers Deliverables insight Developers 10 Keynote Curtis CAST 2011 Application insight Remedial insight

11 AppmarQ The Missing Product Benchmark Organization IT Spend Level Staffing Risks Robustness Performance Product Project Security Level Level Costs Changeability Timelines Costs Size QA 11 Keynote Curtis CAST 2011

12 Security Scores Differ by Language Distribution of Security Scores. 4 Security scores by language ity Secur NET C/C++ COBOL Java EE Oracle 4GL Technologies Bimodal distribution of security scores indicate two types of apps Apps with security scores are predominantly from Financial Services 12 Keynote Curtis CAST 2011

13 Performance Scores Differ by Language Distribution of Performance Scores Performance scores by language. Performa ance Scores NET C/C++ COBOL Java EE Oracle 4GL Technologies Performance distribution is skewed towards higher scores Newer technologies show lower performance scores 13 Keynote Curtis CAST 2011

14 Changeability Scores Differ by Sector Distribution of Changeability Scores. Changeability scores by industry Change eability Sco ores lities nergy & Util E cials Finan Insura ance IT Consu ulting Tech n ology Industry Tele ecom Manufactu uring Governm ment Government applications show poor changeability Outsourcing: Government 75% Industry 50% 14 Keynote Curtis CAST 2011

15 Modularity Minimizes the Effect of Size 3,5 COBOL Applications (TQI vs Size) 120 Tota al Quality In ndex (TQI) 3,4 33 3,3 3,2 3,1 3 2,9 2,8 2,7 cts % of High Com mplex Obje ,6 2,5 R² = NET C/C++ COBOL Java EE Oracle 4GL Size - KLOC Technologies Except for COBOL, size has no impact on application quality Modularity reduces the effect of size on quality 15 Keynote Curtis CAST 2011

16 CISQ An Industry Initiative Co-sponsorship IT Executives es CISQ Technical experts 16 Keynote Curtis CAST 2011

17 CISQ Standards Process Knowledge Discovery Meta-model Technical Work Groups Structured Metrics Meta-model Function Points Defined Measures Maintainability ISO CISQ Exec Forum Reliability & Best Performance OMG ISO Performance Practices Security Methods for Metrics Use Weaknesses & Violations Code Pattern Metamodel ISO CVSS 17 Keynote Curtis CAST 2011