Utilizing measurement in the context of software maturity models Jari Soini Tampere University of Technology Information Technology Department Pori, Finland
Where I come from Pori University Consortium (2 500 students, 190 experts) Tampere University of Technology, Pori unit Information Technology Department
Presentation Content Introduction Background Measurement information system Structure Use Evaluation Summary
Introduction The motivation for the research is based on issues observed in relation to software process quality Finnish software companies have recognized the problems of monitoring and measuring their software processes There is a need for studying the means and tools that could help build a bridge - or at least strengthen the link - between SPI and measurement A brief demonstration of the measurement information system we have developed d
Background Research project SoMe (Software Measurement) Duration 2005 2007 Initiated t by FiSMA (the Finnish Software Measurement Association) Research and project coordination by Tampere University of Technology ogy (TUT) and the University of Joensuu (UJ) The aim to develop a web-based information system for Finnish software companies to help monitor and measure the quality of their own software processes and products Enhancing measurement knowledge sharing between software companies The final outcome of the SoMe project was an information system implemented in a web environment based on a metrics database
Structure of the Information system : The system consists of two main elements: a metrics database and aweb-based based user interface which operate together Metrics database
Metrics database The metrics database is collected from three different sources of information: practical experience, literature, and standards d Experiences (Interviews and questionnaire) Measurement Models (Academic theory) Measurement Standards analysis Metrics database Metric document Metric document Metric document Metric document.
Metrics database Experiences (Interviews and questionnaire) SW companies common characteristics are: they all carry out software development independently they are members of FiSMA (Finnish Software Measurement Association) they have previous experience of software measurement (ranging from 1 to 12 years) Company Business of the SW SW National or Employees Employees SPICE company Business Application Multinational total SE Level A Finance industry, ICT Customer Business Systems (BS) Na 220 220 3 B Finance industry, ICT Customer Business Systems Na 280 150 2 C Finance industry, ICT Customer Business Systems Na 450 35 2 D Finance industry, ICT Customer Business Systems Na 195 195 3 E SW engineering, ICT Customer, Product BS, Package Mn 15000 5000 2 F SW engineering, ITC Customer Business Systems Na 200 60 2 G ITC services Customer Business Systems Na 200 200 3 H Manufacturing Customer, Product Emb Syst, Package Mn 1200 30 1 I Manufacturing Product Emb Syst, Real-Time Mn 24000 200 2 J Automation industry Product Real-Time Mn 3200 120 3
Metric documents (1/2) The collected measurement information was combined and organized Pre-evaluating the information obtained Classifying collected data The captured knowledge was modified in the same framework Creation of a standard form Synchronizing terminology Helps the end user to read, perceive the logic and make a comparison between the knowledge items Metric document The metrics database consists of individual items of information, knowledge items (metrics), and the manifestations of these items i.e. metric documents
Metric documents (2/2) Metric document The following information exists on each individual metric captured in the information system: Purpose Formula (if required) Values (with a possible threshold value) Usage Workloads (establishing the metric, collecting the data, using the metric) Risks Experiences plus other information References
Web based User Interface The information system user interface comprises the main view including several functions related to metrics selection as well as guidelines for using the UI Search taxonomy CMMI SPICE Word search Metrics Glossary (library) Feedback Help Metric document
Example of the search taxonomy: SPICE The process groups of the ISO 15504, aka SPICE standard PRIMARY Life Cycle Processes ORGANIZATIONAL Life Cycle Processes SUPPORTING Life Cycle Processes Acquisition Process Group (ACQ) ACQ.1 Acquisition preparation ACQ.2 Supplier selection ACQ.3 Contract agreement ACQ.4 Supplier monitoring ACQ.5 Customer acceptance Supply Process Group (SPL) Management Process Group (MAN) MAN.1 Organizational alignment MAN.2 Organizational management MAN.3 Project management MAN.4 Quality management MAN.5 Risk management MAN.6 Measurement Support Process Group (SUP) SUP.1 Quality assurance SUP.2 Verification SUP.3 Validation SUP.4 Joint review SUP.5 Audit SUP.6 Product evaluation SPL.1 Supplier tendering SPL.2 Product release SPL.3 Product acceptance support Engineering Process Group (ENG) ENG.1 Requirements elicitation ENG.2 System requirements analysis ENG.3 System architectural design ENG.4 Software requirements analysis ENG.5 Software design ENG.6 Software construction ENG.7 Software integration ENG.8 Software testing ENG.9 System integration ENG.10 System testing ENG.11 Software installation ENG.12 Software and system maintenance Operation Process Group (OPE) OPE.1 Operational use OPE.2 Customer support Process Improvement Process Group (PIM) PIM.1 Process establishment PIM.2 Process assessment PIM.3 Process improvement Resource and Infrastructure Process Group (RIN) RIN.1 Human resource management RIN.2 Training RIN.3 Knowledge management RIN.4 Infrastructure Reuse Process Group (REU) REU.1 Asset management REU.2 Reuse program management REU.3 Domain engineering SUP.7 Documentation SUP.8 Configuration management SUP.9 Problem resolution management SUP.10 Change request management Each measurement factor was mapped so that the metric s purpose for the company and dthe life cycle phase where iti is used ddetermines which hprocess group the factor belongs to
Measurement Information System (Web -based measurement information system: http://gnosis.fi:8080/some-project/welcome.do) v
Measurement Information System v
Measurement Information System
Measurement Information System You can type here Feedback channel 1
Measurement Information System v You can type here
Measurement Information System Alfabethica al order
Measurement Information System Feedback channel 2
Measurement Information System Metrics + database (More information about the measurement information system: www.fisma.fi)
Observations and evaluation Process improvement cannot be measured directly with an individual metric but the general factors that relate to a certain process can The majority of the measurement data (metrics) involved in the metrics database were collected from companies whose capability levels varied between 2 and 3 The allocation and interpretation of the results in the maturity models (SPICE and CMMI) was made based on the authors experience
Summary This gives an example of how to utilize measurement in the context of software maturity models The aim was to point out to the user the connections between metrics and processes The system developed offers a bi-directional link, from software processes to software metrics and vice versa The basic idea behind developing this system was to help organizations utilize measurement knowledge to control their software development process
For future work From the software measurement viewpoint, one relevant topic for future research could be to examine how the capability or maturity level of the organization affects the utilization of the measurement tool and how, and if, the required measurement information varies depending on the level
Thank you for your attention look forward to seeing you in Pori!
Questions?
Measurement Information System
Measurement Information System Feedback channel