Architecture Level Software Quality Prediction

Transcription

1 Architecture Level Software Quality Prediction PerOlof Bengtsson Department of Computer Science and Business Administration University of Karlskrona Ronneby Abstract. This paper describes a method for prediction of software qualities from the software architecture level using scenarios. The method takes the specification of the architecture as input and generates a prediction of the modified volume for the average maintenance task. The scenarios are used to analyse the architecture and to derive a prediction of the maintainability. The method is based on experiences and ideas from the design of a software architecture for a haemo dialysis machine. This far the method has been designed to incorporate only the prediction of maintainability. However, it is ongoing work to incorporate other common software qualities into the method. Experiments for evaluation and validation of the method are ongoing and future work. 1 Introduction One of the biggest issues in software development today is the software quality. The biggest problem is no longer to design and implement the correct functions in products. Instead, software qualities, e.g. performance, reliability, maintainability and flexibility, are often the factors of success or failure. Software Architecture has emerged during the recent years as a means to effectively deal with software qualities. Moreover, it has been recognised [1,2] that the software architecture sets the boundaries for the software qualities of the resulting system. Traditional object oriented software design methods [3,7,11] do not include any particular support for designing software with specific software quality levels, other than reusability and flexibility. However, we have identified that no single system is just a real-time system or a fault-tolerant system, where all other software qualities are of no importance. Instead, systems require a balance of different software qualities, e.g. a real-time system that is impossible to maintain or a high-performance computing system with no reliability is of very little use for anybody. To address these issues we are working on a method for designing software architecture, i.e. the ARCS method [4]. In short, the method starts with an initial architecture where little or no attention has been given to the required software qualities. This architecture is evaluated using a set of possible techniques and the result is compared to the requirements. Unless the requirements are met, the architect transforms the architecture in order to improve the software quality that was not met. Then the architecture is again evaluated and this process is repeated until all the software quality requirements have been met or until it is no longer feasible economically or technically to reach the requirements. In this method, the evaluation of software architecture plays a central and important

2 part. However, software architecture evaluation is not currently understood very well, and only a few methods exist, e.g. [1]. As a part of the research done on the ARCS method, we also investigate the possibilities of evaluating software architecture. Therefore, we propose a Formalised Scenario Prediction Method (FSPM). The method is based on the experiences and ideas from the design of an haemo dialysis architecture. In the project, we used scenarios to make the requirements of the architecture and the derived products, more tangible. The use of scenarios as design drivers made it natural and intuitive to use scenarios for analysing the architecture with respect to the software qualities of resulting products. For example, a driving requirement and actually the most important reason for the initiative was to improve the maintainability of the software for a haemo dialysis machine. To assess the maintainability of a family of dialysis machines derived from the proposed architecture, we used change scenarios. By scripting the scenarios [9], i.e. analysing them with respect to the modifications required to the individual components in the architecture for every scenario, we were able to identify several weaknesses and possible improvements. With these experiences as foundation, we believe that the scenarios, if formalised, can be successfully used to predict software quality factors as early as from architecture design. The main reasons for this is that it differ from ordinary metrics, i.e. scenarios and the analysis of them provide context sensitivity which allows for more accurate identification of exceptional cases and their appropriate analysis. The basis is that supported with a formal method, for making a software quality prediction, the persons involved get the appropriate guidelines to achieve an accurate prediction. Other benefits with the scenario approach are the partial reuse of specification effort. The same super set of scenarios is used as selection base for both design drivers and analysis & prediction. From this super set two disjunct subsets of scenarios are taken. One for use as design drivers and one for use as input to the prediction analysis. The risk of analysing the same scenarios as used for design drivers is now substantially reduced, if not eliminated. The scenarios must be disjunct to provide a valid sample of the scenarios. If the same subset of scenarios, i.e. samples, are used both for design and prediction, the sample are not randomly selected and the prediction will only be valid if the real maintenance case are absolutely identical with the scenario sample. Section 2 presents the steps of the method in detail. Section 3 presents a discussion about the method and the paper is concluded in section 4 with a presentation of ongoing and future work related to the method. 2 Method Step By Step Inputs to the method are the requirement specification, architecture description and optionally a domain specification if available. Out put from the method are; set of scenario categories, set of scenarios for each category with associated weighting, analysis results and a predicted value. The method is divided into a sequential series of steps: 1. Identify categories of maintenance tasks. 2. Synthesise scenarios. 3. Assign each scenario a weight.

3 4. Script the scenarios. 5. Estimate the size of every element in the architecture. 6. Calculate the predicted maintenance effort. 2.1 Identify categories of maintenance tasks Software maintainability is defined by IEEE [6] as: The ease with which a software system or component can be modified to correct faults, improve performance or other attributes, or adapt to a changed environment. This definition renders three categories of maintenance, i.e. corrective, perfective or adaptive. However, these categories are to abstract to be relevant in this process step. Instead, the categories are found in the application or domain description. For example, a dialysis machine might have maintenance scenarios concerned with treatment maintenance, hardware changes, safety regulations changes, etc. These categories reflect the meaning of maintenance in the context of this application or domain and give the designer more clues about the requirements posed on the architecture. Practical hint: Take the specification from the customer and/or the requirements specification. Quickly list from that what the different categories of maintenance are from the customers perspective. Then add what becomes visible from the development perspective. This will give you a good starting point for the next step. 2.2 Synthesise scenarios Now you have a set of categories of maintenance tasks. For each of these categories, synthesise and define as many concrete scenarios as possible. You should have about ten scenarios of each category. The scenarios should be very concrete, avoid specifying types of maintenance or categories again. For example, a scenario could be, Safety regulations change and require one added temperature alarm to the hydraulic module in the dialysis machine. Another example is, the hardware interface for the pump is to be changed from duty cycle into a digital interface, where the set value is in kp (kilo Pascal) Worth noting is that the scenarios used are not what is known from OO-design methods, or the same as use-cases. Instead a scenario is the description of a situation that might occur, e.g. a certain maintenance task. Practical hint: Use the categories and the requirement specification to identify relevant scenarios. Concrete scenarios should have a title to which it can be referred and then a more elaborate textual description. 2.3 Assign each scenario a weight The prediction method is dependent on having probability estimates for each scenario. These probabilities serve as weighting to control the impact on the prediction of more

4 occurring and less occurring maintenance tasks. We define the weight measure as the probability of this scenario resulting in a maintenance task from the release of the software until the succeeding maintenance release. This means that scenarios that describe often-recurring maintenance tasks will get higher probabilities and therefore impact the predicted value more. Consequently, the often-recurring scenario might offer more possibilities in optimising the architecture than the seldom-occurring scenarios that intuitively require more effort. The weight is produced in two ways, depending on the availability of historical/empirical data. The first time, no empirical data available, the scenarios weights should be estimated such that the sum of all the probabilities is one (1.0). The sum of probabilities for a category of scenarios reflect the expected value as agreed with the customer of product management. In the case, when empirical data about maintenance of the product exists in the organisation, the probability data of earlier maintenance efforts should be used. In addition, the categories should be synthesised from that data. Now, it is possible to calculate a probability figure for each category. That figure should serve as a firm guideline for all the weights of that category. The sum of the weights of the scenarios in a category should be the same as the figure for the category as a whole. Practical hint: Use the description about the product, and if available, the domain specification the first time to set a first proposal of probabilities. Then discuss the figures with the customer to validate your estimates. 2.4 Script the scenarios Providing that the selected sub set of scenarios of the different categories are representative as a sample of the actual maintenance tasks we can analyse to what extent the architecture will be modified. For each scenario analyse the impact the realisation of that scenario would have on the architecture and its components, i.e. find what components are affected and to what extent will they be affected. This is called scripting in the 4+1 View Model [9]. For example, adding a temperature alarm in the dialysis machine would require changes to the hydraulic module component, and addition of three (3) new components of type device and controlling algorithm. In addition, the components for system definition and the protective system need to be changed. Practical hint: If you have problem to determine if the component is affected by in a change scenario, the component should be counted as modified. The lack of clearly divided responsibilities in an architecture makes the maintenance harder. 2.5 Estimate the size of every element in the architecture To calculate the maintenance prediction the size of the architecture need to be known and the size of the affected components need to be known. In this case, we have three options. First, the size of every component can be estimated using the estimation technique of choice. Secondly, an adaptation of an OO metric (SIZE2 [5]) metric may be

5 used (SIZE2 ). Finally, when empirical data exists, the size is taken from that and interpolated to new components. At this point in time, we have no information or evidence as to which one is the better. Practical hint: If you have no empirical data available, you might want to consider different methods for estimation, e.g. Putnam [10]. 2.6 Calculate the predicted maintenance effort The prediction value is a weighted average for the effort (expressed as size of modification) for the average maintenance task. To predict the total effort of a maintenance project an estimation or calculation of the number of maintenance tasks has to be done. That figure is then multiplied with the results from this method. However, to compare two architecture alternatives, we only need the weighted average effort of a maintenance task. M tot = k s n = 1 k c PS ( n ) VS ( n, C ) m m = 1 (1) 3 Discussion P(x) the probability weight of scenario x V(x,y) the affected volume of the component y in scenario x k s = number of scenarios k c = number of components in architecture The relevance of this method is dependent on three criteria. First, the effort it takes to perform the method must correspond to the resulting benefits. We believe that this method adds very little work to the state of practise in development. Already today, most developers produce some sort of high-level design. The effort to turn this into a software architecture is motivated, not only, by the results from this method, but also from increased understanding of the system and the gains in detailed design and implementation. The effort of producing the appropriate scenarios is small. For a medium sized system (10-50kLOC) this would not have to require more than a day, or so, for the relevant persons to specify a set of scenarios. Second, the placement in the software life cycle and development process. We need to early assess and predict how software systems will behave in later phases. It is easy to imagine the benefits of stopping an impossible product before it is implemented and tested. Finally, the accuracy of the prediction plays an important role. If the method produces results with an accuracy that could easily be provided by any ad hoc estimation techniques, it would have to be considered irrelevant. Unless, of course, the effort required is less than the ad hoc estimation.

6 4 Ongoing and Future Work The performance of the FSPM method is currently under investigation. Our aim is to evaluate the effort required to use the method, to evaluate the accuracy in the prediction and to benchmark it with other maintainability predictions methods [8]. This is done by experiments using 5 student projects that are developing software systems for industry customers. These projects are divided into a development project (~1500h) and a maintenance project (~700h). The products are developed independent from each other. Currently we hope to do additional experiments and industrial case studies to improve the method. Our goal is to make the method general to allow usage for several different software qualities, e.g. safety, reliability, robustness etc. The reason is the need for balancing of software qualities and that we want to use it for the ARCS method. Acknowledgements Thanks to my advisor Jan Bosch, Michael Mattsson and the ARCS group for inspiring discussions about the method. References 1. L. Bass, P. Clements, R. Kazman, Software Architecture In Practise, Addison Wesley, PO Bengtsson, J. Bosch, Scenario Based Software Architecture Reengineering, Proceedings of International Conference of Software Reuse 5 (ICSR5), G. Booch, Object-Oriented Analysis and Design with Applications, (2nd edition), Benjamin/ Cummings Publishing Company, J. Bosch, P. Molin, Software Architecture Design: Evaluation and Transformation, submitted, S.R. Chidamber and C.F. Kemerer, Towards a metrics suite for object-oriented design, in proceedings: OOPSLA'91, 1991, pp IEEE Standard Glossary of Software Engineering Terminology, IEEE Std I. Jacobson, M. Christerson, P. Jonsson, G. Övergaard, Object-oriented software engineering. A use case approach, Addison-Wesley, W. Li, S. Henry, Object-Oriented Metrics that Predict Maintainability, Elsevier Publishing, New York, Journal of Systems and Software, vol. 23, no. 2, November, 1993, pp P.B. Krutchen, The 4+1 View Model of Architecture, IEEE Software, pp , November L. H. Putnam, Example of and Early Sizing, Cost and Scehdule Estimate for an Application Software System, Proceedings of COMPSAC 78, IEEE, pp , Nov J. Rumbaugh, M. Blaha, W. Premerlani, F. Eddy, W. Lorensen, Object-oriented modeling and design, Prentice Hall, 1991.