Revealing Semantic Quality Characteristics of Free and Open Source Software

Transcription

1 Revealing Semantic Quality Characteristics of Free and Open Source Software Stergios Draganidis, Kerstin V. Siakas Technological Educational Institute of Thessaloniki, Greece Department of Informatics Abstract The increasing contribution to FLOSS 1 development and the amplified use of its products during the past decades has proved to be a viable alternative to proprietary (closed source) software, not just by technically-aware developers, but also by non-developers. The target of FLOSS is mainly public administration bodies and industrial organisations, in particularly Small and Medium Enterprises (SMEs). Different quality models, such as ISO 9126, have been developed for defining quality characteristics and metrics aiming to evaluate the process and the products of proprietary software development [10, 11]. The absence of a common quality model in FLOSS causes confusion regarding Floss quality characteristics. To raise awareness to help potential users to distinguish between different quality level products the quality aspects should be explicit. A research project at the department of Informatics, Alexander Technological Education Institute of Thessaloniki 2, Greece investigates the quality of FLOSS considering underlying human factors that affect the six high level quality characteristics according to ISO The correlation of the quality characteristics described in ISO-9126 with measurable aspects and underlying factors of FLOSS collaborative development is useful for moving beyond superficial evaluation. A more scientific, rational and structured tool for FLOSS product assessment is introduced aiming to be used for further research on a large scale. The final objectives of this research project are to increase the comprehension of FLOSS quality attributes and the awareness of the dynamics that influence the development process. This paper describes the initial stage of the research project by introducing a new approach to determine quality level of FLOSS products. 1 Free/Libre Open Source Software 2 This work is funded by the Greek Ministry of Education (25%) and European Union (75%) under the EPEAEK II program Archimedes ΙΙ.

2 1. Introduction During the last decades a large number of FLOSS products have been developed. Many developers around the world spend a lot of their time to contribute to FLOSS projects that aim to deliver products targeting many application sectors. Trying to meet high quality standards by development of software through collaborative informal networks of professional or amateur programmers, several products have become competitive in a variety of applications, such as Gnome in desktop environments, Linux in operating systems, Mozilla in web browsers and Apache in web servers. FLOSS products are made available through networked distribution to developers and end-users usually free of charge. Although, the first FLOSS products where made by expert developers using appropriate procedures and metrics that ensure high quality products delivery, the quality of FLOSS projects is not based on a specific method or metrics. The evaluation of quality in these projects is based on arbitrary criteria that usually are not sufficient or adequate to represent the quality of the product. Such characteristics are given by projects hosts, such as sourceforge.net that provide some metrics, e.g. projects size, developer number, activity rank and weekly download [1]. Despite the significance of quality, no method or appropriate measures have been made to the majority of FLOSS products. A first attempt for quantification and measurement of some underlying factors of quality in FLOSS projects has taken place in 2002, by a European Commission project, called FLOSS [2]. The FLOSS project delivered a large number of measurements and primary data related to FLOSS usage and development aiming to provide indicators that rationally explain the functioning of the collaborative networks that do not have primary dependence on money. Furthermore the FLOSS project investigated possible impact of such non monetary collaborative development on computer industry, business and economy [2]. The design of semantic measurements for the process of FLOSS development and the derived products together with correlation of measures with quality characteristics is essential for the appropriate comprehension of the measures and quality characteristics. The construction of such measurements enables the appropriate evaluation of quality through rational indicators and provides a method for quality assurance in FLOSS projects. Until today ISO-9126 quality standards have been applied mainly to the internal characteristics of the FLOSS code [3, 4]. Our contribution concentrate on the external characteristics and quality in use of the ISO 9126 standard by taking a more societal approach trying to unfold the human characteristics and dynamics affecting the FLOSS development. ISO-9126 model was chosen because it provides a general framework that allows the estimation of a system s quality characteristics by dividing them into partly independent factors called quality factors.

3 In this paper, we consider quality of FLOSS products based on the six characteristics of ISO 9126 standard. The underlying factors that affect the quality of the products are investigated by quantification of basic attributes of the FLOSS development process. The main goal of this research is to increase the recognition of FLOSS quality characteristics and the awareness of the dynamics that influence the development process. The attempt to correlate the quality characteristics of ISO-9126 with measurable aspects and underlying factors of FLOSS collaborative development is useful for moving beyond superficial evaluation. The above correlation is expected to achieve a more scientific, rational and structured tool for FLOSS product assessment. 2. Free/Libre Open Source Software (FLOSS) The acronym FLOSS refers to software that gives the freedom to the user to run the program for any purpose, to study and modify the code, and distribute the original or modified program without any restriction or any obligation [2]. The above freedoms demand the availability of the source code. The acronym FLOSS was first used in July 2002 when the European Commission published a report titled Free/Libre and Open Software: Survey and Study [2]. The acronym FLOSS is used to describe the combination of the terms Free Software and Open Source Software emphasizing the libre meaning of the word free. Free Software as defined by the Free Software Foundation [5] and Open Source as defined by the Open Source Initiative [6] are two ideological movements that target the same goal, to evolve the knowledge and to develop better products. However, the motivations of each movement differ and many discussions have been made to disambiguate their nature. Despite the ideological disagreements between the two movements the terms free software and open source software are usually used interchangeably to describe the same thing. FLOSS products are developed by collaboration and co-creation of the code by developers and users of varying levels of programming experience. Developers are usually geographically dispersed and work for sake of the community projects independently of any organisation. Most of them work isolated in their own pace, space and time. They do generally not meet each other in person. In a typical open source project, a developer will work from existing or partial code to the development of a new tool or a utility. The developer contributes to the development procedure by posting the evolving code to a newsgroup whose members will try out the program and provide feedback. Members will provide their own programming solutions to problems that the original developer may confront or specify improvements to the original code. The enhanced code is then posted to the group, who accepts or discards it and redistribute the final product freely available to the wider community [7].

4 The most typical characteristic of the FLOSS development process is that developers are themselves users of the software [8]. As a result they are better positioned to understand and solve problems or bugs they encounter in the software. In proprietary software the main problems arise from the fact that the developer has difficulties to understand the often ambiguous user requirements [9]. Inadequate communication between the developer and the user creates inflexible to future changes products and products that are difficult to maintain. Many attempts have been made to formalise the requirements and involve the user to a higher degree in the software development process. One person acting as developer and user simultaneously in the development of FLOSS products seems to be a prominent feature affecting the quality characteristics of the software product. 3. ISO/IEC 9126 Quality Model The objective of the ISO/IEC 9126 Quality Model [10], a standard for software product quality, is to provide a framework for the evaluation of software quality. ISO/IEC 9126 is applicable to every type of software. It defines six high level product quality characteristics (Portability, Efficiency, Reliability, Functionality, Usability, Maintainability) each of which consists of several sub-characteristics. Each quality sub-characteristic is further divided into attributes. An attribute is an entity which can be verified or measured in the software product. Attributes are not defined in the standard, as they vary between different software products. The standard is divided into four parts which address quality model, external metrics, internal metrics and quality in use metrics as follows [10]: ISO/IEC (2001) provides the concepts introduced in the original standard. The recommended quality model categorises software quality in six characteristics, which are further sub-divided into sub-characteristics. Also a definition of quality in use is included, which defines the user's view as a result of using the software. ISO/IEC (2003) provides external metrics for measuring software quality characteristics. An external metric can be used for measuring an attribute or characteristic of a software product, derived from the behaviour of the system of which it is a part. External metrics are applicable to an executable software product. ISO/IEC (2003) provides internal metrics for measuring software quality characteristics that do not rely on software execution. An internal metric can be used for measuring an attribute or characteristic of a software product, derived from the product itself (not from its behaviour), either direct or indirect Internal metrics are applicable to a non executable software product during designing and coding in early stage of development process. ISO/IEC TR (2004) provides quality in use metrics for measuring the attributes defined in ISO/IEC Quality in use metrics are only available when the final product is used in real conditions.

5 4. Research plan and methodology In many situations the quantification of quality attributes is not easy to achieve. The measure and the explanation of many of the attributes create difficulties for the potential users, such as organisations, communities and individuals to implement a systematic approach for quality evaluation. Initially this research concentrated on literature review regarding software quality issues and the existence of previous FLOSS research. The deliverables of the European Commission project in 2002 [2] was the trigger to motivate us to correlate similar measurements with quality characteristics of ISO Demonstration of adequate and well-defined attributes revealing high quality levels of FLOSS product is anticipated as result derived by the investigation of the above correlation. 4.1 Research methodology adopted The research method in this study is a survey using an on-line questionnaire. The objective of the questionnaire is to acquire primary data that describe attributes and processes constituting FLOSS development. The data collected by the questionnaire will be analysed in order to point out quality related attributes of the FLOSS development. The target group for the questionnaire is active persons of FLOSS projects such as project managers, single users, programmers, etc. The research will be carried out from April 2007 until August The collected data will be analysed later using a statistical analysis environment, such as SPSS and appropriate methods of data mining in order to point out quality related attributes of the FLOSS development. For that purpose, the data will be stored in a database system to simplify appropriate transformation of the data before the analysis. An important issue in surveys is the size of the target population. Fowler [12] argues that the size of the population from which a particular size is drawn has virtually no impact on how well that sample is likely to describe that population. He states that most sample size decisions concentrate on the minimum sample sizes that can be tolerated for the smallest subgroups of importance. In our research a relatively big population is required since FLOSS development involve distributed developers and the aims of the project is amongst others to evaluate the profile and the motivation of the developers, the processes used correlated to perception of received product quality. However, sample size may not indicate representativeness. The sample needs to reflect the open source community as a whole. Our target group is the Greek Floss community and may not be representative for the wider FLOSS community. However, the results from this project can contribute to another project on a wider scale.

6 4.2 Questionnaire design For the purpose of measuring the quality characteristics and the underlying factors that affect the quality of FLOSS a survey questionnaire was designed. This is used to correlate response sets between different groups of people. We created a questionnaire with 21 questions relating to significant attributes describing human features and processes that have impact on quality of FLOSS according to ISO The design of the questionnaire was created by having simplicity, user friendliness and wide acceptance in mind. In particular we tried to formulate unbiased understandable questions. The design of our questionnaire was aiming to capture attributes of human factors and development processes leading to high level of FLOSS product quality in the first place to the Greek community and later to the wider public. A small amount of demographic data related to the responders was included to enable clustering of the results and investigation of possible impacts to the quality characteristics. The initial questionnaire was developed in the Greek language aiming to be used in a pilot study. In order to evaluate the questionnaire regarding the validity and outline and to secure future commitment from the FLOSS community it was distributed to many persons in charge of Greek communities that evolve FLOSS development. We received 9 responses with useful comments regarding ambiguity of questions or absence of representative responses. This feedback was used to improve the questionnaire, which also was translated to English in order to reach a greater population [13]. The moderated questionnaire was made available on-line to enable easy completion and analysis of the results. The promotion of the on-line questionnaire to the FLOSS communities should be made in order to collect statistically sufficient responses to produce valid results. 4.3 Data analysis Considerable effort was made to enable the extraction of valid results rapidly from the primary data. To achieve this, a system that collects the questionnaire responses and processes them accordingly has been developed. The main components of the derived system are described in Figure 1.

7 Figure 1: Data Analysis System The statistical analysis of the data reveals factors that can have impact on the quality of FLOSS products. Appropriate statistical analysis of univariate and multivariate cases produce the first results based on the primary data. Data mining methods will be applied to predict the behaviour of the project with respect to quality characteristics and classify projects according to their general quality measurements. The measure and the explanation of many of quality attributes are likely to create difficulties for potential users, such as organisations, communities and individuals to implement a systematic approach for quality evaluation. By scientifically evaluating the quality of a software products and not the process, software engineers can leverage the knowledge achieved in many ways [14]. Our approach evolves the research approaches that concentrate on internal characteristics of ISO to include external characteristics and quality in use. 6. Conclusions and Further Work A new approach to determine quality level of FLOSS products was introduced. The new approach tries to eliminate the absence of method for quality evaluation of FLOSS products by revealing semantic attributes that has an impact on the quality characteristics of FLOSS products regarding ISO This research project described is carried out at the department of Informatics, Alexander Technological Education Institute of Thessaloniki, Greece. The initial stage of the research project was described by introducing the research methodology.

8 Our contribution concentrate on the external characteristics and quality in use of the ISO 9126 standard by taking a more societal approach trying to unfold the human characteristics and dynamics affecting the FLOSS development. The correlation of quality characteristics of ISO-9126 with measurable aspects and underlying factors of FLOSS collaborative development is expected to provide a more scientific, rational and structured tool for FLOSS product assessment. Further research is necessary for receiving valid results. Next steps will include further promotion of the questionnaire and analysis of the collected data. In the future, the analysis of the data will be carried out by using a statistical environment such as SPSS and data mining methods in order to produce different perspectives and summarize them to useful information. 7. References 1 Zhou, Y., Davis, J. (2005): Open source software reliability model: an empirical approach, Open Source Application Spaces: Fifth Workshop on Open Source Software Engineering (5-WOSSE), St Louis, MO, USA 2 Free/Libre and Open Source Software (FLOSS) (2002): Survey and Study. International Institute of Infonomics University of Maastricht, The Netherlands, (visited December 2003) 3 Spinellis D. (2006): Code Quality: The Open Source Perspective, Addison-Wesley Professional 4 Samoladas I., Stamelos I. (2006): Assessing Free/Open Source Software Quality, (visited April 2007) 5 The Free Software Foundation, (visited March 2007) 6 Open Source Initiative, (visited March 2007) 7 Faber B. D. (2002): Educational Models and Open Source: Resisting the Proprietary University, SIGDOC 02, Toronto, Ontario, Canada, October Sowe S. K., Stamelos I., Angelis L. (2006): Identifying knowledge brokers that yield software engineering knowledge in OSS projects. Information & Software Technolog, Vol. 48, Issue 11, pp Wiegers K. E. (2003): Software Requirements, 2nd edition, Microsoft Press, Portsland 10 ISO 9126, (visited March 2007) 11 Siakas K. V., Georgiadou E. (2005): PERFUMES: A Scent of Product Quality Characteristics, The 13th Software Quality Management Conference, SQM 2005, March 2005, Glouchestershire, UK 12 Fowler, F. (1993). Survey Research Methods, Newbury Park, CA: Sage Publications, London 13 On-line FLOSS questionnaire, 14 de Groot, A., Kugler, S., Adams, P. J., Gousios, G. (2006): Call for Quality: Open Source Quality Observation, in IFIP International Federation for Information Processing, Volume 203, in Open Source Systems, eds. E. Damiani, B. Fitzgerald, W. Scacchi, G. Scotto, Boston: Springer, pp