Functional Size Measurement Revisited. Cigdem Gencel 1 Onur Demirors 2. Abstract

Transcription

1 Functional Size Measurement Revisited Cigdem Gencel 1 Onur Demirors 2 Abstract There are various approaches to software size measurement. Among these, the metrics and methods based on measuring the functionality attribute have become widely-used since the original method was introduced in 1979 [7]. Although functional size measurement methods have gone a long way, they still provide challenges for software managers. This paper identifies improvement opportunities based on empirical studies we performed on ongoing projects. We also compare our findings with the extended dataset provided by the International Software Benchmarking Standards Group (ISBSG). 1.Introduction Software size measurement continues to be a significant practical problem in software engineering. Without a solid baseline of size neither estimation and planning nor control for large scale software projects is possible. As a result, estimation errors are reported to be essential causes of poor management [26] [28]. To overcome estimation-based management problems, various approaches to software size measurement and estimation have been developed. Among those, the metrics and methods on measuring the functionality attribute have been widely-used. After the description of the original method in 1979 by Albrecht [7], variations of these methods have been developed to improve the preceding ones. In 1996, ISO established the common principles of FSM methods and published ISO/IEC standard, thereby promoting the consistent interpretation of FSM principles [29][33][34][35][36][37][38]. Although many improvements have been made, FSM methods still do not have the unified acceptance by the software community as the size measurement methods used in traditional engineering disciplines. In this paper we discuss the literature on FSM and the improvement opportunities we identified by performing empirical studies as well as analyzing the projects data in the ISBSG dataset [32]. 1 Dr., Middle East Technical University, Informatics Institute, Ankara, Turkey; cgencel@ii.metu.edu.tr 2 Assoc. Prof., Middle East Technical University, Informatics Institute, Ankara, Turkey; demirors@ii.metu.edu.tr

2 We made a comprehensive literature survey on FSM in order to depict the candidate improvement opportunities and then evaluate them by means of empirical studies. The literature survey revealed similarities and differences between FSM methods as well as suggestions for improvement. We designed and conducted two case studies. The first was a multiple-case study which involved three different cases. In this case study, our objectives were to explore the applicability of two well-known FSM methods - Mk II FPA [41] and COSMIC FFP [39] - to measure the functional size of software systems for different functional domain types and to examine the differences between the measurement processes and the elements of these methods. The second was designed as a single-case study, which was conducted to explore the applicability of four different methods at early phases of the software development life cycle. Empirical studies revealed improvement opportunities on the significance of the FSM process in measuring functional size, convertibility of functional sizes obtained by different methods, applicability of the methods to different functional domains, early reliable functional size estimation and relationship between functional size and the two attributes: length of code and effort. This paper is organized as follows: the related research on software size metrics and size estimation / measurement methods are briefly summarized in the second section. We discuss the case studies we performed in the third section. The identified improvement opportunities of FSM are presented in the fourth section. Finally, conclusions are given in the last section. 2. Related Research There have been several significant attempts to measure functionality of software products to depict size. Initially, in 1979, Albrecht designed Function Points (FP) metric and Function Points Analysis (FPA) method for measuring software size [7]. This is based on the idea of measuring the amount of functionality delivered to the users in terms of Function Points (FP). Later Albrecht and Gaffney improved this method [8]. After that, variants of the method were developed. During the 1980s and 1990s, several authors suggested new counting techniques that intended to improve the original FPA or extend its domain of application [70]. The results of the literature survey on the methods which measure functionality attribute are presented in Table 1.

3 Table 1 Methods measuring the functionality attribute Year Method Developer 1979 Albrecht/ International Function Point Users Group Albrecht [7][30][40] (IFPUG) Function Point Analysis (FPA) 1982 DeMarco s Bang Metrics DeMarco [15] 1986 Feature Points Jones [44] 1988 Mark II FPA Symons [41][68][74] 1990 Netherlands Software Metrics Users Association The Netherlands Software Metrics Users Association [73][42] (NESMA) FPA 1990 ASSET- R Reifer [62] D Function Points Whitmire [77] 1994 Object Points Banker et al. [47][10] 1994 Function Points by Matson, Barret and Mellichamp Matson et al. [53] 1997 Full Function Points (FFP) University of Quebec in cooperation with the Software Engineering Laboratory in Applied Metrics [3] 1997 Early FPA (EFPA) Meli [56][57], Conte et al. [14] 1998 Object Oriented Function Points Caldiera et al. [13] 1999 Predictive Object Points Teologlou [71] 1999 Common Software Measurement International COSMIC [39], [4] Consortium (COSMIC) FFP 2000 Early&Quick COSMIC FFP Meli et al. [14][58] 2001 Object Oriented Method Function Points Pastor and his colleagues [59] 2000 Kammelar s Component Object Points. Kammelar [46] 2004 Finnish Software Metrics Association FSM The Finnish Software Metrics Association (FiSMA) [23] Evolution of these methods in time resulted in conceptual inconsistencies among the methods. In 1996, ISO initiated a workgroup to clarify the conceptual basis and to establish the common principles of FSM methods. In 1998, the first part of ISO/IEC Information Technology - Software Measurement - Functional Size Measurement [33][29] was published. This standard defines the fundamental concepts of FSM such as FURs 3, Functional Size 4, Base Functional Component (BFC) 5, BFC Type 6 and the FSM requirements that should be met by a candidate FSM method in order to be conformant. After that the remaining parts were published [34][35][36][37][38]. As of today, four methods are certified by ISO as an international standard; Mk II FPA [41], IFPUG FPA [40], COSMIC FFP [39] and NESMA FSM [42]. Although all FSM methods measure the functional size by means of the functionality delivered to the users, the main differences between these techniques arise from what they count and how they do it. In Figure 1, FSM Process of IFPUG FPA, Mk II FPA and COSMIC FFP methods are compared. We do not include NESMA FSM method, which has a very similar FSM process to IFPUG FPA. 3 FURs: a sub-set of the user requirements. The FURs represent the user practices and procedures that the software must perform to fulfill the users needs. 4 Functional Size: a size of the software derived by quantifying the FUR. 5 BFC: an elementary unit of FUR defined by and used by an FSM Method for measurement purposes. 6 BFC Type: A defined category of BFCs. A BFC is classified as one and only one BFC Type.

4 Functional User Requirements (FURs) Functional User Requirements Defined Determine Purpose, Scope, Viewpoint & Type of Count IFPUG FPA v.4.1 Mk II FPA v COSMIC FFP v.2.2 Purpose, Scope, Viewpoint & Type of Count Identify Software Layers Software Layers Identify Application Boundary of Count Application Boundary of Count Identify Elementary Components of FURs Functional Processes (COSMIC) Identify Base Functional Components (BFCs) Elementary Processes (IFPUG) Identify Data Functions (IFPUG) Logical Transactions (MkII) Identify & Categorise Data Entity Types (Mk II) Data Movemen Types (COSMIC) Identify Data Groups Classify BFCs into BFC Types Data Functions (Files) Data Entity Types (Mk II) Data Groups (COSMIC) Transaction Functions (EI/EO/EI), Data Functions (EIF/ILF) (IFPUG) E,X,R,W (COSMIC) Determine the Base Counts (#of DETs, #of FTRs) for (EI,EO,EQ), (#of DETs, #of RETs) for (EIF,ILF) (IFPUG) Determine Complexity (# of DETs in the I, # of DETs in the O, # of referenced PEs ) (MkII) (# of E, # of X, # of R, # of W) (COSMIC) Determine Contribution to Functional Size Apply Measurement Function and Calculate Functional Size (Small/Ave./Large) for TFs, DFs (IFPUG) Weight for TFs, DFs according to complexity (IFPUG) (Industrial Weight for I, O, PE) (MkII) Functional Size Measured FunctionalSize (IFPUG FP) FunctionalSize (MkII FP) FunctionalSize (COSMIC FP) Figure 1 FSM Process of IFPUG FPA, Mk II FPA and COSMIC FFP TF: Transactional Function, DF: Data Function, I: Input, O: Output, PE: Processing Entity, EI: External Input, EO: External Output, EQ: External Inquiry, ILF: Internal Logical File, EIF: External Interface File, E: Entry, X: Exit, R: Read, W: Write

5 The software entity measured by FSM methods is the set of FURs. In practice, FUR may exist in the form of a requirements specification document, or they have to be derived from other software engineering artifacts such as architecture, design or even from the artifacts installed on the computer system after it has been implemented [72]. FSM methods define two steps for the abstraction of software that represents the items deemed relevant for functional size [21]. The first step of the FSM process is to extract the FUR from these artifacts and to express them in the form of the FSM method s model suitable for measuring functional size. The FUR are functionally decomposed into Transactions, which are sequences of input, processing and output, not necessarily in that order, triggered by events outside the software [37]. From these Transactions, BFCs are identified and then each of these is categorized to BFC Types and the attributes relevant for obtaining the base counts are identified. The second step of abstraction involves the actual measurement, i.e. the functional size of each BFC is measured by applying a measurement function to the BFC Types and the related attributes. Then the results are summed up to compute the overall size of the software system. The BFCs of Mk II FPA are the LTs. There is only one type of BFC: the LT. In order to assign numeric values to each BFC, some of the FSM methods identify and evaluate the constituent parts from which the BFC types are composed as well. A LT has three constituent parts: input, process and output components. The BFCs of IFPUG FPA are the Elementary Processes. These are further classified as the Transactional Function Types and Data Function Types. A Transactional Function Type can be of the type: External Input, External Output, or External Inquiry, whereas a Data Function may be an External Interface File or an Internal Logical File. COSMIC FFP method introduces another concept; Elementary Components. Each FUR is decomposed into its elementary components, called Functional Processes. In COSMIC FFP method, each of these Functional Processes is assumed to comprise a set of sub-processes, called Data Movement Types, which are declared as the BFCs of this method. A Data Movement Type is defined as a BFC which moves one or more data attribute types belonging to a single data group type [39]. There are four kinds of Data Movement Types: Entry, Exit, Read, and Write. Each of these is defined as a BFC Type. In the literature there exist a number of studies on the evaluation and comparison of the FSM methods. In [52], Lother and Dumke evaluated FSM methods with respect to a number of criteria and discussed the issues of FSM. Fetcke et al [21] proposed a model as a generalized representation of different FSM methods stressing their commonalities and differences. In another study [17], Demirors and Gencel evaluated three estimation methods - Mk II FPA, Jones Very

6 Early Size Predictor and Early FPA - by applying them to a case project at different phases early in the life cycle. In [67] Rule discussed the similarities and differences between IFPUG FPA and Mk II FPA in his study. In [63], Rollo discussed the problems associated with sizing web applications and evaluated IFPUG FPA, Mk II FPA and COSMIC FFP by applying them to a sample e-commerce application. In [24], Gencel et al. presented the results obtained by applying Mk II FPA and COSMIC FFP to a real-time software system. In this study, the similarities and the differences between the measurement processes of these methods and the difficulties faced during the measurement are discussed. Besides the work on clarifying the conceptual basis of FSM methods, significant work has been carried out on their theoretical basis as well. Lokan studied the correlations between the BFC types in FPA [50]. He analyzed ISBSG dataset [32] to gain further insight into the correlations. In [48] Kitchenham and Kansala and in [43], Jeffery and Stathis studied the correlations between the BFC types of IFPUG FPA. Although some of their findings agreed, they found out different correlations in others. The outcomes of these studies showed that the presence of these correlations cause counting the same things more than once in FPA. Moreover, Kitchenham stated in [49] that the different results of studies on correlations showed that any predictive model based on the sum of the elements would not be stable for different datasets. In another important line of research on the theoretical basis of the FSM methods, Fenton called for a rigor in software engineering through measurement theory [19], [20]. A number of authors discussed the fundamental flaws in the construction of FSM methods with respect to measurement theory, especially issues related to scale types [1], [2], [49]. Representation condition of Measurement Theory requires that every measure should be associated with a model of how the measure maps the entities and attributes in the real world to the elements of a numerical system [19]. The mappings from the real world domain to the metric models of FSM are usually not well defined and there is a lack of sound empirical relations. Another issue is related to the additivity of the functional sizes of different BFC Types or the constituent parts of BFCs. The measurement function of all ISO-certified FSM methods involve adding together the functional sizes of different BFC types to obtain a total functional size. This is possible if the assignment of weights to the various types of functions has transformed these different BFC types into a single type. Abran stated this problem for IFPUG FPA in the following way: The additivity of functions poses a question, namely the relevance of adding elements which are of different types and mean different things [1]. Thus, he suggested that it would be more appropriate to call the final result an index rather than a measurement of the size of an application and that

7 the FP count could be used as measurements of size that enables reflect various points of view with different units. All these dimensions in one or several subsets could be used to define and measure a functional structure of software. While discussing indirect measures, Fenton suggested using vectors of measures with rules for combining the vector elements into a larger, indirect measure [19]. Kitchenham also mentioned this problem and suggested not adding or combining the resulting counts together, and instead using basic counts that are not weighted as a vector of measures that describe the system [49]. The results of the literature survey provide insight that there are still some improvement opportunities for FSM. Based on the empirical studies we performed, we evaluated these issues and identified the required improvements in practice in the next section. 3.Empirical Studies on FSM We conducted two independent case studies to understand FSM methods from different perspectives. We focused on usability of FSM methods in different functional domains and in early phases of the software development. Accordingly, we studied the effects of the FSM process on measuring the functional size as well as the relationship of functional size with both length of code (SLOC) and the development effort. Yin defined two types of case study design strategy as single-case and multiple-case designs [78]. The objectives of doing empirical study determine which one to apply. Single-case design strategy involves only one case, whereas multiple-case design strategy involves more than one case. The evidence from multiple-case studies is often considered more compelling and the overall study is regarded as being more robust [78]. In the first case study, our objective was to explore the applicability of FSM methods to measure the size of the projects in different functional domain types, to examine the differences between these methods and, by evaluating the methods, to bring into light the improvement opportunities related to FSM methods. Therefore, we designed this case study as a multiple-case study which involves three different cases. We followed the replication approach to multiple-case studies defined in [78] and selected the cases in such a way that the applications, the functional sizes of which were to be measured, were of different functional domain types. We used single-case design strategy for the second case study since we conducted it as a prelude and an exploratory device for further study. We explored the applicability of a number of functional size estimation and measurement

8 methods for estimating the functional size of an application at different phases of the software development life cycle. The details of a part of this case study are discussed in [17] Case Study 1 Description of the Cases. Project-1 is a development project of one of the subsystems of an avionics managements system for small to medium size commercial aircrafts on a Flight Display System. It consists of three subsystems. It is developed according to RTCA/DO-178B standard [65] and will be certified by Federal Aviation Administration. The software complies with DO-257A, Minimum Operational Performance Standards for the Depiction of Navigation Information on Electronic Maps as a basis and additional user requirements are integrated. The project was started in November 2003 and completed in September Project-1 was developed by a software company which is certified as CMMI/SW Maturity Level 3. The project staff consisted of 13 persons: 1 project manager, 1 senior software engineer (development team leader), 6 software engineers (development team), 1 senior software test engineer (test team leader), 2 junior software test engineers (test team), 1 software quality engineer and 1 software configuration management specialist. Project-2 is a Collision Avoidance Subsystem (CAS) of the Traffic Alert and Collision Avoidance System (TCAS). CAS functionality is specified in detail in RTCA/DO-185A [66]. In this study, we measured the size of CAS-Own Aircraft Algorithm. Own Aircraft function determines the TCAS operational mode, effective sensitivity level and other operation parameters used by the collision avoidance logic. The project was started in September 2004 and completed in July Project-2 was developed by the same software company as Project-1. The project staff consisted of 9 persons: 1 project manager, 1 senior software engineer (development team leader), 3 software engineers (development team), 1 software test engineer (test team leader), 1 junior software test engineer (test team), 1 software quality engineer and 1 software configuration management specialist. Project-3 involves development of a military inventory management system integrated with a document management system. The software development organization is an independent supplier, which is a CMMI/SW Maturity Level 3 company. The organization focuses mostly on web-based projects and has its own framework to develop web applications rapidly. The project was started in October 2004 and completed in August persons worked for the project, namely 1 project manager, 1 senior software engineer, 2 software engineers (development team full-time), 2 software engineers (development team - part-time), 1 software engineer (test team part-time).

9 We used the CHAR Method defined in [37] to determine Functional Domains of the case projects (see Section 4.3). The functional domains of Project-1, Project-2 and Project-3 are Complex Data Driven Control System, Complex Control System and Information System respectively. The efforts utilized for the development life cycle processes of all projects are given in Table 2. For Project-1 and Project-2, the efforts were collected on a daily basis in 15 minutes intervals for each work breakdown structure task. Case Study Conduct and Data Collection. We implemented Mk II FPA and COSMIC FFP methods in order to measure the functional size of all three projects in this case study. Among other FSM methods, we selected Mk II FPA and COSMIC FFP methods due to the fact that they are designed to be applicable to both data-rich and control and communication-rich systems [37]. Their being international ISO standards and having detailed measurement manuals for reliable measurement results were also other factors. In addition, in ISBSG dataset [32], there exist measurement data on projects, the sizes of which are measured by these methods. This would help us to compare the results of our case studies with other projects data. Table 2 Development efforts for Project-1, Project-2 and Project-3 Software Development Life Cycle Phase Project-1 Effort (person-hours) Project-2 Effort (person-hours) Project-3 Effort (person-hours) Development 18, , , Software Requirements Analysis 2, Software Design (Architectural-Detailed) 3, Software Coding & Unit Testing 5, , , Testing (continue for Project-1 & Project-2) 5, , Management 2, , Training 1, Supporting 2, , Total 24, , , For size measurement, we used the SRS documents of the projects. The number of FUR of Project-1, Project-2 and Project-3 are 835, 158 and 127 respectively. Two people performed the size measurement of all projects. One of them is an author of this paper and the other persons involved in the measurement processes work for the development organization of the projects and were involved in the development of projects. Although all staff are experienced in using FSM methods, they are not certified by UKSMA and COSMIC.

10 The functional sizes of Project-1, Project-2 and Project-3 were measured as 5, Mk II FP, 1, Mk II FP and 1, Mk II FP, respectively (see Table 3 - the detailed results can be found in [25]). The results are given in subsystem level for Project-1. Table 3 Size measurement details of the projects by Mk II FPA Project No Subsystem Name Number of FUR Number of Logical Transactions Number of Input DETs Number of Output DETs Number of Data Entity Types Referenced Functional Size (Mk II FP) A ,344 2,037 4, B C Total ,660 2,404 5, , , , By applying COSMIC FFP, the functional sizes of Project-1, Project-2 and Project-3 were measured as 4,036 Cfsu, 945 Cfsu and 1,020 Cfsu, respectively (see Table 4- the detailed results can be found in [25]). The efforts utilized to make measurement by both methods are given in Table 5. Table 4 Size measurement details of the projects by COSMIC FFP Project No 1 Subsystem Name Number of FUR Number of Functional Processes Number of Entries Number of Exits Number of Reads Number of Writes Functional Size (Cfsu) A , , B C Total , , , Table 5 Efforts utilized for functional size measurement in Case Study 1 Project No Effort Utilized By Mk II FPA (person-hours) Effort Utilized by COSMIC FFP (person-hours) The SLOC values of the projects in Case Study 1 are given in Table 6. We obtained the SLOC values for all three subsystems of Project-1 and Project-2 by using Understand for C++ which is a source code analyzer [76]. Both logical and physical un-commented SLOC values are measured. Since the user interface and the database components of Project-

11 3 are developed by using the Internal Development Framework (IDF) which generated XML files, we could not obtain meaningful SLOC values for these components. The interface and database components are generated in parallel with implementing IDF. For the processing part, Java was used as the primary programming language. The SLOC values for the processing part are obtained by using Borland Together Architect [12]. The logical SLOC value for the process component is measured as 11,817 SLOC. Table 6 Length of code (SLOC) of the projects in Case Study 1 Project No Subsystem Name SLOC (Physical, Un-commented) SLOC (Logical, Un-commented) A 20,196 12,143 1 B 6,115 3,449 C 6,698 3,914 Total 33,009 19, The results of this case study are discussed while identifying improvement opportunities in the next section Case Study 2 Description of the Case. The case was a project which targeted the requirements elicitation for a model Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance - C4ISR sub-system for the Turkish Land Forces Command. The project outcomes formed the major parts of the Request for Proposal (RFP) currently issued by the Turkish Armed Forces. In this project we applied the requirements elicitation approach that emphasizes business process modeling to elicitate requirements as defined in [16]. The project started in October 2002 and was completed in thirteen months. The project staff consisted of 11 part-time personnel. The total effort spent during the project was 24 person-months. Case Study Conduct and Data Collection. In this case study, the functional size of one of the subsystems of the case project - Subsystem A1 - is estimated by different methods at different phases of the development life cycle. Mk II FPA [41], IFPUG FPA [40] and COSMIC FFP [39] methods, which are approved by ISO as being international standard FSM methods, were implemented after the detailed system-level functional requirements were identified. This stage is earlier than the phase when these methods are designed to be used to measure the functional size reliably. The results of the size estimation are given in Table 7.

12 Table 7 Size estimation of Subsystem A1 after the system-level requirements identification a) Mk II FPA Number of Input Data Number of Number of Data Entity Functional Size Element Types (DETs) Output DETs Types Referenced (Mk II FP) , b) COSMIC FFP Number of Entries Number of Exits Number of Reads Number of Writes Functional Size (Cfsu) ,873 c) IFPUG FPA Transactional Functions Data Functions Functional Size Number of Number of Number of Number of Internal Number of External (IFPUG FP) External Inputs External Outputs External Inquiries Logical Files Interface Files ,305 Early Function Point Analysis (EFPA) [14][56][57], which is designed as a quick and early estimation technique based on IFPUG FPA, was implemented at five consecutive stages during the requirements analysis phase, starting after the feasibility study until the system level requirements were generated. The results of the size estimation are given in Table 8. Table 8 Size estimation of Subsystem A1 at five consecutive stages by EFPA Unadjusted EFPs Stage No Minimum Average Maximum Stage ,222 Stage ,048 1,318 Stage 2 1,204 1,461 1,796 Stage 3 1,454 1,793 2,155 Stage 4 1,707 2,089 2,554 Size estimation by Mk II FPA was performed by the requirements analysis team of the case project, which involved 4 software analysts. One of the software analysts who is also an author of this paper, performed IFPUG FPA, COSMIC FFP and EFPA estimations herself. All the staff was experienced in using the methods. The efforts utilized to make estimation by Mk II FPA, IFPUG FPA, COSMIC FFP and EFPA were 35 person-hours, 24 person-hours, 15 person-hours and 24

13 person-hours, respectively. The timing of Mk II FPA, COSMIC FFP and IFPUG FPA estimation of Subsystem A corresponds to Stage 4 of EFPA. The results of this case study are discussed while identifying improvement opportunities in Section Improvement Opportunities In the light of the results of the literature review and our observations resulting from the case studies, we classified the improvement opportunities for FSM methods into six categories: - Significance of the FSM Process on Measuring the Functional Size Effects of the Abstraction Level of Elementary Components of FUR Effects of the Granularity Level of FSM Process - Different Scales of FSM Methods and Convertibility - Measuring the Functional Size of Software Systems from Different Functional Domains - Functional Size Estimation Early in the Life Cycle - Size Conversion for Later Phases (Functional Size Length of Code) - The Relationship between Functional Size and Effort Each of these is discussed in the following sub-sections Significance of the FSM Process on Measuring the Functional Size All ISO certified methods have defined roughly the same level of abstraction to measure functional size of elementary components of FUR [21][24]. The Transaction 7, which is Logical Transaction in Mk II FPA, Elementary Process in IFPUG FPA and Functional Process in COSMIC FFP, correspond to each other (see Section 2). What differs among the measurement processes of FSM methods are the BFC Types used, the detailed views of BFC Types, how the base counts are obtained and the measurement functions used [24]. Therefore, after fixing the level of abstraction by defining the elementary unit of FUR, the FSM methods measure the functional size of each BFC at different levels of granularity. 7 Transaction: a sequence of input, processing and output - not necessarily in that order - triggered by events outside the software and is complete when the software is left in a self-consistent state with respect to the external event [37]

14 In the following sub-sections, we discuss two significant effects of the FSM process on measuring the functional size: a) effects of the abstraction level of elementary components of FUR b) effects of the granularity level of FSM process Effects of the Abstraction Level of Elementary Components of FUR In all FSM methods, FURs are mapped to Transactions. During the measurement process of FSM methods, the functionality of each Transaction is measured and then summed up to compute the overall functionality. Some FSM methods measure the functionality of Data Groups in addition to Transactions as well. Mapping FURs to Transactions enables FSM methods to make measurement at a fixed level of abstraction from the users viewpoint. The abstraction level of the BFCs affects not only the effort required to map FURs to Transactions, but the amount of functional size as well. In Case Study 1, one of the difficulties of the measurement process of Project-1 and Project-2 was identifying the Transactions (LTs in Mk II FPA and Functional Processes in COSMIC FFP). Generally, a FUR consists of one or more Transactions. In order to identify them, FURs are decomposed into their elementary components. However, for Project-1 and Project-2, we needed to gather and group two or more FURs in order to form one Transaction. In Project-1 and Project-2, the total number of FURs is 835 and 158 whereas the total number of LTs is 521 and 99, respectively. This is due to the fact that the levels of FURs in the SRS documents of the projects were very detailed since they are prepared according to a specific standard [65]. The FURs are defined at the functional transactions level. In addition, the related requirements which would form one Transaction were organized such that each was specified in different parts of the document. As a result, for Project-1, one quarter of the total effort on measurement was utilized for identification of Transactions. In the SRS document of Project-2, FURs are organized such that the functionality is specified with respect to a specific standard [48]. The algorithmic operations (functions and macros) are specified in separate sections and the FURs give reference to the macros and functions they make use of. In addition, each macro and function gives reference to others as well. Therefore, we had to trace these paths in order to identify each Transaction. As a result, in Project-2, about half of the effort on measurement was utilized for identification of Transactions. These difficulties are related to the effort to be utilized for identifying Transactions during FSM process. The abstraction level of software requirements documents should be aligned with the requirements of FSM methods in order to make reliable and less time consuming measurement.

15 Another point that should be considered is the effect of the abstraction levels of Transactions on the amount of functional size. An important point when identifying Transactions is that each shall be unique and distinct. A minor difference between two Transactions, which are nearly identical in terms of processing logic, causes these two to be regarded as two distinct Transactions. Therefore, at a lower level of abstraction, same functionalities are being measured more than once if required by the user in different BFCs. We faced this situation in Project-1 and Projects-2, which are real-time systems. Let s examine the following example; Transaction - 1: IF ( Map Option selected AND State 1 AND (( ActivePage is X AND X_Place is A ) AND Data ABC valid) THEN (State 3 AND Output_1 (1 attribute)) Transaction - 2: IF ( Map Option selected AND State 2 AND (( ActivePage is X AND X_Place is A ) AND Data ABC valid) THEN (State 3 AND Output_1 (1 attribute)) Since one of the references to Data Entity Types is different, these two Transactions are regarded as distinct and measured separately. However, at a lower level of abstraction, the processing logic used in the input components and the output components are the same. Only one of the Data Entity Types referenced within the process component is different, i.e. State 1 versus State 2. This might also be one of the causes of the variation in the functional size to SLOC ratio discussed in Section 4.5. This is because the Transactions are not designed and coded one by one, but rather designed and coded taking the similar kinds of functionalities into account. Moreover, this might also have impact on the functional size and effort relationship as well (see Section 4.6). Therefore, new extensions to the existing FSM methods, which consider the functional similarity, are required Effects of the Granularity Level of FSM Process During FSM process, the BFCs defined by each FSM method are identified after identifying Transactions within FURs. The functional size of each BFC is measured by categorizing them into BFC Types and applying the measurement function to calculate the Functional Size. Then, the results are summed up to compute the overall size of the software system.

16 In [33], Functional Size is defined as a size of the software derived by quantifying the FUR. Each FSM method makes this quantification differently. The definitions of BFCs, BFCs Types and how the base counts are obtained are different. These differences determine the level of granularity, i.e. the precision, of the FSM process of each method. Mk II FPA method derives functional size by counting the input DETs, output DETs and entity references in each BFC, whereas IFPUG FPA assigns a complexity weight to each BFC. This weight depends on the predetermined interval values of DETs. Therefore, the exact number of DETs is not required in IFPUG FPA. Determining the interval in which the number of DETs falls is sufficient. By IFPUG FPA, the functional size of two BFCs might be measured as being the same although the number of DETs is different. The size of maintenance in a project in which no new Transactions are being added, changed or deleted but small changes in the number of DETs are made, may be measured as 0 by IFPUG FPA. Let s examine this situation in an example Transaction: The system shall enable adding new customers (name, surname, customer id, customer account number, address). If two more attributes ( address, fax number) are added to the existing attributes during maintenance, the size of this change can not be measured by IFPUG FPA. Thus, the granularity level of MkII FPA measurement is lower than IFPUG FPA. COSMIC FFP does not take into account the number of DETs manipulated by each sub-process. It is stated that although the movement of a single data attribute could be used as a sub-unit of measure, measurements on a small sample of software in the field trials of this method indicated that the average number of DETs per data movement did not vary much across the four types of data movement [39]. Therefore, the COSMIC FFP unit of measurement, 1 Cfsu, has been fixed at the level of one data movement. The granularity levels of these methods are concluded as follows: GL IFPUG FPA > GL COSMIC FFP > GL Mk II FPA The granularity level of a method determines at what detail level the FUR shall be defined in order to make reliable measurement. The lower the granularity level of the method, the more detailed the FUR shall be in order to obtain the base counts. This is directly related to the phase of the software development life cycle when the method can be applied reliably. Since the FUR becomes more detailed in the later stages of the development life cycle, the methods which are of low granularity can be applied later than the ones which are at a higher level of granularity. Thus, one of the conclusions

17 of this study is that IFPUG FPA can be applied earlier than COSMIC FFP and that COSMIC FFP can be applied earlier than Mk II FPA during the life cycle. In addition, granularity level largely determines the required effort for measurement. As more detailed information is needed, the effort required to get that information from FUR for measurement increases. In Case Study 1, the efforts utilized to make the measurement by Mk II FPA are given in Table 5. We utilized greater effort for Mk II FPA measurement for all the projects of Case Study 1. In Case Study 2, the efforts utilized to make estimation by Mk II FPA, IFPUG FPA, EFPA, COSMIC FFP were 35 person-hours, 24 person-hours, 24 person-hours and 15 person-hours, respectively. We utilized the greatest effort for Mk II FPA measurement since finding the number of DETs takes longer. Since IFPUG FPA uses ranges for DETs and RETs while giving weights, the effort needed to count the exact number of DETs and RETs decreased. Measurement by COSMIC FFP took the shortest time. We should also note that these differences are not only due to level of granularity but also due to the order of measurement. We performed measurement by Mk II FPA, IFPUG FPA, and COSMIC FFP, consecutively. Therefore, we obtained required information during Mk II FPA measurement for finding the data groups and DETs to be counted by IFPUG FPA and COSMIC FFP. So, the efforts spent should not be directly compared to determine the counting productivity. The granularity level, i.e. the precision, of a FSM method affects the timing of measurement during the software development life cycle. Making measurement by the methods which have lower granularity requires more information and can therefore be applied later during the life cycle than the ones which are at a higher level of granularity. Therefore, existing and new methods shall report the level granularity of their FSM process so that the users of the methods will be aware of the precision of the measurement results while making effort and cost estimates from those figures Different Scales of FSM Methods and Convertibility Although FSM methods give roughly similar sizes on average, they have not been designed to give similar sizes. In [29], it is stated that there is some controversy in the FSM community regarding the functional size, and it is explained that a piece of software has several functional sizes when measured with different methods. In order to be able to compare functional sizes of software products measured by different methods, conversion formula are needed. Convertibility is defined as the ability to convert the results from applying two or more FSM Methods in the measurement of a Functional Size of the same set of FUR [35].

18 In Case Study 2, we compared the functional sizes estimated by EFPA and IFPUG FPA since these two methods use the same metrics. We found the relative percentage errors as % at Stage 4 and % at Stage 0. Convertibility between Mk II FPA-IFPUG FPA. A formula to convert Mk II FPA and IFPUG FPA sizes to each other is defined by mapping the BFC types of IFPUG FPA to Mk II FPA. [69]. In order to compare IFPUG FPA and Mk II FPA estimates, a conversion of Mk II FPA size estimate to IFPUG FPA size estimate is performed by utilizing the formula defined in [69]. For the projects, sizes of which are above 1500 IFPUG FP s or 2,500 Mk II FP, the ratio of the functional sizes is given by the following formula: MarkII _ FP No of Entity References = 0.16 IFPUG _ FP No of Entity Types (1) For this case, the average number of references of each entity type was found as 9. Thus, by using the above formula, the ratio of functional sizes obtained by Mk II FPA to IFPUG FPA is calculated as Accordingly, the size of Subsystem A1 is found as 2, IFPUG FP. By applying IFPUG FPA, we estimated the size of Subsystem A as 2, This shows that there is an error of about - 13 %. Although this formula is very valuable, this difference shows that other factors shall be considered as well as the average number of references. Convertibility between IFPUG FPA-COSMIC FFP. In [6], a survey of previous convertibility studies is presented and convertibility analyses from IFPUG FPA to COSMIC FFP size are discussed. However, since the formula varies among the organizations, a unique conversion formula could not be obtained from the datasets used. In [72], it is stated that an average conversion formula to convert COSMIC FFP size to an IFPUG FPA size would be grossly misleading. Two main factors might give rise to divergences between IFPUG and COSMIC FFP sizes. The first one is that if the software being measured has a high proportion of files which are not much referenced by the processes, measurements made by IFPUG scale tend to result in higher sizes than those made by the COSMIC FFP scale. The second factor arises from allocating size to each BFC whether within limited ranges or with no upper limit. In IFPUG FPA, an External Input can have a size in the range of 3 to 6 FP. In COSMIC FFP, there is no upper limit to the size of a functional

19 process. If the number of sub-processes in a Functional Process is high, the functional size obtained by COSMIC FFP would be much higher. In Case Study 2, the functional size of Subsystem A obtained by COSMIC FFP is % smaller than the one obtained by IFPUG FPA. This shows that the numbers of sub-processes are not high for this Subsystem. Convertibility between Mk II FPA-COSMIC FFP. In the literature, no conversion formula is defined between Mk II FPA and COSMIC FFP sizes. Therefore, in Case Study 1 and Case Study 2, we could not compare the Mk II FPA and COSMIC FFP sizes since each of these methods use different metrics. Instead, we compared the results of both methods in Case Study 1 according to the base counts in order to depict what kind of factors might give rise to obtaining different functional sizes. In Mk II FPA, the size of the processing component of a LT is defined to be proportional to the number of referenced Data Entity Types. A Data Entity Reference in Mk II FPA is generally equivalent to a data group Read or Write in COSMIC FFP. Therefore, the sizes of the processing component are roughly equivalent on both scales [39]. The first distinction between these two methods is the assumptions of the methods when measuring the size of the processing component. Mk II FPA assumes that each LT must have at least 1 input DET, must make 1 reference to a Data Entity Type and must have 1 output DET as a minimum. On the other hand, COSMIC FFP principles say that A Functional Process comprises at least two data movements, an entry plus either an exit or a write [39]. Therefore, in Mk II FPA, for a specific LT, we should count at least 1 entity reference for the processing component in case a Data Entity Type is not accessed. In COSMIC FFP, we do not count anything related to the processing components if there is no Read or Write to a data group. By Mk II FPA, the numbers of references to Data Entity Types were found as 2,404, 592, and 343 in Project-1, Project- 2, and Project-3, respectively. By COSMIC FFP, the total numbers of data groups that are read or written were found as 2,620 in Project-1, 688 in Project-2 and 488 in Project-3. In all three cases, the base counts related to the processing components are higher in COSMIC FFP than in Mk II FPA. This means that there exist Data Entity Types which are both read and written in a Transaction. These are counted only once in Mk II FPA, whereas they are counted separately as Entries and Exits in COSMIC FFP. By Mk II FPA, the functional sizes of the processing component of Project-1, Project-2, and Project-3 are 3,990.64, , Mk II FP, respectively. By COSMIC FFP, the functional size of the processing component of Project-1 is

20 2,620 Cfsu, Project-2 is 688 Cfsu and Project-3 is 488 Cfsu. Although we expect higher functional size by COSMIC FFP, the weight factor used in Mk II FPA changes the result. That is, when calculating the functional size of the processing component, we multiply the number of references by 1.66 in Mk II FPA, whereas there is no weight factor in COSMIC FFP. The second distinction between Mk II FPA and COSMIC FFP is the difference in the granularity level of each method s FSM process (see Section 4.1.2). COSMIC FFP method estimates functional size at a higher level of granularity than Mk II FPA. The COSMIC FFP unit of measurement, 1 Cfsu, has been fixed at the level of one data movement. On the other hand, in Mk II FPA method, the size of the input and output components of a LT is defined to be proportional to the number of DETs in the input and output components. For Project-1, the number of input DETs is 824, the number of output DETs is 2,660 in Mk II FPA measurement and the functional size of input and output components are found to be Mk II FP and Mk II FP, respectively. By implementing COSMIC FFP, the number of Entries and the number of Exits are found to be 615 and 801, respectively. The functional size of Entries is 615 Cfsu and Exits is 801 Cfsu. For Project-2, by implementing Mk II FPA, the number of input DETs is found to be 283 and the number of output DETs as 126. The functional size of input and output components are Mk II FP and Mk II FP, respectively. In COSMIC FFP measurement, the number of Entries is 206 and the number of Exits is 51. The functional sizes of Entries and Exits are 206 Cfsu and 51 Cfsu, respectively. For Project-3, the number of input DETs is 560, the number of output DETs is 1,707 in Mk II FPA measurement and the functional size of input and output components are Mk II FP and Mk II FP, respectively. By COSMIC FFP, the number of Entries is 154 and the number of Exits is 378. The functional sizes of Entries and Exits are calculated as 154 Cfsu and 378 Cfsu, respectively. The third distinction between Mk II FPA and COSMIC FFP is related to the relationship between the functional sizes of different BFC Types and constituent parts of BFC Types. Mk II FPA makes use of weight factors which are calibrated to industry-average relative effort to develop each component. This enables these three kinds of functionality to be combined into a single value for a functional size. On the other hand, COSMIC FFP assumes that the average number of DETs per data movement does not vary much across the four BFC Types, i.e. Entry, Exit, Read, Write. Thus, the contribution of each to functional size is assumed to be the same.

21 In Case Study 1, the distinctions of MkII FPA and COSMIC FFP discussed above, resulted in higher size values by Mk II FPA than values by COSMIC FFP, with a difference of 22 % in Project-1, 20 % in Project-2, and 24 % in Project-3. In Case Study 2, the functional size obtained by Mk II FPA is % greater than the one by COSMIC FFP. Therefore, the practitioners should consider this variance when making effort estimation from functional size figures of different FSM methods. The designers of COSMIC FFP stated that an average conversion formula would result in a project being under-sized or over-sized [72]. We agree with this statement by adding that an average conversion formula of Mk II FPA measurement to COSMIC FFP is possible, but the reverse is not true. If the system is estimated by Mk II FPA and the result is to be converted to COSMIC FFP, we have detailed information on the number of data groups and data movement types. However, when we want to convert COSMIC FFP size to Mk II FPA, we do not have information on the number of DETs, i.e. we do not know if the system has a high or low number of DETs to decide on a formula. Another point stated in [72] is that the lack of enough data to develop statistically-based conversion formulae and having no definite conceptual mapping between the BFCs of one method and of the other make it difficult to develop an exact mathematically-based conversion formula. Our concern about the reason why conversion of one size measure to another is difficult is due to measuring the same attribute in different scales. If the empirical relations about the functional size attribute were defined, the different mappings of these relations to mathematical relations would not cause any problem since the measures would be using the same scale types. Moreover, the problems related to scale types used in FSM methods [49] make conversion very difficult or impossible, because the admissible transformation can not be defined based on measures which do not obey the principles of measurement theory. Fetcke proposed a model as a generalized representation of IFPUG FPA, Mk II FPA and COSMIC FFP focusing on their commonalities and differences [21]. One of the conclusions of this study is that it is not possible to truly convert functional sizes obtained by different methods and that their model defines elements that are sufficient to obtain measurement with any of the FSM methods. If the experience data are stored in this form, the functional size can be obtained for any of the methods. Thus, both empirical and theoretical work on the convertibility issue is demanded. The scales and metrics which FSM methods use shall be explored. The differences between the FSM methods and the relationships between the BFC Types of a method as well as among different methods shall be explored.