Role of Technical Complexity Factors in Test Effort Estimation Using Use Case Points

Transcription

1 Role of Technical ity s in Test Effort Estimation Using Use Case Points Dr. Pradeep Kumar Bhatia pkbhatia.gju@gmail.com Ganesh Kumar gkyaduvansi@gmail.com Abstarct-The increasing popularity of use-case driven development methodologies has generated an industrial interest in software size and effort estimation based on Use Case Points (UCP). The caveat here is that the V-model must be in use and use case generation must start becoming available right at the requirements gathering phase. The acceptance test plan is then prepared with the use cases from the requirement documents as input. This paper provides a test effort estimation model using use case point method. It is extension of model provided by [1] on the basis of Technical ity s that employs a project s complexity and required man-hours. 1. Introduction Testing is an important activity that ensures the quality of the software. Estimation plays a crucial role in project planning and plays a vital role in all the phases of SDLC that includes very uncertain elements. To estimate the time make a product from scratch, and in many cases, without prior experience of the technology is no mean feat. However, conventional estimation techniques address only the development effort that goes into it. It is known that a use case to test case mapping is possible. This means that the UCP figure for development can be indirectly used to provide a figure for the number of test cases. Using organizational test execution time metrics it is now possible to arrive at a figure for the total test effort. This is a viable and systematic approach towards test effort estimation and it makes a leap in providing more realistic figures. This means that the cost of testing can now be factored into projects. Method of effort estimation Actually Test effort estimation at the time of making proposals is very difficult because at that time requirements are not so much clear. But in a real scenario some time we are not writing the code from scratch and products used for creating 5 the application/websites gives us a freedom such that we can easily customize the available functionalities in the products as per our needs and requirements. Definitely it saves a lot of time in development. So in this way we are saving lot of development efforts during the execution of a project but when it comes to testing, irrespective of this fact, the testing effort would not reduce since testing needs to be done end to end making sure that application does not break at any level in any workflow. There is great benefit in estimating the number of test cases. Understanding the number of test cases leads to the logical analysis of comparing actual test cases with expected test cases. If actual test cases fall short of expected test cases, then there is inadequate test coverage. Testing coverage is a direct indication of potential defects and future maintenance costs. The larger the gap between expected test cases and actual test cases, the greater the potential of defects not being detected during testing. Estimation modes for software development estimation are not appropriate to estimate the effort for Testing. Because their estimations are based on software size and development complexity rather test case size and test execution complexity. Mapping Use Cases to Test Cases Use Case Modeling is an accepted and widespread technique to capture the business requirements of software application. Since Use Case provide the functional scope it is easier to have insight in effort estimation. Since use cases are available early in the software development life cycle and each scenario or flow in the Use Case can be an input for a test case. So is it right to map use case to test case and use 'Use Case Points' for Test Effort Estimation instead of FP. The V model given below [figure 1] shows that software development activity is a corresponding testing activity. The effort estimation using Use Case Point (UCP) of software development activity [1] is same for effort estimation of Testing activity with a slight change in the environmental factors.

2 Figure 1. Software Development Activity Producing V Model Estimation of testing effort Effort estimation is basically done for different test activities like Test Plan, Test Design, Test Case Preparation, Test Execution and Test Reporting. In order to estimate the effort required to execute test cases, we first evaluate their execution complexity. For example, creating a message in mobile phone is more complex than reading it. Regarding test cases written, we can evaluate their execution complexity based on their specification. We evaluate the execution complexity of a test case by performing the following activities for each test step defined in the specification: Evaluate the complexity level for executing the action described by the test step based on the functional and non-functional system characteristics exercised by it. Rate the execution complexity level in execution points, a quantitative measure of execution complexity defined by this work. 6 Examples of Functional characteristics are the number of navigations between screens, average number of pressed keys and the number of items to verify during the test. Regarding the Non-functional characteristics, we have characteristics as system performance and the use of network. For each characteristic, we evaluate it as low, average or high complexity. Each complexity value has assigned a specific number of weight. Guidelines were defined through expert judgment and historical data for evaluating and rating the complexity levels. Finally, the execution complexity of a test case is measured by the sum of the execution points of each of its test steps. After evaluating the execution complexity of each test case, we estimate the effort in man-hours required to execute them. In a stable environment, we can do these estimations multiplying the total number of execution points by the historical test productivity. Here, we consider test productivity as the average proportion between execution time and number of execution points of each test case (time

3 spent per each execution point).but the environmental factor plays a role in the estimation. So the environmental factors are analyzed for each testing environment and add that to the test effort. 2. Effort Estimation using Use Case Points The Use Case Points (UCP) method provides the ability to estimate the man-hours a software project requires from its use cases. Based on work by [2], the UCP method analyzes the use case actors, scenarios, and various technical and environmental factors and abstracts them into an equation. Readers familiar with Allan Albrecht s Function Point Analysis [] will recognize its influence on UCP; function point analysis inspired UCP. The UCP equation is composed of three variables:- Unadjusted Use Case Points (UUCP). The Technical ity (TCF). The Environment ity (ECF). Each variable is defined and computed separately using weighted values, subjective values, and constraining constants. The weighted values and constraining constants were initially based on Albrecht, but subsequently modified by people at Objective Systems, LLC, based on their experience with Objectory a methodology created by Ivar Jacobson for developing object-oriented applications [4]. The subjective values are determined by the development team based on their perception of the project s technical complexity and efficiency. Additionally, when productivity is included as a coefficient that expresses time, the equation can be used to estimate the number of man-hours needed to complete a project. Here is the complete equation with a Productivity (PF) included: Effort Estimation = UUCP * TCP * ECF * PF The necessary steps to generate the estimate based on the UCP method are the following:-- i. Determine and compute the UUCPs in terms of use case point. ii. Determine and compute the TCFs in terms of use case point. iii. Determine and compute the ECFs in terms of use case point. iv. Determine the PF in terms of man-hours per use case point. v. Compute the effort estimation in terms of manhours. 7 i. Unadjusted Use Case Points Unadjusted Use Case Points (UUCP) are computed based on two computations:- a.the Unadjusted Use Case (UUCW) based on the total number of activities (or steps) contained in all the use case scenarios. The UUCW is calculated by tallying the number of use cases in each category, multiplying each total by its specified weighting factor, and then adding the products, i.e. UUCW = W i * N i b. The Unadjusted Actor (UAW) based on the combined complexity of all the actors in all the use cases. The UAW is calculated by totaling the number of actors in each category, multiplying each total by its specified weighting factor, and then adding the products. i.e. UAW = Here is the complete equation is UUCP = W i * N i + W i * A i i(a). Unadjusted Use Case W i * A i Unadjusted Use Case (UUCW) is derived from the number of use cases in three categories: simple, average, and complex (see Table 1). Each use case is categorized by the number of steps its scenario contains, including alternative flows. Table 1: Use Case Categories Use Case Category Simple Simple user interface. Touches only a single database entity. Its success scenario has 0 steps or less. Its implementation involves less than 05 classes. 5 N 1 No. of Use Case ( N i )

4 Average More interface design. Touches 02 or more database entities. Between 04 and 07 steps. Its implementation involves between 05 and 10 classes. user interface or processing. Touches 0 or more database entities. More than 07 steps. Its implementation involves more than 10 classes. 10 N 2 15 N Keep in mind the number of steps in a scenario affects the estimate. A large number of steps in a use case scenario will bias the UUCW toward complexity and increase the UCPs. A small number of steps will bias the UUCW toward simplicity and decrease the UCPs. Sometimes, a large number of steps can be reduced without affecting the business process. i(b). Unadjusted Actor In a similar manner, the Actor Types are classified as simple, average, or complex as shown in Table 2. Table 2: Actor Classifications Actor Type Simple Average The actor represents another system with a defined application programming interface. The actor represents another system interacting through a protocol, like Transmission Control Protocol/Internet Protocol. Or Human interactions via a Command Line. The actor is a person interacting via a graphical user Interface. ii. Technical ity s 1 A 1 2 A 2 A No. of Actors ( A i ) Seventeen standard technical factors exist to estimate the impact on productivity that various technical 8 issues have on a project shown in Table. Each factor is weighted according to its relative impact. Table : Technical ity s Technical T1 Distributed System 2 C 1 T2 Performance 1 C 2 T End User Efficiency/ Efficiency of interface T4 Internal Processing / interfacing T5 Reusability / Test ware reuse / Reusable code T6 Easy to Install / Installability T7 Easy to Use / Operability 1 C 1 C 4 1 C C C 7 T8 Portability 2 C 8 T9 Easy to Change / 1 C 9 Maintainability T10 Concurrency 1 C 10 T11 Special Security Features 1 C 11 T12 Provides Direct 1 C 12 Access for Third Parties T1 Special User 1 C 1 Training Facilities Are Required T14 Test Tools 2 C 14 T15 T16 Documented inputs Development Environment 2 C 15 1 C 16 T17 Test Environment 1 C 17 perceived complexity ( C i ) For each project, the technical factors are evaluated by the development team and assigned a perceived complexity value between zero and five. The perceived complexity factor is subjectively determined by the development team s perception of the project s complexity concurrent applications, for example, require more skill and time than singlethreaded applications. A perceived complexity of 0

5 means the technical factor is irrelevant for this project, is average, and 5 is strong influence. When in doubt, use. Each factor s weight is multiplied by its perceived complexity factor to produce the calculated factor. The calculated factors are summed to produce the Technical Total, i.e. 17 Technical Total = W i * C i Two constants are computed with the Technical Total to produce the TCF. The constants constrain the effect the TCF has on the UCP equation from a range of 0.60 (perceived complexities all zero) to a maximum of 1.0 (perceived complexities all five). TCF values less than one reduce the UCP because any positive value multiplied by a positive fraction decreases in magnitude: 100 * 0.60 = 60 (a reduction of 40 percent). TCF values greater than one increase the UCP because any positive value multiplied by a positive mixed number increases in magnitude: 100 * 1.0 = 10 (an increase of 0 percent). Since the constants constrain the TCF from a range of 0.60 to 1.0, the TCF can impact the UCP equation from - 40 percent (.60) to a maximum of +0 percent(1.0). For the mathematically astute, the complete formula to compute the TCF is: 17 TCF = ( 0.01 * W i * C i ) iii. Environmental ity s The ECF provides a concession for the development team s experience, shown in Table 4. More experienced teams will have a greater impact on the UCP computation than less experienced teams. Table 4: Environmental ity s Environmental E1 E2 E Familiarity With UML* Part-Time Workers Analyst Capability W i 1.5 I 1-1 I I Perceived Impact ( I i ) 9 E4 Application Experience 0.5 I 4 E5 Object- 1 I 5 Oriented Experience E6 Motivation 1 I 6 E7 E8 Difficult Programming Language Stable Requirements -1 I 7 2 I 8 The development team determines each factor s perceived impact based on its perception the factor has on the project s success. A value of 1 means the factor has a strong, negative impact for the project; is average; and 5 means it has a strong, positive impact. A value of zero has no impact on the project s success. For example, team members with little or no motivation for the project will have a strong negative impact (1) on the project s success while team members with strong object-oriented experience will have a strong, positive impact (5) on the project s success. Each factor s weight is multiplied by its perceived impact to produce its calculated factor. The calculated factors are summed to produce the Environmental Total. 8 Environmental Total = W i * I i Larger values for the Environment Total will have a greater impact on the UCP equation. To produce the final ECF, two constants are computed with the Environmental Total. The constants, based on interviews with experienced Objectory users at Objective Systems [1], constrain the impact the ECF has on the UCP equation from (Part-Time Workers and Difficult Programming Language = 0, all other values = 5) to 1.4 (perceived impact all 0). Therefore, the ECF can reduce the UCP by 57.5 percent and increase the UCP by 40 percent. The ECF has a greater potential impact on the UCP count than the TCF. The formal equation is: 8 ECF = ( -0.0 * W i * I i ) iv. Productivity The Productivity (PF) is the ratio of development man-hours needed per use case point.

6 Statistics from past projects provide the data to estimate the initial PF. If no historical data has been collected, a figure between 15 and 0 is suggested by industry experts. A typical value is 20. v. Effort Estimation in Terms of Man-Hours. At this stage, we have all the values of required variables. Using these values, the total effort estimated numbers of hours for the project is determined by the following formula. Effort Estimation = UUCP * TCP * ECF * PF If no historical data has been collected, consider two possibilities: I. Establish a baseline by computing the UCP for previously completed projects (as was done with the empirical study in this paper). II. Use a value between 15 and 0 depending on the development team s overall experience and past accomplishments (Do they normally finish on time? Under budget? etc.). If it is a brand-new team, use a value of 20 for the first project. After the project completes, divide the number of actual hours it took to complete the project by the number of UCPs. The product becomes the new PF.. Empirical Study In this section we analysis the test effort estimation on web application developed by [1]. (i). Unadjusted Use Case Points calculation. i(a). Unadjusted Use Case Use Case Category Simple Average Simple user interface. Touches only a single database entity. Its success scenario has 0 steps or less. Its implementation involves less than 05 classes. More interface design. Touches 02 or more database entities. Between 04 and 07 steps. Its Implementation involves between Number of Use Cases ( N i ) Res ult W i * N i and 10 classes. user interface or processing. Touches 0 or more database entities. More than 07 steps. Its implementation involves more than 10 classes. Resource data is used from [1]. i(b). Unadjusted Actor Actor Type Simple Average The actor represents another system with a defined application programming interface. The actor represents another system interacting through a protocol, like Transmission Control Protocol/Internet Protocol. Or Human interactions via a Command Line. The actor is a person interacting via a graphical user Interface. Resource data is used from [1]. Now UUCP = UUCW + UAW UUCP = W i * N i + W i * A i UUCP = = Total UUCW 140 No. of Actors ( A i ) Total UAW 1 ii. Technical ity s Computation Technical T1 Distributed System ( W i) perceived complexity ( C i ) T2 Performance 1 T End User Efficiency/ Efficiency of interface T4 Internal Processing / interfacing T5 Reusability / Test ware reuse / Reusable code Calculated (W i * C i) Result W i * A i

7 T6 Easy to Install / Installability T7 Easy to Use / Operability T8 Portability T9 Easy to Change / Maintainability 1 T10 Concurrency T11 Special Security Features T12 Provides Direct 1 Access for Third Parties T1 Special User Training Facilities Are Required T14 Test Tools T15 T16 T17 Documented inputs Development Environment Test Environment Resource data is used from [1]. Now Total Technical 21.5 TCF =0.6 + (0.01 * Total Technical ) 17 TCF =0.6 + (0.01 * W i * C i ) TCF= (0.01 * 21.5) = iii. Environmental ity s Computation Environmental Perceived Impact ( I i ) Calculated (W i * I i) Familiarity With E1 UML* E2 Part-Time Workers E Analyst Capability E4 Application E5 Experience Object-Oriented Experience E6 Motivation E7 Difficult Programming Language E8 Stable Requirements Resource data is used from [1]. Total Environmental 27 iv. Effort Estimation in Terms of Man-Hours. Effort Estimate = (UUCP * TCP * ECF * PF) in term of UCP = 15 * * 0.59 * = UCP Project ity needs 15% of the estimated effort to be added. 10% is spent in co-ordination and management activity. Total Effort Estimate = = i.e. 189 man-hours = i.e. 20 man-days 4. Industry Case Studies Four studies have been identified on the use of variants of UCP. No other detailed study has been identified documenting the use of UCP as is for estimation purposes. On the basis of a single case study, [5] reports that the UCP method can be more reliable than FPA to predict testing effort. The approach described in [5] is a variant of Karner s [2] proposing nine technical qualities (instead of thirteen) associated with testing activities (for example, test tools and test environment) and ignores the development resource factors (EF). For the single project studied, the effort estimated by the UCP method was reported to be only 6% lower than the actual effort. Nageswaran s extensions to the UCP equation produced an estimate of 67 man-days, a deviation of 6 percent of the actual effort of 90 man-days. The adapted UCP method described in [6] suggests that iterative projects need special rules to account for the ongoing re-estimation of changing requirements during the course of a project. Realizing that an organization s resources are more or less constant, the adapted method replaces the resource factor (EF) by a simple increase in the productivity constant (MR). In this variant of the UCP method, another dimension is introduced: the overhead factor (OF), which represents the effort of project management and another activities not directly related to functionality. For the two projects presented in [6], the efforts estimated by UCP were 21% and 17% lower than the actual effort. Again, this study refers to only two projects and lacks generalization power. The method used in [7] is, in essence, the same as Karner s, but with the addition of a new risk coefficient, specific to each project, to account for risk factors not already covered by the basic model

8 (for instance, reuse effort. The risk coefficient is a simple percentage of effort added according to the opinion of an estimation expert. The method described in the [7] study was reportedly used with over 200 projects over a 5-year period, and produced an average deviation of less than 9% between estimated effort (using the UCP method) and recorded effort; no information is provided about the dispersion across this average, nor about the statistical technique used to build the estimation model or the classical quality criteria of estimation models such as the coefficient of determination (R2) and Mean Relative Error. In brief, no documented details are given to support the claim about the accuracy of the estimates. On the basis of a case stud y in 2006[1], published the results of a UCP estimation effort for a product support web application which extends the UCP equation to include thirteen technical factors to derive a more accurate estimate. The main goal of this study was to analyze the accuracy of the test effort estimation when using the estimation model proposed in this paper. In this paper, we have used resource data from [1]. We extend UCP equation to include seventeen technical factors instead of thirteen which was used in [1], to derive a more accurate estimate. We put forth a comparative study for the test effort estimation, which is as follows:- Variables NAG01 K. In this clemmons06 Paper TCF 9 s 1 s 17 s UCP PF Estimated 67 mandayday 274 man- 20 mandays effort In summary, the UCP measurement method itself was modified in the three studies surveyed, and care must be exercised when comparing data from these three method variants, since no information about convertibility back to the original UCP is provided. Similarly, the empirical information given in these three studies cannot be generalized to a larger context due to either the very small sample (one or two case studies) or lack of supporting evidence. actual effort, and often, closer to the actual effort than experts and other estimation methodologies [7]. Moreover, in many traditional estimation methods, influential technical and environmental factors are not given enough consideration. The UCP method quantifies these subjective factors into equation variables that can be tweaked over time to produce more precise estimates. Finally, the UCP method is versatile and extensible to a variety of development and testing projects. It is easy to learn and quick to apply. The author encourages more projects to use the UCP method to help produce software on time and under budget. 6. References [1.] K. Clemmons Roy, Project Estimation with Use Case Points in February [2.] Karner, Gustav. Resource Estimation for Objectory Projects. Objective Systems SF AB, 199. [.] Albrecht, A.J. Measuring Application Development Productivity. Proc. Of IBM Applications Development Symposium, Monterey, CA, Oct. 1979: 8. [4.] Jacobson, I., G. Booch, and J. Rumbaugh. The Objectory Development Process. Addison-Wesley, [5.] Nageswaran, Suresh, Test Effort Estimation Using Use Case Points, Quality Week 2001, San Francisco, CA, USA, June [6.] Mohagheghi, Parastoo, Effort Estimation of Use Cases for Incremental Large-Scale Software Development, IEEE International Conference on Software Engineering ICSE 05, May 15-21, 2005, St.Louis, MI, USA, pp [7.] Carroll, Ed, Software Estimating Based on Use Case Points, The Cursor, Software Association of Oregon, Website: [8.] Carroll, Edward R. Estimating Software Based on Use Case Points Object- Oriented, Programming, Systems, Languages, and Applications (OOPSLA) Conference, San Diego, CA, [9.] Anda, Bente. Improving Estimation Practices By Applying Use Case Models. June 200. [10.] Jon Smith, The estimation of Effort based on Use Cases, Rational Software. 5. Conclusion An early project estimate helps managers, developers, and testers plan for the resources a project requires. As the case studies indicate, the UCP method can produce an early estimate within 20 percent of the 12