Genetic Algorithm for Optimizing Neural Network based Software Cost Estimation

Size: px

Start display at page:

Download "Genetic Algorithm for Optimizing Neural Network based Software Cost Estimation"

Hilary Greene
5 years ago
Views:

1 Genetic Algorithm for Optimizing Neural Network based Software Cost Estimation Tirimula Rao Benala 1, S Dehuri 2, S.C.Satapathy 1 and Ch Sudha Raghavi 1, 1 Anil Neerukonda Institute of Technology and Sciences Sangivalasa, Visakhapatnam, Andhra Pradesh, India b.tirimula@gmail.com, sureshsatapathy@ieee.org, sudharaghavich@yahoo.com 2 Department of Information & Communication Technology Fakir Mohan University, Vyasa Vihar, Balasore , India satchi.lapa@gmail.com Abstract. Software engineering cost models and estimation techniques are used for number of purposes. These include budgeting, tradeoff and risk analysis, project planning and control, software improvement and investment analysis. The proposed work uses neural network based estimation, which is essentially a machine learning approach, is one of the most popular techniques. In this paper the author has proposed a 2 step process for software effort prediction. In first phase known as training phase neural network selects the matching class (datasets) for the given input, which is improved by optimizing the parameters of each individual dataset by Genetic algorithm. In second step known as testing phase, the prediction process is done by adaptive neural networks. The proposed method uses COCOMO-II as base model. The experimental results show that our method could significantly improve prediction accuracy of conventional Artificial Neural Networks (ANN) and has potential to become an effective method for software cost estimation. Keywords: Software cost estimation, Genetic algorithm, ANN, BP-Learning, and COCOMO-II. 1 Introduction Software cost estimation is critical for the success of software project management. It affects management activities including resource allocation, project bidding and project planning. The importance of accurate estimation has led to extensive research efforts to software cost estimation methods. From a comprehensive review, these methods could be classified into following six categories: parametric models including COCOMO, SLIM and SEER-SEM, expert judgment including Delphi technique and work break down structure based methods, learning oriented techniques including machine learning methods and analogy based estimation; regression based methods including ordinary least square regression and robust regression; dynamics based model, composite methods. [5] In this paper, we are concerned with cost estimation models that are based on artificial neural networks.the artificial neural network (ANN), inspired by biological nervous system, are nonlinear information processing models, which are built from interconnected elementary processing elements called neurons. In general for software cost estimation modeling, the most commonly adopted architecture, learning algorithm and the activation functions are respectively feed forward, multi layer

2 perceptron and the Back propagation algorithm and the Sigmoid function. Many researchers have applied the neural networks approach to estimate software development effort. However some difficulties are still confronted by Neural Network techniques such as neural networks are better suited for classification and categorization problems whereas software cost estimation is more of generalization rather than classification problem [1, 4]; non-normal characteristics of software engineering data set. The large and non-normal data sets always lead ANN methods to low prediction accuracy and high computational complexity. To alleviate these drawbacks our proposed idea has been devoted to simultaneously optimize selected class of projects and their feature selection by genetic algorithm (GA). In this paper we have incorporated Nasser Tadayon [9] s adaptive learning based machine using neural networks to estimate the software cost using neural network. GA is used to optimize the selected class of projects and their feature weights. Neural network based techniques and cost estimation fundamentals are briefly reviewed in section 2. The proposed GA approach for optimizing the selected class of projects is described in section 3.In section 4, numerical examples from COCOMO dataset is used to illustrate the performance. A conclusion and overview of future work conclude this paper. 2 Background 2.1 The COCOMO The Constructive Cost Model, COCOMO, was introduced by Boehm [3].It has become one of the most widely used software cost estimation models in the industry. To support new life cycles and capability, it has evolved into a more comprehensive estimation model, called COCOMO II [2]. COCOMO II consists of three sub models, each one offering increased fidelity. Listed in increased fidelity, these sub models are called Application Composition, Early design and Post Architecture models. Until recently, only the last and most detailed sub model, Post Architecture had been implemented in a calibrated software tool. As such, unless otherwise explicitly specified, all further references in this study to COCOMO II can be assumed to be the Post Architecture Model [7]. The cost effort equation was calibrated and developed by Barry Boehm using the COCOMO II [2] post architecture model for software cost estimation. It computes the efforts (in Person-Months) required for a project based on project s size in KSLOC (Kilo Source Lines Of Code) as well as 22 cost factors (5 Scale factors and 17 Effort Multipliers) by: S F i i 1 P M A. S I Z E. E M where A is a multiplicative constant, and the set of SF (Scale Factor), and EM (Effort Multiplier). There are 17 Effort Multipliers capturing the characteristics of the software development that affect the effort to complete the project. These weighted multipliers are grouped into four categories (Product, platform, personnel, and project), and their product is used to adjust the effort. The nominal weight assigned to 1 7 i 1 i (1)

3 each multiplier is 1.0. If a rating level has detrimental effect on effort, its corresponding multiplier is greater than 1.0. Conversely if the rating level reduces the effort then the corresponding multiplier is less than 1.0.There are five exponent scale factors (Precedenteness, development flexibility, architecture/risk resolution, team cohesion, and process maturity). They account for the relative economies or diseconomies of scale encountered as a software project increases its size based on based on different nominal values and rating schemes [7]. 2.2 Neural Network Based Cost Estimation. The neural network architecture for software cost estimation is given as bias 1 log ( EM 1 ) log( EM 2 ) log( EM 17 ) Bias SF 1 2 P 1 size q j log( size ) 17 Bias P log( EM ) i 1 i i1 5 2 [ q j j 1 Bias log( size )] SF j S T PM SF 2 SF 5 Figure 1: Network Architecture We compute the effort (PM) using the mathematical approach given by Tadayon [11]. 3 Framework Genetic algorithm (GA) is a stochastic global optimization technique initially introduced by Holland in 1970 s [6]. By mimicking biological selection and reproduction, GA can efficiently search through the solution space of complex problems and it is naturally parallel and provides opportunity to escape from local optimum. GA has become one of the most popular algorithms for optimization problems. In this section, we construct the OCFWANN system (stands for Optimal projects of predicted Class and Feature Weighting and Artificial Neural Network

4 based Estimation) which can perform simultaneous optimization of N projects of the predicted class and their feature weights. GA is selected as optimization tool for OCFWANN system. The detailed description is presented in section 3.2. In order to introduce the fitness function in GA algorithm, performance metrics for estimation accuracy are firstly presented in the section Performance Evaluation Metrics To measure the accuracies of the proposed methods, three performance metrics are considered: Mean Magnitude of Relative Error (MMRE), Median Magnitude of relative error (MdMRE),and PRED (0.25), because these measures are widely accepted in literature [6]. The MMRE is defined as: MMRE = MRE = ( ) MRE (2) (3) Where n denotes the total number of projects, E i denotes the actual cost of ith project, and E denotes the estimated cost of the ith project. Small MMRE value indicates the low level of estimation error. However this metric is unbalanced and penalizes overestimation more than underestimation. The MdMRE is the median of all the MREs. MdMRE = Median (MRE) (4) It exhibits similar pattern to MMRE but it is more likely to select the true model especially in the underestimation case since it is less sensitive to extreme outlier [6]. The PRED (0.25) is the percentage of prediction that fall within 25 percent of actual cost PRED(q) = (5) Where n denotes the total number of projects and k denotes the number of projects whose MRE is less than or equal to q. Normally, q is set to be The PRED(0.25) identifies cost estimations that are generally accurate, while MMRE is biased and not always reliable as a performance metric. However, MMRE has been de facto standard in the software cost estimation literature. Thus MMRE is selected for the fitness function in GA. More specifically, for each combination of N project parameters and cost driver weights, MMRE is computed across the validation dataset. Then GA searches through the project parameter space to minimize MMRE. 3.2 GA for Optimization The procedure for OPFWANN system via Genetic Algorithm is presented in this section. The system consists of two stages: the first one is training stage and the

5 second one is testing stage. In the training stage 93 NASA data points are presented to the system, the ANN is configured with cost drivers to produce the cost prediction for the given input data point. The class label of the input data is determined basing on PM obtained. GA explores the class space to minimize the error (in terms of MMRE) of ANN on the training projects by the following steps: Encoding To apply GA for searching, the cost drivers are encoded as binary string chromosome. Each individual chromosome consists of number of binary digits. There are six features for each driver (very low- vl, low-l, nominal-n, high-h, very high-vh, extra high-xh). Here we encode cost driver weights with 3 bits (0-n,1-vl,2-l,3-n,4-h,5- vh,6-xh,7-n). 0 and 7 are assumed default values nominal(n). Since the cost driver weights are decimal values, before entering into ANN model these binary codes are transformed into decimal numbers Population Generation and fitness function After encoding the individual chromosome, the system then generates a population of chromosomes. Each chromosome is evaluated by the fitness function in GA. Since the GA is designed to maximize the fitness value and the smaller MMRE value indicates more accurate prediction, we set the fitness function as the reciprocal of MMRE. f = (6) Given one training project as input, ANN predicts the PM for the project, basing on the person month, the class is identified for the project, which contains set of similar projects as input. To evaluate the prediction performance of the ANN model, the error metric MMRE, PRED (0.25), and MdMRE applied on the training project set in the class. Then, the reciprocal of MMRE is used as the fitness value for each cost driver combination (or chromosome) Rules for selection, extinction and multiplication The standard roulette wheel is used to select chromosomes from the current population. The selected chromosome were consecutively paired with a probability of 0.8 was used to produce new chromosome in each pair. The newly created chromosome constituted a new population. The population is evolved by GA algorithm using evolutionary rules described above. The individual with best fitness value is in the population in every cycle.

6 3.2.3 Completion of Evaluation The population is evolved by the GA algorithm in the first stage until the number of generations is equal to or excess 2000 or the best fitness value did not change in the last 200 generations. The second stage is the testing stage. In this stage the system receives the optimized parameters from the training stage to configure the ANN model. The optimal ANN is then applied to the testing project to evaluate the performance of the trained ANN. 4 Experimentations and Results The COCOMO NASA 2 Dataset containing 93 data points have been taken for our experiment [8]. The data is in COCOMO-I format calibrated to COCOMO-II using Rosetta stone [2]. COCOMO measures efforts in the calendar months of 152 hours (and includes development and management hours). COCOMO assumes that the effort grows more than linearly on software size. There are total of 17 effort multipliers in COCOMO II. These cost factors are expressed in 6 forms i.e vl,l,n, h, vh,hx. For our experimental results we have included 5 scaling factors and assumed their values as nominal. Along with these there are two more attributes, namely, KSLOC (Kilo Source Lines of Code) and actual effort. There are 11 classes distributed across 93 data points. For the purpose of validation, we adopt three-fold cross validation [6] to evaluate accuracy of the methods. In this scheme the NASA dataset is randomly divided into three nearly equal sized subsets. At each time one of three subsets is used as the test sets which is exclusively used to evaluate the estimation performance, and other two subsets treated as Validation data set and training data set exclusively used to optimize the cost drivers. This process is repeated three times. Then the average training error and testing error across all three trials are computed. The advantage of this scheme is it matters less how the data is split since each data point is assigned into a test set, a training set and a validation set respectively once. In the experiment we apply three types of ANN based models. The first model is conventional ANN [11], the second model is FWANN (GA optimizing feature weights(cost drivers) for Neural Network based cost estimation) which does not optimize the projects data points in the corresponding class. The third model is OCFWANN which uses GA simultaneously optimize the class and the feature weights (cost drivers).for comparison, other popular estimation models including Analogy Based Estimation(ABE), Step wise regression(swr)[9],artificial Neural Networks(ANN)[11],classification and regression trees (CART) [10], are also included in the experiments.

7 The experimental results are summarized in table 2. It shows that OCFWANN achieves the best level of prediction performance (0.32 for MMRE, 0.24 for MdMRE, and 0.31 for PRED (0.25)). Methods MMRE MdMRE PRED(0.25) Training Testing Training Testing Training Testing OCFWANN OFWANN ANN SWR CART Table 1 Results and Comparisons on NASA Dataset 5 Conclusion and Future work On appraising the above novel technique the hybrid system of GA and ANN provides better prediction accuracy compared to ANN. GA is used as a tool for simultaneously optimizing the concerned class to which the input project belongs and cost drivers. The experimental results show that our method gives pacifying optimal performance as compared to conventional ANN and outperform the comparative techniques such as OFWANN, SWR and CART. Motivation is therefore exploring the scope of application of soft computing in the field of Software Cost Estimation.. We have done our research in the direction of software cost estimation by hybrid system using GA as it has not been explored extensively till date. There are numerous cost estimation techniques that have been proposed in different real-world applications. We extend connotations to our work with Artificial Bee Colony (ABC), Differential Evolution (DE), Artificial Immune System (AIS), Bacterial foraging optimization algorithm, Neuro Fuzzy, Neuro Genetic, Simulated Annealing and fuzzy logic. 6 References 1. AttarZadeh, I., Siew Hock Ow., Proposing a New Software Cost Estimation Model Based On Artificial Neural Networks, In : ICCET,IEEE 2 nd Conference on Computer Engineering and Technology, ISBN /10 (2010) 2. Boehm, B., Abts, C., Brown, A., Chulani, S., Clark, B., Horowitz, E., Madach, R., Reifer, D., Steece, B., Software Cost Estimation with COCOMO II, Prentice Hall, Upper Saddle River, NJ, Boehm, B., Software Engineering Economics, Prentice Hall, 1981.

8 4. Idri, A., Khoshgoftaar, T.M., Abran, A., Can Neural Networks be easily interpreted in Software Cost Estimation? In:FUZZ-IEEE 02, In Proceeding of IEEE International Conference on Fuzzy Systems, ISBN /02 (2002) 5. Li, Y.F., Xie, M., Goh, T.N., A study Of Project Selection Feature Weighting For Analogy Based Software Cost Estimation, In The Journal Of Systems and Software 82 (2009) Li, Y.F., Xie, M., Goh, T.N., Optimization Of Feature Weights and Number Of Neighbors For Analogy Based Cost Estimation in Software Project Management, In Proceedings Of the 2008 IEEE IEEM, ISBN: /08 (2008). 7. Musilek, P., Pedrycz, W., Sun, N., On the Sensitivity of COCOMO II Software Cost Estimation Model, In: METRICS 02, The Proceedings Of 8 th IEEE Symposium On Software Metrics, ISBN /02 (2002). 8. Menzies, T., The PROMISE Repository Of Software Engineering Databases, School Of Information Technology and Engineering, University Of Ottawa, Canada, Available from 9. Shepperd, M., Kadoda, G , Comparing Software Prediction Techniques using Simulation, IEEE Transaction On Software Engineering, 27(11), Stensrud, E , Alternative Approaches to Software Prediction of ERP Projects, Information and Software Technology, 43(7), Tadayon, N., Neural Network approach for Software Cost Estimation, In: ITCC 05, In Proceeding of IEEE International Conference on Information Technology: Coding and Computing, ISBN /05 (2005).

Proposing a New Software Cost Estimation Model Based on Artificial Neural Networks

Proposing a New Software Cost Estimation Model Based on Artificial Neural Networks Iman Attarzadeh, Siew Hock Ow Department of Software Engineering Faculty of Computer Science & Information Technology