Defect-oriented Approach to Software Development and Suite of Defect Metrics

Defect-oriented Approach to Software Development and Suite of Defect Metrics 31 Defect-oriented Approach to Software Development and Suite of Defect Metrics Nasib S. Gill 1 & Sunil Sikka 2 1 Head, Department of Computer Science & Applications, M.D. University, Rohtak (Haryana) 2 Centre Head, ZAD Institute of IT & Management, Rohtak (Haryana) ABSTRACT Objective of any ideal software development methodology is to produce high quality software for its clients aiming at zero defects. This can t be achieved until intermediate deliverables conform to the same uniform standard. Several measures are used on account of this in every phase of software development. The foremost aim of these measures is to prevent defects, discover defects and further eliminating these defects. Late detection of defects and its remedies not only incur a heavy cost but also delay the delivery of software product. The present paper is aimed at proposing defect oriented perspective of development phase along with necessary notations and finally a suit of defect metrics resulting into the cost effective target product is proposed. Keywords: Software Quality, Defect Prevention, Defect Detection, Defect. 1. INTRODUCTION Defects may be defined as variance from desired attributes. These attributes include complete and correct requirement specifications, design that meets requirements, programs that observe requirements and business rules [1]. Dealing with defects is not only essential but also wastes valuable resources including time if not dealt meticulously [2]. Defects can be classified in several ways, however, most of the researchers use standard orthogonal defect classification (ODC). Eight types of defects defined with in ODC include - Interface, Function, Build/Package/Merge, Assignment, Documentation, Checking, Algorithm and Timing/ Serialization defects [3]. Defect is an unwanted term during and post development. One of the quality metric of final software delivered to client is defect density which computes total number of defects delivered to client per thousand line of source code (KLOC). Defect density can be used to predict the remaining defects when compared to the expected defect density, determine if the amount of testing is sufficient and establish a database of standard defect densities [4]. Another important IEEE defect metric is man-hours per major defect which can be used to measure effort for defect discovery during design, code inspection and testing [4]. Defective software not only increases the development and maintenance cost but also decreases customer satisfaction. Failure cost is very high in safety critical software. The cost of defective software can be * Correspondence: Email: nsgill_2k4@yahoo.com, sunil_sikka@rediffmail.com as high as 50% of the investments in the software development. Yet, the potential to improve software quality and reduce project cost is enormous [5]. Quality of the software can be amplified by delivering zero defects with software to its clients. To achieve zero defects in software, primary objective is to prevent defects from being injected. Unfortunately, complete defect prevention is not possible. Software complexity and development time constraints make it difficult to prevent and avoid defects. Secondary objective is early detection and correction of defects if occurs. If any defect occurs some cost is associated with its detection and correction. With the increase of time span of defect, cost of defect detection and correction also increases. If defect is delivered to a client then detecting and correcting the defect after delivery is more expensive as compare to detecting and correcting it prior to its delivery [6]. Therefore, detection and correction of defect near to where it is injected is essential. Otherwise it can defeat the fundamental principle of software engineering Developing high quality software at low cost. Several tools and techniques are used during software development to minimize defects delivered to its clients. Defect delivered to client can be minimized only if maximum defects are prevented, detected and removed at each phase of software development. Many activities such as inspections, walkthroughs and reviews are performed in this context. These activities must be implemented in an effective manner to help developer to improve quality, to identify defect prone module and finally to reduce the cost of the software. Software metrics

32 Nasib S. Gill & Sunil Sikka play very important role to assess effectiveness and cost of defect related activities. This paper presents a defect oriented approach of software development. This approach mainly includes four activities-defect prevention, defect origination, defect detection and defect correction. These activities are discussed in Section 2. Necessary notations are proposed in Section 3. These notations denote the essential information required for evaluating defects and related activities. A suite of defect metrics for quantitative assessment of defect life, defect proneness, undetected defect, defect handling cost, defect correction side effect, defect prevention effectiveness, defect detection effectiveness, defect correction effectiveness and defect correction backlog is proposed in Section 4. 2. DEFECT ORIENTED VIEW OF SOFTWARE DEVELOPMENT LIFE CYCLE PHASE Final product delivered to client can t have high quality until intermediate product doesn t have high quality at each phase of development. In order to achieve desired quality at each level various activities other than development activities are performed in each phase. Software development can be considered as a set of ordered sequence of phases and each phase further includes a set of activities [7]. It is quite necessary to identify the various activities that create a phase. From defect viewpoint all the activities required in any software development phase can be classified into four categories: 1. Defect Prevention Activities (DPA) 2. Defect Origination Activities (DOA) 3. Defect Detection Activities (DDA) 4. Defect Activities (DCA) In general, each phase of software development consists of above four activities. General view of any software development phase is shown in Figure 1. DCA may insert new defects on correction of a defect, therefore, DDA should also be performed to detect the defects occurred due to DCA. Most researcher use the word injection instead of origination, but we have used the word origination because sometime injection word mislead that defect is introduced intentionally. Defect Prevention Activities (DPA) Undoubtedly, defect is undesirable term in software. It can jeopardize the success of software, therefore, it should not occur at all. DPA uses the experiences of similar software to restrict a defect to occur. Defect prevention increases the productivity and quality while decreases the cost of software. Nobody likes the defect to occur. Defect prevention identifies the root cause of defects and prevents them to re-occur [3]. Defect prevention should be performed at a fine granularity level, such as method level instead, practitioners should collect measurement and defect data at a macro level [1, 8]. Defect prevention is also one of the key process areas at level 5 in SEI-CMM. As complete defect prevention is not possible, therefore, various activities are required to detect and remove the defects. Various defect prevention models are available, which are used depending upon development environment. In general, defect prevention process consists of following four key activities [9]. Casual analysis meetings to identify root causes of defects and suggest prevention actions. Action team to implement prevention actions. Stage kickoff meetings to provide feedback to developers at each stage of development cycle. Data collection and tracking to prevent suggestions from being lost over time, to aid action implementation and to enhance communication among group. Defect Origination Activities (DOA) Each phase of software development consist of set of activities. Any activity which introduces defect can be treated as defect origination activity. For example, development activities such as requirements analysis, requirements specifications, high level design, low level design, coding, testing, implementation and maintenance all are defect origination activities. Misunderstood requirement, ambiguous SRS, invalid number of Undetected Defects of Phase (p i ) to Phase (p i+1 ) Figure 1: Software Development Phase View

Defect-oriented Approach to Software Development and Suite of Defect Metrics 33 parameters, incorrect data structure, bad program structure, miss test case, invalid file changeover and side effected change are defects injected in requirement analysis, requirements specifications, high level design, low level design, coding, testing, implementation and maintenance respectively. Two main reasons of higher defect origination rate are poor design and complexity of software. The moment defect is injected in software its life starts and its life ends when it is removed. Life of defects is correlated negatively with software quality and positively with cost of software. Therefore various defect detection and correction activities are used with each phase to shorten the life of defect. Defect Detection Activities (DDA) Defect detection activities identifies whether intermediate product or final product is defect free or not. Structural and functional testing is used to uncover the defects in testing phase. But testing phase is not enough activity to detect defects because it is applied late in development process and very costly [10]. Many other quality assurance activities such as Inspection, Review and Walkthrough are used to detect the defects near to the origination point. DDA contribute lot in the cost of software development. Undetected defects are delivered to the client with final software. The failure cost of defected software may be very high. By using the effective DDA quality of the software can be enhanced. Economically DDA are investments to optimize the cost of final software[10]. Tools, techniques, time invested and skills can have significant impact on the effectiveness of the defect detection activities. Effectiveness is the evaluation criteria of various defect detection techniques [11]. Defect Activities (DCA) During DDA software remains static; the software is modified in DCA to remove the defect detected by DDA. Debugging is the key activity in defect correction. DCA are performed until all the defects detected by DDA have been removed. DCA may introduce new defects during modification of defected code/document. Therefore DCA must be verified by DDA. For example modification of a module may be followed by regression testing to uncover new defects introduced. 3. PHASE AND DEFECT NOTATION A software development phase(p) can be denoted as P <p, N prevent, N enter, N originate, N detect > Where p = unique phase number N prevent = total number of defects prevented in the phase N enter N originate N detect = total number of defects enters from (p-1)th phase = total number of new defects occurred in the phase = total number of defects detected in the phase Aim of any phase s defect related activities is to maximize N prevent, N detect and minimize N enter, N originate. A defect is represented by its identity, origination time, correction time and amplification factor. An undetected defect of a phase when enters into next phase may create many other defects. Defect amplification factor of a defect is no. of defects created by the defect when it enters into the next phase. A defect can be denoted as D<d, p i, p c, > Where d = unique identity of the defect p i = defect origination time i.e. phase number in which defect is occurred into software document/code. p c = defect correction time i.e. phase number in which defect is corrected from defected software document/code = amplification factor of defect First priority of defect related activities is that p i =0 i.e. defect prevention. If p i 0 then p c p = 0 is the next i priority. For example D<d1, 1 3, 5> is a defect whose identity is d1, occurred in I st phase and corrected in 3 rd phase of software development. Amplification factor is 5 which mean the defect is converted into 5 defects when it enters into 2 nd phase. 4. DEFECT METRICS SUITE Nine metrics are proposed to assess defects and its related activities. These metrics are used to give quantitative measures to different characteristics of defect and its related activities. Information gained from these metrics can help quantify relative improvement [4]. Table 1 lists the metrics, their level and concerned activities. 4.1 Defect Life (DL) Life of a defect is the time span during which defect exists in the software. It is the phase interval between defect origination and correction. DL of a defect D<d, p i, p c, > can be computed as DL p c p If DL=0 then it indicates that defect is corrected in the same phase in which it is originated. Any defect i

34 Nasib S. Gill & Sunil Sikka Table 1 List of Proposed Metrics S.No. Metric Acronym Level Concerned Activities 4.1 Defect Life DL Defect Origination, 4.2 Defect Proneness DP Phase Detection 4.3 Undetected Defect UD Phase Origination, Detection, 4.4 Defect Handling Cost DHC Defect Detection, 4.5 Defect DCSE Defect Side Effect 4.6 Defect Prevention DPE Phase Prevention, Effectiveness Detection 4.7 Defect Detection DDE Phase Detection Effectiveness 4.8 Defect DCE Phase Detection, Effectiveness 4.9 Defect DCB Phase Detection, BackLog should not move to the next phase if it occurs. Positive (>0) value of DL indicates defect is removed in later phase. Late correction may be due to the late detection. Therefore it is indication of ineffectiveness of all the defect detection activities performed prior to the activity which detects the defect. Obviously higher value of DL is not desirable. Higher the value of DL more is the cost associated with it. 4.2 Defect Proneness (DP) The phase which introduces maximum number of defects in the software is most defect prone phase. If T is the no. of defects detected from all phases of software development and Dp is the total no. of defects detected having origination time equal to p then defect proneness(dp) of pth phase can be computed as DP However this metric is based only on detected defects but still it is good indicator of defect prone area of software. Higher value of DP indicates that more defect prone phase. Phase having highest value of DP among all the phases is most defect prone phase. Higher value of DP may be due to complexity of task involved in the phase or poor development activities. DP is not indicative of effectiveness of defect detection techniques of phase because it is not necessary that defect is detected in the same phase in which it is originated. Dp T 4.3 Undetected Defect (UD) This metric computes the total number of defects remains undetected at the end of a phase. UD of pth phase can be computed as Where N enter N originate N detect UD ( N N ) N p enter inject det ect = Total number of defects enters in the pth phase from (p-1)th phase = Total number of new defects occurred in pth phase = Total number of defects detected in pth phase N enter can be computed as N enter UDp1 i1 ip1 ip-1 is amplification factor of ith defect of (p-1)th phase. UD p-1 is total number of undetected defects of (p-1)th phase. Therefore, UDp1 UD p N ip-1 inject N det ect i 1 UD of last phase gives the total number of defects delivered to the client. The term N enter and N originate is unknown it can be replaced by approximated value obtained from previously developed similar projects. Higher value of UD means less effective defect detection activities. UD can be minimized by minimizing N originate and maximizing N detect. 4.4 Defect Handling Cost (DHC) The only cost of defect handling is time which includes review time, rework time (modification of document/ code) testing time and debugging time etc. DHC of a defect can be computed as DHC=cost of detection + cost of correction nd nc ' ' i i i i i1 i1 DHC c t c t Where nd and nc is total no. of detection and correction activities performed respectively. c i and c i is the cost per unit time of ith defect detection and correction activity respectively. t i and t i is the time required for ith detection and correction activity respectively. Higher value of DHC of a defect is indicative of higher DL. Higher time consumption also indicates that defect detection and correction tools and techniques are not efficient. DHC can be reduced by using effective automated tools, techniques and by reducing defect life.

Defect-oriented Approach to Software Development and Suite of Defect Metrics 35 4.5 Defect Side Effect (DCSE) When a defect is corrected then it may introduce new defects. DCSE of a defect is defined as DCSE = total no of new defects originated on correction of defect Higher value of DCSE may be indicative of bad design or high coupling. For lesser value of DCSE coupling in software should be minimum. Higher value of DCSE may also be indicative of ineffectiveness of defect correction techniques. 4.6 Defect Prevention Effectiveness (DPE) DPE of a phase can be defined as percentage of defect prevented in the phase. It is computed as follows N prevent DPE N N N det ect undet ect prevent 100 Where N prevent, N detect and N undetect are total no. of defects prevented, detected and undetected in the phase respectively. This metric can also be applied at system level. Higher value of DPE is always desirable. It can be achieved by maximizing N prevent. Effective defect prevention activities can dramatically improve productivity and product quality. Higher defect prevention also reduces the cost by lowering the cost of review and testing. Defect prevention activities motivate the developer to record complete and accurate defect data. 4.7 Defect Detection Effectiveness (DDE) DDE measures the effectiveness of defect detection tools and techniques used. DDE of a phase is defined as percentage of defect detected in the phase. It can be calculated as follows DDE N N N det ect det ect undet ect 100 Where N detect and N is total no. of defects detected undetect and undetected in the phase respectively. Higher value of DDE is always desirable. Higher value of DDE indicates defect detection tools and techniques used are working effectively. It can be achieved by maximizing N detect. 4.8 Defect Effectiveness (DCE) DCE of a phase measures the effectiveness of defect correction activities used in the phase. It can be found by calculating percentage of defect corrected. DCE of a phase can be calculated as N DCE DCSE 100 N det ect correct 100 % DCE indicates that all the detected defects of the phase are corrected in the same phase without any side effect. Negative value of DCE indicates DCSE is higher than the no. of defects corrected in the phase. Defect correction side effects reduce the defect correction effectiveness. 4.9 Defect BackLog (DCB) DCB of a phase can be defined as DCB = N detect N correct Where N detect, N correct are total no. of defects detected and corrected respectively. All the defects must be corrected in the same phase in which these are detected. The value of DCB must be zero to move to next phase. 5. CONCLUSION AND FUTURE DIRECTIONS Developing complex software with high quality at minimum cost is a great challenge that software industries are facing today. We can not do much for increasing quality of software after developing it. It is economically costly and also technically hard to improve low quality developed software. Therefore, quality need to be ensured at each phase. This paper presented a defect oriented approach of software development life cycle phase and its activities. Defect prevention and detection are the key activities of each phase to improve software quality. Aim of defect prevention activities is to identify root causes of defects and prevents them from recurring. If defect occurs it must be detected and corrected near to its origination time. Proposed set of metrics are useful for quantitative assessment of defect and its related activities. In the intensifying need of high quality software, metrics presented in this paper are very useful. Quality of software can be enhanced based on the results obtained from these metrics. For further scope, metrics presented in this paper can be analyzed with industrial project data. REFERENCES [1] A. Gunes Koru, Hongfong Liu, Building Effective Defect-Prediction Models in Practice IEEE Software, (November/December 2005). [2] Prathibha Tammana and Danny Faught, Software Defect Isolation http://www.tejaconsulting.com/papers/ iworks98/defect_isol.pdf. [3] Meng Li, He Xiaoyuan, Ashok Sontakke, Defect Prevention: A General Framework and its Application Proceedings of the sixth International Conference on Quality Software, (2006), IEEE. [4] Gill, Nasib Singh, Software Engineering, Khanna Book Publishing Co.(P) Ltd., 359-362, (2008). [5] Bala Subramaniam, Effective Software Defect Tracking Reducing Project Costs and Enhancing Quality, ISSRe

36 Nasib S. Gill & Sunil Sikka Systems Inc. Crosstalk The Journal of Defense Software Engineering, (April 1999). [6] Barry Bohem, Victor R.Basili Software Defect Reduction Top 10 List, Software Management, 135-137, (January 2001). [7] J. H. Van Moll, J. C. Jacobs, B. Freimut, J. J. M.Trienekens, The Importance of Life Cycle Modeling to Defect Detection and Prevention, Proceedings of the 10 th International Workshop on Software Technology and Engineering Practice, (2002), IEEE. [8] Pankaj Jalote, K. Dinesh, S. Raghavan, M. R. Bhashyam, M. Ramakrishanan Quantitative Quality Management through Defect Prediction and Statistical Process Control (April 2003), http://www.cse.iitk.ac.in/users/jalote/ papers/2wcsqpaper.pdf. [9] Robert G. Mays, Applications of Defect Prevention in Software Development, IEEE Journal on Selected Areas in Communication, 8 (2), (February 1990). [10] Stefan Biffl, Michael Halling, Investigating the Defect Detection Effectiveness and Cost Benefit of Nominal Inspection Teams, IEEE Transactions on Software Engineering, 29 (5), (May 2003). [11] Per Runeson, Carina Andersson, Thomas Thelin, What Do We Know about Defect Detection Methods?, IEEE Software, 82-90, (May/June 2006).