Diagnosing breast cancer with an improved artificial immune recognition system

Transcription

1 Soft Comput DOI /s METHODOLOGIES AND APPLICATION Diagnosing breast cancer with an improved artificial immune recognition system Mahmoud Reza Saybani 1,2 TehYingWah 1 Saeed Reza Aghabozorgi 1 Shahaboddin Shamshirband 3 Miss Laiha Mat Kiah 3 Valentina Emilia Balas 4 Springer-Verlag Berlin Heidelberg 2015 Abstract Breast cancer is the top cancer in women worldwide. Scientists are looking for early detection strategies which remain the cornerstone of breast cancer control. Consequently, there is a need to develop an expert system that helps medical professionals to accurately diagnose this disease. Artificial immune recognition system (AIRS) has been used successfully for diagnosing various diseases. However, little effort has been undertaken to improve its classification accuracy. To increase the classification accuracy, this study introduces a new hybrid system that incorporates support vector machine, fuzzy logic, and real tournament selection mechanism into AIRS. The Wisconsin Breast Cancer data set was used as the benchmark data set; it is available through the machine learning repository of the University of California at Irvine. The classification performance was measured through tenfold cross-validation, student s t test, sensitivity Communicated by V. Loia. B Mahmoud Reza Saybani saybani@gmail.com B Shahaboddin Shamshirband shamshirband@um.edu.my 1 Department of Information Systems, Faculty of Computer Science and Information Technology, University of Malaya, Kula Lumpur, Malaysia 2 Department of Computer Networks, Markaz-e Elmi Karbordi Bandar Abbas 1, University of Applied Science, Bandar Abbas, Iran 3 Department of Computer System and Technology, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia 4 Department of Automation and Applied Informatics, Aurel Vlaicu University of Arad, Arad, Romania and specificity. With an accuracy of 100 %, the proposed method was able to classify breast cancer dataset successfully. Keywords Artificial intelligence Artificial immune recognition system Breast cancer diagnosis Classification Data mining 1 Introduction It is an unfortunate fact that breast cancer is increasing worldwide. Mammography screening is very expensive and only countries with a good health system can benefit from it. Therefore, early detection strategies are needed which remain the cornerstone of breast cancer control (WHO 2012). Artificial intelligence and data mining techniques have been increasingly applied in the medical domain to identify hidden regularities or patterns and help medical practitioners to make decisions (Elouedi et al. 2014). Although there are advancements in cancer treatment, early diagnosis is still very important for increasing the survival rates of breast cancer patients (Lavanya and Rani 2012). Medical practitioners use their experience to draw diagnostic inference from patients examination, physical condition, and history. The obtained information can sometimes be uncertain, insufficient, and misleading; this makes the diagnostic tasks difficult even for experienced doctors (Sarkar and Leong 2009). Modern computer technologies have enabled medical doctors and scientists to store huge amount of records. It is obviously impossible for an unassisted human to process this tremendous amount of data (Cios and William Moore 2002). Combinations of mathematical models, biology, and artificial intelligence have entered a period of rapid discovery in

2 M. R. Saybani et al. medicine and are used to help physicians to more accurately diagnose various diseases. An expert system helps medical professionals to diagnose diseases faster and reduce human errors (Polat et al. 2007). The human immune system acts like a distributed learning system. It learns how to identify patterns and then uses memory cells to remember previously identified patterns (Dasgupta 1998). Scientists have been motivated by these very significant computational features to create artificial immune systems (AISs) (Timmis et al. 2004). AIS isa hybrid system (De Castro and Timmis 2002) and a class of adaptive algorithm (Brownlee 2005) that abstracts the structure and function of the vertebrate immune system into computational systems (Cai 2014; Yang 2011). AIS was built chiefly for feature extraction or data clustering, therefore its performance for classification was not satisfactory. Watkins (2001) introduced Artificial Immune Recognition System (AIRS) and focused on classification problems (Brownlee 2005). AIRS improved significantly the classification performance (Polat et al. 2007). A human being uses his intelligence to make decisions such as categorizing items or classifying patients. Applications of artificial intelligence techniques in diagnosing diseases have been increasing, and AIRS is one of these techniques that has been successfully used in medical classification problems (Chikh et al. 2012). There are different classification tools that offer different advantages; for instance, k-nearest neighbor is one of the best classifiers for specific problems; however, it is computationally expensive (Goodman et al. 2002). Artificial neural networks perform well, once trained, and can generalize from the training data set; however, very often the best network architecture for a specific task is unknown (Goodman et al. 2002). One of the advantages of AIRS is that one does not need to know the appropriate settings in advance and most importantly, it is self-determined. It has been shown that on certain problems, random settings of AIRS parameters lead to classification accuracies that are only a few percentage point less accurate than when optimized parameter settings are used (Goodman et al. 2002). AIRS is considered to be a clever (Brownlee 2011) supervised classifier that has shown significant achievements on a wide range of classification problems (Brownlee 2005; Pendharkar et al. 1999). Cuevas et al. (2012) argued that bio-inspired computing has proven to be useful for classification problems. Some of applications of AIRS will be discussed later. Although AIRS has been shown to have many good features, it still has potential to perform even better. Jenhani and Elouedi showed that relatively few efforts have been made to improve the classification accuracy of AIRS. The majority of published articles have applied AIRS to solve a particular classification problem (Jenhani and Elouedi 2012). AIRS2 is a more efficient version of the AIRS algorithm. AIRS3 keeps track of the number of antigens for each memory cells, and while AIRS2 uses regular k-nearest Neighbor for classification, AIRS3 uses weighted k-nearest Neighbor (Jenhani and Elouedi 2012). Golzari et al have identified that AIRS selection pressure is very high during resource competition, and this leads to loss of diversity, and may cause premature memory cells. The consequence of high selection pressure is decreased classification accuracy (Golzari et al. 2009a). Polat et al. (2007) have identified that AIRS classification performance can be improved by reducing the number of resources and training time. In addition, AIRS uses k-nearest neighbor (knn) as a classifier, and in machine learning, knn was developed in such a way that it identifies patterns of data without demanding for an exact match to any cases or stored patterns, which means relatively speaking, low accuracy. Furthermore, selecting k for knn affects the performance of knn, if k is chosen to be too small, the result might be sensitive to noise in data, and if k is chosen to be too large, then the neighborhood may include points from other classes (Wu et al. 2008). This paper proposes the following modifications to the process of AIRS2 with the goal of improving its classification accuracy. To reduce and control the resource numbers, this paper uses Fuzzy logic in resource allocation phase of the algorithm; To reduce loss of diversity and obtaining premature memory cells, real-world tournament selection (RWTS) method is introduced into the resource competition phase of the algorithm, and instead of using knn as a classifier, this paper uses support vector machine (SVM) which is a very robust classifier and has shown good performance in many applications (Joachims 1998). The effects of these changes were analyzed by applying the new model to realworld Wisconsin breast cancer dataset (WBCD). The WBCD was taken from UCI machine learning repository (Frank and Asuncion 2010). Obtained classification accuracy was then compared with other classifiers using the same dataset. We call this new hybrid FUZZY-SVM-RWTS-based AIRS2 system FSRAIRS2 and show that it obtains the highest classification accuracy among the classifiers reported by other researchers. The rest of this paper is divided into four major parts. The first part, background, presents human immune principles and their applications in the artificial immune systems. The second part, literature review, summarizes and analyzes the work of previous related studies. The third part, material and methods, describes the methodology and the algorithm in detail. The fourth part, the results, presents the result of this study and compares it with the results obtained through other well-known algorithms. This section is followed by the discussion and conclusion of the study.

3 Diagnosing breast cancer with an improved... Table 1 Mapping between the immune system and AIRS Immune system Antibody Recognition ball (RB) Shape-space Clonal expansion Antigen Affinity maturation Immune memory Metadynamics AIRS Feature vector Combination of feature vector and vector class The possible values of the data vector Reproduction of ARBs that are well matched with antigens Training data Random mutation of ARB and removal of the lowest stimulated ARBs Memory set of mutated ARBs Continual removal and creation of ARBs and Memory cells 2 Background 2.1 Human immune principles and artificial immune systems When infectious organisms or other invaders enter human body, the immune system s task is to attack and try to eliminate infectious organisms or other pathogens that enter human body (Timmis et al. 2004). The human immune system contains B- and T- lymphocytes. B lymphocytes originate from bone marrow where they can mature into B cells, and T lymphocytes originate from thymus where they can mature into T cells (Clark 2007). Once pathogens have been identified, B cells are stimulated and use defense mechanism to lock onto pathogens, with the help of T cells. B cells undergo somatic and hyper-mutation cloning and antibodies are produced. T cells have the function of destroying the invading germs which have been identified by the immune system (Timmis and Neal 2001). Once antibodies are produced, they will be distributed all over the body, so that if the same antigen attacks the body, the immune system uses the existing antibodies to lock onto the antigens and asks T cells to destroy them. Detailed information about immune system can be found in Sompayrac (2012) and Travers et al. (2008). Table 1 shows the mapping between terminologies used for immune system and AIRS (Watkins and Timmis 2002). 2.2 Artificial immune recognition system AIRS is a supervised learning algorithm that was inspired by natural immune system, it was developed by Watkins and Boggess (2002) who draw the idea from the work of Timmis and Neal (2001). This was the first version of AIRS that later was dubbed as AIRS1. Watkins and Boggess (2002) showed that performance of AIRS1 when applied on various benchmark classification problems was comparable to other highly regarded supervised learning techniques for the same benchmarks. Based on the experience gained from AIRS1, Watkins and Timmis decided to refine the process of AIRS1 by reducing the complexity of AIRS1 s approach while maintaining the accuracy of results, and introduced AIRS2 (Brownlee 2011). Watkins et al. showed that AIRS2 is simpler and computationally more efficient than AIRS1 and declared AIRS2 as the standard AIRS implementation (Joachims 1998). Details about the differences between AIRS1 and AIRS2 and information about AIRS2 s algorithm can be found in Watkins and Timmis (2002). Since this paper introduces a new hybrid system, the FSRAIRS2, it is important to note that also AIRS is an assembly of elements and procedures that were developed for supervised and unsupervised AIS algorithms (Brownlee 2005). Watkins used knn inside AIRS to perform the classification tasks. Here, knn is one of the components that have been added to other elements and procedures to make what is known as AIRS; therefore, it could also be interpreted as a hybrid algorithm of AIRS-KNN. Some researchers consider AIRS as pre-processor for KNN (Seeker and Freitas 2007). Literature review reveals that most researchers have used the term AIRS in their work; however, most of them do not explicitly differentiate whether they refer to AIRS1 or AIRS2. The focus of this paper is to make changes to AIRS2 algorithm. 2.3 Literature review Goodman et al. compared the performance of AIRS with other classifiers on the same multiple-class problems and concluded that AIRS was competitive with the top five to eight of those classifiers (Meng et al. 2005; Goodman et al. 2003). Comprehensive benchmark studies by Meng et al. (2005) on AIRS revealed that AIRS delivers reasonable results and can be used for real-world classification tasks. Marwah and Boggess (2002) investigated several different algorithms for circumstances when two or more classes were tied on the number of memory cells among the k strongest stimulated memory cells. Their report showed that accuracy of AIRS was in average higher for one of the testbeds than any reported at the UCI repository for that testbed.

4 M. R. Saybani et al. Goodman et al. investigated the behavior of AIRS on a number of publicly available classification problems. As they increased the number of classes, they kept the number of features constant and compared their result with Kohonen s learning vector quantization (LVQ), which is a well-known classifier. They traced the behavior of both classifiers across those publicly available classification problems. They concluded that AIRS s average performance on one of the problems was the best for that problem (Goodman et al. 2002). Conducting another study, Goodman et al. (2003) examined AIRS empirically. They replaced one of the two possible sources of its classification power with alternative modifications. They reported that results were slightly less effective, however, not statistically significantly so. Their modifications provided fast test version of AIRS. They concluded that AIRS s classification power lies in its replacement and maintenance of its memory cell population. The effects of adding non-euclidean distance measures to the classical AIRS algorithm were explored by Hamaker and Boggess they used four famous classification problems with various proportions of nominal, real and discrete features. They suggested to search further for non-euclidean distance metric that has the best performance for a certain data set (Hamaker and Boggess 2004). In another research, Polat et al. increased classifier s performance by incorporating fuzzy logic into resource allocation mechanism of AIRS. They argued that linearity of resource allocation mechanism of classical AIRS is the reason for excessive resource usage and therefore the classifier takes longer for training and requires a higher number of memory cells. They stated that their algorithm achieved higher classification accuracy than other known classifiers on the same data sets (Polat et al. 2007; Polat and Güneş 2008). Golzari et al. incorporated the real-world tournament selection algorithm into the resource competition phase of AIRS1 and were able to achieve higher classification accuracy. According to their report during competition phase of AIRS1, the algorithm uses a mechanism that has high selection pressure and causes loss of diversity and thus it may generate premature memory cells. They concluded that their experimental results on benchmark datasets from the UCI machine learning repository obtained significantly higher classification accuracy than AIRS1 in all cases (Golzari et al. 2009a, b). A hybrid method involving AIRS and fuzzy-weighted preprocessing was developed by Polat et al. (2007). They used this method to classify thyroid disease dataset. A comparison of their method with the classical AIRS on the same data set showed that their methodology performed better. Saidi et al. and/or Chikh et al. used fuzzy k-nearest neighbor in connection with the reduced memory cell pool of AIRS2 to assign a class membership to each instance, they claimed that the classification time of knn depends on the number of used data points; they argued that reducing this number is useful for the algorithm. They compared their results with results obtained from AIRS2 and claimed to have achieved higher classification accuracy than AIRS2 (Chikh et al. 2012; Saidi et al. 2011). Another application of AIRS can be seen in the work of Forouzideh et al. They used AIRS for text-document classification. Forouzideh et al. compared different versions of AIRS with multi-layer perceptron (MLP) and RBF as a simple neural approach; they concluded that different versions of AIRS performed better than both MLP and RBF. They claimed that because of high performance of AIRS, more successful applications of AIRS in a wide range of content mode classification are expected in the future (Forouzideh et al. 2011). Chen et al. (2008) reported a promising ability of AIRS in analyzing microarray data. In another study, Le and Mo- Yuen incorporated E-algorithm into AIRS in order to create a new Fuzzy-AIRS (FAIRS); they used the new algorithm to identify the cause of power outage. According to their report, results of FAIRS were compared with results of AIRS and E-algorithm. They reported that FAIRS achieved comparable performance with other algorithms; however, FAIRS was significantly faster in computing time than E-algorithm (Le and Mo-Yuen 2008). 2.4 Application of AIRS in diagnosing diseases Classification system has found many applications with significant results in diagnosing various diseases. One of the most important decision making tools in medicine is classification (Kahramanli and Allahverdi 2008), and data classification problems are common in many fields including medicine (Tunç 2012). Huang et al. (2012) indicated that for saving patients lives, classification accuracy of medical data sets has to be optimal. Kodaz et al. (2009) developed an AIRS-based model for diagnosing thyroid disease. They reported to have achieved a classification accuracy of %. Polat et al. (2006) developed a hybrid method of fuzzy-weighted pre-processing and AIRS to diagnose heart disease and reported their method was capable of achieving a classification accuracy of 96.3 %. Latifoǧlu et al. (2007) showed that their AIRS-based method could assist physicians in making the final judgment in diagnosing atherosclerosis patients. They achieved a classification accuracy of %. Another medical application of AIRS can be seen in the work of Kara et al. (2009) they used information gain-based AIRS to classify microorganism species, they have achieved a classification accuracy of %. A recent application of AIRS for use in the field of medical diagnosis was developed by Chikh et al. (2012) who presented a model to classify dia-

5 Diagnosing breast cancer with an improved... betes diseases. They have reported that application of their model, MAIRS2, on diabetes data set has achieved an accuracy of 89.1 %. In another study on predicting type 2 diabetes among pregnant women, Lin et al. (2011) used AIRS and compared its result with that of SVM and logistic regression; according to their report, AIRS achieved highest classification accuracy among these three classifiers, the obtained accuracy with AIRS was 62.8 % on the diabetes data set. Recently, Shamshirband et al. (2014) used AIRS and fuzzy labeling to diagnose tuberculosis disease. They have reported that their method achieved an accuracy of %. 2.5 Related work with Wisconsin breast cancer dataset There are many researches on medical diagnosis of breast cancer using WBCD in the literature. The following shows the classification accuracy achieved by the researchers, when they applied their models against WBCD. Golzari et al. (2009b, 2011) achieved accuracies of 97.17, 97.04, 97.1, and % with AIRS1, EXP1AIRS, EXP2AIRS, and RWT- SAIRS1, respectively. Daoudi et al. used median filtering for cloning in their AIS-based model and achieved a classification accuracy of 94.4 % for WBCD data set (Daoudi et al. 2013). Lavanya and Rani (2012) introduced a hybrid CART classifier with feature selection ad bagging techniques and applied their model against WBCD data set, they have reported a classification accuracy of %. Elouedi et al. (2014) proposed a hybrid clustering model to prognose the breast cancer using WBCD data set. They have achieved a classification accuracy of %. Polat et al. (2007) introduced Fuzzy-AIRS and applied their model against WBCD, and reported a classification accuracy of %. Accuracies of 96.7, 96.8, and 97.2 % obtained through Optimized-LVQ, big-lvq, and AIRS methods respectively were reported by Goodman et al. (2002). Setiono (2000) achieved with his application of neuro-rule method an accuracy of 98.1 %. Classification accuracy obtained by Peña-Reyes and Sipper (1999) was % they used Fuzzy-Genetic Algorithm (Fuzzy-GA). A classification accuracy of 97.2 % was reported by Bennett and Blue (1998)they used SVM as a classifier. Nauck and Kruse (1999) used neuro-fuzzy techniques and obtained an accuracy of %. Hamilton et al. (1996) reported a classification accuracy of % they used Rule Induction through Approximate Classification (RIAC) method, they achieved a classification accuracy of 96 % for C4.5 for the same dataset. Using Linear Discriminant Analysis (LDA) method, Ster and Dobnikar (1996) reported an accuracy of 96.8 %. Simplified Artificial Immune System (SAIS) was developed by Leung et al. (2007). To ensure global optimization of AIS, this classifier was also tested on WBCD, the classification accuracy that was reported for this data set was 96.6 %. Oliveira et al. (2012) developed a new algorithm by the name of Clonal Selection Classifier with Data Reduction (CSCDR) with the goal of optimizing the clonal selection process, when this algorithm was applied to WBCD, it achieved a classification accuracy of %. An approach to classify breast cancer using advanced multidimensional fuzzy neural network was introduced by Naghibi et al. (2012) they used Fuzzy Gaussian potential neural network (FGPNN) and hierarchical fuzzy neural network (HFNN) to classify breast cancer. They reported that simulation results demonstrated the effectiveness of both FGPNN and HFNN; highest reported accuracy on WBCD is 98.2 %. 3 Material and methods 3.1 Fuzzy resource allocation Fuzzy logic (FL) was invented by Zadeh (1965), he presented a new way of processing data, instead of using a crisp set of rules, he proposed using a partial set membership. FL has found applications in various fields including artificial intelligence. One of the stages of the AIRS algorithm deals with resource competition. The goal of resource competition is to improve the selection probability of high-affinity ARBs for subsequent steps. Resource competition depends on the number of allocated resources for each ARB, this allocation is based on the affinity between the ARB and the antigen, and its class. The resource allocation for each ARB in AIRS2 is calculated by multiplying the stimulation value by clonal rate as it is shown in Eq. (1): ARB.res = ARB.stim clonalrate (1) where ARB.res, ARB.stim and clonal rate are the number of allocated resources, stimulation value and a user defined value respectively. Marwah and Boggess (2002) employed a different resource allocation method, there the antigen classes that occurred more frequently obtained more resources. Both AIRS2 and the algorithm proposed by Marwah use linear resource allocation, and this linearity results in a higher number of memory cells and longer classification time (Polat et al. 2007; Golzari et al. 2008). To solve these problems, Eq. (1) is replaced with Eq. (2). ARB.res = FuzzyStimulation (ARB.stim) clonalrate (2) where FuzzyStimulation method was developed to control the stimulation value in a non-linear way. Figure 1 illustrates the pseudocode for resource allocation of FSRAIRS2. The algorithm uses fuzzy control language to convert the input variable stimulation into degrees of membership for the membership functions defined on this variable; the term resources has been chosen for the output

6 M. R. Saybani et al. Fig. 1 Resource allocation for FSRAIRS2 FOREACH ARB DO FIS fis null // FuzzyStimulation fis FIS.load (control file) //LOAD fuzzy control file (see Figure 2) input stimulation //set input variable fis.evaluate()//evaluate output fis.resources //return resources // use output (= resources) to // calculate resource allocation res output * ClonalRate resources resources + res ARB.orderByResources //order by allocated resources return resources ENDFOR variable because this study wants to use the output value for controlling resource allocation of FSRAIRS2. The linguistic variable for resources has to be converted into a value; this is done through a process called defuzzification. Figure 2 shows the fuzzy control language that is used for resource allocation of FSRAIRS2. To simplify the defuzzification, special membership functions, the singletons, are used. This paper uses Center of gravity for singletons (COGS) as a defuzzification method, which is defined as: ARB.stim on the x-axis gets values between 0 and 1, and its membership value is determined through the input membership functions. The amount of allocated resource is calculated through membership functions; it can be any number between 0 and 10 and is denoted on the x-axis of Fig. 4. The FuzzyStimulation method returns a real value that is then used to calculate the allocated resource according to Eq. (2). Result of defuzzication = ni=1 r i μ i ni=1 μ i (3) where r is output variable, n is the number of singeltons, and μ is membership function. Selection of linguistic values was chosen in a way that ARBs with stimulation values between 0.5 and 1 would get a higher allocated resource number than those ARBs with stimulation values between 0 and 0.5. For the case that no rule activates the defuzzifier, the default value for the output variable is set to zero. Figure 2 shows the rules that were used in this study to infer the fuzzy algorithm. This study used MIN fuzzy operator for activation process, by which the degree of fulfillment of rules acts on the output. MIN operator is defined as: Min(μ 1 (x), μ 2 (x)). MAX fuzzy operator was used for the accumulation of final results. MAX operator is defined as: Max(μ 1 (x), μ 2 (x)). Figures 3 and 4 illustrate the membership functions for the input and output variables stimulation and resources respectively. 3.2 Resource competition with real-world tournament selection mechanism One of useful, popular and robust mechanisms in genetic algorithm is tournament selection. The selection pressure of tournament selection depends directly on the tournament size. When the number of competitors increases, the resulting selection pressure increases too (Miller and Goldberg 1995). Lee et al. (2008) compared RWTS and conventional tournament selection (TS) mechanism and found that the former has a higher selection pressure with a relatively small loss of diversity and higher sampling accuracy than the later. They concluded that under similar selection pressure RWTS maintains more diversity than TS, and that higher pressure and sampling accuracy would improve the performance in a selection strategy. Lee et al. (2008) also argued that excessive use of high pressure in a selection strategy is not appropriate because it may cause a situation of premature convergence. Their results showed that RWTS causes only a small loss of

7 Diagnosing breast cancer with an improved... Fig. 2 Fuzzy control language for resource allocation of FSRAIRS2 FUNCTION_BLOCK Fuzzy_Command_File // Define input variables VAR_INPUT stimulation: REAL; END_VAR // Define output variable VAR_OUTPUT resources : REAL; END_VAR // Fuzzify input variable 'stimulation' FUZZIFY stimulation TERM VeryLow := (0, 1) (0.2, 0) ; TERM Low := (0.1, 0) (0.2,1) (0.3,1) (0.4,0); TERM Medium := (0.3, 0) (0.4,1) (0.5,1) (0.6,0); TERM High := (0.5, 0) (0.6,1) (0.7,1) (0.8,0); TERM VeryHigh := (0.7, 0) (0.8,1) (1,1); END_FUZZIFY // Defuzzify output variable 'resources' DEFUZZIFY resources TERM VeryLow := 0.1; TERM Low := 0.35; TERM Medium := 0.5; TERM High := 0.75; TERM VeryHigh := 1; // Use 'Center Of Gravity Singleton' as // defuzzification method METHOD : COGS; // Default value is 0 (if no rule activates defuzzifier) DEFAULT := 0; END_DEFUZZIFY RULEBLOCK RB // Use 'min' for 'and' AND: MIN; // Use 'min' for activation method ACT: MIN; // Use 'max' for accumulation method ACCU: MAX; RULE 1: IF stimulation IS VeryLow THEN resources IS VeryLow; RULE 2: IF stimulation IS Low THEN resources IS Low; RULE 3: IF stimulation IS Medium THEN resources IS Medium; RULE 4: IF stimulation IS High THEN resources IS High; RULE 5: IF stimulation IS VeryHigh THEN resources IS VeryHigh; END_RULEBLOCK END_FUNCTION_BLOCK diversity and this is an indication that RWTS does not have an excessive high selection pressure. RWTS is widely used to find a winner in a sport game. The players or teams are randomly grouped pair wise in sequence with a neighbor. If the number of players is an odd number, the last individual will be paired with someone from the same tournament level randomly; pairs compete and the winner of each pair goes to the next tournament level. In the next round this procedure repeats by building new pairs and starting the competition. At the end, one champion will be identified. The champion of the tournament is the player with the best fitness value among all participating players. For the resource competition, ARBs of each class represent individuals. ARBs enter the competition. Each ARB competes with a neighbor. This algorithm considers ARB A as a neighbor of ARB B, if ARB B is the first ARB from the same class as ARB A,

8 M. R. Saybani et al. Fig. 3 Membership function for stimulation Fig. 4 Membership function for resources and it is generated in the system after ARB A. The competition depends on the amount of allocated resources for ARBs. This amount is calculated for each ARB during the resource allocation process. This paper uses real-world tournament selection (RWTS) which has shown good result in connection with AIRS1 in theworkofgolzari et al. (2009a) and Golzari et al. (2009b). Figure 5 illustrates the pseudocode for resource competition of FSRAIRS2. As seen in Fig. 5, resources are allocated to ARBs, based on their fuzzy stimulation value and then pairs of ARBs are built as described above. These pairs undergo a competition, and the winner goes to next level and the looser will be removed from the list. At the end the strongest will survive, and resources allocated to the looser are removed. 3.3 Support vector machines Support vector machine (SVM) has a root in advanced statistical learning theory (SVMS.org 2010) and is known as one of the most accurate and robust algorithms among all well-known algorithms. It has profound theoretical foundations, is insensitive to the number of dimensions, and needs only a dozen instances for training (Wu et al. 2008). Superior performances of SVMs in comparison with other algorithms have been documented in many types of data including biomedical data (Statnikov 2011). SVM is able to maximize the predictive accuracy of a model without overfitting the training instances. Applications of SVM can be found in various disciplines, including bioinformatics, bio sequence analysis, classification, customer relationship management, concept extraction, text mining, character recognition, and voice and speech recognition. Vapnik et al. at AT&T Bell Laboratories introduced SVM in 1995; however, the soft margin version of SVM was introduced later by Cortes and Vapnik (1995). SVM tries to find an optimal hyperplanes in a multidimensional space which separate cases of different classes. An ideal hyperplane can be constructed through an iterative training algorithm. In a two-class learning task, for example, SVM makes sure that the best hyperplane is found through maximizing the margin between the two classes. The margin is the shortest distance between the closest data points to a point on the hyperplane (Wu et al. 2008). In cases when training data cannot be separated without error, SVM needs to separate the training set with a minimal number of errors, which are also known as classification error (Cortes and Vapnik 1995). SVM tries to exclude training errors from the training set. The error-free part of the dataset can be separated without errors (Cortes and Vapnik 1995). SVM models use a cost parameter, C, to allow some flexibility in separating the classes. C creates a soft margin that allows some misclassifications (Cortes and Vapnik 1995). Increasing C improves the classification accuracy for the training data; however, this may also lead to overfitting, because it increases the cost of misclassifying points too (Dtreg.com 2012). The problem of finding the optimal hyperplane is an optimization problem and Cortes et al. showed that the solution of this optimization problem is obtained through the saddle point of Lagrangian function (Cortes and Vapnik 1995). This classical optimization problem is solved by standard Quadratic Programming (QP) techniques and programs (Cortes and Vapnik 1995). The complexity of calculations does not depend on the dimension of the input space but on the number of support vectors (SVs), which is a small subset of the training vector (Laura Auria 2008) they assists model interpretation (Campbell and Ying 2011). Similarity and dissimilarity of data objects are quantified by kernels, which can be constructed for a broad range of data objects (Campbell and Ying 2011). There are a number of kernels that can be used in SVM models. These include linear, polynomial, radial basis function (RBF) and sigmoid (Statsoft 2013). This research uses RBF which is the most popular choice of kernel types used in SVMs. This is chiefly due to their finite and localized

9 Diagnosing breast cancer with an improved... Fig. 5 Pseudocode for resource competition of FSRAIRS2 Resources Allowed Total Resources Fuzzy-Stimulate ARBs //Allocate resources to ARBs based on their Fuzzy-Stimulated value Allocated Resource Sum of allocated resources of ARBs Current pair first ARB pair //Continue until the resources are below a threshold DOWHILE (Allocated Resource > Allowed Resource) NumResToBeRemoved Allocated Resource Resource Allowed FOR Current pair MinARB.resource Minimum (of resources allocated to the elements of the pair) //Check if element can be removed IF MinARB.resource NumResToBeRemoved Allocated Resource Allocated Resource MinARB.resource Delete MinARB.resource from population ELSE //Decrement resources MinARB.resource MinARB.resource NumResToBeRemoved Allocated Resource Allocated Resource NumResToBeRemoved ENDIF IF current pair is last ARB pair GOTO first ARB pair ELSE GOTO next ARB pair ENDIF ENDFOR ENDDOWHILE //Compare Allocated Resources of ARBs //The best ARB is always the one with the most resources BestARB ARB with the most Allocated Resources Return BestARB responses across the full range of the real x-axis (Statsoft 2013). The RBF is defined as: K (X i, Y i ) = exp( γ X i Y i 2 ), where γ is a kernel parameter. Increasing the value of γ improves the classification accuracy, but this can also lead to overfitting. A comprehensive explanation of SVM can be found in the work of Statnikov (2011), Cortes and Vapnik (1995) and Chang and Lin (2011). 3.4 Pseudocode of FSRAIRS2 In general, AIRS algorithm was developed with the goal of setting memory cells that can be used to classify data. The artificial memory cells represent several characteristics of human immune systems. These memory cells represent memory B Cells which have undergone a maturation process in the human body. FSRAIRS2 has inherited many features from AIRS2 and AIRS1, therefore following description of pseudocode has common structure that was introduced by Watkins et al. (2004). Figure 6 illustrates the pseudocode for FSRAIRS2 algorithm that is used in this paper. This is a new hybrid algorithm that uses fuzzy logic, RWTS, SVM, and general concept of AIRS2. FSRAIRS2 consists of five stages: in the first stage, data are normalized and initialized and there is an option for seeding the memory cell pool. This pool represents a collection of recognition elements that form the classifier which is produced at the end of training phase. In the second stage, the algorithm uses training data only once to prepare the classifier. One at a time, each antigen gets exposed to the memory pool. The antigen stimulates the recognition cells in the pool, and each cell is assigned a stimulation value. For the affinity maturation process, the memory cell with the maximum stimulation value is selected as the best match memory cell. The selected memory cell is used to create a number of mutated clones which are then added to the ARB pool. FSRAIRS2 uses this pool to refine the mutated clones of the best match memory cell for a specific antigen. In the third stage, the algorithm deals with competition for resources in the development process of a candidate memory cell. After the ARB pool has been generated, the process

10 M. R. Saybani et al. I. Initialize and normalize dataset II. Seed the memory cell pool (M), if desired. III. For each training example (antigen) do the following: 1. If M is empty, add antigen to M. 2. Select the memory cell (mc) in M of the same classification having the highest affinity to antigen. 3. Clone mc in proportion to its affinity to antigen. 4. Mutate each clone and add to the B-cell pool (ARB). 5. Use Fuzzy-logic to allocate resources to ARB Use RWTS-mechanism for removing the weak cells (population control of ARB). 6. Calculate the average stimulation of ARB to antigen and check for termination. If the termination condition is satisfied, goto step Clone and mutate a random selection of B-cells in ARB based upon their stimulation. 8. Loop back to step Select the B-cell in ARB with the highest affinity to antigen (candidate). If candidate has a higher affinity to antigen than mc add candidate to M. If mc and candidate are sufficiently similar, then remove mc from M. return M //prepare content of M for SVM IV. Transfer M to SVM format //run SVM as classifier V. Perform SVM classification using M. Fig. 6 Pseudocode for FSRAIRS2 of competition starts. Competition for resources is necessary to control the size of the ARB pool, as well as to promote such ARBs that have greater affinity (stimulation) to the antigen being trained on. The goal of this stage is to develop a memory cell that is most successful in accurately classifying a given antigen. To achieve this goal the algorithm uses three mechanisms. The first mechanism is about system wide resource competition. For a given ARB, resources are allocated. The criterion for the size of resource allocation is normalized stimulation value of the ARB, this value also represents the fitness of ARB to become a recognizer of an antigen. The second mechanism is about using mutation for shape-space exploration and diversification. The third mechanism determines the stop condition which depends on the average stimulation threshold. In step 5 of the 3rd stage, FSRAIRS2 uses fuzzy logic to refine resource allocation for ARBs and then it uses RWTS for determining and removing the weak cells during resource competition (population control of memory cell pool). Then the potential candidate memory cell is introduced into the set of already established memory cells for training. If the memory cell candidate is more stimulated by the training antigen than memory cell match, then the memory cell candidate will be added to the set of memory cells. If the affinity between memory cell candidate and memory cell match is less than a threshold, then memory cell candidate replaces memory cell match in the pool of memory cells. The above process repeats until all antigens have been introduced to the system.

11 Diagnosing breast cancer with an improved... Table 2 Parameters used for AIRS1, AIRS2 and FSRAIRS2 algorithms Parameters AIRS1 AIRS2 FSRAIRS2 Affinity threshold scalar (ATS) Clonal rate Hyper-mutation rate Seed cell Stimulation value Total resources kofknn 3 3 n/a SVM Type n/a n/a V-SVC Kernel function n/a n/a RBF γ n/a n/a 1 C n/a n/a 7 Cash memory size n/a n/a 100 MB In stage 4, the proposed algorithm prepares the content of M for the SVM classifier by reformatting the content to the format used by the SVM. In stage 5, when training for all antigens is completed, this algorithm presents the formatted M to the SVM classifier for classification. This research used WLSVM which is a wrapper of Lib- SVM (Chang and Lin 2011)insideWEKA(Hall et al. 2009) to address the classification problem. 3.5 Used data set This study used WBCD, taken from UCI machine learning repository (Frank and Asuncion 2010). The dataset consists of 699 samples that were collected by Dr. W.H. Wolberg at the University of Wisconsin Madison Hospitals taken from needle aspirates from human breast cancer tissue. The WBCD consists of nine features obtained from fine needle aspirates, each of which is represented as an integer value between 1 and 10. The measured features are as follows: (F1) Clump Thickness, (F2) Uniformity of Cell Size, (F3) Uniformity of Cell Shape, (F4) Marginal Adhesion, (F5) Single Epithelial Cell Size, (F6) Bare Nucleoi, (F7) Bland Chromatin, (F8) Normal Nucleoi, and (F9) Mitoses. 458 (65.5 %) of the samples belong to benign, and remaining 241 (34.5 %) samples belong to malignant class. Table 2 shows parameter settings that were used to run the algorithms. The values for the parameters were obtained through stepwise modifications of each parameters aiming at maximizing the classification accuracy. For determining a value for the ATS, we used values in the range of [0.1, 0.5] with an interval of 0.1; for the clonal rate, we used the values in the range of [8, 12] with an interval of 1; for the hypermutation rate, we used values in the range of [0.1, 3] with the of interval 0.1; for the seed cell, we used values in the range of [1, 10] with and interval of 1; for the stimulation value, we used values in the range of [1, 10] with an interval of 1; for the total resources, we used values in the range of [100, 300] with an interval of 50; for the k, we used the values in the range of [1, 5] with an interval of 1; for the SVM Type, we used C-SVC and nu-svc; for the kernel function, we used linear, RBF, polynomial and sigmoid functions; and for the γ, we used values in the range of [1, 5] with an interval of 1; for the C, we used values in the range of [1, 10] with an interval of 1; and for the memory size, we used values in the range of [50, 200] with an interval of 50. To reduce the number of all possible situations for so many parameters, as soon as we reached optimal solution with the first parameter, we kept this constant and moved to vary the second parameter. As soon as the optimal value was reached, we kept the first two parameters constant and moved to the next parameter. We repeated this process for all parameters, when all values were determined then we moved to determine values for the SVM part. The displayed values are the optimal values we obtained for each algorithm (AIRS1, AIRS2, and FSRAIRS2). As it can be seen from the Table 2, FSRAIRS2 has more parameters than the AIRS1 and the AIRS2 that can be set by the user, this gives the user more flexibility in optimizing the model. Best results were obtained with the above settings. 3.6 Applied parameters 4 Performance and evaluation measurements 4.1 Classification accuracy The accuracy of a classifier refers to the ability of a given classifier to correctly predict the class label of new or previously unseen data. The accuracy is calculated by dividing the number of truly classified instances by the number of instances

12 M. R. Saybani et al. in the test phase. The classification accuracy for the dataset was measured according to Eq. (4)(Watkins 2001): T i=1 accuracy (T ) = assess (t i), t i T (4) T { 1, if classify (t) = t c aasset (t) = (5) 0, otherwise where T is the set of data items to be classified (the test set), t T, t c is the class of the item t, and classify (t) returns the classification of t by AIRS. 4.2 N-fold cross-validation Classification accuracy is a critical design goal of any classifier (Watkins 2001). To demonstrate that test results are more valuable, researchers use n-fold cross-validation. It helps to minimize the bias associated with the random sampling used during training phase (Delen et al. 2005). N-fold cross-validation is a robust approach to estimate the predictive accuracy of classification algorithms. In this approach, the instances are randomly divided to n equal stratified subsets. Each instance is put in one subset. At each iteration, n 1 subsets are merged to form the training set and the classification accuracy of the algorithm is measured on the remaining subset. This process is repeated n times, choosing a different subset as the test set each time. Therefore, all data instances have been used n 1 times for training and once for testing. The final predictive accuracy is computed over all folds in the usual manner with dividing the number of correct classifications taken over all folds by the number of data instances in all folds. This study used tenfold cross-validation for evaluation purposes. 4.3 Data reduction One important feature of AIRS2 is its capability to perform the classification task with the reduced number of data instances. These reduced data sets are evolved memory cells that are produced from the original training instances, which are used to characterize a given class of data from the original training data to the evolved set of memory cells (Watkins et al. 2004). Therefore, it is important to verify how the proposed algorithm performs. Data reduction percentage is calculated from the Eq. (6)(Brownlee 2005). Data ( Reduction = 1 Total Number Memory Cells Total Number Training Instances ) 100 (6) 4.4 Student s t test This paper uses Student s t test for testing the statistical significance between the mean of two results (Jackson 2012; Ian et al. 2011). In order to make decisions, it is common to make guesses or assumptions about the populations involved. Such assumptions are called statistical hypotheses, which may or may not be true. The hypotheses consist of a null hypothesis (Ho) which is tested against an alternative hypothesis denoted by (H 1 ). There are two types of errors when we make decisions. Type I error occurs when we decide to reject a hypothesis when it should be accepted; however, when we accept a hypothesis when it should be rejected, then we say, we made a Type II error. In both cases a wrong decision has been made. In practice, the maximum probability with which we accept to make a Type I error is known as the level of significance of the test. It is common to represent this probability by α, and (1 α) represents the level of confidence of the test. Equations (7) and (8) present the null hypothesis and its alternative hypothesis: H 0 : AIRS2 performance mean = Proposed Algorithm performance mean (7) H 1 : AIRS2 performance mean = Proposed Algorithm performance mean (8) In other words this paper examines the hypothesis (H 0 ) that the mean values of two results are the same. The two-tailed t test is used in this paper to decide whether we can accept or reject the null hypothesis. We determine the p value which is the probability that the two means are the same. If p <α, then we infer that this probability is not enough for the difference between the means to be based merely on chance, and therefore we conclude that there is a statistically significant difference in the means. This paper uses α = Results Comparing the mean classification accuracy for the WBCD (Original) obtained with the AIRS2 (n 1 = 100, M = %, SD = 2.19) to the one obtained with the FSRAIRS2 (n 2 = 100, M = 100 %, SD = 0.0, p < 0.05) the results show that the differences are statistically significant at the 95 % significant level (α = 0.05) for this data set and that FSRAIRS2 achieved significantly higher classification accuracy than AIRS2. Both sensitivity and specificity for FSRARIAS2 on WBCD were 100 %. For the WBCD data set, reported and obtained classification accuracies of AIRS-based classifiers are shown in Table 3: Polat et al. (2007) achieved an accuracy of 97.2 and % for AIRS and Fuzzy-AIRS, respectively. Golzari Hormozi (2011) reported accuracies of 97.1 % and %

13 Diagnosing breast cancer with an improved... Table 3 Comparison of classification accuracy obtained by AIRS-based models on WBCD Researcher Method Classification accuracy (%) This study FSRAIRS Polat et al. (2007) Fuzzy-AIRS Golzari Hormozi (2011) RWTSAIRS Polat et al. (2007) AIRS Goodman et al. (2002) AIRS 97.2 Golzari Hormozi (2011) AIRS This study AIRS This study AIRS Table 4 Comparing classification accuracy of FSRAIRS2 with other classifiers on WBCD Researcher Method Classification accuracy (%) This study FSRAIRS2* 100 Polat et al. (2007) Fuzzy-AIRS* Naghibi et al. (2012) FGPNN, FNN 98.2 Setiono (2000) Neuro-Rule 2a 98.1 Peña-Reyes and Sipper (1999) Fuzzy-GA Bennett and Blue (1998) SVM This study AIRS1* Ster and Dobnikar (1996) LDA* 96.8 Goodman et al. (2002) Big-LVQ* 96.8 Goodman et al. (2002) Optimized LVQ* 96.7 This study MLP* 96.6 This study AIRS2* Nauck and Kruse (1999) NEFCLASS* Hamilton et al. (1996) RIAC* Quinlan (1996) C4.5* This study Naïve Bayes* 93.3 This study J48* 93 This study LibSVM* * tenfold cross-validation, +5-fold cross-validation for AIRS1 and RWTSAIRS1 respectively. Goodman et al. (2002) obtained an accuracy of 97.2 %. All results of this study were obtained from averaging 10 runs and 10-way cross-validation. The high classification accuracy obtained with the proposed model can be explained as the positive effects of multi parameters, our results for data reduction (see below subsection Data Selection) suggest that some unimportant memory cells have been discarded and this may have led to improved classification performance. This research introduced SVM into AIRS and SVM is also known to be a robust classifier that has shown good performance in many applications (Joachims 1998). Although both SVM and KNN are considered to be among the top 10 classifiers (Wu et al. 2008), independent of the application of KNN in AIRS, result of studies has shown that SVM outperforms knn (Huang et al. 2011) especially when the number of features increases (Hmeidi et al. 2008). Researchers of a recent study have also argued that using knn reduces the classification accuracy of AIRS (Jenhani and Elouedi 2012). Putting together, improving data reduction capability and using SVM with optimized parameter settings have led to this high classification performance of the proposed model. 5.1 Comparing classification accuracy of FSRAIRS2 with other classifiers The classification accuracy obtained with FSRAIRS2 for the Wisconsin breast cancer dataset is the highest among the classifiers reported by other researchers. The comparison of FSRAIRS2 with these classifiers with respect to the classification accuracy is illustrated in Table Data reduction Results show that both AIRS2 and FSRAIRS2 are capable of reducing the data sets needed to perform the classification tasks. As Table 5 shows, FSRAIRS2 was able to perform the