A Fuzzy Parallel Island Model Multi Objective Genetic Algorithm Gene Feature Selection For Microarray Classification.

Transcription

1 A Fuzzy Parallel Island Model Multi Objective Genetic Algorithm Gene Feature Selection For Microarray Classification. A. Natarajan Assistant Professor, Department of Information Technology, Jayaraj Annapackiam CSI College of Engg, Nazareth, Tamilnadu, India. Dr. R. Balasubramanian Professor and Head, Department of Computer Science and Engineering, M.S University, Tirunelveli, Tamilnadu, India. Abstract: Gene feature selection is necessary for microarray based cancer classification. The high performance parallel computing is used for gene expression analysis and finding the thousands of genes simultaneously. The selection of good genes will give a good classification result. The microarray data analysis is very much important to extract biologically useful data from the huge amount of expression data to know the current state of the cell.the main objective of this paper is to increase the classification accuracy and reduce the computing time. In this paper we propose a FPIMMOGA feature selection for identifying the best features and the best solution is evaluated by parallel version of SVM Classifier. Our breast cancer data is also experimented with the existing feature selections methods and evaluated by different data mining classifiers. Keywords: Microarray, Fuzzy logic, Multiobjective Genetic Algorithm, Gene Feature selection, Island model, parallel GA. Introduction DNA microarray experiments play a very important role in cancer classification and prediction. The microarray technology has been used in many cancer researches. The analysis of the large volumes of data generated under different experimental conditions is important and it requires advanced knowledge discovery methods. Many data mining techniques [20] have been proposed to analyze microarray data [21]. Feature Selection (FS) is a very important task in data mining for cancer classification with the goal of identifying very important features subsets in a microarray data. It is one of the most key problems in the field of machine learning. The classification and validation of molecular biomarkers for cancer diagnosis is an important problem in cancer genomics. The selection of candidate genes is very crucial to identify accurately the origin of cancer, its treatment and diagnosis too. With the appearance and fast development of DNA microarray technologies, making gene expression profiles for different cancer types has already become a hopeful means for cancer classification. Genetic algorithms (GAs) [6], a form of inductive learning strategies are adaptive search techniques initially introduced by Holland (Holland, 1975). Genetic Algorithms are inspired from Darwin s theory of evolution. By simulating nature evolution and emulating biological selection and reproduction techniques, the GA can solve complex problems in a strong search domain. The algorithm starts with a set of randomly generated solutions called population. The population size remains constant throughout the genetic algorithm. At each iteration the populations are evaluated based on their fitness quality with respect to the given application domain to form new solutions called offspring which retains many features of their parents. Offsprings are formed by two main genetic algorithm operators such as crossover and mutation. Crossover operates by randomly selecting a point in the two selected parent gene structures and exchanging the remaining segments of the parents to create new offspring. Therefore, crossover combines the features of two individuals to create two similar offspring. Mutation operates by randomly changing one or more components of a selected individual. It acts as a population perturbation operator and is a means for inserting new information into the population. This operator prevents any stagnation that might occur during the search process. In this paper we propose fuzzy logic based preprocessing technique to reduce the redundant information and grouping the similar genes from large amount of microarray data. The propose Parallel Island Model GA is implemented for gene feature selection process. Our propose feature selection algorithm is implemented based on multi objective genetic algorithm. This uses a different operator called multi objective operator. Multi objective aspect is defined to find the pareto optimal solutions for ranking. Since the search space is large and requires a good diversity, island model has been proposed. Finally the Fuzzy Based Parallel Island Model GA has been implemented by using parallelization tool [15] Open MP.This FPIMMOGA is used to progress the gene subsets and whose fitness is calculated by parallel version of SVM classifier.figure 1 shows the process of the work. 2761

2 serial programming is used for classification which slows down the computational speed. Proposed Method Microarray Data preprocessing by using Fuzzy Logic: Fuzzy logic groups the similar genes and reduces the large dimension of the data. It consists of four steps. In the first step gene expression levels are converted into Gaussian representational fuzzy subsets. In the second step fuzzy similarity of genes are accessed by the cosine amplitude method. Similarity matrices are made which is then converted by different composition to a matrixof fuzzy equivalence relation. In the third step groups of similar genes that correspond to fuzzy equivalent classes of genes are obtained by using α cuts with decreasing value of α.in the fourth step we select one gene randomly as the representation of the group and the others are ignored. By applying this method to the data set of genes for breast cancer data set is reduced to 8280 genes. Figure 1: The General Process of work. Related Work Soumen Kumar Patel, at el [16] has explained a novel feature selection method which was based on Multiobjective Genetic Algorithm using rough set theory. This method proposed to choose an important informative gene set, which classify the cancer dataset very efficiently. This method has been used two fitness functions individually based on the concepts of strong mathematics such as rough set theory and probability theory. The lack of diversity of the population is overcome by jumping gene mutation. The only drawback of this method is that the population size can be set within the range 100 to 1000 only. Asha Gowda Karegowda [2] has proposed a wrapper approach with genetic algorithm for generation of subset of attributes with different classifiers such as Naïve Bayes, Bayes Networks,C4.5 and Radial basis functions. The above classifiers are experimented on the Diabetes datasets, Breast cancer datasets, Heart Statlog and Wisconsin Breast cancer. The main disadvantage of this approach is that the computing time is very high for the large data sets. In 2009, QingzhongLiu, Andrew H. Sung [14] has proposed a new feature selection method called Recursive Feature Addition method on microarray based breast cancer data. The RFA gene feature selection method provides good classification accuracy than the other methods. In this method, The Island Model Genetic Algorithm Operators for generating Population: Our proposed FPIMMOGA is implemented for selecting the most important genes from the breast cancer data set. The population generation is defined in GA by two operators [11]- Crossover and Mutation. The Crossover operator has two versions 1. Crossover by Exchanging Values 2.Crossover by Inserting Values. The crossover operator mixes the features of two rules by the combination of their attributes. In our proposed work, crossover operators can be defined as: 1. Crossover by Exchanging values. If two rules X and Y have one or several common attribute(s) in their C parts, one common attribute is randomly selected. The value of the selected attribute in X is exchanged with its counterpart in Y (Fig. 2). Figure 2: Crossover by exchanging value 2. Crossover by Inserting values. Conversely, if X and Y have no common attribute, one term is randomly selected in the C part of X and inserted in Y with a probability inversely proportional to the length of Y. The related operation has performed to insert one term of Y in X (Fig.3). 2762

3 replacement strategy. Figure 4 presents the Multiobjective Genetic Algorithm scheme. Figure 3: Crossover by insertion of attributes Mutation is a genetic algorithm operator. This operator changes at random the value of a gene in a newly created individual. The population mutation rate used is very small, in the order of one mutation per thousand genes transfer. Thus mutation is considered to be a secondary mechanism of genetic algorithms. It is still used to introduce new solutions into the population and to protect the algorithm from premature loss of important genetic material by reintroduction of genes. Genetic algorithm has four mutation operators 1.Value mutation operator 2.Attribute mutation 3.Insertion operator 4.Delete operator. In this research work, we proposed adaptive strategy for calculating the rate of application of each mutation. We compute the new rate of mutation by calculating the progress of the jth application of mutation M i for an individual mutated into an individual mutation as follows: Progress j (M i ) = Max(fitness(ind, fitness(mut)) fitness(ind)) (1) Then, for each mutation operator M i, assume Nb_mut(M i ) applications of the mutation are done during a given generation ( j = 1,..., Nb_mut(M i )). Then, we can compute the profit of a mutation M k Profit(M k ) = j Progress j (M k ) / Nb_mut(M k ) (2) i ( j Progress j (M i ) / Nb_mut(M i ) We set a minimum rate δ and a global mutation rate p mutation for N mutation operators. The new mutation ratio for each M i is calculated using the following formula [8]: p(m i ) = Profit(M i ) (p mutation-n ᵟ) + ᵟ (3) The sum of all the mutations is equal to the global rate of mutation p mutation. The initial rate of application of each mutation operator is set to p mutation / N. Multi Objective Genetic Algorithm In the Single Objective Genetic algorithm, the classification accuracy is low and the time taken is very high using serial computing. To overcome this problem, we proposed PFIMMOGA that has been implemented in parallel computing. The population generation is one of basic step in multi-objective genetic algorithm applied here for dimension reduction. The population generation is derived by using the multiobjective optimization problem [19]. In a multi-objective optimization, all the solutions are best compromise. The best solutions encountered over generations are filed into a secondary population called the Pareto archive and the solutions can be selected from the Pareto archive during the production process. This method is called as elitism. The offspring solutions replace their parents according to the Figure 4: A Multi Objective GA Fuzzy Parallel Island Model Multi-Objective Genetic Algorithm The population generation is one of the basic step in parallel multi-objective genetic algorithm. The Parallel Genetic Algorithm [20] has been classified into three main models: global, fine cellular and island. The global model uses parallelism to speed up the Sequential GA. This model uses a global shared population and the fitness evaluation is done on different processors. The cellular model seeks to exploit the fine-gained, massively parallel architectures. The population is separated into a large number of very small subpopulations, which are maintained by different processors. In the island model, the population is divided into a few large independent subpopulations called islands. Each processor evolves their population using a parallel GA. For each island, some solutions rarely migrate to another island. We choose the island model for Parallel GA. Parallel Island Model: The island model is implemented in parallel programming and many islands are connected by using Fig.5. This model typically runs on a parallel multi-objective GA. In this, each processor is called an island with independent populations and Pareto archives (Fig.6). Each GA starts with its proper parameters such as population, parameters of GA. Periodically each island sends some solutions from its Pareto archive randomly selected to the neighboring island. The island model has the following steps: 1. Each island model creates its population. 2. Each model develops its population for a global number of generations and updates its archives for every generation. 3. Each island sends some solutions of its Pareto archive to the neighboring island according to the migration policy. 4. Finally, the island receives all the migrating solutions and replaces its worst solutions by those immigrants according to the ranking. At the end, a specific island waits for all the others to finish their execution and collects all the final Pareto archives to create the global Pareto archive. This island model has been 2763

4 implemented by using parallelization tools like Open MPI [1], [17]. preprocessing technique.the stopping criterion used is the no amelioration. Once the minimal number of generations has been over passed and after taking the best solution for all 10 generations, the iteration stops. The fig 7 shows the pseudo code for the Parallel Island Model. Island Model(E,N,µ) // overall population P of M = P individuals that is partioned into N subpopulations. begin Concurrently for each of the i 1 to N subpopulations Initialize(P i, µ) // For an even partition each subpopulation has µ =M/N individuals, but for generality Pi = µ so that each subpopulation might have a distinct size. P i is a initial subpopulation For epoch 1 to E do // The overall structure of the island model process comprises E major iterations called epochs Concurrently for each of the i 1 to N subpopulations do // During an epoch each subpopulation process independently executes a sequential evolutionary algorithm for Gi generations. Figure 5: Parallel Island model GA Figure 6: Island Model Parameters of GA for Breast cancer Data: Table 1: Fpimoga Parameters Of Gene Subset Selection S.No Parameters Values 1 Population Size Chromosome length No of Generations Crossover rate Mutation rate Elitism rate 10% 7 No of Generations 500 The above parameters in the table1are used in parallel island model GA for gene selection for the breast cancer. The input data used by this algorithm has normalized by fuzzy logic Sequential _GA(P i, G i) od; For i 1 to n do For each neighbour j of i Migration (P ip j) // After each epoch there is a communication phase during which individuals migrate between neighboring subpopulations Assimilate (Pi); // After each phase of migration each subpopulation must assimilate the migrants. This assimilation step is dependent on the details of the migration process od //loop termination od // loop termination Problem solution = best individual of all subpopulations // the size of each subpopulation remains the same after migration and the assimilation is simply a fitness recalculation 15.End Figure 7: Island Model Pseudo code The sequential GA has implemented using the following pseudo code which is shown in Fig 8. Sequential_GA (P i,,p j) / each subpopulation process executes a sequential evolutionary algorithm Begin For generation 1 to G i do P new Pᵩ // New population For offspring 1 to Max_offspring do P α selection (P i) // The selection strategy, which is responsible for choosing mates for the crossover procedure P β selection (P i) Pnew = pnew ʊ crossover (P α, P β ) // Two individuals are combined to create a single offspring 2764

5 Od // Loop termination Fitness_calculation(P i ʊ P new) // raw fitness function is calculated as the inverse of the objective function P i Reduction (P i ʊ P new)// The reduction strategy combines the current subpopulation with the newly created set of offspring, it then simply chooses the fittest individuals from the combined set to be the subpopulation in the next generation, thus keeping the subpopulation size constant. Mutation (P i); // The mutation operator performs random modifications on an individual, i.e. changes the routing solution randomly Fitness_calculation(P i) End Figure 8: Sequential GA Pseudo code The SVM Classification Support Vector Machines [18] are fundamentally binary classification algorithms.for linearly separable data, SVM computes the hyper plane that maximizes the margin between the training examples and the class boundary. For non linearly separable data the examples are mapped to a high dimensional space where such a separating hyper plane can be found. This mapping process mechanism is called the kernel function. In the domain of Microarray data SVM is one of the powerful good performance classifiers.svm can deal with great number of genes data, but researches prove that their performance is increased by reducing the number of genes. In our FPIMMOGA, we use a SVM classifier to assess the quality of a gene subset. For a chromosome x that correspond to a gene subset, we apply a Leave-One-Out Cross-Validation (LOOCV) method to compute the subset [13]. The LOOCV method means that one sample from the dataset is calculated as a test case while a SVM is trained on all the other samples, and this evaluation is repetitive for each sample. So the each chromosome x,fitness(x) = accuracysvm (x).one of the key elements of a SVM classifier concerns the choice of its kernel. Figure 9: Measured run time We carried out this experiment on publicly available microarray breast cancer datasets[15] available at Kent Ridge Bio-medical Data Set Repository ( existing feature selection methods are applied for this breast cancer data sets and classification accuracy are measured using Orange data mining and machine Learning tool[23].the feature selection methods[4],[9] like Novel Hybrid, Wrapper approach, Consistency Subset Selection (CON) [10], Correlated Feature Selection (CFS) [7], Single Objective Genetic Algorithm (SOGA) [8], Multi-Objective GA (MOGA) and the proposed Fuzzy Parallel Island model Multi Objective GA(FPIMMOGA) are applied on breast cancer data sets. It is then measured by various classification methods like Random Forest, K-NN, Classification tree, SVM,CN2 rules and Interactive Tree builder in Orange tool. Table 3: Classification Accuracy (%) of Breast Cancer Data Set Experiments Our proposed method has been implemented in Multi core processor environment[5]. The measured run times for 4,8,16,32 processors are shown in the Table 2 and Barchart.The best features have been taken from Fuzzy parallel island model GA gene feature selection method and the Parallel version of SVM classifier is used to evaluate the gene subsets. Table 2: Measured Run Time No of Processors Run time(h) Our proposed FPIMOGA feature selection method provides better classification accuracy than the other feature selection algorithm and the computing time is very less than the other methods. Table 3 shows the classification accuracy of various methods. The Cross-fold validation is used for measuring the classification accuracy in terms of time which is available at Orange tool and FPIMOGA is implemented by us. From Table 3 we observe that the proposed FPIMMOGA method is better than other methods in terms of time consuming, feature selection and classification accuracy. Our executing system consists of core i5 processor with NVIDIA [12] [24]Graphics processor running at 4 GHz clock frequency. 2765

6 Some statistical measurements like True Positive Rate, False Positive Rate of the classifiers are calculated using equation (4), (5). TPR = TP / P = TP / (TP+FN) (4) FPR = FP / N = FP / (P+TN) (5) Here the TP is positive object classified as positive, FP is positive object classified as negative, TN is negative object classified as negative and FN is negative object classified as positive. Fig 10 Show the classification accuracy of different classifiers. Figure 10: Classification Accuracy ROC Curve of Breast Cancer Data sets for Data mining classifiers. Figure 12: ROC Analyses for CN2 Classifier Figure 11: ROC Analyses for SVM Classifier Fig Figure 13: ROC Analysis for knn Classifier 2766

7 Figure 14: ROC Analysis for Classification Tree Figure 16: ROC Analysis for Random Forest Classifier Figure 15: ROC Analysis for ITB rules Classifier Figure 17: Proposed ROC for SVM 2767

8 Figure 18: Proposed ROC for RF Figure 20: Proposed ROC for CT Figure 19: Proposed ROC for knn Figure 21: Proposed ROC for ITB 2768

9 References Figure 22: Proposed ROC for CN2 Results The ROC curves visualized by orange data mining tool for the existing feature selection methods for different classifiers. It shows the classification performance of the different classifier. The proposed ROC curve is shown in Fig.16 to Fig 21. From the graph we observed that the true positive rate is better than the existing methods. Our proposed method curve indicates that the best features are taken and provides accurate classification performance on the breast cancer data set. Conclusion and Future Work We have done Fuzzy preprocessing technique for reducing the input data and implemented our FPIMMOGA using parallel programming (Open MP).The best features are selected in short time. The best identified gene subsets are evaluated by parallel version of SVM Classifier. Our method has given good classification accuracy than other methods. This method uses the island model for generating the best population. The multiple islands are implemented in parallel, which has significantly reduced the execution time in the process of best feature selection. In this work standard microarray breast cancer data sets are taken from Kent Ridge Biomedical Data Set Repository. In future Fuzzy Neural Network Next Generation Distributed Computing can be applied for Data management and Data analysis for speedy and accurate results. [1] August A.D., Chiou K.P.D, Sendag R., Yi J.J., (2010). Programming Multicores: Do Application Programmers Need to Write Explicitly Parallel Programs?, Computer Architecture Debates in IEEE MICRO, pp [2] Asha Gowda Karegowda, M.A.Jayaram, A.S. Manjunath, Feature Subset Selection Problem using Wrapper Approach in Supervised Learning, International Journal of Computer Applications 1(7):13 17, February [3] Alexandre Mendes,: Identification of Breast Cancer Subtypes Using Multiple Gene Expression Microarray Datasets,LNAI 7106,pp ,2011,Springer [4] Aboul Ella Hassaneian, Classification and feature selection of breast cancer data based on decision tree algorithm, Studies and Informatics Control,vol12, no1,march [5] Charles Severance,Kevin Dowd,: High Performance Computing,revised edition [6] Goldberg, D.E., Holland, J.H.: Genetic algorithms and machine learning. Machine Learning 3(2), (1988). [7] Hall, M.A.: Correlation-based feature selection for machine learning. Diss., The University of Waikato (1999) [8] Hong T.P,Wang H, and ChenH, Simultaneously applying multiple mutation operators in genetic algorithms, J. Heuristics, 6, (2000). [9] Kemal Polat, Seral Sahan, Halife Kodaz and Salih Günes, A New Classification Method for Breast Cancer Diagnosis: Feature Selection Artificial Immune Recognition System (FS-AIRS), In Proceedings of ICNC (2)'2005. pp.830~838. [10] Liu Yu, L.,H.: Efficient feature selection via analysis of relevance and redundancy. The Journal of Machine Learning Research 5, (2004) [11] Mohammed Khabzaoui, Clarisse Dhaenens, A Cooperative Genetic Algorithm for Knowledge Discovery in Microarray Experiments,Parallel Computing for Bioinformatics and computational Biology,2006 JohnWiley & Sons, Inc. [12] Oiso M., Matumura Y., (2011). Accelerating Steady-state genetic algorithms based on CUDA architecture,, in 2011 IEEE Congress on Evolutionary Computation, pp [13] Peng S, Ling, X. Peng, and Chen L. Molecular classification of cancer types from microarray data using the combination of genetic algorithms and support vector machines. FEBS Letter, 555(2): , [14] Qingzhong Liu,Andrew H. Sung,: Feature Selection and Classification of MAQC-II Breast Cancer and Multiple Myeloma Microarray Gene Expression Data [15] Quinn M. J. Parallel Programming in C with MPI and OpenMP, McGraw-Hill, NewYork,2004. [16] Soumen Kumar Patel, Asit Kumar,: Gene Selection Using Multi-objective Genetic Algorithm Integrating 2769

10 Cellular Automata and Rough Set Theory, SEMCCO 2013, Part II, LNCS 8298, pp , Springer International Publishing Switzerland [17] Umbarkar A. J., Joshi M. S. Dual Population Genetic Algorithm (GA) versus OpenMP GA for Multimodal Function Optimization.IJCA( ),64,2013. [18] V. N. Vapnik. Statistical Learning Theory. Wiley N.Y., [19] VanVeldhuizen D. A. and Lamont G, On Measuring Multiobjective Evolutionary Algorithm Performance, in In 2000 Congress on Evolutionary Computation, Piscataway, New Jersey, Vol. 1, July, 2000, pp [20] Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, USA (2005) [21] Xiaosheng Wang1 and Osamu gotoh A Robust Gene selection Method for Microarray-based cancer Classification Cancer Informatics [22] Zdenek Konfrst : Parallel Genetic Algorithms: Advances, Computing Trends, Applications and Perspectives,In Proceedings of the 18th International Parallel and Distributed Processing Symposium (IPDPS 04),IEEE [23] [24]