Subcortical structures segmentation on MRI using support vector machines

Transcription

1 J. Dolz et al. 1 Subcortical structures segmentation on MRI using support vector machines Jose Dolz 1, 2*, Hortense A. Kirisli 1, Maximilien Vermandel 2 and Laurent Massoptier 1 1 AQUILAB, Lille, France 2 Inserm U703, Université Lille 2, CHRU Lille, Loos, France * jose.dolz@aquilab.com Abstract. Medical imaging has evolved during the last years to become a fundamental tool for diagnosis, treatment and follow-up of patient diseases. Particularly, in oncology, medical imaging plays a key role in the diagnosis, treatment and follow-up of brain tumors. Magnetic resonance imaging (MRI) is often the medical imaging method of choice when soft tissue delineation is necessary. However, in clinical practice, organs at risk (OARs) delineation is often still performed manually by experts, or with very few machine assistance. As a consequence, the current delineation process has two major drawbacks: it is time consuming, and achieves low reproducibility. Although several methods to (semi-) automatically segment subcortical structures on MRI have been proposed to overcome these limitations, segmentation still remains challenging, with no general and unique solution. Among these methods, machine learning techniques, and more specifically support vector machines (SVM), have proved to outperform most of the proposed methods. Hence, SVM can be considered as state of the art in regards to the segmentation of subcortical structures. Support Vector Machines, MRI, subcortical structures, segmentation INTRODUCTION Medical imaging, which was initially used for basic visualization and inspection of anatomical structures, has evolved during the last years to become an essential tool for virtually all major medical conditions and diseases. Particularly, in oncology, advanced medical imaging techniques are used for tumor resection surgery and for subsequent radiotherapy treatment planning (RTP). Medical imaging plays a key role in the diagnosis, treatment and follow-up of brain tumors, which are nowadays the second most common cause of cancer death in men ages 20 to 39 and the fifth most common cause of cancer among women age 20 to 39 [1]. In daily clinical practice, magnetic resonance imaging (MRI) is often the medical imaging method of choice when soft tissue delineation is necessary. During RTP, the tumor to irradiate, i.e. clinical target volume (CTV), as well as healthy structures to be spared, i.e. the organs at risk (OARs), must be precisely delineated. These segmentations are crucial inputs for the RTP, in order to compute the parameters for the accelerators, and to verify the dose constraints. However, in clinical practice, OARs delineation on medical images is still performed manually by experts, or with very few machine assistance [2]. As a consequence, the current delineation process has two major drawbacks: it is time consuming, and achieves poor SUMMER Project (PITN-GA ) is funded by the 7 th Framework Programme of the European Commission

2 2 Segmentation of subcortical structures on MRI by using machine learning techniques reproducibility. To overcome these major issues, various computer-aided systems to (semi-) automatically segment anatomical structures in medical images have been developed and published in recent years. However, (semi-) automatic segmentation of subcortical brain structures still remains challenging, with no general and unique solution. Initial approaches of brain segmentation on MRI focused on the classification of the brain into three main classes: white matter (WM), gray matter (GM) and cerebral-spinal fluid (CSF) [3]. More recent methods include tumors and adjacent regions, such as necrotic areas, in addition to the primary cerebrum tissues [4]. Those methods are only based on image intensity. Because of the weak visible boundaries and similar intensity values between different subcortical structures (i.e. OARs), segmentation of subcortical structures can hardly be achieved based solely on signal intensity. Consequently, additional information, such as prior shape, appearance and expected location, is therefore required to perform the segmentation. The terminology subcortical structures as used in this chapter refers to subcortical GM structures within the brain that are not included as part of the cortex and are present in the depth side of the brain. In addition, the hippocampus, which is often considered a cortical structure, is included in our definition of subcortical structures. Several methods to segment subcortical structures on MRI have been proposed [5-8]. Atlas-based segmentation methods are among the most used techniques to perform the segmentation of such structures. These models rely on comparing the images under study with a pre-computed anatomical atlas of the brain. In [5], an extended review of the use of atlas-based segmentation methods to segment subcortical structures on MRI is presented. In addition to atlas-based methods, which only use a priori shape information, statistical models of shape and texture have been also employed [6-7]. In these approaches, correspondences across a training dataset are established, and the statistics of shape and intensity variation are learned and parameterized in terms of mean and eigenvectors, often by using principal component analysis (PCA). New instances are therefore constrained to a subspace of allowable shapes and textures, which are defined by the eigenvectors and their modes of variation. As a consequence, statistical models may be over-constrained, not generalizing well to unsampled population, particularly for small amounts of training data relative to the dimensionality. Contrary to statistical models, deformable models provide flexibility and do not require explicit training. Deformable models are defined as curves or surfaces, which are deformed under the influence of internal and external forces. While internal forces are related to curve features and try to keep the model smooth during the deformation process, external forces are the responsible of attracting the model toward object boundaries, and are related to image features of regions adjacent to the curve. Nevertheless, they are sensitive to initialization, noise and complex topologies. This makes deformable based segmentation methods being used in combination with other approaches, like in [8], where the evolution of the deformation is constrained by using a statistical model. Machine Learning techniques have been extensively used in the MRI analysis domain almost since its creation. Among all the existing machine learning techniques, support vector machines (SVM) represents one of the latest and most successful statistical pattern classifiers, and it has received a lot of attention from the machine learning and pattern recognition community. Although SVM approaches have been mainly employed for brain tumor recognition [9] in the field of medical image classification, recent works have also used them for tissue classification Multimodal imaging towards individualized radiotherapy treatments Summer-school of SUMMER project (July 2014)

3 J. Dolz et al. 3 [10] and segmentation of anatomical human brain structures [11-13]. By introducing machine learning methods, algorithms developed for medical image processing often become more intelligent than conventional techniques. Powell et al. [11] showed the improvements in the resulting relative overlaps when using machine learning methods (artificial neural networks (ANN) and SVM) to segment subcortical structures [11]. In this work, four methods were compared: template based, probabilistic atlas, ANN and SVM. It was showed that machine learning algorithms outperformed the template and probabilistic-based methods when comparing relative overlap between results obtained. Because of their incapability to generate new unsampled shapes, which considerably differs from the shapes in the training set, most of the techniques previously presented (except machine learning methods) might fail in the presence of brain lesions, such as tumors. This makes machine learning methods, and particularly SVM approach, more suitable techniques to perform the segmentation of subcortical structures, especially in such situations. Hence, SVM and its application to the segmentation of subcortical structures will be the focus of this chapter. In the next section, details of the basis of SVM and its use as optimizer for the segmentation problem applied to subcortical structures are presented. In addition, the experimental work carried out is also described in that section. In Results, some outcomes of the experimental work using the proposed approach are presented. The paper concludes with some outlines plans for future work. Support vector machines: the basics. MATERIALS AND METHODS Support vector machines and their variants and extensions, often called kernel-based methods, have been studied extensively and applied to various pattern classification and function approximation problems. Basically, the main idea behind SVM is to find the largest margin hyperplane that separates two classes. The minimal distance from the separating hyperplane to the closest training example is called margin. Thus, the optimal hyperplane is the one showing the maximal margin, which represents the largest separation between the classes (Fig.1.b). The training samples that lie on the margin are referred as support vectors, and conceptually are the most difficult data points to classify. Therefore, support vectors define the location of the separating hyperplane, being located at the boundary of their respective classes. a) b) Solution c) Mapping Kernel Function Figure 1. Process of mapping input samples to a higher dimensionality space to make the data linearly separable. The growing interest on SVM for classification problems lies in its good generalization ability and its capability to successfully classify non-linearly separable data. First, SVM attempts to SUMMER Project (PITN-GA ) is funded by the 7 th Framework Programme of the European Commission

4 4 Segmentation of subcortical structures on MRI by using machine learning techniques maximize the separation margin i.e., hyperplane- between classes, so the generalization performance does not drop significantly even when the training data are limited. Second, by employing kernel transformations to map the objects from their original space into a higher dimensional feature space [14], SVM can separate objects which are not linearly separable (Fig 1). Moreover, they can accurately combine many features to find the optimal hyperplane. Therefore, SVM globally and explicitly maximize the margin while minimizing the number of wrongly classified examples, using any desired linear or non-linear hypersurface. Use of SVM to segment subcortical structures. SVM approach has been successfully applied to the segmentation of subcortical structures. Powell et al. [11] compared the performance of ANN and SVM when segmenting subcortical (caudate, putamen, thalamus and hippocampus) and cerebellar brain structures. In their study the same input vector was used in both machine learning approaches, which was composed by the following features: probability information, spherical coordinates, area iris values, and signal intensity along the image gradient. Although obtained results were very similar, ANN based segmentation approach slightly outperformed SVM. However, they employed a reduced number of brains to test (only 5 brains), and 25 manually selected features, which means that generalization to other datasets was not guarantee. In machine learning, during the training of classifiers, if the number of image features is large, it can lead to ill-posing and over fitting, and reduce the generalization of classifiers. One way to overcome this problem is to reduce feature dimensionality. For this purpose, PCA was used in [12], followed by a SVM classification to identify statistical differences in hippocampus. However, selection of the proper discriminative features is not a trivial task, which has already been explored in the SVM domain. To overcome this problem, AdaBoost algorithm was combined with a SVM formulation [13]. In a first step, AdaBoost was used to select the features that most accurately span the classification problem. Then, SVM fused the selected features together to create the final classification. Furthermore, four automated methods for hippocampal segmentation using different machine learning algorithms were compared: hierarchical AdaBoost, SVM with manual feature selection, hierarchical SVM with automated feature selection (Ada- SVM), and a publicly available brain segmentation package (FreeSurfer). In their proposed study, the benefits of combining AdaBoost and SVM approaches were evaluated sequentially. Experimental set-up. To present robustness and efficacy of the use of SVM to optimize segmentation problem, corpus callosum was segmented in a set of sagittal MRI images. A set of 16 sagittal images containing the corpus callosum and 16 manual labeled masks were used. Each input vector for the SVM classifier consisted of 21 elements, and it was formed by the elements shown in Table 1. Regarding the kernel selection to map the training samples, a Radial Basis kernel was used for the purpose of this chapter. SVM segmentation method was divided into two steps: training and classification. While for the training step 7 cases were selected, for the classification step the 16 available cases were used. The first step was to create a binary mask with the manual labeled masks. This mask was computed by applying an or operation to all input labels in the training set. To make sure that corpus callosum was inside the mask in all input images, a security margin was provided to the Multimodal imaging towards individualized radiotherapy treatments Summer-school of SUMMER project (July 2014)

5 J. Dolz et al. 5 created mask. This mask was applied to all the images both in the training and in the classification steps in order to prune the image pixels. Features selected as components of the input SVM vectors are therefore extracted from the inner mask region (Fig 2). Input SVM Vector Element V 1 V 2 V 3 V 4 V 5 V 6 V 9 V 10 -V 21 Explanation Intensity value of the pixel under examination Angle between pixel and center point with respect to the horizontal line Distance from the pixel under examination to the center point Probability Map value Geodesic Map distance 4 Gradient image values across the largest gradient 12 signal intensity values along each of the two axis (i.e. 3 pixels for each side) Table 1. Features that are used in the SVM input vector. The probability map was derived from the manual label associated to each image. The skeleton was extracted from this label, and a Gaussian distribution from the resulted skeleton was applied to obtain a simulated probability map. Instead of using the whole skeleton, only its center point was used as mask to compute the geodesic distance [15] in the training step. For the classification step, however, the used mask is the computed skeleton of the input label (Fig 3). 2D Features SVM Training Model MR Images Manual Labels Common Mask Common Mask applied to all the images Training Set Map pixel pruning using the common mask Figure 2. SVM Training Process. Extract Features Training Input Data Manual Approximated Label Skeleton from the manual label Probability Map from skeleton Figure 3. Process of obtaining the SVM input vector features of an input image. Geodesic Map from the center of the skeleton To test the reliability of the segmentation algorithm, two segmentation results were compared to manually defined regions. First, the output of the classification proposed approach with no post processing was used. Second, a post processing step was applied to the classifier output. In this process, isolated small blobs were removed from the segmentation result. The results reported in this chapter were provided by computing the Dice similarity coefficient (DSC). The DSC(X,Y) is defined as the ratio of twice the intersection over the sum of the two segmented results, X and Y. According to this, DSC > 0.8 represents high agreement, 0.6 < DSC 0.8 indicates substantial agreement, and 0.4 < DSC 0.6 moderate agreement. SUMMER Project (PITN-GA ) is funded by the 7 th Framework Programme of the European Commission

6 6 Segmentation of subcortical structures on MRI by using machine learning techniques RESULTS Experiments demonstrated that the classification approach proposed in this chapter performed well when segmenting the corpus callosum. From Fig 4.a, it can be observed that 13 out of 16 cases reported DSC values higher than 0.8 for both with and without post-processing. Mean DSC values obtained by the proposed approach were 0.85 with a standard deviation value of 0.07 for the non post processing cases, and 0.89 with a standard deviation value of 0.05 for the cases where the post processing was applied. Regarding the influence of the post processing step, it increased the DSC values of the non-processed results around 3-4% as average. (a) (b) (c) Figure 4. (a) DSC obtained by the proposed approach for all the brain cases used in this work. Result segmentation example of the proposed approach without (b) and with (c) post processing. An important aspect to take into account when working with learning algorithms is the time required for the search, as well as for the training. In the proposed experiment, MATLAB was the language chosen, and it run over an Intel Xeon processor at 3.06 GHz. The time required to extract all the features in all the images used as training set was around 24 seconds in total. With this set of input features, the SVM training took nearly 18 seconds. In the other hand, the proposed approach segmented each of the input images in a time close to 3.5 seconds, where the classification process represented around 25% of this time, and the rest was the feature extraction. DISCUSSION MRI is widely used to identify subcortical brain structures for diagnosis, treatment and follow-up in brain tumors cases. During RTP, these subcortical structures (i.e. OARs) must be precisely delineated, which is currently done manually by experts, or with very few machine assistance. OARs delineation is therefore a time consuming process with poor reproducibility in clinical practice. The automatic segmentation method presented in this chapter is motivated by these limitations. By introducing machine learning methods, algorithms developed for medical image processing often become more intelligent than conventional techniques. Machine learning techniques - and SVM in particular - outperform classical segmentation methods, leading to improvements in the resulting relative overlaps as reported in the work of Powell et al [11]. Particularly, segmentation of the corpus callosum has been proposed and evaluated in this Multimodal imaging towards individualized radiotherapy treatments Summer-school of SUMMER project (July 2014)

7 J. Dolz et al. 7 chapter. The use of SVM - as trained in the proposed experiment - to automatically segment the corpus callosum evidences a high agreement between automatic segmentation result provided and manual labels. Experiments also showed that in some cases, overlapping between automatic segmentation and manual labels is considerably lower than in other cases (Fig 4.a). These cases showed to have an intensity distribution of the corpus callosum different from the samples in the training set. During the training phase, these intensity values were not sampled as a part of the input vector belonging to the corpus callosum. As a consequence, during the search process, input samples containing these intensity values were not properly classified. The introduction in the dataset of a range of samples that can represent a wide variability would improve the segmentation in such situations. In addition, although a post processing step of the output does not highly increase the DSC, it removes small labels that do not belong to the object to segment (Fig 4.b and 4.c). The time required by the proposed approach (both for training and search) was not considerably high. However, it has to be noted that only 2D images were used. Since the inclusion of more features and the use of volumes instead of images may be required to improve the segmentation result, the time required might dramatically increase, becoming an impractical approach. One solution to overcome this issue is to reduce the dimension of the input features, removing redundant features from the input vectors. As in the work of [12], PCA can be successfully applied to solve this. In the presented experiment, the probability map used as feature of the input vector was extracted from the manually labeled mask. Ideally, this probability map would be obtained from the registration step, as in [11], where the input image is registered with an atlas and the labels are propagated. This step makes the segmentation challenging, particularly in those subcortical structures which are close to brain lesions. If deformation caused by the lesion is not accordingly interpreted, the probability map, and consequently the segmentation result might fail. CONCLUSION An automatic approach to segment the subcortical structures on MRI has been presented. Although it has been only evaluated in one subcortical structure, there are some recent works that have proved the efficiency of machine learning methods when segmenting subcortical structures. The purpose of this chapter is to give an idea of how SVM works and some applications that have already used it for the segmentation. The main direction for future research is to examine the extension of this method to a set of subcortical structures which are involved in external radiotherapy and radio-surgery. Since a fully automatic approach is highly demanded in clinical practice, the use of the propagated labels to create the probability map is inside of our scope for this research. Additionally, we aim to extend this approach to its use in 3D images. However, as explained before, some considerations have to be taken into account in this case. As a consequence, the inspection of the dimensionally reduction of the features used as input vector is also required. ACKNOWLEDGMENT The research leading to these results (or invention) has received funding from the European SUMMER Project (PITN-GA ) is funded by the 7 th Framework Programme of the European Commission

8 8 Segmentation of subcortical structures on MRI by using machine learning techniques Union Seventh Framework Programme (FP7-PEOPLE-2011-ITN) under grant agreement PITN- GA The information and views set out in this publication are those of the authors and do not necessarily reflect the official opinion of the European Union. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained therein. REFERENCES [1] American Cancer Society. Cancer Facts & Figures Atlanta: American Cancer Society; [2] Whitfield, Gillian A., et al. "Automated delineation of radiotherapy volumes: are we going in the right direction?." The British journal of radiology (2013): [3] Xuan, Jianhua, Tülay Adali, and Yue Wang. "Segmentation of magnetic resonance brain image: integrating region growing and edge detection." Image Processing, Proceedings., International Conference on. Vol. 3. IEEE, [4] Ahirwar, Anamika. "Study of Techniques used for Medical Image Segmentation and Computation of Statistical Test for Region Classification of Brain MRI." International Journal of Information Technology & Computer Science 5.5 (2013). [5] Cabezas, Mariano, et al. "A review of atlas-based segmentation for magnetic resonance brain images." Computer methods and programs in biomedicine (2011): e158-e177. [6] Babalola, Kolawole O., et al. "3D brain segmentation using active appearance models and local regressors." Medical Image Computing and Computer-Assisted Intervention MICCAI Springer Berlin Heidelberg, [7] Rao, Anil, Paul Aljabar, and Daniel Rueckert. "Hierarchical statistical shape analysis and prediction of sub-cortical brain structures." Medical image analysis 12.1 (2008): [8] McIntosh, Chris, and Ghassan Hamarneh. "Medial-based deformable models in nonconvex shape-spaces for medical image segmentation." Medical Imaging, IEEE Transactions on 31.1 (2012): [9] Zhou, J., et al. "Extraction of brain tumor from MR images using one-class support vector machine." Engineering in Medicine and Biology Society, IEEE-EMBS th Annual International Conference of the. IEEE, 2006 [10] Akselrod-Ballin, Ayelet, et al. "Atlas guided identification of brain structures by combining 3D segmentation and SVM classification." Medical Image Computing and Computer-Assisted Intervention MICCAI Springer Berlin Heidelberg, [11] Powell, Stephanie, et al. "Registration and machine learning-based automated segmentation of subcortical and cerebellar brain structures." Neuroimage 39.1 (2008): [12] Golland, Polina, et al. "Detection and analysis of statistical differences in anatomical shape." Medical image analysis 9.1 (2005): [13] Morra, Jonathan H., et al. "Comparison of AdaBoost and support vector machines for detecting Alzheimer's disease through automated hippocampal segmentation." Medical Imaging, IEEE Transactions on 29.1 (2010): [14] Burges, Christopher JC. "A tutorial on support vector machines for pattern recognition." Data mining and knowledge discovery 2.2 (1998): [15] Criminisi, Antonio, Toby Sharp, and Andrew Blake. "Geos: Geodesic image segmentation." Computer Vision ECCV Springer Berlin Heidelberg, Multimodal imaging towards individualized radiotherapy treatments Summer-school of SUMMER project (July 2014)

9 J. Dolz et al. 9 Jose Dolz attended the Polytechnics University of Valencia (Spain) as an undergraduate, where he received his MSc degree in telecommunications and electrical engineering in After earning his MSc degree, he has worked at the university and in the private industry as computer vision researcher. He is currently an Early Stage Researcher at Aquilab, Lille, France. In addition, he is also enrolled as PhD candidate at the Ecole Doctorale Biologie et Santé of Lille 2 University. His research lies primarily within the fields of image processing and computer vision, where his work and research interests within these fields are image segmentation, feature extraction, image tracking and augmented reality. His work in image segmentation has been motivated and directed toward problems in medical imaging, especially in radiology treatment plans and radio-surgery. Hortense Kirişli comes from Chamonix-Mont-Blanc, France. In 2008, she obtained her Engineering degree in Electronics from the ENSEEIHT (Toulouse, France). Then, her research focused in cardiovascular image analysis at the Biomedical Imaging Group Rotterdam,Rotterdam, the Netherlands, where she obtained her Doctorate degree in June 2013.Since, Hortense is working as a R&D Engineer at Aquilab,Lille, France. She is developing software prototypes that make use of multi-modality imaging techniques for improved personalized external radiotherapy treatment planning, as part of the European 'SUMMER' project. She is leading the technical research integration and quality assurance work-package, and contributes to two other work-packages, dealing with the design of evaluation protocols, the clinical database, the design of user-interfaces, as well as user-testing studies. Hortense's main expertise are medical imaging technology research and its translation for clinical end users. SUMMER Project (PITN-GA ) is funded by the 7th Framework Programme of the European Commission