Statistical Applications in Genetics and Molecular Biology

Transcription

1 Statstcal Applcatons n Genetcs and Molecular Bology Volume 5, Issue Artcle 6 Dmenson Reducton for Classfcaton wth Gene Expresson Mcroarray Data Jan J. Da Lnh Leu Davd Rocke Unversty of Calforna, Davs, jjda@ucdavs.edu Unversty of Calforna, Los Angeles, lleu@math.ucla.edu Unversty of Calforna, Davs, dmrocke@ucdavs.edu Copyrght c 2006 The Berkeley Electronc Press. All rghts reserved.

2 Dmenson Reducton for Classfcaton wth Gene Expresson Mcroarray Data Jan J. Da, Lnh Leu, and Davd Rocke Abstract An mportant applcaton of gene expresson mcroarray data s classfcaton of bologcal samples or predcton of clncal and other outcomes. One necessary part of multvarate statstcal analyss n such applcatons s dmenson reducton. Ths paper provdes a comparson study of three dmenson reducton technques, namely partal least squares (PLS), slced nverse regresson (SIR) and prncpal component analyss (PCA), and evaluates the relatve performance of classfcaton procedures ncorporatng those methods. A fve-step assessment procedure s desgned for the purpose. Predctve accuracy and computatonal effcency of the methods are examned. Two gene expresson data sets for tumor classfcaton are used n the study. KEYWORDS: partal least squares, slced nverse regresson, feature extracton, gene expresson, tumor classfcaton The research reported n ths paper was supported by grants from the Natonal Scence Foundaton (ACI , and DMS ), the Natonal Insttute of Envronmental Health Scences (P43-ES04699 and P01-ES11269), the Natonal Cancer Insttute (P30-CA093373), and the UC Davs Health System. We thank the edtor and two revewers for helpful comments and suggestons.

3 Da et al.: Dmenson Reducton for Class Predcton 1 1. Introducton A characterstc of gene expresson mcroarray data s that the number of varables (genes) p far exceeds the number of samples n, commonly known as the large p, small n problem (West et al., 2001; Dudot et al., 2002). In addton, the gene expresson measures can be hghly correlated. These features present a challenge to modelng the relatonshp between phenotype and gene expresson profles and classfyng (predctng) samples nto known categores such as tumor types n cancer research (Speed, 2003). There are several ways to deal wth the problem. One can reduce dmenson of the data by selectng a subset of nterestng genes (gene selecton), or producng gene components or super genes combnatons of genes (dmenson reducton), or usng combnaton of the strateges. Gene selecton s usually based on some unvarate measure related to the classfcaton (e.g. Hedenfalk et al., 2001; Dettlng and Buhlmann, 2003). Gene components can be constructed usng multvarate technques wth the premse that, although the mcroarray data contan numerous genes, there may be actually a small number of underlyng varables that account for most of the varaton n the data (West et al., 2001). For example, a few lnear combnatons of genes may explan most of the response varaton. Each approach has ts own advantages and lmtatons (Boulestex, 2004). A combnaton of the strateges s often used n practce for classfcaton wth gene expresson data. Such classfcaton procedures often consst of the followng steps: the frst step s gene selecton/dmenson reducton, n whch a few gene components are constructed from a large number of genes; the second step s classfcaton, n whch the samples are classfed nto categores by applyng standard statstcal models on the gene components (Nguyen and Rocke, 2002a, 2002b). Dmenson reducton s a subject of study n several research areas ncludng hgh-dmensonal data analyss, pattern recognton, and machne learnng, where one seeks to explan observed hgh-dmensonal data usng an underlyng low-dmensonal representaton. Dmenson reducton has many applcatons n bonformatcs and computatonal bology. The purpose of ths study s to evaluate some of those recently proposed for tumor classfcaton wth gene expresson data. Specfcally, we focus on three dmenson reducton methods: partal least squares (PLS) (Nguyen and Rocke, 2002a, 2002b; Huang and Pan, 2003; Boulestex, 2004), slced nverse regresson (SIR) (Charomonte and Martnell, 2002; Antonads et al., 2003; Bura and Pfeffer, 2003), and prncpal component analyss (PCA) (Ghosh, 2002). These methods have been shown hghly useful for classfcaton wth gene expresson data. However, there s lack of comparson studes on those methods. For example, the relatve performance of PLS and SIR dmenson reducton for classfcaton s largely unknown. Produced by The Berkeley Electronc Press, 2006

4 2 Statstcal Applcatons n Genetcs and Molecular Bology Vol. 5 [2006], No. 1, Artcle 6 In ths paper, we evaluate the relatve performance of several classfcaton procedures ncorporatng those dmenson reducton methods (PLS, SIR and PCA). We dscuss the methodologcal presumptons of the methods, address ssues nvolved n comparng the models, desgn a fve-step assessment procedure, and present results of evaluatons based on two gene expresson data sets n cancer research: the leukema data set of Golub et al. (1999) and the colon data set of Alon et al. (1999). The paper s organzed as follows. In Secton 2, we descrbe the methods of dmenson reducton, classfcaton, gene selecton, model selecton and valdaton, and desgn a procedure for assessng the relatve performance of the models. In Secton 3, we descrbe the mcroarray data sets and the experments, and present the results of evaluatons. Summares and dscussons are presented n Secton Methods The applcaton context s predcton of response classes such as tumor types usng gene expresson mcroarray data. We vew the problem as a multvarate regresson problem where the number of varables far exceeds the number of observatons (Stone and Brooks, 1990; Frank and Fredman, 1993; Krzanowsk, 1995; Kers, 1997). A classfcaton procedure for the purpose may consst of two basc steps: the frst step s dmenson reducton, n whch the data are reduced from the hgh p-dmensonal gene space to a lower K-dmensonal (K<n) gene component space; the second step s class predcton, n whch response classes are predcted usng a standard class predcton model on the gene components. A step of prelmnary gene selecton can be easly ncorporated nto the procedure. In ths secton, we frst dscuss three dmenson reducton methods (PLS, SIR and PCA) and a standard classfcaton model (logstc dscrmnaton), and then descrbe the methods for gene selecton, model selecton and valdaton, and fnally desgn and present a fve-step procedure for model assessment Dmenson Reducton: PCA, PLS and SIR One way to acheve dmenson reducton s to transform the large number of orgnal varables (genes) to a new set of varables (gene components), whch are uncorrelated and ordered so that the frst few account for most of the varaton n the data. The K new varables (gene components) can then replace the ntal p varables (genes), thereby reducng the data from the hgh p-dmenson to a lower K-dmenson. PCA, PLS and SIR are three of such methods for dmenson reducton. To descrbe them, let be the n p matrx of n tssue samples and p

5 Da et al.: Dmenson Reducton for Class Predcton 3 genes, y be the n 1 vector of response values, and S be the p p covarance matrx of the gene expressons Prncpal Component Analyss PCA s a well-known method of dmenson reducton (Jollffe, 1986). The basc dea of PCA s to reduce the dmensonalty of a data set, whle retanng as much as possble the varaton present n the orgnal predctor varables. Ths s acheved by transformng the p orgnal varables = [x 1, x 2,, x p ] to a new set of K predctor varables, T = [t 1, t 2,, t K ], whch are lnear combnatons of the orgnal varables. In mathematcal terms, PCA sequentally maxmzes the varance of a lnear combnaton of the orgnal predctor varables, u K = arg max Var( u) (1) u' u= 1 subject to the constrant u ' S u j = 0, for all 1 < j. The orthogonal constrant ensures that the lnear combnatons are uncorrelated,.e. Cov( u, u j ) = 0, j. These lnear combnatons t = u (2) are known as the prncpal components (PCs) (Massey, 1965). Geometrcally, these lnear combnatons represent the selecton of a new coordnate system obtaned by rotatng the orgnal system. The new axes represent the drectons wth maxmum varablty and are ordered n terms of the amount of varaton of the orgnal data they account for. The frst PC accounts for as much of the varablty as possble, and each succeedng component accounts for as much of the remanng varablty as possble. Computaton of the prncpal components reduces to the soluton of an egenvalue-egenvector problem. The projecton vectors (or called the weghtng vectors) u can be obtaned by egenvalue decomposton on the covarance matrx S, S u = λ u (3) where λ s the -th egenvalue n the descendng order for =1,,K, and u s the correspondng egenvector. The egenvalueλ measures the varance of the -th PC and the egenvector u provdes the weghts (loadngs) for the lnear Produced by The Berkeley Electronc Press, 2006

6 4 Statstcal Applcatons n Genetcs and Molecular Bology Vol. 5 [2006], No. 1, Artcle 6 transformaton (projecton). The maxmum number of components K s determned by the number of nonzero egenvalues, whch s the rank of S, and K mn(n,p). The computatonal cost of PCA, determned by the number of orgnal predctor varables p and the number of samples n, s n the order of mn(np 2 + p 3, pn 2 + n 3 ). In other words, the cost s O(pn 2 + n 3 ) when p > n Partal Least Squares The objectve of constructng components n PLS s to maxmze the covarance between the response varable y and the orgnal predctor varables, w K = arg max Cov( w, y) (4) w' w= 1 subject to the constrant w ' S w j = 0, for all 1 < j. The central task of PLS s to obtan the vectors of optmal weghts w (=1,,K) to form a small number of components that best predct the response varable y. Note that PLS s a supervsed method because t uses nformaton on both and y n constructng the components, whle PCA s an unsupervsed method that utlzes the data only. To derve the components, [t 1, t 2,, t K ], PLS decomposes and y to produce a blnear representaton of the data (Martens and Naes, 1989): and = t1 w' 1+ t 2w' t K w' K + E (5) y t q + t q t + F (6) = q K K where w s are vectors of weghts for constructng the PLS components t=w, q s are scalars, and E and F are the resduals. The dea of PLS s to estmate w and q by regresson. Specfcally, PLS fts a sequence of blnear models by least squares, thus gven the name partal least squares (Wold, 1966, 1973, 1982). At each step (=1,,K), the vector w s estmated n such a way that the PLS component, t, has maxmal sample covarance wth the response varable y subject to beng uncorrelated wth all prevously constructed components. The frst PLS component t 1 s obtaned based on the covarance between and y. Each subsequent component t (=2,,K), s computed usng the resduals of and y from the prevous step, whch account for the varatons left by the prevous components. As a result, the PLS components are uncorrelated and ordered (Garthwate, 1994; Helland, 1988, 1990).

7 Da et al.: Dmenson Reducton for Class Predcton 5 The maxmum number of components, K, s less than or equal to the smaller dmenson of,.e. K mn(n,p). The frst few PLS components account for most of the covaraton between the orgnal predctors and the response varable and thus are usually retaned as the new predctors. The computaton of PLS s smple and a number of algorthms are avalable (Martens and Naes, 1989). In ths study, we used a standard PLS algorthm (Denham, 1995). Lke PCA, PLS reduces the complexty of mcroarray data analyss by constructng a small number of gene components, whch can be used to replace the large number of orgnal gene expresson measures. Moreover, obtaned by maxmzng the covarance between the components and the response varable, the PLS components are generally more predctve of the response varable than the prncpal components. The number of components, K, to be used n the class predcton model s consdered to be a meta parameter and must be estmated n the applcaton, whch we wll dscuss later. PLS s computatonally very effcent wth cost only at O(np),.e. the number of calculatons requred by PLS s a lnear functon of n and p. Thus t s much faster than the other two methods (PCA and SIR) Slced Inverse Regresson SIR, one of the suffcent dmenson reducton methods (L, 1991, 2000; Duan and L, 1991; Cook 1998), s a supervsed approach, whch utlzes response nformaton n achevng dmenson reducton. The dea of SIR s smple. Conventonal regresson models deal wth the forward regresson functon, E(y ), whch s a p-dmensonal problem and dffcult to estmate when p s large. SIR s based on the nverse regresson functon, η ( y) = E( y) (7) whch conssts of p one-dmensonal regressons and s easer to deal wth. The SIR drectons v can be obtaned as the soluton of the followng optmzaton problem, v K v' Cov( E( y)) v = arg max (8) v' v=1 v' S v subject to the constrant v ' S v j = 0, for all 1 < j. Algebracally, the SIR components t =v (=1,,K) are lnear combnatons of the p orgnal predctor varables defned by the weghtng vectors v. Geometrcally, SIR projects the data Produced by The Berkeley Electronc Press, 2006

8 6 Statstcal Applcatons n Genetcs and Molecular Bology Vol. 5 [2006], No. 1, Artcle 6 from the hgh p-dmensonal space to a much lower K-dmensonal space spanned by the projecton vectors v. The projecton vectors v are derved n such a way that the frst a few represent drectons wth maxmum varablty between the response varable and the SIR components. Computaton of v s straghtforward. Let Sη = Cov(E( y)) be the covarance matrx of the nverse regresson functon defned n (7) and recall that S s the varance-covarance matrx of. The vectors v (=1,,K) can be obtaned by spectral decomposton of Sη wth respect tos, S = η v λs v (9) where λ s the -th egenvalue n descendng order for =1,,K; v s the correspondng egenvector, and v ' S v j = 1. SIR s mplemented by approprate dscretzaton of the response. Let T(y) be a dscretzaton of the range of y. SIR computes Cov(E( T(y))), the covarance matrx for the slce means of, whch can be thought of as the between covarance for the subpopulatons of defned by T(y). Usually, f the response s contnuous, one dvdes ts range nto H slces. If the response s categorcal, one smply consders ts categores. In class predcton problems, the number of classes G s a natural choce for H,.e. H=G. The maxmum number of SIR components s H mnus one,.e. K mn(h-1,n,p). As dscussed before, K s consdered to be a meta-parameter and may be estmated by cross-valdaton. The cost of computng SIR drectons usng the standard algorthm s O(np 2 + p 3 ), whch s qute expensve comparng to the cost of PLS. We used a standard SIR algorthm (Härdle et al., 1995) n ths study Class Predcton: Logstc Dscrmnaton After dmenson reducton, standard statstcal models can be used for class predcton based on a small number of new predctors. The class predcton model we use for ths study s the logstc dscrmnaton (LD). Ths model has been wdely used for two-calss predcton problems and has been shown to perform well n prevous studes (e.g. Nguyen and Rocke, 2002a). A number of statstcal models can be used for the purpose (e.g. Dudot et al., 2002). To descrbe the model, let Z be the n by K matrx of predctor values (gene components) and y be the vector of bnary responses (class labels), for example, y =1 for tumor type A, and y=0 for tumor type B. We want to predct the

9 Da et al.: Dmenson Reducton for Class Predcton 7 probablty that the -th tssue sample s of tumor type A gven the gene expresson profle Z π = P( y = 1/ Z ) (10) and then use the probablty to classfy the sample. In LD, ths probablty s computed usng the logstc functon (Hosmer and Lemeshow, 2000) exp( Zβ) π = (11) 1 + exp( Z β) where β s a vector of coeffcents n the logstc regresson, whose values can be estmated by the method of maxmum lkelhood estmaton (MLE). The predcted probabltes πˆ are computed by replacng β wth the MLE estmates βˆ. These probabltes are then used to classfy each of the samples, (for =1,,n), yˆ = 1 ( ˆ π > 1 ˆ π ), where 1(.) s the ndcator functon. 1(A)=1 f condton A s true and 1(A)=0 otherwse. The classfcaton rule s smple: a tumor sample s classfed as type A ( y ˆ = 1 ) f the predcted probablty that the sample s of type A s greater than the probablty that the sample s of type B; otherwse, the sample s classfed as type B Methods of Assessment Wth dmenson reducton/logstc dscrmnaton one can predct the response classes usng gene expresson data. The observed error rates can be used to compare the accuracy of the classfers. Several ssues need to be addressed n desgnng a procedure for the assessment. Frst, the procedure must provde protecton aganst over-fttng the data. Cross-valdaton and re-randomzaton studes can be used for ths purpose. Second, due to repeatedly fttng hgh dmensonal data, the assessment studes can be very tme-consumng. It would be useful to add a step of gene selecton. Thrd, the number of gene components to be retaned s a meta-parameter n the procedure and ts value must be estmated. We now dscuss these methods and desgn a fve-step procedure for evaluatng the relatve performance of the classfcaton procedures Cross-valdaton and re-randomzaton study Produced by The Berkeley Electronc Press, 2006

10 8 Statstcal Applcatons n Genetcs and Molecular Bology Vol. 5 [2006], No. 1, Artcle 6 Cross-valdaton of predcton results can be acheved by leavng out part of the data, tranng a predcton rule on the remanng data (the tranng set), and predctng response values usng the left-out data (the test set) (Stone, 1974). The predcton errors are used to evaluate the predcton accuracy of a model. Leaveone-out (LOO) cross-valdaton s often used when the number of samples n the data s relatvely small. By ths method, one of the samples s left out and a model s ftted based on all but the left-out sample. The ftted model s then used to predct the left-out sample. Ths s repeated for all samples. The error rate estmated through cross-valdaton s unbased. It s mportant to treat dmenson reducton as a step n buldng the predcton rule and therefore subject to crossvaldaton. We use cross-valdaton to choose the number of gene components (estmate the value of K). Cross-valdaton provdes some protecton aganst overfttng the data, yet t may not be suffcent, because relatvely small cross-valdated errors can be acheved by captalzng on chance propertes. A further step to protect aganst overfttng s to do re-randomzaton studes. That s to re-randomze the entre data and then repeat the modelng and valdaton steps. Re-randomzaton studes help stablze predcton errors Selecton of gene subset Although dmenson reducton va PLS, SIR or PCA can handle a large number of genes, t s useful to nclude gene subset selecton as part of the procedure. Frst, the assessment studes requre fttng the data many tmes due to cross-valdaton and re-randomzatons. A vary large p (number of genes) can be an mpedment to the studes due to large computatonal tme and other challenges. A usual approach to ths s to select a subset of genes and use the subset for model comparsons. Second, t s often the case that only a subset of genes s of nterest n practce. Thus, we nclude gene subset selecton nto the procedure and use subsets of genes n the assessment studes. There are dfferent methods for subset selecton and each has ts own lmtatons (Parmgan et al., 2003). The smplest and fastest one s to form random subsets, each consstng of p* (p* < p) genes from the set of all genes. Ths can be done by random partton of the whole gene set or smple random samplng. The use of random subsets works for the purpose of ths study,.e. assessng the relatve performance of the models, whch doesn t requre a gene subset to be optmal. In other words, regardless of whether a subset contans good or bad (more or less predctve) genes, all models wll be appled to the same subset of genes, and thus the comparson of model performance s vald.

11 Da et al.: Dmenson Reducton for Class Predcton 9 A subset of genes can also be selected based on measures related to the classfcaton. Use of the class nformaton n gene selecton can help select genes whose expressons are more correlated to the response and thus mprove predcton accuracy of the models. Most gene selecton methods use some unvarate measures related to classfcaton. For a two-class applcaton, gene selecton can be based on the smple t-statstc (Nguyen and Rocke, 2002a): t = s 2 1 x 1 x 1 2 / n + s / n where n k, x k and 2 sk are the sze, mean and varance of class k, k=1,2. Usng ths method, t-scores are computed for all genes and the top p* genes wth the best scores are retaned. We use both random subset selecton and the t-score based gene selecton n the assessment studes Selectng the number of components The number of components (K) s a meta parameter n the procedure. It can be estmated by cross-valdaton (CV) on the learnng set usng leave-one-out (LOO) or leave percentage out procedures. The leave-one-out valdaton procedure s as follows: one of the samples n the learnng set s left-out, and a subset of genes (p* genes, p* < p) s selected. The models are ftted to all but the left-out sample. The ftted models are then used to predct the left-out sample. Ths s repeated for all samples n the learnng data set wth K takng successvely dfferent values. The predcted resdual sum of squares (PRESS) s computed for each value of K, and the one that mnmzes PRESS s chosen and denoted as K*. In our studes, the K values are set to be 1,2,3,4,5, whch seems to be a good balance between computaton tme and estmaton accuracy for bnary classfcaton (Boulestex, 2004; Nguyen and Rocke, 2002b) Assessment procedure The assessment procedure conssts of the followng steps: 1. Form a learnng set L wth n L samples and a test set T wth n T samples (n L +n T =n). Denote L as the learnng data matrx of sze n L by p, and T as the test data matrx of sze n T by p. Use the learnng set to determne the number of gene components, K*, by cross-valdaton (See 2.3.3). Produced by The Berkeley Electronc Press, 2006

12 10 Statstcal Applcatons n Genetcs and Molecular Bology Vol. 5 [2006], No. 1, Artcle 6 2. Select a subset of p* genes from the set of all genes usng one of the gene selecton methods, resultng n * L (n L by p* matrx) and * T ( n T by p* matrx). 3. Perform dmenson reducton usng PLS, SIR, or PCA. Let W denote the p* by K* matrx contanng the projecton vectors. Compute the matrx Z L of gene components for the learnng data set: Z L = * L W, and the gene components for the test data set: Z T = * T W. 4. Ft the class predcton model (logstc regresson) to the learnng components Z L. Predct the classes of samples n the test set usng the ftted classfer and the test components, Z T. 5. Repeat all above steps R tmes wth re-randomzatons of the whole data set. The total class predcton error (TCPE) for each method s computed by TCPE = R n T r= 1 = 1 1 ( y yˆ ) where y s the observed response class, ŷ s the predcted response class, 1() s an ndcator functon, n T s the number of test samples, and R s the number of re-randomzaton studes. The error rate (proporton of msclassfcaton) s computed by TCPE/(n T R) based on the test data only. 3. Results In ths secton, we present the results of evaluatons of the relatve performance of three classfcaton procedures: PLSLD, SIRLD and PCALD, whch combne PLS, SIR, PCA wth logstc dscrmnaton (LD). The assessment studes are based on two mcroarray data sets: the leukema data set of Golub et al. (1999) and the colon cancer data set of Alon et al. (1999). Both data sets are from Affymetrx hgh densty olgonucleotde mcroarrays and are publcly avalable. The leukema data set contans 72 tssue samples on 7129 genes; the colon data set has 62 tssue samples on 2000 genes. We mplemented the fve-step assessment procedure n the R software envronment (Ihaka and Gentleman,

13 Da et al.: Dmenson Reducton for Class Predcton ) and then appled t to each of the data sets. We descrbe the leukema data set frst. The acute leukema data contan 72 bone marrow samples on 7129 genes from patents wth ether acute lymphoblastc leukema (ALL) or acute myelod leukema (AML). The orgnal data consst of a tranng set of 38 samples wth 27 ALL and 11 AML and a test set of 34 samples wth 20 ALL and 14 AML. We preprocessed the gene expresson data usng the standard procedure ncludng background correcton, transformaton and normalzaton (Kerr, 2000; Rocke and Durbn, 2001). In data transformaton, we appled the generalzed logarthm 2 (glog), ln( x + x + λ ), where x s an ntensty value (background corrected) and λ s a transformaton parameter whose value can be estmated usng the method of maxmum lkelhood (Durbn et al., 2002; Durbn and Rocke, 2003; Huber et al., 2002; Munson, 2001). The glog transformaton s a generalzaton of and mprovement over the log transformaton as the latter can nflate the varance of the expresson values near background. After data preprocessng, we appled the 5-step assessment procedure on the data set. We consdered p* = 200, 500, 1000 genes and used 100 random subsets (R=100) for each p*. We randomly splt each subset of genes nto two data sets: a tranng set wth 36 samples (n L =36) and a test set wth 36 samples (n T =36). We used leave-one-out cross-valdaton on the tranng set to determne the number of gene components Kˆ, and the test set for evaluatng predcton errors. In total, 3600 class predctons (36*100=3600) were made usng each of the three classfers based on 100 random subsets. The predcton error rates of the classfers were computed. Table 1. Classfcaton error rates of the three methods on the leukema data set wth 36/36 splt of tssue samples averaged over 100 randomzaton studes. p* PLSLD SIRLD PCALD (0.051) (2.1) (0.051) (1.0) (0.069) (3.4) (0.045) (1.8) (0.041) (1.0) (0.065) (3.6) (0.033) (1.9) (0.032) (1.0) (0.055) (3.7) Produced by The Berkeley Electronc Press, 2006

14 12 Statstcal Applcatons n Genetcs and Molecular Bology Vol. 5 [2006], No. 1, Artcle 6 The estmated error rates are presented n Table 1. The standard devaton of an error rate s shown n the frst parentheses and the average value of the estmated meta parameter (K*) s n the second parentheses. It can be seen from the table that the error rates decrease wth the ncrease of sze of gene subset (p*). At any of the subset szes, the error rates of PLS and SIR based procedures (PLSLD and SIRLD) are smlar and they are lower than that of the PCA based procedure (PCALD). It suggests that PLS and SIR are about equally effectve comparng to each other and are more effectve than PCA n dmenson reducton. To compare the relatve computatonal cost, we computed the ratos of computatonal tme of PLSLD and SIRLD to that of PLSLD. The ratos are 7.8 for PCALD/PLSLD and 30.8 for SIRLD/PLSLD wth p*=500. It s clear that the PLS based procedure s much faster than those based on SIR or PCA. The second data set used n ths study s the colon data, whch consst of gene expressons of 2000 human genes wth 62 colon tssue samples (40 tumor and 22 normal). In pre-processng the data, we dd background correcton, glog transformaton and normalzaton. For assessment, we consdered p*=200, 500 and 1000 genes and generated 100 random subsets for each p*. Each subset was randomly parttoned nto two parts: a tranng set wth 36 samples (n L =36) and a test set wth 26 samples (n T =26). The tranng set was used for dmenson reducton and model selecton by leave-one-out cross-valdaton, and the test set was used for predcton. In total, 2600 (26*100) class predctons were made usng each of the three classfers and msclassfcaton rates were computed. The error rates and ther standard devatons, based on 100 randomzaton studes, are reported n Table 2. It can be seen from the table that the classes n the colon data are less well separated than the leukema data, as noted n the prevous studes (Nguyen and Rocke, 2002a; Antonads et al., 2003). Table 2. Classfcaton error rates of the three methods on the colon data set wth 36/26 splt of tssue samples averaged over 100 randomzaton studes. p* PLSLD SIRLD PCALD (0.055) (2.2) (0.054) (1.0) (0.096) (3.5) (0.063) (2.1) (0.046) (1.0) (0.086) (3.9) (0.057) (2.4) (0.040) (1.0) (0.087) (3.7)

15 Da et al.: Dmenson Reducton for Class Predcton 13 The pattern of performance of the methods on the colon data s smlar to that on the leukema data. It s observed from Table 2 that the average error rates of PLSLD and SIRLD are consstently lower than that of PCALD. It s also noted that PLSLD and SIRLD have smlar error rates although the msclassfcaton rate of SIRLD seems to be a lttle hgher than that of PLSLD. In terms of relatve computatonal cost, the results are also comparable to those from the leukema data. The ratos of computaton tme are 7.2 for PCALD/PLSLD and 27.2 for SIRLD/PLSLD as p*=500. Agan, PLS dmenson reducton s much faster than the other two. The results presented so far for both the leukema data and the colon data are based on the studes usng random subsets of genes. Next we descrbe the results of evaluatons usng subsets of genes selected based on the t-scores, a unvarate measure related to the classfcaton. Use of ths supervsed gene selecton method should mprove the accuracy of the classfcaton procedures. As descrbed n Secton 2.3.2, we computed t-scores for all genes n the data sets, ranked the genes by the scores, and selected top p* genes. For both the leukema and the colon data sets, we used p*=1000 genes for llustraton. We performed gene selecton and dmenson reducton wthn cross-valdaton usng the tranng set only and estmated the error rates usng the test set. The results on both data sets are presented n Table 3 below. Shown n the table are the estmated error rates, ther standard devatons and the average values of estmated K* based on 100 re-randomzatons studes. Table 3. Classfcaton error rates of the three methods on the leukema data set wth 36/36 splt and the colon data set wth 36/26 splt, averaged over 100 rerandomzaton studes. p*=1000 top genes selected usng the t-statstc. Data Set PLSLD SIRLD PCALD Leukema (0.021) (1.7) (0.020) (1.0) (0.025) (2.1) Colon (0.045) (2.2) (0.038) (1.0) (0.069) (2.8) Produced by The Berkeley Electronc Press, 2006

16 14 Statstcal Applcatons n Genetcs and Molecular Bology Vol. 5 [2006], No. 1, Artcle 6 Comparng the results n Table 3 wth those n Table 1 and Table 2, one can see that the predcton accuracy of all three methods (PLSLD, SIRLD and PCALD) has been mproved: the average predcton errors of all three methods are reduced and the mean squared errors of predcton are also decreased. The relatve performance of the methods, however, remans bascally the same: PLSLD and SIRLD have smlar msclassfcaton rates and both have done better than PCALD. 4. Dscusson An mportant applcaton of mcroarray data s to classfy bologcal samples or predct clncal or other outcomes. In ths paper, we vewed the class predcton problem as a multvarate regresson problem where the number of varables far exceeds the number of samples, and evaluated several classfcaton procedures for dealng wth the problem. Specfcally, we compared three dmenson reducton methods (PLS, SIR, PCA), examned the relatve performance of classfcaton procedures ncorporatng those methods, and desgned a fve-step procedure for assessment studes. The emprcal analyses were based on two publshed gene expresson data sets. We found that PLS and SIR were both effectve n dmenson reducton and they were more effectve than PCA. The PLS and SIR based classfcaton procedures performed consstently better than the PCA based procedure n predcton accuracy. The emprcal results are consstent wth the analyss of the technques. PLS and SIR construct new predctors usng nformaton on the response varable whle PCA does not; thus PLS and SIR components are more lkely to be good predctors than those from PCA. For smlar reason, the use of supervsed gene selecton methods would be lkely to mprove the classfcaton accuracy. We showed that a smple t-score based gene selecton method worked well for two-class problems. In the study, we also evaluated the computatonal effcency of the three dmenson reducton methods and found that PLS had sgnfcant advantage over the other two. Consderng both predctve accuracy and computatonal effcency, we conclude that the PLS based procedure has provded the best performance among the three classfcaton procedures. Dmenson reducton s a necessary part of multvarate analyss of hghthroughput assay data such as gene expresson data. Dmenson reducton methods are frequently used but ther relatve performance has not been well studed. It would be dffcult to compare the performance of dmenson reducton methods based on results of publshed studes due to dfferences among the studes n data sets, data preprocessng, and methods of gene selecton, model selecton and valdaton. Ths study provdes a systematc comparson of the three

17 Da et al.: Dmenson Reducton for Class Predcton 15 dmenson reducton methods. Moreover, the assessment procedure developed n ths study can be easly extended to nclude more methods nto evaluaton. The scope of the study s however qute lmted. Many methods of gene selecton/dmenson reducton are avalable. In further studes, we ntend to contnue the nvestgaton and nclude more methods nto our evaluatons. References Alon, U., Barka, N., Notterman, D.A., Gsh, K., Ybarra, S., Mack, D. and Levne, A.J. (1999) Broad patterns of gene expresson revealed by clusterng analyss of tumor and normal colon tssues probed by olgonucleotde arrays. Proc. Natl Acad. Sc. USA, 96, Antonads, A., Lambert-Lacrox, S. and Leblanc, F. (2003) Effectve dmenson reducton methods for tumor classfcaton usng gene expresson data. Bonformatcs, 19, Boulestex, A. (2004) PLS Dmenson reducton for classfcaton wth mcroarray data. Statstcal applcatons n genetcs and molecular bology, 3, Bura, E. and Pfeffer, R.M. (2003) Graphcal methods for class predcton usng dmenson reducton technques on DNA mcroarray data. Bonformatcs, 19, Charomonte, F. and Martnell, J. (2002) Dmenson reducton strateges for analyzng global gene expresson data wth a response. Mathematcal Boscences, 176, Cook, R.D. (1998) Regresson Graphcs. John Wley & Sons, New York. Denham, M.C. (1995) Implementng partal least squares. Statstcs and Computng, 5, Dettlng, M. and Buhlmann, P. (2003) Boostng for tumor classfcaton wth gene expresson data. Bonformatcs, 19, Duan, N. and L, K.C. (1991) Slcng regresson: a lnk-free regresson method. The Annals of Statstcs, 19, Produced by The Berkeley Electronc Press, 2006

18 16 Statstcal Applcatons n Genetcs and Molecular Bology Vol. 5 [2006], No. 1, Artcle 6 Dudot, S., Frdlyard, J. and Speed, T.P. (2002) Comparson of dscrmnaton methods for the classfcaton of tumors usng gene expresson data. Journal of the Amercan Statstcal Assocaton, 97, Dudot, S., Shaffer, J.P. and Boldrck, J.C. (2003) Multple hypothess testng n mcroarray experments. Statstcal Scence, 18, Durbn, B., Hardn, J., Hawkns, D.M., and Rocke, D.M. (2002) A varancestablzng transformaton for gene-expresson mcroarray data. Bonformatcs, 18, 105S-110S. Durbn, B. and Rocke, D.M. (2003) Estmaton of transformaton parameters for mcroarray data. Bonformatcs, 19, Frank, I.E. and Fredman, J.H. (1993) A statstcal vew of some chemometrcs regresson tools (wth dscusson). Technometrcs, 35, Garthwate, P.H. (1994) An nterpretaton of partal least squares. Journal of Amercan Statstcal Assocaton, 89, Ghosh, D. (2002) Sngular value decomposton regresson modelng for classfcaton of tumors from mcroarray experments. Proceedngs of the Pacfc Symposum on Bocomputng, Golub, T.R., Slonm, D.K., Tamayo, P., Huard,C., Gaasenbeek, M., Mesrov, P., Coller, H., Loh, M.L., Downng, J.R., Calgur, M.A., Bloomfeld, C.D. and Lander, E.S. (1999) Molecular classfcaton of cancer: class dscovery and class predcton by gene expresson montorng. Scence, 286, Gruber, M.H.J. (1998) Improvng Effcency by Shrnkage. Statstcs: textbooks and monographs, volume 156. Marcel Dekker, Inc, New York. Härdle, W., Klnke, S. and Turlach, B.A. (1995). plore: an Interactve Statstcal Computng Envronment, Sprnger-Verlag, New York. Hawkns, D.M. and Yn,. (2002) A faster algorthm for rdge regresson of reduced rank data. Computatonal Statstcs & Data Analyss, 40, Hedenfalk, I., Duggan, D., Chen, Y., Radmacher, M., Bttner, M., Smon, R., Meltzer, P., Gusterson, B., Esteller, M., Raffeld, M., Yakhn, Z., Ben-Dor, A., Dougherty, E., Kononen, J., Bubendorf, L., Fehrle, W., Pttaluga, S.,

19 Da et al.: Dmenson Reducton for Class Predcton 17 Gruvberger, D., Loman, N., Johannsson, O., Olsson, H., Wlfond, B, Sauter, G., Kallonem, O., Borg, A. and Trent, J. (2001) Gene expresson profles n heredtary breast cancer. New England Journal of Medcne, 244, Helland, I.S. (1988) On the structure of partal least squares. Communcatons n Statstcs: Smulaton and Computaton, 17, Helland, I.S. (1990) Partal least squares regresson and statstcal models. Scandnavan Journal of Statstcs, 17, Hoerl, A.E. and Kennard, R.W. (1970) Rdge regresson: based estmaton for non-orthogonal problems. Technometrcs, 8, Hosmer, D.W. and Lemeshow, S. (2000) Appled Logstc Regresson. Wley, New York. Huang,. and Pan, W. (2003) Lnear regresson and two-class classfcaton wth gene expresson data. Bonformatcs, 19, Huber, W., von Heydebreck, A., Sultmann, H., Poustka, A., and Vngron, M. (2002) Varance stablzaton appled to mcroarray data calbraton and to the quantfcaton of dfferental expresson. Bonformatcs, 18, 96S-104S. Ihaka, R. and Gentleman, R. (1996) R: A language for data analyss and graphcs. Journal of Computatonal and Graphcal Statstcs, 5, Jollffe, I.T. (1986). Prncpal Component Analyss. Sprnger, New York. Kerr, K., Martn, M., and Churchll, G. (2000) Analyss of varance for gene expresson mcroarray data. Journal of Computatonal Bology, 7, Kers, H.A.L. (1997) Dscrmnaton by means of components that are orthogonal n the data space. Journal of Chemometrcs, 11, Krzanowsk, W.J. (1995) Orthogonal canoncal varates for dscrmnaton and classfcaton. Journal of Chemometrcs, 9, L, K.C. (1991) Slced nverse regresson for dmenson reducton. Journal of Amercan Statstcal Assocaton, 86, Produced by The Berkeley Electronc Press, 2006

20 18 Statstcal Applcatons n Genetcs and Molecular Bology Vol. 5 [2006], No. 1, Artcle 6 L, K.C. (2000) Hgh dmensonal data analyss va the SIR/PHS approach. Unpublshed manuscrpt dated Aprl 6, 2000 obtaned at the Internet ste Martens, H. and Naes, T. (1989) Multvarate Calbraton. Wley, New York. Massey, W.F. (1965) Prncpal components regresson n exploratory statstcal research. Journal of Amercan Statstcal Assocaton, 60, Munson, P. (2001) A consstency test for determnng the sgnfcance of gene expresson changes on replcate samples and two convenent varancestablzng transformatons. GeneLogc Workshop of Low Level Analyss of Affymetrx GeneChp Data. Neter, J., Kutner, M.H., Nachtshem, C.J. and Wasserman, W. (1996) Appled lnear statstcal models, 4 th edton, McGraw-Hll, New York. Nguyen, D.V. and Rocke, D.M. (2002a) Tumor classfcaton by partal least squares usng mcroarray gene expresson data. Bonformatcs, 18, Nguyen, D.V. and Rocke, D.M. (2002b) Mult-class cancer classfcaton va partal least squares wth gene expresson profles. Bonformatcs, 18, Parmgan, G., Garrett, E., Irzarry, R. and Zeger, S. (2003) The Analyss of Gene Expresson Data: Methods and Software, Sprnger, New York. Rocke, D.M. and Durbn, B.P. (2001) A model for measurement errors for gene expresson arrays. Journal of Computatonal Bology, 8, Speed, T. (2003) (eds.) Statstcal Analyss of Gene Expresson Mcroarray Data, Chapman & Hall/CRC, New York. Stone, M. (1974). Cross-valdaton choce and assessment of statstcal predctons. Journal of the Royal Statstcal Socety, Seres B, 36, Stone, M. and Brooks, R.J. (1990) Contnuum regresson: cross-valdated sequentally constructed predcton embracng ordnary least squares, partal least squares and prncpal components regresson. Journal of Royal Statstcal Socety, Seres B, 52,

21 Da et al.: Dmenson Reducton for Class Predcton 19 a, Y., Tong, H., L, W.K. and, Z.L. (2002) An adaptve estmaton of dmenson reducton space. J. R. Statst. Soc. B., 64, West, M., Blanchette, C., Fressman, H., Huang, E., Ishda, S., Spang, R., Zuan, H., Marks, J.R. and Nevns, J.R. (2001). Predctng the clncal status of human breast cancer usng gene expresson profles. Proceedngs of the Natonal Academy of Scences of the Unted States, 98, Wold, H. (1966) Nonlnear estmaton by teratve least squares procedures. In Research Papers n Statstcs, ed. F.N. Davd, pp Wley, New York. Wold, H. (1973) Nonlnear teratve partal least squares (NIPALS) modelng: some recent developments. In Multvarate Analyss III, ed. P. Krshnaah, pp , Academc Press, New York. Wold, H. (1982) Soft modelng: the basc desgn and some extensons. In Systems under Indrect Observaton: Causalty-Structure-Predcton, ed. K. G. Joreskog and H. Wold, Vol. II, Ch. 1, pp. 1-54, North-Holland, Amsterdam. Produced by The Berkeley Electronc Press, 2006