Data Mining in Bioinformatics Day 6: Classification in Bioinformatics

Transcription

1 Data Mining in Bioinformatics Day 6: Classification in Bioinformatics Karsten Borgwardt February 25 to March 10 Bioinformatics Group MPIs Tübingen Karsten Borgwardt: Data Mining in Bioinformatics, Page 1

2 Karsten M. Borgwardt Protein function prediction via graph kernels ISMB 2005 Joint work with Cheng Soon Ong and S.V.N. Vishwanathan, Stefan Schönauer, Hans-Peter Kriegel and Alex Smola Ludwig-Maximilians-Universität Munich, Germany and National ICT Australia, Canberra

3 Content Introduction The problem: protein function prediction The method: Support Vector Machines (SVM) Our approach to function prediction Protein graph model Protein graph kernel Experimental evaluation Technique to analyze our graph model Hyperkernels Discussion Karsten Borgwardt et al. - Protein function prediction via graph kernels 2

4 Current approaches to protein function prediction similar structures similar phylogenetic profiles similar motifs similar interaction partners similar function similar surface clefts similar sequences similar chemical properties Karsten Borgwardt et al. - Protein function prediction via graph kernels 3

5 Current approaches to protein function prediction similar structures similar phylogenetic profiles similar motifs similar interaction partners similar function similar sequences similar chemical properties similar surface clefts Karsten Borgwardt et al. - Protein function prediction via graph kernels 4

6 Support Vector Machines Are new data points (x) red or black? The blue decision boundary allows to predict class membership of new data points. Karsten Borgwardt et al. - Protein function prediction via graph kernels 5

7 Kernel trick input space feature space mapping Ф kernel function The kernel trick allows to introduce a separating hyperplane in feature space. Karsten Borgwardt et al. - Protein function prediction via graph kernels 6

8 Feature vectors for function prediction protein structure and/or protein sequence e.g. Cai et al. (2004), Dobson and Doig (2003) hydrophobicity polarity polarizability van der Waals volume fraction of amino acid types fraction of surface area disulphide bonds size of largest surface pocket 7

9 Our approach Sequence + Structure + Chemical properties Graph model SVMs + Graph models Protein function Karsten Borgwardt et al. - Protein function prediction via graph kernels 8

10 Protein graph model protein secondary structure sequence structure Karsten Borgwardt et al. - Protein function prediction via graph kernels 9

11 Protein graph model Node attributes hydrophobicity polarity polarizability van der Waals volume length helix, sheet, loop Edge attributes type (sequence, structure) length Karsten Borgwardt et al. - Protein function prediction via graph kernels 10

12 Protein graph kernel (Kashima et al. (2003) and Gärtner et al. (2003)) compares walks of identical length l l 1 k walk v 1,...,v l, w 1,..., w l = i =1 Walks are similar, if along both walks types of secondary structure elements (SSEs) are the same distances between SSEs are similar chemical properties of SSEs are similar k step v i,v i 1, w i, w i 1 11

13 Example: Protein kernel Protein A S S S S Protein B S Similar (H,10,S,1,S,3,H) (H,9,S,1,S,3,H) 12

14 Example: Protein kernel Protein A S S S S Protein B S Dissimilar (H,10,S,1,S) (S,3,H,5,S) 13

15 Evaluation: enzymes vs. non-enzymes 10-fold cross-validation on 1128 proteins from dataset by Dobson and Doig (2003); 59 % are enzymes. Kernel type accuracy SD Vector kernel Optimized vector kernel Graph kernel Graph kernel without structure Graph kernel with global info DALI classifier Karsten Borgwardt et al. - Protein function prediction via graph kernels 14

16 Attribute selection Which structural or chemical attribute is most important for correct classification? For this purpose, we employ hyperkernels (Ong et. al, 2003). Hyperkernels find an optimal linear combination of input kernel matrices : m i=1 β i K i minimizing training error and fulfilling regularization constraints Karsten Borgwardt et al. - Protein function prediction via graph kernels 15

17 Our approach: Attribute selection Calculate kernel matrix for 600 proteins on graph model with only ONE single attribute! Repeat this for all attributes Normalize these kernel matrices Determine hyperkernel combination Weights then reflect contribution of individual attributes to correct classification 16

18 Attribute selection Attribute EC 1 EC 2 EC 3 EC 4 EC 5 EC 6 Amino acid length bin van der Waals 3-bin Hydrophobicity 3-bin Polarity bin Polarizability d length 0.40 Total van der Waals Total Hydrophobicity Total Polarity Total Polarizability Karsten Borgwardt et al. - Protein function prediction via graph kernels 17

19 Discussion Novel combined approach to protein function prediction integrating sequence, structure and chemical information Reaches state-of-the-art classification accuracy on less information; higher accuracy levels on same amount of information Hyperkernels for finding most interesting protein characteristics Karsten Borgwardt et al. - Protein function prediction via graph kernels 18

20 Discussion More detailed graph models (amino acids, atoms) might be more interesting, yet raise computational difficulties (graphs too large!) Two directions of future research: Efficient, yet expressive graph kernels for structure Integrating more proteomic information, e.g. surface pockets, into our graph model Karsten Borgwardt et al. - Protein function prediction via graph kernels 19

21 The End Thank you! Questions? Karsten Borgwardt et al. - Protein function prediction via graph kernels 20

22 ARTS: Accurate Recognition of Transcription Starts in human Sören Sonnenburg, Alexander Zien,, Gunnar Rätsch Fraunhofer FIRST.IDA, Kekuléstr. 7, Berlin, Germany Friedrich Miescher Laboratory of the Max Planck Society, Max Planck Institute for Biological Cybernetics, Spemannstr , Tübingen, Germany

23 Promoter Detection Overview: Transcription Start Site (TSS) Features to describe the TSS Our approach Evaluation with current methods Example - Protocadherin-α Summary Sonnenburg, Zien, Rätsch 1

24 Promoter Detection Transcription Start Site - Properties POL II binds to a rather vague region of [ 20,+20] bp Upstream of TSS: promoter containing transcription factor binding sites Downstream of TSS: 5 UTR, and further downstream coding regions and introns (different statistics) 3D structure of the promoter must allow the transcription factors to bind Promoter Prediction is non-trivial Sonnenburg, Zien, Rätsch 2

25 Promoter Detection Features to describe the TSS TFBS in Promoter region condition: DNA should not be too twisted CpG islands (often over TSS/first exon; in most, but not all promoters) TSS with TATA box ( 30 bp upstream) Exon content in UTR 5 region Distance to first donor splice site Idea: Combine weak features to build strong promoter predictor Sonnenburg, Zien, Rätsch 3

26 Promoter Detection The ARTS Approach use SVM classifier ( Ns ) f(x) = sign y i α i k(x,x i ) + b i=1 key ingredient is kernel k(x,x ) similarity of two sequences use 5 sub-kernels suited to model the aforementioned features k(x, x ) = k TSS (x, x )+k CpG (x, x )+k coding (x, x )+k energy (x, x )+k twist (x, x ) Sonnenburg, Zien, Rätsch 4

27 Promoter Detection The 5 sub-kernels 1. TSS signal (including parts of core promoter with TATA box) use Weighted Degree Shift kernel 2. CpG Islands, distant enhancers and TFBS upstream of TSS use Spectrum kernel (large window upstream of TSS) 3. Model coding sequence TFBS downstream of TSS use another Spectrum kernel (small window downstream of TSS) 4. Stacking energy of DNA use btwist energy of dinucleotides with Linear kernel 5. Twistedness of DNA use btwist angle of dinucleotides with Linear kernel Sonnenburg, Zien, Rätsch 5

28 Promoter Detection Weighted Degree Shift Kernel x 1 k(x1,x2) = w6,3 + w6,-3 + w3,4 x 2 Count matching substrings of length 1...d Weight according to length of the match β 1...β d Position dependent but tolerates shifts of up to S k(x,x ) = d k=1 L k+1 β k l=1 S s=0 s+l L δ s (I(x[k : l + s]=x [k : l])+i(x[k : l]=x [k : l + s])) x[k : l] := subsequence of x of length k starting at position l Sonnenburg, Zien, Rätsch 6

29 Promoter Detection Training Data Generation True TSS: From dbtssv4 (based on hg16) extract putative TSS windows of size [ 1000, +1000] Decoy TSS: Annotate dbtssv4 with transcription-stop (via BLAT alignment of mrnas) From the interior of the gene (+100bp to gene end) sample negatives for training (10 per positive), again windows [ 1000,+1000] Processing: 8508 positive, negative examples Split into disjoint training and validation set (50% : 50%) Sonnenburg, Zien, Rätsch 7

30 Promoter Detection Training Model Selection 16 kernel parameters + SVM regularization to be tuned! Full grid search infeasible Local axis-parallel searches instead SVM training/evaluation on > 10, 000 examples computationally too demanding Speedup trick: f(x) = N s i=1 α i k(x i, x) + b = N s i=1 α i Φ(x i ) Φ(x) + b = w Φ(x) + b } {{ } w f(x) before: O(N s dls) now: = O(dL) speedup factor up to N s S Large Scale Training and Evaluation possible Sonnenburg, Zien, Rätsch 8

31 Promoter Detection Comparison Current state-of-the-art methods: FirstEF [Davuluri, Grosse, Zhang; 2001, Nat Genet] QDF: for promoter, donor, first exon, WM Range: [ 1500, +500] McPromoter [Ohler, Liao, Niemann, Rubin; 2002, Genome Biol] GHMM with IMC for 6 regions (e.g. upstream, TATA) NN Range: [ 250, +50] Eponine [Down, Hubbard; 2002 Genome Res] RVM: WM with positional distribution for 4 regions (e.g. TATA, CpG) Range: [ 200, +200] Do a genome wide evaluation! How to do a fair comparison? Sonnenburg, Zien, Rätsch 9

32 Promoter Detection Evaluation Idea: Only consider new TSS from dbtssv5-dbtssv4, with max 30% overlap 1. Compute genome wide outputs for each TSF 2. Decrease resolution: divide genome into non-overlapping fixed size chunks (e.g. 50 or 500) 3. Annotate dbtssv5 TSS with gene end 4. Label chunk positive if intersects with [T SS 20bp, T SS + 20bp] 5. Label chunk negative [T SS + 21bp, GeneEnd] Sonnenburg, Zien, Rätsch 10

33 Promoter Detection Results Receiver Operator Characteristic Curve and Precision Recall Curve 35% true positives at a false positive rate of 1/1000 (best other method find about a half (18%)) Sonnenburg, Zien, Rätsch 11

34 5.5 Promoter Detection What does ARTS do better? Entropy and Relative Entropy entropy auroc: 86.5% auprc: 49.8% entropy auroc: 86.5% auprc: 49.8% relative entropy auroc: 86.5% auprc: 49.8% Di-nucleotide Frequency strong discriminative signal around TSS Sonnenburg, Zien, Rätsch 12

35 Promoter Detection Which kernel captures most information? 96 using or removing single kernels area under ROC Curve (in %) TSS WD shift Promotor Spectrum 1st Exon Spectrum Angles Linear Most important Weighted Degree Shift kernel modelling the TSS signal Sonnenburg, Zien, Rätsch 13

36 Promoter Detection Alternative TSS - Protocadherin-α Sonnenburg, Zien, Rätsch 14

37 Promoter Detection Conclusion Developed a new TSF finder, ARTS In genome-wide evaluation achieves state-of-the-art results: ARTS about 35% true positives at a false positive rate of 1/1000 (best other method about a half, 18%) Reason: intensively modelling the TSS region, large scale svm training/evaluation with string kernels Future work: Drosophila, C.elegans, Zebrafish,... Poster: H56 Datasets, Genomebrowser custom track, a lot more details: Source code of SHOGUN toolbox used to train ARTS freely available: Sonnenburg, Zien, Rätsch 15

38 The end See you tomorrow! Next topic: Clustering in Bioinformatics Karsten Borgwardt: Data Mining in Bioinformatics, Page 2