Advanced Introduction to Machine Learning

Size: px

Start display at page:

Download "Advanced Introduction to Machine Learning"

Domenic Melton
6 years ago
Views:

1 Advanced Introduction to Machine Learning 10715, Fall 2014 Structured Sparsity, with application in Computational Genomics Eric Xing Lecture 3, September 15, 2014 Reading: Eric CMU,

2 Structured Sparsity =argminl(x; Y; )+Ð( ) Sparsity Ð( ) = P i j ij Group sparsity y n-1 x 1 x 2 x 3 Ð( ) = P c j G c j 2 = P c q P i2g c 2i n x n-1 Total variation sparsity Ð( ) = P c j G c j TV = P c P i2g c j i i 1 j x n Eric CMU,

3 Genetic Basis of Diseases ACTCGTACGTAGACCTAGCATTACGCAATAATGCGA ACTCGAACCTAGACCTAGCATTACGCAATAATGCGA Healthy TCTCGTACGTAGACGTAGCATTACGCAATTATCCGA ACTCGAACCTAGACCTAGCATTACGCAATTATCCGA ACTCGTACGTAGACGTAGCATAACGCAATAATGCGA TCTCGTACCTAGACGTAGCATAACGCAATAATCCGA ACTCGAACCTAGACCTAGCATAACGCAATTATCCGA Sick Single nucleotide polymorphism (SNP) Causal (or "associated") SNP Eric CMU,

g., disease/control Cancer: Dunning et al. 2009. Diabetes: Dupuis et al. 2010. Atopic dermatitis: Esparza-Gordillo et al. 2009. Arthritis: Suzuki et al.

4 Genetic Association Mapping Data Genotype Phenotype Standard Approach A.... T.. G..... C T..... A.. G. A.... A.. C..... C T..... A.. G. causal SNP T.... T.. G..... G T..... T.. C. A.... A.. C..... C T..... T.. C. A.... T.. G..... G A..... A.. G. T.... T.. C..... G A..... A.. C. A.... A.. C..... C A..... T.. C. a univariate phenotype: e.g., disease/control Cancer: Dunning et al Diabetes: Dupuis et al Atopic dermatitis: Esparza-Gordillo et al Arthritis: Suzuki et al Eric CMU,

5 Genetic Basis of Complex Diseases ACTCGTACGTAGACCTAGCATTACGCAATAATGCGA ACTCGAACCTAGACCTAGCATTACGCAATAATGCGA Healthy TCTCGTACGTAGACGTAGCATTACGCAATTATCCGA ACTCGAACCTAGACCTAGCATTACGCAATTATCCGA ACTCGTACGTAGACGTAGCATAACGCAATAATGCGA TCTCGTACCTAGACGTAGCATAACGCAATAATCCGA ACTCGAACCTAGACCTAGCATAACGCAATTATCCGA Cancer Causal SNPs Eric CMU,

6 Genetic Basis of Complex Diseases ACTCGTACGTAGACCTAGCATTACGCAATAATGCGA Healthy ACTCGAACCTAGACCTAGCATTACGCAATAATGCGA TCTCGTACGTAGACGTAGCATTACGCAATTATCCGA ACTCGAACCTAGACCTAGCATTACGCAATTATCCGA Biological mechanism ACTCGTACGTAGACGTAGCATAACGCAATAATGCGA TCTCGTACCTAGACGTAGCATAACGCAATAATCCGA ACTCGAACCTAGACCTAGCATAACGCAATTATCCGA Cancer Causal SNPs Eric CMU,

7 Genetic Basis of Complex Diseases Association to intermediate phenotypes Intermediate Phenotype Gene expression ACTCGTACGTAGACCTAGCATTACGCAATAATGCGA Healthy ACTCGAACCTAGACCTAGCATTACGCAATAATGCGA TCTCGTACGTAGACGTAGCATTACGCAATTATCCGA ACTCGAACCTAGACCTAGCATTACGCAATTATCCGA ACTCGTACGTAGACGTAGCATAACGCAATAATGCGA TCTCGTACCTAGACGTAGCATAACGCAATAATCCGA ACTCGAACCTAGACCTAGCATAACGCAATTATCCGA Causal SNPs Clinical records Cancer Eric CMU,

8 Genetic Association for Asthma Clinical Traits TCGACGTTTTACTGTACAATT Subnetworks for lung physiology Statistical challenge How to find associations to a multivariate entity in a graph? Subnetwork for quality of life Eric CMU,

9 Gene Expression Trait Analysis TCGACGTTTTACTGTACAATT Samples Genes Statistical challenge How to find associations to a multivariate entity in a tree? Eric CMU,

10 Structured Association Traditional Approach causal SNP ACGTTTTACTGTACAATT Association with Phenome ACGTTTTACTGTACAATT a univariate phenotype: gene expression level Multivariate complex syndrome (e.g., asthma) age at onset, history of eczema genome wide expression profile Eric CMU,

11 Sparse Associations Pleotropic effects Epistatic effects Eric CMU,

12 Structured Sparse Association : a New Paradigm Standard Approach Consider one phenotype at a time vs. New Approach Consider multiple correlated phenotypes (phenome) jointly Phenotypes Phenome Eric CMU,

13 Sparse Learning Linear Model: Lasso (Sparse Linear Regression) [R.Tibshirani 96] Why sparse solution? penalizing constraining Eric CMU,

14 Multi-Task Extension Multi-Task Linear Model: Input: Output: Coefficients for k-th task: Coefficient Matrix: Coefficients for a variable (2 nd ) Coefficients for a task (2 nd ) Eric CMU,

15 Outline Background: Sparse multivariate regression for disease association studies Structured association a new paradigm Association to a graph-structured phenome Graph-guided fused lasso (Kim & Xing, PLoS Genetics, 2009) Association to a tree-structured phenome Tree-guided group lasso (Kim & Xing, ICML 2010) Eric CMU,

16 Multivariate Regression for Single- Trait Association Analysis Trait Genotype T G A A C C A T G A A G T A 2.1 x = Association Strength? y = X x β Eric CMU,

17 Multivariate Regression for Single- Trait Association Analysis Trait Genotype T G A A C C A T G A A G T A 2.1 x = Association Strength Many non-zero associations: Which SNPs are truly significant? Eric CMU,

18 Lasso for Reducing False Positives (Tibshirani, 1996) Trait Genotype Association Strength 2.1 = T G A A C C A T G A A G T A x Lasso Penalty for sparsity J + β j j 1 Many zero associations (sparse results), but what if there are multiple related traits? Eric CMU,

Multivariate Regression for Multiple- Trait Association

9, 1.8) Allergy Lung physiology = T G A A C C A T G A A G T

and Trait i: β j,i + β j,i i, j How to combine information

19 Multivariate Regression for Multiple- Trait Association Analysis Genotype Association Strength (3.4, 1.5, 2.1, 0.9, 1.8) Allergy Lung physiology = T G A A C C A T G A A G T A LD Synthetic lethal x Association strength between SNP j and Trait i: β j,i + β j,i i, j How to combine information across multiple traits to increase the power? Eric CMU,

20 Multivariate Regression for Multiple- Trait Association Analysis Trait Genotype Association Strength (3.4, 1.5, 2.1, 0.9, 1.8) Allergy Lung physiology = T G A A C C A T G A A G T A x Association strength between SNP j and Trait i: β j,i + β j,i i, j + We introduce graph-guided fusion penalty Eric CMU,

Graph constrained fused lasso ACGTTTTACTGTACAATT Fusion Lasso

21 Multiple-trait Association: Graph-Constrained Fused Lasso Step 1: Thresholded correlation graph of phenotypes Step 2: Graph constrained fused lasso ACGTTTTACTGTACAATT Fusion Lasso Penalty Graph-constrained fusion penalty Eric CMU,

22 Fusion Penalty SNP j ACGTTTTACTGTACAATT Association strength between SNP j and Trait k: β jk Association strength between SNP j and Trait m: β jm Trait m Trait k Fusion Penalty: β jk - β jm For two correlated traits (connected in the network), the association strengths may have similar values. Eric CMU,

23 Graph-Constrained Fused Lasso Overall effect ACGTTTTACTGTACAATT Fusion effect propagates to the entire network Association between SNPs and subnetworks of traits Eric CMU,

24 Multiple-trait Association: Graph-Weighted Fused Lasso Overall effect ACGTTTTACTGTACAATT Subnetwork structure is embedded as a densely connected nodes with large edge weights Edges with small weights are effectively ignored Eric CMU,

25 Estimating Parameters Quadratic programming formulation Graph-constrained fused lasso Graph-weighted fused lasso Many publicly available software packages for solving convex optimization problems can be used Eric CMU,

26 Simulation Results 50 SNPs taken from HapMap chromosome 7, CEU population 10 traits SNPs Trait Thresholded Trait Correlation Correlation Network Matrix Phenotypes High association No association True Regression Coefficients Single SNP-Single Trait Test Significant at α = 0.01 Lasso Graph-guided Fused Lasso Eric CMU,

27 Simulation Results Eric CMU,

28 Association to a Tree-structured Phenome TCGACGTTTTACTGTACAATT Eric CMU,

29 Tree-Guided Group Lasso In a simple case of two genes h h Genotypes Low height Tight correlation Joint selection Genotypes Large height Weak correlation Separate selection Eric CMU,

30 Tree-Guided Group Lasso In a simple case of two genes h Select the child nodes jointly or separately? Tree-guided group lasso L 1 penalty Lasso penalty Separate selection Elastic net L 2 penalty Group lasso Joint selection Eric CMU,

31 Tree-Guided Group Lasso For a general tree h 2 h 1 Select the child nodes jointly or separately? Tree-guided group lasso Joint selection Separate selection Eric CMU,

32 Tree-Guided Group Lasso For a general tree h 2 h 1 Select the child nodes jointly or separately? Tree-guided group lasso Joint selection Separate selection Eric CMU,

33 Balanced Shrinkage Previously, in Jenatton, Audibert & Bach, 2009 Eric CMU,

34 Estimating Parameters Second-order cone program Many publicly available software packages for solving convex optimization problems can be used Eric CMU,

35 Illustration with Simulated Data Phenotypes SNPs High association True association strengths Lasso Tree guided group lasso No association Eric CMU,

36 Yeast eqtl Analysis Hierarchical clustering tree Phenotypes SNPs High association No association Single Marker Single Trait Test Tree guided group lasso Eric CMU,

37 Ultimately Pleotropic effects Epistatic effects Eric CMU,

38 Structured Input/Output-Lasso io lasso arg min Output Lasso Input structure: penalty: group within selection group sparsity of of SNPs correlated across multiple epistatic correlated SNPs traits [Lee, Zhu and Xing, submitted 2010] K N p k k Yi j X ij k 1 i 1 j 1 ( r, s) U Z k rs i, rs ( r. s) U j 1 k 1 k m ( r, s) S This full model incorporates input/output structure of the dataset as well as epistatic effects guided by genetic interaction networks U S m : m j k k k 2 j k rs m k j k 2 rs : genetic interaction networks th cluster in SNP network Eric CMU,

39 Sensitivity and Specificity varying the number of SNPs Marginal SNP: Methods taking advantage of output structures outperforms others. Epistatic SNP: Methods taking advantage of input structures outperforms others. IO-Lasso outperforms other methods for detecting both marginal & epsitatic eqtls For each number of SNPs, we show the average of the performance with 5 different simulated data Eric CMU, /15/

40 Computation Time Eric CMU,

41 Proximal Gradient Descent Original Problem: Approximatio n Problem: Gradient of the Approximation : Eric CMU,

42 Geometric Interpretation Smooth approximation Uppermost Line Nonsmooth Uppermost Line Smooth Eric CMU,

43 Convergence Rate Theorem: If we require and set, the number of iterations is upper bounded by: Remarks: state of the art IPM method for for SOCP converges at a rate Eric CMU,

44 Multi-Task Time Complexity Pre-compute: Per-iteration Complexity (computing gradient) Tree: IPM for SOCP Proximal-Gradient Graph: IPM for SOCP Proximal-Gradient Proximal-Gradient: Independent of Sample Size Eric CMU, Linear 2014 in #.of Tasks 44

45 Experiments Multi-task Graph Structured Sparse Learning (GFlasso) Eric CMU,

46 Experiments Multi-task Tree-Structured Sparse Learning (TreeLasso) Eric CMU,

47 Conclusions Novel statistical methods for joint association analysis to correlated phenotypes Graph-structured phenome : graph-guided fused lasso Tree-structured phenome : tree-guided group lasso Advantages Greater power to detect weak association signals Fewer false positives Joint association to multiple correlated phenotypes Eric CMU,

Probabilistic Graphical Models

School of Computer Science Probabilistic Graphical Models Graph-induced structured input/output models - Case Study: Disease Association Analysis Eric Xing Lecture 25, April 16, 2014 Reading: See class