Deep learning frameworks for regulatory genomics and epigenomics

Transcription

1 Deep learning frameworks for regulatory genomics and epigenomics Chuan Sheng Foo Avanti Shrikumar Nicholas Sinnott- Armstrong ANSHUL KUNDAJE Genetics, Computer science Stanford University Johnny Israeli 1

2 Local chromatin architecture of histone marks regulatory elements TFs nucleosomes sequence motifs Adapted from Shlyueva et al. (2014) Nature Reviews Genetics. 2

3 Chromatin states Combinatorial chromatin states define broad classes of elements Promoter Transcribed Enhancer CTCF Heterochromatin Poised Repressed Quiescent Thurman et al. (2012) Nature

4 ATAC-seq: genome-wide chromatin accessibility from low input material Paired-end sequencing ATAC-seq peaks identify chromatin accessible regulatory elements

5 ATAC-seq reveals chromatin architecture in genome-wide fragment length distributions Buenrostro et al. (2013) Nature Methods.

6 Chromatin architecture reflects chromatin state Fragment lengths Promoter CTCF Enhancer Bivalent Transcribed Heterochromatin Represesed Quiescent

7 Fraglen Position-aware 2D fragment length distributions (V-plots) 400 bp Aggregate plot for CTCF state Positioned nucleosome 50 bp -1000bp Accessible binding site Peak summit (Midpoint of fragment) +1000bp Plot at single CTCF site sparse and noisy V-plots were first introduced by Henikoff et al. 2011, PNAS

8 Can we predict chromatin states/histone marks at ATAC-peaks? Which of 8 chromatin states? 2kb around ATAC-peak Image classification task! Which histone mark is present?

9 Deep neural networks (DNNs) for image classification Output label: Trump? Or Sanders? Input: image pixel values Lee et. al. (2009), ICML 9

10 Deep neural networks (DNNs) for V- plot classification Output label: Chromatin state/histone mark Input: V-plots Chromatin state/histone mark Nucleosome arrays, Large regions of open chromatin +1 / -1 nucleosome, Segments of open chromatin Nucleosomes, Open chromatin 10

11 An artificial neuron Sigmoid ReLu (Rectified Linear Unit)

12 Convolutional filters Input image Conv. Filter Feature map

13 Convolutional filters Input image Feature map

14 Convolutional layer: multiple filters learn distinct features

15 Pooling layers: locally smooth signal

16 How does a deep conv. neural network transform the raw V-plot input at each layer -1Kb 500 Pure CTCF Promoter Enhancer Kb Chromatin State Class Probabilities 2nd Fully Connected Layer 1st Fully Connected Layer 3rd Smoothing 2nd set of Convolutional Maps 2nd Smoothing 1st set of Convolutional Maps Initial Smoothing V-Plot Input (300 x 2001)

17 After initial pooling (smoothing) Pure CTCF Chromatin State Class Probabilities 2nd Fully Connected Layer 1st Fully Connected Layer Promoter 3rd Smoothing 2nd set of Convolutional Maps 2nd Smoothing Enhancer 1st set of Convolutional Maps Initial Smoothing V-Plot Input (300 x 2001)

18 Second set of convolutional maps Pure CTCF Chromatin State Class Probabilities 2nd Fully Connected Layer Promoter 1st Fully Connected Layer 3rd Smoothing 2nd set of Convolutional Maps 2nd Smoothing Enhancer 1st set of Convolutional Maps Initial Smoothing V-Plot Input (300 x 2001)

19 Learning from multiple 1D functional data (e.g. DNase, MNase) Chromatin State Class Probabilities 2 nd FC Layer 1 st FC Layer 3 rd Convolution Layer 2 nd Convolution Layer 1 st Convolution Layer 1D MNase signal (1 x 2001) 3 rd Convolution Layer 2 nd Convolution Layer 1 st Convolution Layer 1D DNase signal (1 x 2001) Scan DNase profile using filter 19

20 Learning from raw DNA sequence Class Probabilities Higher layers learn motif combinations Score sequence using filters Convolutional layers learn motif (PWM) like filters

21 The Chromputer Integrating multiple inputs (1D, 2D signals, sequence) to simulatenously predict multiple outputs H3K9me 3 H2A.Z H3K36me3 H3K27me3 H3K4me1 Class Probabilities 2 nd FC Layer 1 st FC Layer H3K4me3 TF Binding Chromatin State Multi-task learning 2nd Combined FC Layer 1st Combined FC Layer 3rd Smoothing 2nd set of Convolutional Maps 2nd Smoothing 1st set of Convolutional Maps 2nd Combined FC Layer 1st Combined FC Layer 3rd Smoothing 2nd set of Convolutional Maps 2nd Smoothing 1st set of Convolutional Maps 2nd Combined FC Layer 1st Combined FC Layer 3rd Smoothing 2nd set of Convolutional Maps 2nd Smoothing 1st set of Convolutional Maps Initial Smoothing Initial Smoothing Initial Smoothing V-Plot Input (300 x 2001)

22 Chromatin architecture can predict chromatin state in held out chromosome (same cell type) Model + Input data types Majority class (baseline) 42% Gene proximity 59% Random Forest: ATAC-seq (150M reads) 61% Chromputer: DNase (60M reads) 68.1% Chromputer: Mnase (1.5B reads) 69.3% Chromputer: ATAC-seq (150M reads) 75.9% Chromputer: DNase + MNase 81.6% Chromputer: ATAC-seq + sequence 83.5% Chromputer: DNase + MNase + sequence 86.2% Label accuracy across replicates (upper bound) 88% 8-class chromatin state accuracy (%)

23 High cross cell-type chromatin state prediction Learn model on DNase and MNase only Learn on GM12878, predict on K562 (and vice versa) Requires local normalization to make signal comparable 8 class chromatin state accuracy Train / Test GM12878 K562 GM K

24 Predicting individual histone marks from ATAC/DNase/MNase/Sequence Area under Precision recall curve CTCF H3K27ac H3K4me3 H3K4me1 H3K9ac H2Az H3K36me3 H3K27me3 H3K9me3

25 Chromputer trained on TF ChIP-seq predicts cross cell-type in-vivo TF binding with high accuracy Area under Precision recall curve Chromputer Inputs: Seq + DNA shape + DNase profile Positives: Reproducible ChIP-seq peaks Negatives: All other DNase peaks + flanks + matched random sites c-myc YY1 CTCF Test sets: Held out chromosomes in held out cell types 25

26 DeepLIFT: Scoring predictive power of features in Deep Neural Networks LIFT: Linear Importance Feature Tracker, or LIFTing the top off the black box. Provides a predictive importance score for any raw input feature (e.g. pixels in V-Plot images, each nucleotide in sequence) intermediate learned features (e.g. convolutional filters) Linear breakdown of contribution of each input to immediate outputs Recursively apply to get contribution of any input to any output Can be computed efficiently with a single backpropagation (unlike insilico mutagenesis) Less susceptible to buffering effects than in-silico mutagenesis Technical details: ReLU networks: equivalent to Taylor approximation of change in softmax/sigmoid logit if input eliminated. i.e. gradient (w.r.t logit) * input

27 What architecture properties of the ATAC-seq V- plots predict different chromatin states? CTCF state: centered binding, symmetric phased nucleosomes Enhancer state: localized signal, heterogeneity Promoter state: broad regions of accessible chromatin what is the change in classification probability relative to an unbiased classifier if we *only* consider the contributions from each pixel

28 Architectural heterogeneity of accessible elements in different chromatin states Low dimensional t-sne embedding (like PCA) of accessible sites (colored by chromatin state) Distinct classes of enhancers 28

29 Top scoring MNase filters and activating input patterns for CTCF state Top scoring MNase filters Maximally activating input MNase profiles Well phased arrays of nucleosomes 29

30 Top scoring MNase filters and activating input patterns for promoter state Top scoring MNase filters Maximally activating input MNase profiles Asymmetric positioning with well positioned +1/-1 nucleosomes 30

31 What useful patterns can we extract from raw DNA sequence models? Gata1 Motif discovery! What PWM like filters are predictive? G A T A A G C A T T A C C G A T A A Which nucleotides in input sequence are contributing to binding!

32 Top sequence filters for CTCF state Canonical motif 32

33 High resolution point binding events and sequence grammars at CTCF peaks MNase-seq DNase-seq CTCF ChIP-seq Nuc. level importance (height of letter) shows coordination of multiple point binding events 33

34 Context-specific reuse of regulatory sequence in chromatin accessibility changes during hematopoiesis Ryan Corces-Zimmerman Jason Buenrostro Will Greenleaf Howard Chang Ravi Majeti ATAC-seq

35 Deep learning sequence determinants of chromatin accessibility Output: Accessible (+1) vs. not accessible (0) HSC B-cells Erythroid Peyton Greenside G C A T T A C C G A T A A Input: Raw DNA sequence

36 Context-specific re-use of regulatory sequence in HSC, B-cells and Erythroblasts Gata (Rc) Position along sequence Gata Gata (Rc) SPI1 Peyton Greenside

37 Importance in HSC s Context-specific re-use of regulatory sequence in HSC, B-cells and Erythroblasts ATAC-seq SPI1 ChIP-seq GATA1 ChIP-seq?? Position along sequence SPI1 Peyton Greenside

38 Importance in B-cells Context-specific re-use of regulatory sequence in HSC, B-cells and Erythroblasts ATAC-seq SPI1 ChIP-seq GATA1 ChIP-seq No peak No peak Not expressed Position along sequence Peyton Greenside

39 Importance in Erythroid Context-specific re-use of regulatory sequence in HSC, B-cells and Erythroblasts ATAC-seq SPI1 ChIP-seq GATA1 ChIP-seq Gata (Rc) Position along sequence Gata Gata (Rc) SPI1 Peyton Greenside

40 YY1 & GATA and much, much more GATA, SPI1, RUNX2 STAT1 & GATA TEAD4 & GATA AP1 in B-cells only ETS & GATA

41 Summary and ongoing work New predictive deep learning framework (Chromputer) for integrative genomics New interpretation engine for deep learning models. We can extract predictive features (motifs, grammars, footprints, architecture features) from the deep neural networks Local chromatin architecture is predictive of chromatin state and histone marks within and across cell types We can predict in-vivo binding profiles of TFs in new cell types from sequence + shape + DNase/ATAC-seq with high accuracy Context-specific reuse of sequence grammars in accessible sites Extensions: From binary to continuous signal prediction Extensions: Functional variant (QTL, GWAS, rare variant) prediction from raw sequence models 41 bioarxiv paper and code coming soon!

42 Acknowledgements Kundaje Lab members Chuan Sheng Foo Funding Avanti Shrikumar Nicholas Sinnott- Armstrong U01HG (GGR) U41-HG S1 R01ES Johnny Israeli Conflict of Interest: Deep Genomics (SAB), Epinomics (SAB) Rahul Mohan Nathan Boley Peyton Greenside Will Greenleaf 42

43 Guess the element from the V-plot AI vs. human What is this regulatory element? Pure CTCF, Promoter, or Enhancer?

44 Its an enhancer! 1st Fully Connected Layer 2nd Fully Connected Layer Enhancer (correct) 44