Introduction to OTU Clustering. Susan Huse August 4, 2016

Size: px

Start display at page:

Download "Introduction to OTU Clustering. Susan Huse August 4, 2016"

Marion Shepherd
6 years ago
Views:

1 Introduction to OTU Clustering Susan Huse August 4, 2016

2 What is an OTU? Operational Taxonomic Units a.k.a. phylotypes a.k.a. clusters aggregations of reads based only on sequence similarity, independent of taxonomic name

3 The Magical 3% NOT! 3% SSU OTUs = Species and 6% SSU OTUs = Genera

4 History of The 3% OTUs Wayne et al (1987) IJSEM 37(4): Uses genome-level phylogeny to define species, using 70% DNA-DNA reassociation. Stackebrandt and Goebel (1994) IJSEM 44(4): % DNA-DNA ~ 97% similarity of full-length 16S Therefore 97% similarity of any hypervariable region in any bacteria or fungus = species?

5 Cumulative Percentage of Non-Singleton Taxa 18% of genera are > 10% OTUs 30% of genera are < 3% OTUs Genus (827) Family (244) Order (92) Class (52) Phylum (31) Fig 1A Schloss & Westcott (2011) Appl Env Microbiol Maximum Intra-Taxon Distance for Full-length 16S

6 Why OTUs? Because we can! We can t: do whole genome sequencing of communities assign complete taxonomic names derive phylogenetic trees from millions of short tags Continuing to develop new methods that are more biologically or evolutionarily meaningful Come back in 10 years

7 Pros and Cons OTUs Novel organisms Taxonomy Universal names Insufficient taxonomy Does not lump together all order or family-level classifications Many names based on phenotype rather than genotype Meaning associated with names Independent of clustering width and algorithm Historically well-studied are split New areas are lumped

8 Cluster Width Diameter Sequences are never more than D apart. (CL) Radius Sequences are never more than R from seed or reference. (Ref, Greedy) Variable Only selected positions (Oligo) Averaged radii (AL)

9 Primary Aggregation Methods Complete linkage - no two sequences are farther apart than the clustering width Average linkage - the average distance between sequences is less than the width Greedy clusters sequences are less than the width to OTU representatives built de novo Reference sequences are less than the width to universal reference set of OTU representatives Oligotyping sequences are equivalent at most informationrich nucleotide positions Probabilistic error modeling of likely clusters

10 De Novo Greedy Clustering Most abundant reads: are likely true sequences least likely to be errors or chimeras, and most likely to spawn errors and chimeras Seed OTUs with most abundant, Assign variation to most abundant source.

11 De Novo Greedy Clustering Sort by abundance First Seq 1 is seed of Cluster 1 Compare next Seq to each cluster in order: if Distance(Seq, Seed) <= W, include it in the cluster if Seq is not a member of any cluster, seed a new cluster Edgar (2010) Bioinformatics 26(19):2460-1

12 Greedy Seeds > W Each tag goes to OTU with nearest representative, not OTU with the nearest sequence

13 5. Bin each tag to that reference s OTU#, or unclustered if > Width Rideout et al (2014) PeerJ in press Closed Reference OTUs 1. Create a set full-length reference sequences (e.g., Greengenes) 2. Cluster reference sequences 3. Select representative sequence for each cluster 4. Map each tag to nearest rep <= Width

14 Open Reference OTUs 1. Perform Closed Reference OTU mapping 2. Perform de novo clustering on unclustered sequences 3. Subsequent datasets can be mapped to the reference and to an expanding set of de novo clusters.

15 Reference OTU Caveat X% clustering of full-length reference sequences does not represent X% for each hypervariable region. Will vary by phylotype

16 Oligotyping Calculated Entropy Alignment positions of greatest information Oligotypes identical in selected positions Ignore the rest as noise Alignment Position Eren et al (2013) Methods in Ecol and Evol 4:1111-9

17 Pros and Cons Greedy, AL - standard de novo methods, abundance dependent, project-specific Reference OTU compare V-regions, universal IDs, dependent on reference database and reference clustering Oligotyping / DADA2 increased sensitivity, no specific radius, projectspecific

18 The Problem of Inflation Clustering algorithms return more OTUs than predicted for mock communities. OTU inflation leads to: alpha diversity inflation beta diversity inflation Where does this inflation come from? How can we adjust for this inflation?

19 OTU inflation varied with sample depth 7000 MS-CL M2FN Rarefaction - PML OTUs ,000 40,000 60,000 80, , ,000 Number of Sequences Sampled 5K 10K 15K 20K 50K 100K

20 Low-Quality Reads and Errant OTUs Word to the wise: minimize adventurous OTUs

21 Different clustering algorithms have very different effects on the size and number of OTUs created

22 Inflation in Action >200,000 V6 reads of E. coli clone (454) 1,042 is a few more than the expected 2 Huse et al (2010) Environ Microbiol 12(7):

23 Sequencing Noise in E.coli Distance Tags Seqs 0 177, % , % % % % % % % % % > % Cluster at 3% using only tags within 3% of the correct sequence

24 Inflation with small sequence error Only 129 OTUs much better!!

25 Clustering V6 sequences within 3% of known templates (outliers removed) E. coli (2) S. epidermidis (1) Clone43 (43) MS-CL 129 / 89 / 694 Clustering method MS-AL 54 / 44 / 218 Alignment method PW-CL 6 / 5 / 308 PW-AL 2 / 1 / 43 MS Multiple Sequence Alignment PW Pairwise Alignment CL Complete Linkage clustering AL Average Linkage clustering

26 Multiple Sequence Alignments Regardless of clustering algorithm, an MSA cannot fully align tags whose sequences are too divergent, more tags cause more problems 18,156 sequences and 392 positions

27 MSA Limitations Distance Comparison Independent Distances Subsampled Distances X-axis: 1000 distances subsampled from MSA of 40,000 sequences Y-axis: 1000 distances from MSA of only the 1,000 bacterial sequences

Cluster Count: 1234567 Complete Linkage propagates OTUs #5 #6 #9 #8 #4 #3 #1 #12 #11 #2 #7 #10 #13 Each V6 variant

28 Cluster Count: Complete Linkage propagates OTUs #5 #6 #9 #8 #4 #3 #1 #12 #11 #2 #7 #10 #13 Each V6 variant with 2nt distance, will start a new OTU. Each V6 variant with 1nt distance, can cluster with the seed or a 2nd variant.

29 Average Linkage collapses errors Cluster Count: 1 #1 Clusters tend to be heavily dominated by their most abundant sequence, which strongly weights the average and smoothes the noise.

30 Greedy Seeds (UClust) Cluster Count: 1

31 Effect of Sampling Depth OTUs MS-CL M2FN Rarefaction - PML 5K 10K 15K 20K 50K 100K Previous algorithms differentially inflated OTUs at a sampling depth dependent upon total sample depth 0-20,000 40,000 60,000 80, , ,000 Number of Sequences Sampled Newer methods do not

32 Errant OTUs If a low-quality sequence is >3% from its source, it can create a new OTU. If the rate of an errant read = 1 in 10 thousand, and we have 1 million reads: * 1,000,000 reads = 100 errant OTUs

33 Impact of Errant OTUs Errant OTUs as percent of OTUs decreases with diversity. If we have 100 errant OTUs: Mock community: 100 / 50 OTUs = +200% Diverse community: 100 / 2,000 OTUs = +5%

34 Relative Inflation Absolute number of errant OTUs will increase with sample size. Relative number of errant OTUs will decrease with sample complexity

35 What is the right cluster width? No.

36 What is the right cluster width? Distances vary by hypervariable region Full-length does not equal V1-V3 which does not equal V3-V5 which does not equal V6, etc. Distance vary by bacterial lineage What specificity are you trying to gain? No single clustering width is the right answer, use several (and wave your hands a lot) or use other methods like Oligotyping

37 Questions about Clustering How meaningful are clusters functionally? When is an errare rare and when is it an error? Should it be included in an existing cluster or start its own? How to assign sequences if OTUs overlap? What is the effect of residual low quality data or chimeras? How sensitive are alpha and beta diversity estimates to clustering results?

Sequencing Errors, Diversity Estimates, and the Rare Biosphere

Sequencing Errors, Diversity Estimates, and the Rare Biosphere or Living in the shadow of Errares Susan Huse Marine Biological Laboratory June 13, 2012 Consistent Community Profile across samples and environments