NGS sequence preprocessing. José Carbonell Caballero

Transcription

1 NGS sequence preprocessing José Carbonell Caballero

2 Contents Data Format Quality Control Sequence capture Fasta and fastq formats Sequence quality encoding Evaluation of sequence quality Quality control tools Identification of typical artifacts Sequence filtering Practical session 2

3 Sequence capture RAW data Propietary format FastQ 3

4 Different technologies 4

5 Genome sequencing Reference genome 5

6 Where we are? Sequence preprocessing NGS pipeline Mapping Variant calling Downstream analysis 6

7 Fasta and Fastq formats Standard formats for sequence storage Text-based formats (easy to use!) (Almost) every programming language has a parser 7

8 Fasta format Nucleotide or peptide sequence >gi gb AAD cytochrome b [Elephas maximus maximus] LCLYTHIGRNIYYGSYLYSETWNTGIMLLLITMATAFMGYVLPWGQMSFWGATVITNLFSAIPYIGTNLV EWIWGGFSVDKATLNRFFAFHFILPFTMVALAGVHLTFLHETGSNNPLGLTSDSDKIPFHPYYTIKDFLG LLILILLLLLLALLSPDMLGDPDNHMPADPLNTPLHIKPEWYFLFAYAILRSVPNKLGGVLALFLSIVIL GLMPFLHTSKHRSMMLRPLSQALFWTLTMDLLTLTWIGSQPVEYPYTIIGQMASILYFSIILAFLPIAGX IENY >BBTBSCRYR tgcaccaaacatgtctaaagctggaaccaaaattactttctttgaagacaaaaactttca aggccgccactatgacagcgattgcgactgtgcagatttccacatgtacctgagccgctg caactccatcagagtggaaggaggcacctgggctgtgtatgaaaggcccaattttgctgg gtacatgtacatcctaccccggggcgagtatcctgagtaccagcactggatgggcctcaa Some typical file extensions (.fasta,.fa,.fna,.ffn,.faa,...) 8

9 Fasta format Allow multi-sequence (typically different chromosomes) >scaffold_1 CAAGGCTATAGCCCACCCGTTTTTGTGGCCTTTTTCCGCTGGACGAACTTGGCGCCCCGGCCTTCGGGTGGTTATTTTTG GGTGCAGCCTAGTGCGCGGCCTATTTTTGGCACACGGAGGCCCCTCGCAAAGTCTCGCGGCATTCCGAGTCCGGCGGACG AAGTTGGCCAGCCCCACCCCCCAAGGCTATAGCCCACCCGTTTTTGTGGCTTTTTTCCGCTGGACGAACTTGGCGCCCCG GCCTTCGGGCGGTTATTTTTGGCCGCGGCTTCGTCCGTGGCCTATTTTTGGCACACGGAGGCCCCCCGCAAACTCTCCCG GCATTCCGAGTCCGGCGGACGAAGTCGGCCAGCTTCACCACCCAGGCTGTTGCCCACCCCTTTTTGTGGCCTTTTTCCGC TGGACGAACTTGGCGCCCCAGCCTTTGGGCTGTTATTTATGGGTGCGGCTTTGTCTACGGCCTATTTTTGGCAGACGCAG GCCCCTCGCAAAGTCTCGCGGCATTCCGAGTCCGGCGGACGAAGTCGGCCAGCCCCACCCCCCAAGGCTATAGCCCACCC GTTTTTGTGGCCTTTTTCCGCTGGACGAACTTGGCGCCCCGGCCTTCGGGCGGTTTTTTTCTGGTGCGGTTTTGCCCGCG GCCTATTTTTGGCACTTGGAGGCCCCTCGTAAAGTCTCGCGGCATTCCGAGTCCGGCGGAAGAAGTTGGCCAGCCCCACC CTCCAAGGCTATAGCCCACCCGTTTTTGTGGCCTTTTTCCGCTGGACGAACTTGGGGCCCCGGCCTTCGGGCGGTTATTT -- >scaffold_2 ACGGTCCGGGGGCATCGGGGTGGGGGGGTAGCCGCGCGGCAGTTTGAACGGCGAAAAATGGGCAAGATCGGGGGCCGCTT AGTAGGCTTTGCCGGTTTCGGCCGTAACTCATGATCCGGGCCTCCGATTCCGTTTCCGTTTGGTCCCACGGGACCAGCGG TCCGGGGGCATCGGCAAGGGGGGGTAGCCGTGCGGCGGTTTGATCGCCGAAAAATGGGTAAGATCGGGGCCGTTTGGCCC GTTTTGCCGGTTTCGGCCCCCCGGGGGGCGGTTCATGCCCCGGGGGGGACAGCGGGGCGGGATCGGCACACGGCGGGTGA GGTGATCGGGTTCAGGCGGGTGCGGTCGGCGGCGGGGGCGCCCGGGGGGCTATAGCGCACCGCCTCGGGGGCCGCAAATT GGGGGCCGTAACTCATGATCGGGGCCCCCGATTCCGTTTCCGTTTCGTACCACGGGACCAGCGGTCCGGGGGCATCGGGA TGGGGGGGTAGCCGCGCGGCAGTTTGAACGGCGAAAAATGGGAAAGATCGGGGGCCGCTTAGTAGGCTTTGCCGGTTTCG GCCGTAACTCATGATCCGGGCCTCCGATTCCGTTTCCGTTTGGTCCCACGGGACCAGCGGTCCGGGGGCATCGGGATGGG GGGGTAGCCGTGCGGCGGTTTGATCGCCGAAAAATGGGTAAGATCGGGGCCGTTTGGCCCGTTTTGCCGGTTTCGGCCCC CCGGGGGGCGGTTCATGCCCCGGGGGGCGGTTCATGCCCCGGGGGGGACAGCGGGGCGGGATCGGTACACGGCGGGGGAG -- >scaffold_21 ACATATATAAAGTATTGTACTAGAAAACATTGTAAATGTATGCCTATTTAAACTTCAAGTATATGTAACACTTTCAAAGT CATAAGTGTAAAACTATTATATTTAAGATGTTTTGAGTTTATAAAAATAAACAACTTAACACTTCTACTCAGACGTTAAA AAATAAAAAACAAACATATTATTCTAATATATACATTCATACCCACTTAGACTCATTCAAGTTTAAGATATAAAAAAAAA AAAAGTGCTACCTAGAAATCCTTAATAGCAAGTCTTGTCTTTTATTTCATTAAGGGTAAAATCCTAAATGTGGCAAATGG CGTGTTATAAGAATTTTTGAGGGGTGCCAAATGCCCAATTCATTCATATTAGGCTTCTGAAAGAAGGCCCATTCATTCAC GACTTCGGGTCGCCCACCTGCTTGAATCCGTTTGTGTTTTTAACGTTGCAAGTATACCCATGTAAGTAAAAACAAACAAA 9

10 Fastq format Let's say fastq is a fasta with GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT +!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65 Fasta storages genomes...and fastq fragments 10

11 Sequence quality encoding Base quality must be encoded in just 1 GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT +!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65 Each base has a corresponding quality value (quality in position n, corresponds to base of position n) How is the encoding? Error probability Phred transformation (inversed integer value) ASCII encoding 11

12 Sequence quality encoding Phred + 33 Sanger [0,40] Illumina 1.8 [0,41] Phred + 64 Illumina 1.3 [0,40] Illumina 1.5 [3,40] 12

13 Sequence quality encoding Phred scores 13

14 Sequence quality encoding Phred scores Error probability Phred transformation (inversed integer value) ASCII GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT +!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF>>>>>>CCCCCCC65 P = 0.01 Q = -10*log10(0.01) = 20 Phred+33 ASCII = 53 => 5 P = 1 Q = -10*log10(0.01) = 0 Phred+33 ASCII 33+0 = 33 =>! 14

15 Evaluation of sequence quality Primary tool to assess sequencing If we evaluate our sequence quality in deep then we will known how reliable are our results QC determines posterior filtering We must be consistent with any filtering decision......if not, dowstream analysis will suffer the consequences QC must be test after every critical step 15

16 Evaluation of sequence quality How? quality per base Quality (or error probability) will be also a topic in next pipeline steps 16

17 Quality control tools FastQC 17

18 Quality control tools fastx-toolkit 18

19 Quality control tools NGS QC Toolkit 19

20 Quality control tools Example GOOD quality sequence_short_fastqc/fastqc_report.html POOR quality _sequence_fastqc/fastqc_report.html 20

21 Typical artifacts Sequence adapters 21

22 Typical artifacts Poor quality data 22

23 Typical artifacts Platform dependent 23

24 Sequence filtering (and editing) remove bad quality data Improve confidence of downstream analysis 24

25 Sequence filtering (and editing) Tail quality trimming minimum quality threshold 25

26 Sequence filtering (and editing) Mean quality Read length Read length after trimming Percentage of bases above Q Adapter trimming Adapter reads 26

27 Sequence filtering tools Fastx-toolkit Galaxy ( SeqTK ( Cutadapt ( And more... 27

28 Any question? 28