Genomics AGRY Michael Gribskov Hock 331

Transcription

1 Genomics AGRY Michael Gribskov Hock 331

2 Computing Essentials Resources In this course we will assemble and annotate both genomic and transcriptomic sequence assemblies We will use a wiki, to handle course logistics We will use computing facilities at the Rosen Center for Advance Computing (RCAC), in particular we will use the server scholar.rcac.purdue.edu You must have access to a computer, and preferably bring it to class on "computational days" You must understand how to use your computer to connect to RCAC

3

4 NGS Sequence Analysis General Process Simpler version of the first two bubbles of fig 1 in Ekblom Sample Preparation Draft Genome Annotation Sequencing Validation and QC Data Cleaning Scaffold Assembly Quality Control Contig Assembly

5 Genome Assembly Original plan for human genome Isolate chromosomes clone in Bacterial Artificial chromosome (BAC) vectors Find "golden path" to minimize sequencing Subclone BACs into plasmids and sequence using dideoxy chain terminating nucleotides (Sanger sequencing) Optimistically estimated to take $3 billion and take 15 years Initiated in 1990 claimed to be the largest collaborative project

6 Genome Assembly Whole Genome Shotgun (WGS) Assembly 1998 Crag Venter Why mess around with all the subcloning and tiling, why not just fragment the whole genome randomly and sequence all the pieces 1998 NIH It'll never work You have to sequence too many clones You won't be able to put it together 2000 Celera completes draft sequence using WGS approach Funding $300 Million

7 Genome Assembly How much sequence do you need (ca. 2004) Lander ES, Waterman MS, Genomic mapping by fingerprinting random clones: a mathematical analysis, Genomics 2: (1988) Depends on genome size (G) sequence length (L) number of sequences (N) coverage = L N / G In 1 st generation sequencing 16 X coverage was a good target For the human genome 15 X coverage 500 base reads 3.3 x 10 9 bp 99 million reads = $ 3 $30/base, = $9.9 $0.10/base

8 Genome Assembly

9 Genome Assembly Monascus Purpureus Used to make red yeast rice ( beni-koji and ang-kak) Also produces statins More on the wiki - genome size about ~50 Mb? has introns and other typical eukaryotic features Data 149,983,522 DNA reads (JGI) 150 base TruSeq paired-end 230 k RNA reads

10 Genome Assembly Illumina TruSeq System universal adapter Primer insert Primer index adapter bar code Paired end reads, each 150 bases Vocabulary paired-end mate-pair contig scaffold Ekblom, fig 2 (partial)

11 Quality and Cleaning TruSeq Adapters Universal adapter, 58 bases, same for all sequences, primer location > TruSeq_Universal_Adapter AATGATACGGCGACCACCGAGATCTACACTCTTTCCCTACACGACGCTCTTCCGATCT > TruSeq_Universal_Adapter_Reversed AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT Index adapter, 63 bases, contains Barcode, primer location > TruSeq_Index_Adapter-GTAGAG TCGATCGGAAGAGCACACGTCTGAACTCCAGTCACGTAGAGATCTCGTATGCCGTCTTGCTTG > TruSeq_Index_Adapter-GTAGAG_Reversed CAAGCAGAAGACGGCATACGAGATCTCTACGTGACTGGAGTTCAGACGTGTGCTCTTCCGATC

12 Quality and Cleaning What should we clean?

13 Quality and Cleaning Is it important? Depends on the assembler Depends on level of contamination Depends on depth Zhou & Rokas, Molec.Ecol. 23, ,2014.

14 Quality and Cleaning What is quality? Introduced in the Phred (Phil Green, UWa) program Quality Score, Q = -10 log 10 ε where ε is the expected error rate (probability of calling an incorrect base) 20 (P=0.01) is a commonly used cutoff Phred quality score Error Probability 10 1:10 90% 20 1:100 99% Base Call Accuracy 30 1:1, % 40 1:10, % 50 1:100, %

15 Genome Assembly Sequence files fastq Combines sequence an quality Beginning of sequence is marked rest of line has sequence ID and documentation Quality section begins with + Quality values are converted to letters in the ASCII alphabet by adding 33 to the Phred quality Typical ascii value = quality + 33 Some companies use 64 instead of 33

16 Quality and Cleaning Typical 1 st gen quality >jgi JGI_CAOP10014.rev JGI_CAOP10014.rev

17 Quality and Cleaning Fastq 1:N:0:GTAGAG GACCCATCCATTGTTGGACAGCTGAAGACGGGACGATCGTGCTCGTGTTTTGAATGCGAGAATCCCTGCAGAGGCTGCCTGCTTCGGNNNNNNNNNNTCCTCGACAGCC + CCCFFFFFHHHHHJIJJJJGIJJJJJJJJJJJIIJIJJJIIJIIHAFGIJJEHHHHFFFDCDDDDDDCDDDDDDBBDDDDDDCCDDB##########++28<<@BB>BD I = ascii 73 Quality = = 40 Quality = -10 log 10 ε ε = 10-4 # = ascii 35 Q = = 2 ε = = 0.63 = totally bogus

18 Quality and Cleaning FastQC Available on RCAC servers. You will use it. A good data set Zhou & Rokas, Molec.Ecol. 23, ,2014.

19 Quality and Cleaning FastQC a,b before and after quality trimming c sequence composition bias at 5' end d kmer enrichment. adapter dimer?? e non-random priming in RNAseq Zhou & Rokas, Molec.Ecol. 23, ,2014.

20 Quality and Cleaning FastQC - Monpu1.genome.rawReads.fastq

21 Quality and Cleaning Quick and Dirty check for primers Universal primer reverse, first 22 bases expected for read 2 (from index adapter) 99.99% are in read 2 (62228/62232) universal adapter Primer Forward Reverse Primer index adapter

22 Quality and Cleaning Quick and Dirty check for primers Sequence: Monpu1.genome.rawReads.fastq TruSeq index adapter (forward) first 22 bases TCGATCGGAAGAGCACACGTCTGAACTCCAGTCACGTAGAGATCTCGTATGCCGTCTTGCTTG 92% are read 1 universal adapter Primer Forward Reverse Primer index adapter

23 Quality and Cleaning Quick and Dirty check for primers Sequence: Monpu1.genome.rawReads.fastq TruSeq index adapter (forward) first 22 bases TCGATCGGAAGAGCACACGTCTGAACTCCAGTCACGTAGAGATCTCGTATGCCGTCTTGCTTG universal adapter Primer Forward Reverse Primer index adapter

24 Quality and Cleaning Quick and Dirty check for primers Sequence: Monpu1.genome.rawReads.fastq TruSeq index adapter (forward) first 22 bases TCGATCGGAAGAGCACACGTCTGAACTCCAGTCACGTAGAGATCTCGTATGCCGTCTTGCTTG TruSeq Index Adapter (Reverse) CAAGCAGAAGACGGCATACGAGATCTCTACGTGACTGGAGTTCAGACGTGTGCTCTTCCGATC

25 Quality and Cleaning Quick and Dirty check for primers Sequence: Monpu1.genome.rawReads.fastq TruSeq universal primer: AATGATACGGCGACCACCGAGATCTACACTCTTTCCCTACACGACGCTCTTCCGATCT grep AATGATACGGCGACCACCGAGA Monpu1.genome.rawReads.fastq more AATGATACGGCGACCACCGAGATCTCGGATGCCGTCTTCTGCTTGAAAAATTAAGGATGATGAACTGCCGCGCAAGATCTTGTTAGAAATCTTGCTGCTGCGGGTACTTTCGGGGGAAATATTTCCTTGCAATCGGGGCCGAGCTTTGGG AATGATACGGCGACCACCGAGATCTACACTCTTTCTTCTTCTACTTCTCCTCCTTAACCACTCTCCTCTTTTCTCTTTCTACTTCTCCTTCTACCACTCTTCTACCACTTCTCCCTTTTCTCCCTCTCTGTCTTCCTCCACTTCTCCTTC AATGATACGGCGACCACCGAGATCTACACTCCTTCCCCAACCACTCCTCCACTCTTCCCCTTCTACTTCTCCTCCCCCACCACTTCTTTACTTCACCTCTCTTACACCTCCCATCTTTCTTCTTCTTCTCATCCTTCTCCTTCTACCACC AATGATACGGCGACCACCGAGATCTACACTCTTTCCCGTCCGTTCCCTACGCTCCATATTTCTCAACCCCCCGGCCTTGGACGGGGGGGGCGGACCGGCCCGGCGGAGCCCACGCGGCGCAGCTTGCTGCTCCTCGTGGTCGCGGCAACA AATGATACGGCGACCACCGAGATCTCGCATGCCGTCTTCTGCTTGAAAAATAAAGCCGTAGAGGGAGAGCGGATGGTCGACGTTGTGCAGCAACCGGCACGGCATGCTGGCGTTGGTGGTGGTCACGGAGTGGGAGACGGTTTAGGGAAG AATGATACGGCGACCACCGAGATCTACACTCTTCTCCTTCTCCTCTTTCTTCTTCTCCTTCTACCACTCTTCTCTTCTCCTTCTACTTCTTCTTCTTCTCCTTCTCCCTAACCACTCTTTCACTTCTCCTTCCTCCTCTTCTCCCTTACC AATGATACGGCGACCACCGAGATCTCTACGTGTCTTGACATCCCCCGCCTCCTCTTCCGACCCAATCCCTTTTTCAAAAACACCCCAGCGGGTGGGGAGGAACACCCTACACTTCCTTCCACCCCACCCTTTCCCAAACACAAAACCTCC AATGATACGGCGACCACCGAGATCTCGTATGCCGTCTTCTGCTTGAAAAAAATCAAACAGATGTCCGCGACGTCGCAACGCCCCGTTTCGCAGCCGTCGGCTCGGGAACCTGCCCAAGCACACCAACAGACGGCAAGCCACCATCACGAA AATGATACGGCGACCACCGAGATCTTCACCCTTTCCCTTCACATCTAAATCCACTCAGGTGGATACCAAATCGTTCTTTTTCAATTCTCCCCCCTCCCCCCGTACATCTTCGTACTTTTACTCACGTGCTACTGTCACGCACTCGTCCAC AATGATACGGCGACCACCGAGATCTACACTCTTTCCCTACACGACGCTCTTCTGCTTAGATCGGAAGAGCACACGTCTGAACTCCAGTCACGTAGAGATCTCGTATGCCGTCTTCTGCTTGAAAAAAACAACACAACAGCAGCGTCTGCC AATGATACGGCGACCACCGAGATCTACACTCTATATGGTATCCCTGGAGGATCAATCCTCGCATACACGCAAGGATTGATCCTCCAGGGATACCATATAGAGTGTTCCGATCAATGTGTTACGGCATAGAGGGATGTAAGGAATGCAGCG AATGATACGGCGACCACCGAGATCTACACTATTTTCTCCGTTCTGAGCTCTTACTGCTCTTACTGGTTCACCAGGTGTCCATCTGGGTCGGCGTTTTTGGGGAATCAATGCCTACGTATTAATATCATGTGCCCTCTAAAGACTGTTTTT AATGATACGGCGACCACCGAGATCTACACTCCTTTCCTCCTTCCTTCCCTCTTATTTGATCTTTTTGTATTTAAACATGGAGTAGAGTGCAGTAAAATTTTAAGACCTTCCTTTATATTAGTAATAAAGATTATTAAATACGCTGGAAGC AATGATACGGCGACCACCGAGATCTACACTCTTTCTTCTCCTGACTACCTTTCCCCTCATGTTCGTGACCCTGCTCTTCCCTGGCTTCACTTTCTGGGTCGCCCAAAAAAGCGCCGCCCGCAAAGACTGCGGGCTACCACGCTACATCCT AATGATACGGCGACCACCGAGATCTACACTCTTTCCCATACCAGCCCAGACACCCCTCCACCTACTGTCAGCGAGAAAAGGTTAAAATGATGGAGCTTCTGAAAACACACCTGAATAACATCAAGATCCTCTGCACGCGCGCGGACAAAC AATGATACGGCGACCACCGAGATCTTCACTCTTTTCTTGCTCTTCTGATAATCTGGTGTTGGTTGGTGGTCGTTCTCATAGATTTGATCTTGCGCTTCAACCGGCGGTGGCGGTCCCGGCCTGCGGCTGGAGCGCTCGTGGCAACCGTCC AATGATACGGCGACCACCGAGATCTACACTCTGTCCCCACAGCTACGCTCTTACTCAAATCCTGATGTCTTCGGATGGATTTGAGTAAGAGCGGAGCTGTGGGGACCCGGAAGATGGTGGAGCATTGCTCAATATGGCGCAACAAGAGGA AATGATACGGCGACCACCGAGATCTACACTCTCCCCCTATGCATAGCTCCGACGTTGACGAGAAGGGTACTCTTCGCTCCCATCCCCTCGTCCTCCCCCCCAAAACCGTTCTGCGAGTGACCCCGCTGGCCCCTTTGTGCGCCGCACACC AATGATACGGCGACCACCGAGATCTACACTCTTTCTCTCTATTTATTCTTTATTTTTCTCTTTTTTCATTTCTCTTCCTCCCCAACCACTCTCTCTTTTTTCCACTTCTCTTTCCCCTTTAATATTCTATTTTTCTTCTCTTTATTCATT AATGATACGGCGACCACCGAGATCTACACTCTTTCCCCACCTCATCCTTCGCAAGCCACACCCCGTACAAACTACCCAGCTCTTATTTTCTCCTCGCGGTTGTTGGGGGCTGGTCGCCCCCGGGGCTCGGGCGCCGCTTTCGCCTCCCGA AATGATACGGCGACCACCGAGATCTACACTCTATCTCCTCAACCAGTCGAGGAGATAGAGGGTCAGACACTTCCGGTTCAGGGTCCTTGGGGCATGTTCCGGGCTAGGGCTGTGGGGGGTGGGGGTGATGCTGCTATTCTTCTTGGCACG AATGATACGGCGACCACCGAGATCTACACTCTTTTCTTAGCCCCTCTAAAGTCTTTTGATCTTGGGGTTGGGGCTGTCGTGTAAACCGAAACATCAAAGTGAGCCACTGGCAAAAAAACTTTTTACCAACCCTGCCTCCAGACCCACAAA AATGATACGGCGACCACCGAGATCTACACTCTTTCCCTACACGACGCTCTTCTGCTTAGATCGGAAGAGCACACGTCTGAACTCCAGTCACGTAGAGATCTCGTATGCCGTCTTCTGCTTGAAAAAAAACAAACTAACATTCCTAGACCG AATGATACGGCGACCACCGAGATCTACACTCTTTCCCCCCACCCCCCCCCTATCTTTGTCACAACTGCACCTACAACCCCCGCCCCTCTCCTTTTCGGGGGCAGGATGCGCCCACACTTTCTCTTGTTTAATACAGTTCTTTTCCACCCC AATGATACGGCGACCACCGAGATCTACACTCTTTCCCCGCGACTGTAATTCGTCAAAGCCTCGACCTTTTCTCTTTGGAATATTAGCTTCCTGTCTTCTTTTTTCTTCTTCTCATTCTCCCTCACCTCATAATTCTGTCTTCAACATATA AATGATACGGCGACCACCGAGATCTACATTCCTTTTCTCCCTTCGTCTTAGTCCTACGTCGACTTGGTGAAGTCGACGTAGGACTAAGACGAAGGGAGAAAAGGAATGTTAGCAAGTTCCGCGCGTTTAACGCTAGGAAAGGAGAGGAAA CAAGCAGAAGACGGCATACGAGATCTCTACGTGACTGGAGTTCAGACGTGTGCTCTTCCGATCTAATGATACGGCGACCACCGAGATCTCGTATGCCCGCTTCTGCTTGAAAAAAAAAAAAAGAGGCGACGAGACGTACGAAGACCGCAC AATGATACGGCGACCACCGAGATCTACACTCTTTCTTCTCATCATCGTCCTCCCTTTCGAAGTAGGGATGCATCTTTTTTGGCCCTTTTAGCTTTGTGCTGAAAAATACTATGTTTCTCATAATTCTTTTGTGAACACCATCCACTCCAC

26 Quality and Cleaning matches to reverse of index adapter universal adapter Primer Forward Reverse Primer index adapter

27 Quality and Cleaning reverse of index adapter adapter should end 41 bases to the right less than 1% error CAAGCAGAAGACGGCATACGAGATCTCTACGTGACTGGAGTTCAGACGTGTGCTCTTCCGATC

28 Quality and Cleaning How many have exact 22 base matches? universal adapter forward grep c AATGATACGGCGACCACCGAGA Monpu1.genome.rawReads.fastq 2694 universal adapter reverse grep c AGATCGGAAGAGCGTCGTGTAG Monpu1.genome.rawReads.fastq index adapter forward grep c GATCGGAAGAGCACACGTCTGA Monpu1.genome.rawReads.fastq index adapter, reverse grep c CAAGCAGAAGACGGCATACGAG Monpu1.genome.rawReads.fastq =166,922 / 149,983,353 total reads = 0.11 % most are what is expected for small or no insert in expected orientation What about mismatches?

29 5' adapter matches vs length Universal forward Universal reverse Index forward Index reverse Random Expected

30 RCAC Modules On RCAC servers many bioinformatics tools have been installed. These are referred to as modules. This is a system specific to RCAC and Purdue Module commands module avail show available modules. To see available modules you must first run module use /apps/group/bioinformatics/modules. Put this in your.bash_profile module load load a module module list show currently loaded modules module show show details about the installation of a module The module system makes it unnecessary to load specific paths, environment symbols an program names on your own Because bioinformatics modules change rapidly, multiple versions are often available

31 RCAC Modules Module avail When there are different versions, one is the default Default is sometimes but not always shown Different versions

32 RCAC Other programs You are not limited to modules, you can download an run programs on your own. This is the main use for your home directory For instance sickle, which is available on github

33 RCAC Other Programs I want to download onto the scholar server not my PC Option one: download on PC/Mac and transfer Option two: right click on the download Zip button to copy the URL in my home directory type wget unzip the resulting file result

34 RCAC Other programs Read the file README.md This will tell you, amongst a lot of other stuff To build Sickle, enter: make After running make I have a new file called sickle Try sickle h or sickle help this will usually give you some brief directions Also look for a doc directory Also look for files named README or MANUAL

35 RCAC Batch jobs not using a module The RCAC server are not designed to run jobs interactively. Instead, jobs are submitted to a queuing system called PBS (or Torque) Since you cannot run jobs on the frontend systems you will need to make job files to submit your jobs

36 RCAC Batch job using a module

37 Assignment Adapter Cleaning See the wiki page: For 5 pts, choose one of the installed module programs and run it on the monascus sequences. Be prepared to explain why you chose the settings you chose. For 10 pts, install and run one of the non-module adapter cleaners. You may use one not on the wiki page. Explain why you chose settings. You may do both of the above Check how well it worked Using grep as shown in class Using FastQC (installed module, run as batch job) Upload relevant information onto your group page Add comments about download sites and papers to the wiki page