The case for cloud computing in genome informatics.
Pubmed
Direct Link
Here's an opinion piece arguing for using cloud computing by a "Giant" in the field: Lincoln Stein. This is a great read for anyone thinking about staying in NGS for the long-haul. Quote of the month: "The cost of genome sequencing is now decreasing several times faster than the cost of storage, promising that at some time in the not too distant future it will cost less to sequence a base of DNA than to store it on a hard disk."
I will discuss some of my criticisms and ideas on the topic related to author's points.
Firstly, the author defines the standard model is hub-and-spoke model, with archival sites (UCSC, SRA, Genebank, etc.) the hub, and sequencing centers and bioinformaticians on the end of the spoke. We all have to submit our NGS data to SRA per NIH policy, but the idea that we submit to SRA then re-download the data is ludicrous. All Genome centers have their own compute farms that analyze the data, which is directly obtained from the sequencers. Whole groups of Bioinformaticians are co-located around these centers for easy access to the data. While other institutions must access the data from SRA, the investigators who generated the data do not. Presumably the original investigators will distill their results into a paper and resource, and have the most incentive to analyze the data. Anyhow, my main point that sequencing centers have massive clusters of computers and Bioinformaticians to analyze their data locally.
The author's solution to giving easy access to data to all (not just those who generated the data) is to use the cloud. This requires everyone to transfer their data to the cloud via fiber (internet) or mail (fed-ex hard drives). Bandwidth costs money and is certainly limited. I will be impressed to see how one genome per day per institution will be feasible. Fed-ex would love us to send hard-drives on a a daily basis. Two other solutions come to mind.
The first solution is to house the major genome centers near the cloud, so that a direct pipe could be installed. Nonetheless, if the West-coast center uploads their data to their west-coast cloud, then the data needs to be migrated to the other clouds so the east-coast can access the data on their cloud (presumably everyone could also access the west-coast cloud). Anyhow, this doesn't solve the problem as migration of data still needs to occur.
The second solution is to use a tried-and-true technology: Bittorrent. Bittorrent is great, although somewhat frowned upon. Having only a modest number of sites hosting this content via Bittorrent would allow the fast and efficient distribution of content, even between clouds. Sites like http://www.biotorrents.net/ already have bittorrent trackers for bio-datasets.
Cloud computing seems like it will become a large tool for NGS analysis. Nevertheless, the clouds need to be more configurable. Does the cloud have high memory (1TB) compute nodes for de novo assembly? Another option would be pool our money together to get an "NIH" part fo the cloud, rather than using for-profit clouds, where the cost of using this part of the cloud is subsidized.
The data deluge is an opportunity to tackle some large data processing questions. Given the size and importance of this data, will academia be squeezed out with industry taking over (like drug development research)?
Tuesday, May 11, 2010
Monday, May 10, 2010
Transcript assembly and quantification by RNA-Seq reveals unannotated transcripts and isoform switching during cell differentiation
Transcript assembly and quantification by RNA-Seq reveals unannotated transcripts and isoform switching during cell differentiation.
Pubmed Direct Link
Similar to another paper I blogged about, this paper describes a novel method to analyze NGS data wrapped in a biology coating (mouse genome analysis). I really like this way of describing a method since this highlights the motivation for the method's creation: to answer a question in biology.
This paper describes a method called Cufflinks, which uses TopHat to find splice junctions, which uses bowtie for mapping. Here you can see the recursive dependency tree on which this method is built. You must trust the upstream methods first.
Off to the methods section. Just like any nature paper, the methods are hidden in in the supplemental materials. I am glad to see the supplementary methods written in latex using the hyperref package. This makes it easy to move around the document quickly, plus latex is much easier to write and format than Word or other programs.
The main contribution is the transcript abundance calculation. I would strongly recommend reading the supplementary methods as this is a great Bayesian discussion of transcript assembly and abundance estimation. It also covers some nice computer science Theorems. The basic idea is to find the minimal set of transcripts that explain the fragments, then quantify each transcript. Seems simple right?
Anyhow, take a look at this paper as Cufflinks (with TopHat (with bowtie)) is a very popular pipeline for RNA-SEQ. You should probably understand what it is doing before you use it. Kudos to the authors.
Pubmed Direct Link
Similar to another paper I blogged about, this paper describes a novel method to analyze NGS data wrapped in a biology coating (mouse genome analysis). I really like this way of describing a method since this highlights the motivation for the method's creation: to answer a question in biology.
This paper describes a method called Cufflinks, which uses TopHat to find splice junctions, which uses bowtie for mapping. Here you can see the recursive dependency tree on which this method is built. You must trust the upstream methods first.
Off to the methods section. Just like any nature paper, the methods are hidden in in the supplemental materials. I am glad to see the supplementary methods written in latex using the hyperref package. This makes it easy to move around the document quickly, plus latex is much easier to write and format than Word or other programs.
The main contribution is the transcript abundance calculation. I would strongly recommend reading the supplementary methods as this is a great Bayesian discussion of transcript assembly and abundance estimation. It also covers some nice computer science Theorems. The basic idea is to find the minimal set of transcripts that explain the fragments, then quantify each transcript. Seems simple right?
Anyhow, take a look at this paper as Cufflinks (with TopHat (with bowtie)) is a very popular pipeline for RNA-SEQ. You should probably understand what it is doing before you use it. Kudos to the authors.
Sunday, May 9, 2010
BS Seeker: precise mapping for bisulfite sequencing
BS Seeker: precise mapping for bisulfite sequencing
Pubmed Direct Link
Yet another alignment algorithm (YAAA), this time for bisulfite sequencing. This algorithm, developed in the same lab as the BS-SEQ (Jacobsen at UCLA) and called BS-SEEKER, purports to be more versatile and faster. I am of the philosophy that an aligner should be "as fast as it needs to be" to achieve the desired level of sensitivity, so the latter claim is not as interesting as the former claim. Basically, it needs to find what you are looking for first, then lets worry about speed. The abstract claims sensitivity and accuracy, so lets see if it delivers.
Surprisingly, the authors do not implement their own algorithm, but instead use bowtie. I do not recommend bowtie since it does not handle insertions or deletions, which can significantly impact mapping accuracy (see this partial discussion). Nevertheless, bowtie should be able to be replaced by other algorithms, since the novel contribution of BS-SEEKER is the separation of the four types of BS reads.
The evaluations were very lite on information, mapping reads to only human chromosome 21, and not accounting for higher sequencing error rates or indels. Furthermore, BS-SEEKER is not the best algorithm in every situation according to their own results, in some cases MAQ or RMAP performing better in terms of accuracy/sensitivity/timing. Their novel contribution is the application of bowtie, but beyond that I am not seeing the value of this method, with the exception if you use the Cokus et al. library construction. More evaluation of this method is necessary to convince others to use it.
Pubmed Direct Link
Yet another alignment algorithm (YAAA), this time for bisulfite sequencing. This algorithm, developed in the same lab as the BS-SEQ (Jacobsen at UCLA) and called BS-SEEKER, purports to be more versatile and faster. I am of the philosophy that an aligner should be "as fast as it needs to be" to achieve the desired level of sensitivity, so the latter claim is not as interesting as the former claim. Basically, it needs to find what you are looking for first, then lets worry about speed. The abstract claims sensitivity and accuracy, so lets see if it delivers.
Surprisingly, the authors do not implement their own algorithm, but instead use bowtie. I do not recommend bowtie since it does not handle insertions or deletions, which can significantly impact mapping accuracy (see this partial discussion). Nevertheless, bowtie should be able to be replaced by other algorithms, since the novel contribution of BS-SEEKER is the separation of the four types of BS reads.
The evaluations were very lite on information, mapping reads to only human chromosome 21, and not accounting for higher sequencing error rates or indels. Furthermore, BS-SEEKER is not the best algorithm in every situation according to their own results, in some cases MAQ or RMAP performing better in terms of accuracy/sensitivity/timing. Their novel contribution is the application of bowtie, but beyond that I am not seeing the value of this method, with the exception if you use the Cokus et al. library construction. More evaluation of this method is necessary to convince others to use it.
Saturday, May 8, 2010
Genome-wide mapping and assembly of structural variant breakpoints in the mouse genome
Genome-wide mapping and assembly of structural variant breakpoints in the mouse genome.
Pubmed Direct Link
This paper presents a new method to detect structural variant (SV) breakpoints called HYDRA. Since it is a Genome Research (GR) paper, the paper focuses on answering a question in biology, namely detecting SVs in the mouse genome; I will focus on the method.
SVs are detected by clustering discordant matepairs. Matepairs are pairs of reads from the same DNA fragment with some base distance (insert size) between the ends. Discordant reads are reads for which the insert size is outside the expected distribution. Hydra seeks to use multiple mappings per read, where each read may map to multiple locations, albeit with possibly different quality/probability. The method then seeks to select the correct mapping for each read collectively for all reads parsimoniously. This is to detect SVs in regions with segmental duplications, copy number changes, or other repetitive or difficult to map regions. This is an extremely important feature. A minor point is how to store multiple mappings, as a whole genome human resequencing BAM file may be hundreds of gigabytes when there is only one mapping per read. Nevertheless, HYDRA does not report what type of SV occurs, but only that there exists a SV. Further analysis needs to be performed to determine the type of SV (insert/deletion/CNV/inversion/etc.). The authors suggest to use bedtools to do this (their own software). It is yet to be seen if author software will come out that automate this whole process
Pubmed Direct Link
This paper presents a new method to detect structural variant (SV) breakpoints called HYDRA. Since it is a Genome Research (GR) paper, the paper focuses on answering a question in biology, namely detecting SVs in the mouse genome; I will focus on the method.
SVs are detected by clustering discordant matepairs. Matepairs are pairs of reads from the same DNA fragment with some base distance (insert size) between the ends. Discordant reads are reads for which the insert size is outside the expected distribution. Hydra seeks to use multiple mappings per read, where each read may map to multiple locations, albeit with possibly different quality/probability. The method then seeks to select the correct mapping for each read collectively for all reads parsimoniously. This is to detect SVs in regions with segmental duplications, copy number changes, or other repetitive or difficult to map regions. This is an extremely important feature. A minor point is how to store multiple mappings, as a whole genome human resequencing BAM file may be hundreds of gigabytes when there is only one mapping per read. Nevertheless, HYDRA does not report what type of SV occurs, but only that there exists a SV. Further analysis needs to be performed to determine the type of SV (insert/deletion/CNV/inversion/etc.). The authors suggest to use bedtools to do this (their own software). It is yet to be seen if author software will come out that automate this whole process
Wednesday, April 21, 2010
Next-Generation Instrument Google Map
Next-Generation Instrument Google Map
http://map.seqanswers.com
A map of NGS Instruments from around the world can be found my following the link above. This map is far from complete, so if you know of or have not included a sequencing facility, please add it to this map. It is amazing the number of sequencers in the field compared to only a few years ago. It is also surprising that given this global sequencing capacity, less than 20 whole genome human genomes have been published. There are an infinite number of other fun projects that do not involve human resequencing, but would it not be fun to truly have the 1000 Genomes Project completed to a high depth (more than 60X)?
I would say it is a pretty poor showing from Africa and Antartica, although BGI and BIG in China are not well reported (128 Illumina HighSeqs anyone?). The one sequencer farther from any other seems to be the one located at the "
Institute for Chemical Biology and Fundamental Medicine".
http://map.seqanswers.com
A map of NGS Instruments from around the world can be found my following the link above. This map is far from complete, so if you know of or have not included a sequencing facility, please add it to this map. It is amazing the number of sequencers in the field compared to only a few years ago. It is also surprising that given this global sequencing capacity, less than 20 whole genome human genomes have been published. There are an infinite number of other fun projects that do not involve human resequencing, but would it not be fun to truly have the 1000 Genomes Project completed to a high depth (more than 60X)?
I would say it is a pretty poor showing from Africa and Antartica, although BGI and BIG in China are not well reported (128 Illumina HighSeqs anyone?). The one sequencer farther from any other seems to be the one located at the "
Institute for Chemical Biology and Fundamental Medicine".
Subscribe to:
Posts (Atom)
