The interpretation of chromatin immunoprecipitation sequencing experiments hinges on everything from sequencing bias and depth of coverage to the analytical methods used to assess read data, according to a recent study.
The study, published in Nature Methods last week, was led by investigators at the Harvard School of Public Health, the University of Chicago, and the University of North Carolina at Chapel Hill.
"We knew there are a lot of factors influencing the final results of ChIP-seq," co-first author Yiwen Chen told In Sequence. "But there was no systematic evaluation of that yet — especially of the most important factor, the sequencing depth."
Chen is a postdoctoral researcher in co-senior author Xiaole Shirley Liu's biostatistics and computational biology lab at Harvard and the Dana-Farber Cancer Institute.
To look at this in more detail, the international team did a series of ChIP-seq experiments in fruit fly cells with an eye to understanding how the information gleaned from ChIP-seq data relates to read bias, control choice, methodology, and more.
One factor the authors did not assess, however, is the choice of sequencing platform. All sequencing for the study was done using the Illumina GAIIx — a platform selected to reflect the preponderance of Illumina-based read data that the authors found when they sifted through read data in existing ChIP-seq databases.
In an effort to be as comprehensive as possible, the researchers generated ChIP-seq data for a transcription factor known as "suppressor of hairy wing," or Su(Hw), enriched at a relatively narrow set of binding sites in the Drosophila genome. They also did a set of experiments focused on the histone mark H3K36me3, which is sprinkled more liberally across the fruit fly genome.
The binding motifs and profiles associated with the transcription factor and histone mark are fairly well established, Chen explained, making it easier to tease apart authentic ChIP-seq signals from false positive peaks for the two marks.
The analysis uncovered a chromatin state bias that leads to greater sequence depth across parts of the genome with open chromatin or euchromatin compared to regions with tightly packed heterochromatin — a read bias that appeared to produce false positive results if not controlled for appropriately.
The researchers also looked at how ChIP-seq results differ when paired-end rather than single-end sequencing is partnered with chromatin immunoprecipitation, and used their Drosophila Su(Hw) and H3K36me3 data to compare the performance of seven algorithms most often used to assess ChIP-seq data.
Along with the insights gained from these analyses, the Drosophila dataset could prove useful for other groups interested in evaluating ChIP-seq performance or even developing new methods for analyzing such data, according to Chen, who called the dataset "a great resource for the community."
By making it possible to nab components of chromatin by immunoprecipitation and sequence-associated DNA, ChIP-seq can be used to track down the genomic localizations of transcription factors, histone modifications, and other epigenetic marks.
But although the approach has found favor in a range of studies exploring such DNA-protein interactions, the researchers explained, several sequencing and analysis-related aspects of ChIP-seq had not been systematically tested prior to the current study.
In an effort to begin exploring the factors that affect ChIP-seq outcomes and interpretation, the team used ChIP-seq to interrogate Su(Hw) and H3K36me3 binding sites across the genome in Drosophila melanogaster S2 cells.
When they initially compared reads generated for two different control samples — one comprised of genomic DNA and another obtained by extracting chromatin from cells without enriching for any particular chromatin mark — the researchers found that both controls showed a bias toward reads from parts of the genome with higher-than-usual guanine and cytosine content.
That GC bias was anticipated based on previous studies and known Illumina read profiles. But the extent of the GC bias differed between the genomic DNA and chromatin input controls.
When they looked at this more closely, the investigators traced the discrepancy back to differences in read counts across euchromatin and heterochromatin regions in the chromatin control, with more reads stemming from parts of the genome with open chromatin and active histone marks than from regions with compact chromatin and repressive histone signatures.
While the ChIP-seq community had some inkling that chromatin state could be an issue, the extent of its effect on read bias had not been fully appreciated previously, according to Chen.
"It's not completely surprising to the community that open chromatin has higher coverage — people talk about it and think about it," he said. "But something interesting we find is that the chromatin, in fact, actually has a much stronger impact than base composition, like the GC content that people always talk about."
For example, when the team failed to control for both the GC and open chromatin biases, it found many ChIP-seq signals for the transcription factor Su(Hw) that did not correspond to the transcription factor's known binding motif. On the flip side, the study authors estimated that they would miss between 4 percent and 10 percent of authentic Su(Hw) peaks by not controlling for both GC- and chromatin-related biases.
So while including a genomic DNA control in ChIP-seq experiments controls for GC bias in read data, Chen explained, the chromatin input control appears to be more crucial, since it can be used to weed out read depth effects related to both GC bias and chromatin state.
"Without chromatin input [as a control], you will have many more false positives detected," Chen said. "So I think that is really an important message for the biology community: you have to use a chromatin input."
And regardless of the control used, the researchers reported, it's going to take a great deal of sequence coverage to find absolutely all of the marks of interest using ChIP-seq. Based on results on the Drosophila transcription factor and histone marks studied, they concluded that "the regularly adopted sequencing depth … in humans may be insufficient to identify the vast majority of enriched regions."
For example, in a comparison with ChIP-on-microarray data, the researchers found that at low sequencing depths — fewer than 900,000 Drosophila reads, which would correspond to just under 18 million reads in human — between 30 percent and 50 percent of the ChIP-chip peaks were missed by ChIP-seq. When the sequencing depth was increased to 2.7 million reads, corresponding to 55 million reads in human, more than 90 percent of ChIP-chip peaks were identified by ChIP-seq.
However, even when the sequencing depth reached 16.2 million reads, which would correspond to around 327 million reads in human, around 1 percent of the ChIP-chip peaks were not detected in the sequencing data.
A bit more information may be gleaned from certain parts of the genome by using paired-end rather than single-end sequencing, the investigators found. While that additional data seemed to do little to increase the overall sensitivity of ChIP-seq, they reported, the paired-end method did help in teasing apart some authentic ChIP-seq peaks from artifacts.
"In the ChIP-seq world, people are not regularly using paired-end [sequencing]," Chen said. "We wanted to see: what are the extra benefits we could gain from paired-end sequencing?"
In the context of standard, single-end ChIP-seq experiments, he explained, redundant reads are often thought to be a consequence of PCR artifacts or other factors reflecting low chromatin immunoprecipitation quality.
By comparing paired-end and single-end sequence data for the same ChIP-seq samples, though, the study authors saw that that is not always the case. Instead, they found that many of their redundant reads mapped back to different parts of the genome when they took into account paired-end read information, suggesting these reads reflect authentic binding peaks.
"When we interpret the single-end data and see a lot of redundant reads, that's not necessarily a sign of bad ChIP quality," Chen said.
On the other hand, the team found that paired-end sequencing offered only modest improvements in ChIP-seq information that could be gleaned from parts of the genome that are prone to redundant reads, such as repetitive regions.
In repeat regions of the Drosophila genome, for example, the researchers saw about a 15 percent jump in the number of Su(Hw)- or H3K36me3-associated signals when they used paired- rather than single-end sequencing.
"That could vary between factors," Chen noted. "But that really gives you a sense, when you decide to embark on ChIP-seq experiments, [of] whether you want to really use paired-end or single-end [sequencing]."
Such increased resolution may be important for answering some research questions, he argued, but for studies where discerning signals from repeat regions is less crucial and
sequencing budgets are constrained, the single-end approach is expected to suffice.
Not surprisingly, the group's analysis indicates that the method selected for analyzing ChIP-seq reads can impact the way results are interpreted as well.
From the more than 30 algorithms available for dealing with ChIP-seq data when the study began, the researchers did a comparison involving seven of the algorithms that were not only frequently cited in the literature but also user friendly for analyzing ChIP-seq data on broad and narrowly distributed chromatin marks.
When they plugged in their Drosophila ChIP-seq data, the investigators found three algorithms that led the way in terms of overall specificity: the site identification from short sequence reads, or SISSRs algorithm; model-based analysis of ChIP-seq, or MACs; and USeq. On the sensitivity side, meanwhile, Useq and the R package spp came out ahead.
But while the general trend was for increased sequencing depth to produce better sensitivity, specificity, and reproducibility across experimental replicates, the researchers also found situations in which algorithms yielded unusual results as sequence depth increased.
Consequently, Chen said it appears that researchers' best bet is to apply at least two of the algorithms that appear best suited for assessing a given chromatin mark rather than relying on a single analytical method.
Beyond giving researchers a resource to help select the most appropriate algorithms for their own ChIP-seq experiments, Chen noted that the algorithm analyses might provide insights for those designing new data analysis software in the future.
An area of particular interest, according to Chen: finding ways to get better information from parts of the genome that are still better represented in ChIP-chip experiments than they are in ChIP-seq experiments.
In the current study, for instance, roughly one percent of the Su(Hw) transcription factor peaks found using ChIP in conjunction with Affymetrix tiling arrays were missed via ChIP-seq even at fairly high sequencing depths.
"These peaks are mostly in low mappability regions," Chen said. "That means that if we only keep uniquely-mapped reads we will miss these regions, but if we keep the reads [that map] to multiple genomic locations we gain a little bit [of information]."
"Therefore, there's lots of room for developing algorithms that can handle reads that map to multiple locations," he added, noting that a few such algorithms are starting to emerge.
Though Chen said he expects many of the ChIP-seq patterns identified in the current analysis to hold true regardless of the high-throughput platform selected, he noted that it might be interesting to look at whether there are any differences or special considerations associated with doing ChIP-seq experiments with single-molecule sequencing platforms.
Meanwhile, the Illumina-generated dataset is being made available online through the Gene Expression Omnibus so that other members of the research community can use it to evaluate and compare additional ChIP-seq analysis methods.
"It's expected that this is going to be a public resource for people doing all kinds of algorithm development [and] algorithm comparisons," Chen said.