Skip to main content

Study of Top Arrays Finds Protocols, Not Platforms, Cause Variation in ChIP Results

A recent study comparing three leading commercial tiling arrays used in chromatin immunoprecipitation (ChIP)-on-chip or copy number-amplification experiments found that the protocols used by individual labs, and not the platforms they use, could be most responsible for the disparities sometimes seen in ChIP-chip experiments.
Additionally, the paper, published in this month’s issue of Genome Research, found all of the platforms involved — Affymetrix, NimbleGen, and Agilent — are relatively equal in their ability to detect protein-DNA interactions.
In the study, members of the National Human Genome Research Institute-led Encyclopedia of DNA Elements, or ENCODE, consortium analyzed the performance of tiling array platforms, amplification procedures, and signal detection algorithms in a simulated ChIP-chip experiment.
Mixtures of human genomic DNA and “spike-ins” comprising nearly 100 human sequences at various concentrations were hybridized to four tiling array platforms by eight independent groups, according to the paper. Blind to the number of spike-ins, their locations, and the range of concentrations, each group made predictions of the spike-in locations.
The results of the study showed that variations in results between labs using the same commercial array platform were as great as among labs using different platforms. This caused the study authors to argue that “microarray platform choice is not the primary determinant of overall performance.” Rather, “variation in performance between labs, protocols, and algorithms within the same array platform was greater than the variation in performance between array platforms.”
The study also stressed that “each array platform had unique performance characteristics that varied with tiling resolution and the number of replicates, which have implications for cost versus detection power.”
The authors suggest that researchers should pay more attention to their protocols and data-analysis algorithms when setting up their experiments, explaining that two characteristics that could help researchers choose a platform for ChIP-chip experiments are the length of the oligonucleotide probes used on a vendor’s arrays and the algorithm used to analyze the data.
“Long oligonucleotide arrays were slightly more sensitive at detecting very low enrichment,” the paper states. Likewise, “performance among signal detection algorithms was heavily dependent on array platform.”
‘Pluses and Minuses’
Kevin Struhl, a professor at Harvard Medical School and co-author of the Genome Research paper, told BioArray News last week that “it’s a little hard to compare platforms because it depends on what you are doing. In the real world, each one of these platforms has its own pluses and minuses.”
For instance, Struhl cited the general low cost of Affymetrix chips as an advantage for ChIP experiments involving whole-genome tiling arrays for mammalian genomes. At the same time, Struhl said that NimbleGen and Agilent have “more flexible” manufacturing processes and longer oligos.
“The question is, ‘When does the benefit of having more oligos hurt or help you compared to having oligos that are intrinsically a bit better?’” Struhl said. “That answer depends on what you are doing. For example, if you are doing a whole-genome human experiment, Affy has an advantage,” he said. “But if you are talking about a smaller genome, like Drosophila, or custom-designed arrays interrogating specific regions, well that makes a big difference.”
According to another of the paper’s authors, perhaps the most surprising finding was that variation in results was as great between the labs using the same platform, as it was among labs using different platforms.
“What you can clearly see is that if you look within a platform, there’s much more variation in performance than there is between platforms,” said Jason Lieb, a paper co-author and an associate professor at the University of North Carolina, Chapel Hill. “The point is that microarrays for this purpose are actually very good at finding the sites of enrichment.”
The authors differed in their opinion of which component of ChIP-chip experiments contributes the most to result disparities. For Struhl, the amplification step is a “much greater source of variation than the platform.”
“I think a huge amount of the errors are coming in from that,” he said. “That has actually not been well investigated. People focus so much on the platform as opposed to the other aspects of the technique when those are actually more important.”
Paper co-author Shirley Liu, an associate professor at the Dana-Farber Cancer Institute, said that choice of algorithm is an important factor.
“I think a lot of the algorithms [available] were designed for a specific type of array,” she told BioArray News last week. “They are usually not that effective when used to analyze data from another type of tiling array.”
Recognizing this, the paper recommends specific algorithms for use with certain platforms. “Among the platforms, the Splitter algorithm was the best on Agilent tiling arrays, while MAT was best for Affymetrix,” the paper states. Liu said that the TAMALg algorithm mentioned in the paper performed best when used with NimbleGen arrays.

“Some people might say that they can never get ChIP-chip to work so they change arrays. More likely they should change the protocol or the algorithm or the postdoc.”

“Some people might say that they can never get ChIP-chip to work on one platform so they change arrays,” Liu said of the findings. “More likely they should change the antibody, the protocol, the algorithm, or the postdoc.”
Bargaining Chips
Though the Genome Research paper and its authors have stressed that there is no winning platform in the study, vendors have used the results to tout their platforms for ChIP-chip applications.
Agilent, which sells a line of arrays for such studies, said in a statement last week that the “study found that longer oligonucleotide microarrays, such as Agilent's, were more sensitive at detecting very low enrichment or copy number.” Additionally, Agilent noted, its platform “demonstrated the highest levels of sensitivity and specificity per probe, in some cases by orders of magnitude, over a range of simulated tiling densities.”
In an e-mail to BioArray News, Agilent pointed to several figures in the study that supported its claims. Among the long-oligo platforms described in the study, which included NimbleGen’s, “Agilent proved to be the most cost-effective over a wide range of tiling densities, relevant to array comparative genomic hybridization and ChIP-on-chip assays, costing at most half the price per performance unit of other long-oligo platforms over this range,” said Agilent spokesperson Stu Matlow.
While the paper did discuss the cost of the arrays used in the experiment, Lieb, Struhl, and Liu all told BioArray News that chip prices have changed since they wrote the paper.
Meantime, Luke Dannenberg, epigenetics product manager at NimbleGen, countered that the study “clearly demonstrates the superior sensitivity of NimbleGen long oligo tiling arrays.” NimbleGen offers the “most sensitive platform at detecting low and ultra-low levels of enrichment on the market,” Dannenburg wrote in an e-mail to BioArray News.
Other platform providers had a different take on the study’s results. Jeremy Preston, senior marketing manager at Affymetrix, told BioArray News in an e-mail last week that the study “proved that Affymetrix offers the most cost-effective solution for whole-genome coverage.”
“The Affymetrix platform features a 35-base-pair resolution tiling design and delivers the most complete picture of genome activity for ChIP-on-Chip studies, all in a seven-array set,” Preston said. “Scientists using the Affymetrix tiling arrays have reported unexpected functions for already known genes and new functions for previously unexplored parts of the genome.”
Struhl said that “basically [all the platforms] work well; that’s the bottom line. Which one you would choose would depend on your experiment.”

The Scan

Pfizer-BioNTech Seek Full Vaccine Approval

According to the New York Times, Pfizer and BioNTech are seeking full US Food and Drug Administration approval for their SARS-CoV-2 vaccine.

Viral Integration Study Critiqued

Science writes that a paper reporting that SARS-CoV-2 can occasionally integrate into the host genome is drawing criticism.

Giraffe Species Debate

The Scientist reports that a new analysis aiming to end the discussion of how many giraffe species there are has only continued it.

Science Papers Examine Factors Shaping SARS-CoV-2 Spread, Give Insight Into Bacterial Evolution

In Science this week: genomic analysis points to role of human behavior in SARS-CoV-2 spread, and more.