Last week the Microarray Quality Control Consortium published the results of a large-scale comparative microarray study in a special issue of Nature Biotechnology. The issue contains three commentaries, four analysis papers, and two articles discussing the results of the project, a US Food and Drug Administration-led effort to evaluate the reproducibility and reliability of microarray data (see related article, this issue).
In tandem with the release, the MAQC has released data from the study, culled from 1,300 microarray experiments on eight microarray platforms, through the National Center for Biotechnology Information’s Gene Expression Omnibus database, the FDA’s ArrayTrack database, and the European Bioinformatics Institute’s ArrayExpress database.
But with so much new information and data available, the question is which segments of the global microarray community are bound to benefit most. Will it benefit industry, larger core labs and testing centers, average academic users, or all of the above?
While the results are scarcely a week old, it appears that array manufacturers and larger testing services and core labs may be the first to benefit from the results of the MAQC project. In particular, chip vendors and larger labs will be able to use the standard assays and data provided by the results to aid in proficiency testing, customer support, and product development.
According to Richard Shippy, an R&D scientist in GE Healthcare’s CodeLink unit and an author on one of the Nature Biotech papers, GE will be able to use the results of the MAQC study to evaluate the scanners its customers use with its CodeLink bioarrays.
“CodeLink is an open platform system,” Shippy told BioArray News this week. “So if a customer needs to purchase a new scanner, they can just run the assay and see if the data is better, or, if it’s worse, by what percentage. There are so many scanners on the market, and this makes evaluating them so much easier,” he said.
Shippy added that GE could use the MAQC results for internal product development.
“This is useful for all array manufacturers. Say they want to modify their labeling procedures. You can see how that performs relative to the baseline and all other platforms,” he said.
Shippy said that GE will be promoting the MAQC results to its customers via its iCenter web portal and through its sales force. GE will also hold a WebEx seminar in October or November to discuss the project’s findings.
More service-oriented companies, like Expression Analysis, also see opportunities for using the MAQC results in their day-to-day operations.
“We have developed proficiency testing as a way of monitoring in microarray labs. We will continue this program using the MAQC resources,” Reid said.
Wendell Jones, senior research statistician at Expression Analysis and MAQC author, said that the results will strengthen confidence in the results of its services offering, particularly among clients that are preparing arrays for clinical usage or using arrays to locate diagnostic and prognostic content.
“We want to make sure that labs, including ourselves, which may be part of the discovery or diagnostic process, are able to demonstrate accurate and reproducible results by using these standard assays,” Jones told BioArray News last week.
“The information resulting from this study is that microarray assay data is very reproducible both within a laboratory and across laboratories as in a multi-site clinical trial when it is carried out using good scientific method that follows recommended operating procedures,” Janet Warrington, vice president of emerging markets and molecular diagnostics R&D at Affymetrix, told BioArray News this week via e-mail.
“The samples that were used in the study are probably now the most well-characterized RNA libraries, by microarray and RT-PCR assay, commercially available and could be very useful for proficiency testing,” she added.
What’s in it for Academia?
As companies like GE Healthcare and Expression Analysis look to integrate the MAQC results into their businesses, some academic researchers are wondering what the MAQC project will mean for them.
According to Zoltan Szallasi, a researcher at Children's Hospital Informatics Program at Harvard Medical School and an MAQC author, “the MAQC results will be more important for industry than for small academic groups.”
“I don’t think the average researcher will do any extensive testing or benchmarking [related to the results] because of a lack of resources. The average researcher uses microarrays more as a fishing tool, so issues about accuracy don’t really come up,” Szallasi told BioArray News this week.
Still, Szallasi said that using the results for proficiency testing in a larger core lab “might be a good idea.”
However, Roger Bumgarner, director of the Center for Expression Analysis at the University of Washington in Seattle who is unaffiliated with MAQC, said the results are unlikely to affect the CEA.
“In general, I don't feel the results will have much impact on our day-to-day lives,” Bumgarner told BioArray News this week. “To a large extent, the works published in this special issue essentially recapitulate what many of us already knew from various test[s] done a few years ago in a variety of settings. That is that expression array technology — when used properly and in the hands of sensible people — produces reliable results that can be interpreted in a sensible way within the constraints of the technology,” he said.
“To a large extent, the works published in this special issue essentially recapitulate what many of us already knew from various test[s] done a few years ago in a variety of settings.”
Naftali Kaminski, thedirector of the Dorothy P. & Richard P. Simmons Center for Interstitial Lung Disease at the University of Pittsburgh School of Medicine and also unaffiliated with MAQC, said that the papers are “reassuring” and show “the maturation of arrays from basic use to clinical usage.”
However, he added that the results might have wielded more influence if they were released earlier.
“Microarray users today are stable. They are pretty committed to their technologies,” Kaminski told BioArray News this week. “It would have been a blow if it had shown that there is no correlation between platforms, but it would have had a bigger impact if it came out three or four years ago.”
Bumgarner also said that the results were unlikely to convince more people to use arrays.
“People often don't let data get in the way of their pre-conceived notions,” he said. He added that the current “poor climate” for funding is likely to have a negative impact on usage, regardless of the study results.
EA’s Reid maintained that there is a place for the MAQC results in smaller labs, however. “Individual microarray labs have access to these materials, can repeat the experiment in their own lab, and compare their results to the MAQC data,” she said.
“We believe this type of proficiency testing is crucial as microarrays are used in clinical and diagnostic applications,” she added.
Margaret Cam, director of the Genomics Core Laboratory at the National Institute of Diabetes and Digestive and Kidney Diseases, suggested that academic researchers and smaller labs could use the raw data from the MAQC project to develop concordance rates for the platforms they are using.
“From my experience at looking at these types of studies, interpretation can be subjective and concordance rates can also depend on many different factors — the way one filters out ‘poorly performing’ probes or ‘bad’ data, for instance,” she told BioArray News this week.
“The best way for a researcher to deal with this is to analyze the raw data and derive concordance rates for platforms they currently use, which will be meaningful to their own research environment or application,” she said.