In a sign that the microarray software sector is reaching a new level of maturity, GeneChip vendor Affymetrix plans to host the bioinformatics community’s first “bake-off” style software comparison via a series of webcast presentations scheduled to take place Feb. 9-19.
Such comparisons are common in the consumer applications market, but they are rare in the scientific computing world. Affymetrix, however, saw an opportunity within the growing microarray analysis sector to “both help our customers find an appropriate tool for their needs, and help the software vendors get some exposure to our customers,” said Scott Jokerst, senior product manager for data management products and head of Affy’s external developers’ program.
The event is an outgrowth of the company’s ongoing effort to encourage software developers to write tools that analyze data produced on the Affymetrix platform, Jokerst said. “We now have several commercial software providers who are in one stage or another of building in integration into the Affy system,” he said, so late last year, the company invited a number of them to analyze the same, unpublished data set, “and told them to show off the best parts of their tools.”
Each company pre-recorded a 30-45 minute presentation based on its analysis, and these sessions will be broadcast one per day over a two-week period. A live Q&A session between viewers and vendors will follow each presentation.
Software firms that will participate in the webcast include: Spotfire, VizX, Silicon Genetics, Inforsense, Iobion, BioDiscovery, Insightful, and Strand Genomics. They will each analyze data from a time-course experiment performed at the Fred Hutchinson Cancer Research Center to study the progression of granulocyte differentiation following retinoic acid simulation in a model cancer cell line.
But unlike software benchmarks that occur in the consumer market, Affymetrix isn’t planning to declare a victor among the eight participants in its webcast. Due to the inherent variability of microarray analysis, which is still as much of an art as it is a science, “they may come up with slightly different gene lists and that type of thing,” Jokerst said. “Actually, I think that’s going to be one of the interesting parts to the webinar, seeing what things do some of the vendors uncover that others don’t, and what are some of the common themes that vendors uncover in the data.”
Nat Goodman, a bioinformaticist at the Institute for Systems Biology, said the webcast is a “cool idea” for providing customers with a level playing field to judge the usability and other features of different software packages. But it is unlikely that the different packages will arrive at drastically different results, he said. “The science behind how you analyze microarray data has become quite mature, and the products all implement the same basic methods,” he said. “It’s all in the hands of the person operating it, so a better analyst will come up with a better answer. Any differences will reflect more the skill of the analysts than the tools.”
For the vendors involved, the event is a valuable forum to showcase those features that set their products apart from the rest of the microarray analysis pack. Elon Gasper, senior vice president at VizX Labs, welcomed the opportunity to take part in what he said is termed “eating your own dog food” in the consumer software market. “The fact is that people haven’t done that in this field very much, because most of the people doing software for scientists build the tools, but they’re not doing the science, too,” he said. The webcast will be a chance to offer the scientific community a glimpse of the vendors’ scientific chops, he said: “These are not product demos — these are actual scientific presentations.”
Jokerst said that Affymetrix is considering sponsoring similar events in the future for application areas other than gene expression analysis, depending on the success of the upcoming webcast.
Further information about the webcast and registration information are available at http://www.affymetrix.com/corporate/events/webinar/ge_data_webinar.affx.