SAN FRANCISCO — No one comes out and says it, but at shows such as Cambridge Healthtech Institute’s High-Content Analysis conference held here last week, scientists inevitably just want to hear the bottom line from other scientists: How was your experience with a particular company’s screening platform?
But just how fair these assessments are is up for debate. In many instances, especially in a nascent field such as high-content cellular analysis, direct comparisons might not always tell the whole story.
Many instrument manufacturers will readily admit that their platform is not suitable for all applications. Different instrument capabilities are often required, depending upon the type of cells being screened, the assay type that is used, the amount of throughput required, and the level of detail or amount of data that needs to be generated.
One particularly contentious case here last week was that of Evotec Technologies’ Opera automated confocal plate reader for high-content screening.
Marjo Simonen, a scientist in the lead discovery center at Novartis’ Institutes for Biomedical Research in Basel, Switzerland, presented a poster at the conference that generated heavy traffic and requests for poster copies.
Simply entitled “Evaluation of Different Cell Imagers,” the poster by Simonen and colleague Fraser Glickman compared the performance of four high-content imaging systems from three companies — the Opera, Cellomics’ ArrayScan VTI, and GE Healthcare’s IN Cell Analyzer 3000 and 1000 — in two types of common drug-screening assays, GPCR internalization and plasma membrane translocation.
In addition to screen shots from each assay for each instrument platform, the poster also contained two grids — one detailing “technical” features of each instrument, such as types and number of cameras, lasers, objectives, and filters, type of plate formats supported, cost, and imaging type, among others; and one detailing “subjective” features, such as assay performance, read time, reliability, ease of use, and pros and cons.
Although the poster did not necessarily portray Evotec’s instrument in a negative light, the Cellomics and GE Healthcare instruments were deemed the clear winners on several levels.
“We found the hardware of all instruments good,” the poster stated in its conclusion. “For image analysis, Cellomics and GE Healthcare rely on ready-to-use analysis modules, whereas Evotec has less ready-to-use modules. Instead, Evotec offers an open system where the user is able to write [his] own algorithms. This can be a benefit or a drawback, depending on the IT skills and needs of the user.
“From the GPCR internalization assay we got comparable results with all instruments,” the poster continued. “More differences were seen in the plasma membrane translocation assay … with the IN Cell Analyzer 3000, good results were obtained fast. With ArrayScan VTI, acceptable results were also obtained. With the Opera, the users of the instruments were unable to find or create algorithms for the analysis of the plasma membrane translocation assay, which demonstrates the difficulty of creating new algorithms for new types of assays.”
Evotec was clearly not pleased with the presentation. At one point, Evotec Technologies’ US-based senior vice president of business development, Robert Rodewald, approached Simonen at the poster and demanded to know where she obtained most of her information. He commented that much of the data was “false,” and pointed out a number of examples from the grid — both technical and subjective — to support his stance.
“You set the instrument up to fail,” Rodewald said.
Simonen defended her data, saying that all the technical aspects were verified by her contact at Evotec Technologies in Germany, and that all the subjective data was just that: subjective.
Rodewald later declined an interview request regarding the matter, and calls to Simonen were not returned in time for this publication.
“We design our software to be open,” Gunter Bauer, Evotec Technologies’ chief business officer, told Inside Bioassays in an interview about a separate matter. “That’s our philosophy, to give users the highest flexibility. We’ve always had software that gave a high degree of freedom.”
Bauer’s comments actually underscore a major issue in comparative studies such as the one in question — for every scientist that complains about the lack of structure in the analysis capabilities of the Opera, there is likely one that will complain about the constraints put on the user by instruments like the ArrayScan and IN Cell Analyzer.
Rodewald also invoked a presentation by Ralph Garippa, research leader at Roche Discovery Technologies, given a day earlier in a conference session on compound screening, which Inside Bioassays also covered. That presentation, in contrast, lauded many of Opera’s features that were called into question by the poster presentation.
Garippa’s presentation, entitled “Use of the Evotec Opera Confocal Microscope Imaging System to Develop and Complete an Orphan GPCR High-Throughput Screening Campaign Using Norak Transfluor Technology,” described a screening campaign against novel members of a GPCR target class involved in metabolic diseases such as obesity.
Regarding the Opera, Garippa stated that Evotec “very cleverly came up with a way … to define the cell borders,” in an assay. Garippa also said that Roche had been “very impressed with the fast ‘out-of-the-gate’ capabilities” of the Opera.
“We had a high-throughput screen up and running within four days with no problems,” Garippa said. Furthermore, Roche was able to request specific modifications to Evotec regarding its GPCR algorithm to better fit its specific screening campaign, he said.
Garippa even cited an advantage of the Opera that is often stated about all high-content imaging systems: “You find a lot of things that you’re looking for, but you find a lot of things that you’re not looking for, as well,” he said.
To be fair, Garippa’s presentation did not present a head-to-head comparison between the Evotec system and any other cell imagers.
In another example of the inherent subjectivity of instrument comparisons, Paul Johnston, an independent consultant and former Eli Lilly research scientist, compared the performances of various Cellomics ArrayScan models against the IN Cell Analyzer 3000 and 1000 at a Cellomics user group meeting.
At the meeting, the Cellomics instrument received the more favorable review — again seemingly in contrast with Simonen’s data, which put the ArrayScan VTI and IN Cell 3000 on essentially the same performance level.
Inside Bioassays was not able to review Johnston’s poster in detail in time for this publication. However, Johnston presumably had no stake in giving a positive review to the ArrayScan.
But just as with the other presentations, this underscores another factor scientists should consider when presented with comparative data from instrument platforms — where is the data being presented and who is presenting it?