UK researchers have compared three commercially available software packages for differential gel electrophoresis using fluorescent dyes, or DIGE, analysis, and have determined that Nonlinear Dynamics' Progenesis SameSpots v.3.0 "outperformed the other two software packages in matching accuracy."
In their study, which appears in the Jan. 9 online edition of the Journal of Proteome Research, the researchers compared the Progenesis package with GE Healthcare's DeCyder v.6.5 and Syngene's Dymension 3.
In the article, the researchers said that while 2D-DIGE is a powerful tool for proteomics research, the technology can provide benefits only when gel images are analyzed with software designed specifically for DIGE-based experiments.
An evaluation of such software, they added, "is necessary, so that users do not compromise valuable DIGE data by using a poor quality evaluation tool."
In an e-mail to ProteoMonitor, Judit Nagy, the corresponding author on the study and director of the proteomics facility at Imperial College London's Institute of Biomedical Engineering, added that a user's ability to use detailed functions plays a significant role in how a gel analysis software performs.
"If users understand the performance of software well, they might be able to minimize the errors caused by automation," she said.
In prior work, she and her colleagues had identified factors in media conditioned with the cell lines HepG2 and END2 and that "had a great effect" on embryonic stem cell development. The cell lines, Nagy said, "presented quite complex proteome maps, so we thought it would be [a] good test to see how software packages deal with a sample of complex nature."
In addition, they tested the three packages for user friendliness. To ensure that that aspect could be evaluated objectively, user intervention was purposely kept to a minimum. Doing so would also help Nagy and her colleagues create a standardized 2D-based proteomics workflow.
The team performed a quantitative evaluation that compared the number of spots detected, cross-matched spots, and expression differences in selected spots, along with a general evaluation of the three software packages. A second study using breast cancer cell lines was done to confirm the software packages on a set of less-complex images.
Compare and Contrast
For a general evaluation of the three software packages, Nagy and her co-researchers compared each product's visualization, normalization method, and workflow. All three packages provided "good" graphical user interfaces, according to the researchers, though Nagy said that Progenesis provided the best user interface.
For normalization analysis, Progenesis, DeCyder, and Dymension each provided univariate and multivariate statistical analysis. Progenesis, however, required less user intervention. The latest version, in particular, "provides prefilter function allowing users to exclude artifacts," while the earlier version required user intervention at the alignment stage of setting landmark vectors.
The authors define a landmark as spots determined by users "to represent [the] same proteins across replicate gels … and aligned with five landmarks."
DeCyder and Dymension required more user intervention at the alignment stage and at spot detection, although using settings recommended by the companies can keep this to a minimum, the authors wrote.
For the workflow comparison, they said that Progenesis was superior to its two rivals. "While spots of each gel were detected before the matching in DeCyder and Dymension, Progenesis SameSpots performed alignment at the beginning and spots were subsequently detected with same spot outlines," the researchers wrote in the paper. Aligned spots were automatically detected across gels in Progenesis "and full matching was performed based on the identical spot outlines without the need [for] user intervention."
[ pagebreak ]
DeCyder had an "exclusion filter" that screened non-protein spots, but the authors found that too many artifacts, nonetheless, circumvented the filter, and a strict threshold in the "exclude filter" thus needs to be applied by the user because the default setting is "too liberal," they said.
Meanwhile, Syngene's Dymension's approach was to accept and match spots only when they were present in all the internal standard gels.
GE's DeCyder was also found to be more liberal at detecting spots than the other two packages. The higher spot numbers, the authors said, resulted from the choice of inflection points for spot splitting, and a single spot detected by Nonlinear Dynamics' Progenesis was "often" interpreted as multiple spots by DeCyder.
"In general, DeCyder is more liberal at spot splitting at inflection points, which often results in oversplits, whereas Progenesis SameSpots and Dymension are more conservative at spot splitting, which results in the amalgamation of potentially different protein spots," the researchers wrote.
However, none of the three software packages completely eliminated the need for manual editing as over- and undersplitting was still a problem, and "manual evaluation of detected spots is essential if minimal user intervention is used as a strategy for alignment and matching," according to the researchers.
In Progenesis, spot detection parameters are already set and the option for user intervention is eliminated. And while DeCyder was too liberal at spot detection, Dymension may be too conservative, the authors said.
Dymension estimated fewer spots than the other two software packages, possibly due to the conservative settings, the authors said. But if segmentation settings are too conservative, "spots are amalgamated and post-translational modified proteins might be missed," they added.
In cross-matching analyses, they determined that all three packages performed poorly when landmark vectors were not used. Using a set of the top 50 spots, ranked by each software package according to the greatest fold changes, the researchers found that Decyder correctly matched 29 spots, Progenesis matched 33 spots, and Dymension matched 12 spots.
Even with five landmark vectors, Dymension still matched only 22 spots correctly, while Progenesis matched 46 spots, and DeCyder matched 36 spots.
For quantitative analysis, the researchers compared how the programs calculate fold changes of the spots. They found significant discrepancies in the estimates, contrary to pre-study expectations that the estimates between the packages would be the same.
In general, fold changes calculated by Progenesis were less than DeCyder or Dymension. The authors attribute this to the way spot boundaries are defined by each software package.
In an e-mail to ProteoMonitor this week, Nonlinear Dynamics CEO Will Dracup pointed out the study's attention to the reduction of user-controlled intervention, which he called an unmet need in the 2D-gel community.
"Our unique approach uses algorithms with an analysis approach specifically designed to address this," he said.
Additionally, while the study quantitatively compared a small number of spots from the HepG2 and END2 cell lines with a second set of less challenging images from breast cancer cell lines, "it would be interesting to compare the variation in quantitation reported for a small number of individual spots in relation to the overall levels of variance within the experiment. This may show a better picture of the overall quantitative precision of the software tested," said Dracup.
GE Healthcare took issue with several points of the study, however, and in an e-mail said that Nagy and her colleagues did not perform a spiked experiment so their study "fails scientifically on the fact that since the true final result in unknown, it is impossible to state which software did a better job."
[ pagebreak ]
A GE scientist wrote that absent a spiked experiment, "any quantitative comparison between the three software packages does only [says] that you may end up with differences in the result."
The company also said that DeCyder is currently available in a version 7, which incorporates a warping step to aid the matching algorithm.
"The warping in combination with color overlay is used to highlight areas in gels that are not entirely aligned and thus more likely to contain mismatched spots," a GE scientist said in the e-mail. "This makes it easier to find potential mismatches and assign suitable landmarks than in previous DeCyder versions."
Syngene did not respond to a request for comment.
The researchers said in their study that a future research goal "should be to eliminate user-controlled settings, either by automatic sample-to-sample optimization by intelligent software or alternative parameter-free segmentation methods."
In her e-mail, Nagy said that eventually what will be needed is intelligent software that gives a "learned decision best for the particular sample type, so that users can be totally free of decision-making, [for example] spot splitting and matching." Several research projects are under way developing such software, she added.