In preliminary results from its first study aimed at addressing a longstanding problem associated with 2D gels, the Industry Advisory Board of the Human Proteome Organization has determined that experiments using the technology can be reproduced.
The study, whose initial results were shared among IAB members during HUPO’s annual conference in Seoul last month, was meant to determine whether a widespread complaint about 2D gel work is valid.
The lesson of the study is that protocols that have been proven to be robust should be followed even if they may not be “the best protocols in the world,” Jan van Oostrum, co-chairman of the IAB, told ProteoMonitor this week.
Comparing scientists to chefs who experiment with recipes to enhance the flavor of dishes, he said researchers are notorious for doing the same with prescribed protocols, often with unwanted effects.
“The minute you start tweaking, you get different results,” he said.
“We didn’t know where the problems were before this, [but] we’ve now got a much, much better handle on the issues, and so we can actually to do something about them,” added Will Dracup, CEO of Nonlinear Dynamics, a member of the IAB.
Because the results are preliminary, van Oostrum, who is also executive director of genome and proteome sciences at Novartis, which is not an IAB member, declined to publicly disclose them. Instead, he and his colleagues plan to present them at the HUPO USA meeting in March.
What he would share is that experiments conducted by five laboratories involving more than 100 2D gels showed that if proper protocols are followed, reproducibility is not an issue with the technology. In particular, study organizers said they were surprised to discover that 1D separation, which has been assumed to be the “main culprit” to preventing 2D work from being reproduced, in fact turned out to be “extremely robust” regardless of the instrument that was used. By eliminating 1D separation as the source of reproducibility issues, the focus has now shifted to running temperatures and sample preparation as the problem.
And by showing that researchers working with 2D gels can, in fact, duplicate each others’ results, the study has partially solved one of the largest issues dogging proteomics.
“There has been a trend to basically state that 2D gels are not reproducible in the wider community, remarks which are mainly made by the people who use an LC-MS approach,” van Oostrum said. “Instead of entertaining an oral battle — ‘it’s reproducible, it’s not’ — why not just do a test and set up an experiment trying to see how we can make 2D gels reproducible, what is needed in order to make them reproducible if they are not in the first instance, and how can we solve this issue?”
Credibility at Stake?
Launched at HUPO’s 2006 conference, the IAB is designed to engage proteomics tools vendors to improve technology, educate the proteomics community about new research and tools, and to help develop standards in research methodology and techniques [See PM 11/02/06 and 04/26/07].
Today, the group consists of 22 companies, including proteomic alpha males Applied Biosystems, Thermo Fisher Scientific, and Waters, as well as smaller shops such as Promega, Zeptosens, and Protein Forest.
“The minute you start tweaking, you get different results.”
According to Nonlinear’s Dracup, the 2D gel study comes at a time when the credibility of proteomics as a research field has been suffering. Gone is the irrational exuberance that had some believing proteomics would be the magic bullet to cure cancer, decipher Alzheimer’s disease, and unravel the mysteries of myriad other diseases.
In its place is the sobering reality that the field is painfully complicated and it may be years before significantly meaningful results can be achieved.
“Proteomics has gotten a bit of bad press and people feel that maybe it hasn’t delivered on its promise, and some funders, pharmaceutical companies, and the like have been losing a little bit of faith in the discipline,” Dracup said.
One of the main bottlenecks to progress in the field has been the lack of reproducibility of results, on any technology platform.
“From my point of view, the ability to reproduce experiments in different labs is a fundamental scientific discipline, and it’s one that heretofore has been perceived to not be feasible in proteomics,” Dracup said. “Now with this reproducibility test that we’ve done, we’ve actually proved that it is feasible and we’ve also started to show … that when you can do cross-lab analyses, you can actually get an awful lot more out of your experiment,” he said.
The next step in the study is a repeat test on a larger scale that could involve as many as 1,000 laboratories, van Oostrum said. Such a study would include inexperienced labs and any success they have in reproducing experiments would provide greater proof of principle.
The five labs that participated in the first test were “high profile and it could be suggested that this success was more due to the skills of the participants than the capabilities of the technologies,” according to the minutes from the October meeting of the IAB.
Bio-Rad Laboratories, Invitrogen, and GE Healthcare have agreed to co-develop a protein test standard for the large-scale study and expressed an interest in commercializing the product, van Oostrum said.
Officials from GE Healthcare and Bio-Rad could not be reached for confirmation or for comment. A spokeswoman for Invitrogen could not confirm the company is involved in the 2D gel project.
Van Oostrum also said that the IAB would like to do a similar study in liquid chromatography-mass spectrometry, though he said that will be a ways off.
“We have not approached the LC people because … although I don’t have hard evidence of it, we feel they are reluctant to do a test like that, and we have not set up a good experimental protocol for it yet,” he said.
Such a test would require outside funding. The 2D gel study was paid for internally, “but that’s not something that could go on forever,” van Oostrum said. Funding efforts for a prospective LC-MS study, as well as other initiatives, will be directed at government entities.