This story originally ran on Oct. 6.
By Tony Fong
In the second phase of a campaign to test the reproducibility of 2D gel technology, 10 out of 17 laboratories, or 60 percent, were able to generate gel images that fell within a 95-percent confidence level in an inter-laboratory study.
As a result, the methods developed in both phases of the campaign, called Fixing Proteomics, now serve as standards by which proteomics researchers using 2D gels can benchmark themselves, according to the campaign's organizers.
Results of the phase 2 study were previewed to ProteoMonitor at last week's annual conference of the Human Proteome Organization in Toronto.
The purpose of the second phase of the 2D gel initiative "is to establish good training, good practice, and for people to have a goal to which they have to aim at to say, 'Now I can run a 2D gel reproducibly,'" said Judit Nagy, director of the proteomics facility at the Institute of Biomedical Engineering at Imperial College, and a member of Fixing Proteomics.
Based on the 2D gel project, the National Institute of Biological Standards and Control in the UK has committed to fund the development of biological reference materials to the proteomics community in an effort called the Biological Reference Material Initiative, which was announced at the HUPO meeting.
NIBSC has committed three years of funding to BRMI, though Nagy said there is an opportunity to extend it, depending on what results from the initiative, which will be working with Fixing Proteomics. BRMI also will collaborate with HUPO's Proteomics Standards Initiative to develop standards for biological reference materials, Nagy added.
"We're working very closely together with that initiative, and then we're working together with other initiatives to ask what biological reference materials they need, for brain, for plasma, for stem cells," she said.
BRMI, she added, will develop, stock, and distribute biological materials supported by NIBSC and the US National Institute of Standards and Technology.
BRMI has already created reference materials for plasma, which is available for free to researchers, and has now turned its attention to stem cell reference materials. "People will be able to use it in 2D and in LC-MS and in other parts of their workflow," she said.
BRMI also is continuing studies into 2D gel technology including conducting a quantitative analysis to evaluate the suitability of certain cells as biological reference materials, and using mass spec to determining which proteins are expressed differently, Nagy said during a session on BRMI during the HUPO conference.
She could not put a dollar figure on how much NIBSC is funding BRMI "because they don't earmark a certain sum," but instead the amount will depend on "what we need to make this project successful." At the moment, NIBSC is funding a full-time postdoc and materials and consumables for the work.
Fixing 2D Gel's Reputation
The interlab study is a follow-up to a study conducted two years ago looking at intralab reproducibility.
The Human Proteome Organization's Industrial Advisory Board spearheaded that phase 1 study [See PM 11/15/07]. The IAB has since relinquished control of the effort, which was begun to reestablish 2D gel technology as a relevant technology in proteomics research, to Fixing Proteomics, a campaign begun in 2007 to improve the quality of the research being done in proteomics and restore credibility to the field [See PM 09/25/08].
Indeed, while 2D gel is a major technology in proteomics — Fixing Proteomics says two-dimensional electrophoresis is "arguably the most widely used method for comparative protein profiling" — it has lost some of its cachet to liquid chromatography and mass spectrometry as a technology.
[ pagebreak ]
Hans Voshol, a senior research investigator and group leader at the Novartis Institutes for BioMedical Research, and who directed analysis of the results of Fixing Proteomics' two studies, told ProteoMonitor earlier this year that 2D gel is "challenged more frequently than LC-MS," and "nobody will challenge mass spectrometry as a useful tool in proteomics, but people do challenge 2D gels."
For both phases of the project, participants received HeLa cell lysates created by CilBiotech in Belgium. Each lab was asked to run the HeLa cell standard by 2D PAGE, following a protocol created by the studies' organizers. They then captured their 2D gel images and uploaded their analysis to a site created by Fixing Proteomics, which then compared each participating lab's results with a set of reference gel image measurements using principal component analysis.
That 10 of the 17 labs that participated in the phase 2 study were able to achieve a 95 percent confidence level is "significant because this was done [under] low stringent conditions," said Kumar Bala, marketing manager for Bio-Rad Laboratories, which participated in both phases of the 2D gel project.
"That means all you've done is give a common sample, a common protocol, and asked them to run the gels and submit an image," said Bala, who also coordinated the second phase of the project with the Novartis Institutes for BioMedical Research and Nonlinear Dynamics.
The study's organizers did not measure the different levels of variation in the seven labs that were unable to meet the 95 percent confidence level, Bala said, but added that they may do so as they perform additional analysis of each lab's results.
The labs that participated in the phase 2 study included Mark Baker's at the Australian Proteomics Facility; Phil Andrews' at the University of Michigan; Mike Dunn's at University College in Dublin; and Alexander Archakov's at the Russian Academy of Medical Sciences.
"These are all key academic thought-leader labs," Bala said. "That way, we have credibility as to where this data is coming from because a lot of people in this industry follow these labs" in how they do their own 2D work.
At first blush, a 60 percent success rate may seem modest, acknowledged Maxey Chung, an associate professor of biochemistry at the National University of Singapore, whose lab also participated in the phase 2 study. But the value of the study is that for the first time, the 2D gel community will have a reference standard to, first, do an experiment that can reproduce results within a set limit, and then run the gel standard, he added.
"Once you know you have got it right, then you do your experiment sample," he said. "Then your confidence is much higher."
Right now, people do 2D gel experiments without any idea about the quality of their gels, he added.
Nagy said that there were also problems with the HeLa samples because they were not lyophilized and so were difficult to transport to the participating labs: some did not receive the samples in time, and as a result, the samples deteriorated, which affected some labs' success.
With standard protocols and a reference set of materials created or being created, and the availability of HeLa cell lysates and a central repository for images, "we will try to make the HeLa cells available to communities that are working in that area," Bala said. "Then we will gauge what the community wants and how they will utilize it," he added, referring to Bio-Rad's commercialization plans.
In addition, training sessions are planned to educate the 2D gel community on best practices based on what was learned from the two studies, which identified some common sources of errors, including improper use of blotter strips at the end of the immobilized pH gradient, which resulted in gel distortion at the high molecular weight end.
Some participants who saw variability in their results also complained about the quality of the IPG strips that were supplied to them. But when Voshul tested the strips, he "got perfect data," Bala said, meaning whatever issues were encountered by the labs were generated on their end.
"What we learned is that it's very important, particularly for inexperienced users, to follow protocols very precisely and not cut corners. That's the key problem, not the consumable, not which vendor. It's use the same sample, and very strictly use the protocol," Nagy said.
Added Bala: "So that is what we are doing — we are forming this global community to educate each other in best practices."