SAN FRANCISCO – With results in from the first set of samples in its effort to create a protein-standard mixture, the Human Proteome Organization now faces the task of figuring out why some laboratories had trouble properly identifying the proteins.
The effort is the next step in HUPO’s initiative to create the protein-standard mixture, which began a year ago [See PM 07/20/06]. In December, the organization sent out its first sample set of 20 proteins in equimolar amounts to 24 laboratories worldwide for analysis [See PM 04/19/07]. The proteins were created by Invitrogen at a purity level of between 95 and 99 percent.
The objective of the initiative, according to John Bergeron, immediate past president of HUPO, is “to assure the community that proteomics is 100 percent successful.
“The idea was to put together a high-quality set of samples, not just to benchmark the community, but also to educate the community in order to assure it can get 100 percent success characterizing these proteins in test samples,” he told ProteoMonitor at the 8th International Symposium on Mass Spectrometry in the Health and Life Sciences held here last month.
But as HUPO began receiving results, coordinators discovered that while some labs were able to identify all of the proteins, others missed some and/or identified some that weren’t there. HUPO contacted those labs and guided them in redoing their analysis until they were able to achieve 100 percent accuracy, but the organization wants to pinpoint exactly what the issues were, Bergeron said.
Currently, HUPO is depositing the raw data from the laboratories’ analyses into the Tranche database so that investigators can try to figure out where those laboratories that had trouble went wrong.
“Through that, we’re going to be to figure out why it is that some of the groups characterized proteins which are not there … and missed several proteins that are definitely there,” Bergeron said, adding that details of the results will be presented at HUPO’s annual conference in October.
“We need to know this because we will have sampled every possible bottleneck that every lab will have around the world. Any HUPO member that comes to us … we can say, no matter where they are, ‘Look, we have this test sample that we can [use to] help you in terms of assuring that you get 100 percent success in terms of characterizing,’” he said.
HUPO’s results mirror those of studies conducted by the Association of Biomolecular Resources and raise questions about the quality of some of the work being done in the proteomics community. Like HUPO, ABRF similarly found mixed results in the ability of researchers to identify and quantify proteins [See PM 02/23-06 and 04/05/07].
When ABRF reported its most recent results during its annual conference in the spring, some conference attendees expressed distress at the findings and the performance of their peers. While Bergeron said that most of the problems encountered by the labs in HUPO’s initiative were easily solved, he also said that the work done by the labs should have been better.
“Part of the issue is the degree of attentiveness that you give a sample,” he said. “In some of these labs, what happened is [this initiative] was work on top of work. The sample is taken in and the head of the lab hands it off to somebody else who hands it off to somebody else, who doesn’t know at the other end is HUPO with a publication,” Bergeron said.
For example, some of the problems encountered by the labs were due to undersampling, where labs chose to take only a small amount of the sample sent to them for analysis, a common issue in proteomics research, Bergeron said. Once HUPO told them to use larger samples, the missing proteins were identified.
“There nevertheless are some fundamental problems with some labs and those we’re going to address.”
Another problem was trypsinization. “But again, labs should be controlling for this,” Bergeron said. “And it was only after we guided them that they recognized there were trypsinization problems, instead of before, which is what we would have wanted. But there nevertheless are some fundamental problems with some labs and those we’re going to address.”
And as in the ABRF studies, the results achieved by labs participating in the HUPO initiative largely depended on the experience level of the researchers, Bergeron said.
In the next phase of the initiative, HUPO is sending out another sample of different proteins in equimolar amounts to once again test the labs’ ability to identify proteins. HUPO will also send samples to mass spectrometry vendors for protein identification.
“We didn’t want them to start too soon because, of course, they’re going to put many more resources into characterizing the test samples than an academic lab does,” Bergeron said. “I can’t conceive that that they’re going to have a problem in characterizing these samples.”
Later HUPO intends to send out another equimolar sample to once again test researchers’ ability to detect and quantify proteins. Once that is finished and the results are analyzed, it will move on to the next phase of the initiative, to have 24 labs identify 20 proteins from a set of proteins at different concentrations. The labs participating in that phase of the initiative have not yet been completely chosen.
That set of proteins is ready to be sent out, but “the only reason we haven’t [done so] is we couldn’t handle the data coming back from them yet,” Bergeron said. “Since these are now going to be at different concentrations spanning several orders of magnitude – handling all of the raw data that [will be] coming back from that is going to be even more taxing” than what HUPO is doing with the first test sample, he said.
Invitrogen has the rights to commercialize the protein standard mixtures. In the fall, Sigma-Aldrich launched the first commercially available mixture, based on a study done by ABRF.