Savannah, Ga. - Q-TOF and TOF/TOF mass spectrometers are better than ion trap mass spectrometers for correctly identifying the sequences of de novo peptides, and de novo sequencing computer software is not reliable, a study by the Association of Biomolecular Resource Facilties' Proteomics Research Group concluded recently.
"We knew that sequencing novel peptides would be challenging," said Brett Phinney, associate director of the Michigan State University Proteome Consortium, who presented results of the PRG's fourth annual study at this year's ABRF conference, held here this week. "From these results, we can see that the average success rate [for correctly identifying de novo sequences] was relatively low, and most core facilities need to improve their capabilities to do de novo sequencing."
For this year's PRG study, relatively large amounts (three to six picomoles) of five de novo sequences were sent out to 106 laboratories, mostly in the US and Europe. Large amounts of the proteins were sent out because the PRG organizers didn't want to make the study into a contest of whose instrument was more sensitive, Phinney explained.
Of the 106 labs, 40, or 37 percent, returned results. Of the 40, 12 used a Q-TOF mass spectrometer alone; seven used a TOF/TOF alone; eight used an ion trap alone; 19 used a Q-TOF plus another type of instrument; nine used a TOF/TOF plus another type of instrument; and 14 used an ion trap plus another type of instrument.
A scoring system was devised based on how well the research groups were able to correctly identify the sequences of the de novo peptides. The maximum score for each peptide was determined by the number of amino acids in the peptide. Non-differentiation between the amino acids isoleucine and leucine was penalized by half a point; non-differentiation between the amino acids glutamine, lysine and hydroxyproline was penalized by 0.7 points; and non-differentiation between the rest of the amino acids was penalized by a full point.
Results comparing scores with instruments showed that Q-TOF and TOF/TOF instruments scored significantly better than ion trap instruments: The average score for Q-TOFs alone was 40.2; the average score for TOF/TOFs alone was 42.5; the average score for ion traps alone was 19.5. For groups that used more than one instrument, those that used a Q-TOF plus another instrument had an average score of 41.3; those that used a TOF/TOF plus another instrument had an average score of 45.4; and those that used an ion trap plus another instrument had an average score of 26.7.
A chart of these results and other results of the ABRF study are available on the ABRF website at http://www.abrf.org/index.cfm/group.show/Proteomics.34.htm. The results are expected to be published later in the Journal of Biomolecular Techniques.
Asked to comment on the results of the PRG study in terms of ion trap versus Q-TOF and TOF/TOF performance, most proteomics researchers said the result was not at all surprising.
"The mass accuracy and resolution of ion traps is just not as good as Q-TOFs and TOF/TOFs," said David Tabb, a postdoctoral fellow at Oak Ridge National Laboratory who gave a talk on data-mining software following the PRG study presentation. "Ion traps can't do very difficult things. But the benefit of the ion trap is that they're a work horse. They can work 24 hours a day."
In terms of software used to help researchers determine the sequence of the de novo peptides, the PRG study concluded that though software saved a bit of time, it was not reliable.
Of the 40 PRG study groups, 28 percent said they used manual interpretation only; about 61 percent said they used software plus manual interpretation; and 5 percent said they used software only.
Software used to aid in sequence determination included BioInformatics Solutions' PEAKS, Applied Biosystem's Bioanalyst, and Thermo Electron's DeNovoX.
Of the groups that used software plus manual interpretation, 45 percent said they used software to aid in manual interpretation, while 55 percent said they used both software and manual interpretation, but considered their manual interpretation to be correct.
The one research group that reported using only software for sequence interpretation scored very poorly — 10.5 out of a total of 70 points.
"I think what this boils down to is that the software wasn't very good, and you all knew that," said Phinney after presenting results related to de novo sequencing software. "Software alone was not sufficient for successful sequence analysis of test peptides."
Asked to comment on results regarding de novo sequencing software, Oak Ridge's Tabb, who has spent some time writing de novo sequencing software himself, said that the software should get better as it incorporates the chemistry of peptide fragmentation into it.
"It's extremely hard" to write de novo software, said Tabb. "As software uses more realistic models, accuracy will get better."
While 28 percent of the laboratories did not identify any of the five de novo sequences 100 percent correctly, one notable group did identify four out of the five peptides 100 percent correctly. This group used a variety of derivitizations, or altering of certain amino acids, before performing the sequencing.
"They were the one exception in terms of sample preparation," said Phinney. "No type of sample preparation was notably more successful in this study, except for with that group."
At the end of his presentation, Phinney encouraged more researchers to submit their results in the future — even if the they think their results are bad.
"The purpose of this is to evaluate the capabilities of the protein analysis community," Thomas Neubert, an assistant professor at the New York University School of Medicine who chairs the PRG, said following Phinney's presentation.. "You who take part —we learn a lot from you."