After five years of planning, discussions, and comments, the Proteomic Standards Initiative of the Human Proteome Organization last week released its first set of modules specifically targeted to the proteomics community.
Published in the current edition of Nature Biotechnology, the three modules are meant to serve as guidelines for how and what researchers should report about experiments. The guidelines cover mass spectrometry, mass spectrometry informatics, and gel electrophoresis.
In the spring, the next set of modules covering gel informatics, column chromatography, and capillary electrophoresis are anticipated to be released, according to Chris Taylor, the cross-workgroup chair for the PSI’s initiative to define experimental reporting standards, called the minimum information about a proteomics experiment, or MIAPE initiative.
He is also a senior software engineer at the European Bioinformatics Institute and the UK’s Natural Environment Research Council’s Environmental Bioinformatics Center.
The first reporting module, which was not aimed directly at proteomics research, was released last year and called the minimum information required for reporting a molecular interaction experiment, or MIMix, [See PM 08/16/07].
Because the modules are recommendations and not rules, researchers are not beholden to them. But as the quality of proteomics experiments and the data generated from them are being increasingly scrutinized, researchers are being pressured into making their work more transparent, and the modules are one way to help them improve data reporting.
“Really, it’s about having enough information to understand the results people present,” Taylor told ProteoMonitor this week, referring to the objective of each module. “The point is at the moment, it can be hard or impossible to get [the data] you need to clearly understand how a result was achieved, and therefore you can’t really trust the conclusions.”
The publication of the three modules also marks a significant step in the PSI’s work to define standards in proteomics. The idea for creating such guidelines originated five years ago, and since then much of the work has been spent trying to reach a consensus on what should be included, rounding up volunteers to help draft the guidelines, and then drafting the guidelines and tweaking them based on feedback PSI received, Taylor said.
While each of the modules is designed for major technologies used in proteomics, the guidelines for mass spectrometry may get the most attention from researchers. In addition to being the dominant technology in proteomics, mass spectrometry is where questions and skepticism about generated scientific data are most pervasive.
|
“Really, it’s about having enough information to understand the results people present.” |
“Obviously, mass spec is a sitter,” Taylor said. “It was always going to be the one that got done first … because it’s absolutely essential to proteomics.”
The MIAPE guidelines for mass spec address the information that should be included by researchers in reporting how they did their experiments but are not meant as recommendations for one research method over another.
Information sought on the mass spec module includes the type of ion source for the instrument, the type of software used, the post-source component, and what was used to generate and annotate spectrum and peak lists.
According to Taylor, the amount of time and effort to comply with the guidelines should be “tiny … because with mass spec it’s very simple.” Similarly, compliance with the mass spec informatics guidelines require minimal work “because the whole thing is computerized. All the information you’d ever want is in a computer somewhere,” he added.
Fulfilling the guidelines outlined in the gel electrophoresis guideline will be more labor-intensive but should become easier after the initial try at compliance.
In trial runs testing the guideline, “The feedback we got was, the first time it took ages. The second time it was much quicker” as researchers became more familiar with the process and had the reporting workflow in place, Taylor said.
For the mass-spec module, tools and instrument vendors make up about one-third to half of the authors of the Nature Biotechnology paper. Despite the potential for a rift between the vendors and the academic researchers and between the competing vendors, all parties worked to develop the module with the understanding that in the end it would benefit the entire community, Taylor said.
“In terms of vested interest, there are no real losers and no real conflicts. I think everybody saw the same thing: that it was beneficial to everybody,” he said.
Still, some give and take was required: With the massive amount of data that a mass spec can generate, it would have been unrealistic to ask researchers to report every step of their work.
“You try to trade off on things that are valuable and things that are less valuable, and things that are easy to get and things that are hard to get,” Taylor said. Feedback he has received from the proteomics community during the review stage of the guideline told him that the current version of the module “is a sensible compromise between how much people are going to get [from the information being reported] and how much [a researcher] is going to have to put in,” he said.
Lifting the Curtain and Raising the Bar
The MIAPE guidelines are among a number of strategies developed in recent years aimed at raising the bar on proteomics experiments by lifting the curtain on how scientists do their work.
In 2004, Molecular & Cellular Proteomics began asking researchers for more data and greater depth of information to be included in manuscripts for publication. Since then, a number of other journals have followed suit with similar requests. Funding agencies, such as the National Institutes of Health, are also asking for more information about how researchers are conducting their experiments as part of the funding process.
The move toward greater transparency and higher reporting standards, however, has not always been welcomed by researchers. Last year, Robert Chalkley at the University of California, San Francisco, told ProteoMonitor that many scientists continue to struggle with how to comply with the journals’ requests for more data reporting [See PM 09/06/07].
And in an attempt to allay concerns that MIAPE reporting standards would create a burden to researchers, and to justify the initiative, PSI last year published a parent paper outlining MIAPE’s goals.
During the five years PSI has been developing MIAPE and the reporting-standards modules, the attitude of the proteomics community has changed, but the need for transparency hasn’t, Taylor said. Indeed, questionable experimental data and the poverty in substantial results from proteomics research have left large pharma skeptical about the science and driven the proteomics field to examine its own shortcomings. As a result, thought leaders are pushing for greater accountability from their fellow researchers.
However, researchers are still not keeping tabs on their experiments in standard ways and information about their experimental steps is getting lost as a result, Taylor said. But “the popular view is that you really need to think hard about how you do your proteomics now, so having the information about other people’s methods that allows you to be sure that they did what you personally consider to be a good job — I think people now look more favorably on that,” he said.
All the MIAPE modules are being created with the idea that they will change as technology evolves. For instance, if another major ion source were to come along and change the field, the mass spec module would be redrafted to reflect that, Taylor said.
In addition to the seven modules released or planned for release, PSI and the Microarray and Gene Expression Data Society have been discussing creating a module for peptide and protein arrays, though significant progress may not happen until next year, Taylor said.
Any such module would have to take into consideration that an array layout is very generic, and the fact that other projects such as the Minimum Information About a Microarray Experiment and the MIMix module published last year address some elements of array use.
“Bottom line [is] it’s complicated if it is to be done right in a way that serves the widest part of the community,” Taylor said.