Washington, DC — As leaders and members of the Human Proteome Organization’s Proteomics Standards Initiatives convened here this week for their annual fall workshop, initial feedback from reviewers of PSI’s plan to develop and issue reporting standards revealed deep skepticism and anxiety.
At the workshop, about 70 scientists and researchers met to get an update on the numerous efforts underway to create standards for proteomics and to continue building a roadmap for further work.
But at a session held during the first day of the three-day workshop, comments by scientists who reviewed PSI’s minimum information about a proteomics experiment’s “parent paper” — a broad overview of the goals of PSI’s standardization efforts — suggested that the real work may be in convincing others in the proteomic field that standards even need to be created.
The underlying principle of the parent paper — and indeed the whole of PSI’s work — according to PSI leader Chris Taylor, is to facilitate the understanding of how proteomic experiments are conducted and to create standards of reporting so that researchers can recreate each others’ work.
It was submitted to the journal Nature Biotechnology, which is planning an issue focusing specifically on standards in the scientific community.
Some of the comments made by the reviewers about the parent paper commended PSI for its efforts, saying that as the amount of proteomic data approaches the amount of gene expression data, the ability to query that information efficiently will become increasingly necessary. Others praised PSI for its balance in its approach.
But many reviewers also questioned the wisdom of PSI’s work. Some wanted to know why HUPO is even trying to create these standards since they see no great clamoring for them. Others wanted to know what MIAPE would enable them to do that they currently can’t do already.
And others complained that writing rules is easier than complying with them, and that MIAPE standards will ultimately create a burden to labs that don’t have the informatics capability to implement the standards.
“The criticism is that there’s a closed set of people who work in journals, people who work in funding, and us,” who are pushing for standards, said Taylor. A whole range of complaints, he said, could be boiled down to essentially saying, “I don’t want to do it.”
Members of PSI responded to some of the criticism by saying that the group had either already addressed or was in the process of addressing the concerns. One reviewer suggested “the consortium behind MIAPE” should provide the tools and methods for complying with MIAPE. In response, Hemming Hermjakob, chairman of the PSI steering committee, said that PSI already had developed some tools and was developing others. The comments, he said, “are responding to one paper and not seeing that PSI has a bigger picture.”
Given the nature of the work and fact that the PSI is trying to push for standards in a field that has virtually none, the criticisms did not surprise Taylor, he said. But the amount of criticism is a signal that PSI may be taking for granted that the proteomics community understands the purpose of the group’s work.
Several attendees said that PSI needs to make clear in the future that even if a researchers cannot see the benefit of MIAPE to their work, in the end, science benefits from MIAPE.
“Generally, we need to do a better selling job,” Taylor said.
Since the spring, the PSI has submitted 12 manuscripts to Nature Biotechnology, and according to Hermjakob, the review process has turned out to be more draconian than originally anticipated mainly because finding reviewers not affiliated with PSI’s work and willing to review the work has turned out to be problematic.
“The criticism is that there’s a closed set of people who work in journals, people who work in funding, and us [who are pushing for this].”
After being reviewed by Nature Biotechnology and invited “opinion leaders,” the manuscripts will be posted on the journal’s Web site for public viewing and comment.
So far, only the guidelines for molecular interactions have reached the public comment stage. Those interested in seeing the guideline and commenting on it can see it here
The MIAPE parent paper has been peer-reviewed while 10 other sets of guidelines are either being peer-reviewed or have been accepted for review by Nature Biotechnology.
In addition to the update on the Nature Biotechnology focus issue, other topics addressed during the three-day workshop focused on whether PSI’s molecular interaction schema can be applied or extended to protein binders and how to provide a format as a means for data exchange between antibody resources such as those being built by the National Cancer Institute and the Swedish Protein Atlas.