SAN FRANCISCO Leaders of the Human Proteome Organization's Proteomics Standards Initiative, together with a group of journal editors and vendor representatives, last week discussed the complementary roles played by reporting standards issued by two proteomics journals and the PSI, all three of which maintain guidelines on the kinds of information researchers must submit with studies they hope to publish.
The discussion, held during a panel session at last week's PSI spring workshop here, also sought to identify the content that should be submitted to satisfy guidelines maintained by the journals Molecular & Cellular Proteomics and Proteomics and by the PSI's MIAPE.
The difference between the publication guidelines and MIAPE is that MIAPE "documents what has been done," while the publications "ask questions about the quality of an experiment, such as 'Have you done replicates?'" said Robert Chalkley, a research scientist at the University of California, San Francisco. Chalkley is a colleague of Al Burlingame, who is director of UCSF's mass spectrometry facility and deputy editor of Molecular & Cellular Proteomics.
MIAPE also includes experimental details that aren't included in the journals' publication guidelines. "There's a difference between what it means to publish a paper and the data needed in order to be able to reprocess the data," said Randall Julian, the chair of the mass spectrometry working group within PSI, and a scientist at Indianapolis-based Indigo BioSystems.
|
"There's a difference between publication and what's needed to reproduce an experiment. I wouldn't be greedy. I wouldn't want what was the date, who did the experiment, what was the phase of the moon, etcetera." |
While journals aim to enforce the quality of science published, as well as the presentation of the science, PSI's MIAPE standards aim to make it easier for scientists to reproduce experiments, and to reprocess data to answer new questions.
"The question is, 'What is the role of the PSI group in the publication process?'" said Robert Barkovich, a representative from Thermo Electron who moderated the panel discussion.
Asked why journals do not adopt emerging MIAPE guidelines wholesale into their publications, Chalkley said the issue is "that some information in MIAPE is difficult to extract out."
For example, MIAPE seeks information about who performed an experiment, when it was performed, and on what instrument was it performed. For some scientists, that information is cumbersome to obtain, said Lukasz Salwinski, an assistant researcher in David Eisenberg's laboratory at the University of California Los Angeles.
"There's a difference between publication and what's needed to reproduce an experiment," said Salwinski. "I wouldn't be greedy. I wouldn't want what was the date, who did the experiment, what was the phase of the moon, etcetera."
However, Chris Taylor, a leader of the PSI who heads the main committee on minimum reporting requirements, pointed out that in some cases, results tend to cluster around the researcher who performed the experiment.
Angel Pizarro, the director of the bioinformatics facility at the University of Pennsylvania's Institute for Translational Medicine and Therapeutics, agreed with Taylor that data does sometimes cluster around a particular person, day of the week, or instrument, but said that nevertheless, those details should not be a reporting requirement.
"We'll never have a complete list of confounding factors," said Pizarro, who used to work on standards development for the Microarray Gene Expression Data Society. "When you dictate what you have to report, the implication is that you're telling scientists they don't know how to do good science. The solution is to let the data speak for itself, and to require quality assessment statistics."
Asked what mechanism journals should employ to determine which facts should be included among reporting requirements, Veronique Kiermer, the editor of Nature Methods, said that the consensus for standards must come from the community, rather than from any particular group of individuals.
"Journals can facilitate the gathering of information," Kiermer said.
Hans-Joachim Kraus, senior publishing editor of life and medical sciences at Wiley-VCH Verlag, which publishes Proteomics, said that the PSI is in a good position to develop reporting standards.
"PSI has a good composition vendors, scientists, major journals in the field," he said. "I think it is the right composition."
Taylor noted that while journals have an ad hoc mechanism for revising their reporting standards once they have been put out to the community, PSI has a formalized mechanism through PSI meetings during which officials can hear feedback and revise reporting standards.
|
"We need a one-stop-shop for those who are trying to find the appropriate guidelines for their workflow." |
To make it easier for researchers to comply with the growing number of reporting standards, scientists at the PSI meeting proposed that a central registry of reporting standards be constructed.
"We need a one-stop-shop for those who are trying to find the appropriate guidelines for their workflow," said Taylor.
Actually, Taylor and some collaborators have begun to develop a concept for such a central registry. For this registry, which Taylor has temporarily named "MIxxx," as in Minimal Information … , Taylor proposed that all relevant reporting standards, such as MIAME, MIAPE, MSI, MIMIx, MIGS, MIACA, MIFACE, MIAMET, MISFISHIE, MIRIAM, and MIAO, be registered in the registry. Then a group of scientific peers will examine the way the standard overlap each other.
MIxxx would then create informal working groups to examine the intersections between different reporting guidelines, and to develop intersection modules to streamline the flow between standards.
"We'll need a Minimum Information About a Minimum Information Document!" said Taylor, laughing. Then, realizing that such a thing actually would be needed, Taylor said, "That is exactly what you need. 'Standards' is not a word to turn around lightly."
As far as enforcing reporting standards, scientists and editors at the PSI meeting agreed that the standards could not be made mandatory until there are some tools developed to enable compliance.
"Unless you really impose [the standards], it's never going to be accepted," said Kiermer. "But until there are the enabling tools in the community, there is a reluctance to impose them."
Tien Shun Lee ([email protected])