Tobacco giant Philip Morris International is partnering with IBM in an effort to advance the industrial applications of systems biology and computational modeling.
The initiative, called Industrial Methodology for Process Verification in Research, or IMPROVER, kicked off this week with the Diagnostic Signature Challenge — the first of several challenges planned over the next four years.
IMPROVER builds upon similar challenge initiatives in the academic world — such as the IBM-led DREAM (Dialogue on Reverse Engineering Assessment and Methods) project and the CASP (Critical Assessment of Protein Structure Prediction) initiative — but is intended to focus on the verification of processes that would be of use in the industrial setting, according to the project organizers.
Hugh Browne, R&D scientific spokesperson for PMI, told BioInform that the effort grew out of the company's internal systems biology research, which is directed toward the aim of developing new tobacco products that are "less risky" than those it markets today.
PMI views systems biology and computational modeling as a promising method for predicting the health risks of these products as a complement to clinical studies, but the company's scientists have been frustrated that there is currently no "standard method of verifying their conclusions," Browne said.
As a result, he said the company began working with IBM in 2009 with the aim of organizing something along the lines of DREAM, but targeted specifically toward the needs of industry.
Browne noted that systems biology approaches have applications in a range of industrial fields, including biopharmaceuticals, nutrition, environmental safety, and consumer product development. As such, he believes that any effort to help companies assess their methods would be of great interest in the commercial world, since such tools have been lacking to date.
"Although industry shares many of the same needs for validation as academia, a methodology for verifying research is needed in the industrial setting that recognizes both speed and protection of proprietary data constraints, as well as the importance of market considerations and consumer protection," researchers from PMI, IBM, and elsewhere noted in a paper describing their vision for IMPROVER, which was published in Nature Biotechnology last September.
Browne explained that IMPROVER participants will not be required to disclose the fine details of their methods if they fear that it will compromise their intellectual property position. "We want to attract the broadest possible community," including commercial firms who may consider one another to be competitors, he said. "IP is important, so if participants feel they need to protect that, then they should do that."
He noted that neither PMI nor IBM will have any claim over any of the methods submitted to the challenge.
Browne said that the IMPROVER organizers view the challenge as a complement to the peer-review process, not a replacement for it. Computational methods published in the literature often include self-assessment against competing methods, but this approach can lead to bias through selective reporting of performance or cherry picking of the best metrics.
An objective, external evaluation framework could help other researchers more easily determine the best computational methods for their needs, he noted.
Joerg Sprengel, senior managing consultant at IBM, added that the IMPROVER framework might even be of interest to publishers themselves since it would provide a third-party means of evaluating the performance of computational methods.
The First Challenge
The goal of the first IMPROVER task — the Diagnostic Signature Challenge — is to assess and verify computational methods used to classify clinical samples based on gene expression data. The challenge comprises four sub-challenges — covering psoriasis, lung cancer, chronic obstructive pulmonary disease, and multiple sclerosis — and is open to academic and research teams or individuals.
Participants will be provided with a training data set in order to build their classifiers. They will test their methods against unpublished datasets containing diseased and normal tissue.
Classifiers will be assessed by an independent panel of scientists from academia and industry. The winner of each challenge will receive a cash award of $50,000. Results of the project will also be published in a peer-reviewed journal.
IBM's Sprengel said that he hopes to see participants apply their methods to all four sub-challenges with the aim of finding a solution that "does well in all cases," though he noted that teams can also select a single sub-challenge.
He added that the IMPROVER organizers decided to kick off the challenge with transcriptomes data because they wanted to "start simple." Researchers have been developing gene expression classifiers for well over a decade, he noted, so it was a logical starting point for the new initiative. "We didn't want to make too complicated a hurdle" for the first challenge, he said.
Future challenges will likely include other types of data, including proteomics, metabolomics, and next-gen sequencing data, he said, though he noted that those details have not yet been decided.
Registration for the Diagnostic Signature Challenge is open now and the submission deadline for predictions is May 30.
The IMPROVER organizers plan to begin evaluating the submissions in June, and will announce the winners at the end of that month. Awards will be presented at a symposium to be held in New York Aug. 28-29.
Have topics you'd like to see covered in BioInform? Contact the editor at btoner [at] genomeweb [.] com.