Five teams established by the US National Cancer Institute are currently trying to explain the reasons behind the wide variability in proteomic results, and to possibility devise ways to eliminate them, according to a report by the Institute issued late last month.
According to the report by the Clinical Proteomic Technologies for Cancer of the NCI, five teams of labs chosen in late 2006 to address the issue of reproducibility in proteomics experiments came up with wide differences in their analysis of a 20-protein mixture, and the labs are now determining “why there is so much variability between laboratories and even within laboratories in an effort to address the sources of variation.”
The teams have now chosen the yeast proteome as a more complex system to continue their work to analyze the reasons for the differences in results and develop standard operating procedures, protocols, and strategies to reduce variability and thus increase the reproducibility of proteomics experiments.
“Although their work has only just begun, the ... teams have made significant progress in establishing a baseline for understanding and reducing intra-laboratory and inter-laboratory variability in the unbiased discovery stage of the proteomics pipeline,” according to the report describing the first year of work done by the five labs. “In addition, valuable resources for the proteomics community have been identified: the NCI-20 protein sample and the yeast proteome sample.”
The teams were put together under an initiative by CPTC, called the Clinical Proteomic Technology Assessment for Cancer, or CPTAC, program, under which the labs were awarded a total of $35.5 million over five years to assess proteomics technologies [See PM 09/28/06
]. Longer-range goals of CPTAC include the development of standard reference material such as samples, antibodies, data, and protocols for use by the general community.
The CPTC, which is the NCI’s principal initiative for improving the proteomics pipeline, includes other components: Advanced Proteomic Platforms and Computational Sciences, to develop new reagents, tools, technologies, and computational and statistical methods for analyzing protein and peptides; and Reagents and Resources, to develop resources such as antibodies.
In its report, the NCI said that protein biomarkers hold great potential value for the early detection of cancer, for monitoring drug reactions, and for detecting post-treatment tumor recurrence. But while more than 1,200 protein biomarkers for cancer have been reported in the scientific literature, few of them have been validated and even fewer “have found their way into clinical practice.”
“It has become increasingly clear that this dichotomy can be traced in large part to several levels of confounding variables,” including technological, organizational, and procedural hurdles, said the NCI.
Technologically, while mass spectrometry and protein arrays have the potential to decipher the complexity of the proteome, “data are being collected at a faster pace than the ability of the researchers to validate, interpret, and integrate them with other known data. The variety of platforms and standards of practice are introducing layers of variability that supplement the biological complexity,” according to the NCI report.
CPTAC was started, in part, to address this bottleneck, one of the most vexing in proteomics. The five teams comprising CPTAC are the Broad Institute; Memorial Sloan-Kettering Cancer Center; Purdue University; the University of California, San Francisco; and Vanderbilt University. An extensive network of facilities also is collaborating with the five teams.
Wide and Variable
For the initial experiments, the teams used mass spectrometry to analyze proteins and peptides in unbiased discovery. The initial work was done on a 20-protein mixture distributed by the National Institute of Standards and Technology. The proteins were cancer-related and spanned a range of concentrations in order to resemble those found in human plasma.
Each team used a platform of its choice “in order to identify how man different platforms and results would be observed across the teams,” the NCI said. Eight different instrument types as well as various liquid chromatography and data analysis tools were used by the CPTAC teams. No SOP was used.
“Although their work has only just begun, the ... teams have made significant progress in establishing a baseline for understanding and reducing intra-laboratory and inter-laboratory variability…”
Not surprisingly, the NCI said, the results showed a high degree of variability between laboratories and instruments.
A separate study was then performed to minimize controllable variables limiting the instrument platform to ion trap mass spectrometers and including a limited SOP. While this reduced variability, “it was still significant, which led to a recommendation for further SOP tightening,” according to the NCI.
Based on the simple protein mixture results and the lessons learned from it, the CPTAC teams decided to tackle a more complex protein mixture — yeast — chosen for its well-characterized proteome and the ease with which materials could be obtained.
The study used designated SOP, optimized based on the results from the simple protein mixture, but despite efforts to control for variables, high variability was observed among the different labs in the peptides they were able to identify.
In spite of this, the CPTAC teams have made significant progress in understanding and reducing variability, according to the NCI.
“Re-analysis of the NCI-20 over time creates a historic record, allowing this sample to be used in subsequent studies as a performance mixture, and the yeast proteome provides an excellent performance standard and could prove to be a valuable resource for the community for benchmarking proteomic platform performance,” the NCI said.
In addition to the reproducibility issue, CPTC is also working on improving verification of protein biomarkers, including “extending the multiple reaction-monitoring assay dynamic range at the low end of the concentration scale in order to detect lower abundance proteins,” the institute said in its report.
Each CPTAC team is applying its own individual research findings to the collaborative project:
- The Broad Institute is developing MRM assays to quantify candidate protein biomarkers in plasma and assessing the use of a workflow involving strong cation exchange chromatographic fractionation of peptides and immunoaffinity enrichment on specific anti-peptide antibodies;