Skip to main content
Premium Trial:

Request an Annual Quote

NCI-Led Study Demonstrates Reproducibility of SID-MRM-MS for Protein Biomarker Verification


This story originally ran on July 8.

By Tony Fong

Addressing one of the most formidable bottlenecks in proteomics, a team of researchers put together by the National Cancer Institute three years ago has demonstrated for the first time that a relatively new verification method is reproducible across different laboratories.

Individual labs have shown that multiple reaction monitoring coupled with isotope dilution mass spectrometry, or SID-MRM-MS, can be used to quantify candidate protein biomarkers in plasma, but until now no one had proven the interlaboratory reproducibility of the method or the transferability of the assays across different labs.

But using "common materials" and standardized protocols, a multisite collaboration has shown that different labs can come up with the same data for the same experiment. The work, described in an article published June 28 in the online edition of Nature Biotechnology, also shows sensitivity for the method to the low microgram per milliliter protein concentrations in unfractionated plasma.

The work is the first of a series of studies to come out from the working groups of NCI's Clinical Proteomic Technology Assessment for Cancer. In 2006 under that program, the NCI awarded five teams $35.5 million over five years to evaluate proteomics technologies with applications to cancer research [see PM 09/28/06].

In addition to the current study, other interlaboratory studies related to discovery proteomics are in review, Steven Carr, senior author on the study and director of the Proteomics Platform at the Broad Institute, told ProteoMonitor last week. And the working group responsible for last week's publication is preparing a "second major study" focusing on "substantially improving sensitivity as well as increasing the plex-level of [approximately] 10-plex to 100-plex," Carr said.

The current study is focused on a new application for SID-MRM-MS, a method that has been in development and use for several years now.

"The primary importance of the work … has to deal with the fact that there is a critical need for a technology that will allow us to move so-called discoveries coming out of the -omics methodologies … forward," Carr said. "The issue is that discovery experiments do not lead to biomarkers in clinical proteomics. They lead to hypotheses or candidates that need to be further credentialed."

Discovery proteomics often results in a list of tens or even hundreds of potential biomarkers, but because analyzing each biomarker can take up to several weeks and the sample numbers are low, the false discovery rates tend to run high, Carr said. That is not necessarily because of technical variability — though he added that that is something that is still not completely understood — but rather is the consequence of biological variability in the samples.

The task is to identify the changes due to pathology, but to do that, researchers need a step to help them bridge the gulf between discovery and "any possibility of clinical utility," Carr said.

Initial potential biomarkers have been typically verified through sensitive and specific high-throughput immunoassays, but such technology has been limited by the availability of well-characterized antibodies, a well-known problem in protein research. Developing high-quality immunoassays also is costly and time-consuming.

As a result, there is a need for more "straightforward quantitative approaches, exploiting the sensitivity and molecular specificity of mass spectrometry," Carr and his co-authors wrote.

In their study, they cite work suggesting SID-MRM-MS as a promising strategy for the direct quantitation of proteins in cell lysates and human plasma and serum. Several members of the CPTAC team, including Carr as well as Mandy Paulovich at the Fred Hutchinson Cancer Research Center, also were already developing new MRM-MS workflows, though other labs participating in the study had little or no experience with the MRM workflow.

At the Broad, Carr's lab has shown that such methods, combined with protein- and peptide-enrichment strategies, are able to "hit target values for limits of quantitation that are in the very bottom of the nanogram per milliliter range for proteins in blood," where many biomarkers of clinical utility reside, he said.

While such work suggests SID-MRM-MS may be suitable for the kind of work called for in biomarker verification — the rapid screening of large numbers of candidate protein biomarkers in large-scale patient samples — the adoption of the method has been tepid because of questions about its interlaboratory reproducibility.

[ pagebreak ]

"The question is 'How good are these methodologies? Are they sufficiently reproducible?'" Carr said. "And this study pretty clearly demonstrates that they are."

"In our study, the NCI-CPTAC groups systematically introduced new sources of variability up to the point where we simulated a real biomarker study. In the course of doing this, we were able to identify the largest sources of variability in the process, learn how to deal with them, and importantly come to an understanding of how the data itself can be visually represented to tell you when you have an error, and when you have an interference," he said.

Reproducing Across Labs

In total, the CPTAC working group spent about two years performing three studies assessing the "reproducibility and quantitative aspects of MRM assays … for measurements of peptides and proteins in human plasma," the authors wrote in the study. Eight labs were involved in the research.

The MRM assay configuration was performed at a single site using a nanoflow liquid chromatography system coupled to the Applied Biosystems 4000 QTrap hybrid triple quadrupole-linear ion trap mass spec. The methods and parameters were then transferred to the eight labs regardless of their instrument platforms in order to minimize variability from data acquisition.

In study 1, variability arising from the digestion of target proteins "was bypassed … by spiking a common pool of reduced, alkylated and trypsin-digested plasma with 11 unlabeled signature peptides derived from the target proteins at nine different concentrations," the authors said. In study 2, seven target proteins were digested separately, then mixed with a stock solution of labeled peptides and digested plasma, and diluted "serially with a labeled peptide/digested plasma stock to generate the same nine concentrations."

And in study 3, "which encompassed nearly all potential sources of analytical variability normally encountered," the scientists produced an equimolar mixture of the same seven proteins in undiluted plasma at the same nine concentrations.

In each of the three studies, the measured concentrations of peptides and proteins were compared to the actual concentrations across the range of spiked-in analytes. The coefficient of variation was determined for the quantitative measurements, and interlaboratory variability and reproducibility were assessed by calculating the coefficient of variation of the quadruplicate analyses at each of the nine final analyte concentrations in plasma.

For nine of the 10 peptides tested across the labs, the researchers reported interlaboratory coefficients of variation ranging from 4 percent to 14 percent for study 1; 4 percent to 13 percent for study 2; and 10 percent to 23 percent for study 3 at a limit of quantitation of 2.92 femtomoles per milliliter.

"Although the current assay performance under real biomarker conditions (study 3) is below that generally stated for clinical assays (typically less than 10 percent to 15 percent), the performance achieved is sufficient for the verification of candidate biomarkers present at more than [approximately] 2 to 6 microgram per milliliter in plasma, with a linear dynamic range spanning three orders of magnitude," the authors said.

From study 1 to study 3, the coefficient of variation progressively increased, indicating that assay variability was due to sample preparation rather than instrument variability, "further highlighting the data quality obtainable from SID-MRM-MS," they added.

"A lot of the problems had more to do with the HPLC than with the mass spectrometry," Carr said.

He and his co-researchers found differences in assay performance for different peptides, however, underscoring the importance of the selection of peptides as surrogates for the target proteins, and recommended that the final selection of signature peptides for SID-MRM-MS biomarker assays be based on multisite studies "so as to ensure the most robust performance."

While the authors wrote that SID-MRM-MS may eventually have the potential to replace certain clinical immunoassays, "especially in cases where interferences are known to exist or multiplex measurements are needed," Carr emphasized that the goal of the study was "not to put [the SID-MRM-MS method] directly in the clinic. The aim is to bridge this gap between discovery and a list of markers that you have any confidence in that you should then put in the next level of effort so that you can do clinical validation."

However, another team member, Leigh Anderson, founder and president of the Plasma Proteome Institute, is speaking with the US Food and Drug Administration about a "mock" 510(k) filing process as an initial step toward developing this for clinical use.

"We have no particular targets in mind for doing this with," Carr said, but proteins in the microgram per milliliter range or hundreds of nanograms per milliliter range and higher "are the proteins that could today, in principle, be [verified] with an MRM-based assay.

"And lower abundance proteins will be accessible using peptide and/or protein enrichment approaches as has already been demonstrated in my laboratory and Amanda Paulovich's lab at the Fred Hutchinson Cancer Institute."

All the raw data from the work has been deposited in Tranche, which Carr said is a valuable resource to the research community as there has been "essentially no source of MRM data, particularly not of this quality or abundance."

The study's protocols are described in a supplement to the Nature Biotechnology study, and all the reagents developed for the study will be freely available through the NCI.

In a statement, John Niederhuber, director of the NCI, called the findings a "potential solution for eliminating one of the major hurdles in validating protein biomarkers for clinical use … [which] has been the lack of standardized technologies and methodologies in the biomarker discovery and validation process."

This research "may solve that dilemma," he added.