Last summer, the National Cancer Institute launched the second phase of its Clinical Proteomic Technologies for Cancer initiative – the Clinical Proteomic Tumor Analysis Consortium, a five-year, $75 million- to $120 million-project aimed at discovering and verifying protein biomarkers linked to various forms of cancer (PM 8/26/2011).
Recently, though, several prominent researchers have called into question the program's design, suggesting that instead of focusing on discovery and verification of new markers, the initiative's funds should be put toward validation and implementation of existing candidates, with the aim of demonstrating proteomics' usefulness as a clinical tool.
These researchers' critiques are part of an ongoing debate within proteomics – and, indeed, the broader medical and biotech community – about the appropriate levels of support for basic research and discovery efforts versus applied and translational work.
As one of the world's largest and best-funded clinical proteomics programs, CPTAC-2 is an obvious target for such concerns.
The initial CPTC initiative, which included the Clinical Proteomic Technology Assessment for Cancer, or CPTAC-1, project, was a five-year, $104-million effort that NCI kicked off in 2006. That program focused on technology development and standardization, addressing the lack of reproducibility and transferability of proteomic techniques across laboratories and experiments.
According to Leigh Anderson, CEO of clinical proteomics firm SISCAPA Assay Technologies and a participant in the CPTAC-1 initiative, many proteomics researchers hoped that CPTAC-2 "would attempt to implement a complete biomarker pipeline and determine for the first time how much clinical value there is to be found among the thousands of published biomarker candidates."
Instead, the new initiative has confined itself to biomarker discovery and verification, with a particular emphasis on linking proteomic characterizations done by the project's eight proteome characterization centers to genomic characterizations done by the NCI-funded Cancer Genome Atlas.
"CPTAC-1 was about providing some analytical control on measurement and systems," Russell Grant, strategic and national director of mass spectrometry at Laboratory Corporation of America, told ProteoMonitor. "CPTAC-2 was where we kind of considered, 'OK, here's where someone says, we've got some degree of [technical] control. Let's take some markers'" through to clinical validation.
"Unfortunately, [based on the current design of CPTAC-2], it doesn't appear that that is going to be the case," he said. "We're going to go back to biomarker discovery and we're never going to understand the steps that will bridge from a biomarker discovery phase through a verification phase … to a clinical validation phase then into a clinical utility phase."
Last month at the Association for Mass Spectrometry's Applications to the Clinical Lab annual meeting, Grant raised questions about CPTAC-2 in response to a presentation by Swiss Federal Institute of Technology Zurich researcher Ruedi Aebersold that suggested that future biomarker discovery would increasingly rely on data from genomics research rather than comparative proteomics experiments and that proteomics would ultimately prove most useful as a tool for validation and clinical implementation of biomarkers.
During the Q&A period following Aebersold's talk, Grant asked who would perform such validation and clinical work, noting that "CPTAC-2 has gone back to biomarker discovery mode."
"[CPTAC-2] was a wonderful opportunity for us to move to analytical and clinical validation," he said. "So that's one opportunity that I think we're missing."
In response, Aebersold agreed that if the CPTAC-2 funds were "used [not] to find new marker candidates [but] to actually test [existing candidates] then the money might lead to better output."
"Given the amount of money available – and we all realize that money is limited – we would get more output for the money if we avoided the extremely expensive and tedious discovery [process] and focused on validation," he said.
In an interview with ProteoMonitor following up on his remarks at MSACL, Aebersold reiterated his critique, noting that, while CPTAC "is a great program" in that "it puts finally a reasonable amount of resources focused on this problem" of clinical proteomics, he was skeptical of the initiative's discovery aims.
Pursuing the Genomic-Proteomic Link
Aebersold applauded CPTAC-2's plans to use the TCGA's genomic data as part of its discovery process, calling efforts in that direction "very encouraging," but he noted that the initiative might not take these efforts as far as he would hope, observing that what he knew of the proteome characterization centers participating in the project suggested that they were, in his opinion, "reasonably conservative in their approaches."
Asked how he might have structured the project, Aebersold said he would have "tried to get tumor and control tissue from patients for whom [these tissues] had been sequenced" and then "tried to infer" from this genomic data the affected pathways and networks.
"From that I would have established proteins that could be indicators of the activity states of these networks, and measured these in tissue and plasma," he added.
CPTAC-2, on the other hand, while using the TCGA's data, appears to be pursuing a less targeted use of that genomic data than Aebersold proposed.
CPTC director Henry Rodriguez declined to be interviewed for this story, but in an e-mail to ProteoMonitor he said that the project's design stemmed from a determination that "complementing a genomic characterization pipeline with a proteomic characterization pipeline could produce a unique continuum that defines the proteins translated from cancer genomes, enabling the research community to better connect cancer genotype to phenotype."
CPTAC "represents a network of proteome characterization centers that coordinate research approaches and data sharing in order to comprehensively interrogate genomically characterized tumors (discovery), followed by downstream measurements (verification) to confirm interesting findings," he said. "The purpose is to identify protein changes that derive from alterations in cancer genomes, and to provide these data with accompanying assays and protocols to the public."
In an e-mail to ProteoMonitor, SAT's Anderson noted that "obviously it is preferable to have discovery informed by genomics." However, he added, CPTAC-2's discovery focus nonetheless "to me looks like a retreat from the hard problem – verification – by kicking the can down the road while looking once more for an easier way around.
"While discovery technology can always be improved, and new features – like splice variants, phosphosites, glycosites, etc. – emerge that might yield new classes of markers, failure to then demonstrate actual clinical value in adequate numbers of real samples renders 'discovery' essentially meaningless," Anderson said. "Since almost 25 percent of all human proteins have been identified as candidate markers through discovery proteomics, and none of these have been approved by the [US Food and Drug Administration] as new clinical tests, it is clear that successful verification has not been occurring."
In order to deliver clinical tests, "we need high-sensitivity, high-throughput specific assays to perform reliable verification of candidate biomarkers, and large sets of high-quality 'unbiased' clinical samples in which to do the testing," he added. "Work on these two components lacks some of the glory of proteomics-style biomarker discovery, but they are the critical missing pieces of the pipeline."
A Shift in Priorities?
The matter of how best to prioritize proteomics discovery versus biomarker validation is not, of course, a new issue for the field. As large numbers of protein markers have continued to stall on the road to the clinic, though, disillusionment with conventional proteomics discovery techniques has grown.
In an e-mail to ProteoMonitor, Leland Hartwell, co-director of the Center for Sustainable Health at Arizona State University's Biodesign Institute, said that in his institution's work with institutes in Taiwan and China, "we are encouraging them not to do discovery but to cull the reported markers in the literature for head-to-head comparison on good clinical samples."
This, Hartwell noted, marks a shift in his views on the question. Several years ago he was a champion of further biomarker discovery efforts, but, he said, "there has [now] been a decade of intense discovery work with transcriptomics and proteomics. The literature is now bulging with undigested preliminary reports. Let's exploit them."
"I think we should at least test what has been reported first with appropriate control and disease biospecimens before doing more discovery," he said. "Until that is done we won't know if real biomarkers exist in the reported literature."
Mary Lopez, director of Thermo Fisher Scientific's BRIMS Center, suggested to ProteoMonitor, however, that the discovery-validation question is something of a false choice.
"It's like asking, 'Do I need my left brain or my right brain, and which one should I prioritize?'" she said. "You can't do without either of them. They're synergistic."
Citing her team's investigations with ASU researcher Randal Nelson into protein microheterogeneity, Lopez suggested that the bulk of existing protein biomarker candidates would prove too non-specific to be clinically useful and that significant discovery work focused on protein isoforms remained to be done.
"I think one of the things we haven't realized in proteomics is that it's not just the complexity of how many proteins are in a sample, it's the complexity of what forms of those proteins are in a sample, and so we need to be specific in which parts or isoforms we measure," Lopez said.
In September, Thermo Fisher acquired Intrinsic Bioprobes – a proteomics firm founded by Nelson that specialized in immunoenrichment-based sample prep tools for mass spec-based biomarker workflows and for quantitating levels of different protein isoforms, in particular (PM 9/2/2011).
Using the IBI-developed mass spectrometric immunoassay, or MSIA, technology, Lopez has conducted a number of studies investigating protein microheterogeneity and its implications for protein-based diagnostics, including work on parathyroid hormone that was published in the February 2010 issue of Clinical Chemistry (PM 7/16/2010) and a study on using apolipoproteins to distinguish between ischemic and hemorrhagic stroke that is currently in press at Proteomics Clinical Applications (PM 2/25/2011).
Lopez said that she and her BRIMS colleagues are also working with two of the CPTAC-2 groups – Dan Chan's at Johns Hopkins University and Reid Townsend's at Washington University in St. Louis – to "provide [them] with the MSIA technology and help them with their discovery work and [SRM-based] verification."
"I believe the pushback people are giving right now, saying, 'We've tried to do proteomics discovery for so many years and have gotten nowhere, let's just focus on validation,' is a reaction to the frustration that the methods and technologies available to do classical [proteomics] discovery have proven to be extraordinarily difficult," she said. "But what that means is not that we throw the tools out or the approach out or the logic out. It means we revisit how we do this, and we learn from some of the mistakes that we've made to come back with an approach that can really yield some fruit."
In addition to the emphasis on protein microheterogeneity, Lopez suggested that advances in mass spec technology have made discovery experiments more feasible, noting that her team at BRIMS has devised a two-stage mass spec workflow consisting of an initial full-spectrum scan followed by a second scan focusing on particular peaks of interest with which they have been able to analyze plasma proteins across seven orders of magnitude. The technique is also high throughput enough to enable the researchers to analyze enough samples to take into account some of the biological variability that has stymied past biomarker discovery efforts, she said.
"We can't just throw our hands up and say we're just going to look at things found from genomics," Lopez said. "Why do you think so much emphasis was put on discovery to begin with? It's because we don't have any good markers."
Anderson acknowledged the potential of investigations into protein microheterogeneity, but, he said, "it's going to take additional time, maybe quite a bit of it, to find out which of the semi-infinite heterogeneous forms are clinically relevant in the face of population heterogeneity." And, ultimately, this too, he added, would require some sort of validation work.
Beyond that, Anderson said, it remains an open question as to whether "the clinical diagnostic information [is] to be found in changes in the amounts of proteins made or released by various cells or tissues" or in measures of different isoforms.
"So far there are more changes in amount … in known markers, and so I'm not in favor of banking everything on a new day of [isoform-based] discovery," he said.
In large part, the critiques of CPTAC-2 stem from frustrations at the difficulty of obtaining resources to do large-scale clinical validation of protein biomarkers and the feeling that the program could have provided such an opportunity.
"There's not a big pot of money in industry or academia [for biomarker validation and translation]," LabCorp's Grant said. "There's more money in biomarker discovery, as is evidenced, I think, by some of where CPTAC-2" has focused.
"That's a problem," he said. "There's no sense in looking for more [markers] if they just fall off a cliff. In some examples, industry will provide [validation], but in many situations, if it's not funded by the appropriate government agencies, we'll just get a lot of lemmings."
"I think there are many reasons why the research community, including CPTAC, is reluctant to push verification," Anderson said. "It's usually much harder to get a grant, or tenure, to verify – usually unsuccessfully – a previous 'discovery,' particularly given the strong resistance to publishing negative results."
"Perhaps more important, I'm afraid that many participants in biomarker proteomics don't have enough confidence in what they find to take the candidates to a serious test," he said. "It's more attractive to run another discovery experiment to get 'better' candidates."
"This," Anderson added," is where the [National Institutes of Health] could provide leadership by expanding and improving verification as a form of put-up-or-shut-up test. There is obvious risk in doing that prematurely, but it has been viewed as premature for a long time now, and serious people begin to wonder if there is a 'there' there."
Lopez acknowledged the difficulty researchers have had obtaining resources for validation work. "I get it," she said. "It's true, and I think that NIH should listen to that complaint and shell out some money for verification of existing markers."
She added that vendors may also have a role to play in this process. "If the government isn't funding that kind of pragmatic work so much, I think a certain amount of responsibility could fall on the vendors to demonstrate that the technology is up to snuff with respect to a particular application," she said. "It certainly interests [Thermo Fisher] to demonstrate that a collection of technologies we are manufacturing can deliver the requisite specificity, sensitivity, and robustness for a clinical application.
"We're going to have to do our homework and the legwork with collaborators to provide the fully developed tools and applications to companies like LabCorp," Lopez said. "It's a problem, because we're a bit at the beginning of the mass spec assay [for proteins] in the clinical lab world. But just because we have a lot of work to do, doesn't mean it's not doable."
Have topics you'd like to see covered in ProteoMonitor? Contact the editor at abonislawski [at] genomeweb [.] com.