BETHESDA, Md. — Could studies of retrospective data be better than sliced bread?
A retrospective study is fundamentally a poor basis for establishing biomarkers, but pre-existing clinical studies — of failed or successful drugs — might offer a wealth of information that could be plumbed for ways to reposition those drugs.
"That's what everyone is doing right now; it's the most immediately relevant" topic at DIA's Pharmacogenomics in Drug Development and Regulatory Decision Making meeting, held here this week, said Tsafrir Vanounou, associate director of biomarker operations at the Novartis Institutes for Biomedical Research. "They're going back and looking at the data to see if they have useful biomarkers."
Pharmacogenomics technology, data, and service providers such as Genaissance Pharmaceuticals and Gene Logic have been using their pharmacogenomics technologies to try to find second or third uses for failed drugs. To be sure, this methodology — shooting randomly yet often enough in the hopes that you'll eventually hit something — is frowned upon by the Food and Drug Administration because of its reliance on retrospective studies. But with careful planning, some studies may turn out to be prospective after all, Gualberto Ruano, CEO of Genomas, told Pharmacogenomics Reporter.
"I think, by far, this probably was one of the most avante-garde topics" at the DIA meeting, said Ruano. "The reason is that it's pointed in the direction of what's going to happen in the next few years … the willingness of the audience and the regulators to at least consider retrospective analysis of data with a few caveats, in terms of what were the markers, what was the biological basis of the markers, and what was the predictive power of the markers in prospective studies," he said.
Because of those caveats, retrospective studies cannot become the order of the day. "Retrospective evaluations, on the whole, are not adequate — but they are very case-specific," said Mark McCamish, chief medical officer at Perlegen.
"If you have two Phase II trials, it's clear you can make a prospective claim for the data for that retrospective data, but it all comes down to the statistical power you have," though such an approach is intended mostly for biomarker discovery, he said. Perlegen is working with seven or eight pharma companies, and "some" of those projects involve studies with retrospective data, he said.
There are two important business issues driving studies with retrospective data, said Ruano. "One is drug safety, the other is 'Can you get more information mileage from clinical trials that ran into the hundreds of millions of dollars? How can you use that information for discovery purposes?'" he said.
The two session discussions of retrospective studies each featured a different example — the first featured a hypothetical low-profit drug repositioned to another indication, while the other session featured a drug with adverse side effects in a definable population.
The process laid out by Ruano should be somewhat familiar: find markers in previously completed clinical research; find associations; and find your target populations. "If you want to use that for drug development, you need a prospective study where you use those markers to select the population," he said. "One of the less contentious issues [at the FDA is] that you need a prospective study," but the size of that study is not yet agreed upon [the FDA], he added. Ruano estimates the prospective study should be about the size of a Phase II trial.
The details of how to make respectable studies using retrospective data are still being worked out [by the pharmaceutical industry and at the FDA.]; in fact, Brian Spear, director of pharmacogenomics at Abbott Laboratories, submitted several scenarios to the audience for yes-or-no votes by a show of hands. Nevertheless, situations in which the genotype of samples is unknown until tested will provide a measure of prospective power since genotypes don't change over time. In this case, a hypothesis and study design can be laid out before the genotyping is performed while investigators can be "blinded" to the study's outcome, he said.
For a biomarker to be accepted, it also helps if it leads to treatment for an unmet medical need, along with other ethical implications, said Michael Ostland, associate director of biostatistics at Genentech Biotechnology.
Markers with some biological connection to the indication being studied are also more likely to be viewed favorably, according to several people interviewed for this article. Also, Genomas' Ruano insists that markers without biological relevance should be included for use as negative controls.
The FDA has good reason to look down its regulatory nose at the use of retrospective controls because there is no control over multiplicity. An unscrupulous investigator "can come back to a database four, five, nine, 10 times and eventually get a positive result," said Ostland. The scientific standard of 5 percent error margins assures that about one result in 20 events will be a false positive, he said. Additionally, formal control of statistical significance is difficult in a retrospective setting, especially when open-ended questions are asked, he said. "There is broad appreciation of that," he added.