A controversial genome-wide association study that identified a set of SNPs that appear useful for predicting human longevity came under increased scrutiny this week when researchers familiar with such studies questioned the design of the project and the platform it used.
In their paper, which appeared July 1 in Science online, the Boston University-led researchers described how they genotyped 1,055 individuals aged 100 years and 1,267 controls and identified 150 SNPs they claimed could predict with 77-percent accuracy those individuals who would likely live beyond 100.
While the study garnered considerable media attention, longevity researchers unrelated to the Science paper have since questioned the study's experimental design and its use of different iterations of Illumina genotyping chips.
These public critiques, which initially came via stories in Newsweek and The New York Times, have prompted the BU researchers to reanalyze their findings and predictive models.
The paper's critics have also raised questions about how such studies are being conducted and what the most appropriate way to perform association studies might be.
"Some researchers have indicated via the press their concern about a couple of genetic markers out of the 150 that are used as a group to differentiate centenarians from controls and for the construction of the genetic signatures," study co-author Thomas Perls, associate professor of medicine and geriatrics at BU, told BioArray News this week.
"It is not surprising that the paper has received an unusual amount of attention given the novel and powerful methodology introduced in the report and the provocative phenotype that we studied," said Perls. "Our initial re-analyses are encouraging, but we need to be sure that all the appropriate analyses are performed and double-checked by other independent and qualified scientists.
"Once this is done, then everyone will have the answers they need to adequately assess the findings of this novel report," he added.
While Perls and fellow researchers reanalyze their data, BioArray News this week spoke with other scientists conducting similar, array-based studies on longevity. In separate interviews, some questioned the BU team's findings and said that the number of variants associated with exceptional longevity seemed unlikely given the results of other association studies to date.
"I read the paper and supplementary material last week and also had serious doubts about the reported findings," said Peter Visscher, a senior research fellow at the Queensland Institute of Medical Research in Australia.
"Even leaving the prediction accuracy question aside, just looking at the results of the SNP-by-SNP association analyses, they seem odd and too good to be true in comparison with the entire body of evidence on genome-wide association studies for complex traits to date," Visscher told BioArray News.
Visscher argued that the BU study was poorly designed, noting that cases were genotyped on two different Illumina arrays, and that the controls were taken from existing databases.
Visscher also faulted what he called the researchers' "apparent lack of experience" dealing with genetic data.
Quality control is "vital for all GWAS studies and a number of the reported results point to potential problems," Visscher said. "Real associations are usually detected with multiple SNPs because SNPs are correlated themselves, yet the authors don’t seem to find that [their] top hits are detected by association with multiple SNPs.
"The authors test each SNP over multiple genetic models, which … create more false positives."
Another scientist, André Uitterlinden, director of the Netherlands Consortium for Healthy Aging, called the BU study "flawed," and "suspiciously surprising with totally unexpected results."
Uitterlinden, a human geneticist at Erasmus Medical Center in Rotterdam, is currently running a GWAS for longevity genes and age-related diseases via the NCHA. Like Visscher, Uitterlinden criticized the design of the BU experiment.
"For a typical GWAS, by current standards, the study has a relatively small sample size of around 900 cases in the discovery set ... and an even smaller sample size in the replication cohort; of around 300 cases," he said.
Citing "current GWAS protocols," Uitterlinden argued that a replication sample "should be at least of the same size and preferably much larger and also consist of multiple, totally independent replication sets." He said the BU team did not meet these "gold-standard requirements."
Uitterlinden also said that the results of the replication, where 70 significant hits were replicated in 33 cases, were "very surprising" and "very rarely seen in GWAS," in which researchers are "accustomed to seeing a handful at most of genome-wide significant hits, if any, given the small sample size in the discovery set of this study."
Uitterlinden also criticized the arrays used in the study. According to the paper, most of the cases were genotyped using Illumina's HumanCNV370-Duo. But about one-tenth of the cases were run on the firm's Human 610-Quad BeadChip.
The controls were ultimately genotyped using four different tools. Some users familiar with the chip, such as Decode Genetics founder Kári Stefánsson, noted publicly that the 610-Quad occasionally contains defective loci, and argued that several of such loci were identified as longevity-associated SNPs in the BU study.
This "technical debate about the particular SNPs they report and the associated Illumina 610-Quad array problems for these SNPs, gives me reason to hold very strong reservations about the validity of the conclusions of the paper," said Uitterlinden.
"I think Science should not have published this paper and it makes me believe that the journal was more impressed by the topic, rather then by the substance and validity of the results," he said.
[ pagebreak ]
'A Yellow Flag'
According to Eric Topol, director of the Scripps Translational Science Institute, one reason the BU study has received so much attention — both laudatory and critical — is because of interest from aging Baby Boomers.
"The response of the media has been surprising because this is very 'inside baseball' technical stuff, but it's mainly because the topic is of such popular interest," Topol told BioArray News this week.
"Longevity, because of the world we live in, with the Baby Boomers aging, is one of the hottest topics today," he said. "This paper touches on a public issue;" he added. "If it hadn't touched on such an area, it might not have received as much attention."
While Topol praised the subject of the study — a paper containing the results of a Scripps-led GWAS of healthy individuals is currently under review — he said that issues with the arrays used in the BU study "mandate" the reanalysis of the results, and that it is not clear how that reanalysis will affect the BU researchers' findings.
Topol said that researchers should "try to avoid" using multiple chips in one study. He noted that the Scripps study relied only on the Illumina Human1M BeadChip. "That's something to keep in mind, that's a lesson," Topol said of using one platform. "That's a yellow flag you always have to be concerned about."
For researchers who use different chips, "you have to look at data as a function of each chip to see if there are any disparate findings," Topol said. "The BU investigators have already agreed that they have a couple of bad SNPs and they'll take them out and do the reanalysis, but the problem is what that reanalysis will look like," he said. "These were among the top SNPs that were spear-headers."
Still, he said that some of the genes that were found are "very strongly implicated in longevity" and Alzheimer's disease. "They are biologically quite plausible and I think there are some important findings within this paper," said Topol.
"My interpretation is that the investigators did the best they could with respect to their analysis," he said. "I don't think there was any intention to be anything other than forthright in their conclusions."
Timothy Donlon, director of the molecular and cytogenetics laboratory at the Queens Medical Center in Honolulu, said that regardless of the outcome of BU's reanalysis, the findings will have to be replicated by other researchers before they become believable.
"Though I was initially excited about these finding I also had serious misgivings about them," Donlon told BioArray News this week. "Whenever one sees the results of a whole-genome scan, one has to be very leery of the results, especially since this study required so much data manipulation."
He noted that GWAS is "very complex and requires, out of necessity, the need for a large amount of data to be discarded." Because of that, Donlon said it is "very easy to run astray" and, "for this reason, no one who is actually doing GWAS believes their own data until another group can replicate their findings. The more the merrier, only then it is time to crack open the bottle of bubbly."
According to Donlon, "time will tell whether these variants are truly associated with exceptional longevity, but I do know one thing: there is no bad press when it comes to aging research. The public will remember these findings, whether they're validated or not."