Skip to main content
Premium Trial:

Request an Annual Quote

DREAM Challenge Models Suggest SNPs Won't Improve Rheumatoid Arthritis Treatment Response Prediction


NEW YORK (GenomeWeb) – A project that brought together computational methods and modeling approaches from dozens of research groups suggests common SNP profiling is likely not the most promising avenue for unearthing markers for predicting treatment response in individuals with rheumatoid arthritis.

Researchers from more than 70 teams put forth modeling methods to evaluate the possibility of using SNP profiles to predict response to so-called anti-TNF treatments, which target the tumor necrosis factor-alpha inflammatory cytokine in individuals with rheumatoid arthritis. 

The effort was organized through a Dialogue for Reverse Engineering Assessment and Methods (DREAM) challenge network, established by IBM and Sage Bionetworks, in collaboration with investigators at Merck Research Labs, Mount Sinai School of Medicine, and the Massachusetts-based registry data and consulting services company Corrona.

"There's a lot of work going on looking at genetic markers of treatment response and not a lot of success," noted Sage Bionetworks President Lara Mangravite, corresponding author on the team's report in Nature Communications this week.

For that study, the group set out to "take a principled, crowdsourced approach to assessing whether this is a useful line of inquiry or not," Mangravite told GenomeWeb, "so drawing a line and saying, 'Should we be doing this as a community or should we be thinking about other approaches? Let's take a principled look at that question.'"

As it turned out, the predictive power of the proposed algorithms for evaluating anti-TNF treatment resistance in rheumatoid arthritis were not enhanced by folding in common SNP data.

Even so, the effort has spurred new lines of inquiry from at least some of the DREAM challenge participants, including co-senior author Yuanfang Guan, a computational medicine and bioinformatics researcher at the University of Michigan.

"I believe this is an essential first step, to have a benchmark [for future studies of treatment response in rheumatoid arthritis]," Guan told GenomeWeb. "We are actively working on finding new biomarkers through functional genomics — kind of complementary to this approach."

The DREAM framework has been used to "study the most fundamental problems in computational medicine," noted Guan, who has participated in many of the challenges. "To me, the most valuable part of all DREAM challenges is that it provides an avenue to exchange technology and ideas from the top experts in the field."

IBM's Gustavo Stolovitzky and Andrea Califano at Columbia University spearheaded the DREAM challenge strategy nearly a decade ago. It has since been applied to a host of problems — from the detection of cancer-related genetic mutations to methods for diagnosing and treating conditions such as Alzheimer's disease and AML.

"DREAM itself has become a pretty well known community — it's a big volunteer community," Mangravite noted. "It's very collaborative. … People approach us with ideas and we vet the idea relative to whether a challenge is a useful approach to address the idea, whether there's enough data to meaningfully address the problem, and those sorts of things. So [the challenges] grow organically out of that."

Her company became involved in the challenges a few years ago and was a leader on the rheumatoid arthritis project — known as the Rheumatoid Arthritis Responder Challenge — that was launched in the fall of 2013.

In addition to designing and leading the challenge, Sage Bionetworks investigators provided technical support for it, along with an online platform that made it possible to do real-time testing of various computational models and doing post-hoc analyses of the data.

The project received funding from several pharmaceutical companies and from the Arthritis Foundation, Mangravite said, and was developed with an eye to potential pharmacogenomic applications.

Anti-TNF treatment can curb rheumatoid arthritis progression in patients who respond to it, the team noted. But only around two-thirds of rheumatoid arthritis cases can be successfully treated in this manner, highlighting a need for biomarkers to discern responders from non-responders.

"Rheumatoid arthritis, and specifically anti-TNF treatment, is a top priority because treatment response is so varied and so many people do fail treatment," Mangravite said. "We were really interested in looking at whether genetics could be used to predict treatment responses in general and using anti-TNF as a use case for that."

Past studies suggested genetic markers might predict anti-TNF treatment effects, though there is not yet a clear benchmark to determine the best possible markers, Guan noted. Her team was one of 73 contributing anti-TNF prediction models, together used to make nearly 4,900 predictions during the training stage of the study.

For the rheumatoid arthritis DREAM challenge, participants had access to a dataset comprised of SNP profiles for more than 2,700 anti-TNF-treated rheumatoid arthritis patients enrolled for more than a dozen prior studies — the set was broken into a training set of 2,031 cases and 675 leaderboard test set cases. Almost 22 percent of individuals in that primary cohort were anti-TNF treatment non-responders.

Another 591 rheumatoid arthritis cases assembled for a Corrona study were used to validate the most promising models. Treatment non-response was slightly higher in that group, coming in at almost 36 percent.

Following a competitive stage of the project based on approaches submitted by the research community, the organizing team analyzed the data alongside the top eight DREAM challenge teams for a six-month, community phase of the challenge.

"We had a long, half a year phase, in which we were meeting regularly with webinars, sharing methods, and learning from each other," Mangravite noted. "The goal there was to see whether we could use these modeling contributions to formally assess the question of whether there was genetic contribution here or not."

The team's statistical analyses hinted that at least some aspects of anti-TNF treatment response do have heritable components. But while several algorithms proposed by DREAM challenge investigators significantly outperformed random predictions of anti-TNF response, results from the study suggested common SNP profiles did not bolster the predictive performance of such models.

Instead, the researchers found that anti-TNF treatment response models based on clinical features alone performed as well as those that included common genetic variants gleaned from genome-wide SNP patterns.

"While we certainly could expand the sample size to see if we could find a significant association, any significant association we found that would require such a large sample size is unlikely to have a large effect size, so it's not going to be useful in the clinical setting," Mangravite said.

"We don't have any plans to follow up on this with genetics," she explained. "I think we would be interested in looking at other data modalities within the context of this question."

For their part, Guan and her colleagues are taking a functional genomics look at potential contributors to anti-TNF response, generating gene-gene networks in the synovial joint tissue that's affected by chronic inflammation in individuals with rheumatoid arthritis. That work is being done in collaboration with other investigators who participated in the rheumatoid arthritis DREAM challenge.

"This challenge brought us the opportunity to have such collaboration," Guan said. If the functional genomics approach yields candidate markers, the researchers plan to test and validate them in some of the cases and controls described in the Nature Communications study.

Guan and her team are also continuing to take part in several other DREAM challenges, including a project aimed at developing algorithms to predict cancer outcomes from clinical data. There are currently half a dozen active challenge projects, spanning topics as diverse as transcription factor binding site analyses and predictions using ENCODE data to methods related to improved breast cancer detection by digital mammography.