According to a team led by researchers at Plymouth University, there are significant flaws in the standard cognitive test used to diagnose and evaluate Alzheimer's disease patients, which could be hampering efforts to develop treatments and biomarkers for the disease.
The test, called the Alzheimer's Disease Assessment Scale – Cognitive Behavior section, or ADAS-Cog, is particularly limited in its capacity to measure early-stage Alzheimer's disease, the researchers said, which could cause difficulties tracking the progression of such patients.
They presented their critique in two papers published this week in the online edition of Alzheimer's & Dementia. In the first they analyzed ADAS-Cog data from 675 participants of the Alzheimer's Disease Neuroimaging Initiative using traditional psychometric techniques, looking at six properties: data completeness, scaling assumptions, targeting, reliability, validity, and responsiveness. In the second they applied Rasch measurement theory techniques to the same data to examine the test's strengths and weaknesses.
The researchers found that while the 11 components of the ADAS-Cog test worked reasonably well when combined together, a number of them performed poorly when examined individually. Specifically, they found that in six of the 11 test components, increasing or decreasing scores did not consistently correlate with worsening or improving patient conditions as intended.
The test is designed such that the severity of a patient's condition is given a number score, so that "if you've got a higher number you have got more of a problem," Jeremy Hobart, a neurologist at Plymouth's Peninsula College of Medicine and Dentistry and leader of the study, told ProteoMonitor.
However, Hobart said, in their work they found that "six of the 11 components don't work the way you would expect them to. A higher score does not necessarily mean more of a problem than a lower score, so the numbers are getting muddled."
In addition, the researchers found that the test lacks sensitivity to make fine distinctions between patient conditions, particularly in the early stages of the disease.
This is problematic not only for Alzheimer's drug development and clinical trials, but for biomarker development, as well, Hobart said.
Alzheimer's biomarkers are a significant area of focus within protein biomarker research. In fact, last year, proteomics firm Proteome Sciences released a report it had commissioned that estimated that protein biomarkers for Alzheimer's disease would represent a cumulative $9 billion market over the next ten years (PM 6/3/2011).
In biomarker research, "you are looking at the relationship between biomarkers and clinical markers [like ADAS-Cog], and these tests are not helping you if they are not discriminating between people," he said. "You will get variance in your biomarkers where there isn't variance in the cognitive tests."
This, he noted, could lead researchers "to believe that there is not a strong clinical relationship between a biomarker and cognitive impairment, and then you'll get put off the scent."
To an extent, the test's flaws are a matter of shifts in the direction of Alzheimer's research in recent decades, Hobart said.
"The ADAS-Cog was developed in the early 1980s when the whole diagnostic arena for Alzheimer's disease and dementia was different," he noted. "And over time as we have become more and more interested in identifying people earlier and earlier and treating people earlier and earlier, we have moved the distribution to the left… to the end of where the scale has the ability to detect change."
In their paper, Hobart and his colleagues suggested that the ADAS-Cog test could be adjusted to better suit these new research needs. However, he told ProteoMonitor that "as you dig deeper into the [test], there are so many more problems with it that ultimately there is a need to re-conceptualize the whole issue and rebuild from the ground up."
Holly Soares, director of clinical neuroscience biomarkers at Bristol-Myers Squibb and chair of ADNI's Biomarker Consortium project team, told ProteoMonitor that, while ADAS-Cog is the gold standard for mild to moderate Alzheimer's disease, "it's well known that it lacks the sensitivity for patients who are in earlier stages of the disease" and therefore "doesn't have the level of granularity needed to assess decline in [such] a patient population."
She noted that this was most problematic in terms of developing biomarkers to track patient decline over time.
"If the [ADAS-Cog] scale isn't sensitive enough but the biomarker is, then you really don't know what the truth is," she said. "Is it because the biomarker isn't valid? Or is it because the scale is not sensitive enough to pick it up? I think that's what we're really struggling with now in the [Alzheimer's biomarker] field."
Soares said that there are efforts underway to modify the ADAS-Cog and combine it with additional cognitive tests to provide a better measure of patient status. These new scales are "still in the research mode, but I imagine that within the next few years we'll probably see a composite scale emerging in our early predementia studies," she said.
Ian Pike, chief operating officer of Proteome Sciences, told ProteoMonitor that his company uses a range of cognitive tools in its ongoing Alzheimer's biomarker collaboration with Kings College London researcher Simon Lovestone.
"We don't really have an alternative" to using such clinical tests as benchmarks for biomarker research, he said, "so I think anyone looking to develop a non-clinical biomarker for [Alzheimer's] has to use several of these tests."
Most concerning to Pike was the demonstration of the test's limited range. "If you've hit [the test's] floor or ceiling when you're trying to predict progression, then you can't tell anything about progression beyond that," he said. "I think that's where the challenge of these clinical assays is going to be most manifest."
"If you're using an ADAS-Cog in a progression cohort to say a drug is effective, and you're trying to benchmark blood markers against those cognitive tests, the fact that those tests are flawed may make your biomarkers equally flawed," he said, noting that this is why the company looks for cohorts with patients that have been assessed by multiple methods at multiple time points.
John Trojanowski, co-director of the University of Pennsylvania's Center for Neurodegenerative Disease Research, told ProteoMonitor via email that, ultimately, no clinical or psychometric test "can accurately predict the underlying cause of dementia," noting that he believes "autopsy is still the best gold standard to know the underlying basis for dementia."
However, he added, "with iterative comparisons of biomarker cutpoints with the neuropathology in patients from whom biomarkers are obtained, I hope we can make [Alzheimer's] biomarker assays the new gold standard to select and track patients for clinical trials and other research."
Hobart, questioned, however, the extent to which biomarker tests could operate independent of clinical measures like ADAS-Cog.
"If you find a biomarker that clearly represents the underlying condition and its changes over time, then obviously you have a very accurate indicator," he said. "But that becomes a retrospective scenario based on the clinical measurements. So you kind of need those."