NEW YORK – Expectations that polygenic risk scores (PRS) can inform population screening and risk stratification as well as predict individual risk may be unrealistic and driven by uninformative performance metrics. Researchers at University College London came to this conclusion in a paper published last week in BMJ Medicine.
PRS have been making their way toward clinical applications, where many hope that they can help inform an individual's likely risk for developing diseases. Genome-wide association studies have, for instance, identified many genetic variants with relatively small effect sizes that nonetheless correlate with illnesses such as diabetes, heart disease, and cancer. PRS resulting from those variants might then be used alongside physical and environmental factors to better gauge one's probable risk for those conditions.
But study first author Aroon Hingorani, a genetic epidemiologist at UCL, said that publications describing PRS tend to report performance measures that are not typically used to evaluate predictive or screening tests. This, he said, calls into question any claims made with respect to the potential clinical utility of PRS.
"What we wanted to do [in the current study] is to see how well polygenic risk scores performed as predictive or screening tests when they were evaluated using the same measures that are used to evaluate the performance of nongenetic predictive and screening tests in medicine," Hingorani said.
Performance metrics commonly used to evaluate PRS, Hingorani explained, largely consist of hazard ratios or odds ratios for an increment of one standard deviation in the score, area under the receiver operating characteristic curve (AUC), and the C index, which is similar to the AUC but incorporates longitudinal data.
Hingorani argued, though, that detection rate — the percentage of people with a polygenic score above a particular cutoff value — and the false positive rate — the ratio between the number of people incorrectly categorized as affected and the total population — would be more clinically meaningful performance metrics.
In their study, Hingorani and his colleagues assessed the performance of scores found in the Polygenic Score Catalog, a comprehensive and open-access directory of studies on PRS for quantitative traits such as high blood pressure and PRS for diseases such as breast cancer. In all, they evaluated 926 PRS for 310 disorders.
By converting measures such as the AUC into detection and false positive rates, Hingorani's team found only minimal differences in the PRS distributions between affected and unaffected individuals.
"That means it's very difficult to differentiate people who will develop the disease on the basis of the score or the particular score cutoff," he said. "In an ideal screening test, the values among those who later develop the disease compared to those who do not would be very different."
When used as a population screening tool for coronary artery disease, for instance, Hingorani's team determined using standard performance measures that PRS would miss approximately 88 percent of affected people. Similarly, approximately 90 percent of people who eventually developed breast cancer in the dataset were missed by PRS when performance was assessed by standard measures.
Hingorani and his colleagues found similar results for individual risk estimation and for population risk stratification.
In a cohort of women aged 50 with a background 10-year odds of breast cancer of 1 in 42, for instance, the UCL team found that while the odds of getting breast cancer nearly doubled for the women in the highest 2.5 percent of the PRS distribution, these women only accounted for 6 percent of those who developed breast cancer.
"Our conclusion was that based on the published evidence of the performance of these scores, you couldn't reasonably conclude from a public health perspective that they have a use currently in screening [for] risk prediction and risk stratification because they're too weak as predictors," Hingorani said.
But PRS are typically used in conjunction with other measures of risk, such as age and sex, rather than as standalone tests, and many, if not most, of the PRS in the Polygenic Score Catalog incorporate age and sex in the score estimation.
Hingorani's group further evaluated PRS performance using the detection and false positive rates alongside conventional risk factors such as high blood pressure and low density lipoprotein cholesterol for coronary artery disease or alongside established screening tests for breast cancer. Similar to their other results, however, they identified little benefit with the addition of PRS. By their calculations, factoring PRS into a risk estimation does not improve disease prediction much and would prove less effective at preventing some diseases, such as coronary artery disease, than simply expanding the use of statins and other blood lipid- and cholesterol-lowering medications.
"We know these are safe, effective, and affordable," Hingorani said, "[and] you could prevent many more cases of heart disease simply by using those drugs more widely, by reducing the risk threshold at which we currently offer them, or even using simply an age threshold."
Not all experts agree with this thinking, however. Ali Torkamani, director of genomics and genome informatics at the Scripps Research Translational Institute, commented that despite being mathematically sound, the idea of ditching PRS in favor of lowering the bar to accessing preventive medication is not "grounded in reality."
"Handing out statins to everyone might be a more effective approach to population health," Torkamani said via email, "but it … ignores the potential negative consequences of statins [such as] diabetes risk."
Torkamani explained that such a blanket approach to population health is also unlikely to work because such approaches typically fail to drive adherence to preventive interventions.
Torkamani led a study last year, for instance, that evaluated self-reported actions taken in response to PRS results related to coronary artery disease, such as beginning lipid-lowering therapy. In that study, higher PRS scores scaled with beginning and adhering to therapy, while lower scores, such as the ones incapable of discerning affected from unaffected populations in Hingorani's study, correlated with poor therapy initiation and adherence.
"Prevention is in part risk detection and in part motivation," he said.
Torkamani also disagreed with Hingorani's argument that cholesterol and blood pressure also amounted to weak predictors of disease risk. "Evaluating one analyte at a time and stating they are weak predictors not worth measuring," he said. "It's just plain wrong."
Meanwhile, companies have been adding PRS to their risk prediction tools for various diseases. Myriad Genetics, MyOme, and Invitae, for instance, have all either fielded PRS-assisted risk prediction tools or have them in development.
Myriad first added RiskScore, a 149-SNP PRS test, to its MyRisk Hereditary Cancer test in 2021. The company has since refined RiskScore to incorporate clinical risk factors based on real-world breast cancer incidence.
Personalized genomics company MyOme, meanwhile, plans to launch a PRS-based breast cancer risk test and has been working to adjust its model to account for differing SNP frequencies in global populations.
Even further afield, a few companies have begun offering prenatal PRS-based tests aimed at assessing an embryo's lifetime risk for certain diseases.
Jonathan Moseley, a professor of medicine and biomedical informatics at Vanderbilt University, said that while the study doesn't present new reasons for being cautious in relying on PRS, Hingorani and his colleagues take a "principled approach" in evaluating them.
"I think that the main message is that results from PRS studies are often presented in a sensational manner," he said by email, "but when it comes to implementing these PRS in real-world populations, the performance is more modest and more nuanced."
The current study agrees with others that have also sought to evaluate the real-world performance of omics-based risk scores.
A study published last month in the Journal of the American Medical Association from researchers at Amgen's Decode Genetics, for instance, showed that a predictor derived from roughly 5,000 proteins found in the plasma performed only modestly well in predicting atherosclerotic cardiovascular disease and showed little advantage over a traditional risk score based on simpler risk factors.
"This doesn't mean that risk factors aren't important," Moseley said. "It also doesn't mean that these findings don't offer important biological insights that may lead to important discoveries."
A key benefit of Hingorani's study, he said, is that it makes this point using hard data and with a focus on PRS.
"A strength of the paper is that it considers both individual risk prediction and population prediction," Moseley said. "It also highlights that applying PRS to low absolute risk populations [such as] young individuals will have little net benefit — a message which contradicts messaging from PRS advocates."
Hingorani also emphasized that his findings don't mean that PRS are not useful in other settings, such as understanding the causes of disease.
"We know from family screening studies or population studies that there are individuals who carry what appear to be pathogenic mutations for a monogenic disease, but who don't exhibit the phenotype of the disease," he said, adding that this raises the question of whether a polygenic score for certain traits might mitigate the effect of a rare mutation.
Conversely, he said, some people in a population may exhibit an extreme phenotype that would make one suspect a monogenic disorder, but who don't have the causative mutation.
"It remains to be seen whether polygenic risk scores might have a role in understanding variable penetrance of monogenic disorders or phenocopy, in other words, clinical manifestations that look like a monogenic disorder but aren't," he said. "That's an area that continues to be written."