At a time when direct-to-consumer genomics companies probably don't need media reports questioning their services, a personal essay in the New York Times this week casts doubt on whether DTC tests can be trusted, or are accurate enough to be useful.
When 28-year-old Kira Peikoff decided she wanted to explore her genetic risks for diseases by buying having a direct-to-consumer company run a genotype test and analyze her genes, she conducted an experiment, Peikoff writes.
Peikoff, a grad student in bioethics and a novelist, wanted to find out her risk for some diseases that run in her family, including coronary heart disease, breast cancer, rheumatoid arthritis, and Alzheimer's. But she also wanted to some way to gauge how accurate her test was.
Her solution: she opted to have three DTC firms assess her disease risk and then compare the results.
Peikoff had cause for concern. She received her results right around the time that the Food and Drug Administration recently warned one of the services she used, 23andMe, to stop advertising its disease risk testing services until it can satisfy the agency's concerns about the clinical utility and validity of its product.
Along with the $99 23andMe test, Peikoff also ordered a $285 test for 25 disease risks from Genetic Testing Laboratories and a $399 test from Pathway Genomics, which provided her a report for 24 disease risks.
She says that when she got the results from the three different tests back, "the discrepancies were striking."
There were discrepancies related to the risk assessment numbers, how the numbers were interpreted, and semantic issues with how those risks were translated, she says.
For example, 23andMe said her most elevated risks, roughly double the average, were for psoriasis and rheumatoid arthritis, (20.2 percent and 8.2 percent, respectively), but GTL said these were her lowest risk areas, (2 percent and 2.6 percent, respectively).
Both 23andMe and GTL said Peikoff has a nearly average risk of coronary heart disease (26 to 29 percent), but Pathway labeled her odds as "above average."
She found the same problem with the Type 2 diabetes risk assessment, which GTL said she was at "medium" risk for while 23andMe said she was at "decreased risk," even though the actual percentage risk assessed by 23andMe was around 5 points higher than GTL.
"Your results are not the least bit surprising," J. Craig Venter told Peikoff, adding that "anything short of sequencing is going to be short on accuracy…."
She also pointed to a need for more industry standards.
“We need to get some kind of agreement on what is high risk, medium risk and low risk," New York University bioethicist Arthur Caplan told her, saying that the "risk is in the eye of the beholder" standard is not going to work."