NEW YORK (GenomeWeb) – Interleukin Genetics' decision to stop selling a genetic test it claims can assess the risk of gum disease is welcome news for some experts in the field of dentistry who have long had serious concerns about the science underlying the product.
They contend that the test should have never come to market in the first place, but remained available to patients far too long because the leadership at Interleukin had influence within the dental research community, a group that is not well-enough versed in genetics to be able to parse the available data themselves.
In April, the American Dental Association published an educational brief for dentists on the genetics of oral health and noted that periodontal disease is a complex disease that may be due to a number of genetic and environmental factors. The ADA said it knows genetic tests are being marketed for gauging risk for gum disease, but its Genetic Testing Workgroup informed dentists that to date no gene has shown to impact the condition nearly as much as conventional factors, such as smoking or diabetes.
"In dentistry, we certainly would welcome a valid and reliable genetic test that could be used to predict risk of developing periodontal disease," said Chris Smiley, a Michigan dentist who has evaluated the underlying data on Interleukin's test and is highly skeptical about its clinical validity and reliability. "It's just that this test is not it."
Genetics is a field with a lot of buzz, and for many in dentistry Interleukin's test offered the first chance to offer cutting-edge care and promote their practices as providing personalized medicine. But dentists aren't the only ones seduced by the promise of precision medicine. Alongside genetic tests that assess people's risk for cancer, companies are marketing tests that they claim can help people lose weight, identify their favorite wine, and find their life partner. With more than 60,000 genetic testing products on the market and, on average, eight to 10 new tests launched daily, it's increasingly challenging for the public to discern which claims are supported by science and which aren't.
The Office of the Inspector General has taken note, and has been tracking fraudulent activities in the genetic testing industry and settled allegations against companies for hefty sums. Insurers have become concerned they are paying for medically unnecessary genetic testing and are instituting mechanisms to ensure appropriate utilization. The US Food and Drug Administration has also expressed concern about the risk unproven genetic testing poses to the public health, and had intended to more tightly regulate the space by overseeing all laboratory-developed tests until the 2016 Presidential election result caused the agency to back off those plans.
"This is another example why it helps to have FDA examine whether important tests are clinically valid," said Joshua Sharfstein, associate dean at Johns Hopkins University's Office of Public Health Practice & Training, and formerly a principal deputy commissioner at the FDA. "Poor data can mislead patients and doctors alike, and lead to unnecessary procedures or missed diagnoses."
In the case of Interleukin, however, it seems FDA and the New York State Department of Health looked into the company's gum disease risk test but didn't limit its availability. There is a large potential market for the type of genetic test Interleukin was selling. The Centers for Disease Control and Prevention estimates that half the adults in the US 30 and older have gum disease. If untreated, periodontal disease can cause tooth loss but the inflammatory condition has also been associated with increased risk for chronic illnesses, such as diabetes, and possibly heart disease and pre-term delivery.
Interleukin was ultimately unable to capture this market with its test. Saddled by $5.6 million worth of debt and unable to defer a payment to a senior lender, the Waltham, Massachusetts-based firm said last week it would stop performing genetic tests within the so-called Ilustra Inflammation Management Program. The company is also laying off more than half of its small staff, and is pursuing strategic alternatives, including other business opportunities, collaborations, selling its assets, and liquidation.
Interleukin declined an interview for this story, but in public statements has maintained that its periodontal disease risk test is well supported by evidence. Geneticists and periodontists who disagree point to a slew of financial conflicts they say enabled the company to falsely promote its test as a tool that dental insurers can use to limit annual cleanings based on people's genetic risk for gum disease, and thus save money.
In 2015, Scott Diehl, a genetics researcher at Rutgers School of Dental Medicine, along with Thomas Hart, another geneticist and dental researcher, reanalyzed a study performed by Interleukin and its collaborators on the test, but concluded that it cannot predict which patients are at risk for periodontal disease. That Interleukin will now stop providing this test, "is a triumph of good science over bad science, eventually," said Diehl, who for years has doggedly tried to get thought leaders in dentistry to hear his concerns about the company's claims, and conveyed his worries to the FDA and the NYSDOH.
This test would have been laughed out of the room if it had been presented to oncologists, or to professionals in medical genetics.
Diehl asserts that Interleukin's test was able to stay on the market without proper vetting of the underlying science or consideration of the potential negative public health impact partly because it was for gum disease. "This is a definite weakness of dentistry," he said. "This test would have been laughed out of the room if it had been presented to oncologists, or to professionals in medical genetics."
Some players in the genetic testing space might dismiss the controversies surrounding Interleukin's test as an isolated example. Others in the field have noted a troubling trend of genetic tests being sold without adequate evidence and believe that this story offers important lessons for the rapidly growing field and the public at large.
"All of this boils down to premature implementation," said James Evans, a professor at University of North Carolina, Chapel Hill, and editor-in-chief of the peer-reviewed journal Genetics in Medicine. "Implementation of any medical test should be driven by evidence of benefit, period. Driving implementation by marketing is a sure way to end up doing inappropriate, wasteful, and ultimately harmful tests on people."
The public should look for "systematic evidence" that a genetic test truly improves health outcomes, Evans reflected, not cherry-pick anecdotes. "A broken clock is right twice a day as the old saying goes," he said.
Growing concerns
Over the years, Interleukin has rebranded its test for periodontal disease risk several times, changing the combinations of genes and SNPs used to gauge disease predisposition. In the mid-to-late 2000s it was called the Periodontitis Susceptibility Test (PST), which assessed one SNP in interleukin-1A (IL-1A) and one in IL-1B. Then, in 2014, the company launched the test as PerioPredict, which gauged four IL-1B variations.
IL-1 genes code for a family of proteins that play a key role in the body's immune and inflammatory responses. These cytokines have been extensively studied and have been implicated in a range of diseases from cancer to rheumatoid arthritis. However, according to research by Diehl and Hart, there seems to be little, if any, consistent association between periodontal disease risk and the specific IL-1 variations gauged by Interleukin's tests.
"This whole idea of a super inflammatory response based on the SNPs proposed by Interleukin has not panned out," said Hart, senior director of the ADA Foundation Volpe Research Center. "It's a complex phenotype."
Last year, Interleukin reintroduced the test again as Ilustra, marketing it as a tool to identify individuals who have a "life-long genetic predisposition to over-produce IL-1." The website for the test claims 3 in 10 people overproduce inflammation, which can lead to other illnesses like gum disease, heart disease, and diabetes. The company reasons that if people deemed to be high risk by Ilustra are identified early and receive more preventive dental care, it can put a dent in the rising costs of treating these conditions.
Interleukin claims in online promotional materials that Ilustra is backed by 23 peer-reviewed publications and five meta-analyses. On the Ilustra website, the company cites an American Journal of Preventive Medicine paper by Jeffcoat et al., which concluded that periodontitis treatment was associated with reducing medical costs for patients with type 2 diabetes, cardiovascular disease, coronary artery disease, and adverse pregnancy outcomes.
However, experts have voiced sharp criticism about the methodology of this study as well. "The data on which the authors base their analysis are very poor," Aubrey Sheiham, a globally renowned (and recently deceased) oral health researcher at the University College London, wrote in a review of the paper by Jeffcoat et al. "The suggested implications for disease management based on the results they report are highly contentious and unjustified."
However, Hart, who has followed Interleukin since its early days, asserts that the periodontal disease risk test was "built on weak science" from the start. In 2002, he published a review article cautioning against the use of an early version of Interleukin's test.
Then, in the Journal of Dental Research in 2013, researchers led by University of Michigan's William Giannobile and Interleukin founder Kenneth Kornman looked at insurance claims from more than 5,000 people over 16 years in order to determine if one or two annual dental checkups impacted tooth loss. The researchers deemed patients at high risk of tooth loss if they were smokers, had diabetes, or had a risky genotype as determined by Interleukin's test; and they were at low risk if they had none of these risk factors.
Giannobile et al. reported that among low-risk patients, the frequency of preventive visits didn't seem to impact tooth loss, but high-risk patients who had two visits had better outcomes. Dentists generally recommend two annual cleanings per year, and typically insurance plans cover this level of care.
The way Interleukin did the statistical analysis, I could demonstrate that dropping a cotton ball on your head would increase your morbidity and mortality when grouped with other risk factors like being shot with a revolver and being shot with a rifle.
But Giannobile and colleagues claim to show in their study that two cleanings a year may not be necessary for everyone. Using a risk-based model that incorporated Interleukin's genetic test, smoking, and diabetes status to determine which patients get one, two, or more cleanings per year, Giannobile and colleagues estimated that insurers could save $4.8 billion.
Diehl and Hart, given their longstanding concerns about Interleukin's test, sought to reanalyze this study. In a gesture highly uncommon in the life sciences, Giannobile agreed to share the raw data with the geneticists, though in the end, he didn't agree with their findings. Hart and Diehl didn't find an association between tooth loss and the genetic test, but they confirmed that diabetes and smoking (well-known risk factors for gum disease) increased risk of tooth loss, while two annual cleanings lowered risk for all patients.
They noted in a Journal of the American Dental Association paper that tooth loss, used in the original study as a proxy for periodontal disease, can be caused by many other factors, and that this further weakened Giannobile et al.'s conclusions. They also pointed out that while Giannobile et al. combined IL-1 genetic test results with diabetes and smoking into a single high-risk group, the authors never reported outcomes dissecting the specific effects of smoking, diabetes, or the genetic test.
"The way Interleukin did the statistical analysis, I could demonstrate that dropping a cotton ball on your head would increase your morbidity and mortality when grouped with other risk factors like being shot with a revolver and being shot with a rifle," Hart said.
Importantly, Giannobile et al. wrote in their paper that the two versions of the gum disease risk test ― PST and PerioPredict ― gave comparable results in terms of the benefit high- and low-risk groups can hope to see from one or two annual cleanings. However, Diehl and Hart used data from the 1,000 Genomes Project to look at the frequency of people deemed to have risky genotypes by the two tests and reported that they classified patients into risk categories very differently. According to the reanalysis, 56 percent of African Americans changed between high and low-risk groups depending on the version of the test used.
Diehl and Hart concluded that these tests should not be used to determine the level of preventive care patients receive.
Following the publication of this reanalysis, Kornman dismissed the paper in an earnings call with investors as "intentionally misleading" and "the perspective of academics" who "like to take pot shots occasionally." Giannobile et al. further countered in JADA that the study was designed to see whether tooth loss differed for patients who got one or two cleanings in high- and low-risk groups, but not whether the risk factors predicted tooth extraction outcomes or had any associations with the number of dental cleanings.
Nonetheless, the company used this study to claim that PerioPredict can help personalize treatment plans and the frequency of dental care visits. "They keep saying their test is for periodontal risk … [but] this study is about tooth loss," said Michigan-based periodontist Brian Cilla, reflecting on the analysis by Giannobile et al. Cilla estimates that 35 percent to 40 percent of tooth loss events are attributable to periodontal disease, "but Interleukin has turned this tooth loss study into a statement about periodontal disease."
John Ioannidis, a professor of health research and policy at Stanford University and a highly cited scientist, wrote in an editorial accompanying Diehl and Hart's paper that their reanalysis was more appropriate than the analysis by Giannobile et al. He agreed with the two geneticists that IL-1 variants gauged by Interleukin's tests do not identify patients who would benefit from one or two annual preventive dental visits.
The Ioannidis editorial was welcome support, according to Diehl, who claims that Interleukin and its supporters in dentistry have unfairly accused him and his colleagues of harboring a vendetta against the company. If he hadn't continued to voice his concerns about the science underlying the test, however, Diehl fears there may have been a far more harmful public health impact. "This is a useless genetic test, and I think we had a close call," he reflected. "This test could have been rolled out and damaged the dental care for the American population."
Regulatory inaction
If, as Hart and Diehl assert, Interleukin's periodontal risk test lacked clinical validity — which the US Food and Drug Administration defines as the ability to accurately predict the presence, absence, or predisposition for a condition in a patient — then why didn't health regulators step in? Diehl tried to convince regulators to limit the genetic test's availability to little effect.
Lab-developed tests, like Interleukin's, are currently overseen by the Centers for Medicare & Medicaid Services, which inspect labs to ensure if they meet federal regulatory standards under the Clinical Laboratory Improvement Amendments. Inspectors evaluating a lab according to CLIA look at a test's analytical validity (the ability to measure the presence or absence of a genetic change), but they do not consider its clinical validity.
Interleukin performs testing in a CLIA-certified lab. However, the company highlighted in 2014 that it received conditional approval for PerioPredict from New York State Department of Health, which is considered to have the highest bar for ensuring both analytical and clinical validity of lab tests.
In early 2015, while his reanalysis of the Giannobile et al. study was in press, Diehl informed the NYSDOH that the independent reevaluation "revealed no clinical support or validity" for PerioPredict or PST. He requested the NYS clinical laboratory evaluation program consider withdrawing its approval of the test. However, the NYSDOH Wasdsworth Center replied a few months later that there is support in the literature for the association of IL-1 polymorphisms and increased risk of periodontal disease.
In emails reviewed by GenomeWeb, the DOH said it requires labs to provide evidence of a test's clinical validity (specificity and sensitivity), but not whether it has clinical utility by improving patient outcomes or lowering healthcare costs. "I believe there is a misunderstanding regarding the results of my recent publication in JADA," Diehl wrote back. "My colleagues and I believe these results definitively show that the tests (both PST and PerioPredict) lack clinical validity (specificity and sensitivity) for predicting risk of tooth loss. It is precisely because of this lack of clinical validity that the tests have no use for patient outcomes or impacts on health care costs. The latter conclusions derive directly from the fact that the tests have no clinical validity."
NYSDOH still maintained its approval for the periodontal risk test. So, Diehl reached out to the FDA, which for some time had expressed concern that lab tests regulated under CLIA were harming patients and wanted to start regulating them. It's unknown if this was prompted by Diehl's letter, but the FDA sent a letter to Interleukin asking why PerioPredict wasn't cleared by the agency. The November 2015 letter was dated a few days before the FDA released a report detailing 20 examples of lab tests sold without its approval that the agency said had harmed the public health or could have. Interleukin's gum disease risk test was not among these examples.
Since then, the FDA has continued to exercise enforcement discretion over lab tests on the whole, and did not require Interleukin to seek clearance or premarket approval for its periodontal disease risk test. Interleukin said it had extensive interactions with the FDA and was allowed to continue marketing the test through its CLIA-certified lab.
Troubling ties
After Amway instituted an employee wellness benefit plan that included Interleukin's Ilustra test last year, Cilla's dentist colleagues in Michigan began complaining to him about patients with periodontal disease who got tested and were found to be low risk. As a consequence, these dentists said their patients were questioning the previous dental treatment that they had been receiving.
Many Michigan dentists and patients have had first-hand experience with this genetic test because Amway is a large employer in the Grand Rapids region and insurer Delta Dental of Michigan in late 2013 instituted a risk-based benefit plan that included Interleukin's genetic test.
In 2013, Delta Dental of Michigan invited Cilla and a few others on the Michigan Dental Association's Board of Trustees to discuss its so-called RightSize policy. Under this plan, Delta Dental employees would get only one covered cleaning a year, unless they were at high risk according to Interleukin's genetic test or based on conventional factors like diabetes. It was voluntary for Delta Dental employees to take the risk assessment survey or the genetic test, but if they declined, they'd be limited to the one covered cleaning.
"That's a big change in standard dental insurance policies," Cilla said. According to a write-up in the ADA's website at the time, depending on the reception of this plan, Delta Dental intended to expand it beyond its employees.
Cilla said he expressed concern at this meeting with the insurer's representatives that a significant gum disease risk factor, smoking, was not among the criteria that would warrant additional preventive care. "Smoking is the number one risk factor for gum disease, nothing else is even close," Cilla said, but he recalled Delta Dental reps' rationale for excluding smoking from coverage criteria was that "they didn't want to reward bad behavior."
I assume an individual investor may be concerned that the marketplace may have been manipulated.
Meanwhile, the Giannobile et al. study that Interleukin used to back the clinical utility of its test found that patients benefitted from two annual cleanings when they were high risk based on genetic testing, had diabetes, or smoked. Delta Dental did not reply to emailed questions for this interview before publication of this article.
Cilla did further research on Interleukin and found out that Delta Dental "had a vested interest in propagating this test within benefit plans." Delta Dental Plan of Michigan invested $3 million into Interleukin in 2012 and held a seat on the company's board of directors until 2014. Additionally, the insurer is an affiliate of Renaissance Health Services Corporation, which funded the Giannobile et al. study.
In contrast, UnitedHealthcare has decided to not cover genetic risk testing for gum disease after a review of the available literature on the test. "The clinical utility of genetic testing for susceptibility to periodontal diseases has not been established," the insurer states in a Feb. 1 dental clinical policy. "Additionally, there is a lack of objective, high-quality clinical evidence to support these tests."
The fact that Giannobile et al. estimated insurers could save $37 per patient per year using a risk-based benefits model that included genetic testing, must have been "powerful financial incentive" for Delta Dental, despite the Diehl/Hart reanalysis, Michigan-based dentist Smiley suspects.
According to Smiley, the Michigan Dental Association has published concerns about Interleukin's test to its members, and also expressed them to Delta Dental, to no avail. In a recent article in the West Michigan District Dental Society's newsletter, Smiley also wrote that local dental offices had been receiving genetic testing results from Interleukin for patients who worked at Amway in the Grand Rapids-region. But based on his interactions with patients who have been tested and other colleagues, he thinks the genetic test wasn't very popular with Amway employees who received flyers and corporate emails urging them to sign up for testing.
However, he still felt concerned that Amway's financial ties to Interleukin weren't made as clear as they should have been. Amway parent company Alticor is a major stockholder in Interleukin, and current and former employees of Alticor hold seats on Interleukin's board.
An Amway spokesperson said in an email that in light of Interleukin's strategic decision, it will now stop offering the genetic test within wellness programs but didn't explain why it continued to offer the test in light of data suggesting it lacked clinical validity.
"Delta Dental of Michigan and Amway each invested in the product and tried to create a market for it," said Smiley. "I assume an individual investor may be concerned that the marketplace may have been manipulated."
Power to influence?
Hart believes that the "evaluation and promotion of these tests is influenced by conflict of interest."
Interleukin's founder and current CSO, Kornman, is influential in dentistry, often described as a "star in periodontics." Giannobile chairs the department of periodontics and oral medicine at the University of Michigan's dental school. The study they led, entitled "Patient stratification for preventive dental care," was printed in the Journal of Dental Research, a publication Giannobile is editor-in-chief of.
The American Academy of Periodontology in 2014 awarded Kornman and the authors of that study a clinical research award. The academy publishes the Journal of Periodontology, which Kornman is editor-in-chief of. Kornman declined an interview for this article and said CEO Mark Carbeau was not available for an interview due to other commitments.
We want to be able to apply genetics to personalize dentistry but we want to be able to do it responsibly.
"The way the test was promoted by Delta Dental of Michigan, the way in which the featured [Giannobile] study was published, and the related financial trail is concerning, and perhaps kept the marketplace from sorting this out sooner," Smiley said. He said he spoke directly with Kornman about his concerns, and while Kornman agreed that Delta Dental's benefits plan was flawed because it didn't include smoking as a risk factor for additional preventive services, he was dismissive about the conflict of interest issues with the publication of the Giannobile et al. paper.
In the years following the publication of their reanalysis, Diehl also wondered why Interleukin's board of directors and clinical and scientific advisors —accomplished experts in periodontology, genomics, inflammation science, and cardiovascular disease — didn't speak up publicly and stop the company from selling its test.
GenomeWeb contacted a number of Interleukin's scientific advisors via email. Most did not reply, and Geoffrey Ginsburg, director of Duke Center for Applied Genomics and Precision Medicine, declined an interview.
Another scientific adviser, Steven Offenbacher, was a coauthor of the ADA's recent educational brief on genetics and oral health. According to Offenbacher, chair of periodontics at the University of North Carolina, the main purpose of the ADA's document is to educate dentists about what is currently understood about the role of genetics in periodontal disease, which has shown to be more complex than previously thought.
Using earlier candidate gene approaches, the IL-1 variants gauged by Interleukin's test were "very promising," and the company "was ahead of its time," Offenbacher said. Since then, many IL-1 markers associated with periodontal disease identified through candidate gene approaches have not been confirmed in larger genome-wide association studies, he acknowledged, but maintained that the company's recent suggestion that these markers may be linked to increased inflammation remains "consistent with their data."
Offenbacher said he did not have any knowledge of the company's current commercial strategy and did not comment on the company's decision to stop selling its periodontal disease risk test.
Diehl scoffed at the characterization of Interleukin as being "ahead of its time," and asserted that the company's gum disease risk test wasn't its only offering based on questionable science. According to filings with the US Securities and Exchange Commission, a significant portion of Interleukin's revenues came from the sale of genetic tests for weight loss and nutrition. Diehl bemoaned the fact that despite a lack of published data showing these tests were clinically valid, the company promoted them in major media outlets.
"One would hope that Interleukin and the clinician-scientists advocating for their useless tests were the exception," he said. "Unfortunately, there has been a plague of genetic testing companies popping up selling tests for diseases and traits with no credible scientific support. Meanwhile, the FDA, [Federal Trade Commission], and SEC are stuck in endless gridlock pondering steps toward regulation, failing to protect the health and financial assets of the American public."
The response from regulators baffled Diehl, but perhaps even more disconcerting was that throughout his efforts to communicate his concerns about Interleukin's test to different segments of the healthcare enterprise, he was told in backroom conversations to not protest too loudly for fear it would quash the potential of genetics-based dentistry.
For Hart, the whole experience speaks of the need to improve genetics education for dentists. "If dentists understood genetics, we wouldn't be having this conversation," he said. "We want to be able to apply genetics to personalize dentistry but we want to be able to do it responsibly."