This story has been updated to include comments from Rob Knight.
NEW YORK – Following months of brewing concerns, the journal Nature on Wednesday retracted a controversial cancer microbiome study from a team led by the University of California, San Diego's Rob Knight.
"After publication, concerns about the robustness of specific microbial signatures reported as associated with cancer were brought to the attention of the editors," Nature said in the retraction note. "Expert post-publication peer review of the issues raised and the authors' responses has confirmed that some of the findings of the article are affected and the corresponding conclusions are no longer supported."
The 2020 Nature study from Knight's team claimed evidence of widespread cancer-specific microbial signatures based on DNA sequencing data from more than 18,000 samples from more than 10,000 patients from The Cancer Genome Atlas (TCGA), covering 33 cancer types.
Last year, however, a number of researchers voiced their skepticism about the study's results, contending the paper has "major data-analysis errors," and therefore, its conclusions should be considered "invalid."
In an August 2023 preprint study, which was subsequently published in the journal mBio in October, researchers led by Abraham Gihawi, a postdoctoral researcher at the University of East Anglia, and Steven Salzberg, a biomedical engineering professor at Johns Hopkins University, delineated their concerns.
For one, Gihawi and Salzberg showed that the raw microbial read counts in the Nature study were "were inflated by many orders of magnitude" due to contamination with human sequences.
Moreover, they illustrated that the paper's strategy for normalizing raw data against technical batch effects, namely Voom-SNM normalization, inadvertently created an artificial signature for each cancer that did not actually exist, which was then exploited by the machine learning model to create highly accurate classifiers despite the absence of any true signal.
The mBio paper was cited in Nature's retraction note.
In a rebuttal published in Oncogene in February, Knight and collaborators said they repeated the analysis following Gihawi's footsteps and still found microbiome signatures specific to different cancer types. "These extensive re-analyses and updated methods validate our original conclusion that cancer type-specific microbial signatures exist in TCGA, and show they are robust to methodology," the team wrote.
The retraction note now stated that "[a]ll authors agree with this retraction."
"All authors of this study agree that the high background and incorrect taxonomy assignments stemming from the incomplete human genome resources available at the time are important enough to warrant the retraction," Knight wrote in an email.
While a manuscript retraction "is never welcomed news," Knight said, his team still "believes that the major conclusions of the original 2020 Nature manuscript regarding the ability of the microbiome to distinguish tumor types are true." As such, Knight said he and colleagues "look forward to further illustrating this view in forthcoming research with improved methodology and more comprehensive datasets."
"[W]e discovered that the authors of the Nature paper had made some huge mistakes — that most of the bacteria they found simply weren't there, or else were present in quantities that were hundreds of times smaller than they reported," Salzberg wrote in an email. "The 2020 paper was simply wrong — essentially, every single machine learning classifier they created was invalid."
"It is proper that Nature has followed up on our concerns and launched an independent review process, which resulted in all of the authors agreeing to the retraction," Gihawi wrote in an email.
He also noted that "the original authors taking the time and effort to be so open with their data and code was commendable and should not be overlooked."
The cancer microbiome findings from the now-retracted 2020 study also spurred the development of an assay by Micronoma, a San Diego-based startup cofounded by Knight and his former graduate student Greg Sepich-Poore, who was the paper's first author.
In January 2023, the company received breakthrough device designation from the US Food and Drug Administration for its OncobiotaLung assay, a blood microbiome-based assay for the detection of lung cancer. At the time, Micronoma touted that the work "that led to the breakthrough device designation is based on the findings of Micronoma's cofounders," including the Nature study.
When concerns were raised last year over the Nature paper, a Micronoma spokesperson told GenomeWeb at that time that while the company's work is based on the findings in that study, "methods have continued to evolve since then, and we are not using the exact same processes."
Micronoma did not respond to a request for comment before deadline.
A spokesperson for the FDA said the agency is unable to confirm the status of the application regarding the OncobiotaLung assay. "Information on any possible applications or applications that have yet to receive an approval or was denied approval generally is not releasable," the spokesperson said in an email.
"I think this saga proves the importance of a dialectic discourse where competing ideas are evaluated. This is the scientific process in action," Ivan Vujkovic-Cvijin, a microbiome researcher at Cedars-Sinai Medical Center who was not involved with either study, wrote in an email. "[B]oth parties should be applauded for hearing each other and having this important dialogue."
Vujkovic-Cvijin said the retraction of the Nature paper "should entrench neither the naysayers nor the proponents of tumor microbiomes," given that it is still possible that there are, or are not, unique microbiome signatures associated with certain tumor types.
Nonetheless, Vujkovic-Cvijin believes the retraction underscores two areas where the field can improve. For one, he said investigators need to be more cognizant of the complexity of machine learning models, which makes their results difficult to test and validate.
As such, he said the field "ought to develop better standards and techniques for validating machine learning models if we are to continue using these methods."
In addition, Vujkovic-Cvijin noted that the controversy "highlights the challenges that face microbial surveys on low biomass samples, like internal organs."
"There are boundless sources of contamination that can outweigh the true signal, and new methodologies in sample collection or processing that enable discovery of true signal should be developed and validated," he added.
"The tumor microbiome field has a colorful and controversial past," East Anglia's Gihawi noted. "I have no doubt that the future of the field is bright and has lots of surprises in store for us that will hopefully result in clinical benefit."