This article has been updated to include additional comments from Karius.
SAN FRANCISCO (GenomeWeb) – Researchers from infectious disease diagnostics firm Karius have published an analytical validation of its cell-free microbial DNA sequencing test as well as results from the first 2,000 clinical cases analyzed from its CLIA-certified and CAP-accredited laboratory in Redwood City, California.
In addition, customers are using the Karius test for indications including pediatric pneumonia and endocarditis, but as a supplement to their conventional diagnostic procedures.
"The study provides a foundational building block," said David Hong, vice president of medical affairs and clinical development at Karius, showing that "yes, we can detect this breadth of organisms at a speed where we can return results in a clinically actionable time frame."
In its study, published in Nature Microbiology, Karius described the results of the SEP-SEQ trial, which was conducted in partnership with Stanford University researchers who analyzed the assay's ability to uncover the cause of sepsis in 350 individuals.
Tim Blauwkamp, chief scientific officer and a company cofounder, said that the main implication of SEP-SEQ was that it shows that "cell-free microbial DNA can identify infections that are difficult or impossible to identify with any other method."
The team also used reference materials to analyze the test's performance for detecting 13 microorganisms and calculated the limit of detection for each reference microorganism, as well as per-analyte sensitivity and specificity. Finally, for the first 2,000 clinical samples analyzed with the test, the company reported that it delivered next-day reports in 87.6 percent of cases and reported a microorganism in 53.7 percent of all samples tested, about half of which were reported as poly-microbial.
The company did not have outcomes data on the patients, so could not determine the performance of the test in this setting, but Hong noted that it is involved in a number of studies with collaborators to do just that.
In the SEP-SEQ trial, the researchers compared the Karius test to both an initial blood culture test as well as to all microbiology testing carried out in the hospital over one week. In addition, a panel of experts evaluated both the Karius results and the microbiology results with the clinical cases to determine the likelihood of a correct diagnosis.
Overall, the researchers found that the Karius test found the likely pathogen in 169 cases. Blood culture alone identified the pathogen in 63 cases, or 18 percent, and with other microbiology tests pathogens were identified in an additional 69 cases. Thus, the Karius test had an overall sensitivity of 92.9 percent, the researchers reported.
For 166 samples for which no likely pathogen could be identified via microbiology tests, the Karius test was also negative in 104 of those cases for an overall specificity of 62.7 percent. And, in 167 asymptomatic cases, the test was negative in 129, or 77.2 percent. In general, for asymptomatic patients for which Karius did identify a cfDNA from a microorganism, it was present at smaller amounts than in affected individuals. Many of the microorganisms were organisms known to be present in humans, even without infection, like Klebsiella pneumoniae.
Charles Chiu, a professor of laboratory medicine at the University of California, San Francisco, who is not affiliated with Karius but was previously a scientific advisor for the company and whose lab has developed a commercial metagenomic sequencing test for meningitis and encephalitis, said that the paper adds to a growing evidence that "using this comprehensive approach of metagenomic NGS is viable for diagnosing infectious diseases." He added that the Karius approach for analyzing cell-free microbial DNA from blood is "very promising."
Nonetheless, he said that the specificity demonstrated in the SEP-SEQ trial was somewhat concerning when thinking about clinical implementation of the test. "That could be a large number of false positives," he said. The study authors noted that a number of discrepant results — where Karius testing for a patient who was suspected of having sepsis identified a microorganism but microbiology testing did not — were organisms known to be present in humans and not likely to be the cause of sepsis, such as a patient with a reactivated herpes virus, or commensal organisms like K. pneumoniae. Chiu said that it's unclear how such results would be reported in a clinical setting. However, Karius said that in these cases the independent clinical committee that adjudicated the results deemed that the identified organism was either not likely to be causative for sepsis or was a possible cause, showing that such results can be interpreted clinically.
Nonetheless, said Chiu, depending on the patient, such results can still be challenging. "If a patient is immuno-compromised sometimes those environmental bacteria can cause disease," he said. "So, it's a challenge to interpret results that may or may not be false positives." He added that the challenge was true of all metagenomic NGS tests, which can identify everything present in a sample, regardless of whether or not it is causing disease, not just Karius' test.
He added that he "would have liked to see more discrepancy testing — further workup of the sample to confirm whether what's identified is correct or not," he said. Karius noted, however, that its use of an independent clinical adjudication committee served as discrepancy testing and that its analytical validation of "contrived infection samples and in silico simulations" gave it further confidence in the results.
Chiu also noted that it would be important for Karius to further define the test's indications. The test is currently being used for a broad range of applications and clinical indications, he said, but the performance reported in the SEP-SEQ trial would likely not be relevant for other indications, he said. And finally, he said, since the test analyzes cell-free DNA, it would not be able to detect RNA viruses.
A test like this could help speed the turnaround time for diagnosing sepsis, he said, but wouldn't be a replacement for blood culture. Typically, if a blood culture is going to yield a positive result, it will do so within 24 to 48 hours, Chiu said. "Usually, during that time frame, we end up sending off a battery of 20 to 30 other tests, especially in critically ill patients," he said. "But, the ability to implement a universal test at that time would be helpful, and I think is where we'll see clinical adoption initially."
Customer feedback
Vance Fowler, a professor of medicine at Duke University School of Medicine, said that his lab has received grants from Karius to test its assay in several different projects. He has used it clinically in patients with endocarditis, an infection in the heart that is rare but can be life threatening and said that it has been especially useful in cases where a blood culture is negative.
"Endocarditis can be caused by a wide range of bacteria, some of which don't grow in conventional culture methods," he said. In such culture-negative cases, physicians have to treat patients with broad antibiotics over a period of six to eight weeks, "which is a long time and the medications can be toxic," he said. In these cases, he noted, the Karius has been useful in identifying a specific organism and enabling a more targeted treatment.
Fowler said that approaches like Karius' and other next-generation sequencing-based tests are a welcome advance in a field that is still based on technology that is 100 years old. "Microbiology diagnostics is fundamentally using the same technology that we used 100 years ago — isolating an organism from an otherwise sterile body site and culturing it out by growth."
Fowler is now looking to determine the best scenarios to use the test and to clarify its performance, including better understanding what a positive or negative test result means.
The issue with these new technologies, is that there is not always a good comparator, Fowler said. "If you get a result from Karius that you don't get from your conventional workup, what does that mean? Is the Karius test wrong? Is the conventional test not sensitive enough? Or is there another explanation?"
These are questions that Fowler and others are still trying to answer. Fowler has not yet published results on these research studies, but in 2016, he and other Duke researchers described a case example in the journal Open Forum Infectious Disease where the Karius test diagnosed within 24 hours a 60-year old man in septic shock with a rare infection, Capnocytophaga canimorsus, which was caused by a dog bite. In that case, blood culture had not yet produced a result and clinicians were treating him with a broad range of antibiotics. The researchers sent a sample to Karius test on a Thursday and had a result by Saturday morning, Fowler said, and were able to focus the treatment regimen on antibiotics known to be effective against C. canimorsus. The patient ultimately succumbed to the infection, which became complicated by an opportunistic fungal infection. A subsequent autopsy analysis confirmed C. canimorsus.
Another hurdle of the test, Fowler noted, is its price tag of around $2,000. "At that price, I'm not sure it will be a first-line test in every infected patient, particularly those where a culture result is likely," he said. Nonetheless, he said there are at least three categories of patients where it could be used based on his experience: culture-negative endocarditis patients, endocarditis patients who have already been treated with antibiotics, and severe sepsis patients where the cause is uncertain.
Lauge Farnaes, who serves as both assistant medical director of Rady Children's Institute for Genomic Medicine and as an assistant clinical professor of pediatric infectious disease, said he has been using the Karius test as his role in the clinical infectious diseases division of Rady. He said the test is ordered "numerous times per week" and has found it to be helpful in a number of different cases, for instance in patients with pneumonia, where imaging has identified an infection but getting a sample to identify the etiology of that infection is difficult.
In those cases, the Karius test is helpful because it does not require a tissue sample, he said. Similar to Fowler's team at Duke, Farnaes said the Rady team has also has success in patients with culture-negative endocarditis. In addition, the test has identified fungal infections in pediatric patients with blood cancers. Such patients are highly immuno-compromised due to chemotherapy treatment, he said.
Nonetheless, he said, "We don't have that clinical evidence yet" to replace conventional testing. "At this point, it just provides additional information."
Because it is a new test, Farnaes said, it does not yet have an established negative predictive value. "If we get a positive result and it makes sense in the clinical context that can be useful. But if we get a negative result we don't know whether there is no infection or if the test didn't pick it up," he said.
Determining an NPV for the test will require "really large studies," Farnaes added. In addition, he said, such a study in one indication will not necessarily be applicable across the broad range of indications that test is currently being used in,. For instance, Farnaes' team published a retrospective analysis earlier this month in Diagnostic Microbiology and Infectious Disease of pediatric patients with pneumonia for whom doctors had ordered the Karius test. In those cases, the researchers knew the patients had pneumonia but did not know the specific causative bacteria.
"By being able to identify which bacteria was causing it, we were able to simplify the antibiotics," he said. In this indication, the Karius test is a "slam dunk," he said, because patients have a high burden of disease so there is more cell-free DNA in their bloodstream. In addition, doing a tissue biopsy of the lung would be invasive, risky, and expensive, so a noninvasive alternative is appealing.
In that recent study, the researchers retrospectively analyzed cases over an eight-month period. During that time, 125 children were admitted to the hospital with pneumonia and doctors had ordered the Karius test for 31 children. Of those, two were excluded from analysis due to being diagnosed with an underlying genetic disorder and 14 were excluded due to having chronic comorbid conditions. For the remaining 15 patients, the researchers reported that the Karius test identified the causative pathogen in 13, compared to seven patients using culture and PCR methods. In addition, the team changed the treatment for seven patients based on the Karius test results.