NEW YORK (GenomeWeb) – A team at the National Institutes of Health Clinical Center recently evaluated a molecular assay for carbapenem-resistant Enterobacteriaceae (CRE), demonstrating that the test has a high negative predictive value but low positive predictive value on clinical samples, prompting an airing of potential issues with molecular diagnostics for this rare but dangerous hospital-acquired infection.
Genes allowing certain bacteria to overcome antibiotics in the carbapenem class — drugs often used as a last resort to treat infections — vary in frequency and mechanism of action, with the most common in the US allowing bacteria to manufacture carbapenemase enzymes. These genes can get swapped around between bacteria in the Enterobacteriaceae family, most often impacting Klebsiella species and Escherichia coli. Patients can develop CRE pneumonia or other types of infections, but the common site of asymptomatic carriage is in the gut, and infected patients are also often colonized in their gastrointestinal tracts.
The NIH study, published last month in the Journal of Clinical Microbiology, evaluated a CRE screening protocol using perirectal swabs from 258 patients in a non-outbreak setting. The research-use-only Check-Direct CPE from Dutch firm Check-Points was compared to reference standards — culture of a PCR-confirmed carbapenemase-containing organism, or a history of colonization within the previous two weeks.
The Check-Points test was found to have 100 percent sensitivity, a negative predictive value of 100 percent, and a specificity of 88 percent.
But results also showed a positive predictive value of 21 percent. All of the 34 false positive samples had high Ct values, suggesting a Ct cut-off may be clinically useful in the future, and the researchers noted a limitation of the study was that they did not directly sequence the amplified products.
"As with other highly sensitive molecular assays, Check-Direct CPE can at times yield results that cannot be confirmed with culture," Joost Thijssen, Check-Points CEO and founder, told GenomeWeb in an email. He noted that in a number of instances these samples have been linked to patients who have previously tested positive or who test culture positive at a later point in time.
Check-Direct CPE was based on initial market feedback showing a need for a highly sensitive commercial assay for rapid identification of CRE carriers, Thijssen noted.
"No case should be missed, as doing so could potentially have a high clinical impact; we therefore set out to design an assay with excellent sensitivity and negative predictive value," he said.
As various published and yet-to-be published studies report Check-Direct CPE to have a 100 percent sensitivity and negative predictive value, Thijssen said the firm had achieved its goal in this regard.
Low PPV may be accounted for by factors such as the particular culture medias used or non-specific binding of primers and probes, NIH Clinical Center Department of Laboratory Medicine's study authors Anna Lau, co-director of bacteriology, parasitology, and molecular epidemiology, and Karen Frank, chief of microbiology service, told GenomeWeb in an email.
"Note that poor PPV is an inherent issue for all molecular assays currently available for [carbapenemase-producing organism] detection directly from rectal swabs or stool samples, as opposed to isolated bacteria," the authors said.
This value is also highly dependent on the patient population tested, they said. "In our study [PPV] was low because our evaluation was conducted on all patients, regardless of risk, in a non-outbreak setting ... [it] may improve if the assay was targeted specifically at high-risk patients in a CPO outbreak setting."
Meanwhile, a commentary on the JCM study published earlier this month called the false positive rate "unacceptable," particularly with respect to using the assay to screen for carriers.
The commentary emphasized that all false positive results were due to targets with a low incidence in the US, and that false positives could lead to unnecessary contact precautions.
The results could even follow patients and impact their medical care in the future, for example through the use of an electronic registry for people who are CRE-colonized like one used in Illinois to curtail extensively drug-resistant organisms.
For hospitals, the cost of false positives could also be significant. The US Centers for Disease Control recommends that a positive CRE result in a facility with low incidence should trigger an epidemiological investigation.
"From a resource management perspective and a patient perspective, having false positives is almost as bad as having false negatives," Romney Humphries, a co-author of the JCM commentary, told GenomeWeb in an interview.
Yet a number of molecular diagnostics manufacturers are developing assays for CRE, and with good reason. The bacteria have a mortality rate as high as 50 percent, according to the CDC, and, "once they get in, they spread like wild fire; it is really hard to contain them," Humphries said.
Additionally, the burden of CRE has steadily increased in the US, with surveillance studies detecting pockets of carriers in places like long-term acute care hospitals and intensive care units in some parts of the country.
Screening and using extra precautions with carriers can potentially eliminate the bacteria, but culture-based susceptibility testing is time-consuming and labor-intensive.
Humphries, who is section chief of clinical microbiology and an assistant professor in the department of pathology and laboratory medicine at the David Geffen School of Medicine at UCLA, said that there are not a lot of molecular options for labs at the moment.
"There are two systems that are currently available that perform off of positive blood cultures; those are the Nanosphere and the BioFire platforms," Humphries noted.
The Nanosphere system "has a pretty good coverage of all the different carbapenemases that have been described to date in the US," whereas "the BioFire system only looks for KPC, which is the most common, but certainly not the only one."
However, CRE bacteria more often cause respiratory or urinary infections, rather than bloodstream infections, "so there's no real good option for molecular testing to look for CRE in those other specimen types," Humphries said.
And the overall rates of these infections is relatively low. "When you start surveilling a large number of people and the pre-test probability of finding anything is low, inherently, with any test, you're going to see false positives," she said.
Industry and clinical lab implications
There is little clinical data available for any of the three commercial molecular assays currently available, all of which target different carbapenemase genes, the NIH's Frank and Lau said.
"Our motivation to perform this extensive evaluation was to provide clinically useful information that is practical for any lab considering implementing molecular CPO surveillance, especially given the limited surveillance options currently available in the US," they said.
Check-Points was the first firm to launch a molecular CRE screening assay covering the most prevalent genes, Thijssen noted. The assay debuted over two years ago, and the firm has begun the process of submitting for 510(k) clearance, as reported by GenomeWeb.
The firm said it recognizes a need to further verify any clinical results that could not be confirmed by culture. "Doing so would help the field as a whole to further learn about the clinical significance of such findings and ultimately help us fine-tune our tools for rapid CRE detection," Thijssen said.
Indeed, the NIH authors concurred, saying, "Published data using direct sequencing of the amplified product to prove specific detection of the carbapenemase gene for culture negative samples would be very helpful; this will help disprove the hypothesis of non-specific binding of the primers and probes [and] ... help guide laboratories as to the clinical usefulness and epidemiological significance for a PCR positive, culture negative sample."
Check-Points currently plans to file for FDA 510(k) clearance in 2016 with an updated assay which will further extend the number of CRE genes and types covered. The updated assay will also reflect the latest insights from clinical CRE studies around the world.
In the meantime, other firms have been homing in on the CRE molecular testing market.
Cepheid's Xpert CARBA‐R was CE-marked in June of 2014 and submitted to the FDA in the middle of September. The European brochure for the test notes that the on-demand test is for rapid screening and surveillance, and provides results for five types of CRE in about an hour.
BioFire Diagnostics' sepsis test has CRE targets and that firm was the first to gain FDA clearance for the molecular KPC assay that is in its FDA-cleared BCID panel, a representative told GenomeWeb. An upcoming test will also cover five types of CRE and will be part of a lower respiratory tract infection panel that should begin clinical studies in the first part of 2016, the representative said.
Great Basin is developing a test directly from whole blood, Becton Dickinson debuted a research-use-only test on the BD Max in 2012 that can be used for surveillance, and firms like Renishaw, Mobidiag, and Curetis are developing tests for CRE or with CRE targets.
So what can molecular diagnostics makers do to avoid these sorts of troubles in the future?
Further developing the product to make it a bit more specific would certainly help, Humphries said. The false positives may be from organisms in the gut that harbor genes similar enough that they cross-react, so investigating this might be helpful to understand the phenomenon, she said.
She also suggested study location choice can play a role. By way of example, Cepheid's MTB/RIF clinical trials were done in Africa and Asia, where the incidence of rifampin resistance is high, "so if you detected a mutation in that gene, chances were it would be a rifampin-resistant isolate," Humphries said.
"If you start doing that test here in the US, where the incidence of rifampin resistance is exceedingly low, more often than not you're just picking up silent mutations that have nothing to do with resistance," she said.
She further suggested that as the community moves forward with more molecular testing, labs really need to evaluate the technology for their particular patient population.
For the NIH group, Humphries noted that it is not necessarily a lost cause for the Check-Points test.
"The performance was good for KPC, and that's mostly what they see; if this was something I was looking at in my lab, I would say the KPC target works so we will depend on that and if we detect someone who has one of these other targets then we need to do a follow-up culture and try to isolate the organism that way," she said. In fact, when labs bring on new molecular microbiology tests they often do this anyway.
The NIH authors agreed, adding that risk-benefit analysis of taking on a new assay should include "cost, workflow, infection control algorithm, and the target patient population," and noted that, in a setting with high prevalence, high NPV enables molecular screening to rapidly rule out patients who are not colonized.
Ultimately, an inexpensive, fast, and accurate assay would be valuable for surveillance in many hospitals, but "it is important to remember that molecular assays for carbapenemases will detect only those genes which they were designed to target, and culture is useful to detect antibiotic resistance that may be due to mechanisms other than a carbapenemase gene," so a combined approach may still be warranted, the authors said.
And Humphries suggested there is hope that proliferation of these so-called superbugs in US facilities can be staved off.
"We're still trying to figure out the whole epidemiology of CRE," she said. "The available data suggest that if you are very aggressive with your infection control — including screening patients, particularly when they come from higher-risk backgrounds [like] multiple hospitalizations, nursing facilities with a lot of CRE, or who were recently hospitalized in another country — then you can really prevent the spread of these organisms."