NEW YORK (GenomeWeb) – Non-profit health services research organization ECRI believes its newly launched interactive genetic testing resource, ECRIgene, can aid stakeholders as they navigate an increasingly complicated maze of genetic tests, with added value over currently available databases and registries.
The institute announced ECRIgene this June, and is now just a few months into offering it to paying subscribers from the clinical, regulatory, reimbursement, and test development spheres.
Vivian Coates, ECRI's vice president of information services and health technology assessment, spoke with GenomeWeb about how the organization is hoping what it has created will help users better engage and grapple with the scientific evidence underlying genetic tests that have the most potential to significantly impact patients' health and outcomes.
"We think this area has tremendous, tremendous promise, but also brings with it challenges for payors, providers, and the public. And ECRIgene is an attempt to help sort through those issues," Coates said.
Toward that end, the new database has been populated through a proprietary approach developed by ECRI that culls from the greater genetic testing market — which some have estimated to represent 60,000 or more tests — plucking out tests that are of particular concern for either clinicians, payors, regulators, or other concerned parties based on their actual, or claimed, impact on patients.
As part of its work in providing governmental and private health technology assessments, ECRI has been paying attention to genetic tests for many years, Coates said. But in the past two, clients began to ask the institute if it could do more "from the perspective of trying to help these stakeholders grapple with what this area has become."
"The answer was yes," she added. "We have seen increasing complexity with the advent of [next-generation sequencing] and multi-gene panels … and as an organization that focuses on evidence, we see huge gaps."
As tests have become more numerous and more complex, it has become more difficult to keep track of them, and to be able to compare them in terms of their evidence base, or even to understand their individual scope.
In the case of multigene panels, for example, Coates said, some genes included in a particular test may have well documented evidence linking the presence of mutations to important clinical factors for the management of a patient. But for other genes on the same panel, the evidence may be much less documented.
"This is in the face of aggressive direct to consumer and … direct to clinician marketing," she said. "So there were a lot of concerns expressed to us in terms of the appropriateness of test ordering, interpretation, and [communication of results to patients]."
According to Coates, ECRI doesn't have the resources, as a non-profit, to tackle the full breadth of the genetic testing realm. Instead, it has developed a filtering method to identify the few thousand tests that are the most important to track and include in its database, via criteria focused on a test's actual, or purported impact.
Inclusion criteria for ECRIgene include "looking for tests where the results either improve patient management or outcomes or the lab is making claims that they do," Coates said. This means tests that represent a change in some sort of care paradigm, for example cancer genomics tests intended to guide treatment with targeted therapies, or pharmacogenomics tests that can drive decision-making around specific drugs.
ECRI is also looking out for tests that may have a large cost impact, either because a test is expensive, or because a patient population is large enough that spending, in aggregate, would be very high. Another indication that a test might make it into ECRIgene is potential to generate ethical or political controversies, so the institute's method for populating the database also includes monitoring for tests that are heavily publicized, promoted, or hyped.
Questions from existing clients have prompted inclusion of tests, as have requests made by labs and manufacturers, and ECRI has been mining CPT code data, payor policies, and employing a team of medical librarians.
According to Coates, the content of the database will be an evolving landscape. It likely won't ever remove tests completely, but if a test is superseded by a newer assay in the same indication or if a company is acquired by another and a test potentially changes names, the older entry may be archived, she said.
ECRI is also closely following regulatory shifts, namely the US Food and Drug Administration's planned mobilization of its oversight over laboratory-developed tests.
"We are staying on top of this and if FDA is able to go forward, for tests that are going through that pathway, we will indicate [in ECRIgene] whether they are under an investigational device exemption or humanitarian device exemption, or PMA or 510K, as all of these labs become essentially medical device manufacturers and are subject not just to premarket submission requirements, but also post-market surveillance requirements," Coates said.
The database is set up so users can filter information in a variety of ways. One of the things we were asked to do, which is actually pretty ambitious, is to include all the gene loci for all the tests in [the resource,]" Coates said. Right now that's about 4,000 genes.
A challenge in this area for the institute is keeping up to date as labs potentially change multigene algorithmic or NGS-based tests.
The database also represents 3,000 searchable clinical conditions at this point, she said. Or users can search by CPT code, which ECRI has licensed the ability to include from the American Medical Association, by test sample type (for example blood-based or liquid biopsy tests), by purpose (diagnostic versus predictive versus screening), or by target (e.g. DNA, RNA, proteins, whole chromosomes).
Each entry in the database includes any information the ECRI can find on a test's content and indication, its regulatory pathway, what professional guidelines include it if any, what payors have policies that mention it, and how it can be coded for billing.
And wherever available, each entry links to an ECRI evidence report, if the institute has created one. If it hasn't subscribers can request one on demand, Coates said.
Each entry also lists when ECRI last added or updated information "to make that completely transparent," she added.
Technology assessment can sometimes feel to those being assessed like technology persecution. And Coates stressed that the role of ECRI and its goal with ECRIgene is not to pass judgement on tests, but rather to do a deep dive on the evidence, providing conclusions about the strength of the evidence overall and with regard to answering specific questions relevant to clinical practice, regulation, and reimbursement.
"Just because there may be gaps. It doesn't mean something isn't useful or doesn't work, it just means we don't know at this point in time," she said.
For example, in ECRIgene's entry for Exact Sciences' Cologuard, the test's methodology is described — including the genes tested and the technologies used. The entry also shows that the assay has been approved by the FDA and provides a link to its premarket application. It reports that Cologuard is included in guidance statements from the American College of Physicians, the National Comprehensive Cancer Network, and the US Preventive Services Task Force.
In the associated and linked test evidence report, ECRI also provides a summary of the available evidence, distilling for users that Cologuard has shown itself to be more sensitive but less specific than fecal immunochemical testing, and giving the strength of evidence underpinning the conclusions about the test a rating of ‘moderate.'
Because of the criteria for how ECRI creates its evidence ratings, they present a high bar for an emerging technology like genetic tests. Moderate "is about as good as you can get for a genetic test," Coates said.
The report also mentions that other technology assessments have been done by groups like Blue Cross Blue Shield, with similar conclusions regarding the evidence of the test's sensitivity and similar recognition that impact on colorectal cancer mortality compared with other methods is still unknown.
"Ideally, future evidence would compare patients who underwent screening with Cologuard with patients who underwent screening with another CRC screening method and report on long-term clinical utility. At this time, clinicians will need to make patient management decisions using available diagnostic accuracy data," the ECRI test evidence report concludes.
Another test included in ECRIgene is Fujirebio Diagnostics' ROMA (Risk of Ovarian Malignancy Algorithm). ECRI hasn't created the same type of in-depth evidence report for ROMA as it has for Cologuard, so the evidence pertaining to the test isn't given a strength rating in terms of its sensitivity, specificity, and clinical utility. However, the ECRI Product Brief on ROMA reviews the evidence and comments on the evidence gaps and theorizes how the test might be useful if clinical utility evidence showed it to be of value in clinical decision making.
This product brief reports that professional guidelines currently do not recommend ROMA, or other protein biomarker tests for ovarian cancer risk assessment.
For example, the NCCN 2015 ovarian cancer guideline states that "It has been suggested that specific biomarkers (serum HE4 and CA-125) along with an algorithm [ROMA] may be useful for determining whether a pelvic mass is malignant or benign. The FDA has approved the use of HE4 and CA-125 for estimating the risk for ovarian cancer in women with a pelvic mass. Currently, the NCCN panel does not recommend the use of these biomarkers for determining the status of an undiagnosed pelvic mass."
In terms of the test's sensitivity and specificity, the brief summarizes that studies have found that ROMA’s diagnostic performance is similar to that of the two protein biomarkers that comprise the test and to other algorithm-based risk assessments that include these biomarkers
"Therefore, ROMA may not provide any additional prognostic benefit if used in place of CA-125 and/or HE4 testing, or in addition to other algorithm-based tests to assess ovarian cancer risk," the report reads.
ECRI also states in the brief that it was not able to identify any studies on the test's clinical utility. However, it says there is an ongoing prospective, observational cohort trial that aims to evaluate the use of ROMA with an expected completion date in 2017.
According to Coates, an important thing for users of ECRIgene to understand is that ECRI's technology assessments and evidence reports are perishable. While there may be evidentiary gaps right now, those could be filled in the future with the publication of ongoing clinical research studies.