The College of American Pathologists has formed a workgroup charged with developing a checklist for certifying labs running clinical tests based on next-generation sequencing technology.
The CAP workgroup, which is still in its early stages of activity, hopes to have an accreditation checklist for next-gen sequencing in place by the end of 2012, according to Nazneen Aziz, director of molecular medicine at CAP and the organization's staff lead for the effort.
The workgroup also aims to put together proficiency-testing guidelines for technicians running next-gen sequencing-based tests, but Aziz told Clinical Sequencing News that that effort would likely have a longer timeline than the accreditation checklist.
Aziz said that CAP is developing the accreditation checklist in cooperation with representatives from the American College of Medical Genetics and the Association for Molecular Pathology in order to ensure consensus across the clinical lab community. "CAP doesn't want to do this in isolation," she said. "This is being done by a community of experts in the NGS field, so what we put out will be well accepted."
There are currently 11 CAP members on the workgroup. Two additional members — one from AMP and one from ACMG — will be named shortly.
Checklists are the cornerstone of CAP certification, which follows more stringent accreditation standards than those enforced under the Clinical Laboratory Improvement Amendments. While some labs with CAP certification already offer tests based on next-gen sequencing platforms — such as Children's Mercy Hospital, the Emory Genetics Laboratory at Emory University School of Medicine, and Illumina — Aziz noted that CAP has only accredited these labs' general laboratory protocols, not the specific sequencing-based tests they are running.
By developing an accreditation checklist specifically for NGS-based clinical testing, the organization hopes to address the many critical differences between NGS-based clinical tests and standard molecular diagnostics.
"These tests are not going to be like a simple molecular genetics test looking at one loci and one SNP, or even using a little bit of Sanger sequencing," Aziz said. "These will require the labs to know a lot of different processes in order to say with confidence that … they have really done everything right."
Karl Voelkerding, medical director for advanced technology at ARUP Laboratories and a member of the CAP workgroup, told Clinical Sequencing News that the effort is intended to give labs confidence in the results of NGS-based tests.
Noting that next-gen sequencing is "a complex technology that is continuing to evolve," Voelkerding said that clinical laboratories adopting the technology "really want to understand the performance characteristics of any given next-generation sequencing-based technology, and then we want to understand the specific performance characteristics for any given diagnostic assay that’s based on next-generation sequencing."
CLIA regulations and CAP's current guidelines governing molecular testing do not address many issues related to next-gen sequencing, Voelkerding said. "When it actually comes down to practice implementation, you have to go at a finer level and ask, 'What is this technology's sensitivity? What is its sensitivity? What is its precision and how did you establish that?'"
CAP's effort to establish specific requirements for next-gen sequencing "acknowledges the fact that this is a new technology moving into clinical laboratories, and therefore there needs to be the appropriate discussion and development of new guidelines and requirements," he said.
"For a laboratory looking to embark upon adopting this new technology, it's very helpful to have guidance and regulatory requirements that have been developed by individuals with expertise in the area to assist them in guiding their own laboratory environment," he added.
A Broad Spectrum
Accreditation issues related to next-gen sequencing will include every step of the workflow, Aziz noted — from sample prep to data analysis and bioinformatics. "This includes handling the data, storing the data, and preparing the library for sequencing," she said. "So, a lot of things are included."
Aziz added that the "mission and goal of this workgroup will be to decide on all of the specific metrics — the parameters that are encountered in analytical validation, quality control, the bioinformatics workflow, all of these — before we develop the final checklist."
Voelkerding added that CAP intends to address "the entire spectrum" of the NGS workflow — "from the receipt of patient samples to the development of interpretive report and all the steps in between."
He noted that many of these steps — particularly those related to sample collection and sample prep — are already addressed in CAP's molecular pathology guidelines, but acknowledged that there are a number of "new and unique aspects of the process that next-gen sequencing brings to table."
Bioinformatics should be a particular challenge since labs use many different types of algorithms and software packages that each have unique strengths and weaknesses, Voelkerding said, though he noted that CAP believes it has "enough diversity of expertise on the workgroup committee to address and discuss the entire process."
[ pagebreak ]
Accreditation will be assay-specific, so the workgroup aims to issue guidelines that account for the unique requirements of next-gen sequencing-based tests. "With other genetic tests you know what you're looking for, so you can be very specific about" the accreditation requirements, Aziz said. "But with next-gen sequencing, in a way, you're discovering at the same time that you're testing."
In line with that, she said that general checklist items might outline quality control parameters or analytical metrics that labs will need to follow for specific tests run on next-gen sequencers, rather than provide a specific list of genes or variants that a test should identify.
"It's a little different from other genetic tests where you know what you're looking for. Here you cannot be that specific," she said. "Every individual is different from another — someone's cancer or even genetic heritable disease might be due to a unique mutation in a known gene. So we can't be that specific."
Likewise, the workgroup will also need to take a more general approach when it turns to proficiency testing, which assesses the capabilities of lab employees running the tests rather than the lab's protocols and workflows.
"We may not be testing gene-specific" outcomes when it comes to proficiency testing, she said, "but we might test [whether an operator knows] how to do deletions when there are deletions in a region, or can you correctly detect translocations, or SNPs, or CNVs?"
A Flexible Framework
Aziz acknowledged that the rapid pace of NGS development will make it difficult for CAP to enforce a strict set of standards for labs running sequencing-based tests. Instead, the organization plans to "allow some flexibility" in its guidelines to account for expected changes in technology and scientific knowledge.
"We'll put it out with version control," she said, explaining that the first version of the checklist may offer general recommendations for certain areas rather than strict requirements. "Then, once we are sure that there is experience in the field and [something] should be done in [a certain] way, then we will make it a requirement."
This flexibility will be necessary, she said, because "we know that this is an evolving technology and we simply don't have enough experience in the clinic. It's barely hitting the research arena and the transition has been so fast … that we believe it needs some discussion and time before we make very rigid requirements."
Voelkerding said that although sequencing is evolving rapidly, it's not the first new technology to find its way into clinical labs. "Laboratories historically have adopted new technologies that then continue to evolve while they're utilizing those technologies for patient practice," he said, citing magnetic resonance imaging and CAT scans as methods that have evolved since they were initially introduced into clinical practice.
He explained that the "first level" of guidelines that CAP will issue for next-gen sequencing "will be those that ask laboratories to describe how they are approaching understanding the performance characteristics of the next-generation sequencing technology that they are using and how they confirm the results generated by next-generation sequencing."
He estimated that it would take several years "to layer onto that additional specific types of requirements and guidelines as the technologies begin to mature."
Voelkerding said that CAP will aim to strike a balance between the rapid development of the field and the need for clinical standards. "If we try to develop guidelines and requirements that are too rigid and don't acknowledge the fact that the technology is rapidly evolving, then that will make it very difficult for laboratories to meet the requirements."
In line with this emphasis on flexibility, CAP aims to make the checklist general enough to cover all available sequencing platforms while accounting for the varying capabilities of different instruments.
"We want to make it very encompassing of all the different platforms, but we will certainly not have one requirement for all of the different platforms because we know that different platforms have different strengths and merits," Aziz said. "So it will be the same checklist, but it will have different requirements for the different platforms."
The CAP workgroup is still considering whether and how it might seek input from sequencing vendors as it puts the accreditation plan together, Aziz said.
Have topics you'd like to see covered in Clinical Sequencing News? Contact the editor at btoner [at] genomeweb [.] com.