This story has been updated to include additional comments from Encarnita Mariotti-Ferrandiz.
NEW YORK – Immune cell profiling products offered by Adaptive Biotechnologies and Takara Bio ranked high in measures of reproducibility, reliability, and sensitivity in a new benchmarking study that compared nine different T cell receptor (TCR) sequencing methods. But the head-to-head comparison revealed "large systemic biases," the study authors said, with respect to the T cell clonotypes the methods uncovered.
Led by Encarnita Mariotti-Ferrandiz, an immunology researcher at Sorbonne University in Paris, an international team sent out the same sample for analysis by nine different TCR sequencing methods, performed by the companies or groups that developed them. In addition to the Adaptive and Takara methods, the team compared commercially available kits or services from iMonitor, Thermo Fisher Scientific's Invitrogen, and iRepertoire, as well as several leading lab protocols. The results appeared earlier this week in Nature Biotechnology.
Each method was analyzed based on sequence read alignment, the ability to detect rare TCRs, and TCR B variable gene (TRBV) usage, and was compared with protein expression quantified by flow cytometry. The researchers also created an in silico "metarepertoire" of clonotypes found by all methods; no method found more than half of all clonotypes in the metarepertoire for both TCR-alpha and -beta chains.
Most of the methods used to sequence the TCR-beta chain, which has more sequence diversity and is therefore more widely used, performed well and there was not much to separate them, Mariotti-Ferrandiz said. Sequencing the alpha chain, however, "is a bit more complicated, and there's definitely some methods that are better than others," she added.
Also, some methods appeared to be better suited to specific questions. "It depends on what you're looking for, whether you would like to have a really large overview of the repertoire, for diversity as far as you can go, or whether you would be just interested in the main expansions," Mariotti-Ferrandiz said. "It's not trivial, you can't just use whatever you want."
The paper helps clarify some differences in the methods, such as whether they're looking at DNA or messenger RNA, explained Suvarna Gandlur, associate director of NGS marketing at Takara Bio USA. Both multiplex PCR, using DNA as input, and rapid amplification of complementary DNA ends (RACE) methods, using RNA, have benefits, she said. "While DNA-based methods can be used for more quantitative studies, RNA-based methods are more sensitive." The use of unique molecular identifiers (UMIs) was another difference between some of the methods. "There is a benefit to UMIs, but they're not always needed," Gandlur said. Takara's SMARTer Human TCR a/b profiling kits use (RACE) PCR to analyze mRNA; the kit evaluated in the study did not use UMIs, but they're now an option for newer kits, she said.
"It's great to see Adaptive perform so well in this assessment," Harlan Robins, the company's cofounder and CSO, said in an email. Adaptive gained exclusive rights to iRepertoire's methods for use in minimal residual disease testing through its 2015 acquisition of Sequenta. Adaptive's immunoSeq assay was used to analyze TCR-beta chains only in the study, though it can also be used to analyze TCR-alpha chains.
The TCR sequencing benchmarking project started in 2016, Mariotti-Ferrandiz said, with discussions among colleagues about comparing several different methods. Adaptive immune repertoire profiling has become critical to several research fields, especially oncology, and has applications in diagnostics, including companion diagnostics and minimal residual disease testing.
Between her postdoc work, collaborations, and the use of some commercial services, Mariotti-Ferrandiz had noticed that the same sample run twice with the same method would not always yield the same results.
She began sharing her data with colleagues and everyone she spoke to "was enthusiastic trying to compare their own protocol to other protocols," she said. By 2017, she had collected a T cell sample, aliquoted it, and sent it out for analysis. The researchers later collected a second sample to run another round of analysis.
"Most of the collaborators were really involved from the beginning in terms of getting the results and designing the proper analysis strategy," she said. "The most important point is, we've been all working with the same sample. Each lab followed their original methods. We did not have an impact on how the protocol could be modified." All data were analyzed by the same person to further reduce investigator bias, she said.
Overall, Mariotti-Ferrandiz said, she did not get a bad impression of any particular method. Still, the study provided rankings for each method based on sensitivity, reproducibility, and reliability. Here, Takara Bio and Adaptive's methods stood tall, as did a TCR-alpha chain sequencing method developed at Germany's Technical University of Dresden.
Takara Bio's SMARTer kit gained the top grade in TCR-beta chain reproducibility and reliability and the second highest grade for sensitivity. For TCR-alpha chain sequencing, the method had the highest grade for reliability and second highest for reproducibility and sensitivity. Adaptive's immunoSeq assay, which was only used for TCR-beta chain sequencing, scored the second highest grade for all three categories.
In Mariotti-Ferrandiz's own work on autoimmune and inflammatory diseases, she has used Takara's kits and has participated in early-access programs with the company providing feedback on the products. "In humans, it's important to get out of the blood a picture of the immune response," she said. "Since most immune cells are organ specific, what you can get out of the blood should be really rare. In my case, what I’m looking for is a collection of rare clonotypes." This has led her to use RACE PCR-based kits.
Cost is also a factor for her and others. The paper suggested that Adaptive's kit starts at approximately $350 per sample, compared to approximately $150 per sample for Takara's kit and approximately $230 per sample for some of the other TCR-beta chain sequencing kits or services.
"There are always going to be situations where one technique is better suited to answer a question than another, but one important factor to keep in mind in this case is the limited sample diversity in this study," said Adaptive's Robins. He noted that the researchers only used one sample type, CD4+ T cells. "An important question to ask here is how each of these methods would perform in more clinically relevant sample types like whole blood, peripheral blood mononuclear cells, and tissue. For example, the majority of tumor biopsies are stored in a formalin-fixed paraffin-embedded format, and RNA-based immunosequencing methods historically have not performed well with this sample type," he said.
"To truly understand which method is best suited for a particular question, that method needs to be tested against a variety of sample types," he said, noting that Adaptive has tested its assay across "a huge variety of samples" and has more than 11,000 human samples in its publicly available database.
Robins also noted that in the reproducibility analysis, the authors removed singletons. "This approach removes clones that were physically present in the sample from the calculation, and the removal of data could skew the analysis in favor of methods that struggle to capture rarer clones," he said.
Removing singletons "is a classical way to reduce the noise," Mariotti-Ferrandiz said, and noted that Adaptive removes sequences in its assay using undisclosed internal filters.
Gandlur noted that given the COVID-19 pandemic, there's interest in the immune system's role in infectious disease. Here, material input is an important factor and the paper suggests that for RNA-based methods, the quantity of starting material affects the diversity of TCR clonotypes. "A low RNA input influences the number of rare TCR sequences detected but not the distribution of the more abundant TCRs," the authors wrote.
Immune cell profiling is a rapidly advancing field and Mariotti-Ferrandiz recognized that the study did not capture data for all the methods around, even those that existed at the time the study was conceived. For example, Adaptive also offers TCR-alpha chain sequencing, which was not included in the study but has been available as a service since 2010.
She plans to continue to compare methods, although she's unsure how those data will be disseminated. She's particularly interested in evaluating the Takara kit with UMIs. "UMIs are believed to be something that can increase the accuracy of your data," she said. "Also, most of the sequencers need this diversity to run," especially Illumina's NextSeq platform, she added.