This story has been updated from a previous version to clarify remarks made by Scipio Bioscience CEO Pierre Chaumat.
NEW YORK – A new study from researchers at Genentech offers one of the broadest comparisons of commercial single-cell RNA sequencing technologies released to date, highlighting both the array of kits available to the field and 10x Genomics' analytical preeminence.
In a BioRxiv preprint posted last month, researchers led by Spyros Darmanis and Daniel Le presented their evaluation of single-cell kits from 10 companies across four technology classes: droplet emulsion kits from 10x and Fluent BioSciences; microwell-based kits from Becton Dickinson, Honeycomb Biotechnologies, and Singleron Biotechnologies; combinatorial indexing-based kits from Parse Biosciences and Scale Biosciences; and Scipio Bioscience's hydrogel-based kit.
They compared performance on a sample of peripheral blood mononuclear cells (PBMCs), as well as sample compatibility, cost, and workflow. For each metric, the technologies were algorithmically placed into one of three tiers — top, middle, and bottom. In addition to headline metrics such as genes and unique molecular identifiers per cell, the researchers also evaluated the technologies on "library-level" metrics, such as cell recovery and sequencing efficiency, and post-analysis metrics such as cluster discrimination.
"All of [the technologies] have advantages and disadvantages," Darmanis said. "We're not trying to say that one is better than the other. … We're presenting this data and saying, 'Depending on your needs and budget, this is how things run. Make a more informed choice.'"
That said, the preprint identified 10x's Gene Expression Flex kit, launched in 2022 to work with fixed samples, as being in a class of its own. In nearly every performance metric, the kit was rated as "top tier." However, that performance comes with high costs — $.17 per cell with upfront equipment costs of $65,000 and a library preparation protocol that lasts just over 24 hours. 10x recently launched a new single-cell gene expression product called GEM-X.
"Additionally, the Rhapsody WTA kit from Becton Dickinson provided a cost-effective balance of performance and expense per cell," the authors wrote.
Many newer entrants noted that the study was not able to make use of improved products that have been recently released or coming soon. Several companies also highlighted the study's inclusion of sequencing efficiency, presented as "read utilization," or the measure of how many sequenced reads yielded usable data.
"Having a higher proportion of usable reads means a lot less waste, as the preprint shows nicely," a 10x spokesperson said in an email.
Darmanis said Genentech embarked on the comparison study about a year ago as new, single-cell methods popped up, offering an alternative to 10x's Chromium platform, which has been used for the "vast majority" of single-cell assays at the company. The driving question was, "How do we make sure that we stay up to date and we have the information we need to make the best choices available for the different types of projects that are being done currently?" he said.
The technologies included show how important throughput has become to the field. When the Association of Biomolecular Resource Facilities' genomics research group did a single-cell comparison study in 2018, they looked at Fluidigm (now part of Standard BioTools), WaferGen (now part of Takara Bio), 10x's Chromium, and Bio-Rad's ddSeq platform.
Only Chromium made the list for Genentech's study. Bio-Rad's Celsee platform, acquired in 2020, and Takara's Icell8 platform were not included on the basis of lower throughput, Darmanis said, though they may be included at some point in the future.
In addition to the 10x Flex kit, the study also looked at the firm's 3' and 5' gene expression assays. All other companies had only one kit tested.
Throughputs observed were between 5,000 cells for Scipio to 154,000 cells for Scale. Cost per cell ranged from $.05 for Fluent and Singleron to $.21 per cell for Scipio.
So-called "library level" metrics included cell recovery percentage, gene and unique molecular identifier (UMI) saturation, and read utilization — defined as the number of UMIs divided by the number of reads. 10x's Flex kit offered read utilization of .44, followed by its 3' kit at .32 and BD's kit at .30; other kits were between .05 and .19.
"We wanted to look a little deeper under the hood," Genentech's Le said. "The percent of reads that actually get mapped defines how many [transcript] counts you're ultimately going to get." But beyond that, certain technologies may create reads that get mapped, "but they're maybe not complete in the sense that we can't actually get the UMI out, so those reads are no longer usable to us."
Knowing this metric is important for both experimental design and budgeting, Darmanis said. Sequencing efficiency is dependent on sample quality, noted Scipio CEO Pierre Chaumat.
Ultimately, the preprint provides a data set of more than 200,000 cells. Researchers may be more interested to find a table comparing every technology on each of the 18 metrics or characteristics considered, as well as "radar plots" visualizing which tier (top, middle, or low) the technology falls into for key usability, sensitivity, and quality control metrics.
In the preprint, the Genentech team noted that they excluded data generated with Singleron's technology, due to cell recovery that was more than double the expected number leading to "unreliable cell calling."
In an email, Singleron CEO Nan Fang said that it had likely provided Genentech with "a wrong [cell] loading table," resulting in overloading the instrument by more than 50 percent. At least one other customer received this table; however, they were aiming to recover fewer cells, "so it was still within the normal range" for the reagents, she said. Still, she questioned the decision to exclude the data from further analysis.
Overall, the study was limited to PBMCs, an important sample type in single-cell studies, especially for benchmarking, as they're relatively easy to work with. The study also only considered whether a technology could do single nuclei or not and did not consider aspects of a method's effect on cell viability. "Those are the kind of application-specific advantages that might drive [researchers] to one platform or another," said Robert Meltzer, Fluent's senior director of applications and strategic partnerships.
Many companies noted that the study provides a snapshot of capabilities that have since evolved. That even includes 10x, which launched a new Chromium chip architecture earlier this year.
In an email, a 10x spokesperson noted data posted to X by the Fred Hutch Innovation Lab on June 27 suggesting that GEM-X 3' kits outperformed Flex kits in median number of genes per cell at 20,000 reads sequenced per cell and showed a 25 percent to 30 percent improvement over the Next GEM technology. Moreover, they said cell recovery increased by about 50 percent for all samples.
Parse CEO and Cofounder Alex Rosenberg noted that the firm's Evercode Mega kit offers throughput of 1 million cells, compared to the 140,000-cell kit used in the study, adding that the preprint "understates the scalability and cost advantage that Parse has."
Fluent launched a fifth version of its chemistry in May. "This has allowed us to directly address areas of needed improvement clearly identified and highlighted by the [Genentech] authors, including read utilization efficiency, gene and transcript sensitivity, differential gene expression, and assay reproducibility," Meltzer said. Improving efficiency and minimizing background noise was a particular area of focus for the firm. Still, he suggested that the preprint's results for PIP-seq version four "are out of bounds of our evaluation" of the same chemistry.
Paris-based Scipio is coming out with a second generation of its technology at the end of the year, Chaumat said. He suggested that the study does not account for the cost to process each sample. Some of the highest throughput kits may offer a low cost per cell, but to do just one sample would burn an entire kit, he said, which can cost thousands of dollars. "We strongly believe that ... the market is asking for kits with suitable performance at lower cost," Chaumat said.
Meanwhile, a BD spokesperson said in an email that "the variation in cell recovery and lower cell recovery rate in this article is not consistent with our claims and customer observations," a BD spokesperson said in an email.
ScaleBio said in an email that its latest kits "offer enhanced performance in UMI and transcript/gene detection" and can go up to 500,000 cells, adding that the study's single-replicate design wasn't able to showcase its technology's ability to minimize batch effects.
The Genentech team is planning to submit the manuscript to a journal. It may include some additional products, such as the 10x GEM-X kits, but is not planning on continually updating its data as new products continue to roll out of these companies' labs.
"We can't be doing this every time there is a new version, otherwise this would be the only thing we're doing," Darmanis said. "We hope at least the way we did it provides a framework to be able to benchmark any of these things that have come out."