NEW YORK – A recent benchmarking study by researchers from Google and Element Biosciences has found that the Element Aviti platform delivered "noticeably higher accuracy" than Illumina sequencing across different ranges of coverage.
While experts believe the study, which was posted as a preprint on BioRxiv in August and has not yet been peer-reviewed, points out potential strengths of the Element sequencing technology, further studies are still needed to systematically evaluate other metrics of the next-generation sequencing newcomer.
According to Andrew Carroll, product lead of genomics at Google Health and the first author of the paper, the study compared PCR-free Illumina whole-genome datasets previously generated on the NovaSeq 6000 sequencer by a third-party lab with whole-genome Aviti data generated internally by Element.
Specifically, Carroll said the Element data reflects two generations of the Aviti sequencing chemistry: the earlier version the company launched in 2022 and a later Cloudbreak chemistry that was released earlier this year.
A unique feature of the Cloudbreak chemistry, he pointed out, is its ability to generate sequence reads with a longer insert size, achieving a medium template length of 1,250 bases.
To perform the benchmarking, Carroll and his team first trained DeepVariant, a deep-learning-based variant caller previously developed by Google for Illumina sequencing, with Element data. "For any new sequencing technology, we always want to see if we can add some of that into the training data for DeepVariant and better support the users of the community," he said.
The training results showed that DeepVariant can "effectively learn from the other data type without having trade-offs," Carroll said, indicating the tool can be deployed for analyzing both Element and Illumina data.
From there, Carroll’s team carried out a head-to-head comparison of the two sequencing platforms by comparing their single nucleotide polymorphism (SNP) and indel sets against the Genome in a Bottle truth set with matched coverage and the same DeepVariant model.
Overall, Element sequencing data had higher accuracy across various ranges of sequencing coverage, from 20X to 50X, Carroll said, although both platforms "performed very well" at high sequencing coverage within the 30X to 50X range.
However, when sequencing coverage drops, the performance of Element sequencing starts to "more substantially diverge" from Illumina, he said, with the former becoming noticeably more accurate in the 20X to 30X coverage range.
In addition, Carroll said the team observed "much cleaner" data with Element sequencing during the heuristic step of DeepVariant’s variant calling method, which uses observed allele frequencies to propose positions as candidate variants, leaving "less noise" for the neural network to sort out.
"One of the key things that seems to be driving this is, when you look at reads in the homopolymer and tandem repeat regions, there is a very clear difference between Element and Illumina," with Element reads better maintaining their quality as they pass through those difficult parts of the genomic sequence, he explained.
Furthermore, by comparing data generated with Element’s new Cloudbreak chemistry with Illumina sequencing, the Google researchers concluded that the longer insert sizes can lead to even higher accuracy across all sequencing depths.
Carroll said the longer insert size and higher accuracy enabled by Element's new chemistry has several benefits. For one, it can improve the ability to map difficult parts of the genome, leading to more accurate analysis of large genetic variants.
Additionally, he said it can help clean up some of the remaining errors in the already very high-quality benchmark datasets and reference sequences, which can be important for the genomic research community.
Beyond these, Carroll believes that "some of the most exciting potential applications probably have not been totally explored" yet, given Element is still a new technology.
The results of the study are "something we have been anticipating," said Catharine Aquino, head of the genomics core facility of the Functional Genomics Center at ETH Zurich, who was not involved in the study. "To have it [contextualized] as a paper is of course better."
Aquino’s team currently operates a suite of Illumina platforms, ranging from NextSeq to NovaSeq, and her lab is slated to receive an Element Aviti sequencer by the end of October.
To some extent, Aquino said, the data presented in the Google study mirrors her group's own observation. A few months ago, her team performed a preliminary study where the researchers ran wastewater surveillance samples with certain known pathogenic variants on different sequencing platforms, including from Element and Illumina.
"It seems that we could actually find the same variants [with Element sequencing] that we find using Illumina sequencing with fewer reads," she said. However, Aquino made clear that the results were based on unpublished preliminary data, and her team will need to repeat the study to confirm them.
While the Google researchers’ study pointed to the potential advantages of Element sequencing, Aquino said she still wants to see how the platform behaves "in the real world" in the hands of independent researchers, given the data used in the study were generated by the company itself.
In addition, she said there is more to consider than just accuracy when choosing between different sequencing technologies. For instance, given that her lab will install the instrument soon, Aquino said she is "constantly thinking about" the size of Element’s reagent kits.
"The consumables are big, and depending on your freezer space, you can only stock up so many runs," she said. In contrast, some reagents for Illumina sequencing can be stable at ambient temperatures and are becoming increasingly compact and simplified.
"Nobody wants to open boxes anymore," she added. "Funnily, that is one prohibitive [factor] of Element sequencing."
"We welcome an objective comparison that includes Illumina’s combined sequencing and Dragen workflow, as the Element paper did not use all parts of the Illumina workflow in their comparison," an Illumina spokesperson wrote in an email. "When data are reanalyzed with Illumina NovaSeq and Dragen secondary analysis, it outperforms Element with fewer false positives and fewer false negatives, for both of Element’s short and long insert protocols."
However, the spokesperson did not further explain which specific workflow component was left out by the Google team and did not provide details on Illumina's internal analysis.
Toumy Guettouche, CSO of Mercy BioAnalytics, who is an NGS expert but was not involved in the study, said the Google preprint "shows what Element has been claiming — that they are more accurate than Illumina sequencing and that they can do longer insert lengths than Illumina."
However, given that the study demonstrated that Element sequencing is more accurate than Illumina mainly at lower sequencing coverages, Guettouche said it remains to be seen how much difference there is between the two technologies at higher sequencing depths beyond 50X.
In the research setting, he explained, people typically sequence at fairly low depth — 60X for whole-exome sequencing and around 30X for whole-genome sequencing. However, for clinical sequencing, people usually sequence at higher depths. "It seems like from the data that they presented, that benefit is less pronounced in that," he noted.
Even though the Element sequencing accuracy seems to be higher than Illumina's, Guettouche said it is unclear what the actual benefits of this are. "Let's say you can sequence at 20X depth of coverage instead of 30X, how much money do you actually save with that?" he asked.
Echoing Aquino’s point, Guettouche also said the current benchmarking study only represents part of the whole picture when comparing different sequencing technologies.
"When you look at a decision in terms of sequencing or what is important for customers, it is not only accuracy," he pointed out, adding that the study did not quite explore other important factors, such as GC bias, uniformity of coverage, sample prep, platform and operational costs, instrument downtime, and customer support.
Furthermore, Guettouche said that while Element’s augmented accuracy is an improvement on short-read sequencing, "it is an evolutionary step, but not a revolutionary one," especially given the backdrop of other short-read companies, such as Singular Genomics, Complete Genomics, and Ultima Genomics, as well as the technological advancements enabled by single-molecule sequencing companies including Pacific Biosciences and Oxford Nanopore Technologies.
Although Guettouche thinks competition is great for customers, he said it remains to be seen if any of the new sequencing companies can make significant inroads and take some market share from Illumina, which he believes is still "essentially a monopoly."
Currently, "nobody is going to be fired because they bought an Illumina sequencer," he said. "But you can't be 100 percent sure about the other companies."