Two recent analyst surveys found that users' experiences with their Illumina HiSeq and Life Technologies' Ion Torrent Proton sequencers vary markedly, illustrating that while Illumina still dominates the market, there may be an opportunity for Life Tech to catch up if it can improve customer satisfaction with Proton's accuracy.
In a survey of 20 Proton customers conducted by Leerink Swann in July, users were generally satisfied with their machines but had concerns about the accuracy of the systems.
Life Tech's Ion Proton "does appear to be meeting expectations for most users," Dan Leonard, an analyst with Leerink Swann, wrote. "However, our data suggest that Proton users are struggling some with accuracy. … While we had thought meaningful accuracy shortfalls might be a symptom of novice users, our data suggest this is not the case."
Meanwhile, a survey of 65 next-gen sequencing users conducted by Macquarie Capital in June indicated that users were slightly more satisfied with their Proton machines compared to their HiSeq systems.
Compared to a similar survey that the investment firm conducted in March, which found that users were happiest with the HiSeq 2500 (IS 4/16/2013), "Ion Torrent made significant gains in terms of instrument satisfaction," Macquarie analyst Jon Groberg wrote. "Most surprisingly, users were slightly more satisfied with Proton than HiSeq."
The 65 users who responded to the June survey own more than 250 instruments in aggregate, including 56 HiSeq 2000s, 55 HiSeq 2500s, 58 MiSeqs, 46 PGMs, and 25 Protons.
About half of users own at least one HiSeq 2000 and 23 percent own a Proton. Forty percent said they had purchased or upgraded to a HiSeq 2500.
The firm estimates that Illumina controls around 64 percent of the $1.1 billion next-gen sequencing market and that Life has around 22 percent of the market.
From quarter to quarter, HiSeq 2000 satisfaction among users was relatively flat, with 87 percent reporting that the system was either above average or excellent, compared to 88 percent who deemed the system above average or excellent in Macquarie's March survey; while HiSeq 2500 satisfaction dropped to 84 percent in the June survey from 100 percent in March.
Meantime, user satisfaction with the Proton increased significantly, with 92 percent of users reporting the system as either above average or excellent, up from 88 percent in March.
Satisfaction with the PGM also increased from 50 percent to 73 percent. As with the previous quarter, users who own only Ion systems are more satisfied with the PGM than users who own both Ion and Illumina systems.
Illumina's MiSeq still appears to be winning the desktop sequencing war among respondents of the Macquarie survey, with 87 percent reporting the system as either above average or excellent, up from 82 percent the previous quarter.
Separately, Leerink Swann found in its survey of 20 Proton users that while customers are overall happy with the instrument, it is falling below expectations in certain key metrics, including accuracy and informatics.
Of the 20 users, 65 percent, or 13, were from a core laboratory or service provider, and 45 percent, or nine, own between six and 10 sequencing instruments. In addition to at least one Proton, 55 percent also own an Illumina HiSeq, 45 percent have a MiSeq, 70 percent a PGM, 35 percent own Roche's 454 GS FLX, 20 percent operate a Pacific Biosciences RS system, and 40 percent have Sanger sequencing capabilities.
Two respondents who had each performed more than 30 runs on the Proton said the system fell "far short of expectations" on accuracy. Among six users who participated in both the current survey and a similar survey that Leerink Swann conducted in April, three reduced their accuracy rating in July, while two gave the same rating, and one gave a higher rating.
"The data suggest that satisfaction with Proton’s performance on accuracy may have deteriorated," Leonard said.
Aside from accuracy, there was also a drop-off in how customers rated informatics and data handling between April and July. While this decline was less substantial than that of the accuracy rating, and the average rating remained at a 'met expectations' level, repeat users and experienced users were more likely to give a lower rating compared to the group as a whole, suggesting that "these results are perhaps reflective of a customer base which has been trained on [Illumina] data," Leonard wrote.
Leerink Swann also asked users about their future purchasing plans. Of the 45 percent, or nine users, who said they planned to purchase a new instrument in the next year, three said they plan to buy a Proton, while four respondents indicated they would buy a HiSeq, four said they plan to buy a MiSeq, and one indicated plans to purchase a PGM. Additionally, two respondents said they would like to buy Oxford Nanopore's GridIon and one indicated interest in the MinIon, although those machines are not commercially available.
Thus far, very few studies have compared the HiSeq and Proton head to head. In what is likely to be the first of many, researchers from the National Cancer Institute recently compared exome sequencing results on the two instruments, finding that they were comparable when calling SNPs, but that there were differences among indels (IS 6/25/2013).
Such head-to-head comparisons are often difficult to do, however, because the systems are amenable to different exome capture kits and informatics pipelines. Additionally, the technology is continually improving, so by the time the study is published, there are often new methods, reagents, or algorithms available that make the results obsolete.
A survey conducted by GenomeWeb and the investment firm Mizuho Securities at the end of the first quarter also found that comparing the HiSeq and Proton is not clear-cut. Respondents rated the HiSeq higher for accuracy, ease of sample prep, current usability, price per base and throughput, while the Proton won out for price, promise, and run time (IS 4/30/2013).