Skip to main content
Premium Trial:

Request an Annual Quote

VBI to Use Convey's Hybrid-Core System to Analyze Data for 1000 Genomes Project, Other Tasks


By Bernadette Toner

This story was posted May 24.

The Virginia Bioinformatics Institute said this week that it has installed a "hybrid-core" computing system from Convey Computer that it will use to analyze data from the 1000 Genomes Project and several other tasks, including text-mining projects and simulations of the spread of infectious diseases through populations.

VBI also plans to open up the HC-1 system, which combines standard multi-core x86 processors with field-programmable gate arrays, to other research groups across the Virginia Tech campus in an exploratory effort to create a larger-scale "computing hub" that will comprise a medley of computing architectures, including standard processors, FPGAs, and graphics processing units.

The aim is to build a "much larger, scalable system for all of Virginia Tech and VBI," Harold "Skip" Garner, executive director of the institute, told BioInform. "The goal is that these new classes of systems can be the basis for new a new type of … supercomputer system that would offer floating point calculations as well as non-floating point calculations, and would be applicable to a variety of domains."

Garner said that Convey's combination of FPGAs and standard processors offers a "fundamentally different way of doing computation" that combines the programming ease of x86 processors with the performance gains of application-specific hardware. FPGAs have traditionally been very difficult to program, Garner said, but Convey "broke the barrier" to enable developers to implement their own code on the system.

More importantly, he added, algorithms that run on the HC-1 are "blindingly fast."

Indeed, Convey announced separately this week that it has achieved a 172-fold speedup for its implementation of the Smith-Waterman alignment algorithm when benchmarked against SSEARCH using the SIMD SSE2 instruction set on an Intel core.

Garner said that he and his colleagues at VBI are using Convey's Smith-Waterman implementation on the HC-1 as part of a project to analyze data for the 1000 Genomes Project. Specifically, Garner's team is responsible for analyzing microsatellite data for the project, which required re-assembling all the data that had previously been generated — more than 340 terabytes of data so far.

"That would be very difficult to do without a machine like this," Garner said. "It's been a real workhorse for analyzing the microsatellite data."

He added that the new system, comprised of three HC-1 servers, runs code "hundreds of times faster" than all the other computers at VBI would if they were to run at the same time — an ability that "enables us to do many things that we couldn't do before because we didn't have the room, or the electricity, or the cooling."

While the 1000 Genomes Project analysis is the "immediate application" for the new system, Garner said that he and his colleagues will also be porting their text-analytics tool called eTBlast to the HC-1. The software currently runs on a Dell server and allows users to enter entire abstracts or other large blocks of texts in order to find similar papers.

Switching to the HC-1 server will allow the service to handle much larger text databases, he said. "It will basically enable us to provide real-time high-performance computing with a web interface. Users do not need to know they're running on unique architecture."

Garner said that in addition to the three HC-1 servers, he has ordered around 100 terabytes of high-speed networked storage from Panasas in order to "match the I/O needs of the ultra-high-throughput machine."

The new Convey system adds to VBI's current computing resources, which includes 750 cores distributed across 150 servers and clusters, and more than 200 terabytes of disk storage. In addition, VBI researchers have access to other computational resources at Virginia Tech, including around 100 petabytes of storage.

However, VBI's computing needs are not only outgrowing the resources on campus, but they're "outgrowing what traditional computers and clusters can do," Garner said. Jobs like the 1000 Genomes Project analysis and other bioinformatics tasks "kind of break the scaling limitations" of traditional processor-based clusters, he said.

[ pagebreak ]

As a result, Garner envisions the HC-1 at VBI will serve as a test case for a broader, campus-wide "supercomputing-on-demand" service that would include "thousands" of processors, including FPGAs and GPUs. "We're just prototyping with these first systems we have," he said.

Garner said that VBI installed the HC-1 several weeks ago and has so far only run the 1000 Genomes Project analysis on it. "We haven't been able to really put it through its paces yet where we're really developing code and capability on it," he said, but added that the VBI programmers will be working with the company to port the newest version of eTBlast to the new system later this summer.

Gaining Traction

In addition to the VBI agreement, Convey announced this week that it is working with Jason Bakos, assistant professor at the University of South Carolina's department of computer science and engineering, to speed up code for computational phylogenetics and other applications "that have never been accelerated before," Bakos said in a statement.

George Vacek, who leads Convey's life sciences business unit, said that Bakos is "close to being done" with an implementation of the MrBayes phylogenetics algorithm that will run on the HC-1.

When complete, the MrBayes implementation will be available for download as a "personality" — an extension to the x86 instruction set that is designed to implement specific calculations on the FPGA.

Vacek said that the company is working on implementations of other common bioinformatics algorithms, such as Blast and HMMer, but did not want to provide a timeline for when they might be available other than to say that "we should have stuff coming out at a fairly good clip."

As far as the expected speedup on other applications, Vacek said they may not be quite as fast as Smith-Waterman, which is particularly "well-suited" to the platform. However, he said he expects most bioinformatics tools to run "a couple orders of magnitude" faster on the HC-1 than standard processors.

Convey was founded in 2006 and began shipping the HC-1 in late 2009 (BioInform 11/20/2009).

In addition to its work with VBI and the University of South Carolina, other life-science customers Lawrence Berkeley National Laboratory, where researchers are using the system to speed up a graph-based algorithm for short-read assembly; and the University of California, San Diego, which has optimized its InsPecT/MS-Alignment proteomics software package for the HC-1.

The performance gains from the hybrid system are coupled with improvements in energy efficiency. For example, according to company literature, UCSD found that one rack of HC-1 servers could replace eight racks of other servers, which corresponded to a 91-percent reduction in energy requirements.

The HC-1 server includes a 2.13 GHz Intel Xeon "host processor" and a Xilinx Virtex 5 FPGA "coprocessor" in a 2U rack-mountable chassis. Developers can use standard Fortran, C, and C++ development tools to deploy applications, which can contain both x86 and coprocessor instructions in a single instruction stream.

Vacek said that the system's specialized "personalities" make it easier for programmers to develop code for the HC-1 than for other FPGA-based systems. "Part of our philosophy is that the platform has to be easy to use, which is not true for other accelerators," he said. Using the personalities, developers "don't have to write in hardware language. They can just compile and go."

The ease of porting new code to the system "depends on the code and the customers," Vacek acknowledged. The Bakos team at the University of South Carolina "did it all themselves," he said, whereas other groups have worked more closely with Convey on the implementations. "It depends a fair bit on how mature our personalities are and how comfortable [the customer is] using them."

Other customers, meantime, may have FPGA programmers on staff who are "happy to write their own personalities down to the bits on the hardware."

Vacek said that the life sciences is one of the company's "more significant" markets. He noted that the field of bioinformatics has historically been open to FPGA computing, but the "ease-of-use issue" has limited adoption.

Before the company launched the HC-1 last year, "we initially thought we'd have to develop a handful of applications before people would even look at the platform," Vacek said. However, over the last several months, "we've been pleasantly surprised to see a lot of uptake even with limited support."

The Scan

Study Links Genetic Risk for ADHD With Alzheimer's Disease

A higher polygenic risk score for attention-deficit/hyperactivity disorder is also linked to cognitive decline and Alzheimer's disease, a new study in Molecular Psychiatry finds.

Study Offers Insights Into Role of Structural Variants in Cancer

A new study in Nature using cell lines shows that structural variants can enable oncogene activation.

Computer Model Uses Genetics, Health Data to Predict Mental Disorders

A new model in JAMA Psychiatry finds combining genetic and health record data can predict a mental disorder diagnosis before one is made clinically.

Study Tracks Off-Target Gene Edits Linked to Epigenetic Features

Using machine learning, researchers characterize in BMC Genomics the potential off-target effects of 19 computed or experimentally determined epigenetic features during CRISPR-Cas9 editing.