BGI's GPU Server Farm Accelerates Genome Analysis Apps

By Matthew Dublin

Yesterday, the Beijing Genomics Institute announced that they have used GPUs to speed up three genome data analysis tools: SOAP3 aligner, GSNP, and GAMA.

"We are drowning in the genome data that our high-throughput sequencing machines create every day," says Bingqiang Wang, head of high performance computing from BGI. "GPU acceleration of our genome analysis applications enables our scientists to crunch through data and gain insights into bacteria, plants and humans faster than was ever possible. It offers the potential for researchers and healthcare professionals to identify highly effective and affordable individualized medicines and treatments."

SOAP3 aligner - Aligns short reads from the sequencing machine against existing reference genome sequences. Through GPU acceleration, the SOAP3 aligner can find all three-mismatch alignments in tens of seconds per one million reads, instead of tens of minutes without GPU acceleration. This means that sequencing and assembling of individual genomes for comparison to those previously sequenced and studied can be performed quickly to understand potential future disease states and treatments. 

GSNP (SNP detection) - A GPU-accelerated version of the widely used SOAPsnp software that detects variation of a single nucleotide polymorphism (SNP) in the DNA of a genome. These DNA SNP variations can be used to study how individuals develop diseases differently and respond to bacteria, viruses and medicines.

GAMA (high resolution genotyping tool) - Finds the distribution of the occurrence or frequency of particular gene variants, such as eye color or the propensity for prostate cancer in a set of genes.

BGI's NVIDIA Tesla GPU-based server farm (credit: BGI)

Highlights from SC11 Part II

This video features Kevin Shinpaugh, director of IT and HPC at Virginia Bioinformatics Institute, discussing uses for HPC at VBI, Ray Bair, chief computational scientist at Argonne National Laboratory, on the Mira supercomputer, and Cycle Computing's Jason Stowe discussing the finalists for Cycle Computing's CycleCloud BigScience Challenge contest. The winners for the contest will be announced next year.

The Making of a Hybrid CPU/GPU Cluster

By Matthew Dublin

In this video, Steve Jones of Stanford University describes the deployment of a hybrid-core HPC cluster that uses both CPUs and GPUs. Jones describes the hardware components as well as the cooling technology that utilizes a liquid cooling system. The video was recorded at the HPC Advisory Council Stanford Workshop on December 6, 2011.

Supercomputer Reveals Mechanisms Behind Drug Detoxification

By Matthew Dublin

Researchers from the University of Tennessee and the Department of Energy's Oak Ridge National Laboratory are using supercomputer simulations to study a class of proteins involved in drug detoxification.

Jerome Baudry and Yinglong Miao, both of whom are jointly affiliated with ORNL and the University of Tennessee, have performed simulations using the Kraken supercomputer, a 1.17 petaflop Cray XT5 Linux system.

With Kraken, they were able to examine the motions of water molecules in a class of enzymes called P450s, which are sometimes responsible for processing a large fraction of drugs taken by humans.

The simulations modeled how water molecules move in and out of the protein's centrally located active site, enabling the team to clarify an apparent contradiction between experimental evidence and theory that had previously puzzled researchers. X-ray crystallography had displayed only six water molecules present in the active site, whereas experimental observations indicated a higher number of water molecules would be present in the enzyme.

"We simulated what happens in this enzyme over a time scale of 0.3 microseconds, which sounds very fast, but from a scientific point of view, it's a relatively long time," Baudry said. "A lot of things happen at this scale that had never been seen before. It's a computational tour de force to be able to follow that many water molecules for that long."

Water molecules (seen in red) move in and out of the active site (seen in blue) of a P450 enzyme. This class of enzymes is responsible for detoxifying a large fraction of drugs taken by humans.

The Kraken supercomputer:

Their research was published in the Biophysical Journal.

Highlights from SC11 Part I

Here is the first video in a four part series featuring highlights from the SC11 conference in Seattle, WA. In this video, we chat about the DOE Genomic Science Program with Betty Mansfield, team leader at Oak Ridge National Laboratory, short-read mapping on FPGAs with Pico Computing's Corey Olson, and the NSF's Center for High-Performance Reconfigurable Computing.

Finding the Purpose of Proteins with Grid Computing

By Matthew Dublin

The international science grid this week has a post that describes how grid computing — not cloud computing — can be employed to aid proteomics research. The University of Nebraska-Lincoln's Robert Powers is using grid computing to increase the amount of data on the purpose of proteins with known structures. Powers has created the Comparison Protein Active Site Structures database, which contains all of the experimentally determined binding sites identified to date – roughly 36,000. The CPASS software uses that database to learn about proteins with unknown functions.

The CPASS software compares all possible orientations of the 3-D structure of the new ligand binding site with the entire database. But with only an in-house 16 CPU cluster at their disposal, each new site search took approximately 24 hours to complete. Over the last few years, the CPASS database had grown by some 40 percent.

To ramp things up a bit, Powers' group worked hand in hand with computing experts from the University of Nebraska-Lincoln's Holland Computing Center to help redesign the back end of database's interface so that users could submit jobs to the Open Science Grid via Glidein, a grid computing solution that is part of the Condor grid project. Condor is a grid paradigm that harnesses unused computer cycles in a network to build a computer grid.

Typical jobs on the CPASS database are now completed in about an hour.

Personalized Medicine and IT

We caught up with Sandy Aronson, executive director of IT of the Partners HealthCare Center for Personalized Genetic Medicine, during the Personalized Medicine Conference in Boston, MA. Aronson discussed IT challenges associated with the $1,000 genome, the GeneInsight Suite and GIGPAD software solutions developed by his lab to facilitate the use of genetic information in the clinic, and more.

Open-Source Solution for Big Data

By Matthew Dublin

Earlier this month at SC11, Clemson University's Walt Ligon presented an overview of OrangeFS, an open source file system for big data. OrangeFS is a highly scalable parallel file system the was originally developed from the Parallel Virtual File System, a file architecture designed to scale to petabytes of storage and data access rates at 100s of GB per second.

Ligon says that OrangeFS is relatively easy to install, use, and maintain because it's a "user level" file system which means that the interface is highly abstracted from the Linux kernel.

GPU Supercomputers Green and Powerful

By Matthew Dublin

Last week, the latest edition of the Green500 list was released, and the verdict seems to be that heterogeneous supercomputers which employ both CPUs and GPUs are the most energy efficient HPC systems out there.

The Green500 ranks supercomputers according to power efficiency — unlike the Top500, which is only concerned with raw computing power. And these supercomputers are not just eco-friendly but capable of serious processing power as well.

The second most powerful system on the Green500 list is the Tsubame 2.0 supercomputer at the Tokyo Institute of Technology, which has been clocked at 1.192 sustained petaflops of LINPACK performance using a mere 1,398.61 Kilowatts of power. Three other GPU-based supercomputers also made the top 10 in terms of computing power on the Green500 — The National Center for Supercomputing Applications (NCSA), Georgia Institute of Technology, and the National Institute for Environmental Studies in Japan ranked #3, #9, and #10 respectively.

The top five most power efficient systems in the world have remained for the last few editions of the list IBM Blue Gene/Q solutions at various locations.
The Green500 list is the brainchild of Virginia Tech's Wu-Chun Feng, an associate professor who also helped developed mpiBLAST, the open-source implementation of NCBI BLAST for parallel computer systems.

I recently had the chance to speak with Feng at last week's SC11 conference in Seattle, WA. A few years ago, Feng initially experienced a significant amount of backlash and vociferous dismissal in the HPC community when he first introduced the concept of power efficient supercomputing. HPC has traditionally always focused on processing power, at all costs, and never concerned with power consumption or the environment. But Feng says that the Green500 list and power-aware HPC have experienced increasing acceptance, and GPU technology development is going to play an increasing role in making supercomputing green moving forward.

Here's a video of Feng discussing the Green500 and GPU systems:

A Billionaire's Bandwidth for Cancer

By Matthew Dublin

Gigaom has a post describing the lofty bandwidth dreams of billionaire physician and biotech entrepreneur Patrick Soon-Shiong, whose recently established Chan Soon-Shiong Institute for Advanced Health is aiming to build a super high-speed Internet connection for cancer research and treatment. The CSS Institute for Advanced Health is investing hundreds of millions of dollars into a nationally distributed computing network, with a focus on the National LambdaRail network /a>, which is comprised of over 12,000 miles of fiberoptics capable of 100 gigabits-per-second.

Cisco has also joined the efforts and will help build out the NLR's 100 Gbps capacity.

To achieve their goal, the institute has partnered with IO Data Centers, and will use IO's Phoenix data center serving as the primary hub for their operations. The data center houses a prototype sequencing appliance comprised of hundreds of high-end multicore Intel processors running on HP servers capable of running 50 gene-sequencing workloads a day.

Flops Fever for iPhone/iPad

By Matthew Dublin

Who said flops can't be fun? Probably the same number of people who said they can be fun. Regardless, if you're feeling bored at work, head over to the Apple app store and download Flops Fever to your iPhone or iPad. Flops Fever is a game developed by the National Center for Supercomputing Applications that challenges players to schedule jobs so that all of the nodes on a supercomputer are kept busy. If you're too slow and the supercomputer goes idle, the job queue gets clogged, and you lose.

Amazon Supersizes Cloud

By Matthew Dublin

Amazon Web Services has announced the release of a super-sized version of its cloud service called Cluster Compute Eight Extra Large, or 'CC2' for short. Each instance has two Intel Xeon processors, each with eight hardware cores, and comes with 60.5 GB of RAM and 3.37 TB of storage.

Currently, customers can expect to pay $2.40 per hour for one on-demand instance that comes loaded with either Linux or Windows Server 2008 R2. There is also a reserved instance offering priced at $4,146 plus $0.054 per hour through a one-year contract or users can bid for time for a lower price tag on the EC2 Spot Market, a sort of cloud computing auction block.

In order to show off their new extra larger offering, AWS put together a cluster on the fly that used 17,024 cores and managed to claim the number 42 spot on the latest version of the Top500 list released this week.

ORNL's Jack Wells on the Titan Supercomputer

By Matthew Dublin

The Exascale Report has an interview with Jack Wells, director of science for the National Center for Computational Sciences at the Oak Ridge National Laboratory, where he discusses Titan — a heterogenous supercomputer slated to come online next year that will be capable of 20 petaflops performance.

Titan is actually ORNL's Jaguar supercomputer but with a makeover thanks to technology development from Cray which, along with the addition of GPU accelerators, will grow the system to contain over 38,400 processors housed in some 200 cabinets.

According to Wells, there are already software development projects in the works that will attempt to fully take advantage of Titan's computational power. These applications include LAAMPS (Los Alamos Molecular Dynamics code), authored by colleagues at Los Alamos, that can be used to simulate lignocellulose, as well as many other material science and physics software.

Whether or not software developers will be able to spread out commonly used bioinformatics code across the Titan system and really get their money's worth remains to be seen. But now that the system will be a virtual warehouse of GPUs, molecular modeling code might really see some massive acceleration and coding innovation.

If you would like to log into Jaguar for some terascale research, click here.