Cleaning up Messy Data with Google Refine

February 06, 2012

Type size: - +
Login or register to post comments
Email
Printer-friendly version
RSS Feed

Rod Page over at iPhylo has a post describing how useful Google Refine is for cleaning up taxonomic databases. Google Refine, formerly known as Freebase Gridworks, is a freely available web-based "power tool" that supports TSV, CSV, Excel, and XML file formats. Among other features, Google Refine allows users to pull together disparate data sets and work with the data in a collated, polished fashion.

Page, a professor of evolutionary biology at the University of Glasgow, is a big fan of Google Refine's "Reconciliation Services," which he uses for matching names to external identifiers.

So far, Page has used Google Refine with EOL, NCBI taxonomy, uBio , WORMS, and GBIF.

Here's an introduction to Google Refine:

Most Viewed
Most Emailed
Blog

The Daily Scan
No Charges for Student
The Daily Scan
Blunt Cuts
The Daily Scan
Into the 'Henhouse'
Careers
Not the Intended Effect?
The Daily Scan
In the Face of Competition

GenomeWebinars

Advances in Single-Cell Genomics: Live Cell RNA and Circulating miRNA Detection

Sponsor: EMD Millipore

Data presented in this webinar illustrates the value of live cell analysis at the single-cell level to identify differences in expression levels across populations of cells. The cells remain intact for downstream analysis. Our experts also discuss the use of SmartFlare RNA detection probes for the direct quantification of circulating miRNAs with rapid processing of blood plasma/serum, which is done without the use of enzymes. Using circulating miRNAs with established roles in cancer and quality control, we can accurately detect these miRNAs in plasma using a microplate fluorometer within an hour after plasma preparation.

On-demand recording is available here.

Optimization of NGS Library Preparation: Low Inputs and Fast, Streamlined Workflows

Sponsor: New England Biolabs

Library preparation methods continue to be challenged by the requirement for faster and more efficient protocols, using lower input amounts. In this online seminar, recorded Feb. 7, 2013, experts discuss new approaches to tackle these challenges, particularly for bacterial and exome sequencing.

Available here.
Young Investigator Profile

Sandra Pankow

Research Associate
Scripps Research Institute

The Cystic Fibrosis Interactome

Sandra Pankow is currently a research associate in John Yates' lab at Scripps Research Institute, studying protein-protein interactions involved in cystic fibrosis.

"I'm interested in trying to understand genetic diseases from a systems biology point of view," she says. "I've found that proteomics is the ideal tool for this because it gives the researcher the possibility to investigate thousands of proteins at the same time rather than just having to guess which one or two you should look at."

During her time in the Yates lab, Pankow has worked on developing a new technique for large-scale profiling of protein-protein interactions called co-purifying protein identification technology, or CoPIT. The method uses co-purification of proteins followed by mass spectrometry analysis and bioinformatic processing to identify not just primary protein-protein interactions, but second-degree and third-degree interactors as well.

With it, Pankow says, "we've reached about 30-fold to 100-fold higher sensitivity than previous methods. We're now able to profile an interactome to near completeness."
Blog

Blunt Cuts

Francis Collins discussed the sequester on Hardball with Chris Matthews.

May 21, 2013

Basic Building Blocks

Synthetic Genomics and ExxonMobil have new agreement for developing algal biofuels.

May 21, 2013

Into the 'Henhouse'

Researchers in New York charged with accepting bribes.

May 21, 2013

This Week in PNAS

In PNAS this week: microRNA signature of chemoresistance, barley powdery mildew genome, and more.

May 21, 2013
Papers of Note

High-resolution transcriptome maps reveal strain-specific regulatory features of multiple Campylobacter jejuni isolates
Dugar, Herbig, et al. PLOS Genetics

The University of Würzburg's Cynthia Sharma and colleagues undertook a transcriptomics-based analysis of the gastroenteritis-causing bacterial species Campylobacter jejuni. The team used its so-called differential RNA sequencing strategy to sequence and compare the transcriptomes of four C. jejuni isolates (three from humans and one from a chicken), applying a new method to automatically annotate transcription start sites in each. "Overall," they write, "our study provides new insights into strain-specific transcriptome organization and [small RNAs], and reveals genes that could modulate phenotypic variation among strains despite high conservation at the DNA level."

Mosaic genome structure of the barley powdery mildew pathogen and conservation of transcriptional programs in divergent hosts
Hacquard, Kracher, et al. Proceedings of the National Academy of Sciences

The barley powdery mildew (Blumeria graminis f. sp. hordei) pathogen genome is comprised of chunks of sequence that are particularly rich or replete in polymorphisms, according to a study by researchers from the Max Planck Institute for Plant Breeding Research. The team sequenced the genomes of two Bgh isolates from Europe, comparing each to the barley powdery mildew reference genome. The newly sequenced isolates each contained distinct combinations of sequence blocks with high or low SNP concentrations — isolate-specific mosaic genomes that point to "exceptionally large standing genetic variation in the Bgh population," study authors say. Meanwhile, their transcriptome sequencing experiments offered a look at genes used by Bgh during attempted infiltration of barley or immunocompromised Arabidopsis.
People on the Move

Kevin Hrusovsky is resigning his post at PerkinElmer as senior VP and president of the Life Science and Technology division. Hrusovsky will serve as a consultant to the company for up to one year, beginning in June. He joined PerkinElmer through the company's acquisition of Caliper Life Sciences, where he was CEO and president.

Hologic has appointed former Beckman Coulter head Scott Garrett to its board of directors, where he will serve on the corporate development committee.
Garrett currently is an operating partner with Water Street Healthcare Partners, a private equity firm. Garrett spent 10 years at Beckman Coulter, where he was chairman, president, and CEO.

Gina Costa is now senior director of genomic applications at Illumina. She joins Illumina from Life Technologies, where she was senior director of genetic analysis, working on development of the Ion Torrent and SOLiD sequencing technologies. She has also held positions at Agencourt Bioscience and Roche's 454 Life Sciences.

Bioinformatics firm Golden Helix has hired Andreas Scherer to be its new president and CEO. Scherer has managed large global software services businesses, and he started his executive career at AOL/Netscape. He will replace Former CEO Christophe Lambert, who will take on the new role of company chairman.
Upcoming Events

Conferences, Meetings & Deadlines

National Advisory Council for Human Genome Research meeting
May 20-21 / Rockville, Md.
National Advisory Council for Human Genome Research

2013 ASCO Annual Meeting: Building Bridges to Conquer Cancer
May 31-Jun 4 / Chicago
American Society of Clinical Oncology

European Human Genetics Conference 2013
Jun 8-11 / Paris
European Society of Human Genetics

ASMS Conference on Mass Spectrometry
Jun 9-13 / Minneapolis
American Society for Mass Spectrometry

International Meeting on Cell-Free DNA
Jun 20-21 / Copenhagen
Copenhagen University Hospital

Abstract & Registration Deadlines

more

Science
An international team has sequenced the genome of the carnivorous bladderwort plant, Utricularia gibba. Their findings suggest that the carnivorous plant has ditched virtually all its non-coding DNA, retaining a set of sequences that's almost exclusively genic. "What that says is that you can have a perfectly good multicellular plant with lots of different cells, organs, tissue types and flowers, and you can do it without the ['junk' DNA]," said co-corresponding author Victor Albert.
Business
Agilent Technologies announced a restructuring program expected to reduce its headcount by about 450 employees and save the company $50 million annually in operating expenses. CEO Bill Sullivan said that the focus of the restructuring will be on Agilent's Electronic Measurement Group and that the company will explore opportunities "to streamline our organization around the world." The firm also announced that its Q2 revenues were flat year over year.
Funding
The US Department of Energy's Joint Genome Institute has funded six new initiatives to develop technologies that will help JGI and its users conduct their research efforts into microbiology, metagenomics, and plant genomics. The projects will be supported under the Emerging Technologies Opportunity Program with a total of around $3.5 million over the next two years. Among the researchers receiving funding are Stephen Quake and Jay Shendure.
GenomeWebinars
GenomeWebinar: Advances in Single-Cell Genomics: Live Cell RNA and Circulating miRNA Detection

Sponsor: EMD Millipore

GenomeWeb and EMD Millipore invite you to view an archived webinar discussing new approaches to detect RNA at the single-cell level as well as new probes for the direct quantification of circulating miRNAs. In this free online seminar, recorded April 25, 2013, our expert panel shares protocols for improved RNA and miRNA detection.

On-demand recording available here.

Search

Cleaning up Messy Data with Google Refine

GenomeWebinars

Young Investigator Profile

Blog

Papers of Note

People on the Move

Upcoming Events

Science

Business

Funding

GenomeWebinars

TechGuides Listings

Poll