NEW YORK (GenomeWeb) – The US Food and Drug Administration has launched the first community challenge for PrecisionFDA — a community-based FDA-led initiative to put in place a cloud-based infrastructure that enables next-generation sequence-based test developers, bioinformaticians, regulatory bodies, and other stakeholders to share genomic datasets, tools and generate reference standards.
The so-called Consistency Challenge was briefly discussed during a White House event earlier today that highlighted commitments from roughly 40 government agencies, academic institutions, and companies to support President Barack Obama's Precision Medicine Initiative.
According to the challenge site, the FDA is providing FastQ files from the Garvan Institute of Medical Research and Human Longevity, containing two datasets of whole genome sequences of the NA12878 human sample at two different sites. Participants are expected to use their informatics pipelines to process these datasets from mapping to variant calling, and then rerun their pipelines on one of the datasets to get a second vcf file for the same input sample.
They'll then have to perform five pairwise comparisons of the data, including comparing the vcf files of the two datasets to evaluate the reproducibility of their pipelines on the same sample across different sites; and comparing the three vcf files to the Genome in a Bottle benchmark vcf. Winners will be selected for eight categories in terms of the reproducibility and accuracy of their comparisons. There's also an extra credit task which challenges participants to provide mechanisms of assessing reproducibility across more than two repeated experiments and for comparing multiple vcf files.
Challenge participants have the option to run their pipeline on their own systems or on the PrecisionFDA infrastructure. The FDA has provided instructions for constructing pipelines on the system here.
The challenge is slated to run from February 25 to April 25. Additional details about the challenge and how to participate are provided on the PrecisionFDA site.