SILVER SPRING, Maryland (GenomeWeb) – The US Food and Drug Administration Thursday evening launched a closed beta of a community platform where participants can share next-generation sequencing datasets and benchmark bioinformatics approaches.
At a public workshop to gather input on NGS test standards, FDA Chief Health Informatics Officer Taha Kass-Hout described the so-called PrecisionFDA platform as a cloud-based environment where NGS test developers, bioinformaticians, standards developing bodies, and regulatory experts can collaboratively advance the regulatory science on genomic tests and software.
"Right now, this is a science tool to be messed around with, contributed to with notes and experiments," Elizabeth Mansfield, director of personalized medicine at FDA's Office of In Vitro Diagnostics and Radiological Health, said at the workshop. "It's not a regulatory tool right now … Right now, it's just for play."
Potential users of the closed beta need to request access through the PrecisionFDA site. According to a timeline on the site, the next beta launch is slated for Dec. 15. There is no cost to participate in PrecisionFDA, an effort the agency believes will be a critical part of implementing the government's Precision Medicine Initiative.
The agency has been working on the PrecisionFDA project for about a year. Over the summer, the FDA awarded DNAnexus an $849,000 research and development contract to develop a cloud-based platform for users to share genomic data and advance reference standards for NGS-based tests.
PrecisionFDA is "really about the community we're trying to build," Kass-Hout said, noting that "anything that comes from it" will be available on the open-source repository GitHub. For now, any group or individual with something to contribute to the effort, such as source code, tools, pipelines, data, or even user feedback, can request access to the closed beta platform. Kass-Hout believes that PrecisionFDA could potentially help groups who really want to enter the genomics field, but lack the computational resources.
He noted that initially the focus will be on enabling users to compare their data and pipelines against a reference set or against data from the PrecisionFDA community.
Within the beta version, participants can upload VCF files to compare variants between a benchmark and a test set. The comparison will yield information about true positive matches between the two variant sets, as well as false positives, false negatives, precision, sensitivity, and F-score. Users can also use a genome browser to visualize comparisons on false positives and false negatives. "This may be a way you can go back and further calibrate your pipeline," Kass-Hout said.
According to the FDA, the variant comparison feature could be used to gauge reproducibility, for example, by running an NGS test twice on the same specimen and seeing how much agreement there is between the two. With a reference sample, for example from Genome in a Bottle, the variant comparison feature could be handy for assessing the accuracy of an NGS test or a mapping and variant calling pipeline.
Beta version users will be able to explore and create apps for mapping, variation calling, or simulation pipelines. There is a section where users can share notes on how they used the platform's various features, as well as document their own procedures and experiments. Users can also create maps to visually track their pipeline processes and share that with their team or with the whole community.
The PrecisionFDA platform has public spaces where members can share their analysis and reference materials within the community, including the FDA. However, there will also be private spaces where members will still have access to shared materials, but can upload their own data and tools and limit who sees that information.
"Even though we're building a community, we are respecting this notion of privacy for each individual or organization," Kass-Hout said. He emphasized that no one without access can enter these private areas of the PrecisionFDA platform, not even the FDA.
"We can't actually see your stuff, if you don't let us see it," Mansfield said. "We'll be watching what happens over time and trying to determine whether we can make this into, at least in part, a regulatory tool for people to be able to deliver their performance metrics to us."
After launching the beta version in December, FDA plans to continue to scale the environment and roll out the platform publicly in March 2016. Kass-Hout said next year, FDA will host community challenges.
At the end of the presentation, Mansfield informally gauged interest in PrecisionFDA by asking for a show of hands at the packed NGS standards workshop. A significant portion of the audience indicated their willingness to partake in the effort.