The MicroArray Quality Control Consortium is grappling with a number of technical issues related to the clinical use of microarrays in hopes of fulfilling the technology’s promise for personalized medicine.
During MAQC’s seventh face-to-face project meeting, held May 24-25 on the campus of the SAS Institute in Cary, NC, consortium participants presented some initial results of the second phase of the project, called MAQC-II.
While MAQC-I evaluated the reproducibility of microarray experiments in basic research, MAQC-II is focusing on the use of arrays in clinical trials, diagnostics and prognostics, safety testing, and other applications that are likely to be the subject of regulatory oversight.
Specifically, the initiative is addressing the challenges of developing and confirming predictive models that use gene-expression profiles to predict disease recurrence, prognosis, response to treatment, and other outcomes of clinical interest. Participating groups are independently analyzing several clinical and toxicogenomics data sets with the goal of identifying “best practices” for developing classifiers that are reliable enough for use in a clinical setting.
Talks at the meeting centered around a number of challenges that the MAQC participants hope to address over the course of the study, including a lack of quality-control standards for array-based clinical experiments and a host of open questions surrounding the best statistical approach for developing and confirming gene signatures used as diagnostic or prognostic markers.
The consortium also launched a new workgroup that will identify best practices for genome-wide association studies — an area of research that is quickly moving into the regulatory realm via the FDA’s Voluntary Exploratory Data Submissions program.
Federico Goodsaid, senior staff scientist in the genomics group at FDA’s Office of Clinical Pharmacology, said in a presentation that the agency has been getting “a number” of GWAS data sets under its VXDS guidelines, but noted that FDA examiners are having a hard time reproducing sponsors’ results.
“How many ways can we not match what the sponsor did?” he asked, explaining that GWAS experiments have “enormous sources of variability at each analysis step,” yet there is currently “no framework to tell us why [submitters] are doing this or that.”
Much like gene-expression studies, Goodsaid noted, there are many unanswered questions regarding the best methods for quality control, normalization, analysis, and biological interpretation for whole-genome studies. Given MAQC’s experience with the former, he and others suggested that the consortium would be well equipped to tackle the latter.
Nick Xiao of SAIC Frederick, agreed. He said that “MAQC has successfully proven that microarray technology can be used for biomarker discovery,” and noted that the group can apply many of the lessons learned from MAQC-I to show that “genotyping technology can be just as trusted and just as robust.”
Goodsaid and Xiao will coordinate the new working group, which plans to identify experts in academia, industry, and government who will be willing to assist with the project. The longer-term goal will be identifying best practices for analyzing GWAS data.
“This will make an enormous difference to what we will be able to do with this data at the FDA,” Goodsaid said.
Leming Shi, a researcher at the US Food and Drug Administration’s National Center for Toxicological Research and coordinator of the MAQC initiative, noted that GWAS experiments present several “new challenges” for the microarray community, “but a lot of similarity with what we saw in gene expression.”
Ultimately, he said that the objectives of the new working group align with the broader goal of MAQC-II, which is to predict health outcomes based on microarray measurements of biological samples.
“For personalized medicine to be realized, we have to be able to make a prediction for each patient,” he said.
NIH Also Tackles GWAS Guidelines
The FDA isn’t the only federal agency struggling with the data-analysis challenges of genome-wide association studies. Teri Manolio, director of the office of population genomics at the National Human Genome Research Institute, said during a presentation at the meeting that there is a “need for consensus on what constitutes replication in association studies.”
As the cost of genotyping continues to fall, the research community is bracing for an “avalanche” of GWAS studies, Manolio said. However, she noted, the “majority” of positive associations in these studies are actually false positives, and the likelihood of a single study establishing an association is low, so it will be crucial for the community to establish better guidelines for replicating experimental results.
“For personalized medicine to be realized, we have to be able to make a prediction for each patient.”
Manolio said that NHGRI and the National Cancer Institute have formed a working group to examine the issue of replication in association studies, and that the group currently has a paper in press at Nature that outlines some recommendations.
One recommendation is that GWAS experiments have a large enough sample size “to distinguish the proposed effect from no effect convincingly,” Manolio said. In addition, she said, follow-on studies must examine the same or similar trait as the original study in the same or a very similar population.
Most importantly, she said, NCI and NHGRI recommend that the authors of GWAS papers describe all the experimental parameters of the study in sufficient detail for future groups to replicate it.
Breast Cancer Tests Under Examination
Fraser Symmans, associate professor of pathology at the MD Anderson Cancer Center, raised another issue related to the use of microarrays in personalized medicine, though one that is not yet being addressed by the MAQC.
Symmans proposed an effort to “establish the reproducibility of microarray-based tests for breast cancer” across different labs, citing the rapid development of commercial array-based breast cancer prognostics from companies like Agendia and Genomic Health.
While both of those firms run their tests in their own labs, Symmans noted that the community will eventually need to ensure reproducibility of array-based tests in multiple reference labs.
And even though MAQC-I proved that microarrays are reproducible for research use, Symmans said those findings are not quite good enough for the clinic.
Symmans proposed a study in which RNA would be profiled in several different reference labs using both the Agilent and Affymetrix platforms in order to establish the reproducibility of microarray-based tests for breast cancer between different laboratories evaluating the same tumor sample and to “establish reproducibility between cytologic samples and tissue samples from the same tumor.”
Symmans noted that he has already received “verbal commitments” from eight medical centers to collect samples with patient consent. He proposed that all the data from the reference labs be provided to a centralized data-analysis center that will evaluate the concordance of the results.