First published in 2009 in the journal Clinical Chemistry, the Minimum Information for Publication of Quantitative Real-Time PCR Experiments guidelines — better known as the MIQE guidelines — were designed to provide researchers with a roadmap for improving the quality and reliability of their qPCR data.
Since that time, the molecular biology research community has slowly adopted the guidelines, and many qPCR instrument and reagent vendors have done their part to help encourage their customers to follow MIQE protocols.
There is still work to be done, however, and some vendors have taken a more active role than others in disseminating information about MIQE to their customers. To wit, Bio-Rad earlier this month published a case study on its website demonstrating how neglecting some of the key steps in the MIQE guidelines can lead to flawed data and erroneous conclusions.
In the study, researchers from Bio-Rad and the Jewish General Hospital at McGill University studied the effect of RNA sample quality and reference gene stability on gene expression data obtained using qPCR.
More specifically, they used the minichromosome maintenance protein MCM7 as a model target gene to investigate the importance of appropriate reference gene selection. They also varied RNA sample quality from their breast cancer samples to determine its effect on data.
Following the MIQE guidelines, they observed a significant increase in gene expression of MCM7 between normal and tumor samples when using high-quality and high-purity RNA with normalization using stable reference genes. However, they obtained inconclusive and even opposite results when using poor-quality RNA samples and unstable reference genes.
Last week, PCR Insider spoke about the case study with its lead author, Sean Taylor, a field application specialist at Bio-Rad, and Francisco Bizouarn, an international field application scientist in Bio-Rad's gene expression division. Following is an edited transcript of the interview.
When did Bio-Rad begin adopting the MIQE guidelines, and training staff and disseminating the guidelines to customers?
Francisco Bizouarn: Back in 1999 — even before the MIQE guidelines came into existence — when we launched our first real-time instrument, the I-Cycler IQ, we were basically the third major company with a real-time [PCR] platform. And we realized that, to make this more accessible to the community at large, [we needed] to educate the scientific community as to how to design qPCR assays from beginning to end.
In 2000 [Bio-Rad hired] a group of researchers, including myself, and our goal was to basically help disseminate this information on how to properly design, prepare, and run qPCR assays. So we started promoting good practices a long time ago. With the advent of the MIQE guidelines, we sort of felt vindicated. We were the group that said, "You have to do things right and follow a certain procedure. You should focus on these different parameters." And we weren't always perceived as the easiest option. But when the MIQE guidelines came about, we felt vindicated.
Why did Bio-Rad feel it was important to conduct this particular case study? Have you done case studies like this in the past?
Sean Taylor: This particular case study is sort of a sequel to a previous case study we did ["A practical approach to RT-qPCR — Publishing data that conform to the MIQE guidelines"], which was published in Methods in 2010.
Soon after the MIQE guidelines were published, we put together this additional resource for our global client base. So we were really pushing the MIQE guidelines at that point. The story behind the case study paper is that I work in the same genre as [Francisco], where I go out and train scientists how to follow best practices for qPCR. I was giving a talk at a McGill University's [Jewish General Hospital] in Montreal … [about] this previously published Methods paper. And one of the researchers at the hospital, after hearing the talk, became very concerned about the way they were conducting qPCR experiments in the lab. They wanted more detail about how to conduct experiments in a more rigorous way … and I agreed to help them on the condition that they would allow us to use some of the data that we would produce in this case study. And they agreed. Basically it went from there. Of course, we provided them with the instrumentation and the reagents to be able to do all this and produce some really nice comparative data on how good results can be if you follow the guidelines, and how poor they can be if you don't.
Regarding that data, it appeared that reference gene selection had a great impact on qPCR data quality, but the sample quality didn't seem to have as much of an impact. Is that correct?
ST: That's true, and has been previously published by other authors — that sample quality may not have as much of an impact on reference gene selection. One of the key mistakes that researchers make when they conduct qPCR experiments is reference gene selection. A lot of researchers choose the usual suspects — essentially GAPDH, beta-actin, and 18S [ribosomal] RNA. Since the change in reference gene expression is directly incorporated into the calculations, reference gene stability between samples will have a direct impact on the final reported values for the normalized fold differences in target gene expression. Therefore if a reference gene with poor stability is chosen to normalize the expression of a target with little or no relative fold expression differences between samples, the reported difference will be an artifact of the change in reference gene expression. We call those reference genes "the usual suspects," and most labs, even today, are taught to do qPCR in a certain way … and never think about going through the literature and finding guidelines — or they just weren't available at the time. And if GAPBH is grossly influenced by my experimental conditions, my results are not going to be due to my gene of interest — they're going to be because of GAPBH expression.
FB: One more thing about some of the commonly used reference genes — some of them are legacy assays from before qPCR, when researchers used to run blocks. You wanted to have a high expresser so you could easily image your blot at a short exposure and determine different input amounts … in whatever columns you had run on your gel. So many of these are not ideal for real gene expression analyses using qPCR.
At Cambridge Healthtech Institute's qPCR for Molecular Diagnostics conference earlier this month in San Francisco, one session and group discussion focused on the inherent difficulty in selecting ideal reference genes for qPCR experiments. What is the nature of this difficulty, and what can be done to address this?
FB: One of the difficulties is that the reference genes that should be used technically have to be the most stably expressed genes in your assay, so you can simply compare variations as far as sample acquisition, sample preparation, and sample processing into cDNA. Depending on the different tissues, cell types, or sources of your RNA, these reference genes will vary, because there are some biases in the degradation patterns of RNA.
So a set of reference genes that can be used in experiment A with, say, liver tissue, may not work if the sample is from muscle tissue, or brain tissue, or some other cell line. Determining the stable expressed genes tends to be something a lot of researchers are hesitant to do because they don't know what to start off with. This is unfortunate because there are lists of commonly used reference genes that can be ordered directly from companies … and all you have to do is take a couple of your control samples, and a couple of your treated or disease samples, and run a very small pilot experiment that can be done in half a day. From then on you can select your reference genes and stay with that for the duration of your experiment. And if a new experiment comes around with a completely different set of parameters, then you can still use this panel of reference genes to do a second screening for the second experiment.
Unfortunately it comes down to a lack of knowledge that these panels are already available out there. And a lot of that stuff is free access.
ST: There are also a couple of software packages out there to help people pick the most stable reference genes after they've tested them, and those are [Aarhus University's] NormFinder and [Biogazelle's] geNorm.
How much can a tech vendor such as Bio-Rad do to ensure that users are complying with MIQE guidelines in their experiments? In the end, isn't it something that the user has to decide to implement?
FB: Yes. We have very little, if any, control over how researchers choose to do their science. It really is up to them and up to the publishing groups, the reviewers, and to the funding agencies to make sure that the data is presented as correct.
As a company, we can provide information on best practices … so researchers can attain the goals that they want. But we can only recommend.
ST: I agree, and the only other thing that we can do is to publish things like this case study and guidelines. Also, as Frank mentioned earlier, we have dedicated and trained staff that actually go and teach customers how to perform experiments according to these guidelines. At least from what I've seen, this really makes customers interested in doing their experiments in an appropriate manner to generate … good quality data that can be reproduced by the rest of the scientific community.
FB: One of the things that we have done to try and promote good practices is participate in various technical seminar series all over the world. We did about 25 different venues in the US; maybe 20-plus venues in Europe; and also [seminars] in Asia-Pacific, India. We invite speakers who are sometimes the co-authors of the MIQE guidelines and present not just the guidelines, but technical strategies for optimizing and validating and complying with the guidelines.
Is it working? What is your sense in speaking with customers on a regular basis as to whether these guidelines are being widely adopted? It seems like many of the same proponents of MIQE are presenting the guidelines at every conference like they are brand new, yet they've been around for several years now.
ST: I'm based in Eastern Canada, but I deal with a large number of scientists across the country, and from what I'm seeing … the scientists that I work with who are reviewers on papers are definitely using the guidelines as a means of reviewing papers. And I'm working directly with scientists who are even writing papers as we speak on their own experiences with their data, and how it changes if they do or don't apply the MIQE guidelines.
So I definitely think [more] people are adopting the guidelines. I think the reason why we're working toward getting the whole scientific community moving in that direction is simply because of the massive number of scientists that have been using qPCR for years and years before the guidelines came out, and they think they're doing it right. They're applying techniques they learned 10 or 20 years ago that seem to give them data that they thought was OK. But as we encounter those groups, we open their eyes to how the data is affected by different aspects of these guidelines — particularly reference gene expression and, to a lesser extent, sample quality — and it really changes the game for them. Bio-Rad is definitely doing its part in this, and I see it happening with a lot of the other biotech companies, as well.
FB: Absolutely, it's more of a legacy issue where researchers have been doing things a certain way for a certain amount of time, and until somebody points out the fact that some of their experiments may not be as accurate as they wish, or may not be accurate at all, they're not going to change, because it's the path of least resistance. If they can publish articles and submit their grants … and have an easy go at it running two samples, and off you go, then they will do it. Once reviewers become more knowledgeable in this technology, things are going to change. Until the appearance of MIQE guidelines, most reviewers assumed that if it was qPCR data, it was done correctly. Now, the crux is on the other side. A reviewer that is familiar with the MIQE guidelines will look at data and say "Well, I'm not sure it's done correctly." And they will check to see if certain parameters are present.
That said, there are still many reviewers out there who do not follow the MIQE guidelines and who do not request them. But slowly the scientific field is moving in that direction.
ST: And as more people become aware, they're pushing researchers to do more with respect to conforming. And the fact of the matter is that the whole premise of science moving forward is to be able to build on what was previously published. So we all, as scientists, need the data that's published to be accurate and reproducible. This is the whole premise of the guidelines and of science [in general]. When a scientist is presented with the MIQE guidelines, it usually makes sense. It's just a practical way of applying a rigorous methodology to qPCR experiments.