The Crash Comes After the Flash

A columnist writes that science "has lost its way" in the push to publish flashy articles in top journals.

Full-text access for registered users only. Existing users login here.
New to GenomeWeb? Register here quickly for free access.

Both The Economist and MIT

Both The Economist and MIT Tech Review picked up on this topic this week as well. While the cited article notes losses in the billions of dollars, that's probably a very low estimate: the pharma industry R&D collapse is readily traced to flaws originating from academic research.

And DO NOT forget - many of the reports that could not be repeated are attempts to connect genomics to disease, drug development and personalized medicine. Didn't work, won't work, can't work.

So much for a glowing return from the Human Genome Project.

I disagree in part with

I disagree in part with rayperkins. Connecting genomics to disease allows researchers to identify a pool of participants for various studies. The results of those studies will then do more good for those screened for, but not yet afflicted by, various diseases as drugs can then be developed that will pinpoint treatments for that specific group. Personalized medicine, i.e., individualized treatment, is foolish because of the uniqueness of all individuals' genomes, but "genetically-categorized " medicine is expanding our basic knowledge of disease processes.

I disagree in part with both

I disagree in part with both previous commenters. there are many many examples of well replicated findings. There are certainly biases, and they are stronger for projects which are either 1) high profile, or 2) very expensive, or both. No one wants to say that a very expensive experiment yielded only negative findings, no one wants to spend money on trivial replications, and there is certainly too much reward given to flashy findings before they are well validated. But to be a bit more realistic, biological research is notoriously difficult because living systems are hugely complex and highly non-linear. Such systems are often very sensitive to local conditions, which easily vary a lot between laboratories. So a certain amount of non-reproducibility does not necessarily indicate false findings, but rather non-usable findings which is not the same thing. As for personalized medicine, to the extent that this refers particularly to genomics and especially genetics, then as with other aspects of genetics there are some very strong results that have measurable impact on patient care, and other results that are more like statistical averages for which individual treatment is not necessarily relevant. It is way too negative to say that personalized medicine simply does not work. It is overhyped for sure, but working as I do in pediatrics we have lots of positive counter-examples.

I totally agree with

I totally agree with columnist. Please, how many times were we told that the genome would revolutionalize medicine and lead to cures for Parkinson and Alzheimers....Let's be honest, the genome project was an incredible scientific endeavor but has not delivered for people suffering with chronic disease. Only a small minority of people with these diseases have chromosomal abnormalities or distinct genetic profiles in terms of causation. I think much of this money would be so much better spent by studying the relationship between genes and ubitquitous neurotoxic triggers in our environment.

Oh god, please, let it go

Oh god, please, let it go man. When was the last time you were at intensive care, its sad to see how much billions we throw at genomics for the big wigs of nasdaq. There is no prevention being done zero. Its scientifically criminal.

My solution would be: Title:

My solution would be:

Title: the new way of publishing
Authors: ben, jim, and jerry
Validation authors: pedro, marta, alexandra

You could use a certain number of validation papers as part of your phd training?

Suppose we have 10 theories

Suppose we have 10 theories that all make good sense (and we should have at least that many for any subject). Which one is correct? Are any of them correct? Nobody knows. Scientists may favor one theory over another (due to experimental findings which support yet never confirm), and be absolutely positively convinced that it's correct; but that's irrelevant because we still don't know the answer. Truth isn't determined by a vote, even a vote of all the experts!

A problem is that we build upon these theories (we must make progress!), implying that these foundational theories are correct. Over the years our structures get taller and taller. We no longer question the lower levels, as time passes they magically transform from theory to dogma.

It's a mistake to believe that these flimsy scientific structures we create are actually sturdy and permanent. One day this house of cards we've built on top of a sand castle will topple over (and we won't be able to hide it with smooth talk and fun-house mirrors). Good luck getting funding to do science after that.

While all comments given here

While all comments given here are valid, I think they are missing the main point of the original post. If I understood correctly, the main point is why so many high profile studies published in high impact journals are not delivering their promise or are outright inaccurate. Of course, while other explanation such as increased exchange of information will definitely lead to increase detection of such problems compared to previous years, I believe this problem stems from the fact that in the last two decades science has become more of a business venture and much less of science. Like with any other business, the most successful scientists now are considered these with highest number of publications and most grant dollars. Since # of published papers and grant dollars amount are highly correlated, in general getting more papers ensures getting more grants, which in turns ensures getting more papers and the vicious circle is closed. So until this circle is broken, this problem of reporting uncorroborated studies with questionable results will continue to persist and likely is going to get only bigger and bigger.

There is a tendency to

There is a tendency to dismiss irreproducibility for experimental setup differences. This is in part true: what happens in Caco2's wont necessarily occur in HeLa's nor primarily lung epithelia, let alone a whole tissue. Similarly, the inner London population genomics will not reflect inner Berlin. However, those institutionally reviewing this kind of discrepancy ought not to dismiss criticism on reproducibility as mere happenstance.

Senior pharma execs have publicly stated they have been unable to reproduce results in the same academic lab, by the same post-doc, when this is attempted. When I was at Pfizer we had our fair share of 'bullshit' science to wade through, but another colleague tallied over 700 IF>10 publications they were unable to reproduce.

Publish or perish has become excel or perish, and this includes patents. Scientists have become marketeers to keep funding, to excel, and thus overstate and overinterpret. It is broken alright, but its not terminally broken.