The Magic of a P Value

Recent studies indicate issues with the statistical cutoff p value of 0.05.

Full-text access for registered users only. Existing users login here.
New to GenomeWeb? Register here quickly for free access.

This insight about

This insight about psychological research results is old news to graduate students trained at the University of Minnesota where many generations have been exposed to the preachings and teachings of the late Paul E. Meehl and David T. Lykken.

from above: "He reports that

from above: "He reports that p values of 0.05 or less "represent only moderate evidence against null hypotheses." Further, he calculates that some 17 percent to 25 percent of marginally significant findings are false."

Well, a p-value of 0.05 means that degree of evidence (e.g., that large a t-test score) would be expected to occur just by random chance 20% of the time -- so no surprise.

"He suggests that scientist adopt a more stringent cutoff for p values at 0.005 or even 0.001."

Of course, if one dials up the threshold, one gets fewer false positives, but then also fails to reject the null hypothesis more often (when one should have) -- and one also needs to keep in mind "effect size", i.e., that statistical significance is not necessarily the same as biological significance.

my understanding is that a

my understanding is that a p-value of 0.05 means the statistics as significant or more so should occur by chance, 5% of the time. I'm not sure how one gets the 20% number... so if 17 to 25 percent of marginally significant findings are false, it means the statistics is not being applied properly or that there's reporting bias in the literature.

I think the '20%' is a bit of

I think the '20%' is a bit of a typo - 5% of occurrences is a 1 in 20 chance.

I agree that 'more stringent is more better', but a consequence of setting a lower threshold is that a greater number of replicates must be analysed in order for an experiment to achieve a satisfactory statistical power. This is historically something that has been difficult to convince all researchers to do...

Hence the adoption of p less than 0.05 as a 'compromise position'.