Changes to the clinical trial reporting system have led to an increase in trials reporting null results, according to an analysis from Department of Health and Human Services' Robert Kaplan and Oregon State University's Veronica Irvin.
As they report in PLOS One this week, Kaplan and Irvin sorted through large randomized controlled trials supported by the National Heart Lung, and Blood Institute between 1972 and 2012. Of the 30 studies that met their criteria that were published before 2000, 17 reported that the intervention studied had significant benefit to the primary outcome, while only two of the 25 post-2000 trials did.
New reporting standards, Kaplan and Irvin note, were phased in around the year 2000 that required researchers to register their trials at ClinicalTrials.gov before beginning data collection and state what their projected outcomes were. These requirements, Irwin says in a statement, made it less likely that researchers would change their analysis plans.
Prior to that, "it was actually pretty common for people to just measure a lot of things" and then inspect the data, Kaplan tells the Chronicle of Higher Education. "If you measure 20 things, one of them is going to be statistically significant by chance."