Concerns about reproducibility in research have abounded for a few years now, and Wired reports that DARPA has an idea for how to rank studies' credibility using a combination of algorithms and human assessment.
In 2015, researchers from the Open Science Collaboration reported that they were only able to reproduce the results of about half the psychology studies they attempted to re-do. Similarly, researchers from Amgen have said they have had trouble reproducing a number of preclinical cancer and high-profile papers. Additionally, there's also an economic cost — a 2015 PLOS Biology paper estimated that about $28 billion a year is spent on preclinical research that's not reproducible.
According to Wired, a DARPA program called Systematizing Confidence in Open Research and Evidence (SCORE) wants to be able to give research findings credibility scores, and it has allotted $7.6 million to the Center for Open Science (COS) to try it out on social science research findings. COS will first develop a database of 30,000 claims and then, for 3,000 of those claims, try to replicate the findings or subject them to human-run prediction market, it adds. And then those results will be analyzed to develop better algorithms that assess credibility, Wired says.