It's Everyone's Problem

At the Gene Expression blog, Razib Khan says bias is "rife in any science which utilizes statistics." A recent post at the GiveWell blog presented some suggestions for the social sciences, including one that said that publication bias is of great concern when examining research. "We wonder both (a) how many studies are done, but never published because people don't find the results interesting or in line with what they had hoped; (b) for a given paper, how many different interpretations of the data were assembled before picking the ones that make it into the final version," GiveWell says. The answer may be to have researchers publish their questions, theories, and planned methods of data collection before their begin their work, the blog adds — that way, if the results come out differently from what the researchers expected, "there's then no way to hide this from a motivated investigator." Khan says this is not just a social science problem but an issue that all of science needs to address. "Statistical methods have become metrics to attain by any means necessary, when in reality they should be guidelines to get a better grasp of reality," he says. "The only way to clean up some of the natural human bias in the enterprise is to shed some light on it."