Big data sets can prove to be a great tool for researchers, but a new paper warns that researchers should use the information carefully, reports Technology Review's Erica Naone. The paper, written by University of South Wales professor Kate Crawford and Microsoft researcher Danah Boyd, suggests that the data in these sets can often be "distorted," Naone says, and Crawford tells Naone that "big data sets are never complete." In addition, Naone says, "Crawford notes that many big data sets — particularly social data — come from companies that have no obligation to support scientific inquiry. Getting access to the data might mean paying for it, or keeping the company happy by not performing certain types of studies." Crawford and Boyd's paper also says that big data sets can raise ethical concerns, as researchers have previously shown that it's not hard to identify individuals from the data sets, even after they've been anonymized.
Oct 11, 2011