In the wake of a data-tampering incident that resulted in the dismissal of an Orchid Cellmark DNA analyst in mid-November, some experts in the forensic bioinformatics community are calling for more transparency from the world’s largest private DNA testing firm.
Last month, Orchid Cellmark, Orchid BioSciences’ DNA forensics unit, said that it had dismissed Sarah Blair, a DNA forensic analyst, because she “did not consistently follow the company’s required procedures to ensure the accuracy of results.”
An investigation into the Blair case, which some believe is part of a systemic problem with Orchid Cellmark’s DNA-testing procedures, is ongoing within the company.
According to court documents and several sources familiar with the incident, Blair on multiple occasions substituted electronic data files for control samples with data files from a prior run. The data-swapping was initially discovered during a routine data review, and did not adversely affect any criminal cases, the company said. The substitutions “were not being made to replace contaminated or failed controls,” but because the control profiles contained “artifacts or had other anomalies indicative of a poor run which would have necessitated a re-run of that sample,” Orchid said.
But rather than go back and perform the appropriate re-runs, Blair simply copied and pasted control data from previous experiments — in at least 20 separate cases.
Robin Cotton, director of technical forensic science at Orchid Cellmark, stated in an affidavit filed in a Maryland court that the company has carried out a rigorous investigation of all of Blair’s casework for the past year — as well as that of the rest of the company’s analysts — and that “no substitution of data files was observed in any other files from any other analyst in the laboratory.” Furthermore, Cotton concluded, “there is no general software problem that contributed to the substitution of data.”
However, Daniel Krane, president of Forensic Bioinformatics Services and a professor at Wright State University, has questioned the depth of Orchid Cellmark’s investigation. In an affidavit filed in response to Cotton’s statement for the Maryland court, Krane said, “I do not agree that the investigation of Sarah Blair’s misconduct as it is described by Dr. Cotton rules out the possibility of a systemic problem with the DNA testing performed by Orchid Cellmark that may predate Ms. Blair’s actions.”
Orchid Cellmark holds exclusive forensic-testing contracts with a number of major police departments worldwide, including the Los Angeles Police Department and Scotland Yard. Cotton’s and Krane’s statements were submitted for a murder case in Maryland in which the defense questioned the quality of Orchid Cellmark’s DNA analysis. Cotton concluded in his affidavit that “the misconduct in our laboratory by Sarah Blair in 2004 has no impact” on the case in question.
Krane, who was retained by the defense in the Maryland case, told BioInform that Cellmark could have conducted a more thorough investigation to rule out that possibility, however. The company, he said, only screened its prior casework for data files that were manipulated in the same manner as those swapped by Blair, in which dates for the original file served as a red flag. Since each file should have a unique time stamp, any duplicates or files that appeared to be out of sequence signaled that the files had been manipulated.
But, Krane said, this doesn’t account for the possibility that an analyst could have changed the file dates — something that several forensics experts verified would be possible — or even manipulated the raw data in a more fundamental way.
“If an analyst was interested in being more sophisticated in this type of misconduct or fraud, it would actually be very easy for them to use software that would allow them to modify those file creation dates such that this method of detection would fail to find it,” Krane told BioInform.
He noted, however, that it’s unlikely that an analyst would tamper with the records beyond changing the file dates. While it would be technically possible, he said, “reaching in and manipulating the individual characters in the files requires a very high level of sophistication, and I don’t think that anybody is presuming that that type of data manipulation is very common or even happens. First, because it requires a lot of sophistication with computer skills, and the people who are doing these tests have been hired not because of their computer skills, but because of their skills at a laboratory bench.”
Secondly, he said, “Even if you were a PhD in computer science and you wanted to do that kind of manipulation, it would just be so much easier for you to accomplish the same results by doing something physical with the experiment itself.”
Nevertheless, he said, Cellmark could and should “do a test in a more sophisticated way that looked at the actual underlying data — as opposed to just the file creation dates — and did essentially all possible pairwise comparisons of samples … [because] you should never find two samples with precisely the same data points,” Krane said, “but they haven’t done that.”
Mark Stolorow, executive director of Orchid Cellmark, confirmed that the company has not conducted a more in-depth investigation of its raw data, and told BioInform that it is not possible for analysts to change file dates or tamper with the data records any further than Blair did.
“We operate under the assumption that we have adequate controls in place, and in the 16-year history of our company here at our Germantown [Md.] site, that has been the case,” he said.
But according to Simon Ford, president of Lexigen Science and Law Consultants, a forensics consulting firm, there is a fairly high degree of flexibility in the type of software that Cellmark uses for its DNA analysis — Applied Biosystems’ GeneScan and Genotyper systems.
ABI designed GeneScan “for a broad range of functions; it was not just designed for forensics,” Ford said. “So it doesn’t have the auditing and tracking functions within it for changes. So you can actually change the fields in it, and no one would know about it.”
Ford doesn’t hold ABI accountable for the Blair incident. Because GeneScan wasn’t designed specifically for forensic use, “I don’t think [ABI] felt there was a need to police the software in that way,” he said. However, he added, “I wonder what precautions are in place within Cellmark in terms of the software and access to the computers, and what rules they have in place to prevent analysts from doing unusual modifications of this type.”
An ABI spokeswoman told BioInform that the company “is looking into the issue that occurred at Orchid Cellmark, and as the leader in the forensic identification industry we aim to provide quality systems for forensic analysis.” The company declined to comment further, however.
Krane and Ford both called for Cellmark to release all of its electronic records over the past year — including sample sheets, injection lists, and log files — to allow for a more systematic evaluation of whether the data tampering extended beyond the “unsophisticated” file-swapping that Blair performed. Both, however, noted that Cellmark has a history of being “very sparing in the discovery that they will actually release,” as Ford explained.
“Most labs, including the FBI, will provide pretty much all of the electronic data. They’ll just burn it onto a CD-ROM and give it to you, and so you can work your way through it and you have absolutely everything there,” Ford said. Cellmark, however, “tends to just give a very small selection of the files, so it means that it’s very difficult for me to do an analysis that would actually spot these types of fraudulent problems.”
Krane echoed Ford’s comments. “Generally speaking, Cellmark has not been particularly good at responding to discovery motions aimed at getting them to reveal the data that underlies their conclusions,” he said. “If they had been as open as the FBI and other testing labs in their responses it is entirely possible that problems such as those related to Sarah Blair’s misconduct would have been brought to their attention and fixed years ago.”
Krane said that it’s very likely that the Blair incident could impact Orchid Cellmark’s credibility, and that defense attorneys could start raising doubts about other cases where samples were analyzed by the company — even though Blair was not directly involved with the analysis. Krane said that he is aware of at least two such cases already, including the one in Maryland for which he submitted the affidavit.
According to a report from the Los Angeles Daily News, the LAPD is re-examining at least 27 cases in light of the incident. Jennifer Friedman, a Los Angeles deputy public defender, told the paper that Cellmark notified the public defender’s office of two cases for which Blair conducted DNA analysis — a homicide and a sex-offenses case. “We were assured that the tests were redone and everything turned out the same, but we aren’t so sure,” Friedman said. “We in the defense community have had problems for years getting full discovery so we can test the results. What this has done is prompt more concerns about how the tests are conducted.”
Friedman said that the public defender’s office intends to review all cases in which Cellmark was used.
“We don’t know how many cases this could be — perhaps in the dozens,” she said.
Orchid Cellmark’s Stolorow declined to comment on whether the Blair case has led to increased requests to reanalyze its data.