Rethinking Reproducibility

The US National Institutes of Health is considering requiring some studies to go through a validation step at an independent lab.

Full-text access for registered users only. Existing users login here.
New to GenomeWeb? Register here quickly for free access.

Reproducibility and technical

Reproducibility and technical validation are two separate and important concerns. Qualification also has a specific meaning. Clearly it is important than non-reproducibility claims of a reported technical method be evaluated and tested before reporting non-reproducibility of clinical data. This variability was recently highlighted for TOMM40 used for Alzheimer Disease age of onset algorithms. [See: Alzheimer’s & Dementia 9 (2013) 132–136] To counter this recurring initial "non-reproduction" claims, the same validated methodology was arranged to be offered by a separate fee for service diagnostic vender at a relatively inexpensive price {1000 samples at < $30. each] for research use - which is less than the partial technician cost alone to develop a new assay. This arrangement was made so that proper qualification studies for clinical reproducibility could use validated technical methods, and to broaden and accelerate the clinical data flow to those clinical centers without specific support for qualification. It would be far less expensive to provide clinical centers small grants for such access than to go through early so-called "non-reproducible" academic wars as had occurred more than 20 years ago regarding early academic reports of APOE associations with age of onset in AD - repeated now early in the story of TOMM40. currently, the costs of FDA validation studies and prospective qualification are being born by the private sector for the now-ongoing clinical trial [TOMMorrow]. Peter Sorger is correct - especially when validation, reproducibility and qualification are not clearly defined and understood. This is a problem for journals, reviewers and scientists: one that could checked by a first step in sharpening up data publication at the time of review - rather than simply another NIH expense program.