Montse Fuentes, Coordinating Editor of JASA and Editor of JASA ACS
Societal impact through scientific advances is predicated on discovery and new knowledge that is reliable and robust and provides a solid foundation on which further advances can be built. Unfortunately, there is evidence many published scientific results will not stand the test of time, in part due to the lack of good scientific practices for reproducibility.
Our statistical profession has a responsibility to establish publication standards that improve the transparency and robustness of what we publish and to promote awareness within the scientific community of the need for rigor in our statistical research to ensure reproducibility of our scientific results. JASA is committed to helping lead the effort by presenting solutions that can help improve research quality and reproducibility.
Reproducibility of scientific research is our ultimate goal, and the code and data requirement is a first step in that direction.
Starting September 1, JASA ACS will require code and data as a minimum standard for reproducibility of statistical scientific research. New infrastructure is being established to support this initiative. Each manuscript will go through the current review process managed by an associate editor (AE), who will assign to one of the reviewers the broad evaluation of the code. A new editorial role—associate editor for reproducibility (AER)—will be added to ensure we meet a standard of reproducibility.
The AER will handle the review of manuscripts that have already gone through the review process and are acceptable for publication based on their scientific merit as judged by the reviewers and AE, pending assessment of the code and data. This assessment will involve the availability of code and data and a broad evaluation of the quality and potential for usability of the code and data by others who might seek to reproduce the work.
Christopher Paciorek, Victoria Stodden, and Julian Wolfson will be JASA’s first group of AERs. Paciorek is a researcher, lecturer, and statistical computing consultant in the department of statistics at the University of California at Berkeley.
Stodden is an associate professor at the University of Illinois, Urbana-Champaign, and Wolfson is an assistant professor at the University of Minnesota. All three have extensive experience in statistical programming.
The computer code and data used to derive the results will be made available through the JASA website. Alternatively, code and/or data that are available in suitable public repositories with appropriate versioning can be linked to from the JASA website.
The submission of code and data will be a condition for acceptance. Exceptions will be made for lack of adherence of this policy for proprietary data, but code will be required in all cases.
Reproducibility of scientific research is our ultimate goal, and the code and data requirement is a first step in that direction. We all have a part in promoting and enabling reproducibility, and we seek your full support to help promote the robustness of our published research.
Leave a Reply