AEA Member Announcements: Updated AEA Data and Code Availability Policy (July 16, 2019)
The AEA established a Data Availability Policy in the early 2000s. Data (and code) have been provided for hundreds of articles, made available via the article landing pages on the AEA website.
On July 10, 2019, the Association adopted an updated Data and Code Availability Policy, which can be found at https://www.aeaweb.org/journals/policies/data-code. The goal of the new policy is to improve the reproducibility and transparency of materials supporting research published in the AEA journals by providing improved guidance on the types of materials required, increased quality control, and more review earlier in the publication process.
What's new in the policy? Several items of note:
- A central role for the AEA Data Editor. The inaugural Data Editor was appointed in January 2018 and will oversee the implementation of the new policy.
- The policy now clearly applies to code as well as data and explains how to proceed when data cannot be shared by an author. The Data Editor will regularly ask for the raw data associated with a paper, not just the analysis files, and for all programs that transform raw data into those from which the paper's results are computed. Replication archives will now be requested prior to acceptance, rather than during the publication process after acceptance, providing more time for the Data Editor to review materials.
- There is a new repository infrastructure, hosted at openICPSR, called the "AEA Data and Code Repository." Data (where allowed) and code (always required) will be uploaded to the repository and shared with the Data Editor prior to publication.
- Each repository will have a unique and distinct DOI, which will be linked back to the article, and the article, as before, will be linked to the data repository. Repositories can be searched using JEL codes, keywords, and other fields where authors fill in the appropriate information such as geographic coverage, time periods, universe, and data types.
- The AEA will migrate the entire back archive of more than 3,000 data and code supplements into the same infrastructure. Authors wishing to enhance the visibility of their data and code archives should contact the Data Editor (email@example.com).
- The Data Editor will assess compliance with this policy and will verify the accuracy of the information. This means that feedback will be provided on the content and quality of the README file, as well as other information that may be part of the replication archive.
- Will the Data Editor’s team run authors' code prior to acceptance? Yes, to the extent that it is feasible. The code will need to produce the reported results, given the data provided. Authors can consult a generic checklist, as well as the template used by the replication teams.
- Will code be run even when the data cannot be posted? This was once an exemption, but the Data Editor will now attempt to conduct a reproducibility check of these materials through a third party who has access to the (confidential or restricted) data. Such checks have already been successfully conducted using the protocol outlined here.
More details can be found in the Frequently Asked Questions (FAQ) at https://www.aeaweb.org/journals/policies/data-code/faq.
The Association recognizes that some of these policy updates will require adjustments as challenges arise, and the AEA Publications Office looks forward to working those out in collaboration with the membership.