American Economic Journal:
Microeconomics
ISSN 1945-7669 (Print) | ISSN 1945-7685 (Online)
Stability of Experimental Results: Forecasts and Evidence
American Economic Journal: Microeconomics
vol. 14,
no. 3, August 2022
(pp. 889–925)
Abstract
How robust are experimental results to changes in design? And can researchers anticipate which changes matter most? We consider a real-effort task with multiple behavioral treatments and examine the stability along six dimensions: (i) pure replication, (ii) demographics, (iii) geography and culture, (iv) the task, (v) the output measure, and (vi) the presence of a consent form. We find near-perfect replication of the experimental results and full stability of the results across demographics, significantly higher than a group of experts expected. The results differ instead across task and output change, mostly because the task change adds noise to the findings.Citation
DellaVigna, Stefano, and Devin Pope. 2022. "Stability of Experimental Results: Forecasts and Evidence." American Economic Journal: Microeconomics 14 (3): 889–925. DOI: 10.1257/mic.20200129Additional Materials
JEL Classification
- C90 Design of Experiments: General
- D82 Asymmetric and Private Information; Mechanism Design
- D91 Micro-Based Behavioral Economics: Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making