Measuring and Bounding Experimenter Demand
American Economic Review
vol. 108,
no. 11, November 2018
(pp. 3266-3302)
Abstract
We propose a technique for assessing robustness to demand effects of findings from experiments and surveys. The core idea is that by deliberately inducing demand in a structured way we can bound its influence. We present a model in which participants respond to their beliefs about the researcher's objectives. Bounds are obtained by manipulating those beliefs with "demand treatments." We apply the method to 11 classic tasks, and estimate bounds averaging 0.13 standard deviations, suggesting that typical demand effects are probably modest. We also show how to compute demand-robust treatment effects and how to structurally estimate the model.Citation
de Quidt, Jonathan, Johannes Haushofer, and Christopher Roth. 2018. "Measuring and Bounding Experimenter Demand." American Economic Review, 108 (11): 3266-3302. DOI: 10.1257/aer.20171330Additional Materials
JEL Classification
- C83 Survey Methods; Sampling Methods
- C90 Design of Experiments: General
- D83 Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
- D91 Micro-Based Behavioral Economics: Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making