Learning from Experiments When Context Matters
American Economic Review
vol. 105,
no. 5, May 2015
(pp. 471-75)
Abstract
Suppose a policymaker is interested in the impact of an existing social program. Impact estimates using observational data suffer potential bias, while unbiased experimental estimates are often limited to other contexts. This creates a practical trade-off between internal and external validity for evidence-based policymaking. We explore this trade-off empirically for several common policies analyzed in development economics, including microcredit, migration, and education interventions. Based on mean-squared error, non-experimental evidence within context outperforms experimental evidence from another context. This advantage declines, but may not reverse, with experimental replication. We offer four reasons these findings are of general relevance to policy evaluation.Citation
Pritchett, Lant, and Justin Sandefur. 2015. "Learning from Experiments When Context Matters." American Economic Review, 105 (5): 471-75. DOI: 10.1257/aer.p20151016Additional Materials
JEL Classification
- C90 Design of Experiments: General
- G21 Banks; Depository Institutions; Micro Finance Institutions; Mortgages
- O16 Economic Development: Financial Markets; Saving and Capital Investment; Corporate Finance and Governance