Measuring Beliefs
Paper Session
Friday, Jan. 3, 2025 8:00 AM - 10:00 AM (PST)
- Chair: Charlie Holt, University of Virginia
Belief Elicitation under Ambiguity
Abstract
Beliefs play a critical role in many economic, political, and life decisions. Hence, researchers have been interested in eliciting individuals’ beliefs to better understand their decision-making processes. In recent years, researchers have developed sophisticated methods to elicit the beliefs of individuals in an incentive compatible manner. To date, much of the discussion around the incentive compatibility of these methods remains agnostic about ambiguity preferences. This is surprising in light of the fact that one could classify many beliefs elicited in experimental settings as being ambiguous. We show theoretically that the binarized scoring rule (BSR) is not incentive compatible in the presence of ambiguity, and we predict the direction of distortions. Our theory-guided experiment tests the extent to which belief reports under the BSR are distorted, and whether these distortions are in the predicted direction. The results further permit recommendations for how to elicit beliefs in the presence of ambiguity.Reduction in Belief Elicitation
Abstract
The state-of-the-art in eliciting probabilistic beliefs, the Binarized Quadratic Scoring Rule (BQSR), relies on an easily overlooked assumption: the reduction of compound lotteries. We find evidence that 70% of people violate the reduction assumption for at least some lotteries involved in the BQSR. Those who reveal preferences consistent with reduction are 33% more likely to report accurate beliefs. As theoretically predicted, the difference between reducers and non-reducers is attenuated in the elicitation of a 50% probability. We implement a novel Rank-Ordered Elicitation (ROE), which does not rely on reduction, to test whether eliminating the need for reduction increases the accuracy of reported beliefs. We find no evidence for this hypothesis. Taken together, the results suggest that preferences inconsistent with reduction contribute to the empirical under-performance of the BQSR, while leaving open a role for other factors in reducing behavioral incentive compatibility.Marginal Incentives in Belief Elicitation
Abstract
Beliefs have become an essential ingredient to understanding economic behavior, where economists have increasingly used incentives to uncover these hidden variables driving choice. The use of incentives (and incentive compatible mechanisms) is predicated on the idea that paying for the belief can lead to a more-accurate measure, with the respondent exerting additional effort when contemplating the question, forming their quantitative response, or overcoming biases with respect to partisanship or ego-relevant concerns. But in this sense, incentivizing beliefs is a real-effort problem. To this end, we outline a number of experiments outlining how marginality in the offered incentives can matter for the accuracy of the provided beliefs.Discussant(s)
Valeria Burdea
,
Ludwig Maximilian University of Munich
Tanjim Hossain
,
University of Toronto
Kirby Nielsen
,
California Institute of Technology
Ryan Oprea
,
University of California-Santa Barbara
Séverine Toussaert
,
University of Oxford
JEL Classifications
- D8 - Information, Knowledge, and Uncertainty
- C9 - Design of Experiments