This website uses cookies.

By clicking the "Accept" button or continuing to browse our site, you agree to first-party and session-only cookies being stored on your device to enhance site navigation and analyze site performance and traffic. For more information on our use of cookies, please see our Privacy Policy.

  1. Home
  2. ASSA Annual Meeting
  3. 2023
  4. Program
  5. Sensitivity Analysis
« Back to Results

Sensitivity Analysis

Paper Session

Sunday, Jan. 8, 2023 8:00 AM - 10:00 AM (CST)

Hilton Riverside, Durham
Hosted By: Econometric Society
  • Chair: Timothy Christensen, New York University

Discordant Relaxations of Misspecified Models

Desire Kedagni
,
Iowa State University
Lixiong Li
,
Johns Hopkins University
Ismael Yacoub Mourifie
,
University of Toronto

Abstract

In many set identified models, it is difficult to obtain a tractable characterization of the identified set. Therefore, empirical works often construct confidence region based on an outer set of the identified set. Because an outer set is always a superset of the identified set, this practice is often viewed as conservative yet valid. However, this paper shows that, when the model is refuted by the data, a nonempty outer set could deliver conflicting results with another outer set derived from the same underlying model structure, so that the results of outer sets could be misleading in the presence of misspecification. We provide a sufficient condition for the existence of discordant outer sets which covers models characterized by intersection bounds and the Artstein (1983) inequalities. We also derive sufficient conditions for the non-existence of discordant submodels, therefore providing a class of models for which constructing outer sets cannot lead to misleading interpretations. In the case of discordancy, we follow Masten and Poirier (2021) by developing a method to salvage misspecified models, but unlike them we focus on discrete relaxations. We consider all minimum relaxations of a refuted model which restores data-consistency. We find that the union of the identified sets of these minimum relaxations is robust to detectable misspecifications and has an intuitive empirical interpretation.

Included and Excluded Instruments in Structural Estimation

Isaiah Andrews
,
Harvard University
Nano Barahona
,
University of California-Berkeley
Matthew Gentzkow
,
Stanford University
Ashesh Rambachan
,
Harvard University
Jesse Shapiro
,
Harvard University

Abstract

We consider the choice of instrumental variables when a researcher’s structural model may be misspecified. We contrast included instruments, which have a direct causal effect on the out- come holding constant the endogenous variable of interest, with excluded instruments, which do not. We show conditions under which the researcher’s estimand maintains an interpretation in terms of causal effects of the endogenous variable under excluded instruments but not under included instruments. We apply our framework to estimation of a linear instrumental variables model, and of differentiated goods demand models under price endogeneity. We show that the distinction between included and excluded instruments is quantitatively important in simulations based on an application. We extend our results to a dynamic setting by studying estimation of production function parameters under input endogeneity.

The Effect of Omitted Variables on the Sign of Regression Coefficients

Matthew A. Masten
,
Duke University
Alexandre Poirier
,
Georgetown University

Abstract

Omitted variables are a common concern in empirical research. We show that "Oster's delta" (Oster 2019), a commonly reported measure of regression coefficient robustness to the presence of omitted variables, does not capture sign changes in the parameter of interest. Specifically, we show that any time this measure is large--suggesting that omitted variables may be unimportant--a much smaller value can actually reverse the sign of the parameter of interest. Relatedly, we show that selection bias adjusted estimands can be extremely sensitive to the choice of the sensitivity parameter. Specifically, researchers commonly compute a bias adjustment under the assumption that Oster's delta equals one. Under the alternative assumption that delta is very close to one, but not exactly equal to one, we show that the bias can instead be arbitrarily large. To address these concerns, we propose a modified measure of robustness that accounts for such sign changes, and discuss best practices for assessing sensitivity to omitted variables. We demonstrate this sign flipping behavior in an empirical application to social capital and the rise of the Nazi party, where we show how it can overturn conclusions about robustness, and how our proposed modifications can be used to regain robustness. We implement our proposed methods in the companion Stata module regsensitivity for easy use in practice.

On Quantile Treatment Effects, Rank Similarity, and the Variation of Instrumental Variables

Sukjin Han
,
University of Bristol
Haiqing Xu
,
University of Texas

Abstract

This paper investigates how certain relationship between observed and counterfactual distributions plays a role in the identification of distributional treatment effects under endogeneity, and shows that this relationship holds in a range of nonparametric models for treatment effects. To motivate the new identifying assumption, we first provide a novel characterization of popular assumptions restricting treatment heterogeneity in the literature, specifically rank similarity. We show the stringency of this type of assumptions and propose to relax them in economically meaningful ways. This relaxation will justify certain parameters (e.g., treatment effects on the treated) against others (e.g., treatment effects for the entire population). It will also justify the quest of richer exogenous variation in the data (e.g., the use of multiple instrumental variables). The prime goal of this investigation is to provide empirical researchers with tools for identifying and estimating treatment effects that are flexible enough to allow for treatment heterogeneity, but still yield tight policy evaluation and are easy to implement.
JEL Classifications
  • C10 - General
  • C54 - Quantitative Policy Modeling