« Back to Results

Causal Inference

Paper Session

Monday, Jan. 4, 2021 12:15 PM - 2:15 PM (EST)

Hosted By: Econometric Society
  • Chair: Peter Hull, University of Chicago

Algorithm is Experiment: Machine Learning, Market Design, and Policy Eligibility Rules

Yusuke Narita
,
Yale University
Kohei Yata
,
Yale University

Abstract

Machine learning, market design, and other algorithms produce a growing portion of decisions and recommendations. Such algorithmic decisions are natural experiments (conditionally quasi-randomly assigned instruments) since the algorithms make decisions based only on observable input variables. We use this observation to characterize the sources of causal-effect identification for a class of stochastic and deterministic algorithms. Data from almost every algorithm is shown to identify some causal effect. This identification result translates into a treatment-effect estimator. We prove that our estimator is consistent and asymptotically normal for well-defined causal effects. The estimator is easily implemented even with high-dimensional data and complex algorithms. Our estimator also induces a high- dimensional regression discontinuity design as a key special case. The proofs use tools from differential geometry and geometric measure theory, which may be of independent interest. A future version will present empirical applications.

Estimating Dynamic Treatment Effects in Event Studies with Heterogeneous Treatment Effect

Liyang Sun
,
Massachusetts Institute of Technology
Sarah Abraham
,
Cornerstone Research

Abstract

To estimate the dynamic effect of an absorbing treatment, researchers often use two-way fixed effects regressions that include leads and lags of the treatment. We show that in settings with variation in treatment timing across units, the coefficient on a given lead or lag can be contaminated by effects from other periods, and apparent pretrends can arise solely from treatment effect heterogeneity. We propose an alternative estimator that is free of contamination, and illustrate the shortcomings of two-way fixed effects regression with leads and lags in comparison to our proposed estimator through an empirical application.

On Policy Evaluation with Aggregate Time-Series Shocks

Dmitry Arkhangelsky
,
CEMFI
Vasily Korovkin
,
CERGE-EI

Abstract

We propose a general strategy for estimating the treatment effects when the only source of exogenous variation is a sequence of aggregate time-series shocks. We start by arguing that commonly used estimation procedures tend to ignore the crucial time-series aspects of the data. Next, we develop a graphical tool and a novel formal test and then illustrate the issues of the design using data from influential studies in development economics [Nunn and Qian, 2014] and macroeconomics [Nakamura and Steinsson, 2014]. Motivated by these studies, we construct a new estimator, which is based on the time-series model for the aggregate shock. We analyze the statistical properties of our estimator in the practically relevant case where both cross-sectional and time-series dimensions are of similar size. Finally, to provide causal interpretation for our estimator, we analyze a new causal model that allows for rich unobserved heterogeneity in potential outcomes and unobserved aggregate shocks.

Measuring Racial Discrimination in Bail Decisions

David Arnold
,
Princeton University
Will Dobbie
,
Harvard University
Peter Hull
,
University of Chicago

Abstract

We develop new quasi-experimental tools to measure racial discrimination, due to either racial bias or statistical discrimination, in the context of bail decisions. We show that the omitted variables bias in observational release rate comparisons can be purged by using the quasi-random assignment of judges to estimate average race-specific misconduct risk. We find that nearly two-thirds of the average release rate disparity between white and Black defendants in New York City is due to racial discrimination. We then develop a hierarchical marginal treatment effects model to study the drivers of discrimination, finding evidence of both racial bias and statistical discrimination.

The Degrees of Freedom of the Synthetic Control Method

Guillaume Pouliot
,
University of Chicago
Zhen Xie
,
University of Chicago

Abstract

We provide closed-form expressions for the degrees of freedom of the synthetic control method, with and without covariates. On the one hand, these results are conceptually informative: in spite of the extensive implicit model selection typically carried out by the method, the degrees of freedom expression suggests that it is in fact not prone to overfitting. On the other hand, these results are methodologically useful: while implementing cross-validation may be challenging with short series, an unbiased estimate of the degrees of freedom allows to circumvent cross-validation altogether and instead rely on information criteria to estimate out-of-sample performance.
JEL Classifications
  • C1 - Econometric and Statistical Methods and Methodology: General