« Back to Results

Adversarial Methods

Paper Session

Friday, Jan. 6, 2023 2:30 PM - 4:30 PM (CST)

Hilton Riverside, Norwich
Hosted By: Econometric Society
  • Chair: Jonas Metzger, Stanford University

Adversarial Estimation of Conditional Moment Models

Nishanth Dikkala
,
Google Research
Greg Lewis
,
Microsoft Research
Lester Mackey
,
Microsoft Research
Vasilis Syrgkanis
,
Microsoft Research

Abstract

We develop an approach for estimating models described via conditional moment restrictions, with a prototypical application being non-parametric instrumental variable regression. We introduce a min-max criterion function, under which the estimation problem can be thought of as solving a zero-sum game between a modeler who is optimizing over the hypothesis space of the target model and an adversary who identifies violating moments over a test function space. We analyze the statistical estimation rate of the resulting estimator for arbitrary hypothesis spaces, with respect to an appropriate analogue of the mean squared error metric, for ill-posed inverse problems. We show that when the minimax criterion is regularized with a second moment penalty on the test function and the test function space is sufficiently rich, then the estimation rate scales with the critical radius of the hypothesis and test function spaces, a quantity which typically gives tight fast rates. Our main result follows from a novel localized Rademacher analysis of statistical learning problems defined via minimax objectives. We provide applications of our main results for several hypothesis spaces used in practice such as: reproducing kernel Hilbert spaces, high dimensional sparse linear functions, spaces defined via shape constraints, ensemble estimators such as random forests, and neural networks. For each of these applications we provide computationally efficient optimization methods for solving the corresponding minimax problem (e.g. stochastic first-order heuristics for neural networks). In several applications, we show how our modified mean squared error rate, combined with conditions that bound the ill-posedness of the inverse problem, lead to mean squared error rates. We conclude with an extensive experimental analysis of the proposed methods.

Generative Adversarial Method of Moments

Elena Manresa
,
New York University
Ignacio Cigliutti
,
New York University

Abstract

We introduce the Generative Adversarial Method of Moments for models defined with moment conditions. The estimator is asymptotically equivalent to optimally--weighted 2--step GMM, but outperforms the GMM estimator in finite samples. We show this both in theory and in simulations. In our theoretical results, we exploit the relationship between AMM and GEL estimators to show, using stochastic expansions, that AMM has smaller bias than optimally--weighted GMM. In our simulation experiments we consider 3 different models: estimation of the variance as in Altonji and Segal (1996), estimation of the autoregressive coefficient in a dynamic panel data model, and estimation of a DSGE model by matching IRFs. We compare the estimator's performance to other commonly--used procedures in the literature, and find that AMM outperforms in cases where other estimators fail.

Adversarial Estimators

Jonas Metzger
,
Stanford University

Abstract

We develop an asymptotic theory of adversarial estimators ('A-estimators'). Like maximum-likelihood-type estimators ('M-estimators'), both the estimator and estimand are defined as the critical points of a sample and population average respectively. A-estimators generalize M-estimators, as their objective is maximized by one set of parameters and minimized by another. The continuous-updating Generalized Method of Moments estimator, popular in econometrics and causal inference, is among the earliest members of this class which distinctly falls outside the M-estimation framework. Since the recent success of Generative Adversarial Networks, A-estimators received considerable attention in both machine learning and causal inference contexts, where a flexible adversary can remove the need for researchers to manually specify which features of a problem are important. We present general results characterizing the convergence rates of A-estimators under both point-wise and partial identification, and derive the asymptotic root-n normality for plug-in estimates of smooth functionals of their parameters. All unknown parameters may contain functions which are approximated via sieves. While the results apply generally, we provide easily verifiable, low-level conditions for the case where the sieves correspond to (deep) neural networks. Our theory also yields the asymptotic normality of general functionals of neural network M-estimators (as a special case), overcoming technical issues previously identified by the literature. We examine a variety of A-estimators proposed across econometrics and machine learning and use our theory to derive novel statistical results for each of them. Embedding distinct A-estimators into the same framework, we notice interesting connections among them, providing intuition and formal justification for their recent success in practical applications.
JEL Classifications
  • C1 - Econometric and Statistical Methods and Methodology: General