« Back to Results

Algorithmic Fairness and Bias

Paper Session

Saturday, Jan. 4, 2020 10:15 AM - 12:15 PM (PST)

Marriott Marquis, Grand Ballroom 1
Hosted By: American Economic Association
  • Chair: Bo Cowgill, Columbia University

Designing Organizations and Incentives with Human and Artificial Intelligence Agents

Susan Athey
,
Stanford University
Kevin Bryan
,
University of Toronto
Joshua Gans
,
University of Toronto

Abstract

This paper analyzes the question of how organizations and incentives should be designed when the organization can use both human and artificial intelligence (AI) agents can be deployed in production. Applications might include hiring, project selection, or resource prioritization. We consider questions such as whether the human or the AI should make the final decision, and how incentives and tasks should be designed for both kinds of agents. We incorporate differences between AI and human agents, including the cost of incorporating a large number of data elements into decisions; the inability of certain types of prior information to be encoded easily for an AI; risk aversion on the part of the agent but not the AI; and preferences of the human agents that differ from those of the firm.

Biased Programmers? Or Biased Data? A Field Experiment in Operationalizing AI Ethics

Bo Cowgill
,
Columbia University
Fabrizio Dell'Aqua
,
Columbia University

Abstract

Why does "algorithmic bias" occur? The two most frequently cited reasons are "biased programmers" and "biased training data." We quantify the effects of these using a field experiment on a diverse group of AI practitioners. In our experiment, machine learning programmers are asked to predict math literacy scores for a representative sample of OECD residents. One group is given perfectly representative training data, and the other is given a "dataset of convenience" -- a biased training sample containing who confirm to common expectations about who is good at math. Using this field experiment, we quantify the benefits of employing programmers who are diversity-aware vs obtaining more representative training data. We also measure the effectiveness of training interventions to reduce algorithmic bias, including both reminders and technical guidance.

Regulating Discrimination in the Presence of Algorithms

Jon Kleinberg
,
Cornell University
Jens Ludwig
,
University of Chicago
Sendhil Mullainathan
,
University of Chicago
Ashesh Rambachan
,
Harvard University

Abstract

The ambiguity of human decision-making often makes it extraordinarily hard for the legal system to know whether anyone has actually discriminated. So, to fully understand how algorithms affect discrimination, we must also understand how they affect the problem of detecting discrimination. In this paper, we present a model that illustrates how the introduction of algorithmic decision-making changes the problem of detecting discrimination in markets. We focus on a regulator that wishes to detect discriminatory hiring practices. At its core, the regulator faces an asymmetric information problem – if the regulator observes that a firm hires few minority workers, the regulator cannot know whether this is due to unobserved differences in productivity or discriminatory hiring practices. We then provide two main results on how algorithmic decision-making interacts with this asymmetric information problem. First, without proper safeguards in place, the introduction of algorithmic decision-making may make it more difficult for the regulator to detect discriminatory practices. In this case, the algorithm becomes a black box that can serve to justify existing discriminatory practices. Second, if the regulator can ``audit’’ the firm’s algorithm by querying its outputs, algorithmic decision-making makes it easier for the regulator to detect discrimination as it provides a new source of transparency that is otherwise not available. Our results highlight that the rise of algorithmic decision-making does not necessarily worsen or alleviate discrimination – its effects depend crucially on the existing regulatory environment.

Algorithmic Risk Assessment in the Hands of Humans

Megan Stevenson
,
George Mason University
Jennifer Doleac
,
Texas A&M University

Abstract

We evaluate the impacts of adopting algorithmic predictions of future offending (risk assessments) as an aid to judicial discretion in felony sentencing. We find that judges' decisions are influenced by the risk score, leading to longer sentences for defendants with higher scores and shorter sentences for those with lower scores. However, we find no robust evidence that this reshuffling led to a decline in recidivism, and, over time, judges appeared to use the risk scores less. Risk assessment's failure to reduce recidivism is at least partially explained by judicial discretion in its use. Judges systematically grant leniency to young defendants, despite their high risk of reoffending. This is in line with a long standing practice of treating youth as a mitigator in sentencing, due to lower perceived culpability. Such a conflict in goals may have led prior studies to overestimate the extent to which judges make prediction errors. Since one of the most important inputs to the risk score is effectively off-limits, risk assessment's expected benefits are curtailed. We find no evidence that risk assessment affected racial disparities statewide, although there was a relative increase in sentences for black defendants in courts that appeared to use risk assessment most. We conduct simulations to evaluate how race and age disparities would have changed if judges had fully complied with the sentencing recommendations associated with the algorithm. Racial disparities might have increased slightly, but the largest change would have been higher relative incarceration rates for defendants under the age of 23. In the context of contentious public discussions about algorithms, our results highlight the importance of thinking about how man and machine interact.
Discussant(s)
Joshua Gans
,
University of Toronto
Erik Brynjolfsson
,
Massachusetts Institute of Technology
Jorge Guzmán
,
Columbia University
Shane Greenstein
,
Harvard Business School
JEL Classifications
  • J7 - Labor Discrimination
  • C6 - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling