« Back to Results

Large Language Models and Generative AI: New Research Tools and Novel Economic Insights

Paper Session

Saturday, Jan. 4, 2025 10:15 AM - 12:15 PM (PST)

Hilton San Francisco Union Square, Plaza A
Hosted By: American Economic Association
  • Chair: Susan Athey, Stanford University

Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?

Apostolos Filippas
,
Fordham University
John Horton
,
Massachusetts Institute of Technology
Benjamin Manning
,
Massachusetts Institute of Technology

Abstract

Large language models (LLM)—because of how they are trained and designed—are implicit computational models of humans—a homo silicus. LLMs can be used like economists use homo economicus: they can be given endowments, information, preferences, and so on, and then their behavior can be explored in scenarios via simulation. Experiments using this approach, derived from Charness and Rabin (2002), Kahneman et al. (1986), and Samuelson and Zeckhauser (1988) show qualitatively similar results to the original, but it is also easy to try variations for fresh insights. LLMs could allow researchers to pilot studies via simulation, first improving their experimental design and searching for novel social science insights to test in the real world.

Occupational Heterogeneity in Exposure to Generative AI

Edward Felten
,
Princeton University
Manav Raj
,
University of Pennsylvania
Robert Seamans
,
New York University

Abstract

Recent dramatic increases in generative Artificial Intelligence (AI), including language modeling and image generation, has led to many questions about the effect of these technologies on the economy. We use a recently developed methodology to systematically assess which occupations are most exposed to advances in AI language modeling and image generation capabilities. We then characterize the profile of occupations that are more or less exposed based on characteristics of the occupation, suggesting that highly-educated, highly-paid, white-collar occupations may be most exposed to generative AI, and consider demographic variation in who will be most exposed to advances in generative AI. The range of occupations exposed to advances in generative AI, the rapidity with its spread, and the variation in which populations will be most exposed to such advances, suggest that government can play an important role in helping people adapt to how generative AI changes work.

What Can Robots Do? Using Large Language Models to Understand How Embodied Computation May Affect Occupations and the Economy

Matt Beane
,
University of California-Santa Barbara
Erik Brynjolfsson
,
Stanford University
Fei-Fei Li
,
Stanford University
J. Frank Li
,
University of British Columbia
Tom Mitchell
,
Carnegie Mellon University
Zanele Munyikwa
,
Massachusetts Institute of Technology
Georgios Petropoulos
,
University of Southern California
Daniel Rock
,
University of Pennsylvania
Daniela Rus
,
Massachusetts Institute of Technology
Jiajun Wu
,
Stanford University
Ruohan Zhang
,
Pennsylvania State University

Abstract

We investigate the impact of robotic technologies on work and home production, with a focus on the U.S. labor market. To facilitate our analysis, we first create a rubric of 24 statements for applying robotic technologies to tasks, activities, and occupations. Using the rubric, we then employ both Large Language Models (LLM) and human expertise to assess the suitability of more than 2000 work tasks and 1000 household task to robots. Next, we create a robust methodology and measurement by evaluating the alignment of LLM and human responses. The results allow us to identify the heterogeneous current state exposure of robotic technologies at task, occupational, industry, and geographic level. The finding implies substantial effects of robots on jobs, wage, standards of living, productivity growth, economic dynamics, and the society.

Augmenting Human Survey Responses with Generative AI: An Application to Economic Research

Erik Brynjolfsson
,
Stanford University
José Ramón Enríquez
,
Stanford University
David Nguyen
,
Stanford University

Abstract

The emergence of Generative AI presents a promising avenue for augmenting traditional survey methodology and improve their accuracy, affordability, scalability, and timeliness. This study demonstrates a new application of Gen AI, specifically Large Language Models (LLMs), to complement and augment human responses in surveys for economic research. We concentrate on two central cases: predicting individual budget constraints and valuations of specific consumer goods and services. We evaluate the efficacy of various LLMs in accurately predicting survey responses from the Panel Study on Income Dynamics and a large study of good and services valuations of more than 20,000 human respondents through multiple waves. In the second part, we expand responses to non-surveyed goods and compare them with aggregate purchase decisions from consumer surveys. We identify the most predictive elements, including individual characteristics, within the prompts to different models, and discuss a range of limitations and pitfalls to avoid. This study offers valuable insights for economic survey researchers and practitioners alike and opens up various avenues for future research in survey analysis.

Discussant(s)
Georgios Petropoulos
,
University of Southern California
Robert Seamans
,
New York University
Anton Korinek
,
University of Virginia
Daniel Rock
,
University of Pennsylvania
JEL Classifications
  • O3 - Innovation; Research and Development; Technological Change; Intellectual Property Rights