« Back to Results

Advances in Machine Learning on Online Job Postings

Paper Session

Friday, Jan. 6, 2023 2:30 PM - 4:30 PM (CST)

Hilton Riverside, Quarterdeck A-B
Hosted By: American Economic Association
  • Chair: Erik Brynjolfsson, Stanford University

work2vec: Measuring the Latent Structure of the Labor Market

Sarah Bana
,
Stanford University
Erik Brynjolfsson
,
Stanford University
Daniel Rock
,
University of Pennsylvania
Sebastian Steffen
,
Massachusetts Institute of Technology

Abstract

Job postings provide unique insights about the demand for skills, tasks, and occupations. Using the full text of data from millions of online job postings, we leverage natural language processing (NLP) in a machine learning model with over 100 million parameters to classify job postings' occupation labels and salaries. To derive additional insights from the model, we develop a method of injecting deliberately constructed text snippets reflecting occupational content into postings. We apply this text injection technique to understand the returns to several information technology skills including machine learning itself. We further extract measurements of the topology of the labor market, building a “jobspace” using the relationships learned in the text structure. Our measurements of the jobspace imply expansion of the types of work available in the U.S. labor market from 2010 to 2019. We compare change rates across occupations, finding substantial heterogeneity across categories. We also demonstrate that this technique can be used to construct indices of occupational technology exposure with an application to remote work. Moreover, our analysis shows that data-driven hierarchical taxonomies can be constructed from job postings to augment existing occupational taxonomies like the SOC (Standard Occupational Classification) system.

Valuing the U.S. Data Economy Using Machine Learning and Online Job Postings

José Bayoán Santiago-Calderón
,
U.S. Bureau of Economic Analysis
Dylan Rassier
,
U.S. Bureau of Economic Analysis

Abstract

With the recent proliferation of data collection and uses in the digital economy, the understanding and statistical treatment of data stocks and flows is of interest among compilers and users of national economic accounts. In this paper, we measure the value of own-account data stocks and flows for the U.S. business sector by summing the production costs of data-related activities implicit in occupations. Our method augments the traditional sum-of-costs methodology for measuring other own-account intellectual property products in national economic accounts by proxying occupation-level time-use factors using a machine learning model and the text of online job advertisements. In our experimental estimates, we find that annual current-dollar investment in own-account data assets for the U.S. business sector grew from $82.6 billion in 2003 to $159.5 billion in 2020, with an average annual growth rate of 3.9 percent. Cumulative current-dollar investment for the period 2003–2020 was $2.1 trillion. In addition to annual current-dollar investment, we present historical-cost net stocks, real growth rates, and effects on value-added by industrial sector.

National Wage Setting

Jonathon Hazell
,
London School of Economics
Christina Patterson
,
University of Chicago
Heather Sarsons
,
University of Chicago
Bledi Taska
,
Lightcast

Abstract

How do firms set wages across space? Using vacancy data with detailed job-level information and a survey of HR managers, we show that 35 percent of multi-establishment firms set wages nationally, meaning they choose rigid pay structures in which they set exactly the same nominal wage for the same job in different regions. We start by showing that a significant minority of firms set identical wages within an occupation across all of their locations. This practice is widespread but most common in high-wage jobs. Next, using the pass-through of local shocks to wages in other locations of the firm, we argue that these identical wages indicate national wage setting. Our survey suggests that one reason firms set wages nationally is that nominal, rather than real, wage comparisons matter to workers.

Understanding Algorithmic Bias in Job Recommender Systems: An Audit Study Approach

Peter Kuhn
,
University of California-Santa Barbara
Shuo Zhang
,
Northeastern University

Abstract

To social science researchers, the recommendation algorithms used by job boards to recommend jobs to workers are a proprietary ‘black box’. To derive insights into how these algorithms work, we conduct an algorithmic audit of four Chinese job boards, where we create fictitious applicant profiles and observe which jobs are recommended to profiles that differ only in age and gender. We then estimate the cumulative effect of pursuing these recommended jobs by applying to them in up to three rounds of successive applications. Focusing on the jobs that were recommended to just one of the two genders that applied, we find that only-to-women jobs propose lower wages, request fewer years of working experience, and are more likely to require literacy skills and administrative skills. Only-to-women (men) jobs also disproportionately contain words related to feminine (masculine) personality characteristics, as measured by three distinct approaches for identifying such characteristics. Finally, we assess the patterns in the recommendations generated by our audit study for their consistency with four processes the algorithms could be using: item-based collaborative filtering, content-based matching, matching based on recruiters’ profile views, and rules-based matching based on employers’ stated gender preferences. We find evidence suggesting that the algorithms are relying on all but the last of these processes.

How are Startups Shaping the Future of Work? The Role of AI Translators

Matthias Qian
,
University of Oxford

Abstract

AI translators — multidisciplinary experts who bridge business and technology expertise — reduce the coordination costs that arise with the difficulties in the communication of hyperspecialized workers who engage in the division of labor to redesign systems of decision making. The organizational inertia of incumbent firms reduces their adoption of AI translators, increasing the risk of failed AI investments and of their creative destruction. This paper asks if AI translators are the basis for the successful entry of new firms and how these VC-funded startups shape the future of work. I identify 14 million AI translator job postings using natural language processing of over one billion task descriptors extracted from the full vacancy text of the near universe of the past decades’ US online job ads. Using a sample of 11,810 venture-capital-funded US startups, I find a positive effect of AI translator use on startup performance, including on successful initial public offerings. These scaled startups rely heavily on AI translators as intermediaries: they post over four times as many AI translator job postings as incumbent firms. The lack of intellectual property protections on the task composition of jobs contributes to strong local knowledge spillover effects that explain the growing importance of AI translators in the labor market.

Discussant(s)
Arthur Turrell
,
UK Office of National Statistics
Laura Pilossoph
,
Federal Reserve Bank of New York
Mária Balgová
,
IZA
Joanna Lahey
,
Texas A&M University
William Kerr
,
Harvard Business School
JEL Classifications
  • O3 - Innovation; Research and Development; Technological Change; Intellectual Property Rights
  • J2 - Demand and Supply of Labor