Journal of Economic Perspectives: Vol. 26 No. 2 (Spring 2012)


Quick Tools:

Print Article Summary
Export Citation
Sign up for Email Alerts Follow us on Twitter


JEP - All Issues

Using Internet Data for Economic Research

Article Citation

Edelman, Benjamin. 2012. "Using Internet Data for Economic Research." Journal of Economic Perspectives, 26(2): 189-206.

DOI: 10.1257/jep.26.2.189


The data used by economists can be broadly divided into two categories. First, structured datasets arise when a government agency, trade association, or company can justify the expense of assembling records. The Internet has transformed how economists interact with these datasets by lowering the cost of storing, updating, distributing, finding, and retrieving this information. Second, some economic researchers affirmatively collect data of interest. For researcher-collected data, the Internet opens exceptional possibilities both by increasing the amount of information available for researchers to gather and by lowering researchers' costs of collecting information. In this paper, I explore the Internet's new datasets, present methods for harnessing their wealth, and survey a sampling of the research questions these data help to answer. The first section of this paper discusses "scraping" the Internet for data—that is, collecting data on prices, quantities, and key characteristics that are already available on websites but not yet organized in a form useful for economic research. A second part of the paper considers online experiments, including experiments that the economic researcher observes but does not control (for example, when Amazon or eBay alters site design or bidding rules); and experiments in which a researcher participates in design, including those conducted in partnership with a company or website, and online versions of laboratory experiments. Finally, I discuss certain limits to this type of data collection, including both "terms of use" restrictions on websites and concerns about privacy and confidentiality.

Article Full-Text Access

Full-text Article (Complimentary)


Edelman, Benjamin (Harvard U)

JEL Classifications

C80: Data Collection and Data Estimation Methodology; Computer Programs: General


View Comments on This Article (0) | Login to post a comment

Journal of Economic Perspectives

Quick Tools:

Sign up for Email Alerts

Follow us on Twitter

Subscription Information
(Institutional Administrator Access)


JEP - All Issues

Virtual Field Journals

AEA Member Login:

AEAweb | AEA Journals | Contact Us