The COVID-19 pandemic put US school administrators in an unprecedented position. Many felt it necessary to adopt virtual learning methods, rather than continue with in-person schooling that could put their staff and students in jeopardy of contracting and spreading COVID-19.

But in a paper in the American Economic Review: Insights, authors Rebecca Jack, Clare Halloran, James Okun, and Emily Oster found that these measures contributed significantly to the decline in students’ pass rates on standardized tests in the 2020–2021 school year.

The results come from district-level, state assessment data for math and English language arts (ELA) in grades 3–8 across 11 states. Figure 1 from the authors’ paper shows the drop in pass rates during the pandemic as compared to previous years.

Figure 1 from Jack et al. (2023)

The black closed circles represent the average percentage point change in pass rates between the spring of 2019 and the spring of 2021. The gray open circles represent the average percentage point change in pass rates in 2019 versus 2018, 2018 versus 2017, and 2017 versus 2016. 

The chart shows little variation in pass rates from 2016 to 2019 and then a sizable drop across all states and demographic groups in 2021. Overall, average pass rates between 2019 and 2021 declined by 12.8 percentage points in math and 6.8 in ELA.

The authors find that there is a clear relationship between the share of in-person schooling and the size of the drop in students’ math pass rates; the more days of in-person schooling districts offered, the smaller the drop in pass rates. A similar, although much less pronounced, pattern appears for ELA.

Using a fixed effects regression, the researchers show that districts with full in-person learning had smaller average declines than districts that were fully virtual—13.4 percentage points less in math and 8.3 percentage points less in ELA. 

The results demonstrate that hybrid and virtual schooling cannot support student learning in the same way as fully in-person instruction. These findings may help school administrators and education policymakers better weigh the tradeoffs of resorting to virtual learning methods in the future.