March 8, 2021
Flexible decision-making and short-term forecasts
How valuable are long-run predictions?
Trismegist8
Historically, humans haven't been much better than groundhogs at making long-term weather predictions. But the shift to scientific, model-driven meteorology has changed things.
Five-day weather forecasts now are as accurate as one-day forecasts were in 1980. Modern 72-hour predictions of hurricane tracks are more accurate than 24-hour forecasts were 40 years ago. No one knows how far meteorologists can accurately push their forecasts, but they are trying.
It’s a hopeful story of scientific advancement, which researchers in numerous fields—from economics to infectious diseases—are looking to replicate. And many policymakers are backing modelers with substantial amounts of resources.
However, it’s not clear to economist Antony Millner that pushing scientific models further and further into the future is always worthwhile.
“In the back of my mind, I had this constant feeling that we're not good at forecasting the long run,” Millner told the AEA in an interview. “But that doesn't mean that we should be putting effort into improving long-term predictions, because we need to filter those metrics of forecast skill through people's decision processes.”
In a paper in the February issue of the American Economic Journal: Microeconomics, Millner and his coauthor Daniel Heyen laid out a model that can help decision-makers think through whether, and when, long-run predictions are important.
Their work has important lessons for many areas of forecasting research and long-run decision-making.
Think about what you're likely to know in the future when you make those decisions and keep conscious of the fact that you can always adjust.
Antony Millner
Some of the earliest efforts at forecasting have been about the weather. Understanding seasonal weather variation is essential for societies that rely on farming.
For instance, farmers need to know what crops to plant at the beginning of the growing season because they can’t change their crops if the weather doesn’t cooperate in a couple months. They have one shot at getting it right, so they need reliable information to make the best decision.
In contrast, Millner and Heyen noticed that many of the problems facing today’s decision-makers seeking to adapt to long-run changes are not like this.
Instead, planners are often able to make adjustments to their decisions repeatedly over time, and have access to a constant flow of new information when doing so. Short-run forecasts play an important role in informing adaptation strategies.
The researchers showed in a simplified model that the relative importance of short-run and long-run predictions depends crucially on how costly it is to make adjustments to one’s decisions. If relatively cheap adjustments can be made, accurate short-run predictions are much more valuable than long-run predictions. This may still be true even if adjustment is costly.
An example: long-term budgeting
Governemts try to budget for the long run. But the chart below shows the challenge that the CBO faces when trying to give the Congress long-run forecasts. It shows projected US debt levels as a percentage of GDP. The CBO knows that complex factors like mortality, productivy, interest rates, and healthcare spending could have huge impacts on deficits, hence the growing gap between the two pink-shaded lines.
One area where this trade-off may be especially important is in adapting to climate change.
While climate models provide high-quality information about long-run changes in the global climate, predictions at the local level are much less certain. Nevertheless, the authors’ research suggests that relatively short-run forecasts could still be valuable to local policymakers. In other words, accurate long-run predictions may not always be necessary to successfully adapt to climate change.
Overall, the authors’ work indicates that, in some cases, policymakers might be better off improving short-term predictions with better data than pushing models further into the future.
“Think about what information would be useful for you to acquire; don't just think about what you don't know,” Millner said. “Think about what you're likely to know in the future when you make those decisions and keep conscious of the fact that you can always adjust.”
♦
“Prediction: The Long and the Short of It” appears in the February issue of the American Economic Journal: Microeconomics. Subscribe to the monthly Research Highlight email digest here.