The Corner

Health Care

How Irritated Should We Be with IHME Right Now?

Social-distancing decals on the floor of Ralphs Kroger grocery store in Los Angeles after California issued a stay-at-home order in an effort to prevent the spread of the coronavirus, March 30, 2020. (Lucy Nicholson/Reuters)

Much as I did Tuesday, I’d like to pop in with a quick word about the Institute for Health Metrics and Evaluation COVID-19 model heavily relied on by the White House. Once again, Tobias Hoonhout and Andy McCarthy got there first.

My last post explained the first downgrade this week; this one will focus on the second. Yes, it’s highly frustrating to have two sizable revisions, in the same direction, just days apart, but here’s an attempt to piece together what happened and why.

What the model aims to do is draw a curve representing the epidemic. It rises exponentially for a while, then levels off and falls. The model uses existing data from the U.S. to draw the beginning of the curve (i.e., the past), and then guesses what the rest of the curve will look like based on how things have played out in other countries.

Unlike the previous revision this week, the most recent one didn’t come with a detailed explanation. But apparently some new data came in better than expected, which IHME finally got around to stating publicly a day later:

So, why did the model not see that these lower numbers were coming? It has always factored in states’ social-distancing policies — and even assumed that states without such policies would implement them soon — so it’s not a case of these policies’ having their expected effect.

One way to spin it is that social distancing is more effective than anticipated, perhaps because a lot of it happened before governments mandated it:

Another, of course, is that the model was or is just off. That would hardly be surprising for an attempt to predict what will happen in a situation as chaotic and unclear as this one, and there have been some solid criticisms of the way the model works (many of which actually say it’s too optimistic). Here, for example, is a thread laying out how the model can behave in volatile ways when small bits of new information are added:

As I’ve said from the beginning, when I first took note of the effort from Imperial College, any model trying to do what these models do is going to be wrong to some extent. I wouldn’t take IHME’s work as gospel, especially given all the volatility in such a short time. (The model doesn’t take itself as gospel, for that matter, given the huge uncertainty ranges it provides.) At most it can give us an informed guess that slowly gets better as new information comes in. If I were managing a hospital or governing a state, I’d definitely check what it says, but I’d consider it alongside other sources.

A few final points.

First, if this model is wrong, it can be wrong in either direction. I’m not aware of any model that says we should just go ahead and reopen, and the experiences of China, Italy, and New York City would suggest it’s unwise.

Second, and relatedly, the model is only of the first wave of this epidemic. It does not try to predict any resurgence that could happen when measures are lifted, whether that’s now or after the first wave is under control.

Third, when something grows exponentially until it’s squelched and rapidly fades out — when it’s measured by how quickly it doubles or is cut in half — revisions that would seem big in other contexts are nearly unavoidable. 

Lastly, if the epidemic ends up killing the current prediction of 60,000 Americans in just a few months with the lockdowns, what does that say about what would have happened if we’d just let it rip through the population?

Most Popular

Economy & Business

Boiling Over

Andrew Ross Sorkin’s frustration over having missed so much of the post-COVID realities in markets and economic life boiled over this morning in one of the more outrageous outbursts I have ever witnessed on financial media. Perhaps this outburst was rivaled only by his behavior during the March COVID market ... Read More
Economy & Business

Boiling Over

Andrew Ross Sorkin’s frustration over having missed so much of the post-COVID realities in markets and economic life boiled over this morning in one of the more outrageous outbursts I have ever witnessed on financial media. Perhaps this outburst was rivaled only by his behavior during the March COVID market ... Read More

How a U2 Anthem Defined Generation X

In Cameron Crowe’s Say Anything... (1989), Lloyd Dobler sketches out a stumbling, uncertain-but-nevertheless-determined path for his and my generation: “I don't want to sell anything, buy anything, or process anything as a career. I don't want to sell anything bought or processed, or buy anything sold or ... Read More

How a U2 Anthem Defined Generation X

In Cameron Crowe’s Say Anything... (1989), Lloyd Dobler sketches out a stumbling, uncertain-but-nevertheless-determined path for his and my generation: “I don't want to sell anything, buy anything, or process anything as a career. I don't want to sell anything bought or processed, or buy anything sold or ... Read More
White House

The Folly of Twitter’s Fact-Check Policy

No American, not even the president, has an inherent right to a social-media account. Tech companies such as Facebook and Twitter are free to ban any user they see fit. They’re free to fact-check anyone they want, to create a framework of acceptable speech, and to enforce their policies either consistently or ... Read More
White House

The Folly of Twitter’s Fact-Check Policy

No American, not even the president, has an inherent right to a social-media account. Tech companies such as Facebook and Twitter are free to ban any user they see fit. They’re free to fact-check anyone they want, to create a framework of acceptable speech, and to enforce their policies either consistently or ... Read More
Film & TV

AT&T Bungles HBO Max Rollout

Well, the big day is here for HBO Max. Or . . . not. The WarnerMedia unit of AT&T decided that it couldn’t compete with Netflix (63 million U.S. subscribers) unless it massively upsized HBO (35 million) with billions of dollars' worth of new programming and more library titles. So Warner bought back the ... Read More
Film & TV

AT&T Bungles HBO Max Rollout

Well, the big day is here for HBO Max. Or . . . not. The WarnerMedia unit of AT&T decided that it couldn’t compete with Netflix (63 million U.S. subscribers) unless it massively upsized HBO (35 million) with billions of dollars' worth of new programming and more library titles. So Warner bought back the ... Read More