The Corner

Health Care

How Irritated Should We Be with IHME Right Now?

Social-distancing decals on the floor of Ralphs Kroger grocery store in Los Angeles after California issued a stay-at-home order in an effort to prevent the spread of the coronavirus, March 30, 2020. (Lucy Nicholson/Reuters)

Much as I did Tuesday, I’d like to pop in with a quick word about the Institute for Health Metrics and Evaluation COVID-19 model heavily relied on by the White House. Once again, Tobias Hoonhout and Andy McCarthy got there first.

My last post explained the first downgrade this week; this one will focus on the second. Yes, it’s highly frustrating to have two sizable revisions, in the same direction, just days apart, but here’s an attempt to piece together what happened and why.

What the model aims to do is draw a curve representing the epidemic. It rises exponentially for a while, then levels off and falls. The model uses existing data from the U.S. to draw the beginning of the curve (i.e., the past), and then guesses what the rest of the curve will look like based on how things have played out in other countries.

Unlike the previous revision this week, the most recent one didn’t come with a detailed explanation. But apparently some new data came in better than expected, which IHME finally got around to stating publicly a day later:

So, why did the model not see that these lower numbers were coming? It has always factored in states’ social-distancing policies — and even assumed that states without such policies would implement them soon — so it’s not a case of these policies’ having their expected effect.

One way to spin it is that social distancing is more effective than anticipated, perhaps because a lot of it happened before governments mandated it:

Another, of course, is that the model was or is just off. That would hardly be surprising for an attempt to predict what will happen in a situation as chaotic and unclear as this one, and there have been some solid criticisms of the way the model works (many of which actually say it’s too optimistic). Here, for example, is a thread laying out how the model can behave in volatile ways when small bits of new information are added:

As I’ve said from the beginning, when I first took note of the effort from Imperial College, any model trying to do what these models do is going to be wrong to some extent. I wouldn’t take IHME’s work as gospel, especially given all the volatility in such a short time. (The model doesn’t take itself as gospel, for that matter, given the huge uncertainty ranges it provides.) At most it can give us an informed guess that slowly gets better as new information comes in. If I were managing a hospital or governing a state, I’d definitely check what it says, but I’d consider it alongside other sources.

A few final points.

First, if this model is wrong, it can be wrong in either direction. I’m not aware of any model that says we should just go ahead and reopen, and the experiences of China, Italy, and New York City would suggest it’s unwise.

Second, and relatedly, the model is only of the first wave of this epidemic. It does not try to predict any resurgence that could happen when measures are lifted, whether that’s now or after the first wave is under control.

Third, when something grows exponentially until it’s squelched and rapidly fades out — when it’s measured by how quickly it doubles or is cut in half — revisions that would seem big in other contexts are nearly unavoidable. 

Lastly, if the epidemic ends up killing the current prediction of 60,000 Americans in just a few months with the lockdowns, what does that say about what would have happened if we’d just let it rip through the population?

Most Popular

Elections

Trump in Trouble

President Trump was disappointed. Bad weather on Wednesday forced a delay in SpaceX's planned launch of the Dragon spacecraft, robbing the president of a prized photo opportunity. He plans to attend the next launch, scheduled for May 30 at 3:22 p.m. EDT, but the spoiled visit to Florida punctuated another week of ... Read More
Elections

Trump in Trouble

President Trump was disappointed. Bad weather on Wednesday forced a delay in SpaceX's planned launch of the Dragon spacecraft, robbing the president of a prized photo opportunity. He plans to attend the next launch, scheduled for May 30 at 3:22 p.m. EDT, but the spoiled visit to Florida punctuated another week of ... Read More
U.S.

First, Restore Order

Doing evil in the service of a just cause does not change either side of the moral equation: Evil remains evil, and the just cause remains just — neither consideration cancels out the other or transmutes it. With riots and violence convulsing American cities after the horrifying death of George Floyd at the ... Read More
U.S.

First, Restore Order

Doing evil in the service of a just cause does not change either side of the moral equation: Evil remains evil, and the just cause remains just — neither consideration cancels out the other or transmutes it. With riots and violence convulsing American cities after the horrifying death of George Floyd at the ... Read More
Politics & Policy

Is It Revolution?

I knew I was tempting fate a week ago when I said that the coming nomination of Joe Biden and the COVID-19 pandemic had put America’s politics on chill during this election year. Little did I know that days later we’d be making analogies to 1968. The killing of George Floyd by a Minneapolis policeman moved ... Read More
Politics & Policy

Is It Revolution?

I knew I was tempting fate a week ago when I said that the coming nomination of Joe Biden and the COVID-19 pandemic had put America’s politics on chill during this election year. Little did I know that days later we’d be making analogies to 1968. The killing of George Floyd by a Minneapolis policeman moved ... Read More
PC Culture

For Looters, Looting Is Fun

One important thing to realize about looting is that it's usually enjoyable for those engaged in it, who exult in the momentary suspension of any rules. Just a couple of examples from the last couple of days (language ... Read More
PC Culture

For Looters, Looting Is Fun

One important thing to realize about looting is that it's usually enjoyable for those engaged in it, who exult in the momentary suspension of any rules. Just a couple of examples from the last couple of days (language ... Read More