NRPLUS MEMBER ARTICLE W e are going to be arguing about the COVID-19 lockdowns for a long time. It’s the kind of topic that gives academics and pundits alike lots of variables to play with. They can reach whatever conclusion they want — and they’re bound to take advantage of the situation, because extraordinary claims require extraordinary . . . media coverage.
But as the lockdowns end, I wanted to offer a brief overview of the work that’s already been done, specifically on the question of whether lockdowns reduce the toll of the virus. Of course, this is just one piece of the puzzle, because lockdowns also harm the economy, damage mental health, and sometimes discourage important medical care. It will probably be years before we even begin to put all that together.
On the narrow question of whether lockdowns control COVID-19, though, I rounded up everything I was aware of, including findings I very much doubt. And I did my best to dredge up more work with Internet searches, though there is so much of this stuff that the studies cited below cannot possibly be all of it.
My own read of the evidence is that government restrictions make a difference — but that voluntary social distancing rooted in fear of the virus does a lot of the work too. Which isn’t that surprising when you think about it, because, at least in democratic countries, it’s very hard to lock down unless the public wants to. By the time U.S. states told everyone to stay home, their restaurant traffic (as measured by OpenTable) had already fallen by more than half.
I know a lot of readers are going to go hunting for studies that support whatever they already think, so I’ve organized the research according to the conclusions it reached, from pro-lockdown to anti-lockdown. It’s the scientific method!
Lockdowns Work in the U.S.
“Strong Social Distancing Measures In The United States Reduced The COVID-19 Growth Rate.” This is probably the most spectacular claim on the pro-lockdown side: “Holding the amount of voluntary social distancing constant, [our] results imply 10 times greater spread by April 27 without [stay-at-home orders] (10 million cases) and more than 35 times greater spread without any of the four measures (35 million).”
“Differential Effects of Intervention Timing on COVID-19 Spread in the United States.” This is the study recently trumpeted by the New York Times, which finds that locking down a week earlier could have reduced the death toll by more than half.
“When Do Shelter-in-Place Orders Fight COVID-19 Best?” Lockdowns make people a bit more likely to stay home full-time, and “approximately three weeks following the adoption of [an order], cumulative COVID-19 cases fell by 44 percent.” The benefits were biggest for states that implemented their policies early or had high population density.
“Comparison of Estimated Rates of Coronavirus Disease 2019 (COVID-19) in Border Counties in Iowa Without a Stay-at-Home Order and Border Counties in Illinois With a Stay-at-Home Order.” This paper compares bordering counties in Iowa and Illinois. The trends were similar before Illinois issued a stay-at-home order, but “after that, cases increased more quickly in Iowa and more slowly in Illinois.” The difference represents “30.4% of the 716 total cases in those Iowa counties by that date.”
“Price of Delay in COVID-19 Lockdowns.” These researchers looked for “natural experiments” where “similar neighboring states” pursued different policies. “Those states that locked down late suffered a 15% to 25% higher penetration of the disease, per week late in date of shutdown, as of May 1st 2020.”
“The Effect of Stay-at-Home Orders on COVID-19 Cases and Fatalities in the United States.” Stay-at-home orders “are associated with a 30.2 percent (11.0 to 45.2) reduction in weekly cases after one week, a 40.0 percent (23.4 to 53.0) reduction after two weeks, and a 48.6 percent (31.1 to 61.7) reduction after three weeks. Stay-at-home orders are also associated with a 59.8 percent (18.3 to 80.2) reduction in weekly fatalities after three weeks.”
“Impact of policy interventions and social distancing on SARS-CoV-2 transmission in the United States.” States that had a stay-at-home order at the time they saw their 500th case tended to have slower spread thereafter.
“Causal Estimation of Stay-at-Home Orders on SARS-CoV-2 Transmission.” This study finds that the orders reduced mobility, and in turn estimates “that reductions in movement have causally reduced SARS-CoV-2 transmission rates by 49%.”
“Effect of Mitigation Measures on the Spreading of COVID-19 in Hard-Hit States.” This paper looks at New York, New Jersey, Michigan, Massachusetts, Illinois, and Louisiana, finding that their overall mitigation efforts worked to slow the virus.
“Stay at Home to Stay Safe: Effectiveness of Stay-at-Home Orders in Containing the COVID-19 Pandemic.” “We use eight states that did not implement the order as a control group and six neighbor states that implemented the order as a treatment group to estimate the effect of the stay-at-home order. We find that, though residents in both groups have already voluntarily stayed at home, the order reduces the number of new COVID-19 cases by 7.6%.”
Lockdowns Work in Other Countries
“Do Lockdown and Testing Help in Curbing Covid-19 Transmission?” Looking at 69 countries, this paper finds that the use of shutdown measures, in the aggregate, “significantly affects the number of confirmed cases after 7 days.”
“Accounting for Global COVID-19 Diffusion Patterns, January-April 2020.” “With a lag, more stringent pandemic policies were associated with significantly lower mortality growth rates.”
“COVID-19: Government Interventions and the Economy.” These researchers say that lockdowns are effective, though “countries may avoid lockdown by imposing school closures, mask wearing and centralised quarantine simultaneously to reach similar COVID-19 infection mitigation outcomes.”
“Shut It Down: A Cross Country Panel Analysis on the Efficacy of Lockdown Measures.” “Our results confirm the efficacy of such measures, and that the average time to have effects in terms of a reduction of cases is of about ten days.”
“Estimating the Number of Infections and the Impact of Non-Pharmaceutical Interventions on COVID-19 in 11 European Countries.” European efforts to control COVID worked, per this paper, but the estimates are imprecise and the methods “assume that changes in the reproductive number — a measure of transmission — are an immediate response to these interventions being implemented rather than broader gradual changes in behaviour.”
“Analysis of the Impact of Lockdown on the Evolution [of] Covid-19 Epidemics in Spain.” “The estimated number of cases shows a sharp increase until the lockdown, followed by a slowing down and then a decrease after full quarantine was implemented. Differences in the basic reproduction ratio are also very significant, dropping from 5.89 . . . to 0.48.”
“Measuring the Impact of the German Public Shutdown on the Spread of Covid-19.” This paper finds “a trend break on 20 March that is in line with the expected lagged impact of the German policies and which reduces the growth rate by 48.2%.”
“One-Month Impact of the French Lockdown on the Epidemic Burden.” France’s lockdown “prevented 587,730 hospitalizations and 140,320 ICU admissions at the national level. The total number of ICU beds required to treat patients in critical conditions would have been 104,550, far higher than the maximum French ICU capacity. This first month of lockdown also permitted [France] to avoid 61,739 hospital deaths, corresponding to a 83.5% reduction of the total number of predicted deaths.”
“The Impact of a Nation-wide Lockdown on COVID-19 Transmissibility in Italy.” Key finding: “Fourteen days after the implementation of the strategy, the net reproduction number has dropped below the epidemic threshold.”
“The Human Cost of Delaying Lockdown.” This modeling finds that an additional week of lockdown could have cut the toll by 30,000 in the U.K. and was waved about by the Telegraph despite being just a blog post.
“Analysis of 25,000 Lab-Confirmed COVID-19 Cases in Wuhan.” This presentation finds that a lockdown helped to control the initial outbreak, but that further, harsher quarantine measures were needed to squelch it.
It Depends What the Meaning of ‘Lockdown’ Is
“The Estimated Impact of Non-Pharmaceutical Interventions on Documented Cases of COVID-19: A Cross-Country Analysis.” This study finds little to no impact for “lockdowns” — i.e., stay-at-home orders — and school closings, but does find a big impact for shutting down non-essential businesses, banning gatherings and events, and closing borders.
“Impact of Non-Pharmaceutical Interventions against COVID-19 in Europe.” This one finds an effect from school closures and certain business shutdowns, but “stay at home orders, closure of all non-[essential] businesses and requiring the wearing of facemasks or coverings in public was not associated with any independent additional impact.”
“Health vs. Wealth?” In an international sample, this study finds evidence that restricting workplaces, public transport, and travel reduces case growth, but the results are less clear when it comes to deaths. The authors also analyze U.S. states and find that stay-at-home orders reduce case growth.
“Social Distancing is Effective at Mitigating COVID-19 Transmission in the United States.” This study finds that social distancing works, but stresses that a lot of it happened without government mandates.
Lockdowns Don’t Work
“Do Lockdowns Work? A Counterfactual for Sweden.” This study compares Sweden with a “synthetic” version of Sweden, a mix of other countries that had similar trends pre-lockdown: 39 percent Netherlands, 26 percent Denmark, 19 percent Finland, 15 percent Norway, and 1 percent Portugal. The upshot is that the real Sweden had no more deaths than the fake Sweden. The real Sweden saw a lot of voluntary social distancing, but its mobility rates didn’t fall quite as much as the fake Sweden’s did.
“Tracking Public and Private Responses to the COVID-19 Epidemic.” This paper focuses on whether lockdowns reduce mobility, not whether they have a concrete effect on virus transmission, but finds “little evidence” that “stay-at-home mandates induced distancing.” More effective were “first case announcements, emergency declarations, and school closures.” (Elsewhere on this list I’ve avoided studies that focus on mobility without tying it explicitly to COVID-19 outcomes, but see here and here for contrary takes and here for a supportive one.)
“The End of Exponential Growth.” This brief study from last month argues that the exponential growth of COVID-19 burns out quickly no matter what policies countries pursue, and that “lockdown policy can be stopped within a few days and replaced by a policy of moderate social distancing.”
“Lockdowns Don’t Work.” This is an online article for the public, not an academic study, but it’s written by the economist Lyman Stone and reports some of his own modeling efforts. His results suggest that school closings and bans on large gatherings work, but stay-at-home orders do not.
“There Is No Empirical Evidence for These Lockdowns.” Like the Stone piece, this is an informal article that reports some regressions. “The impact of state-response strategy on both my cases and deaths measures was utterly insignificant.”
“Full Lockdown Policies in Western Europe Countries Have No Evident Impacts on the COVID-19 Epidemic.” This one looks at “full lockdown strategies applied in Italy, France, Spain and United Kingdom” and finds “no evidence of any discontinuity in the growth rate, doubling time, and reproduction number trends.”
Finally, Carl Quintanilla of CNBC recently circulated an analysis from JPMorgan pointing out that the states that have reopened haven’t seen the virus take off. This could imply that the lockdowns never did anything, that these states’ replacement policies were adequate, or simply that a spike is still to come.
Choose your fighters, everyone! The battle of the statistics has only just begun.