Guesstimating the Toll of Hurricane Maria

The president went on one of his Twitter tears last week, writing: “3000 people did not die in the two hurricanes that hit Puerto Rico. When I left the Island, AFTER the storm had hit, they had anywhere from 6 to 18 deaths. As time went by it did not go up by much. Then, a long time later, they started to report really large numbers, like 3000…”

Hoo boy. And also:

First things first, it seems clear that a lot of people died. Numerous estimates put the number around 1,000 in the first two months. The Puerto Rican government did initially come up with 64, but it was relying on spotty records; a Harvard survey said 4,645 from September 2017 through the end of the year, but it had a “confidence interval” ranging all the way from 793 to 8,498.

The newest and most comprehensive report, from George Washington University (GWU), is the one that places the number at 2,975, and it’s been accepted as official by the Puerto Rican government. It reached a higher result than most others by looking at a full six-month period and finding that Puerto Rico’s death rate was elevated the whole time, driven especially by increased deaths in poor areas and among older men.

The study is carefully done, though its methods are inherently speculative — and the 69-page report that’s been released describes those methods only cursorily. A more technical version of the study will be released soon, allowing experts to critique it more thoroughly. It’s fair to say the estimate should be taken seriously, but not yet as definitive.

A natural disaster can kill people in a lot of different ways, from disrupting medical services to knocking houses down to causing stress, so it can be difficult to specifically identify every disaster-related death. Instead, especially in areas where record-keeping is shoddy, researchers compare the total number of deaths — disaster-related and not — with data from previous years, aiming to calculate the “excess deaths.” In other words, if you normally have 100 deaths in February, but in the February after a natural disaster you have 150, a halfway-decent guess is that the disaster caused 50 deaths.

This gets more complicated when hundreds of thousands of people leave the area, amounting to 8 percent of the population, as evidently happened with Puerto Rico. If you’re trying to tally all the deaths in the total population, hurricane-related and not, you have to account for the fact that some of the population went elsewhere and some of those folks may also have died.

Essentially, what this study aims to do, using complicated statistical models, is predict how many deaths Puerto Rico would have had without the hurricane, compare that with how many deaths were recorded there, and account for the massive number of people displaced as well.

These are some key numbers, as reported in a Washington Post article by one of the study’s authors:

In September 2017 . . . we found there was an excess of 574 deaths above what would have been expected in a year without the storm. The death toll continued to mount every day, with an excess of 697 deaths in October, 347 in November, 479 in December, 558 in January, and 320 in February, for a total of 2,975.

In other words, the hurricane killed two-thirds as many people in months five and six as it killed in months one and two. This justifies some skepticism, but power wasn’t fully restored on the island for nearly a year, and months five and six happened to be January and February. “Mortality always peaks in winter, and in this case it was probably exaggerated by the lack of resources,” says John Sandberg, an associate professor at GWU’s Milken Institute who played a major role in conducting the analysis. He added, though, that the precise nature of the elevated mortality is something that still needs to be studied more.

Then there’s the adjustment for displacement. The adjustment becomes bigger over time, and it becomes truly huge in the last few months. The final month’s death rate is adjusted all the way from about 6.5 to about 8 per 10,000, an increase of maybe 20 to 25 percent. (See Figure 1 on page 8.) That strikes me as a pretty big adjustment when only 8 percent of the population left.

When I asked Sandberg about the adjustment, he gave me some more details on how it’s made. Essentially, the adjustment grows over time because the number of people who’ve left accumulates, and (using data on airline travel) it accounts for the age and sex of those who left, as well as the socioeconomic status of the municipalities they came from. There will be data on this in the technical report, but he was not able to share it before the report’s release. This is an aspect of the study that deserves careful scrutiny once the precise calculations are made clear.

In addition, Sandberg says, the full report will reconcile the various other estimates that have been made — even the Harvard survey is wholly consistent with the GWU report, not just because of that survey’s wide confidence interval, but also because the main estimate comes down dramatically when the proper technical adjustments (for household size) are made.

Again: Take this study seriously even though the president attacked it, but don’t take it as gospel just because the president attacked it. The estimate comes with a confidence interval running from 2,658 to 3,290; its full methodology is not even public yet — and it’s based on counting death certificates, which the report says were often not filled out correctly, adding another layer of uncertainty.

And no matter what the death toll was, let’s think about how government agencies can do better in the future. The GWU study makes a lot of recommendations for Puerto Rico in that regard; if the president wanted to use it for political purposes, that might have been a better place to start.