Earlier this week I gingerly walked through some of the land mines one encounters while exploring COVID-19 data. Here I’d like to add that the data on the flu have similar problems, as explained in John McCormack’s excellent new magazine piece, as well as an article over at Scientific American.
The numbers that have anchored a lot of the discussion so far — such as that the flu has a fatality rate of 0.1 percent and kills 60,000 or so in a bad season — are not as firmly established as one would hope.
As McCormack notes, drawing on an earlier piece from Justin Fox, the 0.1 percent figure does not account for asymptomatic infections, so it can’t be compared with COVID-19 numbers that do. The adjusted number for the flu is probably somewhere around 0.04 percent.
Meanwhile, the Scientific American piece starts with a simple observation from the author: If the flu kills tens of thousands of people a year, he should have seen plenty of these deaths “in four years of emergency medicine residency and over three and a half years as an attending physician” — just as he’d seen other causes of death with similar tolls, such as gun violence and opioid overdoses. But he “could only remember one tragic pediatric case.” He called around the country to ask colleagues about it, and hardly any of them saw flu deaths in significant numbers, either.
Turns out those common statistics are the result of some guesswork:
The 25,000 to 69,000 numbers that Trump cited do not represent counted flu deaths per year; they are estimates that the CDC produces by multiplying the number of flu death counts reported by various coefficients produced through complicated algorithms. . . .
In the last six flu seasons, the CDC’s reported number of actual confirmed flu deaths — that is, counting flu deaths the way we are currently counting deaths from the coronavirus — has ranged from 3,448 to 15,620, . . . far lower than the numbers commonly repeated by public officials and even public health experts. . . . In the fine print, the CDC’s flu numbers also include pneumonia deaths.
First, we adjusted the reported annual hospitalization rates from FluSurv-NET [a system that collects flu data from a small subset of U.S. health-care providers] . . . using multipliers that included the probability of being tested for influenza and the sensitivity of influenza testing. . . . Rates of influenza mortality were calculated by multiplying the adjusted rates of hospitalization by the ratio of deaths to hospitalizations.
That “ratio of deaths to hospitalizations,” in turn, comes from massaging still other numbers:
Not all persons who die with influenza are admitted to a hospital prior to their death, and others may die after hospital discharge, thus hospital surveillance does not fully capture deaths due to influenza in the catchment area. To estimate a more complete ratio of deaths to hospitalizations, we also included data on the probability that a person with a respiratory infection would die outside of a hospital admission. For this we used publically available mortality data from the National Center for Health Statistics for the U.S. population in 2010 to identify the deaths attributable to pneumonia and influenza (ICD-10 codes: J10-J18) and the proportion that occurred while hospitalized vs. outside of a hospital admission (e.g., at home, on arrival, in the emergency department, in hospice or long-term care facility).
Or, if you prefer a flow chart:
So, yeah. These are estimates, not hard facts, just like most of the numbers you’ll see regarding COVID-19.