Why Did Romneycare Save Lives? (If It Did)

by Callie Gable

Last week, Annals of Internal Medicine published the results of a groundbreaking study: When people get health insurance, they’re less likely to die. You’re probably wondering why it took a Harvard economist to figure that out, but the relationship between health insurance and mortality is less obvious than it seems.

The authors of the study conclude Massachusetts’s 2006 health reform was associated with a 4.5 percent decrease in deaths preventable by health care, and that for every 830 people who gain insurance, there’s one fewer death per year.

The study compared pre- and post-health-care-reform mortality rates of nonelderly adults in Massachusetts to mortality rates in control groups in non-reform states. Of course, since this isn’t a randomized control trial, all we have here is correlation, and a sensible causation (that health insurance saves lives). Nonetheless, the findings were significant, represented in the chart below:

Health-amenable mortality in Massachusetts begins a slight decline relative to the control group shortly after the 2006 reforms, while other-cause mortality does not change significantly.

Assuming the change isn’t due to other factors, this would seem to call into question the results of the widely publicized Oregon Health Insurance Experiment, which applied the “gold standard” method of research, a randomized trial, to the subject by comparing health outcomes of a group that received Medicaid coverage with a control group that did not receive coverage.

While OHIE authors concluded that Medicaid recipients used more health care — including more care that has been “linked to improved outcomes” — they also concluded that the effects of increased utilization were unclear:

This randomized, controlled study showed that Medicaid coverage generated no significant improvements in measured physical health outcomes in the first 2 years, but it did increase use of health care services, raise rates of diabetes detection and management, lower rates of depression, and reduce financial strain.

The study also concluded that Medicaid coverage had no significant impact on mortality. So is insurance coverage good for health in Massachusetts and useless in Oregon? Probably not.

Why? The two studies are not measuring quite the same thing. Looking at the type of insurance participants gained in each study may better explain the discrepancy.

The vast majority of newly insured individuals in Massachusetts purchased private insurance, while less than a quarter gained coverage through expansion of public programs:

Conversely, all individuals in the OHIE gained coverage via Medicaid. It would be interesting (and much more useful) to see a study comparing the outcomes of individuals in Massachusetts who gained coverage via private insurance versus Chapter 58 public-insurance options.

Overall, it seems that Massachusetts’s primarily privately driven insurance expansion was associated with great benefits while new Medicaid recipients in Oregon received marginal benefits, most of which weren’t direct health improvements — that seems to be the most obvious answer for the discrepancy.

There are a number of other issues with the comparison, though: Both studies have big flaws.

For one, the Massachusetts study was not a randomized control trial, so the results suffer from potential confounding variables, a fact that the authors note:

Our quasi-experimental approach cannot definitively demonstrate a causal relationship underlying the association between the Massachusetts reform and the state’s declining mortality relative to other states . . . it is critical to note the many dimensions in which Massachusetts differs from the rest of the nation, including lower mortality, higher income and baseline insurance coverage rates, fewer minorities, and the most per capita physicians in the country. The extent to which our results generalize to the United States as a whole is therefore unclear.

The OHIE was an RCT, but the study was “underpowered” due to its small sample size. The confidence intervals for many of the results are extremely large in comparison to the Massachusetts study’s confidence intervals. Most simply: Real effects that will be detected in larger samples (like in the Massachusetts study) will be missed in small sample sizes (like in the Oregon study), so the Oregon study is more likely to find no effects from the expansion than the Massachusetts study was.

There are other issues: As Chris Conover has noted below, Massachusetts’s law also did other things to the states’ health-care market, such as mandating more comprehensive benefits — it’s unclear how much of the decrease in mortality should be attributed to the increase in insurance coverage. Megan McArdle, and Avik Roy have more on other issues with the two studies.

Despite these limitations, it’s safe to say the Massachusetts results add to a growing body of evidence that confirms something we already knew: Private health insurance is probably good for health. But the contrast between the two studies may indicate that Medicaid expansion is not the optimal way to provide universal health care if you are concerned with improving outcomes — and is the extraordinarily expensive task of expanding access to health care worth it if you don’t improve outcomes?

If indeed there is such a large gap between the benefits of Medicaid and private insurance, that has big implications for Obamacare: both for states deciding whether to use federal funds to expand the program, and for the law in general, which may be covering more new enrollees through Medicaid than private insurance. More on this debate soon.