Critical Condition

Critically Reading Polls

More from Kellyanne Conway (president & CEO of the polling company™, inc./WomanTrend) on that Washington Post poll and polls of its kind: 

This week’s Washington Post/ABC News poll is just the latest example of a survey under-representing Republicans and overstating increased public “opinion” on a matter about which the public likely does not know enough to have formed true opinions.  This is agenda-driven polling in that it uses surveys to make public opinion rather than to reflect it.  

The paper’s headline screams: “Public Option Gains Support: Clear Majority Now Backs Plan.” But, a look at the Post’s own numbers, question wording, and methodology seems this to make this one of those conclusions desperately in search of evidence.

When the media conduct and publish polls, there is far less accountability than when private pollsters do.  Unlike private pollsters, the media have no clients, so their existence as “pollsters” is immune to the usual free market forces of hiring and firing based on actual performance and legitimacy.  It is common for private pollsters to be cast as “Republican” or “Democrat,” a tacit insinuation that such professionals would rather appease a client than perform honest work.

But the media polls operate under no such scrutiny. This even though their editorial pages and cable hosts – including The Washington Post – routinely endorse candidates and buck up public policies about which the public may know little (because the media that should be reported on it is instead “polling” about it) but that reporters and editorial board members of the paper themselves support.  Public option is the latest example of this.

There are two major problems: one is methodological, the second is substantive. 

Methodologically, the topline data released by the Post failed to list basic information like gender distribution and regional, age, and racial breakdowns.  This isn’t to say the sample isn’t representative, but the full battery of central demographic questions needs to be made known to assess the representativeness and legitimacy of the poll.  My own firm has faced such routine inquiries from Washington Post reporters over the years when releasing polling data so we hope the request is viewed in the same spirit.

Perhaps the most dramatic and telling aspect of the little demographic information released by the poll was in the party identification:  33% Democrat vs. 20% Republican vs. 42% Independent.  This would mean only one-fifth of the sample belongs to one of the two major political parties in our country. 

Even when you adjust the “leaners,” the party identification in this poll amounts to 53% Democrat vs. 39% Republican, counter to most other polling data:  48% Democrat vs. 42% Republican (Gallup) 38% Democrat vs. 32% Republican (the polling company™, inc./WomanTrend ). 

The party identification in this survey is even worse than what The New York Times/CBS News poll routinely does to under-represent Republicans by a good six to seven points.  In the most recent NYT/CBS poll, the sample naturally fell out to be 27% Republican, 32% Democrat, and 41% Independent.  Employing their own idea as to what the nationwide share of party identification actually is, the sample is morphed to be 22% Republican, 33% Democrat, and 45% Independent.  In consecutive polls taken this summer, the NYT/CBS poll weighted down Republicans to under 25%, going as low as 20% in July.  Only 20% Republicans nationwide. They wish. Seems like each one showed up at a tea party or townhall meeting.

Getting to the substantive part, this Washington Post/ABC News poll presumes a body of knowledge among the public that it is likely wanting and doesn’t exist.  In two other separate polls, Americans have made known their confusion.  Sixty percent say Obama has not clearly explained his plans for healthcare reform (CBS, 8/09) while separately 59% admit they do not “understand most of the major points in Obama’ healthcare reform” (CNN, 8/09).  That makes sense since few would argue that President Obama has stumbled badly in trying to communicate the specifics of his own healthcare plan and, ironically, in trying to convince folks in Congress that meaningful healthcare reform can exist without the public option. It’s why his approval on handling of healthcare has dropped. 

Asking an under-informed public in a poll about “public option” is incomplete. It calls for a response to feel-good phraseology rather than a probing of underlying ideology. “Public option” in health care is not so different from “campaign finance reform,” “Violence Against Women’s Act,” “revenue enhancements” or for that matter, “world peace’ and “no rain this Saturday.”

Certainly the “public option” sounds great when it’s described as “a new health insurance plan to compete with private health insurance plans” (as it is in the Washington Post poll).  If only healthcare reform were as simple as that; then maybe the Democrat President and Democratically-controlled Congress could pass it.  But, the fact is they’re arguing about public option internally.  That’s why their friends in high places, perhaps including newspapers’ polling units, are trying to help.   

Additionally, this very simply-worded question foregoes any mention of cost.  What about the costs to taxpayers?  What about the costs to insurance companies?  Insurance companies will pass along any new taxes they are forced to pay to their consumers, making the costs of coverage even higher, and then leaving consumers with no other “options” but to abandon their private plans and enroll in the public option.  Many employers will follow suit and cut the private coverage they offer for their employees. The Washington Post fails to mention any of these possibilities, consequences, and realities. 

What’s more, following the initial question on the public option, the Washington Post/ABC News poll probes respondents who opposed, but did not bother asking a follow-up question to those who supported it.  The poll asked: “What if this government-sponsored plan was run by state governments and was available only to people who did not have a choice of affordable private insurance? In that case would you support or oppose this idea?”  This caused support for the public “option” to increase.  No doubt probing initial supporters of the plan on the costs and consequences would have caused them to reconsider.  Other surveys, including the well-respected Kaiser Family Foundation’s monthly tracking poll, are unafraid to ask these important and consequential questions about true costs (tangible and intangible). 

When questions about costs are actually put to the test, it is evident that Americans can do the math.  The public “gets it” and understands that cost-shifting becomes their burden.  Sixty-one percent of respondents in this survey opposed taxing the most expensive health insurance policies, even with presented with the do-good opinion in support of such levies (“Supporters say this would help pay for health care reform, and encourage insurers to offer cheaper policies”).  They know that higher costs to insurance companies become their (or their employers’) problems. 

But it’s not just their own pocketbooks over which they express concern.   Americans are worried about the burdens being put upon future generations.  When asked “Just your best guess, do you think health care reform would increase the federal budget deficit, decrease it, or have no effect? (IF INCREASE) Do you think that would be worth it, or not?”,  68% think it will increase the deficit and of those, 37% say it is not worth it, 31% say it is worth it; 10% think it will decrease the deficit.

It is worth noting that the graphic in the print edition of today’s Washington Post highlights how 68% think that healthcare reform would increase the deficit.  Underneath, a call-out box shows: “Of those who said it would increase it, nearly half said it would be worth the cost (45%).”  First, someone needs to recheck the calculator: 31% divided by 68% is 45.6%, which rounds to 46%, but is still not one-half.  What is one-half (and then some) is the point that truly should have been highlighted: the fact that among those who said healthcare would increase the deficit, the majority (54%) said it would not be “worth it.  “

The bogusness of this poll (and others showing “support” for the public option) is also proved by President Obama himself, who is lukewarm on inclusion of public option and has suggested they would move forward with or without it.  In fact, he said in his address to the Joint Session of Congress last month that “the public option is only a means to that end — and we should remain open to other ideas that accomplish our ultimate goal.”  He must realize that honest polling shows no such clamor for a public option. Opponents should take heart since the real argument about public option exists almost exclusively among the Left: unions, Congressional Black Caucus, and even Nancy Pelosi.  To exclude or dilute the public option will likely cost them precious political currency among an increasingly impatient base who thought this would be included in “hope” and “change.”

President Obama also may know something that he did not realize earlier this summer: Most Americans look at healthcare through an economic prism and so one of his greatest vulnerabilities right now is his handling of the deficit.  According to many polls, he’s not trusted on it.

The positive-sounding “public option” should be exposed for what it truly is, by media pollsters and by anyone who truly cares about healthcare reform: a government grab of one of the most intimate matters people face, and one about which most of them are presently content.  Additionally, “private” is the opposite of public, and privacy is an important word in the overall healthcare lexicon.

No one benefits when poorly-gathered information is at the root of any message or strategy.   An issue as complex as healthcare cannot (and should not) be boiled down to a single headline or a single polling question. Those charged with informing the public truthfully about “public option” should not substitute public “opinion” for facts.

Washington Post Topline:

Gallup (Ideology):

Gallup (Party ID):

CBS/NY Times Topline:

The Latest