“Oh, people can come up with statistics to prove anything, Kent. Forty percent of all people know that.” — Homer Simpson.
In the days before the first debate in Denver, President Obama held more than a four-point lead in the Real Clear Politics average, and Romney had been left for dead by most of the media. Then the debate came, and overnight Romney seemingly rid himself of the weaknesses that had been tacked on to him by over $100 million dollars in negative advertising. Now here we are a few weeks later with a dead heat in nationwide polls.
As worry built up among Democrats that Romney had tied the race nationally and had clear momentum heading into the final stretch, they began attaching their hopes to what BuzzFeed’s Ben Smith called “the bulwark against all-out Dem panic” — Nate Silver.
Silver gained fame by correctly predicting 49 of 50 states in the 2008 election using a statistical model that assigns weight to the various polls based on a number of factors. After the 2008 election, Silver partnered with the New York Times, and he has been quoted by many media outlets as the gold standard for predicting what will happen in November.
Some note that 2008 was a wave election, where the enthusiasm and underlying fundamentals were so favorable to Obama that the outcome was easy to foresee, with the exception of a few of the GOP-turned-Democratic states such as Indiana and North Carolina where Obama won a razor-thin victory. Others argue that Silver’s access to the Obama administration’s internal polling gave him information that most other analysts never saw, which allowed him to make more adjustments to his model and increase his accuracy.
Whatever the explanation, Silver’s strong showing in the 2008 election, coupled with his consistent predictions that Obama will win in November, has given Democrats a reason for optimism. While there is nothing wrong with trying to make sense of the polls, it should be noted that Nate Silver is openly rooting for Obama, and it shows in the way he forecasts the election.
On September 30, leading into the debates, Silver gave Obama an 85 percent chance and predicted an Electoral College count of 320–218. Today, the margins have narrowed — but Silver still gives Obama a 67 percent chance and an Electoral College lead of 288–250, which has led many to wonder if he has observed the same movement to Romney over the past three weeks as everyone else has. Given the fact that an incumbent president is stuck at 47 percent nationwide, the odds might not be in Obama’s favor, and they certainly aren’t in his favor by a 67–33 margin.
The main reason that Silver feels Obama is still an overwhelming favorite is that while Romney has surged in the polls to tie (or lead) Obama nationally, the challenger is still, in Silver’s opinion, a long shot to pull together enough battleground states to get to 270 electoral votes. This is the real problem with Silver’s model in the eyes of many Romney backers — the “weighting” he puts into state polls gives an edge to Obama, and the distribution of that weighting is highly subjective. For example, Silver currently gives Obama a 70 percent chance of winning Ohio. A component of this is a weighted “polling average” of Obama’s support at 48.2 percent to Romney’s at 45.2. The current Real Clear Politics average is nearly a full point more favorable to Romney: It has Obama at 48.1 and Romney at 46.0. The difference comes from the fact that Real Clear Politics gives equal weight to all of the polls it includes and uses only the most recent polls from each polling organization in a given timeframe.
While many in the media (and Silver himself) openly mock the idea of Republicans’ “unskewing polls” (and I am not a fan of unskewedpolls.com by any means), Silver’s weighting method is just a more subtle way of doing just that. I outlined yesterday why Ohio is closer than the polls seem to indicate by looking at the full results of the polls as opposed to only the topline head-to-head numbers. Romney is up by well over eight points among independents in an average of current Ohio polls, the overall sample of those same polls is more Democratic than the 2008 electorate was, and Obama’s two best recent polls are among the oldest.
But look at some of the weights applied to the individual polls in Silver’s model. The most current Public Policy Polling survey, released Saturday, has Obama up only one point, 49–48. That poll is given a weighting under Silver’s model of .95201. The PPP poll taken last weekend had Obama up five, 51–46. This poll is a week older but has a weighting of 1.15569.
The NBC/Marist Ohio poll conducted twelve days ago has a higher weighting attached to it (1.31395) than eight of the nine polls taken since. The poll from twelve days ago also, coincidentally enough, is Obama’s best recent poll in Ohio, because of a Democratic party-identification advantage of eleven points. By contrast, the Rasmussen poll from eight days later, which has a larger sample size, more recent field dates, but has an even party-identification split between Democrats and Republicans, has a weighting of .88826, lower than any other poll taken in the last nine days.
Furthermore, Silver explained on Saturday that a tie in the Gravis Marketing Ohio poll is actually a negative for Romney in his forecast because Gravis shows a Republican-leaning bias in polling. But the Gravis poll released Saturday has a nine point advantage in party identification for Democrats — almost double the Democrats’ advantage in the 2008 election. Then, regarding the PPP Ohio poll mentioned above (where Romney cut Obama’s five-point lead to one in a week), Silver notes that “Public Policy Polling has lost most of the strong Democratic lean that it had earlier in the cycle.” He means that PPP’s polling results have tended to favor Obama less than they used to, and thus that the “house effect” of their Democratic tilt has lessened. But this subjective measure fails to take into account the possibility that Romney is doing better among the same samples. The PPP poll of Ohio actually leaned more Democratic this week; Democrats had an eight-point party-ID advantage this week but only a four-point advantage last week. So while the poll swung more to Obama’s advantage in the sample, Silver declares that it has actually lost its “Democratic lean.”
In that same post, Silver touts a “SurveyUSA poll showing Mr. Obama with a one-point lead in Florida is really the slightly better result for him.” That SurveyUSA poll indeed had Obama up by one point, but had a Democratic party-ID advantage of nine points. In 2008 Democrats had a three-point advantage, and in 2010 the parties were even. So the SurveyUSA poll is good news only if you believe Democrats will not only improve on their 2008 turnout, but triple their turnout advantage over Republicans.
This is the type of analysis that walks a very thin line between forecasting and cheerleading. When you weight a poll based on what you think of the pollster and the results and not based on what is actually inside the poll (party sampling, changes in favorability, job approval, etc), it can make for forecasts that mirror what you hope will happen rather than what’s most likely to happen. This is also true of Silver’s dismissal of Romney’s lead in Gallup this week. While Romney is likely not up by seven points nationally, as the poll predicted, you can’t dismiss it while at the same time giving a twelve-day-old Marist/NBC Ohio poll a higher weighting than eight newer polls when Marist has leaned Obama this entire cycle.
All of this also completely ignores the fact that it is more important for Obama to cross the 50 percent threshold in national polls before Election Day than it is for Romney. While it’s impossible to know how the late deciders will break, the historical trend has been for them to break for the challenger. If the Real Clear Politics average is tied at 47 percent, the overwhelming odds are that the last 6 percent will break heavily to Romney. While that would not guarantee an Electoral College victory, it is very difficult to imagine a scenario where Romney wins the national vote by more than a percentage point and loses the Electoral College.
On November 7, after the dust settles from Election Day and we (hopefully) have a winner, it might very well turn out that Nate Silver once again successfully predicted the battleground states and thus the election. But midway through October, 2012 is shaping up to be a far different election than the one in which Silver made his name. One must wonder if Silver’s forecast model includes a little bit too much hope of an Obama victory against what appears to be a surge of Romney momentum. Obama’s aura of inevitability is quickly losing steam, and once that runs out, even Homer Simpson will have a hard time proving otherwise.
— Josh Jordan is a small-business market-research consultant. You can follow him on Twitter @Numbersmuncher.