Google+
Close
Nate Silver’s Flawed Model
The New York Times number cruncher lets his partisanship show.

Statistician Nate Silver

Text  


Comments
577

“Oh, people can come up with statistics to prove anything, Kent. Forty percent of all people know that.” — Homer Simpson.

In the days before the first debate in Denver, President Obama held more than a four-point lead in the Real Clear Politics average, and Romney had been left for dead by most of the media. Then the debate came, and overnight Romney seemingly rid himself of the weaknesses that had been tacked on to him by over $100 million dollars in negative advertising. Now here we are a few weeks later with a dead heat in nationwide polls.

As worry built up among Democrats that Romney had tied the race nationally and had clear momentum heading into the final stretch, they began attaching their hopes to what BuzzFeed’s Ben Smith called “the bulwark against all-out Dem panic” — Nate Silver.

Silver gained fame by correctly predicting 49 of 50 states in the 2008 election using a statistical model that assigns weight to the various polls based on a number of factors. After the 2008 election, Silver partnered with the New York Times, and he has been quoted by many media outlets as the gold standard for predicting what will happen in November.

Advertisement
Some note that 2008 was a wave election, where the enthusiasm and underlying fundamentals were so favorable to Obama that the outcome was easy to foresee, with the exception of a few of the GOP-turned-Democratic states such as Indiana and North Carolina where Obama won a razor-thin victory. Others argue that Silver’s access to the Obama administration’s internal polling gave him information that most other analysts never saw, which allowed him to make more adjustments to his model and increase his accuracy.

Whatever the explanation, Silver’s strong showing in the 2008 election, coupled with his consistent predictions that Obama will win in November, has given Democrats a reason for optimism. While there is nothing wrong with trying to make sense of the polls, it should be noted that Nate Silver is openly rooting for Obama, and it shows in the way he forecasts the election.

On September 30, leading into the debates, Silver gave Obama an 85 percent chance and predicted an Electoral College count of 320–218. Today, the margins have narrowed — but Silver still gives Obama a 67 percent chance and an Electoral College lead of 288–250, which has led many to wonder if he has observed the same movement to Romney over the past three weeks as everyone else has. Given the fact that an incumbent president is stuck at 47 percent nationwide, the odds might not be in Obama’s favor, and they certainly aren’t in his favor by a 67–33 margin.

The main reason that Silver feels Obama is still an overwhelming favorite is that while Romney has surged in the polls to tie (or lead) Obama nationally, the challenger is still, in Silver’s opinion, a long shot to pull together enough battleground states to get to 270 electoral votes. This is the real problem with Silver’s model in the eyes of many Romney backers — the “weighting” he puts into state polls gives an edge to Obama, and the distribution of that weighting is highly subjective. For example, Silver currently gives Obama a 70 percent chance of winning Ohio. A component of this is a weighted “polling average” of Obama’s support at 48.2 percent to Romney’s at 45.2. The current Real Clear Politics average is nearly a full point more favorable to Romney: It has Obama at 48.1 and Romney at 46.0. The difference comes from the fact that Real Clear Politics gives equal weight to all of the polls it includes and uses only the most recent polls from each polling organization in a given timeframe.

While many in the media (and Silver himself) openly mock the idea of Republicans’ “unskewing polls” (and I am not a fan of unskewedpolls.com by any means), Silver’s weighting method is just a more subtle way of doing just that. I outlined yesterday why Ohio is closer than the polls seem to indicate by looking at the full results of the polls as opposed to only the topline head-to-head numbers. Romney is up by well over eight points among independents in an average of current Ohio polls, the overall sample of those same polls is more Democratic than the 2008 electorate was, and Obama’s two best recent polls are among the oldest.

But look at some of the weights applied to the individual polls in Silver’s model. The most current Public Policy Polling survey, released Saturday, has Obama up only one point, 49–48. That poll is given a weighting under Silver’s model of .95201. The PPP poll taken last weekend had Obama up five, 51–46. This poll is a week older but has a weighting of 1.15569.

The NBC/Marist Ohio poll conducted twelve days ago has a higher weighting attached to it (1.31395) than eight of the nine polls taken since. The poll from twelve days ago also, coincidentally enough, is Obama’s best recent poll in Ohio, because of a Democratic party-identification advantage of eleven points. By contrast, the Rasmussen poll from eight days later, which has a larger sample size, more recent field dates, but has an even party-identification split between Democrats and Republicans, has a weighting of .88826, lower than any other poll taken in the last nine days.

Furthermore, Silver explained on Saturday that a tie in the Gravis Marketing Ohio poll is actually a negative for Romney in his forecast because Gravis shows a Republican-leaning bias in polling. But the Gravis poll released Saturday has a nine point advantage in party identification for Democrats — almost double the Democrats’ advantage in the 2008 election. Then, regarding the PPP Ohio poll mentioned above (where Romney cut Obama’s five-point lead to one in a week), Silver notes that “Public Policy Polling has lost most of the strong Democratic lean that it had earlier in the cycle.” He means that PPP’s polling results have tended to favor Obama less than they used to, and thus that the “house effect” of their Democratic tilt has lessened. But this subjective measure fails to take into account the possibility that Romney is doing better among the same samples. The PPP poll of Ohio actually leaned more Democratic this week; Democrats had an eight-point party-ID advantage this week but only a four-point advantage last week. So while the poll swung more to Obama’s advantage in the sample, Silver declares that it has actually lost its “Democratic lean.”



Text  


Sign up for free NRO e-mails today:

Subscribe to National Review