The Corner

Non-Sampling Error

Other than John Miller’s excellent piece in the Journal today, there’s also a piece by Sharon Begley on the problems with opinion polls. It’s a very useful primer on a problem I’ve heard about from pollsters and the like for years and it’s only going to get worse. Alas, the article’s behind their subscription firewall. But two points worth noting:

First, the plus/minus margin of error is the most inconsequential of the problems built into opinion surveys. She explains:

Don’t be fooled. For all its apparent precision, the plus-or-minus statement bears little resemblance to how accurately a poll reflects the opinion of voters. It is error of a completely other kind that trips up polls.

(Math-averse readers are allowed to skip this paragraph.) The sampling error represents the range of possible outcomes from taking a random, representative slice of the population. For practical purposes, it equals one divided by the square root of the number of people surveyed. If you poll 1,600 people, then the sampling error is 1/40, or 2.5%.

Second, the real problem with polls is their non-sampling error. These are the people who don’t get asked questions because they are either unavailable or unwilling. Surly people, for example, are radically under-represented in public opinion polls. If certain attitudes are over-represented or under-represented among the surly, what are pollsters missing? Pollsters randomly dial phone numbers, but since rich people have more phone lines, are they over-represented? Pollsters won’t call cell phones, but young people increasingly have only cell phones. etc etc. Anyway, it’s a useful piece.

Most Popular

Economy & Business

The Swamp: Navarro Nucor Edition

The Wall Street Journal has a story today about the ties between President Trump's trade adviser, Peter Navarro, and the biggest steel company in the U.S. -- Nucor Corp. It is particularly interesting in light of the stiff steel tariffs successfully pushed by Navarro, which he championed ever since he joined the ... Read More


EMPIRICAL   As I can fathom neither endlessness nor the miracle work of deities, I hypothesize, assume, and guess.   The fact that I love you and you love me is all I can prove and proves me. — This poem appears in the April 2 print issue of National Review. Read More

Nancy MacLean Won’t Quit

One of the biggest intellectual jousting matches last year was between Duke history professor Nancy MacLean, who wrote a slimy, dishonest book about Nobel Prize–winning economist James Buchanan and the whole limited-government movement, and the many scholars who blasted holes in it. If it had been a boxing ... Read More
Politics & Policy

Rolling Back Dodd-Frank

The Senate on Wednesday passed a bill that would roll back parts of Dodd-Frank. The vote was 67–31, with 17 members of the Democratic caucus breaking party lines. If the legislation passes the House and is signed, it will be the largest change to the controversial financial-reform package since it became law in ... Read More

How Germany Vets Its Refugees

At the height of the influx of refugees into Germany in 2015–17, there was little doubt that mixed among the worthy cases were economic migrants taking advantage of the chaos to seek their fortunes in Europe. Perhaps out of instinctive pro-immigrant sentiment, Germany’s Left obscured the difference. Its ... Read More