The Number-Crunchers
New data-mining techniques raise questions the Numerati can't answer.


Over the last several years, most Americans have probably noticed that a lot of businesses, especially on the Internet, have been making careful use of statistical analysis. Netflix predicts what star rankings its customers will give certain movies, based on the rankings they’ve given others. Amazon recommends books that fit a customer’s preferences. Websites display different ads to different visitors depending on what other websites they’ve visited.

All of these abilities come from the painstaking analysis of mountains of data, executed by what Business Week’s Stephen Baker calls The Numerati in his new book by that name. And these number-crunchers don’t limit themselves to entertainment: They dig through supermarket shoppers’ purchases, looking to improve advertising and store layout; analyze relationships so they can develop matching technology for dating services; research the genes and behaviors that can cause disease; tease out information that might give insights into voting patterns; and much, much more.

The Numerati is a fascinating and fast read. Baker has a knack for describing statistical techniques in ways that everyone can understand, without formulas and without jargon, while illustrating them with real-world issues.

For example, he takes readers inside’s matching process. The site’s algorithm is based on a system that anthropologist Helen Fisher developed: Various hormones turn people into optimistic Explorers (dopamine), group-focused Builders (serotonin), logical Directors (testosterone), and people-skilled Negotiators (estrogen). After taking a lengthy survey, each potential lovebird is assigned a dominant and secondary type, and these types — along with demographic information — guide the computer’s matches.

And of course, in an election season, who could forget about the data wizards working over poll numbers at campaigns across the country, hoping to convince undecideds and get contributions from more dedicated partisans? The data is increasingly available from aggressive companies — ChoicePoint “quietly amasses court rulings, tax and real estate transactions, birth and death notices”; Yankelovich conducted a huge survey about values; and Acxiom “keeps shopping and lifestyle data on some 200 million Americans” — nearly every adult in the country.

With this data, campaigns can use powerful statistical techniques to divide the country into smaller and smaller segments, and then to target those segments individually. The Yankelovich survey, for example, found ten value “tribes” that politicians can “microtarget.” In looking to reach the survey’s “Right Clicks,” a tech-savvy, Republican-leaning group, a campaign might go after white males with broadband Internet connections. Or rather, whiteness, maleness, and broadband ownership might be three of the countless factors a statistically well-equipped political team might use.

Then there’s the issue of terrorism. By statistics’ very nature, data miners can only narrow down a suspect list. So while these techniques can arguably help make us more secure, lots of innocent people could find themselves under scrutiny, as well. Also, since the Numerati can make lots of money in the private sector, and since the private sector can hire non-citizens, the nation’s counterterrorism operations are at a severe disadvantage when it comes to hiring number-crunching talent.

Despite its wealth of information, The Numerati isn’t a perfect book. Baker tries to humanize his subjects by briefly profiling them, but he never goes beyond a formulaic physical description, and the reader never feels like he knows Baker’s sources. The Numerati are fascinating for what they do, not who they are, so one wonders why Baker didn’t just leave out these mini-profiles entirely.

Also, while Baker neatly summarizes what the Numerati are up to, he seldom tries to analyze these trends or figure out what their activity could mean for society; when he does attempt this, it’s usually not very insightful.

For example, in the book’s conclusion, he notes how employers have called on the Numerati to police our productivity — through such techniques as snooping in search of time-wasting Internet use — but claims we can “turn these tools to our advantage.” How? By “prowling for love on” and using other consumer services (on our own time, presumably). Well, sure: In some ways, the Numerati’s work helps those it touches, and arguably, the good outweighs the bad in the end. But if a guy finds a hot date, that doesn’t change the fact that his workplace has taken the form of a Panopticon, so this really doesn’t address the initial problem of workers losing all privacy. “These tools” are still recording every click he makes.

The biggest problem with the Numerati stems from human nature, and it would have much improved Baker’s book if he dwelt upon it a bit more: People pursue their own interests, and manipulate others when the situation calls for it. Advanced statistics provide a powerful new tool for this. As he mentions, it can be profitable for supermarket chains to tempt customers away from their diets by sending them ads for candy, or arranging the store so they’re more likely to walk past the Doritos display.

Is that a problem that requires government intervention? Or is the real problem that better analytical methods could make Big Government more seductive? After all, Thomas Sowell once noted that Soviet Union planners “had to set over 24 million prices,” an impossible feat under Soviet-era technology. But as today’s private companies learn to collect more and better data from their customers, and it becomes increasingly possible to set more and more prices from a central computer — what happens when the state thinks it can do the same thing for the whole economy? And once it starts along that road — say, with a nationalized health-care system — what else will it do with its powers?

The questions keep piling up. At some point, could advancements in genetics turn the Western world into Gattaca? Is it a problem that the Karl Roves and Mark Penns of the world are ever better at massaging their message to influence voters — or, given that our democracy has always hinged on the whims of clueless “undecided” voters anyway, does it really matter how the nation’s cynical pols convince them? Baker takes no positions, offers no prescriptions.

Bottom line: Stephen Baker’s The Numerati offers a glimpse of the future. But someone else will have to tell us how to live in it.

– National Review associate editor Robert VerBruggen edits the Phi Beta Cons blog.