The Corner

How Narwhal Beat Orca

The Atlantic has the long but engaging and amusing tale of the Obama campaign’s “Tech” team, who designed all the various apps that Obama 2012 used to fundraise, organize, contact voters, etc. There’s been plenty of controversy over the failure of the Romney campaign’s parallel efforts; much of the digital infrastructure for the campaign’s “Orca app” (the orca is the only known predator of the narwhal, the codename for Obama’s efforts) collapsed on Election Day, with Internet service going down, the app being overloaded and unready to handle full demand, lack of telephone communications, etc. The contrast with the Obama campaign’s work could not be more dramatic:

The Obama campaign’s technologists were tense and tired. It was game day and everything was going wrong.

Josh Thayer, the lead engineer of Narwhal, had just been informed that they’d lost another one of the services powering their software. That was bad: Narwhal was the code name for the data platform that underpinned the campaign and let it track voters and volunteers. If it broke, so would everything else.

They were talking with people at Amazon Web Services, but all they knew was that they had packet loss. Earlier that day, they lost their databases, their East Coast servers, and their memcache clusters. Thayer was ready to kill Nick Hatch, a DevOps engineer who was the official bearer of bad news. Another of their vendors, PalominoDB, was fixing databases, but needed to rebuild the replicas. It was going to take time, Hatch said. They didn’t have time.

They’d been working 14-hour days, six or seven days a week, trying to reelect the president, and now everything had been broken at just the wrong time. It was like someone had written a Murphy’s Law algorithm and deployed it at scale.

And that was the point. “Game day” was October 21. The election was still 17 days away, and this was a live action role playing (LARPing!) exercise that the campaign’s chief technology officer, Harper Reed, was inflicting on his team. “We worked through every possible disaster situation,” Reed said. “We did three actual all-day sessions of destroying everything we had built.”

The story also recounts how the team, assembled in the early summer of 2011, underdelivered for months, and much of their work didn’t come to fruition, and become usable by voters, until the spring of 2012. Eventually, though, the world-class team produced a range of products that gave the campaign and its volunteers a range of abilities it had never had before. It’s also a fascinating account of how the precepts of digital-development and start-up management applied, or didn’t apply, to the Obama campaign’s efforts, which were so much more vast than the Romney campaign’s.

In short, though, we never really could have hoped that the GOP would match the quality of Obama’s digital efforts, since that would require Mitt Romney hiring a guy like this:

That’s Obama for America’s tech director, Harper Reed:

If you’ve spent a lot of time around tech people, around Burning Man devotees, around startups, around San Francisco, around BBSs, around Reddit, Harper Reed probably makes sense to you. He’s a cool hacker. He gets profiled by Mother Jones even though he couldn’t talk with Tim Murphy, their reporter. He supports open source. He likes Japan. He says f*** a lot.  He goes to hipster bars that serve vegan Mexican food, and where a quarter of the staff and clientele have mustaches. . . .

Reed has friends like the manager of the hip-hop club Empire who, when we walk into the place early on the Friday after the election, says, “Let me grab you a shot.” Surprisingly, Harper Reed is a chilled vodka kind of guy.

Patrick BrennanPatrick Brennan is a writer and policy analyst based in Washington, D.C. He was Director of Digital Content for Marco Rubio's presidential campaign, writing op-eds, policy content, and leading the ...

Most Popular

Culture

‘Epstein Didn’t Kill Himself’

It was just one more segment to fill out the hour, and thereby fill the long 24 hours of Saturday’s cable news on November 2. Or so it seemed. Navy SEAL Mike Ritland was on the Fox News program Watters World to talk to Jesse Watters about trained German shepherds like the one used in the raid that found ... Read More
Film & TV

The Manly Appeal of Ford v Ferrari

There used to be a lot of overlap between what we think of as a Hollywood studio picture (designed to earn money) and an awards movie (designed to fill the trophy case, usually with an accompanying loss of money). Ford v Ferrari is a glorious throwback to the era when big stars did quality movies about actual ... Read More
White House

Impeachment and the Broken Truce

The contradiction at the center of American politics in Anno Domini 2019 is this: The ruling class does not rule. The impeachment dog-and-pony show in Washington this week is not about how Donald Trump has comported himself as president (grotesquely) any more than early convulsions were about refreshed ... Read More
Politics & Policy

ABC Chief Political Analyst: GOP Rep. Stefanik a ‘Perfect Example’ of the Failures of Electing Someone ‘Because They Are a Woman’

Matthew Dowd, chief political analyst for ABC News, suggested that Representative Elise Stefanik (R., N.Y.) was elected due to her gender after taking issue with Stefanik's line of questioning during the first public impeachment hearing on Wednesday. “Elise Stefanik is a perfect example of why just electing ... Read More
U.S.

What Happened to California Republicans?

From 1967 to 2019, Republicans controlled the California governorship for 31 of 52 years. So why is there currently not a single statewide Republican officeholder? California also has a Democratic governor and Democratic supermajorities in both houses of the state legislature. Only seven of California’s 53 ... Read More