The Polls Ultimately Ended Up Making Sense — But Next Time, Who Knows?

By
Algorithmic Oracle Nate Silver. Photo: Christopher Anderson/Christopher Anderson/ Magnum

No offense to Nate Silver (or Barack Obama), but the biggest winners last night were the pollsters. After all, the vaunted FiveThirtyEight model is only as good as the data it runs through its algorithm, and, while Silver became a punching bag for conservatives over these last few weeks, their real beef was ultimately with the polls themselves, which you didn’t need to be a statistical savant to see showed Obama holding a small but persistent lead over Romney in the crucial battleground states.

Conservatives simply didn’t trust those numbers. While some of that distrust took the form of know-nothing “unskewing,” it wasn’t totally irrational. Even pollsters themselves conceded that the combination of demographic and technological changes had made their supposed science more inexact than ever. “We’re in sort of what I would call polling’s dark age,” Jay Leve, who runs the polling firm Survey USA, told me earlier this fall. “We’re coming out of a period of time where everyone agreed about the right way to conduct research, and we’re entering into a time where no one can agree what the right way to conduct research is.”

But dark ages or not, the polls — with a few notable exceptions (cough, Rasmussen) — turned out to be right. And the polling outfit that had as good a performance as any was one that may have sparked even more pre-election conservative ire than Silver. Public Policy Polling, a small polling firm in North Carolina, conducted a whopping 255 public polls in 2012, and it often seemed like polling skeptics (and even other pollsters) had a bone to pick with each one. This was partly because PPP uses automated dialers. It was also because PPP is a Democratic polling firm. But when the results came in, PPP’s polls had called all 50 states correctly in the presidential race (assuming Florida ultimately goes to Obama), every Senate race, and every important ballot initiative. Its private polling — like the 23 surveys it did of Kentucky legislative races for one client — was similarly on the mark.

When I talked to Tom Jensen, PPP’s director, this morning, he was understandably in the mood to gloat. “These supposed polling experts on the conservative side are morons,” Jensen crowed. “Jay Cost” — the Weekly Standard’s polling expert who’d waged a number-crunching war against PPP — “is an idiot.” But Jensen conceded that the secret to PPP’s success was what boiled down to a well informed but still not entirely empirical hunch. “We just projected that African-American, Hispanic, and young voter turnout would be as high in 2012 as it was in 2008, and we weighted our polls accordingly,” he explained. “When you look at polls that succeeded and those that failed that was the difference.” Given the methodological challenges currently confronting pollsters, those hunches are only going to prove more important. “The art part of polling, as opposed to the science part,” Jensen said, “is becoming a bigger and bigger part of the equation in having accurate polls.”

Which brings us back to our new polling overlord Nate Silver. His appeal, of course, is that he’s scientific. And last night, his science worked because the polls themselves worked. But as polls become more art than science, Silver’s approach could become more tenuous. The good thing about pollsters — at least the good ones — is that they’re constantly reassessing and tweaking their approaches. That’s the bad thing, too, at least when it comes to having any certainty that about how they’ll perform in the future.

What’s most problematic about Silver’s and the pollsters’ triumph last night is what it may herald for the future of campaign coverage. Without a doubt, Silver’s rigorous empiricism is much, much more preferable to the lazy, gassy, vibration-sensing punditry that has made up so much of our political journalism. And yet, the biggest complaint about campaign coverage over the last twenty years has been that it’s too focused on the horse race and doesn’t pay enough attention to the substance. Silver and his fellow polling analysts and aggregators have brought a welcome degree of precision, but they’ve only made the horse race more central to the political conversation. After all, what dominated that conversation for the past month? It wasn’t a conversation about the candidate’s dueling tax plans. Rather, it was a debate about the polls. The fact that the good guys — who put their faith in the data rather than the vibrations — won that debate may turn out to be something of a pyrrhic victory.