So he came up with a system that predicts a pollster’s future performance based on how good it’s been in the past. In finding his average, Silver weights each poll differently—ranking them according to his own statistic, PIE (pollster-introduced error)—based on a number of factors, including its track record and its methodology. One advantage of this system is that, during the primaries, the system actually got smarter. Because each time a poll performed well in a primary, its ranking improved.
For the general election, this gets trickier, since you have polls coming out every single day and you can’t know which ones are getting it right until Election Day. You can, however, weigh these new polls based on the pollster’s history, the poll’s sample size, and how recently the poll was conducted. You can also track trends over time and use these trend lines to forecast where things will end up on November 4. You can also, as Silver has done, analyze all the presidential polling data back to 1952, looking for information as to what is likely to happen next. (For example, how much the polls are likely to tighten in the last month of the race, which they traditionally do.) You can also run 10,000 computer simulations of the election every day based on your poll projections. (Think of this as sort of like that scene at the end of WarGames, where the computer blurs through every possible nuclear-war scenario.) As of October 8, the day after the town-hall debate, Silver’s simulations had Obama winning the election 90 percent of the time.
“Literally, the composition of the North Carolina electorate is different than it was six months ago.”
All of which is very seductive (and heartening to Obamaphiles), especially when you see it laid out on Silver’s site, with its pleasing graphs, compelling charts, and graphically vibrant electoral maps. But in essence, Silver’s whole undertaking is premised on breaking the first rule of reading polls: He’s assuming—in his complex, elegant, partially proprietary and yet-to-be-entirely-validated way—that today’s polls can predict tomorrow’s election. Rival statisticians, in particular Sam Wang at Princeton (who runs his own poll-aggregation blog), have criticized Silver, arguing that polls can’t and shouldn’t be used as a crystal ball. Other critics have argued that the idea of a projection model is inherently flawed because it can’t predict the unpredictable—for example, before the financial meltdown and McCain’s campaign-suspension stunt, the polls were much tighter and Silver’s electoral map had McCain on top.
Silver agrees, to a point, comparing daily polls, especially ones that come out months before the election, to “a 162-game baseball season, where one individual win or loss doesn’t really tell you that much about the ultimate outcome.” But for him, you have to use polls to predict the future—that’s the whole point. Unlike other electoral projection maps, Silver calls each state for one candidate or the other—there are no undecideds—because the goal is to approximate what the map will look like after Election Day. “I think the entire value of the exercise is in predicting the outcome in November,” Silver says, in a response to Wang. “What would happen in an election held today is a largely meaningless question.”
And Silver is right. The truth is that everyone reads polls this way. When you pick up the paper and see McCain up by three or Obama up by six, you assume that means that candidate is on his way to a win. Silver’s goal with FiveThirtyEight, then, is to simply do what everyone does, but do it better—to read the polls in such a way that those assumptions we all naturally make will actually turn out to be true.
For all the numbers and nuance, the adjustments and algorithms, there’s really only one stark, looming, unambiguous test for the political prognosticator. “Pretty soon, there’s going to be an Election Day,” says pollster J. Ann Selzer of Selzer & Co. “And you’re either going to be golden or a goat.” (Selzer, this season, has consistently been golden, calling the Iowa caucus flawlessly, which is partly why Selzer & Co. is Silver’s top-ranked pollster on FiveThirtyEight.)
Like everyone else calling this election, Silver’s day of reckoning will come on November 4. In the meantime, he’s become an increasingly confident commentator, growing into a national role as a calming anti-pundit among the white noise of partisan spin. FiveThirtyEight not only tracks numbers, but features field reports from the 50 states by Silver’s colleague, Sean Quinn, and the documentary photographer Brett Marty. And Silver posts to his blog several times daily, spotting and dissecting surprising trends or aberrations, such as a recent poll in Minnesota that handed McCain a sudden one-point lead. Normally, you’d expect a liberal-leaning commentator to read such a result and blame bias, or error, or voodoo. Silver, however, poked around and determined that McCain had been recently outspending Obama three-to-one in Minnesota, making it the only state in the country where he was out-advertising Obama. “So McCain may literally have bought his way into a competitive race,” Silver wrote. “So, yes, you can beat a state into submission if you really want to … But whether it’s been a good use of resources, we’ll have to see.”