Why Did the Polling Experts — Except Maybe Nate Silver — Miss a Trump Victory?

Polling analysts like those at FiveThirtyEight warned us the polls could be off a few points, but we didn’t believe them. Photo: FiveThirtyEight

A few days before the election, a writer for The Huffington Post wrote a piece more or less accusing FiveThirtyEight founder Nate Silver of altering polling data to produce the impression of a closer race than the data actually justified. Everybody but Nate, you see, had Clinton’s win probability up there in the high 80s or even high 90s. After a Twitter war broke out, Nate’s accuser eventually backed down and even apologized, and most people who read the back-and-forth probably dismissed it as just a schoolyard nerd fight caused by the tensions of this crazy election.

Then November 8 happened, and Nate Silver is probably feeling conflicted between unhappiness that he did not nail the result, and a sense of vindication that he got closer than most people and adamantly insisted Trump could still win.

If Silver is no longer a target of criticism, the polls he analyzes most definitely are. When something as surprising as Donald Trump’s election to the presidency happens, it is natural to blame the non-messenger: the polls that by-and-large predicted a Clinton victory somewhere in the neighborhood of Barack Obama’s in 2012. And because so much political analysis is based largely or entirely on polls, the entire commentariat — or at least the part that was not overtly cheerleading for Trump — was off, too. Hence the battle royal over FightThirtyEight’s refusal to write off Trump.

It will take a while to sort through the debris and figure out how so much data and so much smart analysis got it all wrong. But the beginning point has to be that the final popular-vote margin is not going to be that far off from the final polling averages.

Hillary Clinton now leads in the national popular vote. With a lot of mail ballots still drifting in from heavily pro-Clinton states like California and Washington (where mail ballots postmarked by Election Day still count), she is almost certain to wind up winning the popular vote by about one percent. The final Real Clear Politics polling average had Clinton ahead by 3.3 percent. At HuffPost, the average Clinton lead was 4.6 percent. So they were off by roughly 2 to 3 percent. That is a fairly normal polling error, as Silver pointed out prophetically before the votes started coming in:

The track record of polling in American presidential elections is pretty good but a long way from perfect, and errors in the range of 3 percentage points have been somewhat common in the historical record. Of note, for instance, is that Obama beat his national polling average by nearly 3 points in 2012, although state polls did a better job of pegging his position. In 2000, Al Gore was behind by about 3 points in the final national polling average but won the popular vote. In 1996, Bill Clinton was ahead in national polls by about 12 points, but won by 8.5.

In three of the last five presidential elections, in other words, there was a polling error the size of which would approximately wipe out Clinton’s popular vote lead — or alternatively, if the error were in her favor, turn a solid victory into a near-landslide margin of 6 to 8 percentage points. There’s also some chance of a larger error still. In 1980, Ronald Reagan led in final national polls by slightly less than Clinton does now, but wound up winning the popular vote by almost 10 percentage points.

The bigger polling error, if there was one, was at the state level. Even there, though, there is in some cases less than meets the eye. In Pennsylvania, the state that put Trump over the top, the final Real Clear Politics polling average showed Clinton leading by 1.9 percent. Trump won by 1.1 percent. Once again, that’s a 3 point error. In Florida, the RCP average had Trump up by 0.2 percent. He won by 1.4 percent. That’s a 1.2 percent error. North Carolina? Trump led the polling average by one percent, and won the state by 3.8 percent. Are we seeing a pattern here?

There were a couple of true shockers: Wisconsin, where Clinton led in the polling average by 6.5 percent, only to lose by a point. But there was not a whole lot of polling there for the abundantly good reason that few observers (and until the very end, even the Trump campaign) thought the state was competitive. And even more lightly polled “shocker state” was Michigan, where actually, Clinton’s lead in the polling average was only 3.4 percent, and Trump is currently ahead by a hair.

So why do so many political observers (and well-informed voters) have the sense this morning that we were taken by surprise because the “polls were wrong”? I think there are three key factors.

First, a lot of people were convinced by early voting data that Clinton was going to win states like Florida and North Carolina, making a Trump win impossible. Actually Democrats did not take much of a lead out of early voting in either state, making it entirely feasible for Republicans to “catch up” on November 8. Nevada, by contrast, was a state where (a) Democrats did take a sizable lead in early voting, and (b) early voting was an extremely high percentage of total turnout. Sure enough, Clinton won. But a lot of people over-interpreted early voting in some places, and several of the Rust Belt states that represented Trump’s breakthrough did not offer much in the way of early voting. Early news sticks though.

Second, there was a tendency to mentally add a point or two to Clinton’s poll numbers because of her big advantage in paid media and field operations. Actually, Trump closed the advantage in paid media right at the end, and Clinton did very little advertising in several of the “firewall states” that ultimately did her in. As for the vaunted Clinton get-out-the-vote machine — well, we may have to wait for more information on how that went down. It is entirely possible that the combination of RNC and state GOP resources, plus the galvanizing effect of Trump’s monster rallies, all but eliminated the supposed Clinton advantage. Or maybe she would have lost more decisively without all those field offices. It is too early to tell.

In the end, of course, the real reason Trump’s win came as a shock is because so very many people — Republicans as well as Democrats — simply could not envision the man winning a presidential election. It is still a bit difficult to absorb how he got from where he was to where he is now; like a carnival barker wandering into the Met and delivering a brilliant performance as Iago in Verdi’s Otello. Let’s don’t blame the polls for our struggle to understand the Trump phenomenon. And let’s give Nate Silver an attaboy for sticking by his numbers.

Why Did the Polls Miss a Trump Victory?