Donald Trump likes to call public-opinion surveys that don’t show him doing well “Fake News polling” or even “Fake News Suppression Polls,” whatever that means. Plenty of other people in politics tend to promote poll results that are good for them at the expense of polls results that are bad for them. Among less self-serving observers, there’s a general recognition that while polling quality varies based on sample sizes, polling techniques, and various “weighting” and interpretation practices, most polling data has some value. That is why, say, the nerds of FiveThirtyEight rate pollsters to give everyone a clear way to understand varying poll quality. But in making projections the same nerds tend to use polling averages as a way to reduce the significance of any one survey.
FiveThirtyEight’s Harry Enten, however, is now drawing attention to a largely new and very different problem: people using cheap and easy internet platforms like Google Surveys and SurveyMonkey Audience to do “polls” that either are too loosely conducted to be valid, or are so completely opaque that it’s impossible to tell if they’re just BS.
Enten’s example is a widely publicized survey released in July by a totally unknown firm that purported to show Kid Rock ahead of U.S. senator Debbie Stabenow in a 2018 trial heat in Michigan. The buzzworthiness of the “results” was obvious: It involved a pop celebrity who may or may not be toying with a political career, running against a Democratic senator who was thought to be in pretty good shape heading toward a reelection year. But the more Enten tried to figure out how, specifically, the results were generated, and by whom, the more it became a bit of a mystery. He even thought it possible that the poll was designed to manipulate and profit from betting sites.
The story of Delphi Analytica, its mysterious origins and its Kid Rock poll show that the line between legitimate and illegitimate pollsters is blurring. Much of the polling industry is moving online, where conducting a survey is far less expensive than making thousands of phone calls. But that lower price has also opened up polling to all sorts of new people: Some are seasoned professionals trying an old craft with a new tool or well-informed, well-meaning amateurs trying to break into the industry, but other characters have less noble goals — they’re pranksters seeking attention and scam artists trying to make a quick buck.
So more careful scrutiny of polls, particularly those conducted online, is in order, Enten argues.
But the trouble is: There is no power on earth that can or will keep politicians from cherry-picking polling data to get press, raise donations, or simply boost internal campaign morale. The bad news Enten shares is that the number of “pollsters” generating questionable data is rising, and could rise even more in the very near future.
Some political observers will prove to be suckers for phony or slanted polls, while others will throw the baby out with the bathwater and refuse to look at polling data altogether, relying instead on disingenuous “insider” assessments of elections or, worse yet, making anecdotes a substitute for analysis (anyone can sound superficially informed by talking to preselected groups of voters and gleaning predictable insights).
Now more than ever, strong norms against really shoddy polls, and the use of averages in dealing with others, are called for to avoid a climate of competing half-information that really just turns into preemptive spin. It’s a perspective the chattering classes need to internalize quickly before the midterm election cycle is upon us.
