Advocacy Polls Are Not Real Polls

By
Most public opinion is much less well-informed than this man. Photo: Matt Groening/FOX

Third Way is an intra-party lobbying group that urges Democrats to adopt moderate, pro-business policies. Recently it came out with a poll demonstrating that crucial independent voters coincidentally happen to share Third Way’s views on everything: they care a ton about the deficit, they don’t care about inequality or raising taxes on the rich, and so on. (“although Swing Independents certainly are not fans of bailouts, they are also not raging against the Wall Street machine.”)

Josh Kraushaar and Clive Crook both take the bait, writing columns about how Third Way’s poll indeed demonstrates that the Obama campaign’s message will alienate the center, etc., etc. Let’s leave aside for the moment the question of whether Obama’s message truly is a good way to win an election. Maybe it’s not. I’m not a polling professional. What I do know is that journalists who cite Third Way’s poll, or any poll like it, are being played for suckers.

Public opinion is very, very malleable. Pollsters understand that very slight differences in the wording of a question, or even in the ordering of questions, can produce dramatically different results. Polls that are actually designed to measure public opinion take great precautions to avoid tilting answers one way or another. They try to frame questions in as neutral fashion as possible, and when they do ask questions that gauge people’s ideological views, they measure it by looking at changes.

So, for instance, a poll might ask if you prefer a larger government with more services, or a smaller government with fewer services. That is a classic polling question. It’s not an accurate snapshot of public opinion, though, because even though it’s posed in a completely neutral way, in frames the question in abstract terms rather than specific terms. Its value as a measuring tool is simply that polls as the same question in the same way every year, and the changes in response to the same question can help tell you how public opinion is changing.

Polls from advocacy organizations don’t do this. Their goal is to create a poll “proving” that the group’s position is a political winner. Just about any group, save perhaps NAMBLA, can do this. You simply craft language that may sound plausibly neutral but subtly tilts the question in such a way as to produce the desired results. Third Way’s method involves things like contrasting “fairness” against “opportunity.” But if you frame the question a bit differently, you can “prove” that the public is extremely positive about a populist economic message. Here are some findings from Greenberg Quinlan Rosner:

  • Eighty-one percent of those surveyed agreed that “[r]egular people work harder and harder for less and less, while Wall Street CEOs enjoy bigger bonuses than ever.”
  • Seventy-five percent agreed that “[o]ur economy works for Wall Street CEOs but not for the middle class. America isn’t supposed to only work for the top 1 percent.”
  • Seventy-two percent agreed that “right now, 99 percent of Americans only see the rich getting richer and everyone else getting crushed. And they’re right.”

Wow, that sounds like Obama should be running as the Occupy Wall Street candidate!

Again, it’s just not nearly that simple. A group like Third Way is not conducting polls to measure public opinion. It’s conducting polls in order to help advance its point of view. There are pundits who share its point of view and are receptive to an apparent data point in its favor, but polls purporting to describe public opinion, from this or any advocacy organization, should all be totally ignored.