While Facebook has caught most of the flak over “fake news” in the past year, Google hasn’t escaped unscathed. Two of its main search features, autocomplete and Featured Snippets, had serious problems with offensive, inflammatory, and sometimes outright wrong content. Now, Google is trying to clean up its act.
Autocomplete, first introduced by Google back in 2004, had been giving Google headaches for years. It relied heavily on what users most commonly typed in, and since Google users sometimes search for despicable stuff, you got despicable results. This led to situations where search queries like “are Jews” were being given autocomplete suggestions like “are Jews evil,” which became a minor scandal this December when discovered by the press. Google manually removed that autocomplete suggestion, but it had been playing a game of Whac-a-Mole for years with the feature — someone would find an offensive autocomplete, the story would get picked up by the press, and Google would remove the results. Now, users can manually flag results that are false or offensive, and explain why they flagged them:
Featured Snippets, first introduced in 2014, were an attempt to short-circuit content farms and “What time is the Super Bowl” SEO gaming. The top result of certain search queries, instead of a URL link, is instead a boxed-out quote and answer, what Google’s algorithm deems the “best” answer, given huge prominence at the top of the page. But it, too, quickly became a problem, with false and sometimes offensive results occupying the Featured Snippet spot. This, for example, was the Featured Snippet for “Is Barack Obama planning a coup?” in March:
This also meant it was the reply you’d get if you asked the same question to Google Home, Google’s friendly little AI speaker that’s supposed to answer all of the world’s questions.
To fight back against this, Google is taking the same approach with Featured Snippets, allowing users to flag them and explain why they’ve flagged the results:
Google also uses actual humans to rank its pages, using those to help weigh search results, and made a similar change in its guidelines, telling its human evaluators to look for “misleading information, unexpected offensive results, hoaxes and unsupported conspiracy theories” and flag them. It is similarly making some small changes in how it ranks pages in its algorithm. “We’ve adjusted our signals to help surface more authoritative pages and demote low-quality content, so that issues similar to the Holocaust denial results that we saw back in December are less likely to appear,” writes Ben Gomes, VP of Core Search at Google.
The pressure on Google’s autocomplete and Featured Snippets had been building for a while. Autocomplete had been a weird train wreck for over a decade, and Featured Snippets had started to come under fire roughly in December. There’s now a bigger spotlight on the role of massive monopolies, like Google and Facebook, in shaping how the public gets its information, and Google really had no choice but to step forward.
But there’s an open question: How much can changes in the algorithm and flagging by users fix the problem?
Perhaps the biggest and most effective change Google ever made to its search algorithm was its series of “Panda” updates, seemingly aimed at content farms that were filling up Google results with low-effort spam about anything that ranked highly in search. Panda worked; it destroyed or wildly devalued many sites that relied mainly on gaming search.
Regardless, Google was able to shut down low-quality sites spewing out “How do I get a snowmobile license in Idaho?” articles by making them unprofitable for publishers. But those churning out conspiracy theories and racist tirades aren’t always motivated (purely) by profit. The highly offensive Featured Snippet for “Are black people smart?” was, for a short period, quoting from a WordPress blog run by a man who lives off of a trust fund, and publishes a lot of ugly racial science that would make Charles Murray cringe. Neo-Nazi website Stormfront was the first result for the Google query “Did the Holocaust happen?” along with a host of other anti-Semitic sites, until Google expunged all of them from the first page of results. Put simply, while some fake-news sites are cash grabs by Macedonian teens to afford electric guitars, a lot of the truly vile stuff that Google indexes isn’t published with an eye toward profit; it’s published because of a hateful ideology.
Having users flag results, meanwhile, can also prove problematic. It’s easy when the answer is wrong or obviously racist. But what about highly subjective questions? This, for instance, is currently the Featured Snippet for “Does capitalism work?”
Depending on your politics, the above answer is somewhere between neutral, correct, and completely wrong. Find the right corner of the internet to rile up about this being the top result, and you could get hundreds or thousands of users hammering the flag button, potentially eliminating it from the top spot.
Featured Snippets, in an estimate by Moz.com, are used in about 15 percent of its sample of 10,000 search queries. Autocomplete is used in almost every Google query, except those about sex, violence, and hate speech. Google long ago became an index so large that no human could hope to really run it, and its search algorithm so complex that no one, even inside Google, understands how changes to it will ultimately affect its search results.
In the blog post announcing the changes, Google’s Ben Gomes writes, “[I]t’s become very apparent that a small set of queries in our daily traffic (around 0.25 percent), have been returning offensive or clearly misleading content, which is not what people are looking for.”
Google doesn’t break out daily search volume, simply saying that it gets “trillions” of search queries every year. The last time it confirmed the actual number was when it was getting 1.2 trillion per year in 2012. Taken at the very lowest end, this suggests Google got at least 2 trillion total queries in 2016. That would mean, on average, about 5.5 billion queries per day. If 0.25 percent of them were, in Google’s own estimation, problematic in some way, it’s possible that about 13.7 million search queries were coming back with “offensive or clearly misleading content.” (The math on this, of course, should be taken with a grain of salt — it’s likely that Google’s search traffic is higher than 2 trillion per year, but this is likely global search traffic, and it’s unclear how much autocomplete and Featured Snippets are a problem in non-English queries.)
Tools like flagging, changes to the search algorithm, and using human evaluators as gut checks are helpful, but Google has a problem. If the company is sincere in wanting to keep offensive and false information out of autocomplete and Featured Snippets, and reduce the prominence of Holocaust denial and fake coup attempts in its index, the numbers are against it.