Algorithms, left unchecked, can produce some horrifying results. One Guardian reporter discovered yesterday that a photo of a rape-and-death-threat email she had received and posted on Instagram was being used by Facebook (Instagram’s owner) to advertise the service.
Guardian reporter Olivia Solon found that Facebook had mined her Instagram page for posts that had drawn a lot of engagement, and then automatically used that photo to advertise Instagram to Olivia’s Facebook friends. The problem was that one of Solon’s most engaging Instagram posts was a screenshot of a threatening email she had received.
Solon discovered the ad when her sister was using Facebook and saw it. She had posted the screenshot to her Instagram page nearly a year earlier, writing, “This is an email I received this afternoon. Sadly this is all too common for women on the internet. I am sure this is just an idiot rather than any kind of credible threat but it’s still pretty vile.” The post had gotten a few sympathetic comments and likes from concerned Instagram followers — this was likely enough to trigger Instagram and Facebook’s ad-buying program into determining that Solon’s photo would do well as an advertisement for the service.
The (mostly) automated process of buying ads on Facebook came under fire earlier this month, after a ProPublica report revealed that ad buyers could target users who had interests like “Jew hater,” “How to burn Jews,” or “History of why Jews ruin the world.” Facebook COO Sheryl Sandberg wrote in an apology on Wednesday: “The fact that hateful terms were even offered as options was totally inappropriate and a fail on our part.”
Facebook and Instagram will, no doubt, say they are taking the matter seriously and certain policies or practices will be stopped, or at least “put under review.” Machine vision recognition is sophisticated enough that, if it had been applied to whatever algorithm was spitting out these ads for Instagram, it could read and understand the text in the photos; sentiment analysis could weed out this photo (and others like it) from being used as ads. (Any worry or concern that your old Instagram photos will be mined as advertisements seems almost quaint in 2017 — of course your personal photos and data will be used for advertisements.)
But the bigger problem is simple: Facebook’s “How to burn Jews” advertising option comes from a targeting program that, from Facebook and ad buyer’s perspective, seemed to work very effectively — until this specific instance was found. The bit of code developed to use engaging Instagram posts from friends and family was likely found to be effective as well (and likely underwent public testing to make sure it didn’t display awful results like the above). Google has faced nearly identical problems in its attempts to clean up its “featured snippets” and auto-complete program, resulting in Google (for a time) affirming at the top of search-results pages that Barack Obama was planning a coup, or auto-completing the query “are Jews” into “are Jews evil.”
In each case, the algorithms being used by Facebook and Google likely worked very well in private and public testing. In Solon’s case, it’s easy to imagine that using an engaging photo from a friend’s Instagram feed showed good results in 100,000 tests. But Facebook and Google work with much larger numbers than that; when you have 2 billion users, things with one in a million odds could, theoretically, occur to 2,000 people. Algorithms and machine learning are relentlessly effective at sorting through massive sets of data. But they’re awful at determining whether something is decent, proper, or humane.
