The Trouble With Facebook’s Fake-News Data

By

A new analysis from BuzzFeed fake-news expert Craig Silverman demonstrates that in the final three months of the election, the most successful posts about fake viral news “generated more engagement than the top stories from major news outlets”:

During these critical months of the campaign, 20 top-performing false election stories from hoax sites and hyperpartisan blogs generated 8,711,000 shares, reactions, and comments on Facebook.

Within the same time period, the 20 best-performing election stories from 19 major news websites generated a total of 7,367,000 shares, reactions, and comments on Facebook.

The question is … what does that actually tell us? The data, scraped from specific posts, does not reflect total site engagement on the network — that is, how many people shared, liked, or discussed a given article across the entire platform. It compares how specific posts, published on publishers’ official Facebook pages, performed. It does not distinguish between organic reach and paid placement in the News Feed, known as “boosting.”

Measured in three-month increments, beginning in February, the data shows posts from mainstream news pages substantially ahead in terms of engagement between February and July. But in the final three months of the campaign, the top fake stories promoted by their respective Facebook presences outpaced mainstream sources’ promotion of their own articles. (Why did this balance shift in the final three months of the campaign? The most obvious guess is that the beginning of the final segment coincides nicely with the shift from the primaries to the general election. Both parties turned from internal squabbling to combating each other.)

This data is illuminating, but it is also anecdotal. A news outlet, fake or real, posting a story on its Facebook page is not automatically the most-viewed link to that story. Hell: Some major publishers are paying celebrities with greater organic reach to post their content as well. That’s how inadequate organic reach is. If the page for the television sitcom Friends, which has 1.6 million followers, posted a link to an article claiming that Hillary Clinton has Parkinson’s disease, it would see an enormous number of comments, likes, and shares that BuzzFeed’s data-gathering wouldn’t uncover.

A more precise version of BuzzFeed’s story would be headlined, “[Some] Viral Fake Election News [Posts] Outperformed Real News [Posts] On Facebook In Final Months Of The US Election.” Mainstream sources likely have a wider distribution of readership along the Facebook vector. Still, that the largest fake-news sites can compete with mainstream outlets for Facebook engagement shows how powerful the platform is in distributing information.

That Facebook levels the playing field for publishers of all sizes and shapes, that it “democratizes” the web, is often seen as a plus for discourse. In reality, what we’ve ended up with are sites like Ending the Fed, which can pay minuscule amounts compared to traditional media in order to boost their posts and get them in front of millions of eyeballs. Armed with a small budget and unconstrained from the truth, almost anything can be forced into virality.

But Facebook’s understandable and utterly frustrating inability to be transparent about how their platform works makes it difficult to get an accurate picture about the real reach and presence of these sites. Websites can track how many hits they get from Facebook overall, but they can’t see how individual Facebook posts helped them get there. (It certainly doesn’t help that the company keeps buying third-party tracking solutions like CrowdTangle.) Facebook is a black box, and researchers, publishers, and advertisers are doing their best to figure it out. But Facebook, a publicly traded company, has no financial incentive to be forthcoming.

For now, the best way to understand Silverman’s analysis is holistically (fake news is rampant), and to notice what is absent from Facebook’s response. They told him, “There is a long tail of stories on Facebook. It may seem like the top stories get a lot of traction, but they represent a tiny fraction of the total.” They did not explicitly deny that fake news was performing competitively with mainstream news on the platform.