Facebook can’t control its users. And it has no incentive to do so

Most people I know who use Facebook seem normal. And the uses to which they put the service also seem normal – harmless to the point of banality. So when they see reports of how social media is being used to fuel extremism, violence, racism, intolerance, hatred – even ethnic cleansing – they are puzzled. Who are the people who do things like that? And why doesn’t Facebook stop them?

To answer the first question, let us visit Altena, a town of about 18,000 souls in North Rhine-Westphalia in Germany. After Angela Merkel opened Germany’s doors to refugees, Altena took its quota, like any good German town. When refugees first arrived, so many locals volunteered to help that Anette Wesemann, who runs Altena’s refugee integration centre, couldn’t keep up. She’d find Syrian or Afghan families attended by groups of volunteer German tutors. “It was really moving,” she told a New York Times reporter.

But when Wesemann set up a Facebook page to organise food banks and volunteer activities, things changed. The page filled with anti-refugee vitriol of a sort she hadn’t encountered offline. Some posts came from outsiders, but they were also some from a number of local citizens. Over time, the anger proved infectious, dominating the page. And then one night a young trainee fireman who had hitherto seemed innocuous broke into the attic of a refugee group house, doused it with petrol and set it on fire. Luckily, no one died, but the police found that the culprit had been a dedicated Facebooker who incessantly shared memes and articles attacking foreigners, even though in public he had displayed no such animus towards the refugees.

So what? you say. Every village has its nutters. Yes, but a startling piece of research at Warwick University suggests that this particular small town in Germany is typical of a wider pattern. The researchers, Karsten Müller and Carlo Schwarz, examined every anti-refugee attack in Germany (more than 3,300) over a two-year span. For each attack, they analysed the local community by any variable that seemed relevant – wealth, demographics, support for far-right politics, newspaper readership, number of refugees, history of hate crime, number of anti-refugee protests, etc. One thing stuck out. Towns where Facebook use was higher than average, such as Altena, consistently experienced more attacks on refugees. Wherever per-person Facebook use rose to one standard deviation above the national average, attacks on refugees increased by about 50%. “Our results suggest,” write the researchers, “that social media can act as a propagation mechanism between online hate speech and real-life violent crime.”

Now, as every GCSE maths student knows, correlation is not the same thing as causality. But that particular get-out-of-jail card is proving more and more elusive for Facebook, because we now have abundant proof that what people read and say on its services does indeed induce them to do horrible things offline.

The UN, for example, has concluded that Facebook had “a determining role” in ethnic cleansing in Myanmar. A fake video about child abduction circulating in India on WhatsApp (another Facebook property) has led to innocent people being beaten to death by mobs convinced they were child-kidnappers. And so on.

Which leads to our second question: why isn’t Facebook stopping this rot? There are various answers, but they fall into two broad categories. The first is that the problem is probably too big for it: Mark Zuckerberg and his team have created a monster that no one, not even an autocrat like Zuckerberg, could totally control. The collective ingenuity, benign as well as malevolent, of 2.2bn users is beyond the capacity of any organisation to manage. Even if Facebook employed another 50,000 moderators to weed out the crap it can’t be done, especially given the hyper-centralisation of the company.

The second reason why Facebook isn’t fixing its problems is that it doesn’t seem to want to. Why? Because the solution would cripple its business model, slow its growth dramatically and reduce its profitability. Or, as Frederic Filloux, a seasoned observer of these things puts it: “When it comes to fighting misinformation, Facebook has a problem of will and resolve, which is deep rooted in the questionable set of values the company is built upon. The good news: it can be reversed. The bad news: not with the current management of the company.”

That about sums it up. In the meantime, Facebook is deploying its usual excuse – blaming users. Last week, the Washington Post revealed that the company had begun to assign its users a reputation score, predicting their trustworthiness on a scale from zero to 1. I wonder what my friends will make of that.

What I’ve been reading

1. What hyperinflation looks like. Remember those photographs of people in Weimar Germany taking wheelbarrows of banknotes to pay for a loaf? Much the same has been happening in Venezuela recently – as these photographs from Reuters vividly illustrate.

2. The Hajj – the annual Muslim pilgrimage to Mecca – is the biggest event of its kind in the world. But I hadn’t grasped its scale until Quartz.com unveiled a set of staggering aerial photographs of the crowds.

3. Machine learning is the tech frenzy du jour. If you’d like to understand why some of us are worried about it, then this wonderful YouTube keynote address by Harvard’s James Mickens should help.