Facebook has a team of more than 7,500 underpaid moderators across the globe who are supposed to follow a 1,400-page rulebook to help them enforce the company’s so-called “community standards” on the platform.
On face value, it seems like an honest effort to set things right, after all the scandals and accusations the company has been plagued with for months now, including accusations of data theft, election manipulation, allowing misinformation and hate content to go unchecked on the platform, among other breaches and violations.
However, a New York Times investigation has revealed a plethora of inconsistencies in the content moderation manual, which is likely to do more harm than good.
The rulebook provides the moderators with thousands of day-to-day reference material to help them decide what constitutes a hate speech; what type of content could be a possible threat, or intended to incite violence; which posts have racist undertones, and so on.
We’re talking about thousands of Emojis, slides, words and phrases, PowerPoint presentations, spreadsheets, and what not, that the moderators have to refer to in order to decide whether a post meets the company’s “community standards” criteria.
What’s more is that they have the luxury of ten long seconds in which to allow or reject a post – sarcasm intended.
And, it’s not ten, twenty, or fifty posts we’re talking about here; on an average, they have to judge a thousand posts each day, what with two billion Facebook users around the world, and growing.
“Those moderators, at times relying on Google Translate, have mere seconds to recall countless rules and apply them to the hundreds of posts that dash across their screens each day,” says the NYT report.
It adds: “When is a reference to “jihad,” for example, forbidden? When is a “crying laughter” emoji a warning sign?”
The reference material also includes an internal list of banned groups and organizations for every country that the company has a presence in, according to the Times’ Dec 27 report.
“The closely held rules are extensive, and they make the company a far more powerful arbiter of global speech than has been publicly recognized or acknowledged by the company itself,” the NYT investigation claims to have found.
The content policing guidelines was leaked to the Times by a Facebook employee, who told the paper that the company was wielding excessive power with little discretion and was committing multiple errors.
According to NYT, the documents have revealed embarrassing loopholes – embarrassing for the company, of course – including “numerous gaps, biases and outright errors.”
The paper cites an example where moderators were told to delete a fund-raising post for Indonesian volcano victims just because one of the co-sponsors of the effort was on the company’s internally-generated list of banned groups – choosing to ignore the larger good the post would have done had it been allowed to stay on the platform.
Facebook’s internal list of groups and individuals banned by it as hate figures, although not all of them are fringe elements. Facebook users are prohibited from posting content that is deemed to support or praise them.
Another example talks about “a prominent extremist group, accused of fomenting genocide” in Myanmar, being erroneously allowed to continue on the platform for months, probably spewing venom all the while.
Dozens of Facebook employees, largely comprising young engineers and lawyers, meet every Tuesday morning to set guidelines for moderators the world over to enforce.
The guidelines are, basically, an attempt to address highly complex issues with simple Yes-No solutions, which are then outsourced to various companies for implementation.
A majority of these companies run call centers that are known to hire lowly paid unskilled workers to cut costs and maximize profits, with scant respect for quality.
More often than not, moderators find the guidelines nonsensical and confusing, not to mention the pressure they are under to get it right all the time.
“You feel like you killed someone by not acting,” a moderator told the Times on condition of anonymity, as he was bound by a non-disclosure clause in the agreement he had signed.
Meanwhile, Facebook executives keep harping about all the work that’s being put into making the platform free of controversial and potentially dangerous content.
“It’s not our place to correct people’s speech, but we do want to enforce our community standards on our platform,” said Sara Su, a senior News Feed engineer at Facebook.
“When you’re in our community, we want to make sure that we’re balancing freedom of expression and safety,” she said.
Trying to justify the company’s shortcomings, Facebook’s head of global policy management, Monika Bickert, said that while every effort was being made to make the platform safe, the scale of operations made it near-impossible to get it one hundred percent right.
“We have billions of posts every day, we’re identifying more and more potential violations using our technical systems,” she said.
“At that scale, even if you’re 99 percent accurate, you’re going to have a lot of mistakes.”
The NYT investigation found that the company tries to simply “context-heavy questions” with “one-size-fits-all rules.”
“One document sets out several rules just to determine when a word like “martyr” or “jihad” indicates pro-terrorism speech,” says the paper.
“Another describes when discussion of a barred group should be forbidden.
Words like “brother” or “comrade” probably cross the line. So do any of a dozen emojis,” the investigation revealed.
Based on this rulebook slide on what constitutes hate speech, moderators are expected to make a series of complex, legalistic judgments per post.
These are just a few instances in the discrepancy-filled rulebook that NYT investigation revealed.
The report concludes: “Without a full understanding of the platform’s impact, most policies are just ad hoc responses to problems as they emerge.
“Employees make a tweak, wait to see what happens, then tweak again — as if repairing an airplane mid-flight.”