Requests to remove content on Facebook, whether from official sources or community members, often spark debates about censorship and the spread of misinformation. But a key question remains: Does taking down a post actually have a significant impact?

The Speed Mismatch: Algorithm vs. Moderation

New research suggests that Facebook’s content moderation efforts may be lagging behind the rapid dissemination of information. A study from Northeastern University reveals that by the time a post is removed for violating community standards, it has already reached a significant portion of its intended audience.

On Facebook, content moderation doesn’t have much impact on user experience as it happens too late.
Laura Edelson, assistant professor of computer sciences at Northeastern University

This raises concerns about the effectiveness of current moderation strategies in curbing the spread of harmful content.

Understanding “Prevented Dissemination”

To better understand the impact of content moderation, researchers have proposed a new metric called “prevented dissemination.” This metric uses machine learning to analyze millions of posts and predict their potential reach if they were not taken down.

We wanted to understand what the impact of content moderation was and, in order to do this, the question we’re really asking is, if takedown didn’t happen, what would have happened?
Laura Edelson, assistant professor of computer sciences at Northeastern University

Key Findings: Engagement and Removal Rates

• the study analyzed over 2.6 million Facebook posts from more than 17,500 news and entertainment pages in American english, Ukrainian, and Russian.
• Data was collected regularly between June 17, 2023, and Aug. 1, 2023, monitoring when posts were removed and how quickly they gained engagement.
• A small percentage of posts accounted for the majority of user engagement. Such as, the top 1% of most-engaged content was responsible for 58% of user engagements in American English.
• Engagement happens rapidly: 83.5% of a post’s total engagement occurs within the first 48 hours, with half of the engagement achieved in a median time of three hours.
• Removal rates were low: 0.7% of English posts, 0.2% of ukrainian posts, and 0.5% of Russian posts were removed.
• Removed posts prevented only 24% to 30% of their predicted engagement.

The Focus of Content Moderation

The research indicates that most removed posts were classified as spam, clickbait, or fraudulent content.

This is what most content moderation on platforms are focused on – things that are clickbait, things that are spam and things that are fraud.
Laura Edelson,assistant professor of computer sciences at Northeastern University

The Core Issue: A Mismatch of Speeds

The study highlights a critical issue: the speed at which content is recommended to users far exceeds the speed at which it is moderated.

What this tells us is that if content moderation is going to have an impact on user experience – which is to say, if a platform is going to use content moderation as a strategy to not show users bad stuff – that content moderation needs to happen at the same speed as the content algorithm recommends things to people. In this case, facebook has a fast feed algorithm and slow content moderation.
Laura Edelson, assistant professor of computer sciences at Northeastern University

This mismatch undermines the effectiveness of content moderation efforts.

It’s not necessarily a problem that content moderation is slow; it’s not necessarily a problem when a feed algorithm is fast. The problem is the mismatch between the two.
Laura Edelson, assistant professor of computer sciences at Northeastern University