Ethiopia: Social Media Platforms ‘Failed to Adequately Moderate Genocidal Content’ During Tigray War, Study Finds

Addis Abeba– A recent study by the Distributed AI Research Institute, an interdisciplinary research organization focused on equitable and community-rooted AI development, examining content moderation during the 2020-2022 war in Ethiopia’s Tigray region has found that social media platforms “failed to adequately moderate genocidal content,” allowing hate speech, violent incitement, and denial of atrocities to circulate widely online.

The researchers, a team of journalists, activists, data archivists, and former content moderators–including DAIR founder Timnit Gebru–wrote that platforms “prioritized superficial cultural awareness and language skills of dominant languages in a region,” while neglecting “in-depth familiarity with dialects, cultural practices, and broader social contexts” needed to properly moderate harmful content.

The study focused on moderation failures during the Tigray war, where, according to the authors, platforms fell short. It found that while operating in Ethiopia, “a country with a population of 128 million people speaking an estimated 100 languages,” Facebook “only supported two of those languages for integrity systems,” citing internal documents disclosed by whistleblower Frances Haugen.

To understand the expertise required to moderate wartime content, the researchers conducted a four-month annotation study of 340 X (formerly Twitter) posts drawn from a dataset of 5.5 million. Seven experts fluent in Amharic, Tigrinya, Arabic, and English took part in the labeling process. “We found that dialectical knowledge, including slang terms, was crucial in identifying harmful posts,” the authors wrote.

However, even among these experts, initial disagreement was high. The study reported that “our experts started out disagreeing 71% of the time,” a rate that decreased to 40% following five deliberation meetings. “Each post they disagreed on was annotated by the reason of disagreement, and final labels decided on after discussion,” the paper noted.

One example reviewed by the annotators involved a post containing the hashtag “#FakeAxumMassacre.” The post was classified under the category “Violent Event Denial.” The researchers noted that although “the event was recounted by survivors and investigated and corroborated by the likes of Amnesty International, Human Rights Watch and Associated Press,” the post “claims that the massacre is ‘fake.”‘

In another instance, a post translated from Amharic as “Clean the cockroaches” was not flagged by all annotators. The study stated that “in order to identify this post as genocidal content, one would need to know that the Ethiopian government and its allies regularly described Tigrayans as cockroaches.” One annotator who was in Tigray during the blackout did not recognize the context, while another–based in the diaspora–who had been “independently studying and archiving harmful social media posts during the genocide” quickly identified this post as genocidal content.

The study noted that “even those who had appropriate linguistic and cultural knowledge did not identify harmful content on social media if they did not spend enough time on these platforms to understand the type of content that was disseminated across different networks.”

The report also highlighted the importance of in-depth cultural knowledge in interpreting online content. One example involved the Tigrinya word “ወያነ,” which translates to “revolutionary.” According to the study, “among Eritreans, ‘ወያነ’ is used to describe the Tigray People’s Liberation Front (TPLF), whereas this is not the case among Tigrayans.”

During the war, the Ethiopian government and its allies “often used the Amharic equivalent of the term (ወያነ) to imply that all Tigrayans are members of the TPLF.” Annotators reportedly learned about these differing connotations “when discussing why they disagreed on labeling a post containing this word.” The study added that “the importance of this level of cultural knowledge was also echoed by content moderators in our interview study.”

The researchers also interviewed 15 commercial content moderators who worked in African content markets, including Ethiopia. “Content moderators are prevented from raising such disagreements by organizational hierarchies, exploitative working conditions, and inflexible platform policies,” the report concluded.

Moderators cited vague rules, lack of training, and the inability to influence platform decisions. “We turn [moderators] into robots, we force them to understand policy (…) So it’s not about what you know, what you think you know, it’s what policy says,” one quality analyst is quoted as saying. Another participant stated, “They didn’t even tell me what the job really entails. (…) Unfortunately, when I came here [and worked as a content moderator], that’s [when] I then realized that (…) it’s really not what I thought it is.”

The study found that commercial moderators often lacked the conditions to make informed decisions while also facing surveillance and punitive evaluation metrics. Annotators in the research setting, by contrast, were not time-constrained and had the opportunity to deliberate on disagreements, which the authors described as a crucial factor for consistent labeling.

The study recommends platforms to “ensure dialectical diversity amongst moderators speaking the same language and hire moderators with in-depth cultural and contextual knowledge.” It also calls on companies to “remove overly punitive measures against moderators with lower than standard rates twice in a row,” stating that “the focus on accuracy rates coupled with overly punitive measures discourages moderators who have this knowledge from raising disagreement.”