Social Media Manipulation for Sale - 2024 Experiment on Platform Capabilities to Detect and Counter Inauthentic Social Media Engagement
Social Media Manipulation for Sale - 2024 Experiment on Platform Capabilities to Detect and Counter Inauthentic Social Media Engagement

Author(s): Gundars Bergmanis-Korats, Tetiana Haiduchyk
Contributor(s): Egil Fredheim (Editor)
Subject(s): Media studies, Security and defense, ICT Information and Communications Technologies, Peace and Conflict Studies, Russian Aggression against Ukraine, Russian war against Ukraine, Hybrid Warfare
Published by: NATO Strategic Communications Centre of Excellence
Keywords: Russia; Ukraine; war; media; social media; manipulation;
Summary/Abstract: Two and a half years since the Russian invasion in Ukraine, in addition to newly adopted EU regulations such as the Digital Services Act (DSA)1 and the Digital Market Act (DMA)2 allowed us to continue the series of these reports with high expectations of platforms having developed capabilities to identify and remove commercial manipulations. Although there were minor improvements across most platforms since our last experiment in 2022, our expectations were not fully met, leading us to speculate that EU regulations had minimal impact on detecting inauthentic engagement during our red team experiment. Why was this the case? Several potential reasons, which we explore in detail throughout this report, may address this phenomenon. During the experiment where we purchased inauthentic engagement from commercial social media manipulation services, platforms demonstrated significant variation in resilience to inauthentic activity, with notable differences in the ease of registration and the cost of SMS verification. Most platforms struggled with the removal of fake accounts; X showed good progress by removing 50% of identified fake accounts, while TikTok and VKontakte managed to remove only a small fraction (3% and 2%, respectively). In addition, fake interactions remained prevalent across all platforms, highlighting ongoing challenges. Manipulation services have become increasingly affordable, even from reputable providers in the US and UK, making inauthentic engagement more accessible. While the majority of clients using manipulation services are commercial entities seeking to promote spam, scams, or other commercial topics, we observed the use of bots to amplify political content on a diversity of topics. In the context of the US political environment, we observed bots being utilised to influence public opinion by amplifying divisive content related to the upcoming elections. Bots engaged in promoting and countering narratives about prominent political figures, such as President Joe Biden, Kamala Harris, and Donald Trump, as well as other politically charged content. This trend highlights the ongoing vulnerability of platforms to manipulation in politically sensitive environments, despite efforts to curb inauthentic engagement. The platforms’ efforts to counteract various forms of inauthentic activity, such as fake likes and views, have largely been ineffective, with X standing out as the only platform making significant progress by removing approximately 50% of fake comments and reposts. Most platforms showed a considerable interest in our findings, and many provided insights into their transparency reporting. We reached out to all platforms except VKontakte and received responses from all but X. A key conclusion from our experiment is that, from a red-team perspective, it was successful due to the complex content and behaviour classification systems employed by platforms. These systems rely on multiple indicators to determine whether content should be flagged as problematic, yet our experiment likely remained undetected due to its small scale. X demonstrated that small-scale commercial manipulation can be identified and removed effectively, whereas other platforms continue to struggle with this challenge. A concerning takeaway is that malicious actors can evade detection by breaking large-scale campaigns into smaller micro-scale operations, using different commercial services, and interacting sporadically with platforms. This ability to remain undetected presents a significant threat to the integrity of social media ecosystems.

• Details
• Contents