Terrorist and violent extremist groups have been forced to dramatically reshape their strategies for recruitment, incitement and glorification of violent attacks online over the last few years, shifting their attention from a few large global platforms, to smaller niche services less prepared to respond to this abuse.
Increased enforcement and greater technical sophistication in content removal by large platforms in recent years has prompted this change in tactics. Not only does this content violate the terms of service of most digital platforms, hosting it is often explicitly prohibited by local law. For the smaller platforms now being targeted however, identifying extremist content and removing it to limit its spread and comply with local law has become a major challenge.
To address this, Jigsaw coordinated global, cross-platform organizations across civil society to build Altitude. Altitude, built with Tech Against Terrorism, in partnership with the Global Internet Forum to Counter Terrorism (GIFCT), is the only free, open-source, self-hosted tool to give online platforms a single elevated view of potential terrorist and violent extremist content (TVEC) on their sites. In a few clicks, moderators can view, review and remove content identified by trusted sources, enabling platforms to make clear and confident decisions.
Any platform can now access Altitude directly or work with Tech Against Terrorism to integrate the tool into its workflow and better understand the broader context of potential threats and evolving legislation.
Designing Altitude to meet content moderation needs of small to medium-sized platforms
As detailed in a 2022 report by Tech Against Terrorism, “small platforms are increasingly targeted as a means of sharing this content but are often not included in current crisis response mechanisms, allowing the content to remain online for longer.”
At Jigsaw, we’ve studied how online terrorist and extremist activity can have serious real-world effects. In particular, our past research has pointed to a “boomerang effect,” where extremist content that’s removed from large platforms resurfaces on less moderated spaces, but then can boomerang back to the original, more-moderated spaces. At the same time, a growing body of legislation, including the EU’s Digital Services Act and the EU Regulation on Terrorist Content Online, has emerged that not only proscribes this content but specifies timeframes in which it must be removed — sometimes granting platforms as little as an hour. For many of the smaller social media platforms and file sharing sites that are increasingly being targeted, meeting these requirements has proven almost impossible.
Over the course of a year, Jigsaw conducted interviews with founders, trust and safety leaders, and TVEC content reviewers at eleven platforms, ranging in size. Platforms included image-sharing platforms, social media sites, url shortening services and file and text storage sites. We found that platforms existed along a scale of resourcing and preparedness to manage TVEC, and that that level of preparedness was not neatly related to the size of their organization or user base. Even relatively mature services still sometimes lacked signals to proactively identify harms on their platform. We also saw that, while they may eventually prefer to integrate all tools into an in-house platform, a separate interface for dedicated harms could provide a way to get started in an under-resourced environment.
Our goal was to build a tool that would meet platforms where they were while enabling best-in-class preparedness and responsiveness at even the least resourced of these organizations. We identified several specific insights that indicated a need for an accessible, dedicated tool for moderating TVEC, one that could further provide aggregate signals, rich context, bulk actioning on identical content, and image blurring by default.
Altitude is designed to help small and medium-sized platforms more effectively respond to the onslaught of extremist content. The tool helps platforms prioritize alerts of TVEC by using Meta’s hashing and matching technology to match images from a given platform to content in two leading databases: Tech Against Terrorism’s Terrorist Content Analytics Platform (TCAP) and the GIFCT Hash-Sharing Database (HSDB). Altitude also leverages components from “Prisma,” a design system developed by the User Experience team at Google Trust and Safety for its own reviewer workflows that was built to scale. Altitude makes these design components public to any platform for the first time.
Get started with bespoke onboarding support
We are thrilled to support Tech Against Terrorism as it takes on the maintenance and expansion of Altitude. Tech Against Terrorism is deeply committed to balancing the needs of both under-resourced platforms as well as larger societal needs and the preservation of human rights on the ground. Platforms who are interested in using Altitude can now join the Tech Against Terrorism Network to receive free, bespoke onboarding support including guided installation, testing, and feedback plus they will gain access to a suite of specialist services for members.
Through interviews conducted with global human rights professionals, Jigsaw and Tech Against Terrorism have identified key areas of future development for Altitude: inclusion of additional specialized databases and native translation of both the platform and flagged content. Interviews also stressed the need for continued collaboration between platforms, civil society, and regulators.
As more platforms use Altitude and contribute to the databases themselves, this could lead to a virtuous cycle, in which parties can share timely information on both harmful and non-harmful content, and protect the ecosystem by revealing broader cross-platform trends.
