Countering Violent Extremism: FBI and DHS Need Strategies and Goals for Sharing Threat Information with Social Media and Gaming Companies

What GAO Found

Domestic violent extremists use social media and gaming platforms for several purposes, including to reach wide audiences; to insert their extremist ideas into the mainstream; and to radicalize, recruit, and mobilize others, according to government reports and experts GAO spoke with (see figure). Experts noted that violent extremists generally use a variety of platforms for different purposes, depending on available features, audiences, and content moderation practices.

Example of the Viral Nature of Domestic Violent Extremists' Use of Online Platforms

According to social media and gaming companies GAO met with, they use various content moderation tools to identify and remove content they determine violates their platforms' policies related to domestic violent extremism on their platforms. For example, they report using machine learning tools to scan for content that violates their policies, as well as reviews by employees. However, companies and experts reported that several factors affect these moderation efforts, such as company financial considerations and diversity in standards of acceptable content. For example, content banned on one platform could be allowed on another.

The Federal Bureau of Investigation (FBI) and Department of Homeland Security (DHS) have mechanisms to share and receive domestic violent extremism threat-related information with social media and gaming companies. However, neither agency has developed a strategy that articulates how it identifies and selects companies to engage with or the goals and desired outcomes of those engagements. Without a strategy or goals, the agencies may not be fully aware of how effective their communications are with companies, or how effectively their information-sharing mechanisms serve the agencies' overall missions.

Why GAO Did This Study

In recent years, content on social media and gaming platforms that promotes domestic violent extremism has influenced several high-profile attacks, according to experts and agency officials. As a result, some social media and gaming companies, as well as federal agencies, are making an effort to understand and address online content that promotes domestic violent extremism.

GAO was asked to review domestic violent extremists' use of social media and gaming platforms. This report (1) describes the purposes for which domestic violent extremists use these platforms; (2) describes how selected companies report mitigating content promoting domestic violent extremism; and (3) assesses the extent to which the FBI and DHS have developed goals and strategies for sharing threat-related information with companies.

GAO reviewed FBI and DHS documentation and interviewed officials. GAO obtained views from 16 subject matter experts identified by the National Academies of Sciences and prior work. GAO also interviewed representatives from a non-generalizable sample of five social media and gaming companies.