Researchers’ audit reveals flaws in Facebook’s identification of political ads

Where the platform enforced its policy, or should have, it was wrong more often than right

NEW YORK, LEUVEN — In the first known study to quantify the performance of Facebook’s political ad policy enforcement at a large and representative scale, researchers found that when making decisions on how to classify undeclared ads, Facebook often missed political ads while incorrectly labeling others as political. The political ads that Facebook misses also disappear from its public archive, putting them out of reach for public scrutiny.

Read top findings, download full study, and see gallery of sample misclassified ads by country here.

The researchers from imec-DistriNet at KU Leuven (in Belgium) and NYU Cybersecurity for Democracy examined 33.8 million Facebook ads that ran between July 2020 and February 2021 — a timeframe that included elections in both the U.S. and Brazil. They based their analysis on Facebook’s own criteria for classifying political ads. The researchers will formally present their paper “An Audit of Facebook’s Political Ad Policy Enforcement,” at the 31st USENIX Security Symposium, one of the world’s top security conferences, in August 2022.

Read the report here

Among their top findings:

  • Globally, Facebook made the wrong decision for 83 percent of those ads that had not been declared as political by their advertisers and and that Facebook or the researchers deemed political.
  • Facebook missed a higher proportion of political ads outside the U.S. The platform had the worst record in Malaysia, where it missed as much as 45 percent of ads from obviously political pages or advertisers. In the U.S., 55 percent of detected political ads were actually overcounts, meaning they did not meet Facebook’s definition for political ads.
  • Facebook allowed more than 70,000 political ads to run after its announcement of the moratorium on political ads around the U.S. 2020 elections.

“Facebook was more often wrong than right in distinguishing political from non-political ads. This ultimately harms users, who are unaware that an ad is politically motivated. It also harms advertisers, who see their ads taken down without reason,” said Victor Le Pochat, lead researcher on the study, and a PhD fellow of imec-DistriNet at KU Leuven and the Research Foundation Flanders.

Facebook’s transparency via its Ad Library API was not sufficient to do this analysis. To capture publicly available ads that were not included in the API, the researchers created an alternate pipeline to gather ads from the web portal. This points to the need for mandated universal ad transparency, as defined in “A Standard for Universal Digital Ad Transparency,” by a team of researchers led by Laura Edelson, published December 9 by the Knight First Amendment Institute at Columbia University. The paper defines the criteria for determining which ad platforms would need to comply and what content should be made transparent. Additionally, it compares the proposed standard to the platforms’ current voluntary transparency practices, as well as to related U.S. legislative efforts.

“We can’t trust Facebook to do a good enough job identifying or disclosing political ads on its own. We need mandated transparency powering independent research to identify weaknesses and hold platforms accountable,” said Laura Edelson, co-founder of NYU Cybersecurity for Democracy and co-author of the report.

The authors of “An Audit of Facebook’s Political Ad Policy Enforcement” are: Victor Le Pochat (imec-DistriNet, KU Leuven), Laura Edelson (New York University), Tom Van Goethem (imec-DistriNet, KU Leuven), Wouter Joosen (imec-DistriNet, KU Leuven), Damon McCoy (New York University), and Tobias Lauinger (New York University).

This research is partially funded by the Research Fund KU Leuven, and by the Flemish Research Programme Cybersecurity. Cybersecurity for Democracy at NYU’s Center for Cybersecurity has been supported by Democracy Fund, Luminate, Media Democracy Fund, the National Science Foundation under grant 1814816, Reset, and Wellspring. This material is based upon work supported by the Google Cloud Research Credits program. Victor Le Pochat holds a PhD Fellowship of the Research Foundation Flanders — FWO (11A3421N).

The authors of “A Standard for Universal Digital Ad Transparency” are: Laura Edelson (New York University), Jason Chuang (Mozilla), Erika Franklin Fowler (Wesleyan University), Michael M. Franz (Bowdoin College), and Travis Ridout (Washington University).

About Cybersecurity for Democracy

Cybersecurity for Democracy is a research-based, nonpartisan, and independent effort to expose online threats to our social fabric — and recommend how to counter them. It is a part of the Center for Cybersecurity at the NYU Tandon School of Engineering.

About imec-DistriNet, KU Leuven

Research at imec-DistriNet enables software, systems, and services that are secure, resilient, performant, and robust while open and adaptable. Fundamental, strategic, and applied research activities in the areas of Secure Software, Distributed Software, and Software Engineering converge in this perspective. In addition, imec-DistriNet delivers valorization in an industrial context. The imec-DistriNet research group is part of the Department of Computer Science at KU Leuven.

Click here to view the full paper, read a summary of the findings, and to learn more.

Cybersecurity for Democracy

We work with researchers, journalists & others to expose vulnerabilities to disinformation online.

Cybersecurity for Democracy

Cybersecurity for Democracy (cybersecurityfordemocracy.org) is a research-based effort to expose online threats to our social fabric — and recommend how to counter them. We are part of the Center for Cybersecurity at NYU.

Cybersecurity for Democracy

Written by

Cybersecurity for Democracy is a research-based effort to expose online threats to our social fabric. We are part of the Center for Cybersecurity at NYU.

Cybersecurity for Democracy

Cybersecurity for Democracy (cybersecurityfordemocracy.org) is a research-based effort to expose online threats to our social fabric — and recommend how to counter them. We are part of the Center for Cybersecurity at NYU.