Adversarial attacks and software vulnerabilities are only part of the problem, so in this audit we also considered risks posed by hardware

Go ahead. Bring on the complexity.

In May 2023, IQT Labs released a report detailing the results of our AI assurance audit of SkyScan, a system that collects and automatically assigns labels to images of aircraft. By simultaneously capturing ADS-B (Automatic Dependent Surveillance–Broadcast) signals via a software-controllable radio receiver and images of aircraft in flight, SkyScan can generate an auto-labeled dataset that can be used to train computer vision models to identify various types of aircraft.

As in IQT Labs’ prior audits (FakeFinder, a deepfake detection tool and RoBERTa, a large language model) we assessed a variety of risks, vulnerabilities, and potential concerns posed by the SkyScan system, including:

• the security of hardware in SkyScan’s “Capture Kit”;
• the security of software components in an ML pipeline built from SkyScan data;
• the ethics of auto-labeled data; and
• how collection biases in an auto-labeled dataset might introduce biases in the inferences drawn by a classification model trained on that dataset.

One key finding from this work is the many ways that hardware concerns complicate the auditing process, by adding complexity and increasing attack surfaces. To characterize these risks, we divided them into three categories: technical, mechanical, and architectural. Then, to help us assess the severity and prioritize mitigation efforts, we designed and implemented 10 different attacks –- from GPS and ADS-B spoofing to Model Evasion, Data Poisoning, and a data-science twist on a MITM attack that we called “Model-in-the-Middle.”

The full report is available here.

For more on IQT Labs’ AI Audits, check out Interrogating RoBERTa: Inside the challenge of learning to audit AI models and tools and What happened when we audited a deepfake detection tool called FakeFinder.

— —

A version of this post was originally published on the IQT blog in May 2023.
Image by Domagoj Ćosić on Unsplash.