Sitemap
Voxel51

News, tutorials, tips, and big ideas in computer vision and data-centric machine learning, from the company behind open source FiftyOne. Learn more at https://voxel51.com

ImageNet-D: a new synthetic test set designed to rigorously evaluate the robustness of neural networks.

7 min readFeb 11, 2025

--

ImageNet-D parsed into FiftyOne format
Examples from various synthetic image benchmarks compared to ImageNet-D, taken from Figure 2 in the paper

Image Generation by Diffusion Models

The ImageNet-D creation framework, taken from Figure 4 in the paper.
Prompt format used to create synthetic images. Source.

Hard Image Mining with Shared Perception Failures

The Mining Process

Quality Control by Human-in-the-Loop

The UI for Mechanical Turk workers. Source.

How to Use and Interpret Results

Comparison of model performance on ImageNet-D. Source.

Next Steps

Exploring the ImageNet-D dataset in the FiftyOne app.
import fiftyone as fo
import fiftyone.utils.huggingface as fouh

dataset = fouh.load_from_hub("Voxel51/ImageNet-D")

# Launch the App
session = fo.launch_app(dataset)

Conclusion

--

--

Voxel51
Voxel51

Published in Voxel51

News, tutorials, tips, and big ideas in computer vision and data-centric machine learning, from the company behind open source FiftyOne. Learn more at https://voxel51.com

Harpreet Sahota
Harpreet Sahota

Written by Harpreet Sahota

🤖 Generative AI Hacker | 👨🏽‍💻 AI Engineer | Hacker-in- Residence at Voxel 51

Responses (1)