Discover fastdup Advanced Features: Now Available for Free!
Over the past few months, we’ve been hard at work testing out our beta features, refining their performance, and ensuring they’re ready for prime time.
📣 Today we’re excited to announce all advanced features are out from beta testing and are now available to the public, completely free of charge! To access, head to fastdup GitHub repo.
Here’s a video in a nutshell.
About fastdup
If you’re new, fastdup is an unsupervised and free tool for image and video data analysis.
Over the past year, fastdup has amassed over 247,000 downloads on PyPI and has been used to analyze over 50 billion images in total. This makes fastdup one of the fastest-growing computer vision dataset analysis tools.
If you’d like to try it out yourself head over to fastdup GitHub repo to get started.
Overview of Advanced Features
Today we are releasing many exciting new features that passed our internal beta testing phase.
We categorize these features into 4 sub-categories namely:
- Hugging Face Datasets Support.
- Data Enrichment.
- Embeddings.
- Visual Search.
Hugging Face Datasets Support
Hugging Face Datasets is one of the most widely used libraries to access and share computer vision, audio, and NLP datasets. You can easily load any of the 46,385 datasets (as of July 2023) with just a single line of code.
But what if you’d like to analyze the dataset for issues like duplicates, outliers, or other quality issues?
Now you can easily do that with fastdup.
We loaded the tiny-imagenet dataset from Hugging Face to show how easy it is and analyzed it for issues.
View the notebook on nbviewer or run it yourself on Google Colab.
Data Enrichment
Computer vision datasets are notoriously tricky to analyze. This is when data enrichment plays a role in enhancing the understanding of the images.
With fastdup’s new features, you can enrich your computer vision datasets by adding captions to your image using the BLIP model, detecting visible texts on your images with PaddleOCR, detecting if there are human faces present in the dataset, and detecting if there are objects from the COCO dataset using a pre-trained YOLOv5 model.
By enhancing these images with additional information we can improve the search and retrieval capabilities for images in the dataset. For example with captions, we can easily find relevant images using text queries.
Links to the notebooks are as follows:
- Face Detection in Videos — nbviewer, Google Colab.
- Object Detection in Videos — nbviewer, Google Colab.
- Optical Character Recognition — nbviewer, Google Colab.
- Captioning with BLIP — nbviewer, Google Colab.
Embeddings
Another strategy to enhance search capabilities for image datasets is to use embeddings. Embeddings refer to the vector representation of images that capture their semantic and contextual information.
Using fastdup’s new features you can extract feature vectors of your dataset using the latest DINOv2 model, or any model you prefer. On top of that, you can also use fastdup to read feature vectors and use them for further processing.
Links to the notebooks are as follows:
- DINOv2 Embeddings — nbviewer, Google Colab.
- Use Your Own Feature Vectors — nbviewer, Google Colab.
Visual Search
Finally, we are also happy to release one of the most requested features — large-scale visual search. You can now search through large datasets for similar-looking images using fastdup, on a CPU.
In our example notebook, we searched through the Shopee Price Match Dataset with 32,418 images for similar images in seconds.
View the notebook on nbviewer or run it yourself on Google Colab.
Community and Support
We’re incredibly grateful for our community. Your feedback and contributions have been instrumental in shaping these advanced features.
If you have any questions, feedback or need help, don’t hesitate to reach out via Slack, GitHub, or our discussion forum.
We are also present on LinkedIn, Twitter, or YouTube.
We’re excited to see how you’ll utilize these new features in your projects. Thank you and happy data exploring!
