The Segments.ai Plugin for FiftyOne
At Voxel51, we created FiftyOne to be the most pluggable and extensible tool on the planet to refine datasets and models together. Why? Our ultimate goal is to help AI/ML builders create the highest-performing AI possible.
That’s just one of the reasons why we’re excited about this new plugin…the Segments.ai plugin for FiftyOne!
We caught up with the plugin builder-Tom Roussel, Machine Learning Engineer at Segments.ai-to ask about the plugin and what users can expect. Let’s dive in!
First, for anyone who is new to Segments.ai, can you share some highlights of your offering?
Tom: Our solution is focused on multi-sensor labeling specifically. Our customers are typically building a vehicle or robot with multiple camera, lidar, and radar sensors.
With multi-sensor labeling, customers can annotate different sensors in a single interface, getting consistent labels across all sensors. The main benefits include:
- Label multiple sensors simultaneously
- Visualize multiple sensors together
- Get consistent IDs across modalities and time
- Save time on quality control and edits
Here’s an overview of our products:
- Labeling Tooling: We provide multi-sensor labeling tooling, ready for use by internal, external, or computer vision engineering teams
- Labeling services: We can help you find the right workforce, onboard them quickly, and set up your model-assisted labeling and active learning pipelines
- Labeling advisory: We bring our Ph.D. expertise along to help customers improve their labeling specifications, pipelines, and strategies
To learn more or get started, head over to the Segments.ai website.
Tell us about the Segments.ai plugin for FiftyOne.
Tom: Because data curation and annotation go hand in hand, it’s natural that we want to reduce the friction between these two critical ML tasks. The Segments.ai plugin allows the user to use the powerful set of data curation capabilities FiftyOne provides and easily use our extensive data annotation capabilities to iteratively improve the quality of their datasets.
The best part? The integration is completely seamless, enabling customers to request annotations directly within the FiftyOne interface and eliminating the need to switch between tools.
What are the top benefits the Segments.ai plugin brings to customers?
Tom: This plugin gives customers the ability to connect data exploration, annotation, and quality control in one smooth process, which reduces friction and dataset errors, leading to better datasets and faster iteration cycles.
That said, here are some top benefits the Segments.ai plugin offers for customers:
- A seamless way to integrate data curation capabilities in our customer’s data annotation workflow
- Less friction between finding interesting/hard cases for models and having them annotated
- Less context switching when curating a dataset and communicating with the annotation team
- No separate scripts are needed for Segments.ai, as it’s a one-click integration
Describe how using the plugin works.
Tom: After installing the plugin, users start by uploading a dataset from FiftyOne to the Segments.ai annotation service. They can either create a new “dataset” on our platform or append the data to an existing one. Afterward, they can interact with the Segments.ai dataset in a few different ways.
The user creates a persistent link between a FiftyOne dataset and a Segments.ai dataset. This is a little different from the way FiftyOne integrates with other annotator backends, where the dataset concept in the annotation backend may not be as long-lived. The user can then fetch annotations made on the Segments.ai platform.
Lastly, our platform supports “Issues” linked to samples in datasets. This is a mechanism to communicate errors, edge cases, etc., with the annotation team. Our plugin allows the user to create issues from within FiftyOne by selecting a sample and entering a description of the issue.
What led to the development of this plugin?
Tom: We had multiple customers who contacted us asking if we had an integration with FiftyOne. After reviewing the FiftyOne platform, we believed it would be helpful to tightly integrate our service with it. FiftyOne’s plugin system gave us an easy way of developing this integration.
Please share any resources where people can get started with the Segments.ai plugin.
Tom: There are a few resources we put together for people to get started with the Segments.ai plugin for FiftyOne:
- We posted an announcement blog that includes helpful tutorial videos on how to get started
- When you’re ready to install and configure the plugin, head on over to the GitHub repo
- If you’d like to see the plugin in action, we’d love to show you-reach out for a demo
Originally published at https://voxel51.com on September 25, 2024.
