Introducing Tagbox Custom Tags

TL;DR: You can now teach Tagbox to recognize new things, with the new /tagbox/teach API.

You may have heard of Tagbox before. We released it a few months ago. You can use it in production to get tags that describe the contents of images (like pizza, car, ball, etc). It is pre-trained with 1000 ImageNet tags.

For example, I used Tagbox last week at the London Go Meetup to bust the myth that machines have problems distinguishing between Chihuahuas and Muffins. Turns out that identifying dogs is pretty easy for a machine. Tagbox is 100% correct!

Tagbox is awesome for general purpose use; improving search, better SEO, and relating content. But there are a few drawbacks:

• Tags sometimes are very specific. For example, you want to identify that in an image there is a monkey, or a bird, but you are not interested in the type of monkey or bird.
• Tags can be very general, but you might want them to be more specific. For example, you want to automatically classify images from ‘The Simpsons’ vs ‘Family Guy’.
• Or they could be wrong if Tagbox returns an incorrect tag for a image (false positive).

Using your own Tags

If you’ve used Facebox before, we have an endpoint to teach the box to recognize new people, and we use similar technology, to allow you to teach Tagbox your own tags.

The new /tagbox/teach endpoint allows you to POST any image and tag (you can try it in the console.)

Using your own tags you are able to:

• Generalize your image classification problem, using tags that are relevant to your business.
• Solve simple but specific use cases like classifying concepts that Tagbox would not support before.
• Make corrections in cases where Tagbox returns false positives.

As an example, let’s tackle the first problem: we want to tag some monkeys!

Teaching Tagbox this little baboon monkey

Without teaching, Tagbox will return the tag baboon for the above image. But we want the tag monkeys to return anytime any type of monkey is posted to Tagbox. We can use the console to teach that the image above belongs to the tag monkeys.

Of course if we call /tagbox/check with the same image that we used for /tagbox/teach we get 100% confidence that the image belongs to monkeys. That’s easy. But you realize the true power of Tagbox when you use a different monkey.

Instead we are going to call /tagbox/check with the monkey that we have on the Samples bar (a spider monkey).

We can see that now Custom Tags are returned, and the value of that tag is monkeys, so we are able to use our own tags for any concept that we’d like.

The more samples you provide of the same tag, the more confident Tagbox becomes.

Some of the successful use cases that we tried are:

• We were able to identify photos with Spiderman on the foreground, just teaching two examples of Spiderman.
• We were able to distinguish images from Homer vs images from Peter Griffin (everybody has this problem, don’t they?).
• Classify different images from sports like football vs basketball.

But Tagbox is not the silver bullet of image classification, for some cases /tagbox/teach is not enough, since you can’t beat the accuracy of a custom trained model. Try it to see if it fits your needs, and contact us with any feedback or questions; we can advise you if your case is likely to work, or if we’ll need to build a custom model for you (we do that too).

Search by visual similarity

But wait, there’s more. We implemented /tagbox/similar endpoint, which allows you to get all similar images ordered by confidence.

/tagbox/similar makes easy to implement a visual search engine

That makes it really easy to implement a visual search engine, or to recommend similar images to your costumers. It also gives you a sense of what Tagbox is learning with all those /tagbox/teach calls.

How can I get or update Tagbox?

Signup for an account on https://machinebox.io to get a MB_KEY. Then, with Docker installed, run:

MB_KEY="my_key_here"
docker run -p 8080:8080 -e "MB_KEY=$MB_KEY" machinebox/tagbox

If you already have Tagbox, do a docker pull to get the latest update.

docker pull machinebox/tagbox:latest

Some limits are applied

We want to make Machinebox a sustainable business so for the FREE tier, we’re limiting teaching to 20 images. If the limit is getting in your way, tell us what are you building and we’ll work with you to make sure you aren’t blocked. Otherwise, signing up for a subscription on https://machinebox.io will remove all limits.

We love feedback!

Please tell us if this is useful to you and what your use cases are, we might be able to make your life easier https://machinebox.io/contact.