Try it Yourself: 11 Brick Image Set for Image Recognition and Neural Network exploration

Paco Garcia
Sep 3, 2018 · 3 min read
Download the Lego Image Set

This work is licensed under a Creative Commons Attribution 3.0 International License.

Updated 12/8/2018:

I have now uploaded all the data sets I’ve used to Kaggle, you can download them here:

20 Brick Types for Sorting (4,500+ images)

Lego vs. Generic Brick Identification set (20,000+ images)


5 Parts to This Blog

This part of a 5 Blog Series to cover the mechanical and software design for the Lego Sorter, as well as sharing the training set and some evaluation sets:

1. Lego Sorter using TensorFlow on Raspberry

2. Mechanical Separation (Design, Motors and Sensors)

3. Overview of the Software stack

4. Using Inception V3 to Identify LEGO vs. Generic Bricks

5. Try It Yourself: 2 Big Data Sets so you can Replicate this Project


Perform Image Recognition

This image set is a fantastic opportunity to become familiar with image recognition techniques. Here are some ideas you can test:

There are multiple ways of detecting an object in a image such as Contours and Edge detection. Can you identify the object? Calculate its are? Crop the image?

This would probably be easy once you can identify objects. You will realize that in order to train a Neural Network on this set, you might want to analyze each image contains only 1 brick type.

Identifying the LEGO brick doesn’t necessarily need a Neural network and it is possible to use Image Recognition techniques (combined with geometry) to correctly identify the bricks.


Train and Validate your own Model

I decided to use TensorFlow and Inception V3 to train my model, but there are many other options you can use.

You can choose another neural network (build your own or re-train ones such as VGG16, VGG19, ResNet, Xception, etc.) and also use different frameworks like scikit-learn and XGBoost

I have included an actual set captured from two evaluation runs of my LEGO Sorter. Use these sets to validate your trained Neural Network.


Identify Lego Brand vs non-Lego Brand pieces

I purposely built a sample of bricks that included LEGO Brand and non-LEGO brand.

This would definitely be for bonus points as it is probably ambitious given the size of the data set, but if it’s possible, it would be very interesting to learn how you solved it!

Can you identify the Branded LEGO from the rest?

5 Parts to This Blog

This part of a 5 Blog Series to cover the mechanical and software design for the Lego Sorter, as well as sharing the training set and some evaluation sets:

1. Lego Sorter using TensorFlow on Raspberry

2. Mechanical Separation (Design, Motors and Sensors)

3. Overview of the Software stack

4. Using Inception V3 to Identify LEGO vs. Generic Bricks

5. Try It Yourself: 2 Big Data Sets so you can Replicate this Project

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade