My robot can tell the difference between Ba Gua (肉干)& Ba Zhang (粽子)!

Project Intro

Since CNY is coming soon and I have been missing home for quite a bit, so why not do a little machine learning project about local Malaysian food. This project is quite simple (with a complex theory behind it) and it basically boils down to using Google’s Tensorflow library InceptionV3 to identify images of either Ba Gua or Ba Zhang. Checkout my github code here.

“3 year old also can differentiate ba gua & ba zhang la!”

You might say, “3 year old also can differentiate ba gua & ba zhang la!”. Well that is true, but for a computer, it’s an entirely foreign thing to it and requires a couple hundred of training images for it to eventually recognize the object.

This is Ba Gua
This is Ba Zhang

So, without further ado, let’s get started:

I followed the steps from the codelab tutorial and modified the training images with about 700+ images of Ba Gua and Ba Zhang.

The codelab tutorial is called Tensorflow for Poets (pretty cool name huh) and you can check it out here.

How to download lots of image at once?

Well, there’s a nifty chrome extension called Fatkun Batch Image Downloader. You basically open a tab → google search the images → click on the Fatkun extension → tweak your options and boom! You get a bunch of images in your download folder. Pretty sweet!

The Code

The code is pretty simple and if you face any problem you should just refer back to the codelab tutorial. I’m conducting this project on my Mac machine so this tutorial is based on what I’m doing but other OS should be roughly the same.

Here are the steps:

  1. You need to install docker on your machine → Docker Page
  2. Make a directory called tf_files in your home directory and put your images in the directory.
cd ~
mkdir tf_files

When you ls your tf_files you should see your folders of images in it:

cd tf_files
tf_files $ ls
food
tf_files $ cd food
tf_files $ ls
ba_gua ba_zhang

3. Start a docker virtual container (virtual VM) and copy tf_files to your container

docker run -it -v $HOME/tf_files:/tf_files gcr.io/tensorflow/tensorflow:latest-devel

4. Retrieve the training code:

cd /tensorflow
git pull

5. Start training your model:

# In Docker
python /tensorflow/tensorflow/examples/image_retraining/retrain.py \
--bottleneck_dir=/tf_files/bottlenecks \
--how_many_training_steps 500 \
--model_dir=/tf_files/inception \
--output_graph=/tf_files/retrained_graph.pb \
--output_labels=/tf_files/retrained_labels.txt \
--image_dir /tf_files/food

6. Stop docker (press ctrl-D) and get the the classifying and predicting python file:

# ctrl-D to exit Docker and then:
curl -L https://goo.gl/tx3dqg > $HOME/tf_files/label_image.py

7. Restart docker and viola! You can now classify your image:

docker run -it -v $HOME/tf_files:/tf_files  gcr.io/tensorflow/tensorflow:latest-devel

Classify a Ba Gua:

# In Docker
python /tf_files/label_image.py /tf_files/food/ba_gua/pic_001.jpg

Or….classify a Ba Zhang:

# In Docker
python /tf_files/label_image.py /tf_files/food/ba_zhang/pic_001.jpg

8. You will get a result something like this:

For Ba Gua:

ba gua (score = 0.97737)
ba zhang (score = 0.02263)

For Ba Zhang:

ba zhang (score = 0.96333)
ba gua (score = 0.03667)
Damnnnn son….Your robot is now a Ba Gua & Ba Zhang expert!

Results:

Considering that the tensorflow model only ran 500 training steps, the results was really good. I can consistently get an above 0.9 prediction score for a Ba Gua or Ba Zhang picture.

I hope this post inspires you to do more fun stuff with machine learning! Feel free to use the pictures I downloaded from my github if you want to do a Ba Gua & Ba Zhang classifier as well.

Wow, I’m really getting hungry now after this project!

BONUS:

I am doing another InceptionV3 image classifying project on facial recognition on PM Najib and PM Lee. I will attempt to do face swaps, draw artificial facial hair, change picture angle and see how the machine and the model will fair. Stay tune~! Let me know if anyone has any interesting findings :)