Internet of Things
Smart Garage Control With Firebase AutoML Vision Edge and RaspberryPi
2020 is the year where Internet of Things and Artificial Intelligence continue to expand their influence on our lives. From WiFi-controlled light bulbs to face recognition door locks, regular household items are getting smarter and smarter. Wouldn’t it be nice to have smart garage doors that remind us if we leave them open for too long?
Getting started
The traditional approach involves installing additional sensors on the garage door and connecting them to a central control hub. In this post I will explore a more cost-effective approach — using a Raspberry Pi, a web camera and Firebase, with some magic using machine learning to send us reminders when we forget to close our garage door. It is also a good way to get some hands-on experience with machine learning.
To do this, we need to create an on-device machine learning model that can determine if a garage door is closed based on a picture from the webcam.
Compared to a cloud-based model, an on-device model allows us to get the result without ever sending the image over the network, making prediction results available offline.
To start, we will need to collect enough training data of different states of our garage doors. In this case, with two garage doors, I have 4 states in total.
Collecting training data
I’m using a regular webcam in this example, but if your garage has poor lighting, an infrared camera might provide better results.
After connecting the webcam with the Raspberry Pi, I secured it somewhere that has a good angle on both doors. Then I used crontab and a simple bash script to periodically take images as my training data.
To see my data without having to pull out the SD card, I chose to also upload all the images to a Google Cloud Storage bucket, but this step is optional.
When collecting data, make sure you have images capturing conditions with different lighting, weather, cars and other objects at different positions. If all your sample images with an open door also happen not to have a car inside, the model might incorrectly “learn” that the door is open if there is no car, and closed if there is a car inside the garage.
Train the model with AutoML
After collecting about 400 images in many different conditions, I uploaded them into Firebase AutoML Vision Edge and had it automatically train a model for us.
- Upload the data to the Firebase console. To label your data, you can choose to put images for each label into a separate folder (e.g. put the images of the open garage door in a folder named “open” and the closed on in “closed”). Then zip together all folders and upload to Firebase. The following diagram explains the accepted zip structure.
Alternatively Firebase provides a nice UI that lets you label your images on the fly. Pro tip: you can also shift+click to multi-select!
- Configure training options. Choose the model type (latency vs accuracy) and the number of hours to train (more images will require a longer training time). In this case I selected high accuracy because compared to accuracy, inference speed isn’t important for our use case. To learn more, click here for a rough guideline on how long to train your model for and here for how to balance between model latency and accuracy.
- Evaluate the model. After training for 1 hour I was able to get a model with 95.62% accuracy! I’m pretty happy with this result. So this will be the model I use to monitor my garage door. The confusion matrix also suggests my model sometimes confuses closed with left_open, so I could consider adding more training data for these two labels.
- Test the model. Just to be sure my model is giving the right results, I used the Test model UI in Firebase to test my model with some new pictures. For realistic results, make sure you use an image that’s from the same camera/angle, but different from the ones you uploaded while training the model.
Using the model
If you want to try running the model yourself, the trained model and sample test images are available in this Github repo.
Now that I had a model ready to go, I downloaded it onto my Raspberry Pi. Then I set up another cron task to take a picture every minute.
After installing the necessary dependencies using:
pip install tensorflow numpyThe following Python code will use the picture to determine if the garage doors are closed using the freshly trained machine learning model. When the garage door is open, the script will also use a custom sms module to send me a text reminder.
Finally, I opened my garage door and ran this script on my Raspberry Pi. Woohoo! I got a text reminding me to close the garage door!
Future work
This was a fun project to work on while social distancing at home to get more hands-on knowledge about the power of machine learning. Using the same setup, one can also leverage AutoML to do more cool things such as identifying vehicles or people. If you have any other cool ideas, please share them with me in the comments below!
