A Telegram Bot with the Power of Computer Vision

Snap-it Find-it: Your Shopping Companion Bot

This is a series of Medium posts made by 4 NUS SCALE master students (MSc. in Industry 4.0) who are taking ISY5004 Intelligent Sensing Systems. Here’s a snapshot of what we’ll be sharing:

• Web scrapping, feature extraction using CNN, and image searching.
• Object detection using YOLOv3.
• Object detection and background removal using Detectron2.
• Further improvement using various CNN models.
• Telegram bot and hosting on cloud. (this post)

Thank you for reaching the last article of our posting. Now, we have all the elements, let’s build a telegram bot and integrate them all together, shall we?

There are quite a number of posts on Medium teaching how to create a telegram bot, you may search and read it. But I am going to show you the simplified version of it… First, you would need to register a new bot with BotFather.

The token is needed to communicate with the bot, please keep the token privately.

After creating the robot, you may chat with BotFather to update the profile picture, command, description, and others.

Now, you would need to install a very nice wrapper for Telegram API and it’s call python-telegram-bot which can be installed using the following code:

After this, we can start to integrate all of our hard work into a main.py which is also our Telegram bot script:

There are quite a few sample Telegram bot scripts that you can make reference to:

python-telegram-bot/python-telegram-bot

Here’s a quick look at how a user can interact with the Snap-it Find-it Telegram bot:

Snap-it Find-it allows a user to simply take a photo and upload it to search for similar products from Ikea and Hipvan.

Viola! Here is it! Now it would be better to host this bot on cloud so that we won’t have to run this on our computer isn’t? Let’s do it by deploying it to Google Cloud Compute!

This is the setting we used on Google Cloud Compute to set up our VM.

After we provision the VM, we can SSH to the VM and install the following dependencies:

Now we can upload or git pull the repository and use screen to run the main.py script in the background so that the script won’t stop after disconnection from SSH. (refer to this post on StackOverflow)

That’s all!

You may refer to our GitHub for this project.

Special thanks to our professors from NUS Institute of Systems Science (ISS), Dr. Tian Jing and Dr. Jen Hong for teaching and guidance us in making this project a successful one!