Mario’s Artificial Intelligence Internship at Triple
My name is Mario, I’m 19 years old and currently doing an MBO-4 course in application development. For my final year, I need to finish with a 20-week internship. I chose Triple.
I started at Triple on the 1st of February. I’m currently halfway through the internship and although it feels like I just started, in the past 10 weeks I’ve learned a massive amount, developed myself, met tons of fun people and made some cool products!
First impressions
You feel the Triple vibe as soon as you enter the office. There are fun decorations, lots of LEGO constructions, plants, posters, you name it. It’s almost like a couple of gamers just decided to open up a software company… oh wait, that’s actually how Triple was founded.
What I do
I’m in the AI (Artificial Intelligence) team. My job is to research and realize project ideas that rely on machine learning. I research these ideas or problems, how they’re best tackled and develop the solutions. I generally make use of available tools or models and put them all together to develop a product.
Receipts API
My first project was a website where a receipt could be uploaded in order to get processed data in return. The idea behind this was that we wanted a website where users could send a photo of their receipt to prove that they bought a product from a specific company, in order to receive cashback on their purchase.
Images are processed using AWS Textract, with my own spice on top of it. Textract is a machine learning (ML) service that automatically extracts data from scanned documents. It goes beyond simple optical character recognition (OCR) to identify and understand the data. It’s quite easy to OCR receipts and invoices, but the hard part is understanding what the text is. Unfortunately training my own model is out of the question, it takes too long to implement something that complicated while also having the training dataset for it. One of the problems I encountered is that Textract sees all texts as the same. It doesn’t differentiate between handwriting and printed texts. So I manually trained a tool with images of receipts with and without handwriting.
The quality of some of the images was bad, resulting in Textract not detecting the text correctly. The solution to this problem was image enhancement, I achieved this with an image scaling and noise reduction model. This is what the difference looks like:
The processed data of a receipt is saved in the database and before it gets processed, the API checks whether the receipt was already processed before or not.
It’s also fun to think about some of the edge-cases, for example:
- How can you tell that a picture has a receipt?
- Is it a valid receipt?
- Does it have fake data?
- How can you tell where the items are on the receipt and tell what their prices are?
- What about different receipt formats?
- How should you verify users?
Wozzbot Intent Service
The second project I’ve worked on is an intent service for Wozzbot. You might wonder what Wozzbot is and what an intent service means. Wozzbot is a smart chatbot builder, it allows you to add sentences with labels which you can teach the bot.
Let’s say you own a coffee shop and you’d like the users to be able to make an order through the chatbot. So you create a new intent, called order, and add a list of example sentences that the user could send. You could have as many intents and inputs as you want. The bot then trains on this data:
What the user sends, doesn’t need to exactly match the input in the dataset. This is the cool thing about it, it’s NLP (natural language processing), meaning it tries to understand what the user wants with machine learning.
The chatbot work can actually get a lot more complex than that, this is just an example.
There are two problems with the dataset:
- Some inputs in a different label could have a similar meaning, resulting in a confused chatbot, not understanding what you really mean.
- Most of the inputs do not have a label at all, is there a way that I could suggest an intent (label) for these inputs?
Let’s say you have this in your dataset:
As you see, Input A is similar to Input D, this could cause problems with training and deploying the bot. But what even decides that they’re similar? I use Google’s USE Multilingual (universal sentence encoder), the cool thing about this model is that it’s trained on 16 different languages. Combined with NumPy, the similarity matrix is calculated by finding the inner product of two encoded sentences. This is the result:
You give it two sentences with a custom threshold, if the similarity score is higher than the threshold, the sentences are similar. Now all I need to do is iterate through all the inputs that are in a different label and compare them with each other.
Suggesting new intents for inputs with a label is done in a similar way. Let’s say you have these inputs:
As you can see, the first 3 inputs do not have a label, but the second 3 are all labeled. So how can we suggest a label for these 3 labels? By comparing them with all the other labels!
Input A is compared with Input D, E and F. The one with the highest similarity score is seen as the label.
Events
At Triple, people like to have fun and celebrate whatever they can celebrate and I’m all for that. Here are some of my favorite events.
Funky Friday
Funky Friday is a monthly Friday afternoon drink, each edition has a different theme. It takes place on Friday afternoons, and the theme will be announced in advance. The last one was a Carnaval edition! Everyone dressed up in a fun outfit and we all had drinks in the canteen.
Easter Lunch & Easter Hunt
To celebrate the Easter weekend, the office was full of nicely set tables with delicious sandwiches, eggs and other delicacies for lunch. It was a treat for sure.
During the week of April 11, 3 (chocolate) easter eggs were hidden in the office. Each egg had a different color and therefore represented a different gift, which the finder could collect.
Tropical Tuesday
A weekly event where people get to order food and work on their hobby projects! We recently celebrated the 300th edition of Tropical Tuesday, which was a very special one. We teamed up and played games against each other.
Conclusion
That’s how my experience has been so far. I have about two months left, I’ll be working on two other projects and besides that, you can frequently catch me playing pingpong in the canteen.
If you’re looking for an internship or a new job, I highly recommend Triple. Get ready for constant static shocks around the office because of the carpets though…
Want to have a look? Visit https://jobs.wearetriple.com.
