My Meaningful Internship with DSAID VA Team

Hi! I am Varshini, currently a final year student majoring in Electrical and Electronic Engineering at Nanyang Technological University. Being really passionate about AI and development, I decided to pursue an internship with GovTech under the Data Science and Artificial Intelligence Division Video Analytics (VA) team in January 2023 and took on the role, AI Engineer (Engagement Management).

The experience that I gained over the past seven months was nothing short of interesting or invaluable and I would really love to share more about it here!

My Role in the Engagement Team

Firstly, being part of the engagement team under VA, I had the opportunity to be involved in engagements with various agencies. I kept track of the different use cases presented by different agencies and how they proposed using video analytics to aid them. I also observed how my team analysed the use cases presented, how they went about presenting the capabilities that VA offers and how they proposed moving ahead with the next steps.

Image Classification Project

As an AI engineer I also worked on AI and development projects during the course of my internship.

The first project was an image classification use case where I worked alongside another fellow intern, Yu Hoe. We were tasked with developing a model with the aim of comparing relevancy between user-submitted images and agency-submitted images. We were given a dataset containing 3000 cases, where each case was pre-classified as ‘Relevant’ or ‘ Irrelevant’. We experimented with different models to select the one which achieved the best results.

One experimentation I tried out was to make use of Convolutional Neural Networks (CNN) to train an image classifier. To do so, I split the available dataset into 2 parts, one to train and the other for testing. One of the key challenges which I faced during this process was that the dataset was not balanced between the two classes, thus affecting the accuracy with which the model classified the images.

To overcome this problem, I learnt about and tried methods such as introducing focal loss to balance the weights of both the classes. However, we learnt that another model that utilises Contrastive Language-Image Pretraining (CLIP) outperformed the CNN model.

Throughout the process, I gained more knowledge about image classification and model development which I would find useful when I work on other projects in the future after my internship.

Chatbot for AI

Another project that I worked on was coming up with a Chatbot service using Telegram for a use case by Singapore Food Agency (SFA) to automate the task of manually counting rotifers. For some context, rotifer is a type of zooplankton, which is the first feed for fish larvae and plays a crucial part in large-scale fish hatchery production.

This project was particularly memorable as I worked on the entire project right from the Proof of Concept stage all the way until actual deployment of the Telegram bot.

The project utilised one of our everyday applications, Telegram. All the user would need to do is to take a picture of their sample of rotifers and upload it onto the bot to obtain the rotifer counts instantly. The key idea was to perform image analytics on the image uploaded by sending it to the dedicated Rotifer AI model hosted in the Video Analytics System (VAS) server and returning the result. Sounds simple? But it was not as straightforward and easy as it seemed.

I still remember being extremely intimidated when I was first given the use case to work on as I had no prior experience working with Telegram bots. It was definitely not an easy task to learn something entirely new and come up with a working prototype from scratch within a really short time span. However, with the guidance and suggestions provided by my supervisor, Yong Kiat and colleagues, I could get started and show that it was possible to develop and deliver the project. The actual development work began soon after this.

One of the interesting parts was coming up with a method to log into the system. The unique thing about it was that the bot did not make use of the conventional method of username and password verification. Instead, all the user had to do was upload their own contact to the bot through a dedicated “send contact” button. Since every user on Telegram is tied to a unique user ID, the user is verified by checking if the user ID of the contact sent and the chat ID of the messages sent on the bot are the same. We also added a second layer of security to send an OTP once verification is complete. Upon keying in the correct OTP, the user will be able to view the various functions available for their user type, depending on whether they are “admin” or “non-admin”.

You can see from this picture how a typical message will look like after an admin successfully logs in.

Another interesting part of the development process was the main feature of being able to call the Rotifer model when uploading a picture of rotifer samples in a petri dish. It was during this part that we discovered that Telegram significantly downsized the images uploaded before they were sent to the VA server. As such, the results returned after running the model on the images differed greatly from the ground truth. To resolve this issue, we enabled the option to upload the images as a file. This method of sending in the images prevented them from being downsized and hence allowed more accurate results to be returned.

The pictures below show how it would look like when the user uploads an image and how the returned results look like (with the user’s image cropped, rotifers identified with the bounding boxes and the text message with the relevant counts). How cool is that!

Once we were sure that all the intended features were functioning properly in our local environment, we proceeded to host it on cloud services which hence made our bot “live”.

The most memorable part of being part of this project was the visit to SFA fish farms to get farmers onboard and test the Telegram bot with their rotifer samples. It was extremely satisfying and rewarding when we received feedback that the telegram bot that we had developed was simple and straightforward to use, and returned accurate results.

While I had initially started off alone, Kenneth, a newcomer into the team joined me in this project. I am really grateful for the opportunity to work with him as his expertise and prior experience on working with Telegram bots helped me learn about the best practices to improve code security and improve functionality. Not forgetting the help from Mandis, another fellow intern who rigorously tested the bot that we had developed and found many edge cases and bugs. This helped us as we could work on including more error handling so that the bot did not crash.

Activities outside work

Although the work and projects that I was part of contributed to the bulk of my time during the internship, there were other things which also added value to my time here.

Apart from my projects, I also attended the monthly AI Sharing sessions by the various engineers in my team. I really enjoyed learning about all the new and rapidly evolving technology and how they could be relevant to the work that we do here. Through my interactions with the other interns in VA, I also learnt more about the projects which they were working on and gained more insights about AI and development.

If there is time for work, there is time for play too! I definitely had lots of fun during the occasional team bonding activities where we took some time out of work to interact and bond with one another. Particularly, the time when we went to an escape room and headed to Fort Canning Park for a “Games day” session would be one that I would fondly look back at!

Key Takeaways

Through this internship, I am certain that I have learnt and gained a lot. Being part of the engagement team helped me improve on my communication skills while the projects which I worked on helped me gain new technical skills.

If you want to stretch your horizons and love stepping out of your comfort zone apply to GovTech and be assured that you will not be disappointed! Find out more about GovTech’s internship programme at go.gov.sg/GovTechInternship.