My DSAID Internship Experience
This article was originally posted by Chow Keng Ji on DSAID’s Medium blog.
This summer, I worked for twelve weeks as a software engineering intern with the Transcribe team in GovTech’s Data Science and Artificial Intelligence Division (DSAID). While other internships may have been affected by the circuit breaker measures, GovTech’s internship programme went on digitally with work from home arrangements. Apart from some missed opportunities for face-to-face interaction, my internship was nevertheless a fulfilling one as the team kept in close contact through online channels.
What I did and learnt
The team is currently working on a Speech-to-Text platform built with a microservices architecture. For my internship, I was given two assignments to work on. The first assignment involved developing API test scripts using the k6 Javascript framework for load testing. The framework allows developers to use scripts to simulate the behaviour of multiple users concurrently sending requests to the system. Before I began writing tests, I was tasked to understand the basics of Docker and containerisation. I had to understand Docker so as to connect my test scripts to the component to be tested, as well as connect to a stub service I wrote in Python Flask.
After I was done with API testing, I started learning Go, which is known for its in-built support for concurrency. I then practiced Go by implementing a simple API with a database and object storage. The next task was to design and plan a wrapper which would serve as the external API of an existing transcription engine. This involved thinking about how to ensure that transcription jobs could be distributed over multiple instances of the engine.
The task was challenging as I did not have much experience with architecture design. I wrote several drafts of the proposal, improving on them based on feedback and comments from my mentors. The final approach to task distribution involved using AWS SQS as a queue service shared between the engine instances. It was structured using a hexagonal architecture, also known as a Ports and Adaptors architecture, which prevents circular imports and ensures modularity and testability of each of the application components.
After finalising the design during an engineering discussion with the software engineers in the team, I started implementing the overall project skeleton. By the end of my internship, I had contributed to developing the queue, file storage and database components of the project.
Through the internship, I picked up two new programming languages (Javascript and Go) and learnt how to use Docker to spin up services such as API servers, databases and object storage for local development. As the projects I had previously worked on were self-contained monolithic applications with minimal dependencies, I did not have prior experience working with Docker or external services provided by AWS. Going through the process of designing the engine wrapper with considerations such as concurrency, application deployment and the microservices architecture was thus an eye-opening experience.
Life at work (from home)
The work from home arrangement did not compromise the quality of the internship, as my mentors were approachable and always contactable via Slack to respond to any questions I had. I worked mostly independently on the assigned tasks and made code contributions via pull requests through a Github repo, where my mentor gave detailed feedback through Pull Request (PR) reviews about best coding practices and software design. We also communicated through voice call whenever there was a need for more in-depth discussion.
In this arrangement, I was trusted to work with new tools and programming languages that I only picked up during the internship. I was also not given strict deadlines and had the flexibility of setting my own targets on a daily basis. Every morning, I attended the virtual stand-up meeting where team members would report on their progress and their plans for the day. There are also bi-weekly sprint retrospective meetings where the team meets to consolidate their accomplishments for the sprint and discuss next steps.
During two retro meetings, I presented my work on API testing to the team, and received insights and feedback from other team members who did not supervise me directly. For instance, during the first presentation, I received advice to pay more attention to simulating typical user behaviour as well as the endpoints triggered by GUI buttons on the front-end. During a second presentation, I shared the results of running load tests on the production system. It was only during this presentation that I learned that the performance benchmark results for upload and download of audio were poor from a user’s perspective, and that the results were likely affected by a bottleneck caused by my local network.
Beyond the regular stand-ups and retros, there were team level engineering discussions which provided a big picture view of the project and insight into how project planning is carried out. I also attended several monthly engineering sharing sessions as well as an intern brown bag session organised by GovTech’s HR team. These talks provided exposure to projects that other DSAID teams are working on, as well as other areas outside my job scope, such as front-end development.
Conclusion
Through this 12-week stint, I gained insights into the job scope and expectations of working as a software engineer, and was exposed to some tools and concepts that the core university curriculum does not typically introduce. More importantly, I was given the opportunity to learn new skills to be able to contribute meaningfully to the project. This made the internship a treasured and valuable learning experience.
