Unleash Data Potential @ Analytics.gov — My Experience as a Software Engineer Intern
Hi everyone! 😀 I’m Pun Pun, a third-year Computer Science student at the National University of Singapore (NUS).
For the past 6 months, I had the opportunity to work as a Software Engineer Intern under Analytics.gov — a whole-of-government (WOG) data exploitation platform developed under GovTech’s Data Science and Artificial Intelligence Division (DSAID).
Let me take you through my experiences during this internship journey!
Background
As a Computer Science student, I have always been deeply interested in the fields of Software Engineering and Data Science. Throughout my search for an internship, GovTech DSAID has consistently remained my dream place to intern due to its perfect alignment with my career aspirations and areas of interest. I was particularly drawn to the opportunity of having a holistic internship experience that would allow me to explore and gain insights into both fields.
What further captivated me was the overwhelmingly positive feedback from my friends and schoolmates who had previously interned here. They spoke highly of the supportive and friendly culture that permeates each team. Considering the substantial duration of a 6-month internship, I felt confident that GovTech DSAID would provide an ideal environment for my technical and personal growth.
On a personal level, I believe the opportunity to work on products for the public sector is meaningful. I was driven by the prospect of contributing to the development of solutions that align with my interests, while positively impacting the lives of the people in Singapore. This motivation propelled me to apply for an internship position with DSAID, seeking an opportunity not only to learn but also to contribute to society through my internship.
Motivation
Public agencies often face challenges when analysing data on their GSIB (Government Standard Image Build) — mainly its limited capabilities to download packages and resources from the internet and lack of dedicated hardware to run computationally intensive projects. To address this, Analytics.gov was developed as a secure and user-friendly platform that enables public agencies to support their data exploitation and analysis work beyond the constraints of their GSIB machines.
Overview of Analytics.gov
Analytics.gov is a data exploitation platform for the WOG use. It is a secured platform that enables public agencies to analyse data and develop insights through an easy-to-use and readily available environment accessible from their GSIB machines.
Analytics.gov offers a wide range of tools and features that can be customised to meet the needs of individual agencies, including a virtual environment for data analysis, a wide range of analytical tools, Integrated Development Environments (IDEs), and Machine Learning Operations (MLOps) capabilities.
Here are some of the key features that Analytics.gov offers:
- A diverse range of analytical tools, including visualisations, statistical analysis, and machine learning capabilities.
- Ample computing power for handling computationally-intensive analytics projects.
- Access to readily available and up-to-date code libraries and packages for efficient data analysis.
- Seamless code sharing and collaboration among users within and across agencies.
What I worked on
During my internship, I was involved in a range of tasks to incorporate new functionalities into the platform and elevate the overall user and developer experience.
1. Revamping Analytics.gov Landing Page
I had the opportunity to contribute to the revamp of the platform’s landing page leveraging Next.js and Tailwind CSS along with various packages. The new version of Analytics.gov not only improves user understanding of the product but also incorporates functionalities that streamline the user experience.
One key addition to the revamped landing page is the inclusion of informative sections that provide users with a comprehensive understanding of Analytics.gov. These sections highlight the platform’s features, benefits, and role in facilitating data analysis and exploitation for Singapore’s public agencies. By presenting this information clearly, we aimed to enhance user comprehension and engagement.
Furthermore, we implemented a search bar within the FAQs and announcements sections, enabling users to easily search for specific content they are looking for. This functionality significantly improves user navigation and saves valuable time by swiftly directing them to the relevant information they need.
This experience allowed me to strengthen my technical skills in front-end development, responsive design, and the utilisation of popular frameworks and libraries in real-world projects.
Apart from the technical aspects, I also had the opportunity to work closely with our product manager and I learned to take note of the minor details that could impact user experiences. These details might seem small, but they could significantly influence how users perceive and interact with the product.
2. SageMaker Pre-signed URL
During my internship, one of the new features that we aimed to include in Analytics.gov was the integration of the AWS SageMaker Studio — a cloud native service that allows agencies to work on their end-to-end Machine Learning (ML) projects and leverage the enhanced MLOps capabilities through a wide array of built-in packages, models, and pipelines, to streamline and optimise their ML workflows effectively.
To kick-start this integration, I worked on the creation of pre-signed URLs for the SageMaker Studio, allowing users to securely launch the studio from our landing page.
First, I developed a Lambda function that generated the pre-signed URL for the SageMaker Studio and integrated this with the API Gateway to expose it as an endpoint.
To enhance the security of the function, I implemented a robust authentication mechanism using the OAuth authorisation code grant workflow using AWS Cognito and leveraging the Government Commercial Cloud (GCC) as the identity provider. Users were required to authorise their access using their credentials, ensuring that only authorised individuals could generate the pre-signed URLs.
Through this task, I was able to deepen my understanding of server-less architecture and picked up the knowledge of AWS services. I also gained a comprehensive understanding of how to integrate robust authorisation mechanisms into applications using identity providers, including obtaining authorisation codes, exchanging them for access tokens, and verifying user identities.
3. Codebase Migration to SHIP-HATS GitLab and CI/CD
After the lambda function, I worked on the migration of our landing page code base from AWS CodeCommit to GitLab, implementing a robust CI/CD pipeline in the process. This pipeline facilitated seamless webpage deployment onto our Elastic Kubernetes Service (EKS) cluster which helped to efficiently maintain the build and deployment process of the landing page in different environments (i.e. dev, uat, and prod).
To ensure the security of our application, we integrated Static Application Security Testing (SAST) to scan the code base for vulnerabilities, and Aquasec Trivy scanners to detect vulnerabilities in the built Docker images before deployment. The pipeline triggered CodePipeline, allowing our EKS cluster to pull the latest image for deployment.
This experience provided me with valuable insights into DevOps, specifically, how to configure CI/CD pipelines, integrate security tools, and optimise the deployment process. By automating the deployment workflow and incorporating security measures, we achieved a more efficient and secure development process for our landing page.
Exploring beyond the technical aspects
As GovTech mostly operates in smaller teams, each product team is cross-functional with a mix of product manager and engineers. This allowed me to explore and observe the end-to-end process of how products are built from the ground up from ideation, planning, implementation, testing, and product launch.
From this experience, the key takeaways that I gained beyond the horizon of my software development work are:
Agile Methodology (Scrum): Daily Stand-Ups, Sprint Planning, and Retrospective
As part of the scrum process, we hold daily stand-ups for team synchronisation, sprint planning to prioritise tasks, and retrospectives to identify areas for improvement, ensuring efficient project management and collaboration. After learning about the scrum practices in school, I now understand how these practices are important in the real world as it facilitates better communication among team members and enables us to continuously adapt and deliver high-quality products.
User Acceptance Testing (UAT): Ensuring Product Quality and User Satisfaction
During the development process, one of the critical phases was to push our product to the UAT environment. During UAT, the product is thoroughly evaluated to ensure it meets the users’ expectations and requirements. This process helps us identify any potential usability issues, functional gaps, or performance concerns. By involving and thinking about end-users in the testing process, we gain valuable feedback that enables us to refine the product and enhance its overall quality before launching the product to the public in the production environment.
Marketing and Publicity: Creating Product Videos/Demos and User Guides
As Analytics.gov is a platform meant for technical data exploitation work, during my stint, I had the opportunity to help out with product videos/demos (e.g. Nexus repository, Git, and GitLab) for a visual demonstration to showcase the platform’s offerings. Moreover, I also had the opportunity to draft user guides to provide comprehensive instructions for potential users to use specific features on our platform.
Documentation: Recording Crucial Product Information, Developer Guide, and Best Practices
As our teams include many engineers working collaboratively, I quickly learned the immense significance of maintaining a good developer guide. I realised that projects with a comprehensive developer guide will be notably easier to navigate and understand. They offered clear coding standards, best practices, and architectural insights, ensuring a consistent and unified code base. Moreover, these guides proved invaluable during team transitions, making handovers and on-boarding new members significantly smoother. As a result, I recognised that a well-maintained developer guide is not just a supplementary resource but is critical for the team to efficiently and productively collaborate in software development. It reinforced the importance of investing time and effort in documentation, as it directly impacts the efficiency and effectiveness of software engineering teams in real-world scenarios.
What I Learned
My internship at Analytics.gov was an enriching experience that greatly strengthened my technical skills through tackling challenging projects. What I am really grateful for is the amount of autonomy and empowerment that I was given as an intern at DSAID to undertake tasks. The exceptional level of trust and responsibility created an exhilarating environment for me to tangibly impact and influence the work I was involved in.
I also gained valuable perspective on the realities of software engineering in the real world. One crucial lesson I learned was the importance of building extensible and maintainable implementations for real-world software development. Unlike in an academic context where everything is based on syllabus and structured lesson plans, real-world software development often involves evolving requirements and changing priorities. By focusing on extensibility and maintainability, we ensure that our implementations can adapt to future changes and additions without requiring significant rework or compromising the stability of the system. This approach allows for efficient and cost-effective development, reduces technical debt, and enables the team to respond effectively to shifting project requirements.
Through this internship, I gained a deeper understanding of how software engineering concepts that I learned in school such as modular design, code reuse, and best practices could be translated into practical implementation to create robust and flexible solutions that can withstand the dynamic nature of real-world software development.
Moreover, I was fortunate to have the support of experienced colleagues from Analytics.gov and the Data Engineering team who were always around to guide and assist me whenever I encountered technical difficulties. I was initially surprised at how everyone was very willing to jump on to provide their insights or hop onto a Slack huddle to help me understand why certain parts of my given task implementation did not work. Their mentorship and willingness to share their expertise played a vital role in my growth and development during the internship as it allowed me to easily clarify my doubts, quickly solve impediments, and learn as much as possible during my time here.
Experience outside of work
Continuous learning and development
At GovTech, the commitment to continuous learning and development extends beyond the workplace. Throughout my internship, I also had the chance to attend esteemed tech meetups and conferences like the AWS Summit ASEAN 2023 and the Google Cloud Summit 2023.
These events proved to be invaluable opportunities for expanding my knowledge, staying up to date with the latest industry trends, and connecting with like-minded professionals. From engaging sessions and workshops to insightful keynote speeches, these gatherings provided a platform to explore cutting-edge technologies, broaden my network, and gain fresh perspectives outside the confines of daily work.
Another highlight during my internship was that I was introduced to the AWS certifications by my colleagues and managed to obtain the AWS Certified Solutions Architect - Associate certificate despite not having prior experience in cloud technology. Not knowing about these certifications before, I was also surprised by how many of my colleagues consistently stay updated in the ever-evolving technology landscape and always actively seek ways to enhance their technical skills. While studying for the certificate, I also realised how I was able to quickly understand the concepts tested as I could closely relate the content to real-world applications of these cloud technologies that were used during my internship projects. This hands-on experience and the support provided by the team helped me to bridge the gap between theory and practice, making the learning process more impactful and effective.
Fun and work-life harmony at DSAID
My internship experience has been incredibly enjoyable, thanks to the various fun activities that were organised.
Fridays were particularly exciting, as we often had game sessions that provided an opportunity for everyone to unwind and connect on a more personal level. These games allowed us to relax, have a good laugh, and deepen our relationships within the team.
We often had personal sharing sessions which offered us a chance to learn about everyone’s diverse backgrounds and past experiences, fostering a sense of understanding and camaraderie. The casual lunch talks were another highlight, where we all gathered to unwind and engage in light-hearted conversations.
What made these experiences even better was how approachable everyone was. It created a welcoming and inclusive environment where I felt comfortable reaching out and connecting with others.
Conclusion
My internship at DSAID proved to be a truly enriching experience, where I had the opportunity to delve into a wide array of projects and expand my skill set.
On the technical front, I acquired valuable expertise in Next.js, AWS, and CI/CD, which broadened my exposure to web development and deployment. These newfound skills not only strengthened my technical prowess but also opened doors to new possibilities in my career. Additionally, I gained invaluable soft skills, such as effective communication and collaboration, from the opportunity to work closely with my colleagues at DSAID and learned from their experiences. Not to mention all those fun and interactive team-bonding moments, which made my internship immensely memorable.
If you are seeking a fulfilling and comprehensive technical internship, I wholeheartedly recommend exploring the exciting opportunities within DSAID at GovTech!
