Atom’s Journey to Serverless
What is Serverless Computing?
Serverless Computing is a means to code and build our applications without the need to manage the servers, the resources needed to run our applications.
So how does it work for Frontend?
Modern frontend applications are built as single-page applications using a framework like Angular or libraries such as React/Vue and bundling tools like webpack creates our app with root index.html and the dependent JS/CSS assets. This enables us to rely on the backend as a service for authentication and business logic, and we can focus solely on how to display the data to our users with minimum latency and optimized rendering.
With the help of cloud providers such as AWS, we can build our frontend applications with any framework/library of our choice, and rely on them for managing the infrastructure needed for hosting and even seamless fast content delivery of our assets.
AWS achieves this through S3 for website hosting and Cloudfront for fast content delivery
So let’s get started with how we achieved this, but before that a brief introduction to our current architecture.
We have three major personas — Educator, Author and Students, each of them along with their dependents a microsite created in Angular/React and served through an express server for the assets along with health/heartbeat endpoints. All of them are deployed as containers on AWS ECS (Elastic Container Service). We have a micro-frontend architecture with multi-tenancy in which we have our Nginx reverse proxy that handles requests based on the URI and forwards to the appropriate upstreams microsite containers.
Using ECS, we have created a task definition for our services outlining the image, container port mapping, resource allocation parameters to run our container. Next we have a service for each of our sites to deploy our task definition with desired, maximum and minimum container count. ECS provides a managed service scheduling, enabling spawning/destroying containers based on parameters like CPU/RAM usage.
A major advantage of using ECS is it enables us to to deploy several containers of the same task definition to the same instance where it handles host-container port mapping dynamically through a Load Balancer. We assign a load balancer with a target group to our service that handles routing our requests to the correct instance and host port. But it has a caveat, a need to create a new load balancer for each service, which could be a handful if you have several services to manage.
To avoid the above scenario, we use Consul which provides us with service discovery through a central registry. Each container deployed on the ECS cluster, is associated with a service tag (namely its domain) and is registered with Consul central registry with information about its host and port mapping . When applications need to communicate with each other, it queries the registry for the service it needs to connect with, and Consul returns the host instance and port mapped to the service container to connect.
The challenges with this approach
Reliability
Firstly the containers, as each service is hosted on it, there are chances of it crashing due to a deployment bug or the express server crashing due to a fatal exception making it unreachable. Continuous monitoring by the dev team of a server which serves only assets is an overkill.
Secondly is our reverse proxy Nginx server, we have had instances where our proxy itself has gone down due to a deployment failure of the reverse proxy container, leading to a complete downtime of our entire stack.
Cost
As all our frontend applications are pure assets build during the compile time, a need to have a server running only to serve this assets also adds to the cost to the organization
Scalability
Each time we deploy a new service, we need to create a new ECR image, ECS service. We also need to re-deploy the reverse proxy to manage the route proxying which brings it won deployment overhead. Also a need to create a server for proxy handling, is something we can unload it to our Cloud provider.
The Solution — S3 + Cloudfront.
We already have an architecture running on the above mentioned, moving it all would not be a feasible option considering the resources needed. With the advent of 2022, our team got the responsibility of developing a new portal for the Educator persona, providing the perfect opportunity to implement the new architecture. But we had 2 major challenges , first not interfering with the current architecture and seamless integration with it, secondly the provision of moving the current microsites in a phased manner without the need of much effort in the future.
We have 3 environments: PROD and 2 lower environments, Integration for developers and QA.
Step 1: Create S3 Bucket
We had an already provisioned S3 bucket for web hosting, one for INT/QA and another for PROD. AWS docs provide a detailed section on Web hosting
Rather than creating S3 buckets for each microsite, we decided to serve their assets in their respective folders. So we created a bucket called serverless-apps and inside that a folder for the educator persona. For our lower environments, to handle serving respective environments, a sub folder named int/qa was created inside each persona respectively. A Jenkins job was set up to build and deploy the assets to the respective folder.
Step 2: Create a Cloudfront distribution
Next step was to set up Cloudfront to support both the current architecture (reverse-proxy) and S3 web hosting.
Origins
When you create a Cloudfront distribution, we first have to specify the locations from which your assets will be requested from known as Origins. We can also set up multiple origins in order to serve assets from different locations based on routes. We created 2 origins, one pointing to our S3 buckets and the other to the reverse-proxy Load Balancer (Custom Origin).
Also as we didn’t want to configure 2 distributions for our 2 lower environments, we added 2 alternate domains(CNAMES), one for INT, other for QA. There are few requirements on setting multiple CNAMES to a distribution, more about it can be found on Cloudfront CNAMES
Behaviours
Next comes the behaviors, we wanted to serve assets for our new educator portal through S3 while having the remaining traffic pass through the Load Balancer.
Path based specific routing was set up for the educator portal. We created multiple behaviors, one to serve optimized cache(css/js) and the other for no-caching to serve index.html and certain js/build files.
Remaining traffic was redirected to our Load Balancer to handle all the other routes that are currently under the existing architecture.
With the initial setup done, we still had few more hurdles we needed to overcome
- Sending browser caching and security headers to the client
- Ability to send headers, cookies and query string parameters for our non cached assets and also to our Load Balancer as it is needed for Authentication
- As Dev, QA would be served through the same Cloudfront and in turn the same S3 bucket but different folders, we needed a way to distinguish between the domain
We went on to achieve this through Cloudfront policies and AWS Lambda Edge Functions
Cloudfront Policies
With Cloudfront, there are 3 type of policies that can be created
The cache key is the unique identifier for every object in the Cloudfront cache. When the user requests a resource, it stores the resource based on the key and the policy associated with the behavior route for subsequent requests. We can also decide if we would like Cloudfront to cache a particular resource or not by creating a dedicated no-caching policy for a particular route. The default cache key used by every distribution is the Cloudfront distribution domain name (example.cloudfront.net) and the URL path of the request.
As we are serving both for INT and QA, we also decided to Include the Host header in the cache policy for cached assets. This served 2 purposes, first our assets were cached separately for each environment and second getting the host header for part 2 of our setup which i’ll cover below. For our un-cached assets we used the AWS Managed-CachingDisabled policy.
It enables us to send extra headers, certain cookies and query params to our origin but will not be considered as part of our cache key leading to better caching hit ratio. We used this for our uncached assets and our Load Balancer to send the complete payload of our client to the origin.
The above policy is used to specify the HTTP headers that CloudFront adds to HTTP responses. In order to send valid caching headers and security headers to the browser, we created 2 response header policies.
First for our cached assets with a Caching-Control header with max-age of 3153600 seconds(1 year) and security headers of Strict-Transport-Policy, X-Content-Type and X-XSS-Protection
For our non-cached assets, a Caching-Control header of no-cache
AWS Lambda
Now for the last and the important part of our setup, whenever we set up a behaviour with a route, it forwards the route to the origin by appending to it, but as our assets path was different also containing the environment sub-path, we needed a way to map the host header to the correct folder in S3 and also to handle URL path stripping.
The second was handling 404 routes, as it is an Angular application the routes were handled in the client side, so if the user refreshes or enters a URL that needs to be handled by Angular, the server should send the index. html file for the application to load successfully and Angular take care of the routing
This was achieved by AWS Lambda Edge functions. In Cloudfront by using Lambda functions we can customize the request and response. It can be set between the user and Cloudfront and between Cloudfront and the origin.
It has 4 places it can be set up
- Viewer Request: Request sent from user => Edge function => Cloudfront
- Origin Request: Request sent from Cloudfront => Edge function => origin
- Origin response: Response sent from origin => Edge function => Cloudfront
- Viewer response: Response sent from Cloudfront => Edge function => user
For our use case we used the 2 option Origin Request. With Origin Request, we get access to the request object that will be sent to the origin and options to modify it. A sample of the request can be found here. We created a Lambda function that read the Host header of our request, and updated the URL with the environment route based on that. An important part to remember is that, if we don’t include the Host header in our cache policy or in our Origin request policy the edge function won’t have access to it as Cloudfront strips all headers not needed or updates it with its own.
To handle 404 routes, we used a simple regex to check if it’s part of our assets like js/css or our icons and if it didn’t match any of those, we updated the origin request URL with index.html.
In order to associate a lambda function, first a numbered version of the lambda needs to be created as Cloudfront triggers can only be set to versions as they are immutable
There are 2 ways to add a Cloudfront trigger to lambda
- Using the Lambda console or API
- Using the Cloudfront console or API
We used the latter as it gives better control on which part of the request we need to add the trigger
Last but not least error handling
Error handling is an integral part of the development process, and incomplete without it. There can be scenarios where origin servers can send 4xx or 5xx error codes, and in such cases we need to provide our users with appropriate messages. In Cloudfront we can set our own custom error handling for different error codes. Cloudfront caches the following error codes.
To set up, go to Error pages section for a distribution
On clicking Create Custom error responses, you will get a page to define your custom error page
Response page path is the behavior path pattern that you want Cloudfront to request for your custom error page.
We had a S3 bucket that contains all our custom error pages, and we added it as our origin to our distribution. Next step was to setup a path pattern to match the error response path
Next time, a 503 status is returned by any of the origin, the custom error page is returned to the user.
More about how Cloudfront handles errors can be read here.
Learnings
The whole process did take some time, edging out all the quirks for routing and caching, setting the AWS lambda Edge function and integrating it with the existing architecture, but it was the first step for us to achieve a complete serverless architecture for our frontend apps. Also relying on AWS Cloudfront for URL based routing would one day make our reverse proxy redundant, providing better reliability and performance. Cloudfront with its caching abilities would provides us lower latency and finally the optimization of our assets with caching and security headers providing better performance and higher security.
