Mobile Preview - Part2 - Managing a pool of pods dynamically inside kubernetes
In my previous article, I discussed different kinds of architectures for building and rendering a preview of a web app in the browser or remotely in a vm or a container. In this article, I will focus on a specific part of our solution that was managing a pool of pods dynamically on Kubernetes.
The purpose of this architecture is to assign a dedicated container to each client, allowing them to utilize their own remotely hosted Expo server for building and rendering their application in real-time in their webbrowsers, similar to CodeSandbox.
Before entering into details, some knowledge about deploying application on kubernetes is required, this is a quick reminder:
Deploy a Node.js application on Kubernetes
Deploying a Node.js application on Kubernetes involves a series of steps including building a Docker image, pushing the image to a Docker registry, and deploying the application using a Kubernetes Deployment. Here’s a high-level overview of the process:
- Prepare your Node.js application: Ensure your application is ready for deployment. This involves making sure it works as expected, there’s no sensitive data hardcoded, etc.
- Create a Dockerfile: A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image.
Dockerfile
FROM node:14
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
CMD [ "node", "server.js" ]- Build Docker Image: Use Docker build command to build the Docker image from the Dockerfile.
> docker build -t my-node-app:1.0 .- Push Docker Image to Registry: Push the built image to a Docker registry, such as DockerHub or Google Container Registry.
> docker push my-node-app:1.0- Create a Kubernetes Deployment YAML file: This file will specify the Docker image to pull and the desired state of your deployment. You can also specify the number of replicas (pods) you want to maintain.
# Deployment configuration
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-node-app
spec:
replicas: 3
selector:
matchLabels:
app: my-node-app
template:
metadata:
labels:
app: my-node-app
spec:
containers:
- name: my-node-app
image: my-node-app:1.0
ports:
- containerPort: 8080- Apply the Kubernetes Deployment: Use the
kubectl applycommand to create the deployment on your Kubernetes cluster.
> kubectl apply -f deployment.yaml- Create a Service to Expose Your App: To make your Node.js application accessible from outside your Kubernetes cluster, you need to expose it using a Service. It’s an abstraction layer that allows access to a set of Pods in a Deployment. It provides a stable IP address and DNS name for Pods and enables load balancing and service discovery.
# Service configuration file
apiVersion: v1
kind: Service
metadata:
name: my-node-app-service
spec:
selector:
app: my-node-app
ports:
- protocol: TCP
port: 80
targetPort: 8080
type: LoadBalancer- Apply it with:
>kubectl apply -f service.yamlThis is a basic example of how you might go about deploying a Node.js application on Kubernetes. Depending on the needs of your specific application, you may need additional configurations like ConfigMaps, Secrets, Persistent Volumes, Ingress etc.
However, if we want to create pods dynamically at runtime, we need to take a different approach. To achieve this, we can use the Kubernetes JavaScript API.
Kubernetes Javascript API
The Kubernetes JavaScript API is a client library that allows you to interact with the Kubernetes API from JavaScript. It is implemented in TypeScript, but can be called from either JavaScript or TypeScript.
To use it, you will need to add the following dependency to your Node.js application
> npm install @kubernetes/client-nodeand to use this code to initialise the library
import * as k8s from '@kubernetes/client-node';
const kc = new k8s.KubeConfig();
kc.loadFromDefault();With this client API you can programmatically do anything that you would do with kubectl when applying your Yaml configuration files.
In our case we will use it to Create, Update, Delete deployments and services.
This is an example for creating a deployment programmatically
const k8sAppsApi = kc.makeApiClient(k8s.AppsV1Api);
let deployObj = {
apiVersion: 'apps/v1',
kind: 'Deployment',
metadata: {
name: 'preview',
},
spec: {
replicas: 1,
selector: {
matchLabels: {
app: 'preview',
},
},
template: {
metadata: {
labels: {
app: 'preview',
},
},
spec: {
containers: [{
name: 'preview',
image: 'preview:1.0.0',
ports: [{
containerPort: 19006,
}],
}],
},
},
},
};
k8sAppsApi.createNamespacedDeployment('default', deployObj).then((res) => {
console.log(res.body);
}).catch((err) => {
console.error(err);
});Now that we know how to set up resources programmatically, let’s take a look at the architecture we put in place.
Goal & Design
Our project aims to provide each client with a dedicated Expo server. To ensure speed and readiness, we keep a pool of active pods. This system helps manage unexpected demand surges and promises a seamless user experience.
Here’s a high-level representation of our architecture:
The micro-service responsible to maintain the pool of pod is the preview-management.
As seen before to deploy a micro-service we need two resources:
- Deployment: responsible for managing a group of identical pods, ensuring that the system runs the desired number of pods for your application (in our case only one Pod will suffice) and recreate automatically a new one if the pod crashes or deleted, maintaining the desired number of pods
- Service: provide a stable endpoint, load balancing, routing, service discovery
the preview-management will create a predefined number of deployment and services resources in kube and affect them an unique uuid identifier.
To enable access from outside to the pod, we use another resource from Istio called VirtualService.
VirtualService can route network traffic based on URI paths. In our case, we configured it to match incoming request with the following url **https://mydomain.com/uuid** to be routed to a service name with the same uuid identifier for example preview-service-uuid.
Note: Istio does not support dynamic labels or regex-based hostnames in the destination field of the route in a VirtualService. The host field in the destination expects a static string representing the destination host, not a regex or dynamic string. So we had to create separate VirtualService for each service instead of simply create one configuration file with this regex pattern like this:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: uuid-routing
spec:
hosts:
- "mydomain.com"
http:
- match:
- uri:
regex: "/(.*)"
rewrite:
uri: "/"
route:
- destination:
host: "service-\\1"In the Github repo, you will find the PreviewService class, which facilitates the creation, deletion, modification of Deployment, Service, and VirtualService. It affects them an unique identifier, keeping a kind of atomicity between Deployment, Service and VirtualService that should be created or deleted all at once.
export class PreviewService {
...
async create(): Promise<Preview> {
const id = uuid();
await this.createDeployment(id);
await this.createService(id);
await this.createVirtualService(id);
return new Preview(id, this.getName(id), PreviewState.FREE);
}
...
}You will find another service called PreviewPoolService responsible to initialize and maintain a pool of pods. It uses the PreviewService to manage kubernetes resources.
export class PreviewPoolService {
private previewsPool: Preview[];
private previewAllocationMap: Map<string, Preview>;
private previewAdded$: Subject<Preview> = new Subject();
constructor(private previewService: PreviewService) {
this.previewsPool = [];
this.previewAllocationMap = new Map();
this.initializePool();
this.startWatching();
}
private async initializePool() {
await this.previewService.clean();
const previews = await this.previewService.getAll();
previews.forEach((preview) => {
if (preview.allocatedTo) {
this.previewAllocationMap.set(preview.allocatedTo, preview);
} else {
if (preview.state === PreviewState.FREE) {
this.previewsPool.push(preview);
}
}
});
if (previews.length >= MAX) {
return;
}
if (this.previewsPool.length < POOL_SIZE) {
const additionalPreviewNeeded = POOL_SIZE - this.previewsPool.length;
for (let i = 0; i < additionalPreviewNeeded; i++) {
const preview = await this.previewService.create();
if (preview) {
this.previewsPool.push(preview);
}
}
}
}
....
}This service is tasked with maintaining a predetermined number of pods within a pool. It ensures efficient resource usage by dynamically creating new pods when none are available, and conversely, releasing excess pods when the pool exceeds its specified capacity.
(Note: in the example the pool have been greatly simplified)
