From Monolith to Microservice Architecture on Kubernetes, part 1 — The Api Gateway
In this blog series we’ll discuss our journey at Cupenya of migrating our monolithic application to a microservice architecture running on Kubernetes. There’s a lot of talk on microservices and also a lot of great online resources available, but I noticed that practical & pragmatic guidance is often lacking. Therefore, I’ll try to get really hands on.
If you’re unsure what a microservice is, I suggest reading Martin Fowler’s article about them. For the sake of this article my definition of a microservice is:
A software component that is independently deployable and scalable
I also highly recommend Chris Richardson’s excellent series on microservice architecture. If you’re not familiar with Kubernetes’ basic entities such as pods, deployments and services you can check out this nice hands on introduction by the guys from Codefresh.io.
This series won’t be a beginners guide on building microservices. I’m also not going to advocate (too much) on whether to microservice or not. This is just about sharing our experience with transitioning to microservices and the setup we ended up with at Cupenya. But don’t worry, there will be code.
- Part 1: The Api Gateway (this part)
- Part 2: Authentication with JWT
- Part 3: Deploying our Scala app as a microservice
- Part 4: Monitoring, health checks, logging & tracing
- Part 5: Deployment automation & continuous delivery
Before the migration
Let’s start with a bit of context. We do a lot of Scala development at Cupenya. Also many of the code samples in this series will be written in Scala. Scala provides you with the tools to design a very modular architecture. This is also what we tried to achieve. The application was split up in different components in a service layer, each with its own REST interface. Although the application was very modular during development, when you look at it from a deployment perspective it effectively boils down to a very traditional Java stack, like in the image below.
All the different REST endpoints, business and data layers were wired together as one REST interface and in one runnable jar. The only physically separate component was the frontend, which were just static assets served by nginx. We could do a lot better here. It would be great if we could separate the deployment lifecycle or certain components from our service layer. That way we could:
- Increase agility and have smaller & faster deployments
- Scale services individually
- Have more fine grained control over service SLA’s (i.e. some services are crucial and need to have failover in place and some are not)
- Have autonomous teams responsible for a subset of services
New Architecture Overview
The diagram below provides a simplified (e.g. in reality each microservice has its own database) overview of the architecture we’ve ended up with after our little adventure.
- The dashed box: A logical Kubernetes cluster, running on multiple nodes
- Green boxes: Kubernetes services. They expose a set of one or more pods, typically through HTTP, for use by other pods
- Blue boxes: Kubernetes pods. Typically for us each pod runs one docker container, but if we need co-located applications they could run multiple containers
- Blue cylinders: Off-cluster database, in our case MongoDB. Each microservice has its own database provided by a single MongoDB cluster.
- Blue cloud: A cloud DNS provider, like Google Cloud DNS or Cloudflare
In this first part of the series we’ll focus mostly on one of the emphasised components in the picture: the Api Gateway. This is also the first component we started to build and deploy on Kubernetes. Before converting our application to a microservice and have it running in Kubernetes both the Api Gateway and the Authentication Service need to be in place for a working setup which we could go live with.
The component I’d like to dig deep into in this post is the Api Gateway. The Api Gateway is the entry point of our microservice infrastructure (see api gateway pattern). The main problem it solves is:
How do the clients of a Microservices-based application access the individual services?
Kubernetes has the concept of services which can expose a set of one or more pods to be available for other pods or external applications. So why do we even need this Api Gateway? For us there’s several reasons:
- We wanted to have a single entry endpoint for all API clients where we could handle crosscutting concerns such as authentication and security
- The number of individual Kubernetes services and their locations (host+port) could change dynamically
- Partitioning into services can change over time and should be hidden from clients
- There’s a difference in API calls we wanted to expose outside the cluster and internal API calls
Service registry & service discovery
The Api Gateway keeps track of all available services and the resources they expose. When an API call comes in it basically needs to figure out which service to proxy it to. To do that it needs to maintain some kind of internal mapping, a service registry.
Let’s get our hands dirty and write some code.
As you can see it’s just a simple map between a
String, the resource (e.g. ‘users’) and a
GatewayTargetClient basically holds a connection pool to the target host and a simple proxy route. We are using Akka HTTP so it looks pretty much like the following code.
In reality the main ‘route’ logic on line 6–9 comprises also the basic authentication flow. I’ve left that out for brevity for now, but we’ll get to that in the next post in this series. Now that we have mapping between a resource and a
GatewayTargetClient we can setup the dynamic routes. We use Akka HTTP’s directives API to find the right mapping for a given path.
As you can see the logic basically consists of two tail recursive functions which traverse through each segment of a given path trying to find an available mapping. If it can’t find an available mapping the
Unmatched rejection is returned. Those familiar with Akka HTTP will know this will be picked up by the default
RejectionHandler to return a 404.
serviceRouteForResource directive is then used to construct a dynamic Akka HTTP route as follows:
Every time a request comes in, the current configuration is fetched and the dynamic routing logic is performed.
Alright, so far we’ve seen that the Api Gateway keeps a registry of services mapped to a resource and is able to route requests for that particular resource to the appropriate service. For example if we would have the following mapping
"users" -> GatewayTargetClient("my.user.svc", 9090)
in essence the call
GET "/api/users/123" would be proxied by the Api Gateway to
So it’s pretty neat that we can register services and API calls will be dynamically routed to the appropriate service, but it would be very tedious and cumbersome if we would have to manually manage the registry. This would also defeat the purpose of supporting dynamic changes of services and their locations. After all one of the goals of the API gateway was to hide these internals for clients. Wouldn’t it be great if we could detect such changes in services and automatically synchronise the registry?
This is where service discovery comes in. In the past I’ve looked at tools like Consul. There are even dockerised bridges which could listen for Docker container update events and keep Consul in sync. There’s a nice post about getting this set up. However, Kubernetes has already a built-in service registry exposed through an API so we figured it would be much easier to utilize this. In fact Kubernetes’ internal DNS server, kube-dns, is also relying on this API for synchronising its DNS records.
To access the Kubernetes API we need an access token. The easiest way to do this would be through service accounts. The steps below will create a service account and associated secret. A secret is a nice mechanism to hold sensitive information, like passwords and access tokens without having to directly put it in a docker image or pod specification. Cool, let’s create a service account.
$ kubectl create serviceaccount my-service-account
serviceaccount "my-service-account" created
Great, now a new service account has been created and under the hood also an associated secret which we can retrieve as follows.
$ kubectl get serviceaccounts my-service-account -o yaml
- name: my-service-account-token-1yvwg
As you can see in our example the generated secret is called
my-service-account-token-1yvwg. We can reference this secret in the deployment descriptor of the API gateway to populate an environment variable. We can also manage the Kubernetes API host and port information through environment variables in the descriptor file.
In the above descriptor file you can see that we assign the value of our secret to the environment variable
K8S_API_TOKEN and also set the
K8S_API_PORT environment variables. Since we use Typesafe Config as a configuration library we can simply use these environment variables as overrides for our config.
Enough messing around with configuration files! Now that we have access to the API we can start coding again. The services API has a pretty straight forward REST endpoint to retrieve a list of services and their metadata. There’s even a “watch” option which delivers service updates as a stream. All we need to do is write a small REST client which retrieves the service updates and (un)register the service at the Api Gateway.
We started of with streaming updates from the “watch” endpoint in the Api Gateway , but the tricky part was that you could miss updates on downtime. If, for whatever reason, the Api Gateway would be restarted the service registry could go out of sync. Hence, we decided to go with a rather naive polling solution. First, we created a simple agent Actor that would schedule the calls to the Kubernetes service API.
The signature of the
listServices function is
def listServices: Future[List[ServiceUpdate]]
so it asynchronously returns a list of service updates. The
ServiceUpdate object, which is passed to the callback function, contains all the necessary information to update the Api Gateway’s service registry.
This information we parse from the Kubernetes service metadata. Hence, for each new service we define in Kubernetes we need to populate this metadata in the service descriptor to be picked up by the service discovery logic. Let’s look at an example of such a service descriptor.
This service will expose all pods with the name “user-api” under the endpoint
http://user-api-service.my-namespace/ and bind to the target port 9090. In the
metadata section there’s a
labels subsection which defines the properties
secured. These are custom properties which are used by our service discovery algorithm.
resource property is specified on a service we will register the service in the Api Gateway under the specified value, in this case
users (as explained before the Api Gateway would then proxy calls to
http://user-api-service.my-namespace/users). If the
resource property is omitted, the service will be ignored by service discovery and it will just remain internal only. The
secured flag is to indicate whether a service requires authentication or not. I’ll get to this in my next post.
So this service metadata is returned from the Kubernetes services API, can be parsed into our
ServiceUpdate model and is in its turn passed to the callback function. The corresponding service discovery client implementation therefore looks like the following.
It’s quite a lot of parsing boilerplate code, but essentially it comes down to this: In line 13 we parse the API response into our modeled list of
ServiceObject’s and in line 15–28 we populate our
ServiceUpdate model. The resulting
Future[List[ServiceUpdate]] is then passed along to a callback function by the service discovery agent as we’ve seen above.
The full service discovery implementation is available as an open source library. The callback function used by the Api Gateway basically looks like the code below.
Let’s briefly walk through it. First we filter on namespace, because we only want to keep a registry of the services in the namespaces that we’re watching (as I’ll explain in my following posts in this blog series we are heavily relying on namespaces to for instance setup isolated environments for feature branches). Then we check if all currently registered resources are still available. The ones which are not will be deleted from the registry. Lastly, we’re adding the new services.
Thus far we’ve seen the dynamic routing, the service registry and the service discovery logic. This is basically the core of the Api Gateway. It can now detect new resources being served and proxy incoming API calls to the appropriate service. The full implementation of the Api Gateway is available as an open source project (there’s also other implementations available, such as Kong, Tyk and Zuul). It’s also available on Docker Hub.
The last step is to expose the Api Gateway to the outside world. Remember, this is our single entry point for public API traffic. We started out with using the Kubernetes Ingress resource for that. This resource could be used to give services externally reachable URLs, load balance traffic, etc. It’s basically a collection of rules that allow inbound connections to reach the services inside the cluster.
However, it had a few extra requirements. For instance, it required all backend services to be of
type: NodePort which basically means that each node will get a port allocated to proxy that port into your service. We didn’t really need this. Another requirement was that each backend service needed to implement health checks (i.e. needs to return 200 OK on GET) on
/ endpoint. The health endpoint requirement was not bad in itself. As a matter of fact I’m a huge fan of having health check endpoints as I love to tell you more about in the next post of this blog series. I do have a problem with the requirement to serve it on
/. As far as I know this wasn’t configurable.
It all felt a bit over complicated at the time. In the end we just needed a simple reverse proxy to route traffic to
/api to the Api Gateway and all other traffic to the static resources container. So we decided to just run a simple nginx container deployment with a reverse proxy config.
As you can see we can just refer to the host names based on DNS provided by kube-dns . Since we stay within the same namespace we can also omit the namespace suffix.
Luckily there was no need to create a custom nginx container image based on this config file. Kubernetes has this great concept called ConfigMaps which allow you to decouple configuration artifacts from your container images to increase container portability. They basically create key-value pairs of directories, files or literal values. In your deployment descriptor you can then use these ConfigMaps to populate environment variables or volumes to keep configuration dynamic. This is exactly what we used for our custom ingress container. Below are the steps we followed.
First, we rename our nginx config file to
default.conf (this is the name of nginx’ default config file we need to override) and created a configmap out of it.
$ mv ingress-nginx.conf default.conf
$ kubectl create configmap ingress-conf --from-file default.conf
Second, we create a simple deployment descriptor where we define a volume based on the configmap and mount that volume on
/etc/nginx/conf.d so the container image default config gets overridden by our custom configuration
As with the Api Gateway deployment we needed a Kubernetes service to expose the container ports. This time we needed the service to be publicly available and with a static IP so we could setup DNS with our provider of choice (e.g. Cloudflare). For this we could use the service
type: LoadBalancer which would effectively generate a TCP load balancer. By specifying the
loadBalancerIP property in the service descriptor we could enforce the static IP.
Once we deploy our Api Gateway and Ingress we’ve got the main entry point of our microservice infrastructure ready for business. All we have to do now is tell Kubernetes to create our resources. To do that we can bundle all deployment and service descriptors and run:
$ kubectl --namespace my-namespace apply -f my-descriptor-file.yaml
You can also keep the files separate and run the command above separately for each file. In my next posts we’re going to integrate the Api Gateway with our Authentication Service and we’re going to convert a Scala application to a running Kubernetes service.
I hope you enjoyed the first post of this blog series and that you feel inspired to give Kubernetes a go. It might be a bit daunting at first, but once you get the hang of it you’ll soon benefit from what a powerful platform it is.
Thanks for reading my story. In the next post I’ll show how we built our authentication flow with a little help of JWT. If you like this post, have any questions or think my code sucks, please leave a comment!