Using Kotlin in a Serverless Architecture with AWS Lambda — Part 4: A Multi-layer Core for Your Function
Implementing a Client-Dispatcher-Server pattern and a micro HTTP router in Kotlin
In previous parts, we have covered the basics to set up our serverless project, use a development environment using emulation software, and a general multi-tier architecture for our project. In this part, I’ll present and implement design patterns to develop a serverless architecture following a Function as a Microservice (FaaM) style.
Index
- Setting up the project
- Setting up your local development environment
- Designing and implementing a Multi-Tier Lambda App
- A Multi-layer core for your function: A request-dispatcher and micro HTTP router implementation
- Testing your code
- Integrating a RDBMS to your project (coming soon)
- Authentication & Authorisation using Amazon Cognito (coming soon)
- Creating an end-to-end RESTFul service (coming soon)
Service Granularity
In the mid-90s, the web was starting a historical revolution and worldwide systems needed a way to integrate with each other. During the 70s, we saw some theoretical solutions to integrate remote systems and in the 80s some of them were implemented. However, popular solutions such as Remote Procedure Calls (RPC), CORBA and Distributed Computing Environment (DCE) were not ready for a global scale change and, most of the time, their implementations were not agnostic enough to work in a hybrid environment. Then the world met a new way to integrate their systems: Service Oriented Architecture (SOA).
From its humble beginnings in 1996 to a complete revolution using SOAP in the 2000s, SOA became the software design and implementation that changed the tech landscape forever. Evolving into a new mindset during the 2010s with REST, Event Driven Architectures (EDA) and Microservices, the question that any developer, architect and manager asks is: How big should a service be?
The classic SOA presented two main concepts: Coarse-grained (CA) and Fine-grained (FA). In the past, a CA was defined as a service with a broad business scope, while FA was considered a service that focused on a narrow scope. However, while a huge CA may produce a monster with a lot of business capabilities, a FA service risks to create hundreds or thousands of mini problems. In the end, both extremes can reduce the modifiability, scalability and security of the solution.
For that reason, I like the granularity principle that the Service-Oriented Framework (SOMF) provides. In this framework, a service can be divided into three different kinds: Cluster, Composite and Atomic, which can be defined as:
Atomic: A fine-grained service that is impractical to decompose because of its suggested limited capabilities or processes
Composite: A coarse-grained service comprised of internal fine-grained atomic or composite services, forming hierarchical parent-child associations
Cluster: An association of services grouped by related business or technical processes that collaborate to offer solutions
— SOMF 2.1 Specifications © 2008–2011
But, how can we apply those concepts to our serverless project? Let’s first understand how Serverless Framework + AWS Lambda integrate some similar concepts.
Serverless Granularity
When we’re working with Serverless Framework + AWS Lambda we need to take into account three main concepts for any project:
- Event: Anything that can trigger a function (Http calls, S3 upload, Cloudwatch alarm, etc).
- Function: A piece of code that handles an event and executes program instructions that perform a specific task.
- Service: Where you define your AWS Lambda Functions, the events that trigger them and any AWS infrastructure resources they required.
According to Serverless Framework, we should create an AWS lambda function per single CRUD operation. Meaning that an entity User will become a service with 4 AWS lambda functions. However, if we follow this practice, just having 10 entities will result in 40 functions! So now you need to manage 40 lambda functions, one for every single event. Also, let’s remember that our entry point is going to be the AWS API Gateway and when using services, it creates a different API Gateway for each service. In theory this makes sense, but the idea of having a Gateway is to make the backend services transparent from the final client and there’s no way of creating an orchestration layer between multiple gateways in AWS (like a gateway that redirects to other gateways). According to Serverless Framework, that’s a limitation that they’re working on:
Note: Currently, every service will create a separate REST API on AWS API Gateway. Due to a limitation with AWS API Gateway, you can only have a custom domain per one REST API. If you plan on making a large REST API, please make note of this limitation. Also, a fix is in the works and is a top priority.
— Serverless Framework Guide
Function as a Microservice (FaaM)
For that reason, I like to match the framework concepts with the ones defined by SOMF. With that in mind, I propose a concept called Function as a Microservice (FaaM) that I defined as:
A design pattern where each lambda function behaves as a small cluster with meaningful business composite capabilities, that receives, handles, routes and process multiple atomic events.
Using this pattern instead of having a function to “add a new city”, we have a geolocation function that knows how to CRUD countries, cities and regions. These small microservice functions shouldn’t manage more than 7± 2 entities (following Miller’s Law), otherwise you’re going to have a huge monolith in a function. Note: Keep in mind that this pattern assumes that every single function is responsible for its own data source too.
But how can we implement a FaaM using AWS Lambda? Well, the answer is using good routing patterns!
Request-Dispatcher
In AWS every single Function should have a unique handler to resolve the incoming events. Using the Serverless Framework we can define the handler file that each function is going to use. By default, Serverless creates a Handler.kt file like this:
Instead of copying and pasting the same logic, we can always reuse the generic and abstract code (essentially it’s always the same thing). We just need to know how to route to different logic depending on the event type. For those cases, we can use a Client-Dispatcher-Server Pattern.
In our specific case, this routing will depend on the URI that a client uses to make a request. Meaning that if we make an HTTP request to /movie, we should be capable to route that request to a specific Kotlin function based on its request URI. The next example presents a routing structure using the same handler:
But using the same handler, how can we route properly? Well, I created a different yaml file that using regex knows how to route to a specific class and function depending on the URI. In the resources folder I created a routes.yml file like this:
If we take a look to the User Microservice, the first element of the list means that when there’s a URI that satisfies the regex ^/login(/)?$ it will execute the function login in the class UserController of the package com.myblockbuster.users.controllers. Note: If you have worked with frameworks like Django or Ruby on Rails you should be familiar with this routing.
Nevertheless, this is just the static way that we want to route. Now we need to implement the piece of code for that.
Gradle file
Include the next dependencies to your project:
// Updates the Kotlin Gradle Plugin
dependencies {
classpath 'org.jetbrains.kotlin:kotlin-gradle-plugin:1.2.21'
...
}// Updates the Kotlin Gradle Stdlib
compile group: 'org.jetbrains.kotlin', name: 'kotlin-stdlib', version: '1.2.21'// Add new dependencies
compile group: 'com.fasterxml.jackson.dataformat', name: 'jackson-dataformat-yaml', version: '2.1.2'
compile group: 'org.jetbrains.kotlin', name: 'kotlin-reflect', version: '1.1.1'
Data models
I created the next models.kt file to support all the entities that my new architecture requires.
Exceptions
As usual, it’s always a good practice to manage your own exceptions. In my case, I’m also defining the HTTP code that a specific exception should raise. The exceptions.kt file looks like:
Dispatcher
The next interface presents the basic capabilities that any Dispatcher should provide. Even when we are just going to have one dispatcher in this project, it is always a good practice to isolate implementations from the functional contracts.
Based on the previous interface we can have the next RequestDispatcher implementation.
This dispatcher implementation executes a Kotlin function and returns the result object to our client. To achieve that, I’m using the default Kotlin reflection methods, so you need to be cautious when using it. After executing some tests, this usage of reflection is not affecting performance too much because I’m using the complete Classpath. But if you try more dynamic code, performance can change from milliseconds to seconds.
Response
By default, Serverless creates a Response class that I changed into an interface. Just like with the Dispatcher, the main reason for that it’s just to keep implementation and functional contracts isolated from each other. Note: After this, you can also delete the HelloResponse class.
package com.myblockbuster
interface ResponseApiGatewayResponse
Similar to the class Response, Serverless also creates an ApiGatewayResponse class. To accept the new changes for the Client-Dispatcher-Server, let’s use the next piece of code:
New Handler
Now we can create the reusable handler that multiple functions will use for their routing. The Handler.kt file looks like:
Controllers
Finally, let’s create a layer with our business logic. Following the same principle of an MVC app, we can create some controllers with the necessary logic.
The functions of this Interface/Mixin are some basic capabilities that you may need to process a request. However, most of the time the processing of our request will be quite repetitive. For that reason, I decided to create a Micro-HTTP router that based on the HTTP method it routes to a specific Kotlin function.
Micro HTTP-Router
To create the Micro-Router we can create a generic Service interface that defines a basic CRUD for an object. But first, let’s define some additional models that we need.
Models
Service
This interface defines the basic CRUD capabilities that any entity should provide. This contract is based on frameworks like Loopback, Django RF, RoR and Spring. The Service.kt looks something like:
In this Service I’m assuming that we’ll receive a User to validate permissions to execute that function. We can use an AnonymousUser class to define a user who hasn’t logged into the system. The next models.kt file exposes an example of that.
User Models
HTTP Router
The easiest and most reusable way that I found to develop the router, was using the Controller interface/mixin that we already defined. You can always create another interface called HttpRouter and implement it in the Controller interface if needed.
The previous code fragments show how to route to a specific method according to the HTTP method received as a parameter. With this piece of code, you just need to implement each CRUD method or throw a MyException object with the code 405 (method not allowed).
Implementation Example
The controller can be defined according to your needs, that’s why the mixin doesn’t have any contract to follow (the only constraint is that each function needs to receive a Request object as argument). The next code fragments are examples that you can reuse for your own project.
Movies Exceptions
Movies Models
Movie Service Implementation (mock without Security and Persistence layer)
Movie Controller Implementation
Now just gradle deploy and test it! You should be able to “CRUD” a movie!
BONUS: Using the Factory pattern to behave like Spring
If you don’t like highly coupled relationships, using a factory design pattern to interact with your services is a good choice. As I already mentioned, the use of reflection, Aspect Oriented Programming (AOP) or any code injection can affect the performance of your Lambda function. So using the basic and well-known patterns is always the right way to go.
Service Factory
We can define an interface and it’s implementation for the Service Factory. In this specific case, the service is being returned based on the model we want to CRUD.
Movie Controller Implementation with Service Factory
SOURCE CODE
If you want to see the complete example, the source code is available at this Github repo.
Final Thoughts
This article presented an implementation of a Client-Dispatcher-Server pattern to use in your Kotlin + Lambda Functions as Microservices and an HTTP Micro-router for your CRUD requests. This is a quick example of a library/framework that I’m developing to rock future projects. A framework that pretends to remove all the boilerplate code for typical Kotlin Serverless APIs projects.
The next articles will explain in detail how to create some good unit tests for Kotlin in AWS Lambda, add a relational persistence layer and finally how to integrate security requirements that we may need.
See you soon and don’t forget to clap & subscribe.
