Writing our own serverless router for AWS Lambda functions.

A Workaround for 200 resources limitation on CloudFormation.

Pipeline photo created by evening_tao — www.freepik.com

In my last work project, we had to deal with a common feature when we work with AWS Lambdas functions on Serverless framework…

200 Resources limitation feature

This is not an error, this is a common behavior from AWS CloudFormation, since this is a feature from CloudFormation to avoid have many resources in one stack, forcing to have more than one stack.

Fortunately, we have workarounds for this, like split in microservices on multiple stack in one domain via API Gateway or apply a proxy router to your handler functions like expressjs.

In this post, we ’ll learn to write your own router, through the basics features about a router, so, first let’s see…

About 200 resources limitation feature

AWS CloudFormation is the responsible for create any resource necessary to bind our function with API Gateway and others in the AWS cloud.

For each Lambda function on our serverless.yml CloudFormation needs to create at least six resources more than just one function:

• AWS:Lambda:Function
• AWS:Lambda:Version
• AWS:Logs:LogGroup
• AWS:Lambda:Permission
• AWS:ApiGateway:Resource
• AWS:ApiGateway:Method
• etc…

So, for each function we have this others resources that CloudFormation needs to create, imagine, if you have 35 functions in one stack, you really have 210 resources… Opps, you have the famous…

The CloudFormation template is invalid: Template format error: Number of resources, 210, is greater than maximum allowed, 200

Problem:

Error graphic

Solution:

Solution graphic

Roughly, this is a graphic description from a solution, but what happen’s with our router how it works, well…

For more detailed information about this error, please check this article.

A Router as solution

There are many router libraries to get a serverless router like serverless-http, such library, connect with your favorite API router framework, like expressjs, koa, sails, happi, etc…

However, you have too another option, and this option is…

Creating our own router

Well, first we have to understand, what is a router and how it works, let’s see the next image:

Router in action

In image from above, we see that our Router now, handle all request to API Gateway through ANY methods proxy, after, we see that, the event sent to main handler, which is our router, send the event to the corresponding function handler described in our router logic, which we ‘ll to call…

Logic description

Every router, must have a way to describe the logic to send the event through the functions, by example…

pe = “api/users” > me = “GET” > fc

Where:

• pe: Path endpoint
• me: Method endpoint
• fc: Function controller

If we translate this to Javascript code, will be where the developer can describe the router logic like this…

Logic description example

Familiar?, yes if you has used frameworks like Express or Koa for example, but What I need to give access to register a logic in the router?

Creating the verbose methods functions

That we need, is to create the main functions for verbose main methods:

• GET
• POST
• PUT
• DELETE

Methods functions

We have our first approach to the our Router prototype, where we have an object with methods or functions:

• _routes: Is the routes Array property to save all paths or routes from this router.
• _addRoutes: Is the function responsible to add a new route in our routes list.
• get: Is our get function to verbose method GET.
• post: Is our post function to verbose method POST.

First, our _routes list is an object array with next properties:

Route object prototype example

• path: Route path described for this handlers functions
• method: Verbose method to communicate with request
• middlewares: Array functions where store all functions in order to execute when request match with this route.

Next, we have to handle the route path match with request path from API Gateway.

Mapping and finding paths

For this section, we have a very helpful library (path-to-regexp), which handle all path matches for us, this is a time saver, trust in me.

However, we need to develop the match functions, well, let’s do this.

Match path example

In above code, we create a function to help us to match any path from request with any route described in our router.

First, we need the match method from path-to-regexp library, which compare between two routes or paths and determine, if one route match with path request, also, this function return the regex params (api/users/:userId) described on route selected by match and this way we have the selected route with params object filled with their respective values.

After, we need to execute this function inside an Array filter predicate, to get the routes selected by path-to-regexp and finally get an Array filled with routes, which we need the middlewares functions for each route selected.

Filter predicate generator example

Where _findAndComposeRoute is our function that return other function, yes… closures everywhere :), and the returned function will be our filter predicate function to apply in an Array filter method, so, inside this function there are some validations to know, if the current route is valid regarding with the current path from request.

In summary, we have two validations mainly:

• Method validation: This validation checks the match between method from request and method from current route on router.
• Match validation: Evaluate the the result from our _getPathParams function, with results from library path-to-regexp
• Selected: Finally this is the combination from above validations to get a Boolean result.

Handler function generator

In this part, we have to create a handler function finally to give to the API Gateway a function to connect with the unique endpoint described on our serverless.yml

Handler generation function example

Well, toLambda() is our method from our prototype, such we create the final handler function where, the all request will pass and inside this function there is our router logic before described.

However, let’s see some key points about this last function.

• AWS resource path detection
• Filter routes list
• Routes middleware functions list
• Execution pipeline

AWS Resource Path Detection

First we see in above code, that we split event.resource property from our event request object, with next criteria /{proxy*} , and we get the first element from split result, this is because, in our serverless.yml we have to describe something like:

serverless.yml example

With this we now know where start our router and where ends AWS path resource.

Filter Routes List

Also, we need to apply the predicate filter to our routes list.

Filter predicate application example

Routes Middleware Functions List

After this, we have to get for each route selected by match criteria, all middlewares functions stored on middlewares property.

Getting middleware functions

Where _toFlat is our Flat map to get a Flatten array.

Finally, we get all route middleware functions that we need to execute in order.

Execution pipeline

We have come to a very interesting part, where we need execute in order all middleware functions by route, stored in our fns variable.

Before continue, I believe necessary understand one thing called…

Pipelines

In another article that I wrote, called “De las promesas a los Pipes de promesas”, I’m trying to explain what is a Pipeline, thus, in this article we just extract a roughly, the essence of the concept.

When you have many functions to execute in order, I mean, where the first function result is the arguments to the next function, there are many ways to do this, example:

By chaining:

arr.filter(fnFilter).map(fnMap).reduce(fnReduce)

Readable but with iteration overhead, since we are creating by each dot chaining more than one iteration with same data.

By special methods: like Promise.all() from Promise native API.

Promise.all([Array]).then()

This just apply with promises.

However, there is one more way to do this and it’s about a feature or small approach to FP (Functional programming), welcome to the…

By pipelines:

Pipeline example

Where …fnsare the functions to execute in order, through rest operators to get an Array, before to this array, we apply a reduce method to get an single execution pipeline, we take advantage from accumulator and current object in this case both are functions, whereby we can execute and pass as argument the result from the first promise.

Well, keeping this in mind, we can review this part from our router

Pipeline for execute middlewares function in order

Well, well, slowly, first we need explain two important factors in above code:

• next function
• res function

Next function:

Remember the logic description of our router:

Logic description with next function

The next function is just the execution callback to pass to the next Promise, with the data that we want as result, in other words, the next function is the access to the next middleware function in order from our routes list route.middlewares , so, we need to execute and return the result from the next function.

But who inject these callback function like a Promise?, well…

next function description

Where:

• idx: Is the current function index, this is an indicator to always know where we stay executing.
• length: Is the second indicator, contains the total of middlewares functions to execute in this route.
• e: Is the closure function parameter, to get the entire event object from request.
• returns: A Native Promise with event object as result, to after just execute from logic description like this return next(req)

Then the return statement receive the result from next, which we know now, is a Promise with req object as result and also we know that the req object is our e closure parameter in our _next function.

Res function:

Now let’s see the full logic description example, which have many middleware functions:

Full logic description example

We have a logic description with more than one middleware function, the task of next function, as we seen, is pass the data to the next middleware function in order in our router list route.middlewares and filtered by path match, giving as result the selected route.

Then, is here where Pipeline help us to sequence the data through many functions.

In the last function, we see that we don’t execute the next() function, we execute and return res.send(req.body.auth) function instead.

res function description

Where:

• signal: Is the flag to know if continue with pipeline execution or not.
• returns: A function that returns an object with next properties:
• statusCode: Status code to send to client
• headers: header to return in response
• setHeader: Method to add new header to send in response
• send: Method to send finally the response with data as parameter to send

We ‘ill put on special focus in sendmethod.

send method description

Where:

• data: Is the data to send to client in response, I mean, the body of the response.
• signal.continue = false: Set to false the closure parameter signal, to stop any execution after this function execution in pipeline.
• returns: Promise with result in the correct format to AWS API Gateway

Welcome back to the Pipeline description

Pipeline for execute middlewares function in order

Now, we know the key pieces from above code, is time to explain what are doing this function.

1. Declare in a const variable our signal flag to stop in any time the pipeline execution.
2. The function receive in parameters the functions in Array object.
3. f Array parameter, we take it, if length of this Array, is major to 1, then we can say, that we have more than one function to execute, thus, we can apply a pipeline concept.
4. Just in case if f.length > 1 the pipeline takes all functions through reduce Array method and return a function that receive as parameter e ,which is our event object from request.
5. Now keeping in f.length > 1 case, we execute the first function called before and receive parameter like e=event object, _res(signal)() = our res function that return object with method send to resolve the and stop the execution pipeline, _next(idx, f.length) = our next function to pass the request event to the next pipeline function, and this receive the current function index in pipeline and the total of pipeline functions to execute.
6. After, we evaluate the signal.continue flag, to know if we can continue with pipeline execution, if this condition is true then continue with afterfunction execution, which receive the destructuring from the berforefunction execution result in addition with the res(signal)(),next(idx, f.length) functions.
7. If the condition signal.continue is false, then we just return the result from the first function before and set to true the signal.continue flag condition.
8. But if case f.length > 1 is false, then just we return the first and unique function result, applying the same parameters before described, replacing the next function with other res(signal)().send method, since in this point, we don’t need the next function more.

brianda-router is the result of this

As result from all above, I created brianda-router, available on npmjs.org

Conclusion

By having your own router, brings to you, the opportunity to adapt the logic to your necessities and this way don’t generate dependencies with third party libraries, also, this is a good use case to apply one small approach from FP, like Pipelines, that in combination with Compose and Transducer concepts, are fabulous.