Serverless: What should be tested?
In the past years, a lot has been written about testing strategies. During our careers we saw different architecture shifts and with each of them the test strategy has changed as well. In this article, I do not want to explain them or propose new test strategies because there are many white papers out there describing exactly these patterns.
In the following, I want to explain my point of view on testing a serverless application. Often the main concern for many people who are not familiar with serverless applications is: How do you test it? I would like to run all in docker containers and have it all up and running locally.
From my point of view, everything should be tested. During my career the concept of unit tests did not exist, and magically the software went in production and remained there for many years without any problem. But nowadays it is impossible to have a distributed system without any form of testing strategy. So like everything else, we need to find a compromise and, in our business where developing is pretty much the higher cost in a company, we must really understand where to invest and which is the added value of having a good test strategy.
Let’s take as example a typical serverless microservice:
If we are following the classic test pyramid, we should have a lot of unit tests that, in our case, are written inside our Lambda function.
Are we confident to deploy in production only with Lambda function unit tests?
If your answer is ”no”, you are right. I believe that we always must cover 100 percent of the business logic inside our Lambda function, and we should not have one big test that covers everything just for the sake of coverage lines. Just to give you an example, if in your function you have something like this:
I expect to see five tests to cover each path of the code. Unit tests are important to cover your logic, and this will significantly reduce the number of bugs that you will eventually have. Another important point to consider is that without integration test you might have missed some misconfigurations in the other components of our microservice (APIGW, SQS, DynamoDB).
Let’s go back for a moment to our application:
· Is SQS correctly receiving the message from the APIGW?
· Do we have a correct access policy on the SQS?
· Does the SQS trigger the Lambda?
· Are the DynamoDB queries, correct?
With integration tests you can cover these scenarios and the importance is self-explanatory. One important thing to mention is that for integration tests it is very important to choose the right test coverage. Compared to unit tests, the complexity and overall execution time is higher and for this reason is it better to keep the test suite fast and easy to maintain.
Are we now confident to deploy in production? Almost. With Integration tests and unit tests we are confident that the infrastructure and the compute logic works as expected.
What we really need now is a few e2e tests that will verify the behaviour of our services. In this scenario I like to combine BDD (behaviour-driven development) with Cucumber and having automation tests expressed in this way:
BDD makes use of a simple domain language and describes behaviour/requirements shared between stakeholders, domain experts and engineers.
With this, we are confident to deploy in production — every day of the week and at any time.
Conclusions:
Serverless is a shift of thinking and changes the rules in play when we want to test our application. There is no need to use local mock frameworks or to run the entire infrastructure in docker containers. With Serverless, your microservice is completely isolated, your Lambda function will have most likely a single responsibility and you do not need to test every single time the entire infrastructure like we usually do with our monolith architectures.
One of the benefits of serverless is the pay-per-use pricing model. Instead of configuring several local frameworks to run your services in your local machine, it is easier in terms of time and setup using AWS environments infrastructure or creating temporary stacks to work in complete isolation.
