Going Serverless with AWS Lambda, Serverless Framework and faunaDB.

Serverless application from scratch with Serverless Framework, AWS Lambda and faunaDB

TL; DR

• “serverless” design is using the FaaS (Function as a Service) feature in cloud environments.
• In serverless applications you deploy functions, not servers. Those functions are executed when they are triggered.
• Examples: AWS Lambda, Google Cloud functions, Azure functions
• Serverless Framework can be used to develop and deploy serverless applications
• faunaDB is a noSQL database with some features of relational databases
• Starter code is here https://github.com/budvinchathura/serverless-faunadb-test-starter
• Complete source code is here https://github.com/budvinchathura/serverless-faunadb-test

What is “Serverless”?

“Serverless” technology in cloud environments is relatively a new technology compared to traditional server-based options. This is also called as “Function as a Service” feature. You can think it is as a function deployed in the cloud. It is not like a server which is running continuously; it is only a function definition. Just like in programming languages, you just define a function to be used later. When a user calls that cloud function, cloud vendor spins up an execution environment and execute the function. Hence serverless functions are short-lived and can be considered as “stateless”. (But different cloud vendors may have slightly different implementations where the execution environment or context is sometimes shared between consecutive function invocations.)

Some vendor-specific serverless examples are,

• AWS Lambda functions
• Google Cloud functions
• Azure functions

Source: https://www.freecodecamp.org/news/what-is-serverless-architecture-what-are-its-pros-and-cons/

In most of the times, serverless approach can be considered as a cost-saving way because you are billed only for the execution time of function invocations. (Currently, pricing for AWS Lambda is around $0.20 per 1 million requests and $0.0000166667 for every GB-second). Meanwhile, if you are hosting a traditional server you are billed for each hour (or day) regardless of the traffic to the server. Also, scalability in serverless is more flexible than using a server because you can have a large amount of serverless function executions running at the same moment.

But don’t get too excited about this because serverless has its own drawbacks as well. If your function gets called for too many times for much long-running tasks, it may not be efficient or cost effective as traditional server-based approach. There are other issues like ‘cold-starts’, debugging issues, managing security and vendor locking (for some extent).

Degree of automation when using serverless design (Source: https://specify.io/concepts/serverless-baas-faas)

Shifting to serverless does not mean that we are getting rid of servers. Any cloud service needs running servers for execution. Serverless is a slightly different paradigm which provides much abstraction for the developers with more flexible pricing strategies and scalability options.

You can easily find more insights about serverless architecture and its characteristics in many other articles. In this article, I am going to guide you to build a serverless app from scratch.

I will use these technologies and frameworks for this tutorial.

• Serverless Framework
• AWS Lambda
• faunaDB (faunaDB is a noSQL database with some features of relational databases. You may use any other database service if you have a running instance for that.)

What you will need

• Activated AWS account with AWS CLI configured with an Administrator IAM user.

You can use these user guides for configuring AWS CLI V2.

Installing, updating, and uninstalling the AWS CLI version 2

Configuration basics

Check your AWS CLI configuration by running following command in your terminal.

aws configure list

You should get something similar to this, with ‘XXXX’ replaced by your credentials and your region instead of ‘us-east-1’. (I am using us-east-1 for this tutorial.)

Name                    Value             Type    Location
      ----                    -----             ----    --------
   profile                <not set>             None    None
access_key     ****************XXXX shared-credentials-file
secret_key     ****************XXXX shared-credentials-file
    region                us-east-1      config-file    ~/.aws/config

• Create a new DB in faunaDB from here and obtain access credentials for that DB.

faunDB security key tab for a database

• npm and NodeJS installed.
• Install serverless as a global npm package.

npm install -g serverless

Check your installation with,

serverless -v

You should get something like the following configuration. Exact versions may differ.

Framework Core: 2.20.1
Plugin: 4.4.2
SDK: 2.3.2
Components: 3.5.1

Let’s code

We are going to create a simple CRUD API with entities as ‘users’. ‘user’ records can be added, modified, searched, listed and deleted. Let’s start with the starting code available at https://github.com/budvinchathura/serverless-faunadb-test-starter. Clone this repo and install all the dependencies with

npm install

Since we are using TypeScript for this we have to use webpack to get JavaScript output and then deploy it to AWS Lambda. (Currently AWS Lambda supports several runtimes such as NodeJS, Python, Ruby, Java, Go and .NET. If you decide to use JavaScript for this, you don’t need webpack configurations. JavaScript code can be directly added to AWS Lambda with Serverless). For now, you don’t have to worry about webpack related configurations, I have added them to the repo already.

Before compiling or deploying any code, you have to create env.json file in project root directory and add the following contents where xxxxxxxx is replaced with your access secret for the newly created empty database in faunaDB.

{
    "FAUNADB_SECRET_KEY":"xxxxxxxx"
}

This is obviously to keep our environment variables, in this scenario faunaDB access credentials.

Now let’s look at the contents of each file.

1. fauna-setup.js

This is basically the initial structure of our database. You can see in line #8 we are creating a new client to access faunaDB. In line #9 we are creating a new collection called ‘users’. A ‘collection’ is something like a ‘table’ in relational databases. You can execute this file with just the node execution environment with.

node faunadb-setup.js

If everything goes well, you will see the terminal output as,

"users" collection created.
finished

Now our database is ready.

2. serverless.yml

This is the most important part of our initial code. serverless.yml file defines the infrastructure resources we need inside AWS. Basically we are deploying a service called ‘serverless-faunadb-test-starter’ and inside that service, we have 2 functions.

• create
• list

These are actually 2 AWS Lambda functions that we are going to create inside this service.

Lets consider the ‘create’ function.

create:
  handler: users/create.create
   events:
     - http:
         path: users
         method: post
         cors: true

• ‘handler’ attribute defines the actual function in our code, which should be executed when we call ‘create’ function in our service.
• Meaning of users/create.create is that we are using ‘create’ function of ‘create’ file which is inside ‘users’ directory. (It is just the path to the actual function).
• ‘events’ attribute defines trigger(s) for this function. (specifying what should trigger this function). AWS has many triggers such as http, websocket, schedule, sns etc. Single lambda functions may have more than one trigger. Here we are using only an http event trigger since we need to execute this Lambda function when a user sends an http request to our service.
• Under http event, path attribute is for the URL prefix of our endpoint. Method is POST since we have to get details for the new user in http body. CORS is set to ‘true’ to allow requests from all domains.

If you want, you can change ‘region’ attribute under ‘provider’ to any other AWS region.

For this service we have defined our faunaDB secret under environment variables (line #27, #28). Meaning of ${file(./env.json):FAUNADB_SECRET_KEY} is to read env.json file and get FAUNADB_SECRET_KEY attribute from that json file.

Now we will move into actual handler files.

create.ts

In this function, we are parsing and validating the data attributes which are submitted to POST /users endpoint. After that, the user record is created with faunadbClient.query method. Finally, the response is formatted and sent to the user.

If we look into buildResponse function which is in utils.ts file,

We should set ‘Content-Type’ header and also ‘statusCode’ since this output from our Lambda function is forwarded to the API users via AWS API Gateway. Those attributes are needed to transform Lambda response to an http response.

Code segments for ‘list’ function are similar to ‘create’ function with few changes to query for existing users. Function implementation code is in users/list.ts list function. I assume it will not be much difficult to understand the code related to the ‘list’ function.

Now we have developed 2 functions and it’s time to deploy.

Let’s deploy the code

I am sure you have previously deployed some web server related project to a cloud virtual instance. Most of the times, this deployment process involves provisioning virtual machines, installing dependencies, configuring the server and several other tasks. With serverless design and thanks to Serverless Framework, deploying one service is just a single command. Run this command in the project root directory.

serverless deploy

If everything goes well, after a few seconds, you should see the last few lines of the terminal output as.

......
service: serverless-faunadb-test-starter
stage: dev
region: us-east-1
stack: serverless-faunadb-test-starter-dev
resources: 17
api keys:
  None
endpoints:
  POST - https://xxxx.execute-api.us-east-1.amazonaws.com/dev/users
  GET - https://xxxx.execute-api.us-east-1.amazonaws.com/dev/users
functions:
  create: serverless-faunadb-test-starter-dev-create
  list: serverless-faunadb-test-starter-dev-list
layers:
  None

‘xxxx’ will be replaced by a unique id for your API service.

Yp, that’s it, we just deployed our service with 2 functions (or 2 endpoints).

This serverless deploy command will execute many steps.

1. Compile TypeScript code into JavaScript
2. Bundle with necessary dependencies
3. Upload code to AWS
4. Validate serverless.yml template
5. Create AWS resources
6. Remove previous resources or resource definitions. (only if you are re-deploying after some code changes)

In fact, step 5 creates many AWS resources such as,

• Lambda functions (for each function we have defined)
• API Gateway (to connect these Lambda functions with http triggers)
• Log groups for each Lambda functions. (If you have any console.log statements or any logging commands in your code, those console outputs will be automatically added to these logs. Logs also contain execution time and memory used for each Lambda invocation)
• S3 bucket (to store our code and Serverless template in AWS).
• Cloudformation stack (It is the entity which packages all above resource as a single resource group so that tracking and managing is easier when accessed from one place.)

After login into AWS Console You should be able to view this deployment stack in https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/

There should be stack named ‘serverless-faunadb-test-starter-dev’. If you click and open that stack, you will be able to see all the resources and even get a simple visualization of those resources.

Note: If you have changed your region to something other than ‘us-east-1’, please edit and use above link accordingly.

Testing the deployment

Now we have deployed our code and its time to test the functionality. Remember we got our endpoint URL s in the terminal output just after the deployment.

To create a new user, we are using ‘create’ endpoint. Note that this is a POST request and the content type is json. Remember to replace ‘xxxx’ with the real identifier and replace ‘us-east-1’ with the region. (only if you have changed the region in serverless.yml).

curl --request POST https://xxxx.execute-api.us-east-1.amazonaws.com/dev/users --header "Content-Type: application/json" --data-raw '{"name":"Test user 1", "dob":"2000-01-01","email":"abc@abc.com","city":"Downtown"}'

This will return an output similar to,

{"                           
status":"SUCCESS","data":{"user":{"name":"Test user 1","dob":"2000-01-01","city"                           
:"Downtown","email":"abc@abc.com","id":"288722901473952259"}}}

Value of id field maybe different since this id is auto generated by faunaDB for each record.

We have just created a user in our database. Now let’s view the list of users.

curl https://xxxx.execute-api.us-east-1.amazonaws.com/dev/users

It will return the users as,

{"status":"SUCCESS","data":{"users":[{"name":"Test user 1","dob":"2000-01-01","city":"Downtown","email":"abc@abc.com","id":"288722901473952259"}]}}

I assume now you have a fair knowledge about deploying a service which has several functions by using Serverless Framework. You can now try to add new endpoints like

• Get details of a specific user
• Update a user
• Delete a user

Procedure is same as previous 2 functions we just deployed.

• Add the function definition to serverless.yml file
• Create a new .ts file or edit an existing .ts file to add the function implementation code. Remember to declare the TypeScript function as an export function. If not, Serverless Framework can not access your function code.
• Run serverless deploy in the project root terminal

If you just want to modify .ts files only for a single function, after modifying those files you can run the following command to re-deploy only that function.

serverless deploy -f functionName

For example, if I modify any code only in create.ts, I have to run

serverless deploy -f create

This command will re-deploy only the ‘create’ function.

Note that if you edit any code in types.ts or utils.ts you have to run serverless deploy -f functionName for all the functions, because those files are imported in all the functions that we have defined (or you can run serverless deploy to re-deploy the whole service). Also, if you are doing any modifications to serverless.yml you have to run serverless deploy.

These re-deployments will not change the URL s of existing endpoints. (assuming you are not changing serverless.yml attributes such as stage and region.)

If you want to remove the deployed service and delete all AWS resources, simple run the following command.

serverless remove

This command will remove all the AWS resources related to the service.

Note: If you execute serverless deploy again after serverless remove , ‘xxxx’ part in https://xxxx.execute-api.us-east-1.amazonaws.com will change.

Serverless Framework also offers to execute and debug your functions locally. For VSCode editor, I have included the launch configurations in .vscode -> launch.json file. Or you can run

serverless offline

in the terminal to start executing locally.

When running locally the URL prefix will be changed to localhost. i.e. https://xxxx.execute-api.us-east-1.amazonaws.com will be changed to something like http://localhost:3333. With the offline, feature you can test and debug your functions before deploying to the cloud environments.

After some improvements to the code, I have added some additional features like,

• Get user, update user, delete user
• Add followers to a user, remove followers from a user

For the ‘followers’ feature, you may have to use faunaDB indexes to model the relationship between ‘user’ records.

Complete source code with above functionalities can be accessed in https://github.com/budvinchathura/serverless-faunadb-test

It is not mandatory to use faunaDB for this, but I just thought using it since it will also be a new experience. You can always use any other different databases like PostgreSQL, MySQL, MongoDB, AWS DynamoDB etc.

If you are using a relational database like PostgreSQL or MySQL, pay attention to use a singleton design pattern for database connections. This is because AWS Lambda persists execution environment for consecutive Lambda invocations, up to some limits. (read more about that in here). Hence, you can initialize database connection outside the function handler and re-use that connection. But don’t forget to re-check and initialize the connection when necessary.

You can find a lot more tutorials and examples in https://www.serverless.com/examples/ about Serverless Framework.

You will be able explore many things like,

• Adding authorizers for Lambda functions (custom authorizers or AWS Cognito user pools)
• Using AWS DynamoDB, AWS RDS with Serverless
• Uploading files to S3 or any other storage services
• Creating multiple services under a single API gateway. (in this tutorial we deployed only one service)
• Other Lambda triggers like websocket, data streams, scheduled triggers etc.

I hope now you have a fair understanding of serverless design. But keep in mind that serverless is not a complete replacement for traditional server-based design. This serverless pattern has it own ups and downs. Please read about them and get a good understanding before using serverless in a production environment. Also, don’t forget to try Serverless Framework with other cloud vendors as well.

Good luck with your next serverless project !!!