Microservices Communications with AWS Lambda Invocation Types

Published in

AWS Serverless Microservices with Patterns & Best Practices

10 min readMar 12, 2022

In this article, we are going to see how Microservices Communications handle in Serverless world with using AWS Lambda Invocation Types.

Serverless Event-driven E-commerce Microservices Architecture

We will map these Lambda Invocation Types with our architecture when designing Serverless E-commerce application. So AWS Lambda Invocation Types will be our Microservices communication styles when developing our Serverless Event-driven E-commerce Microservices Architecture application.

Let’s start with remembering AWS Lambda Invocation Types.

Step by Step Design AWS Architectures w/ Course

I have just published a new course — AWS Serverless Microservices with Patterns & Best Practices.

In this course, we’re going to learn how to Design and Develop AWS Serverless Event-driven Microservices with using AWS Lambda, AWS DynamoDB, AWS API Gateway, AWS EventBridge, AWS SQS, AWS CDK for IaC — Infrastructure as Code tool and AWS CloudWatch for monitoring.

Source Code

Get the Source Code from Serverless Microservices GitHub — Clone or fork this repository, if you like don’t forget the star. If you find or ask anything you can directly open issue on repository.

AWS Lambda Invocation Types

AWS Lambda functions is using everywhere for various use cases. Sometimes we want to invoke Lambda functions over HTTP and other times trigger a function based on different events. It is extremely useful to understand the different ways that we can invoke Lambda functions.
Here are 3 different ways we can invoke AWS Lambda function:

AWS Lambda has 3 Invocation Types;

Lambda Synchronous Invocation
Lambda Asynchronous Invocation
Lambda Event Source Mapping with Polling Invocation

https://aws.amazon.com/blogs/architecture/understanding-the-different-ways-to-invoke-lambda-functions/

You can all invocation types in one image.
Synchronous invocation is trigger by push model and execute immediately and wait response. we can say RequestResponse model.
Asynchronous invocation is trigger by event model and executes asynchronous don’t wait immediate response.
Event Source Mapping invocation is triggered by poll-based. That means lambda function receives multiple items.

Lets see one by one and explain these invocation types.

Lambda Synchronous Invocation

This is one of the simple lambda invocation model. In this communication type, lambda functions execute immediately when you perform the Lambda Invoke by API call.
So when we perform a Lambda invoke an API call, we wait for the function to process the function and return back to response. The response data includes the function response and additional data which can be retrieved from DynamoDB or some other resources.

A common scenario for synchronous invocation is the API Gateway + Lambda integration:
You can see in the image of API Gateway — Lambda — DynamoDB that we can perform CRUD operations for a particular DynamoDB table with exposing APIs from API Gateway.

The important thing; The Invocation-type flag should be “RequestResponse”. This instructs AWS to execute our Lambda function and wait for the response of function to complete.
When we perform a synchronous invocation, we are responsible for inspecting the response and determining if there was an error like time out exceptions and decide to retry the invocation.

We can perform Synchronous RequestResponse invocation using the AWS CLI, SDKs and many other options.

Example of synchronous invocation using the AWS CLI:

aws lambda invoke — function-name MyLambdaFunction — invocation-type RequestResponse — payload ‘{ “key”: “value” }’

As you can see that when we specify the Invocation-type flag is RequestResponse. That means AWS execute our function and wait for the function to complete. The important part is we are responsible for checking the response and determining whether it is an error and we should retry the invoke.

There are many AWS services that can trigger lambda function synchronously.
Here are some of them:

ELB (Application Load Balancer)
Amazon Cognito
Amazon Lex
Amazon Alexa
Amazon API Gateway
Amazon CloudFront (Lambda@Edge)
Amazon Kinesis Data Firehose

Lambda Asynchronous Invocation

In this communication type, Lambda functions works with events and invoke by events and not respond immediately.

Basically we can invoke a lambda function asynchronously, after that Lambda sends the event to a internal queue and returns a success response without any additional information. After that a separate process reads events from the queue and runs our lambda function.

S3 + Lambda + DynamoDB

A common scenario for Asynchronous invocation is the S3 / SNS + Lambda + DynamoDB. You can see in the image of S3 / SNS + Lambda + DynamoDB.

In the diagram, Amazon S3 will call the lambda asynchronous when a new object is written into the bucket. In order to invoke a function asynchronously, we should set the invocation type parameter to Event.

Example of an asynchronous invokes using the CLI:

aws lambda invoke — function-name MyLambdaFunction — invocation-type Event — payload ‘{ “key”: “value” }’

When we specify the Invocation-type parameter value as Event if the function failed AWS will automatically retry the invoke twice.

By default AWS Lambda sets a retry policy for asynchronous invocation, your function will be invoked twice if there is an error.

In order to avoid event losses when using asynchronous lambda, always attach a dead-letter queue on the function in order to save events that weren’t successfully processed.

Here is a list of services that invoke Lambda functions asynchronously:

S3
EventBridge
SNS
SES
CloudFormation
CloudWatch Logs
CloudWatch Events
CodeCommit
Pool Based Invokes

Lambda Event Source Mapping with Polling Invocation

Pool-Based invocation model allows us to integrate with AWS Stream and Queue based services. These services don’t invoke Lambda function directly. Lambda will poll from the AWS SQS or Kinesis streams, retrieve records, and invoke functions.
AWS Lambda Event Source Mapping manages the poller and performs Synchronous invokes of our function.

Data stream or queue are read in batches, the function receives multiple items when execute function. Batch sizes can configure according to service types and after that the event source mapping will send data according to batch size. If the batch size is too large to send in one event, it has to be split up, resulting in events with smaller numbers of items than the batch size.

SQS + Lambda

A common scenario for Event Source Mapping with Polling Invocation is the SQS + Lambda. You can see in the image of SQS + Lambda

In the diagram, Amazon SQS has some queue records, this queue record can be added from other AWS Services like EventBridge, SNS and so on.
After that a lambda function poll events from AWS SQS and receive queue records with Event Source Mapping invocations. Lambda function retrieve queue records according to Batch Size property of polling configurations.

Another common scenario for Event Source Mapping with Polling Invocation is stream based processing. These are Kinesis and DynamoDB Streams that Lambda can poll stream records with Event Source Mapping invocations.

Here is a list of services that invoke Lambda functions as event source mapping:

Amazon Kinesis
Amazon DynamoDB
Amazon Simple Queue Service

As you can see we have we have seen AWS Lambda Invocation Types.

Now let’s remember the microservices communication types, after that we will map these two concepts when designing our Serverless E-Commerce architecture.

Microservices Communication Types — Sync or Async Communication

We are going to learn Microservices Communication types — Synchronous — Asynchronous Communication.

Client and services communicate with each other with many different types of communication. Mainly, those types of communications can be classified in two axes.

Synchronous and Asynchronous

Lets start to talk about Synchronous communication.

What is Synchronous communication ?

Basically, we can say that Synchronous communication is using HTTP or gRPC protocol for returning sync response. The client sends a request and waits for a response from the service. So that means client code block their thread, until the response reach from the server.

The synchronous communication protocols can be HTTP or HTTPS.
In synchronous communication, the client sends a request with using http protocols and waits for a response from the service.

So that means the client call the server and block client their operations.
The client code will continue its task when it receives the HTTP server response. So this operation called Synchronous communication. It has pros and cons that we should consider when we pick this way.

Another communication type is Asynchronous communication.

What is Asynchronous communication ?

Basically, In Asynchronous communication, the client sends a request but it doesn’t wait for a response from the service. So the key point here is that, the client should not have blocked a thread while waiting for a response.

The most popular protocol for this Asynchronous communications is AMQP (Advanced Message Queuing Protocol). So with using AMQP protocols, the client sends the message with using message broker systems like Kafka and RabbitMQ queue. The message producer usually does not wait for a response. This message consume from the subscriber systems in async way, and no one waiting for response suddenly.

An asynchronous communication also divided by 2 according to implementation. An asynchronous systems can be implemented in a one-to-one(queue) mode or one-to-many (topic) mode.

In a one-to-one(queue) implementation there is a single producer and single receiver. But in one-to-many (topic) implementation has Multiple receivers. Each request can be processed by zero to multiple receivers. one-to-many (topic) communications must be asynchronous.

So we will see this communication with the publish/subscribe mechanism used in patterns like Event-driven microservices architecture in the upcoming articles. Basically an event-bus or message broker system is publishing events between multiple microservices, and communication provide with subscribing these events in an async way.

Kafka and RabbitMQ is the best tools for this operations.

As you can see that we have understand both Microservices Communication types and AWS Lambda Invocation Types. It’s time to map these two concepts when designing our Serverless E-Commerce architecture.

Serverless E-Commerce Microservices with Lambda Invocation Types

If we map these Lambda Invocation Types with our microservices architecture when developing serverless e-commerce application, you can see the diagram below;

So AWS Lambda Invocation Types will be our Microservices Communications when developing our Serverless Event-driven E-commerce Microservices application.

So during the article, we will follow these Lambda Invocation types as a Microservices Communication types of our serverless e-commerce microservices project.

Lambda Invocation Types = Sync -> Async -> Event Mapping

Following Microservices Communication Types for Serverless

Synchronous Communication
Asynchronous Communication
Event Source Mapping for Poling queue events

So how we can follow these invocation types into our application ?

Synchronous Communication for RESTful Microservices

Synchronous commands are request/response. In a synchronous model, the client makes a request and waits for a response. We will developed RESTful Microservices with API Gateway, Lambda Functions and DynamoDB.

We called this = RESTful Microservices

REST API and CRUD endpoints (AWS Lambda, API Gateway)
Data persistence (AWS DynamoDB)

AWS API Gateway -for- Restful API-Driven Development for performing CRUD operations amongst lambda and DynamoDB.

Asynchronous Communication

In an asynchronous design, the client sends a request and may get an acknowledgement that the event was received, but it doesn’t get a response that includes the results of the request.

We will use

Amazon EventBridge -for- Event-Driven asynchronous communication between Microservices

We will develop Amazon EventBridge with Lambda Basket microservices which publish the event in asynchronous way and this will consume by Ordering queue.

Event Source Mapping for Polling Event Messages

In a polling event, consumers poll the producer for messages in batches and then process the batch before returning for a new batch of records.

We will use

AWS SQS -for- Decouple Microservices and processing events asynchronously using queues

We will develop AWS SQS with Lambda Ordering microservices which consumes by polling the event from the Ordering queue.

As you can see that we follow Serverless Microservices with AWS Lambda Invocation Types; Following Microservices Communication Types for Serverless;

Synchronous Communication
Asynchronous Communication
Event Source Mapping for Poling events

AWS Serverless Microservices for Ecommerce Application Architecture

This is the big picture of what we are going to develop together for AWS Serverless Event-driven E-commerce Microservices application that is Step by Step Implementation together.