Bootstrap SQS with Spring Cloud Messaging
Setup a Simple Queue Service (SQS) consumer in less than 20 lines of code for Asynchronous Message Processing
With some workloads not requiring instantaneous feedback to the end-user (e.g. video processing/conversion), asynchronous message processing has become an extremely appealing offering in recent years. Thanks to AWS, you have a messaging platform at your fingertips where you don’t have to worry about managing the infrastructure but instead focus on providing value to your customers.
What you’ll need
- Java 8 or above
- Gradle/Maven
- AWS Account
- IAM user with SQS Access
Project Setup
As with any Spring project, Spring Initialiser provides a simple, user friendly way to generate a project. Let’s add all the dependencies we are going to need and then open the project in your favorite IDE
Creating SQS Async Client
Notice here we are using DefaultAWSCredentialsProviderChain to get the IAM credentials rather than defining a static credentials provider. This will tell Spring Cloud to use the credentials provided in the Environment to configure access so you don’t have to specify any credentials when you deploy your application (all AWS services have these preconfigured for you).
Configuring Spring Cloud
By default, Spring Cloud Messaging thinks that you are running your application using a CloudFormation template. When you run your application it will throw an exception about the stack not existing so this disables that feature. Unless you are using CloudFormation, set the property above to false.
Setting up SQS
Type SQS into the AWS Console service search and select the first option that appears Simple Queue Service (SQS)
Click on the Get Started Now button and type in a name for your queue. For the purpose of this tutorial, I will be using sqs-demo. Select the type of queue you would like (self-explanatory) and hit Quick Create Queue
Adding SQS Consumer
Setup a new consumer and annotate it with @Component so Spring can bootstrap a new class and @Slf4j so that we can get some logging features provided by Lombok.
Annotating our method with @SqsListener("QUEUE_NAME") tells Spring to grab that queue from SQS and start listening for messages. Run the application and if we now send a test message in the SQS interface, we should see it log to our console.
Processing JSON Messages
Define a new model (doesn’t need to be identical to the one below) that we are going to be using to deserialise the messages we receive on the listener.
Spring Cloud Messaging configures it’s own object mapper when serialising/deserialising messages it sends/receives. Unfortunately this doesn’t support JSON out the box, so we need to configure it.
In essence, we are defining a new message converter using our applications default object mapper, then passing that to a QueueMessageHandlerFactory which tells Spring to use our custom message converter when deserialising the messages it receives in its listeners. Change the input parameter you specified on your @SqsListener to the newly created model and send another message, this time in JSON format and you should see it log to console.
Wrapping things up
By now you can probably see how easy it is to setup and process messages asynchronously. So let’s recap a few things
- AWS Services: removes over head management of infrastructure, scaling etc.
- Spring Cloud Messaging: provides an abstract way of defining a messaging API independent of service provider
- DefaultAWSCredentialsProviderChain: uses environment variables to setup the IAM access
- Jackson Serialisation/Deserialisation: not supported by default, needs additional configuration
The full source code for the project can be found here with all the gists used in this tutorial
