How to Emulate AWS SQS for Development in a Dockerized Ruby on Rails App

Ankit Samarthya
Better Programming
Published in
3 min readSep 4, 2019

In this piece, I’ll be walking you through how to mock AWS SQS for your development environment without even having to create an AWS account. I will pick up the Dockerized Ruby on Rails app ,which I used in “Add background jobs and cron to your dockerized ruby on rails app.” You can clone the app and follow along with me. Hope you are ready.

We are going to use https://github.com/softwaremill/elasticmq, which is converted to the roribio16/alpine-sqs image in Docker Hub. Let’s make edits to docker-compose.yml to add this mock SQS service that we’re going to use and also add this service as a dependency to our web service.

...
sqs:
image: roribio16/alpine-sqs
ports:
- "9324:9324"
- "9325:9325"
volumes:
- ./config/elasticmq.conf:/opt/config/elasticmq.conf
...
web:
...
depends_on:
- sqs
...

After adding the service to your docker-compose.yml file, it should look like this:

docker-compose.yml

As you can see, I am directly using the alpine-sqs image with the 9324 and 9325 ports mapped. 9324 is to use the queue service and 9325 is to access the web interface. Also, I have linked a config file to configure the queues. Content of this file is below (create this file inside a config folder with the name elasticmq.conf):

include classpath("application.conf")                                            

node-address {
protocol = http
host = "*"
port = 9324
context-path = ""
}

rest-sqs {
enabled = true
bind-port = 9324
bind-hostname = "0.0.0.0"
// Possible values: relaxed, strict
sqs-limits = strict
}

queues {
default {
defaultVisibilityTimeout = 10 seconds
delay = 5 seconds
receiveMessageWait = 0 seconds
}
service-queue {
defaultVisibilityTimeout = 10 seconds
delay = 5 seconds
receiveMessageWait = 0 seconds
}
}

An important part to focus on here is the queues section, where you can define as many queues as you want. Here we have two queues, default and service-queue. Now the queue setup is done, but we have to test it. We will test this setup by pushing to queue using aws-cli and polling for messages from our Rails app. Let’s start.

First, we will create a message poller service in our Rails app. Follow these steps to do so:

  1. Add gem ‘aws-sdk-sqs’, ‘~> 1.0.0.rc11’ to Gemfile.
  2. Create an app/subscribers/test_subscriber.rb file with the below content:
app/subscribers/test_subscriber.rb

3. Create aws.rb in your intializers folder with Aws.config = { region: ‘s3.ap-southeast-1’, logger: Rails.logger } content.

4. Create a rake task for the poller in the lib/tasks folder with the name sqs_subscriber.rake.

lib/tasks/sqs_subscriber.rake

5. Finally, add the poller service to docker-compose.yml file.

message_poller:                                             
build: .
command: bundle exec rake sqs_subscriber:subscribe_test
restart: on-failure
depends_on:
- sqs
- db

Notice the line restart: on-failure. This is important as sometimes SQS is not fully ready, so the message poller just throws some error and shuts down.

6. Add the following variables into the .env file, which we used in the test_subscriber.rb file:

SQS_ENDPOINT="http://sqs:9324"                                            SQS_REGION="ap-southeast-1"                                            SQS_QUEUE_NAME="service-queue"                                            AWS_ACCESS_KEY_ID=""                                            AWS_SECRET_ACCESS_KEY=""

7. All the changes are done. Now we can run docker-compose build and then docker-compose up. Once all the services comes up, we are ready to test it.

I assume you have aws-cli installed on your machine. If not, you can install it by following the steps here.

Fire up a terminal, and run the below command to send a message to the queue we just created:

aws --endpoint-url http://localhost:9324 sqs send-message --queue-url http://localhost:9324/queue/service-queue --message-body "Hello, queue"

After you hit return, you will see a log message in the development.log file. It’ll be something like: “Message received: \”Hello world\”” . This means our test subscriber is able to poll for the messages. Amazing — we are able to emulate SQS without even having to create a AWS account. How cool is that?

Let me know in the comments if you have any questions or problems with the code. You can find the full source code here.

--

--

No responses yet