6 Fantastic Mistakes That You can do using RabbitMQ (+NodeJs)

Deshan Madurajith
Oct 13 · 3 min read

RabbitMQ is a very good solution to many problems. If you don't use it correctly, It will be a monster. Yes, it is a monster.

RabbitMQ is a Moster
RabbitMQ is a Moster

1. Don’t use RabbitMQ or any queues unless it is necessary

What? Yes. Most people use message queues to create a Picasso (A Famous Painter) Like Architecture. Most of the cases, you don’t need it. If you add more and more components to your architecture means that you have more and more dependencies. So, it is better to avoid dependence. Always Keep it Simple, Stupid (KISS).

$179 million Picasso Paint

An alternative to Queue — Most of the time database is good enough to achieve your goals. Most of the time Queue increases the complexity. Advice — Evaluate before you implement

2. Don’t implement without doing proper research (learn theories)

There are some theories that you need to learn. Don’t just play around it and use it. You should know some theories, features like How to design queues, exchanges, exchange_type, channel vs connection.

Then you will be able to apply correct theories and implement proper patterns to improve the performance and utilization.

3. Don’t use misuse connections

Anyone knows that we should not create many connections. Each connection uses at least 100 kb of RAM. If the application opens more and more connections without closing them, the application will not be able to create connections after some point because out of memory. That means the RabbitMQ will reject the new connection. Worst case the rabbitMQ will be crashed.

First of all, make sure that you close the connection. To monitor that you can use Management UI. You will be able to see the following UI.

The best practice is to reuse connections and multiplex a connection between threads with channels.

4. Don’t add large messages in queues

If you try to pass a large message in queue, it may be not a good way to do it. For example, at the beginning of a project, we send a large payload via RabbitMQ. It worked as expected without any performance issues. After a few months, we got a lot of messages with a large payload, then the rabbitMQ was crashed. Then we move the payload to S3, then we put S3 URL to rabbitMQ.

Upload S3 then put ref to RabbitMQ

large payload means

  • RabbitMQ keeps in the RAM. If it has large, then put some of them to disk (page out). Then the rabbitMQ can be slow.
  • Page out may block the queue from processing
  • It may take more time for indexing with large files. It also takes more time to restart the rabbitMQ.

5. Auto-delete queues you are not using

Having an unused queue means that rabbitMQ has to allocate resources for unused queue. That could affect performance. You can add the following setting to auto-delete queue

5. Don't forget to use Prefetch

The prefetch means — Number of messages that sends to the consumer at the same time. API should be utilized as much as possible. If we send 1 as prefetch, it may not affect the API because the CPU or RAM will not utilized.

The prefetch value is used to specify how many messages are being sent to the consumer at the same time. It is used to get as much out of your consumers as possible.

A too-small prefetch count means the RabbitMQ is waiting to get permission to send more messages. more about prefetch — https://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth/

Deshan Madurajith

Written by

UI/UX enthusiastic, ❤️ new Technologies and build great things 🤨😉… | Sri Lankan🇱🇰 linkedin.com/in/deshan-m/

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade