A Beginners Guideline to RabbitMq and MassTransit(Part 3): User Access Management and Good Practice for Using RabbitMq
Basic of Authorization and Authentication User in RabbitMq
This article is a three-part series about complete implementation of RabbitMq as message broker . You can navigate to other part by link below :
RabbitMQ provides features for authorization and authentication for each access to it. Different users can be granted access only to specific virtual hosts. Their permissions in each virtual host also can be limited. For the sake of simplicity, we’ll define authentication as “identifying who the user is” and authorization as “determining what the user is and isn’t allowed to do.”.
The Basics
Clients use RabbitMQ features to connect to it. Every connection has an associated user which is authenticated. It also targets a virtual host for which the user must have a certain set of permissions. User credentials, target virtual host, and (optionally) client certificate are specified at connection initiation time.
There is a default pair of credentials called the default user. This user can only be used for host-local connections by default. Remote connections that use it will be refused.
When the server first starts running and detects that its database is uninitialized or has been deleted, it initializes a fresh database with the following resources:
- a virtual host named / (a slash)
- a user named guest with a default password of guest, granted full access to the / virtual host
It is advisable to pre-configure a new user with a generated username and password or delete the guest user or at least change its password to a reasonably secure generated value that won’t be known to the public.
Authentication
After an application connects to RabbitMQ and before it can perform operations, it must authenticate, that is, present and prove its identity. With that identity, RabbitMQ nodes can look up their permissions and authorize access to resources such as virtual hosts, queues, exchanges, and so on.
Two primary ways of authenticating a client are username/password pairs and X.509 certificates. Username/password pairs can be used with a variety of authentication backends that verify the credentials. Connections that fail to authenticate will be closed with an error message in the server log.
To authenticate client connections using X.509 certificate a built-in plugin, rabbitmq-auth-mechanism-ssl must be enabled and clients must be configured to use the EXTERNAL mechanism. With this mechanism, any client-provided password will be ignored.
Authorization
By default, the guest user is prohibited from connecting from remote hosts; it can only connect over a loopback interface (i.e. localhost). This applies to connections regardless of the protocol. Any other users will not (by default) be restricted in this way.
The recommended way to address this in production systems is to create a new user or set of users with the permissions to access the necessary virtual hosts. This can be done using CLI tools, HTTP API, or definitions import. This is configured via the loopback_users item in the configuration file.
It is possible to allow the guest user to connect from a remote host by setting the loopback_users configuration to none like the example below.
Managing Users and Permissions
RabbitMQ provides CLI tools to manage your users and permissions, rabbitmqctl.with this command you can do several actions like add a new user, grant permissions to a user, etc.
Before We Start: Shell Escaping and Generated Passwords. It is a common security practice to generate complex passwords, often involving non-alphanumeric characters. This practice is perfectly applicable to RabbitMQ users. Shells (bash, zsh, and so on) interpret certain characters (!, ?, &, ^, “, ‘, *, ~, and others) as control characters.
When a password is specified on the command line for rabbitmqctl add_user, rabbitmqctl change_password, and other commands that accept a password, such control characters must be escaped appropriately for the shell used. With inappropriate escaping the command will fail or RabbitMQ CLI tools will receive a different value from the shell.
When generating passwords that will be passed on the command line, long (say, 40 to 100 characters) alphanumeric value with a very limited set of symbols (e.g. :, =) is the safest option.
- Adding a User
To add a user, use rabbitmqctl add_user. It has multiple ways of specifying a password:
rabbitmqctl add_user “username”This is a basic way to add users. With this command it will prompt for a password, only use this option interactively
echo ‘password’ | rabbitmqctl add_user ‘username’In an advanced way, you can provide your password via standard output, Note that certain characters such as $, &, &, #, and so on must be escaped to avoid special interpretation by the shell.
rabbitmqctl add_user ‘username’ ‘password’And the last way is directly interpreted by a shell command, you must escape for a special character too before putting the password in your command.
- Listing Users
To list users in a cluster, use rabbitmqctl list_users:
rabbitmqctl list_usersThe output can be changed to be JSON by adding option — formatter=json
rabbitmqctl list_users — formatter=json- Deleting a User
To delete a user, use rabbitmqctl delete_user:
rabbitmqctl delete_user ‘username’- Granting Permissions to a User
Granting permissions to user can be done by command below:
rabbitmqctl set_permissions -p “custom-vhost” “username” “.*” “.*” “.*”First “.*” is used to configure permission on every entity, Second “.*” for write permission on every entity, and Third “.*” for reading permission on every entity. If you don’t want to grant it for specific access just type an empty quotation mark “”.
RabbitMQ accepts wildcards or regular expressions for granting access to specific entities. For example, you can grant read permissions of any exchanges that start with My.Example with the command below
rabbitmqctl set_permissions -p “custom-vhost” “username” “” “” “^[My.Example].*”- Revoke permissions
To revoke permissions from a user in a virtual host, use rabbitmqctl clear_permissions:
rabbitmqctl clear_permissions -p “custom-vhost” “username”Good Practice for User Management in RabbitMQ
With command on top, we can create various users and grant it for different access to achieve a full management architecture. You can separate users by each division, application, or even can be divided into users that can only configure and users that can read and write in the same application.
This is a simple example of user management that you can do in RabbitMQ. Start with an administrator user that has all access to entities in this vhost. Then create a user divide by the application that connects to it. Users for specific applications can access only entities with specific prefixes of this application. Each app has 2 users, 1 for the operator which can view the dashboard and configure entity on it, and the other one is used by the application to do write and read actions. User_app users can be kept by only sysadmin in the production server so developer and QA can only have access to read and configure dashboard entities.
Common mistakes on RabbitMQ and how to avoid it
Below is common mistakes on RabbitMQ implementation and how to avoid them:
1. Don’t open and close connections or channels repeatedly.
Have long-lived connections if possible, and use channels for each task. Don’t open a channel each time you are publishing. If you can’t have long-lived connections, then make sure to gracefully close the connection. The best practice is to reuse connections and multiplex a connection between threads with channels.
2. Don’t use too many connections or channels.
Try to keep the connection/channel count low. Use separate connections to publish and consume. Ideally, you should have one connection per process, and then use one channel per thread in your application.
- Reuse connections
- 1 connection for publishing
- 1 connection for consuming
3. Don’t share channels between threads.
Don’t share channels between threads, as most clients don’t make channels thread-safe for performance reasons.
4. Don’t have queues that are too large or too long.
Having many messages in a queue places a heavy load on RAM usage. When this happens, RabbitMQ will start flushing (page out) messages to the disk to free up RAM, and when that happens queueing speeds will deteriorate. Separate your queue into smaller ones and increase consumer on each queue to handle it
5. Don’t use old RabbitMQ/Erlang versions or RabbitMQ clients/libraries.
Stay up-to-date with the latest stable versions of RabbitMQ and Erlang. Make sure you are using the latest recommended version of client libraries.
6. Don’t have an unlimited prefetch value.
The channel prefetch value defines the max number of unacknowledged deliveries that are permitted on a channel. A typical mistake is to have an unlimited prefetch, where one client receives all messages. This can lead to the client running out of memory and crashing, and then all messages are re-delivered.
7. Don’t ignore lazy queues
Use lazy queues to achieve predictable performance or if you have large queues. With lazy queues, the messages go straight to disk, thereby minimizing the RAM usage, though throughput will be lower. Lazy queues create a more stable cluster, with more predictable performance. Your messages will not, without a warning, get flushed to disk.
8. Limit queue size with TTL or max-length, if possible
Applications that get hit by spikes of messages, and where throughput is a priority, set a max-length on the queue. This keeps the queue short by discarding messages from the head of the queues so that it’s never larger than the max-length setting.
9. Use multiple queues and consumers.
Achieve better throughput on a multi-core system with multiple queues and consumers. You will achieve optimal throughput if you have as many queues as cores on the underlying node(s).
10. Persistent messages and durable queues for a message to survive a server restart
If you cannot afford to lose any messages, make sure that your queue is declared as “durable” and your messages are sent with delivery mode “persistent” (delivery_mode=2). For high throughput send transient messages to non-lazy queues.
11. Split your queues over different cores.
Queue performance is limited to one CPU core. You will, therefore, get better performance if you split your queues into different cores, and also into different nodes if you have a RabbitMQ cluster.
12. Consume (push), don’t poll (pull) for messages.
Make sure that your consumer consumes messages from the queue instead of using basic get actions.
13. Missing a High Availability policy while creating a new vhost on a cluster.
When you create a new vhost on a cluster, don’t forget to enable an HA-policy for that vhost (even if you don’t have an HA setup, you will need it for plan changes when using classic mirrored queue). Messages will not be synced between nodes without an HA policy.
Conclusion
RabbitMq has its personal user management for authentication and authorization. it can be done with management UI or with console command tools called rabbitmqctl. It's better for us to understand those commands to implement RabbitMq with better security control. Also, there is some situation we must avoid when designing our architecture so make sure to review it.
