How to clean install RabbitMQ

How to uninstall and install rabbitMQ the right way ✅

TLDR;

1. RabbitMQ stopped working suddenly with the below error.
2. Tried many things. ( working witherlang.cookie, uninstalling, reinstalling…. )
3. Thought of resetting my homebrew, but I didn’t.
4. Finally fixed it by clean installing RabbitMQ.

{"level":"fatal","error":"dial tcp [::1]:5672: connect: connection refused","caller":"path/to/go/file/connecting/to/rabbitmq"}

Backstory

RabbitMQ stopped working suddenly with the above error.

Skip Backstory & Navigate to Fix

Started debugging from here “error”:”dial tcp [::1]:5672: connect: connection refused”

1. Check if rabbitmq is running brew services list
2. You should see something like rabbitmq stopped
3. Try starting it with brew services start rabbitmq You should see a log saying rabbitmq started successfully.
4. Now go to Step 1 and observe that rabbitmq is still not started
5. Try brew services restart rabbitmq You should see a log saying rabbitmq stopped and started successfully.
6. You will still see rabbitmq stopped when checked in the services list ( Step [1] )

I was blocked by this loop for some days. Then tried to start the server by runningrabbitmq-server then I got some new errors

2023-07-19 17:09:51.744537+05:30 [error] <0.234.0> Feature flags: `maintenance_mode_status`: required feature flag not enabled! It must be enabled before upgrading RabbitMQ.
2023-07-19 17:09:51.749121+05:30 [error] <0.234.0> Failed to initialize feature flags registry: {disabled_required_feature_flag,
2023-07-19 17:09:51.749121+05:30 [error] <0.234.0>                                               maintenance_mode_status}

2023-07-19 17:09:51.753208+05:30 [error] <0.234.0>
2023-07-19 17:09:51.753208+05:30 [error] <0.234.0> BOOT FAILED
2023-07-19 17:09:51.753208+05:30 [error] <0.234.0> ===========
2023-07-19 17:09:51.753208+05:30 [error] <0.234.0> Error during startup: {error,failed_to_initialize_feature_flags_registry}
2023-07-19 17:09:51.753208+05:30 [error] <0.234.0>
BOOT FAILED
===========
Error during startup: {error,failed_to_initialize_feature_flags_registry}

2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>   crasher:
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     initial call: application_master:init/4
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     pid: <0.233.0>
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     registered_name: []
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     exception exit: {failed_to_initialize_feature_flags_registry,
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>                         {rabbit,start,[normal,[]]}}
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>       in function  application_master:init/4 (application_master.erl, line 142)
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     ancestors: [<0.232.0>]
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     message_queue_len: 1
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     messages: [{'EXIT',<0.234.0>,normal}]
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     links: [<0.232.0>,<0.44.0>]
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     dictionary: []
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     trap_exit: true
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     status: running
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     heap_size: 376
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     stack_size: 28
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>     reductions: 173
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>   neighbours:
2023-07-19 17:09:52.755297+05:30 [error] <0.233.0>
2023-07-19 17:09:52.773504+05:30 [notice] <0.44.0> Application rabbit exited with reason: {failed_to_initialize_feature_flags_registry,{rabbit,start,[normal,[]]}}
Kernel pid terminated (application_controller) ({application_start_failure,rabbit,{failed_to_initialize_feature_flags_registry,{rabbit,start,[normal,[]]}}})

Crash dump is being written to: erl_crash.dump...done

Seems to be some error with Feature Flag, updated .erlang.cookie but nothing worked.

Decided to uninstall and re-install Rabbitmq

brew services stop rabbitmq
brew uninstall rabbitmq
rm -rf /opt/homebrew/etc/rabbitmq
rm -rf /opt/homebrew/etc/rabbitmq/enabled_plugins
rm -rf /opt/homebrew/etc/rabbitmq/enabled_plugins.default
rm -rf /opt/homebrew/etc/rabbitmq/rabbitmq-env.conf

Still faced the same issues.

Done the same thing thrice. But errors remained the same.

I understood one thing from this. There must be some files that are not deleted.

I decided to search for the folder rabbitmq on my complete mac

find / -type d -name "rabbitmq"

The above command returns a log for every path it looked in with a status found or not found

We can only list down paths where the rabbitmq directory is present with the below command

find / -type d -name "rabbitmq" -print 2>/dev/null

The operation took ~10mins to complete

Here I can see some paths that might have stale rabbitmq data.

/opt/homebrew/var/lib/rabbitmq
/opt/homebrew/var/log/rabbitmq

Note: There are ~20 other paths containing directory rabbitmq but the issue might be from the above two paths as we have installed rabbitmq with homebrew and any stale data traces of previous version should be here.

There will be some paths starting with /dev/fd that contains homebrew. Don’t touch them, they got nothing to do with our issue.

Let’s remove rabbitmq completely!

Fix

Stop and uninstall rabbitmq

brew services stop rabbitmq
brew uninstall rabbitmq

remove the stale data of RabbitMQ

rm -r /opt/homebrew/etc/rabbitmq                
rm -r /opt/homebrew/var/lib/rabbitmq
rm -r /opt/homebrew/var/log/rabbitmq

re-install rabbitmq

brew update
brew install rabbitmq
brew services start rabbitmq

if the last command throws an error try

brew services restart rabbitmq

if the above command throws an error simply run

rabbitmq-server

Once you are able to start rabbitmq the below commands should work fine

brew services stop rabbitmq
brew services start rabbitmq

PS: I didn't remove Erlang in this procese.

Conclusion

This clean install process should work for any service installed via homebrew

Thanks for reading! Drop a clap if you found it helpful. 👏

Cheers! Anjan Talatam