Organize your development processes with Overmind
Rarely can a developer get away with running just a single process to get their entire development environment running.
To start developing our apps we often have to spin up a bunch of processes, for example:
- the API server (eg: rails)
- the client app (eg: webpack)
- the background job processor (eg: sidekiq)
- and there could be many others…
We usually run these separately in multiple terminal windows, run one script that forks the processes, etc.
One popular way to tame this jumbled mess of processes is to use a Procfile — a format to specify types of processes an application provides and commands to run those services. It is a standard for Heroku and other Platforms-as-a-Service.
A simple Procfile defining a web process that runs a rails server and a worker process that runs sidekiq.
web: rails s
worker: bundle exec sidekiqWe have a bunch of tools to run a Procfile, the most famous being foreman by David Dollar but I’d like to show you how we can take Procfiles to the next level with Overmind.
What is Overmind?
Frustrated with existing Procfile-based process management tools, Sergey Alexandrovich set out to make a tool that solved his problem. As explained on the Github project:
The problem with most of those tools is that processes you want to manage start to think they are logging their output into a file, and that can lead to all sorts of problems: severe lagging, losing or breaking colored output. Tools can also add vanity information (unneeded timestamps in logs).
He solved these problems with the project Hivemind. However, that wasn’t enough, he wanted processes to be bundled together in their output but should still be accessible directly by the developer. He went on to integrate tmux with Hivemind to create Overmind.
So what can Overmind do that other Procfile managers cannot? Here is a comprehensive list from the Github repository:
- It starts processes in a tmux session (we’ll get into what tmux does later) so you can easily connect to any process and gain control over it. This is especially useful because it allows us to connect to debuggers;
- It can restart a single process on the fly — you don’t need to restart the whole stack;
- It allows a specified process to die without interrupting all of the other ones;
- It uses
ptyto capture process output — so it won't be clipped, delayed, and it won't break colored output; - It can read environment variables from a file and use them as parameters so that you can configure Overmind behavior globally and/or per directory.
How to setup Overmind
Note: At the moment, Overmind supports Linux, *BSD, and macOS only.
Now that you’ve heard about what Overmind can do for you, let’s get into setting it up.
Overmind requires tmux, so you need to install it first:
# on macOS (with homebrew)
$ brew install tmux# on Ubuntu
$ apt-get install tmux
If you’re on macOS, it’s relatively easy to install Overmind using Homebrew:
$ brew install overmindOtherwise you can download the latest binary or build from source.
Once you’re done you can get it started with:
$ overmind start -f path/to/your/Procfiletmux
The secret ingredient in Overmind is tmux, a terminal multiplexer. It will let you switch between several programs in one terminal, detach them, while they are running in the background, and reattach them to a different terminal. You don’t need to know tmux to use Overmind but a little knowledge can help.
A few useful commands:
Ctrl + b d — To detach from the tmux window once you get inside with overmind connect
Ctrl + b 0 — The different processes will be assigned different numbers in the tmux window, which you can use to navigate between processes.
Note: My tmux looks different because I’ve customised it. You can customise your tmux by editing your
.tmux.conf. tmux by gpakosz is a good place to start.
Debugging made easy again
Usually when we’re debugging an issue we’d want to use pry or byebug to do some live debugging. However with most Procfile managers or other methods of running processes simultaneously, we aren’t able to access the debugging console for the process. Overmind on the other hand allows us to connect to any of the running processes in a new window using the overmind connect <process name> command. In the below picture you can see that the debugger started on the web process and I was able to use overmind connect web to connect into that process and start debugging.
Restarting quickly
Often there are times we want to restart one of the processes in our Procfile, perhaps the sidekiq worker, and to do so we have to close foreman and all our processes. However in Overmind, we can useovermind restart <process name> to restart just that process.
Procfiles for everything
Generally we don’t have just one stack of processes we need to run. You might a separate set of processes to run your Slack bot or to boot up your acceptance test server. Since Overmind runs on Procfiles we can create different sets for our use cases like a Procfile.slack to boot up the process that listens for events and ngrok, or a Procfile.acceptance to boot up your acceptance server and run the database setup.
# Procfile.slack
listener: bundle exec rake slack:listen_for_events
localtunnel: ngrok http 3000# Procfile.dev
web: rails s
worker: bundle exec sidekiq# Procfile.acceptance
client: rake acceptance:client
server: rake acceptance:server
If you want to run multiple Procfiles simultaneously from the same path, you’ll run into this error:
overmind: it looks like Overmind is already running. If it’s not, remove path/to/project/.overmind.sock and try again
You can fix this by specifying the socket for your processes overmind start -f Procfile.dev -s ./dev.sock. Once you start Overmind with a socket you need to run all your commands with the same socket.
$ overmind connect -s path/to/socket web
$ overmind restart -s path/to/socket sidekiq
$ overmind kill -s path/to/socketThanks for reading! If you have any suggestions or anything else you want to share with us, feel free to reach out by writing a response below.
