Version management with Homebrew and asdf

The Rekki team at work

Hi, I’m Dennis and work in the Rekki engineering team. Every day we make sure that thousands of restaurants, bars, and cafés get their supplies, and that staff in those places spend less time dealing with stuff that doesn’t keep their users (you and me) happy.

At Rekki we try and act in a similar way. Minimise our development time that’s spent on things that do not make our users happy.

I’ll kick off the engineering team’s input to this blog by talking about a small but annoying problem we recently solved, or at least made a lot better.

“But it works on my machine!”

When a team of people are working together and use a set of common tools, issues arise if they are using different versions of those tools.

This problem is amplified when the product runs in multiple environments. For example, every developer has their own local environment, and there could be testing environments, staging environments, and live environments.

At Rekki, we would have issues where we ran different versions of Elixir and those differences broke the CI build a few times even though it worked fine on some of our computers. We also had issues with database migrations that worked fine on one development environment and not the other.

In short, you have a higher risk of shipping software with defects and you lose time troubleshooting issues that are caused by version differences. We can reduce these risk by having environments that are as consistent as possible — no more “It works on my machine” situations.

Potential solutions

Okay, there are benefits to using the same versions of software, but how does this work in a team of software engineers?¹

A common approach is to use Docker in development. I’m not a fan of this approach. I think Docker works great for infrastructure, but not as a development environment. The feedback loop can be too slow, as you’ll spend a lot of time managing the images and containers. I won’t cover this approach because I don’t advocate it.²

Another approach is to use specific version managers for every tool that has them available. For example, you may use rbenv for Ruby, kiex for Elixir, nvm for Node, pyenv for Python, and so on. This is time consuming. For every tool you have to review which version manager best suits your needs (and these choices may be reviewed every few years as tools change), you have to learn how to use every version manager as they all work differently, and you have to individually set them all up.

A simpler approach, which is good enough in my experience, is to use a combination of asdf and Homebrew³.

Homebrew is a popular package manager for macOS, and asdf is an extendable version manager with support for many tools. A caveat is that Homebrew is not a version manager, and the maintainers make this clear, but there are ways to have it behave like one (sort of).

Some common tools in the development environment of a Rekki backend engineer are Elixir and PostgreSQL. We’ll use those as an example here.

Managing tools with brew

Let’s start by installing Homebrew. Run the following in a terminal

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install common dependencies via brew

brew install coreutils automake autoconf openssl libyaml readline libxslt libtool unixodbc

Install PostgreSQL 10 via brew⁴ ⁵ (installs the server and the psql client)

brew install postgresql@10

Pin the PostgreSQL version⁶ via brew

brew pin postgresql@10

Start the PostgreSQL service⁷ ⁸

brew services start postgresql@10

Managing tools with asdf

Install asdf via brew⁹

brew install asdf

Add the Elixir plugin (each supported language uses its own plugin)

asdf plugin-add elixir

Update all the plugins (optional)

asdf plugin-update --all

Install Elixir 1.6.2 compiled against OTP 20

asdf install elixir 1.6.2-otp-20

Set a global Elixir version (optional)

asdf global elixir 1.6.2-otp-20

Project specific versions

You’ll probably work with different tool versions across different projects.

If you’re using different versions of a brew-managed service, you’ll need to stop one and start the other as they run globally. Here we switch from PostgreSQL 10 (and whatever minor version it’s pinned at) to 9.5.

↪ brew services list
Name           Status
postgresql     started
postgresql@9.5 stopped
↪ brew services stop postgresql@10
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql`
↪ brew services start postgresql@9.5
==> Successfully started `postgresql@9.5`

↪ brew services list
Name           Status
postgresql     stopped
postgresql@9.5 started

With asdf it’s even simpler. Add a .tool-versions file to your project root directory. For example, one project’s .tool-versions file could look like

↪ cat .tool-versions
elixir 1.6.2-otp-20
erlang 20.3.8.8

To install the versions listed in the file, run asdf install from the directory. You may need to install the necessary plugins first, asdf plugin-add erlang.

Hope that helps. Leave your thoughts and suggestions as a comment below!

Notes

1. We’re going to focus on a team that uses macOS as their operating system and a stack that’s built on open-source tooling. The tooling we cover is also supported by some Linux distributions, but this guide won’t focus on that.
2. There are lots of posts online about why you should or shouldn’t use Docker as a development environment. Such an environment, for example, could exist as having a Dockerfile for each of your services, using Docker Compose to orchestrate multiple services in their respective containers, having some mounted volumes to avoid rebuilding images whenever a code change is made, and a Docker Repository to push/pull images to/from. You’ll probably end up building some extra tooling around it, like a wrapper to make certain commands or clean up easier, some scripts to ensure services are ready to be used, and so on.
3. Linux users can try Linuxbrew, a fork of Homebrew.
4. You can also install a lower version if Homebrew has the formula available. For example brew install postgresql@9.6.
5. PostgreSQL can also be installed with asdf, which gives you more control over its version. But you won’t be able to manage it with brew services.
6. Pinning helps keep the version locked, unless explicitly requested to upgrade that formula. You can safely perform a brew upgrade --cleanup for example without worrying about accidentally changing the PG version.
7. This will start the service at login. Run brew services to see its help.
8. You may also need to create a postgres superuser for some databases createuser --superuser postgres
9. asdf can also be installed without brew, in case you’re experiencing issues with the brew installation.

—

Want to know more us? Go here.