Colima v0.5.0 increased my productivity

Ever since the product subscription change introduced by docker in January 2022 (see https://www.docker.com/blog/updating-product-subscriptions/) my colleagues and I have been looking at alternatives for running containers on macOS.

After some investigation we came across colima which seemed to handle the majority of our use cases without too much change or effort.

One issue we constantly ran up against was the performance of colima on either x86 or arm (M1) macs. With the use of qemu, mounting and manipulating files from the host machines within the container was slow and encouraged many to install npm and similar tools locally as to not be hampered by the poor filesystem performance. It is hard to recommend a tool and process where engineers feel they are sitting around waiting for a task to complete, especially when an alternative solution is much faster and well known (locally installed software).

For me, I prefer to run as much as I can within a container to ensure my software is isolated and consistent even with the performance downside. I find it much easier to create and recreate a container opposed to installing software on my host and hoping it doesn’t interfere with what is already installed. I also prefer the repeatability that comes with using containers.

I was pleasantly surprised a few days ago to see the new version of colima had been released (as I had seen some useful changes in the main branch but no new release). v0.5.0 release headlined with the following benefits.

This release adds the following for devices running macOS 13.0 or newer.

- Support for native macOS Virtualization.Framework

- Support for faster virtiofs volume mounts

- Support for faster x86_64 virtualization on M1 devices via Rosetta2

This works in combination with the latest lima v0.14.0 release and fixes in v0.14.1.

This was timely as a colleague had just been in conversations with other software engineers as to why building their application with colima / containers on their macs was getting in the way of productivity.

To verify if the latest changes would make any different I updated both my lima and colima to the latest versions (using brew):

❯ limactl --version
limactl version 0.14.1

❯ colima version   
colima version 0.5.0
git commit: 5a94ab4f098ec0fe94e6d0df8b411fb149fe26fe

runtime: docker
arch: aarch64
client: v20.10.22
server: v20.10.20

I needed to delete my existing colima instance to change the vm type. I used the --edit during the initial start to ensure I had the vmType: vzand mountType: virtiofs.

NOTE: Deletion is a destructive event and removes all of the containers stored in your existing instance. If you want to keep existing containers, start colima with the added --profile <unique_name> to create a unique isolated instance.

colima delete
colima start --edit

You verify colima is running with the correct settings via:

❯ colima status
INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] socket: unix:///Users/user/.colima/default/docker.sock

My project is a hugo static website with approximately 21 npm dependencies and generates ~680 pages and ~50 static files.

I ran my tests against a few different variations and with both a clean (empty) node_modules folder and then again against the existing. My test machine is a MacBook Pro 16-inch, 2021, Apple M1 Pro with 16 GB running macOS Ventura 13.0.1 (22A400). My colima has cpu: 2 (cores) disk:60 (GiB) and memory:2 (GiB).

It is always best to start with the original configuration using qemu + sshfs to remove any bias and anecdotal “I think its faster”:

Fresh:
  Total time (npm install and hugo build): 6:53.82 minutes
  Hugo build: 91.01 seconds
Existing: 
  Total time (npm install and hugo build): 2:19.01 minutes
  Hugo build: 92.59 seconds

The npm install is much faster without the need to download all of the files (when using the cache) but hugo still takes a long time to get started. These times are not ideal but I had grow accustomed to this (for better or worse).

Next was to try with the virtiofs volume mount instead of sshfs (still using qemu):

Fresh:
  Total time (npm install and hugo build): 12:01.16 minutes
  Hugo build: 98.78 seconds
Existing: 
  Total time (npm install and hugo build): 2:28.40 minutes
  Hugo build: 96.79 seconds

Here with no existing node_modules directory we are waiting more then 10 minutes to download and install dependencies, not great for productivity and a regression on the previous setup. I wouldn’t be able to recommend this to anybody! Maybe this was just my setup so hard to tell why it performed so much worse.

Now for the big test using virtiofs and the newly supported Virtualization.framework for macOS 13 Ventura. Unsurprisingly and thankfully we see the build times tumbling:

Fresh:
  Total time (npm install and hugo build): 47.549 seconds
  Hugo build: 13.07 seconds
Existing: 
  Total time (npm install and hugo build): 28.756 seconds
  Hugo build: 13.19 seconds

It now only takes 6% of time time to download and build and 20% of the time to rebuild just the hugo site. Not to mention my CPU when running qemu was averaging ~300% in the activity monitor and is now ~100%.

These savings are a game changer for anyone using npm and likely any other tool that is heavy on filesystem changes. I haven’t had a chance to look at other tools and scenarios but I assume this trend will continue, even if to a lesser extent. Saving over a minute for every full rebuild of the hugo site makes using colima and mounted volumes with npm a much easier solution to recommend and a good experience overall.

Likely this won’t change the mindset or workflow of engineers that have all of their tools locally or are not adept in the world of containers but it makes for a great conversation starter and greatly reduces one more barrier for adoption. For those of us who are already container first its going to remove a lot of idle time.

I should note I did run into some stability issues (in general) and when using vz + sshfs I needed to implement the suggestions in https://github.com/abiosoft/colima/issues/515 to mount my files in the container. This will likely be fixed in the next releases with https://github.com/abiosoft/colima/pull/521 already being merged.

Remember, Lima and colima are still not v1 / stable so issues are to be expected by looking at the activity in both the repository and responses to issues I don’t feel like support or interest for either projects is an issue. I feel confident now recommending the use of colima and containers for the majority of local use cases.