New macOS task execution architecture for Cirrus CI

Fedor Korotkov
Jan 26 · 3 min read

We are happy to announce that the macOS tasks on Cirrus CI Cloud have switched to a new virtualization technology as well as overall architecture of the orchestration. This switch should be unnoticeable for the end users except that the tasks should become much faster since now each macos_instance of the Cirrus CI Cloud offering will utilize a full Mac Mini with 12 virtual CPUs and 24G of RAM.

The new architecture is built on top of the recently announced Persistent Workers functionality and can be easily replicated with on-premise Mac hardware by any Cirrus CI user or on any other CI by using Cirrus CLI.

We know from experience that continuous integration for macOS is the hardest and how little information there is about the topic on the internet! And we want to share below how the new simplified architecture looks like and how to replicate it.

Cirrus CI architecture is very simple. There is Cirrus Agent (a self contained binary written in Go) which job is to simply execute scripts, download/upload caches, parse test reports and stream progress via gRPC API. Both Cirrus CI Cloud and Cirrus CLI implement the same gRPC API so the agent binary doesn’t even know in which environment it’s been executed.

Image for post
Image for post
Image for post
Image for post
Schema of executing Cirrus Tasks in the cloud and locally via Cirrus CLI

Cirrus CLI initially was intended to be a local executor of Cirrus Tasks in Docker containers only. Cirrus CLI simply parses Cirrus configuration file and then uses Docker daemon API to start/stop containers to execute parsed tasks. Note that Cirrus CLI doesn’t require to use Cirrus CI Cloud and can be used with any other CI. Once this functionality was ironed out and well tested it was easy to add an option to use Parallels virtualization instead of Docker containers to execute tasks in.

Before that Cirrus used Anka cloud which required a complex setup of Controller/Registry services that were orchestrating execution of Anka VMs on the hosts.

Image for post
Image for post
Old architecture with Anka controller responsible for scheduling

With Persistent Workers we were able not only to dogfood Cirrus CI’s own functionality but also cut the middle man of Anka Controller which was contributing to the “created to execution” metric of macOS tasks. Now macOS tasks will be scheduled even faster! Here is how simple the current architecture look like:

Image for post
Image for post
New architecture that leverages persistent workers

As you can see this new architecture is not rocket science and somewhat very traditional. The key here is that Cirrus CLI can isolate task execution in a Parallels VM. Under the hood the following configuration

Will be translated to:

This configuration can be easily executed locally or in any other CI via Cirrus CLI.

We are very excited about the new architecture and opportunity to dogfood persistent workers functionality at scale! Please let us know how new architecture works for your projects (especially since there are 3x more CPU resources and better network performance) and send us feedback either on GitHub or on Twitter!

CirrusLabs

Making Developers Happy

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store