INTRODUCING SINGULARITY DESKTOP: MOBILE AND REPRODUCIBLE LINUX CONTAINERS FOR COMPUTE-DRIVEN WORKLOADS ON MACOS

TL;DR: Singularity Desktop is available now for macOS; you can download it here, and soon be running Linux containers on your Mac.

A New Product Introduction for macOS

There’s no need to anticipate any longer: the alpha preview of Singularity Desktop is now available for you to download and use. Simply stated, Singularity Desktop from Sylabs now allows users to design, build, test, and sign/verify Linux-based Singularity containers on macOS.

For some, Singularity Desktop will be an end in itself. As the product matures towards a generally available release, it’ll meet their requirements as container prosumers — both consumers and providers of Linux containers on macOS.

For others, however, macOS support is more important from a prototyping perspective, as their laptop use comprises a critical preliminary step to executing their containerized applications and workflows at scale on enterprise-grade platforms. In seamlessly transitioning between macOS-based laptops and Linux-based clusters and supercomputers, in their DevOps process of improvement that transitions iteratively from prototype to production, these users are providing a compelling demonstration of a core competence of containerization — namely, the ability to introduce runtime mobility for applications and workflows. Because the container with a consistent encapsulation of application and workflows is executed on macOS and on clusters and supercomputers, a second core competence of containerization is realized through this product release — namely, the ability to ensure reproducibility irrespective of the underlying platform.

Thus Singularity Desktop now provides those with compute-driven workloads and workflows a solution for containerization that is both mobile and capable of delivering reproducible results consistently on laptops to supercomputers. The gravity of this enabling product introduction is not lost on Singularity community member and HPC veteran Dr. James Cuff who states:

When starting microkernels to test, build and run native Linux applications on non-native host clients, time matters; many systems today are just too slow and cumbersome. I am particularly impressed not only by Sylabs progress with the software itself, but with their community engagement.

Cuff was so taken with the possibilities presented by this alpha preview of Singularity Desktop that he shared some suggestions with the community as to how it might further reduce the already impressive runtime latency in future releases. His rationale is clear:

Having a low latency, interactive and frictionless interface from host systems to native Linux binaries that is both fast and secure dramatically helps the development of local container systems prior to porting to large scale HPC clusters.

When it comes to Singularity Desktop, your macOS platform can now play an active role in all aspects of DevOps; it is no longer merely an attractive spectator to where compute actually happens.

The macOS Implementation

The expectation of macOS support was concretely anticipated at the code level in the version 3.1.0 release , as Singularity’s command line interface clearly and systematically took the Darwin operating environment into account. In fact, with little more than the software development environment provided by the Go programming language , it was possible to build a native port of Singularity for the macOS platform. Though the alpha preview has some inherent limitations, it will ultimately allow users to design, build, test, and sign/verify Linux-based Singularity containers on macOS.

In terms of implementation, Singularity Desktop for macOS (Alpha Preview) is enabled through two components:

• Singularity itself — a native port of the Singularity codebase to macOS that offers a subset of the functionality currently available under Linux
• The macOS compatibility layer — a pre-built package that provides a binary interface between Singularity and macOS

The macOS compatibility layer is comprised of two components:

• SyOS — an extremely pared down, custom implementation of the Linux operating environment from Sylabs that includes a minimal set of Linux kernel functionality and userspace tools to enable execution of Singularity
• The xhyve hypervisor — a userspace hypervisor customized by Sylabs to provide an interface for SyOS with underlying macOS operating environment

In addition to these components, a squashfs utility is included to enable conversion of OCI format images (e.g., from Docker Hub) into the Singularity Image Format (SIF) — the native format for Singularity containers that encapsulates the entire runtime into a single file.

The complete list of open source software used in Singularity Desktop can be found here.

Creating Secure Singularity Containers with the Sylabs Cloud

In addition to serving as an environment in which you can make use of Linux containers on macOS, Singularity Desktop also allows for the creation of containers. Whether your desire is to bootstrap the build process from first principles via definition files, or from a preexisting image available through the Sylabs Cloud Container Library, Singularity Desktop delivers this capability today via the Sylabs Cloud Remote Builder — a cloud-based service offering that allows for remote creation of containers with support for privilege escalation, if necessary.

Once created via the Remote Builder service, the locally available copy residing on macOS-based platforms can be cryptographically signed at the source. Based upon signed SIF files then, containers of known provenance can be shared via the Cloud Library in an immutable fashion, whose authenticity can always be verified upon demand. Thus the Sylabs Keystore is critical in ensuring trust for Singularity containers regardless of the platform on which they are created and/or used.

When it comes to Singularity Desktop, your macOS platform can now play an active role in all aspects of DevOps; it is no longer merely an attractive spectator to where compute actually happens.

Next Steps

Singularity Desktop for macOS (Alpha Preview) allows Linux containers to be designed, built, tested, and signed/verified on macOS. Although the long-term plan is to deliver user experience (UX) parity on macOS when compared to Linux, the current alpha preview implementation is the starting point. Singularity Desktop for macOS (Alpha Preview) is available here. We are delighted to share this exciting and enabling new product with the user, develop, and provider community on the first day of the inaugural meeting of the Singularity User Group (SUG) .

Originally published at https://archive.sylabs.io on March 12, 2019.