Introducing Mamba 2.0
A Major Leap in Package Management
We are excited to present the first release candidate of Mamba 2.0, a significant upgrade to the mamba package manager. This update brings considerable enhancements for both users and developers, following an extensive year-long development effort.
🥚The origins of Mamba
The Mamba project initially started as a modified version of conda, merely substituting the default solver of conda with the libsolv SAT solver. At the time, the conda project lacked the ability to use a different solver from the default one, which led to the development of mamba as an independent application combining the new solver with parts of the conda code base, rather than simply existing as a conda plugin.
As the project expanded, a larger portion of the conda features were included in the mamba codebase, which led to the release of micromamba, a complete C++ implementation of the conda command-line utility. Micromamba’s main advantage over conda lies in its statically linked executable, enabling standalone use with only a 5Mb download, a stark contrast to the typical 100Mb download required for miniconda or miniforge installers. This makes it particularly interesting in the context of CI workflows. Another benefit was the improved error messages in case of conflicts compared to the original conda solver. Eventually, after a few years of coexistence, the conda team introduced a solver plugin system in conda, and switched to using mamba as the default solver for dependency resolution.
Unfortunately, Mamba 1.x faced challenges due to technical debt that accumulated over time. Initially designed as a command-line utility, Mamba’s usage shifted towards being a library, but it lacked a consistent and composable API. Furthermore, the internal workings of libsolv were tightly integrated into the Mamba codebase. Lastly, Mamba found itself managing two implementations of conda functionalities: the one in conda and the expanding C++ reimplementation. These factors combined created a complex situation that necessitated the development of Mamba 2.0.
🐍What is new in Mamba 2.0?
The first major decision with Mamba 2.0 was to completely remove the Python bits of the library. Mamba 2.0 uses the same code as micromamba and is completely standalone and independent of the conda codebase.
Inside mamba, a lot more work has been done than what is visible in the public interface. In mamba 1.0, code was highly coupled to the rest of the library. Attempts to rewrite components from scratch proved challenging to integrate back into the library. Our strategy ended up following the following cycle: simplify, decouple, split, test and repeat. Once a component is finally independent from the rest of the code base, it can be moved to its own space, tested, and exposed publicly. We started from simple components such as utility functions, and moved up to Conda object specifications, up until we reached the big components such as the Solver.
Mamba 2.0 is the outcome of this bottom-up refactor, focusing on the two most critical components: the dependency solver and the downloaders. The dependency solver is responsible for finding compatible versions of the required packages and their dependencies, while the downloaders handle package downloads over various protocols and configurations.
Regarding the dependency solver, we now enjoy a fully independent functional solver interface. This results in an improved interface for users, and greatly improved velocity for developers. Also, previously, Mamba relied on libsolv for much more than solving dependencies. This included relying on it for parsing the repository metadata (repodata.json), and parsing the dependency string format (to make sense of strings like python≥3.10=*_cpython). However, delegating these aspects to libsolv made it harder to maintain and slow to evolve. For this reason, Mamba 2.0 provides its own implementation of these features. The solver interface within Mamba has also been redesigned, using a transactional pattern and decoupling it from the rest of the library. For users, this means improved stability in both Mamba and Conda, as well as better support for complex dependency specifications such as numpy[md5=d8285bea2a350f63fab23bf460221f3f].
Historically, Conda packages have been freely hosted on anaconda.org, a service operated by Anaconda Inc, the company responsible for the creation of conda. This means that even though Conda and Mamba are open-source packages, the ecosystem remained reliant on a single actor for hosting packages. Mamba 2.0 addresses this situation by introducing support for mirrors in the package manager, allowing for the package repository to be hosted by multiple parties. We also added support for hosting packages on OCI registries enabling support for new hosting providers such as GitHub Container registry. Package mirrors can be configured in the .condarc. For example, for using prefix.dev as a secondary hosting provider for conda-forge, you can enter:
mirrored_channels:
conda-forge:
- https://conda.anaconda.org/conda-forge
- https://prefix.dev/conda-forgeFinally, Mamba 2.0 comprises a solid implementation of The Update Framework (TUF) for package signing and general software supply chain security, and an implementation of the conda content trust protocol based upon TUF.
We believe that mamba 2.0 provides a much more solid foundation for building new functionalities in the Conda ecosystem. In the coming weeks, we will be testing this release candidate with various downstream packages to ensure that it can be released safely.
🔬How to try the release candidate?
The mamba 2.0 pre-releases are available on conda-forge, and can be installed with the following command:
mamba install mamba -c conda-forge/label/mamba_devMicromamba is the standalone executable that can be used to create environments and install packages. Micromamba 2.0 rc0 can be installed the following way:
VERSION=2.0.0rc0-1 "${SHELL}" <(curl -L micro.mamba.pm/install.sh)Acknowledgements
The work by Antoine Prouvost, Johan Mabille, Joël Lamotte, and Hind Montassif on Mamba at QuantStack was funded by several organizations in the past few years:
- Bloomberg funded work towards the implementation of TUF and the general refactor of the mamba codebase to mamba 2.0.
- The Chan Zuckerberg Initiative has funded work on Mamba through the EOSS 5 grant program, which included work toward supporting hosting conda packages on OCI registries, included in this release.
We are grateful to Wolf Vollprecht, the original creator of Mamba, for advising on the development of Mamba 2.0. Currently, Wolf is leading the charge in the development of an innovative new stack centered around the conda-forge and mamba ecosystem at Prefix.dev. His work has infused a fresh wave of creativity and progress into our community.
We want to thank Jonas Haag for supporting the Mamba project through his numerous contributions.
We extend our heartfelt gratitude to Jaime Rodríguez-Guerra Pedregal for his instrumental role in integrating the mamba solver into conda. His invaluable feedback and precise bug reports on mamba’s pre-releases have significantly contributed to the development of this release candidate. Furthermore, Jaime’s dedicated efforts in coordinating QuanStack and Quansight’s work on the Chan Zuckerberg Initiative grant for conda-forge are truly commendable.
About the authors
Antoine Prouvost is a senior scientific software engineer, and the lead contributor to the mamba 2.0 refactor. Prior to his time at QuantStack, Antoine was a researcher in Machine Learning and Combinatorial Optimization at the DS4DM research chair and Mila. Currently, he is working on an exciting new venture that will be unveiled to the public soon.
Joël Lamotte is a senior developer at QuantStack with main experience in embedded or high-performance domains usually involving C++. He is also a game developer, music composer and comics maker.
Johan Mabille is a technical director at QuantStack specialized in high-performance computing in C++. He is a core developer of Project Jupyter, and co-authored the xtensor stack.
Hind Montassif is a senior software developer at QuantStack. She is a maintainer of the mamba package manager and contributes to the Jupyter project, notably in the area of language kernels.
