QuantStack: 2023 in review
Since QuantStack was founded in 2016, we have remained committed to releasing our work under permissive open-source licenses.
Most notably, the QuantStack team comprises some of the most active developers of the Jupyter ecosystem and has consistently been the foremost and most unwavering supporter of the project in recent years, both from a technical and organizational perspective.
Here are some highlights of the 2023 achievements. Buckle up!
JupyterLab 4 and Notebook 7
The main event of the year has been the coordinated releases of both JupyterLab Version 4 and Notebook Version 7.
JupyterLab 4 marks a major milestone in the development of the project, with improved performance, a consolidated collaborative editing experience, and a new extension manager. The QuantStack team is responsible for some of the main developments in this release and for the lion’s share of the day-to-day maintenance of the JupyterLab project this year.
Jupyter Notebook 7 is the most significant release of the Jupyter Notebook in years. Some highlights of this release include real-time collaboration, interactive debugging, table of contents, theming and dark mode, internationalization, improved accessibility, and compact view on mobile devices. Unlike earlier versions, Notebook 7 is based on JupyterLab’s internals, consolidating both Jupyter UIs under a single code base, while retaining the familiar document-oriented user experience.
The release of Notebook v7 marks the culmination of 18 months of dedicated work by the QuantStack team, following the acceptance of the Jupyter Enhancement Proposal to adopt a JupyterLab-based UI as the foundation for the next iteration of the Jupyter notebook.
Beyond the core packages themselves, the impact of JupyterLab 4 and Notebook 7 reverberates across the broader Jupyter ecosystem. Shortly after the release of JupyterLab 4, Voilà 0.5 emerged as a significant iteration of the Jupyter dashboarding system. Much like Notebook v7, Voilà 0.5 leverages the JupyterLab application framework for its frontend, leaving behind the classic notebook’s extension system used previously. QuantStack also played a crucial role in upgrading extensions of the Jupyter ecosystem to align with the latest version of JupyterLab: nbgrader, ipywidgets, rise, ipyleaflet, and many more. This concerted effort underscores our commitment to enhancing the overall functionality and interoperability of the Jupyter ecosystem.
Improving Jupyter’s Accessibility
Jupyter has gained a significant number of users over the past few years, especially in educational contexts. The project has become a key component of their digital experience, from the first lines of code written by high-school students to the most sophisticated engineering applications. In this context of global adoption, it is essential to ensure that as many people as possible can use the project.
A primary obstacle to the accessibility of Jupyter was that the text editor underlying the Jupyter Notebook and JupyterLab had major accessibility flaws. Fortunately, those issues have been resolved as QuantStack upgraded JupyterLab to use CodeMirror 6, a complete rewrite with a strong focus on accessibility. While automated audits previously identified hundreds of accessibility issues in the classic Notebook 6 user interface, only a few dozen issues were detected in the Notebook 7 development version, all of which were resolved before the release. This was a first but crucial step towards accessibility of Jupyter. For more details, you can check out our blog post on the accessibility of Notebook 7.
Shortly after the release of JupyterLab 4, QuantStack published a prototype theme editor for JupyterLab 4 and the Notebook 7, enabling the creation and fine-tuning of color themes for the user interface. The ability to fine-tune contrast, color palettes, and fonts can be very valuable for accessibility, an essential requirement for software to be usable by the whole community. Subtle differences can impact comfort significantly.
This prototype helped us discover and fix bugs in how colors were used in the UI before the release of JupyterLab 4. In 2024, we will iterate on the theme editor to improve the current user interface and how color palettes are mapped to JupyterLab colors.
In another iteration on accessibility and usability, QuantStack teamed up with the Jupyter Accessibility Working Group to improve the keyboard navigation of the Jupyter Notebook and JupyterLab UI. The general logic of the keyboard navigation in JupyterLab was reworked and the identified “tab traps” were corrected. For more details about this work, you can consult our joint blog article about the improvements to keyboard navigation.
We are committed to continue the enhancement of accessibility features and have several exciting developments on the 2024 roadmap. Firstly, we will focus on improving keyboard navigability within both the JupyterLab settings editor and the notebook metadata editor. Secondly, we aim to create tools that help users identify accessibility issues in the content of their notebooks. Additionally, we will continue our efforts to standardize JupyterLab widgets using the FAST Web Components library to ensure a consistent user experience throughout the application.
JupyterCon
In addition to our technical contributions, we actively engage in fostering community support for Jupyter.
QuantStack played a pivotal role in orchestrating the JupyterCon 2023 conference, showcasing our commitment by dedicating volunteer time to the organizational efforts. Notably, Sylvain Corlay served as the general chair. Furthermore, we assumed the role of platinum sponsor, emphasizing our financial support for the event.
This significant milestone marked the inaugural JupyterCon installment in Europe, hosted at the Cité des Sciences in Paris from May 10 to May 12, 2023, with an exceptional lineup of keynote speakers and participants, including Nobel Laureate Paul Romer from NYU, Professor Alyssa Goodman from Harvard University, as well as Cory Gwin and Craig Peters from GitHub.
We are proud to have been instrumental in facilitating this landmark conference and remain deeply invested in nurturing the vibrant Jupyter community. This is the second time that we are involved in the organization of JupyterCon. For the 2020 edition, we provided both organizational and technical support for the conference, which had to pivot to an online platform due to the pandemic.
In future years, we will strive to consistently support the Jupyter community and actively contribute to its long-term sustainability.
Beyond JupyterCon, QuantStack also supported the organization of PackagingCon, which took place in Berlin, as a Gold Sponsor. We also helped with the inauguration of the conference in 2020, both as sponsors and co-organizers.
JupyterCAD
In 2023, QuantStack introduced the first version of JupyterCAD, a tool integrating Computer-Aided Design (CAD) capabilities into JupyterLab.
JupyterCAD marks a significant milestone as QuantStack expands its horizons beyond the realms of data science and scientific computing, and steps into the domain of industrial engineering. It does not only mark a milestone for QuantStack but also for the JupyterLab project, which has proved to be a solid foundation for the creation of sophisticated desktop-like web applications.
A standout feature of JupyterCAD is its provision of collaborative editing functionalities for CAD modeling, setting it apart as a truly innovative solution in this domain.
Solutions providing collaborative editing features have become a ubiquitous part of our digital lives — even more so as remote work has become the norm. Collaborative editing has deeply transformed collaboration with word processing software.
We believe that real-time collaboration tools will be even more transformative for complex system design. Whether one is building a football stadium or a jet engine, the integrated model comprising all of the information about the system is a collective creation that could hugely benefit from web-based collaborative user interfaces. JupyterCAD demonstrates that the foundations of collaborative editing built for the JupyterLab IDE can work for non-textual content.
In 2024, we will continue the development of JupyterCAD by adding support for more CAD file formats, and adding features for better collaborative editing such as the ability to visualize the history of changes in a collaborative document. Finally, we are dedicated to addressing key missing elements, ensuring seamless editing capabilities for any FreeCAD file.
JupyterLite
JupyterLite is a JupyterLab distribution that runs entirely in the web browser, including language kernels. Programming languages such as Python and Lua, which are not natively supported by browsers, are enabled by using WebAssembly builds of the corresponding interpreters.
JupyterLite’s journey commenced in 2021 as a proof-of-concept for a grant application submitted to the French government by QuantStack. Although the grant was not secured, QuantStack remained committed to advancing JupyterLite’s development. The vast majority of the development work on JupyterLite was spearheaded by the QuantStack team, with valuable community contributions enhancing the project.
JupyterLite has enjoyed explosive growth since 2021. Major open-source projects have adopted JupyterLite to power interactive embedded consoles and notebooks on their websites and documentation, including NumPy, Scikit-Learn, SymPy, and the main Jupyter website, serving millions of monthly visitors with readily available computing environments.
The scalability of JupyterLite deployments hinges on not needing individual containers for user sessions. Everything required to run the platform can be statically served alongside HTML and JavaScript assets. This opens opportunities for very large-scale deployments where the conventional “one container per user session” model falters. JupyterLite stands out as a compelling solution for education platforms because of its capacity to facilitate coding education for millions of people without requiring a hyperscale cloud provider.
In 2023, we released JupyterLite 0.2, which is based on JupyterLab 4.0 and Notebook 7, and improved the development workflow of custom JupyterLite kernels. In 2024, JupyterLite will be a key priority for QuantStack with more resources being allocated to the project. We believe that the next 10 million Jupyter users will be JupyterLite users.
In 2024, JupyterLite will be at the center of our attention. We will put a strong focus on the software packaging story for the WebAssembly platform, specifically the emscripten-forge software distribution, which is based on mamba and conda-forge’s model. We are working on bringing more programming languages to JupyterLite, such as R and C++.
Advancing the Jupyter project
2023 was a pivotal year for the Jupyter project. It is the first year in which Jupyter operated under its new governance model, which has been progressively developed and implemented in the previous months.
A tangible outcome of this transformative shift is the release of Notebook v7, which is the materialization of the first major outcome from the new decision-making framework. It is a testament to Jupyter’s capacity to make bold, forward-thinking decisions concerning its flagship project.
The involvement of QuantStack team members in various Jupyter subproject councils and other governance bodies underscores our commitment to the project. Throughout the year, the Jupyter Software Steering Council successfully ratified numerous Jupyter Enhancement Proposals, resolving a longstanding backlog and streamlining the processing of proposals that had previously been pending. This proactive approach highlights Jupyter’s dedication to efficient governance and ongoing improvement within its collaborative ecosystem.
Mamba 2.0
Mamba is a reimplementation of the conda package manager in C++, which was started by QuantStack in 2020 with the primary goal of enhancing conda’s performance. The imperative need for such improvements stemmed from the challenges faced by the conda-forge distribution, where existing conda performance proved to be a bottleneck for end users. Mamba now underlies many key projects of our ecosystem, including conda-forge and MyBinder (through repo2docker). Mamba’s solver was also recently adopted by conda itself for resolving the package dependencies so that all users of conda can now benefit from mamba’s superior performance.
One of the main focuses of 2023 for the QuantStack team was the development of Mamba 2.0 for which we published the first pre-release in December. Mamba 2.0 is a major refactor of the mamba codebase. The goal of the refactoring was to make mamba more modular and usable as a framework for package management rather than just a command-line utility. It will make integrations like the one done by conda more streamlined and will facilitate the development of web services relying on mamba in the backend.
Mamba 2.0 will be the first version of mamba to support the downloading of packages from OCI registries, and to natively support the verification of package signatures with the conda-content-trust protocol, ensuring better software supply chain security.
Xeus kernels
The Xeus C++ library is an implementation of the Jupyter kernel protocol, forming the backbone for various Jupyter language kernels like Xeus-Cling (C++), Xeus-Robot (Robot Framework), Xeus-Lua (Lua), and Xeus-Python (Python).
2023 marked a significant chapter for the Xeus project, witnessing notable developments and expansions.
- We published the Xeus-Octave kernel for the GNU Octave programming language. Initially crafted by Giulio Girardi and subsequently enhanced by Antoine Prouvost, Xeus-Octave’s features are extensively detailed in this blog article.
- Our team embarked on the creation of the Xeus-R kernel, an upcoming addition specifically designed for the R programming language. Stay tuned for upcoming updates on the Xeus-R kernel!
- We are also working on a reboot of the Xeus-Cling C++ kernel called Xeus-cpp, in collaboration with a team from CERN. Xeus-cpp will be based on the newly developed “clang-repl” project which upstreams the main features of Cling into the core Clang compiler.
Beyond the new language kernels, Xeus stands out as a lean implementation of the kernel protocol, with a very reduced API surface compared to ipykernel. This approach enhances the overall clarity of the codebase, making it a highly focused and efficient solution, turning Xeus into an excellent sandbox for experimenting with new kernel features.
Xeus has two main extension points: the “xinterpreter” and the “xserver.”
- The “xinterpreter” is the base class that one has to specialize to create a new language kernel.
Kernel authors don’t need to know low-level details of the protocol such as how messages are signed or how connections are established, they just need to develop the language-specific bits in their implementation of xinterpreter.
Relying on a framework for a language kernel rather than re-implementing the protocol offers kernel authors a great way to keep their kernel up to date with the latest changes to the kernel protocol. - The “xserver” is the middleware layer of the kernel and is responsible for sending messages over the communication channels.
The ability to override the xserver implementation is unique to Xeus and opens many possibilities. You can check out our recent article about how to embed Jupyter kernels into Desktop applications with a custom xserver implementation that polls communication channels with the Qt event loop.
Overriding the xserver implementation is also how we used Xeus to make kernels compatible with the JupyterLite system, replacing the ZeroMQ-based communication with direct call to JavaScript using Emscripten’s Embind foreign function interface.
XSimd
The XSimd project provides a unified C++ API for performing SIMD operations, and supporting a large number of SIMD instruction sets, from x86 to ARM.
In 2023, we saw significant projects adopt XSimd, with the notable case of Firefox, following other large open-source projects like Apache Arrow, Krita, and Pythran.
Besides, two new instruction sets have been added to the list, with support for RISC-V and WebAssembly. Notably, the addition of RISC-V was done by a community member.
Joining QuantStack
QuantStack will soon be advertising for open positions. We are looking for talented people to help us tackle the next challenges in real-time collaboration, accessibility, integration of LLMs into the Jupyter UI, and much more.
If you are looking to make open-source development your full-time job, or if you want to write tools for science, discovery, education, or advanced engineering, QuantStack may be the place for you. Our team is distributed across four European countries and we allow for full-remote positions.
You can check out our profile on Welcome to the Jungle for more information on careers at QuantStack.
Past QuantStack review blog posts
These year-end review blog posts are becoming a QuantStack tradition. You can check out earlier publications here:
- 2022 https://medium.com/@QuantStack/quantstack-2022-in-review-7d4704b1db42
- 2021 https://medium.com/@QuantStack/quantstack-2021-in-review-fe5eac2e0f6d
