Rust 2019 — let us pursue composability

TL;DR: An edition release doesn’t mean that the work on that edition is over. In 2019, we should continue to pursue and drive home the goals set in the spirit of productivity. That being said, I’m going to argue that for the upcoming years and the next edition, we should pursue software composability as a new overarching theme and go over some thoughts on what that means in a concrete way.

On editions and themes and the following year

First of all, let me start with an observation about the themes of the past and the current editions. The overarching theme of Rust 2015 was stability. The work towards stabilising the language started in 2014 and was publicly announced in the blog post Road to Rust 1.0. A huge amount of work was coordinated and performed for the effort that culminated in releasing Rust 1.0 in June, 2015.

However, the release, although a culmination point, didn’t mean that the job was done. For months and even years after the big release, the community continued the effort of polishing and stabilising yet unstable parts of the standard library and the community-maintained libraries release by release. One could say that the Libz Blitz initiative in 2017 was still made in the spirit of stability!

Now, in 2017 roadmap it was announced that productivity was going to be the theme of that year, and in 2018 it was solidified as a the overarching theme of the Rust 2018 edition. The effort finally culminated in a big release a few days ago.

Does that mean that the work on productivity is done? I hope the reader is going to agree with me that the answer is clearly no! I think we are seeing a recurring pattern what the edition releases are about: a flurry of preparation, design and implementation work intensifying and finally building up to a release[1], followed by a phase of consolidating work and driving smaller, supporting goals into completion.

Moreover, although there are clear-cut releases, there is no hard cutover for overarching themes. They just express the spirit of the community and give general direction as we are moving forward with the Rust project and the ecosystem — and I think we can be sure that the theme of productivity continues to define the direction we are moving to during the following year.

This was a long-winded way of stating the first point I want to make about Rust in 2019: I think we should continue to pursue the goals we set in the spirit of productivity. To mention some specifics: I think that finishing the the support for asynchronous Rust is incredibly important. Another very important goal we should focus on is improving the compiler; the query-based compilation model and refined incremental compilation are going to be a huge boon for productivity, as they enable faster builds and better support for IDE-like use cases. Completing the refactorings of trait matching and borrow checking should also allow doing some experimental work more easily. Finally, I’d also like to see the 1.0 releases of some important community-maintained libraries such as log, rand, tokio and hyper during the year 2019. All in all, I very much agree with Jonathan Turner in that I’d like the year 2019 to be a “fallow year” of maturing the ecosystem.

Now, that being said, let us talk about the future of Rust on a longer timeline.

Composability as a core value

I think Rust should consider setting composability as the overarching theme for the Rust 2021 edition and possibly upgrading it to a core value of the project. Rust already has almost all the qualities to make it the most composable language in existence — all that is missing is a concentrated community effort to realise those qualities into a polished vision. We’ll see next what I mean by composability and why it is desirable by going through some examples.

Examples from the composability zoo

Software reusability has always been a dream of many programmers. It’s not an easy thing to achieve; I’ve even heard some people calling it a pipe dream (usually in combination of trying to achieve it through some failed promises of object-oriented programming). However one can’t help but accept that it for sure is a desirable thing; reinventing the wheel every time certainly isn’t. I’m going to argue that composability — the ease of composing software from smaller building blocks — is a key property that makes code reuse possible. However, I don’t think there is a single feature that makes code composable. I think there is multiple excellent examples of some programming languages that are very composable in different ways; here I’m going to concentrate on C, Lua, Haskell and JavaScript.

C can be seen as the one of the most composable programming languages in existence, if you consider the vast amount of libraries written in C and the huge software stacks that consist of software written in C. There are two key features in C that make this possible: the lack of a runtime and ABI stability. The lack of a runtime means it doesn’t have much requirements for its runtime environment. This makes it possible for C to run almost anywhere — be it an embedded microcontroller, a mainframe or your smartphone. It also makes it possible to run C code in presence or in spite of a runtime, which makes C code easy to call from Python, Java, Ruby etc. The ABI stability makes also libraries written in C easy to package, distribute, wrap and reuse.

Lua is another example of being able to run almost anywhere. However, as a high-level language it has a totally different flavor; it’s an extremely simple language, and can be implemented as a C library. In this sense, it leapfrogs C’s ability to be run almost everywhere. However, there is another sense in which Lua is very composable: it is self-contained. Its runtime is a sandboxed context with a very clear, non-global state. If you want, you can easily run multiple different Lua instances and they don’t need to care about each others. Lua even makes possible running code in self-contained sandboxes inside its runtime by allowing creating “environments” to run code in. The code running inside an empty environment can’t access global state as it doesn’t have any reference to it. Lua makes it possible to run untrusted code and get away with it. This also enables the main usecase Lua is geared for: describing behaviour as scripts, and making it possible to create flexibly-behaving applications by composing the desired behaviour from those scripts.

There is a lot to love about Haskell. It has multiple features that make it extremely composable. I think the most notable one is the strict control of side-effects. It’s not a security feature and you can’t run untrusted code relying on just to the lack of side effects. However, it makes one confident that library code is self-contained and doesn’t do anything funny. All the side-effects are indicated and documented by the types in the function signatures. Another feature that makes Haskell composable is its generic polymorphism. It makes code composition flexible by not setting in stone the exact types that are needed for interacting with libraries, allowing very general and multi-purpose code to be written.

Finally, there is JavaScript. Most people wouldn’t think of JavaScript as an especially composable language, but I’d like to highlight a single feature that makes it easy to compose code in real life: a good package and dependency manager. Because of NPM, JavaScript library ecosystem is thriving.

Let’s return to our examples from a reverse perspective; there is some recent counterexamples that highlight the uncomposability of JavaScript. There was the eslint-scope hijacking incident and there was the left-pad incident. These were both cases that erode the concept of composing software from libraries. There are some problems with other languages too: Haskell has a runtime and garbage collection which makes it not always straightforward to call from other languages. This, combined with it’s lazyness make it also hard to build embedded software in Haskell. Lua, while having very strong story in embeddability, is as a scripting language not suitable for base systems implementation. It also does not have a strong “batteries included” library story as it is aimed for very specialised, application-specific use cases where a “catch-it-all” default set of libraries wouldn’t make sense. Finally, C, while being able to run almost everywhere, is ironically not easy to get to run almost anywere. This is because of the lack of polished dependency management and complicated, non-portable build systems. C also does nothing to ensure some properties of the code. It’s up to you to verify that it does what it claims to do, and if it segfaults, good luck with debugging nightmares. That’s not encouraging for code reuse!

As we can see, composability is not a single thing, and certainly it’s not just a technical language feature. Composability just means the ease of composing software from building blocks, and everything that makes the process of code composition easier and more robust, increases composability. Everything that makes the process harder or more unreliable decreases it. We can, for example, see that having strong standards for documentation increases composability, because it makes it easier and more accessible to understand each of the building blocks. Here, I find Rust’s story already quite good with the venerable websites like https://docs.rs/ and https://crates.io/.

Enter Rust

I’d like to argue that Rust is one of the most composable language there exists, and with a concentrated community effort it’s possible to polish it to be the composable language that is second to none in almost every aspect. We can already see that Rust shares similar qualities with many of our examples.

The lack of runtime is similar to C, and that makes it possible to run Rust code on embeddable devices and develop libraries that are easy to re-package and wrap for other languages to call.

It doesn’t have a sandboxed model like Lua, but it supports compiling to WebAssembly; I think that in the future we’ll see software even outside web browsers that implement plugins functionality by running sandboxed WASM.

Like Haskell, Rust also has a strong culture of not relying on global state and being explicit (in documentation) about side-effects which helps to assure that libraries you use are not going to interfere with each others (or with themselves, or with your code). This is further improved by recently stabilised support for deterministic const fns that hopefully continue to improve.

Also like Haskell, Rust has very flexible generics that makes writing generic libraries possible.

I’d love to see Rust further polish these aspects. I want Rust to be a language that runs anywhere, interoperates with anything, and gives you the peace of mind that it does what you expect it to. Here’s some ideas of features and tools we could spearhead to make Rust the most composable language in existence:

• More embedded targets
• Streamlined build process for embedded
• Easier and more complete story for no_std
• Complete and stabilise per-object allocators
• Make the WASM story even better
• Finish the work on generic associated types to allow more generic libraries and interfaces to be built
• Support projects that aim to make calling Rust code from other languages easy, such as Helix
• Create a strong culture for community code reviews to improve trust in the ecosystem libraries and support the development of tooling for that. Crev is an extremely promising project.
• Make builds generally more robust by sandboxing the build scripts and procedural macros (how awesome an use case for WASM would that be!)
• Make it easier to standardise build scripts by allowing them to depend on some community-maintained well-behaved libraries (RFC)
• Make separation between private and public dependencies clearer in Cargo, and make tooling for checking it automatically. (Edit: added this item on 2018–12–10)
• Support tooling for automatically detecting semver bumps; make Semverver a more integrated first-class tool. (Edit: added this item on 2018–12–10)
• Assess the problems with the orphan rules / coherence restrictions and try to solve the cental problems of “glue crates”. (Edit: added this item on 2018–12–10)
• Provide a versioned, stable ABI. It doesn’t have to include everything in Rust, but it should allow using some common types such as slices and trait objects (with possibly only some subset of the all features of trait objects) for stable FFI calls.
• Make lifetimes more composable through new typesystem features. I’ve highlighted some problems in my earlier post Things Rust doesn’t let you do. Especially the last three items 12–14 apply here.
• Make it possible to reason about side effects in a generic way, introducing a polymorphic effect-handler system.

I think that the most of the items are relatively uncontroversial. The last ones might raise some objections; there are some opinions about Rust being already having too much features or having already spent its complexity budget. However, I think that with good design and discretion, features such as these are only going to make code more comprehensible and easier to control.

In particular, I’d like to point out that at the moment, it’s hard to control the side effects of code that you call. In Rust the problem is generally not as bad as in some languages where there is a culture of writing very side-effect happy code. However, as const fns have been stabilised, we have already a way to restrict side-effects. While that is great, we now find ourselves on a head-on collision course with the problem of red and blue functions. What if we want to debug log from our const function? That’s a side effect. Should we make logging and log-less versions? At some point, we want to be polymorphic over side effects, and we want our users to decide how to handle them to improve composability. I don’t think it’s realistic nor desirable to do huge changes to the type system at short time scales, but the capability of controlling side effects would make sense in the long run.

All in all, I think we should seriously consider composability as a core value and overarching theme for the upcoming years and start sketching, planning and designing accordingly — as we have seen with the async support, these things take time.

Happy Rusting, and let’s make 2019 a great year for Rust!

Footnotes

[1] To be sure, the release model of Rust is to release often and do small, non-feature-based releases, but when it comes to edition releases, I think we can all accept that they feel “big”.