Rust, first impressions

Photo by Tengyart on Unsplash

Who am I to judge?

As I write this article, I have a little more than twenty years of experience in IT and development.

Inevitably over such a period of time, an engineer/developer has to use different languages in different environments within different business contexts.

Lately, I have been practicing more or less regularly the following development languages: C, C++, Haskell, Elixir, Dart, Scala, Go, Kotlin, and of course Rust.

I exclude my experiences with interpreted languages such as PHP, HTML / CSS, shell scripting, Wolfram Language, or Java Script which are for me rather in the category of scripting tools.

In the last 5 years, I have developed a few hundred projects (from a small CLI tool to a government scale production monitoring system through the Kubernetes plugin or the BtoC platform), mostly in Go, a language for which I have already expressed an opinion here.

About 6 months ago, I became interested in Rust mainly for three reasons:

• I couldn’t find a suitable library in Go for one of my recent projects (medical image processing)
• I was frustrated on one of my rather “memory intensive” developments (an in-memory database) by the inevitable impact of garbage collection on performance
• The permanent desire to discover and learn that guides us all in this job

The discovery process

Rust has the reputation of being a difficult language with a rather long learning curve, at the risk of shortening the suspense: it is true.

Not because its syntax is so exotic (those used to Python or Ocalm won’t be too disoriented), not because its expressiveness makes it unreadable (at least when you’ve done Scala…), but rather because of what makes it so powerful: its total absence of embedded runtime and therefore of a garbage collector.

Rust is indeed a bit of an alien in the development world, it is a modern hyper expressive language, but low-level, two usually antinomic properties.

We’ll come back to the technical consequences of this unusual positioning, but as far as the discovery process is concerned, it implies one thing: don’t throw yourself headlong into your first program without having studied the theoretical aspects of the language beforehand, it’s doomed to failure, even disgust.

Unlike Go for example, where you can master the language by using it in a few days, reading at least one book on Rust is highly recommended.

The reference being the official “Rust Book”.

Alternatively, or in addition, there are video tutorials more or less complete, I would advise the excellent channel of Ryan Levick.

Once you have assimilated the basic concepts of Rust (ownership, borrowing, lifetimes, macros, …) you will be able to start without cursing the compiler until you are sick of it (on the opposite, it will quickly become your best friend).

What makes it so special?

As mentioned before, Rust is an anomaly in the software engineering landscape: it is an advanced and hyper-expressive language, but low-level.

Explanations

Traditionally, development languages are classified into three categories:

• The so-called “system languages” or low-level languages: C is the undisputed king for nearly 50 years. They are not very expressive (i.e. you often have to reinvent the wheel or use libraries developed by others who did it for you). They are not very secure (the stability and security are entirely based on the developer level who is given all freedom), but they allow to obtain a binary very close to the “metal” (E.I. of the CPU) thus allowing its optimal exploitation (if the developer is good).
• The so-called “advanced” or “expressive” languages: were essentially designed to increase the productivity of developers via different paradigms: object, functional... They allow limiting the risks inherent to the first category (automated memory management avoiding stack overflow and other “dangling pointers”). We can put Java, Scala, Kotlin, Go, C#, Swift, …. Their performance and efficiency (ratio between performance and resource consumption) is clearly less good, but in a world where the power of the machines doubled at a constant price every 18 months, nobody ever considered that it was a problem.
• Interpreted or “scripting” languages: they do not produce any compiled code, but are instead interpreted on the fly when they are executed. They push the developer’s productivity axis even further but at the cost of performance and efficiency levels that are often catastrophic. We can mention for example Python, Ruby, PHP, JavaScript, R, Perl.

Rust aims to merge the first and the second category, and in some aspects, it succeeds.

Rust is clearly an advanced language, it doesn’t fully embrace the concepts of object-oriented programming (it’s a good thing since nobody wants it anymore) but like Go, it allows defining data structures and methods applicable to them, interfaces and composition.

It is not a pure functional language either, but it has most of its properties (default immutability, monads, higher order functions, closures, …).

It embeds a very developed genericity management, uses massively type inference or pattern matching.

All these capabilities position it quite high in the evolution hierarchy and make it clearly a modern language… but…

But, its compiler produces machine code that is at least as close to hardware and efficient as a C compiler (or even more efficient in some cases), without garbage collection and with a level of security that is hardly ever reached.

Rust is natively called “memory safe” (a property usually found only in garbage-collected languages) without the developer having to manage allocations/deallocations manually. It provides guarantees against “race conditions” or “deadlocks” in concurrent programming, all with a very high level of abstraction and without any impact on performance (we talk about “zero-cost abstractions”).

The compiler is powerful enough to produce totally deterministic code (including memory management) thus eliminating the need for runtime and a garbage collector at runtime.

How it is done ?

The explanation could fill entire books (and it does), so I will simplify and synthesize as much as possible.

Memory management

How can it be “memory safe” without garbage collection?

It is based on an innovative concept: ownership management.

Several rules follow from it:

• A value (i.e. a memory area, whether it contains a simple integer or a more complex structure) has an owner (I.E. a function)
• It can only have one owner at a time
• When the execution leaves the scope of the owner (I.E. leaves the function), the value is destroyed (deallocated)
• The property can be transferred from one function to another (the original function then loses access to the value)
• A function can “borrow” access to a value from another function while these one remaining the owner (like lending a book)
• There can be as many read-only borrowings as you want
• There can be only one read/write borrowing at a time
• A read/write borrowing prohibits any read-only borrowing
• A borrowing must refer to an active value (not yet destroyed, then no null/nill pointer)

This set of rules allows the compiler in a static way :

• to determine when to allocate and when to free memory, and thus to code it directly in the produced binary
• To guarantee the absence of dangling pointers
• To guarantee the absence of race condition (simultaneous writings in an undetermined order to the same memory zone)
• To guarantee the absence of memory overflow, the references or borrowings being smart pointers subject to the last rule

It is estimated that this last point alone is the cause of 70 to 80% of the bugs and security incidents (CVE) reported in the world.

This mechanism is clearly the most confusing for newcomers (I don’t mention the concept of lifetimes that follows from it).

This causes epic and frequent battles between the developer and the compiler if he stubbornly refuses to respect these rules.

Why is it so confusing when they seem to make so much sense? Because no other language forces you to follow them.

• Traditional system languages such as C let you violate them with impunity (with dire consequences at runtime)
• Traditional advanced languages like Java or Go handle allocations/deallocations for you, so you are never subject to their constraints.

Macros

This imperative of total determinism at compile-time poses a problem, it prohibits (among other things) variadic functions (I.E. functions taking an unknown number of parameters).

But this is a very useful type of function and it is used just as much, at least for strings formatting.

For example, the Rust equivalent of the following Go code is not possible:

fmt.Printf("Hello world, my name is %s I'm %d old", "bastien", 42)

The function fmt.Printf takes as parameters a variable number of parameters (the initial string with its placeholders and the values to be used).

The Go runtime will actually generate the final code at runtime according to the number of parameters actually present. But Rust does not have a runtime.

Deterministically compiling the code equivalent of fmt.Printf would imply producing an infinite number of versions with from 1 to an infinite number of parameters, this is obviously impossible.

The solution is the concept of “macro”.

They are identified by an exclamation mark at the end of their name:

println!("Hello world, my name is {}, I'm {} old", "bastien", 42);

A macro is “compiled” twice.

• The first step will generate a Rust code (we talk about Marco’s expansion) according to the number of parameters actually used (here 3 by analyzing in passing the type of value used in the parameters)
• The second step will compile this Rust code into machine code.

Simple and elegant.

For what daily result?

The absence of runtime/garbage collector opens interesting perspectives.

It allows the use of Rust for the development of operating systems and/or embedded systems.

• Microsoft has announced to include it progressively to replace C and C++ in the heart of its OS
• Linus Torvald has accepted the idea to experiment with the use of Rust in the Linux kernel (which is quite a sign when you know Linus’ strong positions on languages)
• Amazon used it successfully for the development of its micro-virtualization system hidden behind AWS Lambda
• Google uses it for the development of the possible replacement of Android: Fucshia

At the other extreme, it also allows the use of it on the “front” side (a place where we usually meet JavaScript and others) through WebAssembly.

We can therefore reasonably think that it is finally usable from one end of the application spectrum to the other.

Would this make it the holy grail ?, THE universal language that every developer aspires to?

I would be careful not to come to such a conclusion.

On one hand, because only time will tell, and on the other hand because everything is not so perfect.

Weaknesses

The entry cost

First of all, as mentioned, there is a long learning curve.

It can demotivate many, especially the less experienced or the newly “converted”.

In France and in other countries, schools (e.g. 42) and other rapid training centers (e.g. Le Wagon) have been springing up for a few years, allowing anyone who wants to train as a developer to do so.

They aim to make up for the lack of profiles available on the market by completing the cohort of graduates from more traditional schools (IT is a field where demand exceeds supply).

Freshly graduated students (whatever their age) will tend to focus on what they have been trained for: web development, mobile, AI, Big Data (no, this is out of fashion).

To be operational (from the company’s point of view) at the end of the training implies that the latter focuses on technologies and methodologies that produce results quickly (interpreted languages, agile methodology …), even if it means ignoring a whole part of information theory and computer science.

However, one cannot understand and appreciate Rust without being introducted to Von Neumann’s architecture, without understanding the difference between the Stack and the Heap, without knowing the orders of magnitude of the cost of the different allocation methods.

Rust is, therefore, in my opinion, aimed at rather experienced profiles who already have solid experience with other languages and who have already touched the limits of the latter.

The cognitive investment necessary to master it will not, I think, allow it to replace all the others.

The difficulty in reading

I classify Rust as a hyper-expressive language.

As such, it allows implementing in a concise and elegant way problems that are complicated or even complex.

But this does not make it an easy-to-read language.

The mathematical formalism is concise and elegant, but this has not made high-level mathematics a mass pastime.

A developer spends on average 90% of his working life reading code (his own or others’) and only 10% writing it.

The speed of reading and comprehension is therefore crucial for the overall productivity, the more important the project is (number of lines of code and number of participants) the more crucial they are.

Go excels in this field, it is so simple (at the risk of sometimes lacking conciseness or elegance) that a newcomer on a project understands it very quickly, and is just as quickly operational.

The intellectual mechanics of understanding the multiple nesting of Rust functions that can be linked in a single line of code is different, and it draws its philosophy (functional programming) from mathematical formalism.

Reading Rust code will always require more effort than reading Go code, so the question is whether it is worth it.

The power of the packaging system

The packaging and dependency management system of Rust (Cargo + Crate) is extremely powerful.

It may seem strange to classify it as a weakness, but this power has its price.

It requires for the company which wishes to have a private Registry (Crates warehouse) a heavier infrastructure than Go for example (which is satisfied with a simple source repository (ex: Git), often already deployed in the company).

Finally, and as always with powerful systems, mastering it requires a significant intellectual investment.

The lack of maturity of concurrent programming

This is an aspect often cited as a reference when talking about the Go language because it was designed from the start with the idea that it would be executed on multi-core machines connected in a network.

Rust was born around the same year as Go, but the handling of concurrent programming and asynchrony has curiously suffered many hesitations and withdrawals during its short history.

If things seem to stabilize today, we are far from the simplicity of Go-routines for example.

Once again, the selected approach is based on power and freedom of choice: for example, it is possible to choose one’s thread management engine, whether it is “OS thread” or “green thread”.

The slowness of the compilation

As you can see, the Rust compiler has a lot of work to do, between checking the respect of ownership and borrowing rules, the expansion of macros, the management of generic types, or the conversion of high-level abstractions into ultra optimized machine code, it is not idle.

This pays off in processing time, Rust is fast at execution, but certainly not at compilation.

As a result, developer productivity takes a hit.

Although the real-time verification tools allow limiting the call to the compiler, the complexity of the language makes the latter indispensable during the development cycle.

It provides very valuable and detailed explanations of the errors encountered in the code, and sometimes even gives the solution.

It is better to have a fast machine to do Rust.

The strengths

Performance

When it counts above all else (operating systems, video games, AI engines, blockchain, etc.), the price of the cognitive investment mentioned above clearly takes second place.

Performance (which normally goes hand in hand with efficiency) is paid for dearly anyway, and let’s be clear, to this day, nothing goes faster than Rust (except maybe coding in assembler, a form of torture abolished several years ago).

The only alternative on performance levels reached by Rust is C, which has neither its security nor its expressiveness level.

It has only two things going for it: its history (number of available libraries) and the number of profiles available on the market.

On these types of applications, the future of Rust seems to be very clear.

Security

The control mechanisms integrated into the compiler make it particularly attractive for critical systems or systems that are particularly exposed to security risks (cryptographic libraries, identity management systems, firewalls/network proxies, etc.).

When your code compiles (finally) you have the guarantee that it will execute correctly and that it will do what you asked it to do (but not necessarily what you think you asked it to do).

No surprises at runtime, no “random” crashes, and an attack surface that is limited to the logic of your program rather than the traditional vectors (memory manipulation).

Portability

Rust can be compiled, at the time of writing, for 84 different targets (pair of CPU architecture and operating system).

So it has the luxury of being more portable than Go with its 37 supported targets, even if in Rust not all targets have the same level of support.

The power of the packaging system

As mentioned, it is a weakness in some aspects, but also a strength.

Its modularity allows incredible and powerful management: dependencies management, versioning, and even features that you decide to activate, or not when importing a library.

I personally have not experienced anything like this in the other languages I have used.

The documentation

Rust is very well documented, whether it is the language itself (it was necessary), its standard library (split into several modules) or in a general way the publicly available libraries.

This is mainly due to the power of its automatic documentation generation system (via cargo).

By properly commenting on your code (Markdown format is natively supported), a real website will be automatically generated not only on the documentation of your code but also of all its dependencies.

Conclusion

Rust has become an important language in the software development landscape, it is indisputable, despite its young age (about ten years).

After these first six months of use and my first conclusive results, I give you my modest conclusion.

• Will I continue to use it? Certainly.
• Will I replace Go with Rust? Certainly not.

The two technologies are quite complementary.

Even an excellent Rust developer will never reach the productivity level of a good Go developer.

For high-speed projects, where specifications evolve several times a day, where teams are numerous and moving, where collaboration is important, Go remains for me unbeatable.

For critical, functionally complex projects, where performance is measured in nanoseconds (or even picoseconds), Rust is the right solution, provided you accept the costs.

Despite its incredible versatility and its genuine qualities, I don’t think it represents the grail, the universal language that will replace all the others.

But between us, who still believes in it?

Resources to go further

There are a lot of resources available to train and improve in Rust, you will find below my selection of the most interesting (for me, and until now)

Video :

• Ryan Levick’s youtube channel (live coding tutorial): https://www.youtube.com/c/RyanLevicksVideos
• “Let’s get Rusty” youtube channel (quick and effcient tutos) : https://www.youtube.com/c/LetsGetRusty
• Doug Milford’s youtube channel (sugested by a reader) : https://www.youtube.com/channel/UCmBgC0JN41HjyjAXfkdkp-Q

Books :

• “The Rust Programming Language” : official and free book (https://doc.rust-lang.org/book/)
• “Programming Rust, 2nd Edition” : probably the best Rust book I’ve read, good examples, simple explanations, detailed descriptions (https://www.oreilly.com/library/view/programming-rust-2nd/9781492052586/)

Want to know what happens next?https://bastienvigneron.medium.com/rust-second-impression-6ff6c26a95d0