Borrow is not a pointer
Yesterday I visited the DSRV hacker house in Seoul where my friend Ethan Frey was doing his great hands-on session about building CosmWasm applications — from smart contracts to basic dApp. During the session, there was a question from the audience — “So is the borrow a pointer?” After a while of hesitation, Ethan answered: “Yes, it is.” I understand why he responded like that — in the vast majority of cases, you can easily treat borrows as pointers. But I also fully support his hesitation — borrows (or rather references — borrowing is just an act of obtaining one) are, in most cases, implemented as pointers, but their semantics is different. And also — most cases are not all cases.
Disclaimer
The article elaborates on the question whether the borrow (meaning reference) is a pointer asked by Rust newcomer from another tech stack. Rust has its own pointer type, but it behaves a bit different than what we expect, considering pointers from C++ or other common languages. But to make it clear — by `pointer` in this article, I mean the variable containing an address with common operations to be performed on it. The purpose of the article is to emphasize that reference and borrowing in Rust is much more than that, and in some cases, it may be misleading to think about references as addresses.
What is a pointer?
Let’s establish some pointer definitions to discuss the relationship between the pointer and the borrow. A pointer is a variable containing an address of some data in the memory. It typically can be any unsigned number fitting the native machine word — u32for 32-bit machines and u64for 64-bit machines. One standard operation on pointers is the dereference, which reaches the memory addressed by the pointer and retrieves data from it. In all languages I am aware of, there is no reason why a pointer cannot take any arbitrary value — in particular, there is often to have a pointer containing a 0, commonly called nullor nil. Such a pointer is a special value for “this pointer does not point to any data.” Also, pointers being just numbers, can point to data that are nevermore valid. The straightforward case is returning a pointer to local function variables — when the function is done, the frame pointer is closed, and data are invalid. Most languages can detect such obvious cases — C++ throws a warning — but typically, obtaining an invalid pointer causing the “use after free” behavior is not difficult. There is also a problem, what happens if we dereference such a null or invalid pointer? In most languages, there is an undefined behavior — either (if you are lucky) it will cause a crash due to segmentation fault (by memory protection failure). Still, often such operation would succeed, giving the user some invalid and/or inconsistent data. Such behavior tends to cause serious logic bugs, which are very difficult to detect.
The last thing we typically expect from pointers is so-called pointer arithmetic. As pointers are just numbers, there should be a way just to add or subtract them. This technique is often exploited to efficiently iterate over continuous memory of objects of the same type (aka arrays or slices). Some languages don’t support it natively (thinking about Nim right now), but even then, it’s often an easy workaround for that — in Nim, you just cast the pointer to an integer, perform arithmetic, and cast it back with no compiler complaints.
Obviously, I am aware that there are languages handling pointers differently — for example, most GC-driven languages would not allow having “dangling pointers” — but also, they are commonly called “references”, not “pointers” for a good reason — they are not exactly pointers. The same argument for “Smart pointer” — sure, they handle some of the pointers problems still keeping their core “dereference” functionality, but they are “smart pointers”, not “pointers”. Rust borrows are neither of those — they have different semantics closely related to borrowing rules and lifetimes. Now it’s time to take a closer look at them.
Obtaining reference
First, let’s start with the fact that the Rust reference cannot be built from an arbitrary integer value. The only way to create a reference in safe Rust is to borrow it from some existing variable. The reference will then refer to this variable. A significant thing to emphasize is that the borrower will refer to this variable, not point to memory. It may seem like not a difference at all, but it is. Every reference in Rust keeps the additional value being its lifetime — meaning how long this borrow can be alive. And this lifetime is “as long as the original object borrow is created from is alive.” This lifetime is verified by the Rust compiler and will never allow the borrower to be used after the original variable may be dead. Notice that I write “may be dead”, not “is dead” — the check is performed in compile time, so there is no runtime overhead, but it means that if some path leading to the point in the program would destroy the variable, borrow cannot be used. Take a look at some examples:
In the first example, we see that Rust insists on not letting us return a borrow to a local variable in any way. And as I said before — some languages handle it via static code analysis, but in Rust, it is baked into the type system. The whole borrow checker would never allow us to return a borrow to not existing object, and it can detect pretty complex cases. You can see that in examples in the main functions — if we try to use a borrow after dropis called on the original variable, a compiler error occurs. The drop function is a utility to destroy the variable before it goes out of the scope — so using a borrow after this would be illegal.
What is interesting is the example with the mutable borrow. So as reference traces what variable it borrows from, it is possible to prevent having multiple mutable borrows or mutable and shared references simultaneously. It helps us avoid multithreading problems like data race, but it actually prevents even simpler, very common mistakes from happening on a single thread. Consider this C++ snippet:
This code is suspicious at first glance — everyone should see the infinite loop there. And yes, I know I could use ranged for syntax — but I wanted to make it more obvious what happens. But believe me, the infinite loop is not a problem here. In the loop, we modify the vector by adding an element. And it may — and eventually will — cause the underlying memory reallocation. As a result, the itgets invalidated, and technically we have undefined behavior here — in practice, it would always end up in segfault. So what do things about a similar code?
This time we get a compile error. Rust complains that we are trying to borrow datamutably to call the pushfunction, but the datais already borrowed to iterate over it — and the iteration borrow would be released after the whole loop is over. It may seem like a very artificial problem, but only because it is a very small snippet. In large codebases, it is common to have some kind of handle to the object internals (here: an iterator), but mutations of this object invalidate the handle. Rust prevents any kind of such mistake, as any way of keeping “handle” to object internals would require keeping a borrow to the object.
Zero Sized Types
The difference in how borrows are handled compared to pointers leads to a very interesting thing — in Rust, some types are zero-sized (do not confuse with unsized, which is another thing). That means no matter how many of them you keep in your memory, they completely disappear after the binary is compiled! Consider the following snippet:
This program would print 0in standard output. And if it is not surprising for you, I would tell you a secret — similar code in C++ would return a non-zero value (probably 1, but it is technically architecture-dependent). The reason for this is that in C++, there has to be a possibility to create a pointer for every object. Also — the pointers to two distinct objects have to be different. As a result, even objects with no payload have to occupy at least one byte of memory. There is no such thing in Rust. In Rust, every object can be borrowed, but borrow doesn’t have to be an address. In the case of ZST (Zero Sized Types), the borrow contains only type system-related information: its mutability and lifetime, but should not leave any runtime footprint.
This behavior around ZST is also a very important difference in how references are handled in Rust and most other programming languages. In SW architecture, we often treat references more like pointers to a particular object, which are more or less guaranteed to be valid (for eg. in GC collected stack, references would prolong object life). But this leads to the fact that we often expect references to be different for two distinct objects with exactly the same values. For example, in C# and Java, if we have types with no data, comparing references to two different objects of those types with == behaves exactly like comparing pointers in C++ — objects are different and distinguishable. My point here is — that Rust references do not have any guaranteed notion of address or object identity. It just borrows from a given variable, but it is allowed to optimize the address at any point. ZSTs are very much the case of such optimization, and it’s worth being aware of that.
To be honest — for a long time, I advocated for calling rust references (as a type) a borrow — and only recently did I get corrected that it is not a proper name used in the community. I respect that and will switch to common naming — but still emphasize the difference in how they work in Rust compared to other stacks.
The unsafe Rust
Now it’s time to make fun of myself — everything I said is a lie. There is an unsafe Rust, where I can create pointers (yes, there is a notion of a raw, pure pointer in Rust) from random numbers and then cast them to borrows to create dangling references or transmute lifetimes to induce use after free. But the unsafe Rust is the point of this write-up. When I talk about anything in Rust, I always consider its safety guarantees, assuming they are fulfilled. I am also aware of the unsafe layer. The issue is that the idea of unsafe Rust is not to give you a way to burn your entire system. The idea is that sometimes cases are very difficult or impossible to prove by a compiler. And in such cases, we have this tool to be used. But with great power comes great responsibility — when using unsafe Rust, it is a developer’s responsibility to keep all the guarantees that we have in safe Rust. You must ensure your lifetimes are properly bound, and you have to remember that ZST borrows cast to pointers doesn’t give unique addresses. My point here is — as long as you are mid-level Rust dev, you can think about borrows as if they were pointers, and you will never get harmed — the compiler would make you safe. However, I think it is worth exploring differences to take a wider POV about borrows and how strongly they impact the Rust programming language especially if you want to do any unsafe stuff.
Find me on Github and LinkedIn.
Also, check out my CosmWasm book and the CosmWasm academy we are building.
