Go slices, deleting items, and memory usage

Go is one of the best choices for cloud applications. It’s fast, frugal, secure, and provides built-in concurrency. A single server instance can handle hundreds, or thousands, of concurrent users, for a very long uptime.

I had the opportunity to contribute a small but important change in the Go standard library. It’s aiming at reducing the memory usage, making software more frugal and reliable.

The most efficient services strive to not allocate any extra memory at all when handling each incoming request. This makes sense for some specific, limited processings, but is not always possible. We create objects, lists, maps, which often have a non-zero memory footprint in the heap. It is still valuable to minimize our memory usage, especially for objects that outlive the current user request. We don’t want our total footprint to grow after each request. Whether the memory usage is growing for a legit and useful purpose, or because of a memory leak, the server would frequently crash with an “out of memory” error (OOM).

I love it when a programming language comes with a garbage collector (GC), because machines are just better than humans at managing memory. But even with a GC, it is still possible to create too many objects and keep them “alive” (when the program is still holding references to them), and end up in an OOM crash. Let’s be careful!

I also love it when I can use a carefully designed abstraction from the standard library, rather than writing my own toolbox. This is what the slices package added in Go 1.21 is about: various utility functions to manipulate slices of any type.

One of them is Delete:

// Remove s[2], s[3], s[4] from the slice s
s = slices.Delete(s, 2, 5)

This is much nicer than the traditional way of deleting elements from a slice, before Go had generics and before the slices package existed:

// Remove s[2], s[3], s[4] from the slice s 
s = append(s[:2], s[5:]...)

But slices.Delete comes with a few gotchas:

• It does not allocate a copy of the original slice s. Instead, it modifies the slice in-place, reusing the same underlying array.
• Because it’s modifying the length of s, it needs to return a new slice (a new window on the same array). That’s how slices work.
• Accidentally ignoring the return value is a bug. This will trigger a go vet warning in Go 1.23.
• The underlying array never shrinks.
• The original slice s is now “invalid”. Its length is too large and its last elements are not useful data anymore. Any other slices sharing the same memory portion are now invalid as well.
• In Go 1.21, the objects referenced by pointers in the array “beyond the new length” would not be garbage collected either.

The last gotcha caught my attention me because Go developers would not always be aware that, in a slice of pointers to large objects, using slices.Delete could leave some pointers alive in a now-invisible part of the underlying array.

s = slices.Delete(s, 2, 5)

result in Go 1.21

In my opinion, this was a form of memory leak.

This leak would not theoretically affect the program correctness, but it would lead to:

• performance issues
• higher costs associated with memory usage
• higher probability of OOM crash (which, in a way, is a correctness issue)

One possible solution would be, after shifting (copying) the right part to the left, to set the rightmost “obsolete” elements to nil, and let the garbage collector do its job. For the sake of memory frugality, I wrote a formal proposal, implemented it, and published a detailed article about it.

s = slices.Delete(s, 2, 5)

result in Go 1.22

I had initially envisioned a casual, obvious fix: “let’s set the tail elements to nil, just like the ArrayList class does in Java”. But the community was divided at first: for some people, the status quo was better. And in my journey I discovered lots of details that required careful attention: What about runtime performance? What about the elements beyond initial length, within capacity? The capacity cap(s) includes the extra space available after len(s). What about element types that contain both pointers and non-pointers? What about the subtle question of the “ownership” of the original slice? What about accidentally misusing the package API? And what about backward compatibility?

Fortunately, even if the behavior change is user-visible (the zeroed elements are reachable), it is backward compatible:

• The documentation of slices.Delete did not specify if the “obsolete” elements would be modified or not. No promises were made. Codebases were not supposed to rely on this detail of the old behavior.
• In codebases audits, we found that all such occurrences of code relying on the old behavior were in fact misusing the API (e.g. by ignoring the return value of Delete).
• The documentation is now more explicit: Delete zeroes the elements s[len(s)-(j-i):len(s)].

Now that Out of Memory errors are a bit less likely, it’s time to deploy our first Go service to Google Cloud Run :)