Memory Management — Out of Memory Exceptions

This post is an oversimplified explanation on how memory management works; and why buying more memory won’t help you to solve out of memory exceptions.

Virtual mode

Contrary to what many professional programmers think, the amount of physical memory that a machine has, have nothing to do with out of memory exceptions. For instance, if a 32 bit process (that can address up to 4 GB of memory) is crashing on a PC that have 2 GB of RAM, adding a couple of Gigs won’t help at all.

Even if the process it’s running on a machine that has 1 GB of RAM, on 32 bits systems, that process can allocate up to 4 GB of memory (virtual address space, actually). That’s right. On modern systems, memory is virtual, so it doesn’t matter if the amount of physical RAM is low, the program won’t run out of memory because of that.

Sure, more RAM you get the better, because the OS will not have to swap memory to disk and your program will run faster. But the fact that a machine has more RAM, only affects storage speed, not storage capacity. (Even these days that we have solid state drives, RAM is much faster than “disc” access).

Toy memory allocator

Just for illustration purposes, let’s create a toy memory allocator for a platform where the address space is 12 bytes wide, the memory allocation unit is 4 bytes wide, and each memory slot is 1 byte. (In real life this is a nonsense, but is good for illustration purposes)

So, the empty address space of our process would look something like this:

| | | | | | | | | | | | |

If we ask the allocator for a chunk of memory to store a 3 bytes wide variable, it will take 4 bytes from the address space. (Remember that it works on multiples of 4.)

x == used space
- == wasted space
^ == next free slot
|x|x|x|-| | | | | | | | |
         ^

Since the allocator works on 4 bytes basis, each time it allocates a 3 bytes wide variable, it pads the storage to accommodate 4 bytes and wastes a byte on each allocation. (That lost byte in the address space is unreachable for the allocator).

If we ask more memory to store a 2 bytes wide variable, it’ll also take 4 bytes from the address space, wasting two more bytes.

|x|x|x|-|x|x|-|-| | | | |
                 ^

So far we have 8 bytes held; 5 bytes in use and 3 bytes wasted out of our 12 bytes of address space.

You may have heard about memory fragmentation. That is what is happening with our process. Since the allocator uses 4 bytes offsets to address memory it need to pad some allocations so that the storage location for a variable always starts at a 4 bytes offset. (And by doing so, it wastes some memory).

Now let’s say that our process needs to allocate two more variables, one is 3 bytes wide and the other is just 1 byte. What will happen?

If you said out of memory exception, you are right. Let’s take a closer look: The first variable is 3 bytes wide, since the allocator works on 4 bytes basis, it pads the 3 bytes width with an empty byte, wastes that byte and runs out of space for the other variable.

|x|x|x|-|x|x|-|-|x|x|x|-|
                        ^

If the address space wasn’t as fragmented as it was, the allocator would ve been able to store one more 1 byte wide variable (in fact, four of them). But in this case, the wasted space is so big, that the process will crash.

That is what happens when you see out of memory exceptions on processes that are using less memory than what the OS allow them to address. Even though they haven’t hit the OS’s limit, they run out of “reachable” memory and end up crashing.

To keep this post short, I’m not going into the details of how to free or de-frag memory. Maybe I’ll cover that in the future.

Of course this is an extremely simplified description of how memory allocators work, but is accurate in enough to give you a good idea of what happens when your program runs out of addressable memory.

So, that’s it for this post. Thanks for reading! And see you next time!

PS: Don’t forget to clap if you like this post :)