The Memory Hierarchy

Learning about how computers work behind the scenes has always fascinated me. Whether you are an aspiring “tech guy/girl” or you need to buy a new laptop, learning about its configuration can be really useful.

A couple years ago I needed to buy a new computer. I had no idea about what kind of computer I should get, and I ended up deciding on one just because… It was blue. And it looked really cool actually.

So good looking!!!!!

To make a long story boring, it all went downhill when I started school and had to download files, e-books and text-editors. My computer just couldn’t handle it. But how could I have known?

Regarding a computer’s memory, the main goals in its design are having a large capacity, high speed and low cost. Combining these three can be challenging, but in order to achieve it efficiently, hardware designers combine different kinds of memories in a systematic way: a small amount of expensive fast memory and large amounts of inexpensive slow memory, achieving the highest possible access speed while minimizing the total cost of memory system.

All computers need different kinds of memories. These memories can be separated into two categories:

Volatile Memory: This loses its data, when the system or power is switched off. Most of the RAM (Random access memories) are volatile.

Non-Volatile Memory: This is a permanent storage and does not lose any data when power is switched off.

The system of combined memories working together is called the Memory Hierarchy and it can be represented as a pyramid.

Registers

Small memory spaces inside our processor that can each hold only single values. The registers can store a small amount of data really fast. A register may hold an instruction, a storage address, or any kind of data (such as a bit sequence or individual characters).


Cache

Is a high speed random access memory, directly integrated with the CPU.

It stores the data most used by the CPU, making the access for this information faster than in the computer’s main memory, and therefore increasing our program’s speed.

Level 1 cache is very small (usually between 2 and 64 KB).

Level 2 cache typically resides on a memory card located near the CPU. Depending on the CPU, its size is between 256 KB to 2 megabytes. Some of the inexpensive computers don’t have a Level 2 cache, but the high performance ones have their level 2 cache built into the CPU chip itself.

Level 3 cache feeds our Level 2, it is typically slower than the second one, but still faster than main memory.


Main memory

Is the physical hardware where programs and data are kept when our processor is using them during an operation. When the programs and data become active, they are copied from secondary memory into the Main memory where the processor can interact with them. Sometimes the main memory is called RAM, but properly speaking, the concept of ”RAM” is related to the type of silicon chip used to implement main memory.

When people talk about how much of “RAM” a computer has, they are talking about how big its main memory is. Common memory sizes are 256MB, 512MB, and 1GB. Since this amount is limited, instead

of stopping all of your programs from working, your computer starts enlarging the memory by storing data in the secondary memory (or hard disk space). This process is called virtual memory. Since the hard drive is much slower than the RAM, data stored in the secondary memory needs to be transferred back to the main memory, which will make our programs runtime slower.


Secondary memory

This is where the programs and data are stored in a long-term basis. This memory is not connected directly to the processor, which means it is significantly slower than the main memory, but can be over 100 times larger. Whenever we save a file in our computer, it is copied to a file in our hard disk. ROM (reading only memory) is an important type of secondary memory. It’s a ‘built-in’ computer memory that can only be read and not written to. This is a non-volatile memory, therefore its data is not lost when the computer power is turned off. This memory is part of the BIOS chip in the motherboard and it contains basic instructions to get the computer started.

As we can see in the pyramid, the memories in a lower level are larger, slower and less expensive. Understanding the memory access patterns was fundamental for writing code in lower level programming languages that allow you to have access and register directly to different memory locations. When we’re dealing with higher level languages, our compiler takes care of it more effectively. So, well, don’t panic if all these new names and concepts seem too difficult to memorize.

Now I’d like to share this short video that I had to watch more times than I can count prior to presenting this blog post:

Yep, sometimes it sucks to be a non-native English speaker.