Memory Management in Linux
You are studying for your end semester. Given that you haven’t paid attention the entire semester, you start cramming up all your subjects at once. There goes concurrency control of DBMS, Genetic Algorithms from ML, and Device Drivers from System — and these are just single topics. There are tons of such subtopics from 6 different subjects. Is your memory gonna explode or is it gonna EXPLODE! You need to play smart here and manage your memory. Imagine now how this would be in the case of a computer system!
Memory Management is the idea of using a variety of methods to store and track data and programs in memory (RAM) and restore space when it is no longer needed. It is one of the most important parts of any operating system and when I say important, I mean it here!
Linux OS is a broadly used open-source operating system that is based on Linux Kernel. Memory management under Linux has 2 main components:
a). The first is about sharing and releasing physical memory blocks, page groups, and small memory blocks
b). The second handles virtual memory memory-mapped into the address space of active processes.
Management of Physical Memory
Linux divides physical memory into three separate regions or zones:
- ZONE_DMA
- ZONE_NORMAL
- ZONE_HIGHMEM
ZONE_DMA
DMA here stands for Direct Memory Access i.e data is directly transferred between the memory and the peripheral devices without the intervention of the CPU. Now this differs from architecture to architecture. For example:- In the Intel 80x86 architecture, some devices can only access up to 16 MB of virtual memory using DMA. For such devices, the first 16 MB of portable memory includes ZONE_DMA.
ZONE_NORMAL
This indicates the physical memory mapped in the CPU address space. This zone is used for most common memory applications.
ZONE_HIGHMEM
Any real memory not mapped into the kernel address space comes under this (ZONE_HIGHMEM = High Memory). For example:- In 32-bit Intel architecture(2³² = 4 GB) the kernel is delineated into the first 896 MB of address space; The other remaining memory is what we know as high memory and allocated to ZONE_HIGHMEM.
REAL-TIME FACTS ABOUT LINUX MEMORY
1. Each address space is represented in Linux Kernel by an object called mm_structure (struct_mm_struct in file include/linux/sched.h).
2. Many tasks can share the same address space, so mm_structure is something that counts as an existing reference as long as one function uses the address space it represents.
3. There is a mm_pointer to mm_structure that defines the address space of the task.
4. As a special feature, functions known only for accessing kernel space (such as the kswapd-Kernel swap Daemon) are said to have unknown address space, and mm_pointer for such tasks is set to NULL.
How is memory actually allocated in these ZONES?
The primary physical-memory manager in LINUX Kernel is the Page Allocator. Each zone has its own allocator which is responsible for allocating and freeing all physical pages for the zone and is capable of allocating ranges of ‘physically contiguous pages’ on request.
HOW IS IT ABLE TO ALLOCATE CONTIGUOUS PAGES?
For this, it makes use of something called a Buddy System. The allocator, with the help of a buddy system, keeps track of available physical pages. Each allocated memory region has an adjacent partner (or buddy). Whenever two allocated partner regions are freed up, they are combined to form a larger region called BUDDY HEAP. If a small memory request can not be satisfied by allocating an existing small free region, then a larger free region will be subdivided into two buddies to satisfy the request.
IS THIS THE ONLY METHOD AVAILABLE?
NOPE!
Memory allocation in Linux kernel occurs either dynamically, as seen above, or statically (through drivers).
In the static method, drivers reserve a contiguous area of memory during system boot time. But Static partition schemes suffer from the limitation of having a fixed number of active processes and the usage of space may also not be optimal.
For maintaining pages of file contents, a separate cache memory called Page Cache is maintained by the kernel’s main cache. It stores entire pages of file contents for local and network file I/O.
For Kernel Memory, Linux uses an allocation strategy named SLAB ALLOCATION STRATEGY.
SLAB ALLOCATION IN LINUX
- A Slab is used to allocate memory to kernel data structures and is made up of one or more physically contiguous pages.
- A Cache consists of one or more slabs. There is a single cache for each unique data structure — a cache for process descriptor, a cache for file objects, a cache for semaphores, and so forth.
- Each cache is populated with objects of the kernel data structure the cache represents. For instance, a cache representing semaphores stores instances of semaphore objects, and the cache representing process descriptors stores instances of process descriptor objects.
THE SLAB ALLOCATION ALGORITHM:
Whenever a cache is created, a number of objects are allocated to the cache which depends on the size of the associated slab. Like say, for instance, a 12 KB slab, made up of three contiguous 4 KB pages, could store six 2 KB objects.
Initially, all objects in the cache are marked as free. When a new object for the kernel data structure is needed, the allocator can assign any free object from the cache to satisfy the requirement. Post this, the object assigned from the cache is marked used.
Any slab in Linux can be present in one of the three possible states:
- Full — All objects are used.
- Empty — All objects are free.
- Partial — Some objects are free while others are used.
Wasn’t this interesting to know! Well, ever wondered how a 10 GB game like God of War runs on your 2 GB RAM computer?
Virtual Memory
In layman’s terms — Something which appears to be present when actually is not. The address space visible to each process is what comprises the Virtual memory.
Management of Virtual Memory:
The pages of virtual memory are created on-demand and the management consists of loading those pages from disk or swapping them back to disk as required.
The Virtual Memory Manager maintains two separate views of a processor’s address space:
Logical View
This view describes instructions received by virtual memory concerning the layout of address space.
The address space consists of a set of non-overlapping regions, each representing a continuous, page-aligned subset of address space.
Each region is described internally by a single vm_area_struct structure that defines the properties of the region, including process’s read, write and execute permissions in the region as well as information about any files associated with the region.
The regions of each address space are linked into a balanced binary tree to allow a fast lookup of regions corresponding to any virtual address.
Physical View
This view is that view of virtual memory which is represented by the information stored in the hardware page tables of the process.
Every page table entry identifies the exact current location of each page of virtual memory, whether it is on disk or in physical memory.
The physical view is managed by a set of routines, which are invoked from the kernel’s software-interrupt handlers whenever a process tries to access a page that is not currently present in the page tables.
Each vm_area_struct in the address-space description contains a field that points to a table of functions that implement the key page-management functions for any given virtual memory region.
All requests to read or write an unavailable page are eventually dispatched to the appropriate handler in the function table for the vm_area_struct.
A new virtual address space is created when — 1). When a process runs a new program with the exec() system call. 2). Upon creation of a new process by the fork() system call.
Linux implements several types of virtual memory. Most memory regions are either backed by a file or by nothing. The one backed by nothing is the simplest type of region and represents demand-zero memory. When a process tries to read a page in such a region, it is simply given back a page of memory filled with zeroes. In the case of file-backed, a page containing the offset of the file will be returned.
That is it for this blog. I appreciate each one of you for taking the time to read this till the end.🌟 If anything seems off, do mention it in the comments, I would gladly improve.
Make sure to clap if this is helpful and for more such blogs, do follow me on medium.
