Kernels and Logging in Linux
The booting process in a Linux system is quite a complete journey, it has a bunch of stages that accomplish certain tasks just to tune up your workstation or server in an efficient way.
The whole idea is to write a very compact overview on how kernels are triggered and how logging is performed in Linux systems, specifically on CentOS versions. I’m going to be pretty direct on some terminology because there’s a lot of information available for this, however, I’ll try to cover the much as possible in a concise and clear way.
The booting system
Starting from the firmware code execution, to boot loaders, kernels, ramdisk initialization up to whatever process management system, your server or workstation needs to perform all cycles/stages just to be ready for work in your system.
Just to warm up a little bit, we can set that the firmware executes code in two ways. In old systems the computer executes code in the BIOS, actually, this approach is mostly found in legacy systems. On the other side, newer computers perform code execution in the
The common approach here is that both are executed during the
power-on-self-test known as
To better understand this, you can refer to this article definition article from Geeksforgeeks.
POST process is performed successfully, the
Bootloader it’s started. At this point, the firmware executes the famous boot loader
grub2 code on the drive, the
Bootloader reads its configuration and boot the Linux kernel.
In cases when the
BIOS is present, the configuration file for
grub will be located at
/boot/grub2/grub.cfg , in other cases of
UEFI based systems, you should take a look at the
After it, the next big thing is the
Kernel initialization, in this step the
ramdisk is loaded into RAM memory, the
ramdisk serves as a temporary filesystem that includes significant kernel modules, drivers, and some Kickstarter files, after that, the kernel itself unmount the
ramdisk and mounts the actual root filesystem.
You can find more information about ramdisk here.
Once the filesystem is mounted, the initialization stage goes on. Here, you will be ruled by a certain service management system, whether using
A simple grub overview
As you may know, CentOS 7 uses
grub2 as their boot loader.
You can check the complete configuration of the grub loader in the
/boot/grub2/grub.cfg file. Here are a few code lines that are worth mentioning
## BEGIN /etc/grub.d/00_headerinternally create the GRUB menu header and sets up the screen for it
## BEGIN /etc/grub.d/10_linuxcontains the actual kernel choices that we see at the boot-up. All entries start with menu entry and include options for the bootloader and the Kernel
linux16 /vmlinuz-3-10.0 …is the actual Linux default system, this includes a route to the root filesystem.
initrd16 /initramsfs-3.10.0 …specifies the
ramdiskwhich matches the kernel. This is the previous step before mounting the root filesystem
### BEGIN /etc/grub.d/30_os-proberthis one can be useful if you have different partitions on the drive, the
os-probewould find it and create a boot entry for it.
### BEGIN /etc/grub.d/40_customand this one is used for custom menu entries
All the names can be changed for modifying the sort order of execution. If you perform any changes, you will need to some sort of save the
grub configuration just by running the
Additionally, we have the default grub settings file located at
/etc/default/grub which contains the full dump content used to set up
grub properly. A cool example is the
GRUB_TIMEOUT property that sets a timeout to the displayed menu before automatically booting the default kernel entry
The default kernel configuration is located at
/boot/grub2/grubenv and for this, it is worth mentioning the
saved_entry list that is nothing more than the menu items for the
In order to change the default
grub2 loader, you can use the
grub2-set-default command by adding the kernel list number to it.
After that, you’ll need to reconfigure the grub config file, using:
grub2-mkconfig -o /boot/grub2/grub.cfg
grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
Understanding the semantic versioning will be helpful to know what type of kernel updates can be applicable to a certain system. In CentOS systems, you can check a list of installed kernels by running:
sudo yum -q install list kernel-*
Here’s a kernel example version output:
- 3: is the major version
- 10: is the major revision
- 0: is the kernel patch
- 514: refers to the Red Hat version
- el7: refers to Enterprise Linux 7
The kernel files reside on the
/boot directory, so, when you list it you’ll find
vmlinuz files that represent the Linux kernel itself, the
initramfs as the ramdisk directories, and additionally, there are some grub-related directories.
Another important directory is the
/proc this one is created in memory as the system startup and gets deleted when the machine is shut down. The
/proc directory as a direct interface to the kernel itself. Internally you will see a bunch of numeric folders, which represents each running PID with execution details, and the no-numeric folders contain information about the memory system that are represented in text files.
In addition to the kernel, Linux has modules that have to match the kernel version, these modules add functionality to the kernel, which can include filesystems and drivers. You can find it on the
/lib and the
Here’s how you can filter it:
ls /lib/modules/$(uname -r)/kernel Internally there are libraries for architecture, crypto, drivers, filesystems, kernels, network, and more, however, you can check some of them with these util commands.
lsmodfor showing modules loaded in memory
modinfo [module]to show information to a particular module
modprobe -v [module]outputs information and its dependencies
And so on.
Linux handles hardware drivers automatically, but sometimes it’s required need to force issues, and this is because devices addressed across the network has no way of knowing what’s in there, this is a common scenario for a storage area, network, and printers, but if the hardware is local, there’s another util tool called
depmod So, for loading automatically at boot, you must:
mkdir /etc/modules-load.d(should include the module name)
sudo vi /etc/modules-load.d/dm_mirror.conf(each file for each module), inside type the name of the module:
dm_mirrorand that’s it
For Backlisting (the opposite):
sudo vi /etc/modprobe.d/ctxfi.confand inside you should type the name of the module:
blacklist [module]and that’s it
- When booting, the module will not be loaded.
Services, in a nutshell, are processes started by the OS that is managed by the not-so-recent but famous
systemctl command lets you manage (start, stop, restart, enable, disable, etc) a service.
In CentOS systems the
SystemD manages services and other objects like devices, timers, and targets, where every object is called
Units and for each unit, there’s a file for configuration.
Here are some examples:
systemctl list-unit-files -at servicethis basically show all enabled unit files, services configured to start up automatically
systemctl list-units -at serviceshows the enabled running services
The common output information consists of a service name, unit file status, a generated state, a current state, and a description. You can filter it as:
systemctl list-units -t service --state running
However, you can output detailed information like this:
systemctl cat [service] that will show detailed information about the service, including dependencies, command execution, fails, and more.
For management purposes, we have the following examples:
sudo systemctl stop atdwhere
atdis the name of the process
sudo systemctl status [process]for knowing the status
sudo systemctl start [process]to starting a process
The same pattern applies to anything else, you can also run:
sudo systemctl is-active [service]to know if a service is started or not
sudo systemctl disable [service]to disabling
sudo systemctl enable [service]to enable a service, basically, this makes it permanent
A log is basically a file that contains a message about the system, there are different logs for different tasks, you can find logs for critical errors, failed errors, cronjobs statutes, and way more.
CentOS systems have 2 logging systems
rsyslog and the
rsyslogit’s compatible with
sysklogdand it’s used for persistent logs. These types of logs are long text files that can be captured remotely via
journaldis non-persistent by default, they don’t survive to reboot. These are in a binary format and are stored on RAM and also have a fixed size. But they can be persisted by doing some strategies
You can interact with the
rsyslog service with the help of the
systemctl command. All the configurations settings will be at
rsyslog has some sort of rules where you can set which subsystem can produce the system message and its criteria (priority). Additionally, all the messages will be written in the
On the other side, the
journalctl log service is handled differently. Here are a bunch of command execution examples
sudo journalctl(all entries)
sudo journalctl -k
sudo journalctl /sbin/crond
sudo journalctl -u crond
In order to make it permanent, you’ll need to create a particular directory for it
sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald
ls -l /var/log/journal
This is all for now. I’m pretty sure that more concepts are missing at this time, so please stay tuned, this probably will get updated periodically.
All these concepts are a must-know for Linux administrators, so you can see this as a refresher or as a learning material as well.
Happy coding :)