Introduction to syscalls in Linux

In the previous article, we talked about how libc-free programming works. It’s time to talk about syscalls, especially in Linux.

In Linux, we have several different syscalls that are responsible for doing many different things. You can look at the Linux syscall table by Chromium to see a list of them, and what arguments each syscall requires. Each syscall has a number and requires specific arguments. For making a syscall on an x86_64 Linux machine, we store the syscall number in the “rax” register that indicates which kind of syscall we’re marking. Then we store the arguments needed. Arguments 1, 2, 3, 4, 5, and 6 should be stored in registers “rdi”, “rsi”, “rdx”, “r10, “r8”, and “r9” respectively. All of the said registers can hold up to 64 bits, which is why they’re called 64-bit registers. The result of our syscall is stored in the returning register “rax” by the OS. The type and usage are dependent on the type of the syscall.

Here’s an example in C:

// The write syscall number on x86_64 is one
#define SYS_write 1

// It's pretty common to see "syscallN" functions where 
// N is the number of arguments for a syscall. The write
// syscall takes three arguments.
long syscall3(long type, long a1, long a2, long a3) {
    // The type of syscall or the syscall number
    // goes to "rax".
    register long _rax __asm__("rax") = type;
    // The first argument of the syscall goes to "rdi".
    register long _rdi __asm__("rdi") = a1;
    // The second argument of the syscall goes to "rsi".
    register long _rsi __asm__("rsi") = a2;
    // The third argument of the syscall goes to "rdx".
    register long _rdx __asm__("rdx") = a3;
    // We make the actual syscall here. "r" stands for register.
    __asm__ __volatile__("syscall"
    // The returning register is "rax".
                         : "=r"(_rax)
    // Here we pass the registers as arguments.
                         : "r"(_rax), "r"(_rdi), "r"(_rsi), "r"(_rdx)
    // Then we tell the compiler about some potential
    // changes to some flag registers.                         
                         : "rcx", "r11", "memory");

    // Finally we return the "rax" also known as the returning
    // register
    return _rax;
}

int main() {
    // The write syscall takes three arguments.
    // The first one is the file descripter which we pass
    // 0 meaning we're willing to write to the standard output.
    // The second argument is the string we want to write. We
    // cast it as long to take the address in memory.
    // The third argument is the number of bytes our buffer
    // has that we passed as the second parameter.
    syscall3(SYS_write, 0, (long)"Successful write syscall!\n", 27);
    // After running this you should see the message printed
    // to your screen.
}

congratulations! You just learned how to make syscalls in Linux. Don’t forget to read the upcoming article.