Describing UNIX command line interpreter (Shell)
This post describes step by step how to understand Shell program and what happens when you type “ls -l”and hit Enter. Writing this post is one of the Simple Shell pair-project tasks for Holberton School.
In a word, typing ls -l displays the content of the current working directory in a long format, i.e. you can see all the files and directories along with permissions, owners and created date and time:
Now let’s look at that on the deeper level.
Before moving forward it is important to understand what the terms kernel and shell mean and what is the difference between sh and bash.
Kernel is the central and most fundamental part of a computer operating system. It is a “brain” of a computer which is responsible for interfacing all of your applications that are running in “user mode” down to the physical hardware. It allows processes, known as servers, to get information from each other using inter-process communication (IPC).
Shell is a command line interface (CLI) program that takes commands from the keyboard and gives them to the operating system to perform. It is the way a user “talks” to the kernel, by typing commands into the command line.
Sh (or the Shell Command Language) is a programming language described by the POSIX standard. It is a detailed description of the syntax and semantics of that language.
Bash is the shell for the GNU operating system. Bash is a programming language that can be thought of as an implementation of sh (although that has changed with time). So sh is a specification, bash is the implementation.
Three main things Shell implements:
- Initialize: In this step, a typical shell would read and execute its configuration files. These change aspects of the shell’s behavior.
- Interpret: Next, the shell reads commands from stdin (which could be interactive, or a file) and executes them.
- Terminate: After its commands are executed, the shell executes any shutdown commands, frees up any memory, and terminates.
Step 1: Reading a line
a). Alias
In the newer versions of the C library there is a getline() function in stdio.h header file. So once you’ve typed in your command and hit enter, Shell reads the line and allocates a buffer for storing the line. When recreating the Shell we used getline() function for that:
Then Shell checks for any aliases associated with the command you’ve entered. If an alias is found, it replaces ls with its value.
b). Special Characters
If no aliases are found for the command, shell program looks for special characters such as: ", \, ', *, #, &and executes the logic associated with each special character.
c). Built-in
Bash built-in commands (also known as “internal commands”) are part of the shell itself. Each built-in command is executed directly in the shell itself, instead of an external program which the bash would load and run. For example, alias is an internal or built-in command, and ls is an external command.
Step 2: Parsing a line
a). Tokens
Now, we need to parse the command line into a list of arguments. Calling strtok() function we can split the line into separate strings of words. Function returns a pointer to the first token. Actually function returns pointers to within the string you give it, and place ‘\0’ bytes at the end of each word (token). We store each pointer in an array (buffer) of character pointers.
Finally, we reallocate the array of pointers if necessary. The process repeats until no token is returned by the function, and we need to null-terminate the list of tokens. Type “man 3 strtok” in the terminal for more details.
b). $PATH
There are many pieces of information that your shell compiles to determine its behavior and access to resources. Some of these settings are contained within configuration settings and others are determined by user input.
One way that the shell keeps track of all of these settings and details is through an area it maintains called the environment. The environment is an area that the shell builds every time that it starts a session that contains variables that define system properties:
One of the environment variables is a PATH. The PATH variable holds as its value a list of directories the Shell searches every time a command is entered. Shell program looks through each directory separated by a ‘:’ to search for the executable program for ls command, which is token[0]. There is a function called stat() that does the job. The prototype of the function is as follows:
int stat(const char *path, struct stat *buf);The error message is displayed, if no executable program was found.
Step 3: Executing the command
a). Parent and child processes
Since ls isn’t built-in command, one practical way for processes to get started is the fork() system call. When this function is called, the operating system makes a duplicate of the process and starts them both running. The original process is called the “parent”, and the new one is called the “child”. fork() returns 0 to the child process, and it returns the process ID number (PID) of its child to the parent. So why do we need that duplicate process?
When we want to run a new process, we want to run a different program in it. So we need execve() system call for that. It replaces the current running program with an entirely new one. So when we call execve(), the operating system stops the process, loads up the new one, and starts running it. A process never returns from the execve() call, unless there’s an error.
The input to execve() for ls -l after the PATH search will be:
execve(“/bin/ls”, {“/bin/ls”, “-l”, NULL}, NULL); Tying it all together, each time you enter a command a new child process needs to be created and the parent process needs to wait for the child process to execute before proceeding forward. We can use wait() system call for that. Once the child process executes, we need to free all allocated memories before passing control back to the parent process.
if execve() returns -1, the error message should be displayed. We use perror() to print the system’s error message, along with our program name, so users know where the error came from. Then, we exit so that the shell can keep running.
Step 4: “Re-prompt”
After ls -l is executed, the shell executes shutdown commands, frees up memory, exits, and re-prompts the user for input.
Note to reader:
Since you know what’s is “under the hood”, why not to practice writing your simple shell…
For deeper understanding read more about the following functions and system calls from man page:
access(man 2 access)chdir(man 2 chdir)close(man 2 close)closedir(man 3 closedir)execve(man 2 execve)exit(man 3 exit)_exit(man 2 _exit)fflush(man 3 fflush)fork(man 2 fork)free(man 3 free)getcwd(man 3 getcwd)getline(man 3 getline)isatty(man 3 isatty)kill(man 2 kill)malloc(man 3 malloc)open(man 2 open)opendir(man 3 opendir)perror(man 3 perror)read(man 2 read)readdir(man 3 readdir)signal(man 2 signal)stat(__xstat) (man 2 stat)lstat(__lxstat) (man 2 lstat)fstat(__fxstat) (man 2 fstat)strtok(man 3 strtok)wait(man 2 wait)waitpid(man 2 waitpid)wait3(man 2 wait3)wait4(man 2 wait4)write(man 2 write)
Once you want to practice writing your own simple shell, don’t forget to add the following header files:
#include <stdarg.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/wait.h>
#include <sys/stat.h>
#include <signal.h>
#include <errno.h>
extern char **environ;
Good luck!
You can find some inspiration from the link below:
https://github.com/patrickdeyoreo/simple_shellAuthors:
Patrick DeYoreo and Banu Sapakova