Magic in the Shell: A Deeper Look at the Command Line Interface
The longer I gaze into the crystal ball of knowledge, the more I realize there’s more to learn than I could in this lifetime.
The worlds of technology, mathematics, psychology, and science are colliding in ways I never imagined. The topics I’ve unearthed are creating parallels that would’ve been impossible to discover on their own.
Learning the fundamentals of networking and operating systems has allowed me to better understand how to use command-line tools and make server requests. Understanding how the mind works has allowed me to make analytical decisions instead of emotional ones while problem-solving.
This week’s journey took an unexpected turn while learning about command line interfaces. I found myself asking the philosophical question: “What kind of magic happens when I execute this command?” I’ve been using CLI tools for most of my life, but I never took the time to truly understand what was going on behind the scenes.
Until now.
The Early Years
Command Line Interfaces (CLIs) have a long history dating back to the 1960s. They were created as a way for users to interact with operating systems and perform simple computing tasks.
One of the very first CLIs was the Command Line Interpreter for the MIT Compatible Time-Sharing System (CTSS) developed in 1961. This was followed by the Multics shell, developed in 1965, which introduced many of the features that are still present in modern shells, such as command history and tab completion.
In the 1970s, the Unix operating system was developed. It’s the parent to many of the modern operating systems we use today like macOS and Linux. Unix featured the Bourne shell (sh) as its default CLI which became the foundation for many other shells developed in the following decades, including the C shell (csh), the Korn shell (ksh), and the GNU Bourne-Again shell (bash).
In the 1980s and 1990s, personal computers and graphical user interfaces (GUIs) led to a decline in CLIs. However, these interfaces remained popular among developers and administrators due to their flexibility.
CLIs are widely used in many areas of computing such as:
- web development
- system administration
- data science
- game design & testing
Many modern CLIs, such as the Windows PowerShell and the Azure Cloud Shell, have been developed to make it easier for users to interact with operating systems and perform network tasks like SSH.
Cracking the Shell
With a better grasp of CLIs, we can start to understand what exactly happens when we use them. The easiest way to do that is by thinking of them in two parts; the terminal and the shell.
The shell acts like the brains of a terminal application. It takes the string you enter as a command and parses it into an executable. It allows a user to navigate the files of an operating system, run programs, start scripts, and perform other tasks on the computer.
In Unix-based systems, such as Linux and macOS, the default shell is often set to Bash (Bourne Again SHell), but there are other options like Zsh and Fish. The terminal provides an interface that allows the user to type commands, the shell executes them and allows you to view the results.
Without a shell, your terminal would be closer to a word processor than a mighty programming tool.
Harnessing the power of a shell allows you to do some pretty amazing things. Here’s an example:
The macOS terminal allows you to use a wide variety of Linux-style commands. One of my favorites is cat (concatenate.) It reads data from a file and gives its content as output.
The command takes one or more filenames as arguments and displays the contents of each file:
That’s a lot of code to look at in the terminal!
If you wanted to know how many lines of code, you could tell the shell to execute a second command after receiving the output from cat.
You can accomplish this by using the pipe | character following the filename:
Then use another one of my favorite commands, word count (wc) following the pipe:
The wc command is used to find the number of lines, words, and bytes in the files specified in the arguments:
Now we can see that the index.js file from my last blog project has 91 lines of code, 327 words, and 3046 bytes.
Looking Behind the Curtain
When a command is executed in the terminal shell, it goes through a series of steps before the command is completed. The process begins when the user types a command and hits enter. The terminal shell then interprets the command and checks to see if it's a built-in command or an external command.
When you execute a command in the macOS terminal, the following steps happen:
- The terminal interprets the command and checks if it’s a built-in command or an external command.
- If the command is a built-in command, such as ‘cd’ or ‘pwd’, the terminal will execute the command directly, without the need to spawn a new process.
- If the command is an external command, the terminal will check if the command is in the system’s PATH. (The PATH is a list of directories that the terminal checks for executable files.)
- If the command is found in the PATH, the terminal executes the command by spawning a new process. If the command is not found in the PATH, the terminal will return an error message indicating that the command could not be found.
- When a new process gets spawned, the terminal creates a new process object and assigns it a unique process ID (PID). (The process object contains information about the command, its arguments, and the environment it’s running in.)
- The terminal then passes the process object to the operating system’s process scheduler, which assigns the process to the CPU for execution.
- Once the process is executed, the terminal will wait for the process to complete and then return the output to the user. If the command produces any error, the error message will be displayed in the terminal.
To find a list of processes with the macOS terminal type:
ps -ax
To end any process on the list type kill PID#
Learning to use these tools will help you navigate challenging situations throughout your career.
The PATH variable was a particularly difficult monster for me to tame while attending Bootcamp. Installing Node.js with homebrew (instead of using npm install -g) created some major issues with the PATH location for my project dependencies on an M1 MacBook. Connecting to a PostgreSQL database with Ruby on Rails forced me to find and kill many processes while attempting to deploy to Heroku.
Curtain Call
With a deeper understanding of CLIs, you can truly appreciate the hidden world that exists on the other side of your enter key. The invisible processes powering each request are vital to the tasks we rely on each day.
The next time you type “cd” or “ls,” think of how far we’ve come from the early days of the command line interface and how these tools continue to be simple solutions in a complex world.
Resources:
https://nodejs.org/dist/latest-v18.x/docs/api/process.html
https://www.postgresql.org/docs/
Sources:
https://en.wikipedia.org/wiki/Compatible_Time-Sharing_System
