Introduction to Unix Command Line

10 min readFeb 3, 2022

Command vs Shell vs Terminal
Structure of a command
Getting help with man pages
Redirection & Piping
Move faster in commands
Signal
File system
Environment Variables
Process
File permissions
Alias
Bash script
Curl
Symbolic links
Process JSON files with jq
Customize command line experience
Other helpful tips

Command vs Shell vs Terminal

Commands are simply texts but when we press Enter in terminal.

The text is interpreted by a computer program called the shell. A shell can also be referred as command line interpreter. Some popular shells are Bash, Z Shell, C Shell etc.

Terminal is a wrapper program that runs a shell and lets us enter commands and see output.

Structure of a command

command_name options input

Example:

ls -a -h -l Desktop

Option in single - can be chained:

ls -ahl Desktop

A command accepts options and input to work on. Each command is a small program. command_name lets shell know which program to run. The shell will first search that program in shell path. A shell path is list of folders that contain these commands. we can find path for commands using:

echo $PATH

It would output something like this:

/Users/administrator/.nvm/versions/node/v16.13.1/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

The shell will look to the leftmost folder and look for file called echo Then, it would move to next and so on until it finds the file. If it finds the file, it executes command echo , else it returns error message.

We can add other directories to PATH variable to ask shell to look into those directories for commands to execute.

We can also find a folder where command resides using which command.

which cat
which node

Getting help with man pages

man refers to manual structure. Unix systems come with manual pages that provide detail about commands. Understanding how to use man command can help us efficiently use unix commands with ease.

Let’s understand how to use ls with man command.

man ls

This command should output the following man page:

The syntax for option is crucial to running a command. Please refer to the table below for more information.

Example:

Since, the options for ls command are all in [] brackets, all options are optional and doe not require mandatory option. So we can use ls without any option.

[file ...] denotes we can pass a file . Please note that everything in unix can be considered as a file. So, directory can also be considered a file and file should accept directory name as well.

... in [file ...] denotes that file can be repeated.

We can also use man command to find a command. Suppose we want to find a command that removes a directory:

man -k remove a directory

It would output:

We can see rmdir(1) listed there which is our command. Then we can run use man to see its detail and use the command.

man rmdirrmdir Music

Redirection & piping

Types of output & input

There are two types of outputs:

Standard output

When command executes successfully, the output from the command is on channel called standard output. By default, standard output is displayed in the terminal.

Standard error:
When command executes with error, the error message is on channel called standard error.

In unix, there are two ways a command can get input:

Standard input

It accepts text as its input. We can use cat to accept standard input.

Command line arguments

Inputs are passed as command line arguments.

ls -a

Redirection

Redirection is a way of changing the way from where command reads input from or where command sends output.

Syntax

Standard output is numbered as stdin(1), standard error is numbered as stdin(2) and standard input numbered as stdin(0)

Redirection to file

1> Or > : Standard output


# Overwrite
cat > file.txt# Append
cat >> file.txt

2> : Standard Error

dog 2> error.txt
dog 2>> error.txt

We can use 2> /dev/null to filter out the errors so that they will not be # output to our console.

0< Or < : Standard Input

cat < file.txt

Redirection to program with piping

In Unix, output of one command can become input to another program .Piping takes standard output of one command and connects it to standard input of another command.

ls | cat
ls | lessls | cat > output.txt# Search from history 
history | grep aws

xargs command

If we try to pipe to commands like echo , the output will not get piped.

# Will not display anything
ls | echo

There are commands which do not accept input. These commands only accept command line arguments. The xargs command allows us to convert piped data into command line arguments for commands that only accept command line arguments so that we can continue piping without breaking our flow.

ls | xargs echo
cat files-to-delte.txt | xargs rm

Move faster in commands

In MacOS, to enable Opt key to move back and forward, we have to enable a setting:

Go to Preferences from Terminal → Go to Keyboard tab → Check Use option as Meta key

However, this only works with MacOS default terminal. In order to make it work with iTerm, we have to add custom key binding.

Reference: Stackoverflow

Preferences > Keys
Click the plus

move forward one word

option+right
send escape sequence
f

move back one word

option+left
send escape sequence
b

Signal

A signal is a notification that is sent to a program. A program can receive a signal and do something in response to the signal.

Here are few important signals:

SIGINT Ctrl + C: Tell program to interrupt.
SIGQUIT Ctrl + D: Exit in bash prompt, python shells.
SIGKILL: Interrupt program immediately. ie kill command will send SIGKILL signal to the program to exit immediately without cleaning up.

File system

Everything in the system can be considered as file. Files are structured as trees.

Wildcard

A wildcard is a symbol or set of symbols that can be used used to substitute other characters.

Working with files and directories

Note: mdfind or locate uses previously built database to search for files whereas find traverses through file hierarchy.

Use cases:

# List files starting with S
ls S*# List files with tsx extension
ls *.tsx# Creates A.txt, B.txt & C.txt
touch {A,B,C}.${tsx}
# Creates directories Jan_2021, Feb_2021, Jan_2022 and so on
mkdir {Jan, Feb}_{2021..2023}# Removes files starting with d and ending with r
rm d*r# Remove all files recursively
rm -r *

Environment variables

Environment variables are dynamic values stored in the system that can be used by applications used in shell. AWS CLI and other applications can make use the environment variables.

To list environment variables stored in our system, we can use:

# List environment variables
printenv# Print value of environment variable
echo $HOME# Check if a variable is an environment variable
#printenv VARIABLE_NAME

We can create a new environment variable as below:

export API_KEY="334343"echo $API_KEY

In order to append to existing environment variable, we can use following command:

export PATH=/usr/node/bin:$PATH

Environment variables created with this approach only persist in the current session.

In order to make environment variables persistent across sessions, we can declare the variable in shell specific configuration file ie .zshrc for Z Shell

# Add following line in ~/.zshrc
export API_KEY=14343534

To load new environment variables in the shell session:

source ~/.zshrc# Outputs 14343534
echo API_KEY

We can use unset command to remove environment variable.

unset API_KEY

Process

A process is an active instance of a program.

# What we as user is running
ps# What everyone is running
ps aux

We can use & to make a process run in background

# Run a process in background
sleep 100 &# List running process
ps

And We can use kill pid to stop any running process.

When a process is running, we can use Ctrl Z to suspend a process so that we can interact with shell to do other tasks. jobs command lists the suspended processes. And we can hit bg %NUMBER to continue the suspended process. The NUMBER is a number assigned to suspended process that can be retrieved from jobs command.

File permissions

Unix has three types of owners:

user (u)
group (g)
others (a)
Read (r): Gives authority to to open & read content of a file.
Write (w): Gives authority to edit , remove or rename a file.
Execute (x): Gives authority to execute a program.

Setting permissions with chmod

chmod command is used to change the permission flags on existing files. It can be invoked in two ways:

With Symbolic representation of flags

With octal values representing permission flags.

The octal values have following representation:

Using table, we can deduce that:

777 = rwxrwxrwx

So, if we want to provide all permissions to user, group and others, we can run:

chmod 777 file

Alias

We can use alias to simplify typing command repeatedly. To see available aliases, we can simply run:

alias

To define an alias, we can use the following syntax:

alias app="cd ~/Projects/repos/app"

Using this approach is not persistent across sessions. In order to make alias persistent, we can declare the alias in shell specific configuration file ie .zshrc for Z Shell

# Add following line in ~/.zshrc
alias app="cd ~/Projects/repos/app"

To load new alias in the shell session:

source ~/.zshrc

Then, we can simply type app command which is equivalent to running:

cd ~/Projects/repos/app

Bash script

Bash script is plain text file which contains series of commands. We can use bash scripts to run multiple commands together, perform task automation etc

We can create a bash script file as below:

#! /bin/bashecho "Unix command line is amazing"mkdir folder
cd folder
touch file.txt

#! is called shabang. The first line denotes that it is not a normal file and needs to be interpreted as a bash script.

We can execute this script as below:

bash script.sh

If we are creating a python script, the header line will probably be:

#!/usr/bin/pythonprintf("Hello from Python")

Simplify running scripts

We can also execute the script using ./script.sh With this approach, program can determine which interpreter the script is requiring. However, running ./script.sh will result in Permission denied error. We have to provide execute permission to the file.

chmod +x ./script.sh# Now, it should run successfully 
./script.sh

Curl

Curl command can be used to transfer data to and from the server. It can also be used to test Restful apis.

Symbolic links

A symbolic link or symlink is a special file that links to another file or directory .

# create symlink to a file
ls -s ~/files/file.txt file.txt# create symlink to directory
ln -s ~/documents/projects ~/projects

We can check if the symlink was successful using:

ls -l file.txt
ls -l ~/projects

And we can remove symlink using:

unlink file.txt
unlink ~/projects

Process JSON files with jq

jq is a lightweight and flexible command-line JSON processor. We can use it to slice and filter and map and transform json files with ease. Here are some use cases that might be helpful:

Pretty print JSON in terminal

jq '.' package.json

Retrieve a value from key JSON

jq '.name' package.json
jq '.scripts.start' package.json

Customize command line experience

Use 3rd party terminals

While the default terminal can be quite good, the 3rd party solutions can provide new features and functionalities to make us master with split screens, key mappings and customization options. I can recommend the following terminals for different platforms:

MacOS: iTerm2

Windows: Windows Terminal

Ubuntu: Guake

Enable color syntax for vi

Create .vmrc file in user directory

cd ~
touch .vmrc

2. Edit .vmrc file with following content

syntax on  
colorscheme desert

Upgrade to better terminal experience with Z Shell & Oh My Zsh

Z Shell is an extended version of the bash with support for new features, plugins, themes and customization options. Oh my zsh is an awesome framework for managing the Zsh configuration. It has an impressive collection of plugins which we can use to extend and customize our terminal to our liking.

Here are my favorite plugins:

zsh-autosuggestions: Provide suggestions of previously used command
git: Improve git terminal experience
zsh-syntax-highlighting: enables highlighting of commands whilst they are typed
web-search: Enables searching with Google, YouTube etc from terminal
colored-man-pages: Adds colors to man pages

Other helpful tips

Use pbcopy & pbpaste for copy and paste (MacOS)

# Copy contents of a file
pbcopy < ~/.ssh/id_rsa.pub# Paste contents
pbpaste

Use VS Code as default editor for git

We can leverage VS code to edit git commit messages by setting VS Code as git editor.

git config --global core.editor "code -w"

Use htop to manage processes

Htop is an interactive process viewer. It is similar to Activity Monitor on MacOS. We can see our memory usage, troubleshoot and kill processes slowing down the system with ease.

Use networkQuality to check our internet speed (MacOS)

networkQuality gives summary of internet upload / download speed and responsiveness.

Make your mac read out texts or files

You can use say command in MacOS to read out texts & files.

say "Unix is awesome"say -f package.json