Understanding & Executing - Linux Command Line - Part 2

The second part of the walkthrough of the Linux (& macOS) command line

Part 1: https://medium.com/@AndreasSterneer/unixcommandline1-76e1d6721614

Introduction

This will be the second part of my walkthrough of the Linux & macOS command line. If you haven’t read the first part I recommend you go read it before continuing to read this one. However, if you feel confident that you know the basics of the command line you can just continue reading.

In this post we’ll be looking at more advanced commands & go deeper in how the command line & Linux works. As I mentioned before, this is the second part of my walkthrough, I will not explain everything as thoroughly as I did in the last post to avoid unnecessarily long explanations (I will only skip the explanations of things I went over in the last post).

Note: I recommend you keep reading even if you don’t understand something at first as chances are, it will make sense later when you get more context.

Quick Recap

If you haven’t read my previous post and don’t intend to read it, here’s a quick recap of what I went over and also what I will assume you know.

• How the command line works together with the kernel
• How the filesystem works
• How command flags work
• Listing content, changing directories & see your location in the filesystem
• Creating and deleting files & directories
• Standard streams
• Moving & copying files
• Searching & locating files, directories and file contents

If you’re unsure about what these are/mean or works I suggest you go to the first article & make sure you understand everything I mentioned in this list.

File & Directory Permissions

So first up we got permissions. This was one of the more difficult things for me to fully grasp when I was learning the command line myself but I’ll try to keep it as simple as possible.

Now there are 3 classes you should know of before we continue; User, Group & Other. These are the classes that the users on your computer (including you) belong to. The classes you belong to changes depending on where in the file system you’re located & which files/directories you’re trying to do something with.

• User - File owner
• Group - Users who are members of the file’s group
• Other - Users who are not the owner of the file or members of the group
• All - All of the above

So first of all I want you to go to your desktop (or a folder where you have a lot of files) in your terminal & run the command

ls -l 

to the left you should see something looking like this:

These are the permissions for each file & directory. It may look strange now, but it’s a lot easier than it looks.

So to start off, there are 10 hyphens/spots available, the first one let’s you know if it’s a directory or not; By showing the character d if it is a directory & a hyphen if it’s a file. As you can see from my screenshot, there are 5 directories on my desktop & 5 files. After the first hyphen you should be able to see a pattern. Since there are 9 hyphens/modes left, you can split them by 3. This should leave you with 3 parts, each having 3 modes.

So the three parts are actually the three classes I mentioned earlier (user, group, other - in that very order). And the three hyphens/modes are the permissions (r, w, x / read, write, execute).

directory/user/group/other = -/---/---/---
- /rwx/rw-/r--
d /rw-/r--/--

chmod

They way you can change these is by using the chmod command. Syntax:

chmod <class>[+-=]<mode(s)> <file(s)>
chmod g-r image.jpg //revokes group permission to read the file
chmod ug+rw note.txt //adds user & group permissions to read & write
chmod o-rwx,u+rwx file.txt //revokes group other permissions to read, write, execute & also adds user permission to read, write & execute

Note: You can’t use any spaces when using a comma , to apply different permissions to different classes as shown in the example above.

Numerics

There’s only one more thing I wish to go over regarding permissions here & that’s setting & changing permissions with numbers rather than changing them with characters in the example above. It may appear as more difficult & complex but I still recommend that you learn this way of doing it as it’s both easier and faster once you get the hang of it.

You will still be using the command chmod to change permissions, however you will do it with numbers. In order to understand this part you will have to understand the basics of binary.

Now for permissions, a 0 would simply be a hyphen & a character/mode would be a 1. The permissions are set with the decimal total, rw- is therefore 110 = 4+2+0 which is equal to 6.

Binary to decimal (111 = 4 + 2 + 1 = 7)
Permission Number | Permissions Text | Permission Mode
| 7 | read, write and execute | rwx |
| 6 | read and write          | rw- |
| 5 | read and execute        | r-x |
| 4 | read only               | r-- |
| 3 | write and execute       | -wx |
| 2 | write only              | -w- |
| 1 | execute only            | --x |
| 0 | none                    | --- |
777 = rwxrwxrwx
707 = rwx---rwx
534 = r-x-wxr--

So -rw-r--r-- is 420 for the user class, 400 for the group class & 400 for the other class as well, which is equal to 644.

Another example would be -rwxrw-rw- which would be 421 420 420 which is equal to 766.

Redirection & Pipes

Standard Output

Let’s move on with redirecting standard output with this operator

>

Example:

cat > fruits.txt

Will create a file with the name fruits and let you enter the data you want to be displayed.

head -n1 <file>

Will “flow” the first line of the file’s data to it’s default output, however, we can use the > operator to have the data flow to a new file.

Therefore,

head -n1 fruits.txt > fruits-2.txt

Will take the first line of data and “flow” it into the new file rather than printing it on the screen. Keep in mind that this will not affect the file we take the data from, it will only redirect the data we where given to another location than the screen.

We can also pass the data of several files at once, for example:

cat fruits-2.txt fruits-3.txt fruits-4.txt > fruits-5.txt

Will pass the data from all files behind the > operator into the file fruits-5.txt
Keep in mind that this type of redirection will always overwrite all data in the file if the data is to be redirected to an already existing file.

The Append Operator

In order to avoid the mistake of overwriting contents of an existing file you can use the append operator

>>

Example:

 cat >> fruits.txt

will append the data you enter into the fruits file. We can also append several files into an existing file or merge the data from several files into a new file, for example:

cat fruits-2 fruits-3 >> fruits-4

Will append the data from the files behind the append operator into fruits-4 Either just appending the data to the existing file or create a file with the name we entered if a file with the name doesn’t already exist.

Standard Input

Standard Input works essentially the same way just the other way around. Instead of redirecting the output of something we can redirect input from a file to another command. Simply put: inserting data to a specified target.

mail john.appleseed@email.com < KeynoteNotes.docx

Will send a mail with all the data from KeynoteNotes.docx to john.appleseed@email.com.

Standard Error

If you run a command and wish to save the error messages to a file you can use the standard error operator

2>          // Only redirects the error messages
&>          // Redirects both output and error messages

So if you run

find / -name "*.pid" 2> error.txt

In your terminal the error messages will not be displayed but put in your file error.txt (if you don’t have a file with the specified name, a file with the specified name will be created automatically).

I recommend you to read this answer on Unix & Linux (StackExchange) to learn about more operators.

The heredoc

The “here” document is a unix term for a temporary document that preserves tabs, spaces, line breaks.

Syntax:

<program/command> << <document name>

Example:

cat << EOF

The heredoc launches the program left to the operator & creates the document. The heredoc can have as many lines as you want, you close the heredoc by writing the name you chose for it when you created the document. In this case, that would be EOF.

When you close the document it will send all of the data except the closing line to the standard input of the program on the left of the operator.

The here document does environment variable expansions such as:

my default shell is $SHELL

which will print (for me) as

my default shell is /bin/zsh/ 

It also does variable substitution,

my account name is `whoami`

will print (for me) as

my account name is andreas

Pipes

Pipes (in short) is sending the output of one command as input to another command. The pipe operator is a straight line ( | )where the command to the left standard output is sent as input to the command on the right side of the operator.

Lets say we use the heredoc to fill with data that needs to be sorted, this would be done like this:

<<MUSIC | sort

when we end/close the heredoc, the sort command will be fed the data from the heredoc and sort it as well as printing it to your console.

Another example of using pipes would be to grep a string from the output of the ls -l command:

ls -l | grep fruit

this would give the output of all items in the directory that has “fruit” in it’s
name, such as “fruit.software” and “fruits-2.txt”.

DF Command

Before we move on to working with process there are two commands I wish to go over. First, the DF (disk space) command.

df prints a few lines with information about the disk space being used in the file system, however, it could be hard or time consuming to read long numbers, the h flag (which stands for human readable) simply makes the numbers easier to read— it reduces the amount of numbers and uses prefixes to avoid excessively long numbers (1K 243M 2G).

df -h

Tee Command

The tee command copies from standard input to standard output making a copy in zero or more files. So lets use the df command with pipes and the tee command to save the disk space output into a file:

df -h | tee diskInfo.txt

Working with process

The definition of process is the instance of a computer program being executed. A bit vague, I know, in other words, a process is whenever a command is run in the command line or when a program is started from the applications folder via finder.

PID = Process ID
PPID = Parent Process ID
TTY = The controlling terminal of the process
STIME = Starting time of the process
TIME = CPU time
CMD = The command the process represents
S = Sleeping Process
S+ = In the foreground of the controlling terminal

The ps command

ps stands for process status, which is exactly what the command does, running the command ps will show the process status.

Useful flags: -f (extra info), -o (customised output example: ps -o user, PPID, PID, state, command), -a (shows processes of other users).

Note: The ps command only shows a snapshot of the processes being run at the very instance the command is executed.

The top command

The top command shows periodically taken snapshots of a sorted list of information of processes running on our system.

To more effectively review the data displayed from top you can enter a question mark “?” while running top to open the help menu. The help screen can for example help you with how to sort which processes are using the cpu more than others.

You can use the flags when starting top to start top with certain parameters of which information to display, example:

top -s 5 -o cpu

will set the update interval to 5 seconds and show the most cpu extreme processes at the top.

Foreground & Background Process

There are two different types of processes, foreground process and background processes.

A foreground process is any command or task you run directly and wait for it to complete. […] Unlike with a foreground process, the shell does not have to wait for a background process to end before it can run more processes. — Juergen Haas

A foreground process is impossible to tell from commands like cd, ls and most other commands we’ve used so far because they finish in an instant, so to use a command that doesn’t die as quickly we’ll run a command to find all files in the root directory that ends with .pid

 find / -name “*.pid”

While this command is running you won’t be able to execute another command in your command line because it’s running in the foreground.

If you wish to kill this process before it finishes you can use ctrl+c to terminate it. However if you only wish to temporarily do something else and simply suspend the process you can press ctrl+z.

You can run a command in the background by appending the & operator to the end of a command.

find / -name “*.pid” &

The process will still send its standard output to your console, however, you will now be able to run a second command while the first command is still running.

Jobs

The jobs command allows us to inspect background and foreground processes. The number at the very left is the process / job ID.

The fg & bg commands

The command fg and bg works essentially the same way. They resume a suspended job in either the foreground or the background. Syntax:

fg %job_ID // resumes the process the foreground with the job ID behind the % operator
bg %job_ID // resumes the process in the background with the job ID behind the % operator