Linux 40+ useful command

Published in

Chen-Chih’s Portfolio Page

47 min readFeb 26, 2024

I have been writing several posts about Linux tutorials, but I think Linux command is pretty important. Today I would like to write the most common and useful Linux command. Please note I won’t mention all of them, there are many useful commands out there, but I will choose what I think is important. Please note the Linux distribution i am using is Ubuntu or Debian

So I will categorize it into this section:

- Common and Basic command (use often)
  1. cd: change directory
  2. ls : list your directory
  3. ln: as a symbolic link
  4. pwd: show your full path in this directory
  5. cat: reading a file
  6. vi : edit file and its flag options
  7. touch : create an empty file
  8. mkdir: create an empty directory
  9. cp: copy file
  10.mv :move file or rename file
  11. rm: remove or delete file and directory
  12 alias : custom command
  13. whoami :which user am I
  14. su: switch user
  15. man: manual page
  16 history: search for history command you use
- customize command on in bin or sbin(similar to alias)
  17 tail or head: File Viewing and Manipulation Commands 
  18. more, less viewing large data 
  19. > or >>: Input/Output Redirection Operators 
  20. grep: Text Searching Commands 
  21. SED text manipulation and editing file
- Set Permission file and directory
  22. set ownership and group name 
  23. Setting permission for file or folder
- System and Monitoring:
  24. uname:check kernel version 
  25. dpkg: package management 
  26. systemctl checking services
  27. system restart or shutdown 
  28. dh and du : Check Disk Space and file 
  29. ps:  check process running or PID 
  30: free , top, htop: resource for CPU and memory loading  
  31. kill: kill the process to exit program  
- Archive File Create Compress Extract
  32. tar : to create an archive
- Network command
  33. ifconfig: check IP address 
  34. IP address: show and set IP
  35.Add IP address 
  36. Ethtool: check your network status 
  37. Adding Routing 
  38. tcpdump : capture packet 
  39. Ping: check network connection 
  40. ssh and scp : remote and transfer files
- Storage command ¹
  41. fdisk check disk name
  42 mount 
  43 fdisk and parted : Create Partition 
  44. mkfs format disk
- Shell command

Common Command (Often used) ¹

1. cd: change directory
2. ls : list your directory
3. ln: as a symbolic link
4. pwd: show your full path in this directory
5. cat: reading a file
6. vi : edit file and its flag options
7. touch : create an empty file
8. mkdir: create an empty directory
9. cp: copy file
10.mv :move file or rename file
11. rm: remove or delete file and directory
12 alias : custom command
13. whoami :which user am I
14. su: switch user
15. man: manual page
16 history: search for history command you use
customize command on in bin or sbin(similar to alias)

`1. cd:` change directory ¹

if you want to change or go to a specific directory or folder just use the command cd <folder or directory> , this command is the same as the window cd .

cd is similar to chdir, they both do the same thing to change directory or folder, basley chdir is called by cd to change the directory.

Let me talk about if want to go to the upper level’s specific directory from the current directory, you can use the cd ../<folder directory> , or you can go to upper-level by cd ..

Notice when you use cd .. to upper level and access to another folder cd foldername, type two times. Instead we can use with one command go upper-level and access to folder cd ../foldername

2. `ls` : list your directory ¹

https://www.rapidtables.com/code/linux/ls.html

ls which also means list, this command lists all your directories or folders in their current location. So if you’re in /home/username then you use the ls which will list all your folders in this current location

you can also use ls -al which means listing all your directories and files including hidden files. The folder that starts with the . period are hidden files. You can also use the ll command which is the same as ls -al , which ll is been alias by default so both commands can be use to show all file. This command is often used, listing all your working directories including files, folders, and hidden files. Please refer below example

Linux files are case sensitive, which means if your file is set lowercase but you try to open it as uppercase will not be found. From below example i want to show the file HIDDEN which is uppercase, if i use ll HIDDEN it will be able to display the file information, but if i use the lowercase it will display not file or directory found. This tell us that linux all string are case sensitive which mean upper case and lower case are different.

But there’s some way you can ignore the case sensitive, I will show you two methods.

Method1: using grep(I will introduce in the below section), ex: ll |grep -i <file or directory> , which i means insensitive meaning ignore upper or lowercase.

Method2: You need to enable case-insensitive which will ignore upper or lower case.

enable:shopt -s nocaseglobdisable(default): shopt -u nocaseglob

In order to ignore case sensitive, you need to enable case-insensitive with the -s option as set

Note: In window also has this command but instead of ls it uses dir

3. ln: as a symbolic link ¹

A link is a special type of file that serves as a pointer or reference to another file or directory in the file system. There are two types of Link type, Hard Link as default, and Soft Link which we call symbolic link. Below is the syntax, hard link without soption, and soft link with s option.

syntax:
soft link:ln -s <sourceFile> <link_path >
hard link:ln <sourceFile> <link_path >

symbolic Link or soft link

Soft Links as symbolic links act like shortcuts in Windows, allowing you to access a file or directory through a different path. When you use the ls -l to list directories if you see location1-> location2this part is called a symbolic link.

You can remove symbolic link by unlink < symlink> or rm <symlink> symlink_to_remove>

different between Hard Link and Softlink:

Hard Link means the file points to the same inode as the original file /directory .
Soft Link points to a different inode original file /directory. The file that just stores the path of the original file and not its contents, so when you move or remove the original file the soft link will not work

From the below picture, you can see the soft and hard links the node id will be different after you create a link.

Node ID will be different:

Original File does not exist not work

In the below picture, I show you when I add links on both hard and soft links when moving the original code, the soft link will not work whereas the hard link file still works. This seems reasonable from above same node and different node, when we have different nodes no matter the file move it still link together.

4. `pwd:` show your full path in this directory ¹

This command is pretty useful when you enter a specific subdirectory you can use the command to show your full path, it will list your current location

5. `cat:` reading a file ¹

This command is the opposite of vi the edit command. Cat basely allows you to read a file, and will read all the data out. I will talk about the vi command in the next one.

But this command has one disadvantage that’s is if you file is long, it will cover all the content in your terminal which is pretty messy.

syntax: cat option <filename>

6. `vi` : edit file and its flag options ¹

vi command basely is to edit a file, vi filename but some people will use vim or nanorelated command. If you decide to use vim then you need to install the Vim editor sudo apt-get install vim .

If you’re Linux is text mode, then you need to know some of the edit mode’s parameters, I will explain i use the most often one:

Edit the file syntax: vi <filename> , ex: $vi filename

i insert mode, this is to edit or write the file.

esc : exit the mode and change mode
:wq! save and quit, you can use it separately w as save, q quit. you can also use the x it saves and quits, not many people know this command.
From the previous example insert, after editing it you can press esc and then press wq to save and quit.

Now you can check file is been saved, and you can use the cat command to read or vi the command to edit.

copy cut paste mode: this is an important part

ctrl+v: visual mode select cursor to select which string to copy or cut
y: copy
d: cut
p: paste
u: undo

whenever you want to copy or cut the string, you need to go to enable the visual block to press the arrow ← ↓→ to select text or string then press v copy or x cut the sting, then press p to paste.

Above picture I demonstrate how it works, for example, the copy example, I press enter to enter a new line to paste the text. After the new line press i to insert string and press esc to exit mode, then press v paste it.

When you want to cut or paste please make sure all modes are off, which means below should not occur insert or visual block.

Step1: vi filename #edit a file
Step2: enter editor , press i to insert and press enter
Step3: press esc to exit insert mode and move cursor or anchor to string to copy
Step4: press ctrl+v to enable visual block mode
Step5: select the text to copy or cut by arrow key 
Step6: press copy or cut #after press it wil auto exit the visualblock mode
step7: press v for paste

You don’t have to follow my steps, it’s just an example, indeed there are many ways to achieve. I hope this is a clear understanding of the copy, paste, cut, and undo in editor mode.

:set number display the number line of the file

let's edit a file $vi /var/lib/dhcp/dhcpd.leases specific file and display the number line in the editor, use the command :set number

/: Finding string in a file. When you edit a file you will find a specific string. we can use the n to find the next finding string.

go to a specific line in a file, there are many methods you can use

Method1: vi +<line> filename, ex: vi +10 filenameMethod2: vi filename and :line ex: :10
Method3: vi filename and press ctrl+g <currentline> -1 , for example you want to go to line 10, then you need to change the line to 9 like ctrl+g + 9, then it will go to line 10. The reason is that vi starts the counter from 1 not 0, it depends on some version.

7. `touch` : create an empty file ¹

when you use the touch command it will create or generate an empty file. You can also use the vi command to edit a file, but you have to save it or else it will not create the file.

8. `mkdir:` create an empty directory ¹

create an empty folder mkdir newfoldername , but when you want to create a directory in which the parent directory is missing then it will cause an error.

-p for parent: Creates missing parent directories
-v for verbose: Prints a message for each created directory

you can also create a multiply folder like this:
mkdir -pv /tftpboot/{images,kernels}

9. cp: copy file ¹

syntax: cp [options] source dest

different option:

ref:https://www.rapidtables.com/code/linux/cp.html

copy file: cp file file.bkcopy all files into the directory: cp *.txt dest
Copy directory into different directory:
method1: cp -av sourcedir dest_dir or cp -av . dest_dir
method2: cp -R sourcedir dest_dir
Copy all files and directories subdirectory: cp -R dev bak

if you use the cp command to copy a folder will occur warming, so in order to copy the directory or folder, please use the option command or the -R

warming message without the option

copy all files or specific file format, you can use the * or the . option. The . allow you to copy entire files or folders, and * allow you to copy specific file types.

10.`mv` :move file or rename file ¹

syntax: mv [options] source dest

You can think command as moving a file or rename as file. So if you do like this mv file file.txt, this will rename the file to file.txt. This is a powerful tool, so please be careful to use it. If you’re afraid of it, you can use the cp copy command.

move /rename the file or directory

rename file: mv test test.txt
move directory : mv test /home/chenchih/

move entire file to another directory

move only file with txt format : mv *.txt /newfoldermove subdirectory to directory: mv /folder/* /newfolder

you can add -v option to display move source to desstioantion. without it will not show anything as below.

11. rm: remove or delete file and directory ¹

If you want to remove the file you can use the rm command which stands for remove.

rm -rf filename: remove folder or directory. r for recusrsive and f for force.
rm folder: remove file

remove file with extension: rm *.<filetype> ex: rm *.tx

remove file contain pattern: rm *.txt*

12 alias : custom command ¹

This is very useful, you can customize your own command instead of using the read command, but you need to set it up. For example, if ls -al is hard to remember you can use the ll, the default already helps you add ls -al as ll so this is the reason they display the same thing.

you can check your alias which will list all the shortcut commands. Below you can see I added desktop path to the alias, which means when I press desktop it will direct me to the desktop directory, I don’t have to type the full path. This is pretty convenient especially when you deal with many long paths.

To set an alias there’s a couple of ways:

syntax: alias <command shorcut command>=<your custom command here>

Method1: alias desktop=’cd /home/chenchih/Desktop’this is not the best way, this only allows you to set it temporarily after reboot this alias will be gone.

Method 2: write the above into .bashrc a file, which is a hidden file. All the files start with a period are hidden files, so run this command ll to list all files including hidden files or directories or ll|grep . to list all hidden file

Now vi .bashrc and find the alias string like my file paste or type the above alias command inside like below on the underneath it.

However, there’s another way that is better which allows you to add into another file based on the record of your alias.

Method3 : create .bash_aliases a file to keep your alias. Due to .bashrc is a system file if you break this file system might have a problem booting, instead we can create this file.bash_aliases which will read your alias here.

when you go to .bashrc, there’ 's a code like below, which means if the file exists, it sources (reads and executes) the contents of ~/.bash_aliases into the current shell environment, if not exist then it not source and continue to execute .bashrc.

so let's create this file .bash_aliases and put your alias command in it, let me read my file of mine as below. As you know ipconfig is check ip address in windows, so let's add ipconfig which identical means ifconfig.

then after saving the file, run this command source ~/.bash_aliases . This command means activating the aliases and functions defined in ~/.bash_aliases in your current shell session.

Now if you run the alias ipconfig will add inside.

so recap to prevent modifying the .bashrc system file directly, we can use an alternative file called .bash_aliases . This avoids modifying or damaging system configuration, as .bashrc . By placing aliases and functions in .bash_aliases, we keep our customizations separate and minimize the risk of accidentally breaking the system configuration.

13. `whoami` :which user am I ¹

This command is not used often, but I still want to mention it when you log in to a specific account and forget this command allows you to show what’s is my current user.

When you use the sudo su it will change to the root user or other users, when you switch many users sometimes will not remember it, so this command will help you identify which user are I using right now.

14. `su:` switch user ¹

you can switch to root by sudo su , this is why when using Ubuntu we often use the sudo command, which means run by root. If you want to switch to a normal account you can do this command sudo su - username

15. man: manual page ¹

if you see some command you’re not sure with the command you can use the man <command> for help just like below: man tar or man tar -- help

16 history: search for history command you use ¹

This is a pretty useful tool I like to use, when you press the arrow up and down will show your previous command. But when you have pressed so many commands previously, the best way is to use the history command will list all the commands you press.

customize command on in bin or sbin ¹

If you want to create your customer command, you can use this method just create a file. This is similar to alias method, but you can customize it.

Let me show you an example, if I have a directory that records many logs and each log’s name will change or update like the below:

If I want to see the log I might use this command tail -f <path and filename> which both the path and file are long. So the solution to this is can instead of that command I can just type cdulog la will parse the right file will parse the cdulog l3 left file.

You just have to go to /usr/sbin or /usr/bin either one is ok, and create a file name anything you like that will be your command. In my case, I will create file as cdulog which essential is the command.

#go to directory /usr/bin
cd /usr/bin 
cd /usr/sbin 
#Step1: create file 
vi cdulog or touch cdulog
#step2: add the code into the file cdulog
tail -f <your full path>/*$1*

So let me show before and after will look like this:

original command: tail -f /var/gnb/elog/elog_gnb_cu_l3.0.2024….
customize command: cdulog l3Syntax: <command thefile your create> <variable you enter>

So from above you can see cdulog will use the tail to parse the string or variable you enter in the $1which will parse the specific path and file. Notice I enter l3 the file name contains l3 then it will use that file.

To shorten description: cdulog as command, and $1 as your file name

As you can see from a long command we can shorten it down this is pretty convenient especially when you have a long file name in a different location. This is similar to what alias will do, but this is more powerful and allows you to customize your command. Alias only allows you to add commands, but can’t customize.

Command with Operators and searching ¹

17 tail or head: File Viewing and Manipulation Commands
18. more, less viewing large data
19. > or >>: Input/Output Redirection Operators
20. grep: Text Searching Commands
21. SED text manipulation and editing file

17 .tail or head: File Viewing and Manipulation Commands¹

-f : Follow mode, keeps the command running and displays new lines as they’re appended to the file.
-n :Specifies the number of lines to display from the end of the file.

Tail: starts at the end of a file and displays a specified number of lines backward (default 10) or until the beginning of the file is reached.

Head: starts at the beginning of a file and displays a specified number of lines (default 10) or until the end of the file is reached.

It reads the last N line of the file, so the syntax is [tail| head] -f N which N is the last N line. So if I want to display the last 20th line, then I will use this: tail -f 20 . This is often used to see the log, we don’t want to display all of the log.

tail -f myfile.log : without specifying the N, default N=10, which means will display the last 10 lines, and continue display line if added.
tail -n 20 myfile.log : display the last 20 lines and exit
tail -f -n 10 myfile.log: : display the last 10 lines and continue display line if added

18. more, less viewing large data ¹

In the previous section the cat command is to view file, but it can only allow viewing the entire file, but this section introduces another viewing command to customize viewing. When reading a file too large to fit the screen, you want to display the file content by pages using the more or less command. more is the oldest, less is an improvement.

This happens when you are in text mode when you reading a large file or a command ifconfig, then you can use these commands to change to pages mode which allows you to use the arrow to switch up and down. So you can use with cat command of ifconfig like this:

less :display file page by page in an interactive view. You can also use the arrow up or down or page up or page bottom to do scrolling.
more: display file now screen at a time, scrolling with space or enter.

The difference between these two is that, less can move up and down a page, whereas more can only move forward and not allow them to move up. So less is the improvement of more.

There is a most command, but it seems like needs to install which default might not install. I haven’t used most commands, but browser on this command I saw some people mention this also, you can check this link.

ifconfig | less #or more
cat file | less #or more

Just think like when you open a file you can scroll down, but in Linux text mode, you aren’t able to use the mouse to scroll. For example when you cat a file that can’t fit one page, and you want to see the top content, then you can use this command. ifconfig is also a good example when running the command, when you want to check what’s your IP or MAC address, then use this less option ifconfig | less which will display like a page using the arrow to scroll up or down. I think it’s pretty useful.

`19. >` or `>>:` Input/Output Redirection Operators ¹

This Command is to dump one file to another file, often on writing a shell script. Ex: file1 > file2

syntax: command [>|>>] filename

Use > overwrite, when you want to replace the contents of a file completely.

Use >> append, when you want to add content to the end of an existing file.

# Create a file named "original.txt" with some content
echo "This is the original content" > original.txt
# output: 
# This is the original content

# Overwrite "original.txt" with new content
echo "This is the overwritten content" > original.txt
# output: 
# This is the overwritten content# Append to "original.txt"
echo "This is appended content" >> original.txt
# output: 
# This is the overwritten content
# This is appended content

20. grep: Text Searching Commands ¹

We you want to filter a specific string, file, or directory by string then you will use the grep command with the pipeline|. The grep command can be complex so I am not going to take in-depth, this is a good document you can reference with regular expression.

Linux is case sensitive, which means if your file is capitalized but you filter as lower case it will not be able to be found. You can use the grep -i option also mean insensitive. As you can see below when I filter ls |grep hello, no match but when I use the -i will find matching because it will ignore the case sensitive.

you can also filter the content of the file, i show an example of filtering the the *.txt file, which will display all related strings. As you can see the command cat <filename>|grep ‘match’> is same as grep |‘match’ <filename>

If you want to filter multiple files you can use like below example, which gives the same result as searching on both HELLO or test .

The e option is also the same as egrep, which uses extended regular expressions. This also means you can add an additional regular expression operator in it to filter like grouping (), and quantifiers like +, ?, and {}.

ls | grep -ie 'hello\|test'
ls | grep -ie 'hello' -e 'test'
ls | egrep 'hello|test'

There are grep, egrep, and fgrep you can use, if you are interested you can search on it.

grep: normal grep using basic regular expression for pattern matching
egrep: extended regular expressions for more complex pattern matching.
fgrep: Searches for fixed strings in text files, meaning it looks for exact matches of the specified string.

#egrep example:
egrep "pattern" file.txt #Searching pattern string for single file
egrep "pattern" file1.txt file2.txt #Searching multiply file
cat file.txt | egrep "pattern" 
find . -type f -exec egrep 'search' {} + #Searching recursively in directories

#fgrep example
fgrep "apple" file.txt#Searching fixed string for single file
fgrep "apple" file1.txt file2.txt#Searching multiply file
cat file.txt | fgrep "apple" 
find . -type f -exec fgrep "apple" {} +#Searching recursively in directories

21. SED text manipulation and editing file ¹

sed: Simple tool for text manipulation and editing file content based on the pattern. Need to be careful when using it because a mistake might damage a file. I am not going to dive in depth SED, but I will explain some common examples. SED is a powerful tool that allows you to search and write into files. There are some great post on SED.

This topic contains many examples, I have marked a ² this link will direct to here in case you want to see a different topic.

The feature allows you to do:

Reads text line by line.
Uses regular expressions to find and replace patterns.
Can insert, delete, or modify lines based on patterns.

syntax: sed [-OPTION] [ADD1][,ADD2] [COMMAND] [/PATTERN][/REPLACEMENT]/[FLAG] [FILE]

option:
-i: Edits the source file directly
-e: Allows you to combine multiple sed commands in one go.
- n: Doesn’t write changes to the file, just shows or displays the result pattern. Safer for testing.
Flag:
g: replace all substitution on all occurrences of the pattern
p: print the pattern space only if the substitution was successful
w<file>: work just as pflag, but the pattern will written to <file>
I: ignore case-sensitive upper or lower case
substitution command:
s/pattern/replacement/: Substitute occurrences of pattern with replacement. Search and replace
c/pattern/replacement/: Change the entire line if it matches the pattern. Replace line
/ : It specifies the delimiters
other commands :
a or i or a\or i\: Insert text before the current line.
d: Delete the current line.
$: The last line of the file.
=: Print the line number of the current line.

Replace string if match using g flag ²
Search the string related pattern and replace it

#Ex1 default without g flag
# Replace Upper Case W to lower case
echo 'What is your name' |sed 's/W/w/'
#output:
# what is your name

#Ex2 set g will replace all match 
echo 'aaadddaa aaadddaa' | sed 's/d/b/g'
# aaabbbaa aaabbbaa

#Ex3 replace ONLY N position or address
echo 'aaadddaa aaadddaa' | sed 's/d/b/5'
#output: 
#aaadddaa aaadbdaa

#Ex4 replace start from N position
echo 'aaadddaa aaadddaa' | sed 's/d/b/5g'
#output:
#aaadddaa aaadbbaa

Ignore case sensitive using I flag ²
So when using the I flag it will ignore the case sensitive, in the below example my target search uses lowercase, but my original string is uppercase. It ignores the case sensitive, so it still replaces the string.

echo 'my name is Chenchih' | sed 's/chenchih/Chen-Chih/I'
#my name is Chen-Chih

Print or display pattern match with p flag ²
The -p flag in sed tells the command to print the modified pattern space after applying any editing commands.

# ex1 using -p with -s to replace the string
echo 'My name is chenchih' | sed 's/chenchih/Mark/p'
My name is Mark #print the modify 
My name is Mark # print after modify 

#ex2 using the -n just diplay the line if match 
 echo -e 'Line1\nLine2\nLine3' | sed -n '/[13]/p'
#Line 1
#Line 2
 echo -e 'Line1\nLine2\nLine3' | sed -n '/[3]/p'
#Line 3

using n andp option will allow us to print specific line

Write into file pattern match with w flag ²
ifconfig | sed 's/inet addr:/{&}/w ipaddress.txt'you can also add a multiply flag: ifconfig | sed 's/inet addr:/{&}/Igpw ipaddress.txt'

Example1: Replace pattern and modify in a file²

1.1Replace matching word: sed -i 's/oldstring/newstring/g' filename1.2Replace matching character: sed -i 's/A/a/g' file.txt

This will replace matching strings or characters and write them into a file when using the i option. In 1.1 it replaces the word of the entire content in a file and writes. In 1.2 it replaces the character change uppercase A to lowercase.

Note: When using i option it’s pretty dangerous and will modify the file, you can instead use the -n to print the pattern.

So let breakdown the options and flag we use here
i: Edits the file in place.
s: Searches for the pattern.
g: Replaces all occurrences of the pattern.

you can also replace with all files in the directory, so below replace all text files: sed -i 's/oldstring/newstring/g' *.txt

if you don't want to replace all occurrences instead with specific ones you can use this syntax: sed -i 's/oldstring/newstring/N occurance' filename , replace g with a specific occurrence. In the below example, I want to replace only the second one, so I place 2.

echo 'hello world hello world' | sed 's/world/World/2'.Only the second matching will change. sed

Example 2 : Search specific line and string patterns and replace specific strings ²

This will not be written into a file, instead it will only display. It a great practice for testing, instead of writing like the above example

search line 1~3 of a file and replace the match string:
sed -e'1,3s/[Tt]he/THE/g'MyFile.txt

if you want to replace only a specific N line just put N line like this
sed -e'2 s/../g' filename , 2 is my N line, which means only search at line 2 and replace the second line.

So let breakdown the options and flag we use here
e:Edit and allows you to specify multiple editing commands on the same command line
s: Searches for the pattern.
g: Replaces all occurrences of the pattern.

Example 3 delete and add line d and i option ²

i: insert string after N line, and d: delete line

Delete a line: (deletes lines matching the pattern)
In the below example if I want to remove the first line, just type line with d , as you can see apple is been removed
sed -e '<line>d' file.txt

you can also exclude specific line sed '1,3d' file.txt the first three lines will not display

Add a line: (inserts text after line) ²
You can insert a string after a specific line like below, but it will not write into file. It will show pattern replace result.
sed '<line> i<string>' <File>

Example4: Replace string write into a new file ²
Replace line1 ~ line2 string installed to install and write into a new file. This method is better than i option which will edit and modify the file, but this will modify with the new file.
cat <file> | sed '<add> <add> s/<string>/<repalce>/g' ><newfile>

Example5: Replace number or digit with a sting ²

If you want to search on a specific line start the string from a specific sting and replace number to the string like below:
sed '<line>,/<start string>/ s/<search pattern>/<replace>/g' <file>

other example:²


#replace name to postition
echo 'What is your name'| sed 's/\<name\>/position/g'
#What is your position

# replace multiply condition
echo 'My name is chenchih' | sed 's/my/My/' | sed 's/chenchih/ChenChih/'
#My name is ChenChih

#replace /slash to \
echo '/home/test/' | sed 's#/#\\#g'
#\home\test\

#replace start sting in N line
sed '2,$ s/^v/D/' status.txt 
#before: 
#vsftp installed ok
#after output:
#Ｄsftp installed ok

#replace end string .com to org
echo "www.google.com" |sed 's/com.*/org/g'
#replace start string http to https
echo "http://www.google.com" |sed 's/^http/https/g'
#https://www.google.com

#replace space from hello world to hello-world
 echo 'hello world' |sed 's/hello\ world/hello-word/g'
#hello-world

#change end string to specfic string
echo 'name: tttt' | sed 's/name:.*/name: Linux/g'
#name: Linux

Change permission and ownership ¹

This concept is pretty important to change the permission of a file or folder name. If you ever write a bash script, then without changing it will not be able to run. This is a good post for permission commands.

chown : change file owner and group
chgrp: change group ownership
chmod: to change the permissions of a file or directory

Let’s understand three types of permission and ownership

22. set ownership and group name
23. Setting permission for file or folder

22. chown and chgrp : set ownership and group name ¹

change ownership name
chown name <filename> chown name <foldername>
change groupname
chgrp groupname <filename> chgrp groupname <foldername>

23 Setting permission for file or folder ¹

I have marked a ² this link will direct to here in case you want to see a different topic.

Before introducing chmod this command, need to know the group type and permission type as below:

Groups type:

u owner: permissions will only apply to owners and will not affect other groups
g group: assign a group of users with specific permissions and will effect within group
o other , everone, or all user:will apply to all users, and as a result
a: all

Permission type:

rread: allows a user or group to view a file
w write: allows the user to write , delete or modify a file or directory.
x execute: allow A user or group with execute permissions can execute a file or view a directory.
-: this is the file type: so if your file is file will display -, directory as d,and symbolic link as i

Below is a picture of understanding the group type and permission type.

So from below, I gave them all ownership permission rwx, on file:4 and directory:test1234. This means both the file and directory permission is read write, and execute, also anyone can access or write it.

change directory or file permissions as below command:

syntax: chmod <option> <filename or directory>

Method1: use chmod with rwx option ²

If used without specifying a specific user, group, or other, it applies to all three categories: owner, group, and others.
chmod +rwx <file or folder> : add permissions to all ownership
chmod -rwx <file or folder> :remove permissions to ownership
chmod +x <file or folder> :add executable permissions to ownership
chmod -wx <file or folder> :write and execute permission to ownership

Please note when I use the chmod +rwx , this mean owner, group, and other have full access however, the other write positions is — not granted. This mean it’s doesn’t have access to write or delete a file. Just imagine you login into an account, and you’re not able to delete or modify other people's files. So when we set full access it will look like -rwxrwxr-x .

When you add chmod -rwx, the file will not have any permission even reading, please refer above third command.

group and other: ²
chmod g+w <file or folder>:group add write permissions
chmod g-wx <file or folder>:group remove write and execute permissions
chmod o+w <file or folder>: other user write permission
chmod o-rwx <file or folder>:other user remove permissions

as you can see from the above example when adding a group to permission, only the group position will change. The same as adding or removing the other, only the other position will change. The example I showed above is pretty intuitive i show examples for commands and results.

set same permission to all ownership ²
chmod ugo+rwx <file or folder> :assign read write and execute to everyone
chmod a=r <file or folder> :assign read to everyone

Method2: use chmod with numeric option

0 = No Permission
1 = x for Execute
2 = wfor write
4 = r for Read
0 =---
1 =--x
2 = -w-
3 = -wx
4 = r--
5 = r-x
6 = rw-
7 = rwx

so to to give all access to file read write execute, we add 1+2+4, which is 7, so you will often see people use chmod 777 , you add it up using the above number option. So let me show you common numeric example used often:

chmod 777 :(-rwxrwxrwx) Give read, write, and execute permissions to the owner, group, and others.
chmod 700: (-rwx------) Give read, write, and execute permissions to the owner only. This command restricts all permissions to only the owner of the file or directory.
chmod 755:(-rwxr-xr-x) Give read, write, and execute permissions to the owner, and read and execute permissions to the group and others.
chmod 327 :(--wx-w-rwx)3 means the owner has read and write permissions.2 means the group has write permissions.7 means others have read, write, and execute permissions.

3. Special file permissions ²

if you see other than x in the execute position(the last position) then it’s a special file permission(refer above picture). Below are the descriptions of each description:

s in owner position as SUID Bit: User ID. Only set executeable file, which allow user to run the file with the same permission as the owner of the file.
s in group position as SGID bit: group id. Only set executeable file,which allow user to run the file with the same permission as the group of the file.
t in execute position as a sticky bit: sticky. Only set directory, and only the owner of the file or root have permssion to write or delete the file. This mean With the sticky bit set, users can upload files to /var/upload and modify or delete their own files, but they cannot delete files owned by other users.

System and Monitoring ¹

24. uname:check kernel version
25. dpkg: package management
26. systemctl checking services
27. system restart or shutdown
28. dh and du : Check Disk Space and file
29. ps: check process running or PID
30: free , top, htop: resource for CPU and memory loading
31. kill: kill the process to exit program

24. uname: check kernel version ¹

uname -a check kernel and version

25. dpkg: package management ¹

if you want to check or package you can use the below command or remove it.

dpkg -list <pkg-name>: check specific package install or not
dpkg -s <pkg-name> | grep -i version : check version
dpkg -i <package.deb>: install deb package
dpkg -r <services> or dpkg --purge <services> : remove package or service

From above you can see if I can use the dpkg -l to list all the packages that have been installed in the system, however, we can use grep to filter which package is been installed. I also show how to remove deb packages with -r the option. In my example basely I want to remove iperf3, so I need to use show all dpkg to list the name of iperf3 and then remove it. To remove the the services or package, need to know the name same as shown in the list.

26. systemctl checking services ¹

This command are able to show your services status, and you are able to start, restart or shut down services.

Syntax: systemctl <start|restart|status|stop|> <services name>
Ex: sudo systemctl start nginx

Listing Targets:
This command systemctlis powerful which list various of ststemd units, services, socket, mounts, and many more. So when you enter just systemctl it will list everything, you can filter specific services like systemctl --type=service --state=running . This command will filter type as service, and the state as running.

[Managing Service]-Auto enable or disable state after boot

systemctl <enable/disable> <services name> example: systemctl enable nginx

If you want to enable or disable specific services after boot, then you might want to set this. If set disabled, services will not activate, you have to manually enable it by the below start command.

[Managing Service]-Check the state of service
In the previous one you enabled or disable it, now this command can check the state of all services.

check all state: systemctl list-unit-filesfilter state enable:systemctl list-unit-files --state=enabled

[Managing Service]-start and stop services
You can use this to start stop or restart services

Stop services: systemctl stop <service name>Start services: systemctl start <service name>Retart services: systemctl restart <service name>reboot systemctl:systemctl rebootshutdown systemctl:systemctl poweroff

[Managing Service]-Check services Status:
can provide information about the overall system health, by checking the service's current status. You can use the below State and type to filter the common information you want.

check specfic services : systemctl status <service name>list services current running: systemctl list-units --type=service --state=runningor another shortcut command: systemctl --type=service --state=runningfilter fail services:systemctl --type=service --state=failed,exited

Show the services unit information like cofnigure : systemctl show <services>

Check the log of services:
can display logs related to a service usingjournalctl -u service_name

overall make a recap of the checking the status in systemctl, list-unit-files, and list-unit two modes.

list-unit-files: state status of services automatically activate or not during boot
list-units: check the service's health status such as start, stop, or fail. So this means list-unit-files must be enabled first to start the services.

reference: https://www.howtogeek.com/839285/how-to-list-linux-services-with-systemctl/

27. system restart or shutdown ¹

you can use the reboot command which is the easiest way.

For Shutdown, you can use the run level init 0 or halt, shutdown this command or below:

shutdown: shutdown -h nowor init 0
reboot: shutdown -r now or reboot

you can also use the run-level command for the system

Runlevel 0 (Halt/Shutdown)
Runlevel 1 (Single User Mode)
Runlevel 2 (Multi-User Mode, No Networking)
Runlevel 3 (Multi-User Mode with Networking)
Runlevel 4 (Reserved/User-Defined)
Runlevel 5 (Graphical User Interface)

28. dh and du : Check Disk Space and file ¹

dh: report file system disk space usage
you can check disk usage: df -h

du: estimate file space usage

syntax:du -sh [filename|path]

you can check your file size: du -sh filename

29. ps: check process running or PID ¹

This is a pretty useful tool and powerful command that allows you to troubleshoot such as performance problems, monitor your system activity, check applications, and remove the process.

It allows you to monitor and check which application is running, and you can check the PID, memory CPU, and more. When your application seems to hang, then checking the PID and killing that process, pretty cool.

syntax: ps [options] [ — help]

ps is a command but you can add many options to filter much information. I often use the ps -aux , and to kill the process you can use kill <pid> or kill -9 <pid> .

ps : display a list of processes for running the current terminal session, without any option. This will display pretty less information so we need to add an option to show more detail.

PID: Process ID numbers for the listed processes
TTY: The terminals (or pseudo-terminals) associated with the process
Time: The total CPU time used by each process since it started.
CMD: name of command start the process

ps aux

You can also combine ps with options like aux , and this is the most usage people will use. Please refer to each option’s description:

a: show all processes including those owned by other user
u: show username of process owner instead of user ID
x: show information about the process including memory, CPU, and command argument.

I use them often to see the process that exists which is the COMMAND column, and the PID column which allows me to kill it. I only list aux , there are more you can search for which include:

it will list all running processes which include process ID, user ID, command line, and resource usage. I made some descriptions of each piece of information about the process in below, and example of it.

PID: The process ID.
USER: The user who owns the process.
%CPU: The percentage of CPU time that the process is using.
%MEM: The percentage of memory that the process is using.
VSZ: The virtual memory size of the process.
RSS: The resident set size of the process.
TTY: The terminal that the process is attached to
STAT: The status of the process.
START: The time at which the process started.
TIME: The total amount of CPU time that the process has used.
COMMAND: The command that was used to start the process.

common options you can use with ps, I will not mention all of them, you can search online.

1. General information:
show long format: l . In default ps it shows less information, expand the detail
Show information about the process of all progress: e
Specify which column to display : o <column>, ex: ps o pid,comm Show information about processes owned by a specific user: U <username>2.filtering:
- Show processes owned by specfic user -u or -U option:ps -u <username>. U:show process by EUID, and U by RUID.ps-U<user>-u<user>- using grep to filter ps aux | grep command- Show processes in a tree-like format based on parent-child relationships.Ex ps aux --forest or ps --forext 3.Resource Usage
- Show processes with extended resource usage information x
- how the real user ID of the process owner. ruser
- Show memory usage of the process: %mem
- Show the percentage of CPU used by the process. %cpu

30: free , top, htop: resource for CPU and memory loading ¹

From above I mention using ps command to check resources, however, there’s also a top command we can check, let me show you.

free ²

use the free command to check memory usage

top ²

use the top to check CPU memory and PID. This command monitors your CPU and memory current status and checks idle. There are some option you can use after entering top :

Display options:

1 toggle between single and overall CPU usage

f:toggle between different columns to display

I provide many other options below, generated by GPT. I rarely use these options, if you’re interested you can play around with it.

htop ²

If you run into a problem, probably it’s not been installed by default, install using this command:sudo apt-get install htop. It will look like the below, which provides a dynamic and real-time overview of the system’s resource usage, allowing users to monitor and manage processes in a more user-friendly compared with top. It uses color to represent, as you can see red color means it’s overloading. In the below memory seems to be ok, and so does the CPU.

I think that’s about it for the monitor using top, free, htop, and ps the command should cover it.

31. kill: kill the process to exit program ¹

I have mentioned the kill command in ps command, I would like to explain it here. You need to kill the process you need to know the process ID. The syntax is:

kill <PID>: will gracefully terminate a process , allowing it to perform cleanup before exiting
kill -9 <PID>: forcefully terminate process, which mean immediately terminated without any chance to perform cleanup operations

let me show you a demo, below is a bash script:

#!/bin/bash

echo "Script is running. Press Ctrl+C to exit.$$"
#$$ mean show the pID

while true; do
    sleep 1
done

please change the permission chmod 777 testscript.sh, and run it ./testscript.sh

so as you can see when I run the script, it shows PID 30806, and then I run px aux | grep testscript.sh , the PID also shows 30806, which means this is the script we are running. To terminate or close this you can use the kill 30806, the script will automatically terminate itself.

Archive File Create Compress Extract ¹

32. tar : to create an archive

I often use the tar.gz command to compress and extract file, however, there are many other commands you can use. This command tar is just archiving the file. Archive means collecting multiple data files together into a single file, and the file size will be the same as the original file. You can compress the file into gzip as .gz or bzip2 .bz2 .

syntax: tar -[option] [file.tar.gx] [direcotry or full path]

Below is the tar option:

create an archive file: tar -cvf archive.tar directoryname

c create archive
v mean verbose which will show the process output
f use to specify the filename of the archive

2. create a compressed file: tar -cvzf archive.tar.gz directoryname

you can use to compress bzip or gzip file according to your need, option a below

c create archive
v verbose which will show the process output
z compress gzip extension file
j compress bzip2extension file
f use to specify the filename of the archive

Let me show you the difference between the compressed and archive files the size is different. Notice if the file is been compressed its size will be smaller.

However, you can also use it without the z or j compress command, it will just treat the file as an archive file. From the below picture basely I created an archive file without compress and named the file as .tar.gz, the size will be the same as .tar which means the file is not been compressed.

create multiple files or directories: tar -cvf filearchieve.tar file1 file2 direcory1 directory2.It works on both archived and compressed files.

3. list the content of the archive tar file: tar -tvf testfolder.tar

you can also use this tar -tvf filename.tar.gz for compress file

t list the content inside archive
v verbose which will show the process output
f use to specify the filename of the archive

When you create an archive or compress you can use this t to list the content without extracting it. This is pretty useful when making sure your file or directory has been archived or compressed.

From the below picture when I use the t flag it shows all the content in the archive without extracting the archive.

4. Update or add files or directories :

tar -rvf tarName.tar newfile or tar -uvf tarName.tar newfile

r: append : if file not exist inside archive add file into archive. If exist, if file newer than inside archive file then will overwrite the older file inside archive, if not newer will not add into archive.
u:update if file newer than inside archive then update file, if not will not update. If file not exist will also not update inside archive.

These two flags look similar they have a common thing that is if the file is newer than the file inside the archive it will update or add inside. But one difference is u flag will only update if the file exists, whereas r the flag will add either exists or not, but the newer file will be added. Please refer above I made a diagram for easier understanding.

So from below I updated file ystq into filearc.tar, and show the content will exist in the new file.

5. Deleting a file from archive: tar -- delete -f archives.tar filename

6. Extract archive tar file: tar -xvf testfolder.tar

x extracts file from archive
v verbose which will show the process output
f use to specify the filename of the archive

Now since we created an archive and compressed the file, we need to extract the file with the x flag for extract.

Extract to the current directory: tar -xvf testfolder.tarExtraction in a separate directory: tar -xvf testfolder.tar /test

Extract specific file: tar -xvf filesarc.tar <specficfile or directory>

Network command ¹

I am going to show the network command I use often. You can reference more commands on the Ubuntu Offical Network command.

33. ifconfig: check IP address
34. IP address: show and set IP
35.Add IP address
36. Ethtool: check your network status
37. Adding Routing
38. tcpdump : capture packet
39. Ping: check network connection
40. ssh and scp : remote and transfer files

33. ifconfig: check IP address ¹

This is older command is used to check all your IP addresses ifconfig or ifonfig -a to show all interfaces. In window also has this command ipconfig . You can also use the -a option to show all interfaces. This command will show detailed information about the network interface such as IP address , and MAC Address.

you can also show specific interfaces by using ifconfig <intername>

34. IP address: show and set IP ¹

This command is a new command that is much more powerful which allows to check IP addresses and set IP addresses.

You can check all the LAN interfaces using the command ip -a or ip address . This command and the previous command both can check the IP interface

If you want to show a specific interface IP address you can use ip show <interface>

you can also just show only the information of the interface not IP address

show all interface: ip link show
show active interface: ip link show <interface> up
show specfic interface: ip link show <interface>

35 Add IP address ¹

add ip address : ip addr add <IPAddress> <interface>delete ip address : ip addr del <IPAddress> <interface>Bring down the interface:ifdown <interface>Bring up the interface: ifup <interface>remove IP address :ip addr flush dev <interface>

This command only sets IP temporary, after reboot this IP will return to the original or default IP. Often this command to set IP when people don’t want to reboot their IP after setting or for debugging for some purpose.

If you exist IP in an interface you can use the ip addr flush dev <interface> or you can do ifdown or ifup and change IP address

To set your IP address you need to modify the network interface and do a reboot to activate it, after reboot the IP will be what you set.

Ubuntu 16.04 network interface configure: /etc/network/interfaceand restart by/etc/init.d/network-manager restart
Ubuntu18 or above are managed by NetworkManager: /etc/netplan/inerfacename.yaml , restart by netplan apply

If you want to set a different static IP /etc/network/interface without rebooting it, you have to do this step:

#edit network interface
vi /etc/network/interface

# restart network services, can choose either one
sudo /etc/init.d/networking restart
sudo systemctl restart networkin
# systemctl 
#clear exist IP address
ip addr flush dev <interface>
ifdown <interface>
ifup <interface>

#check IP address will update
ifonfig <interface>
#or 
ip a

you can also set it like this ip addr add <IPAddress> <interface> , but next time boot will return the default setting(interface file setting)

restart network service:
You can choose any of these commands:

# sudo /etc/init.d/networking restart
or
# sudo /etc/init.d/networking stop
# sudo /etc/init.d/networking start
else
# sudo systemctl restart networking

36. Ethtool: check your network status ¹

This command is useful to debug whether your network state is linked or not, speed, etc. Please refer below picture on what it will display.

Syntax: ethtool <interface name>

37. Adding Routing ¹

show route table: ip route
add and delete route: ip route [add/del] [sourceIP]/24 via [desIP] dev <interface>

38. tcpdump : capture packet ¹

all interface: taskset -c 32 tcpdump -i lo -w file.pcap
specific interface tcpdump -i enp4s0 -w file.pcap

Ping is a command that allows you to check between connections of devices. Each device obtains an IP, and we use the ping command to check whether that device is online or available. We often use the command ping <ipaddress>, but there are many more options we can use but I will not show all of the options, instead, I will show options I use often.

Syntax：ping [options] [destination]

ping localhost without parameter as default will keep pinging until you stop it by ctrl+c

c count option: specify the number of ICMP packets that the ping command should send

ping -c <count> <destination>

if you set -c 10 , this means it will send 10 ICMP packets then stop

i interval option: specify the interval between sending ICMP packets

ping -i <interval> <destination>

if you set -i 5 , which means every 5 seconds will send a ping or ICMP packet.

I also want to show ping to display the timestamp using the -D option

But this method is not the best way, the time is hard to read, instead, we can use ts the command this option to display the date and time:

ping localhost | ts

But you need to install a package sudo apt-get install moreutils

40. ssh and SCP : remote and transfer files ¹

I will not introduce to much on these commands, i have written another post related to SSH and remote command.

ssh

normal ssh: ssh <username>@<hostname or IP>
ssh with password: sshpass -p ‘<password’ ssh <username>@<hostname or IP> Note: Need to install sshpass package

scp transfer file

scp <username>@<hostname or IP>:<source path/filename> <destination path.filename>Example:
Upload: scp ufo.csv chenchih@192.168.2.100:/home/chenchih/Downloads
download: scp username@192.168.2.100:/home/chenchih/Downloads /download path

if you want to upload or download a folder or directory add a -r option also mean recursive, please refer below example: scp -r Links chenchih@192.168.2.100:/home/chenchih/Downloads

Storage command ¹

41. fdisk check disk name
42 mount
43 fdisk and parted : Create Partition
44. mkfs format disk

Check Disk Name:</path/to/disk/partition>

41. fdisk : check your disk name ¹

fdisk -l : check your disk device name and path, because you need to know the path so that you can mount it on a directory to copy files from the disk locally. Below is an example of listing your device's name in this case sda1 is the device's name.

If you have multiply partition it will occur sda1 ….N. SDA means the first disk, if you insert another one will name SDB and SDC and so on.

Name of disk:
hd(x) - IDE
sd(x) - SATA,SSD,ISCI,SAS
nvme0n1p(x):SSD

you can use the ls -la to show all your disks: ls -la /dev/sd*

42. mount command ¹

mount: mount </path/to/disk/partition> <system path>
umount or eject: umount </path/to/disk/partition> <system path>mount advance with rw: mount -t ext4 -o rw </path/to/disk/partition> <system path>

if you know your disk name and path which I mentioned previously now we can mount your disk into a specific directory or path in the below example I mounted in the /mnt location. Now you can copy the local file to mnt, which also means to the disk. Your disk now is mounted into mnt which means it’s your USB thumb drive. To eject your disk from /mnt just just umount /dev/sdb1

43. fdisk and parted : Create Partition ¹

fdisk : use when drive less than 2TB, it wasn’t designed for large partition
parted: is an alternative of fdisk

fsdisk: sudo fdisk </path/to/disk/partition>

parted: sudo parted </path/to/disk/partition>

you can use the p to print the partition or h to see the help command

44. mkfs: Format disk ¹

Syntax: mkfs.<filesystem> </path/to/disk/partition>

There’re more format I just list some of them
ext4 format for linux: mkfs.ext4 /dev/sdb1vfat format for window:mkfs.vfat /dev/sdb1nfs format for window:sudo mkfs.NTFS /dev/sdb1fat format for window: mkfs.fat -F 32 /dev/sda2 , -f Fat type 12,16,32bit

Other checking disk commands ¹

view existing partition or mount disk: lsblkshow disk information: lshw -class disk

Shell command ¹

I think this is also an important topic or command to mention it here, but it’s not used often, only if you are writing a bash script. You can use the cat /etc/shells to list the valid shell that are available in your system and echo $SHELL command to check the default shell for the current user.

Running a shell script using ./script.sh or bash script.sh will both use the bash shell by default. You will see many people add this #!/bin/bash on top of the .sh file, which indicates that the script should be interpreted by the Bash shell.

Conclusion and Summary ¹

I didn’t mention all of the cli Linux commands, but most of the commands I mention are much to use. There are many more commands you can search by yourself and options. I didn’t mention many of the flags or options, I only mentioned what I used often and some I think are important.

Lastly, this command might be hopeful for people who never use the command. Linux command and Mac(based on unix) are somehow the same, but there are some parts different.

My intention in this post is to record the command I used and share it with people who haven’t used it before, and indeed to keep it as my note.

If you reach up to here thanks for reading up till here, and from the bottom of my heart I appreciate it. Any suggestion or feedback please leave a comment