AWK command in Linux

The awk command in Linux is a powerful text-processing utility that allows you to manipulate and analyze text data in files or streams. It operates on a per-line basis, splitting each line into fields based on a specified delimiter (by default, whitespace), and then allows you to perform various operations on these fields.

Some more details about AWK :

• Searches files for lines that contain certain patterns
• Performs operation described in AWK body on line with cerain pattern
• Performs operation described in AWK body on choosen line
• In AWK body bash features dosen’t work
• Completely new language in AWK body
• Completely new features we can call on specific line (specific text)

AWK can filter the file both horizontally and vertically at the same time.

Basic Structure of AWK command :

awk 'BEGIN{code_in_BEGIN_section}{code_in_main_body_section} End {Code_in_END_section} input-file1 input-file2
awk 'BEGIN{code_in_BEGIN_section}
          {code_in_main_body_section}
       END{code_in_END_section}' input-file1 input-file2

awk -F: '{print$0}' /etc/passwd /etc/shadow /etc/shadow
awk 'BEGIN {FS=":"} {print $1,$2}' /etc/passwd #We can also give Field Seperator in BEGIN part
awk 'BEGIN {RS=":"} {print$0}' /etc/passwd #Record Seprator can seperate records, by default it is \\n

here we used awk command and set the delimiter to ‘ : ‘ using -F switch and printed all fields of the three files.We can select different fields to print by $ sign

(*If we dont use -F switch it will consider the delimiter as space)

awk -F: '{print$1,$6,$7}' /etc/passwd #Uses space as delimiter while using(,)
awk -F: '{print$1,"-",$6}' /etc/passwd #Adds costum symbol in between the values
awk -F: '{print$1,"\\\\",$6}' /etc/passwd #To Use backlash we need to give two backslash 
awk -F: '{print$1,"text ",$6}' /etc/passwd #We can add any text also in between

Now to filter out a specific string :

awk -F: '/^root/ {print$0}' /etc/passwd
#filtering strings starting with "root"

ifconfig | awk '/ inet /{print$2}'
#filtering ip address using awk

We can also add begin and end part in awk like this :

ifconfig | awk 'BEGIN {print"===IP Address==="} / inet /{print$2} 
END {print "===== END ====="}'

OUTPUT ->
===IP Address===
192.168.29.21
127.0.0.1
192.168.122.1
===== END =====

We can also use begin, middle and ending part as we wish and can also use them individually:

echo "one two three four" | awk 'BEGIN{print"demo1"} {print"demo2"} 
END{print"demo3"}'
demo1
demo2
demo3

We mainly define the middle part usually, BEGIN & END part is not used much.

We can also modify any $ value :

echo Armour Infosec | awk '{$1 = "ARMOUR" ; print$0}’
ARMOUR Infosec

We can use == and != to filter out output in awk command like :

ifconfig | awk'$1=="inet" {print$2}'
#field 2nd whose first field is inet

ifconfig | awk'$1!="inet" {print$0}'
#all the fields whose 1st field is inet

awk -F: '$1=="root" {print$0}' /etc/passwd
#all the fields whose 1st field is root

awk -F: '$1!="root" {print$0}' /etc/passwd
#all the fields whose 1st field is not root

awk -F: '$3>=1000 {print$0}' /etc/passwd 
#all the fields who have value greater than 1000 in 3rd field

awk -F : '/^bash/ {print$0}' /etc/passwd
#fields starting with "bash"

awk -F : '/bash&/ {print$0}' /etc/passwd
#fields ending with bash

awk -F : '/\\/sbin\\/nologin$/ {print$0}' /etc/passwd
#use of opposite slash to search for slash values (/sbin/nologin)

echo "one two three four" | awk '{print NF}'
#prints number of fields(Output = '4')

echo "one two three four" | awk '{print $NF}'
#prints last field value

echo "one two three four" | awk '{print $(NF-1)}'
#prints second last field value and so on {(NF-2) (NF-3)...}

Using Pre-Made pattern file :

Sometime the middle file pattern can be of multiple lines which can increase the confusion and things can become haywire, we can eliminate this by saving our pattern in a text file like :

vim awk_patterns.txt
  BEGIN{
  print "Passwd File"
  }
  {
  print $1 , "home at" , $6
  }
  END{
  print "End of passwd file"
  }
#We can define the content in multiple lines as well as single line :
BEGIN{print "Passwd File"}{print $1 , "home at" , $6}END{print "End of passwd file"}

Now we can use the pattern file by -f switch :

awk -f awk_patters.txt -F : /etc/passwd

We can also create good looking patters like :

BEGIN{
print "User and their corresponding home"
print "User Name \\t \\t Home Path"
print "---------------\\t  \\t---------------"
FS=":"
}
{
print$1,"\\t \\t \\t "$6
}
END{
print"-------------END of the file--------------"
}

OUTPUT ->
User and their corresponding home
User Name 	 	 Home Path
---------------	  	---------------
root 	 	 	 /root
bin 	 	 	 /bin
daemon 	 	 	 /sbin
adm 	 	 	 /var/adm
lp 	 	 	 /var/spool/lpd
sync 	 	 	 /sbin
shutdown 	 	 	 /sbin
halt 	 	 	 /sbin
mail 	 	 	 /var/spool/mail
postfix 	 	 	 /var/spool/postfix
user1 	 	 	 /home/user1
userx 	 	 	 /home/userx
-------------END of the file--------------

ps aux | awk -F" " 'BEGIN{print"USERNAME \\t PID \\t CMD"} $1==root {print$1,"\\t \\t",$2,"\\t",$11,$12,$13,$14}'
USERNAME 	 PID 	 CMD
root 	 	 1 	 /usr/lib/systemd/systemd --switched-root --system --deserialize
root 	 	 2 	 [kthreadd]   
root 	 	 4 	 [kworker/0:0H]   
root 	 	 6 	 [ksoftirqd/0]   
root 	 	 7 	 [migration/0]   
root 	 	 8 	 [rcu_bh]   
......

We can also use awk for calculations like :

echo 4 6 | awk '{print$1+$2}'
10
echo 4 6 | awk '{print$1-$2}'
-2
echo 4 6 | awk '{print$1*$2}'
24
echo 4 6 | awk '{print$1/$2}'
0.666667