Troubleshooting Application issues using find and grep in Linux

How to troubleshoot the Application issues in Linux?

In this article, you will learn about “Managing the Files and Folders in Linux” with Hands-on scenarios.

👨🏽‍💻🧑🏻‍💻For more ARTICLES, FOLLOW📍DevOps Engineering on Cloud

Let’s get started.

Review Standard Location for Application Logs on Linux

You can find the application logs of Linux servers in /var/log directory. It's a very important directory to troubleshoot the application.

🚨👉🏼 You can also check the complete udemy course (Linux Shell Commands for Absolute Beginners using Ubuntu 20x)🔗Referral link

You can get all the application logs in /var/log directory in Linux.

How do troubleshoot the application logs on Linux systems?

Setup Log Files to explore grep and find to troubleshoot the issues

You can use the following repository to get the log files for the hands-on scenarios of how to troubleshoot the applications on Linux using log files.

you can clone the above repository for the practice of log file troubleshooting on Linux

After cloning the above repository you need to copy the log folders to the /var/log directory. You need to be a super user to run the below command successfully.

sudo cp -rf loghub/* /var/log

Now you can see the logs from different applications in the /var/log directory.

Log files to troubleshoot the application in Linux.

You can use the following command in Linux to get the files with properties which are ending with .log an extension.

find . -name "*.log"

How to find the log files in Linux /var/log directory?

Let's troubleshoot the logs generated by the Hadoop application at /var/log/hadoop directory. Inside that directory, you can see the Hadoop_2k.log file. In this log file, you can see the logs.

Coming to .log file it has the information like the Timestamp of the log generation, Whether the log is INFO or ERROR or WARN and the description.

How does the log file look?

How to troubleshoot the log files in Linux?

Quick Review of the find command to get log file names based on criteria

You can use the following command in Linux to get the files with properties that are updated in last 24 hours.

find . -type f -mtime -1 -exec ls -ltr {} +;

Get the properties of the files which are ending with .log extension using find command.

If you want to get the messages with the error in them from the .log files then you can use the find command along with the grep .

find . -type f -mtime -1 -exec grep -w error {} +;

This command is very powerful and time-saving to troubleshoot the errors in the log files.

If you want to get the messages with the error keyword in them which are generated in the last 24 hours then you can use the following command with -l control argument in the command.

find . -type f -mtime -1 -exec grep -lw error {} +;

Get all the error messages which are generated in the last 24 hours from the log file.

How to use the find and grep command to get the error log message from the log files?

Getting help or official documentation or man page on Linux grep command

The grep utility searches any given input files, selecting lines that match one or more patterns. You can get the complete usage of the grep command in Linux by running grep --help or man grep .

Grep command in Linux

Grep command In Linux introduction.

Redirect Linux Command or Program Results to a file

You can redirect the output of one command to the file and save it by using > Linux. For example, you can use the following command.

grep COMPLETE data/retail_db/orders/part-00000 > /tmp/orders_complete

Redirect the output of the grep command to another file in Linux

In the above screenshot, you can clearly see the creation of orders_complete a file in Linux. When you run the grep command with CLOSED a keyword then it had replaced the orders_complete file.

If you use >> instead of > then instead of replacing the existing file, it will append data to already existing data.

Append the new data into already existing data by using >> in Linux.

How to redirect the output of the command to the file?

Redirect standard output and error to files and null device in Linux

You can use the following command to get the .log files in the Linux /var directory.

find /var -name "*.log"

Get the .log files from the /var directory

In the above screenshot, all the lines which are with permission denied are standard errors. And one with .log standard out results.

The following command will ignore the non-error messages and print only the error messages in the Linux.

find /var/ -name "*.log" 1>/dev/null

Error messages in Linux.

And if you want to ignore the error messages and print only the non-error messages in the Linux then use the following command.

find /var/ -name "*.log" 2>/dev/null

Print only non-error message files in Linux.

You can redirect the outputs of error and non-error messages to the files by using the following command in Linux.

find /var/ -name "*.log" 1>/tmp/find.out 2>/tmp/find.err

Redirect the error and non-error messages to the files.

Redirecting the error and non-error messages to troubleshoot the Linux apps.

Search for error lines from files in Linux using find and grep

You can search the lines which contain the ERROR or WARN or INFO in the log files by using grep command in Linux.

grep WARN HDFS_2k.log

get the lines that contain the word WARN in them using grep in Linux

You can use any of the following find commands to get the lines that contain the word ERROR in them from all the .log files

find . -name "*.log" 2>/dev/null | xargs grep ERROR | wc -l
or 
find . -name "*.log" -exec grep ERROR {} + 2>/dev/null | wc -l

Get the lines that contain the ERROR in them by using the find command

You can perform case insensitive search by using -i control argument in the grep command.

find . -name "*.log" 2>/dev/null | xargs grep -i ERROR | wc -l

Performing the case insensitive search of ERROR keyword using grep and find command in Linux.

How to perform the search on .log files using find and grep commands in Linux?

Get File Names with errors using grep and find in Linux

You can get the file names instead of the lines of the error message by passing the -l control argument to the grep command in Linux.

grep -l ERROR */*.log

Using the grep command in Linux to get the file names of the error messages.

You can also perform the case insensitive search by passing the -i control argument to the grep command in Linux.

grep -il ERROR */*.log

Performing case insensitive search of the log files using grep command

If you want to find the files with an error message in the parent directories and sub-directories then you can use find command as follows.

find . -name "*.log" 2>/dev/null | xargs grep -il ERROR

Performing a search of file names with error messages by using the find command.

Find a command in Linux to get the file names with error messages in them.

Perform Whole Word Search using Grep Command

The whole word search will take the complete word and perform the search. It will not work even if a single letter is missing from the whole word. Check out the following command for the whole word search.

grep -w Received HDFS/HDFS_2k.log

You can also perform a whole word search with case insensitive search by using -i control argument.

grep -iw Received HDFS/HDFS_2k.log

How to perform whole word search in Linux?

Search for multiple patterns in the files using find and grep in Linux

You can pass the multiple patterns to the same command by using piping in Linux. For example, If you want to get the union of ERROR and EXCEPTION lines from the log file you can use the following command.

grep 'ERROR\|EXCEPTION' Zookeeper/Zookeeper_2k.log

Each line of this output contains both the ERROR and exception words.

Now you can pass this grep command to find the command to get all the files that have the union of Error and exception keywords in them.

find . -name "*.log" -exec grep -ilw 'ERROR\|EXCEPTION' {} + 2>/dev/null

The following output contains the files with both error and exception keywords in a single line.

How to search for multiple patterns in the files using find and grep in Linux?

Get the number of lines with a pattern using find and grep in Linux

You can get the number of lines with a certain pattern which you pass as input to the grep command by using the following command in Linux.

grep -i 'ERROR\|EXCEPTION' Zookeeper/Zookeeper_2k.log | wc -l

Get the number of lines that contain the pattern passed as input to the grep command in Linux.

Now, You can pass the above grep command to the find command to get the file names that contain the lines with both error and exception keywords in them along with the count.

find . -name "*.log" -exec grep -ic 'ERROR\|EXCEPTION' {} + 2>/dev/null

Get the file names along with the count of errors and exception keywords that occurred in each line of the .log file.

You can also use the grep -ic ‘ERROR\|EXCEPTION’ */*.log to get the similar pattern as above. But you can see only a few files in the output.

How to get the number of lines with patterns using find and grep in Linux?

Get line number and output line using grep in Linux

The normal output of the grep command will not display the line number. To get the line number in the output you need to use -n the control argument along with the grep command in Linux.

grep -in exception OpenStack_2k.log

Print the Line number along with the output in Linux by using the grep command.

You can also view the specific log file by running view [FILE NAME] . Once the file is opened you can press the :(colon) set nu then the line numbers will be displayed.

How to get the line number from the log file in Linux to troubleshoot issues?

Get additional lines along with error or exception lines using grep in Linux

If you want to get the 3 lines before the exception line from the log file then you can use the -B control argument with the grep command in Linux.

grep -in -B 3 exception OpenStack_2k.log

Print the before 3 lines of the exception lines to get more data to troubleshoot the issue in the Linux log file

If you want to get the 3 lines After the exception line from the log file then you can use the -A control argument with the grep command in Linux.

grep -in -A 3 exception OpenStack_2k.log

If you want to get the 3 lines before and 3 lines After the exception line from the log file then you can use the -C control argument with the grep command in Linux.

grep -in -C 3 exception OpenStack_2k.log

How to Get additional lines along with error or exception lines using grep in Linux?

🙏🏼Thank you, for reading the article. If you find it valuable please follow our publication DevOps Engineering on Cloud

🚨👉🏼 You can also check the complete udemy course (Linux Shell Commands for Absolute Beginners using Ubuntu 20x)🔗Referral link