How to Check Server Logs for Security and Software Issues
How-to-Guides from Kryotech
What Do I Need?
What are Log Files?
Log files are the all-important records that Linux stores for administrators to keep track and monitor important events on the webserver, kernel, services, and the applications running on it. Linux provides a centralized repository of log files that can be located under the /var/log directory. All log files generated in a Linux environment can typically be classified into four different categories:
- application logs,
- event logs,
- service logs,
- system logs.
Monitor Log Files
Monitoring and analyzing log files can be a challenging task. The sheer volume of logs can sometimes make it frustrating to drill down and find the right file that contains the required information.
Messages
cat /var/log/messages
This log file contains generic system activity logs. It’s mainly used to store informational and non-critical system messages.
Using these logs, you can track non-kernel boot errors, application-related service errors, and the messages that are logged during system startup. It’s the first log file any Linux administrator should check if something goes wrong.
Auth.log
cat /var/log/auth.log
All authentication-related events in Debian and Ubuntu servers are logged here. If you’re looking for anything involving the user authorization mechanism, you’ll find it here.
If you suspect there’s been a security breach of your server, this is where you may find indicators. If you notice a suspicious javascript file where it shouldn’t be, this is where you’d see it.
Secure.log
RedHat and CentOS-based systems use this log file instead of /var/log/auth.log. It’s mainly used to track the usage of authorization systems. It stores all security-related messages, including authentication failures and various others. It’s also responsible for tracking sudo logins, ssh logins, and other errors logged by security systems daemons or services.
All user authentication events are logged. This file can provide detailed insights into unauthorized and failed login attempts and can be useful for detecting possible hacking attempts. It also stores useful information about successful logins and tracks the activities of valid users.
Boot.log
cat /var/log/boot.log
The system initialization script, /etc/init.d/bootmisc.sh, sends all bootup messages to this log file. This is the repository of booting related information and messages logged during the system startup process. You should analyze this log file to investigate issues related to improper shutdown, unplanned reboots, or booting failures. You can also determine the duration of system downtime caused by an unexpected shutdown.
Dmesg
cat /var/log/dmesg
This file contains kernel ring buffer messages. Information related to hardware devices and their drivers is logged here. As the kernel detects physical hardware devices associated with the webserver during the booting process, it captures the device status, hardware errors, and other generic messages.
This log file is useful for dedicated server users mostly. If certain hardware is functioning improperly or not getting detected, then you can rely on this log file to troubleshoot the issue.
Kern.log
cat /var/log/kern.log
This is a very important log file as it contains information logged by the kernel; perfect for troubleshooting kernel-related errors and warnings.
Kernel logs can be helpful to troubleshoot a custom-built kernel and can be extremely useful in debugging hardware and connectivity issues.
Faillog
cat /var/log/faillog
This file contains information on failed login attempts. It works best to find out any attempted security breaches involving username/password hacking and brute-force attacks.
Next Steps
I’d recommend looking at the variety of other logs also available. For example, it’s always a good idea to check the following:
- /var/log/cron
- cat /var/log/cron
- /var/log/yum.log
- cat /var/log/yum.log
- /var/log/maillog or /var/log/mail.log
- cat /var/log/mail.log
- /var/log/httpd/
- cat /var/log/httpd/
- /var/log/mysqld.log or /var/log/mysql.log
- cat /var/log/mysqld.log
Conclusion
While monitoring and analyzing all the log files generated by the system can be a difficult task, you can make use of a centralized log monitoring tool to simplify the process. Personally, as opposed to ‘handing off’ inspection and control to outsourced elements, I suggest getting to grips with these log files and monitoring them manually.
