Linux Log Files Every Admin Should Know

Understanding Linux log files is fundamental to effective system administration. Whether you’re troubleshooting issues, monitoring performance, or investigating security incidents, log files are your first line of defense and your most valuable diagnostic tool. This comprehensive guide covers the essential log files every Linux administrator should know and how to use them effectively.

Table of Contents

• What Are Linux Log Files?
• The /var/log Directory Structure
• Essential Log Files You Must Know
• Authentication and Security Logs
• Application-Specific Logs
• Tools for Viewing and Analyzing Logs
• Log Rotation and Management
• Best Practices for Log Management

What Are Linux Log Files?

Linux log files are text-based records that document system events, application activities, security incidents, and user actions. These files are automatically generated by the operating system, daemons, and applications to provide administrators with detailed information about what’s happening on their systems.

Most Linux distributions store log files in the /var/log directory. The logs are typically managed by system logging daemons such as rsyslog or systemd-journal, which collect messages from various sources and write them to appropriate log files.

The /var/log Directory Structure

The /var/log directory is the central repository for system logs. Here’s what you’ll typically find:

• Plain text log files that can be read with standard tools
• Compressed archived logs with extensions like .gz or .bz2
• Subdirectories for specific applications and services
• Binary log files that require special tools to read

To view the contents of your log directory, simply run:

ls -lh /var/log

Essential Log Files You Must Know

/var/log/syslog or /var/log/messages

This is the general system activity log that captures almost everything happening on your system. On Debian-based systems, it’s called syslog, while Red Hat-based systems use messages. This log contains information from the kernel, system daemons, and various services.

tail -f /var/log/syslog

/var/log/kern.log

The kernel log file contains messages from the Linux kernel, including hardware errors, driver issues, and kernel-level warnings. This is crucial for diagnosing hardware problems and low-level system issues.

/var/log/dmesg

This file contains kernel ring buffer messages, primarily showing boot-time hardware detection and initialization messages. You can also view this with the dmesg command:

dmesg | less

/var/log/boot.log

Contains information about system startup, including which services started successfully or failed during boot. This is invaluable when troubleshooting boot issues.

Authentication and Security Logs

/var/log/auth.log or /var/log/secure

This critical security log records all authentication attempts, including successful and failed login attempts, sudo usage, and SSH connections. Debian systems use auth.log, while Red Hat systems use secure.

grep "Failed password" /var/log/auth.log

For cloud-based Linux instances, especially those running on platforms like Kamatera, monitoring authentication logs is essential for detecting unauthorized access attempts and maintaining security compliance.

/var/log/faillog

This binary file tracks failed login attempts. Use the faillog command to read it:

faillog -a

/var/log/lastlog

Records the last login time for all users. View it with:

lastlog

Application-Specific Logs

/var/log/apache2/ or /var/log/httpd/

Web server logs are stored here, including access.log for all HTTP requests and error.log for server errors. These are essential for web administrators monitoring site traffic and troubleshooting web applications.

/var/log/mysql/ or /var/log/mariadb/

Database server logs contain query errors, slow queries, and database performance information. The error.log file is particularly useful for database troubleshooting.

/var/log/cron

Records all cron job executions. When scheduled tasks aren’t running as expected, this is the first place to check:

grep CRON /var/log/cron

/var/log/mail.log

Mail server logs track email sending and receiving activities, including SMTP transactions and delivery status.

Tools for Viewing and Analyzing Logs

Linux provides several powerful tools for working with log files:

Basic Viewing Commands

• cat – Display entire file contents
• less – Page through logs interactively
• tail – View the last lines of a file
• head – View the first lines of a file
• grep – Search for specific patterns

tail -n 100 /var/log/syslog
tail -f /var/log/apache2/access.log
grep "error" /var/log/syslog | less

Journalctl for Systemd Systems

Modern Linux distributions using systemd store logs in a binary format accessible through journalctl:

journalctl -xe
journalctl -u nginx.service
journalctl --since "1 hour ago"
journalctl -p err -b

For those looking to deepen their Linux administration skills, platforms like DataCamp offer comprehensive courses on system administration and log analysis that can help you master these essential tools.

Log Rotation and Management

Log files can grow rapidly and consume significant disk space. Linux uses logrotate to manage this automatically. Configuration files are located in /etc/logrotate.conf and /etc/logrotate.d/.

Logrotate performs several important functions:

• Rotates logs based on size or time
• Compresses old log files to save space
• Deletes very old logs automatically
• Creates new log files with proper permissions

Example logrotate configuration:

/var/log/myapp/*.log {
    daily
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 www-data adm
}

Best Practices for Log Management

Regular Monitoring

Set up automated monitoring and alerting for critical log entries. Tools like Logwatch can send daily email summaries of important log events.

Centralized Logging

For environments with multiple servers, implement centralized logging using tools like rsyslog, syslog-ng, or the ELK stack (Elasticsearch, Logstash, Kibana).

Security Considerations

• Restrict log file permissions to prevent unauthorized access
• Monitor authentication logs for suspicious activity
• Ensure adequate disk space for log storage
• Consider forwarding logs to remote servers for security

Retention Policies

Establish clear retention policies based on compliance requirements and storage capacity. Some regulations require specific log retention periods.

Documentation

Document your logging infrastructure, including what gets logged where and retention schedules. This helps during incident response and audits.

Conclusion

Mastering Linux log files is essential for effective system administration. From troubleshooting application errors to detecting security breaches, logs provide the detailed information needed to maintain healthy, secure systems. Start by familiarizing yourself with the essential log files covered in this guide, practice using the viewing tools, and establish good log management practices. As you gain experience, you’ll develop the ability to quickly locate relevant information and diagnose issues efficiently, making you a more effective Linux administrator.

Remember that log files are living documents—they’re constantly being updated with new information. Regular review and monitoring of these files should be part of your daily administrative routine, helping you catch problems early and maintain system reliability.