Q I tried to review /var/log/messages and found the file to be empty. Listing the contents of /var/log I found /var/log/messages to be dated two days ago while /var/log/messages.1 was dated today. I created a few entries using logger and saw that my messages were in fact being written to /var/log/messages.1. I rebooted the system and from then on new entries were being written to /var/log/messages. I have checked the Syslog configuration at /etc /syslog.conf and confirmed that messages were meant to be written to /var/log/messages. Am I overlooking something? I run Red Hat Enterprise Linux 4.
A On Unix filesystems, 'inodes' describe the type, permissions, ownership, timestamps and the data that make up a file. In fact, the filename is just a link in a directory to the inode as identified by its inode number. The Syslog daemon, Syslogd, is responsible for the entries written to /var/log/messages. Using lsof, it is possible to determine the inode that corresponded to /var/log/messages when you started Syslogd, like this:
syslogd 3579 root 1w REG 3,5 926461 7898395 /var/log/messages
Calling ls with -lai includes the inode number in the listing, giving
7898395 -rw------- 1 root root 933307 Sep 14 20:13 /var/log/messages
It follows that Syslogd will keep on writing to the same inode regardless of any link name change, be it a rename or a deletion. The link /var/log/messages should have been rotated to /var/log/messages.1 by Logrotate. Under Red Hat ES 4, logrotate is run daily as specified by the crontab entry /etc/cron.daily/logrotate, while /etc/logrotate.conf and the included /etc/logrotate.d/* files dictate which logs are to be rotated and how often. Inspecting the logrotate configuration file illustrates how, after log files are renamed, new log files are to be recreated. Also, /etc/logrotate.d/syslog dictates that Syslogd is to be restarted to reload the configuration while closing and reopening all log files for append. This last step must have failed, and Syslogd kept on appending to the inode linked by /var/log/messages the last time Syslogd had been properly restarted. Rebooting restarts Syslog, which is why the issue was, on this occasion, resolved.