rsyslog sisteminin nasıl çalıştığını çoğu insan tam anlamıyla bilmez. Bu kayıt “log” sisteminin nasıl çalıştığını ve nasıl yönetilebileceğini anlatmaya çalışalım.
Aşağıda rsyslog version 5.8 ve yeni 8.4 versiyonu ile ilgili değişiklikler anlatılıyor.
1. Varsayılan rsyslog.conf ayarları
Varsayılan /etc/rsyslog.conf konumunda bulunan ayar dosyası aşağıdaki şekildedir:
auth,authpriv.* /var/log/auth.log *.*;auth,authpriv.none -/var/log/syslog daemon.* -/var/log/daemon.log kern.* -/var/log/kern.log lpr.* -/var/log/lpr.log mail.* -/var/log/mail.log user.* -/var/log/user.log mail.info -/var/log/mail.info mail.warn -/var/log/mail.warn mail.err /var/log/mail.err *.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug *.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages *.emerg :omusrmsg:*
Burada bulunan kodların amacı hata durumlarını filtreleyerek farklı dosyalarda tutulmasını sağlamak.
The first column is a filter to capture a subset of messages and pipe them into a specific log file, or take other action. The destination log files appear on the right.
For the curious, omusrmsg stands for “User Message Output Module” and combined with ‘*’ will send a console alert to all logged in users. For example, when the system is going down for reboot.
2. Security Levels
The available security levels are defined in RFC 5424 as follows:
|0||emerg, panic||Emergency: system is unusable|
|1||alert||Alert: action must be taken immediately|
|2||crit||Critical: critical conditions|
|3||err, error||Error: error conditions|
|4||warning, warn||Warning: warning conditions|
|5||notice||Notice: normal but significant condition|
|6||info||Informational: informational messages|
|7||debug||Debug: debug-level messages|
So when we read from rsyslog.conf that /var/log/mail.info receives messages from the ‘mail’ facility with a security level of “info or higher” what we mean is that the mail.info log also collects entries destined for: mail.notice, mail.warn, mail.err, mail.crit, mail.alert and mail.emerg (so higher in severity), but not mail.debug which is less severe than ‘info’.
3. How the filters work
As described earlier, the column on the left is a filter, and in that column you have entries using the format:
For example:auth,authpriv.*matches all messages from the auth or authpriv facilities.*.*;auth,authpriv.nonematches all messages from all facilites, but excluding any from auth or authpriv.mail.infomatches messages from the mail facility with security levels of ‘info’ or higher.mail.warnmatches messages from the mail facility with security levels of ‘warn’ or higher.*.=debug;matched messages from all facilities with the ‘debug’ security level and no other.
You can deduce from this that some messages will appear in multiple log files, which is something we want to keep an eye on and control.
For example, with the above settings, an incoming message with status mail.info will appear in three (3) log files: syslog, mail.log and mail.info. While a message with status mail.err will be logged to five (5) log files, namely: syslog, mail.log, mail.info, mail.warn, and mail.err.
And a message logged from iptables with –log-level 7 will arrive with a status of kern.debug and be logged to three (3) separate log files: syslog, kern.log and debug.
You can monitor these files in real-time from the command-line:
tail -fn0 syslog mail.err mail.warn mail.info mail.log kern.log debug
Prefix your commands with sudo if you’re not running as root or don’t have read access.
4. Less duplication in the mail logs
Focusing on the ‘mail’ facility for now, we don’t really need that many copies of every log entry. So let’s make some changes.
Now we have just two logfiles for ‘mail’. The mail.log for all entries, and mail.err for ‘warn’, ‘error’, and any more severe messages.
|Security level||Old settings log targets||New settings|
|mail.info||syslog, mail.log, mail.info||mail.log|
|mail.warn||syslog, mail.log, mail.info, mail.warn||mail.log, mail.err|
|mail.err||syslog, mail.log, mail.info, mail.warn, mail.err||mail.log, mail.err|
What a difference this makes already on an active mail server!
5. Less duplication in the iptables logs
Similar to the situation to mail, we have an issue with iptables logging to multiple locations.
At the end of our firewall script we have the following catch-all to LOG details of any access attempts that weren’t already either ACCEPT’ed or REJECT’ed by earlier rules:
$IPTABLES -A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7
$IPTABLES --policy INPUT DROP
With a log level of 7 (debug) this rule generates log messages with a status of kern.debug. And with the default configuration rsyslog will write them to three (3) locations: syslog, kern.log and debug. Let’s clean that up.
New settings (old syntax)
New settings (new syntax)
We’ve excluded kernel messages from the syslog, and put a stop on kern.debug messages to prevent them being picked up by subsequent filters. This ensures that our iptables LOG messages now only appear in kern.log and not in the syslog or debug logs.
The syntax for the stop has changed from tilde (~) to the word ‘stop’ in the latest version.
You’ll notice that the minuses have also been removed in the new syntax. They are used in the old version to prevent excess disk writes, but not needed and not used in the new version of rsyslog.
An alternative approach used by some is:
This creates a new log file just for iptables debugging.
Remember that any log files you add will need to be rotated using logrorate or a similar facility so they can’t grow to fill up the partition.
6. Restarting rsyslog
For any configuration changes to take affect you need to restart the rsyslog daemon
Under the old ‘init’ system:
service rsyslog restart
And under the new ‘systemd’ system:
systemctl restart rsyslog.service
7. Creating a basic filter
Here’s a quick example showing how you can split off certain entries into a new log file. In this case we’re separating out spamd log entries from the main mail.log.
Create a file /etc/rsyslog.d/spamd.conf with:
if $programname == 'spamd' then /var/log/spamd.log
This will now intercept any log entries matching spamd[PID] and pipe them into a new spamd.log.
After this you should also update /etc/logrotate.d/rsyslog to recognise and rotate the new log file along with the standard mail logs:
8. More advanced filtering
There are now three types of filters available in rsyslog. We’ve been working with the “traditional” selectors, but there are also RainerScript-based filters and property-based filters which you can read up on here. They allow for regular expression matching among other things.
9. rsyslog errors after upgrade
On a separate topic, you may find that after upgrading rsyslog you start seeing errors similar to the following:
May 24 06:25:33 hostname rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="1812" x-info="http://www.rsyslog.com"] rsyslogd was HUPed
May 24 06:26:39 hostname rsyslogd0: action 'action 17' resumed (module 'builtin:ompipe') [try http://www.rsyslog.com/e/0 ]
May 24 06:26:39 hostname rsyslogd-2359: action 'action 17' resumed (module 'builtin:ompipe') [try http://www.rsyslog.com/e/2359 ]
May 24 06:55:58 hostname rsyslogd-2007: action 'action 17' suspended, next retry is Sun May 24 06:56:28 2015 [try http://www.rsyslog.com/e/2007 ]
These stem from rsyslog trying to write to a destination that doesn’t exist. In particular you’re probably not running an xconsole so shouldn’t be piping anything to /dev/xconsole.
The fix is to comment out the /dev/xconsole section of rsyslog.conf as follows:
# *.=notice;*.=warn |/dev/xconsole
And then restart the rsyslog daemon.