Setting Up a Syslog Server

In the field of networking it is very useful to have a centralized location for your logs as the system itself (the network) is distributed. Syslog is the usual method of collecting the logs. There are lots of different solutions to collect syslogs, ranging from general-purpose servers or virtual machines running some syslog daemon software to dedicated appliances.

My chosen and quick model of implementing a syslog collector for my needs is a Debian GNU/Linux server with rsyslog daemon. Here is a quick guide for installing it (as of version 7.7 of Debian) in a VMware virtual machine.

Note that creating and administrating a real-life log server is a specific kind of art, including securing the server platform itself and taking care of the log content confidentiality and integrity. Take it very seriously if you are going to log other people’s actions and information some way. Consider this post good for labbing purposes only.

• Install the base system. I use the latest netinst image for amd64 (64-bit) architecture. I’m not going into the base installation details. Deselect the workstation-related options and select the SSH Server option in the process. After getting the system up and running, let’s see some additional packages and configurations.
• Install VIM. Why? Because it just happens to work nicer than the default VI editor. I don’t care about the details, I just run “apt-get install vim” (the standard install method in this post) on every Debian install and I get better experience of VI. If you prefer some other editors, go ahead and install them (I heard there are some newcomers like pico or nano), or maybe they are installed already.
• Install open-vm-tools. VMware recommends it (in their KB2073803) (vmware.com) so you don’t have to install VMware Tools manually. Open-vm-tools installs nicely and easily. It automatically suggests the dependencies that are needed for it.
• Install ntp to keep server clock in good shape. Edit the /etc/ntp.conf to reflect your specific NTP server addresses if needed. Restart the NTP server (“service ntp restart“, the standard way of restarting a service in this post) if you are not rebooting the system otherwise to get the configuration change in effect.
• Install unattended-upgrades. It will install the most important OS upgrades automatically every day. You can also configure it further in /etc/apt/apt.conf.d/50unattended-upgrades if needed. For kernel upgrades you will need to reboot the server anyway at some point.
• Remove some unnecessary running stuff, somewhat hardening or otherwise cleaning the system: “apt-get remove rpcbind nfs-common mpt-status“.
• I tend to replace Exim mail server with Postfix, just because I don’t know Exim at all but know at least something about Postfix. So I install postfix, and it removes Exim automatically in the process. Enter the address of your (or your ISP’s) SMTP relay in the Smarthost setting when asked if you want to send email out. I also edit /etc/postfix/main.cf to say “inet_interfaces = loopback-only” instead of all interfaces because I don’t expect incoming SMTP connections. Again, practicing the habbit of smallest intrusion surface. Restart postfix to get the change deployed.
• Finally we get in configuring the syslog server itself. rsyslog is already running automatically, but we will edit it a bit to get it listening to syslog traffic. Edit /etc/rsyslog.conf and uncomment these lines (remove the # characters from the beginning of the lines):

$ModLoad imudp
$UDPServerRun 514

• In the #### RULES #### section add these:

:HOSTNAME, !isequal, "syslog01"      /var/log/remotelogs
:HOSTNAME, !isequal, "syslog01"      ~

• Note that “syslog01” above is the name of this server itself, so replace it to match your server hostname. These configuration lines mean: When there is an incoming log message, check the hostname in the message, and if it doesn’t match “syslog01” (= it is basically coming from some other host with syslog protocol) then log the message to /var/log/remotelogs and stop processing further rules. This way only the syslog messages go to this additional log file, and other system logs (the local logs) still go to the normal places, as configured in the default rsyslog.conf.
• Restart rsyslog to get the new config deployed.
• Create a new file /etc/logrotate.d/remotelogs and enter this configuration in it:

/var/log/remotelogs {
        rotate 185
        daily
        dateext
        missingok
        compress
        postrotate
                invoke-rc.d rsyslog rotate > /dev/null
        endscript
}

• This configuration instructs logrotate (that runs daily) to take care of our remotelogs log file and keep 185 days worth of logs. No need to restart logrotate for this change as the configuration files are read in each daily run. The older log files will be gzipped and the current date will be added to the filenames.

That’s about it in the server side. To see that rsyslog is listening to the syslog (514/UDP) traffic in both IPv4 and IPv6:

root@syslog01:~# netstat -ln | grep :514
udp        0      0 0.0.0.0:514             0.0.0.0:*
udp6       0      0 :::514                  :::*

Then you can reconfigure your network devices to start sending syslogs to your new syslog server. Check /var/log/remotelogs for any incoming messages, for example with “tail -f /var/log/remotelogs” command.

If you have syslog implementation hints for other operating systems or distributions, feel free to comment below.