Configuring SNMP Trap Receiver for Zabbix on Debian

Monitoring
Comments

In order to handle SNMP traps in Zabbix you need to configure your server to receive the traps. Here are the steps, tested with Zabbix 5.4 on Debian Linux 10 (Buster), assuming Zabbix server has already been installed from the official repository:

(Note: Long commands and paths below can appear split incorrectly, so be careful with them)

  1. Install the required packages:
    sudo apt install snmptrapd libsnmp-perl
  2. Get the zabbix_trap_receiver.pl script from Zabbix source repository:
    sudo wget https://git.zabbix.com/projects/ZBX/repos/zabbix/raw/misc/snmptrap/zabbix_trap_receiver.pl -O /etc/zabbix/zabbix_trap_receiver.pl
  3. Edit the SNMPTrapperFile variable in /etc/zabbix/zabbix_trap_receiver.pl:
    $SNMPTrapperFile = '/var/log/snmptrap/snmptrap.log';
  4. Create the directory:
    sudo mkdir /var/log/snmptrap
  5. Configure snmptrapd with these commands (where my_trap is your desired trap community string):
    echo 'authCommunity execute my_trap' | sudo tee -a /etc/snmp/snmptrapd.conf
    echo 'perl do "/etc/zabbix/zabbix_trap_receiver.pl";' | sudo tee -a /etc/snmp/snmptrapd.conf
  6. Edit and/or verify the parameters in /etc/zabbix/zabbix_server.conf:
    SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
    StartSNMPTrapper=1
  7. Restart the services:
    sudo systemctl restart snmptrapd zabbix-server
  8. Set the trap receiver service to start automatically at reboot:
    sudo systemctl enable snmptrapd

Note that if you want to receive the traps on a Zabbix proxy instead of Zabbix server, the steps are pretty much the same, you just need to edit zabbix_proxy.conf instead of zabbix_server.conf and restart zabbix-proxy after that.

Now the trap receiving should work and the traps should show up in /var/log/snmptrap/snmptrap.log.

Testing the traps

For testing you can use the following snmptrap command (where x.x.x.x is the IP address of your Zabbix server where you installed the trap receiver on; install snmp package with sudo apt install snmp if the snmptrap command is not present yet):

snmptrap -v 2c -c my_trap x.x.x.x "" 1.3.6.1.4.1.8072.9999.9999

You can ignore the “read_config_store open failure on /var/lib/snmp/snmpapp.conf” error messages for purpose of this testing.

Now you can check the trap log file and you should see similar results to this:

$ cat /var/log/snmptrap/snmptrap.log
22:12:33 2021/05/21 ZBXTRAP x.x.x.x
PDU INFO:
  requestid                383425449
  transactionid            5
  community                my_trap
  notificationtype         TRAP
  errorstatus              0
  errorindex               0
  messageid                0
  version                  1
  receivedfrom             UDP: [x.x.x.x]:52828->[x.x.x.x]:162
VARBINDS:
  iso.3.6.1.2.1.1.3.0      type=67 value=Timeticks: (173991) 0:28:59.91
  iso.3.6.1.6.3.1.1.4.1.0  type=6  value=OID: iso.3.6.1.4.1.8072.9999.9999

If that is fine, you should also see this in /var/log/zabbix/zabbix_server.log:

$ sudo cat /var/log/zabbix/zabbix_server.log
...
   731:20210521:221235.131 unmatched trap received from "x.x.x.x": 22:12:33 2021/05/21 PDU INFO:
...

Note: If you don’t see the unmatched trap error in the Zabbix server log (but you see the trap saved in snmptrap.log), there is a setting in Zabbix GUI that affects the logging of unmatched traps: Administration – General – Other – Log unmatched SNMP traps. The setting is enabled by default. It is meant to get you an indication about traps that you receive but you haven’t configured any item in Zabbix. Naturally this error is also not present if you already have configured Zabbix host with a matching SNMP trap item.

Loading the MIBs

In the example above the object identifiers are shown in numerical form (like iso.1.3.6.1.4.1.8072.9999.9999). If you want to resolve and use the names, you need to download the MIB files and enable loading them. There are a couple of steps required to do that on Debian:

  1. Edit /etc/apt/sources.list to include the non-free repository on each non-commented line. For example, when the lines are something like “deb <url> buster main“, add “non-free” there on each line, like this:
    “deb <url> buster main non-free“.
  2. Update the package data:
    sudo apt update
  3. Install the SNMP MIBs:
    sudo apt install snmp-mibs-downloader
  4. Edit /etc/snmp/snmp.conf and comment out the “mibs :” line, like this:
    #mibs :
  5. Restart the trap receiver:
    sudo systemctl restart snmptrapd

Test the trap sending again, and you will see something like this in /var/log/snmptrap/snmptrap.log:

VARBINDS:
  DISMAN-EVENT-MIB::sysUpTimeInstance type=67 value=Timeticks: (244631) 0:40:46.31
  SNMPv2-MIB::snmpTrapOID.0  type=6 value=OID: NET-SNMP-MIB::netSnmpPlaypen

The difference is that all the OIDs have been resolved to names that are defined in the MIB files.

You can also test with a longer command: snmptrap -v 2c -c my_trap x.x.x.x "" 1.3.6.1.4.1.8072.9999.9999 1.3.6.1.4.1.8072.9999.9999 s "My testing trap"

It results trap log like this:

VARBINDS:
  DISMAN-EVENT-MIB::sysUpTimeInstance type=67 value=Timeticks: (272411) 0:45:24.11
  SNMPv2-MIB::snmpTrapOID.0    type=6 value=OID: NET-SNMP-MIB::netSnmpPlaypen
  NET-SNMP-MIB::netSnmpPlaypen type=4 value=STRING: "My testing trap"

Configuring trap items in Zabbix

Now there is the basic capability completed to receive the SNMP traps in the server level. You will also need to configure relevant items in your hosts in Zabbix. See the Zabbix documentation about configuring SNMP traps for more information.

Some ideas:

  • If you want to save and handle all the incoming traps for the host you are configuring, add an item with type of SNMP Trap and key of snmptrap.fallback. Then Zabbix will happily ingest all the traps that the configured host sends.
  • If you only want to save and/or handle some specific traps, then use the item key snmptrap[<regex>] with an appropriate regular expression to match your trap. You will need to create one item for each specific match you want to handle in Zabbix.
  • Type of information for the item is Log, and log time format is hh:mm:ss yyyy/MM/dd (as configured in zabbix_trap_receiver.pl)
  • In triggers you can use for example the expression (in Zabbix 5.4 syntax) “find(/My_host/snmptrap.fallback,,,"Error") > 0” to create a problem event when the received trap data contains word Error. Enable “Allow manual close” in the trigger to be able to manually close the problem later.
  • Problem expression for triggering an interface down event for interface index 5 of host Switch:
    find(/Switch/snmptrap.fallback,,"regexp","IF-MIB::linkDown(\n.*)+IF-MIB::ifIndex.5 ")=1
  • Recovery expression for the same trigger:
    find(/Switch/snmptrap.fallback,,"regexp","IF-MIB::linkUp(\n.*)+IF-MIB::ifIndex.5 ")=1
    • Line breaks (“\n“) need to be explicitly matched in the regex as “.*” does not match them
  • Note that in order to Zabbix to link the incoming trap to the correct host the host in Zabbix needs to have an SNMP interface configured with the same IP address that the trap contains. Otherwise the trap will end up being unmatched.

Trap log file rotation

It is also a good idea to add rotation for the trap log file, for example with the following configuration file saved in /etc/logrotate.d/snmptrap:

/var/log/snmptrap.log {
    weekly
    rotate 12
    compress
    delaycompress
    missingok
    notifempty
}

Leave a Reply