Pollux's corner

[archives] [tags]

links

Tags

ulogd2: the new userspace logging daemon for netfilter/iptables (part 2)

Permalink 2008-04-04 20:39:00+02:00 By Pollux Category General Tags Firewall Logs Netfilter MySQL PostgreSQL Ulogd

This article explains how to build, install and configure ulogd 2 for use with netfilter/iptables. It explains how to use plugins to store logs in databases (MySQL and PostgreSQL), use plugins to filter data, and gives some iptables rules to log packets.

Get the sources

You can use the official repository:

git clone git://git.netfilter.org/ulogd2.git/

Prerequisites

Build

Use the standard autotools method for configure, build and install:

./autogen.sh
./configure --prefix=/path/to/prefix
make
sudo make install

Configuration

Edit ulogd.conf

1. enable plugins

You will have to choose the input and output plugins according to your setup. NFLOG is present in recent kernels (and iptables installation), and should be preferred if possible.

  • Input plugin: ULOG or NFLOG
  • Output: MySQL or PostgreSQL

You have to enable the corresponding in the configuration before you can use them:

plugin="/path/to/prefix/lib/ulogd/ulogd_inppkt_ULOG.so"
plugin="/path/to/prefix/lib/ulogd/ulogd_output_MYSQL.so"

See “Stack configuration” later.

2. buid the stack

For MySQL, we will use a very simple plugin stack. As MySQL is quite inefficient in storing IP addresses (and has no standard type for it), we will log the IP addresses in binary format using the IP2BIN plugin. You won’t be able to run SQL commands directly, but GUI tools (like Nulog can decode them.

stack=log1:ULOG,base1:BASE,ip2bin1:IP2BIN,mysql1:MYSQL

For PostgreSQL, the stack is similar, except that we choose to log IP addresses using the standard inet type.

stack=log1:NFLOG,base1:BASE,ifi1:IFINDEX,ip2str1:IP2STR,pgsql1:PGSQL

3. configure input plugin

[log1]
# netlink multicast group (the same as the iptables --ulog-nlgroup param)
nlgroup=1

4. configure output plugin

[mysql1]
db="ulog2"
host="localhost"
user="ulog2"
table="ulog"
pass="password"
procedure="INSERT_PACKET_FULL"

The configuration is the same as usual and should be easy, except maybe for the last parameter: it is the name of a procedure which will be used to insert data.

For PostgreSQL, you can use the following:

[pgsql1]
db="ulog2"
host="localhost"
user="ulog2"
table="ulog"
pass="ulog2"
procedure="INSERT_PACKET_FULL"

The configuration is the same, assuming you are running on the default port and have configured PostgreSQL, and can connect.

Stack configuration

Each module is a plugin which is dynamically loaded during ulogd start. A module has:

  • Input keys: It defined the entry which are needed (mandatory or not) by the module to be able to work (ie output other keys)
  • Output keys: The module outputs key->value association with key in the output list.

Each key has a type and may have a default value.

To have a working stack, you will need to provide an input module and filter which will be able to modify the initial set of key->value provided by the input module in a set of key->value which will contain all the input keys which are mandatory for the output plugin.

Information about a given module can be obtained through the info command of ulogd:

# /opt/ulogd2/sbin/ulogd --info /opt/ulogd2/lib/ulogd/ulogd_filter_IP2STR.so
Name: IP2STR
Input keys:
       Key: oob.family (unsigned int 8)
       Key: oob.protocol (unsigned int 16)
       Key: ip.saddr (IP addr)
       Key: ip.daddr (IP addr)
       [...]
Output keys:
       Key: ip.saddr.str (string)
       Key: ip.daddr.str (string)
       [...]

From this output, we see that the IP2STR module output IP in string format and take in input IP in rw format and some additional information (oob.family and oob.protocol).

Input plugins

  • ULOG: get packet select via the iptables target ULOG
  • NFLOG: get packet from NFLOG target which is the successor of ULOG
  • NFCT: get flow information from Netfilter connection tracking via libnetfilter_conntrack

Output plugins

  • LOGEMU: log packet/flow into a file
  • OPRINT: log packet/flow to a file in multiline format
  • SYSLOG: log packet/flow to syslog system
  • MYSQL: log packet/flow to a MySQL database
  • PGSQL: log packet/flow to a PGSQL server
  • SQLITE3: log packet to a SQLITE3 file
  • PCAP: log packet to a Pcap file
  • IPFIX: log flow via IP Flow Information Export
  • NACCT: log flow to a nacct compatible format (accounting)

Create database schema

PostgreSQL

1. create user and database

# su - postgres
$ createuser -P ulog2
$ createdb -O ulog2 ulog2

2. add support for plpgsql language (for procedures)

$ createlang plpgsql ulog2

3. insert schema

$ psql -U ulog2 -h localhost ulog2 -f doc/pgsql-ulogd2.sql

MySQL

1. create user and database

# mysql -uroot mysql
> CREATE DATABASE ulog2;
> GRANT ALL PRIVILEGES TO 'ulog2'@'localhost' IDENTIFIED BY 'password';

2. insert schema

$ mysql -uulo2 -ppassword ulog2 < doc/mysql-ulogd2.sql

Start ulogd2

As root, start the process:

/path/to/prefix/sbin/ulogd

If an error happens, it will only print:

Fatal error, check logfile.

Edit /var/log/ulogd.log (be careful, you will have to scroll many lines, so be sure not to use tail).

Wed Mar 26 22:19:54 2008 <7> ulogd.c:698 cannot find key `timestamp' in stack
Wed Mar 26 22:19:54 2008 <1> ulogd.c:807 destroying stack

Add iptables rules

iptables -A OUTPUT -p tcp --dport 80 -m state --state NEW -j ULOG

Check the result

mysql> select count(*) from ulog2;
+--+
| count(*) |
+--+
|        5 |
+--+
1 row in set (0.00 sec)

So now, we are logging to the database. Next time, we will detail the SQL schema: how to get information, how to insert data, tables, view, procedures, etc.

References