iWebDev's blog

In the previous post you installed the syslog-ng 3.2.2. Now you have to configure our syslog-ng daemon to collect events to database; for this tutorial we choosed a MySQL and Postgres databases. First of all you have to configure the syslog-ng configuration file.

nano /opt/syslog-ng/etc/syslog-ng.conf

Syslog-ng receives log messages from a source. To define a source you should follow the following syntax:

source <identifier> { source-driver(params); source-driver(params); … };

For example you have to define the following source:

source my_source{ tcp ( port ( 614 ) ); };

In syslog-ng log messages are sent to files. The destination syntax is very similar to sources:

destination <identifier> {destination-driver(params); destination-driver(params); … };

You will be normally logging to a file, but you could log to a different destination-driver: pipe, unix socket, TCP-UDP ports, terminals or to specific programs.

destination my_dest{ file(“/var/log/mylog.txt”); };

CREATE DATABASE `syslog` DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;

USE `syslog`;

CREATE TABLE IF NOT EXISTS `logs` (
`id` bigint(20) unsigned NOT NULL auto_increment,
`host` varchar(128) collate utf8_unicode_ci default NULL,
`facility` varchar(10) collate utf8_unicode_ci default NULL,
`priority` varchar(10) collate utf8_unicode_ci default NULL,
`level` varchar(10) collate utf8_unicode_ci default NULL,
`tag` varchar(10) collate utf8_unicode_ci default NULL,
`datetime` datetime default NULL,
`program` varchar(15) collate utf8_unicode_ci default NULL,
`msg` text collate utf8_unicode_ci,
`seq` bigint(20) unsigned NOT NULL default ’0′,
`counter` int(11) NOT NULL default ’1′,
`fo` datetime default NULL,
`lo` datetime default NULL,
PRIMARY KEY (`id`),
KEY `datetime` (`datetime`),
KEY `sequence` (`seq`),
KEY `priority` (`priority`),
KEY `facility` (`facility`),
KEY `program` (`program`),
KEY `host` (`host`) )
ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

GRANT SELECT , INSERT , UPDATE , DELETE , CREATE , DROP , INDEX , ALTER ON `syslog` . * TO ‘syslog’@'localhost’;

SET PASSWORD FOR ‘syslog’@'localhost’ = PASSWORD( ‘syslog’ )

Edit syslog-ng config appropriately; add these rows in the destination section (if you want to use Postgres you have to change mysql to pgsql):

sql(type(mysql)
host(“localhost”)
username(“syslog”)
password(“syslog”)
database(“syslog”)
table(“logs”)
columns(“host”, “facility”, “priority”, “level”, “tag”, “datetime”, “program”, “msg”, “seq”)
values(“$HOST_FROM”, “$FACILITY”, “$PRIORITY”, “$LEVEL”, “$TAG”, “$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC”, “$PROGRAM”, “$MSG”, “$SEQNUM”)
indexes(“host”, “facility”, “priority”, “datetime”, “program”, “seq”));

Syslog-ng connects sources, filters and destinations with log statements. The syntax is:

log { source(src); filter(f_mail); filter(f_info); destination(mailinfo); };

So you have to connect my_source with my_dest:

log { source( my_source ); destination( my_dest ); };

Syslog-ng is an open source implementation of the Syslog protocol for Unix and Unix-like systems. It extends the original syslogd model with content-based filtering, rich filtering capabilities, flexible configuration options and adds important features to syslog, like using TCP for transport. In syslog-ng starting from version 3.0 there is a great option of forward logs directly to database (Postgres, or for that matter to MySQL, Firebird or sqlite database). In comparison with the old way of doing that, namely using a pipe and executing either a wrapper script or mysql client directly, the new way saves a great deal of resources as syslog-ng does not need to start a process every time there is a log message to log. So if you want this features you have to install syslog-ng of version 3.0 or greater with use flag sql enabled. In order to install syslog-ng you have to download the right version from the official site. For our purpose we download the syslog-ng 3.2.2 version (3.2.2/setups/linux-glibc2.3.6-i386).

wget http://www.balabit.com/downloads/files?path=/syslog-ng/sources/3.2.2/setups/linux-glibc2.3.6-i386/syslog-ng-3.2.2-linux-glibc2.3.6-i386.run

Once you downloaded the file you have to grant execute permission to syslog-ng-3.2.2-linux-glibc2.3.6-i386.run.

chmod +x syslog-ng-3.2.2-linux-glibc2.3.6-i386.run

Now you are ready to install the syslog-ng.

./syslog-ng-3.2.2-linux-glibc2.3.6-i386.run

The first screen shows the path where the syslog-ng will be installed; you have to presso “continue”.

The second screen resumes the parameters about your system; press “yes” if the information are corrects.

The third screen suggest user to check if the “/opt/syslog-ng/bin” and “/opt/syslog-ng/sbin” directory are in the search PATH. In order to do so, please add the following line into the shell profile:

PATH=/opt/syslog-ng/bin:$PATH

The fourth step checks if there is old version of syslog-ng installed. If the installer has detected a configuration file from a previous syslog-ng installation, the user can use this old configuration file. We choose “no”.

The installer generates a simple configuration file and asks if user wants to receive log messages from the network. We choose “yes”.

The last step asks user if he wants forward the log messages to a remote server; we choose “skip”.

Congratulation, we installed syslog-ng 3.2.2.