Quick Setup Guide for super_mediator

This tutorial is a step-by-step guide of setting up yaf and super_mediator. For a detailed tutorial on super_mediator and the different configuration options, see this tutorial. This particular tutorial takes the DPI data produced by yaf and imports the data into a MySQL database. This also enables DNS deduplication in super_mediator This tutorial does not give an example of how to setup SiLK. See this page for a tutorial that includes SiLK.

Install Procedure

Install prerequisites

$ sudo yum groupinstall "Development Tools"
$ sudo yum install libpcap-devel pcre-devel mysql-server* mysql-devel*

*RHEL7 mariadb-server mariadb-devel

Build libfixbuf:

$ tar -xvzf libfixbuf-1.7.0.tar.gz
$ cd libfixbuf-1.7.0
$ ./configure
$ make
$ sudo make install

Build yaf:

$ tar -xvzf yaf-2.7.0.tar.gz
$ cd yaf-2.7.0
$ ./configure --enable-applabel --enable-plugins
$ make
$ sudo make install

Build super_mediator:

$ tar -xvzf super_mediator-1.2.0.tar.gz
$ cd super_mediator-1.2.0
$ ./configure --with-mysql
$ make
$ sudo make install

Setup mysqld

$ sudo service [mysqld|mariadb] start

Setup a password for the root user:

$ /usr/bin/mysqladmin -u root password '<SuperSecretPassword>'

Login to the database (It will prompt you for the password you created in the last step):

$ mysql -u root -p

Create the database you intend to use for super_mediator:

mysql> create database smediator;

Create a user for super_mediator to access the database:

mysql> CREATE USER 'mediator'@'localhost' IDENTIFIED BY '<SuperSecretPassword>';

Giver permissions to user to access only the smediator database:

mysql> GRANT ALL ON smediator.* TO mediator@'localhost';

Create MySQL Tables

Use super_table_creator to create all the tables in your database:

$ /usr/local/bin/super_table_creator --name mediator \
        --pass=<SuperSecretPassword> --database=smediator
$ /usr/local/bin/super_table_creator --name mediator \
        --pass=<SuperSecretPassword> --database=smediator --dns-dedup

Configure super_mediator

Create output directories:

$ sudo mkdir -p /data/smediator/dpi
$ sudo mkdir -p /data/smediator/dns

Create your super_mediator.conf file. One is installed by default into /usr/local/etc. The following one will get you started (you should add your <SuperSecretPassword>):

COLLECTOR TCP
  PORT 18000
COLLECTOR END

#dedup process
EXPORTER TEXT
  PATH "/data/smediator/dns/yaf2dns"
  DELIMITER "|"
  ROTATE 1200
  DNS_DEDUP_ONLY
  LOCK
  MYSQL_USER "mediator"
  MYSQL_PASSWORD "<SuperSecretPassword>"
  MYSQL_TABLE "dns-dedup"
  MYSQL_DATABASE "smediator"
EXPORTER END

#dpi 2 database
EXPORTER TEXT
  PATH "/data/smediator/dpi"
  ROTATE 1200
  MULTI_FILES
  DPI_ONLY
  LOCK
  MYSQL_USER "mediator"
  MYSQL_PASSWORD "<SuperSecretPassword>"
  MYSQL_DATABASE "smediator"
EXPORTER END

DNS_DEDUP
  MAX_HIT_COUNT 5000
DNS_DEDUP END

LOGLEVEL DEBUG

LOG "/var/log/super_mediator.log"

PIDFILE "/data/super_mediator.pid"

Start the Tools

Start super_mediator

$ super_mediator -c /usr/local/etc/super_mediator.conf --daemonize

Confirm super_mediator is running:

$ ps -ef | grep super

If super_mediator is not running, check for any errors:

$ cat /var/log/super_mediator.log

Start YAF:

$ sudo mkdir /var/log/yaf

$ export LTDL_LIBRARY_PATH=/usr/local/lib/yaf

Run YAF over PCAP file:

$ /usr/local/bin/yaf
        --in <PCAP FILE> \
        --ipfix tcp \
        --out localhost \
        --log /var/log/yaf/yaf.log \
        --verbose \
        --silk \
        --ipfix-port=18000 \
        --applabel \
        --max-payload 2048 \
        --plugin-name=/usr/local/lib/yaf/dpacketplugin.so

OR Run YAF on interface eth0:

$ sudo /usr/local/bin/yaf
        --in eth0 \
        --live pcap \
        --ipfix tcp \
        --out localhost \
        --log /var/log/yaf/yaf.log \
        --verbose \
        --silk \
        --ipfix-port=18000 \
        --applabel \
        --max-payload 2048 \
        --plugin-name=/usr/local/lib/yaf/dpacketplugin.so