Install Prerequisites

You should have a fully functional YUM/RPM-based system; these instructions use CentOS 7. Run the following command to install the basic development tools (e.g., a C compiler) required to build SiLK and YAF.

sudo yum -y install gcc gcc-c++ make pkgconfig

Alternatively, you may install the tools for a complete development environment:

sudo yum -y group install "Development Tools"

Install the rpm-build package, and create the RPM build hierarchy. (These instructions use ~/rpm as the root of the build hierarchy, but it may be located anywhere.)

sudo yum -y install rpm-build
cd ~
mkdir rpm rpm/BUILD rpm/RPMS rpm/SOURCES rpm/SPECS rpm/SRPMS

Create the ~/.rpmmacros file with your favorite editor, using this as a template:

%packager     your name <you@example.com>
%_topdir      /home/you/rpms

Run the following commands to install the GLib-2, PCAP, Python, LZO, zlib, and GnuTLS development libraries:

sudo yum -y install glib2-devel libpcap-devel python-devel \
    lzo-devel zlib-devel gnutls-devel

If you want to use the MaxMind country-code mapping capability of SiLK, you must install the libmaxminddb-devel package which is available from the Extra Packages for Enterprise Linux (EPEL) repository:

sudo yum -y install epel-release
sudo yum -y update epel-release
sudo yum -y install libmaxminddb-devel

Download Software

Download the libfixbuf, SiLK, and YAF source code from the tools.netsa.cert.org web site.

cd ~/rpm/SOURCES
wget https://tools.netsa.cert.org/releases/silk-3.19.1.tar.gz
wget https://tools.netsa.cert.org/releases/libfixbuf-2.4.1.tar.gz
wget https://tools.netsa.cert.org/releases/yaf-2.12.2.tar.gz

Each NetSA package has an RPM .spec file template. When you run the package's configure script, the template is customized using the features (command line options) you specified to the script.

Create RPMs for libfixbuf and Install Them

Expand the libfixbuf source code, run configure, and copy the generated libfixbuf.spec file into the RPM build hierarchy:

cd /tmp
tar -zxf ~/rpm/SOURCES/libfixbuf-2.4.1.tar.gz
cd libfixbuf-2.4.1
./configure
cp libfixbuf.spec ~/rpm/SPECS/.

Build the libfixbuf RPMs:

cd ~/rpm
rpmbuild -ba SPECS/libfixbuf.spec
ls RPMS/x86_64

The output from the last command should resemble:

x86_64/libfixbuf-2.4.1-1.el7.x86_64.rpm
x86_64/libfixbuf-debuginfo-2.4.1-1.el7.x86_64.rpm
x86_64/libfixbuf-devel-2.4.1-1.el7.x86_64.rpm
x86_64/libfixbuf-ipfixDump-2.4.1-1.el7.x86_64.rpm

Install the libfixbuf and libfixbuf-devel packages which are prerequisites for building SiLK and YAF.

sudo yum -y install x86_64/libfixbuf-2*.rpm \
    x86_64/libfixbuf-devel-2*.rpm

Create RPMs for SiLK and Install Them

The following commands unpack the SiLK sources and create the silk.spec file. The switches to the configure command do the following:

• set the default location for SiLK's hourly repository of flow files to /var/silk/data (The default location is /data.)
• enable support for IPv6 addresses and flow records (The default is to support only IPv4 flow records.)
• enable creating IPset files in the most compact format (The format may be changed at run-time by providing the --record-version switch to an IPset tool; see also the SILK_IPSET_RECORD_VERSION environment variable.)
• enable automatic (de-)compression of binary SiLK files when reading and writing (The default is no compression. The compression may be changed at run-time by providing the --compression-method switch when invoking a tool; see also the SILK_COMPRESSION_METHOD environment variable.)
• enable the SiLK Python extension (PySiLK)

cd /tmp
tar -zxf ~/rpm/SOURCES/silk-3.19.1.tar.gz
cd silk-3.19.1
./configure                              \
    --enable-data-rootdir=/var/silk/data \
    --enable-ipv6                        \
    --enable-ipset-compatibility=3.14.0  \
    --enable-output-compression          \
    --with-python
cp silk-3.19.1.spec ~/rpm/SPECS/.

Build the SiLK RPMs:

cd ~/rpm
rpmbuild -ba SPECS/silk-3.19.1.spec
ls RPMS/x86_64/silk*

The output from the final command should look similar to:

RPMS/x86_64/silk-analysis-3.19.1-1.el7.x86_64.rpm
RPMS/x86_64/silk-common-3.19.1-1.el7.x86_64.rpm
RPMS/x86_64/silk-debuginfo-3.19.1-1.el7.x86_64.rpm
RPMS/x86_64/silk-devel-3.19.1-1.el7.x86_64.rpm
RPMS/x86_64/silk-flowcap-3.19.1-1.el7.x86_64.rpm
RPMS/x86_64/silk-rwflowappend-3.19.1-1.el7.x86_64.rpm
RPMS/x86_64/silk-rwflowpack-3.19.1-1.el7.x86_64.rpm
RPMS/x86_64/silk-rwpollexec-3.19.1-1.el7.x86_64.rpm
RPMS/x86_64/silk-rwreceiver-3.19.1-1.el7.x86_64.rpm
RPMS/x86_64/silk-rwsender-3.19.1-1.el7.x86_64.rpm

Install at least the silk-analysis and silk-rwflowpack packages (silk-common is a prerequisite for both):

sudo yum -y install x86_64/silk-common*.rpm \
    x86_64/silk-analysis*.rpm x86_64/silk-rwflowpack*.rpm

Create an RPM for YAF and Install It

The following commands create the yaf.spec file, enabling all of YAF's application labeling and deep packet inspection (DPI) capabilities, use the file to create the RPM packages for YAF, and install the YAF application. If you are building yaf-3.0.0 or later, add --enable-dpi to the configure options.

cd /tmp
tar -zxf ~/rpm/SOURCES/yaf-2.12.2.tar.gz
cd yaf-2.12.2
./configure                 \
    --enable-applabel       \
    --enable-metadata       \
    --enable-plugins
cp yaf.spec ~/rpm/SPECS/.
cd ~/rpm
rpmbuild -ba SPECS/yaf.spec
sudo yum -y install x86_64/yaf-2.*.rpm

Install libfixbuf

Unpack, configure, and install libfixbuf into the /usr/local directory. (This section assumes you downloaded the source code to /tmp.)

cd /tmp
tar -zxf /tmp/libfixbuf-2.4.1.tar.gz
cd libfixbuf-2.4.1
./configure               \
    --prefix=/usr/local   \
    --enable-silent-rules
make
sudo make install

Install SiLK

Unpack, configure, and install SiLK into the /usr/local directory. Most of the options to configure are described in the previous section; the additional switch --with-python-prefix causes the PySiLK files to be installed under the SiLK installation tree (/usr/local) instead of with the other Python packages.

cd /tmp
tar -zxf /tmp/silk-3.19.1.tar.gz
cd silk-3.19.1
./configure                              \
    --prefix=/usr/local                  \
    --enable-silent-rules                \
    --enable-data-rootdir=/var/silk/data \
    --enable-ipv6                        \
    --enable-ipset-compatibility=3.14.0  \
    --enable-output-compression          \
    --with-python                        \
    --with-python-prefix
make
sudo make install

Install YAF

Unpack, configure, and install YAF into the /usr/local directory, enabling YAF's application labeling and deep packet inspection (DPI) capabilities. If you are building yaf-3.0.0 or later, add --enable-dpi to the configure options.

cd /tmp
tar -zxf /tmp/yaf-2.12.2.tar.gz
cd yaf-2.12.2
./configure                 \
    --prefix=/usr/local     \
    --enable-silent-rules   \
    --enable-applabel       \
    --enable-metadata       \
    --enable-plugins
make
sudo make install

Update Dynamic Linker

Instead of exporting LD_LIBRARY_PATH each time you use SiLK, it is easier to add the /usr/local/lib paths to ld.so.conf. Create a file named silk.conf containing the following lines that specify the library directories for SiLK:

/usr/local/lib
/usr/local/lib/silk

Now copy the file into the /etc/ld.so.conf.d directory and run ldconfig.

sudo mv silk.conf /etc/ld.so.conf.d/.
sudo ldconfig

Configure SiLK

The first step to configuring SiLK is to create the data repository directory (/var/silk/data) and add the silk.conf file, which defines how your data is stored (see the manual page for details). Use the default silk.conf file for the twoway site, which is installed at /usr/share/silk/twoway-silk.conf. You may edit the sensor descriptions if desired. The default settings cause the SiLK analysis program rwfilter to consider only incoming data unless the user provides the --type or --flowtypes switch. If you want rwfilter to look at both incoming and outgoing data by default, modify the default-types line to include in inweb out outweb. If desired, also add int2int ext2ext. See the twoway packing logic manual page for the meanings of these types.

sudo mkdir -p /var/silk/data
sudo chmod go+rx /var/silk /var/silk/data
sudo cp /usr/share/silk/twoway-silk.conf /var/silk/data/silk.conf

Configure rwflowpack

Next create the sensors.conf file that is used by rwflowpack for collecting data from yaf. Details on the file's syntax are in its manual page. Use your text editor to create a file that contains the following lines.

IMPORTANT: Make sure the ipblocks in the my-network section match your "internal" network blocks according to the ethernet interface (e.g., enp0s3, eth0) where you will configure yaf to listen below.

probe S0 ipfix
 listen-on-port 18001
 protocol tcp
 listen-as-host 127.0.0.1
end probe

group my-network
 ipblocks 192.168.1.0/24  # address of ethernet interface. CHANGE THIS.
 ipblocks 10.0.0.0/8      # other blocks considered internal. OPTIONAL.
end group

sensor S0
 ipfix-probes S0
 internal-ipblocks @my-network
 external-ipblocks remainder
end sensor

(Once the entire system is running, if you discover that all your records show up as type ext2ext, it means you did not configure your internal netblocks correctly.)

Move this file into place.

sudo mv sensors.conf /var/silk/sensors.conf

SiLK comes with traditional "init script" start-up files, with two files per daemon: One file is invoked by the system to start the daemon, and the other contains configuration settings used by the first. Edit the start-up configuration script for rwflowpack, /etc/sysconfig/rwflowpack.conf to change these variables to the values shown here:

ENABLED=1
statedirectory=/var/silk
SENSOR_CONFIG=/var/silk/sensors.conf
ARCHIVE_DIR=  # empty
LOG_TYPE=legacy
LOG_DIR=/var/log

If you used the instructions to install the tools outside of YUM, then:

• Copy the start-up configuration file /usr/local/share/silk/etc/rwflowpack.conf to /usr/local/etc/rwflowpack.conf and edit it as shown above.
• Copy the start-up script /usr/local/share/silk/etc/init.d/rwflowpack to /etc/init.d/rwflowpack before the next step.

Start rwflowpack:

sudo systemctl start rwflowpack.service

Look at the rwflowpack log file, /var/log/rwflowpack-YYYYYMMDD.log to ensure that it is running.

Start YAF

Like SiLK, YAF has two "init script" start-up files. If you used the instructions to install the tools outside of YUM, you must copy the yaf start-up script from the source directory to /etc/init.d/yaf and enable its execute permissions:

sudo cp /tmp/yaf-2.12.2/etc/init.d/yaf /etc/init.d/yaf
chmod a+w /etc/init.d/yaf

Edit the YAF start-up configuration file, /etc/yaf.conf, setting these values:

IMPORTANT: Make sure the interface (enp0s3) matches the interface on which you want to capture.

ENABLED=1
YAF_CAP_IF=enp0s3      # Ensure this is correct for your machine
YAF_IPFIX_PORT=18001   # Must match value in sensors.conf
YAF_EXTRAFLAGS="--silk --applabel --max-payload=512"

Start yaf:

sudo systemctl start yaf.service

Look at the yaf log file, /var/log/yaf.log to ensure that it is running.

Check the rwflowpack log file again to ensure that it received a connection from yaf. There should be a message similar to the following (the port number will likely differ):

'S0': accepted connection from 127.0.0.1:36734

If nothing is happening, it could be that yaf is waiting for network traffic. Follow the steps in the next section to generate some.

Generate traffic

Generate some traffic and wait records to be flushed. Run the following ping command, or use your web browser or the wget command to access web sites.

ping -c 4 8.8.8.8

You may have to wait 5 or 10 minutes for the first records to be flushed, but you should see messages in the rwflowpack log file that it is creating files. You may check the status of YAF and rwflowpack with the following commands:

systemctl status yaf.service
systemctl status rwflowpack.service

Run a test query

Run the following query to get data for the current day:

/usr/local/bin/rwfilter --sensor=S0 --type=all --all=stdout \
| rwcut --tail-recs=10

You now have a standalone flow collection and analysis machine.

See SiLK's documentation page for more information on the SiLK tool suite. In particular,

• the SiLK Installation Handbook provides additional information on installing SiLK
• the Analysts' Handbook: Using SiLK for Network Traffic Analysis contains more examples of the use of SiLK for analysis
• the manual pages provide details for each individual SiLK tool