Install Prerequisites

You should have a fully functional apt/deb-based system; these instructions use Debian 13 Trixie. Run the following command to install the basic development tools (e.g., a C compiler) required to build SiLK and YAF.

$ sudo apt install build-essential

(All commands in this section are run as the root user.)

Run the following command to ensure all of the required software development tools and libraries are available:

$ sudo apt install \
    gcc \
    libglib2.0-dev \
    libgnutls28-dev \
    liblzo2-dev \
    libpcap-dev \
    libssl-dev \
    make \
    perl \
    pkg-config \
    python3-dev \
    zlib1g-dev

To use the MaxMind country-code mapping capability of SiLK, you must install the libmaxminddb-dev package.

$ sudo apt install libmaxminddb-dev

Download Software

Download the libfixbuf, SiLK, and YAF source code from the tools.netsa.cert.org web site using wget or another tool. In the example, we download it to /tmp, but it could be downloaded to and compiled in any directory.

If there are new versions of these software packages, we recommend building the latest versions even if this document has not yet been updated.

Note: On some SELinux systems, files cannot be executed in /tmp, so this software must be built in a different location.

$ cd /tmp
$ wget https://tools.netsa.cert.org/releases/silk-3.24.2.tar.gz
$ wget https://tools.netsa.cert.org/releases/libfixbuf-2.5.4.tar.gz
$ wget https://tools.netsa.cert.org/releases/yaf-2.20.3.tar.gz

Build and install libfixbuf

Unpack, configure, and build libfixbuf, then install it into the /usr/local directory.

$ tar -xvzf libfixbuf-2.5.4.tar.gz
$ cd libfixbuf-2.5.4
$ ./configure --prefix=/usr/local --enable-silent-rules
$ make
$ sudo make install

Build and install SiLK

Unpack, configure, build, and install SiLK. The switches to the configure command do the following:

--enable-data-rootdir

sets the default location for SiLK's hourly repository of flow files to /var/silk/data. (The default location is /data.)

--enable-ipv6

enables support for recording IPv6 addresses and flow records.

--enable-ipset-compatibility

enables creating IPset files in the most compact format.

--enable-output-compression

enables automatic compression of SiLK data.

--with-python

enables the SiLK Python extension.

--with-python-prefix

enables installation of the PySiLK files under the SiLK installation tree (/usr/local) instead of with the other Python packages.

$ tar -xvzf silk-3.24.2.tar.gz
$ cd silk-3.24.2
$ ./configure --prefix=/usr/local --enable-silent-rules \
    --enable-data-rootdir=/var/silk/data \
    --enable-ipv6 \
    --enable-ipset-compatibility=3.14.0 \
    --enable-output-compression \
    --with-python \
    --with-python-prefix
$ make
$ sudo make install

Build and install YAF

Unpack, configure, build, and install YAF, enabling YAF's application labeling and deep packet inspection (DPI) capabilities. The switches to the configure command do the following:

--enable-applabel

enables application labeling

--enable-plugins

enables plugins, including DPI

$ tar -xvzf yaf-2.20.3.tar.gz
$ cd yaf-2.20.3
$ ./configure --prefix=/usr/local --enable-silent-rules \
    --enable-applabel \
    --enable-plugins
$ make
$ sudo make install

Update Dynamic Linker Path

Finally, you should update the cache of the dynamic linker. If you do not, it may be necessary to set the LD_LIBRARY_PATH environment variable to /usr/local/lib when you use SiLK or YAF.

Typically an entry for /usr/local/lib will already exist in the /etc/ld.so.conf.d/ directory. To confirm:

$ grep local /etc/ld.so.conf /etc/ld.so.conf.d/*
/etc/ld.so.conf.d/libc.conf:/usr/local/lib

To update the cache in this case, run the ldconfig program.

However, if your machine does not have such an entry (or you installed to a location other than /usr/local), you should create. Create a file named silk-so.conf containing the following line that specifies the library directory for SiLK and YAF:

/usr/local/lib

Now copy the file into the /etc/ld.so.conf.d directory and run ldconfig.

$ sudo install -m 644 silk-so.conf /etc/ld.so.conf.d/silk-so.conf
$ sudo ldconfig

Configure SiLK

The first step to configuring SiLK is to create the data repository directory (/var/silk/data) and add the silk.conf file, which defines how your data is stored (see the manual page for details). Use the default silk.conf file for the twoway site, which is installed at /usr/local/share/silk/twoway-silk.conf. You may edit the sensor descriptions if desired. The default settings cause the SiLK analysis program rwfilter to consider only incoming data unless the user provides the --type or --flowtypes switch. If you want rwfilter to look at both incoming and outgoing data by default, modify the default-types line to include in inweb out outweb. If desired, also add int2int ext2ext. See the twoway packing logic manual page for the meanings of these types. Run these commands as root:

$ sudo install -d /var/silk/data
$ sudo install -m 644 /usr/local/share/silk/twoway-silk.conf \
    /var/silk/data/silk.conf

SiLK comes with systemd unit files and also with traditional "init script" start-up files. In addition to either of these startup methods, configuration files are required for each service. Here we discuss the systemd configuration, which is the recommended method for modern Linux systems.

Copy SiLK's systemd unit files to /usr/local/lib/systemd/system, and startup scripts to /usr/local/libexec/silk:

$ sudo install -d /usr/local/lib/systemd/system
$ sudo install -m 644 /usr/local/share/silk/lib/systemd/system/* \
    /usr/local/lib/systemd/system
$ sudo install -d /usr/local/libexec/silk
$ sudo install -m 755 /usr/local/share/silk/libexec/silk/* \
    /usr/local/libexec/silk

Installing these files makes all of SiLK's services available to run but does not enable any of them to actually run or automatically start. The further steps to accomplish that are detailed below for each service.

Configure rwflowpack

Before the SiLK packing system can operate, you must create the sensor.conf file that is used by rwflowpack for collecting data from yaf. Details on the file's syntax are in its manual page. Use your text editor to create a file that contains the following lines.

IMPORTANT: Make sure the ipblocks in the my-network section match your "internal" network blocks according to the ethernet interface (e.g., enp0s3, eth0) where you will configure yaf to listen below.

probe S0 ipfix
 listen-on-port 18001
 protocol tcp
 listen-as-host 127.0.0.1
end probe

group my-network
 ipblocks 192.168.1.0/24  # address of ethernet interface. CHANGE THIS.
 ipblocks 10.0.0.0/8      # other blocks considered internal. OPTIONAL.
end group

sensor S0
 ipfix-probes S0
 internal-ipblocks @my-network
 external-ipblocks remainder
end sensor

(Once the entire system is running, if you discover that all your records show up as type ext2ext, it means you did not configure your internal netblocks correctly.)

(All commands in this section are run as the root user.)

Move this file into place.

$ sudo install -m 644 sensor.conf /var/silk/data/sensor.conf

Copy rwflowpack's configuration settings script to /usr/local/etc/rwflowpack.conf:

$ sudo install -m 644 /usr/local/share/silk/etc/rwflowpack.conf \
    /usr/local/etc/rwflowpack.conf

Open the /usr/local/etc/rwflowpack.conf file in an editor and change these variables to the values shown here:

ENABLED=1
statedirectory=/var/silk
SENSOR_CONFIG=/var/silk/data/sensor.conf
ARCHIVE_DIR=  # empty

Now enable rwflowpack to run automatically at system boot, and start rwflowpack running now:

$ sudo systemctl enable rwflowpack.service
$ sudo systemctl start rwflowpack.service

Look at the rwflowpack status using systemctl status rwflowpack.service to ensure that it is running. You can view its logs with journalctl -u rwflowpack.service.

Start YAF

Like SiLK, YAF has systemd unit files and configuration. Install the YAF systemd unit files and startup scripts:

$ sudo install -m 644 /usr/local/share/yaf/lib/systemd/system/* \
    /usr/local/lib/systemd/system
$ sudo install -d /usr/local/libexec/yaf
$ sudo install -m 755 /usr/local/share/yaf/libexec/yaf/* \
    /usr/local/libexec/yaf

Copy the YAF start-up configuration file from /usr/local/share/yaf/etc/yaf.conf to /usr/local/etc/yaf.conf using:

$ sudo install -m 644 /usr/local/share/yaf/etc/yaf.conf \
    /usr/local/etc/yaf.conf

Edit /usr/local/etc/yaf.conf, setting these values:

IMPORTANT: Make sure the interface (enp0s3) matches the interface on which you want to capture.

ENABLED=1
YAF_CAP_IF=enp0s3      # Ensure this is correct for your machine
YAF_IPFIX_HOST=127.0.0.1
YAF_LOG=stderr
YAF_IPFIX_PORT=18001   # Must match value in sensors.conf
YAF_EXTRAFLAGS="--silk --applabel --max-payload=512"

As the root user, start yaf:

$ sudo systemctl enable yaf
$ sudo systemctl start yaf.service

Use systemctl status yaf.service to check that YAF is running.

Check the rwflowpack log file again to ensure that it received a connection from yaf. It may take a little while, but eventually there should be a message similar to the following (the port number will likely differ):

'S0': accepted connection from 127.0.0.1:36734

If nothing is happening, it could be that yaf is waiting for network traffic. Follow the steps in the next section to generate some.

Generate traffic

Generate some traffic and wait records to be flushed. Run the following ping command, or use your web browser or the wget command to access web sites.

$ ping -c 4 8.8.8.8

You may have to wait 5 or 10 minutes for the first records to be flushed, but you should see messages in the rwflowpack log file that it is creating files. You may check the status of YAF and rwflowpack with the following commands:

$ sudo systemctl status yaf.service
$ sudo systemctl status rwflowpack.service

(You may need to include the --no-pager and -l switches to get useful information from systemctl.)

Run a test query

Run the following query to get data for the current day:

$ /usr/local/bin/rwfilter --sensor=S0 --type=all --all=stdout \
    | rwcut --tail-recs=10

You now have a standalone flow collection and analysis machine.

See SiLK's documentation page for more information on the SiLK tool suite. In particular,

• the SiLK Installation Handbook provides additional information on installing SiLK
• the Analysts' Handbook: Using SiLK for Network Traffic Analysis contains more examples of the use of SiLK for analysis
• the manual pages provide details for each individual SiLK tool