This page describes how to install and configure SiLK and YAF (and their prerequisite libfixbuf) on a single CentOS machine (RedHat and Fedora are similar) for standalone flow collection and analysis. (See the instructions for an Ubuntu or Debian machine if that applies to you).
When installing the NetSA applications on an YUM/RPM-based machine, it is highly recommended you use RPMs to install the tools. This makes it easier to upgrade or uninstall the tools. Follow the instructions under Creating RPMS of the tools to create your own RPMs and then install them.
You are not required to install the NetSA tools as RPMs. Use the instructions in Installing the tools outside of YUM if you prefer.
After installing the tools, use the instructions in Configuring the tools to start the tools and collect data.
This section describes how to build your own RPMs of the NetSA tools.
You should have a fully functional YUM/RPM-based system; these instructions use CentOS 7. Run the following command to install the basic development tools (e.g., a C compiler) required to build SiLK and YAF.
sudo yum -y install gcc gcc-c++ make pkgconfig
Alternatively, you may install the tools for a complete development environment:
sudo yum -y group install "Development Tools"
Install the rpm-build package, and create the RPM
build hierarchy. (These instructions use ~/rpm
as the
root of the build hierarchy, but it may be located anywhere.)
sudo yum -y install rpm-build cd ~ mkdir rpm rpm/BUILD rpm/RPMS rpm/SOURCES rpm/SPECS rpm/SRPMS
Create the ~/.rpmmacros
file with your favorite editor,
using this as a template:
%packager your name <you@example.com> %_topdir /home/you/rpms
Run the following commands to install the GLib-2, PCAP, Python, LZO, zlib, and GnuTLS development libraries:
sudo yum -y install glib2-devel libpcap-devel python-devel \ lzo-devel zlib-devel gnutls-devel
If you want to use the MaxMind country-code mapping capability of SiLK, you must install the libmaxminddb-devel package which is available from the Extra Packages for Enterprise Linux (EPEL) repository:
sudo yum -y install epel-release sudo yum -y update epel-release sudo yum -y install libmaxminddb-devel
Download the libfixbuf, SiLK, and YAF source code from the tools.netsa.cert.org web site.
cd ~/rpm/SOURCES wget https://tools.netsa.cert.org/releases/silk-3.23.1.tar.gz wget https://tools.netsa.cert.org/releases/libfixbuf-2.5.0.tar.gz wget https://tools.netsa.cert.org/releases/yaf-2.16.1.tar.gz
In the August 2024 releases of the tools, the process to build an RPM
changed. In these releases, the rpmbuild
specification file (.spec
file) included in the
downloaded file is usable as-is, and custom configuration of the
binaries is done by using feature flags on the
rpmbuild command line, for example
rpmbuild -ba silk.spec --with pysilk --without ipv6
For prior releases, each NetSA package has an RPM .spec
file template. When you run the package's configure
script, the template is customized using the features (command line
options) you specified to configure.
For libfixbuf-2.5.0 and later, extract the libfixbuf.spec
file from the downloaded file by specifying that file as an argument
to the tar command, then move the file into place.
cd /tmp tar -zxf ~/rpm/SOURCES/libfixbuf-2.5.0.tar.gz libfixbuf-2.5.0/libfixbuf.spec mv libfixbuf-2.5.0/libfixbuf.spec ~/rpm/SPECS/.
For earlier releases, expand the libfixbuf source code, run
configure, and copy the generated
libfixbuf.spec
file into the RPM build hierarchy:
cd /tmp tar -zxf ~/rpm/SOURCES/libfixbuf-2.4.1.tar.gz cd libfixbuf-2.4.1 ./configure cp libfixbuf.spec ~/rpm/SPECS/.
Once the libfixbuf.spec
file is in place, build the
libfixbuf RPMs:
cd ~/rpm rpmbuild -ba SPECS/libfixbuf.spec ls RPMS/x86_64
The output from the last command should resemble:
x86_64/libfixbuf-2.5.0-1.el9.x86_64.rpm x86_64/libfixbuf-debuginfo-2.5.0-1.el9.x86_64.rpm x86_64/libfixbuf-devel-2.5.0-1.el9.x86_64.rpm x86_64/libfixbuf-ipfixDump-2.5.0-1.el9.x86_64.rpm
Install the libfixbuf and libfixbuf-devel packages which are prerequisites for building SiLK and YAF.
sudo yum -y install x86_64/libfixbuf-2*.rpm \ x86_64/libfixbuf-devel-2*.rpm
For SiLK 3.23.0 and later, follow these instructions to create the SiLK RPMs. Install at least the silk-common, silk-analysis and silk-rwflowpack packages as described at the end of this section.
For prior releases, the following commands unpack the SiLK sources and
create the silk.spec
file. The switches to the
configure command do the following:
/data
.)
cd /tmp tar -zxf ~/rpm/SOURCES/silk-3.19.1.tar.gz cd silk-3.19.1 ./configure \ --enable-data-rootdir=/var/silk/data \ --enable-ipv6 \ --enable-ipset-compatibility=3.14.0 \ --enable-output-compression \ --with-python cp silk-3.19.1.spec ~/rpm/SPECS/.
Build the SiLK RPMs:
cd ~/rpm rpmbuild -ba SPECS/silk-3.19.1.spec ls RPMS/x86_64/silk*
The output from the final command should look similar to:
RPMS/x86_64/silk-analysis-3.19.1-1.el9.x86_64.rpm RPMS/x86_64/silk-common-3.19.1-1.el9.x86_64.rpm RPMS/x86_64/silk-debuginfo-3.19.1-1.el9.x86_64.rpm RPMS/x86_64/silk-devel-3.19.1-1.el9.x86_64.rpm RPMS/x86_64/silk-flowcap-3.19.1-1.el9.x86_64.rpm RPMS/x86_64/silk-rwflowappend-3.19.1-1.el9.x86_64.rpm RPMS/x86_64/silk-rwflowpack-3.19.1-1.el9.x86_64.rpm RPMS/x86_64/silk-rwpollexec-3.19.1-1.el9.x86_64.rpm RPMS/x86_64/silk-rwreceiver-3.19.1-1.el9.x86_64.rpm RPMS/x86_64/silk-rwsender-3.19.1-1.el9.x86_64.rpm
Install at least the silk-analysis and silk-rwflowpack packages (silk-common is a prerequisite for both):
sudo yum -y install x86_64/silk-common*.rpm \ x86_64/silk-analysis*.rpm x86_64/silk-rwflowpack*.rpm
For YAF 2.16.0 and later, extract the yaf.spec
file from
the downloaded file by specifying that file as an argument to the
tar command, then move the file into place.
cd /tmp tar -zxf ~/rpm/SOURCES/yaf-2.16.1.tar.gz yaf-2.16.1/yaf.spec mv yaf-2.16.1/yaf.spec ~/rpm/SPECS/.
For earlier releases, use the following commands create the
yaf.spec
file, enabling all of YAF's application labeling and deep packet inspection (DPI)
capabilities. If you are building yaf-3.0.0 or later, add
--enable-dpi
to the configure options.
cd /tmp tar -zxf ~/rpm/SOURCES/yaf-2.12.2.tar.gz cd yaf-2.12.2 ./configure \ --enable-applabel \ --enable-metadata \ --enable-plugins cp yaf.spec ~/rpm/SPECS/.
Once the yaf.spec
file is in place, build and install the
yaf RPMs:
cd ~/rpm rpmbuild -ba SPECS/yaf.spec sudo yum -y install x86_64/yaf-2.*.rpm
This section describes how to build the NetSA tools using the
traditional
configure && make && make install
method if you cannot or do not want to install the tools using RPMs.
Follow the instructions in the previous
section through the invocations of wget to download
the NetSA tools' source code. (Those instructions install the packages
required to build the NetSA tools.) The rpm-build and
~/.rpmmacros
steps may be omitted.
Unpack, configure, and install libfixbuf into the
/usr/local
directory. (This section assumes you
downloaded the source code to /tmp
.)
cd /tmp tar -zxf /tmp/libfixbuf-2.5.0.tar.gz cd libfixbuf-2.5.0 ./configure \ --prefix=/usr/local \ --enable-silent-rules make sudo make install
Unpack, configure, and install SiLK into the /usr/local
directory. Most of the options to configure are
described in the previous section; the additional switch
--with-python-prefix
causes the PySiLK files to be
installed under the SiLK installation tree (/usr/local
)
instead of with the other Python packages.
cd /tmp tar -zxf /tmp/silk-3.23.1.tar.gz cd silk-3.23.1 ./configure \ --prefix=/usr/local \ --enable-silent-rules \ --enable-data-rootdir=/var/silk/data \ --enable-ipv6 \ --enable-ipset-compatibility=3.14.0 \ --enable-output-compression \ --with-python \ --with-python-prefix make sudo make install
Unpack, configure, and install YAF into the /usr/local
directory, enabling YAF's application
labeling and deep packet inspection
(DPI) capabilities. If you are building yaf-3.0.0 or later, add
--enable-dpi
to the configure options.
cd /tmp tar -zxf /tmp/yaf-2.16.1.tar.gz cd yaf-2.16.1 ./configure \ --prefix=/usr/local \ --enable-silent-rules \ --enable-applabel \ --enable-metadata \ --enable-plugins make sudo make install
Instead of exporting LD_LIBRARY_PATH each time you use SiLK, it is
easier to add the /usr/local/lib
paths to
ld.so.conf
. Create a file named silk.conf
containing the following lines that specify the library directories
for SiLK:
/usr/local/lib /usr/local/lib/silk
Now copy the file into the /etc/ld.so.conf.d
directory
and run ldconfig.
sudo mv silk.conf /etc/ld.so.conf.d/. sudo ldconfig
The instructions in this section assume you have installed the NetSA tools using either of the RPM instructions above. The paths will be different if you installed the tools outside of YUM.
The first step to configuring SiLK is to create the data repository
directory (/var/silk/data
) and add the
silk.conf
file, which defines how your data is stored
(see the manual page for details). Use
the default silk.conf
file for the
twoway site, which is installed at
/usr/share/silk/twoway-silk.conf
. You may edit the
sensor descriptions if desired. The default settings cause the SiLK
analysis program rwfilter
to consider only incoming data unless the user provides the
--type or --flowtypes switch. If
you want rwfilter to look at both incoming and
outgoing data by default, modify the default-types
line
to include in inweb out outweb. If desired,
also add int2int ext2ext. See the twoway packing logic manual page for
the meanings of these types.
sudo mkdir -p /var/silk/data sudo chmod go+rx /var/silk /var/silk/data sudo cp /usr/share/silk/twoway-silk.conf /var/silk/data/silk.conf
Next create the sensors.conf
file that is used by rwflowpack for collecting
data from yaf. Details on the file's syntax are in
its manual page. Use your text editor
to create a file that contains the following lines.
IMPORTANT: Make sure the ipblocks in the my-network
section match your "internal" network blocks according to the ethernet
interface (e.g., enp0s3
, eth0
) where you
will configure yaf to listen below.
probe S0 ipfix listen-on-port 18001 protocol tcp listen-as-host 127.0.0.1 end probe group my-network ipblocks 192.168.1.0/24 # address of ethernet interface. CHANGE THIS. ipblocks 10.0.0.0/8 # other blocks considered internal. OPTIONAL. end group sensor S0 ipfix-probes S0 internal-ipblocks @my-network external-ipblocks remainder end sensor
(Once the entire system is running, if you discover that all your records show up as type ext2ext, it means you did not configure your internal netblocks correctly.)
Move this file into place.
sudo mv sensors.conf /var/silk/sensors.conf
SiLK comes with traditional "init script" start-up files, with two
files per daemon: One file is invoked by the system to start the
daemon, and the other contains configuration settings used by the
first. Edit the start-up configuration script for
rwflowpack,
/etc/sysconfig/rwflowpack.conf
to change these variables
to the values shown here:
ENABLED=1 statedirectory=/var/silk SENSOR_CONFIG=/var/silk/sensors.conf ARCHIVE_DIR= # empty LOG_TYPE=legacy LOG_DIR=/var/log
If you used the instructions to install the tools outside of YUM, then:
/usr/local/share/silk/etc/rwflowpack.conf
to
/usr/local/etc/rwflowpack.conf
and edit it as shown
above.
/usr/local/share/silk/etc/init.d/rwflowpack
to
/etc/init.d/rwflowpack
before the next step.
SiLK 3.23.0 and later support systemd and should be started using these instructions.
For prior releases, start rwflowpack:
sudo systemctl start rwflowpack.service
Look at the rwflowpack log file,
/var/log/rwflowpack-YYYYYMMDD.log
to ensure that
it is running.
Like SiLK, YAF has two "init script" start-up files. If you used the
instructions to install the tools outside of
YUM, you must copy the yaf start-up script from
the source directory to /etc/init.d/yaf
and enable its
execute permissions:
sudo cp /tmp/yaf-2.16.1/etc/init.d/yaf /etc/init.d/yaf chmod a+w /etc/init.d/yaf
Edit the YAF start-up configuration file, /etc/yaf.conf
,
setting these values:
IMPORTANT: Make sure the interface (enp0s3
) matches
the interface on which you want to capture.
ENABLED=1 YAF_CAP_IF=enp0s3 # Ensure this is correct for your machine YAF_IPFIX_PORT=18001 # Must match value in sensors.conf YAF_EXTRAFLAGS="--silk --applabel --max-payload=512"
YAF 2.16.0 and later support systemd and should be started using these instructions.
For earlier releases, start yaf:
sudo systemctl start yaf.service
Look at the yaf log file,
/var/log/yaf.log
to ensure that it is running.
Check the rwflowpack log file again to ensure that it received a connection from yaf. There should be a message similar to the following (the port number will likely differ):
'S0': accepted connection from 127.0.0.1:36734
If nothing is happening, it could be that yaf is waiting for network traffic. Follow the steps in the next section to generate some.
Generate some traffic and wait records to be flushed. Run the following ping command, or use your web browser or the wget command to access web sites.
ping -c 4 8.8.8.8
You may have to wait 5 or 10 minutes for the first records to be flushed, but you should see messages in the rwflowpack log file that it is creating files. You may check the status of YAF and rwflowpack with the following commands:
systemctl status yaf.service systemctl status rwflowpack.service
Run the following query to get data for the current day:
/usr/local/bin/rwfilter --sensor=S0 --type=all --all=stdout \ | rwcut --tail-recs=10
You now have a standalone flow collection and analysis machine.
See SiLK's documentation page for more information on the SiLK tool suite. In particular,