The Analysis Pipeline is designed to run as a daemon as part
of the SiLK
collection and packing
process, where it inspects every SiLK Flow record as the records
are created. The Analysis Pipeline supports several analyses,
including watch list alerting, beacon detection, passive FTP
detection, and IPv6 tunnel detection. The textual output from the
Analysis Pipeline can be fed to a security information and event
The Analysis Pipeline 5.5 is a streaming analysis tool than can process
more than just SiLK
flows as done in version
4.x. It can now process YAF records and raw IPFIX records. It can do all
of the analyses available in version 4.x. A notable enhancement is
expansive DNS record processing. This includes fast flux detection and
domain name watchlisting.
The fixbuf library provides a set of functions for processing the
IPFIX protocol message format. Using fixbuf, developers can build
IPFIX Collecting and Exporting Processes. pyfixbuf provides a
Python API to the fixbuf library.
IP Association (IPA) is a suite of tools and libraries which aims
to provide a flexible repository of IP address data and metadata.
iSiLK is a graphical front-end for the SiLK tools, designed to
work with an existing installation of the SiLK analysis suite. The
application uses the SSH protocol to connect to an analysis
server, run SiLK command-line tools and copy data files. It
provides an easy-to-use alternative interface to the core
functionality of the SiLK tool suite.
The netsa-python library is a grab-bag of Python routines and
frameworks that we have found helpful when developing analyses
using the SiLK toolkit.
Rayon is a Python library and set of tools for generating basic
two-dimensional statistical visualization. Rayon can be used to
automate reporting; provide command-line, GUI or web applications;
or do ad-hoc exploratory data analysis.
schemaTools is a library of middleware for the Analysis Pipeline that
provides a standard way of describing data upon arrival.
is a distributed alert reporting system. Applications can
`s libraries to send network alert messages, which can
then be routed to multiple destinations in a configurable
is designed to allow application and script
developers to emit network alert messages without being concerned with
the details of how the messages will be formatted downstream, or what
destinations they will be routed to.
The System for Internet Level Knowledge (SiLK) is an efficient
network flow collection and storage infrastructure that will
accept flow data from a variety of sensors. SiLK also provides a
suite of efficient command-line tools for analysis.
Yet Another Flow Sensor (YAF) processes packet data into
bidirectional flow records that can be used as input to an IPFIX
Collecting Process. YAF's output can be used with super_mediator,
Pipeline 5, and the SiLK tools.