The Network Situational
Awareness (NetSA) group at CERT has developed and maintains a
suite of open source tools for monitoring large-scale networks
using flow data. These tools have grown out of the work of the
AirCERT project, the SiLK project and the effort to integrate this
work into a unified, standards-compliant flow collection and
CERT is a part of the Software Engineering Institute
(SEI), a federally funded research and development center
(FFRDC) operated by Carnegie Mellon
The System for Internet Level Knowledge (SiLK) is an efficient
network flow collection and storage infrastructure that will
accept flow data from a variety of sensors. SiLK also provides a
suite of efficient command-line tools for analysis.
Yet Another Flow Sensor (YAF) processes packet data into
bidirectional flow records that can be used as input to an IPFIX
Collecting Process. YAF's output can be used with super_mediator,
Pipeline 5, and the SiLK tools.
The Analysis Pipeline 5.3 is a streaming analysis tool than can process
more than just SiLK
flows as done in version
4.x. It can now process YAF records and raw IPFIX records. It can do all
of the analyses available in version 4.x. A notable enhancement is
expansive DNS record processing. This includes fast flux detection and
domain name watchlisting.
super_mediator is an IPFIX mediator for use with the YAF and SiLK
tools. It collects and filters YAF output data to various IPFIX
collecting processes and/or csv files. super_mediator can be
configured to perform de-duplication of DNS resource records as
exported by YAF.