Why does the world need another network flow event generator? yaf was originally intended as an experimental implementation tracking developments in the IETF IPFIX working group, specifically bidirectional flow representation, archival storage formats, and structured data export with Deep Packet Inspection. It is designed to perform acceptably as a flow sensor on any network on which white-box flow collection with commodity hardware is appropriate. yaf can and should be used on specialty hardware when scalability and performance are of concern.
The YAF toolchain presently consists of two primary tools, yaf itself, and yafscii. The YAF applications require the libairframe and libyaf libraries, which are included and installed as part of the YAF distribution. libairframe installs two additional tools, filedaemon and airdaemon. libyaf implements YAF file and network I/O, and contains YAF packet decoder, fragment assembler, and flow table. In addition, two tools to assist in PCAP analysis are also installed with YAF.
Yet Another Flowmeter. Processes packet data from pcap(3) dumpfiles as generated by tcpdump(1) or via live captures from an interface into bidirectional flows, then exports those flows to IPFIX Collecting Processes or in an IPFIX-based file format.
YAF Flow printer. yafscii takes IPFIX flow data files generated by yaf and prints them in ASCII format loosely analogous to that produced by tcpdump(1), with one flow per line.
yaf PCAP metadata file parser and PCAP file creator. yafMeta2Pcap takes the metadata files produced by yaf and some additional flow information provided by getFlowKeyHash to create PCAP files for a particular flow.
yaf flow key calculator. getFlowKeyHash takes information about a flow and calculates the yaf flow key hash and prints the flow records along with the hash to the screen. In addition, it will convert the flow's start time to milliseconds since Epoch time. Together, the flow key hash and the start time can be used as a unique identifier for a flow.
Run a program as a daemon process, restarting the program if it dies.
filedaemon can invoke another program on files matching a glob pattern. It is often used to poll a directory and move files from one directory to another.
The following manuals provide general information about specific features of yaf. These features are not enabled by default, and require them to be enabled at compile time.
Information about the yaf configuration file which is an alternative to configuring yaf with command line options. The syntax of the configuration file is explained by examples.
Information about yaf application labeling, signature detection, and setting up the configuration file. Also provides a table of current application labels.
Provides information about the deep packet inspection capabilities in yaf, including the available protocols, setting up the configuration file, and export fields.
yaf DHCP fingerprinting capability information. Provides information on how to use it, the configuration file, and the fields exported.
yaf requires glib 2.6.4 or later. Build and install glib before building YAF. Note that glib is also included in many operating environments or ports collections. If installing via rpm, please note that you must install the glib2-devel package as well.
yaf requires libpcap. Note that libpcap is included with many operating environments or ports collections. If installing via rpm, please note that you must install the libpcap-devel package as well.
yaf can process compressed PCAP files if the zlib library is installed and yaf is run with the
yaf requires libfixbuf 1.7.0 or later.
yaf provides support for the Endace/Emulex, Napatech, and Netronome
capture cards. yaf can be configured to use the custom libpcap on
these cards by using the
--with-libpcap option or by setting
CFLAGS and LDFLAGS when configuring yaf. However, if yaf
is compiled with libdag, libntapi, or NFM and the appropriate name is given
to --live, yaf, by default, will record the
physical interface the packet was received on. To export these values, use the
--export-interface option when running yaf.
Interface values can be used to determine directionality of a flow in some cases.
To disable interface collection, configure yaf with
--enable-interface=no. To separate traffic received on separate ports
into different flows, use the
--enable-daginterface option when
Endace DAG live input support requires libdag. Use the
--with-dag option to
./configure to enable DAG support. Standard interface recording is
enabled by default when running yaf with
Napatech live input support requires libntapi and the 3rd generation Napatech drivers. Use the
--with-napatech option to
./configure to enable Napatech support.
Standard interface recording is enabled by default when running yaf
Netronome live input support requires the Netronome Flow Manager (NFM) which includes
the NFM PCAP library and NFM software. Use the
./configure to enable Netronome support. Standard interface recording
is enabled by default when running yaf with
Support for Bivio interface labeling requires yaf to be configured
Support for application labeling requires PCRE 7.3 or
later. Build and install PCRE before building YAF. (Many Linux systems already have
PCRE installed.) If installing via rpm, you must install the pcre-devel package.
Support for application labeling requires giving the
--enable-applabel option to
Support for p0f requires libp0f. Build and install libp0f before building YAF. You may need to set the PKG_CONFIG_PATH environment variable if libp0f is not installed in the default location.
Spread support requires Spread 4.1 or later. Build and install Spread before building YAF.
yaf contains support for PF_RING and PF_RING ZC (ZERO COPY) if yaf
is compiled with libpfring by giving the
./configure. PF_RING is available through ntop.
Download and install PF_RING (v.6.2.0 or higher) before installing yaf.
Install the PF_RING kernel modules, drivers, and library. PF_RING ZC requires a license
purchase through ntop.
To use PF_RING ZC, you are required to run yafzcbalance,
a tool installed with yaf, or a similar application
which will load balance the traffic on one
or more interfaces to one or more yaf applications.
yaf uses a reasonably standard autotools-based build system.
The customary build procedure
./configure && make && make install)
should work in most environments. Note that yaf finds
libfixbuf and libairframe using the
so you may have to set the
on the configure command line if these libraries are installed in a
nonstandard location, other than the prefix to which you are installing
Support for application labeling requires giving the
--enable-applabel option to
Support for p0f requires giving the
Deep Packet Inspection (DPI) requires plugin support. Use the
--enable-plugins option to
yaf can generate Multiprotocol Label Switching (MPLS)-Aware
flow data. yaf exports the first three MPLS labels
from the label stack along with the traditional flow data. When this
feature is enabled, yaf will also export non-IP
flow data. To enable MPLS-aware flow, use the
This presentation from FloCon 2015 describes how to use yaf and super_mediator to index large PCAP files. A possible method for identifying and classifying malware is also presented.
yaf presented at LISA'10 Proceedings.
yaf presention slides from LISA'10 Proceedings.
As of yaf 2.0.0, yaf uses a subTemplateMultiList to export certain flow elements. See yaf for more information. Older versions of yaf can read yaf 2.0 flow files, but will ignore anything contained in the subTemplateMultiList.
It is suggested to use
--silk when running yaf
with SiLK. If SiLK version 2.x is used,
--no-stats should be used due to how
yaf exports TCP flow information and
yaf process statistics.
Presently, the destinationTransportPort information element contains ICMP type and code information for ICMP or ICMP6 flows; this is nonstandard and may not be interoperable with other IPFIX implementations.
By default YAF generates flows based on the standard 5-tuple and VLAN tag, if
available. The 5-tuple consists of the source IP address,
destination IP address, source port, destination port, and protocol.
If YAF is configured with MPLS support
--enable-mpls, YAF will use the top
three MPLS labels from the MPLS label stack in addition to the 5-tuple and vlan
to determine the flow. In MPLS mode, it will also export the top three MPLS
labels in the IPFIX record. The exported fields will not include the
experimental bits and the bottom of stack bit. In addition, if YAF is
--enable-nonip, YAF will accept non-IP data and generate
flow data using just the MPLS labels. The 5-tuple and VLAN fields
will be set to 0, and the exported flow will contain start and end times,
packet counts, byte counts, and MPLS labels. Since the byte count is typically
taken from the length in the IP header,YAF will use the length provided
by libpcap. Non-IP data can only be exported if MPLS mode is enabled.
Please send bug reports, feature requests, and questions to . We welcome bug fixes and patches.