YAF
===

YAF is Yet Another Flowmeter. It processes packet data from pcap(3) dumpfiles
as generated by tcpdump(1) or via live capture from an interface using
pcap(3), an Endace DAG card, or a Napatech adapter into bidirectional flows,
then exports those flows to IPFIX Collecting Processes or in an IPFIX-based
file format. YAF's output can be used with the SiLK flow analysis tools and
the NetSA Aggregated Flow (NAF) toolchain.

YAF also supports partial payload capture - this feature is intended for use
in "banner grabbing" for protocol verification and service presence detection,
and is presently experimental.

Why does the world need another network flow event generator? YAF is intended
as an experimental implementation tracking developments in the IETF IPFIX
working group, specifically bidirectional flow representation, archival
storage formats, and structured data export with Deep Packet Inspection.  It
is designed to perform acceptably as a flow sensor on any network on which
white-box flow collection with commodity hardware is appropriate, but
tradeoffs between raw performance and clarity of design have generally been
made in favor of the latter.

The YAF toolchain presently consists of two primary tools, yaf itself, and
yafscii, which converts yaf output into ASCII format. getFlowKeyHash and
yafMeta2Pcap are also distributed with YAF and are helpful tools if using
yaf's pcap export options.

By default YAF generates flows based on the standard 5-tuple and VLAN tag, if
available.  The 5-tuple consists of the source IP address, destination IP
address, source port, destination port, and protocol.  If YAF is configured
with MPLS support, YAF will use the top three MPLS labels from the MPLS label
stack in addition to the 5-tuple and vlan to determine the flow.  In MPLS
mode, it will also export the top three MPLS labels in the IPFIX record.  The
exported fields will not include the experimental bits and the bottom of stack
bit.  In addition, if YAF is configured with --enable-nonip, YAF will accept
non-IP data and generate flow data using just the MPLS labels.  The 5-tuple
and VLAN fields will be set to 0, and the exported flow will contain start and
end times, packet counts, byte counts, and MPLS labels.  Since the byte count
is typically taken from the length in the IP header, YAF will use the length
provided by libpcap.  Non-IP data can only be exported if MPLS mode is
enabled.


Building
========

YAF requires GLib 2.34.0 or later; GLib is available at
https://www.gtk.org. Build and install GLib before building YAF. Note that
GLib is also included in many operating environments or ports collections.

YAF requires libfixbuf 2.3.0 or later releases with the same major
version number.  libfixbuf is available at
https://tools.netsa.cert.org/fixbuf. Build and install libfixbuf before
building YAF.

Spread support requires Spread 4.1 or later.  Build and install
Spread before building YAF.

YAF requires libpcap from https://www.tcpdump.org. Note that libpcap is
included with many operating environments or ports collections.

YAF requires libairframe which is built and installed with the YAF tools
distribution.

Endace DAG live input support requires libdag. Use the --with-dag option to
./configure to enable DAG support.

Napatech live input support requires libpcapexpress. libpcapexpress library is
available to Napatech customers for download at www.pcapexpress.com.  Use the
--with-napatech option to ./configure to enable Napatech support.  Before
starting YAF with --live napatech, reload Napatech drivers using pcapxctl -r
(Napatech Driver Load/Unload Script).

The YAF application labeling functionality requires the Perl regular
expression library, PCRE.  This library is available at https://www.pcre.org/.
If YAF is installed in a nonstandard location, it may be necessary to set the
LTDL_LIBRARY_PATH environment variable to the location where the application
labeling plugins are installed. By default, YAF installs the application
labeling plugins in /usr/local/lib/yaf.

PF_RING and PF_RING ZC (ZERO COPY) support requires libpfring (v. 6.2.0 or
higher), available at https://www.ntop.org/get-started/download/.  Download
and install the library before installing YAF. Install the PF_RING kernel
modules and drivers (if necessary).  To use PF_RING ZC, you are required to
run yafzcbalance which will load balance the traffic on one or more interfaces
to one or more yaf applications.

The YAF applications also require the included libyaf library.  libyaf
implements YAF file and network I/O, packet decoding, fragment assembly, and
flow generation. This library is built and installed with the YAF tools
distribution.

YAF uses a reasonably standard autotools-based build system. The customary
build procedure (./configure && make && make install) should work in most
environments. Note that YAF finds libfixbuf and libairframe using the
pkg-config(1) facility, so you may have to set the PKG_CONFIG_PATH variable
on the configure command line if these libraries are installed in a
nonstandard location, other than the prefix to which you are installing
YAF itself.

Common Issues when Installing or Running YAF
============================================

Configure Error: configure: error: Cannot find a suitable libfixbuf (>= 2.3.0)
(Try setting PKG_CONFIG_PATH): No package 'libfixbuf' found

Solution: export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig if libfixbuf was
installed in the default location (Otherwise /$prefix/lib/pkgconfig)

---

Runtime Error: yaf: error while loading libraries: libairframe-2.3.0.so.4:
cannot open share object file: No such file or directory

Solution: Most likely yaf libraries were installed in a nonstandard location.
Try running `ldconfig` or setting LD_LIBRARY_PATH to the location of
libairframe.

---

Runtime Error: "couldn't open library "dnsplugin": file not found"

Solution: Most likely yaf application labeling libraries were installed in a
nonstandard location (default: /usr/local/lib/yaf).  Set LTDL_LIBRARY_PATH to
the location of those libraries ($prefix/lib/yaf).  If you are starting yaf
via a startup script, it may be necessary to export this environment variable
from the startup script.

---

Error: yaf terminating on error: Failed to load certificate file:
error:0906D06C:PEM routines:PEM_read_bio:no start line

Solution: When running yaf exporting via TLS, the certificate files given to
--tls-ca and --tls-cert must be in PEM format.  DER format is not accepted.

---

Error: yaf terminating on error: Failed to load private key file:
error:0906A068:PEM routines:PEM_do_header:bad password read

Solution: Most likely the key file given to --tls-key requires a password and
the YAF_TLS_PASS environment variable was not set.  Set the YAF_TLS_PASS
environment variable to the correct password for the --tls-key or remove the
password from the key file (openssl rsa -in key.key -out key.key).

---

Error: yaf terminating on error: Failed to load private key file:
error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt

Solution: Most likely the key file given to --tls-key requires a password and
the YAF_TLS_PASS environment variable was set to the incorrect password.  Set
the YAF_TLS_PASS environment variable to the correct password or remove the
password from the key file (openssl rsa -in key.key -out key.key)


Known Issues
============

YAF BPF Filtering is ignored when using --live dag because libpcap is not
used.

If YAF is intended to export to SiLK, the --silk command line flag should be
used.

YAF will not rotate output files if it is not seeing any flow data.  However,
it will continue to write process statistics messages at the configured
interval time to the most recent output file.

Presently, the destinationTransportPort information element contains ICMP type
and code information for ICMP or ICMP6 flows; this is nonstandard and may not
be interoperable with other IPFIX implementations.

Please send bug reports, feature requests, and questions to
<netsa-help@cert.org>.

COPYRIGHT
=========
@DISTRIBUTION_STATEMENT_BEGIN@
YAF 2.16

Copyright 2024 Carnegie Mellon University.

NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING
INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON
UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED,
AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR
PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF
THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF
ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT
INFRINGEMENT.

Licensed under a GNU GPL 2.0-style license, please see LICENSE.txt or
contact permission@sei.cmu.edu for full terms.

[DISTRIBUTION STATEMENT A] This material has been approved for public
release and unlimited distribution.  Please see Copyright notice for
non-US Government use and distribution.

This Software includes and/or makes use of Third-Party Software each
subject to its own license.

DM24-1063
@DISTRIBUTION_STATEMENT_END@
