NAME

applabel - yaf application labeling

SYNOPSIS

yaf --applabel --max-payload=MAX_PAYLOAD [--applabel-rules=FILE] ...

DESCRIPTION

yaf(1) can examine a flow record's packet payloads, determine the application protocol in use within the flow, and export a 16-bit application label (often called an applabel) with each flow. (This feature must be enabled when yaf is built by supplying the --enable-applabel option to the configure script.)

The exported application label typically uses the assigned or common port number for the protocol. For example, if yaf determines a flow contains HTTP traffic, yaf assigns an applabel of 80 (the default HTTP port) for the flow independent of what ports the flow record contains.

Labels and rules are taken from a configuration file read by yaf at startup time. The location of the rule file may be given on the command line using the --applabel-rules option. If the location is not specified, yaf attempts to read it from the default location of /usr/local/etc/yafApplabelRules.conf. The application labeling system also uses compiled plug-ins (dynamically loaded libraries) that yaf must load. If yaf is unable to find these plug-ins, it may be necessary to set the LTDL_LIBRARY_PATH environment variable to the location of the application label plugins. yaf installs the application labeling plugins in /usr/local/lib/yaf/yaf.

Application labeling examines the payload of the flow records, and it requires payload capture to be enabled with the --max-payload option. A minimum payload capture length of 384 bytes is recommended for best results.

Application labeling is not guaranteed to be 100% accurate. Application labels are supported in other tools in the NetSA suite, including yafscii(1), super_mediator(1), and SiLK via rwflowpack(8), flowcap(8), and rwipfix2silk(1).

CONFIGURATION FILE FORMAT

The yafApplabelRules.conf file is the source of information by which yaf determines application labels, and the file is required for application labeling support. The file's default location is set during installation; for this installation, the file is /usr/local/etc/yafApplabelRules.conf.

The file may contain comments, blank lines, label statements, and port statements.

Comments and Blank Lines

A '#' symbol starts a comment in the rules file, and the rest of the line is a comment. The comment and any whitespace that precedes it is ignored. Comments must appear on their own line; they are not recognized if they appear after another keyword. Example:

# this is a comment

A completely empty line or a line containing only whitespace is considered a blank line and is ignored.

Label Statements

A label statement begins with the keyword 'label', and has the following format:

label APP { regex | plugin | signature } LABEL-RULE-ARGUMENTS

where

• APP is a unsigned 16-bit decimal integer (in the range 1 to 65535) and usually represents the application label to set in the flow record.

• regex, plugin, or signature is the rule type and specifies how to recognize the given application protocol.

• LABEL-RULE-ARGUMENTS are additional arguments that depend on the rule type.

The APP value must be unique across all regex and plugin rules, and it is recommended, but not required, that it also be unique across signature rules.

Rule Execution

Once a label is assigned to a flow record, the application labeling process stops; there is no attempt to find a "better" match.

The signature rule tests are performed first in the order they appear in the configuration file (regardless of where they appear in the file), and these rules are tested against both the forward and reverse payload. If a rule matches, the applabel is assigned and no other rules are tested.

If none of the signature rules match, the regex rule and plugin rule tests are checked. These are port-based tests: If the source or destination port of a flow record matches a rule's APP value (see also "Port Statements" below), that rule's test is performed. If the test passes, yaf assigns an applabel to the flow record and no other rules are tested.

For application protocols that use common ports (such as HTTP on port 80), the APP used in the rule should the common port. This helps yaf run the most-like-to-succeed test first and improve performance.

If the port-specific test fails or if the flow's ports do not match a specific rule, yaf begins testing the record against the regex and plugins rules in the order they appear in the configuration file regardless of type. If a rule succeeds, an applabel is assigned to the record and processing stops.

The order of the regex and plugin rules makes a difference. More common protocols (especially if known to run on arbitrary ports) should be specified earlier in the file to increase efficiency. Rules that have few false positives and use simple regular expressions or heuristics should also appear closer to the beginning of the file.

For regex and plugin rules, the rules are executed first using the forward payload. If no applabel is assigned, the rules are then executed using the reverse payload.

As of YAF-2.18, if --applabel-check-early=OCTETS is used, yaf tests the payload for a given direction once the captured payload in that direction reaches OCTETS octets.

Regex Rules

A label statement that uses a regex (regular expression) rule has the the following form:

label APP regex EXPRESSION

where

• APP is the label to assign to the record, and records that use APP as their source or destination port check this rule first.

• EXPRESSION is a PCRE2 regular expression (see pcre2pattern(1). Note that EXPRESSION is not delimited: it is separated from the regex keyword by whitespace and continues to the end of the line.

The APP value must be unique across all regex and plugin rules. The rule is ignored if it uses a previously defined label.

The EXPRESSION is compared against the available application payload for one direction of the flow. If the expression matches, the label APP is applied to the flow and the labeling process stops.

Regular expressions specifically crafted for reverse payloads are not recommended unless there is no chance they will match another protocol. Be aware that poorly crafted regular expressions can be detrimental to the efficiency of the software.

For example, the regex rule to match HTTP traffic is

label 80 regex HTTP/\d

where the final '\d' matches any decimal digit (0-9).

Plugin Rules

A plugin rule label statement examines a flow record's application payload using a dynamically loaded library, written in C. Its syntax is

label APP plugin LIBRARY FUNCTION-NAME [ ARG [ ARG... ] ]

where

• APP tells yaf that flow records that use that port should rule this test first

• LIBRARY is the name of the dynamically loadable library that exists somewhere within the LD_LIBRARY_PATH, the LTDL_LIBRARY_PATH, or a system library path, without the library extension name (typically .so)

• FUNCTION-NAME is the name of the function to call within the library

• [ARG [ARG...]] represents an optional, space-separated list of arguments that are passed to the function in the style of the argc and argv parameters to a C program's main() function

The APP value must be unique across all plugin and regex rules. The rule is ignored if it uses a previously defined label.

During the application labeling process, yaf calls FUNCTION-NAME with the flow record and either its forward or reverse payload as arguments (plus any ARG values specified in the yafApplabelRules.conf file).

If FUNCTION-NAME returns 0, no applabel is assigned. Any other return value (except 1) becomes the applabel assigned to the flow, and the application labeling process stops. This allows a single plugin to identify multiple protocols. A return value of 1 tells yaf to use the APP value specific in the yafApplabelRules.conf file, but this feature is deprecated as it does not interoperate well with early payload detection (--applabel-check-early).

See the source code to the plugins included with yaf for details on the specific protocol implementations.

The plugin rule to match TLS traffic is

label 443 plugin tlsplugin tlsplugin_LTX_ycTlsScanScan

where tlsplugin refers to the dynamic library (likely named tlsplugin.so and compiled from the C source file tlsplugin.c), and tlsplugin_LTX_ycTlsScanScan is a function defined in that file.

Signature Rules

The syntax of a signature rule label statement is similar to a regex rule:

label APP signature EXPRESSION

where EXPRESSION is a PCRE2 regular expression and it is not delimited: it is separated from the signature keyword by whitespace and continues to the end of the line. yaf checks whether EXPRESSION matches either the forward or reverse application payload of a flow record and assigns the label APP to the flow if a match is found.

Signature rules differ from Regex Rules in that they are tested before the other rules and the flow's ports are not considered: yaf tests a flow's payload with all the signature rules' regular expressions in the order the rules appear in the configuration file, stopping if a match is found.

Multiple signature rules may use the same APP value, and a signature rule's APP value may also be used by a regex or plugin rule.

For example, to apply the applabel 9876 to flow records that have the phrase "foo bar" anywhere within the application payload, add the following to the yafApplabelRules.conf file:

label 9876 signature foo bar

Port Statements

As of YAF-2.19, the port statement allows one to provide additional hints to yaf that traffic on a particular port is likely to match a particular applabel.

The syntax of the port statement is

port PORT label APP

where PORT is a port number present on the flow record and APP is a previously defined label-regex rule or label-plugin rule. The PORT value must be unique among all port statements, and it may not duplicate an APP value in a previous label-regex statement or label-plugin statement.

As mentioned in the "Rule Execution" section above, for port-based tests yaf uses the rule's APP number (e.g., 443 for TLS) as a hint that if a flow record's source or destination port matches that value (443), yaf should first check whether the record's payload matches the rule for that protocol (TLS).

The port statement extends this to tell yaf that traffic on additional ports should also first be checked against a particular rule. For example, port 993 is typically used by IMAPS (IMAP over TLS). To tell yaf to run the TLS test first on a flow record using port 993, specify this in the yafApplabelRules.conf file:

port 993 label 443

Port statements are useful in three instances:

1. When the common port is not the assigned port, such as IRC using port 6667 instead of the IANA-assigned port 194.

2. When the same protocol is used by services on multiple ports, such as the IMAPS example above.

3. When multiple protocols use common plugin code, such as the DNS-plugin recognizing DNS (port 53), NetBIOS (137), MDNS (5353), and LLMNR (5355).

Proxy Traffic

If yaf is seeing traffic behind a web proxy, it may incorrectly label https (443) traffic as http (80) due to the HTTP Connect method that occurs before the certificate exchange. To accurately label https traffic, uncomment the following line in the yafApplabelRules.conf file:

label APP plugin proxyplugin proxyplugin_LTX_ycProxyScanScan

and set APP to the port on which the proxy is listening for connections. This does label https flows as APP; instead It sets the application label to 443 and allows the DPI plugin to capture and export X.509 Certificates.

DEFINED LABELS

As of YAF-2.19, the following application labels are enabled by default. They are presented in alphabetical order.

Notes:

1. IRC servers typically run on port 6667 but 194 is the port assigned by IANA

2. LLMNR, MDNS, and NETBIOS are not included in the configuration file. They are contained in the DNS decoder due to their similarities with the DNS Protocol.

3. RTCP is not included in the configuration file. It is contained in the RTP decoder due to similarities in the protocols.

SEE ALSO

yaf(1), yafdpi(1), yafscii(1), rwipfix2silk(1), rwflowpack(8), flowcap(8), https://www.pcre.org/, pcre2pattern(3) or pcre2pattern