NAME

flowcap - Capture, temporarily store, and forward flow data

SYNOPSIS

  flowcap --destination-directory=DIR
        --sensor-configuration=FILENAME [--probes=NAME[,NAME...]]
        --max-file-size=SIZE [--fc-version=NUM]
        [--timeout=TIMEOUT] [--clock-time[=OFFSET]]
        [--freespace-minimum=SIZE] [--space-maximum-percent=NUM]
        [--compression-method=COMP_METHOD]
        { --log-destination=DESTINATION
          | --log-pathname=FILE_PATH
          | --log-directory=DIR_PATH [--log-basename=LOG_BASENAME]
            [--log-post-rotate=COMMAND] }
        [--log-level=LEVEL] [--log-sysfacility=NUMBER]
        [--pidfile=FILE_PATH] [--no-chdir] [--no-daemon]

Help options:

  flowcap --sensor-configuration=FILE_PATH
        { --verify-sensor-config | --verify-sensor-config=VERBOSE }

  flowcap --help

  flowcap --version

DESCRIPTION

flowcap is a daemon that collects data from devices that produce flow data (such as a router producing NetFlow v5 or a flow meter producing IPFIX (Internet Protocol Flow Information eXport)), temporarily stores the data to files on its local disk, and forwards these files to rwflowpack(8) for packing.

As flowcap receives flow records, it stores them in files in the location specified by the --destination-directory switch. These files are closed on quantum boundaries, with one file per flow source per quantum. A quantum is either the amount of time represented by the --timeout switch or the file size represented by the --max-file-size switch, whichever is reached first.

To transfer the files to rwflowpack, flowcap works in tandem with the rwsender(8) program. rwsender polls the storage directory and sends the files it finds there to an rwreceiver(8) process for processing by rwflowpack.

flowcap produces files that are named PROBE_YYYYMMDDhhmmss.XXXXXX, where PROBE is the name of the probe, YYYY is the current year, MM is the current month, DD is the current day, hh is the current hour, mm is the current minute, ss is the current second, and XXXXXX is a random six-character string.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

For the following options, a SIZE may be given as an ordinary integer, or as a real number followed by a suffix K, M, G, or T, which represents the numerical value multiplied by 1,024 (kilo), 1,048,576 (mega), 1,073,741,824 (giga), and 1,099,511,627,776 (tera), respectively. For example, 1.5K represents 1,536 bytes, or one and one-half kilobytes.

General Configuration Switches

--destination-directory=DIR

Store aggregated packed flow files in this directory for processing by rwsender. This switch is required.

--sensor-configuration=FILENAME

Give the path to the configuration file that flowcap will consult to determine how to collect flow records. The complete syntax of the configuration file is described in the sensor.conf(5) manual page; see also the SiLK Installation Handbook. This switch is required.

--probes=NAME[,NAME...]

Choose which of the probes described in the sensor configuration file will be used by flowcap. The default is to use all of the probes defined in the configuration file. This switch instructs flowcap to only use the specifically named probes.

--max-file-size=SIZE

Set the approximate maximum size of flowcap files to SIZE bytes. If a flowcap file's exceeds SIZE bytes, it will be closed, and a new file will be created and used. In addition, before opening an output file, flowcap ensures there are SIZE bytes of free space available, and exits if there is not. This switch is required.

--timeout=TIMEOUT

Set the maximum duration that a flowcap output file remains open to TIMEOUT seconds. When the --clock-time switch is given, the first duration will be less than TIMEOUT seconds. If the --timeout switch is not specified, flowcap uses a default of 60 seconds.

--clock-time[=OFFSET]

Force flowcap to close its files at predictable times. When this switch is provided, flowcap closes its output files at OFFSET seconds after midnight (UTC of the current day) and at every TIMEOUT seconds thereafter. The default value of OFFSET is 0. For example, --timeout=900 --clock-time=300 causes flowcap to close its output files at the 05, 20, 35, and 50 minute points in each hour. Even with this switch, files will still be closed if they exceed the size specified by --max-file-size.

--fc-version=NUM

Choose the record version for the files of IPv4 flow records that flowcap produces. Valid values are 2, 3, 4, and 5, and the default is 5. This switch is ignored for probes that support IPv6 addresses.

--freespace-minimum=SIZE

Set the minimum free space to maintain on the file system where the --destination-directory is located. By default, flowcap assumes that it has full rein over the file system on which it writes its files. The default is to leave 1GB of free space. If flowcap fills this space, it will exit. Flows arriving during this time will be lost. See also --space-maximum-percent.

--space-maximum-percent=NUM

Use no more than this percentage of the file system containing the --destination-directory. The default is to use no more than 98% of the file system. If flowcap fills this space, it will exit. See also --freespace-minimum.

--compression-method=COMP_METHOD

Specify the compression library to use when writing output files. When no compression method is specified, flowcap files are compressed using the best method, regardless of the default chosen when SiLK was compiled. The valid values for COMP_METHOD are determined by which external libraries were found when SiLK was compiled. To see the available compression methods and the default method, use the --help or --version switch. SiLK can support the following COMP_METHOD values when the required libraries are available.

none

Do not compress the output using an external library.

zlib

Use the zlib(3) library for compressing the output. Using zlib produces the smallest output files at the cost of speed.

lzo1x

Use the lzo1x algorithm from the LZO real time compression library for compression. This compression provides good compression with less memory and CPU overhead.

snappy

Use the snappy library for compression. This compression provides good compression with less memory and CPU overhead. Since SiLK 3.13.0.

best

Use lzo1x if available, otherwise use snappy if available, otherwise use zlib if available.

--verify-sensor-config
--verify-sensor-config=VERBOSE

Verify that the syntax of the sensor configuration file is correct and then exit flowcap. If the file is incorrect or if it does not define any probes, an error message is printed and flowcap exits abnormally. If the file is correct and no argument is provided to the --verify-sensor-config switch, flowcap simply exits with status 0. If an argument (other than the empty string and 0) is provided to the switch, the names of the probes found in the sensor configuration file are printed to the standard output, and then flowcap exits.

--help

Print the available options and exit.

--version

Print the version number and information about how SiLK was configured, then exit the application.

Logging and Daemon Configuration Switches

The switches in this section determine the type of log messages that flowcap generates and where those messages are written.

One of the following switches are required:

--log-destination=DESTINATION

Specify the destination where logging messages are written. When DESTINATION begins with a slash /, it is treated as a file system path and all log messages are written to that file; there is no log rotation. When DESTINATION does not begin with /, it must be one of the following strings:

none

Messages are not written anywhere.

stdout

Messages are written to the standard output.

stderr

Messages are written to the standard error.

syslog

Messages are written using the syslog(3) facility.

both

Messages are written to the syslog facility and to the standard error (this option is not available on all platforms).

--log-directory=DIR_PATH

Use DIR_PATH as the directory to which the log files are written; DIR_PATH must be a complete directory path. The log files have the form

 DIR_PATH/LOG_BASENAME-YYYYMMDD.log

where YYYYMMDD is the current date and LOG_BASENAME is the application name or the value passed to the --log-basename switch when provided. The log files are rotated: At midnight local time, a new log is opened, the previous file is closed, and the command specified by --log-post-rotate is invoked on the previous day's log file. (Old log files are not removed by flowcap; the administrator should use another tool to remove them.) When this switch is provided, a process-ID file (PID) is also written in this directory unless the --pidfile switch is provided.

--log-pathname=FILE_PATH

Use FILE_PATH as the complete path to the log file. The log file is not rotated.

The following switches are optional:

--log-level=LEVEL

Set the severity of messages that will be logged. The levels from most severe to least are: emerg, alert, crit, err, warning, notice, info, debug. The default is info.

--log-sysfacility=NUMBER

Set the facility that syslog(3) uses for logging messages. This switch takes a number as an argument. The default is a value that corresponds to LOG_USER on the system where flowcap is running. This switch produces an error unless --log-destination=syslog is specified.

--log-basename=LOG_BASENAME

Use LOG_BASENAME in place of the application name in the name of log files in the log directory. See the description of the --log-directory switch. This switch does not affect the name of the process-ID file.

--log-post-rotate=COMMAND

Run COMMAND on the previous day's log file after log rotation. When this switch is not specified, the previous day's log file is compressed with gzip(1). When the switch is specified and COMMAND is the empty string, no action is taken on the log file. Each occurrence of the string %s in COMMAND will be replaced with the full path to the log file, and each occurrence of %% will be replaced with %. If any other character follows %, flowcap exits with an error. Specifying this switch without also using --log-directory is an error.

--pidfile=FILE_PATH

Set the complete path to the file in which flowcap writes its process ID (PID) when it is running as a daemon. No PID file is written when --no-daemon is given. When this switch is not present, no PID file is written unless the --log-directory switch is specified, in which case the PID is written to LOGPATH/flowcap.pid.

--no-chdir

Do not change directory to the root directory. When flowcap becomes a daemon process, it changes its current directory to the root directory so as to avoid potentially running on a mounted file system. Specifying --no-chdir prevents this behavior, which may be useful during debugging. The application does not change its directory when --no-daemon is given.

--no-daemon

Force flowcap to run in the foreground---it does not become a daemon process. This may be useful during debugging.

ENVIRONMENT

SILK_IPFIX_PRINT_TEMPLATES

When set to 1, flowcap writes messages to the log file describing each IPFIX and NetFlow v9 template it receives. The first message includes the domain, the template identifier, the number of information elements in the template, and the name of this environment variable. Next, a message is printed for each information element in the template where the message contains the domain id, the template id, and the element's position in the template, length in octets, numeric information element identifier, and name. For elements defined by a private enterprise, the IE number has two parts: the private enterprise number and the information element number, separated by a slash (/). (Requires libfixbuf 1.4.0 or later.) Since SiLK 3.8.2.

SILK_LIBFIXBUF_SUPPRESS_WARNINGS

When set to 1, flowcap disables all warning messages generated by libfixbuf. These warning messages include out-of-sequence packets, data records not having a corresponding template, record count discrepancies, and issues decoding list elements. Since SiLK 3.10.0.

FILES

sensor.conf

The location of this file must be specified by the --sensor-configuration switch. This file specifies probe blocks that tell flowcap how to capture data. The syntax of this file is described in the sensor.conf(5) manual page.

SEE ALSO

sensor.conf(5), rwflowpack(8), rwsender(8), rwreceiver(8), silk(7), syslog(3), zlib(3), gzip(1), SiLK Installation Handbook