NAME

rwset - Generate binary IPset files of unique IP addresses

SYNOPSIS

  rwset [--sip-file=FILE] [--dip-file=FILE] [--nhip-file=FILE]
        [--print-filenames] [--copy-input=PATH]
        [--note-add=TEXT] [--note-file-add=FILE]
        [--compression-method=COMP_METHOD]
        [--site-config-file=FILENAME] [FILES...]

DESCRIPTION

rwset reads SiLK Flow data from the standard input, a named pipe, or named input files and generates binary IPset file(s) containing the list of the unique source IP addresses, destination IP addresses, and/or next-hop IP addresses.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

One of the following output switches is required:

--sip-file=FILE: Store the unique source IP addresses in the binary IPset file FILE. rwset will write the IPset file to the standard output when FILE is stdout and the standard output is not a terminal.
--dip-file=FILE: Store the unique destination IP addresses in the binary IPset file FILE. rwset will write the IPset file to the standard output when FILE is stdout and the standard output is not a terminal.
--nhip-file=FILE: Store the unique next-hop IP addresses in the binary IPset file FILE. rwset will write the IPset file to the standard output when FILE is stdout and the standard output is not a terminal.

Only one of the above switches my use stdout as the name of the file.

rwset supports these additional switches:

--print-filenames: Prints to the standard error the names of input files as they are opened.
--copy-input=PATH: Copy all binary input to the specified file or named pipe. PATH can be stdout to print flows to the standard output as long as no IPset files are being written there.
--note-add=TEXT: Add the specified TEXT to the header of every output file as an annotation. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool.
--note-file-add=FILENAME: Open FILENAME and add the contents of that file to the header of every output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to ensure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation.
--compression-method=COMP_METHOD: Set the compression method of the output to COMP_METHOD. Some SiLK tools can use an external library to compress their binary output. The list of available compression methods and the default method are set when SiLK is compiled (the --help and --version switches print the available and default compression methods) and depend on which supported libraries are found. SiLK can support:
--site-config-file=FILENAME: Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, the location specified by the SILK_CONFIG_FILE environment variable is used if that variable is not empty. The value of SILK_CONFIG_FILE should include the name of the file. Otherwise, the application looks for a file named silk.conf in the following directories: the directory specified in the SILK_DATA_ROOTDIR environment variable; the data root directory that is compiled into SiLK (use the --version switch to view this value); the directories $SILK_PATH/share/silk/ and $SILK_PATH/share/; and the share/silk/ and share/ directories parallel to the application's directory.

EXAMPLES

rwset is intended to work tightly with rwfilter. For example, consider generating two sets: one of all IP addresses which have 3 packets or less in their sessions, and one with four or more packets.

The first set, for TCP traffic on 03/01/2003 can be generated with:

  rwfilter --start-date=2003/03/01:00 --end-date=2003/03/01:23 \
        --proto=6 --packets=1-3 --pass=stdout | \
        rwset --sip-file=low_packet_tcp.set

The second set with:

  rwfilter --start-date=2003/03/01:00 --end-date=2003/03/01:23
        --proto=6 --packets=4- --pass=stdout | \
        rwset --sip-file=high_packet_tcp.set

ENVIRONMENT

SILK_CONFIG_FILE: This environment variable is used as the value for the --site-config-file when that switch is not provided.
SILK_DATA_ROOTDIR: When the --site-config-file switch is not provided and the SILK_CONFIG_FILE environment variable is not set, rwset looks for the site configuration file in $SILK_DATA_ROOTDIR/silk.conf.
SILK_PATH: This environment variable gives the root of the install tree. As part of its search for the SiLK site configuration file, rwset checks for a file named silk.conf in the directories $SILK_PATH/share/silk and $SILK_PATH/share.

NOTES

rwset does not support IPv6 addresses. Any IPv6 data in the input will be ignored.

IPset files are in a binary format. To create a binary IPset file from a list of dotted decimal IP addresses, use rwsetbuild(1).

Use rwsetcat(1) to see the IP addresses in a binary set file.

rwsettool(1) allows you to perform set operations on binary IPset files.

The command:

  rwset --sip-file=stdout | rwsetcat

will be faster than rwuniq, but will cannot report total volume nor do the thresholding that rwuniq supports.

BUGS

Currently there is no support for IPv6 addresses in IPsets.

When used in an IPv6 environment, rwset will attempt to convert any IPv6 addresses to IPv4. Records that can be converted will be processed, all other records will be silently ignored.