SiLK

NAME

rwnetmask - Zero out lower bits of IP addresses in SiLK Flow records

SYNOPSIS

rwnetmask [--4sip-prefix-length=N] [--6sip-prefix-length=N]
      [--4dip-prefix-length=N] [--6dip-prefix-length=N]
      [--4nhip-prefix-length=N] [--6nhip-prefix-length=N]
      [--sip-prefix-length=N] [--dip-prefix-length=N]
      [--nhip-prefix-length=N] [--output-path=PATH]
      [--print-filenames] [--ipv6-policy=POLICY]
      [--note-add=TEXT] [--note-file-add=FILE]
      [--compression-method=COMP_METHOD]
      [--site-config-file=FILENAME]
      {[--xargs] | [--xargs=FILENAME] | [FILE [FILE ...]]}

rwnetmask --help

rwnetmask --version

DESCRIPTION

rwnetmask reads SiLK Flow records, sets the prefix of the source IP, destination IP, and/or next hop IP to the specified value(s) by masking the least significant bits of the address(es), and writes the modified SiLK Flow records to the specified output path. Modifying the IP addresses allows one to group IPs into arbitrary CIDR blocks. Multiple prefix-lengths may be specified; at least one must be specified.

When SiLK is compiled with IPv6 support, a separate mask can be specified for IPv4 and IPv6 addresses. Records are processed using the IP-version in which they are read. The --ipv6-policy switch can be used to force the records into a particular IP-version or to ignore records of a particular IP-version.

rwnetmask reads SiLK Flow records from the files named on the command line or from the standard input when no file names are specified and --xargs is not present. To read the standard input in addition to the named files, use - or stdin as a file name. If an input file name ends in .gz, the file is uncompressed as it is read. When the --xargs switch is provided, rwnetmask reads the names of the files to process from the named text file or from the standard input if no file name argument is provided to the switch. The input to --xargs must contain one file name per line.

When no output path is specified and the standard output is not connected to a terminal, rwnetmask writes the records to the standard output.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

One of these switches must be provided:

--4sip-prefix-length=N

--sip-prefix-length=N

For IPv4 addresses, specify the number of most significant bits of the source address to keep. The default is to not mask off any bits (i.e., N=32).

--4dip-prefix-length=N

--dip-prefix-length=N

For IPv4 addresses, specify the number of most significant bits of the destination address to keep. The default is to not mask off any bits (i.e., N=32).

--4nhip-prefix-length=N

--nhip-prefix-length=N

For IPv4 addresses, specify the number of most significant bits of the next-hop address to keep. The default is to not mask off any bits (i.e., N=32).

--6sip-prefix-length=N

For IPv6 addresses, specify the number of most significant bits of the source address to keep. The default is to not mask off any bits (i.e., N=128).

--6dip-prefix-length=N

For IPv6 addresses, specify the number of most significant bits of the destination address to keep. The default is to not mask off any bits (i.e., N=128).

--6nhip-prefix-length=N

For IPv6 addresses, specify the number of most significant bits of the next-hop address to keep. The default is to not mask off any bits (i.e., N=128).

These switches are optional:

--output-path=PATH

Write the binary SiLK Flow records to PATH, where PATH is a filename, a named pipe, the keyword stderr to write the output to the standard error, or the keyword stdout or - to write the output to the standard output. If PATH names an existing file, rwrwnetmask exits with an error unless the SILK_CLOBBER environment variable is set, in which case PATH is overwritten. If this switch is not given, the output is written to the standard output. Attempting to write the binary output to a terminal causes rwrwnetmask to exit with an error.

--print-filenames

Print to the standard error the names of the input files as the files are opened.

--ipv6-policy=POLICY

Determine how IPv4 and IPv6 flows are handled when SiLK has been compiled with IPv6 support. When the switch is not provided, the SILK_IPV6_POLICY environment variable is checked for a policy. If it is also unset or contains an invalid policy, the POLICY is mix. When SiLK has not been compiled with IPv6 support, IPv6 flows are always ignored, regardless of the value passed to this switch or in the SILK_IPV6_POLICY variable. The supported values for POLICY are:

ignore

Ignore any flow record marked as IPv6, regardless of the IP addresses it contains. Only records marked as IPv4 will be processed.

asv4

Convert IPv6 flow records that contain addresses in the ::ffff:0:0/96 netblock (that is, IPv4-mapped IPv6 addresses) to IPv4 and ignore all other IPv6 flow records.

mix

Process the input as a mixture of IPv4 and IPv6 flows.

force

Convert IPv4 flow records to IPv6, mapping the IPv4 addresses into the ::ffff:0:0/96 netblock.

only

Process only flow records that are marked as IPv6 and ignore IPv4 flow records in the input.

--note-add=TEXT

Add the specified TEXT to the header of the output file as an annotation. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool.

--note-file-add=FILENAME

Open FILENAME and add the contents of that file to the header of the output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to ensure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation.

--compression-method=COMP_METHOD

Specify the compression library to use when writing output files. If this switch is not given, the value in the SILK_COMPRESSION_METHOD environment variable is used if the value names an available compression method. When no compression method is specified, output to the standard output or to named pipes is not compressed, and output to files is compressed using the default chosen when SiLK was compiled. The valid values for COMP_METHOD are determined by which external libraries were found when SiLK was compiled. To see the available compression methods and the default method, use the --help or --version switch. SiLK can support the following COMP_METHOD values when the required libraries are available.

none

Do not compress the output using an external library.

zlib

Use the zlib(3) library for compressing the output, and always compress the output regardless of the destination. Using zlib produces the smallest output files at the cost of speed.

lzo1x

Use the lzo1x algorithm from the LZO real time compression library for compression, and always compress the output regardless of the destination. This compression provides good compression with less memory and CPU overhead.

snappy

Use the snappy library for compression, and always compress the output regardless of the destination. This compression provides good compression with less memory and CPU overhead. Since SiLK 3.13.0.

best

Use lzo1x if available, otherwise use snappy if available, otherwise use zlib if available. Only compress the output when writing to a file.

--site-config-file=FILENAME

Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, rwnetmask searches for the site configuration file in the locations specified in the "FILES" section.

--xargs

--xargs=FILENAME

Read the names of the input files from FILENAME or from the standard input if FILENAME is not provided. The input is expected to have one filename per line. rwnetmask opens each named file in turn and reads records from it as if the filenames had been listed on the command line.

--help

Print the available options and exit.

--version

Print the version number and information about how SiLK was configured, then exit the application.

EXAMPLES

In the following example, the dollar sign ($) represents the shell prompt. The text after the dollar sign represents the command line. Lines have been wrapped for improved readability, and the back slash (\) is used to indicate a wrapped line.

To summarize the TCP traffic from your network to each /24 on the Internet, use:

$ rwfilter --type=out,outweb --proto=6 --pass=stdout       \
  | rwnetmask --dip-prefix-length 24                       \
  | rwaddrcount --use-dest --sort --print-rec
IP Address| Bytes|Packets|Records|         Start Time|...
10.10.35.0|  2345|     52|      6|01/15/2003 19:30:31|
 10.23.3.0|   118|      2|      1|01/16/2003 19:38:40|
 10.23.4.0| 20858|    263|     16|01/16/2003 16:54:25|
10.31.49.0|266920|   3885|   1092|01/11/2003 02:04:11|
10.126.7.0| 36912|    260|      9|01/16/2003 17:03:28|
....

ENVIRONMENT

SILK_IPV6_POLICY

This environment variable is used as the value for --ipv6-policy when that switch is not provided.

SILK_CLOBBER

The SiLK tools normally refuse to overwrite existing files. Setting SILK_CLOBBER to a non-empty value removes this restriction.

SILK_COMPRESSION_METHOD

This environment variable is used as the value for --compression-method when that switch is not provided. Since SiLK 3.13.0.

SILK_CONFIG_FILE

This environment variable is used as the value for the --site-config-file when that switch is not provided.

SILK_DATA_ROOTDIR

This environment variable specifies the root directory of data repository. As described in the "FILES" section, rwnetmask may use this environment variable when searching for the SiLK site configuration file.

SILK_PATH

This environment variable gives the root of the install tree. When searching for configuration files, rwnetmask may use this environment variable. See the "FILES" section for details.

FILES

${SILK_CONFIG_FILE}

${SILK_DATA_ROOTDIR}/silk.conf

/data/silk.conf

${SILK_PATH}/share/silk/silk.conf

${SILK_PATH}/share/silk.conf

/usr/share/silk/silk.conf

/usr/share/silk.conf

Possible locations for the SiLK site configuration file which are checked when the --site-config-file switch is not provided.

SEE ALSO

rwfileinfo(1), silk(7), zlib(3)