NAME

rwcat - Concatenate SiLK Flow files into single stream

SYNOPSIS

  rwcat [--output-path=FILE] [--note-add=TEXT] [--note-file-add=FILE]
        [--print-filenames] [--byte-order={big | little | native}]
        [--ipv4-output] [--compression-method=COMP_METHOD]
        [--site-config-file=FILENAME]
        {[--xargs] | [--xargs=FILENAME] | [ input-files ... ]}

  rwcat --help

  rwcat --version

DESCRIPTION

rwcat reads SiLK Flow records from the specified input files and writes the records in the standard binary SiLK format to the specified output-path; rwcat will write the records to the standard output when stdout is not the terminal and --output-path is not provided.

When the --xargs switch is provided, rwcat will read the names of the files to process from the named text file, or from the standard input if no file name argument is provided to the switch. The input should contain one filename per line.

If the input file names end in .gz, they will be uncompressed as they are read. When stdin is provided as an input file name, rwcat will read records from the standard input.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--output-path=FILE: Write the SiLK Flow records to FILE, which must not exist. If the switch is not provided or if FILE is stdout, flows are written to the standard output. If the name ends in .gz, the output will be compressed using gzip(1).
--note-add=TEXT: Add the specified TEXT to the header of the output file as an annotation. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool.
--note-file-add=FILENAME: Open FILENAME and add the contents of that file to the header of the output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to ensure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation.
--byte-order=ENDIAN: Set the byte order for the output SiLK Flow records. The argument is one of the following:
--ipv4-output: Force the output to contain only IPv4 addresses. When this switch is specified, IPv6 addresses are ignored unless the IPv6 address is an encapsulation of an IPv4 address, in which case the IPv4 address will be written to the output. By default, rwcat writes IP addresses in the same format as the input file. When SiLK has not been compiled with IPv6 support, this switch has no effect.
--compression-method=COMP_METHOD: Set the compression method of the output to COMP_METHOD. Some SiLK tools can use an external library to compress their binary output. The list of available compression methods and the default method are set when SiLK is compiled (the --help and --version switches print the available and default compression methods) and depend on which supported libraries are found. SiLK can support:
--print-filenames: Print the names of input files and the number of records each file contains as the files are read.
--site-config-file=FILENAME: Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, the location specified by the SILK_CONFIG_FILE environment variable is used if that variable is not empty. The value of SILK_CONFIG_FILE should include the name of the file. Otherwise, the application looks for a file named silk.conf in the following directories: the directory specified in the SILK_DATA_ROOTDIR environment variable; the data root directory that is compiled into SiLK (use the --version switch to view this value); the directories $SILK_PATH/share/silk/ and $SILK_PATH/share/; and the share/silk/ and share/ directories parallel to the application's directory.
--xargs
--xargs=FILENAME: Causes rwcat to read file names from FILENAME or from the standard input if FILENAME is not provided. The input should have one file name per line. rwcat will open each file in turn and read records from it, as if the files had been listed on the command line.
--help: Print the available options and exit.
--version: Print the version number and information about how SiLK was configured, then exit the application.

EXAMPLES

To combine the results of several rwfilter runs---stored in the files run1.rwf, run2.rwf, ... runN.rwf---together, you can use:

  rwcat --output=combined.dat  *.rwf

If the shell complains about too many arguments, you can use the UNIX find(1) function and pipe its output to rwcat:

  find . -name '*.rwf' -print | \
        rwcat --xargs --output=combined.dat

ENVIRONMENT

SILK_CONFIG_FILE: This environment variable is used as the value for the --site-config-file when that switch is not provided.
SILK_DATA_ROOTDIR: When the --site-config-file switch is not provided and the SILK_CONFIG_FILE environment variable is not set, rwcat looks for the site configuration file in $SILK_DATA_ROOTDIR/silk.conf.
SILK_PATH: This environment variable gives the root of the install tree. As part of its search for the SiLK site configuration file, rwcat checks for a file named silk.conf in the directories $SILK_PATH/share/silk and $SILK_PATH/share.

BUGS

Although rwcat will read from the standard input, this feature should be used with caution. rwcat will treat the standard input as a single file, as it has no way to know when one file ends and the next begins. The following will not work:

  cat run1.rwf run2.rwf | rwcat --output=combined.dat  # WRONG!

The header of run2.rwf will be treated as data of run1.rwf, resulting in corrupt output.