NAME

rwtuc - Text Utility Converter - rwcut output to SiLK flows

SYNOPSIS

  rwtuc [--fields=FIELDS] [--column-separator=CHAR]
        [--output-path=FILEPATH] [--bad-input-lines=FILEPATH]
        [--note-add=TEXT] [--note-file-add=FILE]
        [--compression-method=COMP_METHOD]
        [--site-config-file=FILENAME] [--saddress=IPADDR]
        [--daddress=IPADDR] [--sport=NUM] [--dport=NUM]
        [--protocol=NUM] [--packets=NUM] [--bytes=NUM]
        [--flags-all=TCPFLAGS] [--stime=TIME] [--duration=NUM]
        [--etime=TIME] [--sensor=SID] [--input-index=NUM]
        [--output-index=NUM] [--next-hop-ip=IPADDR]
        [--flags-initial=TCPFLAGS] [--flags-session=TCPFLAGS]
        [--attributes=ATTR] [--application=NUM] [--class=NAME]
        [--type=NAME] [--stime+msec=TIME] [--etime+msec=TIME]
        [--duration+msec=NUM] [FILES]

  rwtuc --help

  rwtuc --version

DESCRIPTION

rwtuc reads text files that have a format similar to that produced by rwcut(1) and attempts to create a SiLK Flow record for each line of input.

The field which make up a single record should be separated by the pipe character ('|'); use the --column-separator switch to change this delimiter. Note that space will not work as delimiter since several fields (e.g., time, tcp-flags) contain embedded spaces.

The fields to be read from each line can be specified with the --fields switch; if the switch is not provided, rwtuc treats the first line as a title and attempts to determine the fields from the title strings.

Fields can be forced to have a fixed value, which causes rwtuc to override the value read from the input file (if any).

The textual input is read from the files named on the command line; if no files are specified, rwtuc attempts to read the text from the standard input if it is not connected to a terminal.

When the --output-path switch is not provided, output is sent to the standard output when it is not connected to a terminal.

Lines that cannot be parsed are ignored (unless rwtuc is attempting to determine the fields from the title line). These lines can be copied to another output stream with the --bad-input-lines switch. Each bad line will have the source file name and line number prepended to it, separated from each other and the source line by colons (':').

Due to the way records are stored, some field combinations cannot be supported. The end time is computed by adding the duration to the start time; when start time and duration are specified, any end-time value is ignored. The initial TCP flags and session TCP flags fields must both be present or absent. Using the attributes or application fields requires using the initial TCP flags field. The class and type field must both be present for the fields to have any affect on the SiLK flow record.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--fields=FIELDS: FIELDS contains the list of fields (columns) to parse. FIELDS is a comma separated list of field-names, field-integers, and ranges of field-integers; a range is specified by separating the start and end of the range with a hyphen (-). Field-names are case insensitive.; The field names and their descriptions are:
--column-separator=CHAR: Expect the character CHAR to used as the delimiter between columns instead of the default '|'.
--output-path=FILEPATH: Write the SiLK Flow records to FILEPATH. The strings stdout and stderr may be used for the standard output and standard error, respectively, as long as they are not connected to a terminal.
--bad-input-lines=FILEPATH: Copy any lines which could not be parsed to FILEPATH. The strings stdout and stderr may be used for the standard output and standard error, respectively. Each bad line will be prepended by the source input file, a colon, the line number, and a colon. On exit, rwtuc will remove FILEPATH if all input lines were successfully parsed.
--note-add=TEXT: Add the specified TEXT to the header of the output file as an annotation. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool.
--note-file-add=FILENAME: Open FILENAME and add the contents of that file to the header of the output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to ensure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation.
--compression-method=COMP_METHOD: Set the compression method of the output to COMP_METHOD. Some SiLK tools can use an external library to compress their binary output. The list of available compression methods and the default method are set when SiLK is compiled (the --help and --version switches print the available and default compression methods) and depend on which supported libraries are found. SiLK can support:
--site-config-file=FILENAME: Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, the location specified by the SILK_CONFIG_FILE environment variable is used if that variable is not empty. The value of SILK_CONFIG_FILE should include the name of the file. Otherwise, the application looks for a file named silk.conf in the following directories: the directory specified in the SILK_DATA_ROOTDIR environment variable; the data root directory that is compiled into SiLK (use the --version switch to view this value); the directories $SILK_PATH/share/silk/ and $SILK_PATH/share/; and the share/silk/ and share/ directories parallel to the application's directory.
--help: Print the available options and exit.
--version: Print the version number and information about how SiLK was configured, then exit the application.

The following switches can be used to set fields to fixed values:

--saddress=IPADDR: Set the source address field to IPADDR for all records. IPADDR can be in canonical notation or an unsigned integer.
--daddress=IPADDR: Set the destination address field to IPADDR for all records. IPADDR can be in canonical notation or an unsigned integer.
--sport=NUM: Set the source port field to NUM for all records; a value between 0 and 65535.
--dport=NUM: Set the destination port field to NUM for all records; a value between 0 and 65535.
--protocol=NUM: Set the protocol field to NUM for all records; a value between 0 and 255.
--packets=NUM: Set the packets field to NUM for all records; the value must be non-zero.
--bytes=NUM: Set the bytes field to NUM for all records; the value must be non-zero.
--flags-all=TCPFLAGS: Set the TCP flags field to TCPFLAGS for all records.
--stime=TIME: Set the start time field to TIME for all records.
--duration=NUM: Set the duration field to NUM for all records.
--etime=TIME: Set the end time field to TIME for all records.
--sensor=SID: Set the sensor field to SID for all records. This can either be a sensor name or sensor ID.
--input-index=NUM: Set the SNMP input index field to NUM for all records; a value between 0 and 65535.
--output-index=NUM: Set the SNMP output index field to NUM for all records; a value between 0 and 65535.
--next-hop-ip=IPADDR: Set the next-hop-ip field to IPADDR for all records. IPADDR can be in canonical notation or an unsigned integer.
--flags-initial=TCPFLAGS: Set the initial TCP flags field to TCPFLAGS for all records.
--flags-session=TCPFLAGS: Set the session TCP flags field to TCPFLAGS for all records.
--attributes=ATTR: Set the attributes field to ATTR for all records.
--application=NUM: Set the application field to NUM for all records; a value between 0 and 65535.
--class=NAME: Set the class field to NAME for all records. Should be a valid class name; use rwfilter --help to see the list of valid classes for your site.
--type=NAME: Set the type field to NAME for all records. Should be a valid type name; use rwfilter --help to see the list of valid types for your site.
--stime+msec=TIME: Set the start time field (including milliseconds) to TIME for all records.
--etime+msec=TIME: Set the end time field (including milliseconds) to TIME for all records.
--duration+msec=NUM: Set the duration field (including milliseconds) to NUM for all records.

EXAMPLES

 $ rwcut data.rwf > cut.txt
 $ md5 < cut.txt
 7e3d693cd2cba2510803935274e1debd
 $ rwtuc < cut.txt | rwcut | md5
 7e3d693cd2cba2510803935274e1debd

ENVIRONMENT