SiLK

NAME

rwcut - Print selected fields of binary SiLK Flow records

SYNOPSIS

rwcut [{--fields=FIELDS | --all-fields}]
      {[--start-rec-num=START_NUM] [--end-rec-num=END_NUM]
       | [--tail-recs=TAIL_START_NUM]}
      [--num-recs=REC_COUNT] [--dry-run] [--icmp-type-and-code]
      [--timestamp-format=FORMAT] [--epoch-time]
      [--ip-format=FORMAT] [--integer-ips] [--zero-pad-ips]
      [--integer-sensors] [--integer-tcp-flags]
      [--no-titles] [--no-columns] [--column-separator=CHAR]
      [--no-final-delimiter] [{--delimited | --delimited=CHAR}]
      [--print-filenames] [--copy-input=PATH] [--output-path=PATH]
      [--pager=PAGER_PROG] [--site-config-file=FILENAME]
      [--ipv6-policy={ignore,asv4,mix,force,only}]
      [{--legacy-timestamps | --legacy-timestamps={1,0}}]
      [--plugin=PLUGIN [--plugin=PLUGIN ...]]
      [--python-file=PATH [--python-file=PATH ...]]
      [--pmap-file=MAPNAME:PATH [--pmap-file=MAPNAME:PATH ...]]
      [--pmap-column-width=NUM]
      {[--xargs] | [--xargs=FILENAME] | [FILE [FILE ...]]}

rwcut [--pmap-file=MAPNAME:PATH [--pmap-file=MAPNAME:PATH ...]]
      [--plugin=PLUGIN ...] [--python-file=PATH ...] --help

rwcut [--pmap-file=MAPNAME:PATH [--pmap-file=MAPNAME:PATH ...]]
      [--plugin=PLUGIN ...] [--python-file=PATH ...] --help-fields

rwcut --version

DESCRIPTION

rwcut reads binary SiLK Flow records and prints the user-selected record attributes (or fields) to the terminal in a textual, bar-delimited (|) format. See the "EXAMPLES" section below for sample output.

rwcut reads SiLK Flow records from the files named on the command line or from the standard input when no file names are specified and --xargs is not present. To read the standard input in addition to the named files, use - or stdin as a file name. If an input file name ends in .gz, the file is uncompressed as it is read. When the --xargs switch is provided, rwcut reads the names of the files to process from the named text file or from the standard input if no file name argument is provided to the switch. The input to --xargs must contain one file name per line.

The user may provide the --fields switch to select the record attributes to print. When --fields is not specified rwcut prints the source and destination IP address, source and destination port, protocol, packet count, byte count, TCP flags, start time, duration, end time, and the sensor name. The fields are printed in the order in which they occur in the --fields switch. Fields may be repeated.

A subset of the input records may be selected by using the --start-rec-num, --end-rec-num, --num-recs, and --tail-recs switches.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--fields=FIELDS

FIELDS contains the list of flow attributes (a.k.a. fields or columns) to print. The columns will be displayed in the order the fields are specified. Fields may be repeated. FIELDS is a comma separated list of field-names, field-integers, and ranges of field-integers; a range is specified by separating the start and end of the range with a hyphen (-). Field-names are case-insensitive. Example:

--fields=stime,10,1-5

If the --fields switch is not given, FIELDS defaults to:

sIP,dIP,sPort,dPort,protocol,packets,bytes,flags,sTime,dur,eTime,sensor

The complete list of built-in fields that the SiLK tool suite supports follows, though note that not all fields are present in all SiLK file formats; when a field is not present, its value is 0.

sIP,1

source IP address

dIP,2

destination IP address

sPort,3

source port for TCP and UDP, or equivalent

dPort,4

destination port for TCP and UDP, or equivalent

protocol,5

IP protocol

packets,pkts,6

packet count

bytes,7

byte count

flags,8

bit-wise OR of TCP flags over all packets

sTime,9

starting time of flow in microsecond resolution

duration,10

duration of flow in microsecond resolution

eTime,11

end time of flow in microsecond resolution

sensor,12

name or ID of sensor at the collection point

class,20

class of sensor at the collection point

type,21

type of sensor at the collection point

iType

the ICMP type value for ICMP or ICMPv6 flows and empty for non-ICMP flows. This field was introduced in SiLK 3.8.1.

iCode

the ICMP code value for ICMP or ICMPv6 flows and empty for non-ICMP flows. See note at iType.

icmpTypeCode,25

equivalent to iType,iCode. This field is deprecated as of SiLK 3.8.1.

Many SiLK file formats do not store the following fields and their values will always be 0; they are listed here for completeness:

in,13

router SNMP input interface or vlanId if packing tools were configured to capture it (see sensor.conf(5))

out,14

router SNMP output interface or postVlanId

nhIP,15

router next hop IP

Enhanced flow metering software (such as yaf(1)) may provide flow information elements in addition to those found in NetFlow. SiLK stores some of these elements in the fields named below. For flows without this additional information, the field's value is always 0.

initialFlags,26

TCP flags on first packet in the flow

sessionFlags,27

bit-wise OR of TCP flags on the second through final packets in the flow

attributes,28

flow attributes set by the flow generator:

S

all the packets in this flow record are exactly the same size

F

flow generator saw additional packets in this flow following a packet with a FIN flag (excluding ACK packets)

T

flow generator prematurely created a record for a long-running connection due to a timeout. (When the flow generator yaf(1) is run with the --silk switch, it will prematurely create a flow and mark it with T if the byte count of the flow cannot be stored in a 32-bit value.)

C

flow generator created this flow as a continuation of long-running connection, where the previous flow for this connection met a timeout (or a byte threshold in the case of yaf).

Consider a long-running ssh session that exceeds the flow generator's active timeout. (This is the active timeout since the flow generator creates a flow for a connection that still has activity). The flow generator will create multiple flow records for this ssh session, each spanning some portion of the total session. The first flow record will be marked with a T indicating that it hit the timeout. The second through next-to-last records will be marked with TC indicating that this flow both timed out and is a continuation of a flow that timed out. The final flow will be marked with a C, indicating that it was created as a continuation of an active flow.

application,29

guess as to the content of the flow. Some software that generates flow records from packet data, such as yaf, will inspect the contents of the packets that make up a flow and use traffic signatures to label the content of the flow. SiLK calls this label the application; yaf refers to it as the appLabel. The application is the port number that is traditionally used for that type of traffic (see the /etc/services file on most UNIX systems). For example, traffic that the flow generator recognizes as FTP will have a value of 21, even if that traffic is being routed through the standard HTTP/web port (80).

The following fields provide a way to label the IPs or ports on a record. These fields require external files to provide the mapping from the IP or port to the label:

sType,16

for the source IP address, the value 0 if the address is non-routable, 1 if it is internal, or 2 if it is routable and external. Uses the mapping file specified by the SILK_ADDRESS_TYPES environment variable, or the address_types.pmap mapping file, as described in addrtype(3).

dType,17

as sType for the destination IP address

scc,18

for the source IP address, a two-letter country code abbreviation denoting the country where that IP address is located. Uses the mapping file specified by the SILK_COUNTRY_CODES environment variable, or the country_codes.pmap mapping file, as described in ccfilter(3). The abbreviations are those defined by ISO 3166-1 (see for example https://www.iso.org/iso-3166-country-codes.html or https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) or the following special codes: -- N/A (e.g. private and experimental reserved addresses); a1 anonymous proxy; a2 satellite provider; o1 other

dcc,19

as scc for the destination IP

src-map-name

label contained in the prefix map file associated with map-name. If the prefix map is for IP addresses, the label is that associated with the source IP address. If the prefix map is for protocol/port pairs, the label is that associated with the protocol and source port. See also the description of the --pmap-file switch below and the pmapfilter(3) manual page.

dst-map-name

as src-map-name for the destination IP address or the protocol and destination port.

sval

as src-map-name when no map-name is associated with the prefix map file

dval

as dst-map-name when no map-name is associated with the prefix map file

Finally, the list of built-in fields may be augmented by the run-time loading of PySiLK code or plug-ins written in C (also called shared object files or dynamic libraries), as described by the --python-file and --plugin switches.

--all-fields

Instruct rwcut to print all known fields. This switch may not be combined with the --fields switch. This switch suppresses error messages from the plug-ins.

--plugin=PLUGIN

Augment the list of fields by using run-time loading of the plug-in (shared object) whose path is PLUGIN. The switch may be repeated to load multiple plug-ins. The creation of plug-ins is described in the silk-plugin(3) manual page. When PLUGIN does not contain a slash (/), rwcut will attempt to find a file named PLUGIN in the directories listed in the "FILES" section. If rwcut finds the file, it uses that path. If PLUGIN contains a slash or if rwcut does not find the file, rwcut relies on your operating system's dlopen(3) call to find the file. When the SILK_PLUGIN_DEBUG environment variable is non-empty, rwcut prints status messages to the standard error as it attempts to find and open each of its plug-ins.

--start-rec-num=START_NUM

Begin printing with the START_NUM'th record by skipping the first START_NUM-1 records. The default is 1; that is, to start printing at the first record; START_NUM must be a positive integer. If START_NUM is greater than the number of input records, rwcut only outputs the title. This switch may not be combined with the --tail-recs switch. When using multiple input files, records are treated as a single stream for the purposes of the --start-rec-num, --end-rec-num, --tail-recs, and --num-recs switches. This switch does not affect the records written to the stream specified by --copy-input.

--end-rec-num=END_NUM

Stop printing after the END_NUM'th record. When END_NUM is 0, the default, printing stops once all input records have been printed; that is, END_NUM is effectively infinity. If this value is non-zero, it must not be less than START_NUM. This switch may not be combined with the --tail-recs switch. When using multiple input files, records are treated as a single stream for the purposes of the --start-rec-num, --end-rec-num, --tail-recs, and --num-recs switches. This switch does not affect the records written to the stream specified by --copy-input.

--tail-recs=TAIL_START_NUM

Begin printing once rwcut is TAIL_START_NUM records from end of the input stream, where TAIL_START_NUM is a positive integer. rwcut will print the remaining records in the input stream unless --num-recs is also specified and is less than TAIL_START_NUM. The --tail-recs switch is similar to the --start-rec-num switch except it counts from the end of the input stream. This switch may not be combined with the --start-rec-num and --end-rec-num switches. When using multiple input files, records are treated as a single stream for the purposes of the --start-rec-num, --end-rec-num, --tail-recs, and --num-recs switches. This switch does not affect the records written to the stream specified by --copy-input.

--num-recs=REC_COUNT

Print no more than REC_COUNT records. Specifying a REC_COUNT of 0 will print all records, which is the default. This switch is ignored under the following conditions: When both --start-rec-num and --end-rec-num are specified; when only --end-rec-num is given and END_NUM is less than REC_COUNT; when --tail-recs is specified and TAIL_START_NUM is less than REC_COUNT. When using multiple input files, records are treated as a single stream for the purposes of the --start-rec-num, --end-rec-num, --tail-recs, and --num-recs switches. This switch does not affect the records written to the stream specified by --copy-input.

--dry-run

Causes rwcut to print the column headers and exit. Useful for testing.

--icmp-type-and-code

Unlike TCP or UDP, ICMP messages do not use ports, but instead have types and codes. Specifying this switch will cause rwcut to print, for ICMP records, the message's type and code in the sPort and dPort columns, respectively. Use of this switch has been discouraged since SiLK 0.9.10. As for SiLK 3.8.1, this switch is deprecated and it will be removed in SiLK 4.0; use the iType and iCode fields instead.

--timestamp-format=FORMAT

Specify the format, timezone, and/or precision (representation of fractional seconds) to use when printing timestamps and the duration. When this switch is not specified, the SILK_TIMESTAMP_FORMAT environment variable is checked for a format, timezone, and precision. If it is empty or contains invalid values, timestamps are printed in the default format with microseconds, and the timezone is UTC unless SiLK was compiled with local timezone support. FORMAT is a comma-separated list of a format, a timezone, and/or a precision in any order. The format is one of:

default

Print the timestamps as YYYY/MM/DDThh:mm:ss.sss.

iso

Print the timestamps as YYYY-MM-DD hh:mm:ss.sss.

m/d/y

Print the timestamps as MM/DD/YYYY hh:mm:ss.sss.

epoch

Print the timestamps as the number of seconds since 00:00:00 UTC on 1970-01-01.

The --timestamp-format switch may change the representation of fractional seconds, or precision, of the timestamp and duration fields from their default of microseconds. Note: When using a precision less than that used by SiLK internally, the printed start time and duration may not equal the printed end time. The available precisions are:

no-frac

Truncate the fractional seconds value on the timestamps and on the duration field. Previously this was called no-msec. Since SiLK 3.23.0.

milli

Print the fractional seconds to 3 decimal places. Since SiLK 3.23.0.

micro

Print the fractional seconds to 6 decimal places. Since SiLK 3.23.0.

nano

Print the fractional seconds to 9 decimal places. Since SiLK 3.23.0.

no-msec

Truncate the fractional seconds value on the timestamps and on the duration field. This is an alias for no-frac and is deprecated as of SiLK 3.23.0.

When a timezone is specified, it is used regardless of the default timezone support compiled into SiLK. The timezone is one of:

utc

Use Coordinated Universal Time to print timestamps.

local

Use the TZ environment variable or the local timezone.

--epoch-time

Print timestamps as epoch time (number of seconds since midnight GMT on 1970-01-01). This switch is equivalent to --timestamp-format=epoch, it is deprecated as of SiLK 3.0.0, and it will be removed in the SiLK 4.0 release.

--ip-format=FORMAT

Specify how IP addresses are printed, where FORMAT is a comma-separated list of the arguments described below. When this switch is not specified, the SILK_IP_FORMAT environment variable is checked for a value and that format is used if it is valid. The default FORMAT is canonical according to whether the individual flow record is marked as IPv4 or IPv6. Since SiLK 3.7.0.

canonical

Print IP addresses in the canonical format. For an IPv4 record, use dot-separated decimal (192.0.2.1). For an IPv6 record, use either colon-separated hexadecimal (2001:db8::1) or a mixed IPv4-IPv6 representation for IPv4-mapped IPv6 addresses (the ::ffff:0:0/96 netblock, e.g., ::ffff:192.0.2.1) and IPv4-compatible IPv6 addresses (the ::/96 netblock other than ::/127, e.g., ::192.0.2.1).

no-mixed

Print IP addresses in the canonical format (192.0.2.1 or 2001:db8::1) but do not used the mixed IPv4-IPv6 representations. For example, use ::ffff:c000:201 instead of ::ffff:192.0.2.1. Since SiLK 3.17.0.

decimal

Print IP addresses as integers in decimal format. For example, print 192.0.2.1 and 2001:db8::1 as 3221225985 and 42540766411282592856903984951653826561, respectively.

hexadecimal

Print IP addresses as integers in hexadecimal format. For example, print 192.0.2.1 and 2001:db8::1 as c00000201 and 20010db8000000000000000000000001, respectively.

zero-padded

Make all IP address strings contain the same number of characters by padding numbers with leading zeros. For example, print 192.0.2.1 and 2001:db8::1 as 192.000.002.001 and 2001:0db8:0000:0000:0000:0000:0000:0001, respectively. For IPv6 addresses, this setting implies no-mixed, so that ::ffff:192.0.2.1 is printed as 0000:0000:0000:0000:0000:ffff:c000:0201. As of SiLK 3.17.0, may be combined with any of the above, including decimal and hexadecimal.

The following arguments modify certain IP addresses prior to printing. These arguments may be combined with the above formats.

map-v4

Change IPv4 addresses to IPv4-mapped IPv6 addresses (addresses in the ::ffff:0:0/96 netblock) prior to formatting. Since SiLK 3.17.0.

unmap-v6

Change any IPv4-mapped IPv6 addresses (addresses in the ::ffff:0:0/96 netblock) to IPv4 addresses prior to formatting. Since SiLK 3.17.0.

The following argument is also available:

force-ipv6

Set FORMAT to map-v4,no-mixed.

--integer-ips

Print IP addresses as integers. This switch is equivalent to --ip-format=decimal, it is deprecated as of SiLK 3.7.0, and it will be removed in the SiLK 4.0 release.

--zero-pad-ips

Print IP addresses as fully-expanded, zero-padded values in their canonical form. This switch is equivalent to --ip-format=zero-padded, it is deprecated as of SiLK 3.7.0, and it will be removed in the SiLK 4.0 release.

--integer-sensors

Print the integer ID of the sensor rather than its name.

--integer-tcp-flags

Print the TCP flag fields (flags, initialFlags, sessionFlags) as an integer value. Typically, the characters F,S,R,P,A,U,E,C are used to represent the TCP flags.

--no-titles

Turn off column titles. By default, titles are printed.

--no-columns

Disable fixed-width columnar output.

--column-separator=C

Use specified character between columns and after the final column. When this switch is not specified, the default of '|' is used.

--no-final-delimiter

Do not print the column separator after the final column. Normally a delimiter is printed.

--delimited

--delimited=C

Run as if --no-columns --no-final-delimiter --column-sep=C had been specified. That is, disable fixed-width columnar output; if character C is provided, it is used as the delimiter between columns instead of the default '|'.

--print-filenames

Print to the standard error the names of input files as they are opened.

--copy-input=PATH

Copy all binary SiLK Flow records read as input to the specified file or named pipe. PATH may be stdout or - to write flows to the standard output as long as the --output-path switch is specified to redirect rwcut's textual output to a different location.

--output-path=PATH

Write the textual output to PATH, where PATH is a filename, a named pipe, the keyword stderr to write the output to the standard error, or the keyword stdout or - to write the output to the standard output (and bypass the paging program). If PATH names an existing file, rwcut exits with an error unless the SILK_CLOBBER environment variable is set, in which case PATH is overwritten. If this switch is not given, the output is either sent to the pager or written to the standard output.

--pager=PAGER_PROG

When output is to a terminal, invoke the program PAGER_PROG to view the output one screen full at a time. This switch overrides the SILK_PAGER environment variable, which in turn overrides the PAGER variable. If the --output-path switch is given or if the value of the pager is determined to be the empty string, no paging is performed and all output is written to the terminal.

--ipv6-policy=POLICY

Determine how IPv4 and IPv6 flows are handled when SiLK has been compiled with IPv6 support. When the switch is not provided, the SILK_IPV6_POLICY environment variable is checked for a policy. If it is also unset or contains an invalid policy, the POLICY is mix. When SiLK has not been compiled with IPv6 support, IPv6 flows are always ignored, regardless of the value passed to this switch or in the SILK_IPV6_POLICY variable. The supported values for POLICY are:

ignore

Ignore any flow record marked as IPv6, regardless of the IP addresses it contains. Only records marked as IPv4 will be printed.

asv4

Convert IPv6 flow records that contain addresses in the ::ffff:0:0/96 netblock (that is, IPv4-mapped IPv6 addresses) to IPv4 and ignore all other IPv6 flow records.

mix

Process the input as a mixture of IPv4 and IPv6 flow records.

force

Convert IPv4 flow records to IPv6, mapping the IPv4 addresses into the ::ffff:0:0/96 netblock.

only

Print only flow records that are marked as IPv6 and ignore IPv4 flow records in the input.

--site-config-file=FILENAME

Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, rwcut searches for the site configuration file in the locations specified in the "FILES" section.

--legacy-timestamps

--legacy-timestamps=NUM

When NUM is not specified or is 1, this switch is equivalent to --timestamp-format=m/d/y,no-msec. Otherwise, the switch has no effect. This switch is deprecated as of SiLK 3.0.0, and it will be removed in the SiLK 4.0 release.

--xargs

--xargs=FILENAME

Read the names of the input files from FILENAME or from the standard input if FILENAME is not provided. The input is expected to have one filename per line. rwcut opens each named file in turn and reads records from it as if the filenames had been listed on the command line.

--help

Print the available options and exit. Specifying switches that add new fields or additional switches before --help will allow the output to include descriptions of those fields or switches.

--help-fields

Print the description and alias(es) of each field and exit. Specifying switches that add new fields before --help-fields will allow the output to include descriptions of those fields.

--version

Print the version number and information about how SiLK was configured, then exit the application.

--pmap-file=PATH

--pmap-file=MAPNAME:PATH

Load the prefix map file located at PATH and create fields named src-map-name and dst-map-name where map-name is either the MAPNAME part of the argument or the map-name specified when the file was created (see rwpmapbuild(1)). If no map-name is available, rwcut names the fields sval and dval. Specify PATH as - or stdin to read from the standard input. The switch may be repeated to load multiple prefix map files, but each prefix map must use a unique map-name. The --pmap-file switch(es) must precede the --fields switch. See also pmapfilter(3).

--pmap-column-width=NUM

When printing a label associated with a prefix map, this switch gives the maximum number of characters to use when displaying the textual value of the field.

--python-file=PATH

When the SiLK Python plug-in is used, rwcut reads the Python code from the file PATH to define additional fields for possible output. This file should call register_field() for each field it wishes to define. For details and examples, see the silkpython(3) and pysilk(3) manual pages.

EXAMPLES

In the following examples, the dollar sign ($) represents the shell prompt. The text after the dollar sign represents the command line. Lines have been wrapped for improved readability, and the back slash (\) is used to indicate a wrapped line.

The standard output from rwcut resembles the following (with the text wrapped for readability):

          sIP|            dIP|sPort|dPort|pro|\
  10.30.30.31|    10.70.70.71|   80|36761|  6|\

      packets|     bytes|    flags|\
            7|      3227|FS PA    |\

                  sTime| duration|                  eTime|senso|
2003/01/01T00:00:14.625|    3.959|2003/01/01T00:00:18.584|EDGE1|

The first line of the output is the title line which shows the names of the selected fields; the --no-titles switch will disable the printing of the title line. The second line and onward will contain the printed representation of the records, with one line per record.

A common use of rwcut is to read the output of rwfilter(1). For example, to see representative TCP traffic:

$ rwfilter --start-date=2002/01/19:00 --end-date=2002/01/19:01     \
       --proto=6 --pass=stdout                                     \
  | rwcut

To see only selected fields, use the --fields switch. For example, to print only the protocol for each record in the input file data.rw, use:

$ rwcut --fields=proto  data.rw

The silkpython(3) manual page provides examples that use PySiLK to create and print arbitrary fields for rwcut.

The order of the FIELDS is significant, and fields can be repeated. For example, here is a case where in addition to the default fields of 1-12, you also to prefix each row with an integer form of the destination IP and the start time to make processing by another tool (e.g., a spreadsheet) easier. However, within the default fields of 1-12, you want to see dotted-decimal IP addresses. (The num2dot(1) tool converts the numeric fields in column positions three and four to dotted quad IPs.)

$ rwfilter ... --pass=stdout \
  | rwcut --fields=2,9,1-12 --ip-format=decimal --timestamp-format=epoch \
  | num2dot --ip-field=3,4

Both of the following commands print the title line and the first record in the input stream:

$ rwcut --num-recs=1  data.rw

$ rwcut --end-rec-num=1  data.rw

The following prints all records except the first (plus the title):

$ rwcut --start-rec-num=2  data.rw

These three commands print only the second record:

$ rwcut --no-title --start-rec-num=2 --num-recs=1  data.rw

$ rwcut --no-title --start-rec-num=2 --end-rec-num=2  data.rw

$ rwcut --no-title --end-rec-num=2 --num-recs=1  data.rw

This command prints the title line and the final record in the input stream:

$ rwcut --tail-recs=1  data.rw

This command prints the next to last record in the input stream:

$ rwcut --no-title --tail-recs=2 --num-recs=1  data.rw

Using the sIP and dIP fields can be confusing when the file you are examining contains both incoming and outgoing flow records. To make the output more clear, consider using the int-ext-fields(3) plug-in. The plug-in defines four additional fields representing the external IP address, the external port, the internal IP address, and the internal port. The plug-in requires the user to specify which class/type pairs are incoming and which are outgoing. See its manual page for additional information.

$ rwcut --fields=sip,sport,dip,dport,proto,type \
       --num-rec=8 data.rw
            sIP|sPort|            dIP|dPort|pro|   type|
192.168.111.201|29617|   172.24.2.123|   53| 17|    out|
   172.24.2.123|   53|192.168.111.201|29617| 17|     in|
192.168.111.201|29618|  10.252.217.50|   22|  6|    out|
  10.252.217.50|   22|192.168.111.201|29618|  6|     in|
192.168.204.193|   68|    172.30.2.67|   67| 17|    out|
    172.30.2.67|   67|192.168.204.193|   68| 17|     in|
  10.239.85.193|29897|192.168.228.153|   25|  6|     in|
192.168.228.153|   25|  10.239.85.193|29897|  6|    out|

$ export INCOMING_FLOWTYPES=all/in,all/inweb
$ export OUTGOING_FLOWTYPES=all/out,all/outweb
$ rwcut --plugin=int-ext-fields.so                         \
       --fields=int-ip,int-port,ext-ip,ext-port,proto,type \
       --num-rec=8 data.rw
         int-ip|int-p|         ext-ip|ext-p|pro|   type|
192.168.111.201|29617|   172.24.2.123|   53| 17|    out|
192.168.111.201|29617|   172.24.2.123|   53| 17|     in|
192.168.111.201|29618|  10.252.217.50|   22|  6|    out|
192.168.111.201|29618|  10.252.217.50|   22|  6|     in|
192.168.204.193|   68|    172.30.2.67|   67| 17|    out|
192.168.204.193|   68|    172.30.2.67|   67| 17|     in|
192.168.228.153|   25|  10.239.85.193|29897|  6|     in|
192.168.228.153|   25|  10.239.85.193|29897|  6|    out|

ENVIRONMENT

SILK_IPV6_POLICY

This environment variable is used as the value for --ipv6-policy when that switch is not provided.

SILK_IP_FORMAT

This environment variable is used as the value for --ip-format when that switch is not provided. Since SiLK 3.11.0.

SILK_TIMESTAMP_FORMAT

This environment variable is used as the value for --timestamp-format when that switch is not provided. Since SiLK 3.11.0.

SILK_PAGER

When set to a non-empty string, rwcut automatically invokes this program to display its output a screen at a time. If set to an empty string, rwcut does not automatically page its output.

PAGER

When set and SILK_PAGER is not set, rwcut automatically invokes this program to display its output a screen at a time.

PYTHONPATH

This environment variable is used by Python to locate modules. When --python-file is specified, rwcut must load the Python files that comprise the PySiLK package, such as silk/__init__.py. If this silk/ directory is located outside Python's normal search path (for example, in the SiLK installation tree), it may be necessary to set or modify the PYTHONPATH environment variable to include the parent directory of silk/ so that Python can find the PySiLK module.

SILK_PYTHON_TRACEBACK

When set, Python plug-ins will output traceback information on Python errors to the standard error.

SILK_COUNTRY_CODES

This environment variable allows the user to specify the country code mapping file that rwcut uses when computing the scc and dcc fields. The value may be a complete path or a file relative to the SILK_PATH. See the "FILES" section for standard locations of this file.

SILK_ADDRESS_TYPES

This environment variable allows the user to specify the address type mapping file that rwcut uses when computing the sType and dType fields. The value may be a complete path or a file relative to the SILK_PATH. See the "FILES" section for standard locations of this file.

SILK_CLOBBER

The SiLK tools normally refuse to overwrite existing files. Setting SILK_CLOBBER to a non-empty value removes this restriction.

SILK_CONFIG_FILE

This environment variable is used as the value for the --site-config-file when that switch is not provided.

SILK_DATA_ROOTDIR

This environment variable specifies the root directory of data repository. As described in the "FILES" section, rwcut may use this environment variable when searching for the SiLK site configuration file.

SILK_PATH

This environment variable gives the root of the install tree. When searching for configuration files and plug-ins, rwcut may use this environment variable. See the "FILES" section for details.

TZ

When the argument to the --timestamp-format switch includes local or when a SiLK installation is built to use the local timezone, the value of the TZ environment variable determines the timezone in which rwcut displays timestamps. (If both of those are false, the TZ environment variable is ignored.) If the TZ environment variable is not set, the machine's default timezone is used. Setting TZ to the empty string or 0 causes timestamps to be displayed in UTC. For system information on the TZ variable, see tzset(3) or environ(7). (To determine if SiLK was built with support for the local timezone, check the Timezone support value in the output of rwcut --version.)

SILK_PLUGIN_DEBUG

When set to 1, rwcut prints status messages to the standard error as it attempts to find and open each of its plug-ins. In addition, when an attempt to register a field fails, rwcut prints a message specifying the additional function(s) that must be defined to register the field in rwcut. Be aware that the output can be rather verbose.

FILES

$SILK_ADDRESS_TYPES

$SILK_PATH/share/silk/address_types.pmap

$SILK_PATH/share/address_types.pmap

/usr/share/silk/address_types.pmap

/usr/share/address_types.pmap

Possible locations for the address types mapping file required by the sType and dType fields.

${SILK_CONFIG_FILE}

${SILK_DATA_ROOTDIR}/silk.conf

/data/silk.conf

${SILK_PATH}/share/silk/silk.conf

${SILK_PATH}/share/silk.conf

/usr/share/silk/silk.conf

/usr/share/silk.conf

Possible locations for the SiLK site configuration file which are checked when the --site-config-file switch is not provided.

$SILK_COUNTRY_CODES

$SILK_PATH/share/silk/country_codes.pmap

$SILK_PATH/share/country_codes.pmap

/usr/share/silk/country_codes.pmap

/usr/share/country_codes.pmap

Possible locations for the country code mapping file required by the scc and dcc fields.

${SILK_PATH}/lib64/silk/

${SILK_PATH}/lib64/

${SILK_PATH}/lib/silk/

${SILK_PATH}/lib/

/usr/lib64/silk/

/usr/lib64/

/usr/lib/silk/

/usr/lib/

Directories that rwcut checks when attempting to load a plug-in.

NOTES

Fields sTime+msec, eTime+msec, dur+msec, and their aliases (22, 23, 24) were removed in SiLK 3.23.0. Use fields sTime, eTime, and duration instead.

If you are interested in only a few fields, use the --fields option to reduce the volume of data to be produced. For example, if you are checking to see which internal host got hit with the slammer worm (signature: UDP, destPort 1434, pkt size 404), then the following rwfilter, rwcut combination will be much faster than simply using default values:

$ rwfilter --proto-17 --dport=1434 --bytes-per-packet=404-404      \
  | rwcut --fields=dip,stime

SEE ALSO

rwfilter(1), num2dot(1), rwpmapbuild(1), addrtype(3), ccfilter(3), int-ext-fields(3), pmapfilter(3), silk-plugin(3), silkpython(3), pysilk(3), sensor.conf(5), silk(7), yaf(1), dlopen(3), tzset(3), environ(7)