NAME

rwgroup - Group multiple SiLK records together with a common tag

SYNOPSIS

  rwgroup [--id-field=FIELD] [--delta-field=FIELD]
        [--delta-value=DELTA] [--objective]
        [--rec-threshold=THRESHOLD] [--summarize]

DESCRIPTION

rwgroup reads sorted SiLK Flow records from the standard input, marks records it decides form a group with an identifier in the Next Hop IP field, and prints the binary SiLK Flow records to the standard output. Two SiLK records are defined as being in the same group when the fields specified in the --id-fields switch match exactly and when the fields listed in the --delta-fields match with a value less than or equal to the value given by the --delta-value switch.

Grouped records will be assigned a common id which is stored in the Next Hop IP field on each SiLK Flow record; the --rec-threshold switch may be used to only print groups that contain a certain number of records.

rwgroup requires that the records be sorted on the fields listed in the --id-fields and --delta-fields switches. For example, a call using

  rwgroup --id-field=2 --delta-field=9

should read the output of

  rwsort --field=2,9

otherwise the results are unpredictable.

rwgroup is an interstitial piping command, it is placed between two other SiLK applications.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

At least one value for --id-field or --delta-field must be provided; rwgroup will terminate with an error if no fields are specified.

--id-fields=FIELDS

Numeric identity of the fields that must match exactly among records in a group. FIELDS is a list of the values:

source IP
destination IP
source port
destination port
protocol
packets
bytes
flags
start time
duration
end time
sensor

FIELDS is read as a list, so 2,9,1 is acceptable.

--delta-field=FIELD

Numeric identity of a single field that can differ by a delta value among the SiLK records in a group. The FIELD identifiers are the same as for --id-fields. The switch takes a single argument, multiple delta fields cannot be specified.

--delta-value=VALUE

Acceptable difference between the values of the --delta-field; if two consecutive records have values less than or equal to VALUE, then the records are considered members of the same group. The --delta-value switch is required when the --delta-field switch is provided.

--objective

This flag changes the behavior of the --delta-value switch. With --objective set, a record is considered part of a group if the value of its --delta-field is within the delta of the first record in the group.

--rec-threshold=THRESHOLD

Minimum number of SiLK records a group must contain before it will be written to the output stream. The default is 1; i.e., write all records.

--summarize

This flag causes rwgroup to print a single record for each group. Normally, all the records in each group having --rec-threshold members is printed.

LIMITATIONS

rwgroup requires sorted data. The application works by comparing records in the order that the records are received (similar to the UNIX uniq(1) command), odd orders will produce odd groupings.

EXAMPLES

As a rule of thumb, the --id-fields and --delta-field parameters should match rwsort's call, with --delta-field being the last parameter. A call to group all web traffic by queries from the same addresses (field=2) within 10 seconds (field=9) of the first query from that address will be:

  rwfilter --proto=6 --dport=80 --pass=stdout | \
        rwsort --field=2,9 | \
        rwgroup --id-field=2 --delta-field=9 --delta-value=10
        --objective

SEE ALSO

rwfilter(1), rwsort(1), rwmatch(1)

BUGS

When used in an IPv6 environment, rwgroup will attempt to convert any IPv6 addresses to IPv4. Records that can be converted will be processed, all other records will be silently ignored.