NAME

rwbagbuild - Create a binary Bag from non-flow data.

SYNOPSIS

  rwbagbuild { --set-input=SETFILE | --bag-input=TEXTFILE }
        [--delimiter=C] [--default-count=DEFAULTCOUNT]
        [--note-add=TEXT] [--note-file-add=FILE]
        [--compression-method=COMP_METHOD] [--output-path=OUTPUTFILE]

DESCRIPTION

rwbagbuild builds a binary Bag file from an IPset file or from textual input.

When creating a Bag from an IPset, the value associated with each IP address is the value given by the --default-count switch, or 1 if the switch isn't provided.

The textual input read from the argument to the --bag-input switch is processed a line at a time. Comments begin with a '#'-character and continue to the end of the line; they are stripped from each line. Any line that is blank or contains only whitespace is ignored. All other lines must contain a valid key or key-count pair; whitespace around the key and count is ignored.

If the delimiter character (specified by the --delimiter switch and having pipe ('|') as its default) is not present, the line must contain only an IP address or an integer key. If the delimiter is present, the line must contain an IP address or integer key before the delimiter and an integer count after the delimiter. These lines may have a second delimiter after the integer count; the delimiter and any text to the right of it are ignored.

When the --default-count switch is specified, its value will used as the count for each key, and the count value parsed from each line, if any, is ignored. Otherwise, the parsed count is used, or 1 is used as the count if no delimiter was present.

For each key-count pair, the key will be inserted into Bag with its count or, if the key is already present in the Bag, its total count will be incremented by the count from this line.

The IP address or integer key must be expresed in one of these formats:

Dotted decimal---all 4 octets are required:

  10.1.2.4

An unsigned 32-bit integer:

  167838212

Either of the above with a CIDR designation---for dotted decimal all four octets are still required:

  10.1.2.4/31
  167838212/31

SiLK wildcard notation: Four octets separated by periods where each octet may be a single number, a range of numbers, e.g., 1-10, a comma separated list of numbers and ranges, or the character 'x' used to represent all values in an octet, that is 0-255:

  10.x.1-2.4,5

If an IP address or count cannot be parsed, or if a line contains a delimiter character but no count, rwbagbuild prints an error and exits.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

The following two switches control the type of input; one and only one must be provided:

--set-input=SETFILE

Create a Bag from an IPset. SETFILE is a filename, a named pipe, or the keyword stdin. Counts have a volume of 1 unless overridden with --default-count.

--bag-input=TEXTFILE

Create a Bag from a delimited text file. TEXTFILE is a filename, a named pipe, or the keyword stdin. See the DESCRIPTION section for the syntax of the TEXTFILE.

--delimiter=C

The delimiter to expect between each key-count pair of the TEXTFILE read by the --bag-input switch. The delimiter is ignored if the --set-input switch is specified. Since '#' is used to denote comments and newline is used to used to denote records, neither is a valid delimiter character.

--default-count=DEFAULTCOUNT

Override the counts of all values in the input bag or set with the value of DEFAULTCOUNT. DEFAULTCOUNT must be a positive integer.

--note-add=TEXT

Add the specified TEXT to the header of the output file as an annotation. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool.

--note-file-add=FILENAME

Open FILENAME and add the contents of that file to the header of the output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to ensure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation.

--compression-method=COMP_METHOD

Set the compression method of the output to COMP_METHOD. Some SiLK tools can use an external library to compress their binary output. The list of available compression methods and the default method are set when SiLK is compiled (the --help and --version switches print the available and default compression methods) and depend on which supported libraries are found. SiLK can support:

none

Do not compress the output using an external library

zlib

Use the zlib(3) library for compressing the output

lzo1x

Use the lzo1x algorithm from the LZO real time compression library for compression

best

Use whichever available method gives the best compression in general, though not necessarily the best for this particular output.

--output-path=OUTPUTFILE

Redirect output to OUTPUTFILE. OUTPUTFILE is a filename, named pipe, or the keyword stdout.

EXAMPLES

Assume the file mybag.txt contains the following (ignore leading whitespace and every line ends with a newline):

  192.168.0.1|5
  192.168.0.2|500
  192.168.0.3|3
  192.168.0.4|14
  192.168.0.5|5

To build a bag with it:

  rwbagbuild --bag-input=mybag.txt > mybag.bag

Or given the IP set myset.set, create a bag where every entry in the set has a count of 3:

  rwbagbuild --set-input=myset.set --default-count=3 \
        --out=mybag2.bag

SEE ALSO

rwbag(1), rwbagcat(1), rwbagtool(1), rwfileinfo(1), rwset(1)