NAME

rwbagcat - Output a binary Bag as text.

SYNOPSIS

  rwbagcat [--stats[=OUTFILE]] [--tree-stats[=OUTFILE]]
        [ --network-structure[=[T][A][B][C][X][H][S]] |
          --bin-ips[=SCALE]]
        [--mincounter=VALUE] [--maxcounter=VALUE]
        [--minkey=VALUE] [--maxkey=VALUE] [--zero-counts]
        [--integer-keys | --zero-pad-ips] [--output-path=OUTPUTFILE]
        [--no-columns] [--column-separator=C] [--no-final-delimiter]
        [{--delimited | --delimited=C}] [--pager=PAGER_PROG]
        [BAGFILE...]

DESCRIPTION

rwbagcat reads a binary Bag, converts it to text, and outputs it to the standard output or the specified file. It can also print various statistics and summary information about the Bag.

rwbagcat reads the BAGFILEs specified on the command line; if no BAGFILE arguments are given, rwbagcat attempts to read the Bag from the standard input. BAGFILE may also explicitly be the keyword stdin to allow rwbagcat to combine files and piped input. If any input does not contain a Bag, rwbagcat prints an error to stderr and exits abnormally.

When multiple BAGFILEs are specified, each is handled individually; to process the combination of the BAGFILEs, invoke rwbagcat on the output from rwbagtool(1).

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--network-structure

--network-structure={T | A | B | C | X | H}...[S]

Print the network structure around the keys.

T

total sum of all keys

A

class A (/8) network header and subtotal

B

class B (/16) network header and subtotal

C

class C (/24) network header and subtotal

X

(/27) network header and subtotal

H

individual IP addresses and counter

S

count of /8,/16,/24,/27s in each smaller subnet

If no paramter is specified to the --network-structure switch, the default is TS. Any combination of T,A,B,C,X,H may be specified; the value S by itself is illegal.

--bin-ips

--bin-ips=SCALE

Invert the bag and count the total number of unique IP addresses for a given value of the volume bin. For example, turn a Bag {sip:flow} into {flow:count(sip)}. SCALE is a string containing the value linear, binary, or decimal.

The default behavior is linear: Each distinct counter gets its own bin. Any counter in the input Bag file that is larger than the maximum possible key will be attributed to the maximum key; to prevent this, specify --maxcounter=4294967295.
binary creates a bag of {log2(flow):count(sip)}. Bin n contains counts in the range [ 2^n, 2^(n+1) ).
decimal creates one hundred bins for each counter in the range [1,100), and one hundred bins for each counter in the range [100,1000), each counter in the range [1000,10000), etc. Counters are logarithmically distributed among the bins.

--stats

--stats=OUTFILE

Print out breakdown of the network hosts seen, and print out general statistics about the keys and counters.

count of unique keys
sum of all the counters
minimum key
maximum key
minimum counter
maximum counter
mean of counters
variance of counters
standard deviation of counters
skew of counters
kurtosis of counters

OUTFILE is a filename, named pipe, or one of the keywords stdout or stderr. Defaults to printing on stderr unless output is being paged, in which case output is to stdout.

--tree-stats

--tree-stats=OUTFILE

Print out metadata about how the bag is performing:

count of nodes allocated
total bytes allocated for nodes
count of leaves allocated
total bytes allocated for leaves
count of keys entered
density of data

OUTFILE is a filename, named pipe, or one of the keywords stdout or stderr. Defaults to printing on stdout.

--mincounter=VALUE

Only output records whose minimum counter value is VALUE or higher. The valid range of VALUE is 1 to 18446744073709551615. The default is to print all records with non-zero counter; use --zero-counts to show records whose counter is 0.

--maxcounter=VALUE

Only output records whose maximum counter value is VALUE or lower. The valid range of VALUE is 1 to 18446744073709551615, with the default being the maximum counter value.

--minkey=VALUE

Only output records whose minimum key value is VALUE or higher. The valid range is of VALUE 0 to 4294967295, or 0.0.0.0 to 255.255.255.255. Default is 0 (for port or protocol) or 0.0.0.0 (for IP address). Accepts dotted decimal or integer notation.

--maxkey=VALUE

Only output records whose maximum key value is VALUE or lower. The valid range of VALUE is 0 to 4294967295, or 0.0.0.0 to 255.255.255.255. Default is all ports or protocols, or the maximum IP address 255.255.255.255. Accepts dotted decimal or integer notation.

--zero-counts

Print keys whose counter is zero. Normally, keys with a counter of zero are suppressed since all keys have a default counter of zero. In order to use this flag, --minkey and --maxkey must be specified. When this switch is specified, any counter limit explicitly set by the --maxcounter switch will still be applied.

--output-path=OUTPUTFILE

Redirect output of the --network-structure or --bin-ips options to OUTPUTFILE. OUTPUTFILE is a filename, named pipe, or the keyword stdout.

--zero-pad-ips

Pad IP address octets with zeros so that every octet is three characters wide.

--integer-keys

Print the keys as integers. This flag should be used if the bag is a port or protocol bag.

--no-columns

Disable fixed-width columnar output.

--column-separator=C

Use specified character between columns and after the final column. When this switch is not specified, the default of '|' is used.

--no-final-delimiter

Do not print the column separator after the final column. Normally a delimiter is printed. When the network summary is requested (--network-structure=S), the separator is always printed before the summary column and never that column.

--delimited

--delimited=C

Run as if --no-columns --no-final-delimiter --column-sep=C had been specified. That is, disable fixed-width columnar output; if character C is provided, it is used as the delimiter between columns instead of the default '|'.

--pager=PAGER_PROG

When output is to a terminal, invoke the program PAGER_PROG to view the output one screen full at a time. This switch overrides the SILK_PAGER environment variable, which in turn overrides the PAGER variable. If the value of the pager is determined to be the empty string, no paging will be performed and all output will be printed to the terminal.

EXAMPLES

To print the bag:

  $ rwbagcat mybag.bag 
       172.23.1.1|              5|
       172.23.1.2|            231|
       172.23.1.3|              9|
       172.23.1.4|             19|
    192.168.0.100|              1|
    192.168.0.101|              1|
    192.168.0.160|             15|
   192.168.20.161|              1|
   192.168.20.162|              5|
   192.168.20.163|              5|

To print it with full network:

  $ rwbagcat --network-structure=TABCHX mybag.bag
            172.23.1.1      |              5|
            172.23.1.2      |            231|
            172.23.1.3      |              9|
            172.23.1.4      |             19|
          172.23.1.0/27     |            264|
        172.23.1.0/24       |            264|
      172.23.0.0/16         |            264|
    172.0.0.0/8             |            264|
            192.168.0.100   |              1|
            192.168.0.101   |              1|
          192.168.0.96/27   |              2|
            192.168.0.160   |             15|
          192.168.0.160/27  |             15|
        192.168.0.0/24      |             17|
            192.168.20.161  |              1|
            192.168.20.162  |              5|
            192.168.20.163  |              5|
          192.168.20.160/27 |             11|
        192.168.20.0/24     |             11|
      192.168.0.0/16        |             28|
    192.0.0.0/8             |             28|
  TOTAL                     |            292|

Or an abbreviated network structure by class A and C only, including summary information:

  $ rwbagcat --network-structure=ACS mybag.bag 
      172.23.1.0/24     |            264| 4 hosts in 1 /27
  172.0.0.0/8           |            264| 4 hosts in 1 /16, 1 /24, and 1 /27
      192.168.0.0/24    |             17| 3 hosts in 2 /27s
      192.168.20.0/24   |             11| 3 hosts in 1 /27
  192.0.0.0/8           |             28| 6 hosts in 1 /16, 2 /24s, and 3 /27s

To bin by number of unique IP addresses by volume:

  $ rwbagcat --bin-ips mybag.bag
                1|              3|
                5|              3|
                9|              1|
               15|              1|
               19|              1|
              231|              1|

This means there were 3 source hosts in the bag that had a single flow; 3 hosts that had 5 flows; and one host each that had 9, 15, 19, and 231 flows.

For a log2 breakdown of the counts:

  $ rwbagcat --bin-ips=binary mybag.bag
     2^0 to 2^1-1|              3|
     2^2 to 2^3-1|              3|
     2^3 to 2^4-1|              2|
     2^4 to 2^5-1|              1|
     2^7 to 2^8-1|              1|

Statistics:

  $ rwbagcat --stats mybag.bag

  Statistics
                keys:  10
     sum of counters:  292
         minimum key:  172.23.1.1
         maximum key:  192.168.20.163
       minimum count:  1
       maximum count:  231
                mean:  29.2
            variance:  5064
  standard deviation:  71.16
                skew:  2.246
            kurtosis:  8.1

  $ rwbagcat --tree-stats mybag.bag 
     nodes allocated:  5 (10240 bytes)
    leaves allocated:  4 (1024 bytes)
       keys inserted:  10 (10 unique)
     counter density:  7.81%

ENVIRONMENT

SILK_PAGER

When set to a non-empty string, rwbagcat automatically invokes this program to display its output a screen at a time. If set to an empty string, rwbagcat does not automatically page its output.

PAGER

When set and SILK_PAGER is not set, rwbagcat automatically invokes this program to display its output a screen at a time.

SEE ALSO

rwbag(1), rwbagbuild(1), rwbagtool(1)