Monitoring for Large-Scale Networks

SiLK

rwsiteinfo

NAME

rwsiteinfo - Print information from the silk.conf site configuration file

SYNOPSIS

  rwsiteinfo --fields=FIELD[,FIELD...]
        { [--classes=CLASS[,CLASS...]] [--types=TYPE[,TYPE...]]
          | [--flowtypes=CLASS/TYPE[,CLASS/TYPE...]] }
        [--sensors=SENSOR[,SENSOR...]] [--groups=GROUP[,GROUP...]]
        [--data-rootdir=ROOT_DIRECTORY] [--site-config-file=FILENAME]
        [--timestamp-format=FORMAT] [--no-titles]
        [--no-columns] [--column-separator=CHAR]
        [--no-final-delimiter] [{--delimited | --delimited=CHAR}]
        [--list-delimiter=CHAR] [--output-path=PATH]
        [--pager=PAGER_PROG]

  rwsiteinfo --help

  rwsiteinfo --help-fields

  rwsiteinfo --version

DESCRIPTION

rwsiteinfo is a utility to print selected information about the classes, types, flowtypes, sensors, and groups (collections of sensors) that are defined in the silk.conf(5) site configuration file. The --fields switch is required, and its argument is a comma-separated list of field names selecting the fields to be printed. The output from rwsiteinfo consists of multiple columns and rows, where each column contains one of the FIELDs and where each row has a unique value for one of the FIELDs. rwsiteinfo prints rows until all possible combinations of fields is exhausted. By default, the information is printed in a columnar, bar-delimited (|-delimited) format.

As of SiLK 3.11.0, rwsiteinfo can visit the files in the data repository to report the date of the earliest (oldest) file in the repository, the date of the latest (most recent) file in the repository, and the number of files in the repository. These values are reported individually for each row in the output. Note: If your data repository is large, scanning it may take a long time.

The --classes, --types, --flowtypes, --sensors, and --groups switches allow the user to limit the amount of information printed. (These switches operate similarly to their namesakes on rwfilter(1) and rwfglob(1).) If none of these switches are given, rwsiteinfo prints information for all values defined in the silk.conf file. If one or more of these switches is specified, rwsiteinfo limits its output to the specified values. The limit is applied even if that field is not listed in --fields. For example, specifying --sensors=S1 --fields=class tells rwsiteinfo to limit the output to classes that have sensor S1 as a member. To print information about the default class or the default types within a class, use the at-sign (@) as the name of the class or type, respectively. The --flowtypes switch must be used independently of the --classes and --types switches.

SiLK 3.21.0 added support for sensor groups to rwsiteinfo. Sensor groups are defined in the silk.conf(5) file within a group block. The --sensors switch accepts sensor group names to limit the output to sensors defined with the named groups. The --groups switch may be used to limit the output to specified sensor groups. Using the same argument to --groups or --sensors produces the same result unless --fields includes group and the silk.conf file has sensors as members of multiple groups.

As of SiLK 3.20, the --classes, --types, --flowtypes, --sensors, and --groups switches accept a value in the form @PATH, where PATH names a file (that is, an @ character followed by a file or path name). The format of this file is described in the "Read Argument Values from a File" section below.

As stated above, rwsiteinfo prints unique rows given a list of FIELDs. As an example, suppose the user entered the command rwsiteinfo --fields=class,type,sensor. rwsiteinfo will print a row containing the first class defined in the silk.conf file, the first type defined for that class, and the first sensor name defined for that class/type pair. On the next row, the class and type will be the same and the second sensor name will be printed. Once all sensors have been printed, rwsiteinfo repeats the process for the second type defined for the first class, and so on. Once all information for the first class has been printed, the process would repeat for the next class, until all classes have been printed.

The order of the FIELDs determines how rwsiteinfo iterates through the possible values. The last FIELD will change most rapidly, and the first field will change most slowly. Two invocations of rwsiteinfo where the first specifies --fields=class,sensor and the second specifies --fields=sensor,class produce the same number of rows, and each invocation has an outer and inner iterator. In the first invocation, the outer iterator is over the classes, and the inner iterator is over each sensor defined in that class. In the second invocation, the outer iterator is over the sensors, and the inner is over the classes to which that sensor belongs.

In general, the output will contain some combination of class, type, flowtype, and sensor. For flowtype and sensor, the numeric ID may be printed instead of the name. For class and type, the default values may be printed or they may be identified by a symbol. Most field names support a FIELD:list variant that puts all possible values for that field into a single column. See the description of the --fields switch below for details.

Read Argument Values from a File

As of SiLK 3.20, the --classes, --types, --flowtypes, --sensors, and --group switches accept a value in the form @PATH, where @ is the "at" character (ASCII 0x40) and PATH names a file or a path to a file. For example, the following reads the name of types from the file t.txt and uses the sensors S3, S7, and the names and/or IDs read from /tmp/sensor.txt:

 rwsiteinfo --type=@t.txt --sensors=S3,@/tmp/sensor.txt,S7

Multiple @PATH values are allowed within a single argument. If the name of the file is -, the names are read from the standard input.

The file must be a text file. Blank lines are ignored as are comments, which begin with the # character and continue to the end of the line. Whitespace at the beginning and end of a line is ignored as is whitespace that surrounds commas; all other whitespace within a line is significant.

A file may contain a value on each line and/or multiple values on a line separated by commas and optional whitespace. For example:

 # Sensor 4
       S4
 # The first sensors
 S0, S1,S2
 S3     # Sensor 3

An attempt to use an @PATH directive in a file is an error.

When rwsiteinfo is parsing the name of a file, it converts the sequences @, and @@ to , and @, respectively. For example, --class=@cl@@ss.txt@,v reads the class from the file cl@ss.txt,v. It is an error if any other character follows an embedded @ (--flowtypes=@f@il contains @i) or if a single @ occurs at the end of the name (--sensor=@errat@).

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--fields=FIELD[,FIELD...]

Specify the fields to print as a comma-separated list of names. The names are case-insensitive. The fields will be displayed in the order the names are specified. The --fields switch is required, and rwsiteinfo will fail when it is not provided.

The list of possible field names is:

class: the class name, e.g., all
type: the type name, e.g., inweb
flowtype: the flowtype name, e.g., iw. The flowtype name is a combination of the class name and type name, and it is used to name files in the SiLK data repository.
id-flowtype: the integer identifier for the flowtype, e.g., 2
sensor: the sensor name, e.g., S3
id-sensor: the integer identifier for the sensor, e.g., 3
describe-sensor: the sensor description, when present
group: the sensor group name Since SiLK 3.21.0.
default-class: the default class name
default-type: the default type name
mark-defaults: a two-character wide column that contains a plus '+' on a row that contains the default class and an asterisk '*' on a row that contains a default type
repo-start-date: the earliest date for a file in the repository that matches the values in this row or empty when no files match Since SiLK 3.11.0
repo-end-date: the latest date for a file in the repository that matches the values in this row or empty when no files match Since SiLK 3.11.0
repo-file-count: the number of files in the repository that match the values in this row or zero when no files match Since SiLK 3.11.0
class:list: instead of printing class names on separate rows, join all the classes in a single row separated using the list-delimiter
type:list: instead of printing type names on separate rows, join all the types in a single row separated using the list-delimiter
flowtype:list: instead of printing flowtype names on separate rows, join all the flowtypes in a single row separated using the list-delimiter
id-flowtype:list: instead of printing flowtype identifiers on separate rows, join all the flowtype identifiers in a single row separated using the list-delimiter
sensor:list: instead of printing sensor names on separate rows, join all the sensors in a single row separated using the list-delimiter
id-sensor:list: instead of printing sensor identifiers on separate rows, join all the sensor identifiers in a single row separated using the list-delimiter
group:list: instead of printing sensor group names on separate rows, join all the group names in a single row separated using the list-delimiter Since SiLK 3.21.0.
default-class:list: equivalent to default-class, but provided for consistency
default-type:list: instead of printing the default type names on separate rows, join all the default type names in a single row separated using the list-delimiter

--classes=CLASS[,CLASS,@PATH...]

Restrict the output using the class(es) named in the comma-separated list. The default class may be specified by using an at-sign (@) as the name of a class. As of SiLK 3.20.0, if an argument is "@PATH", rwsiteinfo attempts to open the file PATH and read the names of classes from it; see "Read Argument Values from a File" for details.

--types=TYPE[,TYPE,@PATH...]

Restrict the output using the type(s) named in the comma-separated list. The default types for a class may be specified by using an at-sign (@) as the name of a type. An argument of "@PATH" causes rwsiteinfo to read type names from the file PATH; see "Read Argument Values from a File".

--flowtypes=CLASS/TYPE[,CLASS/TYPE,@PATH...]

Restrict the output using the class/type pairs named in the comma-separated list, where the class name and type name are separated by a slash (/). The keyword all may be used for the CLASS and/or TYPE to select all classes and/or types. As of SiLK 3.20.0, the arguments may also include "@PATH" which causes rwsiteinfo to open the file PATH and read the class/type pairs from it; see "Read Argument Values from a File".

--sensors=SENSOR[,SENSOR,SENSOR-GROUP,@PATH...]

Restrict the output to the sensors(s) identified in the comma-separated list of sensor names, sensor IDs (integers), ranges of sensor IDs, sensors added to the specified sensor group names, and names and/or IDs of sensors and sensor groups read from the file PATH. Using a sensor group name adds all sensors defined within that group. The ability to use sensor group names was added in SiLK 3.21.0.

--groups=SENSOR-GROUP[,SENSOR-GROUP,@PATH...]

Restrict the output using the sensor groups named in the comma-separated list. Naming a group in either --groups or --sensors produces the same result unless --fields includes group and the silk.conf puts sensors in multiple groups. If an argument is "@PATH", rwsiteinfo attempts to open the file PATH and read the names of sensor groups from it; see "Read Argument Values from a File" for details. Since SiLK-3.21.0.

--data-rootdir=ROOT_DIRECTORY

Use ROOT_DIRECTORY as the root of the data repository, which overrides the location given in the SILK_DATA_ROOTDIR environment variable, which in turn overrides the location that was compiled into rwsiteinfo (/data). This directory is one of the locations where rwsiteinfo attempts to find the silk.conf file, and it is the repository that is scanned when the repo-start-date, repo-end-date, or repo-file-count field is specified.

--site-config-file=FILENAME

Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, rwsiteinfo searches for the site configuration file in the locations specified in the "FILES" section.

--timestamp-format=FORMAT

Specify the format and/or timezone to use when printing timestamps. When this switch is not specified, the SILK_TIMESTAMP_FORMAT environment variable is checked for a default format and/or timezone. If it is empty or contains invalid values, timestamps are printed in the default format, and the timezone is UTC unless SiLK was compiled with local timezone support. FORMAT is a comma-separated list of a format and/or a timezone. The format is one of:

default: Print the timestamps as YYYY/MM/DDThh:mm:ss.
iso: Print the timestamps as YYYY-MM-DD hh:mm:ss.
m/d/y: Print the timestamps as MM/DD/YYYY hh:mm:ss.
epoch: Print the timestamps as the number of seconds since 00:00:00 UTC on 1970-01-01.

When a timezone is specified, it is used regardless of the default timezone support compiled into SiLK. The timezone is one of:

utc: Use Coordinated Universal Time to print timestamps.
local: Use the TZ environment variable or the local timezone.

--no-titles

Turn off column titles. By default, titles are printed.

--no-columns

Disable fixed-width columnar output.

--column-separator=C

Use specified character between columns and after the final column. When this switch is not specified, the default of | is used.

--no-final-delimiter

Do not print the column separator after the final column. Normally a delimiter is printed.

--delimited

--delimited=C

Run as if --no-columns --no-final-delimiter --column-sep=C had been specified. That is, disable fixed-width columnar output; if character C is provided, it is used as the delimiter between columns instead of the default |.

--list-delimiter=C

Specify the character to use between items that comprise a FIELD:list column. The default list delimiter is comma ,.

--output-path=PATH

Write the textual output to PATH, where PATH is a filename, a named pipe, the keyword stderr to write the output to the standard error, or the keyword stdout or - to write the output to the standard output (and bypass the paging program). If PATH names an existing file, rwsiteinfo exits with an error unless the SILK_CLOBBER environment variable is set, in which case PATH is overwritten. If this switch is not given, the output is either sent to the pager or written to the standard output. Since SiLK 3.15.0.

--pager=PAGER_PROG

When output is to a terminal, invoke the program PAGER_PROG to view the output one screen full at a time. This switch overrides the SILK_PAGER environment variable, which in turn overrides the PAGER variable. If the --output-path switch is given or if the value of the pager is determined to be the empty string, no paging is performed and all output is written to the terminal.

--help

Print the available options and exit. Options that add fields can be specified before --help so that the new options appear in the output.

--help-fields

Print a description for each field and exit.

--version

Print the version number and information about how SiLK was configured, then exit the application.

EXAMPLES

In the following examples, the dollar sign ($) represents the shell prompt. The text after the dollar sign represents the command line. Lines have been wrapped for improved readability, and the back slash (\) is used to indicate a wrapped line.

The output from these examples is based on the sample silk.conf as distributed for the twoway site (c.f. packlogic-twoway(3)).

Displaying the sensors with various options

This displays all known sensors using the default display options:

 $ rwsiteinfo --fields=sensor
 Sensor|
     S0|
     S1|
     S2|
     S3|
     S4|
     S5|
     S6|
     S7|
     S8|
     S9|
    S10|
    S11|
    S12|
    S13|
    S14|

The following prints all known sensor names, one name per line:

 $ rwsiteinfo --fields=sensor --no-titles --delimited
 S0
 S1
 S2
 S3
 S4
 S5
 S6
 S7
 S8
 S9
 S10
 S11
 S12
 S13
 S14

The following prints all known sensor names on a single line with the names separated by comma:

 $ rwsiteinfo --fields=sensor:list --no-titles --delimited
 S0,S1,S2,S3,S4,S5,S6,S7,S8,S9,S10,S11,S12,S13,S14

This changes the output from the previous example to use a space as the separator:

 $ rwsiteinfo --fields=sensor:list --no-titles --delimited \
        --list-delimiter=' '
 S0 S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14

The following prints the sensor names for the default class on a single line (since there is a single class, the output is the same as that shown above):

 $ rwsiteinfo --fields=sensor:list --class=@ --no-titles --delimited
 S0,S1,S2,S3,S4,S5,S6,S7,S8,S9,S10,S11,S12,S13,S14

This shows the numeric sensor IDs:

 $ rwsiteinfo --fields=id-sensor:list
                     Sensor-ID:list|
 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14|

The following prints four columns: (1) the sensor identifier, (2) the sensor name, (3) the list of classes for that sensor, and (4) a description of the sensor. This output mimics the output of the deprecated mapsid(1) tool.

 $ rwsiteinfo --fields=id-sensor,sensor,class:list,describe-sensor
 Sensor-ID|Sensor|Class:list|                Sensor-Description|
         0|    S0|       all|         Description for sensor S0|
         1|    S1|       all|                                  |
         2|    S2|       all|Optional description for sensor S2|
         3|    S3|       all|                                  |
         4|    S4|       all|                                  |
         5|    S5|       all|                                  |
         6|    S6|       all|                                  |
         7|    S7|       all|                                  |
         8|    S8|       all|                                  |
         9|    S9|       all|                                  |
        10|   S10|       all|                                  |
        11|   S11|       all|                                  |
        12|   S12|       all|                                  |
        13|   S13|       all|                                  |
        14|   S14|       all|                                  |

Displaying classes and types

This prints three columns: the first contains the class, the second contains the type, and the third uses a + to mark rows for the default class and a * to mark rows for a default type.

 $ rwsiteinfo --fields=class,type,mark-default
 Class|   Type|Defaults|
   all|     in|      +*|
   all|    out|      + |
   all|  inweb|      +*|
   all| outweb|      + |
   all| innull|      + |
   all|outnull|      + |
   all|int2int|      + |
   all|ext2ext|      + |
   all| inicmp|      +*|
   all|outicmp|      + |
   all|  other|      + |

The following prints two columns, the first containing a class name and the second the list of default types for that class:

 $ rwsiteinfo --fields=class,default-type:list
 Class|Default-Type:list|
   all|  in,inweb,inicmp|

The following prints the default types. (The output contains the default type for each class, but twoway site has a single class.)

 $ rwsiteinfo --fields=default-type --no-titles --delimited
 in
 inweb
 inicmp

This does the same (by limiting the output the default types).

 $ rwsiteinfo --fields=type --types=@ --no-titles --delimited
 in
 inweb
 inicmp

The following prints the class, the sensor, and the type. The number of rows of output (excluding the title) is the product of the number of classes, number of types, and number of sensors.

 $ rwsiteinfo --fields=class,sensor,type
 Class|Sensor|   Type|
   all|    S0|     in|
   all|    S0|    out|
   all|    S0|  inweb|
   all|    S0| outweb|
   all|    S0| innull|
   all|    S0|outnull|
   all|    S0|int2int|
   all|    S0|ext2ext|
   all|    S0| inicmp|
   all|    S0|outicmp|
   all|    S0|  other|
   all|    S1|     in|
   all|    S1|    out|
 ...
   all|   S14|outicmp|
   all|   S14|  other|

Displaying sensor groups

The --group switch was added in SiLK 3.21.0 and limits the output to particular sensor groups. In addition, the --sensors switch accepts sensor group names. Given this silk.conf file:

 sensor 1 S1
 sensor 2 S2
 sensor 3 S3
 group G12
   sensors S1 S2
 end group
 group G13
   sensors S1 S3
 end group
 ...

Using --group=G12 limits the output to that group:

 $ rwsiteinfo --fields=group,sensor --group=G12
 Group|Sensor|
   G12|    S1|
   G12|    S2|

If --sensors=G12 is used, rwsiteinfo limits the output to the sensors that group contains (S1 and S2). The result includes group G13 since that group contains sensor S1:

 $ rwsiteinfo --fields=group,sensor --sensor=G12
 Group|Sensor|
   G12|    S1|
   G12|    S2|
   G13|    S1|

This occurs when sensors appear in multiple groups and group is one of the output --fields.

Displaying information about the flow data repository

The repo-start-date, repo-end-date, and repo-file-count fields print the range of available dates for the files in the repository. The following shows information about files in the repository for the repository as a whole:

 $ rwsiteinfo --fields=repo-start-date,repo-end-date,repo-file-count
          Start-Date|           End-Date|File-Count|
 2009/02/12T00:00:00|2009/02/14T23:00:00|      2880|

This breaks down the file information per type:

 $ rwsiteinfo --fields=type,repo-start-date,repo-end-date,repo-file-count
    Type|         Start-Date|           End-Date|File-Count|
      in|2009/02/12T00:00:00|2009/02/14T23:00:00|       720|
     out|2009/02/12T00:00:00|2009/02/14T23:00:00|       720|
   inweb|2009/02/12T00:00:00|2009/02/14T23:00:00|       720|
  outweb|2009/02/12T00:00:00|2009/02/14T23:00:00|       720|
  innull|                   |                   |         0|
 outnull|                   |                   |         0|
 int2int|                   |                   |         0|
 ext2ext|                   |                   |         0|
  inicmp|                   |                   |         0|
 outicmp|                   |                   |         0|
   other|                   |                   |         0|

This shows the information for each sensor:

 $ rwsiteinfo --fields=type,repo-start-date,repo-end-date,repo-file-count
 Sensor|         Start-Date|           End-Date|File-Count|
     S0|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
     S1|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
     S2|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
     S3|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
     S4|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
     S5|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
     S6|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
     S7|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
     S8|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
     S9|2009/02/12T00:00:00|2009/02/14T23:00:00|       288|
    S10|                   |                   |         0|
    S11|                   |                   |         0|
    S12|                   |                   |         0|
    S13|                   |                   |         0|
    S14|                   |                   |         0|

Restricting the output by reading names from a file

Suppose the file sensors.txt exits in the current directory and has the content shown here:

 $ cat sensors.txt
 # Sensor 4
       S4
 # The first sensors
 S0,S1,S2
 S3     # Sensor 3

When using SiLK 3.20.0 or later, the file can be used to restrict which sensors rwsiteinfo displays:

 $ rwsiteinfo --fields=class,sensor  --sensor=@/tmp/foobart
 Class|Sensor|
   all|    S0|
   all|    S1|
   all|    S2|
   all|    S3|
   all|    S4|

An example that shows use of the @ escapes when parsing the name of a file (file weird,n@me becomes weird@,n@@me), and the error generated when attempting to use @PATH within a file:

 $ cat weird,n@me
 S1
 @another-file

 $ rwsiteinfo --fields=sensor --sensor=@weird@,n@@me
 rwsiteinfo: Error parsing sensors '@weird@,n@@me': Bad
     token '@another-file' found in file 'weird,n@me': May not
     recursively use @FILE construct within a file

ENVIRONMENT

SILK_TIMESTAMP_FORMAT: This environment variable is used as the value for --timestamp-format when that switch is not provided. Since SiLK 3.11.0.
SILK_PAGER: When set to a non-empty string, rwsiteinfo automatically invokes this program to display its output a screen at a time. If set to an empty string, rwsiteinfo does not automatically page its output.
PAGER: When set and SILK_PAGER is not set, rwsiteinfo automatically invokes this program to display its output a screen at a time.
SILK_CLOBBER: The SiLK tools normally refuse to overwrite existing files. Setting SILK_CLOBBER to a non-empty value removes this restriction.
SILK_CONFIG_FILE: This environment variable is used as the value for the --site-config-file when that switch is not provided.
SILK_DATA_ROOTDIR: This environment variable specifies the root directory of data repository. As described in the "FILES" section, rwsiteinfo may use this environment variable when searching for the SiLK site configuration file. In addition, rwsiteinfo visits all the files in this directory when the repo-start-date, repo-end-date, or repo-file-count fields are specified in the --fields switch.
SILK_PATH: This environment variable gives the root of the install tree. When searching for configuration files and plug-ins, rwsiteinfo may use this environment variable. See the "FILES" section for details.
TZ: When the argument to the --timestamp-format switch includes local or when a SiLK installation is built to use the local timezone, the value of the TZ environment variable determines the timezone in which rwsiteinfo displays timestamps. (If both of those are false, the TZ environment variable is ignored.) If the TZ environment variable is not set, the machine's default timezone is used. Setting TZ to the empty string or 0 causes timestamps to be displayed in UTC. For system information on the TZ variable, see tzset(3) or environ(7). (To determine if SiLK was built with support for the local timezone, check the Timezone support value in the output of rwsiteinfo --version.)

FILES

${SILK_CONFIG_FILE}
${SILK_DATA_ROOTDIR}/silk.conf
/data/silk.conf
${SILK_PATH}/share/silk/silk.conf
${SILK_PATH}/share/silk.conf
/usr/share/silk/silk.conf
/usr/share/silk.conf: Possible locations for the SiLK site configuration file which are checked when the --site-config-file switch is not provided. The location of the SILK_DATA_ROOTDIR may be specified using the --root-directory switch.
${SILK_DATA_ROOTDIR}/
/data/: Locations for the root directory of the data repository when the --data-rootdir switch is not specified.

NOTES

rwsiteinfo added support for sensor groups in SiLK 3.21.0. For information on sensor groups, see the documentation for the group block command in the silk.conf(5) manual page.

The ability to read classes, types, sensors, and flowtypes from files via the @PATH directive was added in SiLK 3.20.0.

The --output-path switch was added in SiLK 3.15.0.

The repo-start-date, repo-end-date, and repo-file-count fields were added in SiLK 3.11.0.

rwsiteinfo was added in SiLK 3.0.

rwsiteinfo duplicates the functionality found in mapsid(1). mapsid is deprecated, and it will be removed in the SiLK 4.0 release. Examples of using rwsiteinfo in place of mapsid are provided in the latter's manual page.