NAME

rwfglob - Print files that rwfilter's File Selection switches will access

SYNOPSIS

  rwfglob [--start-date=YYYY/MM/DD[:HH] [--end-date=YYYY/MM/DD[:HH]]]
        { [--class=CLASS] [--type={all | TYPE[,TYPE ...]}]
         | [--flowtype=CLASS/TYPE[,CLASS/TYPE ...]] }
        [--sensors=SENSOR[,SENSOR ...]]
        [--data-rootdir=PATH] [--site-config-file=FILENAME]
        [--print-missing-files] [--no-file-names] [--no-summary]

  rwfglob [--data-rootdir=PATH] [--site-config-file=FILENAME] --help

  rwfglob --version

DESCRIPTION

rwfglob accepts the normal File Selection options of rwfilter(1) and prints, to the standard output, the names of the files that would normally be accessed. At the end, a summary is printed of the number of files that exist and the number of those files that are on tape. (The on tape number is determined by seeing how many files had 0 blocks allocated to them.) By default, rwfglob only prints the names of files that exist; to see the names of files that it did not find, supply the --print-missing-files switch.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--start-date=YYYY/MM/DD[:HH]

--end-date=YYYY/MM/DD[:HH]

The date predicates indicate which days and hours to consider when creating the list of files. The dates are expressed in YYYY/MM/DD:HH format. For example, 2003/01/18:00 represents the first hour of January 18th, 2003, while 2002/10/01:22 corresponds to 22:00 on October 1st, 2002.

Whether the date strings represent times in GMT or the local timezone depend on how SiLK was compiled. See the output from --help or check the Timezone support setting in the --version output to determine how your version of SiLK was compiled.

When both --start-date and --end-date are specified to hour precision, all hours within that time range are processed.

When --start-date is specified to day precision, the hour specified in --end-date (if any) is ignored, and files for all dates between midnight on start-date and 23:59 on end-date are processed.

When --end-date is not specified and --start-date is specified to day precision, files for that complete day are processed.

When --end-date is not specified and --start-date is specified to hour precision, files for that single hour are processed.

It is an error to specify --end-date without specifying --start-date.

When neither --start-date nor --end-date is given, rwfglob prints all files for the current day.

--class=CLASS

The --class switch is used to specify a group of data to process. Only a single class may be selected. Classes are defined in the silk.conf(5) site configuration file. If the --class option is not given, the default-class as specified in silk.conf is used. Use the --help option to see the list of available classes and the default class.

--type={all | TYPE[,TYPE]}

The --type predicate further specifies data within the selected CLASS by listing the TYPEs of traffic to process. The switch takes a comma-separated list of types or the keyword all which specifies all types for the specified CLASS. Types are defined in silk.conf, they typically refer to the direction of the flow, and they may vary by class. Classes typically define default-types to use when the --type switch is not specified. Use the --help option to get the list of available types for each class.

--flowtypes=CLASS/TYPE[,CLASS/TYPE ...]

The --flowtype predicate provides an alternate way to specify class/type pairs. The --flowtype switch allows a single rwfglob invocation to print data from multiple classes. The keyword all may be used for the CLASS and/or TYPE to select all classes and/or types.

--sensors=SENSOR[,SENSOR ...]

The --sensor switch is used to select data from specific sensors. The parameter is a comma separated list of sensor names, sensor IDs (integers), and/or ranges of sensor IDs. Sensors are defined in the silk.conf(5) site configuration file, and the mapsid(1) command can be used to print a mapping of sensor names to IDs and classes. When the --sensor switch is not specified, the default is to use all sensors which are valid for the specified class(es).

--data-rootdir=PATH

This option causes rwfglob to use PATH as the root of the data store directory, which overrides the location given in the SILK_DATA_ROOTDIR environment variable, which overrides the location that was compiled into rwfglob. The default data store directory will be shown when the --version option is given.

--site-config-file=FILENAME

Read the SiLK site configuration from the named file FILENAME. When this switch is not provided, the location specified by the SILK_CONFIG_FILE environment variable is used if that variable is not empty. The value of SILK_CONFIG_FILE should include the name of the file. Otherwise, the application looks for a file named silk.conf in the following directories: the root of the data directory (see --data-rootdir); the directories $SILK_PATH/share/silk/ and $SILK_PATH/share/; and the share/silk/ and share/ directories parallel to the application's directory.

--print-missing-files

This option prints to the standard error file names that rwfglob expected to find but did not. This switch is useful for debugging, but the list of files it produces can be misleading. For example, suppose there is a decommissioned sensor that still appears in the silk.conf file to permit retrieval of historical data; these data files will be missing even though their absence is expected. Use the output from this switch judiciously.

--no-file-names

This option instructs rwfglob not to print the names of the files that it successfully finds. By default, rwfglob prints the names of the files it finds and a summary line showing the number of files it found.

--no-summary

This option instructs rwfglob not to print the summary line (that is, the line that shows the number of files found). By default, rwfglob prints the names of the files it finds and a summary line showing the number of files it found.

--help

Print the available options and exit. The available classes and types will be included in output; you may specify a different root directory or site configuration file before --help to see the classes and types available for that site.

--version

Print the version number and information about how SiLK was configured, then exit the application.

EXAMPLES

Looking at a day on a single sensor:

  $ rwfglob --start=2003/10/11 --sensor=2
  /data/in/2003/10/11/in-GAMMA_20031011.23
  /data/in/2003/10/11/in-GAMMA_20031011.22
  /data/in/2003/10/11/in-GAMMA_20031011.21
  /data/in/2003/10/11/in-GAMMA_20031011.20
  /data/in/2003/10/11/in-GAMMA_20031011.19
  /data/in/2003/10/11/in-GAMMA_20031011.18
  /data/in/2003/10/11/in-GAMMA_20031011.17
  /data/in/2003/10/11/in-GAMMA_20031011.16
  /data/in/2003/10/11/in-GAMMA_20031011.15
  /data/in/2003/10/11/in-GAMMA_20031011.14
  /data/in/2003/10/11/in-GAMMA_20031011.13
  /data/in/2003/10/11/in-GAMMA_20031011.12
  /data/in/2003/10/11/in-GAMMA_20031011.11
  /data/in/2003/10/11/in-GAMMA_20031011.10
  /data/in/2003/10/11/in-GAMMA_20031011.09
  /data/in/2003/10/11/in-GAMMA_20031011.08
  /data/in/2003/10/11/in-GAMMA_20031011.07
  /data/in/2003/10/11/in-GAMMA_20031011.06
  /data/in/2003/10/11/in-GAMMA_20031011.05
  /data/in/2003/10/11/in-GAMMA_20031011.04
  /data/in/2003/10/11/in-GAMMA_20031011.03
  /data/in/2003/10/11/in-GAMMA_20031011.02
  /data/in/2003/10/11/in-GAMMA_20031011.01
  /data/in/2003/10/11/in-GAMMA_20031011.00
  globbed 24 files; 0 on tape

If you only want the summary, specify --no-file-names

  $ rwfglob --start-date=2003/10/11 --sensor=2 --no-file-names
  globbed 24 files; 0 on tape

ENVIRONMENT

SILK_CONFIG_FILE

This environment variable is used as the value for the --site-config-file when that switch is not provided.

SILK_DATA_ROOTDIR

When set, overrides the compiled-in value for the location of the directory tree containing the files of SiLK Flow records collected and stored by the packing system (rwflowpack(8)). In addition, when the --site-config-file switch is not provided and the SILK_CONFIG_FILE environment variable is not set, rwfglob looks for the site configuration file in $SILK_DATA_ROOTDIR/silk.conf.

SILK_PATH

This environment variable gives the root of the install tree. As part of its search for the SiLK site configuration file, rwfglob checks for a file named silk.conf in the directories $SILK_PATH/share/silk and $SILK_PATH/share.

SEE ALSO

rwfilter(1), mapsid(1), silk.conf(5)

BUGS

The --print-missing-files option needs to be smarter about what files are really missing.

The block size check is of unknown portability across different tape-farm systems.