silk.conf - SiLK site configuration file


The silk.conf SiLK site configuration file is used to associate symbolic names with flow collection information stored in SiLK Flow records.

In addition to the information contained in the NetFlow or IPFIX flow record (e.g., source and destination addresses and ports, IP protocol, time stamps, data volume), every SiLK Flow record has two additional pieces of information that is added when rwflowpack(8) converts the NetFlow or IPFIX record to the SiLK format:

Note that the binary form of SiLK flow records represent the sensor and flowtype by their numeric IDs, not by their names.

For historic reasons, one rarely speaks of the flowtype of a SiLK Flow record, but instead refers to its class and type. Every flowtype maps to a unique class/type pair. The classes and types have names only; they do not have numeric IDs. Note that flowtype and type are different concepts despite the similarity of their names.

A class is generally used to represent topological features of the network with different collections of sensors, since every active sensor must belong by one or more classes. Every class must have a unique name.

A type is used to distinguish traffic within a single topological area based on some other dimension. For example, incoming and outgoing traffic is generally distinguished into different types. Web traffic is also frequently split into a separate type from normal traffic in order to partition the data better. The type names within a class must be unique, but multiple classes may have a type with the same name.

As stated above, each class/type pair maps to a unique flowtype.

The silk.conf file defines

In normal usage, the silk.conf file will be located at the root of the SiLK data spool referenced by the SILK_DATA_ROOTDIR environment variable, or specified on the command line using the --data-rootdir flag. This ensures that the sensor and class definitions in the site configuration match the data in the flow records you retrieve.

If you cannot place the site configuration file in the data root directory, or the file in that location is incorrect, you can use the SILK_CONFIG_FILE environment variable to specify the location of your configuration file (including the file name). Many SiLK commands provide the --site-config-file switch which allows you to specify the name of the site configuration file on the command line.

By having the site configuration information outside of the SiLK tools, a single SiLK installation can be used to query different data stores (though each invocation of a command can only query one storage location).

Any additions or modifications to the silk.conf file will be seen by all SiLK applications upon their next invocation. There are some important things to keep in mind when modifying the silk.conf file:


In the site configuration file, each line may be blank, or contain any amount of leading whitespace, which is ignored. At any location in a line, the character # indicates the beginning of a comment, which reaches until the end of the line. (If a literal # symbol is required in the argument of any command, it may be quoted as described below.) These comments are ignored.

Each non-empty line begins with a command name, followed by one or more arguments. Command names are a sequence of non-whitespace characters, not including the characters # or " (see below for valid commands). Arguments may either be textual atoms (any sequence of non-whitespace characters, non-#-or-" characters, including numerals and punctuation), or quoted strings. Quoted strings begin with the character " and end with the character ", and allow for C-style backslash escapes in between. The character # inside a quoted string does not begin a comment, and whitespace is allowed inside a quoted string.

For the commands supported by silk.conf and described below, unless a command explicitly states that it is used by particular applications, it should be considered used by all of the SiLK analysis tools and the packing tools flowcap(8), rwflowpack(8), and rwflowappend(8).

There are three contexts for commands: top-level, class block, and group block contexts. The class block and group block contexts are used to describe individual features of classes and groups, while top-level commands are used to describe the entire configuration, and to define sensors.

The valid commands for each context are described below.

Top-Level Commands

class class-name

The class command begins a new class block. It takes as an argument the name of the class being defined. Each class must have a unique name. A class block is closed with the end class command. See below for a list of commands valid inside class blocks.

The class name must begin with a letter, must not be longer than 32 characters, and may not contain whitespace characters or the character slash (/).

A site that does not use multiple classes should define a single class with a name like all or default.

To be valid, a configuration file must contain at least one class definition.

Example: class all

default-class class-name

rwfilter(1) and rwfglob(1) will use a default class when the user does not specify an explicit --class. This command specifies that default class; the class must have been created prior to this command. If more than one default class is set, the last definition encountered is used.

Example: default-class all

group group-name

Sensor groups are a convenient way of defining named groupings of sensors for inclusion in classes. They cannot currently be used in the SiLK command-line tools, but only in the configuration file. The group command takes as an argument the group to be defined, and begins a group block. A group block is closed using the end group command. See below for details on valid commands within group blocks.

Example: group test-sensors

include "file-name"

The include command is used to include the contents of another file. This may be used to separate large configurations into logical units. (Note, however, that all sensors, classes, groups, and types must be declared before they may be referenced.)

Example: include "silk-2.conf"

packing-logic "file-name"

The packing-logic command provides a default value for the --packing-logic switch on rwflowpack(8). The value is the path to a plug-in that rwflowpack loads; the plug-in provides functions that determine into which class and type a flow record will be categorized. The path specified here will be ignored when the --packing-logic switch is explicitly specified to rwflowpack or when SiLK has been configured with hard-coded packing logic.

Example: packing-logic ""

path-format "format-string"

File and directory locations relative to the SILK_DATA_ROOTDIR may be defined using the path-format command. The path-format is used by rwflowpack and rwflowappend(8) when writing data to the data repository, and it is used by rwfilter and rwfglob when reading or listing files in the data repository. This command takes a format string specification that supports the following %-conversions:


The textual class name


The textual flowtype name for this class/type pair (see also %f)


The hour (24-hour clock) as a two-digit, zero-padded number


The textual sensor name (see also %n)


The textual type name


The year as a four-digit, zero-padded number


The day of the month as a two-digit, zero-padded number


The flowtype ID, as an unpadded number (see also %F)


The month of the year as a two-digit, zero-padded number


The sensor ID, as an unpadded number (see also %N)


The default file name, which is equivalent to %F-%N_%Y%m%d.%H


A literal % character

A % followed by any other character is an error.

For example, to place all spooled files directly in the data root directory, the path format %x could be used. To use two levels of hierarchy, the first containing the year and month, and the second containing the day and sensor name, like 2006-01/23-alpha/..., the format would be %Y-%m/%d-%N/%x.

If no path format is set by the configuration file, the default path format of %T/%Y/%m/%d/%x is used.

All path formats are currently required to end in /%x so that information may be extracted from the file name. This requirement may be lifted in the future.

Example: %C/%T/%Y/%m/%d/%x

sensor sensor-id sensor-name
sensor sensor-id sensor-name "sensor-description"

Individual sensor definitions are created with the sensor command. This command creates a new sensor with the given name and numeric ID. Sensor names must begin with a letter, must not be longer than 64 characters, and may not contain whitespace characters or the characters slash (/) or underscore (_).

The sensor line may may also provide an optional description of the sensor, enclosed in double quotes. The description can be used however your installation chooses to use it. The description may be viewed by specifying the describe-sensor field to rwsiteinfo(1). (When using sensor descriptions, the file's version must be 2.)

It is an error to define two different sensors with the same sensor ID or the same sensor name.

NOTE: It is extremely important not to change the sensor-id or sensor-name for a given sensor once that sensor is in use. The sensor-id field is stored numerically in SiLK data files, and the sensor-name field is used to construct file names within the data root directory.

Example: sensor 0 S001

Example: sensor 0 S001 "Primary connection to ISP"

version version-number

The version command declares that this configuration file conforms to a given version of the configuration file format. If the tools do not support this version of the configuration file, they will report an error. Currently, versions 1 and 2 of the format is defined, where version 2 indicates that sensor descriptions are present.

It is a recommended practice to include the version number at the beginning of all configuration files for compatibility with future versions.

Example: version 1

Class Block Commands

The commands inside a class block define the class's types, its default types, the sensors that belong to it, and the mapping from the class/type pair to the flowtype name and flowtype ID.

end class

The end class command ends the definition of a class. Following an end class command, top-level commands are again accepted.

Example: end class

default-types type-name ...

When no types are specified for the rwfilter or rwfglob commands, the default set of types for the selected class is used. Each of the types listed in this command is included as a default type of the class.

Example: default-types in inweb

sensors sensor-name-or-group-ref ...

The sensors command is used to associate sensors with a class. In short, to declare that these sensors have data for this class. Each item in the list must be either the name of a sensor or the name of a sensor group preceded by an at (@) character. When you add a sensor group, it is the same as individually adding each sensor in that group to the class.

Example: sensors my-sensor-1 my-sensor-2 @my-group-1

type flowtype-id type-name [ flowtype-name ]

The type command defines a type name within the current class, and it specifies the flowtype ID to use for that class/type pair. In addition, the type command may specify a flowtype name. The flowtype ID and flowtype name must be unique across the entire silk.conf file (and any included files). If a flowtype name is not specified, a default flowtype name is constructed by concatenating the name of the class and the name of the type. (e.g. the type in in the class all would have a flowtype name of allin.) Within a class, each type must have a unique name, but multiple classes may use the same type name. The type name and flowtype name must begin with a letter, must not be longer than 32 characters, and may not contain whitespace characters or the character slash (/).

As with sensors, it is important to be careful when renumbering flowtype IDs or renaming types or flowtypes because the numeric IDs are stored in data files, and the textual names are used as portions of file and path names.

Example: type 0 in

Example: type 1 out out

Group Block Commands

A group block is a convenience used to define a list of sensors.

end group

Close a group block by using the end group command. Following this command, top-level commands are again accepted.

Example: end group

sensors sensor-name-or-group-ref ...

Sensors are associated with a sensor group by means of the sensors command within a group block. Each item in the list must be either the name of a sensor or the name of a sensor group preceded by an at (@) character. When you add a sensor group, it is the same as individually adding each sensor in that group to the group being defined.

Example: sensors my-sensor-1 my-sensor-2 @my-group-1


rwfilter(1), rwfglob(1), rwsiteinfo(1), sensor.conf(5), flowcap(8), rwflowpack(8),packlogic-twoway(3), packlogic-generic(3), rwflowappend(8), silk(7), SiLK Installation Handbook