NAME

sensor.conf - Sensor Configuration file for rwflowpack and flowcap

DESCRIPTION

As part of collecting flow data, the rwflowpack(8) and flowcap(8) daemons need to know what type of data they are collecting and how to collect it (e.g., listen on 10000/udp for NetFlow v5; listen on 4740/tcp for IPFIX). In addition, the rwflowpack daemon needs information on how to categorize the flow to specify its flowtype (for example, to label the flows collected at a border router as incoming or outgoing). The Sensor Configuration file, sensor.conf, contains this information, and this manual page describes the syntax of the file.

The sensor.conf file may have any name, and it does not have to reside in a particular location. The location of the the file is specified by the --sensor-configuration switch to rwflowpack and flowcap.

The Sensor Configuration File defines the following concepts:

probe: A probe specifies a source for flow data. The source could be a port on which flowcap or rwflowpack collects NetFlow or IPFIX data from a flow generator such as a router or the yaf software (http://tools.netsa.cert.org/yaf/). In rwflowpack, the source can be a directory to periodically poll for files containing NetFlow v5 PDUs, IPFIX records, or SiLK Flow records. When defining a probe, you must specify a unique name for the probe and the probe's type.
group: A group is a named list that contains either CIDR blocks or integers to treat as SNMP interfaces. A group may also contain groups that have been previously defined. The use of groups is optional; the primary purpose of a group is to allow the administrator to specify a commonly used list (such as the IP space of the network being monitored) in a single location. Groups were introduced in SiLK 2.1.
sensor: A sensor represents a logical collection point for the purposes of analysis. Usually one sensor corresponds to one probe; however, a sensor could be comprised of multiple probes, or the flow data collected at a single probe could be packed by multiple sensors. The sensor contains the information that rwflowpack uses to categorize each flow depending on how the flow moves between networks at the collection point. Since the sensors and the categories (known as flowtypes or as class/type pairs) are also used for analysis, they are defined in the Site Configuration file, as described silk.conf(5). In the Sensor Configuration file, you specify how sensors map to probes, and give the rules required to categorize the data.

SYNTAX

When parsing the Sensor Configuration file, blank lines are ignored. At any location in a line, the character # indicates the beginning of a comment, which continues to the end of the line. These comments are ignored.

All other lines begin with optional leading whitespace, a command name, and one or more arguments to the command. Command names are a sequence of non-whitespace characters, not including the character #. Arguments are textual atoms: any sequence of non-whitespace, non-# characters, including numerals and punctuation.

There are four contexts for commands: top-level, probe block, group block, and sensor block. The probe block, group block, and sensor block contexts are used to describe individual features of probes, groups, and sensors, respectively.

The valid commands for each context are described below.

Top-Level Commands

In addition to the commands to begin a probe, group, or sensor block, the top-level context supports the following command:

include ``path'': The include command is used to include the contents of another file whose location is path. This may be used to separate large configurations into logical units.

Probe Block

The first command below is used at top-level to begin a probe definition block, and the remaining commands are accepted within the probe context. Note that one and only one of listen-on-port, listen-on-unix-socket, read-from-file, or poll-directory must be specified.

probe probe-name probe-type

The probe command begins a new probe block and it continues to the end probe command. The arguments to the probe command are the name of the probe being defined and the probe type. The probe-name must be unique among all probes. It must begin with a letter, and it may not contain whitespace characters or the slash character (/). When a probe is associated with a single sensor, it is good practice to give the probe the same name as the sensor; however, when rwflowpack from SiLK-1.0 is reading flowcap files created by earlier versions of SiLK, the probe name must follow the rules specified in the rwflowpack(8) manual page. The probe-type must be one of the following:

netflow-v5: This probe processes NetFlow v5 protocol data units (PDU) that the daemon reads from a UDP port or from a file. NetFlow may be generated by a router or by software that reads packet capture (pcap(3)) data and generates NetFlow v5 records.
netflow: This is an alias for netflow-v5 for backwards compatibility. This alias is deprecated, and it may be removed in a future release.
ipfix: An IPFIX probe processes Internal Protocol Flow Information eXchange records that the daemon reads over the network from an IPFIX source such as yaf(1). An IPFIX probe can also poll a directory for files generated by the yaf program. To support IPFIX probes, SiLK must be built with support for the external library libfixbuf. Both yaf and libfixbuf are available from http://tools.netsa.cert.org/.
netflow-v9: This probe processes NetFlow v9 protocol data units (PDU) that the daemon reads from a UDP port from a router. To support NetFlow v9 probes, SiLK must be built with support for the external library libfixbuf, version 0.8 or later.
silk: A SiLK probe processes the records contained in SiLK Flow files created by previous invocations of rwflowpack. The flows will be completely re-packed, as if they were just received over the network. The sensor and flowtype values in each flow will be ignored. Note that SiLK usually removes the SNMP interfaces from its flow records, and it is likely that you will be unable to use the SNMP interfaces to pack the flows.

end probe

The end probe command ends the definition of a probe. Following an end probe command, top-level commands are again accepted.

listen-on-port port

This command specifies the network port where the probe should collect flow data. When listening to NetFlow from a Cisco router, this is the port that was specified to the Cisco ISO command

  ip flow-export [ip-address] [port]

protocol { tcp | udp }

This command, required when listen-on-port is given, specifies whether the port is a tcp or udp port. IPFIX probes support both types; the only permitted value for all other probe types is udp.

accept-from-host host-name

This optional command specifies the IP or name of the host that is allowed to connect to the port where the probe is listening. When this command is not present, any host may connect. The command may only be specified when the listen-on-port command is also present. When listening for NetFlow, this attribute would be the IP address of the router as seen from the machine running rwflowpack or flowcap. NOTE: Currently IPFIX probes and NetFlow v9 probes ignore this setting and accept flows from any host.

listen-as-host ip-address

This optional command is used on a multi-homed machine to specify the address the probe should listen on (bind(2) to). Its value is a network IP address in dotted-decimal notation. If not present, the program will listen on all the machine's addresses. The command may only be specified when the listen-on-port command is also present. For listening to NetFlow, the value would be the ip-address that was specified to the Cisco ISO command

  ip flow-export [ip-address] [port]

listen-on-unix-socket path-to-unix-socket

The value contains the path name to a UNIX domain socket where the flow generator writes its data. The parent directory of path-to-unix-socket must exist.

poll-directory directory-path

When this command is given, rwflowpack will periodically poll the directory-path to look for files to process. flowcap will exit with an error if you attempt to use probes that contain this command since flowcap does not support reading data from files. When polling the directory, zero length files and files whose name begin with a dot (.) are ignored. This command may be with the following probe types:

For SiLK probes, each file must be a valid SiLK Flow file.
IPFIX probes can process files created by the yaf program. There is no standard format for files of IPFIX records.
A NetFlow v5 probe will process files containing NetFlow v5 PDUs. The format of these files is described in the following command.

read-from-file dummy-value

When this command is given, rwflowpack will read NetFlow v5 records from the the file specified by the --netflow-file command line switch. The value to the read-from-file command is completely ignored, and we recommend you use /dev/null as the value. flowcap will exit with an error if you attempt to use probes that contain this command since flowcap does not support reading data from files. The format of a NetFlow v5 file is that the file's length should be an integer multiple of 1464 bytes, where 1464 is the maximum length of the NetFlow v5 PDU. Each 1464 block should contain the 24-byte NetFlow v5 header and space for thirty 48-byte flow records, even if fewer NetFlow records are valid. rwflowpack will accept NetFlow v5 files that have been compressed with the gzip(1) program.

log-flags { none | all | { bad | missing ... } }

This optional command gives specific logging instructions for the probe. Some probe types will keep track of the sequence numbers in the packets they receive; these probes will write messages to the log file about missing or bad packets. By default, all messages are written by the probes that keep track of this information, but sometimes it is useful to reduce the verbosity of this information or eliminate it entirely. The log-flags command accepts a list of the following values:

none: Log nothing.
all: Log all the conditions listed below; this is the default.
bad: Write messages about bad packets. An example of a bad packet is a NetFlow v5 PDU that reports no flow records.
missing: Examine the sequence numbers and write messages about missing packets.

interface-values { snmp | vlan }

This optional command specifies the values that should be stored in the input and output fields of the SiLK Flow records that are read from the probe. If this command is not given, the default is snmp. Note that NetFlow v5 probes only support snmp.

snmp: Store the index of the network interface card (ifIndex) where the flows entered and left the router, respectively.
vlan: Store the VLAN identifier for the source and destination networks, respectively. If only one VLAN ID is available, input is set to that value and output is set to 0.

priority value

This optional command is ignored. It is maintained for backwards compatibility.

To summarize the probe types and the input they can accept:

 Probe Type   Berkeley    Directory  UnixDomain    Single
               Socket      Polling     Socket       File
 ==========  ==========  ==========  ==========  ==========
 ipfix        tcp/udp        yes
 netflow-v5     udp          yes                    yes
 netflow-v9     udp
 silk                        yes

Group Block

The use of group blocks is optional. They are a convenience to define a list of commonly used CIDR blocks or SNMP interfaces. Groups are only used in sensor blocks as described in the descriptions for the discard-when, discard-unless, network-name-ipblocks, and network-name-interfaces statements, below.

The statements in a group definition must all be of the same type. That is, you cannot mix interfaces and ipblocks statements in a single group definition.

The first command below is used at top-level to begin a group definition block, and the remaining commands are accepted within the group block.

group group-name: The group command begins a new group definition block which continues to the end group command. The argument to the group command is the name of the group being defined. The group-name must be unique among all groups. It must begin with a letter, and it may not contain whitespace characters or the slash character (/).
end group: The end group command ends the definition of a group. Following an end group command, top-level commands are again accepted.
interfaces integer-or-group-ref [integer-or-group-ref...]: Interfaces values are added to a group by means of the interfaces command within a group block. Existing interface groups may be added by prefixing each group's name with an at (@) character. When you add an interface group, it is the same as individually adding each interface to the group being defined. It is an error to add an ipblock group to an interface group. The interfaces command may appear multiple times in a group block.
ipblocks cidr-block-or-group-ref [cidr-block-or-group-ref...]: CIDR block values are added to a group by means of the ipblocks command within a group block. Existing ipblock groups may be added by prefixing each group's name with an at (@) character. When you add an ipblock group, it is the same as individually adding each CIDR block to the group being defined. It is an error to add an interface group to an ipblock group. The ipblocks command may appear multiple times in a group block.

Sensor Block

The information from the sensor block is used by rwflowpack to determine how to categorize a flow; that is, in which file the flow record is stored.

When the Sensor Configuration file is used with flowcap, no sensors need to be defined. In fact, flowcap completely ignores all text inside each Sensor Block.

The logic to categorize a flow for the twoway site is described below in the PACKING LOGIC section. The packing logic is normally loaded into rwflowpack as a run-time plug-in. The packing logic for your version of rwflowpack may be different.

The packing logic plug-in specifies a list of network names, and you will refer to these networks when you configure the sensor block. Most plug-ins provide the external, internal, and null names, where internal refers to network being monitored, null are flows that were blocked by the router's access control list, and external is everything else.

Several of the commands in the sensor block require as an argument a list of interfaces or a list of CIDR blocks. These are defined as follows:

interface-list: An interface-list contains one or more non-negative integers ranging from 0 to 65535 that represent SNMP interface index(es). The interface-list can be creating by specifying numbers and/or references to groups containing interfaces. To add a reference to a group, prefix the group's name with an at (@) character. It is an error to add an ipblock group to an interface-list.
cidr-block-list: An cidr-block-list contains one or more CIDR blocks that represent an address space. The cidr-block-list can be creating by specifying CIDR blocks and/or references to groups containing CIDR blocks (ipblocks). To add a reference to a group, prefix the group's name with an at (@) character. It is an error to add an interface group to a cidr-block-list.

The first command below is used at top-level to begin a sensor configuration block, and the remaining commands are accepted within the sensor block.

sensor sensor-name: The sensor command begins a new sensor configuration block. It takes as an argument the name of the sensor being configured, and that sensor must be defined in the Site Configuration file (see silk.conf(5)). A sensor block is closed with the end sensor command. You may have multiple sensor blocks that have the same sensor-name, but the list of probes within each sensor block must be unique.
end sensor: The end sensor command ends the configuration of a sensor. Following an end sensor command, top-level commands are again accepted.
probe-type-probes probe-name [probe-name ...]: This command associates the listed probe names of the given probe type with the sensor. The probes do not have to be defined before they are used. (Note this also means that a mistyped probe-name will not be detected.) For example, netflow-v5-probes S1 says that S1 is a netflow-v5 probe; whenever flow data arrives on the S1 probe, the sensor associated with the probe notices that data is available and processes it.
discard-when source-interfaces interface-list: This optional command instructs rwflowpack to discard a flow record whose input SNMP interface is in interface-list. When the flow's input interface matches a value in the list, rwflowpack immediately stops processing that record and it will not pack the flow. When this command is present, no other discard-* source-interfaces commands may be specified on this sensor, but you may specify a discard-* source-ipblocks statement. When paired with VLAN tagging (see the interface-values statement in the probe block), this allows the administrator to discard flows that have a specific VLAN tag.
discard-when destination-interfaces interface-list: As above, but discards a flow when its output SNMP interface matches a value in interface-list.
discard-when any-interfaces interface-list: As above, but discards a flow when either its input or output SNMP interface matches a value in interface-list.
discard-when source-ipblocks cidr-block-list
discard-when destination-ipblocks cidr-block-list
discard-when any-ipblocks cidr-block-list: These optional commands are similar to the discard-when *-interfaces commands above except they operate on IP addresses. That is, the first discards a flow when its source IP address is in cidr-block-list, the second ignores a flow when its destination IP address is in cidr-block-list, and the third discards a flow when either its source IP or destination IP address is in cidr-block-list.
discard-unless source-interfaces interface-list: This optional command is the similar to the discard-when command described above, except this command instructs rwflowpack to discard the flow record unless its input SNMP interface is in interface-list. That is, the flow record will be packed only when all the discard-unless statements are matched. When this command is present, no other discard-* source-interfaces commands may be specified on this sensor, but you may specify a discard-* source-ipblocks statement. When paired with VLAN tagging (see the interface-values statement in the probe block), this allows one to discard flows that do not have a specific VLAN tag.
discard-unless destination-interfaces interface-list: As above, but discards a flow unless its output SNMP interface matches a value in interface-list.
discard-unless any-interfaces interface-list: As above, but discards a flow unless either its input or output SNMP interface matches a value in interface-list.
discard-unless source-ipblocks cidr-block-list
discard-unless destination-ipblocks cidr-block-list
discard-unless any-ipblocks cidr-block-list: These optional commands are similar to the discard-unless *-interfaces commands above, except they operate on IP addresses. That is, the first discards a flow unless its source IP address is in cidr-block-list, the second discards a flow unless its destination IP address is in cidr-block-list, and the third discards a flow unless either the source IP or destination IP address is in cidr-block-list.
source-network network-name: This command causes the sensor to assume that all flows originated from the network named network-name. For example, if a sensor is associated a probe that only monitors incoming traffic, you could use source-network external to specify that all traffic originated from the external network.
destination-network network-name: This command causes the sensor to assume that all flows were sent to the network named network-name.
network-name-ipblocks {cidr-block-list | remainder}: This command specifies the IP-space that are assigned to the network named network-name. The value of the command can be the keyword remainder or a list containing CIDR blocks and/or group references. When the value is the keyword remainder, the IP-space for network-name is computed by finding all IPs not assigned to other networks on this sensor. The remainder keyword can only appear one time within a sensor block.
network-name-interfaces {interface-list | remainder}: This command specifies the SNMP interface index(es) that are assigned to the network named network-name. The value of the command can be the keyword remainder or a list containing interface numbers and/or group references. When the value is the keyword remainder, the interface list is computed by finding all interface values not assigned to other networks on this sensor. The remainder keyword can only appear one time within a sensor block.
isp-ip ip-address [ip-address ...]: This optional command may be used for a sensor that processes NetFlow data. The value to the command is a list of IP addresses in dotted-decimal notation, where the IPs are the addresses of the NICs on the router. For traffic that doesn't leave the router (and thus was sent to the router's null-interface), some packing-logic plug-ins use these IPs to distinguish legitimate traffic for the router (e.g., routing protocol traffic, whose destination address would be in this list) from traffic that violated the router's access control list (ACL).

PACKING LOGIC

The packing logic plug-in that rwflowpack loads uses the values in the Sensor Block to determine how to categorize a flow and where on disk to store it. This section describes the how a flow's category (or flowtype) is set when the logic in the packlogic-twoway.so file is used.

The packlogic-twoway.so plug-in specifies three network names: the internal network refers to network being monitored, the null network is the destination for flows that were blocked by the router's access control list, and the external network is everything else.

You may categorize a flow by specifying the source-network and destination-network. This can be used when the physical network device only sees one sort of traffic; for example, if NetFlow is only enabled on a router's external-facing interfaces. Most of the time, more complex categorization is desired.

You may categorize any flow record, regardless of the type of probe that collected it, based on the IP addresses the flow contains and the IP space of the monitored network. Flows collected by a NetFlow probe may instead be categorized using the SNMP interface information in the flow.

The packlogic-twoway.so packing logic requires you specify for each sensor either the IP space outside the monitored network (external-ipblock) or, for sensors that use NetFlow probes, the SNMP interfaces that face outside the network (external-interfaces). In addition to specifying the external side, you may specify the values on the internal side (internal-ipblock or internal-interfaces). When the internal value is not specified, it is taken to be all values not used on the external side. It is an error to specify both the IP space and SNMP interfaces on a sensor.

When a router generates a flow record for packets that the router did not route (that never left the router), the outgoing SNMP interface is set to the null interface. A packet may be not-routed because the packet is meant for the router itself (e.g., a BGP message) or because it matches an ACL violation. On a Cisco router, the null interface has the destination ID of 0. You may specify the null-interfaces to use this destination in your packing logic. (For consistency, there is a null-ipblock that can be set.)

Note: As of SiLK-1.0, the null-interface must be explicitly set. It is no longer set by default.

Given a flow traveling between the external, internal, and null networks, the following rules specify the flowtype that is assigned to the flow. This description uses IP-based splitting; for SNMP-based splitting, the logic is identical, except the SNMP interfaces are checked. These rules also apply when the source- and destination-networks have been set explicitly.

ext2ext: Contains flows where both the source IP and destination IP are listed in the external-ipblock.
innull: Contains flows where the source IP is listed in the external-ipblock and the destination IP is listed in the null-ipblock.
in,inicmp,inweb: Contains flows where the source IP is listed in the external-ipblock and the destination IP is listed in the internal-ipblock. A flow's flowtype is inicmp if the protocol is ICMP (i.e., 1). The flow's flowtype is inweb if the protocol is TCP (6) and either the source port or destination port is one of 80, 443, 8080. Otherwise the flow's flowtype is in.
int2int: Contains flows where both the source IP and destination IP are listed in the internal-ipblock.
outnull: Contains flows where the source IP is listed in the internal-ipblock and the destination IP is listed in the null-ipblock.
out,outicmp,outweb: Contains flows where the source IP is listed in the internal-ipblock and the destination IP is listed in the external-ipblock.
other: Contains flows where either

EXAMPLES

All these examples assume you are using the packlogic-twoway.so packing logic plug-in to rwflowpack.

NetFlow Categorized by SNMP Interface

The following two blocks define a probe that listens on 9900/udp for NetFlow v5 from a router. The probe only accepts traffic originating from 172.16.22.22. The associated sensor uses the SNMP interfaces to categorize the flows, where traffic that enters the router on interface 1 and leaves on interface 8 is in, traffic entering on 8 and leaving on 1 is out, traffic from 1 to 0 is innull, traffic from 8 to 8 is ext2ext, etc.

  probe S1 netflow-v5
      listen-on-port 9900
      protocol udp
      accept-from-host 172.16.22.22
  end probe

  sensor S1
      netflow-v5-probes S1
      external-interfaces 1
      internal-interfaces 8
      null-interfaces 0
  end sensor

NetFlow Categorized by IP Address

The probe in this example is the same as above, except the administrator has chosen to log only messages about bad packets (messages about missing packets will be ignored). The sensor is categorizing flows by the source and destination IP address in the flow record. The internal network is defined as 128.2.0.0/16, and all other IPs are defined as external. HTTP traffic whose source is 128.2.0.1 and destination is google.com will be categorized as outweb; the reply (source of google.com and destination 128.2.0.1) will be inweb.

  probe S2 netflow-v5
      listen-on-port 9901
      protocol udp
      accept-from-host 172.16.22.22
      log-flags bad                     # ignore missing pkts
  end probe

  sensor S2
      netflow-v5-probes S2
      internal-ipblocks 128.2.0.0/16
      external-ipblocks remainder
  end sensor

IPFIX Categorized by IP Address

This example uses the same sensor configuration as above, but it uses an IPFIX probe to collect the flows on port 9902/tcp. Also, we instruct the probe to bind to address 192.168.1.92. This example includes a group definition to define the internal network.

  probe S3 ipfix
      listen-on-port 9902
      protocol tcp
      listen-as-host 192.168.1.92
  end probe

  group my-network
      ipblocks 128.2.0.0/16
  end group

  sensor S3
      ipfix-probes S3
      internal-ipblocks @my-network
      external-ipblocks remainder
  end sensor

IPFIX Read from Files

This example uses the same sensor configuration as above. The probe processes files that have been created by yaf(1) and stored in the directory /tmp/var/yaf/.

  probe S4 ipfix
      poll-directory /tmp/var/yaf
  end probe

  sensor S4
      ipfix-probes S4
      internal-ipblock 128.2.0.0/16
      external-ipblock remainder
  end sensor

Multiple IPFIX

Consider the case of using yaf on a monitor at the border of a network where all traffic entering the network enters the monitor on one network interface card (NIC) and all traffic leaving the network enters the monitor on a different NIC. Since yaf does not support multiple interfaces yet, you must run two yaf processes, one for each NIC. The sensor configuration for this monitor would list two probes, each listening on a different port, and two sensor blocks both packing to the same sensor.

  probe S5-in ipfix
      listen-on-port 9902
      protocol tcp
  end probe

  probe S5-out ipfix
      listen-on-port 9907
      protocol tcp
  end probe

  sensor S5
      ipfix-probes S5-in
      source-network external
      destination-network internal
  end sensor

  sensor S5
      ipfix-probes S5-out
      source-network internal
      destination-network external
  end sensor

One Probe to Two Sensors

Suppose you have instrumented a single router but you wish to split the traffic into two sensors, where one part of the network (monitored by sensor S6) is defined as 128.2.0.0/17, and the other (sensor S7) as 128.2.128.0/17. Traffic between 128.2.0.1 and google.com will be assigned to sensor S6, but it will so appear as ext2ext traffic for sensor S7 unless you explicitly discard that traffic using the discard-unless statement.

  probe S6-S7 ipfix
      listen-on-port 9903
      protocol tcp
  end probe

  group S6-space
      ipblocks 128.2.0.0/17
  end group

  group S7-space
      ipblocks 128.2.128.0/17
  end group

  sensor S6
      ipfix-probes S6-S7
      discard-unless any-ipblock @S6-space
      internal-ipblocks @S6-space
      external-ipblocks remainder
  end sensor

  sensor S7
      ipfix-probes S6-S7
      discard-unless any-ipblock @S7-space
      internal-ipblocks @S7-space
      external-ipblocks remainder
  end sensor

Discarding Flows Using VLAN Tags

You can configure rwflowpack to discard flows that do not have a particular VLAN tag. First, you use the interface-values statement to instruct the probe to put the VLAN id into the fields that typically store the SNMP interfaces. On the sensor, you use the discard-unless command to discard flows that do not have the desired VLAN tag (27 in this example). Normally you do not use the VLAN tags to determine a flow's direction (category) since there is often a single VLAN tag on each flow, and you must specify the IP space of the monitored network in the sensor.

  probe S8 ipfix
      listen-on-port 9904
      protocol tcp
      interface-values vlan
  end probe

  sensor S8
      ipfix-probes S8
      discard-unless any-interface 27
      internal-ipblocks 128.2.0.0/16
      external-ipblocks remainder
  end sensor

NetFlow From a File Categorized by SNMP Interfaces

Instead of listening on a UDP port for NetFlow traffic, you can configure the probe to process a file containing NetFlow v5 PDUs. This example assumes you are running rwflowpack with the switches --input-mode=file --netflow-file=FILENAME. The --netflow-file switch overrides the read-from-file command on the probe.

  probe S9 netflow-v5
      log-flags bad                     # ignore missing pkts
      read-from-file /dev/null          # use --netflow-file=<file>
  end probe

  sensor S9
      netflow-v5-probes S9
      external-interface 2
      internal-interface 9
      null-interface 0
  end sensor

Repacking of SiLK Flows by IP Address

A SiLK probe must get its flows by polling a directory of SiLK Flow files. The flows can be re-categorized based on the IP addresses or based on the SNMP interfaces (beware: often the SNMP interface values are 0 in SiLK Flow data). In this example, the files in the directory /var/tmp/old-data/ are processed. The internal network is defined as 128.2.0.0/16, and all other IPs are defined as external.

  probe S10 silk
      poll-directory /var/tmp/old-data
  end probe

  sensor S10
      silk-probes S10
      internal-ipblock 128.2.0.0/16
      external-ipblock remainder
  end sensor

NOTES

The following statements did not exist prior to SiLK 2.1: group, discard-unless, discard-when, interface-values.

The ability to repack SiLK Flow files and to handle files created by yaf was added in SiLK 2.0.

You may need to use specific probe names when rwflowpack is processing files created by a pre-SiLK-1.0 version of flowcap. See the NOTES section of the rwflowpack man page for details.

The syntax of sensor.conf differs greatly between SiLK-0.11.x and SiLK-1.0. SiLK provides the update-sensor-conf script to upgrade older versions of sensor.conf to the new syntax. When making the change by hand, this old sensor-probe block:

 sensor-probe S01
      probe-name netflow
      probe-type netflow
      priority 8
      log-flags none all bad missing
      listen-on-port 9999
      protocol udp
      listen-as-host 192.168.1.1
      listen-on-unix-domain-socket /tmp/sock
      read-from-file /var/tmp/flow-file
      accept-from-host 172.16.22.22
      null-index 0
      isp-ip 10.10.10.10,10.10.10.11
      external-index 1,2,3,4
      internal-index 8,9
      null-ipblock 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
      internal-ipblock 128.2.0.0/16
      external-ipblock remainder

becomes:

 probe S01 netflow-v5
      priority 8
      log-flags none all bad missing
      listen-on-port 9999
      protocol udp
      listen-as-host 192.168.1.1
      listen-on-unix-domain-socket /tmp/sock
      read-from-file /var/tmp/flow-file
      accept-from-host 172.16.22.22
 end probe

 sensor S01
      netflow-v5-probes S01
      isp-ip 10.10.10.10,10.10.10.11
      null-interface 0
      external-interface 1,2,3,4
      internal-interface 8,9
      null-ipblock 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
      internal-ipblock 128.2.0.0/16
      external-ipblock remainder
 end sensor

Note the following:

Move the probe-type to the same line as the name of the probe.
Change the probe-type from netflow to netflow-v5.
Add a probe-type-probes statement to the sensor.
Change any occurrence of -index or -indexes to -interface.
For netflow-v5 probes, rwflowpack no longer sets the null-interface to 0 by default. You must specify it explicitly.
The protocol statement is now required for all probes whenever listen-on-port is specified.
The probe block gets all the statements except for those that define ipblocks, interfaces, or isp-ips.
Each block must have an end statement.