Partition SiLK Flow records into one or more 'pass' and/or 'fail' output streams. rwfilter is the primary tool for pulling flows from the data store.

Read SiLK Flow records, sort them by the specified key, and write the records to the named output path or to the standard output.

Print SiLK Flow records in a delimited, columnar, human-readable format. The default delimiter is the pipe (|).

The PySiLK extension allows one to read, manipulate, and write SiLK Flow records and IPsets from within Python. PySiLK may be used in a stand-alone Python script or as a plug-in. This document describes the objects, methods, and functions that PySiLK provides. The next entry describes using PySiLK from within a plug-in.

The SiLK Python plug-in provides a way to use PySiLK to define arbitrary partitioning rules for rwfilter and arbitrary fields for rwcut, rwuniq, and rwsort.

Bin SiLK Flow records by a user-defined key and print the total byte, packet, and/or flow counts for each bin. rwuniq can also print distinct source IP and destination IP counts.

Summarize SiLK Flow records across time, producing textual output with counts of bytes, packets, and flow records for each time bin.

Summarize SiLK Flow records by one of a limited number of key/value pairs and display the results as a Top-N or Bottom-N list.

Summarize SiLK Flow records by a specified key and print the byte, packet, and flow counts for flows matching the key.

Summarize SiLK flow records by source or destination IP and print the byte, packet, and flow counts for each IP.

Read SiLK Flow records and generate binary IPset file(s) containing the source IP addresses or destination IP addresses seen on the flow records.

Read (textual) IP addresses in dotted-quad or CIDR notation from an input file or from the standard input and write a binary IPset file.

Print the contents of a binary IPset files as text. Additional information about the IPset file can be printed.

Perform union, intersection, difference, and sampling functions on the input IPset files, generating a new IPset file.

Determine whether the IP address specified on the command line is contained in an IPset.

This command is deprecated; use rwsettool instead. Generate a new IPset file by performing intersection and/or difference operations on the IPset files listed on the command line.

This command is deprecated; use rwsettool instead. Merge the input binary IPset files into the output IPset; an IP in any input file will be in the output file.

Read SiLK Flow records and build binary Bag(s) containing key-count pairs. An example may be a Bag containing the byte count for each source port seen on the flow records.

Create a binary Bag file from a binary IPset file or from a textual input file.

Print binary Bag files as text.

Perform operations (addition, subtraction, etc) on binary Bag files.

Read textual input and create a binary Prefix Map (pmap) file for use with the Address Type (addrtype) and Prefix Map (pmapfilter) Plug-Ins.

Print information about a Prefix Map (pmap) file at text. By default, print each IP range in the pmap and its label.

Import a SiLK IPset, Bag, or Prefix Map file into the IP Address Association (IPA) library.

Export a set of IP addresses from the IP Address Association (IPA) library to a SiLK IPset, Bag, or Prefix Map.

The Address Type plug-in provides a way to map an IP address to an integer denoting the IP as internal, external, or non-routable.

The Country Code plug-in provides a mapping from an IP address to two-letter, lowercase abbreviation of the country that "owns" the IP address.

The Flowrate plug-in, which must be loaded explicitly, adds switches and fields to compute packets/second, bytes/second, bytes/packet, payload-bytes, and payload-bytes/second.

The Prefix Map plug-in provides a way to map field values to string labels based on a user-defined map file. The map file is created by rwpmapbuild.

Group flows together by specified the id fields, and mark the records with a group ID that is stored in the next-hop IP field. rwgroup requires that its input is sorted.

Match (mate) records as queries and responses and mark mated records with an ID that is stored in the next-hop IP field. rwmatch requires that its input is sorted.

Read SiLK Flow records from the standard input, mask off the lower bits of the source, destination, and/or next-hop IP, and write the resulting records to the standard output.

Convert a tcpdump file to a single file of SiLK Flow records. rwp2yaf2silk assumes that the yaf and rwipfix2silk commands are available on your system as it is a simple Perl wrapper around those commands.

Convert a stream of IPFIX (Internet Protocol Flow Information eXport) records to the SiLK Flow record format.

Convert a stream of SiLK Flow records to an IPFIX (Internet Protocol Flow Information eXport) format.

Output a tcpdump file as ASCII, in a form similar to rwcut.

Detect and eliminate duplicate records from multiple tcpdump input files. See also rwdedupe.

Filter a tcpdump file by outputting only packets whose 5-tuple and timestamp match corresponding flows in a SiLK Flow file.

Read a tcpdump file and generate a SiLK Flow record for every packet.

Attempt to detect scanning activity from SiLK Flow records. rwscan can produce files that can be loaded into a database and queried with rwscanquery.

Query the scan database that has been populated from database load files generated by rwscan.

Map between sensor names and sensor IDs using the values stored in the silk.conf file.

Read delimited text from the standard input, convert integer values in the specified column(s) to dotted-decimal IP addresss, and print the result to the standard output.

Append the SiLK Flow records contained in the second through final filename arguments to the records contained in the first filename argument.

Read SiLK Flow records from the files named on the comamnd line, or from the standard input when no files are provided, and write the SiLK records to the specified output file or to the standard output if it is not connected to a terminal.

Read SiLK Flow records from files named on the command line or from the standard input and write the records to the named output path or to the standard output, removing any duplicate flow records. Note that rwdedupe will reorder the records as part of its processing.

Print to the standard output the list of files that rwfilter would normally process for a given set of file selection switches.

Print information (type, version, etc.) about a SiLK Flow, IPset, Bag, or Prefix Map file.

Create the country code Prefix Map required by the ccfilter plug-in from the MaxMind GeoIP database.

Invoke rwfilter to find flow records matching Snort signatures.

Map a (textual) list of IP addresses to their country code.

Generate a new SiLK Flow file by substituting a pseudo-random IP address for the source and destination IP addresses in given input file.

Read delimited text from the standard input, attempt to resolve the IP addresses in the specified column(s) to host names, and print the result to the standard output.

Read SiLK Flow records and generate a set of subfiles from the input. The subfiles can be limited by flow-, byte-, or packet-counts, or by unique IP count. In addition, the subfile may contain all the flows or only a sample of them.

Generate a new SiLK Flow file by changing the byte order of values in a given input SiLK Flow file.

Generate SiLK Flow records from textual input; the input should be in a form similar to what rwcut generates.

Listen to flow generators (devices which produce network flow data) and store the data in temporary files prior to transferring the files to a remote machine for processing by rwflowpack.

Read flow data either directly from a flow generator or from files generated by flowcap, convert the data to the SiLK Flow record format, categorize the flow records, and write the records either to hourly flat-files organized in a time-based directory structure or into files for transfer to a remote machine for processing by rwflowappend.

Watch a directory for files containing small numbers of SiLK flow records and append those records to hourly files stored in a time-based directory tree.

Watch an incoming directory for files, move the files into a processing directory, and transfer the files to one or more rwreceiver processes. Either rwsender or rwreceiver may act as the server (i.e., listen for incoming network connections) with the other acting as the client.

Accept files transferred from one or more rwsender processes and store them in a destination directory. Either rwsender or rwreceiver may act as the server (i.e., listen for incoming network connections) with the other acting as the client.

Read SiLK Flow records and look for unusual patterns that may indicate data file corruption.

Read a file containing NetFlow v5 PDU records and print the SNMP interfaces that are used most often and the number of records seen for each interface.

Configuration file for sensors and probes used by rwflowpack and flowcap.

An overview of SiLK and a list of environment variables that affect SiLK.

Configuration file naming the Classes, Types, and Sensors available at your installation.