Monitoring for Large-Scale Networks

SiLK

rwpmatch

NAME

rwpmatch - Filter a tcpdump file using a SiLK Flow file

SYNOPSIS

rwpmatch --flow-file=FLOW_FILE [--msec-compare] [--ports-compare]
      TCPDUMP_INPUT > TCPDUMP_OUTPUT

rwpmatch --help

rwpmatch --version

DESCRIPTION

rwpmatch reads each packet from the pcap(3) (tcpdump(1)) capture file TCPDUMP_INPUT and writes the packet to the standard output if the specified FLOW_FILE contains a matching SiLK Flow record. It is designed to reverse the input from rwptoflow(1).

rwpmatch will read the pcap capture data from its standard input if TCPDUMP_INPUT is specified as stdin. The application will fail when attempting to read or write binary data from or to a terminal.

The SiLK Flow records in FLOW_FILE should appear in time sorted order.

OPTIONS

Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--flow-file=FLOW_FILE: FLOW_FILE refers to a file, named pipe, or the string stdin. The flow file determines which packet records should be output to the new packet file. This switch is required.
--msec-compare: Compare times down to the millisecond (rather than the default of second).
--ports-compare: For TCP and UDP data, compare the source and destination ports when matching.
--help: Print the available options and exit.
--version: Print the version number and information about how SiLK was configured, then exit the application.

EXAMPLES

In the following examples, the dollar sign ($) represents the shell prompt. The text after the dollar sign represents the command line.

Given the pcap capture file data.pcap, use rwptoflow(1) to convert it to a SiLK flow file:

$ rwptoflow data.pcap --packet-pass=good.pcap --flow-out=data.rw

With rwfilter(1), select the SiLK Flow records whose source IPs are found in the IPset file sip.set:

$ rwfilter --sipset=sip.set --pass=filtered.rw  data.rw

Match the original pcap file against the filtered SiLK file, in effect generating a pcap file which has been filtered by sip.set:

$ rwpmatch --flow-file=filtered.rw good.pcap > filtered.pcap

NOTES

For best results, the tcpdump input to rwpmatch should be the output from --packet-pass-output switch on rwptoflow. This ensures that only well-behaved packets are given to rwpmatch.

The flow file input to rwpmatch should contain single-packet flows originally derived from a tcpdump file using rwptoflow. If a flow record is found which does not represent a corresponding tcpdump record, rwpmatch will return an error.

Both the tcpdump and the SiLK file inputs must be time-ordered.

rwpmatch is an expensive I/O application since it reads the entire tcpdump capture file and the entire SiLK Flow file. It may be worthwhile to optimize an analysis process to avoid using rwpmatch until payload filtering is necessary. Saving the output from rwpmatch as a partial-results file, and matching against that in the future (rather than the original tcpdump file) can also provide significant performance gains.

SiLK releases before 3.23 supported only millisecond timestamps. When reading packets whose timestamps have finer precision, the times are truncated at the millisecond position.