NAME
rwpmapbuild - Create a binary prefix map from a text file
SYNOPSIS
rwpmapbuild [--input-file=FILENAME] [--output-file=FILENAME]
[--note-add=TEXT] [--note-file-add=FILENAME]
rwpmapbuild --help
rwpmapbuild --version
DESCRIPTION
Prefix maps provide a way to map field values (specifically either IPv4 addresses or protocol-port pairs) to string labels based on a user-defined map file. Using the prefix map plug-in, pmapfilter(3), you can you can partition, count, sort and display records based on arbitrary labels. rwpmapbuild reads textual input to create a binary prefix map file that the plug-in can use; the syntax of this input is described below.
The textual input is read from the specified input file, or from the standard input when the --input-file switch is not provided. The binary output is written to the named output file, or to the standard output when the --output-file switch is not provided and the standard output is not connected to a terminal.
OPTIONS
Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.
- --input-file=FILENAME
-
Read the textual input from FILENAME. You may use
stdinor-to represent the standard input. When this switch is not provided, the input is read from the standard input. The input file format is described below. - --output-file=FILENAME
-
Write the binary prefix map to FILENAME. You may use
stdoutor-to represent the standard output. When this switch is not provided, the prefix map is written to the standard output unless stdout is connected to a terminal. - --note-add=TEXT
-
Add the specified TEXT to the header of the output file as an annotation. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool.
- --note-file-add=FILENAME
-
Open FILENAME and add the contents of that file to the header of the output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to ensure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation.
- --help
-
Print the available options and exit.
- --version
-
Print the version number and information about how SiLK was configured, then exit the application.
INPUT FILE FORMAT
The input file format consists of any number of input lines of the
forms described below. Note that there is not a form that accepts
a single IP address and a label; therefore, to provide a label for a
single IP address you must append /32 to the IPv4 address.
Blank lines in the input file are ignored, as are comments. Comments
begin with the first # character on a line and extend to the end of
the line.
In the following, the label-value represents either a numerical label identifier that was created with the label command or label-text.
label-text is a textual string that begins at the first non-whitespace character and extends to the final non-whitespace character on that line that does not appear in a comment. The label-text may include embedded whitespace and non-alphanumeric characters. When the parser reads label-text, it checks to see if it has seen the text before. If the label-text exists, the numeric identifier for that text is used. If the label-text has not been seen before, a new label identifier is created; if a label statement occurring later in the file attempts to use this label-text or automatically generated identifier, an error message is printed.
The following statements are supported:
- map-name simple-string
-
Creates a name for the data in this prefix map file. The simple-string cannot contain whitespace, a comma, or a colon. When the prefix map file is used by rwfilter(1), the simple-string is used to generate the filtering switch names. When the prefix map file is used by rwcut(1), rwgroup(1), rwsort(1), or rwuniq(1), the simple-string is used to generate the field names. See pmapfilter(3) for details.
- default label-value
-
Make the given label identifier or label text the default value for any ranges not explicitly mentioned in this input file. You may not provide a numeric label that has not yet been defined.
- label num label-text
-
Associate the numeric identifier with the given label text. Note that any label text used in the remainder of the file will be automatically assigned a number. It is an error to give the same label text to multiple names, or the same text to multiple label identifiers.
- mode { ip | proto-port }
-
Process this file in IPv4 address or protocol/port mode. By default, processing is done in IPv4 address mode. (IPv6 addresses are not yet supported).
IPv4 Address Mode
- cidr-block label-value
-
Associate the given label identifier or label text with this CIDR block.
- low-ip-or-int high-ip-or-int label-value
-
Associate the given label identifier or label text with this IPv4 range (both IPs given as either integers or IPv4 addresses.)
Protocol/Port Mode
- proto/port proto/port label-value
-
Associate the given label identifier or label text with all protocols and port numbers between these two values inclusive. Note that while port is not meaningful for all protocols (specifically, it is meaningful for TCP and UDP and may contain type/code information for ICMP), this file allows port numbers to be given for any protocol.
- proto proto label-value
-
Associate the given label identifier or label text with all protcols between these two values.
NOTES
The IP Adddress input file can contain nested CIDR blocks. They should be ordered with the more general blocks first, and the more specific blocks last. That is, use:
10.0.0.0/8 My-space 10.1.0.0/16 Special-Subnet-1 10.1.2.0/24 Special-Subnet-2
Likewise, the protocol/port data can be nested:
6 6 TCP 6/0 6/1024 TCP/Generic reserved 6/22 6/22 TCP/SSH 6/25 6/25 TCP/SMTP 6/80 6/80 TCP/HTTP
EXAMPLE
Reading and writing to a file:
$ echo "10.1.2.3/32 my favorite host" > fav.txt $ rwpmapbuild -i fav.txt -o fav.pmap
Reading and writing to stdin and stdout:
$ echo "10.9.8.128/27 suspicious subnet" \
| rwpmapbuild --input-file=stdin --output-file=stdout > suspicious.pmap
Complex IP File
# Numerical mappings of labels
label 0 non-routable label 1 internal label 2 external
# Default to "external" for all un-defined ranges.
default external
# Force IP-mode
mode ip
# Create a name # # This will add --pmap-src-network and --pmap-dst-network # switches to rwfilter, and src-network and dst-network # fields to rwcut, rwgroup, rwsort, and rwuniq
map-name network
## Reserved and non-routable blocks ###########################
# Addresses in this block refer to source hosts on "this" # network. Address 0.0.0.0/32 may be used as a source # address for this host on this network; other addresses # within 0.0.0.0/8 may be used to refer to specified hosts # on this network [RFC1700, page 4].
0.0.0.0/8 non-routable
# This block is set aside for use in private networks. Its # intended use is documented in [RFC1918]. Addresses within # this block should not appear on the public Internet.
10.0.0.0/8 non-routable
# This block is assigned for use as the Internet host # loopback address. A datagram sent by a higher level # protocol to an address anywhere within this block should # loop back inside the host. This is ordinarily # implemented using only 127.0.0.1/32 for loopback, but no # addresses within this block should ever appear on any # network anywhere [RFC1700, page 5].
127.0.0.0/8 non-routable
# This is the "link local" block. It is allocated for # communication between hosts on a single link. Hosts # obtain these addresses by auto-configuration, such as when # a DHCP server may not be found.
169.254.0.0/16 non-routable
# This block is set aside for use in private networks. Its # intended use is documented in [RFC1918]. Addresses within # this block should not appear on the public Internet.
172.16.0.0/12 non-routable
# This block is assigned as "TEST-NET" for use in # documentation and example code. It is often used in # conjunction with domain names example.com or example.net # in vendor and protocol documentation. Addresses within # this block should not appear on the public Internet.
192.0.2.0/24 non-routable
# This block is set aside for use in private networks. # Its intended use is documented in [RFC1918]. Addresses # within this block should not appear on the public Internet.
192.168.0.0/16 non-routable
# 240.0.0.0/4 - This block, formerly known as the Class E # address space, is reserved. The "limited broadcast" # destination address 255.255.255.255 should never be # forwarded outside the (sub-)net of the source. The # remainder of this space is reserved for future use. # [RFC1700, page 4]
255.255.255.255/32 non-routable
# -- Below this line, would add any mappings appropriate to # -- the local network.
Complex Protocol/Port File
# Default to a hyphen ("-") for all un-defined ranges.
default -
# Force Protocol/Port-mode # # This MUST be present, since IP mode is the default.
mode proto-port
# Protocol Overview
1 1 ICMP 6 6 TCP 17 17 UDP 50 50 ESP 58 58 ICMPv6
# TCP -- Specific Ports
6/0 6/1024 TCP/Generic Reserved 6/21 6/21 TCP/ftp 6/22 6/22 TCP/ssh 6/25 6/25 TCP/smtp 6/53 6/53 TCP/dns 6/80 6/80 TCP/http 6/6000 6/6063 TCP/X11
# UDP -- Specific Ports
17/53 17/53 UDP/dns
# ICMP -- Specific Type/Code # # To convert a type/code to a "port" value as stored in SiLK: # (type << 8) | code OR (type * 256) + code # so 3/3 (Destination Unreachable/Port Unreachable) becomes:
1/771 1/771 ICMP/Destination Unreachable/Port Unreachable
SEE ALSO
pmapfilter(3), rwfilter(1), rwpmapcat(1). rwcut(1), rwgroup(1), rwsort(1), rwuniq(1)
BUGS
Prefix map files do not support IPv6 addresses. Any IPv6 addresses in the input will be ignored.