Monitoring for Large-Scale Networks

YAF

getFlowKeyHash

NAME

getFlowKeyHash - YAF Flow Key Calculator

SYNOPSIS

getFlowKeyHash [--in INPUT_SPECIFIER]
               [--out OUTPUT_SPECIFIER]
               [--sip4 IPV4_ADDRESS] [--dip4 IPV4_ADDRESS]
               [--sip6 IPV6_ADDRESS] [--dip6 IPV6_ADDRESS]
               [--sport SOURCE_PORT] [--dport DESTINATION_PORT]
               [--protocol PROTOCOL] [--vlan VLAN_ID]
               [--date FLOW_START_DATE] [--time FLOW_START_TIME]
               [--ipfix][--reverse][--snmp]

DESCRIPTION

getFlowKeyHash takes information about a flow and calculates the yaf(1) flow key hash and prints the flow records along with the hash to the screen or to the output specifier in a textual, bar (|) delimited format. Together, the flow key hash and the start time of the flow will create a unique identifier for a flow. The flow key hash and start time are used to identify flows with yaf's pcap export options. If given the start time, getFlowKeyHash will print the file path to the flow's pcap (helpful if yaf was run with the --pcap-per-flow option) and the milliseconds since epoch time (helpful for use with yaf's pcap-meta-file).

getFlowKeyHash reads IPFIX from the file provided to --in or from standard input when no file names are specified and --sip[4|6] and --dip[4|6] are not available.

getFlowKeyHash will export the flow key hash and start time in IPFIX if --ipfix is provided.

OPTIONS

Input Options

Either --in or the remaining options are required for getFlowKeyHash to calculate the flow key.

--in INPUT_SPECIFIER

INPUT_SPECIFIER is an input specifier. The default is to read from standard input. Otherwise, it is a filename. The string - may be used to read from standard input. The input should be IPFIX. The internal template contains the following fields:

flowStartMilliseconds
sourceIPv4Address, sourceIPv6Address
destinationIPv4Address, destinationIPv6Address
sourceTransportPort
destinationTransportPort
ingressInterface
vlanId
protocolIdentifier

The ingressInterface is included because rwsilk2ipfix(1) exports the vlanId in this field. The ingressInterface field is only used if it is populated and vlanId is not. If both fields contain non-zero values, vlanId is used to calculate the flow key hash, and ingressInterface is ignored.

--sip4 IPV4_ADDRESS

IPV4_ADDRESS should be the source address of the flow in the form 127.0.0.1. Either --sip4 or --sip6 must be present if --in is not present.

--dip4 IPV4_ADDRESS

IPV4_ADDRESS should be the destination address of a flow in the form 127.0.0.1. Either --dip4 or --dip6 must be present if --in is not present.

--sip6 IPV6_ADDRESS

IPV6_ADDRESS should be the source IPv6 address of a flow in the form 2001:48af::1:1. Either --sip6 or --sip4 must be present if --in is not present.

--dip6 IPV6_ADDRESS

IPV6_ADDRESS should be the destination IPv6 address of a flow in the form 2001:48af::1:1. Either --dip6 or --dip4 must be present if --in is not present.

--sport SOURCE_PORT

SOURCE_PORT should be the source port of the flow. This number should be between 0 and 66535. The default value is 0.

--dport DESTINATION_PORT

DESTINATION_PORT should be the destination port of the flow. This number should be between 0 and 65535. The default value is 0.

--protocol PROTOCOL

PROTOCOL is the protocol of the flow. For TCP, PROTOCOL is 6. For UDP, PROTOCOL is 17. The default value is 0.

--vlan VLAN

VLAN is the vlan ID of the flow, if available. The default value is 0.

Output Options

This option controls where getFlowKeyHash will send its output. The default is to stdout.

--out OUTPUT_SPECIFIER

OUTPUT_SPECIFIER is an output specifier. The default is to write to stdout. Otherwise, a filename can be given. The string - may be used to write to stdout. The IPFIX export template contains the following fields:

flowStartMilliseconds
flowEndMilliseconds
packetTotalCount
yafFlowKeyHash
reverseFlowKeyHash

--ipfix

If provided, getFlowKeyHash will export the flow key hash, start time, end time (if available), packet count (if available), and reverse flow key hash (if --reverse is present) in IPFIX to the OUTPUT_SPECIFIER provided to --out or stdout by default. This option is meant to be used when using with yafMeta2Pcap(1).

Optional Arguments

The following options are optional and are only necessary to calculate the file path of the pcap for the flow. yaf(1) uses the milliseconds of the flow's start time as the file directory. The date and time is also used in pcap file's name. If the date and time are given, getFlowKeyHash will also calculate the milliseconds since epoch, which is useful when using yaf's pcap-meta-files and the yafMeta2Pcap(1) tool.

--date FLOW_START_DATE: Specify the date in YYYY-MM-DD format; time is in UTC.
--time FLOW_START_TIME: Specify the time in HH:MM:SS[.sss] format.
--reverse: Calculate the reverse flow key hash by swapping the source and destination IP addresses and ports. If this option is present and --ipfix is present, getFlowKeyHash will also export the reverse flow key hash in IPFIX.
--snmp: If present, getFlowKeyHash will ignore the value in ingressInterface field. SiLK encodes either the VLAN ID or the SNMP interface in the ingressInterface IPFIX field. If SiLK was configured to pack the SNMP interface, this value will interfere with the correct calculation of the getFlowKeyHash. You also may want to ignore this field if yaf is run with the --no-vlan-in-key option which forces yaf to not use the VLAN in the flow key hash calculation. Alternatively, you may set the environment variable YAF_IGNORE_SNMP to 1 to always ignore this value in flow key hash calculations.

Examples

In the following examples, the dollar sign ("$") represents the shell prompt. The text after the dollar sign represents the command line. Lines have been wrapped for improved readability, and the back slash ("\") is used to indicate a wrapped line.

$ getFlowKeyHash --sip4 10.10.25.11 --dip4 10.10.26.12 --sport 1586 \
                 --dport 80 --protocol 6 --vlan 3
           sIP|            dIP|sPort|dPort|pro| vlan|      hash
   10.10.25.11|    10.10.26.12| 1586|   80|  6|    3| 100819735

$ getFlowKeyHash --sip4 10.10.25.11 --dip4 10.10.26.12 --sport 1586 \
              --dport 80 --protocol 6 --vlan 3  --date 2011-03-21 \
              --time 16:15:54.256
           sIP|            dIP|sPort|dPort|pro| vlan|      hash|                  ms
   10.10.25.11|    10.10.26.12| 1586|   80|  6|    3| 100819735|       1300724154256

$ rwfilter --pass-destination=stdout --application=0 \
           --start-date=2014/01/29:16:27 \
           --type=all --max-pass-records=1
           | rwsilk2ipfix | getFlowKeyHash
           sIP|            dIP|sPort|dPort|pro| vlan|      hash|                  ms
   10.20.11.51|    10.64.22.15|61902| 8080|  6|    0|4053426924|       1391012769175

$ yaf --in mypcap.pcap --out - | getFlowKeyHash
           sIP|            dIP|sPort|dPort|pro| vlan|      hash|                  ms
 127.0.224.172|        5.5.5.5| 1324|   80|  6|    0|2133427641|       1207802501882

$ rwfilter --pass-dest=stdout --application=53 \
           --start-date=2014/01/29:16:27 \
           --type=all --max-pass-records=2 \
           | rwsilk2ipfix | getFlowKeyHash -I | \
           yafMeta2Pcap --pcap-meta-file="/data/pcapmeta/*" \
           --out myDNSpcap.pcap

Known Issues

Bug reports may be sent directly to the Network Situational Awareness team at <netsa-help@cert.org>.

AUTHORS

Emily Sarneso and the CERT Network Situational Awareness Group Engineering Team, <http://www.cert.org/netsa>.