sFlow Collectors

How to use libfixbuf to collect sFlow v5

libfixbuf can be used to collect sFlow and convert sFlow to IPFIX. Follow the steps above to create an fbListener. After creating the listener, retrieve the collector by calling fbListenerGetCollector() before calling fbCollectorSetSFlowTranslator(). Essentially, the libfixbuf translator is an IPFIX mediator which converts sFlow to IPFIX. sFlow v5 is a fixed format protocol. The same steps are used to retrieve flow records from the buffer, call fBufNext(). The internal template should contain some subset of the fields listed below. sFlow Data Records will have a template ID of 0xEEEE and the Options Records will have a template ID of 0xEEEF.

Fixbuf first reads the sFlow header to ensure the buffer contains sFlow v5. Fixbuf currently only has support for sFlow v5. The sFlow header only contains the time since the device last rebooted (but not the time of reboot) and this time will be reported in the systemInitTimeMilliseconds field. Fixbuf records the time that the sFlow message was received in the collectionTimeMilliseconds field. Once the first message has been received, the translator will create an external buffer and export the fixed templates to the fixbuf session. Note: the first sFlow message that fixbuf receives will not be processed - this is used to setup the translation process. The translator will keep track of sequence numbers per peer (IP)/observation domain (agent ID) by default. There are multiple sequence numbers in sFlow. Each sFlow message has a sequence number and each sample has a sequence number. The sFlow message sequence number is used to determine if sFlow messages have been dropped. Fixbuf will report if either sequence number is out of sequence and emit a warning. The warning is just for notification, libfixbuf will process all well-formed samples that it receives.

libfixbuf will process Flow Samples (1), Extended Flow Samples (3), Counter Samples (2), and Extended Counter Samples (4). Any other format will return an FB_ERROR_SFLOW. Applications should ignore (and potentially log) FB_ERROR_SFLOW errors. FB_ERROR_SFLOW errors are not fatal. With an sFlow sample, fixbuf can handle the following formats:

  • Raw Packet Data, enterprise = 0, format = 1
  • Ethernet Frame Data, enterprise = 0, format = 2
  • IPv4 Data, enterprise = 0, format = 3
  • IPv6 Data, enterprise = 0, format = 4
  • Extended Switch data, enterprise = 0, format = 1001
  • Extended Router data, enterprise = 0, format = 1002
  • Extended Gatway Data, enterprise = 0, format = 1003

Any other flow sample format will be silently ignored. Each sFlow flow record can contain the following fields, formats are listed in the parenthesis:

IPFIX FIELDS sFlow FIELDS Reduced Length
sourceIPv6Address Ipv6 Address in IPv6 (4) or Raw Packet (1) Data N
destinationIPv6Address Ipv6 Address in IPv6 (4) or Raw Packet (1) DataN
ipNextHopIPv6Address Extended Router Data (1002) N
bgpNextHopIPv6Address Extended Gateway Data (1003) N
collectorIPv6Address Message Header Data N
collectionTimeMilliseconds Message Header Data N
systemInitTimeMilliseconds Message Header Data N
collectorIPv4Address Message Header Data N
protocolIdentifier IPv4 (3) or IPv6 (4) or Raw Packet (1) Data N
ipClassOfService IPv4 (3) or IPv6 (4) or Raw Packet (1) Data N
sourceIPv4PrefixLength Extended Router Data (1002) N
destinationIPv4PrefixLength Extended Router Data (1002) N
sourceIPv4Address IPv4 (3) or Raw Packet (1) Data N
destinationIPv4Address IPv4 (3) or Raw Packet (1) DataN
octetTotalCount Flow Sample Header Data 4
packetTotalCount Flow Sample Header Data 4
ingressInterface Flow Sample Header Data N
egressInterface Flow Sample Header Data N
sourceMacAddress Ethernet (2), IPv4 (3), IPv6 (4) or Raw Packet (1) Data N
destinationMacAddress Ethernet (2), IPv4 (3), IPv6 (4) or Raw Packet (1) Data N
ipNextHopIPv4Address Extended Router Data (1002) N
bgpSourceAsNumber Extended Gateway Data (1003)N
bgpDestinationAsNumberExtended Gateway Data (1003) N
bgpNextHopIPv4AddressExtended Gateway Data (1003) N
samplingPacketInterval Message Header Data N
samplingPopulationMessage Header Data N
droppedPacketTotalCountMessage Header Data 4
selectorIdMessage Header Data 4
vlanId IPv4 (3) or IPv6 (4) or Raw Packet (1) Data N
sourceTransportPort IPv4 (3) or IPv6 (4) or Raw Packet (1) Data N
destinationTransportPort IPv4 (3) or IPv6 (4) or Raw Packet (1) Data N
tcpControlBits IPv4 (3) or IPv6 (4) or Raw Packet (1) Data 2
dot1qVlanId Extended Switch Data (1001) N
postDot1qVlanId Extended Switch Data (1001) N
dot1qPriority Extended Switch Data (1001) N

libfixbuf will also convert sFlow Counter Records to Options Records in IPFIX. libfixbuf will only process the Generic Interface Counters (format = 1). Other formats will be silently ignored. The following fields are present in the Counter (Options) Template/Record:

IPFIX FIELDS sFlow FIELDS Reduced Length
collectorIPv6Address Message Header Data N
collectionTimeMilliseconds Message Header Data N
systemInitTimeMilliseconds Message Header Data N
collectorIPv4Address Message Header Data N
ingressInterface Counter Sample Header Data N
octetTotalCount ifINOctets (1) N
ingressInterfaceType ifType (1) N
packetTotalCount ifInUcastPkts (1) 4
ingressMulticastPacketTotalCount ifInMulticastPkts (1) 4
ingressBroadcastPacketTotalCount ifInBroadcastPkts (1) 4
notSentPacketTotalCount ifInDiscards (1) 4
droppedPacketTotalCount ifInErrors (1) 4
postOctetTotalCount ifOutOctets (1) N
ignoredPacketTotalCount ifInUnknownProtos (1) 4
postPacketTotalCount ifOutUcastPkts (1) 4
egressBroadcastPacketTotalCount ifOutBroadcastPkts (1) 4
selectorId Message Header Data 4

fbCollectorSFlowMissed() can be used to retrieve the number of potential missed export packets. This is not the number of FLOW samples that the collector has missed. Fixbuf tries to account for any reboot of the device and not count large sequence number discrepancies in it's missed count.

Fixbuf will return FB_ERROR_SFLOW if it tries to process any malformed samples.