How-To Use libfixbuf to Collect sFlow v5
libfixbuf can be used to collect sFlow and convert sFlow to IPFIX.
Follow the UDP instructions to create an fbListener_t, retrieve the fbCollector_t by calling fbListenerGetCollector(), and call fbCollectorSetSFlowTranslator().
Essentially, the libfixbuf translator is an IPFIX mediator which converts sFlow to IPFIX. sFlow v5 is a fixed format protocol. The same steps are used to retrieve flow records from the buffer, call fBufNext(). The internal template should contain some subset of the fields listed below. sFlow Data Records will have a template ID of 0xEEEE and the Options Records will have a template ID of 0xEEEF.
Fixbuf first reads the sFlow header to ensure the buffer contains sFlow v5. Fixbuf currently only has support for sFlow v5. The sFlow header only contains the time since the device last rebooted (but not the time of reboot) and this time will be reported in the systemInitTimeMilliseconds field. Fixbuf records the time that the sFlow message was received in the collectionTimeMilliseconds field. Once the first message has been received, the translator will create an external buffer and export the fixed templates to the fixbuf session. Note: the first sFlow message that fixbuf receives will not be processed - this is used to setup the translation process. The translator will keep track of sequence numbers per peer (IP)/observation domain (agent ID) by default. There are multiple sequence numbers in sFlow. Each sFlow message has a sequence number and each sample has a sequence number. The sFlow message sequence number is used to determine if sFlow messages have been dropped. Fixbuf will report if either sequence number is out of sequence and emit a warning. The warning is just for notification, libfixbuf will process all well-formed samples that it receives.
libfixbuf will process Flow Samples (1), Extended Flow Samples (3), Counter Samples (2), and Extended Counter Samples (4). Any other format will return an FB_ERROR_SFLOW. Applications should ignore (and potentially log) FB_ERROR_SFLOW errors. FB_ERROR_SFLOW errors are not fatal. With an sFlow sample, fixbuf can handle the following formats:
- Raw Packet Data, enterprise = 0, format = 1
- Ethernet Frame Data, enterprise = 0, format = 2
- IPv4 Data, enterprise = 0, format = 3
- IPv6 Data, enterprise = 0, format = 4
- Extended Switch data, enterprise = 0, format = 1001
- Extended Router data, enterprise = 0, format = 1002
- Extended Gatway Data, enterprise = 0, format = 1003
Any other flow sample format will be silently ignored. Each sFlow flow record can contain the following fields, formats are listed in the parenthesis:
| IPFIX FIELDS | sFlow FIELDS | Reduced Length |
|---|---|---|
| sourceIPv6Address | Ipv6 Address in IPv6 (4) or Raw Packet (1) Data | N |
| destinationIPv6Address | Ipv6 Address in IPv6 (4) or Raw Packet (1) Data | N |
| ipNextHopIPv6Address | Extended Router Data (1002) | N |
| bgpNextHopIPv6Address | Extended Gateway Data (1003) | N |
| collectorIPv6Address | Message Header Data | N |
| collectionTimeMilliseconds | Message Header Data | N |
| systemInitTimeMilliseconds | Message Header Data | N |
| collectorIPv4Address | Message Header Data | N |
| protocolIdentifier | IPv4 (3) or IPv6 (4) or Raw Packet (1) Data | N |
| ipClassOfService | IPv4 (3) or IPv6 (4) or Raw Packet (1) Data | N |
| sourceIPv4PrefixLength | Extended Router Data (1002) | N |
| destinationIPv4PrefixLength | Extended Router Data (1002) | N |
| sourceIPv4Address | IPv4 (3) or Raw Packet (1) Data | N |
| destinationIPv4Address | IPv4 (3) or Raw Packet (1) Data | N |
| octetTotalCount | Flow Sample Header Data | 4 |
| packetTotalCount | Flow Sample Header Data | 4 |
| ingressInterface | Flow Sample Header Data | N |
| egressInterface | Flow Sample Header Data | N |
| sourceMacAddress | Ethernet (2), IPv4 (3), IPv6 (4) or Raw Packet (1) Data | N |
| destinationMacAddress | Ethernet (2), IPv4 (3), IPv6 (4) or Raw Packet (1) Data | N |
| ipNextHopIPv4Address | Extended Router Data (1002) | N |
| bgpSourceAsNumber | Extended Gateway Data (1003) | N |
| bgpDestinationAsNumber | Extended Gateway Data (1003) | N |
| bgpNextHopIPv4Address | Extended Gateway Data (1003) | N |
| samplingPacketInterval | Message Header Data | N |
| samplingPopulation | Message Header Data | N |
| droppedPacketTotalCount | Message Header Data | 4 |
| selectorId | Message Header Data | 4 |
| vlanId | IPv4 (3) or IPv6 (4) or Raw Packet (1) Data | N |
| sourceTransportPort | IPv4 (3) or IPv6 (4) or Raw Packet (1) Data | N |
| destinationTransportPort | IPv4 (3) or IPv6 (4) or Raw Packet (1) Data | N |
| tcpControlBits | IPv4 (3) or IPv6 (4) or Raw Packet (1) Data | 2 |
| dot1qVlanId | Extended Switch Data (1001) | N |
| postDot1qVlanId | Extended Switch Data (1001) | N |
| dot1qPriority | Extended Switch Data (1001) | N |
libfixbuf will also convert sFlow Counter Records to Options Records in IPFIX. libfixbuf will only process the Generic Interface Counters (format = 1). Other formats will be silently ignored. The following fields are present in the Counter (Options) Template/Record:
| IPFIX FIELDS | sFlow FIELDS | Reduced Length |
|---|---|---|
| collectorIPv6Address | Message Header Data | N |
| collectionTimeMilliseconds | Message Header Data | N |
| systemInitTimeMilliseconds | Message Header Data | N |
| collectorIPv4Address | Message Header Data | N |
| ingressInterface | Counter Sample Header Data | N |
| octetTotalCount | ifINOctets (1) | N |
| ingressInterfaceType | ifType (1) | N |
| packetTotalCount | ifInUcastPkts (1) | 4 |
| ingressMulticastPacketTotalCount | ifInMulticastPkts (1) | 4 |
| ingressBroadcastPacketTotalCount | ifInBroadcastPkts (1) | 4 |
| notSentPacketTotalCount | ifInDiscards (1) | 4 |
| droppedPacketTotalCount | ifInErrors (1) | 4 |
| postOctetTotalCount | ifOutOctets (1) | N |
| ignoredPacketTotalCount | ifInUnknownProtos (1) | 4 |
| postPacketTotalCount | ifOutUcastPkts (1) | 4 |
| egressBroadcastPacketTotalCount | ifOutBroadcastPkts (1) | 4 |
| selectorId | Message Header Data | 4 |
fbCollectorGetSFlowMissed() can be used to retrieve the number of potential missed export packets. This is not the number of FLOW samples that the collector has missed. Fixbuf tries to account for any reboot of the device and not count large sequence number discrepancies in its missed count.
Fixbuf will return FB_ERROR_SFLOW if it tries to process any malformed samples.
Previous: NetFlow v9 Collectors | Next: Connection-less Collector
