Template Definition

How-To Define Templates

As mention previously, you must define an internal and external fbTemplate_t to describe each record you want to write, and you must define an internal Template that describes each record you want to read (the external Templates are defined by the incoming data).

An internal Template represents the layout of the record in your application, and the external Template represents the layout of the record in the IPFIX stream. The internal and external Templates do not have to match. libfixbuf copies the common fields, ignores extra fields in the source Template, and zeroes extra fields in the destination.

fbTemplateAlloc() allocates a Template. Typically one uses fbTemplateAppendSpecArray() or fbTemplateAppendArraySpecId() to define the information elements in a Template. libfixbuf uses the fbTemplateField_t type to represent an information element that appears in a Template.

fbSessionAddTemplate() and fbSessionAddTemplatesForExport() add the Template to an fbSession_t. Templates use internal reference counting, so they may be added to multiple Sessions, to the same Session using multiple template IDs or multiple observation domains, or as both an internal and an external Template on the same session.

The layout of the template usually matches the C struct that holds the record. Any gaps (due to alignment) in the C struct must be noted in both the data structure and the template.

Example code:

    fbInfoElementSpec_t collectTemplate[] = {
        {"flowStartMilliseconds",               8, 0 },
        {"flowEndMilliseconds",                 8, 0 },
        {"sourceIPv4Address",                   4, 0 },
        {"destinationIPv4Address",              4, 0 },
        {"sourceTransportPort",                 2, 0 },
        {"destinationTransportPort",            2, 0 },
        {"protocolIdentifier",                  1, 0 },
        {"paddingOctets",                       3, 0 },
        {"packetTotalCount",                    8, 0 },
        {"octetTotalCount",                     8, 0 },
        {"ipPayloadPacketSection",              0, 0 },
        FB_IESPEC_NULL
    };
    struct collectRecord_st {
        uint64_t      flowStartMilliseconds;
        uint64_t      flowEndMilliseconds;
        uint32_t      sourceIPv4Address;
        uint32_t      destinationIPv4Address;
        uint16_t      sourceTransportPort;
        uint16_t      destinationTransportPort;
        uint8_t       protocolIdentifier;
        uint8_t       padding[3];
        uint64_t      packetTotalCount;
        uint64_t      octetTotalCount;
        fbVarfield_t  payload;
    } collectRecord;
    tmpl = fbTemplateAlloc(model);
    if (!fbTemplateAppendSpecArray(tmpl, collectTemplate, ~0, &err))
        FATAL(err);
    if (!(tid = fbSessionAddTemplate(
              session, TRUE, FB_TID_AUTO, tmpl, NULL, &err)))
        FATAL(err);

The argument to fbTemplateAppendSpecArray() is an array of fbInfoElementSpec_t definitions. Typically the array is a static definition. The fbInfoElementSpec_t includes the name of the fbInfoElement_t, the length for the element in this template, and a flags value. The argument to fbTemplateAppendArraySpecId() is an array of fbInfoElementSpecId_t definitions. An fbInfoElementSpecId_t is similar to an fbInfoElementSpec_t except it uses the element's numeric identifier and private enterprise number (PEN).

The flags value allows creation of multiple Templates from a single fbInfoElementSpec_t array. The Template-building functions fbTemplateAppendSpecArray(), fbTemplateAppendSpec(), fbTemplateAppendArraySpecId(), and fbTemplateAppendSpecId() as well as the Template-querying function fbTemplateContainsAllFlaggedElementsByName() take a parameter named wantedFlags. The two values are used as follows:

• If an fbInfoElementSpec_t's or fbInfoElementSpecId_t flags is 0, the spec is always included in the Template.
• When a spec's flags value is non-zero, the spec is used only if the spec's flags intersected with (bit-wise AND) the function's wantedFlags parameter equals the spec's flags. That is, the high bits of the wantedFlags parameter must include all the high bits of the spec's flags.

In general, the flags member of an fbInfoElementSpec_t or fbInfoElementSpecId_t should have few high bits. The functions' wantedFlags parameter generally contains multiple high bits to include multiple specs. Consider this spec array:

fbInfoElementSpec_t spec_array[] = {
    {"protocolIdentifier",       1,  0},
    {"paddingOctets",            3,  8},
    {"sourceTransportPort",      2,  1},
    {"destinationTransportPort", 2,  1},
    {"paddingOctets",            5, 16},
    {"icmpTypeIPv4",             1,  2},
    {"icmpCodeIPv4",             1,  2},
    {"icmpTypeIPv6",             1,  4},
    {"icmpCodeIPv6",             1,  4},
    FB_IESPEC_NULL
};

The protocol is included in every template since its flags value is 0. The ports are included when the wantedFlags parameter has a 1 in bit position 1. When building a template that maps to the following C struct (where padding is necessary for member alignment), wantedFlags should be 9 (1 | 8).

struct proto_ports = {
    uint8_t   protocol;
    uint8_t   padding[3];
    uint16_t  source_port;
    uint16_t  dest_port;
};

For ICMP records, wantedFlags should have a 1 in position 2 or 4 depending on whether the record is IPv4 or IPv6, respectively. To map the template to the following struct, wantedFlags should be 18 (2 | 16) for IPv4 and 20 (4 | 16) for IPv6.

struct proto_ports = {
    uint8_t   protocol;
    uint8_t   padding[5];
    uint8_t   icmp_type;
    uint8_t   icmp_code;
};

When writing these records, including the padding would make them larger than necessary. You could eliminate the padding by defining external Templates that do not include it: Use 1, 2, or 4 for the wantedFlags parameter to fbSessionAddTemplate(). (In libfixbuf-3.0.0, you may also use fbTemplateCopy() or fbSessionAddTemplatesForExport().)

To query whether an incoming template matches these structs, use fbTemplateContainsAllFlaggedElementsByName(). To avoid testing for the presence of the padding, use 1, 2, or 4 for its wantedFlags parameter.

If you pass 0 as the value for wantedFlags, you only get elements where the flags value of the fbInfoElementSpec_t is 0. To get all elements, ensure all bits of wantedFlags are high by passing 0xffffffff, UINT32_MAX, or ~0.

Previous: IPFIX File Collectors | Next: Network Collectors