Monitoring for Large-Scale Networks

SiLK

silk-plugin

NAME

silk-plugin - Creating a SiLK run-time plug-in using C

SYNOPSIS

sk_cc=`silk_config --compiler`
sk_cflags=`silk_config --cflags`
$sk_cc $sk_cflags -shared -o FILENAME.so FILENAME.c

rwfilter --plugin=FILENAME.so [--plugin=FILENAME.so ...] ...

rwcut --plugin=FILENAME.so [--plugin=FILENAME.so ...]
      --fields=FIELDS ...

rwgroup --plugin=FILENAME.so [--plugin=FILENAME.so ...]
      --id-fields=FIELDS ...

rwsort --plugin=FILENAME.so [--plugin=FILENAME.so ...]
      --fields=FIELDS ...

rwstats --plugin=FILENAME.so [--plugin=FILENAME.so ...]
      --fields=FIELDS --values=VALUES ...

rwuniq --plugin=FILENAME.so [--plugin=FILENAME.so ...]
      --fields=FIELDS --values=VALUES ...

DESCRIPTION

Several of the SiLK analysis tools allow the user to augment the tools' functionality through the use of plug-ins that get loaded at run-time. These tools are:

rwfilter(1): Supports adding new switches to determine whether each SiLK Flow record should be written in the --pass or the --fail output stream.
rwcut(1): Supports adding new output fields that, when selected using the --fields switch, appear as a column in the output.
rwsort(1): Supports adding new key fields that, when selected using the --fields switch, are used to determine the order in which records are sorted.
rwgroup(1): Supports adding new key fields that, when selected using the --id-fields switch, are used to determine how records are grouped.
rwuniq(1): Supports adding new key fields that, when selected using the --fields switch, are used to bin (i.e., group) the records. In addition, rwuniq supports adding new aggregate value fields that, when selected using the --values switch, will be computed for each bin. The key and value fields will appear in the output.
rwstats(1): Supports adding new key fields that, when selected using the --fields switch, are used to bin (i.e., group) the records. In addition, rwstats supports adding new aggregate value fields that, when selected using the --values switch, will be computed for each bin and can be used to determine the top-N (or bottom-N) bins. The key and value fields will appear in the output for bins that meet the top-N threshold.
rwptoflow(1): Supports adding functionality to ignore packets in the pcap(3) input stream or to modify the SiLK Flow records as the records are generated.

In addition, all of the above tools support adding new command line switches that can be used to initialize the plug-in itself (for example, to load an auxiliary file that the plug-in requires).

The plug-ins for all tools except rwptoflow can be written in either C or using PySiLK (the SiLK Python extension, see pysilk(3)). Although the execution time for PySiLK plug-ins is slower than for C plug-ins, we encourage you to use PySiLK for your plug-ins since the time-to-result can be faster for PySiLK: The faster development time in Python typically more than compensates for the slower execution time. Once you find that your PySiLK plug-in is seeing a great deal of use, or that PySiLK is just too slow for the amount of data you are processing, then re-write the plug-in using C. Even when you intend to write a plug-in using C, it can be helpful to prototype your plug-in using PySiLK.

The remainder of this document explains how to create a plug-in for the SiLK analysis tools (except rwptoflow) using the C programming language. For information on creating a plug-in using PySiLK, see silkpython(3).

A template file for plug-ins is included in the SiLK source tree, in the silk-VERSION/src/template/c-plugin.c file.

The setup function

When you provide --plugin=my-plugin.so on the command line to an application, the application loads the my-plugin.so file and calls a setup function in that file to determine the new switches and/or fields that my-plugin.so provides.

This setup function is called with three arguments: the first two describe the version of the plug-in API, and the third is a pointer that is currently unused.

skplugin_err_t SKPLUGIN_SETUP_FN(
    uint16_t    major_version,
    uint16_t    minor_version,
    void       *plug_in_data)
{
    ...
}

There are several tasks this setup function may do: (1) check the API version, (2) register new command line switches (if any), (3) register new filters (if any), and (4) register new fields (if any). Let's describe these in more detail.

(1) Check the API version

The setup function should ensure that the plug-in and the application agree on the API to use. This provides protection in case the SiLK API to plug-ins changes in the future. To make this determination, call the skpinSimpleCheckVersion() function. A typical invocation is shown here, where the major_version and minor_version were passed into the SKPLUGIN_SETUP_FN, and PLUGIN_API_VERSION_MAJOR and PLUGIN_API_VERSION_MINOR are macros defined in the template file to the current version of the API.

#define PLUGIN_API_VERSION_MAJOR 1
#define PLUGIN_API_VERSION_MINOR 0

/* Check the plug-in API version */
rv = skpinSimpleCheckVersion(major_version, minor_version,
                             PLUGIN_API_VERSION_MAJOR,
                             PLUGIN_API_VERSION_MINOR,
                             skAppPrintErr);
if (rv != SKPLUGIN_OK) {
    return rv;
}

(2) Register command line switches

If the plug-in wants to define new command line switches, those switches must be registered in the setup function. A typical use of a command line switch is to allow the user to configure the plug-in; for example, the switch may allow the user to specify the location of an auxiliary input file that the plug-in requires, or to set a parameter used by the plug-in.

A second use for a command line switch is more subtle. When creating a plug-in for rwfilter, you may want your plug-in to provide several similar features, and only enable each feature when the user requests it via a command line switch. For this case, you want to delay registering the filter until the command line switch is seen, in which case the filter registration function should be invoked in the switch's callback function.

Information on registering a command line switch is available below ("Registering command line switches").

(3) Register filters

You only need to register filters when the plug-in will be used by rwfilter(1). You may choose to register the filters in the setup function; if you do, the filter will always be used when the plug-in is loaded by rwfilter. If you the plug-in provides several filtering functions that the user may choose from via command line switches, you should call the filter registration function in the callback function for the command line switch.

See "Registering filter functions" for details on registering a function to use with rwfilter.

(4) Register fields

If you want your plug-in to create a new printable field for rwcut(1), a new sorting field for rwsort(1), a new grouping field for rwgroup(1), rwstats(1), or rwuniq(1), or a new aggregate value field for rwstats or rwuniq, you should register those fields in the setup function. (While you can register the fields in a switch's callback function, there is usually little reason to do so.)

There are two interfaces to registering a new field:

The advanced interface provides complete control over how the field is defined, and allows (or forces) you to specify exactly how to map from a SiLK Flow record to a binary representation to a textual representation. To use the advanced interface you will need to define several functions and fill in a C structure with pointers to those functions. This interface is described in the "Advanced field registration function" section below.
The simple interface can be used to define fields that map to an integer value, an IP address, or text that is index by an integer value. To use this interface, you need to define only one or two functions. The simple interface should handle many common cases, and it is described in "Simple field registration functions".

Registering command line switches

When you register a switch, the two important pieces of information you must provide are a name for the switch and a callback function. When the application encounters the command line switch registered by your plug-in, the application will invoke the callback function with the parameter that the user provided (if any) to the command line switch.

To register a command line switch, call the skpinRegOption2() function:

skplugin_err_t skpinRegOption2(
    const char           *option_name,
    skplugin_arg_mode_t   mode,
    const char           *option_help_string,
    skplugin_help_fn_t    option_help_fn,
    skplugin_option_fn_t  opt_process_fn,
    void                 *opt_callback_data,
    int                   num_fn_mask,
    ...);  /* list of skplugin_fn_mask_t */

The parameters are

option_name

Specifies the command line switch to create. Do not include the leading -- characters in the name.

mode

Determines whether the switch takes an argument. It should be one of

NO_ARG: when the command line option acts as an on/off switch
OPTIONAL_ARG: when the command line option has a default value, or
REQUIRED_ARG: when the user of the plug-in must provide an argument to the command line option.

option_help_string

This parameter specifies the usage string to print when the user requests --help from the application. This parameter may be NULL. Alternatively, you may instruct the application to generate a help string by invoking a callback function your plug-in provides, as described next.

option_help_fn

This parameter specifies a pointer to a function that the application will to call to print a help message for the command line switch when the user requests --help from the application. This parameter may be NULL; if it is not NULL, the option_help_string value is ignored. The signature of the function to provide is

void option_help_fn(
    FILE                *file_handle,
    const struct option *option,
    void                *opt_callback_data);

The file_handle argument is where the function should print its help message. The opt_callback_data is the value provided to skpinRegOption2() when the option was registered. The struct option parameter has two members of interest: name contains the number used to register the option, and has_arg contains the mode that was used when the option was specified.

opt_process_fn

Specifies the callback function, whose signature is

skplugin_err_t opt_process_fn(
    const char *opt_arg,
    void       *opt_callback_data);

The application will call opt_process_fn(opt_arg,opt_callback_data) when --option_name is seen as a command line argument. opt_arg will be the parameter the user passed to the switch, or it will be NULL if no parameter was given.

opt_callback_data

Will be passed back unchanged to the plug-in as a parameter in the opt_process_fn() and option_help_fn() callback functions.

num_fn_mask

Specifies the number of skplugin_fn_mask_t values specified as the final argument(s) to skpinRegOption2().

...

Specifies a list of skplugin_fn_mask_t values. The length of this list must be specified in the num_fn_mask parameter. A plug-in file (e.g., my-plugin.so) can be loaded into any SiLK tool that supports plug-ins, but you may want a command line switch to appear only in certain applications. For example, the flowrate(3) plug-in can be used in both rwfilter and rwcut. When used by rwfilter, flowrate provides a --bytes-per-second switch; when used by rwcut, that switch is not available, and instead the bytes/sec field becomes available. This list determines in which applications the switch gets defined, and the list should contain the SKPLUGIN_FN_* or SKPLUGIN_APP_* macros defined in skplugin.h. To make the switch available in all applications, specify SKPLUGIN_FN_ANY. When skpinRegOption2() is called in an the application that does not match a value in this list, the function returns SKPLUGIN_ERR_DID_NOT_REGISTER, indicating that this option is not applicable to the application.

Registering filter functions

When you register a filter function, you are specifying a function that rwfilter will call for every SiLK Flow record that rwfilter reads from its input files. If the function returns SKPLUGIN_FILTER_PASS, rwfilter writes the record into the stream(s) specified by --pass. The record goes to the --fail streams if the function returns SKPLUGIN_FILTER_FAIL.

(The previous paragraph is true only when the plug-in is the only filtering predicate. When multiple tests are specified on the rwfilter command line, rwfilter will put the record into the fail destination as soon as any test fails. If there are multiple tests, your plug-in function will only see records that have not yet failed a test. If a plug-in filter function follows your function, it may fail a record that your filter function passed.)

To register a filter function, call the following function:

skplugin_err_t skpinRegFilter(
    skplugin_filter_t         **filter_handle,
    const skplugin_callbacks_t *regdata,
    void                       *cbdata);

filter_handle

When this parameter is not NULL, skpinRegFilter() will set the location it references to the newly created filter. Currently, no other function accepts the skplugin_filter_t as an argument.

cbdata

This parameter will be passed back unchanged to the plug-in as a parameter in the various callback functions. It may be NULL.

regdata

This structure has a member for every possible callback function the SiLK plug-in API supports. When used by skpinRegFilter(), the following members are supported.

filter

rwfilter invokes this function for each SiLK flow record. If the function returns SKPLUGIN_FILTER_PASS, the record is accepted; if it returns SKPLUGIN_FILTER_FAIL, the record is rejected. The type of the function is a skplugin_filter_fn_t, and its signature is:

skplugin_err_t filter(
    const rwRec  *rec,
    void         *cbdata,
    void        **extra);

where rec is the SiLK Flow record, cbdata is the cbdata specified in skpinRegFilter(), and extra will likely be unused.

init

rwfilter invokes this function for all registered filter predicates. It is called after argument processing and before reading records. The function's type is skplugin_callback_fn_t and the function pointer may be NULL. The callback's signature is

skplugin_err_t init(
    void *cbdata);

cleanup

When this function pointer is non-NULL, rwfilter calls this function after all records have been processed. This function has the same type and signature as the init function.

The function's return value will be SKPLUGIN_OK unless the filter member of the regdata structure is NULL.

If your plug-in registers a filter function and the plug-in is used in an application other that rwfilter, the call to skpinRegFilter() is a no-op.

Simple field registration functions

Using a plug-in, you can augment the keys available in the --fields switch on rwcut(1), rwgroup(1), rwsort(1), rwstats(1), and rwuniq(1), and provide new aggregate value fields for the --values switch on rwstats and rwuniq.

The standard field registration function, skpinRegField(), is powerful---for example, you can control exactly how the value you compute will be printed. However, that power comes with complexity. Many times, all your plug-in needs to do is to compute a value, and having to write a function to print a number is work with little reward. The functions in this section handle the registration of common field types.

All of these functions require a name for the new field. The name is used as one of the arguments to the --fields or --values switch, and the name will also be used as the title when the field is printed (as in rwcut). Field names are case insensitive, and all field names must be unique within an application. You will get a run-time error if you attempt to create a field whose name already exists. (In rwuniq and rwstats, you may have a --fields key and a --values aggregate value with the same name.)

The callback functions dealing with integers use uint64_t for convenience, but internally the value will be stored in a smaller integer field if possible. Specifying the max parameter to the largest value you actually use may allow SiLK to use a smaller integer field.

The functions in this section return SKPLUGIN_OK unless the callback function is NULL.

Integer key field

The following function is used to register a key field whose value is an unsigned 64 bit integer.

skplugin_err_t skpinRegIntField(
    const char              *name,
    uint64_t                 min,
    uint64_t                 max,
    skplugin_int_field_fn_t  rec_to_int,
    size_t                   width);

name

The name of the new key field.

min

A number representing the minimum integer value for the field.

max

A number representing the maximum integer value for the field. If max is 0, a value of UINT64_MAX is used instead.

rec_to_int

A callback function that accepts a SiLK Flow record as its sole argument, and returns an unsigned integer (in host byte order) which represents the value of the name field for the given record. The signature is

uint64_t rec_to_int(
    const rwRec *rec);

width

The column width to use when displaying the field. If width is 0, it will be computed to be the number of digits necessary to display the integer max.

IPv4 key field

The following function registers a new key field whose value is an IPv4 address.

skplugin_err_t skpinRegIPv4Field(
    const char               *name,
    skplugin_ipv4_field_fn_t  rec_to_ipv4,
    size_t                    width);

name

The name of the new key field.

rec_to_ipv4

A callback function that accepts a SiLK Flow record as its sole argument, and returns a 32 bit integer (in host byte order) which represents the IPv4 addresses for the name field for the given record. The signature is

uint32_t rec_to_ipv4(
    const rwRec *rec);

width

The column width to use when displaying the field. If width is 0, it will be set to 15.

IP key field

The following function is used to register a key field whose value is any IP address (an skipaddr_t).

skplugin_err_t skpinRegIPAddressField(
    const char             *name,
    skplugin_ip_field_fn_t  rec_to_ipaddr,
    size_t                  width);

name

The name of the new key field.

rec_to_ipaddr

A callback function that accepts a SiLK Flow record and an skipaddr_t as arguments. The function should fill in the IP address as required for the name field. The signature is

void rec_to_ipaddr(
    skipaddr_t  *dest,
    const rwRec *rec);

width

The column width to use when displaying the field. If width is 0, it will be set to 39 when SiLK has support for IPv6 addresses, or 15 otherwise.

Text key field (from an integer)

The following function is used to register a key field whose value is an unsigned 64 bit integer (similar to skpinRegIntField()), but where the printed representation of the field is determined by a second callback function. This allows the plug-in to create arbitrary text for the field.

skplugin_err_t skpinRegTextField(
    const char               *name,
    uint64_t                  min,
    uint64_t                  max,
    skplugin_int_field_fn_t   value_fn,
    skplugin_text_field_fn_t  text_fn,
    size_t                    width);

name

The name of the new key field.

min

A number representing the minimum integer value for the field.

max

A number representing the maximum integer value for the field. If max is 0, a value of UINT64_MAX is used instead.

value_fn

uint64_t rec_to_int(
    const rwRec *rec);

text_fn

A callback function that provides the textual representation of the value returned by value_fn. The function's signature is

void text_fn(
    char     *dest,
    size_t    dest_len,
    uint64_t  val);

The callback should fill the character array dest with the printable representation of val. The number of characters in dest is given by dest_len. Note that dest_len may be different than the parameter width passed to skpinRegTextField(), and text_fn must NUL-terminate the string.

width

The column width to use when displaying the field.

Text key field (from a list)

The following function is used to register a field whose value is one of a list of strings. The plug-in provides the list of strings and a callback that takes a SiLK Flow record and returns an index into the list of strings.

skplugin_err_t skpinRegStringListField(
    const char               *name,
    const char              **list,
    size_t                    entries,
    const char               *default_value,
    skplugin_int_field_fn_t   rec_to_index,
    size_t                    width);

name

The name of the new key field.

list

List is the list of strings. The list should either be NULL terminated, or entries should have a non-zero value.

entries

The number of entries in list. If entries is 0, SiLK determines the number of entries by traversing list until it finds a element whose value is NULL.

default_value

The value to use when rec_to_index returns an invalid value.

rec_to_index

A callback function that accepts a SiLK Flow record as its sole argument, and returns an unsigned integer (in host byte order) which represents an index into list. If the return value is beyond the end of list, default_value will be used instead. The signature of this callback function is

uint64_t rec_to_int(
    const rwRec *rec);

width

The column width to use when displaying the field. If width is 0, it is defaulted to the width of the longest string in list and default_value.

Integer sum aggregate value field

The following function registers an aggregate value field that maintains a running unsigned integer sum. That is, the values returned by the callback are summed for every SiLK Flow record that matches a bin's key. The sum is printed when the bin is printed.

skplugin_err_t skpinRegIntSumAggregator(
    const char              *name,
    uint64_t                 max,
    skplugin_int_field_fn_t  rec_to_int,
    size_t                   width);

name

The name of the new aggregate value field.

max

A number representing the maximum integer value for the field. If max is 0, a value of UINT64_MAX is used instead.

rec_to_int

A callback function that accepts a SiLK Flow record as its sole argument, and returns an unsigned integer (in host byte order) which represents the value of the name value field for the given record. The signature is

uint64_t rec_to_int(
    const rwRec *rec);

width

The column width to use when displaying the value. If width is 0, it will be computed to be the number of digits necessary to display the integer max.

Integer minimum or maximum aggregate value field

The following function registers an aggregate value field that maintains the minimum integer value seen among all values returned by the callback function.

skplugin_err_t skpinRegIntMinAggregator(
    const char              *name,
    uint64_t                 max,
    skplugin_int_field_fn_t  rec_to_int,
    size_t                   width);

This function is similar, except it maintains the maximum value.

skplugin_err_t skpinRegIntMaxAggregator(
    const char              *name,
    uint64_t                 max,
    skplugin_int_field_fn_t  rec_to_int,
    size_t                   width);

name

The name of the new aggregate value field.

max

A number representing the maximum integer value for the field. If max is 0, a value of UINT64_MAX is used instead.

rec_to_int

uint64_t rec_to_int(
    const rwRec *rec);

width

The column width to use when displaying the value. If width is 0, it will be computed to be the number of digits necessary to display the integer max.

Unsigned integer aggregate value field

The following function registers an aggregate value field that can be represented by a 64 bit integer. The plug-in must register two callback functions. The first takes a SiLK Flow record and returns an integer value; the second takes two integer values (as returned by the first callback function) and combines them to form a new aggregate value.

skplugin_err_t skpinRegIntAggregator(
    const char              *name,
    uint64_t                 max,
    skplugin_int_field_fn_t  rec_to_int,
    skplugin_agg_fn_t        agg,
    uint64_t                 initial,
    size_t                   width);

name

The name of the new aggregate value field.

max

A number representing the maximum integer value for the field. If max is 0, a value of UINT64_MAX is used instead.

rec_to_int

uint64_t rec_to_int(
    const rwRec *rec);

agg

A callback function that combines (aggregates) two values. For example, if you wanted to create a new aggregate value that contained a bit-wise OR of the TCP flags seen on every packet, your agg function would OR the values. The signature is

uint64_t agg(
    uint64_t current,
    uint64_t operand);

initial

Specifies the initial value for the aggregate value. The first time the agg function is called on a bin, operand will be the value returned by rec_to_int, and current will be the value given in initial. The value in initial must be less than or equal to the value in max.

width

The column width to use when displaying the value. If width is 0, it will be computed to be the number of digits necessary to display the integer max.

Advanced field registration function

When the simple field registration functions do not provide what you need, you can use the skpinRegField() function that gives you complete control over the field.

skpinRegField() registers a new derived field for record processing. The plug-in must supply the name of the new field. The name is used as one of the arguments to the --fields switch (for key fields) or --values switch (for aggregate value fields). Field names are case insensitive, and all field names must be unique within an application. You will get a run-time error if you attempt to create a field whose name already exists. (In rwuniq and rwstats, you may have a --fields key and a --values aggregate value with the same name.)

The skpinRegField() function requires you initialize and pass in a structure. In this structure you will specify the callback functions that the application will call, as well as additional information required by some applications. Although the structure is complex, not all applications use all members.

If the plug-in is loaded by an application that does not support fields (such as rwfilter), the function is a no-op.

The advanced field registration function is

skplugin_err_t skpinRegField(
    skplugin_field_t          **return_field,
    const char                 *name,
    const char                 *description,
    const skplugin_callbacks_t *regdata,
    void                       *cbdata);

return_field: When this value is not NULL, skpinRegField() will set the location it references to the newly created field.
name: This sets the primary name of the field, and by default will be the title used when printing the field.
description: The description provides a textual description of the field. Currently this is unused.
regdata: The regdata structure provides the application with the callback functions and additional information it needs to use the plug-in. The members that must be set vary by application. It is described in more detail below.
cbdata: This parameter will be passed back unchanged to the plug-in as a parameter in the various callback functions. It may be NULL.

The structure used by the skpinRegField() (and skpinRegFilter()) functions to specify callback functions is shown here:

typedef struct skplugin_callbacks_st {
    skplugin_callback_fn_t     init;
    skplugin_callback_fn_t     cleanup;
    size_t                     column_width;
    size_t                     bin_bytes;
    skplugin_text_fn_t         rec_to_text;
    skplugin_bin_fn_t          rec_to_bin;
    skplugin_bin_fn_t          add_rec_to_bin;
    skplugin_bin_to_text_fn_t  bin_to_text;
    skplugin_bin_merge_fn_t    bin_merge;
    skplugin_bin_cmp_fn_t      bin_compare;
    skplugin_filter_fn_t       filter;
    skplugin_transform_fn_t    transform;
    const uint8_t             *initial;
    const char               **extra;
} skplugin_callbacks_t;

All of the callback functions reference in this structure take cbdata as a parameter, which is the value that was specified in the call to skpinRegField(). The extra parameter to the callback functions is used in complex plug-ins and can be ignored.

The members of the structure are:

init

This specifies a callback function which the application will call when it has determined this field will be used. (In the case of skpinRegFilter(), the function is called for all registered filters.) The application calls the function before processing data. It may be NULL; the signature of the callback function is

skplugin_err_t init(
    void *cbdata);

cleanup

When this callback function is not NULL, the application will call it after all records have been processed. It has the same signature as the init function.

column_width

The number of characters (not including trailing NUL) required to hold a string representation of the longest value of the field. This value can be 0 if not used (e.g., rwsort does not print fields), or if it will be set later using skpinSetFieldWidths().

bin_bytes

The number of bytes (octets) required to hold a binary representation of a value of the field. This value can be 0 if not used (e.g., rwcut does not use binary values), or if it will be set later using skpinSetFieldWidths().

rec_to_text

The rwcut application uses this callback function to fetch the textual value for the field given a SiLK Flow record. The signature of this function is

skplugin_err_t rec_to_text(
    const rwRec  *rec,
    char         *dest,
    size_t        width,
    void         *cbdata,
    void        **extra);

The callback function should fill the character array dest with the textual value, and the value should be NUL-terminated. width specifies the overall size of dest, and it may not have the same value as specified by the column_width member. For proper formatting, the callback function should write no more than column_width characters into dest. Note that if an application requires a rec_to_bin function and rec_to_bin is NULL, the application will use rec_to_text if it is provided. The application will use column_width as the width for binary values (zeroing out the destination area before it is written to).

rec_to_bin

This callback function is used by the application to fetch the binary value for this field given the SiLK Flow record. The signature of this function is:

skplugin_err_t rec_to_bin(
    const rwRec  *rec,
    uint8_t      *dest,
    void         *cbdata,
    void        **extra);

The callback function should write exactly bin_bytes of data into dest (where bin_bytes was specified in the call to skpinRegField() or skpinSetFieldWidths()). See also the rec_to_text member.

add_rec_to_bin

This callback function is used by rwuniq and rwstats when computing aggregate value fields. The application expects this function to get the binary value for this field from the SiLK Flow record and merge it (e.g., add it) to the current value. That is, the function should update the value in current_and_new_value with the value that comes from the current rec. The signature is:

skplugin_err_t add_rec_to_bin(
    const rwRec  *rec,
    uint8_t      *current_and_new_value,
    void         *cbdata,
    void        **extra);

The callback function should write exactly bin_bytes of data into current_and_new_value.

bin_to_text

This callback function is used to get a textual representation of a binary value that was set by a prior call to the rec_to_bin or add_rec_to_bin functions. The function signature is

skplugin_err_t bin_to_text(
    const uint8_t *bin,
    char          *dest,
    size_t         width,
    void          *cbdata);

The binary input value is in bin, and it is exactly bin_bytes in length. The textual output must be written to dest. The overall size of dest is given by width, which may be different than the column_width value that was previously specified. For proper formatting, the callback function should write no more than column_width characters into dest.

bin_merge

When rwstats and rwuniq are unable to store all values in memory, the applications write their current state to temporary files on disk. Once all input data has been processed, the temporary files are combined to produce the output. When a key appears in multiple temporary files, the aggregate values must be merged (for example, the byte count for two keys would be added). This callback function is used to merge aggregate value fields defined by the plug-in. The function signature is below. The src1_and_dest parameter will contain a binary aggregate value from one of the files, and the src2 parameter a value from the other. These should be combined and the (binary) result written to src1_and_dest. The byte length of both parameters is bin_bytes.

skplugin_err_t bin_merge(
    uint8_t        *src1_and_dest,
    const uint8_t  *src2,
    void           *cbdata);

bin_compare

This callback function is used by rwstats when determining the top-N (or bottom-N) bins based on the binary aggregate values. The function accepts two binary values, value_a and value_b, each of length bin_bytes. The function must set cmp_result to an integer less than 0, equal 0, or greater than 0 to indicate whether value_a is less than, equal to, or greater than value_b, respectively. If this function is NULL, memcmp() will be used on the binary values instead.

skplugin_err_t bin_compare(
    int            *cmp_result,
    const uint8_t  *value_a,
    const uint8_t  *value_b,
    void           *cbdata);

filter

This callback function is only required when the plug-in will be used by rwfilter, as described above. When defining a field, filter is ignored.

transform

This callback function is only required when the plug-in will be used by rwptoflow. This callback allows the plug-in to modify the SiLK Flow record, rec, before it is written to the output. The callback function should modify rec in place; the signature is

skplugin_err_t transform(
    rwRec  *rec,
    void   *cbdata,
    void  **extra);

initial

When the initial member is not NULL, it should point to a value containing at least bin_bytes bytes. These bytes will be used to initialize the binary aggregate value. As an example use case, when the plug-in is computing a minimum, it may choose to initialize the field to contain the maximum value. When initial is NULL, binary aggregate values are initialized using bzero().

extra

This member is usually NULL. When not NULL, it points to a NULL-terminated constant array of strings representing "extra arguments". These are not often used, and they will not be discussed in this manual page.

Once a field is registered, you may make changes to it by calling the additional functions described below. In each of these functions, the field parameter is the handle returned when the field was registered.

By default, the name will also be used as the field's title. To specify a different title, the plug-in may call

skplugin_err_t skpinSetFieldTitle(
    skplugin_field_t    field,
    const char          title);

To create an alternate name for the field (that is, a name that can be used in the --fields or --values switches) call

skplugin_err_t skpinAddFieldAlias(
    skplugin_field_t    field,
    const char          alias);

To set or modify the textual and binary widths for a field, use the following function. This function should called in the field's init callback function.

skplugin_err_t skpinSetFieldWidths(
    skplugin_field_t    field,
    size_t              field_width_text,
    size_t              field_width_bin);

The following table shows when a member of the skplugin_callbacks_t structure is required or optional. (Where the table shows column_width and bin_bytes as required, the values can be set in the structure or via the skpinSetFieldWidths() function.)

               rwfilter rwcut rwgroup rwsort rwstats rwuniq rwptoflow
init              r       f      f      f      f,a    f,a       r
cleanup           r       f      f      f      f,a    f,a       r
column_width      .       F      .      .      F,A    F,A       .
bin_bytes         .       .      F      F      F,A    F,A       .
rec_to_text       .       F      .      .       .      .        .
rec_to_bin        .       .      F      F       F      F        .
add_rec_to_bin    .       .      .      .       A      A        .
bin_to_text       .       .      .      .      F,A    F,A       .
bin_merge         .       .      .      .       A      A        .
bin_compare       .       .      .      .       A      .        .
initial           .       .      .      .       a      a        .
filter            R       .      .      .       .      .        .
transform         .       .      .      .       .      .        R
extra             r       f      f      f      f,a    f,a       r

The legend is

F: required for a key field
A: required for an aggregate value field
R: required for a non-field application (e.g., rwfilter)
f: optional for a key field
a: optional for an aggregate value field
r: optional for a non-field application
.: ignored

Miscellaneous functions

The following registers a cleanup function for the plug-in. This function will be called by the application after any field- or filter-specific cleanup functions are called. Specifically, this is the last callback that the application will invoke on a plug-in.

skplugin_err_t skpinRegCleanup(
    skplugin_cleanup_fn_t cleanup);

The signature of the cleanup function is:

void cleanup(void);

The plug-in author should invoke the following function to tell rwfilter that this plug-in is not thread safe. Calling this function causes rwfilter not use multiple threads; as such, this function should only be called when the plug-in has registered an active filter function.

void skpinSetThreadNonSafe(void);

Compiling the plug-in

Once you have finished writing the C code for the plug-in, save it in a file. The following uses the name my-plugin.c for the name of this file.

In the following, the leading dollar sign ($) followed by a space represents the shell prompt. The text after the dollar sign represents the command line. Lines have been wrapped for improved readability, and the back slash (\) is used to indicate a wrapped line.

When compiling a plug-in, you should use the same compiler and compiler-options as when SiLK was compiled. The silk_config(1) utility can be used to obtain that information. To store the compiler used to compile SiLK into the variable sk_cc, specify the following at a shell prompt (note that those are backquotes, and this assumes a Bourne-compatible shell):

$ sk_cc=`silk_config --compiler`

To get the compiler flags used to compile SiLK:

$ sk_cflags=`silk_config --cflags`

Using those two variables, you can now compile the plug-in. The following will work on Linux and Mac OS X:

$ $sk_cc $sk_cflags -shared -o my-plugin.so my-plugin.c

For Mac OS X:

$ $sk_cc $sk_cflags -bundle -flat_namespace -undefined suppress    \
       -o my-plugin.so my-plugin.c

If there are compilation errors, fix them and compile again.

Notes: The preceding assumed you were building the plug-in after having installed SiLK. The paths given by silk_config do not work if SiLK has not been installed. To compile the plug-in, you must have access to the SiLK header files. (If you are using an RPM installation of SiLK, ensure that the silk-devel RPM is installed.)

Once you have created the my-plugin.so file, you can load it into an application by using the --plugin switch on the application as shown in the "SYNOPSIS". When loading a plug-in from the current directly, it is best to prefix the filename with ./:

$ rwcut --plugin=./my-plugin.so ...

If there are problems loading the plug-in into the application, you can trace the actions the application is doing by setting the SILK_PLUGIN_DEBUG environment variable:

$ SILK_PLUGIN_DEBUG=1  rwcut --plugin=./my-plugin.so ...

EXAMPLES

rwfilter

Suppose you want to find traffic destined to a particular host, 10.0.0.23, that is either ICMP or coming from 1434/udp. If you attempt to use:

$ rwfilter --daddr=10.0.0.23 --proto=1,17 --sport=1434     \
       --pass=outfile.rw  flowrec.rw

the --sport option will not match any of the ICMP traffic, and your result will not contain ICMP records. To avoid having to use two invocations of rwfilter, you can create the following plug-in to do the entire check in a single pass:

#include <silk/silk.h>
#include <silk/rwrec.h>
#include <silk/skipaddr.h>
#include <silk/skplugin.h>
#include <silk/utils.h>

/* These variables specify the version of the SiLK plug-in API. */
#define PLUGIN_API_VERSION_MAJOR 1
#define PLUGIN_API_VERSION_MINOR 0

/* ip to search for */
static skipaddr_t ipaddr;

/*
 *  status = filter(rwrec, reg_data, extra);
 *
 *    The function should examine the SiLK flow record and return
 *    SKPLUGIN_FILTER_PASS to write the rwRec to the
 *    pass-destination(s) or SKPLUGIN_FILTER_FAIL to write it to the
 *    fail-destination(s).
 */
static skplugin_err_t filter(
    const rwRec    *rwrec,
    void           *reg_data,
    void          **extra)
{
    skipaddr_t dip;

    rwRecMemGetDIP(rwrec, &dip);
    if (0 == skipaddrCompare(&dip, &ipaddr)
        && (rwRecGetProto(rwrec) == 1
            || (rwRecGetProto(rwrec) == 17
                && rwRecGetSPort(rwrec) == 1434)))
    {
        return SKPLUGIN_FILTER_PASS;
    }
    return SKPLUGIN_FILTER_FAIL;
}

/* The set-up function that the application will call. */
skplugin_err_t SKPLUGIN_SETUP_FN(
    uint16_t    major_version,
    uint16_t    minor_version,
    void       *plug_in_data)
{
    uint32_t ipv4;
    skplugin_err_t rv;
    skplugin_callbacks_t regdata;

    /* Check the plug-in API version */
    rv = skpinSimpleCheckVersion(major_version, minor_version,
                                 PLUGIN_API_VERSION_MAJOR,
                                 PLUGIN_API_VERSION_MINOR,
                                 skAppPrintErr);
    if (rv != SKPLUGIN_OK) {
        return rv;
    }

    /* set global ipaddr */
    ipv4 =  ((10 << 24) | 23);
    skipaddrSetV4(&ipaddr, &ipv4);

    /* register the filter */
    memset(&regdata, 0, sizeof(regdata));
    regdata.filter = filter;
    return skpinRegFilter(NULL, &regdata, NULL);
}

Once this file is created and compiled, you can use it from rwfilter as shown here:

$ rwfilter --plugin=./my-plugin.so --pass=outfile.rw  flowrec.rw

Additional examples

For additional examples, see the source files in silk-VERSION/src/plugins.

ENVIRONMENT

SILK_PATH: This environment variable gives the root of the install tree. When searching for plug-ins, a SiLK application may use this environment variable. See the "FILES" section for details.
SILK_PLUGIN_DEBUG: When set to 1, the SiLK applications print status messages to the standard error as they attempt to find and open each plug-in. In addition, when an attempt to register a field fails, the application prints a message specifying the additional function(s) that must be defined to register the field in the application. Be aware that the output can be rather verbose.

FILES

${SILK_PATH}/lib64/silk/
${SILK_PATH}/lib64/
${SILK_PATH}/lib/silk/
${SILK_PATH}/lib/
/usr/lib64/silk/
/usr/lib64/
/usr/lib/silk/
/usr/lib/: Directories that a SiLK application checks when attempting to load a plug-in.