This is documentation for Mothra, a collection of Scala and Spark
library functions for working with Internet-related data.
This is documentation for Mothra, a collection of Scala and Spark
library functions for working with Internet-related data. Some modules
contain APIs of general use to Scala programmers. Some modules make
those tools more useful on Spark data-processing systems.
Please see the documentation for the individual packages for more
details on their use.
Scala Packages
These packages are useful in Scala code without involving Spark:
The "junk drawer" of netsa-util so far provides only two features:
First, a method for equipping Scala scala.collection.Iterators with
exception handling. And second, a way to query the versions of NetSA
libraries present in a JVM at runtime.
Spark datasources for CERT file types. This package contains utility
features which add methods to Apache Spark
DataFrameReader objects,
allowing IPFIX and SiLK flows to be opened using simple spark.read...
calls.
The mothra-datasources library contains both IPFIX and SiLK
functionality, while mothra-datasources-ipfix and
mothra-datasources-silk contain only what's needed for the named
datasource.
This is documentation for Mothra, a collection of Scala and Spark library functions for working with Internet-related data. Some modules contain APIs of general use to Scala programmers. Some modules make those tools more useful on Spark data-processing systems.
Please see the documentation for the individual packages for more details on their use.
Scala Packages
These packages are useful in Scala code without involving Spark:
org.cert.netsa.data
This package, which is collected as the
netsa-datalibrary, provides types for working with various kinds of information:org.cert.netsa.data.net- types for working with network dataorg.cert.netsa.data.time- types for working with time dataorg.cert.netsa.data.unsigned- types for working with unsigned integral valuesorg.cert.netsa.io.ipfix
The
netsa-io-ipfixlibrary provides tools for reading and writing IETF IPFIX data from various connections and files.org.cert.netsa.io.silk
To read and write CERT NetSA SiLK file formats and configuration files, use the
netsa-io-silklibrary.org.cert.netsa.util
The "junk drawer" of
netsa-utilso far provides only two features: First, a method for equipping Scala scala.collection.Iterators with exception handling. And second, a way to query the versions of NetSA libraries present in a JVM at runtime.Spark Packages
These packages require the use of Apache Spark:
org.cert.netsa.mothra.datasources
Spark datasources for CERT file types. This package contains utility features which add methods to Apache Spark DataFrameReader objects, allowing IPFIX and SiLK flows to be opened using simple
spark.read...calls.The
mothra-datasourceslibrary contains both IPFIX and SiLK functionality, whilemothra-datasources-ipfixandmothra-datasources-silkcontain only what's needed for the named datasource.org.cert.netsa.mothra.analysis
A grab-bag of analysis helper functions and example analyses.
org.cert.netsa.mothra.functions
This single Scala object provides Spark SQL functions for working with network data. It is the entirety of the
mothra-functionslibrary.