Packages

•  package root

  This is documentation for Mothra, a collection of Scala and Spark library functions for working with Internet-related data.

  This is documentation for Mothra, a collection of Scala and Spark library functions for working with Internet-related data. Some modules contain APIs of general use to Scala programmers. Some modules make those tools more useful on Spark data-processing systems.

  Please see the documentation for the individual packages for more details on their use.

  Scala Packages

  These packages are useful in Scala code without involving Spark:

  org.cert.netsa.data

  This package, which is collected as the netsa-data library, provides types for working with various kinds of information:

  • org.cert.netsa.data.net - types for working with network data
  • org.cert.netsa.data.time - types for working with time data
  • org.cert.netsa.data.unsigned - types for working with unsigned integral values

  org.cert.netsa.io.ipfix

  The netsa-io-ipfix library provides tools for reading and writing IETF IPFIX data from various connections and files.

  org.cert.netsa.io.silk

  To read and write CERT NetSA SiLK file formats and configuration files, use the netsa-io-silk library.

  org.cert.netsa.util

  The "junk drawer" of netsa-util so far provides only two features: First, a method for equipping Scala Iterators with exception handling. And second, a way to query the versions of NetSA libraries present in a JVM at runtime.

  Spark Packages

  These packages require the use of Apache Spark:

  org.cert.netsa.mothra.datasources

  Spark datasources for CERT file types. This package contains utility features which add methods to Apache Spark DataFrameReader objects, allowing IPFIX and SiLK flows to be opened using simple spark.read... calls.

  The mothra-datasources library contains both IPFIX and SiLK functionality, while mothra-datasources-ipfix and mothra-datasources-silk contain only what's needed for the named datasource.

  org.cert.netsa.mothra.analysis

  A grab-bag of analysis helper functions and example analyses.

  org.cert.netsa.mothra.functions

  This single Scala object provides Spark SQL functions for working with network data. It is the entirety of the mothra-functions library.

  Definition Classes

  root

•  package org

  Definition Classes

  root

•  package cert

  Definition Classes

  org

•  package netsa

  Definition Classes

  cert

•  package io

  Definition Classes

  netsa

•  package silk

  SiLK file formats, data types, and methods to read them, including support for reading them from Spark.

  SiLK file formats, data types, and methods to read them, including support for reading them from Spark.

  RWRec is the type of SiLK flow records.

  You can use RWRecReader to read SiLK files from Scala, including compressed files if Hadoop native libraries are available. For example:

  import org.cert.netsa.io.silk.RWRecReader
  import java.io.FileInputStream
  
  val inputFile = new FileInputStream("path/to/silk/rw/file")
  
  for ( rec <- RWRecReader.ofInputStream(inputFile) ) {
    println(rec.sIP)
  }

  Definition Classes

  io

  See also

  org.cert.netsa.mothra.datasources.silk.flow for working with SiLK data in Spark using the Mothra SiLK datasource.

•  object FileFormat extends ByteEnum[FileFormat]

  Definition Classes

  silk

• FT_ADDRESSES
• FT_BLOOM
• FT_EXPERIMENT1
• FT_EXPERIMENT2
• FT_FLOWCAP
• FT_GRAPH
• FT_IPFIX
• FT_IPSET
• FT_NIDSMAP
• FT_PDUFLOWCAP
• FT_PORTMAP
• FT_PREFIXMAP
• FT_RESERVED_09
• FT_RESERVED_0F
• FT_RESERVED_17
• FT_RWACL
• FT_RWAUGMENTED
• FT_RWAUGROUTING
• FT_RWAUGSNMPOUT
• FT_RWAUGWEB
• FT_RWBAG
• FT_RWCOUNT
• FT_RWDAILY
• FT_RWFILTER
• FT_RWGENERIC
• FT_RWIPV6
• FT_RWIPV6ROUTING
• FT_RWNOTROUTED
• FT_RWPRINTSTATS
• FT_RWROUTED
• FT_RWSCAN
• FT_RWSPLIT
• FT_RWWWW
• FT_SERVICEMAP
• FT_SHUFFLE
• FT_TAGTREE
• FT_TCPDUMP
• FT_TEMPFILE

org.cert.netsa.io.silk.FileFormat

FT_RWAUGWEB