class ExportStream extends AnyRef
ExportStream supports writing IPFIX templates and records to a file.
The caller creates an ExportStream instance with an output channel and an associated Session. The caller may then add Templates or Records to the ExportStream, and the ExportStream writes the IPFIX data to the output file.
Since the Session is part of the ExportStream's constructor, all IPFIX Messages in the stream use the same observation domain.
By default, each exported IPFIX Message uses the current time
as its timestamp. Set the makeTimestamp
variable to change this
behavior.
By default, an ExportStream adds an options record to describe
each Information Element that is not part of the
standard information model, and ExportStream ignores any such
options records in the input stream. Set the describeElements
variable to false
to change this behavor, and use
elementDescriptionTID
to view or set the template ID used by
these records.
By default, an ExportStream adds an options record to describe
each Template it exports, and ExportStream ignores any such
options records in the input stream. Set the describeTemplates
variable to false
to change this behavor, and use
templateDescriptionTID
to view or set the template ID used by
these records.
NOTE: When copying data from an IPFIX input stream, ExportStream may change the order in which templates and records appear---specifically, templates that appear after a record in the input may be written before that record. ExportStream is aware of the Meta-Data Options templates and records that describe Information Elements and Templates, and, when ExportStream is not creating and exporting those items itself, those templates and records are copied from the input and written to the output in the required order. The user may call flush() to ensure some items appear in the output before others. The authors of this package may need to consider providing some way for the users to know when to call flush(), such as adding a callback methods to be invoked when a new IpfixSet is read.
- Alphabetic
- By Inheritance
- ExportStream
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new ExportStream(out: WritableByteChannel, session: Session)
- out
Where to write the IPFIX data.
- session
The session to use.
- Attributes
- protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def add(reader: MessageReader): ExportStream
Add all records from a MesageReader to this ExportStream
- def add(record: Record): ExportStream
Adds
record
to the output stream.Adds
record
to the output stream. Prior to addingrecord
, adds any Templates used byrecord
that have not been seen before.If
record
appears to contain either information element meta-data or template meta-data, then the record is ignored if the Export Stream is exporting that meta-data itself (see thedescribeElements
anddescribeTemplates
variables). - def add(template: Template): ExportStream
Adds
template
to the output stream unless the template has already been seen by this stream.Adds
template
to the output stream unless the template has already been seen by this stream.If
template
appears to be a template for describing either information element meta-data or template meta-data, then the template is ignored if the Export Stream is exporting that meta-data itself (see thedescribeElements
anddescribeTemplates
variables). - final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def close(): Unit
Flushes and closes the output stream.
- def dataWrittenCallback: Option[DataWrittenCallback]
Gets the current callback invoked when bytes are written to the outputStream as an Option.
Gets the current callback invoked when bytes are written to the outputStream as an Option.
- Since
1.3.1
- def dataWrittenCallback_=(callback: DataWrittenCallback): Unit
Sets a callback to be invoked when bytes are written to the outputStream.
Sets a callback to be invoked when bytes are written to the outputStream.
- Since
1.3.1
- var describeElements: Boolean
Whether to export option records that describe private enterprise information elements.
Whether to export option records that describe private enterprise information elements.
If this is
true
, ExportStream generates these records itself and any such records given to theadd
method are ignored. The records are generated as-needed and may appear anywhere in the output.If this is
false
, input records that describe private enterprise elements are written to the output without ExportStream examining their content. This may result in duplicate descriptions in the output.In either case, ExportStream attempts to use the same template ID for the element metadata template as seen in the incoming data.
- var describeTemplates: Boolean
Whether to export option records that include the name and description of each template.
Whether to export option records that include the name and description of each template.
If this is
true
, ExportStream generates these records itself and any such records given to theadd
method are ignored. The records are generated as-needed and may appear anywhere in the output.If this is
false
, input records that describe templates are written to the output without ExportStream examining their content. This may result in duplicate descriptions in the output.In either case, ExportStream attempts to use the same template ID for the template metadata template as seen in the incoming data.
- def elementDescriptionTID: Option[Int]
Returns the template ID for the options template used by records that describe private enterprise information elements.
Returns the template ID for the options template used by records that describe private enterprise information elements. Returns scala.None if no ID has been assigned.
- def elementDescriptionTID_=(tid: Int): Unit
Sets the template ID for the options template used by records that describe private enterprise information elements and exports the template to ensure the template ID is reserved.
Sets the template ID for the options template used by records that describe private enterprise information elements and exports the template to ensure the template ID is reserved.
- Exceptions thrown
IllegalTemplateRecordException
whentid
is illegal or is already in use by the session.java.lang.RuntimeException
whentid
cannot be used for the template because the template already exists.
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- def flush(): ExportStream
Flushes any data currently sitting in memory.
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- var knownTmpl: Set[Template]
The set of templates that have been exported
- var makeTimestamp: MessageTimestamp
Callback to generate a timestamp
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def outputStream: WritableByteChannel
Returns the file channel to which the data is being written
- def recordCount: Int
Returns the number of data records added to the stream.
- val session: Session
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def templateCount: Int
Returns the number of templates added to the stream.
- def templateDescriptionTID: Option[Int]
Returns the template ID for the options template used by records that describe templates.
Returns the template ID for the options template used by records that describe templates. Returns scala.None if no ID has been assigned.
- def templateDescriptionTID_=(tid: Int): Unit
Sets the template ID for the options template used by records that describe templates and exports the template to ensure the template ID is reserved.
Sets the template ID for the options template used by records that describe templates and exports the template to ensure the template ID is reserved.
- Exceptions thrown
IllegalTemplateRecordException
whentid
is illegal or is already in use by the session.java.lang.RuntimeException
whentid
cannot be used for the template because the template already exists.
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
This is documentation for Mothra, a collection of Scala and Spark library functions for working with Internet-related data. Some modules contain APIs of general use to Scala programmers. Some modules make those tools more useful on Spark data-processing systems.
Please see the documentation for the individual packages for more details on their use.
Scala Packages
These packages are useful in Scala code without involving Spark:
org.cert.netsa.data
This package, which is collected as the
netsa-data
library, provides types for working with various kinds of information:org.cert.netsa.data.net
- types for working with network dataorg.cert.netsa.data.time
- types for working with time dataorg.cert.netsa.data.unsigned
- types for working with unsigned integral valuesorg.cert.netsa.io.ipfix
The
netsa-io-ipfix
library provides tools for reading and writing IETF IPFIX data from various connections and files.org.cert.netsa.io.silk
To read and write CERT NetSA SiLK file formats and configuration files, use the
netsa-io-silk
library.org.cert.netsa.util
The "junk drawer" of
netsa-util
so far provides only two features: First, a method for equipping Scala scala.collection.Iterators with exception handling. And second, a way to query the versions of NetSA libraries present in a JVM at runtime.Spark Packages
These packages require the use of Apache Spark:
org.cert.netsa.mothra.datasources
Spark datasources for CERT file types. This package contains utility features which add methods to Apache Spark DataFrameReader objects, allowing IPFIX and SiLK flows to be opened using simple
spark.read...
calls.The
mothra-datasources
library contains both IPFIX and SiLK functionality, whilemothra-datasources-ipfix
andmothra-datasources-silk
contain only what's needed for the named datasource.org.cert.netsa.mothra.analysis
A grab-bag of analysis helper functions and example analyses.
org.cert.netsa.mothra.functions
This single Scala object provides Spark SQL functions for working with network data. It is the entirety of the
mothra-functions
library.