Analysis Pipeline Release 5.11.4, 2022-Apr-7

Downloads

analysis-pipeline-5.11.4.tar.gz (2MiB)

(SHA256=71416d1f9738b28ba32a63ad05aff3c2d1e3a4ea357363804f77c9dc1d45fa1a)

Notes

  • Updated to allow compilation with fixbuf-2.0.0 or fixbuf-3.0.0. Notes:
    • Processing IPFIX produced by YAF 3 is experimental and not recommended.
    • Ensure the SchemaTools and SiLK dependencies use the same version of libfixbuf.
  • Fixed an issue on macOS that prevented configure from finding SiLK.
  • Updated CERT IPFIX elements to include those added in YAF 3. This update does not include the element name changes that were part of that release.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.17.0 or later with libfixbuf 2.0 or SiLK 3.19.3 or later with libfixbuf 3.0.
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.3 with libfixbuf 2.0 or SchemaTools 1.4 or later with libfixbuf 3.0.
  • fixbuf — Analysis Pipeline requires libfixbuf 2.4 or later
  • json-c — Recommended: Analysis Pipeline can produce JSON alerts when compiled with json-c 0.11 or later.
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.11.3, 2019-Sep-17

Downloads

analysis-pipeline-5.11.3.tar.gz (2MiB)

(SHA256=0b9bed44e92eb03a883cfc075a04a72da21c581d96513a6b2a1fd6ef7704a96d)

Notes

  • Fixed a bug with how HAS_CHANGED was parsed in the config file.
  • Fixed a bug where marking a DISTINCT eval inactive would cause errors.
  • Fixed a bug where CALC_STATS evals would not properly recover after overflowing.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.17.x or later (Due to libfixbuf 2.0)
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.3 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 2.4 or later
  • json-c — Recommended: Analysis Pipeline can produce JSON alerts when compiled with json-c 0.11 or later.
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.11.2, 2019-Jul-3

Downloads

analysis-pipeline-5.11.2.tar.gz (2MiB)

(SHA256=03a0a3be797756a96d1932c35941a29334a698415d788fe25f969ad649dc23de)

Notes

  • Fixed a bug in the Makefile logic that would cause a build failure on newer compilers.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.17.x or later (Due to libfixbuf 2.0)
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.3 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 2.0 or later
  • json-c — Recommended: Analysis Pipeline can produce JSON alerts when compiled with json-c 0.11 or later.
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.11.1, 2019-Jun-20

Downloads

analysis-pipeline-5.11.1.tar.gz (2MiB)

(SHA256=d0879f8f0c4c60c0eba69796b0bec6893707ea11dda51429c96188628baf2cfd)

Notes

  • Fixed a memory leak in certain alerting scenarios when compiled with json-c but not using --json.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.17.x or later (Due to libfixbuf 2.0)
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.3 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 2.0 or later
  • json-c — Recommended: Analysis Pipeline can produce JSON alerts when compiled with json-c 0.11 or later.
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.11, 2019-Jun-1

Downloads

analysis-pipeline-5.11.tar.gz (2MiB)

(SHA256=c92ad70364de488b68959f9c8d2f7a01ef6fc35c86532e7eca9dc3d579fee8cb)

Notes

  • Added support for JSON output using the json-c library.
  • Added support for the NO_FILTER configuration keyword to pass all records to a statistic or evaluation.
  • Pipeline now prints the age range of the processed records in the file ingest log line.
  • "make check" now runs a more comprehensive test suite.
  • Various speed improvements and bugfixes.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.17.x or later (Due to libfixbuf 2.0)
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.3 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 2.0 or later
  • json-c — Recommended: Analysis Pipeline can produce JSON alerts when compiled with json-c 0.11 or later.
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.10, 2019-Mar-11

Downloads

analysis-pipeline-5.10.tar.gz (2MiB)

(SHA256=7755735278bf3e977c1ad4720bb3feda42be2f442ee1500b4b1ba4234e9237d9)

Notes

  • Added directory polling support for YAF input files.
  • Both new and old tombstone records are now handled properly.
  • Added ability to alert all tombstone and/or yaf_stats records.
  • Other bug fixes.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.17.x or later (Due to libfixbuf 2.0)
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.3 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 2.0 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.9, 2018-Nov-30

Downloads

analysis-pipeline-5.9.tar.gz (2MiB)

(SHA256=025be4d7ee638f592946fc15266fd2968d4dd54eaedf51b369c2c2a1b948077b)

Notes

  • Filters now properly iterate over individual DNS records in YAF data when checking multiple DNS fields.
  • New HAS CHANGED primitive to check whether a field changes values between records.
  • FOREACH now works with the EVERYTHING PASSES primitive.
  • Tombstone records are now logged when received by pipeline.
  • Other bug fixes.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.17.x or later (Due to libfixbuf 2.0)
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.3 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 2.0 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.8, 2018-May-31

Downloads

analysis-pipeline-5.8.tar.gz (1.9MiB)

(SHA256=5fc51e2f852154893c1f761ffaf2c4e9d7ef1f6c8d912fd99b581abbc0bb0b7c)

Notes

  • New EWMA primitive to calculate the Exponentially Weighted Moving Average and corresponding standard deviation.
  • New CALCULATE STATS primitive to calculate the common statistical values such as the standard deviation, mean and count.
  • Records can now be put into bins based on time windows to increase efficiency in certain situations and allow for better control of updating logic.
  • FILTERS can now be put into MANIFOLDS to increase efficiency in certain situations.
  • Other bug fixes.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.17.x or later (Due to libfixbuf 2.0 changes)
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.3 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 2.0 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.7, 2017-Sep-29

Downloads

analysis-pipeline-5.7.tar.gz (1.9MiB)

(SHA256=b996d31607c6788ade5d545d7974b0a3b49b6fd41aad6fcbec73f3b0a05737c3)

Notes

  • EVALUATIONS can be forced to wait a minimum amount of time before alerting
  • STATISTICS can now have a minimum number of records before updating.
  • Other bug fixes.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.0 or later
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.2 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 1.4 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.6, 2017-Jan-7

Downloads

analysis-pipeline-5.6.tar.gz (1.7MiB)

(SHA256=0a26a8f2ac95582804903c6d06c72bc10a3107de855d0600b34c066d5aaac7a4)

Notes

  • All fields can use a SEED file of any type
  • More than one EXTRA ALERT FIELDs is now allowed.
  • EXTRA ALERT FIELDs can now be derived fields
  • Added EXTRA AUX ALERT FIELD to add fields to auxilliary alerts
  • STATISTICs can now updated EVERY HOUR, or EVERY DAY
  • STATISTICs will send one final update after processing a list of files using --name-files
  • Other bug fixes.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.0 or later
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.2 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 1.4 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.5, 2016-Oct-18

Downloads

analysis-pipeline-5.5.tar.gz (1.7MiB)

(SHA256=6d4f2e5d7409142d911000fff45b7bb7caa5f3f71ffd68416a359686c46a47f9)

Notes

  • New PERSISTENCE primitive to detect a FOREACH tuple's presence for a specified number of HOURS or DAYS.
  • A minimum number of records requirement can be added to primitives, either at the overall EVALUATION level, or for each value of the FOREACH field. Alerts will not be sent until the minimum number of records is seen.
  • Other bug fixes.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.0 or later
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.2 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 1.4 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.4.1, 2016-Jul-13

Downloads

analysis-pipeline-5.4.1.tar.gz (1.6MiB)

(SHA256=1236147d6121a76e66e01caa3c45a058b64602c3ca44b35adaab5e2f3f5927d7)

Notes

  • List configuration can now write files with the contents of the list without sending an alert.
  • ICMP fields are fixed.
  • Filtering by comparing two fields works with derived fields.
  • Other bug fixes.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.0 or later
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.2 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 1.4 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.4, 2016-Jun-3

Downloads

analysis-pipeline-5.4.tar.gz (1.6MiB)

(SHA256=4e31c01543449c9cd04504a893e66f8024d20c4ecfa2d03790f0051a803cb8eb)

Notes

  • Significant memory and prossecing efficiency improvements.
  • Streamlined Statistic processing
  • Reloading of bag files used as custom thresholds upon update.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.0 or later
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.2 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 1.4 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.3.2, 2016-Feb-17

Downloads

analysis-pipeline-5.3.2.tar.gz (1.6MiB)

(SHA256=a050abe81593251a1f7272b1c63f7d90eebfe9ec586eccad06c14bd79d9e28db)

Notes

  • Pmaps are IP version agnostic. Pmaps can have both v4 and v6 address that can be used with SIP and SIP_V6.
  • Small bug fixes with Ubuntu compiling and domain name processing.
  • Unit test improvements

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.0 or later
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.2 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 1.4 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.3.1, 2016-Jan-22

Downloads

analysis-pipeline-5.3.1.tar.gz (4.9MiB)

(SHA256=e14f713c877a73e1bd71dffa6acca22beffbf8a0d5e40907805850865ef0749d)

Notes

  • Changed Snarf alerts when using FOREACH. Rather than a single string containing a comma separated field list and a single string for the values, each value and field will be in parallel arrays, with values in appropriate format.
  • Small bug fixes.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.0 or later
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.2 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 1.4 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.

Analysis Pipeline Release 5.3, 2015-Sep-30

Downloads

analysis-pipeline-5.3.tar.gz (1.6MiB)

(SHA256=044e4a12470b6adcce2215488d10fb1f9b242da62dca676f7e4043e422baa85e)

Notes

  • Expanded data inputs to include records from YAF (including all deep packet inspection fields), and any flat IPFIX records.
  • The handling of multiple data sources at once.
  • FAST FLUX primitive to detect fast flux networks from DNS records
  • Derived fields, that operate on values from the records, such as the length of a string, the second level domain from a fully qualified domain name, and pulling the day of the week from a timestamp.
  • The ability to have a watchlist using any type of field, paired with the having a LIST CONFIGURATION write the contents to file regardless of field type.
  • A special type of watchlist for DNS that checks each part of a domain name, rather than a generic string match.
  • First public release of Analysis Pipeline version 5.

Dependencies

  • SiLK — Analysis Pipeline requires SiLK 3.0 or later
  • SchemaTools — Analysis Pipeline requires SchemaTools 1.1 or later
  • fixbuf — Analysis Pipeline requires libfixbuf 1.4 or later
  • Snarf — Recommended: Analysis Pipeline can interact with the snarf alerting library.