Download Now
This Lambda-based script optimizes AWS VPC Flow Logs by eliminating redundant cloud metadata fields, reducing log size by 60%. The script responds to S3 Event Notifications (configured either directly or through Amazon SNS), pulls raw VPC Flow Logs from S3, removes repetitive columns, and writes the optimized logs to a destination S3 bucket using compression. It can optionally retain the original flow log headers, allowing compatibility with Splunk's standardized ingestion format.
Despite removing fields, the reduced logs preserve full analytical capabilities through optional enrichment pipelines that add cloud context on-demand. This reduction decreases S3 storage costs, lowers Splunk licensing costs, improves query performance, and maintains searchability while using existing VPC Flow Log collection configurations.
Download Now
This is a collection of scripts for working with Azure flow logs. Currently includes a script that restructures the deeply nested, event-driven JSON Azure
NSG logs into a more analyst-friendly mode.
Download Now
The Prism trend script is a tool for quickly visualizing flow data
as a time-series broken down into several configurable bins by
SiLK's rwfilter tool. The script can be
used directly, or might be used as a component in other more
specialized scripts.
Download Now
This is a plug-in for CIF that consists of Perl and Python
modules. Perl-based CIF plug-in passes JSON-like (objects in {},
but no commas in between) formatted result of CIF query to
Python-based STIX/Cybox document builder.
Download Now
This is a plug-in for CIF that consists of a Python module. It parses
STIX/Cybox documents into JSON CIF Feed files with corresponding
configuration files for each source document and feed it to CIF.