Installation

Requirements

In addition to Python, the following software packages and tools are needed for installation and operation:

Required software
Software version Download source
fixbuf 1.3.0+ http://tools.netsa.cert.org/fixbuf/download.html
glib 2.6.4+ http://ftp.gnome.org/pub/gnome/sources/glib/
netsa-python 1.4.3+ http://tools.netsa.cert.org/netsa-python/download.html)

For raw pcap processing support:

yaf 2.4.0+ http://tools.netsa.cert.org/yaf/download.html

For Oracle support:

cx_Oracle 5.1.2+ http://cx-oracle.sourceforge.net/

For PostgreSQL support:

psycopg2 2.4.5+ http://www.initd.org/psycopg/download/

Listed version numbers were used in testing.

Building and Installation

To build the system, run the following command from the top-level directory:

python setup.py build

If you wish to use an alternative version of Python, rather than the system default, then use that version of Python to run setup.py. For example, if the version of Python you wish to use is python2.7, you should run:

python2.7 setup.py build

During the build, you may get an error like this:

The following required packages are missing from pkg-config:
    libfixbuf
Check to ensure PKG_CONFIG_PATH is set correctly

This indicates either that the listed packages are not installed or that they’re not being found by pkg-config. You may need to set your PKG_CONFIG_PATH environment variable to include directories where pkg-config information may be found. For example, if you have packages installed in /usr/local, you may need to do:

export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig

After the build is complete, run the following command as a user with permission to install software in the desired location:

python setup.py install

If you wish to install in a different location that the default (under the site-packages directory of the Python distribution), you should specify the desired install prefix like:

python setup.py install --prefix=/opt/orcus

Note that if you do this, you may need to set the PYTHONPATH environment variable when running Orcus. Consider looking into the virtualenv tool for a cleaner alternative way to install Python packages and tools outside the standard locations.

Database Configuration and Schema Installation

The Orcus database schema must be installed before Orcus can load data into the database. Note that before you can install the schema, you must create whatever database users or roles that you will be using. Orcus allows for separate roles to be used for loading data and for querying data.

Database schemas can be found in the sql subdirectory. These include schema creation scripts and drop scripts for each supported RBMS.

Oracle

On Oracle, you could use the following commands as an administrator in sqlplus to create roles for Orcus:

create user orcus identified by <password>;
grant connect to orcus;
grant resource to orcus;
create user orcususer identified by <password>;
grant connect to orcususer;

These users may be named anything, suitable to your local Oracle configuration. The above commands will create a user for loading (orcus), with permission to connect to the database (connect) and create new database objects (resource). They will also create a user for querying (orcususer), with permission to connect to the database (connect).

See Orcus File Configuration for more details about configuring the Orcus tools to properly connect to the database.

On Oracle, the schema can be installed by running a command like:

sqlplus orcus @create-sa_orcus-1.0.0-oracle.sql

If you later wish to drop the database schema (and destroy all information stored by Orcus), you can do so with the following command:

sqlplus orcus @drop-sa_orcus-1.0.0-oracle.sql

PostgreSQL

On PostgreSQL, you could use the following commands as an administrator in psql to create roles for Orcus:

create user orcus login password '<password>';
create user orcususer login password '<password>';

You should also create a database instance for Orcus (although you may use an already-existing database):

create database orcus owner orcus;

As with Oracle, the users may be named anything, and so may the database. The above commands will create a user for loading data (orcus), and one for querying data (orcususer), and create a new database instance which is owned by the loading user (orcus).

Note that Orcus requires the PL/PgSQL language to be available. On some installations of Postgres, this may not be available in new databases by default and you’ll have to create the language in the orcus DB with a command like:

create language plpgsql;

See Orcus File Configuration for more details about configuring the Orcus tools to properly connect to the database.

Or PostgreSQL, the schema can be installed by running a command like:

psql -U orcus -q -f create-sa_orcus-1.0.0-postgres.sql

If you later wish to drop the database schema (and destroy all information stored by Orcus), you can do so with the following command:

psql -U orcus -q -f drop-sa_orcus-1.0.0-postgres.sql

Orcus File Configuration

The Orcus tools look for a configuration file, first in a directory relative to where they were installed, then in a system-wide directory. If Orcus is installed in /usr/local, first /usr/local/etc/orcus.conf will be checked for, then if that’s not found /etc/orcus.conf will be used. (The location of this configuration file may also be overridden via the --config-file command-line switch.)

There are a number of options that may be set in this file, along with a few that must be set. Full documentation is available in the orcus.conf man page. An example is provided with the source code as orcus.conf.sample.

The following items should be set in the configuration file:

Database URI

The Orcus tools look for two files in a directory relative to where they were installed. If Orcus is installed in /usr/local, these files are /usr/local/etc/orcus-db-load.uri and /usr/local/etc/orcus-db.uri. If the files are not found at these locations, system-wide /etc is checked as well.

This search is overridden if the db-uri or db-uri-file options are specified in the orcus.conf file.

These files are expected to contain netsa-python database connection URIs describing how to contact the database in order to load data (used by orloader), and in order to query data (used by orlookup and orquery).

The URI format for Oracle is:

nsql-oracle:;user=<username>;password=<password>

For example, to connect as user “orcus” with password “pass”:

nsql-oracle:;user=orcus;password=pass

The URI format for PostgreSQL is:

nsql-postgres:<database>;user=<username>;password=<password>

For example, to connect to the database instance “orcus” as user “orcususer” with password “pass”:

nsql-postgres:orcus;user=orcususer;password=pass

In order to prevent unauthorized users from learning how to connect to the database with write privileges, orcus-db-load.uri should be set to be readable only by the user who will run orloader.

On the other hand, orcus-db.uri should be readable to all users who are allowed to query the Orcus database.

Processing Directories

In order to load data into Orcus, you need to set up the following directories, with locations configured via orcus.conf:

  1. An “incoming” directory for fresh files that need to be loaded. (incoming-directory)
  2. A “loading” directory where files are processed. (loading-directory)
  3. An “error” directory where any files that produce errors during loading are placed for later inspection. (error-directory)
  4. Optionally, an “archive” directory where files are placed after being processed. (archive-directory)

Create these directories and make sure they are readable and writable to the user who is to run the Orcus loader process. Note that the “loading” and “error” directories should not be world-readable, because they may at some times contain database password information.

The input files placed into the “incoming” directory may be IPFIX files containing packet capture data, IPFIX files containing DNS deep-packet-inspection data, or pcap files (which will be processed by YAF to produce IPFIX files containing packet capture data.)

If you place pcap files in this directory, take care that they have the suffix ”.pcap”.

If the name of a file includes a hyphen (“sensor-...”), the portion before the first hyphen will be taken as the sensor name. If no hyphen appears in the filename, the sensor name UNKNOWN will be used.

Once the directories are set up and orcus.conf is correct, run:

orloader

orloader will watch for new files placed in “incoming” and then process them. Watch for log messages in the orloader log file to see if there are errors during startup or processing. If there are processing errors, the input file that produced the error will appear in a subdirectory of “error”, along with intermediate files created during processing that input file.