Overview¶

netsa.dist provides a set of extensions to distutils, along with an alternative API for specifying the contents of the distribution. The extensions provide for automatic generation of documentation using Sphinx (including PDF, HTML, and man pages), automatic generation of Python-readable version information from configuration metadata, and targets for running automated tests. The alternative API allows for specification of project metadata in a similar style to the metadata of netsa.script.

New setup.py Commands¶

When running setup.py for a project that uses netsa.dist, all of the normal commands (build, install, sdist, etc.) are available, along with the following:

check

Run all automated tests for the project.

check_unit

Run automated unit tests for the project.

check_other

Run all other automated tests for the project.

gen_version

Generates any “version” files required by the project. See add_version_file.

gen_doc_html

Generates an HTML manual for this project, placing it in doc/html. This manual is in the normal style for Python documentation. It is never automatically generated.

gen_doc_tools_web

Generates an HTML manual for this project and create a tarball out of it, placing the results in dist/<name>-<version>-doc-web.tar.gz. This manual is designed for use on the NetSA Tools website, and this command is used only to generate documentation to be deployed at that site.

gen_doc_man

Creates generated man pages for the project, placing them under doc/man. This is automatically called when generating a source distribution, and the results will be included in the source tarball. When installing, netsa.dist will attempt to run this command, but if it fails it will use a pre-generated copy if available.

gen_doc_pdf

Generates a PDF manual for this project, placing it in the top level directory. This is automatically called when generating a source distribution, and the resulting manual will be included in the source tarball. This manual is not installed, however, only included with the distribution.

netsa_dist

Generates the standard items for a NetSA distribution. Namely, a source code release tarball in dist/<name>-<version>.tar.gz and a documentation tarball in dist/<name>-<version>-doc-web.tar.gz for deployment to the NetSA Tools website.

netsa_src_license

Modifies the source files in place to update their license section. The license in LICENSE-xxx.txt is used for a section that begins with @xxx_HEADER_START@. Backup files are created, just in case.

Project Layout¶

The files of the project are expected to be arranged mostly as follows:

The following directories are used to contain outputs, and any extra files in them may be automatically destroyed by the clean command:

Documentation Configuration¶

To support simpler common configuration for documentation output, a convenience module is create during documentation generation. Under most circumstances, you should be able to use the following conf.py without changes:

from netsa_sphinx_config import *

add_static_path("static_html")

Importing all symbols from netsa_sphinx_config sets all of the settings to their normal values for a NetSA project, including producing output appropriate for use on the NetSA Tools website automatically.

If you need to make modifications, just replace or modify the values of standard Sphinx build options.

The following function is provided to allow automatic generation of man pages. Any man page generated from the documentation will be automatically generated and included in source distributions, and automatically installed in the appropriate location.

netsa_sphinx_config.add_man_page(source_file : str, man_page : str, description : str[, section = 1])¶

Add a new man page to be generated from the given source_file (without extension). The name of the resulting file is man_page.*section*. Note that when Sphinx generates man pages, the top-level heading from the input file is ignored, and the title used is “man_page - description” instead. This way, you can use the same input file to produce installed man pages and to produce man pages for display in the HTML output.

Project Configuration¶

The following functions are used to set metadata for the project:

netsa.dist.set_name(project_name : str)¶

Sets the name of the project. This name is used as part of the name of produced tarballs and documentation files.

netsa.dist.set_title(project_title : str)¶

Sets the title for this project. This should be the human-readable name of the project. It is displayed in most places as the project name.

netsa.dist.set_description(project_description : str)¶

Sets the long-form description for this project. This should be a detailed explanation of the project’s purpose.

netsa.dist.set_version(project_version : str)¶

Sets the version number for this project. The version number is used as part of the filename of distribution files, is included in the documentation, and may be written out as version files (see add_version_file and netsa.find_version).

netsa.dist.set_copyright(project_copyright : str)¶

Sets the copyright date for this project. This is used in documentation generation, and in the project metadata. For example:

dist.set_copyright("2008-2011, Carnegie Mellon University")

netsa.dist.set_license(project_license : str)¶

Sets the license type for this project, which defaults to 'GPL'. This is used for project distribution metadata.

netsa.dist.set_maintainer(project_maintainer : str)¶

Given a name and email address (i.e. 'Harry Q. Bovik <bovik@sample.samp>') sets the maintainer name and email address metadata for the project.

netsa.dist.set_author(project_author : str)¶

Given a name and email address (i.e. 'Harry Q. Bovik <bovik@sample.samp>') sets the author name and email address metadata for the project.

netsa.dist.set_url(project_url : str)¶

Sets the home page URL metadata for this project.

netsa.dist.set_download_url(project_download_url : str)¶

Sets the download page URL metadata for this project.

Choosing which files should be installed where is accomplished with the following functions:

netsa.dist.add_package(package_name)¶

Adds a Python package to be installed, by package name. For example:

dist.add_package("netsa.data")

The files for this Python package would be found under src/netsa/dist. Remember that the package directory (and every directory leading up to it) must include an __init__.py file to be accepted as a Python package.

netsa.dist.add_package_data(package_name, data_file_glob)¶

Adds one or more data files to be installed within a package. Each file or directory that data_file_glob expands to is included. The files and directories should be stored under src/<package_name>, just like the Python source files for the package. For a method of installing files in different places, see add_install_data.

netsa.dist.add_module_py(module_name)¶

Adds a single module by module name. For example:

dist.add_module_py("netsa.util.shell")

Thie file for this module would be found at src/netsa/util/shell.py. Remember that the package directory (and every directory leading up to it) must include an __init__.py file, which will also be installed.

netsa.dist.add_module_ext(module_name, module_sources, **kwargs)¶

Adds a single C extension module, given a module name, a list of sources, and optional keyword arguments as accepted by distutils.core.Extension. For example:

dist.add_module_ext('foo', ['foo.c', 'bar.c'])

netsa.dist.add_script(script_name)¶

Adds a single script by script name. For example:

dist.add_script("helloworld")

The file for this script would be found at bin/helloworld. When installed, if the script has a #! line and contains python, it will automatically be modified to point to the version of Python being used to install this project.

netsa.dist.add_install_data(install_path, data_file_name)¶

Adds an extra data file that should be installed when the project is installed. The data_file_name should be the path to the file from the top level of the project. install_path should be the path to the installation directory from the install prefix. For example:

dist.add_install_data("share/doc/helloworld", "samples/helloworld.ini")

This would install the file found at samples/helloworld.ini as .../share/doc/helloworld/helloworld.ini under the installation prefix.

netsa.dist.add_extra_files(extra_glob)¶

Given a glob string, adds files which match that glob to the distribution. This is used to add any extra files (README, etc.) that should be included in a source distribution but are not to be installed. If you do include in this list a file that’s already to be installed, it will still be installed, and it will still be included in the distribution.

In order to avoid recording the version number in both the setup.py file and the source code, you can use the following functions to automatically generate a file with the version number in it, and read it back at run time:

netsa.dist.add_version_file(version_file_name : str[, version_file_template = "%s\n"])¶

Adds a “version file” to the project, with the given path and template. By default the template is "%s\n", which simply includes the version number and a newline. The path should be given relative to the base of the project. The version file will be generated automatically before any other processing is done.

See netsa.find_version for a convenient method for retrieving the version number from this file for your package.

Example:

# in setup.py
dist.add_version_file("src/netsa/VERSION")

# in netsa/__init__.py
__version__ = netsa.find_version(__file__)

netsa.find_version(source_file : str[, num_levels = 3]) → str¶

Given the path to a Python source file, read in a version number from a file VERSION in the same directory, or look for a setup.py file in up to num_levels directories above the file and attempt to find the version there.

The following functions allow automated tests to be added and run from setup.py:

netsa.dist.add_unit_test_module(script_unit_test_module : str)¶

Adds a unit test module to be run, by module name. For example:

dist.add_unit_test_module("netsa.data.test")

The provided module is expected to be a unittest test module, and the tests will be run in a separate process from the process running setup.py. Running tests automatically builds the project, and places the build area in the PYTHONPATH for the test process.

netsa.dist.add_other_test_module(script_other_test_module : str)¶

Adds a module to be run for testing, by module name. For example:

dist.add_other_test_module("crunchy.test")

The provided module is called in a subprocess like this:

python -m crunchy.test ${source_dir}

Where ${source_dir} is the top level source directory of the project. Running tests automatically builds the project, and places the build area in the PYTHONPATH for the test process.

Finally, once the project is fully configured, use this function to handle command-line options and actually running the tasks:

netsa.dist.execute()¶

Using the project as so far specified, parse command line options and does what is required to build, install, test, or make a distribution for the project. This should be called as the last thing in setup.py.