Building and Distributing Packages with Setuptools
Setuptools
is a collection of enhancements to the Python distutils
that allow developers to more easily build and distribute Python packages, especially ones that have dependencies on other packages.
Packages built and distributed using setuptools
look to the user like ordinary Python packages based on the distutils
.
Feature Highlights:
- Create Python Eggs – a single-file importable distribution format
- Enhanced support for accessing data files hosted in zipped packages.
- Automatically include all packages in your source tree, without listing them individually in setup.py
- Automatically include all relevant files in your source distributions, without needing to create a
MANIFEST.in
file, and without having to force regeneration of theMANIFEST
file when your source tree changes. - Automatically generate wrapper scripts or Windows (console and GUI) .exe files for any number of “main” functions in your project. (Note: this is not a py2exe replacement; the .exe files rely on the local Python installation.)
- Transparent Cython support, so that your setup.py can list
.pyx
files and still work even when the end-user doesn’t have Cython installed (as long as you include the Cython-generated C in your source distribution) - Command aliases – create project-specific, per-user, or site-wide shortcut names for commonly used commands and options
- Deploy your project in “development mode”, such that it’s available on
sys.path
, yet can still be edited directly from its source checkout. - Easily extend the distutils with new commands or
setup()
arguments, and distribute/reuse your extensions for multiple projects, without copying code. - Create extensible applications and frameworks that automatically discover extensions, using simple “entry points” declared in a project’s setup script.
- Full support for PEP 420 via
find_namespace_packages()
, which is also backwards compatible to the existingfind_packages()
for Python >= 3.3.
Table of Contents
- Building and Distributing Packages with Setuptools
- Developer’s Guide
- Command Reference
alias
– Define shortcuts for commonly used commandsbdist_egg
– Create a Python Egg for the projectdevelop
– Deploy the project source in “Development Mode”egg_info
– Create egg metadata and set build tagsrotate
– Delete outdated distribution filessaveopts
– Save used options to a configuration filesetopt
– Set a distutils or setuptools option in a config filetest
– Build package and run a unittest suiteupload
– Upload source and/or egg distributions to PyPI
- Configuring setup() using setup.cfg files
- Extending and Reusing Setuptools
Developer’s Guide
Installing setuptools
To install the latest version of setuptools, use:
pip install --upgrade setuptools
Refer to Installing Packages guide for more information.
Basic Use
For basic use of setuptools, just import things from setuptools instead of the distutils. Here’s a minimal setup script using setuptools:
from setuptools import setup, find_packages
setup(
name="HelloWorld",
version="0.1",
packages=find_packages(),
)
As you can see, it doesn’t take much to use setuptools in a project. Run that script in your project folder, alongside the Python packages you have developed.
Invoke that script to produce distributions and automatically include all packages in the directory where the setup.py lives. See the Command Reference section below to see what commands you can give to this setup script. For example, to produce a source distribution, simply invoke:
setup.py sdist
Of course, before you release your project to PyPI, you’ll want to add a bit more information to your setup script to help people find or learn about your project. And maybe your project will have grown by then to include a few dependencies, and perhaps some data files and scripts:
from setuptools import setup, find_packages
setup(
name="HelloWorld",
version="0.1",
packages=find_packages(),
scripts=["say_hello.py"],
# Project uses reStructuredText, so ensure that the docutils get
# installed or upgraded on the target machine
install_requires=["docutils>=0.3"],
package_data={
# If any package contains *.txt or *.rst files, include them:
"": ["*.txt", "*.rst"],
# And include any *.msg files found in the "hello" package, too:
"hello": ["*.msg"],
},
# metadata to display on PyPI
author="Me",
author_email="me@example.com",
description="This is an Example Package",
keywords="hello world example examples",
url="http://example.com/HelloWorld/", # project home page, if any
project_urls={
"Bug Tracker": "https://bugs.example.com/HelloWorld/",
"Documentation": "https://docs.example.com/HelloWorld/",
"Source Code": "https://code.example.com/HelloWorld/",
},
classifiers=[
"License :: OSI Approved :: Python Software Foundation License"
]
# could also include long_description, download_url, etc.
)
In the sections that follow, we’ll explain what most of these setup()
arguments do (except for the metadata ones), and the various ways you might use them in your own project(s).
Specifying Your Project’s Version
Setuptools can work well with most versioning schemes; there are, however, a few special things to watch out for, in order to ensure that setuptools and other tools can always tell what version of your package is newer than another version. Knowing these things will also help you correctly specify what versions of other projects your project depends on.
A version consists of an alternating series of release numbers and pre-release or post-release tags. A release number is a series of digits punctuated by dots, such as 2.4
or 0.5
. Each series of digits is treated numerically, so releases 2.1
and 2.1.0
are different ways to spell the same release number, denoting the first subrelease of release 2. But 2.10
is the tenth subrelease of release 2, and so is a different and newer release from 2.1
or 2.1.0
. Leading zeros within a series of digits are also ignored, so 2.01
is the same as 2.1
, and different from 2.0.1
.
Following a release number, you can have either a pre-release or post-release tag. Pre-release tags make a version be considered older than the version they are appended to. So, revision 2.4
is newer than revision 2.4c1
, which in turn is newer than 2.4b1
or 2.4a1
. Postrelease tags make a version be considered newer than the version they are appended to. So, revisions like 2.4-1
and 2.4pl3
are newer than 2.4
, but are older than 2.4.1
(which has a higher release number).
A pre-release tag is a series of letters that are alphabetically before “final”. Some examples of prerelease tags would include alpha
, beta
, a
, c
, dev
, and so on. You do not have to place a dot or dash before the prerelease tag if it’s immediately after a number, but it’s okay to do so if you prefer. Thus, 2.4c1
and 2.4.c1
and 2.4-c1
all represent release candidate 1 of version 2.4
, and are treated as identical by setuptools.
In addition, there are three special prerelease tags that are treated as if they were the letter c
: pre
, preview
, and rc
. So, version 2.4rc1
, 2.4pre1
and 2.4preview1
are all the exact same version as 2.4c1
, and are treated as identical by setuptools.
A post-release tag is either a series of letters that are alphabetically greater than or equal to “final”, or a dash (-
). Post-release tags are generally used to separate patch numbers, port numbers, build numbers, revision numbers, or date stamps from the release number. For example, the version 2.4-r1263
might denote Subversion revision 1263 of a post-release patch of version 2.4
. Or you might use 2.4-20051127
to denote a date-stamped post-release.
Notice that after each pre or post-release tag, you are free to place another release number, followed again by more pre- or post-release tags. For example, 0.6a9.dev-r41475
could denote Subversion revision 41475 of the in- development version of the ninth alpha of release 0.6. Notice that dev
is a pre-release tag, so this version is a lower version number than 0.6a9
, which would be the actual ninth alpha of release 0.6. But the -r41475
is a post-release tag, so this version is newer than 0.6a9.dev
.
For the most part, setuptools’ interpretation of version numbers is intuitive, but here are a few tips that will keep you out of trouble in the corner cases:
-
Don’t stick adjoining pre-release tags together without a dot or number between them. Version
1.9adev
is theadev
prerelease of1.9
, not a development pre-release of1.9a
. Use.dev
instead, as in1.9a.dev
, or separate the prerelease tags with a number, as in1.9a0dev
.1.9a.dev
,1.9a0dev
, and even1.9.a.dev
are identical versions from setuptools’ point of view, so you can use whatever scheme you prefer. -
If you want to be certain that your chosen numbering scheme works the way you think it will, you can use the
pkg_resources.parse_version()
function to compare different version numbers:>>> from pkg_resources import parse_version >>> parse_version("1.9.a.dev") == parse_version("1.9a0dev") True >>> parse_version("2.1-rc2") < parse_version("2.1") True >>> parse_version("0.6a9dev-r41475") < parse_version("0.6a9") True
Once you’ve decided on a version numbering scheme for your project, you can have setuptools automatically tag your in-development releases with various pre- or post-release tags. See the following sections for more details:
New and Changed setup()
Keywords
The following keyword arguments to setup()
are added or changed by setuptools
. All of them are optional; you do not have to supply them unless you need the associated setuptools
feature.
include_package_data
- If set to
True
, this tellssetuptools
to automatically include any data files it finds inside your package directories that are specified by yourMANIFEST.in
file. For more information, see the section below on Including Data Files. exclude_package_data
- A dictionary mapping package names to lists of glob patterns that should be excluded from your package directories. You can use this to trim back any excess files included by
include_package_data
. For a complete description and examples, see the section below on Including Data Files. package_data
- A dictionary mapping package names to lists of glob patterns. For a complete description and examples, see the section below on Including Data Files. You do not need to use this option if you are using
include_package_data
, unless you need to add e.g. files that are generated by your setup script and build process. (And are therefore not in source control or are files that you don’t want to include in your source distribution.) zip_safe
- A boolean (True or False) flag specifying whether the project can be safely installed and run from a zip file. If this argument is not supplied, the
bdist_egg
command will have to analyze all of your project’s contents for possible problems each time it builds an egg. install_requires
- A string or list of strings specifying what other distributions need to be installed when this one is. See the section below on Declaring Dependencies for details and examples of the format of this argument.
entry_points
- A dictionary mapping entry point group names to strings or lists of strings defining the entry points. Entry points are used to support dynamic discovery of services or plugins provided by a project. See Dynamic Discovery of Services and Plugins for details and examples of the format of this argument. In addition, this keyword is used to support Automatic Script Creation.
extras_require
- A dictionary mapping names of “extras” (optional features of your project) to strings or lists of strings specifying what other distributions must be installed to support those features. See the section below on Declaring Dependencies for details and examples of the format of this argument.
python_requires
- A string corresponding to a version specifier (as defined in PEP 440) for the Python version, used to specify the Requires-Python defined in PEP 345.
setup_requires
-
A string or list of strings specifying what other distributions need to be present in order for the setup script to run.
setuptools
will attempt to obtain these (using pip if available) before processing the rest of the setup script or commands. This argument is needed if you are using distutils extensions as part of your build process; for example, extensions that process setup() arguments and turn them into EGG-INFO metadata files.(Note: projects listed in
setup_requires
will NOT be automatically installed on the system where the setup script is being run. They are simply downloaded to the ./.eggs directory if they’re not locally available already. If you want them to be installed, as well as being available when the setup script is run, you should add them toinstall_requires
andsetup_requires
.) dependency_links
- A list of strings naming URLs to be searched when satisfying dependencies. These links will be used if needed to install packages specified by
setup_requires
ortests_require
. They will also be written into the egg’s metadata for use during install by tools that support them. namespace_packages
- A list of strings naming the project’s “namespace packages”. A namespace package is a package that may be split across multiple project distributions. For example, Zope 3’s
zope
package is a namespace package, because subpackages likezope.interface
andzope.publisher
may be distributed separately. The egg runtime system can automatically merge such subpackages into a single parent package at runtime, as long as you declare them in each project that contains any subpackages of the namespace package, and as long as the namespace package’s__init__.py
does not contain any code other than a namespace declaration. See the section below on Namespace Packages for more information. test_suite
-
A string naming a
unittest.TestCase
subclass (or a package or module containing one or more of them, or a method of such a subclass), or naming a function that can be called with no arguments and returns aunittest.TestSuite
. If the named suite is a module, and the module has anadditional_tests()
function, it is called and the results are added to the tests to be run. If the named suite is a package, any submodules and subpackages are recursively added to the overall test suite.Specifying this argument enables use of the test command to run the specified test suite, e.g. via
setup.py test
. See the section on the test command below for more details.New in 41.5.0: Deprecated the test command.
tests_require
-
If your project’s tests need one or more additional packages besides those needed to install it, you can use this option to specify them. It should be a string or list of strings specifying what other distributions need to be present for the package’s tests to run. When you run the
test
command,setuptools
will attempt to obtain these (using pip if available). Note that these required projects will not be installed on the system where the tests are run, but only downloaded to the project’s setup directory if they’re not already installed locally.New in 41.5.0: Deprecated the test command.
test_loader
-
If you would like to use a different way of finding tests to run than what setuptools normally uses, you can specify a module name and class name in this argument. The named class must be instantiable with no arguments, and its instances must support the
loadTestsFromNames()
method as defined in the Pythonunittest
module’sTestLoader
class. Setuptools will pass only one test “name” in the names argument: the value supplied for thetest_suite
argument. The loader you specify may interpret this string in any way it likes, as there are no restrictions on what may be contained in atest_suite
string.The module name and class name must be separated by a
:
. The default value of this argument is"setuptools.command.test:ScanningLoader"
. If you want to use the defaultunittest
behavior, you can specify"unittest:TestLoader"
as yourtest_loader
argument instead. This will prevent automatic scanning of submodules and subpackages.The module and class you specify here may be contained in another package, as long as you use the
tests_require
option to ensure that the package containing the loader class is available when thetest
command is run.New in 41.5.0: Deprecated the test command.
eager_resources
-
A list of strings naming resources that should be extracted together, if any of them is needed, or if any C extensions included in the project are imported. This argument is only useful if the project will be installed as a zipfile, and there is a need to have all of the listed resources be extracted to the filesystem as a unit. Resources listed here should be “/”-separated paths, relative to the source root, so to list a resource
foo.png
in packagebar.baz
, you would include the stringbar/baz/foo.png
in this argument.If you only need to obtain resources one at a time, or you don’t have any C extensions that access other files in the project (such as data files or shared libraries), you probably do NOT need this argument and shouldn’t mess with it. For more details on how this argument works, see the section below on Automatic Resource Extraction.
use_2to3
- Convert the source code from Python 2 to Python 3 with 2to3 during the build process. See Supporting both Python 2 and Python 3 with Setuptools for more details.
convert_2to3_doctests
- List of doctest source files that need to be converted with 2to3. See Supporting both Python 2 and Python 3 with Setuptools for more details.
use_2to3_fixers
- A list of modules to search for additional fixers to be used during the 2to3 conversion. See Supporting both Python 2 and Python 3 with Setuptools for more details.
project_urls
- An arbitrary map of URL names to hyperlinks, allowing more extensible documentation of where various resources can be found than the simple
url
anddownload_url
options provide.
Using find_packages()
For simple projects, it’s usually easy enough to manually add packages to the packages
argument of setup()
. However, for very large projects (Twisted, PEAK, Zope, Chandler, etc.), it can be a big burden to keep the package list updated. That’s what setuptools.find_packages()
is for.
find_packages()
takes a source directory and two lists of package name patterns to exclude and include. If omitted, the source directory defaults to the same directory as the setup script. Some projects use a src
or lib
directory as the root of their source tree, and those projects would of course use "src"
or "lib"
as the first argument to find_packages()
. (And such projects also need something like package_dir={"": "src"}
in their setup()
arguments, but that’s just a normal distutils thing.)
Anyway, find_packages()
walks the target directory, filtering by inclusion patterns, and finds Python packages (any directory). Packages are only recognized if they include an __init__.py
file. Finally, exclusion patterns are applied to remove matching packages.
Inclusion and exclusion patterns are package names, optionally including wildcards. For example, find_packages(exclude=["*.tests"])
will exclude all packages whose last name part is tests
. Or, find_packages(exclude=["*.tests", "*.tests.*"])
will also exclude any subpackages of packages named tests
, but it still won’t exclude a top-level tests
package or the children thereof. In fact, if you really want no tests
packages at all, you’ll need something like this:
find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"])
in order to cover all the bases. Really, the exclusion patterns are intended to cover simpler use cases than this, like excluding a single, specified package and its subpackages.
Regardless of the parameters, the find_packages()
function returns a list of package names suitable for use as the packages
argument to setup()
, and so is usually the easiest way to set that argument in your setup script. Especially since it frees you from having to remember to modify your setup script whenever your project grows additional top-level packages or subpackages.
find_namespace_packages()
In Python 3.3+, setuptools
also provides the find_namespace_packages
variant of find_packages
, which has the same function signature as find_packages
, but works with PEP 420 compliant implicit namespace packages. Here is a minimal setup script using find_namespace_packages
:
from setuptools import setup, find_namespace_packages
setup(
name="HelloWorld",
version="0.1",
packages=find_namespace_packages(),
)
Keep in mind that according to PEP 420, you may have to either re-organize your codebase a bit or define a few exclusions, as the definition of an implicit namespace package is quite lenient, so for a project organized like so:
├── namespace
│ └── mypackage
│ ├── __init__.py
│ └── mod1.py
├── setup.py
└── tests
└── test_mod1.py
A naive find_namespace_packages()
would install both namespace.mypackage
and a top-level package called tests
! One way to avoid this problem is to use the include
keyword to whitelist the packages to include, like so:
from setuptools import setup, find_namespace_packages
setup(
name="namespace.mypackage",
version="0.1",
packages=find_namespace_packages(include=["namespace.*"])
)
Another option is to use the “src” layout, where all package code is placed in the src
directory, like so:
├── setup.py
├── src
│ └── namespace
│ └── mypackage
│ ├── __init__.py
│ └── mod1.py
└── tests
└── test_mod1.py
With this layout, the package directory is specified as src
, as such:
setup(name="namespace.mypackage",
version="0.1",
package_dir={"": "src"},
packages=find_namespace_packages(where="src"))
Automatic Script Creation
Packaging and installing scripts can be a bit awkward with the distutils. For one thing, there’s no easy way to have a script’s filename match local conventions on both Windows and POSIX platforms. For another, you often have to create a separate file just for the “main” script, when your actual “main” is a function in a module somewhere. And even in Python 2.4, using the -m
option only works for actual .py
files that aren’t installed in a package.
setuptools
fixes all of these problems by automatically generating scripts for you with the correct extension, and on Windows it will even create an .exe
file so that users don’t have to change their PATHEXT
settings. The way to use this feature is to define “entry points” in your setup script that indicate what function the generated script should import and run. For example, to create two console scripts called foo
and bar
, and a GUI script called baz
, you might do something like this:
setup(
# other arguments here...
entry_points={
"console_scripts": [
"foo = my_package.some_module:main_func",
"bar = other_module:some_func",
],
"gui_scripts": [
"baz = my_package_gui:start_func",
]
}
)
When this project is installed on non-Windows platforms (using “setup.py install”, “setup.py develop”, or with pip), a set of foo
, bar
, and baz
scripts will be installed that import main_func
and some_func
from the specified modules. The functions you specify are called with no arguments, and their return value is passed to sys.exit()
, so you can return an errorlevel or message to print to stderr.
On Windows, a set of foo.exe
, bar.exe
, and baz.exe
launchers are created, alongside a set of foo.py
, bar.py
, and baz.pyw
files. The .exe
wrappers find and execute the right version of Python to run the .py
or .pyw
file.
You may define as many “console script” and “gui script” entry points as you like, and each one can optionally specify “extras” that it depends on, that will be added to sys.path
when the script is run. For more information on “extras”, see the section below on Declaring Extras. For more information on “entry points” in general, see the section below on Dynamic Discovery of Services and Plugins.
“Eggsecutable” Scripts
Deprecated since version 45.3.0.
Occasionally, there are situations where it’s desirable to make an .egg
file directly executable. You can do this by including an entry point such as the following:
setup(
# other arguments here...
entry_points={
"setuptools.installation": [
"eggsecutable = my_package.some_module:main_func",
]
}
)
Any eggs built from the above setup script will include a short executable prelude that imports and calls main_func()
from my_package.some_module
. The prelude can be run on Unix-like platforms (including Mac and Linux) by invoking the egg with /bin/sh
, or by enabling execute permissions on the .egg
file. For the executable prelude to run, the appropriate version of Python must be available via the PATH
environment variable, under its “long” name. That is, if the egg is built for Python 2.3, there must be a python2.3
executable present in a directory on PATH
.
IMPORTANT NOTE: Eggs with an “eggsecutable” header cannot be renamed, or invoked via symlinks. They must be invoked using their original filename, in order to ensure that, once running, pkg_resources
will know what project and version is in use. The header script will check this and exit with an error if the .egg
file has been renamed or is invoked via a symlink that changes its base name.
Declaring Dependencies
setuptools
supports automatically installing dependencies when a package is installed, and including information about dependencies in Python Eggs (so that package management tools like pip can use the information).
setuptools
and pkg_resources
use a common syntax for specifying a project’s required dependencies. This syntax consists of a project’s PyPI name, optionally followed by a comma-separated list of “extras” in square brackets, optionally followed by a comma-separated list of version specifiers. A version specifier is one of the operators <
, >
, <=
, >=
, ==
or !=
, followed by a version identifier. Tokens may be separated by whitespace, but any whitespace or nonstandard characters within a project name or version identifier must be replaced with -
.
Version specifiers for a given project are internally sorted into ascending version order, and used to establish what ranges of versions are acceptable. Adjacent redundant conditions are also consolidated (e.g. ">1, >2"
becomes ">2"
, and "<2,<3"
becomes "<2"
). "!="
versions are excised from the ranges they fall within. A project’s version is then checked for membership in the resulting ranges. (Note that providing conflicting conditions for the same version (e.g. “<2,>=2” or “==2,!=2”) is meaningless and may therefore produce bizarre results.)
Here are some example requirement specifiers:
docutils >= 0.3
# comment lines and \ continuations are allowed in requirement strings
BazSpam ==1.1, ==1.2, ==1.3, ==1.4, ==1.5, \
==1.6, ==1.7 # and so are line-end comments
PEAK[FastCGI, reST]>=0.5a4
setuptools==0.5a7
The simplest way to include requirement specifiers is to use the install_requires
argument to setup()
. It takes a string or list of strings containing requirement specifiers. If you include more than one requirement in a string, each requirement must begin on a new line.
This has three effects:
- When your project is installed, either by using pip,
setup.py install
, orsetup.py develop
, all of the dependencies not already installed will be located (via PyPI), downloaded, built (if necessary), and installed. - Any scripts in your project will be installed with wrappers that verify the availability of the specified dependencies at runtime, and ensure that the correct versions are added to
sys.path
(e.g. if multiple versions have been installed). - Python Egg distributions will include a metadata file listing the dependencies.
Note, by the way, that if you declare your dependencies in setup.py
, you do not need to use the require()
function in your scripts or modules, as long as you either install the project or use setup.py develop
to do development work on it. (See “Development Mode” below for more details on using setup.py develop
.)
Dependencies that aren’t in PyPI
Warning
Dependency links support has been dropped by pip starting with version 19.0 (released 2019-01-22).
If your project depends on packages that don’t exist on PyPI, you may still be able to depend on them, as long as they are available for download as:
- an egg, in the standard distutils
sdist
format, - a single
.py
file, or - a VCS repository (Subversion, Mercurial, or Git).
You just need to add some URLs to the dependency_links
argument to setup()
.
The URLs must be either:
- direct download URLs,
- the URLs of web pages that contain direct download links, or
- the repository’s URL
In general, it’s better to link to web pages, because it is usually less complex to update a web page than to release a new version of your project. You can also use a SourceForge showfiles.php
link in the case where a package you depend on is distributed via SourceForge.
If you depend on a package that’s distributed as a single .py
file, you must include an "#egg=project-version"
suffix to the URL, to give a project name and version number. (Be sure to escape any dashes in the name or version by replacing them with underscores.) EasyInstall will recognize this suffix and automatically create a trivial setup.py
to wrap the single .py
file as an egg.
In the case of a VCS checkout, you should also append #egg=project-version
in order to identify for what package that checkout should be used. You can append @REV
to the URL’s path (before the fragment) to specify a revision. Additionally, you can also force the VCS being used by prepending the URL with a certain prefix. Currently available are:
svn+URL
for Subversion,git+URL
for Git, andhg+URL
for Mercurial
A more complete example would be:
vcs+proto://host/path@revision#egg=project-version
Be careful with the version. It should match the one inside the project files. If you want to disregard the version, you have to omit it both in the requires
and in the URL’s fragment.
This will do a checkout (or a clone, in Git and Mercurial parlance) to a temporary folder and run setup.py bdist_egg
.
The dependency_links
option takes the form of a list of URL strings. For example, this will cause a search of the specified page for eggs or source distributions, if the package’s dependencies aren’t already installed:
setup(
...
dependency_links=[
"http://peak.telecommunity.com/snapshots/"
],
)
Declaring “Extras” (optional features with their own dependencies)
Sometimes a project has “recommended” dependencies, that are not required for all uses of the project. For example, a project might offer optional PDF output if ReportLab is installed, and reStructuredText support if docutils is installed. These optional features are called “extras”, and setuptools allows you to define their requirements as well. In this way, other projects that require these optional features can force the additional requirements to be installed, by naming the desired extras in their install_requires
.
For example, let’s say that Project A offers optional PDF and reST support:
setup(
name="Project-A",
...
extras_require={
"PDF": ["ReportLab>=1.2", "RXP"],
"reST": ["docutils>=0.3"],
}
)
As you can see, the extras_require
argument takes a dictionary mapping names of “extra” features, to strings or lists of strings describing those features’ requirements. These requirements will not be automatically installed unless another package depends on them (directly or indirectly) by including the desired “extras” in square brackets after the associated project name. (Or if the extras were listed in a requirement spec on the “pip install” command line.)
Extras can be used by a project’s entry points to specify dynamic dependencies. For example, if Project A includes a “rst2pdf” script, it might declare it like this, so that the “PDF” requirements are only resolved if the “rst2pdf” script is run:
setup(
name="Project-A",
...
entry_points={
"console_scripts": [
"rst2pdf = project_a.tools.pdfgen [PDF]",
"rst2html = project_a.tools.htmlgen",
# more script entry points ...
],
}
)
Projects can also use another project’s extras when specifying dependencies. For example, if project B needs “project A” with PDF support installed, it might declare the dependency like this:
setup(
name="Project-B",
install_requires=["Project-A[PDF]"],
...
)
This will cause ReportLab to be installed along with project A, if project B is installed – even if project A was already installed. In this way, a project can encapsulate groups of optional “downstream dependencies” under a feature name, so that packages that depend on it don’t have to know what the downstream dependencies are. If a later version of Project A builds in PDF support and no longer needs ReportLab, or if it ends up needing other dependencies besides ReportLab in order to provide PDF support, Project B’s setup information does not need to change, but the right packages will still be installed if needed.
Note, by the way, that if a project ends up not needing any other packages to support a feature, it should keep an empty requirements list for that feature in its extras_require
argument, so that packages depending on that feature don’t break (due to an invalid feature name). For example, if Project A above builds in PDF support and no longer needs ReportLab, it could change its setup to this:
setup(
name="Project-A",
...
extras_require={
"PDF": [],
"reST": ["docutils>=0.3"],
}
)
so that Package B doesn’t have to remove the [PDF]
from its requirement specifier.
Declaring platform specific dependencies
Sometimes a project might require a dependency to run on a specific platform. This could to a package that back ports a module so that it can be used in older python versions. Or it could be a package that is required to run on a specific operating system. This will allow a project to work on multiple different platforms without installing dependencies that are not required for a platform that is installing the project.
For example, here is a project that uses the enum
module and pywin32
:
setup(
name="Project",
...
install_requires=[
"enum34;python_version<'3.4'",
"pywin32 >= 1.0;platform_system=='Windows'"
]
)
Since the enum
module was added in Python 3.4, it should only be installed if the python version is earlier. Since pywin32
will only be used on windows, it should only be installed when the operating system is Windows. Specifying version requirements for the dependencies is supported as normal.
The environmental markers that may be used for testing platform types are detailed in PEP 508.
Including Data Files
The distutils have traditionally allowed installation of “data files”, which are placed in a platform-specific location. However, the most common use case for data files distributed with a package is for use by the package, usually by including the data files in the package directory.
Setuptools offers three ways to specify data files to be included in your packages. First, you can simply use the include_package_data
keyword, e.g.:
from setuptools import setup, find_packages
setup(
...
include_package_data=True
)
This tells setuptools to install any data files it finds in your packages. The data files must be specified via the distutils’ MANIFEST.in
file. (They can also be tracked by a revision control system, using an appropriate plugin. See the section below on Adding Support for Revision Control Systems for information on how to write such plugins.)
If you want finer-grained control over what files are included (for example, if you have documentation files in your package directories and want to exclude them from installation), then you can also use the package_data
keyword, e.g.:
from setuptools import setup, find_packages
setup(
...
package_data={
# If any package contains *.txt or *.rst files, include them:
"": ["*.txt", "*.rst"],
# And include any *.msg files found in the "hello" package, too:
"hello": ["*.msg"],
}
)
The package_data
argument is a dictionary that maps from package names to lists of glob patterns. The globs may include subdirectory names, if the data files are contained in a subdirectory of the package. For example, if the package tree looks like this:
setup.py
src/
mypkg/
__init__.py
mypkg.txt
data/
somefile.dat
otherdata.dat
The setuptools setup file might look like this:
from setuptools import setup, find_packages
setup(
...
packages=find_packages("src"), # include all packages under src
package_dir={"": "src"}, # tell distutils packages are under src
package_data={
# If any package contains *.txt files, include them:
"": ["*.txt"],
# And include any *.dat files found in the "data" subdirectory
# of the "mypkg" package, also:
"mypkg": ["data/*.dat"],
}
)
Notice that if you list patterns in package_data
under the empty string, these patterns are used to find files in every package, even ones that also have their own patterns listed. Thus, in the above example, the mypkg.txt
file gets included even though it’s not listed in the patterns for mypkg
.
Also notice that if you use paths, you must use a forward slash (/
) as the path separator, even if you are on Windows. Setuptools automatically converts slashes to appropriate platform-specific separators at build time.
If datafiles are contained in a subdirectory of a package that isn’t a package itself (no __init__.py
), then the subdirectory names (or *
) are required in the package_data
argument (as shown above with "data/*.dat"
).
When building an sdist
, the datafiles are also drawn from the package_name.egg-info/SOURCES.txt
file, so make sure that this is removed if the setup.py
package_data
list is updated before calling setup.py
.
(Note: although the package_data
argument was previously only available in setuptools
, it was also added to the Python distutils
package as of Python 2.4; there is some documentation for the feature available on the python.org website. If using the setuptools-specific include_package_data
argument, files specified by package_data
will not be automatically added to the manifest unless they are listed in the MANIFEST.in file.)
Sometimes, the include_package_data
or package_data
options alone aren’t sufficient to precisely define what files you want included. For example, you may want to include package README files in your revision control system and source distributions, but exclude them from being installed. So, setuptools offers an exclude_package_data
option as well, that allows you to do things like this:
from setuptools import setup, find_packages
setup(
...
packages=find_packages("src"), # include all packages under src
package_dir={"": "src"}, # tell distutils packages are under src
include_package_data=True, # include everything in source control
# ...but exclude README.txt from all packages
exclude_package_data={"": ["README.txt"]},
)
The exclude_package_data
option is a dictionary mapping package names to lists of wildcard patterns, just like the package_data
option. And, just as with that option, a key of ""
will apply the given pattern(s) to all packages. However, any files that match these patterns will be excluded from installation, even if they were listed in package_data
or were included as a result of using include_package_data
.
In summary, the three options allow you to:
include_package_data
- Accept all data files and directories matched by
MANIFEST.in
. package_data
- Specify additional patterns to match files that may or may not be matched by
MANIFEST.in
or found in source control. exclude_package_data
- Specify patterns for data files and directories that should not be included when a package is installed, even if they would otherwise have been included due to the use of the preceding options.
NOTE: Due to the way the distutils build process works, a data file that you include in your project and then stop including may be “orphaned” in your project’s build directories, requiring you to run setup.py clean --all
to fully remove them. This may also be important for your users and contributors if they track intermediate revisions of your project using Subversion; be sure to let them know when you make changes that remove files from inclusion so they can run setup.py clean --all
.
Accessing Data Files at Runtime
Typically, existing programs manipulate a package’s __file__
attribute in order to find the location of data files. However, this manipulation isn’t compatible with PEP 302-based import hooks, including importing from zip files and Python Eggs. It is strongly recommended that, if you are using data files, you should use the ResourceManager API of pkg_resources
to access them. The pkg_resources
module is distributed as part of setuptools, so if you’re using setuptools to distribute your package, there is no reason not to use its resource management API. See also Importlib Resources for a quick example of converting code that uses __file__
to use pkg_resources
instead.
Non-Package Data Files
Historically, setuptools
by way of easy_install
would encapsulate data files from the distribution into the egg (see the old docs). As eggs are deprecated and pip-based installs fall back to the platform-specific location for installing data files, there is no supported facility to reliably retrieve these resources.
Instead, the PyPA recommends that any data files you wish to be accessible at run time be included in the package.
Automatic Resource Extraction
If you are using tools that expect your resources to be “real” files, or your project includes non-extension native libraries or other files that your C extensions expect to be able to access, you may need to list those files in the eager_resources
argument to setup()
, so that the files will be extracted together, whenever a C extension in the project is imported.
This is especially important if your project includes shared libraries other than distutils-built C extensions, and those shared libraries use file extensions other than .dll
, .so
, or .dylib
, which are the extensions that setuptools 0.6a8 and higher automatically detects as shared libraries and adds to the native_libs.txt
file for you. Any shared libraries whose names do not end with one of those extensions should be listed as eager_resources
, because they need to be present in the filesystem when he C extensions that link to them are used.
The pkg_resources
runtime for compressed packages will automatically extract all C extensions and eager_resources
at the same time, whenever any C extension or eager resource is requested via the resource_filename()
API. (C extensions are imported using resource_filename()
internally.) This ensures that C extensions will see all of the “real” files that they expect to see.
Note also that you can list directory resource names in eager_resources
as well, in which case the directory’s contents (including subdirectories) will be extracted whenever any C extension or eager resource is requested.
Please note that if you’re not sure whether you need to use this argument, you don’t! It’s really intended to support projects with lots of non-Python dependencies and as a last resort for crufty projects that can’t otherwise handle being compressed. If your package is pure Python, Python plus data files, or Python plus C, you really don’t need this. You’ve got to be using either C or an external program that needs “real” files in your project before there’s any possibility of eager_resources
being relevant to your project.
Extensible Applications and Frameworks
Dynamic Discovery of Services and Plugins
setuptools
supports creating libraries that “plug in” to extensible applications and frameworks, by letting you register “entry points” in your project that can be imported by the application or framework.
For example, suppose that a blogging tool wants to support plugins that provide translation for various file types to the blog’s output format. The framework might define an “entry point group” called blogtool.parsers
, and then allow plugins to register entry points for the file extensions they support.
This would allow people to create distributions that contain one or more parsers for different file types, and then the blogging tool would be able to find the parsers at runtime by looking up an entry point for the file extension (or mime type, or however it wants to).
Note that if the blogging tool includes parsers for certain file formats, it can register these as entry points in its own setup script, which means it doesn’t have to special-case its built-in formats. They can just be treated the same as any other plugin’s entry points would be.
If you’re creating a project that plugs in to an existing application or framework, you’ll need to know what entry points or entry point groups are defined by that application or framework. Then, you can register entry points in your setup script. Here are a few examples of ways you might register an .rst
file parser entry point in the blogtool.parsers
entry point group, for our hypothetical blogging tool:
setup(
# ...
entry_points={"blogtool.parsers": ".rst = some_module:SomeClass"}
)
setup(
# ...
entry_points={"blogtool.parsers": [".rst = some_module:a_func"]}
)
setup(
# ...
entry_points="""
[blogtool.parsers]
.rst = some.nested.module:SomeClass.some_classmethod [reST]
""",
extras_require=dict(reST="Docutils>=0.3.5")
)
The entry_points
argument to setup()
accepts either a string with .ini
-style sections, or a dictionary mapping entry point group names to either strings or lists of strings containing entry point specifiers. An entry point specifier consists of a name and value, separated by an =
sign. The value consists of a dotted module name, optionally followed by a :
and a dotted identifier naming an object within the module. It can also include a bracketed list of “extras” that are required for the entry point to be used. When the invoking application or framework requests loading of an entry point, any requirements implied by the associated extras will be passed to pkg_resources.require()
, so that an appropriate error message can be displayed if the needed package(s) are missing. (Of course, the invoking app or framework can ignore such errors if it wants to make an entry point optional if a requirement isn’t installed.)
Defining Additional Metadata
Some extensible applications and frameworks may need to define their own kinds of metadata to include in eggs, which they can then access using the pkg_resources
metadata APIs. Ordinarily, this is done by having plugin developers include additional files in their ProjectName.egg-info
directory. However, since it can be tedious to create such files by hand, you may want to create a distutils extension that will create the necessary files from arguments to setup()
, in much the same way that setuptools
does for many of the setup()
arguments it adds. See the section below on Creating distutils Extensions for more details, especially the subsection on Adding new EGG-INFO Files.
“Development Mode”
Under normal circumstances, the distutils
assume that you are going to build a distribution of your project, not use it in its “raw” or “unbuilt” form. If you were to use the distutils
that way, you would have to rebuild and reinstall your project every time you made a change to it during development.
Another problem that sometimes comes up with the distutils
is that you may need to do development on two related projects at the same time. You may need to put both projects’ packages in the same directory to run them, but need to keep them separate for revision control purposes. How can you do this?
Setuptools allows you to deploy your projects for use in a common directory or staging area, but without copying any files. Thus, you can edit each project’s code in its checkout directory, and only need to run build commands when you change a project’s C extensions or similarly compiled files. You can even deploy a project into another project’s checkout directory, if that’s your preferred way of working (as opposed to using a common independent staging area or the site-packages directory).
To do this, use the setup.py develop
command. It works very similarly to setup.py install
, except that it doesn’t actually install anything. Instead, it creates a special .egg-link
file in the deployment directory, that links to your project’s source code. And, if your deployment directory is Python’s site-packages
directory, it will also update the easy-install.pth
file to include your project’s source code, thereby making it available on sys.path
for all programs using that Python installation.
If you have enabled the use_2to3
flag, then of course the .egg-link
will not link directly to your source code when run under Python 3, since that source code would be made for Python 2 and not work under Python 3. Instead the setup.py develop
will build Python 3 code under the build
directory, and link there. This means that after doing code changes you will have to run setup.py build
before these changes are picked up by your Python 3 installation.
In addition, the develop
command creates wrapper scripts in the target script directory that will run your in-development scripts after ensuring that all your install_requires
packages are available on sys.path
.
You can deploy the same project to multiple staging areas, e.g. if you have multiple projects on the same machine that are sharing the same project you’re doing development work.
When you’re done with a given development task, you can remove the project source from a staging area using setup.py develop --uninstall
, specifying the desired staging area if it’s not the default.
There are several options to control the precise behavior of the develop
command; see the section on the develop command below for more details.
Note that you can also apply setuptools commands to non-setuptools projects, using commands like this:
python -c "import setuptools; with open('setup.py') as f: exec(compile(f.read(), 'setup.py', 'exec'))" develop
That is, you can simply list the normal setup commands and options following the quoted part.
Distributing a setuptools
-based project
Detailed instructions to distribute a setuptools project can be found at Packaging project tutorials.
Before you begin, make sure you have the latest versions of setuptools and wheel:
pip install --upgrade setuptools wheel
To build a setuptools project, run this command from the same directory where setup.py is located:
setup.py sdist bdist_wheel
This will generate distribution archives in the dist directory.
Before you upload the generated archives make sure you’re registered on https://test.pypi.org/account/register/. You will also need to verify your email to be able to upload any packages. You should install twine to be able to upload packages:
pip install --upgrade twine
Now, to upload these archives, run:
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
To install your newly uploaded package example_pkg
, you can use pip:
pip install --index-url https://test.pypi.org/simple/ example_pkg
If you have issues at any point, please refer to Packaging project tutorials for clarification.
Setting the zip_safe
flag
For some use cases (such as bundling as part of a larger application), Python packages may be run directly from a zip file. Not all packages, however, are capable of running in compressed form, because they may expect to be able to access either source code or data files as normal operating system files. So, setuptools
can install your project as a zipfile or a directory, and its default choice is determined by the project’s zip_safe
flag.
You can pass a True or False value for the zip_safe
argument to the setup()
function, or you can omit it. If you omit it, the bdist_egg
command will analyze your project’s contents to see if it can detect any conditions that would prevent it from working in a zipfile. It will output notices to the console about any such conditions that it finds.
Currently, this analysis is extremely conservative: it will consider the project unsafe if it contains any C extensions or datafiles whatsoever. This does not mean that the project can’t or won’t work as a zipfile! It just means that the bdist_egg
authors aren’t yet comfortable asserting that the project will work. If the project contains no C or data files, and does no __file__
or __path__
introspection or source code manipulation, then there is an extremely solid chance the project will work when installed as a zipfile. (And if the project uses pkg_resources
for all its data file access, then C extensions and other data files shouldn’t be a problem at all. See the Accessing Data Files at Runtime section above for more information.)
However, if bdist_egg
can’t be sure that your package will work, but you’ve checked over all the warnings it issued, and you are either satisfied it will work (or if you want to try it for yourself), then you should set zip_safe
to True
in your setup()
call. If it turns out that it doesn’t work, you can always change it to False
, which will force setuptools
to install your project as a directory rather than as a zipfile.
In the future, as we gain more experience with different packages and become more satisfied with the robustness of the pkg_resources
runtime, the “zip safety” analysis may become less conservative. However, we strongly recommend that you determine for yourself whether your project functions correctly when installed as a zipfile, correct any problems if you can, and then make an explicit declaration of True
or False
for the zip_safe
flag, so that it will not be necessary for bdist_egg
to try to guess whether your project can work as a zipfile.
Namespace Packages
Sometimes, a large package is more useful if distributed as a collection of smaller eggs. However, Python does not normally allow the contents of a package to be retrieved from more than one location. “Namespace packages” are a solution for this problem. When you declare a package to be a namespace package, it means that the package has no meaningful contents in its __init__.py
, and that it is merely a container for modules and subpackages.
The pkg_resources
runtime will then automatically ensure that the contents of namespace packages that are spread over multiple eggs or directories are combined into a single “virtual” package.
The namespace_packages
argument to setup()
lets you declare your project’s namespace packages, so that they will be included in your project’s metadata. The argument should list the namespace packages that the egg participates in. For example, the ZopeInterface project might do this:
setup(
# ...
namespace_packages=["zope"]
)
because it contains a zope.interface
package that lives in the zope
namespace package. Similarly, a project for a standalone zope.publisher
would also declare the zope
namespace package. When these projects are installed and used, Python will see them both as part of a “virtual” zope
package, even though they will be installed in different locations.
Namespace packages don’t have to be top-level packages. For example, Zope 3’s zope.app
package is a namespace package, and in the future PEAK’s peak.util
package will be too.
Note, by the way, that your project’s source tree must include the namespace packages’ __init__.py
files (and the __init__.py
of any parent packages), in a normal Python package layout. These __init__.py
files must contain the line:
__import__("pkg_resources").declare_namespace(__name__)
This code ensures that the namespace package machinery is operating and that the current package is registered as a namespace package.
You must NOT include any other code and data in a namespace package’s __init__.py
. Even though it may appear to work during development, or when projects are installed as .egg
files, it will not work when the projects are installed using “system” packaging tools – in such cases the __init__.py
files will not be installed, let alone executed.
You must include the declare_namespace()
line in the __init__.py
of every project that has contents for the namespace package in question, in order to ensure that the namespace will be declared regardless of which project’s copy of __init__.py
is loaded first. If the first loaded __init__.py
doesn’t declare it, it will never be declared, because no other copies will ever be loaded!
TRANSITIONAL NOTE
Setuptools automatically calls declare_namespace()
for you at runtime, but future versions may not. This is because the automatic declaration feature has some negative side effects, such as needing to import all namespace packages during the initialization of the pkg_resources
runtime, and also the need for pkg_resources
to be explicitly imported before any namespace packages work at all. In some future releases, you’ll be responsible for including your own declaration lines, and the automatic declaration feature will be dropped to get rid of the negative side effects.
During the remainder of the current development cycle, therefore, setuptools will warn you about missing declare_namespace()
calls in your __init__.py
files, and you should correct these as soon as possible before the compatibility support is removed. Namespace packages without declaration lines will not work correctly once a user has upgraded to a later version, so it’s important that you make this change now in order to avoid having your code break in the field. Our apologies for the inconvenience, and thank you for your patience.
Tagging and “Daily Build” or “Snapshot” Releases
When a set of related projects are under development, it may be important to track finer-grained version increments than you would normally use for e.g. “stable” releases. While stable releases might be measured in dotted numbers with alpha/beta/etc. status codes, development versions of a project often need to be tracked by revision or build number or even build date. This is especially true when projects in development need to refer to one another, and therefore may literally need an up-to-the-minute version of something!
To support these scenarios, setuptools
allows you to “tag” your source and egg distributions by adding one or more of the following to the project’s “official” version identifier:
- A manually-specified pre-release tag, such as “build” or “dev”, or a manually-specified post-release tag, such as a build or revision number (
--tag-build=STRING, -bSTRING
) - An 8-character representation of the build date (
--tag-date, -d
), as a postrelease tag
You can add these tags by adding egg_info
and the desired options to the command line ahead of the sdist
or bdist
commands that you want to generate a daily build or snapshot for. See the section below on the egg_info command for more details.
(Also, before you release your project, be sure to see the section above on Specifying Your Project’s Version for more information about how pre- and post-release tags affect how version numbers are interpreted. This is important in order to make sure that dependency processing tools will know which versions of your project are newer than others.)
Finally, if you are creating builds frequently, and either building them in a downloadable location or are copying them to a distribution server, you should probably also check out the rotate command, which lets you automatically delete all but the N most-recently-modified distributions matching a glob pattern. So, you can use a command line like:
setup.py egg_info -rbDEV bdist_egg rotate -m.egg -k3
to build an egg whose version info includes “DEV-rNNNN” (where NNNN is the most recent Subversion revision that affected the source tree), and then delete any egg files from the distribution directory except for the three that were built most recently.
If you have to manage automated builds for multiple packages, each with different tagging and rotation policies, you may also want to check out the alias command, which would let each package define an alias like daily
that would perform the necessary tag, build, and rotate commands. Then, a simpler script or cron job could just run setup.py daily
in each project directory. (And, you could also define sitewide or per-user default versions of the daily
alias, so that projects that didn’t define their own would use the appropriate defaults.)
Generating Source Distributions
setuptools
enhances the distutils’ default algorithm for source file selection with pluggable endpoints for looking up files to include. If you are using a revision control system, and your source distributions only need to include files that you’re tracking in revision control, use a corresponding plugin instead of writing a MANIFEST.in
file. See the section below on Adding Support for Revision Control Systems for information on plugins.
If you need to include automatically generated files, or files that are kept in an unsupported revision control system, you’ll need to create a MANIFEST.in
file to specify any files that the default file location algorithm doesn’t catch. See the distutils documentation for more information on the format of the MANIFEST.in
file.
But, be sure to ignore any part of the distutils documentation that deals with MANIFEST
or how it’s generated from MANIFEST.in
; setuptools shields you from these issues and doesn’t work the same way in any case. Unlike the distutils, setuptools regenerates the source distribution manifest file every time you build a source distribution, and it builds it inside the project’s .egg-info
directory, out of the way of your main project directory. You therefore need not worry about whether it is up-to-date or not.
Indeed, because setuptools’ approach to determining the contents of a source distribution is so much simpler, its sdist
command omits nearly all of the options that the distutils’ more complex sdist
process requires. For all practical purposes, you’ll probably use only the --formats
option, if you use any option at all.
Making “Official” (Non-Snapshot) Releases
When you make an official release, creating source or binary distributions, you will need to override the tag settings from setup.cfg
, so that you don’t end up registering versions like foobar-0.7a1.dev-r34832
. This is easy to do if you are developing on the trunk and using tags or branches for your releases – just make the change to setup.cfg
after branching or tagging the release, so the trunk will still produce development snapshots.
Alternately, if you are not branching for releases, you can override the default version options on the command line, using something like:
setup.py egg_info -Db "" sdist bdist_egg
The first part of this command (egg_info -Db ""
) will override the configured tag information, before creating source and binary eggs. Thus, these commands will use the plain version from your setup.py
, without adding the build designation string.
Of course, if you will be doing this a lot, you may wish to create a personal alias for this operation, e.g.:
setup.py alias -u release egg_info -Db ""
You can then use it like this:
setup.py release sdist bdist_egg
Or of course you can create more elaborate aliases that do all of the above. See the sections below on the egg_info and alias commands for more ideas.
Distributing Extensions compiled with Cython
setuptools
will detect at build time whether Cython is installed or not. If Cython is not found setuptools
will ignore pyx files.
To ensure Cython is available, include Cython in the build-requires section of your pyproject.toml:
[build-system]
requires=[..., "cython"]
Built with pip 10 or later, that declaration is sufficient to include Cython in the build. For broader compatibility, declare the dependency in your setup-requires of setup.cfg:
[options]
setup_requires =
...
cython
As long as Cython is present in the build environment, setuptools
includes transparent support for building Cython extensions, as long as extensions are defined using setuptools.Extension
.
If you follow these rules, you can safely list .pyx
files as the source of your Extension
objects in the setup script. If it is, then setuptools
will use it.
Of course, for this to work, your source distributions must include the C code generated by Cython, as well as your original .pyx
files. This means that you will probably want to include current .c
files in your revision control system, rebuilding them whenever you check changes in for the .pyx
source files. This will ensure that people tracking your project in a revision control system will be able to build it even if they don’t have Cython installed, and that your source releases will be similarly usable with or without Cython.
Command Reference
alias
– Define shortcuts for commonly used commands
Sometimes, you need to use the same commands over and over, but you can’t necessarily set them as defaults. For example, if you produce both development snapshot releases and “stable” releases of a project, you may want to put the distributions in different places, or use different egg_info
tagging options, etc. In these cases, it doesn’t make sense to set the options in a distutils configuration file, because the values of the options changed based on what you’re trying to do.
Setuptools therefore allows you to define “aliases” – shortcut names for an arbitrary string of commands and options, using setup.py alias aliasname expansion
, where aliasname is the name of the new alias, and the remainder of the command line supplies its expansion. For example, this command defines a sitewide alias called “daily”, that sets various egg_info
tagging options:
setup.py alias --global-config daily egg_info --tag-build=development
Once the alias is defined, it can then be used with other setup commands, e.g.:
setup.py daily bdist_egg # generate a daily-build .egg file
setup.py daily sdist # generate a daily-build source distro
setup.py daily sdist bdist_egg # generate both
The above commands are interpreted as if the word daily
were replaced with egg_info --tag-build=development
.
Note that setuptools will expand each alias at most once in a given command line. This serves two purposes. First, if you accidentally create an alias loop, it will have no effect; you’ll instead get an error message about an unknown command. Second, it allows you to define an alias for a command, that uses that command. For example, this (project-local) alias:
setup.py alias bdist_egg bdist_egg rotate -k1 -m.egg
redefines the bdist_egg
command so that it always runs the rotate
command afterwards to delete all but the newest egg file. It doesn’t loop indefinitely on bdist_egg
because the alias is only expanded once when used.
You can remove a defined alias with the --remove
(or -r
) option, e.g.:
setup.py alias --global-config --remove daily
would delete the “daily” alias we defined above.
Aliases can be defined on a project-specific, per-user, or sitewide basis. The default is to define or remove a project-specific alias, but you can use any of the configuration file options (listed under the saveopts command, below) to determine which distutils configuration file an aliases will be added to (or removed from).
Note that if you omit the “expansion” argument to the alias
command, you’ll get output showing that alias’ current definition (and what configuration file it’s defined in). If you omit the alias name as well, you’ll get a listing of all current aliases along with their configuration file locations.
bdist_egg
– Create a Python Egg for the project
Warning
eggs are deprecated in favor of wheels, and not supported by pip.
This command generates a Python Egg (.egg
file) for the project. Python Eggs are the preferred binary distribution format for EasyInstall, because they are cross-platform (for “pure” packages), directly importable, and contain project metadata including scripts and information about the project’s dependencies. They can be simply downloaded and added to sys.path
directly, or they can be placed in a directory on sys.path
and then automatically discovered by the egg runtime system.
This command runs the egg_info command (if it hasn’t already run) to update the project’s metadata (.egg-info
) directory. If you have added any extra metadata files to the .egg-info
directory, those files will be included in the new egg file’s metadata directory, for use by the egg runtime system or by any applications or frameworks that use that metadata.
You won’t usually need to specify any special options for this command; just use bdist_egg
and you’re done. But there are a few options that may be occasionally useful:
--dist-dir=DIR, -d DIR
- Set the directory where the
.egg
file will be placed. If you don’t supply this, then the--dist-dir
setting of thebdist
command will be used, which is usually a directory nameddist
in the project directory. --plat-name=PLATFORM, -p PLATFORM
- Set the platform name string that will be embedded in the egg’s filename (assuming the egg contains C extensions). This can be used to override the distutils default platform name with something more meaningful. Keep in mind, however, that the egg runtime system expects to see eggs with distutils platform names, so it may ignore or reject eggs with non-standard platform names. Similarly, the EasyInstall program may ignore them when searching web pages for download links. However, if you are cross-compiling or doing some other unusual things, you might find a use for this option.
--exclude-source-files
- Don’t include any modules’
.py
files in the egg, just compiled Python, C, and data files. (Note that this doesn’t affect any.py
files in the EGG-INFO directory or its subdirectories, since for example there may be scripts with a.py
extension which must still be retained.) We don’t recommend that you use this option except for packages that are being bundled for proprietary end-user applications, or for “embedded” scenarios where space is at an absolute premium. On the other hand, if your package is going to be installed and used in compressed form, you might as well exclude the source because Python’straceback
module doesn’t currently understand how to display zipped source code anyway, or how to deal with files that are in a different place from where their code was compiled.
There are also some options you will probably never need, but which are there because they were copied from similar bdist
commands used as an example for creating this one. They may be useful for testing and debugging, however, which is why we kept them:
--keep-temp, -k
- Keep the contents of the
--bdist-dir
tree around after creating the.egg
file. --bdist-dir=DIR, -b DIR
- Set the temporary directory for creating the distribution. The entire contents of this directory are zipped to create the
.egg
file, after running various installation commands to copy the package’s modules, data, and extensions here. --skip-build
- Skip doing any “build” commands; just go straight to the install-and-compress phases.
develop
– Deploy the project source in “Development Mode”
This command allows you to deploy your project’s source for use in one or more “staging areas” where it will be available for importing. This deployment is done in such a way that changes to the project source are immediately available in the staging area(s), without needing to run a build or install step after each change.
The develop
command works by creating an .egg-link
file (named for the project) in the given staging area. If the staging area is Python’s site-packages
directory, it also updates an easy-install.pth
file so that the project is on sys.path
by default for all programs run using that Python installation.
The develop
command also installs wrapper scripts in the staging area (or a separate directory, as specified) that will ensure the project’s dependencies are available on sys.path
before running the project’s source scripts. And, it ensures that any missing project dependencies are available in the staging area, by downloading and installing them if necessary.
Last, but not least, the develop
command invokes the build_ext -i
command to ensure any C extensions in the project have been built and are up-to-date, and the egg_info
command to ensure the project’s metadata is updated (so that the runtime and wrappers know what the project’s dependencies are). If you make any changes to the project’s setup script or C extensions, you should rerun the develop
command against all relevant staging areas to keep the project’s scripts, metadata and extensions up-to-date. Most other kinds of changes to your project should not require any build operations or rerunning develop
, but keep in mind that even minor changes to the setup script (e.g. changing an entry point definition) require you to re-run the develop
or test
commands to keep the distribution updated.
Here are some of the options that the develop
command accepts. Note that they affect the project’s dependencies as well as the project itself, so if you have dependencies that need to be installed and you use --exclude-scripts
(for example), the dependencies’ scripts will not be installed either! For this reason, you may want to use pip to install the project’s dependencies before using the develop
command, if you need finer control over the installation options for dependencies.
--uninstall, -u
-
Un-deploy the current project. You may use the
--install-dir
or-d
option to designate the staging area. The created.egg-link
file will be removed, if present and it is still pointing to the project directory. The project directory will be removed fromeasy-install.pth
if the staging area is Python’ssite-packages
directory.Note that this option currently does not uninstall script wrappers! You must uninstall them yourself, or overwrite them by using pip to install a different version of the package. You can also avoid installing script wrappers in the first place, if you use the
--exclude-scripts
(aka-x
) option when you rundevelop
to deploy the project. --multi-version, -m
-
“Multi-version” mode. Specifying this option prevents
develop
from adding aneasy-install.pth
entry for the project(s) being deployed, and if an entry for any version of a project already exists, the entry will be removed upon successful deployment. In multi-version mode, no specific version of the package is available for importing, unless you usepkg_resources.require()
to put it onsys.path
, or you are running a wrapper script generated bysetuptools
. (In which case the wrapper script callsrequire()
for you.)Note that if you install to a directory other than
site-packages
, this option is automatically in effect, because.pth
files can only be used insite-packages
(at least in Python 2.3 and 2.4). So, if you use the--install-dir
or-d
option (or they are set via configuration file(s)) your project and its dependencies will be deployed in multi- version mode. --install-dir=DIR, -d DIR
- Set the installation directory (staging area). If this option is not directly specified on the command line or in a distutils configuration file, the distutils default installation location is used. Normally, this will be the
site-packages
directory, but if you are using distutils configuration files, setting things likeprefix
orinstall_lib
, then those settings are taken into account when computing the default staging area. --script-dir=DIR, -s DIR
- Set the script installation directory. If you don’t supply this option (via the command line or a configuration file), but you have supplied an
--install-dir
(via command line or config file), then this option defaults to the same directory, so that the scripts will be able to find their associated package installation. Otherwise, this setting defaults to the location where the distutils would normally install scripts, taking any distutils configuration file settings into account. --exclude-scripts, -x
- Don’t deploy script wrappers. This is useful if you don’t want to disturb existing versions of the scripts in the staging area.
--always-copy, -a
- Copy all needed distributions to the staging area, even if they are already present in another directory on
sys.path
. By default, if a requirement can be met using a distribution that is already available in a directory onsys.path
, it will not be copied to the staging area. --egg-path=DIR
- Force the generated
.egg-link
file to use a specified relative path to the source directory. This can be useful in circumstances where your installation directory is being shared by code running under multiple platforms (e.g. Mac and Windows) which have different absolute locations for the code under development, but the same relative locations with respect to the installation directory. If you use this option when installing, you must supply the same relative path when uninstalling.
In addition to the above options, the develop
command also accepts all of the same options accepted by easy_install
. If you’ve configured any easy_install
settings in your setup.cfg
(or other distutils config files), the develop
command will use them as defaults, unless you override them in a [develop]
section or on the command line.
rotate
– Delete outdated distribution files
As you develop new versions of your project, your distribution (dist
) directory will gradually fill up with older source and/or binary distribution files. The rotate
command lets you automatically clean these up, keeping only the N most-recently modified files matching a given pattern.
--match=PATTERNLIST, -m PATTERNLIST
- Comma-separated list of glob patterns to match. This option is required. The project name and
-*
is prepended to the supplied patterns, in order to match only distributions belonging to the current project (in case you have a shared distribution directory for multiple projects). Typically, you will use a glob pattern like.zip
or.egg
to match files of the specified type. Note that each supplied pattern is treated as a distinct group of files for purposes of selecting files to delete. --keep=COUNT, -k COUNT
- Number of matching distributions to keep. For each group of files identified by a pattern specified with the
--match
option, delete all but the COUNT most-recently-modified files in that group. This option is required. --dist-dir=DIR, -d DIR
- Directory where the distributions are. This defaults to the value of the
bdist
command’s--dist-dir
option, which will usually be the project’sdist
subdirectory.
Example 1: Delete all .tar.gz files from the distribution directory, except for the 3 most recently modified ones:
setup.py rotate --match=.tar.gz --keep=3
Example 2: Delete all Python 2.3 or Python 2.4 eggs from the distribution directory, except the most recently modified one for each Python version:
setup.py rotate --match=-py2.3*.egg,-py2.4*.egg --keep=1
saveopts
– Save used options to a configuration file
Finding and editing distutils
configuration files can be a pain, especially since you also have to translate the configuration options from command-line form to the proper configuration file format. You can avoid these hassles by using the saveopts
command. Just add it to the command line to save the options you used. For example, this command builds the project using the mingw32
C compiler, then saves the –compiler setting as the default for future builds (even those run implicitly by the install
command):
setup.py build --compiler=mingw32 saveopts
The saveopts
command saves all options for every command specified on the command line to the project’s local setup.cfg
file, unless you use one of the configuration file options to change where the options are saved. For example, this command does the same as above, but saves the compiler setting to the site-wide (global) distutils configuration:
setup.py build --compiler=mingw32 saveopts -g
Note that it doesn’t matter where you place the saveopts
command on the command line; it will still save all the options specified for all commands. For example, this is another valid way to spell the last example:
setup.py saveopts -g build --compiler=mingw32
Note, however, that all of the commands specified are always run, regardless of where saveopts
is placed on the command line.
Configuration File Options
Normally, settings such as options and aliases are saved to the project’s local setup.cfg
file. But you can override this and save them to the global or per-user configuration files, or to a manually-specified filename.
--global-config, -g
- Save settings to the global
distutils.cfg
file inside thedistutils
package directory. You must have write access to that directory to use this option. You also can’t combine this option with-u
or-f
. --user-config, -u
- Save settings to the current user’s
~/.pydistutils.cfg
(POSIX) or$HOME/pydistutils.cfg
(Windows) file. You can’t combine this option with-g
or-f
. --filename=FILENAME, -f FILENAME
- Save settings to the specified configuration file to use. You can’t combine this option with
-g
or-u
. Note that if you specify a non-standard filename, thedistutils
andsetuptools
will not use the file’s contents. This option is mainly included for use in testing.
These options are used by other setuptools
commands that modify configuration files, such as the alias and setopt commands.
setopt
– Set a distutils or setuptools option in a config file
This command is mainly for use by scripts, but it can also be used as a quick and dirty way to change a distutils configuration option without having to remember what file the options are in and then open an editor.
Example 1. Set the default C compiler to mingw32
(using long option names):
setup.py setopt --command=build --option=compiler --set-value=mingw32
Example 2. Remove any setting for the distutils default package installation directory (short option names):
setup.py setopt -c install -o install_lib -r
Options for the setopt
command:
--command=COMMAND, -c COMMAND
- Command to set the option for. This option is required.
--option=OPTION, -o OPTION
- The name of the option to set. This option is required.
--set-value=VALUE, -s VALUE
- The value to set the option to. Not needed if
-r
or--remove
is set. --remove, -r
- Remove (unset) the option, instead of setting it.
In addition to the above options, you may use any of the configuration file options (listed under the saveopts command, above) to determine which distutils configuration file the option will be added to (or removed from).
test
– Build package and run a unittest suite
Warning
test
is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
When doing test-driven development, or running automated builds that need testing before they are deployed for downloading or use, it’s often useful to be able to run a project’s unit tests without actually deploying the project anywhere, even using the develop
command. The test
command runs a project’s unit tests without actually deploying it, by temporarily putting the project’s source on sys.path
, after first running build_ext -i
and egg_info
to ensure that any C extensions and project metadata are up-to-date.
To use this command, your project’s tests must be wrapped in a unittest
test suite by either a function, a TestCase
class or method, or a module or package containing TestCase
classes. If the named suite is a module, and the module has an additional_tests()
function, it is called and the result (which must be a unittest.TestSuite
) is added to the tests to be run. If the named suite is a package, any submodules and subpackages are recursively added to the overall test suite. (Note: if your project specifies a test_loader
, the rules for processing the chosen test_suite
may differ; see the test_loader documentation for more details.)
Note that many test systems including doctest
support wrapping their non-unittest
tests in TestSuite
objects. So, if you are using a test package that does not support this, we suggest you encourage its developers to implement test suite support, as this is a convenient and standard way to aggregate a collection of tests to be run under a common test harness.
By default, tests will be run in the “verbose” mode of the unittest
package’s text test runner, but you can get the “quiet” mode (just dots) if you supply the -q
or --quiet
option, either as a global option to the setup script (e.g. setup.py -q test
) or as an option for the test
command itself (e.g. setup.py test -q
). There is one other option available:
--test-suite=NAME, -s NAME
-
Specify the test suite (or module, class, or method) to be run (e.g.
some_module.test_suite
). The default for this option can be set by giving atest_suite
argument to thesetup()
function, e.g.:setup( # ... test_suite="my_package.tests.test_all" )
If you did not set a
test_suite
in yoursetup()
call, and do not provide a--test-suite
option, an error will occur.
New in 41.5.0: Deprecated the test command.
upload
– Upload source and/or egg distributions to PyPI
The upload
command was deprecated in version 40.0 and removed in version 42.0. Use twine instead.
For more information on the current best practices in uploading your packages to PyPI, see the Python Packaging User Guide’s “Packaging Python Projects” tutorial specifically the section on uploading the distribution archives.
Configuring setup() using setup.cfg files
Note
New in 30.3.0 (8 Dec 2016).
Important
A setup.py
file containing a setup()
function call is still required even if your configuration resides in setup.cfg
.
Setuptools
allows using configuration files (usually setup.cfg
) to define a package’s metadata and other options that are normally supplied to the setup()
function (declarative config).
This approach not only allows automation scenarios but also reduces boilerplate code in some cases.
Note
This implementation has limited compatibility with the distutils2-like setup.cfg
sections used by the pbr
and d2to1
packages.
Namely: only metadata-related keys from metadata
section are supported (except for description-file
); keys from files
, entry_points
and backwards_compat
are not supported.
[metadata]
name = my_package
version = attr: src.VERSION
description = My package description
long_description = file: README.rst, CHANGELOG.rst, LICENSE.rst
keywords = one, two
license = BSD 3-Clause License
classifiers =
Framework :: Django
License :: OSI Approved :: BSD License
Programming Language :: Python :: 3
Programming Language :: Python :: 3.5
[options]
zip_safe = False
include_package_data = True
packages = find:
scripts =
bin/first.py
bin/second.py
install_requires =
requests
importlib; python_version == "2.6"
[options.package_data]
* = *.txt, *.rst
hello = *.msg
[options.extras_require]
pdf = ReportLab>=1.2; RXP
rest = docutils>=0.3; pack ==1.1, ==1.3
[options.packages.find]
exclude =
src.subpackage1
src.subpackage2
[options.data_files]
/etc/my_package =
site.d/00_default.conf
host.d/00_default.conf
data = data/img/logo.png, data/svg/icon.svg
Metadata and options are set in the config sections of the same name.
-
Keys are the same as the keyword arguments one provides to the
setup()
function. -
Complex values can be written comma-separated or placed one per line in dangling config values. The following are equivalent:
[metadata] keywords = one, two [metadata] keywords = one two
-
In some cases, complex values can be provided in dedicated subsections for clarity.
-
Some keys allow
file:
,attr:
, andfind:
andfind_namespace:
directives in order to cover common usecases. -
Unknown keys are ignored.
Using a src/
layout
One commonly used package configuration has all the module source code in a subdirectory (often called the src/
layout), like this:
├── src
│ └── mypackage
│ ├── __init__.py
│ └── mod1.py
├── setup.py
└── setup.cfg
You can set up your setup.cfg
to automatically find all your packages in the subdirectory like this:
# This example contains just the necessary options for a src-layout, set up
# the rest of the file as described above.
[options]
package_dir=
=src
packages=find:
[options.packages.find]
where=src
Specifying values
Some values are treated as simple strings, some allow more logic.
Type names used below:
str
– simple stringlist-comma
– dangling list or string of comma-separated valueslist-semi
– dangling list or string of semicolon-separated valuesbool
–True
is 1, yes, truedict
– list-comma where keys are separated from values by=
section
– values are read from a dedicated (sub)section
Special directives:
attr:
– Value is read from a module attribute.attr:
supports callables and iterables; unsupported types are cast usingstr()
.file:
– Value is read from a list of files and then concatenated
Note
The file:
directive is sandboxed and won’t reach anything outside the directory containing setup.py
.
Metadata
Note
The aliases given below are supported for compatibility reasons, but their use is not advised.
Key | Aliases | Type | Minimum Version | Notes |
---|---|---|---|---|
name | str | |||
version | attr:, file:, str | 39.2.0 |
|
|
url | home-page | str | ||
download_url | download-url | str | ||
project_urls | dict | 38.3.0 | ||
author | str | |||
author_email | author-email | str | ||
maintainer | str | |||
maintainer_email | maintainer-email | str | ||
classifiers | classifier | file:, list-comma | ||
license | str | |||
license_file | str | |||
license_files | list-comma | |||
description | summary | file:, str | ||
long_description | long-description | file:, str | ||
long_description_content_type | str | 38.6.0 | ||
keywords | list-comma | |||
platforms | platform | list-comma | ||
provides | list-comma | |||
requires | list-comma | |||
obsoletes | list-comma |
Note
A version loaded using the file:
directive must comply with PEP 440. It is easy to accidentally put something other than a valid version string in such a file, so validation is stricter in this case.
Notes: 1. The version file attribute has only been supported since 39.2.0.
Options
Key | Type | Minimum Version | Notes |
---|---|---|---|
zip_safe | bool | ||
setup_requires | list-semi | ||
install_requires | list-semi | ||
extras_require | section | ||
python_requires | str | ||
entry_points | file:, section | ||
use_2to3 | bool | ||
use_2to3_fixers | list-comma | ||
use_2to3_exclude_fixers | list-comma | ||
convert_2to3_doctests | list-comma | ||
scripts | list-comma | ||
eager_resources | list-comma | ||
dependency_links | list-comma | ||
tests_require | list-semi | ||
include_package_data | bool | ||
packages | find:, find_namespace:, list-comma | ||
package_dir | dict | ||
package_data | section |
|
|
exclude_package_data | section | ||
namespace_packages | list-comma | ||
py_modules | list-comma | ||
data_files | dict | 40.6.0 |
Note
packages – The find:
and find_namespace:
directive can be further configured in a dedicated subsection options.packages.find
. This subsection accepts the same keys as the setuptools.find_packages and the setuptools.find_namespace_packages function: where
, include
, and exclude
.
find_namespace directive – The find_namespace:
directive is supported since Python >=3.3.
Notes: 1. In the package_data section, a key named with a single asterisk (*) refers to all packages, in lieu of the empty string used in setup.py.
Configuration API
Some automation tools may wish to access data from a configuration file.
Setuptools
exposes a read_configuration()
function for parsing metadata
and options
sections into a dictionary.
from setuptools.config import read_configuration
conf_dict = read_configuration("/home/user/dev/package/setup.cfg")
By default, read_configuration()
will read only the file provided in the first argument. To include values from other configuration files which could be in various places, set the find_others
keyword argument to True
.
If you have only a configuration file but not the whole package, you can still try to get data out of it with the help of the ignore_option_errors
keyword argument. When it is set to True
, all options with errors possibly produced by directives, such as attr:
and others, will be silently ignored. As a consequence, the resulting dictionary will include no such options.
Extending and Reusing Setuptools
Creating distutils
Extensions
It can be hard to add new commands or setup arguments to the distutils. But the setuptools
package makes it a bit easier, by allowing you to distribute a distutils extension as a separate project, and then have projects that need the extension just refer to it in their setup_requires
argument.
With setuptools
, your distutils extension projects can hook in new commands and setup()
arguments just by defining “entry points”. These are mappings from command or argument names to a specification of where to import a handler from. (See the section on Dynamic Discovery of Services and Plugins above for some more background on entry points.)
Adding Commands
You can add new setup
commands by defining entry points in the distutils.commands
group. For example, if you wanted to add a foo
command, you might add something like this to your distutils extension project’s setup script:
setup(
# ...
entry_points={
"distutils.commands": [
"foo = mypackage.some_module:foo",
],
},
)
(Assuming, of course, that the foo
class in mypackage.some_module
is a setuptools.Command
subclass.)
Once a project containing such entry points has been activated on sys.path
, (e.g. by running “install” or “develop” with a site-packages installation directory) the command(s) will be available to any setuptools
-based setup scripts. It is not necessary to use the --command-packages
option or to monkeypatch the distutils.command
package to install your commands; setuptools
automatically adds a wrapper to the distutils to search for entry points in the active distributions on sys.path
. In fact, this is how setuptools’ own commands are installed: the setuptools project’s setup script defines entry points for them!
Adding setup()
Arguments
Warning
Adding arguments to setup is discouraged as such arguments are only supported through imperative execution and not supported through declarative config.
Sometimes, your commands may need additional arguments to the setup()
call. You can enable this by defining entry points in the distutils.setup_keywords
group. For example, if you wanted a setup()
argument called bar_baz
, you might add something like this to your distutils extension project’s setup script:
setup(
# ...
entry_points={
"distutils.commands": [
"foo = mypackage.some_module:foo",
],
"distutils.setup_keywords": [
"bar_baz = mypackage.some_module:validate_bar_baz",
],
},
)
The idea here is that the entry point defines a function that will be called to validate the setup()
argument, if it’s supplied. The Distribution
object will have the initial value of the attribute set to None
, and the validation function will only be called if the setup()
call sets it to a non-None value. Here’s an example validation function:
def assert_bool(dist, attr, value):
"""Verify that value is True, False, 0, or 1"""
if bool(value) != value:
raise DistutilsSetupError(
"%r must be a boolean value (got %r)" % (attr,value)
)
Your function should accept three arguments: the Distribution
object, the attribute name, and the attribute value. It should raise a DistutilsSetupError
(from the distutils.errors
module) if the argument is invalid. Remember, your function will only be called with non-None values, and the default value of arguments defined this way is always None. So, your commands should always be prepared for the possibility that the attribute will be None
when they access it later.
If more than one active distribution defines an entry point for the same setup()
argument, all of them will be called. This allows multiple distutils extensions to define a common argument, as long as they agree on what values of that argument are valid.
Also note that as with commands, it is not necessary to subclass or monkeypatch the distutils Distribution
class in order to add your arguments; it is sufficient to define the entry points in your extension, as long as any setup script using your extension lists your project in its setup_requires
argument.
Customizing Distribution Options
Plugins may wish to extend or alter the options on a Distribution object to suit the purposes of that project. For example, a tool that infers the Distribution.version
from SCM-metadata may need to hook into the option finalization. To enable this feature, Setuptools offers an entry point “setuptools.finalize_distribution_options”. That entry point must be a callable taking one argument (the Distribution instance).
If the callable has an .order
property, that value will be used to determine the order in which the hook is called. Lower numbers are called first and the default is zero (0).
Plugins may read, alter, and set properties on the distribution, but each plugin is encouraged to load the configuration/settings for their behavior independently.
Adding new EGG-INFO Files
Some extensible applications or frameworks may want to allow third parties to develop plugins with application or framework-specific metadata included in the plugins’ EGG-INFO directory, for easy access via the pkg_resources
metadata API. The easiest way to allow this is to create a distutils extension to be used from the plugin projects’ setup scripts (via setup_requires
) that defines a new setup keyword, and then uses that data to write an EGG-INFO file when the egg_info
command is run.
The egg_info
command looks for extension points in an egg_info.writers
group, and calls them to write the files. Here’s a simple example of a distutils extension defining a setup argument foo_bar
, which is a list of lines that will be written to foo_bar.txt
in the EGG-INFO directory of any project that uses the argument:
setup(
# ...
entry_points={
"distutils.setup_keywords": [
"foo_bar = setuptools.dist:assert_string_list",
],
"egg_info.writers": [
"foo_bar.txt = setuptools.command.egg_info:write_arg",
],
},
)
This simple example makes use of two utility functions defined by setuptools for its own use: a routine to validate that a setup keyword is a sequence of strings, and another one that looks up a setup argument and writes it to a file. Here’s what the writer utility looks like:
def write_arg(cmd, basename, filename):
argname = os.path.splitext(basename)[0]
value = getattr(cmd.distribution, argname, None)
if value is not None:
value = "\n".join(value) + "\n"
cmd.write_or_delete_file(argname, filename, value)
As you can see, egg_info.writers
entry points must be a function taking three arguments: a egg_info
command instance, the basename of the file to write (e.g. foo_bar.txt
), and the actual full filename that should be written to.
In general, writer functions should honor the command object’s dry_run
setting when writing files, and use the distutils.log
object to do any console output. The easiest way to conform to this requirement is to use the cmd
object’s write_file()
, delete_file()
, and write_or_delete_file()
methods exclusively for your file operations. See those methods’ docstrings for more details.
Adding Support for Revision Control Systems
If the files you want to include in the source distribution are tracked using Git, Mercurial or SVN, you can use the following packages to achieve that:
- Git and Mercurial: setuptools_scm
- SVN: setuptools_svn
If you would like to create a plugin for setuptools
to find files tracked by another revision control system, you can do so by adding an entry point to the setuptools.file_finders
group. The entry point should be a function accepting a single directory name, and should yield all the filenames within that directory (and any subdirectories thereof) that are under revision control.
For example, if you were going to create a plugin for a revision control system called “foobar”, you would write a function something like this:
def find_files_for_foobar(dirname):
# loop to yield paths that start with `dirname`
And you would register it in a setup script using something like this:
entry_points={
"setuptools.file_finders": [
"foobar = my_foobar_module:find_files_for_foobar",
]
}
Then, anyone who wants to use your plugin can simply install it, and their local setuptools installation will be able to find the necessary files.
It is not necessary to distribute source control plugins with projects that simply use the other source control system, or to specify the plugins in setup_requires
. When you create a source distribution with the sdist
command, setuptools automatically records what files were found in the SOURCES.txt
file. That way, recipients of source distributions don’t need to have revision control at all. However, if someone is working on a package by checking out with that system, they will need the same plugin(s) that the original author is using.
A few important points for writing revision control file finders:
- Your finder function MUST return relative paths, created by appending to the passed-in directory name. Absolute paths are NOT allowed, nor are relative paths that reference a parent directory of the passed-in directory.
- Your finder function MUST accept an empty string as the directory name, meaning the current directory. You MUST NOT convert this to a dot; just yield relative paths. So, yielding a subdirectory named
some/dir
under the current directory should NOT be rendered as./some/dir
or/somewhere/some/dir
, but always as simplysome/dir
- Your finder function SHOULD NOT raise any errors, and SHOULD deal gracefully with the absence of needed programs (i.e., ones belonging to the revision control system itself. It may, however, use
distutils.log.warn()
to inform the user of the missing program(s).
Mailing List and Bug Tracker
Please use the distutils-sig mailing list for questions and discussion about setuptools, and the setuptools bug tracker ONLY for issues you have confirmed via the list are actual bugs, and which you have reduced to a minimal set of steps to reproduce.
Categories: Uncategorized