mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-04 17:08:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13

Add documentation for the packaging module.

Browse source 
This updates the user guide to refer to Packaging instead of Distutils.
Some files still require an update.
This commit is contained in:
Éric Araujo 2011-06-01 20:42:49 +02:00
parent a003af1ce9
commit 3a9f58f6b3
42 changed files with 7388 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

307
Doc/packaging/builtdist.rst Normal file
View file

@ -0,0 +1,307 @@
.. _packaging-built-dist:
****************************
Creating Built Distributions
****************************
A "built distribution" is what you're probably used to thinking of either as a
"binary package" or an "installer" (depending on your background). It's not
necessarily binary, though, because it might contain only Python source code
and/or byte-code; and we don't call it a package, because that word is already
spoken for in Python. (And "installer" is a term specific to the world of
mainstream desktop systems.)
A built distribution is how you make life as easy as possible for installers of
your module distribution: for users of RPM-based Linux systems, it's a binary
RPM; for Windows users, it's an executable installer; for Debian-based Linux
users, it's a Debian package; and so forth. Obviously, no one person will be
able to create built distributions for every platform under the sun, so the
Distutils are designed to enable module developers to concentrate on their
specialty---writing code and creating source distributions---while an
intermediary species called *packagers* springs up to turn source distributions
into built distributions for as many platforms as there are packagers.
Of course, the module developer could be his own packager; or the packager could
be a volunteer "out there" somewhere who has access to a platform which the
original developer does not; or it could be software periodically grabbing new
source distributions and turning them into built distributions for as many
platforms as the software has access to. Regardless of who they are, a packager
uses the setup script and the :command:`bdist` command family to generate built
distributions.
As a simple example, if I run the following command in the Distutils source
tree::
python setup.py bdist
then the Distutils builds my module distribution (the Distutils itself in this
case), does a "fake" installation (also in the :file:`build` directory), and
creates the default type of built distribution for my platform. The default
format for built distributions is a "dumb" tar file on Unix, and a simple
executable installer on Windows. (That tar file is considered "dumb" because it
has to be unpacked in a specific location to work.)
Thus, the above command on a Unix system creates
:file:`Distutils-1.0.{plat}.tar.gz`; unpacking this tarball from the right place
installs the Distutils just as though you had downloaded the source distribution
and run ``python setup.py install``. (The "right place" is either the root of
the filesystem or Python's :file:`{prefix}` directory, depending on the options
given to the :command:`bdist_dumb` command; the default is to make dumb
distributions relative to :file:`{prefix}`.)
Obviously, for pure Python distributions, this isn't any simpler than just
running ``python setup.py install``\ ---but for non-pure distributions, which
include extensions that would need to be compiled, it can mean the difference
between someone being able to use your extensions or not. And creating "smart"
built distributions, such as an executable installer for
Windows, is far more convenient for users even if your distribution doesn't
include any extensions.
The :command:`bdist` command has a :option:`--formats` option, similar to the
:command:`sdist` command, which you can use to select the types of built
distribution to generate: for example, ::
python setup.py bdist --format=zip
would, when run on a Unix system, create :file:`Distutils-1.0.{plat}.zip`\
---again, this archive would be unpacked from the root directory to install the
Distutils.
The available formats for built distributions are:
+-------------+------------------------------+---------+
| Format | Description | Notes |
+=============+==============================+=========+
| ``gztar`` | gzipped tar file | (1),(3) |
| | (:file:`.tar.gz`) | |
+-------------+------------------------------+---------+
| ``ztar`` | compressed tar file | \(3) |
| | (:file:`.tar.Z`) | |
+-------------+------------------------------+---------+
| ``tar`` | tar file (:file:`.tar`) | \(3) |
+-------------+------------------------------+---------+
| ``zip`` | zip file (:file:`.zip`) | (2),(4) |
+-------------+------------------------------+---------+
| ``wininst`` | self-extracting ZIP file for | \(4) |
| | Windows | |
+-------------+------------------------------+---------+
| ``msi`` | Microsoft Installer. | |
+-------------+------------------------------+---------+
Notes:
(1)
default on Unix
(2)
default on Windows
(3)
requires external utilities: :program:`tar` and possibly one of :program:`gzip`,
:program:`bzip2`, or :program:`compress`
(4)
requires either external :program:`zip` utility or :mod:`zipfile` module (part
of the standard Python library since Python 1.6)
You don't have to use the :command:`bdist` command with the :option:`--formats`
option; you can also use the command that directly implements the format you're
interested in. Some of these :command:`bdist` "sub-commands" actually generate
several similar formats; for instance, the :command:`bdist_dumb` command
generates all the "dumb" archive formats (``tar``, ``ztar``, ``gztar``, and
``zip``). The :command:`bdist` sub-commands, and the formats generated by
each, are:
+--------------------------+-----------------------+
| Command | Formats |
+==========================+=======================+
| :command:`bdist_dumb` | tar, ztar, gztar, zip |
+--------------------------+-----------------------+
| :command:`bdist_wininst` | wininst |
+--------------------------+-----------------------+
| :command:`bdist_msi` | msi |
+--------------------------+-----------------------+
The following sections give details on the individual :command:`bdist_\*`
commands.
.. _packaging-creating-dumb:
Creating dumb built distributions
=================================
.. XXX Need to document absolute vs. prefix-relative packages here, but first
I have to implement it!
.. _packaging-creating-wininst:
Creating Windows Installers
===========================
Executable installers are the natural format for binary distributions on
Windows. They display a nice graphical user interface, display some information
about the module distribution to be installed taken from the metadata in the
setup script, let the user select a few options, and start or cancel the
installation.
Since the metadata is taken from the setup script, creating Windows installers
is usually as easy as running::
python setup.py bdist_wininst
or the :command:`bdist` command with the :option:`--formats` option::
python setup.py bdist --formats=wininst
If you have a pure module distribution (only containing pure Python modules and
packages), the resulting installer will be version independent and have a name
like :file:`foo-1.0.win32.exe`. These installers can even be created on Unix
platforms or Mac OS X.
If you have a non-pure distribution, the extensions can only be created on a
Windows platform, and will be Python version dependent. The installer filename
will reflect this and now has the form :file:`foo-1.0.win32-py2.0.exe`. You
have to create a separate installer for every Python version you want to
support.
.. TODO Add :term: markup to bytecode when merging into the stdlib
The installer will try to compile pure modules into bytecode after installation
on the target system in normal and optimizing mode. If you don't want this to
happen for some reason, you can run the :command:`bdist_wininst` command with
the :option:`--no-target-compile` and/or the :option:`--no-target-optimize`
option.
By default the installer will display the cool "Python Powered" logo when it is
run, but you can also supply your own 152x261 bitmap which must be a Windows
:file:`.bmp` file with the :option:`--bitmap` option.
The installer will also display a large title on the desktop background window
when it is run, which is constructed from the name of your distribution and the
version number. This can be changed to another text by using the
:option:`--title` option.
The installer file will be written to the "distribution directory" --- normally
:file:`dist/`, but customizable with the :option:`--dist-dir` option.
.. _packaging-cross-compile-windows:
Cross-compiling on Windows
==========================
Starting with Python 2.6, packaging is capable of cross-compiling between
Windows platforms. In practice, this means that with the correct tools
installed, you can use a 32bit version of Windows to create 64bit extensions
and vice-versa.
To build for an alternate platform, specify the :option:`--plat-name` option
to the build command. Valid values are currently 'win32', 'win-amd64' and
'win-ia64'. For example, on a 32bit version of Windows, you could execute::
python setup.py build --plat-name=win-amd64
to build a 64bit version of your extension. The Windows Installers also
support this option, so the command::
python setup.py build --plat-name=win-amd64 bdist_wininst
would create a 64bit installation executable on your 32bit version of Windows.
To cross-compile, you must download the Python source code and cross-compile
Python itself for the platform you are targetting - it is not possible from a
binary installtion of Python (as the .lib etc file for other platforms are
not included.) In practice, this means the user of a 32 bit operating
system will need to use Visual Studio 2008 to open the
:file:`PCBuild/PCbuild.sln` solution in the Python source tree and build the
"x64" configuration of the 'pythoncore' project before cross-compiling
extensions is possible.
Note that by default, Visual Studio 2008 does not install 64bit compilers or
tools. You may need to reexecute the Visual Studio setup process and select
these tools (using Control Panel->[Add/Remove] Programs is a convenient way to
check or modify your existing install.)
.. _packaging-postinstallation-script:
The Postinstallation script
---------------------------
Starting with Python 2.3, a postinstallation script can be specified with the
:option:`--install-script` option. The basename of the script must be
specified, and the script filename must also be listed in the scripts argument
to the setup function.
This script will be run at installation time on the target system after all the
files have been copied, with ``argv[1]`` set to :option:`-install`, and again at
uninstallation time before the files are removed with ``argv[1]`` set to
:option:`-remove`.
The installation script runs embedded in the windows installer, every output
(``sys.stdout``, ``sys.stderr``) is redirected into a buffer and will be
displayed in the GUI after the script has finished.
Some functions especially useful in this context are available as additional
built-in functions in the installation script.
.. currentmodule:: bdist_wininst-postinst-script
.. function:: directory_created(path)
file_created(path)
These functions should be called when a directory or file is created by the
postinstall script at installation time. It will register *path* with the
uninstaller, so that it will be removed when the distribution is uninstalled.
To be safe, directories are only removed if they are empty.
.. function:: get_special_folder_path(csidl_string)
This function can be used to retrieve special folder locations on Windows like
the Start Menu or the Desktop. It returns the full path to the folder.
*csidl_string* must be one of the following strings::
"CSIDL_APPDATA"
"CSIDL_COMMON_STARTMENU"
"CSIDL_STARTMENU"
"CSIDL_COMMON_DESKTOPDIRECTORY"
"CSIDL_DESKTOPDIRECTORY"
"CSIDL_COMMON_STARTUP"
"CSIDL_STARTUP"
"CSIDL_COMMON_PROGRAMS"
"CSIDL_PROGRAMS"
"CSIDL_FONTS"
If the folder cannot be retrieved, :exc:`OSError` is raised.
Which folders are available depends on the exact Windows version, and probably
also the configuration. For details refer to Microsoft's documentation of the
c:function:`SHGetSpecialFolderPath` function.
.. function:: create_shortcut(target, description, filename[, arguments[, workdir[, iconpath[, iconindex]]]])
This function creates a shortcut. *target* is the path to the program to be
started by the shortcut. *description* is the description of the shortcut.
*filename* is the title of the shortcut that the user will see. *arguments*
specifies the command-line arguments, if any. *workdir* is the working directory
for the program. *iconpath* is the file containing the icon for the shortcut,
and *iconindex* is the index of the icon in the file *iconpath*. Again, for
details consult the Microsoft documentation for the :class:`IShellLink`
interface.
Vista User Access Control (UAC)
===============================
Starting with Python 2.6, bdist_wininst supports a :option:`--user-access-control`
option. The default is 'none' (meaning no UAC handling is done), and other
valid values are 'auto' (meaning prompt for UAC elevation if Python was
installed for all users) and 'force' (meaning always prompt for elevation).

31
Doc/packaging/commandhooks.rst Normal file
View file

@ -0,0 +1,31 @@
=============
Command hooks
=============
Packaging provides a way of extending its commands by the use of pre- and
post- command hooks. The hooks are simple Python functions (or any callable
objects) and are specified in the config file using their full qualified names.
The pre-hooks are run after the command is finalized (its options are
processed), but before it is run. The post-hooks are run after the command
itself. Both types of hooks receive an instance of the command object.
Sample usage of hooks
=====================
Firstly, you need to make sure your hook is present in the path. This is usually
done by dropping them to the same folder where `setup.py` file lives ::
# file: myhooks.py
def my_install_hook(install_cmd):
print "Oh la la! Someone is installing my project!"
Then, you need to point to it in your `setup.cfg` file, under the appropriate
command section ::
[install_dist]
pre-hook.project = myhooks.my_install_hook
The hooks defined in different config files (system-wide, user-wide and
package-wide) do not override each other as long as they are specified with
different aliases (additional names after the dot). The alias in the example
above is ``project``.

349
Doc/packaging/commandref.rst Normal file
View file

@ -0,0 +1,349 @@
.. _packaging-command-reference:
*****************
Command Reference
*****************
This reference briefly documents all standard Packaging commands and some of
their options.
.. FIXME does not work: Use pysetup run --help-commands to list all
standard and extra commands availavble on your system, with their
description. Use pysetup run <command> --help to get help about the options
of one command.
Preparing distributions
=======================
:command:`check`
----------------
Perform some tests on the metadata of a distribution.
For example, it verifies that all required metadata fields are provided in the
:file:`setup.cfg` file.
.. TODO document reST checks
:command:`test`
---------------
Run a test suite.
When doing test-driven development, or running automated builds that need
testing before they are installed for downloading or use, it's often useful to
be able to run a project's unit tests without actually installing the project
anywhere. The :command:`test` command runs project's unit tests without
actually installing it, by temporarily putting the project's source on
:data:`sys.path`, after first running :command:`build_ext -i` to ensure that any
C extensions are built.
You can use this command in one of two ways: either by specifying a
unittest-compatible test suite for your project (or any callable that returns
it) or by passing a test runner function that will run your tests and display
results in the console. Both options take a Python dotted name in the form
``package.module.callable`` to specify the object to use.
If none of these options are specified, Packaging will try to perform test
discovery using either unittest (for Python 3.2 and higher) or unittest2 (for
older versions, if installed).
.. this is a pseudo-command name used to disambiguate the options in indexes and
links
.. program:: packaging test
.. cmdoption:: --suite=NAME, -s NAME
Specify the test suite (or module, class, or method) to be run. The default
for this option can be set by in the project's :file:`setup.cfg` file::
.. code-block:: cfg
[test]
suite = mypackage.tests.get_all_tests
.. cmdoption:: --runner=NAME, -r NAME
Specify the test runner to be called.
:command:`config`
-----------------
Perform distribution configuration.
The build step
==============
This step is mainly useful to compile C/C++ libraries or extension modules. The
build commands can be run manually to check for syntax errors or packaging
issues (for example if the addition of a new source file was forgotten in the
:file:`setup.cfg` file), and is also run automatically by commands which need
it. Packaging checks the mtime of source and built files to avoid re-building
if it's not necessary.
:command:`build`
----------------
Build all files of a distribution, delegating to the other :command:`build_*`
commands to do the work.
:command:`build_clib`
---------------------
Build C libraries.
:command:`build_ext`
--------------------
Build C/C++ extension modules.
:command:`build_py`
-------------------
Build the Python modules (just copy them to the build directory) and
byte-compile them to .pyc files.
:command:`build_scripts`
------------------------
Build the scripts (just copy them to the build directory and adjust their
shebang if they're Python scripts).
:command:`clean`
----------------
Clean the build tree of the release.
.. program:: packaging clean
.. cmdoption:: --all, -a
Remove build directories for modules, scripts, etc., not only temporary build
by-products.
Creating source and built distributions
=======================================
:command:`sdist`
----------------
Build a source distribution for a release.
It is recommended that you always build and upload a source distribution. Users
of OSes with easy access to compilers and users of advanced packaging tools will
prefer to compile from source rather than using pre-built distributions. For
Windows users, providing a binary installer is also recommended practice.
:command:`bdist`
----------------
Build a binary distribution for a release.
This command will call other :command:`bdist_*` commands to create one or more
distributions depending on the options given. The default is to create a
.tar.gz archive on Unix and a zip archive on Windows or OS/2.
.. program:: packaging bdist
.. cmdoption:: --formats
Binary formats to build (comma-separated list).
.. cmdoption:: --show-formats
Dump list of available formats.
:command:`bdist_dumb`
---------------------
Build a "dumb" installer, a simple archive of files that could be unpacked under
``$prefix`` or ``$exec_prefix``.
:command:`bdist_wininst`
------------------------
Build a Windows installer.
:command:`bdist_msi`
--------------------
Build a `Microsoft Installer`_ (.msi) file.
.. _Microsoft Installer: http://msdn.microsoft.com/en-us/library/cc185688(VS.85).aspx
In most cases, the :command:`bdist_msi` installer is a better choice than the
:command:`bdist_wininst` installer, because it provides better support for Win64
platforms, allows administrators to perform non-interactive installations, and
allows installation through group policies.
Publishing distributions
========================
:command:`register`
-------------------
This command registers the current release with the Python Package Index. This
is described in more detail in :PEP:`301`.
.. TODO explain user and project registration with the web UI
:command:`upload`
-----------------
Upload source and/or binary distributions to PyPI.
The distributions have to be built on the same command line as the
:command:`upload` command; see :ref:`packaging-package-upload` for more info.
.. program:: packaging upload
.. cmdoption:: --sign, -s
Sign each uploaded file using GPG (GNU Privacy Guard). The ``gpg`` program
must be available for execution on the system ``PATH``.
.. cmdoption:: --identity=NAME, -i NAME
Specify the identity or key name for GPG to use when signing. The value of
this option will be passed through the ``--local-user`` option of the
``gpg`` program.
.. cmdoption:: --show-response
Display the full response text from server; this is useful for debugging
PyPI problems.
.. cmdoption:: --repository=URL, -r URL
The URL of the repository to upload to. Defaults to
http://pypi.python.org/pypi (i.e., the main PyPI installation).
.. cmdoption:: --upload-docs
Also run :command:`upload_docs`. Mainly useful as a default value in
:file:`setup.cfg` (on the command line, it's shorter to just type both
commands).
:command:`upload_docs`
----------------------
Upload HTML documentation to PyPI.
PyPI now supports publishing project documentation at a URI of the form
``http://packages.python.org/<project>``. :command:`upload_docs` will create
the necessary zip file out of a documentation directory and will post to the
repository.
Note that to upload the documentation of a project, the corresponding version
must already be registered with PyPI, using the :command:`register` command ---
just like with :command:`upload`.
Assuming there is an ``Example`` project with documentation in the subdirectory
:file:`docs`, for example::
Example/
example.py
setup.cfg
docs/
build/
html/
index.html
tips_tricks.html
conf.py
index.txt
tips_tricks.txt
You can simply specify the directory with the HTML files in your
:file:`setup.cfg` file:
.. code-block:: cfg
[upload_docs]
upload-dir = docs/build/html
.. program:: packaging upload_docs
.. cmdoption:: --upload-dir
The directory to be uploaded to the repository. By default documentation
is searched for in ``docs`` (or ``doc``) directory in project root.
.. cmdoption:: --show-response
Display the full response text from server; this is useful for debugging
PyPI problems.
.. cmdoption:: --repository=URL, -r URL
The URL of the repository to upload to. Defaults to
http://pypi.python.org/pypi (i.e., the main PyPI installation).
The install step
================
These commands are used by end-users of a project using :program:`pysetup` or
another compatible installer. Each command will run the corresponding
:command:`build_*` command and then move the built files to their destination on
the target system.
:command:`install_dist`
-----------------------
Install a distribution, delegating to the other :command:`install_*` commands to
do the work.
.. program:: packaging install_dist
.. cmdoption:: --user
Install in user site-packages directory (see :PEP:`370`).
:command:`install_data`
-----------------------
Install data files.
:command:`install_distinfo`
---------------------------
Install files recording details of the installation as specified in :PEP:`376`.
:command:`install_headers`
--------------------------
Install C/C++ header files.
:command:`install_lib`
----------------------
Install C library files.
:command:`install_scripts`
--------------------------
Install scripts.

125
Doc/packaging/configfile.rst Normal file
View file

@ -0,0 +1,125 @@
.. _packaging-setup-config:
************************************
Writing the Setup Configuration File
************************************
Often, it's not possible to write down everything needed to build a distribution
*a priori*: you may need to get some information from the user, or from the
user's system, in order to proceed. As long as that information is fairly
simple---a list of directories to search for C header files or libraries, for
example---then providing a configuration file, :file:`setup.cfg`, for users to
edit is a cheap and easy way to solicit it. Configuration files also let you
provide default values for any command option, which the installer can then
override either on the command line or by editing the config file.
The setup configuration file is a useful middle-ground between the setup script
---which, ideally, would be opaque to installers [#]_---and the command line to
the setup script, which is outside of your control and entirely up to the
installer. In fact, :file:`setup.cfg` (and any other Distutils configuration
files present on the target system) are processed after the contents of the
setup script, but before the command line. This has several useful
consequences:
.. If you have more advanced needs, such as determining which extensions to
build based on what capabilities are present on the target system, then you
need the Distutils auto-configuration facility. This started to appear in
Distutils 0.9 but, as of this writing, isn't mature or stable enough yet
for real-world use.
* installers can override some of what you put in :file:`setup.py` by editing
:file:`setup.cfg`
* you can provide non-standard defaults for options that are not easily set in
:file:`setup.py`
* installers can override anything in :file:`setup.cfg` using the command-line
options to :file:`setup.py`
The basic syntax of the configuration file is simple::
[command]
option = value
...
where *command* is one of the Distutils commands (e.g. :command:`build_py`,
:command:`install_dist`), and *option* is one of the options that command supports.
Any number of options can be supplied for each command, and any number of
command sections can be included in the file. Blank lines are ignored, as are
comments, which run from a ``'#'`` character until the end of the line. Long
option values can be split across multiple lines simply by indenting the
continuation lines.
You can find out the list of options supported by a particular command with the
universal :option:`--help` option, e.g. ::
> python setup.py --help build_ext
[...]
Options for 'build_ext' command:
--build-lib (-b) directory for compiled extension modules
--build-temp (-t) directory for temporary files (build by-products)
--inplace (-i) ignore build-lib and put compiled extensions into the
source directory alongside your pure Python modules
--include-dirs (-I) list of directories to search for header files
--define (-D) C preprocessor macros to define
--undef (-U) C preprocessor macros to undefine
--swig-opts list of SWIG command-line options
[...]
.. XXX do we want to support ``setup.py --help metadata``?
Note that an option spelled :option:`--foo-bar` on the command line is spelled
:option:`foo_bar` in configuration files.
For example, say you want your extensions to be built "in-place"---that is, you
have an extension :mod:`pkg.ext`, and you want the compiled extension file
(:file:`ext.so` on Unix, say) to be put in the same source directory as your
pure Python modules :mod:`pkg.mod1` and :mod:`pkg.mod2`. You can always use the
:option:`--inplace` option on the command line to ensure this::
python setup.py build_ext --inplace
But this requires that you always specify the :command:`build_ext` command
explicitly, and remember to provide :option:`--inplace`. An easier way is to
"set and forget" this option, by encoding it in :file:`setup.cfg`, the
configuration file for this distribution::
[build_ext]
inplace = 1
This will affect all builds of this module distribution, whether or not you
explicitly specify :command:`build_ext`. If you include :file:`setup.cfg` in
your source distribution, it will also affect end-user builds---which is
probably a bad idea for this option, since always building extensions in-place
would break installation of the module distribution. In certain peculiar cases,
though, modules are built right in their installation directory, so this is
conceivably a useful ability. (Distributing extensions that expect to be built
in their installation directory is almost always a bad idea, though.)
Another example: certain commands take options that vary from project to
project but not depending on the installation system, for example,
:command:`test` needs to know where your test suite is located and what test
runner to use; likewise, :command:`upload_docs` can find HTML documentation in
a :file:`doc` or :file:`docs` directory, but needs an option to find files in
:file:`docs/build/html`. Instead of having to type out these options each
time you want to run the command, you can put them in the project's
:file:`setup.cfg`::
[test]
suite = packaging.tests
[upload_docs]
upload-dir = docs/build/html
.. seealso::
:ref:`packaging-config-syntax` in "Installing Python Projects"
More information on the configuration files is available in the manual for
system administrators.
.. rubric:: Footnotes
.. [#] This ideal probably won't be achieved until auto-configuration is fully
supported by the Distutils.

334
Doc/packaging/examples.rst Normal file
View file

@ -0,0 +1,334 @@
.. _packaging-examples:
********
Examples
********
This chapter provides a number of basic examples to help get started with
Packaging.
.. _packaging-pure-mod:
Pure Python distribution (by module)
====================================
If you're just distributing a couple of modules, especially if they don't live
in a particular package, you can specify them individually using the
:option:`py_modules` option in the setup script.
In the simplest case, you'll have two files to worry about: a setup script and
the single module you're distributing, :file:`foo.py` in this example::
<root>/
setup.py
foo.py
(In all diagrams in this section, *<root>* will refer to the distribution root
directory.) A minimal setup script to describe this situation would be::
from packaging.core import setup
setup(name='foo',
version='1.0',
py_modules=['foo'])
Note that the name of the distribution is specified independently with the
:option:`name` option, and there's no rule that says it has to be the same as
the name of the sole module in the distribution (although that's probably a good
convention to follow). However, the distribution name is used to generate
filenames, so you should stick to letters, digits, underscores, and hyphens.
Since :option:`py_modules` is a list, you can of course specify multiple
modules, e.g. if you're distributing modules :mod:`foo` and :mod:`bar`, your
setup might look like this::
<root>/
setup.py
foo.py
bar.py
and the setup script might be ::
from packaging.core import setup
setup(name='foobar',
version='1.0',
py_modules=['foo', 'bar'])
You can put module source files into another directory, but if you have enough
modules to do that, it's probably easier to specify modules by package rather
than listing them individually.
.. _packaging-pure-pkg:
Pure Python distribution (by package)
=====================================
If you have more than a couple of modules to distribute, especially if they are
in multiple packages, it's probably easier to specify whole packages rather than
individual modules. This works even if your modules are not in a package; you
can just tell the Distutils to process modules from the root package, and that
works the same as any other package (except that you don't have to have an
:file:`__init__.py` file).
The setup script from the last example could also be written as ::
from packaging.core import setup
setup(name='foobar',
version='1.0',
packages=[''])
(The empty string stands for the root package.)
If those two files are moved into a subdirectory, but remain in the root
package, e.g.::
<root>/
setup.py
src/
foo.py
bar.py
then you would still specify the root package, but you have to tell the
Distutils where source files in the root package live::
from packaging.core import setup
setup(name='foobar',
version='1.0',
package_dir={'': 'src'},
packages=[''])
More typically, though, you will want to distribute multiple modules in the same
package (or in sub-packages). For example, if the :mod:`foo` and :mod:`bar`
modules belong in package :mod:`foobar`, one way to lay out your source tree is
::
<root>/
setup.py
foobar/
__init__.py
foo.py
bar.py
This is in fact the default layout expected by the Distutils, and the one that
requires the least work to describe in your setup script::
from packaging.core import setup
setup(name='foobar',
version='1.0',
packages=['foobar'])
If you want to put modules in directories not named for their package, then you
need to use the :option:`package_dir` option again. For example, if the
:file:`src` directory holds modules in the :mod:`foobar` package::
<root>/
setup.py
src/
__init__.py
foo.py
bar.py
an appropriate setup script would be ::
from packaging.core import setup
setup(name='foobar',
version='1.0',
package_dir={'foobar': 'src'},
packages=['foobar'])
Or, you might put modules from your main package right in the distribution
root::
<root>/
setup.py
__init__.py
foo.py
bar.py
in which case your setup script would be ::
from packaging.core import setup
setup(name='foobar',
version='1.0',
package_dir={'foobar': ''},
packages=['foobar'])
(The empty string also stands for the current directory.)
If you have sub-packages, they must be explicitly listed in :option:`packages`,
but any entries in :option:`package_dir` automatically extend to sub-packages.
(In other words, the Distutils does *not* scan your source tree, trying to
figure out which directories correspond to Python packages by looking for
:file:`__init__.py` files.) Thus, if the default layout grows a sub-package::
<root>/
setup.py
foobar/
__init__.py
foo.py
bar.py
subfoo/
__init__.py
blah.py
then the corresponding setup script would be ::
from packaging.core import setup
setup(name='foobar',
version='1.0',
packages=['foobar', 'foobar.subfoo'])
(Again, the empty string in :option:`package_dir` stands for the current
directory.)
.. _packaging-single-ext:
Single extension module
=======================
Extension modules are specified using the :option:`ext_modules` option.
:option:`package_dir` has no effect on where extension source files are found;
it only affects the source for pure Python modules. The simplest case, a
single extension module in a single C source file, is::
<root>/
setup.py
foo.c
If the :mod:`foo` extension belongs in the root package, the setup script for
this could be ::
from packaging.core import setup, Extension
setup(name='foobar',
version='1.0',
ext_modules=[Extension('foo', ['foo.c'])])
If the extension actually belongs in a package, say :mod:`foopkg`, then
With exactly the same source tree layout, this extension can be put in the
:mod:`foopkg` package simply by changing the name of the extension::
from packaging.core import setup, Extension
setup(name='foobar',
version='1.0',
packages=['foopkg'],
ext_modules=[Extension('foopkg.foo', ['foo.c'])])
Checking metadata
=================
The ``check`` command allows you to verify if your project's metadata
meets the minimum requirements to build a distribution.
To run it, just call it using your :file:`setup.py` script. If something is
missing, ``check`` will display a warning.
Let's take an example with a simple script::
from packaging.core import setup
setup(name='foobar')
.. TODO configure logging StreamHandler to match this output
Running the ``check`` command will display some warnings::
$ python setup.py check
running check
warning: check: missing required metadata: version, home_page
warning: check: missing metadata: either (author and author_email) or
(maintainer and maintainer_email) must be supplied
If you use the reStructuredText syntax in the ``long_description`` field and
`Docutils <http://docutils.sourceforge.net/>`_ is installed you can check if
the syntax is fine with the ``check`` command, using the ``restructuredtext``
option.
For example, if the :file:`setup.py` script is changed like this::
from packaging.core import setup
desc = """\
Welcome to foobar!
===============
This is the description of the ``foobar`` project.
"""
setup(name='foobar',
version='1.0',
author=u'Tarek Ziadé',
author_email='tarek@ziade.org',
summary='Foobar utilities'
description=desc,
home_page='http://example.com')
Where the long description is broken, ``check`` will be able to detect it
by using the :mod:`docutils` parser::
$ python setup.py check --restructuredtext
running check
warning: check: Title underline too short. (line 2)
warning: check: Could not finish the parsing.
.. _packaging-reading-metadata:
Reading the metadata
====================
The :func:`packaging.core.setup` function provides a command-line interface
that allows you to query the metadata fields of a project through the
:file:`setup.py` script of a given project::
$ python setup.py --name
foobar
This call reads the ``name`` metadata by running the
:func:`packaging.core.setup` function. When a source or binary
distribution is created with Distutils, the metadata fields are written
in a static file called :file:`PKG-INFO`. When a Distutils-based project is
installed in Python, the :file:`PKG-INFO` file is copied alongside the modules
and packages of the distribution under :file:`NAME-VERSION-pyX.X.egg-info`,
where ``NAME`` is the name of the project, ``VERSION`` its version as defined
in the Metadata, and ``pyX.X`` the major and minor version of Python like
``2.7`` or ``3.2``.
You can read back this static file, by using the
:class:`packaging.dist.Metadata` class and its
:func:`read_pkg_file` method::
>>> from packaging.metadata import Metadata
>>> metadata = Metadata()
>>> metadata.read_pkg_file(open('distribute-0.6.8-py2.7.egg-info'))
>>> metadata.name
'distribute'
>>> metadata.version
'0.6.8'
>>> metadata.description
'Easily download, build, install, upgrade, and uninstall Python packages'
Notice that the class can also be instantiated with a metadata file path to
loads its values::
>>> pkg_info_path = 'distribute-0.6.8-py2.7.egg-info'
>>> Metadata(pkg_info_path).name
'distribute'
.. XXX These comments have been here for at least ten years. Write the
sections or delete the comments (we can maybe ask Greg Ward about
the planned contents). (Unindent to make them section titles)
.. multiple-ext::
Multiple extension modules
==========================
Putting it all together
=======================

95
Doc/packaging/extending.rst Normal file
View file

@ -0,0 +1,95 @@
.. _extending-packaging:
*******************
Extending Distutils
*******************
Distutils can be extended in various ways. Most extensions take the form of new
commands or replacements for existing commands. New commands may be written to
support new types of platform-specific packaging, for example, while
replacements for existing commands may be made to modify details of how the
command operates on a package.
Most extensions of the packaging are made within :file:`setup.py` scripts that
want to modify existing commands; many simply add a few file extensions that
should be copied into packages in addition to :file:`.py` files as a
convenience.
Most packaging command implementations are subclasses of the
:class:`packaging.cmd.Command` class. New commands may directly inherit from
:class:`Command`, while replacements often derive from :class:`Command`
indirectly, directly subclassing the command they are replacing. Commands are
required to derive from :class:`Command`.
.. .. _extend-existing:
Extending existing commands
===========================
.. .. _new-commands:
Writing new commands
====================
Integrating new commands
========================
There are different ways to integrate new command implementations into
packaging. The most difficult is to lobby for the inclusion of the new features
in packaging itself, and wait for (and require) a version of Python that
provides that support. This is really hard for many reasons.
The most common, and possibly the most reasonable for most needs, is to include
the new implementations with your :file:`setup.py` script, and cause the
:func:`packaging.core.setup` function use them::
from packaging.core import setup
from packaging.command.build_py import build_py as _build_py
class build_py(_build_py):
"""Specialized Python source builder."""
# implement whatever needs to be different...
setup(..., cmdclass={'build_py': build_py})
This approach is most valuable if the new implementations must be used to use a
particular package, as everyone interested in the package will need to have the
new command implementation.
Beginning with Python 2.4, a third option is available, intended to allow new
commands to be added which can support existing :file:`setup.py` scripts without
requiring modifications to the Python installation. This is expected to allow
third-party extensions to provide support for additional packaging systems, but
the commands can be used for anything packaging commands can be used for. A new
configuration option, :option:`command_packages` (command-line option
:option:`--command-packages`), can be used to specify additional packages to be
searched for modules implementing commands. Like all packaging options, this
can be specified on the command line or in a configuration file. This option
can only be set in the ``[global]`` section of a configuration file, or before
any commands on the command line. If set in a configuration file, it can be
overridden from the command line; setting it to an empty string on the command
line causes the default to be used. This should never be set in a configuration
file provided with a package.
This new option can be used to add any number of packages to the list of
packages searched for command implementations; multiple package names should be
separated by commas. When not specified, the search is only performed in the
:mod:`packaging.command` package. When :file:`setup.py` is run with the option
:option:`--command-packages` :option:`distcmds,buildcmds`, however, the packages
:mod:`packaging.command`, :mod:`distcmds`, and :mod:`buildcmds` will be searched
in that order. New commands are expected to be implemented in modules of the
same name as the command by classes sharing the same name. Given the example
command-line option above, the command :command:`bdist_openpkg` could be
implemented by the class :class:`distcmds.bdist_openpkg.bdist_openpkg` or
:class:`buildcmds.bdist_openpkg.bdist_openpkg`.
Adding new distribution types
=============================
Commands that create distributions (files in the :file:`dist/` directory) need
to add ``(command, filename)`` pairs to ``self.distribution.dist_files`` so that
:command:`upload` can upload it to PyPI. The *filename* in the pair contains no
path information, only the name of the file itself. In dry-run mode, pairs
should still be added to represent what would have been created.

45
Doc/packaging/index.rst Normal file
View file

@ -0,0 +1,45 @@
.. _packaging-index:
##############################
Distributing Python Projects
##############################
:Authors: Greg Ward, Anthony Baxter and Packaging contributors
:Email: distutils-sig@python.org
:Release: |version|
:Date: |today|
This document describes Packaging for Python authors, describing how to use the
module to make Python applications, packages or modules easily available to a
wider audience with very little overhead for build/release/install mechanics.
.. toctree::
:maxdepth: 2
:numbered:
tutorial
setupcfg
introduction
setupscript
configfile
sourcedist
builtdist
packageindex
uploading
examples
extending
commandhooks
commandref
.. seealso::
:ref:`packaging-install-index`
A user-centered manual which includes information on adding projects
into an existing Python installation. You do not need to be a Python
programmer to read this manual.
:mod:`packaging`
A library reference for developers of packaging tools wanting to use
standalone building blocks like :mod:`~packaging.version` or
:mod:`~packaging.metadata`, or extend Packaging itself.

193
Doc/packaging/introduction.rst Normal file
View file

@ -0,0 +1,193 @@
.. _packaging-intro:
*****************************
An Introduction to Packaging
*****************************
This document covers using Packaging to distribute your Python modules,
concentrating on the role of developer/distributor. If you're looking for
information on installing Python modules you should refer to the
:ref:`packaging-install-index` chapter.
Throughout this documentation, the terms "Distutils", "the Distutils" and
"Packaging" will be used interchangeably.
.. _packaging-concepts:
Concepts & Terminology
======================
Using Distutils is quite simple both for module developers and for
users/administrators installing third-party modules. As a developer, your
responsibilities (apart from writing solid, well-documented and well-tested
code, of course!) are:
* writing a setup script (:file:`setup.py` by convention)
* (optional) writing a setup configuration file
* creating a source distribution
* (optional) creating one or more "built" (binary) distributions of your
project
All of these tasks are covered in this document.
Not all module developers have access to multiple platforms, so one cannot
expect them to create buildt distributions for every platform. To remedy
this, it is hoped that intermediaries called *packagers* will arise to address
this need. Packagers take source distributions released by module developers,
build them on one or more platforms and release the resulting built
distributions. Thus, users on a greater range of platforms will be able to
install the most popular Python modules in the most natural way for their
platform without having to run a setup script or compile a single line of code.
.. _packaging-simple-example:
A Simple Example
================
A setup script is usually quite simple, although since it's written in Python
there are no arbitrary limits to what you can do with it, though you should be
careful about putting expensive operations in your setup script.
Unlike, say, Autoconf-style configure scripts the setup script may be run
multiple times in the course of building and installing a module
distribution.
If all you want to do is distribute a module called :mod:`foo`, contained in a
file :file:`foo.py`, then your setup script can be as simple as::
from packaging.core import setup
setup(name='foo',
version='1.0',
py_modules=['foo'])
Some observations:
* most information that you supply to the Distutils is supplied as keyword
arguments to the :func:`setup` function
* those keyword arguments fall into two categories: package metadata (name,
version number, etc.) and information about what's in the package (a list
of pure Python modules in this case)
* modules are specified by module name, not filename (the same will hold true
for packages and extensions)
* it's recommended that you supply a little more metadata than we have in the
example. In particular your name, email address and a URL for the
project if appropriate (see section :ref:`packaging-setup-script` for an example)
To create a source distribution for this module you would create a setup
script, :file:`setup.py`, containing the above code and run::
python setup.py sdist
which will create an archive file (e.g., tarball on Unix, ZIP file on Windows)
containing your setup script :file:`setup.py`, and your module :file:`foo.py`.
The archive file will be named :file:`foo-1.0.tar.gz` (or :file:`.zip`), and
will unpack into a directory :file:`foo-1.0`.
If an end-user wishes to install your :mod:`foo` module all he has to do is
download :file:`foo-1.0.tar.gz` (or :file:`.zip`), unpack it, and from the
:file:`foo-1.0` directory run ::
python setup.py install
which will copy :file:`foo.py` to the appropriate directory for
third-party modules in their Python installation.
This simple example demonstrates some fundamental concepts of Distutils.
First, both developers and installers have the same basic user interface, i.e.
the setup script. The difference is which Distutils *commands* they use: the
:command:`sdist` command is almost exclusively for module developers, while
:command:`install` is more often used by installers (although some developers
will want to install their own code occasionally).
If you want to make things really easy for your users, you can create more
than one built distributions for them. For instance, if you are running on a
Windows machine and want to make things easy for other Windows users, you can
create an executable installer (the most appropriate type of built distribution
for this platform) with the :command:`bdist_wininst` command. For example::
python setup.py bdist_wininst
will create an executable installer, :file:`foo-1.0.win32.exe`, in the current
directory. You can find out what distribution formats are available at any time
by running ::
python setup.py bdist --help-formats
.. _packaging-python-terms:
General Python terminology
==========================
If you're reading this document, you probably have a good idea of what Python
modules, extensions and so forth are. Nevertheless, just to be sure that
everyone is on the same page, here's a quick overview of Python terms:
module
The basic unit of code reusability in Python: a block of code imported by
some other code. Three types of modules are important to us here: pure
Python modules, extension modules and packages.
pure Python module
A module written in Python and contained in a single :file:`.py` file (and
possibly associated :file:`.pyc` and/or :file:`.pyo` files). Sometimes
referred to as a "pure module."
extension module
A module written in the low-level language of the Python implementation: C/C++
for Python, Java for Jython. Typically contained in a single dynamically
loaded pre-compiled file, e.g. a shared object (:file:`.so`) file for Python
extensions on Unix, a DLL (given the :file:`.pyd` extension) for Python
extensions on Windows, or a Java class file for Jython extensions. Note that
currently Distutils only handles C/C++ extensions for Python.
package
A module that contains other modules, typically contained in a directory of
the filesystem and distinguished from other directories by the presence of a
file :file:`__init__.py`.
root package
The root of the hierarchy of packages. (This isn't really a package,
since it doesn't have an :file:`__init__.py` file. But... we have to
call it something, right?) The vast majority of the standard library is
in the root package, as are many small standalone third-party modules that
don't belong to a larger module collection. Unlike regular packages,
modules in the root package can be found in many directories: in fact,
every directory listed in ``sys.path`` contributes modules to the root
package.
.. _packaging-term:
Distutils-specific terminology
==============================
The following terms apply more specifically to the domain of distributing Python
modules using Distutils:
module distribution
A collection of Python modules distributed together as a single downloadable
resource and meant to be installed all as one. Examples of some well-known
module distributions are NumPy, SciPy, PIL (the Python Imaging
Library) or mxBase. (Module distributions would be called a *package*,
except that term is already taken in the Python context: a single module
distribution may contain zero, one, or many Python packages.)
pure module distribution
A module distribution that contains only pure Python modules and packages.
Sometimes referred to as a "pure distribution."
non-pure module distribution
A module distribution that contains at least one extension module. Sometimes
referred to as a "non-pure distribution."
distribution root
The top-level directory of your source tree (or source distribution). The
directory where :file:`setup.py` exists. Generally :file:`setup.py` will
be run from this directory.

104
Doc/packaging/packageindex.rst Normal file
View file

@ -0,0 +1,104 @@
.. _packaging-package-index:
**********************************
Registering with the Package Index
**********************************
The Python Package Index (PyPI) holds metadata describing distributions
packaged with packaging. The packaging command :command:`register` is used to
submit your distribution's metadata to the index. It is invoked as follows::
python setup.py register
Distutils will respond with the following prompt::
running register
We need to know who you are, so please choose either:
1. use your existing login,
2. register as a new user,
3. have the server generate a new password for you (and email it to you), or
4. quit
Your selection [default 1]:
Note: if your username and password are saved locally, you will not see this
menu.
If you have not registered with PyPI, then you will need to do so now. You
should choose option 2, and enter your details as required. Soon after
submitting your details, you will receive an email which will be used to confirm
your registration.
Once you are registered, you may choose option 1 from the menu. You will be
prompted for your PyPI username and password, and :command:`register` will then
submit your metadata to the index.
You may submit any number of versions of your distribution to the index. If you
alter the metadata for a particular version, you may submit it again and the
index will be updated.
PyPI holds a record for each (name, version) combination submitted. The first
user to submit information for a given name is designated the Owner of that
name. They may submit changes through the :command:`register` command or through
the web interface. They may also designate other users as Owners or Maintainers.
Maintainers may edit the package information, but not designate other Owners or
Maintainers.
By default PyPI will list all versions of a given package. To hide certain
versions, the Hidden property should be set to yes. This must be edited through
the web interface.
.. _packaging-pypirc:
The .pypirc file
================
The format of the :file:`.pypirc` file is as follows::
[packaging]
index-servers =
pypi
[pypi]
repository: <repository-url>
username: <username>
password: <password>
The *packaging* section defines a *index-servers* variable that lists the
name of all sections describing a repository.
Each section describing a repository defines three variables:
- *repository*, that defines the url of the PyPI server. Defaults to
``http://www.python.org/pypi``.
- *username*, which is the registered username on the PyPI server.
- *password*, that will be used to authenticate. If omitted the user
will be prompt to type it when needed.
If you want to define another server a new section can be created and
listed in the *index-servers* variable::
[packaging]
index-servers =
pypi
other
[pypi]
repository: <repository-url>
username: <username>
password: <password>
[other]
repository: http://example.com/pypi
username: <username>
password: <password>
:command:`register` can then be called with the -r option to point the
repository to work with::
python setup.py register -r http://example.com/pypi
For convenience, the name of the section that describes the repository
may also be used::
python setup.py register -r other

648
Doc/packaging/setupcfg.rst Normal file
View file

@ -0,0 +1,648 @@
.. highlightlang:: cfg
*******************************************
Specification of the :file:`setup.cfg` file
*******************************************
.. :version: 1.0
This document describes the :file:`setup.cfg`, an ini-style configuration file
(compatible with :class:`configparser.RawConfigParser`) configuration file used
by Packaging to replace the :file:`setup.py` file.
Each section contains a description of its options.
- Options that are marked *multi* can have multiple values, one value per
line.
- Options that are marked *optional* can be omitted.
- Options that are marked *environ* can use environment markers, as described
in :PEP:`345`.
The sections are:
global
Global options not related to one command.
metadata
Name, version and other information defined by :PEP:`345`.
files
Modules, scripts, data, documentation and other files to include in the
distribution.
command sections
Options given for specific commands, identical to those that can be given
on the command line.
Global options
==============
Contains global options for Packaging. This section is shared with Distutils.
commands
Defined Packaging command. A command is defined by its fully
qualified name. *optional*, *multi*
Examples::
[global]
commands =
package.setup.CustomSdistCommand
package.setup.BdistDeb
compilers
Defined Packaging compiler. A compiler is defined by its fully
qualified name. *optional*, *multi*
Example::
[global]
compilers =
hotcompiler.SmartCCompiler
setup_hook
defines a callable that will be called right after the
:file:`setup.cfg` file is read. The callable receives the configuration
in form of a mapping and can make some changes to it. *optional*
Example::
[global]
setup_hook = package.setup.customize_dist
Metadata
========
The metadata section contains the metadata for the project as described in
:PEP:`345`. Field names are case-insensitive.
Fields:
name
Name of the project.
version
Version of the project. Must comply with :PEP:`386`.
platform
Platform specification describing an operating system
supported by the distribution which is not listed in the "Operating System"
Trove classifiers (:PEP:`301`). *optional*, *multi*
supported-platform
Binary distributions containing a PKG-INFO file will
use the Supported-Platform field in their metadata to specify the OS and
CPU for which the binary distribution was compiled. The semantics of
the Supported-Platform field are free form. *optional*, *multi*
summary
A one-line summary of what the distribution does.
(Used to be called *description* in Distutils1.)
description
A longer description. (Used to be called *long_description*
in Distutils1.) A file can be provided in the *description-file* field.
*optional*
description-file
path to a text file that will be used for the
**description** field. *optional*
keywords
A list of additional keywords to be used to assist searching
for the distribution in a larger catalog. Comma or space-separated.
*optional*
home-page
The URL for the distribution's home page.
download-url
The URL from which this version of the distribution
can be downloaded. *optional*
author
Author's name. *optional*
author-email
Author's e-mail. *optional*
maintainer
Maintainer's name. *optional*
maintainer-email
Maintainer's e-mail. *optional*
license
A text indicating the term of uses, when a trove classifier does
not match. *optional*.
classifiers
Classification for the distribution, as described in PEP 301.
*optional*, *multi*, *environ*
requires-dist
name of another packaging project required as a dependency.
The format is *name (version)* where version is an optional
version declaration, as described in PEP 345. *optional*, *multi*, *environ*
provides-dist
name of another packaging project contained within this
distribution. Same format than *requires-dist*. *optional*, *multi*,
*environ*
obsoletes-dist
name of another packaging project this version obsoletes.
Same format than *requires-dist*. *optional*, *multi*, *environ*
requires-python
Specifies the Python version the distribution requires.
The value is a version number, as described in PEP 345.
*optional*, *multi*, *environ*
requires-externals
a dependency in the system. This field is free-form,
and just a hint for downstream maintainers. *optional*, *multi*,
*environ*
project-url
A label, followed by a browsable URL for the project.
"label, url". The label is limited to 32 signs. *optional*, *multi*
Example::
[metadata]
name = pypi2rpm
version = 0.1
author = Tarek Ziadé
author-email = tarek@ziade.org
summary = Script that transforms an sdist archive into a RPM package
description-file = README
home-page = http://bitbucket.org/tarek/pypi2rpm/wiki/Home
project-url:
Repository, http://bitbucket.org/tarek/pypi2rpm/
RSS feed, https://bitbucket.org/tarek/pypi2rpm/rss
classifier =
Development Status :: 3 - Alpha
License :: OSI Approved :: Mozilla Public License 1.1 (MPL 1.1)
You should not give any explicit value for metadata-version: it will be guessed
from the fields present in the file.
Files
=====
This section describes the files included in the project.
packages_root
the root directory containing all packages and modules
(default: current directory). *optional*
packages
a list of packages the project includes *optional*, *multi*
modules
a list of packages the project includes *optional*, *multi*
scripts
a list of scripts the project includes *optional*, *multi*
extra_files
a list of patterns to include extra files *optional*,
*multi*
Example::
[files]
packages_root = src
packages =
pypi2rpm
pypi2rpm.command
scripts =
pypi2rpm/pypi2rpm.py
extra_files =
setup.py
README
.. Note::
The :file:`setup.cfg` configuration file is included by default. Contrary to
Distutils, :file:`README` (or :file:`README.txt`) and :file:`setup.py` are
not included by default.
Resources
---------
This section describes the files used by the project which must not be installed
in the same place that python modules or libraries, they are called
**resources**. They are for example documentation files, script files,
databases, etc...
For declaring resources, you must use this notation::
source = destination
Data-files are declared in the **resources** field in the **file** section, for
example::
[files]
resources =
source1 = destination1
source2 = destination2
The **source** part of the declaration are relative paths of resources files
(using unix path separator **/**). For example, if you've this source tree::
foo/
doc/
doc.man
scripts/
foo.sh
Your setup.cfg will look like::
[files]
resources =
doc/doc.man = destination_doc
scripts/foo.sh = destination_scripts
The final paths where files will be placed are composed by : **source** +
**destination**. In the previous example, **doc/doc.man** will be placed in
**destination_doc/doc/doc.man** and **scripts/foo.sh** will be placed in
**destination_scripts/scripts/foo.sh**. (If you want more control on the final
path, take a look at base_prefix_).
The **destination** part of resources declaration are paths with categories.
Indeed, it's generally a bad idea to give absolute path as it will be cross
incompatible. So, you must use resources categories in your **destination**
declaration. Categories will be replaced by their real path at the installation
time. Using categories is all benefit, your declaration will be simpler, cross
platform and it will allow packager to place resources files where they want
without breaking your code.
Categories can be specified by using this syntax::
{category}
Default categories are:
* config
* appdata
* appdata.arch
* appdata.persistent
* appdata.disposable
* help
* icon
* scripts
* doc
* info
* man
A special category also exists **{distribution.name}** that will be replaced by
the name of the distribution, but as most of the defaults categories use them,
so it's not necessary to add **{distribution.name}** into your destination.
If you use categories in your declarations, and you are encouraged to do, final
path will be::
source + destination_expanded
.. _example_final_path:
For example, if you have this setup.cfg::
[metadata]
name = foo
[files]
resources =
doc/doc.man = {doc}
And if **{doc}** is replaced by **{datadir}/doc/{distribution.name}**, final
path will be::
{datadir}/doc/foo/doc/doc.man
Where {datafir} category will be platform-dependent.
More control on source part
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Glob syntax
"""""""""""
When you declare source file, you can use a glob-like syntax to match multiples file, for example::
scripts/* = {script}
Will match all the files in the scripts directory and placed them in the script category.
Glob tokens are:
* ``*``: match all files.
* ``?``: match any character.
* ``**``: match any level of tree recursion (even 0).
* ``{}``: will match any part separated by comma (example: ``{sh,bat}``).
.. TODO Add examples
Order of declaration
""""""""""""""""""""
The order of declaration is important if one file match multiple rules. The last
rules matched by file is used, this is useful if you have this source tree::
foo/
doc/
index.rst
setup.rst
documentation.txt
doc.tex
README
And you want all the files in the doc directory to be placed in {doc} category,
but README must be placed in {help} category, instead of listing all the files
one by one, you can declare them in this way::
[files]
resources =
doc/* = {doc}
doc/README = {help}
Exclude
"""""""
You can exclude some files of resources declaration by giving no destination, it
can be useful if you have a non-resources file in the same directory of
resources files::
foo/
doc/
RELEASES
doc.tex
documentation.txt
docu.rst
Your **files** section will be::
[files]
resources =
doc/* = {doc}
doc/RELEASES =
More control on destination part
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. _base_prefix:
Defining a base prefix
""""""""""""""""""""""
When you define your resources, you can have more control of how the final path
is compute.
By default, the final path is::
destination + source
This can generate long paths, for example (example_final_path_)::
{datadir}/doc/foo/doc/doc.man
When you declare your source, you can use whitespace to split the source in
**prefix** **suffix**. So, for example, if you have this source::
docs/ doc.man
The **prefix** is "docs/" and the **suffix** is "doc.html".
.. note::
Separator can be placed after a path separator or replace it. So these two
sources are equivalent::
docs/ doc.man
docs doc.man
.. note::
Glob syntax is working the same way with standard source and splitted source.
So these rules::
docs/*
docs/ *
docs *
Will match all the files in the docs directory.
When you use splitted source, the final path is compute in this way::
destination + prefix
So for example, if you have this setup.cfg::
[metadata]
name = foo
[files]
resources =
doc/ doc.man = {doc}
And if **{doc}** is replaced by **{datadir}/doc/{distribution.name}**, final
path will be::
{datadir}/doc/foo/doc.man
Overwriting paths for categories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This part is intended for system administrators or downstream OS packagers.
The real paths of categories are registered in the *sysconfig.cfg* file
installed in your python installation. This file uses an ini format too.
The content of the file is organized into several sections:
* globals: Standard categories's paths.
* posix_prefix: Standard paths for categories and installation paths for posix
system.
* other ones XXX
Standard categories paths are platform independent, they generally refers to
other categories, which are platform dependent. :mod:`sysconfig` will choose
these category from sections matching os.name. For example::
doc = {datadir}/doc/{distribution.name}
It refers to datadir category, which can be different between platforms. In
posix system, it may be::
datadir = /usr/share
So the final path will be::
doc = /usr/share/doc/{distribution.name}
The platform-dependent categories are:
* confdir
* datadir
* libdir
* base
Defining extra categories
^^^^^^^^^^^^^^^^^^^^^^^^^
.. TODO
Examples
^^^^^^^^
These examples are incremental but work unitarily.
Resources in root dir
"""""""""""""""""""""
Source tree::
babar-1.0/
README
babar.sh
launch.sh
babar.py
:file:`setup.cfg`::
[files]
resources =
README = {doc}
*.sh = {scripts}
So babar.sh and launch.sh will be placed in {scripts} directory.
Now let's move all the scripts into a scripts directory.
Resources in sub-directory
""""""""""""""""""""""""""
Source tree::
babar-1.1/
README
scripts/
babar.sh
launch.sh
LAUNCH
babar.py
:file:`setup.cfg`::
[files]
resources =
README = {doc}
scripts/ LAUNCH = {doc}
scripts/ *.sh = {scripts}
It's important to use the separator after scripts/ to install all the shell
scripts into {scripts} instead of {scripts}/scripts.
Now let's add some docs.
Resources in multiple sub-directories
"""""""""""""""""""""""""""""""""""""
Source tree::
babar-1.2/
README
scripts/
babar.sh
launch.sh
LAUNCH
docs/
api
man
babar.py
:file:`setup.cfg`::
[files]
resources =
README = {doc}
scripts/ LAUNCH = {doc}
scripts/ *.sh = {scripts}
doc/ * = {doc}
doc/ man = {man}
You want to place all the file in the docs script into {doc} category, instead
of man, which must be placed into {man} category, we will use the order of
declaration of globs to choose the destination, the last glob that match the
file is used.
Now let's add some scripts for windows users.
Complete example
""""""""""""""""
Source tree::
babar-1.3/
README
doc/
api
man
scripts/
babar.sh
launch.sh
babar.bat
launch.bat
LAUNCH
:file:`setup.cfg`::
[files]
resources =
README = {doc}
scripts/ LAUNCH = {doc}
scripts/ *.{sh,bat} = {scripts}
doc/ * = {doc}
doc/ man = {man}
We use brace expansion syntax to place all the shell and batch scripts into
{scripts} category.
Command sections
================
To pass options to commands without having to type them on the command line
for each invocation, you can write them in the :file:`setup.cfg` file, in a
section named after the command. Example::
[sdist]
# special function to add custom files
manifest-builders = package.setup.list_extra_files
[build]
use-2to3 = True
[build_ext]
inplace = on
[check]
strict = on
all = on
Option values given in the configuration file can be overriden on the command
line. See :ref:`packaging-setup-config` for more information.

689
Doc/packaging/setupscript.rst Normal file
View file

@ -0,0 +1,689 @@
.. _packaging-setup-script:
************************
Writing the Setup Script
************************
The setup script is the center of all activity in building, distributing, and
installing modules using Distutils. The main purpose of the setup script is
to describe your module distribution to Distutils, so that the various
commands that operate on your modules do the right thing. As we saw in section
:ref:`packaging-simple-example`, the setup script consists mainly of a
call to :func:`setup` where the most information is supplied as
keyword arguments to :func:`setup`.
Here's a slightly more involved example, which we'll follow for the next couple
of sections: a setup script that could be used for Packaging itself::
#!/usr/bin/env python
from packaging.core import setup, find_packages
setup(name='Packaging',
version='1.0',
summary='Python Distribution Utilities',
keywords=['packaging', 'packaging'],
author=u'Tarek Ziadé',
author_email='tarek@ziade.org',
home_page='http://bitbucket.org/tarek/packaging/wiki/Home',
license='PSF',
packages=find_packages())
There are only two differences between this and the trivial one-file
distribution presented in section :ref:`packaging-simple-example`: more
metadata and the specification of pure Python modules by package rather than
by module. This is important since Ristutils consist of a couple of dozen
modules split into (so far) two packages; an explicit list of every module
would be tedious to generate and difficult to maintain. For more information
on the additional metadata, see section :ref:`packaging-metadata`.
Note that any pathnames (files or directories) supplied in the setup script
should be written using the Unix convention, i.e. slash-separated. The
Distutils will take care of converting this platform-neutral representation into
whatever is appropriate on your current platform before actually using the
pathname. This makes your setup script portable across operating systems, which
of course is one of the major goals of the Distutils. In this spirit, all
pathnames in this document are slash-separated.
This, of course, only applies to pathnames given to Distutils functions. If
you, for example, use standard Python functions such as :func:`glob.glob` or
:func:`os.listdir` to specify files, you should be careful to write portable
code instead of hardcoding path separators::
glob.glob(os.path.join('mydir', 'subdir', '*.html'))
os.listdir(os.path.join('mydir', 'subdir'))
.. _packaging-listing-packages:
Listing whole packages
======================
The :option:`packages` option tells the Distutils to process (build, distribute,
install, etc.) all pure Python modules found in each package mentioned in the
:option:`packages` list. In order to do this, of course, there has to be a
correspondence between package names and directories in the filesystem. The
default correspondence is the most obvious one, i.e. package :mod:`packaging` is
found in the directory :file:`packaging` relative to the distribution root.
Thus, when you say ``packages = ['foo']`` in your setup script, you are
promising that the Distutils will find a file :file:`foo/__init__.py` (which
might be spelled differently on your system, but you get the idea) relative to
the directory where your setup script lives. If you break this promise, the
Distutils will issue a warning but still process the broken package anyways.
If you use a different convention to lay out your source directory, that's no
problem: you just have to supply the :option:`package_dir` option to tell the
Distutils about your convention. For example, say you keep all Python source
under :file:`lib`, so that modules in the "root package" (i.e., not in any
package at all) are in :file:`lib`, modules in the :mod:`foo` package are in
:file:`lib/foo`, and so forth. Then you would put ::
package_dir = {'': 'lib'}
in your setup script. The keys to this dictionary are package names, and an
empty package name stands for the root package. The values are directory names
relative to your distribution root. In this case, when you say ``packages =
['foo']``, you are promising that the file :file:`lib/foo/__init__.py` exists.
Another possible convention is to put the :mod:`foo` package right in
:file:`lib`, the :mod:`foo.bar` package in :file:`lib/bar`, etc. This would be
written in the setup script as ::
package_dir = {'foo': 'lib'}
A ``package: dir`` entry in the :option:`package_dir` dictionary implicitly
applies to all packages below *package*, so the :mod:`foo.bar` case is
automatically handled here. In this example, having ``packages = ['foo',
'foo.bar']`` tells the Distutils to look for :file:`lib/__init__.py` and
:file:`lib/bar/__init__.py`. (Keep in mind that although :option:`package_dir`
applies recursively, you must explicitly list all packages in
:option:`packages`: the Distutils will *not* recursively scan your source tree
looking for any directory with an :file:`__init__.py` file.)
.. _packaging-listing-modules:
Listing individual modules
==========================
For a small module distribution, you might prefer to list all modules rather
than listing packages---especially the case of a single module that goes in the
"root package" (i.e., no package at all). This simplest case was shown in
section :ref:`packaging-simple-example`; here is a slightly more involved
example::
py_modules = ['mod1', 'pkg.mod2']
This describes two modules, one of them in the "root" package, the other in the
:mod:`pkg` package. Again, the default package/directory layout implies that
these two modules can be found in :file:`mod1.py` and :file:`pkg/mod2.py`, and
that :file:`pkg/__init__.py` exists as well. And again, you can override the
package/directory correspondence using the :option:`package_dir` option.
.. _packaging-describing-extensions:
Describing extension modules
============================
Just as writing Python extension modules is a bit more complicated than writing
pure Python modules, describing them to the Distutils is a bit more complicated.
Unlike pure modules, it's not enough just to list modules or packages and expect
the Distutils to go out and find the right files; you have to specify the
extension name, source file(s), and any compile/link requirements (include
directories, libraries to link with, etc.).
.. XXX read over this section
All of this is done through another keyword argument to :func:`setup`, the
:option:`ext_modules` option. :option:`ext_modules` is just a list of
:class:`Extension` instances, each of which describes a single extension module.
Suppose your distribution includes a single extension, called :mod:`foo` and
implemented by :file:`foo.c`. If no additional instructions to the
compiler/linker are needed, describing this extension is quite simple::
Extension('foo', ['foo.c'])
The :class:`Extension` class can be imported from :mod:`packaging.core` along
with :func:`setup`. Thus, the setup script for a module distribution that
contains only this one extension and nothing else might be::
from packaging.core import setup, Extension
setup(name='foo',
version='1.0',
ext_modules=[Extension('foo', ['foo.c'])])
The :class:`Extension` class (actually, the underlying extension-building
machinery implemented by the :command:`build_ext` command) supports a great deal
of flexibility in describing Python extensions, which is explained in the
following sections.
Extension names and packages
----------------------------
The first argument to the :class:`Extension` constructor is always the name of
the extension, including any package names. For example, ::
Extension('foo', ['src/foo1.c', 'src/foo2.c'])
describes an extension that lives in the root package, while ::
Extension('pkg.foo', ['src/foo1.c', 'src/foo2.c'])
describes the same extension in the :mod:`pkg` package. The source files and
resulting object code are identical in both cases; the only difference is where
in the filesystem (and therefore where in Python's namespace hierarchy) the
resulting extension lives.
If you have a number of extensions all in the same package (or all under the
same base package), use the :option:`ext_package` keyword argument to
:func:`setup`. For example, ::
setup(...,
ext_package='pkg',
ext_modules=[Extension('foo', ['foo.c']),
Extension('subpkg.bar', ['bar.c'])])
will compile :file:`foo.c` to the extension :mod:`pkg.foo`, and :file:`bar.c` to
:mod:`pkg.subpkg.bar`.
Extension source files
----------------------
The second argument to the :class:`Extension` constructor is a list of source
files. Since the Distutils currently only support C, C++, and Objective-C
extensions, these are normally C/C++/Objective-C source files. (Be sure to use
appropriate extensions to distinguish C++\ source files: :file:`.cc` and
:file:`.cpp` seem to be recognized by both Unix and Windows compilers.)
However, you can also include SWIG interface (:file:`.i`) files in the list; the
:command:`build_ext` command knows how to deal with SWIG extensions: it will run
SWIG on the interface file and compile the resulting C/C++ file into your
extension.
.. XXX SWIG support is rough around the edges and largely untested!
This warning notwithstanding, options to SWIG can be currently passed like
this::
setup(...,
ext_modules=[Extension('_foo', ['foo.i'],
swig_opts=['-modern', '-I../include'])],
py_modules=['foo'])
Or on the command line like this::
> python setup.py build_ext --swig-opts="-modern -I../include"
On some platforms, you can include non-source files that are processed by the
compiler and included in your extension. Currently, this just means Windows
message text (:file:`.mc`) files and resource definition (:file:`.rc`) files for
Visual C++. These will be compiled to binary resource (:file:`.res`) files and
linked into the executable.
Preprocessor options
--------------------
Three optional arguments to :class:`Extension` will help if you need to specify
include directories to search or preprocessor macros to define/undefine:
``include_dirs``, ``define_macros``, and ``undef_macros``.
For example, if your extension requires header files in the :file:`include`
directory under your distribution root, use the ``include_dirs`` option::
Extension('foo', ['foo.c'], include_dirs=['include'])
You can specify absolute directories there; if you know that your extension will
only be built on Unix systems with X11R6 installed to :file:`/usr`, you can get
away with ::
Extension('foo', ['foo.c'], include_dirs=['/usr/include/X11'])
You should avoid this sort of non-portable usage if you plan to distribute your
code: it's probably better to write C code like ::
#include <X11/Xlib.h>
If you need to include header files from some other Python extension, you can
take advantage of the fact that header files are installed in a consistent way
by the Distutils :command:`install_header` command. For example, the Numerical
Python header files are installed (on a standard Unix installation) to
:file:`/usr/local/include/python1.5/Numerical`. (The exact location will differ
according to your platform and Python installation.) Since the Python include
directory---\ :file:`/usr/local/include/python1.5` in this case---is always
included in the search path when building Python extensions, the best approach
is to write C code like ::
#include <Numerical/arrayobject.h>
.. TODO check if it's d2.sysconfig or the new sysconfig module now
If you must put the :file:`Numerical` include directory right into your header
search path, though, you can find that directory using the Distutils
:mod:`packaging.sysconfig` module::
from packaging.sysconfig import get_python_inc
incdir = os.path.join(get_python_inc(plat_specific=1), 'Numerical')
setup(...,
Extension(..., include_dirs=[incdir]))
Even though this is quite portable---it will work on any Python installation,
regardless of platform---it's probably easier to just write your C code in the
sensible way.
You can define and undefine preprocessor macros with the ``define_macros`` and
``undef_macros`` options. ``define_macros`` takes a list of ``(name, value)``
tuples, where ``name`` is the name of the macro to define (a string) and
``value`` is its value: either a string or ``None``. (Defining a macro ``FOO``
to ``None`` is the equivalent of a bare ``#define FOO`` in your C source: with
most compilers, this sets ``FOO`` to the string ``1``.) ``undef_macros`` is
just a list of macros to undefine.
For example::
Extension(...,
define_macros=[('NDEBUG', '1'),
('HAVE_STRFTIME', None)],
undef_macros=['HAVE_FOO', 'HAVE_BAR'])
is the equivalent of having this at the top of every C source file::
#define NDEBUG 1
#define HAVE_STRFTIME
#undef HAVE_FOO
#undef HAVE_BAR
Library options
---------------
You can also specify the libraries to link against when building your extension,
and the directories to search for those libraries. The ``libraries`` option is
a list of libraries to link against, ``library_dirs`` is a list of directories
to search for libraries at link-time, and ``runtime_library_dirs`` is a list of
directories to search for shared (dynamically loaded) libraries at run-time.
For example, if you need to link against libraries known to be in the standard
library search path on target systems ::
Extension(...,
libraries=['gdbm', 'readline'])
If you need to link with libraries in a non-standard location, you'll have to
include the location in ``library_dirs``::
Extension(...,
library_dirs=['/usr/X11R6/lib'],
libraries=['X11', 'Xt'])
(Again, this sort of non-portable construct should be avoided if you intend to
distribute your code.)
.. XXX Should mention clib libraries here or somewhere else!
Other options
-------------
There are still some other options which can be used to handle special cases.
The :option:`optional` option is a boolean; if it is true,
a build failure in the extension will not abort the build process, but
instead simply not install the failing extension.
The :option:`extra_objects` option is a list of object files to be passed to the
linker. These files must not have extensions, as the default extension for the
compiler is used.
:option:`extra_compile_args` and :option:`extra_link_args` can be used to
specify additional command-line options for the respective compiler and linker
command lines.
:option:`export_symbols` is only useful on Windows. It can contain a list of
symbols (functions or variables) to be exported. This option is not needed when
building compiled extensions: Distutils will automatically add ``initmodule``
to the list of exported symbols.
The :option:`depends` option is a list of files that the extension depends on
(for example header files). The build command will call the compiler on the
sources to rebuild extension if any on this files has been modified since the
previous build.
Relationships between Distributions and Packages
================================================
.. FIXME rewrite to update to PEP 345 (but without dist/release confusion)
A distribution may relate to packages in three specific ways:
#. It can require packages or modules.
#. It can provide packages or modules.
#. It can obsolete packages or modules.
These relationships can be specified using keyword arguments to the
:func:`packaging.core.setup` function.
Dependencies on other Python modules and packages can be specified by supplying
the *requires* keyword argument to :func:`setup`. The value must be a list of
strings. Each string specifies a package that is required, and optionally what
versions are sufficient.
To specify that any version of a module or package is required, the string
should consist entirely of the module or package name. Examples include
``'mymodule'`` and ``'xml.parsers.expat'``.
If specific versions are required, a sequence of qualifiers can be supplied in
parentheses. Each qualifier may consist of a comparison operator and a version
number. The accepted comparison operators are::
< > ==
<= >= !=
These can be combined by using multiple qualifiers separated by commas (and
optional whitespace). In this case, all of the qualifiers must be matched; a
logical AND is used to combine the evaluations.
Let's look at a bunch of examples:
+-------------------------+----------------------------------------------+
| Requires Expression | Explanation |
+=========================+==============================================+
| ``==1.0`` | Only version ``1.0`` is compatible |
+-------------------------+----------------------------------------------+
| ``>1.0, !=1.5.1, <2.0`` | Any version after ``1.0`` and before ``2.0`` |
| | is compatible, except ``1.5.1`` |
+-------------------------+----------------------------------------------+
Now that we can specify dependencies, we also need to be able to specify what we
provide that other distributions can require. This is done using the *provides*
keyword argument to :func:`setup`. The value for this keyword is a list of
strings, each of which names a Python module or package, and optionally
identifies the version. If the version is not specified, it is assumed to match
that of the distribution.
Some examples:
+---------------------+----------------------------------------------+
| Provides Expression | Explanation |
+=====================+==============================================+
| ``mypkg`` | Provide ``mypkg``, using the distribution |
| | version |
+---------------------+----------------------------------------------+
| ``mypkg (1.1)`` | Provide ``mypkg`` version 1.1, regardless of |
| | the distribution version |
+---------------------+----------------------------------------------+
A package can declare that it obsoletes other packages using the *obsoletes*
keyword argument. The value for this is similar to that of the *requires*
keyword: a list of strings giving module or package specifiers. Each specifier
consists of a module or package name optionally followed by one or more version
qualifiers. Version qualifiers are given in parentheses after the module or
package name.
The versions identified by the qualifiers are those that are obsoleted by the
distribution being described. If no qualifiers are given, all versions of the
named module or package are understood to be obsoleted.
.. _packaging-installing-scripts:
Installing Scripts
==================
So far we have been dealing with pure and non-pure Python modules, which are
usually not run by themselves but imported by scripts.
Scripts are files containing Python source code, intended to be started from the
command line. Scripts don't require Distutils to do anything very complicated.
The only clever feature is that if the first line of the script starts with
``#!`` and contains the word "python", the Distutils will adjust the first line
to refer to the current interpreter location. By default, it is replaced with
the current interpreter location. The :option:`--executable` (or :option:`-e`)
option will allow the interpreter path to be explicitly overridden.
The :option:`scripts` option simply is a list of files to be handled in this
way. From the PyXML setup script::
setup(...,
scripts=['scripts/xmlproc_parse', 'scripts/xmlproc_val'])
All the scripts will also be added to the ``MANIFEST`` file if no template is
provided. See :ref:`packaging-manifest`.
.. _packaging-installing-package-data:
Installing Package Data
=======================
Often, additional files need to be installed into a package. These files are
often data that's closely related to the package's implementation, or text files
containing documentation that might be of interest to programmers using the
package. These files are called :dfn:`package data`.
Package data can be added to packages using the ``package_data`` keyword
argument to the :func:`setup` function. The value must be a mapping from
package name to a list of relative path names that should be copied into the
package. The paths are interpreted as relative to the directory containing the
package (information from the ``package_dir`` mapping is used if appropriate);
that is, the files are expected to be part of the package in the source
directories. They may contain glob patterns as well.
The path names may contain directory portions; any necessary directories will be
created in the installation.
For example, if a package should contain a subdirectory with several data files,
the files can be arranged like this in the source tree::
setup.py
src/
mypkg/
__init__.py
module.py
data/
tables.dat
spoons.dat
forks.dat
The corresponding call to :func:`setup` might be::
setup(...,
packages=['mypkg'],
package_dir={'mypkg': 'src/mypkg'},
package_data={'mypkg': ['data/*.dat']})
All the files that match ``package_data`` will be added to the ``MANIFEST``
file if no template is provided. See :ref:`packaging-manifest`.
.. _packaging-additional-files:
Installing Additional Files
===========================
The :option:`data_files` option can be used to specify additional files needed
by the module distribution: configuration files, message catalogs, data files,
anything which doesn't fit in the previous categories.
:option:`data_files` specifies a sequence of (*directory*, *files*) pairs in the
following way::
setup(...,
data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
('config', ['cfg/data.cfg']),
('/etc/init.d', ['init-script'])])
Note that you can specify the directory names where the data files will be
installed, but you cannot rename the data files themselves.
Each (*directory*, *files*) pair in the sequence specifies the installation
directory and the files to install there. If *directory* is a relative path, it
is interpreted relative to the installation prefix (Python's ``sys.prefix`` for
pure-Python packages, ``sys.exec_prefix`` for packages that contain extension
modules). Each file name in *files* is interpreted relative to the
:file:`setup.py` script at the top of the package source distribution. No
directory information from *files* is used to determine the final location of
the installed file; only the name of the file is used.
You can specify the :option:`data_files` options as a simple sequence of files
without specifying a target directory, but this is not recommended, and the
:command:`install_dist` command will print a warning in this case. To install data
files directly in the target directory, an empty string should be given as the
directory.
All the files that match ``data_files`` will be added to the ``MANIFEST`` file
if no template is provided. See :ref:`packaging-manifest`.
.. _packaging-metadata:
Metadata reference
==================
The setup script may include additional metadata beyond the name and version.
This table describes required and additional information:
.. TODO synchronize with setupcfg; link to it (but don't remove it, it's a
useful summary)
+----------------------+---------------------------+-----------------+--------+
| Meta-Data | Description | Value | Notes |
+======================+===========================+=================+========+
| ``name`` | name of the project | short string | \(1) |
+----------------------+---------------------------+-----------------+--------+
| ``version`` | version of this release | short string | (1)(2) |
+----------------------+---------------------------+-----------------+--------+
| ``author`` | project author's name | short string | \(3) |
+----------------------+---------------------------+-----------------+--------+
| ``author_email`` | email address of the | email address | \(3) |
| | project author | | |
+----------------------+---------------------------+-----------------+--------+
| ``maintainer`` | project maintainer's name | short string | \(3) |
+----------------------+---------------------------+-----------------+--------+
| ``maintainer_email`` | email address of the | email address | \(3) |
| | project maintainer | | |
+----------------------+---------------------------+-----------------+--------+
| ``home_page`` | home page for the project | URL | \(1) |
+----------------------+---------------------------+-----------------+--------+
| ``summary`` | short description of the | short string | |
| | project | | |
+----------------------+---------------------------+-----------------+--------+
| ``description`` | longer description of the | long string | \(5) |
| | project | | |
+----------------------+---------------------------+-----------------+--------+
| ``download_url`` | location where the | URL | |
| | project may be downloaded | | |
+----------------------+---------------------------+-----------------+--------+
| ``classifiers`` | a list of classifiers | list of strings | \(4) |
+----------------------+---------------------------+-----------------+--------+
| ``platforms`` | a list of platforms | list of strings | |
+----------------------+---------------------------+-----------------+--------+
| ``license`` | license for the release | short string | \(6) |
+----------------------+---------------------------+-----------------+--------+
Notes:
(1)
These fields are required.
(2)
It is recommended that versions take the form *major.minor[.patch[.sub]]*.
(3)
Either the author or the maintainer must be identified.
(4)
The list of classifiers is available from the `PyPI website
<http://pypi.python.org/pypi>`_. See also :mod:`packaging.create`.
(5)
The ``description`` field is used by PyPI when you are registering a
release, to build its PyPI page.
(6)
The ``license`` field is a text indicating the license covering the
distribution where the license is not a selection from the "License" Trove
classifiers. See the ``Classifier`` field. Notice that
there's a ``licence`` distribution option which is deprecated but still
acts as an alias for ``license``.
'short string'
A single line of text, not more than 200 characters.
'long string'
Multiple lines of plain text in reStructuredText format (see
http://docutils.sf.net/).
'list of strings'
See below.
In Python 2.x, "string value" means a unicode object. If a byte string (str or
bytes) is given, it has to be valid ASCII.
.. TODO move this section to the version document, keep a summary, add a link
Encoding the version information is an art in itself. Python projects generally
adhere to the version format *major.minor[.patch][sub]*. The major number is 0
for initial, experimental releases of software. It is incremented for releases
that represent major milestones in a project. The minor number is incremented
when important new features are added to the project. The patch number
increments when bug-fix releases are made. Additional trailing version
information is sometimes used to indicate sub-releases. These are
"a1,a2,...,aN" (for alpha releases, where functionality and API may change),
"b1,b2,...,bN" (for beta releases, which only fix bugs) and "pr1,pr2,...,prN"
(for final pre-release release testing). Some examples:
0.1.0
the first, experimental release of a project
1.0.1a2
the second alpha release of the first patch version of 1.0
:option:`classifiers` are specified in a Python list::
setup(...,
classifiers=[
'Development Status :: 4 - Beta',
'Environment :: Console',
'Environment :: Web Environment',
'Intended Audience :: End Users/Desktop',
'Intended Audience :: Developers',
'Intended Audience :: System Administrators',
'License :: OSI Approved :: Python Software Foundation License',
'Operating System :: MacOS :: MacOS X',
'Operating System :: Microsoft :: Windows',
'Operating System :: POSIX',
'Programming Language :: Python',
'Topic :: Communications :: Email',
'Topic :: Office/Business',
'Topic :: Software Development :: Bug Tracking',
])
Debugging the setup script
==========================
Sometimes things go wrong, and the setup script doesn't do what the developer
wants.
Distutils catches any exceptions when running the setup script, and print a
simple error message before the script is terminated. The motivation for this
behaviour is to not confuse administrators who don't know much about Python and
are trying to install a project. If they get a big long traceback from deep
inside the guts of Distutils, they may think the project or the Python
installation is broken because they don't read all the way down to the bottom
and see that it's a permission problem.
.. FIXME DISTUTILS_DEBUG is dead, document logging/warnings here
On the other hand, this doesn't help the developer to find the cause of the
failure. For this purpose, the DISTUTILS_DEBUG environment variable can be set
to anything except an empty string, and Packaging will now print detailed
information about what it is doing, and prints the full traceback in case an
exception occurs.

273
Doc/packaging/sourcedist.rst Normal file
View file

@ -0,0 +1,273 @@
.. _packaging-source-dist:
******************************
Creating a Source Distribution
******************************
As shown in section :ref:`packaging-simple-example`, you use the :command:`sdist` command
to create a source distribution. In the simplest case, ::
python setup.py sdist
(assuming you haven't specified any :command:`sdist` options in the setup script
or config file), :command:`sdist` creates the archive of the default format for
the current platform. The default format is a gzip'ed tar file
(:file:`.tar.gz`) on Unix, and ZIP file on Windows.
You can specify as many formats as you like using the :option:`--formats`
option, for example::
python setup.py sdist --formats=gztar,zip
to create a gzipped tarball and a zip file. The available formats are:
+-----------+-------------------------+---------+
| Format | Description | Notes |
+===========+=========================+=========+
| ``zip`` | zip file (:file:`.zip`) | (1),(3) |
+-----------+-------------------------+---------+
| ``gztar`` | gzip'ed tar file | \(2) |
| | (:file:`.tar.gz`) | |
+-----------+-------------------------+---------+
| ``bztar`` | bzip2'ed tar file | |
| | (:file:`.tar.bz2`) | |
+-----------+-------------------------+---------+
| ``ztar`` | compressed tar file | \(4) |
| | (:file:`.tar.Z`) | |
+-----------+-------------------------+---------+
| ``tar`` | tar file (:file:`.tar`) | |
+-----------+-------------------------+---------+
Notes:
(1)
default on Windows
(2)
default on Unix
(3)
requires either external :program:`zip` utility or :mod:`zipfile` module (part
of the standard Python library since Python 1.6)
(4)
requires the :program:`compress` program. Notice that this format is now
pending for deprecation and will be removed in the future versions of Python.
When using any ``tar`` format (``gztar``, ``bztar``, ``ztar`` or
``tar``) under Unix, you can specify the ``owner`` and ``group`` names
that will be set for each member of the archive.
For example, if you want all files of the archive to be owned by root::
python setup.py sdist --owner=root --group=root
.. _packaging-manifest:
Specifying the files to distribute
==================================
If you don't supply an explicit list of files (or instructions on how to
generate one), the :command:`sdist` command puts a minimal default set into the
source distribution:
* all Python source files implied by the :option:`py_modules` and
:option:`packages` options
* all C source files mentioned in the :option:`ext_modules` or
:option:`libraries` options
* scripts identified by the :option:`scripts` option
See :ref:`packaging-installing-scripts`.
* anything that looks like a test script: :file:`test/test\*.py` (currently, the
Packaging don't do anything with test scripts except include them in source
distributions, but in the future there will be a standard for testing Python
module distributions)
* the configuration file :file:`setup.cfg`
* all files that matches the ``package_data`` metadata.
See :ref:`packaging-installing-package-data`.
* all files that matches the ``data_files`` metadata.
See :ref:`packaging-additional-files`.
Contrary to Distutils, :file:`README` (or :file:`README.txt`) and
:file:`setup.py` are not included by default.
Sometimes this is enough, but usually you will want to specify additional files
to distribute. The typical way to do this is to write a *manifest template*,
called :file:`MANIFEST.in` by default. The manifest template is just a list of
instructions for how to generate your manifest file, :file:`MANIFEST`, which is
the exact list of files to include in your source distribution. The
:command:`sdist` command processes this template and generates a manifest based
on its instructions and what it finds in the filesystem.
If you prefer to roll your own manifest file, the format is simple: one filename
per line, regular files (or symlinks to them) only. If you do supply your own
:file:`MANIFEST`, you must specify everything: the default set of files
described above does not apply in this case.
:file:`MANIFEST` files start with a comment indicating they are generated.
Files without this comment are not overwritten or removed.
See :ref:`packaging-manifest-template` section for a syntax reference.
.. _packaging-manifest-options:
Manifest-related options
========================
The normal course of operations for the :command:`sdist` command is as follows:
* if the manifest file, :file:`MANIFEST` doesn't exist, read :file:`MANIFEST.in`
and create the manifest
* if neither :file:`MANIFEST` nor :file:`MANIFEST.in` exist, create a manifest
with just the default file set
* if either :file:`MANIFEST.in` or the setup script (:file:`setup.py`) are more
recent than :file:`MANIFEST`, recreate :file:`MANIFEST` by reading
:file:`MANIFEST.in`
* use the list of files now in :file:`MANIFEST` (either just generated or read
in) to create the source distribution archive(s)
There are a couple of options that modify this behaviour. First, use the
:option:`--no-defaults` and :option:`--no-prune` to disable the standard
"include" and "exclude" sets.
Second, you might just want to (re)generate the manifest, but not create a
source distribution::
python setup.py sdist --manifest-only
:option:`-o` is a shortcut for :option:`--manifest-only`.
.. _packaging-manifest-template:
The MANIFEST.in template
========================
A :file:`MANIFEST.in` file can be added in a project to define the list of
files to include in the distribution built by the :command:`sdist` command.
When :command:`sdist` is run, it will look for the :file:`MANIFEST.in` file
and interpret it to generate the :file:`MANIFEST` file that contains the
list of files that will be included in the package.
This mechanism can be used when the default list of files is not enough.
(See :ref:`packaging-manifest`).
Principle
---------
The manifest template has one command per line, where each command specifies a
set of files to include or exclude from the source distribution. For an
example, let's look at the Packaging' own manifest template::
include *.txt
recursive-include examples *.txt *.py
prune examples/sample?/build
The meanings should be fairly clear: include all files in the distribution root
matching :file:`\*.txt`, all files anywhere under the :file:`examples` directory
matching :file:`\*.txt` or :file:`\*.py`, and exclude all directories matching
:file:`examples/sample?/build`. All of this is done *after* the standard
include set, so you can exclude files from the standard set with explicit
instructions in the manifest template. (Or, you can use the
:option:`--no-defaults` option to disable the standard set entirely.)
The order of commands in the manifest template matters: initially, we have the
list of default files as described above, and each command in the template adds
to or removes from that list of files. Once we have fully processed the
manifest template, we remove files that should not be included in the source
distribution:
* all files in the Packaging "build" tree (default :file:`build/`)
* all files in directories named :file:`RCS`, :file:`CVS`, :file:`.svn`,
:file:`.hg`, :file:`.git`, :file:`.bzr` or :file:`_darcs`
Now we have our complete list of files, which is written to the manifest for
future reference, and then used to build the source distribution archive(s).
You can disable the default set of included files with the
:option:`--no-defaults` option, and you can disable the standard exclude set
with :option:`--no-prune`.
Following the Packaging' own manifest template, let's trace how the
:command:`sdist` command builds the list of files to include in the Packaging
source distribution:
#. include all Python source files in the :file:`packaging` and
:file:`packaging/command` subdirectories (because packages corresponding to
those two directories were mentioned in the :option:`packages` option in the
setup script---see section :ref:`packaging-setup-script`)
#. include :file:`README.txt`, :file:`setup.py`, and :file:`setup.cfg` (standard
files)
#. include :file:`test/test\*.py` (standard files)
#. include :file:`\*.txt` in the distribution root (this will find
:file:`README.txt` a second time, but such redundancies are weeded out later)
#. include anything matching :file:`\*.txt` or :file:`\*.py` in the sub-tree
under :file:`examples`,
#. exclude all files in the sub-trees starting at directories matching
:file:`examples/sample?/build`\ ---this may exclude files included by the
previous two steps, so it's important that the ``prune`` command in the manifest
template comes after the ``recursive-include`` command
#. exclude the entire :file:`build` tree, and any :file:`RCS`, :file:`CVS`,
:file:`.svn`, :file:`.hg`, :file:`.git`, :file:`.bzr` and :file:`_darcs`
directories
Just like in the setup script, file and directory names in the manifest template
should always be slash-separated; the Packaging will take care of converting
them to the standard representation on your platform. That way, the manifest
template is portable across operating systems.
Commands
--------
The manifest template commands are:
+-------------------------------------------+-----------------------------------------------+
| Command | Description |
+===========================================+===============================================+
| :command:`include pat1 pat2 ...` | include all files matching any of the listed |
| | patterns |
+-------------------------------------------+-----------------------------------------------+
| :command:`exclude pat1 pat2 ...` | exclude all files matching any of the listed |
| | patterns |
+-------------------------------------------+-----------------------------------------------+
| :command:`recursive-include dir pat1 pat2 | include all files under *dir* matching any of |
| ...` | the listed patterns |
+-------------------------------------------+-----------------------------------------------+
| :command:`recursive-exclude dir pat1 pat2 | exclude all files under *dir* matching any of |
| ...` | the listed patterns |
+-------------------------------------------+-----------------------------------------------+
| :command:`global-include pat1 pat2 ...` | include all files anywhere in the source tree |
| | matching --- & any of the listed patterns |
+-------------------------------------------+-----------------------------------------------+
| :command:`global-exclude pat1 pat2 ...` | exclude all files anywhere in the source tree |
| | matching --- & any of the listed patterns |
+-------------------------------------------+-----------------------------------------------+
| :command:`prune dir` | exclude all files under *dir* |
+-------------------------------------------+-----------------------------------------------+
| :command:`graft dir` | include all files under *dir* |
+-------------------------------------------+-----------------------------------------------+
The patterns here are Unix-style "glob" patterns: ``*`` matches any sequence of
regular filename characters, ``?`` matches any single regular filename
character, and ``[range]`` matches any of the characters in *range* (e.g.,
``a-z``, ``a-zA-Z``, ``a-f0-9_.``). The definition of "regular filename
character" is platform-specific: on Unix it is anything except slash; on Windows
anything except backslash or colon.

112
Doc/packaging/tutorial.rst Normal file
View file

@ -0,0 +1,112 @@
==================
Packaging tutorial
==================
Welcome to the Packaging tutorial! We will learn how to use Packaging
to package your project.
.. TODO merge with introduction.rst
Getting started
---------------
Packaging works with the *setup.cfg* file. It contains all the metadata for
your project, as defined in PEP 345, but also declare what your project
contains.
Let's say you have a project called *CLVault* containing one package called
*clvault*, and a few scripts inside. You can use the *pysetup* script to create
a *setup.cfg* file for the project. The script will ask you a few questions::
$ mkdir CLVault
$ cd CLVault
$ pysetup create
Project name [CLVault]:
Current version number: 0.1
Package description:
>Command-line utility to store and retrieve passwords
Author name: Tarek Ziade
Author e-mail address: tarek@ziade.org
Project Home Page: http://bitbucket.org/tarek/clvault
Do you want to add a package ? (y/n): y
Package name: clvault
Do you want to add a package ? (y/n): n
Do you want to set Trove classifiers? (y/n): y
Please select the project status:
1 - Planning
2 - Pre-Alpha
3 - Alpha
4 - Beta
5 - Production/Stable
6 - Mature
7 - Inactive
Status: 3
What license do you use: GPL
Matching licenses:
1) License :: OSI Approved :: GNU General Public License (GPL)
2) License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)
Type the number of the license you wish to use or ? to try again:: 1
Do you want to set other trove identifiers (y/n) [n]: n
Wrote "setup.cfg".
A setup.cfg file is created, containing the metadata of your project and the
list of the packages it contains::
$ cat setup.cfg
[metadata]
name = CLVault
version = 0.1
author = Tarek Ziade
author_email = tarek@ziade.org
description = Command-line utility to store and retrieve passwords
home_page = http://bitbucket.org/tarek/clvault
classifier = Development Status :: 3 - Alpha
License :: OSI Approved :: GNU General Public License (GPL)
[files]
packages = clvault
Our project will depend on the *keyring* project. Let's add it in the
[metadata] section::
[metadata]
...
requires_dist =
keyring
Running commands
----------------
You can run useful commands on your project once the setup.cfg file is ready:
- sdist: creates a source distribution
- register: register your project to PyPI
- upload: upload the distribution to PyPI
- install_dist: install it
All commands are run using the run script::
$ pysetup run install_dist
$ pysetup run sdist
$ pysetup run upload
If you want to push a source distribution of your project to PyPI, do::
$ pysetup run sdist register upload
Installing the project
----------------------
The project can be installed by manually running the packaging install command::
$ pysetup run install_dist

80
Doc/packaging/uploading.rst Normal file
View file

@ -0,0 +1,80 @@
.. _packaging-package-upload:
***************************************
Uploading Packages to the Package Index
***************************************
The Python Package Index (PyPI) not only stores the package info, but also the
package data if the author of the package wishes to. The packaging command
:command:`upload` pushes the distribution files to PyPI.
The command is invoked immediately after building one or more distribution
files. For example, the command ::
python setup.py sdist bdist_wininst upload
will cause the source distribution and the Windows installer to be uploaded to
PyPI. Note that these will be uploaded even if they are built using an earlier
invocation of :file:`setup.py`, but that only distributions named on the command
line for the invocation including the :command:`upload` command are uploaded.
The :command:`upload` command uses the username, password, and repository URL
from the :file:`$HOME/.pypirc` file (see section :ref:`packaging-pypirc` for more on this
file). If a :command:`register` command was previously called in the same
command, and if the password was entered in the prompt, :command:`upload` will
reuse the entered password. This is useful if you do not want to store a clear
text password in the :file:`$HOME/.pypirc` file.
You can specify another PyPI server with the :option:`--repository=*url*`
option::
python setup.py sdist bdist_wininst upload -r http://example.com/pypi
See section :ref:`packaging-pypirc` for more on defining several servers.
You can use the :option:`--sign` option to tell :command:`upload` to sign each
uploaded file using GPG (GNU Privacy Guard). The :program:`gpg` program must
be available for execution on the system :envvar:`PATH`. You can also specify
which key to use for signing using the :option:`--identity=*name*` option.
Other :command:`upload` options include :option:`--repository=<url>` or
:option:`--repository=<section>` where *url* is the url of the server and
*section* the name of the section in :file:`$HOME/.pypirc`, and
:option:`--show-response` (which displays the full response text from the PyPI
server for help in debugging upload problems).
PyPI package display
====================
The ``description`` field plays a special role at PyPI. It is used by
the server to display a home page for the registered package.
If you use the `reStructuredText <http://docutils.sourceforge.net/rst.html>`_
syntax for this field, PyPI will parse it and display an HTML output for
the package home page.
The ``description`` field can be filled from a text file located in the
project::
from packaging.core import setup
fp = open('README.txt')
try:
description = fp.read()
finally:
fp.close()
setup(name='Packaging',
description=description)
In that case, :file:`README.txt` is a regular reStructuredText text file located
in the root of the package besides :file:`setup.py`.
To prevent registering broken reStructuredText content, you can use the
:program:`rst2html` program that is provided by the :mod:`docutils` package
and check the ``description`` from the command line::
$ python setup.py --description | rst2html.py > output.html
:mod:`docutils` will display a warning if there's something wrong with your
syntax.