mirror of
				https://github.com/python/cpython.git
				synced 2025-11-03 19:34:08 +00:00 
			
		
		
		
	
		
			
				
	
	
		
			1412 lines
		
	
	
	
		
			56 KiB
		
	
	
	
		
			TeX
		
	
	
	
	
	
			
		
		
	
	
			1412 lines
		
	
	
	
		
			56 KiB
		
	
	
	
		
			TeX
		
	
	
	
	
	
\documentclass{howto}
 | 
						|
\usepackage{distutils}
 | 
						|
 | 
						|
% $Id$
 | 
						|
 | 
						|
\title{Distributing Python Modules}
 | 
						|
 | 
						|
\author{Greg Ward}
 | 
						|
\authoraddress{Email: \email{gward@python.net}}
 | 
						|
 | 
						|
\makeindex
 | 
						|
 | 
						|
\begin{document}
 | 
						|
 | 
						|
\maketitle
 | 
						|
\begin{abstract}
 | 
						|
  \noindent
 | 
						|
  This document describes the Python Distribution Utilities
 | 
						|
  (``Distutils'') from the module developer's point-of-view, describing
 | 
						|
  how to use the Distutils to make Python modules and extensions easily
 | 
						|
  available to a wider audience with very little overhead for
 | 
						|
  build/release/install mechanics.
 | 
						|
\end{abstract}
 | 
						|
 | 
						|
% The ugly "%begin{latexonly}" pseudo-environment supresses the table
 | 
						|
% of contents for HTML generation.
 | 
						|
%
 | 
						|
%begin{latexonly}
 | 
						|
\tableofcontents
 | 
						|
%end{latexonly}
 | 
						|
 | 
						|
 | 
						|
\section{Introduction}
 | 
						|
\label{intro}
 | 
						|
 | 
						|
In the past, Python module developers have not had much infrastructure
 | 
						|
support for distributing modules, nor have Python users had much support
 | 
						|
for installing and maintaining third-party modules.  With the
 | 
						|
introduction of the Python Distribution Utilities (Distutils for short)
 | 
						|
in Python 1.6, this situation should start to improve.
 | 
						|
 | 
						|
This document only covers using the Distutils to distribute your Python
 | 
						|
modules.  Using the Distutils does not tie you to Python 1.6, though:
 | 
						|
the Distutils work just fine with Python 1.5.2, and it is reasonable
 | 
						|
(and expected to become commonplace) to expect users of Python 1.5.2 to
 | 
						|
download and install the Distutils separately before they can install
 | 
						|
your modules.  Python 1.6 (or later) users, of course, won't have to add
 | 
						|
anything to their Python installation in order to use the Distutils to
 | 
						|
install third-party modules.
 | 
						|
 | 
						|
This document concentrates on the role of developer/distributor: if
 | 
						|
you're looking for information on installing Python modules, you
 | 
						|
should refer to the \citetitle[../inst/inst.html]{Installing Python
 | 
						|
Modules} manual.
 | 
						|
 | 
						|
 | 
						|
\section{Concepts \& Terminology}
 | 
						|
\label{concepts}
 | 
						|
 | 
						|
Using the Distutils is quite simple, both for module developers and for
 | 
						|
users/administrators installing third-party modules.  As a developer,
 | 
						|
your responsibilities (apart from writing solid, well-documented and
 | 
						|
well-tested code, of course!) are:
 | 
						|
\begin{itemize}
 | 
						|
\item write a setup script (\file{setup.py} by convention)
 | 
						|
\item (optional) write a setup configuration file
 | 
						|
\item create a source distribution
 | 
						|
\item (optional) create one or more built (binary) distributions
 | 
						|
\end{itemize}
 | 
						|
Each of these tasks is covered in this document.
 | 
						|
 | 
						|
Not all module developers have access to a multitude of platforms, so
 | 
						|
it's not always feasible to expect them to create a multitude of built
 | 
						|
distributions.  It is hoped that a class of intermediaries, called
 | 
						|
\emph{packagers}, will arise to address this need.  Packagers will take
 | 
						|
source distributions released by module developers, build them on one or
 | 
						|
more platforms, and release the resulting built distributions.  Thus,
 | 
						|
users on the most popular platforms will be able to install most popular
 | 
						|
Python module distributions in the most natural way for their platform,
 | 
						|
without having to run a single setup script or compile a line of code.
 | 
						|
 | 
						|
 | 
						|
\subsection{A simple example}
 | 
						|
\label{simple-example}
 | 
						|
 | 
						|
The setup script is usually quite simple, although since it's written in
 | 
						|
Python, there are no arbitrary limits to what you can do with
 | 
						|
it.\footnote{But be careful about putting arbitrarily expensive
 | 
						|
  operations in your setup script; unlike, say, Autoconf-style configure
 | 
						|
  scripts, the setup script may be run multiple times in the course of
 | 
						|
  building and installing your module distribution.  If you need to
 | 
						|
  insert potentially expensive processing steps into the Distutils
 | 
						|
  chain, see section~\ref{extending} on extending the Distutils.}  If
 | 
						|
all you want to do is distribute a module called \module{foo}, contained
 | 
						|
in a file \file{foo.py}, then your setup script can be as little as
 | 
						|
this:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
from distutils.core import setup
 | 
						|
setup(name="foo",
 | 
						|
      version="1.0",
 | 
						|
      py_modules=["foo"])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
Some observations:
 | 
						|
\begin{itemize}
 | 
						|
\item most information that you supply to the Distutils is supplied as
 | 
						|
  keyword arguments to the \function{setup()} function
 | 
						|
\item those keyword arguments fall into two categories: package
 | 
						|
  meta-data (name, version number) and information about what's in the
 | 
						|
  package (a list of pure Python modules, in this case)
 | 
						|
\item modules are specified by module name, not filename (the same will
 | 
						|
  hold true for packages and extensions)
 | 
						|
\item it's recommended that you supply a little more meta-data, in
 | 
						|
  particular your name, email address and a URL for the project
 | 
						|
  (see section~\ref{setup-script} for an example)
 | 
						|
\end{itemize}
 | 
						|
 | 
						|
To create a source distribution for this module, you would create a
 | 
						|
setup script, \file{setup.py}, containing the above code, and run:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py sdist
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
which will create an archive file (e.g., tarball on \UNIX, ZIP file on
 | 
						|
Windows) containing your setup script, \file{setup.py}, and your module,
 | 
						|
\file{foo.py}.  The archive file will be named \file{Foo-1.0.tar.gz} (or
 | 
						|
\file{.zip}), and will unpack into a directory \file{Foo-1.0}.
 | 
						|
 | 
						|
If an end-user wishes to install your \module{foo} module, all she has
 | 
						|
to do is download \file{Foo-1.0.tar.gz} (or \file{.zip}), unpack it,
 | 
						|
and---from the \file{Foo-1.0} directory---run
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py install
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
which will ultimately copy \file{foo.py} to the appropriate directory
 | 
						|
for third-party modules in their Python installation.
 | 
						|
 | 
						|
This simple example demonstrates some fundamental concepts of the
 | 
						|
Distutils: first, both developers and installers have the same basic
 | 
						|
user interface, i.e. the setup script.  The difference is which
 | 
						|
Distutils \emph{commands} they use: the \command{sdist} command is
 | 
						|
almost exclusively for module developers, while \command{install} is
 | 
						|
more often for installers (although most developers will want to install
 | 
						|
their own code occasionally).
 | 
						|
 | 
						|
If you want to make things really easy for your users, you can create
 | 
						|
one or more built distributions for them.  For instance, if you are
 | 
						|
running on a Windows machine, and want to make things easy for other
 | 
						|
Windows users, you can create an executable installer (the most
 | 
						|
appropriate type of built distribution for this platform) with the
 | 
						|
\command{bdist\_wininst} command.  For example:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist_wininst
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
will create an executable installer, \file{Foo-1.0.win32.exe}, in the
 | 
						|
current directory.
 | 
						|
 | 
						|
Currently (Distutils 0.9.2), the only other useful built
 | 
						|
distribution format is RPM, implemented by the \command{bdist\_rpm}
 | 
						|
command.  For example, the following command will create an RPM file
 | 
						|
called \file{Foo-1.0.noarch.rpm}:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist_rpm
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
(This uses the \command{rpm} command, so has to be run on an RPM-based
 | 
						|
system such as Red Hat Linux, SuSE Linux, or Mandrake Linux.)
 | 
						|
 | 
						|
You can find out what distribution formats are available at any time by
 | 
						|
running
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist --help-formats
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
 | 
						|
\subsection{General Python terminology}
 | 
						|
\label{python-terms}
 | 
						|
 | 
						|
If you're reading this document, you probably have a good idea of what
 | 
						|
modules, extensions, and so forth are.  Nevertheless, just to be sure
 | 
						|
that everyone is operating from a common starting point, we offer the
 | 
						|
following glossary of common Python terms:
 | 
						|
\begin{description}
 | 
						|
\item[module] the basic unit of code reusability in Python: a block of
 | 
						|
  code imported by some other code.  Three types of modules concern us
 | 
						|
  here: pure Python modules, extension modules, and packages.
 | 
						|
\item[pure Python module] a module written in Python and contained in a
 | 
						|
  single \file{.py} file (and possibly associated \file{.pyc} and/or
 | 
						|
  \file{.pyo} files).  Sometimes referred to as a ``pure module.''
 | 
						|
\item[extension module] a module written in the low-level language of
 | 
						|
  the Python implementation: C/C++ for Python, Java for JPython.
 | 
						|
  Typically contained in a single dynamically loadable pre-compiled
 | 
						|
  file, e.g. a shared object (\file{.so}) file for Python extensions on
 | 
						|
  \UNIX, a DLL (given the \file{.pyd} extension) for Python extensions
 | 
						|
  on Windows, or a Java class file for JPython extensions.  (Note that
 | 
						|
  currently, the Distutils only handles C/C++ extensions for Python.)
 | 
						|
\item[package] a module that contains other modules; typically contained
 | 
						|
  in a directory in the filesystem and distinguished from other
 | 
						|
  directories by the presence of a file \file{\_\_init\_\_.py}.
 | 
						|
\item[root package] the root of the hierarchy of packages.  (This isn't
 | 
						|
  really a package, since it doesn't have an \file{\_\_init\_\_.py}
 | 
						|
  file.  But we have to call it something.)  The vast majority of the
 | 
						|
  standard library is in the root package, as are many small, standalone
 | 
						|
  third-party modules that don't belong to a larger module collection.
 | 
						|
  Unlike regular packages, modules in the root package can be found in
 | 
						|
  many directories: in fact, every directory listed in \code{sys.path}
 | 
						|
  can contribute modules to the root package.
 | 
						|
\end{description}
 | 
						|
 | 
						|
 | 
						|
\subsection{Distutils-specific terminology}
 | 
						|
\label{distutils-term}
 | 
						|
 | 
						|
The following terms apply more specifically to the domain of
 | 
						|
distributing Python modules using the Distutils:
 | 
						|
\begin{description}
 | 
						|
\item[module distribution] a collection of Python modules distributed
 | 
						|
  together as a single downloadable resource and meant to be installed
 | 
						|
  \emph{en masse}.  Examples of some well-known module distributions are
 | 
						|
  Numeric Python, PyXML, PIL (the Python Imaging Library), or
 | 
						|
  mxDateTime.  (This would be called a \emph{package}, except that term
 | 
						|
  is already taken in the Python context: a single module distribution
 | 
						|
  may contain zero, one, or many Python packages.)
 | 
						|
\item[pure module distribution] a module distribution that contains only
 | 
						|
  pure Python modules and packages.  Sometimes referred to as a ``pure
 | 
						|
  distribution.''
 | 
						|
\item[non-pure module distribution] a module distribution that contains
 | 
						|
  at least one extension module.  Sometimes referred to as a ``non-pure
 | 
						|
  distribution.''
 | 
						|
\item[distribution root] the top-level directory of your source tree (or 
 | 
						|
  source distribution); the directory where \file{setup.py} exists and
 | 
						|
  is run from
 | 
						|
\end{description}
 | 
						|
 | 
						|
 | 
						|
\section{Writing the Setup Script}
 | 
						|
\label{setup-script}
 | 
						|
 | 
						|
The setup script is the centre of all activity in building,
 | 
						|
distributing, and installing modules using the Distutils.  The main
 | 
						|
purpose of the setup script is to describe your module distribution to
 | 
						|
the Distutils, so that the various commands that operate on your modules
 | 
						|
do the right thing.  As we saw in section~\ref{simple-example} above,
 | 
						|
the setup script consists mainly of a call to \function{setup()}, and
 | 
						|
most information supplied to the Distutils by the module developer is
 | 
						|
supplied as keyword arguments to \function{setup()}.
 | 
						|
 | 
						|
Here's a slightly more involved example, which we'll follow for the next
 | 
						|
couple of sections: the Distutils' own setup script.  (Keep in mind that
 | 
						|
although the Distutils are included with Python 1.6 and later, they also
 | 
						|
have an independent existence so that Python 1.5.2 users can use them to
 | 
						|
install other module distributions.  The Distutils' own setup script,
 | 
						|
shown here, is used to install the package into Python 1.5.2.)
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
#!/usr/bin/env python
 | 
						|
 | 
						|
from distutils.core import setup
 | 
						|
 | 
						|
setup(name="Distutils",
 | 
						|
      version="1.0",
 | 
						|
      description="Python Distribution Utilities",
 | 
						|
      author="Greg Ward",
 | 
						|
      author_email="gward@python.net",
 | 
						|
      url="http://www.python.org/sigs/distutils-sig/",
 | 
						|
      packages=['distutils', 'distutils.command'],
 | 
						|
     )
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
There are only two differences between this and the trivial one-file
 | 
						|
distribution presented in section~\ref{simple-example}: more
 | 
						|
meta-data, and the specification of pure Python modules by package,
 | 
						|
rather than by module.  This is important since the Distutils consist of
 | 
						|
a couple of dozen modules split into (so far) two packages; an explicit
 | 
						|
list of every module would be tedious to generate and difficult to
 | 
						|
maintain.
 | 
						|
 | 
						|
Note that any pathnames (files or directories) supplied in the setup
 | 
						|
script should be written using the \UNIX{} convention, i.e.
 | 
						|
slash-separated.  The Distutils will take care of converting this
 | 
						|
platform-neutral representation into whatever is appropriate on your
 | 
						|
current platform before actually using the pathname.  This makes your
 | 
						|
setup script portable across operating systems, which of course is one
 | 
						|
of the major goals of the Distutils.  In this spirit, all pathnames in
 | 
						|
this document are slash-separated (MacOS programmers should keep in
 | 
						|
mind that the \emph{absence} of a leading slash indicates a relative
 | 
						|
path, the opposite of the MacOS convention with colons).
 | 
						|
 | 
						|
This, of course, only applies to pathnames given to Distutils functions.
 | 
						|
If you, for example, use standard python functions such as glob.glob
 | 
						|
or os.listdir to specify files, you should be careful to write portable
 | 
						|
code instead of hardcoding path separators:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
    glob.glob(os.path.join('mydir', 'subdir', '*.html'))
 | 
						|
    os.listdir(os.path.join('mydir', 'subdir'))
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
\subsection{Listing whole packages}
 | 
						|
\label{listing-packages}
 | 
						|
 | 
						|
The \option{packages} option tells the Distutils to process (build,
 | 
						|
distribute, install, etc.) all pure Python modules found in each package
 | 
						|
mentioned in the \option{packages} list.  In order to do this, of
 | 
						|
course, there has to be a correspondence between package names and
 | 
						|
directories in the filesystem.  The default correspondence is the most
 | 
						|
obvious one, i.e. package \module{distutils} is found in the directory
 | 
						|
\file{distutils} relative to the distribution root.  Thus, when you say
 | 
						|
\code{packages = ['foo']} in your setup script, you are promising that
 | 
						|
the Distutils will find a file \file{foo/\_\_init\_\_.py} (which might
 | 
						|
be spelled differently on your system, but you get the idea) relative to
 | 
						|
the directory where your setup script lives.  (If you break this
 | 
						|
promise, the Distutils will issue a warning but process the broken
 | 
						|
package anyways.)
 | 
						|
 | 
						|
If you use a different convention to lay out your source directory,
 | 
						|
that's no problem: you just have to supply the \option{package\_dir}
 | 
						|
option to tell the Distutils about your convention.  For example, say
 | 
						|
you keep all Python source under \file{lib}, so that modules in the
 | 
						|
``root package'' (i.e., not in any package at all) are right in
 | 
						|
\file{lib}, modules in the \module{foo} package are in \file{lib/foo},
 | 
						|
and so forth.  Then you would put
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
package_dir = {'': 'lib'}
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
in your setup script.  (The keys to this dictionary are package names,
 | 
						|
and an empty package name stands for the root package.  The values are
 | 
						|
directory names relative to your distribution root.)  In this case, when
 | 
						|
you say \code{packages = ['foo']}, you are promising that the file
 | 
						|
\file{lib/foo/\_\_init\_\_.py} exists.
 | 
						|
 | 
						|
Another possible convention is to put the \module{foo} package right in 
 | 
						|
\file{lib}, the \module{foo.bar} package in \file{lib/bar}, etc.  This
 | 
						|
would be written in the setup script as
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
package_dir = {'foo': 'lib'}
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
A \code{\var{package}: \var{dir}} entry in the \option{package\_dir}
 | 
						|
dictionary implicitly applies to all packages below \var{package}, so
 | 
						|
the \module{foo.bar} case is automatically handled here.  In this
 | 
						|
example, having \code{packages = ['foo', 'foo.bar']} tells the Distutils
 | 
						|
to look for \file{lib/\_\_init\_\_.py} and
 | 
						|
\file{lib/bar/\_\_init\_\_.py}.  (Keep in mind that although
 | 
						|
\option{package\_dir} applies recursively, you must explicitly list all
 | 
						|
packages in \option{packages}: the Distutils will \emph{not} recursively
 | 
						|
scan your source tree looking for any directory with an
 | 
						|
\file{\_\_init\_\_.py} file.)
 | 
						|
 | 
						|
 | 
						|
\subsection{Listing individual modules}
 | 
						|
\label{listing-modules}
 | 
						|
 | 
						|
For a small module distribution, you might prefer to list all modules
 | 
						|
rather than listing packages---especially the case of a single module
 | 
						|
that goes in the ``root package'' (i.e., no package at all).  This
 | 
						|
simplest case was shown in section~\ref{simple-example}; here is a
 | 
						|
slightly more involved example:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
py_modules = ['mod1', 'pkg.mod2']
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
This describes two modules, one of them in the ``root'' package, the
 | 
						|
other in the \module{pkg} package.  Again, the default package/directory
 | 
						|
layout implies that these two modules can be found in \file{mod1.py} and
 | 
						|
\file{pkg/mod2.py}, and that \file{pkg/\_\_init\_\_.py} exists as well.
 | 
						|
And again, you can override the package/directory correspondence using
 | 
						|
the \option{package\_dir} option.
 | 
						|
 | 
						|
 | 
						|
\subsection{Describing extension modules}
 | 
						|
\label{describing-extensions}
 | 
						|
 | 
						|
Just as writing Python extension modules is a bit more complicated than
 | 
						|
writing pure Python modules, describing them to the Distutils is a bit
 | 
						|
more complicated.  Unlike pure modules, it's not enough just to list
 | 
						|
modules or packages and expect the Distutils to go out and find the
 | 
						|
right files; you have to specify the extension name, source file(s), and
 | 
						|
any compile/link requirements (include directories, libraries to link
 | 
						|
with, etc.).
 | 
						|
 | 
						|
All of this is done through another keyword argument to
 | 
						|
\function{setup()}, the \option{extensions} option.  \option{extensions}
 | 
						|
is just a list of \class{Extension} instances, each of which describes a
 | 
						|
single extension module.  Suppose your distribution includes a single
 | 
						|
extension, called \module{foo} and implemented by \file{foo.c}.  If no
 | 
						|
additional instructions to the compiler/linker are needed, describing
 | 
						|
this extension is quite simple:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
Extension("foo", ["foo.c"])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
The \class{Extension} class can be imported from
 | 
						|
\module{distutils.core}, along with \function{setup()}.  Thus, the setup
 | 
						|
script for a module distribution that contains only this one extension
 | 
						|
and nothing else might be:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
from distutils.core import setup, Extension
 | 
						|
setup(name="foo", version="1.0",
 | 
						|
      ext_modules=[Extension("foo", ["foo.c"])])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
The \class{Extension} class (actually, the underlying extension-building
 | 
						|
machinery implemented by the \command{build\_ext} command) supports a
 | 
						|
great deal of flexibility in describing Python extensions, which is
 | 
						|
explained in the following sections.  
 | 
						|
 | 
						|
 | 
						|
\subsubsection{Extension names and packages}
 | 
						|
 | 
						|
The first argument to the \class{Extension} constructor is always the
 | 
						|
name of the extension, including any package names.  For example,
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
Extension("foo", ["src/foo1.c", "src/foo2.c"])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
describes an extension that lives in the root package, while
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
Extension("pkg.foo", ["src/foo1.c", "src/foo2.c"])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
describes the same extension in the \module{pkg} package.  The source
 | 
						|
files and resulting object code are identical in both cases; the only
 | 
						|
difference is where in the filesystem (and therefore where in Python's
 | 
						|
namespace hierarchy) the resulting extension lives.
 | 
						|
 | 
						|
If you have a number of extensions all in the same package (or all under
 | 
						|
the same base package), use the \option{ext\_package} keyword argument
 | 
						|
to \function{setup()}.  For example,
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
setup(...
 | 
						|
      ext_package="pkg",
 | 
						|
      ext_modules=[Extension("foo", ["foo.c"]),
 | 
						|
                   Extension("subpkg.bar", ["bar.c"])]
 | 
						|
     )
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
will compile \file{foo.c} to the extension \module{pkg.foo}, and
 | 
						|
\file{bar.c} to \module{pkg.subpkg.bar}.
 | 
						|
 | 
						|
 | 
						|
\subsubsection{Extension source files}
 | 
						|
 | 
						|
The second argument to the \class{Extension} constructor is a list of
 | 
						|
source files.  Since the Distutils currently only support C/C++
 | 
						|
extensions, these are normally C/C++ source files.  (Be sure to use
 | 
						|
appropriate extensions to distinguish C++ source files: \file{.cc} and
 | 
						|
\file{.cpp} seem to be recognized by both \UNIX{} and Windows compilers.)
 | 
						|
 | 
						|
However, you can also include SWIG interface (\file{.i}) files in the
 | 
						|
list; the \command{build\_ext} command knows how to deal with SWIG
 | 
						|
extensions: it will run SWIG on the interface file and compile the
 | 
						|
resulting C/C++ file into your extension.
 | 
						|
 | 
						|
\XXX{SWIG support is rough around the edges and largely untested;
 | 
						|
  especially SWIG support of C++ extensions!  Explain in more detail
 | 
						|
  here when the interface firms up.}
 | 
						|
 | 
						|
On some platforms, you can include non-source files that are processed
 | 
						|
by the compiler and included in your extension.  Currently, this just
 | 
						|
means Windows message text (\file{.mc}) files and resource definition
 | 
						|
(\file{.rc}) files for Visual C++. These will be compiled to binary resource
 | 
						|
(\file{.res}) files and linked into the executable.
 | 
						|
 | 
						|
 | 
						|
\subsubsection{Preprocessor options}
 | 
						|
 | 
						|
Three optional arguments to \class{Extension} will help if you need to
 | 
						|
specify include directories to search or preprocessor macros to
 | 
						|
define/undefine: \code{include\_dirs}, \code{define\_macros}, and
 | 
						|
\code{undef\_macros}.
 | 
						|
 | 
						|
For example, if your extension requires header files in the
 | 
						|
\file{include} directory under your distribution root, use the
 | 
						|
\code{include\_dirs} option:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
Extension("foo", ["foo.c"], include_dirs=["include"])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
You can specify absolute directories there; if you know that your
 | 
						|
extension will only be built on \UNIX{} systems with X11R6 installed to
 | 
						|
\file{/usr}, you can get away with
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
Extension("foo", ["foo.c"], include_dirs=["/usr/include/X11"])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
You should avoid this sort of non-portable usage if you plan to
 | 
						|
distribute your code: it's probably better to write your code to include
 | 
						|
(e.g.) \code{<X11/Xlib.h>}.
 | 
						|
 | 
						|
If you need to include header files from some other Python extension,
 | 
						|
you can take advantage of the fact that the Distutils install extension
 | 
						|
header files in a consistent way.  For example, the Numerical Python
 | 
						|
header files are installed (on a standard \UNIX{} installation) to
 | 
						|
\file{/usr/local/include/python1.5/Numerical}.  (The exact location will
 | 
						|
differ according to your platform and Python installation.)  Since the
 | 
						|
Python include directory---\file{/usr/local/include/python1.5} in this
 | 
						|
case---is always included in the search path when building Python
 | 
						|
extensions, the best approach is to include (e.g.)
 | 
						|
\code{<Numerical/arrayobject.h>}.  If you insist on putting the
 | 
						|
\file{Numerical} include directory right into your header search path,
 | 
						|
though, you can find that directory using the Distutils
 | 
						|
\module{sysconfig} module:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
from distutils.sysconfig import get_python_inc
 | 
						|
incdir = os.path.join(get_python_inc(plat_specific=1), "Numerical")
 | 
						|
setup(...,
 | 
						|
      Extension(..., include_dirs=[incdir]))
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
Even though this is quite portable---it will work on any Python
 | 
						|
installation, regardless of platform---it's probably easier to just
 | 
						|
write your C code in the sensible way.
 | 
						|
 | 
						|
You can define and undefine pre-processor macros with the
 | 
						|
\code{define\_macros} and \code{undef\_macros} options.
 | 
						|
\code{define\_macros} takes a list of \code{(name, value)} tuples, where
 | 
						|
\code{name} is the name of the macro to define (a string) and
 | 
						|
\code{value} is its value: either a string or \code{None}.  (Defining a
 | 
						|
macro \code{FOO} to \code{None} is the equivalent of a bare
 | 
						|
\code{\#define FOO} in your C source: with most compilers, this sets
 | 
						|
\code{FOO} to the string \code{1}.)  \code{undef\_macros} is just
 | 
						|
a list of macros to undefine.
 | 
						|
 | 
						|
For example:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
Extension(...,
 | 
						|
          define_macros=[('NDEBUG', '1')],
 | 
						|
                         ('HAVE_STRFTIME', None),
 | 
						|
          undef_macros=['HAVE_FOO', 'HAVE_BAR'])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
is the equivalent of having this at the top of every C source file:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
#define NDEBUG 1
 | 
						|
#define HAVE_STRFTIME
 | 
						|
#undef HAVE_FOO
 | 
						|
#undef HAVE_BAR
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
 | 
						|
\subsubsection{Library options}
 | 
						|
 | 
						|
You can also specify the libraries to link against when building your
 | 
						|
extension, and the directories to search for those libraries.  The
 | 
						|
\code{libraries} option is a list of libraries to link against,
 | 
						|
\code{library\_dirs} is a list of directories to search for libraries at 
 | 
						|
link-time, and \code{runtime\_library\_dirs} is a list of directories to 
 | 
						|
search for shared (dynamically loaded) libraries at run-time.
 | 
						|
 | 
						|
For example, if you need to link against libraries known to be in the
 | 
						|
standard library search path on target systems
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
Extension(...,
 | 
						|
          libraries=["gdbm", "readline"])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
If you need to link with libraries in a non-standard location, you'll
 | 
						|
have to include the location in \code{library\_dirs}:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
Extension(...,
 | 
						|
          library_dirs=["/usr/X11R6/lib"],
 | 
						|
          libraries=["X11", "Xt"])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
(Again, this sort of non-portable construct should be avoided if you
 | 
						|
intend to distribute your code.)
 | 
						|
 | 
						|
\XXX{Should mention clib libraries here or somewhere else!}
 | 
						|
 | 
						|
\subsubsection{Other options}
 | 
						|
 | 
						|
There are still some other options which can be used to handle special
 | 
						|
cases.
 | 
						|
 | 
						|
The \option{extra\_objects} option is a list of object files to be passed
 | 
						|
to the linker. These files must not have extensions, as the default
 | 
						|
extension for the compiler is used.
 | 
						|
 | 
						|
\option{extra\_compile\_args} and \option{extra\_link\_args} can be used
 | 
						|
to specify additional command line options for the compiler resp.
 | 
						|
the linker command line.
 | 
						|
 | 
						|
\option{export\_symbols} is only useful on windows, it can contain a list
 | 
						|
of symbols (functions or variables) to be exported. This option
 | 
						|
is not needed when building compiled extensions: the \code{initmodule}
 | 
						|
function will automatically be added to the exported symbols list
 | 
						|
by Distutils.
 | 
						|
 | 
						|
\subsection{Listing scripts}
 | 
						|
So far we have been dealing with pure and non-pure Python modules,
 | 
						|
which are usually not run by themselves but imported by scripts.
 | 
						|
 | 
						|
Scripts are files containing Python source code, indended to be started
 | 
						|
from the command line.
 | 
						|
Distutils doesn't provide much functionality for the scripts: the only
 | 
						|
support Distutils gives is to adjust the first line of the script
 | 
						|
if it starts with \code{\#!} and contains the word ``python'' to refer
 | 
						|
to the current interpreter location.
 | 
						|
 | 
						|
The \option{scripts} option simply is a list of files to be handled
 | 
						|
in this way.
 | 
						|
 | 
						|
 | 
						|
\subsection{Listing additional files}
 | 
						|
 | 
						|
The \option{data\_files} option can be used to specify additional
 | 
						|
files needed by the module distribution: configuration files,
 | 
						|
data files, anything which does not fit in the previous categories.
 | 
						|
 | 
						|
\option{data\_files} specify a sequence of \code{(directory, files)}
 | 
						|
pairs in the following way:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
setup(...
 | 
						|
      data_files=[('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
 | 
						|
                  ('config', ['cfg/data.cfg'])])
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
Note that you can specify the directory names where the data files
 | 
						|
will be installed, but you cannot rename the data files themselves.
 | 
						|
 | 
						|
You can specify the \option{data\_files} options as a simple sequence
 | 
						|
of files without specifying a target directory, but this is not recommended,
 | 
						|
and the \command{install} command will print a warning in this case.
 | 
						|
To install data files directly in the target directory, an empty
 | 
						|
string should be given as the directory.
 | 
						|
 | 
						|
 | 
						|
\section{Writing the Setup Configuration File}
 | 
						|
\label{setup-config}
 | 
						|
 | 
						|
Often, it's not possible to write down everything needed to build a
 | 
						|
distribution \emph{a priori}: you may need to get some information from
 | 
						|
the user, or from the user's system, in order to proceed.  As long as
 | 
						|
that information is fairly simple---a list of directories to search for
 | 
						|
C header files or libraries, for example---then providing a
 | 
						|
configuration file, \file{setup.cfg}, for users to edit is a cheap and
 | 
						|
easy way to solicit it.  Configuration files also let you provide
 | 
						|
default values for any command option, which the installer can then
 | 
						|
override either on the command-line or by editing the config file.
 | 
						|
 | 
						|
(If you have more advanced needs, such as determining which extensions
 | 
						|
to build based on what capabilities are present on the target system,
 | 
						|
then you need the Distutils ``auto-configuration'' facility.  This
 | 
						|
started to appear in Distutils 0.9 but, as of this writing, isn't mature 
 | 
						|
or stable enough yet for real-world use.)
 | 
						|
 | 
						|
The setup configuration file is a useful middle-ground between the setup
 | 
						|
script---which, ideally, would be opaque to installers\footnote{This
 | 
						|
  ideal probably won't be achieved until auto-configuration is fully
 | 
						|
  supported by the Distutils.}---and the command-line to the setup
 | 
						|
script, which is outside of your control and entirely up to the
 | 
						|
installer.  In fact, \file{setup.cfg} (and any other Distutils
 | 
						|
configuration files present on the target system) are processed after
 | 
						|
the contents of the setup script, but before the command-line.  This has 
 | 
						|
several useful consequences:
 | 
						|
\begin{itemize}
 | 
						|
\item installers can override some of what you put in \file{setup.py} by
 | 
						|
  editing \file{setup.cfg}
 | 
						|
\item you can provide non-standard defaults for options that are not
 | 
						|
  easily set in \file{setup.py}
 | 
						|
\item installers can override anything in \file{setup.cfg} using the
 | 
						|
  command-line options to \file{setup.py}
 | 
						|
\end{itemize}
 | 
						|
 | 
						|
The basic syntax of the configuration file is simple:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
[command]
 | 
						|
option=value
 | 
						|
...
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
where \var{command} is one of the Distutils commands (e.g.
 | 
						|
\command{build\_py}, \command{install}), and \var{option} is one of the
 | 
						|
options that command supports.  Any number of options can be supplied
 | 
						|
for each command, and any number of command sections can be included in
 | 
						|
the file.  Blank lines are ignored, as are comments (from a
 | 
						|
\character{\#} character to end-of-line).  Long option values can be
 | 
						|
split across multiple lines simply by indenting the continuation lines.
 | 
						|
 | 
						|
You can find out the list of options supported by a particular command
 | 
						|
with the universal \longprogramopt{help} option, e.g.
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
> python setup.py --help build_ext
 | 
						|
[...]
 | 
						|
Options for 'build_ext' command:
 | 
						|
  --build-lib (-b)     directory for compiled extension modules
 | 
						|
  --build-temp (-t)    directory for temporary files (build by-products)
 | 
						|
  --inplace (-i)       ignore build-lib and put compiled extensions into the
 | 
						|
                       source directory alongside your pure Python modules
 | 
						|
  --include-dirs (-I)  list of directories to search for header files
 | 
						|
  --define (-D)        C preprocessor macros to define
 | 
						|
  --undef (-U)         C preprocessor macros to undefine
 | 
						|
[...]
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
Or consult section \ref{reference} of this document (the command
 | 
						|
reference).
 | 
						|
 | 
						|
Note that an option spelled \longprogramopt{foo-bar} on the command-line 
 | 
						|
is spelled \option{foo\_bar} in configuration files.
 | 
						|
 | 
						|
For example, say you want your extensions to be built
 | 
						|
``in-place''---that is, you have an extension \module{pkg.ext}, and you
 | 
						|
want the compiled extension file (\file{ext.so} on \UNIX, say) to be put
 | 
						|
in the same source directory as your pure Python modules
 | 
						|
\module{pkg.mod1} and \module{pkg.mod2}.  You can always use the
 | 
						|
\longprogramopt{inplace} option on the command-line to ensure this:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py build_ext --inplace
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
But this requires that you always specify the \command{build\_ext}
 | 
						|
command explicitly, and remember to provide \longprogramopt{inplace}.
 | 
						|
An easier way is to ``set and forget'' this option, by encoding it in
 | 
						|
\file{setup.cfg}, the configuration file for this distribution:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
[build_ext]
 | 
						|
inplace=1
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
This will affect all builds of this module distribution, whether or not
 | 
						|
you explcitly specify \command{build\_ext}.  If you include
 | 
						|
\file{setup.cfg} in your source distribution, it will also affect
 | 
						|
end-user builds---which is probably a bad idea for this option, since
 | 
						|
always building extensions in-place would break installation of the
 | 
						|
module distribution.  In certain peculiar cases, though, modules are
 | 
						|
built right in their installation directory, so this is conceivably a
 | 
						|
useful ability.  (Distributing extensions that expect to be built in
 | 
						|
their installation directory is almost always a bad idea, though.)
 | 
						|
 | 
						|
Another example: certain commands take a lot of options that don't
 | 
						|
change from run-to-run; for example, \command{bdist\_rpm} needs to know
 | 
						|
everything required to generate a ``spec'' file for creating an RPM
 | 
						|
distribution.  Some of this information comes from the setup script, and
 | 
						|
some is automatically generated by the Distutils (such as the list of
 | 
						|
files installed).  But some of it has to be supplied as options to
 | 
						|
\command{bdist\_rpm}, which would be very tedious to do on the
 | 
						|
command-line for every run.  Hence, here is a snippet from the
 | 
						|
Distutils' own \file{setup.cfg}:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
[bdist_rpm]
 | 
						|
release = 1
 | 
						|
packager = Greg Ward <gward@python.net>
 | 
						|
doc_files = CHANGES.txt
 | 
						|
            README.txt
 | 
						|
            USAGE.txt
 | 
						|
            doc/
 | 
						|
            examples/
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
Note that the \option{doc\_files} option is simply a
 | 
						|
whitespace-separated string split across multiple lines for readability.
 | 
						|
 | 
						|
 | 
						|
\begin{seealso}
 | 
						|
  \seetitle[../inst/config-syntax.html]{Installing Python
 | 
						|
            Modules}{More information on the configuration files is
 | 
						|
            available in the manual for system administrators.}
 | 
						|
\end{seealso}
 | 
						|
 | 
						|
 | 
						|
\section{Creating a Source Distribution}
 | 
						|
\label{source-dist}
 | 
						|
 | 
						|
As shown in section~\ref{simple-example}, you use the
 | 
						|
\command{sdist} command to create a source distribution.  In the
 | 
						|
simplest case,
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py sdist
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
(assuming you haven't specified any \command{sdist} options in the setup
 | 
						|
script or config file), \command{sdist} creates the archive of the
 | 
						|
default format for the current platform.  The default format is gzip'ed
 | 
						|
tar file (\file{.tar.gz}) on \UNIX, and ZIP file on Windows.
 | 
						|
\XXX{no MacOS support here}
 | 
						|
 | 
						|
You can specify as many formats as you like using the
 | 
						|
\longprogramopt{formats} option, for example:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py sdist --formats=gztar,zip
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
to create a gzipped tarball and a zip file.  The available formats are:
 | 
						|
\begin{tableiii}{l|l|c}{code}%
 | 
						|
  {Format}{Description}{Notes}
 | 
						|
  \lineiii{zip}{zip file (\file{.zip})}{(1),(3)}
 | 
						|
  \lineiii{gztar}{gzip'ed tar file (\file{.tar.gz})}{(2),(4)}
 | 
						|
  \lineiii{bztar}{bzip2'ed tar file (\file{.tar.gz})}{(4)}
 | 
						|
  \lineiii{ztar}{compressed tar file (\file{.tar.Z})}{(4)}
 | 
						|
  \lineiii{tar}{tar file (\file{.tar})}{(4)}
 | 
						|
\end{tableiii}
 | 
						|
 | 
						|
\noindent Notes:
 | 
						|
\begin{description}
 | 
						|
\item[(1)] default on Windows
 | 
						|
\item[(2)] default on \UNIX
 | 
						|
\item[(3)] requires either external \program{zip} utility or
 | 
						|
  \module{zipfile} module (not part of the standard Python library)
 | 
						|
\item[(4)] requires external utilities: \program{tar} and possibly one
 | 
						|
  of \program{gzip}, \program{bzip2}, or \program{compress}
 | 
						|
\end{description}
 | 
						|
 | 
						|
 | 
						|
 | 
						|
\subsection{Specifying the files to distribute}
 | 
						|
\label{manifest}
 | 
						|
 | 
						|
If you don't supply an explicit list of files (or instructions on how to
 | 
						|
generate one), the \command{sdist} command puts a minimal default set
 | 
						|
into the source distribution:
 | 
						|
\begin{itemize}
 | 
						|
\item all Python source files implied by the \option{py\_modules} and
 | 
						|
  \option{packages} options
 | 
						|
\item all C source files mentioned in the \option{ext\_modules} or
 | 
						|
  \option{libraries} options (\XXX{getting C library sources currently
 | 
						|
    broken -- no get\_source\_files() method in build\_clib.py!})
 | 
						|
\item anything that looks like a test script: \file{test/test*.py}
 | 
						|
  (currently, the Distutils don't do anything with test scripts except
 | 
						|
  include them in source distributions, but in the future there will be
 | 
						|
  a standard for testing Python module distributions)
 | 
						|
\item \file{README.txt} (or \file{README}), \file{setup.py} (or whatever 
 | 
						|
  you called your setup script), and \file{setup.cfg}
 | 
						|
\end{itemize}
 | 
						|
Sometimes this is enough, but usually you will want to specify
 | 
						|
additional files to distribute.  The typical way to do this is to write
 | 
						|
a \emph{manifest template}, called \file{MANIFEST.in} by default.  The
 | 
						|
manifest template is just a list of instructions for how to generate
 | 
						|
your manifest file, \file{MANIFEST}, which is the exact list of files to
 | 
						|
include in your source distribution.  The \command{sdist} command
 | 
						|
processes this template and generates a manifest based on its
 | 
						|
instructions and what it finds in the filesystem.
 | 
						|
 | 
						|
If you prefer to roll your own manifest file, the format is simple: one
 | 
						|
filename per line, regular files (or symlinks to them) only.  If you do
 | 
						|
supply your own \file{MANIFEST}, you must specify everything: the
 | 
						|
default set of files described above does not apply in this case.
 | 
						|
 | 
						|
The manifest template has one command per line, where each command
 | 
						|
specifies a set of files to include or exclude from the source
 | 
						|
distribution.  For an example, again we turn to the Distutils' own
 | 
						|
manifest template:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
include *.txt
 | 
						|
recursive-include examples *.txt *.py
 | 
						|
prune examples/sample?/build
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
The meanings should be fairly clear: include all files in the
 | 
						|
distribution root matching \code{*.txt}, all files anywhere under the
 | 
						|
\file{examples} directory matching \code{*.txt} or \code{*.py}, and
 | 
						|
exclude all directories matching \code{examples/sample?/build}.  All of
 | 
						|
this is done \emph{after} the standard include set, so you can exclude
 | 
						|
files from the standard set with explicit instructions in the manifest
 | 
						|
template.  (Or, you can use the \longprogramopt{no-defaults} option to
 | 
						|
disable the standard set entirely.)  There are several other commands
 | 
						|
available in the manifest template mini-language; see
 | 
						|
section~\ref{sdist-cmd}.
 | 
						|
 | 
						|
The order of commands in the manifest template matters: initially, we
 | 
						|
have the list of default files as described above, and each command in
 | 
						|
the template adds to or removes from that list of files.  Once we have
 | 
						|
fully processed the manifest template, we remove files that should not
 | 
						|
be included in the source distribution:
 | 
						|
\begin{itemize}
 | 
						|
\item all files in the Distutils ``build'' tree (default \file{build/})
 | 
						|
\item all files in directories named \file{RCS} or \file{CVS}
 | 
						|
\end{itemize}
 | 
						|
Now we have our complete list of files, which is written to the manifest
 | 
						|
for future reference, and then used to build the source distribution
 | 
						|
archive(s).
 | 
						|
 | 
						|
You can disable the default set of included files with the
 | 
						|
\longprogramopt{no-defaults} option, and you can disable the standard
 | 
						|
exclude set with \longprogramopt{no-prune}.
 | 
						|
 | 
						|
Following the Distutils' own manifest template, let's trace how the
 | 
						|
\command{sdist} command builds the list of files to include in the
 | 
						|
Distutils source distribution:
 | 
						|
\begin{enumerate}
 | 
						|
\item include all Python source files in the \file{distutils} and
 | 
						|
  \file{distutils/command} subdirectories (because packages
 | 
						|
  corresponding to those two directories were mentioned in the
 | 
						|
  \option{packages} option in the setup script---see
 | 
						|
  section~\ref{setup-script})
 | 
						|
\item include \file{README.txt}, \file{setup.py}, and \file{setup.cfg}
 | 
						|
  (standard files)
 | 
						|
\item include \file{test/test*.py} (standard files)
 | 
						|
\item include \file{*.txt} in the distribution root (this will find
 | 
						|
  \file{README.txt} a second time, but such redundancies are weeded out
 | 
						|
  later)
 | 
						|
\item include anything matching \file{*.txt} or \file{*.py} in the
 | 
						|
  sub-tree under \file{examples},
 | 
						|
\item exclude all files in the sub-trees starting at directories
 | 
						|
  matching \file{examples/sample?/build}---this may exclude files
 | 
						|
  included by the previous two steps, so it's important that the
 | 
						|
  \code{prune} command in the manifest template comes after the
 | 
						|
  \code{recursive-include} command
 | 
						|
\item exclude the entire \file{build} tree, and any \file{RCS} or
 | 
						|
  \file{CVS} directories
 | 
						|
\end{enumerate}
 | 
						|
Just like in the setup script, file and directory names in the manifest
 | 
						|
template should always be slash-separated; the Distutils will take care
 | 
						|
of converting them to the standard representation on your platform.
 | 
						|
That way, the manifest template is portable across operating systems.
 | 
						|
 | 
						|
 | 
						|
\subsection{Manifest-related options}
 | 
						|
\label{manifest-options}
 | 
						|
 | 
						|
The normal course of operations for the \command{sdist} command is as
 | 
						|
follows:
 | 
						|
\begin{itemize}
 | 
						|
\item if the manifest file, \file{MANIFEST} doesn't exist, read
 | 
						|
  \file{MANIFEST.in} and create the manifest
 | 
						|
\item if neither \file{MANIFEST} nor \file{MANIFEST.in} exist, create a
 | 
						|
  manifest with just the default file set\footnote{In versions of the
 | 
						|
    Distutils up to and including 0.9.2 (Python 2.0b1), this feature was
 | 
						|
    broken; use the \programopt{-f} (\longprogramopt{force-manifest})
 | 
						|
    option to work around the bug.}
 | 
						|
\item if either \file{MANIFEST.in} or the setup script (\file{setup.py})
 | 
						|
  are more recent than \file{MANIFEST}, recreate \file{MANIFEST} by
 | 
						|
  reading \file{MANIFEST.in}
 | 
						|
\item use the list of files now in \file{MANIFEST} (either just
 | 
						|
  generated or read in) to create the source distribution archive(s)
 | 
						|
\end{itemize}
 | 
						|
There are a couple of options that modify this behaviour.  First, use
 | 
						|
the \longprogramopt{no-defaults} and \longprogramopt{no-prune} to
 | 
						|
disable the standard ``include'' and ``exclude'' sets.\footnote{Note
 | 
						|
  that if you have no manifest template, no manifest, and use the
 | 
						|
  \longprogramopt{no-defaults}, you will get an empty manifest.  Another
 | 
						|
  bug in Distutils 0.9.2 and earlier causes an uncaught exception in
 | 
						|
  this case.  The workaround is: Don't Do That.}
 | 
						|
 | 
						|
Second, you might want to force the manifest to be regenerated---for
 | 
						|
example, if you have added or removed files or directories that match an
 | 
						|
existing pattern in the manifest template, you should regenerate the
 | 
						|
manifest:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py sdist --force-manifest
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
Or, you might just want to (re)generate the manifest, but not create a
 | 
						|
source distribution:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py sdist --manifest-only
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
\longprogramopt{manifest-only} implies \longprogramopt{force-manifest}.
 | 
						|
\programopt{-o} is a shortcut for \longprogramopt{manifest-only}, and
 | 
						|
\programopt{-f} for \longprogramopt{force-manifest}.
 | 
						|
 | 
						|
 | 
						|
\section{Creating Built Distributions}
 | 
						|
\label{built-dist}
 | 
						|
 | 
						|
A ``built distribution'' is what you're probably used to thinking of
 | 
						|
either as a ``binary package'' or an ``installer'' (depending on your
 | 
						|
background).  It's not necessarily binary, though, because it might
 | 
						|
contain only Python source code and/or byte-code; and we don't call it a
 | 
						|
package, because that word is already spoken for in Python.  (And
 | 
						|
``installer'' is a term specific to the Windows world.  \XXX{do Mac
 | 
						|
  people use it?})
 | 
						|
 | 
						|
A built distribution is how you make life as easy as possible for
 | 
						|
installers of your module distribution: for users of RPM-based Linux
 | 
						|
systems, it's a binary RPM; for Windows users, it's an executable
 | 
						|
installer; for Debian-based Linux users, it's a Debian package; and so
 | 
						|
forth.  Obviously, no one person will be able to create built
 | 
						|
distributions for every platform under the sun, so the Distutils are
 | 
						|
designed to enable module developers to concentrate on their
 | 
						|
specialty---writing code and creating source distributions---while an
 | 
						|
intermediary species of \emph{packager} springs up to turn source
 | 
						|
distributions into built distributions for as many platforms as there
 | 
						|
are packagers.
 | 
						|
 | 
						|
Of course, the module developer could be his own packager; or the
 | 
						|
packager could be a volunteer ``out there'' somewhere who has access to
 | 
						|
a platform which the original developer does not; or it could be
 | 
						|
software periodically grabbing new source distributions and turning them
 | 
						|
into built distributions for as many platforms as the software has
 | 
						|
access to.  Regardless of the nature of the beast, a packager uses the
 | 
						|
setup script and the \command{bdist} command family to generate built
 | 
						|
distributions.
 | 
						|
 | 
						|
As a simple example, if I run the following command in the Distutils
 | 
						|
source tree:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
then the Distutils builds my module distribution (the Distutils itself
 | 
						|
in this case), does a ``fake'' installation (also in the \file{build}
 | 
						|
directory), and creates the default type of built distribution for my
 | 
						|
platform.  The default format for built distributions is a ``dumb'' tar
 | 
						|
file on \UNIX, and an simple executable installer on Windows.  (That tar
 | 
						|
file is considered ``dumb'' because it has to be unpacked in a specific
 | 
						|
location to work.)
 | 
						|
 | 
						|
Thus, the above command on a \UNIX{} system creates
 | 
						|
\file{Distutils-0.9.1.\filevar{plat}.tar.gz}; unpacking this tarball
 | 
						|
from the right place installs the Distutils just as though you had
 | 
						|
downloaded the source distribution and run \code{python setup.py
 | 
						|
  install}.  (The ``right place'' is either the root of the filesystem or 
 | 
						|
Python's \filevar{prefix} directory, depending on the options given to
 | 
						|
the \command{bdist\_dumb} command; the default is to make dumb
 | 
						|
distributions relative to \filevar{prefix}.)  
 | 
						|
 | 
						|
Obviously, for pure Python distributions, this isn't a huge win---but
 | 
						|
for non-pure distributions, which include extensions that would need to
 | 
						|
be compiled, it can mean the difference between someone being able to
 | 
						|
use your extensions or not.  And creating ``smart'' built distributions,
 | 
						|
such as an RPM package or an executable installer for Windows, is a big
 | 
						|
win for users even if your distribution doesn't include any extensions.
 | 
						|
 | 
						|
The \command{bdist} command has a \longprogramopt{formats} option,
 | 
						|
similar to the \command{sdist} command, which you can use to select the
 | 
						|
types of built distribution to generate: for example,
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist --format=zip
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
would, when run on a \UNIX{} system, create
 | 
						|
\file{Distutils-0.8.\filevar{plat}.zip}---again, this archive would be
 | 
						|
unpacked from the root directory to install the Distutils.
 | 
						|
 | 
						|
The available formats for built distributions are:
 | 
						|
\begin{tableiii}{l|l|c}{code}%
 | 
						|
  {Format}{Description}{Notes}
 | 
						|
  \lineiii{gztar}{gzipped tar file (\file{.tar.gz})}{(1),(3)}
 | 
						|
  \lineiii{ztar}{compressed tar file (\file{.tar.Z})}{(3)}
 | 
						|
  \lineiii{tar}{tar file (\file{.tar})}{(3)}
 | 
						|
  \lineiii{zip}{zip file (\file{.zip})}{(4)}
 | 
						|
  \lineiii{rpm}{RPM}{(5)}
 | 
						|
  \lineiii{srpm}{source RPM}{(5) \XXX{to do!}}
 | 
						|
  \lineiii{wininst}{self-extracting ZIP file for Windows}{(2),(4)}
 | 
						|
\end{tableiii}
 | 
						|
 | 
						|
\noindent Notes:
 | 
						|
\begin{description}
 | 
						|
\item[(1)] default on \UNIX
 | 
						|
\item[(2)] default on Windows \XXX{to-do!}
 | 
						|
\item[(3)] requires external utilities: \program{tar} and possibly one
 | 
						|
  of \program{gzip}, \program{bzip2}, or \program{compress}
 | 
						|
\item[(4)] requires either external \program{zip} utility or
 | 
						|
  \module{zipfile} module (not part of the standard Python library)
 | 
						|
\item[(5)] requires external \program{rpm} utility, version 3.0.4 or
 | 
						|
  better (use \code{rpm --version} to find out which version you have)
 | 
						|
\end{description}
 | 
						|
 | 
						|
You don't have to use the \command{bdist} command with the
 | 
						|
\longprogramopt{formats} option; you can also use the command that
 | 
						|
directly implements the format you're interested in.  Some of these
 | 
						|
\command{bdist} ``sub-commands'' actually generate several similar
 | 
						|
formats; for instance, the \command{bdist\_dumb} command generates all
 | 
						|
the ``dumb'' archive formats (\code{tar}, \code{ztar}, \code{gztar}, and
 | 
						|
\code{zip}), and \command{bdist\_rpm} generates both binary and source
 | 
						|
RPMs.  The \command{bdist} sub-commands, and the formats generated by
 | 
						|
each, are:
 | 
						|
\begin{tableii}{l|l}{command}%
 | 
						|
  {Command}{Formats}
 | 
						|
  \lineii{bdist\_dumb}{tar, ztar, gztar, zip}
 | 
						|
  \lineii{bdist\_rpm}{rpm, srpm}
 | 
						|
  \lineii{bdist\_wininst}{wininst}
 | 
						|
\end{tableii}
 | 
						|
 | 
						|
The following sections give details on the individual \command{bdist\_*}
 | 
						|
commands.
 | 
						|
 | 
						|
 | 
						|
\subsection{Creating dumb built distributions}
 | 
						|
\label{creating-dumb}
 | 
						|
 | 
						|
\XXX{Need to document absolute vs. prefix-relative packages here, but
 | 
						|
  first I have to implement it!}
 | 
						|
 | 
						|
 | 
						|
\subsection{Creating RPM packages}
 | 
						|
\label{creating-rpms}
 | 
						|
 | 
						|
The RPM format is used by many of popular Linux distributions, including
 | 
						|
Red Hat, SuSE, and Mandrake.  If one of these (or any of the other
 | 
						|
RPM-based Linux distributions) is your usual environment, creating RPM
 | 
						|
packages for other users of that same distribution is trivial.
 | 
						|
Depending on the complexity of your module distribution and differences
 | 
						|
between Linux distributions, you may also be able to create RPMs that
 | 
						|
work on different RPM-based distributions.
 | 
						|
 | 
						|
The usual way to create an RPM of your module distribution is to run the 
 | 
						|
\command{bdist\_rpm} command:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist_rpm
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
or the \command{bdist} command with the \longprogramopt{format} option:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist --formats=rpm
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
The former allows you to specify RPM-specific options; the latter allows 
 | 
						|
you to easily specify multiple formats in one run.  If you need to do
 | 
						|
both, you can explicitly specify multiple \command{bdist\_*} commands
 | 
						|
and their options:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist_rpm --packager="John Doe <jdoe@python.net>" \
 | 
						|
                bdist_wininst --target_version="2.0"
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
Creating RPM packages is driven by a \file{.spec} file, much as using
 | 
						|
the Distutils is driven by the setup script.  To make your life easier,
 | 
						|
the \command{bdist\_rpm} command normally creates a \file{.spec} file
 | 
						|
based on the information you supply in the setup script, on the command
 | 
						|
line, and in any Distutils configuration files.  Various options and
 | 
						|
sections in the \file{.spec} file are derived from options in the setup
 | 
						|
script as follows:
 | 
						|
\begin{tableii}{l|l}{textrm}%
 | 
						|
  {RPM \file{.spec} file option or section}{Distutils setup script option}
 | 
						|
  \lineii{Name}{\option{name}}
 | 
						|
  \lineii{Summary (in preamble)}{\option{description}}
 | 
						|
  \lineii{Version}{\option{version}}
 | 
						|
  \lineii{Vendor}{\option{author} and \option{author\_email}, or \\&
 | 
						|
                  \option{maintainer} and \option{maintainer\_email}}
 | 
						|
  \lineii{Copyright}{\option{licence}}
 | 
						|
  \lineii{Url}{\option{url}}
 | 
						|
  \lineii{\%description (section)}{\option{long\_description}}
 | 
						|
\end{tableii}
 | 
						|
 | 
						|
Additionally, there many options in \file{.spec} files that don't have
 | 
						|
corresponding options in the setup script.  Most of these are handled
 | 
						|
through options to the \command{bdist\_rpm} command as follows:
 | 
						|
\begin{tableiii}{l|l|l}{textrm}%
 | 
						|
  {RPM \file{.spec} file option or section}%
 | 
						|
  {\command{bdist\_rpm} option}%
 | 
						|
  {default value}
 | 
						|
  \lineiii{Release}{\option{release}}{``1''}
 | 
						|
  \lineiii{Group}{\option{group}}{``Development/Libraries''}
 | 
						|
  \lineiii{Vendor}{\option{vendor}}{(see above)}
 | 
						|
  \lineiii{Packager}{\option{packager}}{(none)}
 | 
						|
  \lineiii{Provides}{\option{provides}}{(none)}
 | 
						|
  \lineiii{Requires}{\option{requires}}{(none)}
 | 
						|
  \lineiii{Conflicts}{\option{conflicts}}{(none)}
 | 
						|
  \lineiii{Obsoletes}{\option{obsoletes}}{(none)}
 | 
						|
  \lineiii{Distribution}{\option{distribution\_name}}{(none)}
 | 
						|
  \lineiii{BuildRequires}{\option{build\_requires}}{(none)}
 | 
						|
  \lineiii{Icon}{\option{icon}}{(none)}
 | 
						|
\end{tableiii}
 | 
						|
Obviously, supplying even a few of these options on the command-line
 | 
						|
would be tedious and error-prone, so it's usually best to put them in
 | 
						|
the setup configuration file, \file{setup.cfg}---see
 | 
						|
section~\ref{setup-config}.  If you distribute or package many Python
 | 
						|
module distributions, you might want to put options that apply to all of
 | 
						|
them in your personal Distutils configuration file
 | 
						|
(\file{\textasciitilde/.pydistutils.cfg}).
 | 
						|
 | 
						|
There are three steps to building a binary RPM package, all of which are 
 | 
						|
handled automatically by the Distutils:
 | 
						|
\begin{enumerate}
 | 
						|
\item create a \file{.spec} file, which describes the package (analogous 
 | 
						|
  to the Distutils setup script; in fact, much of the information in the 
 | 
						|
  setup script winds up in the \file{.spec} file)
 | 
						|
\item create the source RPM
 | 
						|
\item create the ``binary'' RPM (which may or may not contain binary
 | 
						|
  code, depending on whether your module distribution contains Python
 | 
						|
  extensions)
 | 
						|
\end{enumerate}
 | 
						|
Normally, RPM bundles the last two steps together; when you use the
 | 
						|
Distutils, all three steps are typically bundled together.
 | 
						|
 | 
						|
If you wish, you can separate these three steps.  You can use the
 | 
						|
\longprogramopt{spec-only} option to make \command{bdist\_rpm} just
 | 
						|
create the \file{.spec} file and exit; in this case, the \file{.spec}
 | 
						|
file will be written to the ``distribution directory''---normally
 | 
						|
\file{dist/}, but customizable with the \longprogramopt{dist-dir}
 | 
						|
option.  (Normally, the \file{.spec} file winds up deep in the ``build
 | 
						|
tree,'' in a temporary directory created by \command{bdist\_rpm}.)
 | 
						|
 | 
						|
\XXX{this isn't implemented yet---is it needed?!}
 | 
						|
You can also specify a custom \file{.spec} file with the
 | 
						|
\longprogramopt{spec-file} option; used in conjunction with
 | 
						|
\longprogramopt{spec-only}, this gives you an opportunity to customize
 | 
						|
the \file{.spec} file manually:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
> python setup.py bdist_rpm --spec-only
 | 
						|
# ...edit dist/FooBar-1.0.spec
 | 
						|
> python setup.py bdist_rpm --spec-file=dist/FooBar-1.0.spec
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
(Although a better way to do this is probably to override the standard
 | 
						|
\command{bdist\_rpm} command with one that writes whatever else you want
 | 
						|
to the \file{.spec} file; see section~\ref{extending} for information on
 | 
						|
extending the Distutils.)
 | 
						|
 | 
						|
 | 
						|
\subsection{Creating Windows installers}
 | 
						|
\label{creating-wininst}
 | 
						|
 | 
						|
Executable Windows installers are the natural format for binary
 | 
						|
distributions on Windows.  They display a nice graphical user interface,
 | 
						|
display some information of the module distribution to be installed, taken
 | 
						|
from the meta-data in the setup script, let the user select a few
 | 
						|
(currently maybe too few) options, and start or cancel the installation.
 | 
						|
 | 
						|
Since the meta-data is taken from the setup script, creating
 | 
						|
Windows installers is usually as easy as running:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist_wininst
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
or the \command{bdist} command with the \longprogramopt{format} option:
 | 
						|
 | 
						|
\begin{verbatim}
 | 
						|
python setup.py bdist --formats=wininst
 | 
						|
\end{verbatim}
 | 
						|
 | 
						|
If you have a pure module distribution (only containing pure
 | 
						|
Python modules and packages), the resulting installer will be
 | 
						|
version independent and have a name like \file{Foo-1.0.win32.exe}.
 | 
						|
These installers can even be created on \UNIX{} or MacOS platforms.
 | 
						|
 | 
						|
If you have a non-pure distribution, the extensions can only be
 | 
						|
created on a Windows platform, and will be Python version dependend.
 | 
						|
The installer filename will reflect this and now has the form
 | 
						|
\file{Foo-1.0.win32-py2.0.exe}. You have to create a separate installer
 | 
						|
for every Python version you want to support.
 | 
						|
 | 
						|
The installer will try to compile pure modules into bytecode after
 | 
						|
installation on the target system in normal and optimizing mode.
 | 
						|
If you don't want this to happen for some reason, you can run
 | 
						|
the bdist_wininst command with the \longprogramopt{no-target-compile} and/or
 | 
						|
the \longprogramopt{no-target-optimize} option.
 | 
						|
 | 
						|
%\section{Examples}
 | 
						|
%\label{examples}
 | 
						|
 | 
						|
 | 
						|
%\subsection{Pure Python distribution (by module)}
 | 
						|
%\label{pure-mod}
 | 
						|
 | 
						|
 | 
						|
%\subsection{Pure Python distribution (by package)}
 | 
						|
%\label{pure-pkg}
 | 
						|
 | 
						|
 | 
						|
%\subsection{Single extension module}
 | 
						|
%\label{single-ext}
 | 
						|
 | 
						|
 | 
						|
%\subsection{Multiple extension modules}
 | 
						|
%\label{multiple-ext}
 | 
						|
 | 
						|
 | 
						|
%\subsection{Putting it all together}
 | 
						|
 | 
						|
 | 
						|
 | 
						|
%\section{Extending the Distutils}
 | 
						|
%\label{extending}
 | 
						|
 | 
						|
 | 
						|
%\subsection{Extending existing commands}
 | 
						|
%\label{extend-existing}
 | 
						|
 | 
						|
 | 
						|
%\subsection{Writing new commands}
 | 
						|
%\label{new-commands}
 | 
						|
 | 
						|
%\XXX{Would an uninstall command be a good example here?}
 | 
						|
 | 
						|
 | 
						|
 | 
						|
\section{Reference}
 | 
						|
\label{reference}
 | 
						|
 | 
						|
 | 
						|
%\subsection{Building modules: the \protect\command{build} command family}
 | 
						|
%\label{build-cmds}
 | 
						|
 | 
						|
%\subsubsection{\protect\command{build}}
 | 
						|
%\label{build-cmd}
 | 
						|
 | 
						|
%\subsubsection{\protect\command{build\_py}}
 | 
						|
%\label{build-py-cmd}
 | 
						|
 | 
						|
%\subsubsection{\protect\command{build\_ext}}
 | 
						|
%\label{build-ext-cmd}
 | 
						|
 | 
						|
%\subsubsection{\protect\command{build\_clib}}
 | 
						|
%\label{build-clib-cmd}
 | 
						|
 | 
						|
 | 
						|
\subsection{Installing modules: the \protect\command{install} command family}
 | 
						|
\label{install-cmd}
 | 
						|
 | 
						|
The install command ensures that the build commands have been run and then
 | 
						|
runs the subcommands \command{install\_lib},
 | 
						|
\command{install\_data} and
 | 
						|
\command{install\_scripts}.
 | 
						|
 | 
						|
%\subsubsection{\protect\command{install\_lib}}
 | 
						|
%\label{install-lib-cmd}
 | 
						|
 | 
						|
\subsubsection{\protect\command{install\_data}}
 | 
						|
\label{install-data-cmd}
 | 
						|
This command installs all data files provided with the distribution.
 | 
						|
 | 
						|
\subsubsection{\protect\command{install\_scripts}}
 | 
						|
\label{install-scripts-cmd}
 | 
						|
This command installs all (Python) scripts in the distribution.
 | 
						|
 | 
						|
 | 
						|
%\subsection{Cleaning up: the \protect\command{clean} command}
 | 
						|
%\label{clean-cmd}
 | 
						|
 | 
						|
 | 
						|
\subsection{Creating a source distribution: the
 | 
						|
            \protect\command{sdist} command}
 | 
						|
\label{sdist-cmd}
 | 
						|
 | 
						|
 | 
						|
\XXX{fragment moved down from above: needs context!}
 | 
						|
 | 
						|
The manifest template commands are:
 | 
						|
\begin{tableii}{ll}{command}{Command}{Description}
 | 
						|
  \lineii{include \var{pat1} \var{pat2} ... }
 | 
						|
    {include all files matching any of the listed patterns}
 | 
						|
  \lineii{exclude \var{pat1} \var{pat2} ... }
 | 
						|
    {exclude all files matching any of the listed patterns}
 | 
						|
  \lineii{recursive-include \var{dir} \var{pat1} \var{pat2} ... }
 | 
						|
    {include all files under \var{dir} matching any of the listed patterns}
 | 
						|
  \lineii{recursive-exclude \var{dir} \var{pat1} \var{pat2} ...}
 | 
						|
    {exclude all files under \var{dir} matching any of the listed patterns}
 | 
						|
  \lineii{global-include \var{pat1} \var{pat2} ...}
 | 
						|
    {include all files anywhere in the source tree matching\\&
 | 
						|
     any of the listed patterns}
 | 
						|
  \lineii{global-exclude \var{pat1} \var{pat2} ...}
 | 
						|
    {exclude all files anywhere in the source tree matching\\&
 | 
						|
     any of the listed patterns}
 | 
						|
  \lineii{prune \var{dir}}{exclude all files under \var{dir}}
 | 
						|
  \lineii{graft \var{dir}}{include all files under \var{dir}}
 | 
						|
\end{tableii}
 | 
						|
The patterns here are \UNIX-style ``glob'' patterns: \code{*} matches any
 | 
						|
sequence of regular filename characters, \code{?} matches any single
 | 
						|
regular filename character, and \code{[\var{range}]} matches any of the
 | 
						|
characters in \var{range} (e.g., \code{a-z}, \code{a-zA-Z},
 | 
						|
\code{a-f0-9\_.}).  The definition of ``regular filename character'' is
 | 
						|
platform-specific: on \UNIX{} it is anything except slash; on Windows
 | 
						|
anything except backslash or colon; on MacOS anything except colon.
 | 
						|
 | 
						|
\XXX{Windows and MacOS support not there yet}
 | 
						|
 | 
						|
 | 
						|
%\subsection{Creating a built distribution: the
 | 
						|
%  \protect\command{bdist} command family}
 | 
						|
%\label{bdist-cmds}
 | 
						|
 | 
						|
 | 
						|
%\subsubsection{\protect\command{bdist}}
 | 
						|
 | 
						|
%\subsubsection{\protect\command{bdist\_dumb}}
 | 
						|
 | 
						|
%\subsubsection{\protect\command{bdist\_rpm}}
 | 
						|
 | 
						|
%\subsubsection{\protect\command{bdist\_wininst}}
 | 
						|
 | 
						|
 | 
						|
\input{sysconfig}
 | 
						|
 | 
						|
 | 
						|
\end{document}
 |