mirror of
https://github.com/python/cpython.git
synced 2025-08-28 12:45:07 +00:00
b193fa996a
In some development setups it is inconvenient or impossible to write bytecode caches to the code tree, but the bytecode caches are still useful. The PYTHONPYCACHEPREFIX environment variable allows specifying an alternate location for cached bytecode files, within which a directory tree mirroring the code tree will be created. This cache tree is then used (for both reading and writing) instead of the local `__pycache__` subdirectory within each source directory. Exposed at runtime as sys.pycache_prefix (defaulting to None), and can be set from the CLI as "-X pycache_prefix=path". Patch by Carl Meyer.
269 lines
10 KiB
ReStructuredText
269 lines
10 KiB
ReStructuredText
:mod:`compileall` --- Byte-compile Python libraries
|
|
===================================================
|
|
|
|
.. module:: compileall
|
|
:synopsis: Tools for byte-compiling all Python source files in a directory tree.
|
|
|
|
**Source code:** :source:`Lib/compileall.py`
|
|
|
|
--------------
|
|
|
|
This module provides some utility functions to support installing Python
|
|
libraries. These functions compile Python source files in a directory tree.
|
|
This module can be used to create the cached byte-code files at library
|
|
installation time, which makes them available for use even by users who don't
|
|
have write permission to the library directories.
|
|
|
|
|
|
Command-line use
|
|
----------------
|
|
|
|
This module can work as a script (using :program:`python -m compileall`) to
|
|
compile Python sources.
|
|
|
|
.. program:: compileall
|
|
|
|
.. cmdoption:: directory ...
|
|
file ...
|
|
|
|
Positional arguments are files to compile or directories that contain
|
|
source files, traversed recursively. If no argument is given, behave as if
|
|
the command line was ``-l <directories from sys.path>``.
|
|
|
|
.. cmdoption:: -l
|
|
|
|
Do not recurse into subdirectories, only compile source code files directly
|
|
contained in the named or implied directories.
|
|
|
|
.. cmdoption:: -f
|
|
|
|
Force rebuild even if timestamps are up-to-date.
|
|
|
|
.. cmdoption:: -q
|
|
|
|
Do not print the list of files compiled. If passed once, error messages will
|
|
still be printed. If passed twice (``-qq``), all output is suppressed.
|
|
|
|
.. cmdoption:: -d destdir
|
|
|
|
Directory prepended to the path to each file being compiled. This will
|
|
appear in compilation time tracebacks, and is also compiled in to the
|
|
byte-code file, where it will be used in tracebacks and other messages in
|
|
cases where the source file does not exist at the time the byte-code file is
|
|
executed.
|
|
|
|
.. cmdoption:: -x regex
|
|
|
|
regex is used to search the full path to each file considered for
|
|
compilation, and if the regex produces a match, the file is skipped.
|
|
|
|
.. cmdoption:: -i list
|
|
|
|
Read the file ``list`` and add each line that it contains to the list of
|
|
files and directories to compile. If ``list`` is ``-``, read lines from
|
|
``stdin``.
|
|
|
|
.. cmdoption:: -b
|
|
|
|
Write the byte-code files to their legacy locations and names, which may
|
|
overwrite byte-code files created by another version of Python. The default
|
|
is to write files to their :pep:`3147` locations and names, which allows
|
|
byte-code files from multiple versions of Python to coexist.
|
|
|
|
.. cmdoption:: -r
|
|
|
|
Control the maximum recursion level for subdirectories.
|
|
If this is given, then ``-l`` option will not be taken into account.
|
|
:program:`python -m compileall <directory> -r 0` is equivalent to
|
|
:program:`python -m compileall <directory> -l`.
|
|
|
|
.. cmdoption:: -j N
|
|
|
|
Use *N* workers to compile the files within the given directory.
|
|
If ``0`` is used, then the result of :func:`os.cpu_count()`
|
|
will be used.
|
|
|
|
.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash]
|
|
|
|
Control how the generated pycs will be invalidated at runtime. The default
|
|
setting, ``timestamp``, means that ``.pyc`` files with the source timestamp
|
|
and size embedded will be generated. The ``checked-hash`` and
|
|
``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based
|
|
pycs embed a hash of the source file contents rather than a timestamp. See
|
|
:ref:`pyc-invalidation` for more information on how Python validates bytecode
|
|
cache files at runtime.
|
|
|
|
.. versionchanged:: 3.2
|
|
Added the ``-i``, ``-b`` and ``-h`` options.
|
|
|
|
.. versionchanged:: 3.5
|
|
Added the ``-j``, ``-r``, and ``-qq`` options. ``-q`` option
|
|
was changed to a multilevel value. ``-b`` will always produce a
|
|
byte-code file ending in ``.pyc``, never ``.pyo``.
|
|
|
|
.. versionchanged:: 3.7
|
|
Added the ``--invalidation-mode`` parameter.
|
|
|
|
|
|
There is no command-line option to control the optimization level used by the
|
|
:func:`compile` function, because the Python interpreter itself already
|
|
provides the option: :program:`python -O -m compileall`.
|
|
|
|
Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix`
|
|
setting. The generated bytecode cache will only be useful if :func:`compile` is
|
|
run with the same :attr:`sys.pycache_prefix` (if any) that will be used at
|
|
runtime.
|
|
|
|
Public functions
|
|
----------------
|
|
|
|
.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP)
|
|
|
|
Recursively descend the directory tree named by *dir*, compiling all :file:`.py`
|
|
files along the way. Return a true value if all the files compiled successfully,
|
|
and a false value otherwise.
|
|
|
|
The *maxlevels* parameter is used to limit the depth of the recursion; it
|
|
defaults to ``10``.
|
|
|
|
If *ddir* is given, it is prepended to the path to each file being compiled
|
|
for use in compilation time tracebacks, and is also compiled in to the
|
|
byte-code file, where it will be used in tracebacks and other messages in
|
|
cases where the source file does not exist at the time the byte-code file is
|
|
executed.
|
|
|
|
If *force* is true, modules are re-compiled even if the timestamps are up to
|
|
date.
|
|
|
|
If *rx* is given, its search method is called on the complete path to each
|
|
file considered for compilation, and if it returns a true value, the file
|
|
is skipped.
|
|
|
|
If *quiet* is ``False`` or ``0`` (the default), the filenames and other
|
|
information are printed to standard out. Set to ``1``, only errors are
|
|
printed. Set to ``2``, all output is suppressed.
|
|
|
|
If *legacy* is true, byte-code files are written to their legacy locations
|
|
and names, which may overwrite byte-code files created by another version of
|
|
Python. The default is to write files to their :pep:`3147` locations and
|
|
names, which allows byte-code files from multiple versions of Python to
|
|
coexist.
|
|
|
|
*optimize* specifies the optimization level for the compiler. It is passed to
|
|
the built-in :func:`compile` function.
|
|
|
|
The argument *workers* specifies how many workers are used to
|
|
compile files in parallel. The default is to not use multiple workers.
|
|
If the platform can't use multiple workers and *workers* argument is given,
|
|
then sequential compilation will be used as a fallback. If *workers* is
|
|
lower than ``0``, a :exc:`ValueError` will be raised.
|
|
|
|
*invalidation_mode* should be a member of the
|
|
:class:`py_compile.PycInvalidationMode` enum and controls how the generated
|
|
pycs are invalidated at runtime.
|
|
|
|
.. versionchanged:: 3.2
|
|
Added the *legacy* and *optimize* parameter.
|
|
|
|
.. versionchanged:: 3.5
|
|
Added the *workers* parameter.
|
|
|
|
.. versionchanged:: 3.5
|
|
*quiet* parameter was changed to a multilevel value.
|
|
|
|
.. versionchanged:: 3.5
|
|
The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
|
|
no matter what the value of *optimize* is.
|
|
|
|
.. versionchanged:: 3.6
|
|
Accepts a :term:`path-like object`.
|
|
|
|
.. versionchanged:: 3.7
|
|
The *invalidation_mode* parameter was added.
|
|
|
|
.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP)
|
|
|
|
Compile the file with path *fullname*. Return a true value if the file
|
|
compiled successfully, and a false value otherwise.
|
|
|
|
If *ddir* is given, it is prepended to the path to the file being compiled
|
|
for use in compilation time tracebacks, and is also compiled in to the
|
|
byte-code file, where it will be used in tracebacks and other messages in
|
|
cases where the source file does not exist at the time the byte-code file is
|
|
executed.
|
|
|
|
If *rx* is given, its search method is passed the full path name to the
|
|
file being compiled, and if it returns a true value, the file is not
|
|
compiled and ``True`` is returned.
|
|
|
|
If *quiet* is ``False`` or ``0`` (the default), the filenames and other
|
|
information are printed to standard out. Set to ``1``, only errors are
|
|
printed. Set to ``2``, all output is suppressed.
|
|
|
|
If *legacy* is true, byte-code files are written to their legacy locations
|
|
and names, which may overwrite byte-code files created by another version of
|
|
Python. The default is to write files to their :pep:`3147` locations and
|
|
names, which allows byte-code files from multiple versions of Python to
|
|
coexist.
|
|
|
|
*optimize* specifies the optimization level for the compiler. It is passed to
|
|
the built-in :func:`compile` function.
|
|
|
|
*invalidation_mode* should be a member of the
|
|
:class:`py_compile.PycInvalidationMode` enum and controls how the generated
|
|
pycs are invalidated at runtime.
|
|
|
|
.. versionadded:: 3.2
|
|
|
|
.. versionchanged:: 3.5
|
|
*quiet* parameter was changed to a multilevel value.
|
|
|
|
.. versionchanged:: 3.5
|
|
The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
|
|
no matter what the value of *optimize* is.
|
|
|
|
.. versionchanged:: 3.7
|
|
The *invalidation_mode* parameter was added.
|
|
|
|
.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP)
|
|
|
|
Byte-compile all the :file:`.py` files found along ``sys.path``. Return a
|
|
true value if all the files compiled successfully, and a false value otherwise.
|
|
|
|
If *skip_curdir* is true (the default), the current directory is not included
|
|
in the search. All other parameters are passed to the :func:`compile_dir`
|
|
function. Note that unlike the other compile functions, ``maxlevels``
|
|
defaults to ``0``.
|
|
|
|
.. versionchanged:: 3.2
|
|
Added the *legacy* and *optimize* parameter.
|
|
|
|
.. versionchanged:: 3.5
|
|
*quiet* parameter was changed to a multilevel value.
|
|
|
|
.. versionchanged:: 3.5
|
|
The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
|
|
no matter what the value of *optimize* is.
|
|
|
|
.. versionchanged:: 3.7
|
|
The *invalidation_mode* parameter was added.
|
|
|
|
To force a recompile of all the :file:`.py` files in the :file:`Lib/`
|
|
subdirectory and all its subdirectories::
|
|
|
|
import compileall
|
|
|
|
compileall.compile_dir('Lib/', force=True)
|
|
|
|
# Perform same compilation, excluding files in .svn directories.
|
|
import re
|
|
compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True)
|
|
|
|
# pathlib.Path objects can also be used.
|
|
import pathlib
|
|
compileall.compile_dir(pathlib.Path('Lib/'), force=True)
|
|
|
|
.. seealso::
|
|
|
|
Module :mod:`py_compile`
|
|
Byte-compile a single source file.
|