Patch #1810 by Thomas Lee, reviewed by myself:

allow compiling Python AST objects into code objects
in compile().
This commit is contained in:
Georg Brandl 2008-03-28 12:11:56 +00:00
parent b9803421d2
commit fc8eef3c78
11 changed files with 3256 additions and 62 deletions

View file

@ -12,16 +12,16 @@ Abstract Syntax Trees
.. versionadded:: 2.5
The ``_ast`` module helps Python applications to process trees of the Python
abstract syntax grammar. The Python compiler currently provides read-only access
to such trees, meaning that applications can only create a tree for a given
piece of Python source code; generating :term:`bytecode` from a (potentially modified)
tree is not supported. The abstract syntax itself might change with each Python
release; this module helps to find out programmatically what the current grammar
looks like.
abstract syntax grammar. The abstract syntax itself might change with each
Python release; this module helps to find out programmatically what the current
grammar looks like.
An abstract syntax tree can be generated by passing ``_ast.PyCF_ONLY_AST`` as a
flag to the :func:`compile` builtin function. The result will be a tree of
objects whose classes all inherit from ``_ast.AST``.
An abstract syntax tree can be generated by passing :data:`_ast.PyCF_ONLY_AST`
as a flag to the :func:`compile` builtin function. The result will be a tree of
objects whose classes all inherit from :class:`_ast.AST`.
A modified abstract syntax tree can be compiled into a Python code object using
the built-in :func:`compile` function.
The actual classes are derived from the ``Parser/Python.asdl`` file, which is
reproduced below. There is one class defined for each left-hand side symbol in
@ -41,12 +41,15 @@ attribute ``left`` of type ``_ast.expr``. Instances of ``_ast.expr`` and
``_ast.stmt`` subclasses also have lineno and col_offset attributes. The lineno
is the line number of source text (1 indexed so the first line is line 1) and
the col_offset is the utf8 byte offset of the first token that generated the
node. The utf8 offset is recorded because the parser uses utf8 internally.
node. The utf8 offset is recorded because the parser uses utf8 internally.
If these attributes are marked as optional in the grammar (using a question
mark), the value might be ``None``. If the attributes can have zero-or-more
values (marked with an asterisk), the values are represented as Python lists.
The constructors of all ``_ast`` classes don't take arguments; instead, if you
create instances, you must assign the required attributes separately.
Abstract Grammar
----------------

View file

@ -28,12 +28,6 @@ code.
This chapter explains how the various components of the :mod:`compiler` package
work. It blends reference material with a tutorial.
The following modules are part of the :mod:`compiler` package:
.. toctree::
_ast.rst
The basic interface
===================

View file

@ -190,21 +190,27 @@ available. They are listed here in alphabetical order.
.. function:: compile(source, filename, mode[, flags[, dont_inherit]])
Compile the *source* into a code object. Code objects can be executed by an
:keyword:`exec` statement or evaluated by a call to :func:`eval`. The
*filename* argument should give the file from which the code was read; pass some
recognizable value if it wasn't read from a file (``'<string>'`` is commonly
used). The *mode* argument specifies what kind of code must be compiled; it can
be ``'exec'`` if *source* consists of a sequence of statements, ``'eval'`` if it
consists of a single expression, or ``'single'`` if it consists of a single
interactive statement (in the latter case, expression statements that evaluate
to something else than ``None`` will be printed).
Compile the *source* into a code or AST object. Code objects can be executed
by an :keyword:`exec` statement or evaluated by a call to :func:`eval`.
*source* can either be a string or an AST object. Refer to the :mod:`_ast`
module documentation for information on how to compile into and from AST
objects.
When compiling multi-line statements, two caveats apply: line endings must be
represented by a single newline character (``'\n'``), and the input must be
terminated by at least one newline character. If line endings are represented
by ``'\r\n'``, use the string :meth:`replace` method to change them into
``'\n'``.
When compiling a string with multi-line statements, two caveats apply: line
endings must be represented by a single newline character (``'\n'``), and the
input must be terminated by at least one newline character. If line endings
are represented by ``'\r\n'``, use the string :meth:`replace` method to
change them into ``'\n'``.
The *filename* argument should give the file from which the code was read;
pass some recognizable value if it wasn't read from a file (``'<string>'`` is
commonly used).
The *mode* argument specifies what kind of code must be compiled; it can be
``'exec'`` if *source* consists of a sequence of statements, ``'eval'`` if it
consists of a single expression, or ``'single'`` if it consists of a single
interactive statement (in the latter case, expression statements that
evaluate to something else than ``None`` will be printed).
The optional arguments *flags* and *dont_inherit* (which are new in Python 2.2)
control which future statements (see :pep:`236`) affect the compilation of
@ -224,6 +230,9 @@ available. They are listed here in alphabetical order.
This function raises :exc:`SyntaxError` if the compiled source is invalid,
and :exc:`TypeError` if the source contains null bytes.
.. versionadded:: 2.6
Support for compiling AST objects.
.. function:: complex([real[, imag]])

View file

@ -15,6 +15,7 @@ These modules include:
.. toctree::
parser.rst
_ast.rst
symbol.rst
token.rst
keyword.rst