Add more documentation around attribute visitors.

Adds a bit more documentation to various pieces of attribute visitors.
Adds a section around traversal order of visitors given a tree.
This commit is contained in:
Jennifer Taylor 2019-08-29 12:13:15 -07:00 committed by Jennifer Taylor
parent ce47f864a2
commit 0d2b66a1e7
3 changed files with 103 additions and 16 deletions

View file

@ -9,16 +9,20 @@ Visit and Leave Helper Functions
--------------------------------
While it is possible to subclass from :class:`~libcst.CSTVisitor` or :class:`~libcst.CSTTransformer`
and override the ``on_visit``/``on_leave`` functions directly, it is not recommended. The default
implementation for both visitors will look up a ``visit_<Type[CSTNode]>`` and ``leave_<Type[CSTNode]>``
function on the visitor subclass and call it directly. If such a function exists for the node in
question, the visitor base class will call the relevant function, respecting the above outlined
semantics. If the function does not exist, the visitor base class will assume that you do not care
about that node and visit its children for you without requiring a default implementation.
and override the ``on_visit``/``on_leave``/``on_visit_attribute``/``on_leave_attribute`` functions
directly, it is not recommended. The default implementation for both visitors will look up any
``visit_<Type[CSTNode]>``, ``leave_<Type[CSTNode]>``, ``visit_<Type[CSTNode>]_<attribute>`` and
``leave__<Type[CSTNode>]_<attribute>`` method on the visitor subclass and call them directly.
If such a function exists for the node in question, the visitor base class will call the relevant
function, respecting the above outlined semantics. If the function does not exist, the visitor base
class will assume that you do not care about that node and visit its children for you without
requiring a default implementation.
As a convenience, you can return ``None`` instead of a boolean value from your ``visit_<Type[CSTNode]>``
functions. Returning a ``None`` value is treated as a request for default behavior, which causes the
visitor to traverse children. It is equivalent to returning ``True``, but requires no explicit return.
Much like ``on_visit``, ``visit_<Type[CSTNode]>`` return a boolean specifying whether or not LibCST
should visit a node's children. As a convenience, you can return ``None`` instead of a boolean value
from your ``visit_<Type[CSTNode]>`` functions. Returning a ``None`` value is treated as a request for
default behavior, which causes the visitor to traverse children. It is equivalent to returning
``True``, but requires no explicit return.
For example, the below visitor will visit every function definition, traversing to its children only
if the function name doesn't include the word "foo". Notice that we don't need to provide our own
@ -44,6 +48,84 @@ An example Python REPL using the above visitor is as follows::
'abc'
'123'
Traversal Order
---------------
Traversal of any parsed tree directly matches the order that tokens appear in the source which
was parsed. LibCST will first call ``on_visit`` for the node. Then, for each of the node's
child attributes, LibCST will call ``on_visit_attribute`` for the node's attribute, followed
by running the same visit algorithm on each child node in the node's attribute. Then,
``on_leave_attribute`` is called. After each attribute has been fully traversed, LibCST will
call ``on_leave`` for the node. Note that LibCST will only call ``on_visit_attribute`` and
``on_leave_attribute`` for attributes in which there might be a LibCST node as a child. It
will not call attribute visitors for attributes which are built-in python types.
For example, take the following simple tree generated by calling ``parse_expression("1+2")``.
.. code-block:: python
BinaryOperation(
left=Integer(
value='1',
lpar=[],
rpar=[],
),
operator=Add(
whitespace_before=SimpleWhitespace(
value='',
),
whitespace_after=SimpleWhitespace(
value='',
),
),
right=Integer(
value='2',
lpar=[],
rpar=[],
),
lpar=[],
rpar=[],
)
Assuming you have a visitor that overrides every convenience helper method available,
methods will be called in this order:
.. code-block:: python
visit_BinaryOperation
visit_BinaryOperation_lpar
leave_BinaryOperation_lpar
visit_BinaryOperation_left
visit_Integer
visit_Integer_lpar
leave_Integer_lpar
visit_Integer_rpar
leave_Integer_rpar
leave_Integer
leave_BinaryOperation_left
visit_BinaryOperation_operator
visit_Add
visit_Add_whitespace_before
visit_SimpleWhitespace
leave_SimpleWhitespace
leave_Add_whitespace_before
visit_Add_whitespace_after
visit_SimpleWhitespace
leave_SimpleWhitespace
leave_Add_whitespace_after
leave_Add
leave_BinaryOperation_operator
visit_BinaryOperation_right
visit_Integer
visit_Integer_lpar
leave_Integer_lpar
visit_Integer_rpar
leave_Integer_rpar
leave_Integer
leave_BinaryOperation_right
visit_BinaryOperation_rpar
leave_BinaryOperation_rpar
leave_BinaryOperation
Batched Visitors
----------------

View file

@ -20,7 +20,8 @@ class RemovalSentinel(Enum):
The parent node should make a best-effort to remove the child, but may raise an
exception when removing the child doesn't make sense, or could change the semantics
in an unexpected way. E.g. a function with no name doesn't make sense.
in an unexpected way. For example, a function definition with no name doesn't make
sense, but removing one of the arguments is valid.
In we can't automatically remove the child, the developer should instead remove the
child by constructing a new parent in the parent's :meth:`~CSTTransformer.on_leave`

View file

@ -74,8 +74,10 @@ class CSTTransformer(CSTTypedTransformerFunctions, MetadataDependent):
def on_visit_attribute(self, node: "CSTNode", attribute: str) -> None:
"""
Called before a node's attribute is visited and after we have called
:func:`~libcst.CSTTransformer.on_visit` on the node.
Called before a node's child attribute is visited and after we have called
:func:`~libcst.CSTTransformer.on_visit` on the node. A node's child
attributes are visited in the order that they appear in source that this
node originates from.
"""
visit_func = getattr(self, f"visit_{type(node).__name__}_{attribute}", None)
if visit_func is not None:
@ -83,7 +85,7 @@ class CSTTransformer(CSTTypedTransformerFunctions, MetadataDependent):
def on_leave_attribute(self, original_node: "CSTNode", attribute: str) -> None:
"""
Called after a node's attribute is visited and before we have called
Called after a node's child attribute is visited and before we have called
:func:`~libcst.CSTTransformer.on_leave` on the node.
Unlike :func:`~libcst.CSTTransformer.on_leave`, this function does
@ -136,8 +138,10 @@ class CSTVisitor(CSTTypedVisitorFunctions, MetadataDependent):
def on_visit_attribute(self, node: "CSTNode", attribute: str) -> None:
"""
Called before a node's attribute is visited and after we have called
:func:`~libcst.CSTVisitor.on_visit` on the node.
Called before a node's child attribute is visited and after we have called
:func:`~libcst.CSTTransformer.on_visit` on the node. A node's child
attributes are visited in the order that they appear in source that this
node originates from.
"""
visit_func = getattr(self, f"visit_{type(node).__name__}_{attribute}", None)
if visit_func is not None:
@ -145,7 +149,7 @@ class CSTVisitor(CSTTypedVisitorFunctions, MetadataDependent):
def on_leave_attribute(self, original_node: "CSTNode", attribute: str) -> None:
"""
Called after a node's attribute is visited and before we have called
Called after a node's child attribute is visited and before we have called
:func:`~libcst.CSTVisitor.on_leave` on the node.
"""
leave_func = getattr(