This was a pain, because apparently we can't refer to
"static_analysis.libcst" as a module before its finished executing, due
to https://bugs.python.org/issue25294. To combat this, I had to change
to direct importing inside parser, which was a bit of a pain due to the
number of nodes used. But, it works.
This implements the required nodes, but not the parsing logic (that'll
be a later diff).
To support `Dict`, this diff also introduces `BaseDictElement`,
`DictElement`, `StarredDictElement`, and `BaseDict`.
It also refactors some of the logic in `BaseSet` into `_BaseSetOrDict`
so that it can be shared between `BaseSet` and `BaseDict`.
`DictComp` will come later.
Go through and clean up where we export things from in the top level
directories. This is the first pass at cleaning up the public API. In
this diff I clean up all top-level exports that were coming out of .py
files and instead export them from libcst's base module itself.
I also didn't export a few of the things defined in newly underscored
files because they were only used internally.
Without this option, autodoc defaults to ordering all members of
modules and classes alphabetically.
While this is normally fine (and probably desirable), the ordering of
fields in dataclasses is significant, so we need to change this behavior
to preserve the original member ordering.
These aren't really matchers as much as they are the auto-generated
function stubs for visit/leave functions. So, rename them as such. This
isn't the best name either, but I think it gets us closer to what's
really the purpose.
Give node-based visitors the ability to state that they don't want to
visit their children. I use this in add_none_throws to skip entire
modules that I don't want to modify (non-tests) and to skip looking at
comments that are irrelevant. This should also make add_none_throws
faster since we skip visiting a lot of node types.
We already have several places in unit tests that need this, as well
as a spot in an existing codemod that could benefit from it. So,
implement ensure_type which can be used to refine the type of a node to
an exact node type. I purposefully left the node type as 'object'
instead of libcst.CSTNode so that it could be used with RemovalSentinel
or MaybeSentinel without needing to import these.
All classes that inherit from `_MetadataInterface` now have access to
any metadata declared in any of their base classes. Inherited metadata
is resolved by the metadata runner upon visiting a module.
This is a combination of:
- Fixing some issues upstream.
- Modifying our cleanup script to carry over noqa comments, and
copying over the new versions of some files.
- A small addition to our flake8 config.
Some of these crept in before we had type-checking on node
constructors or decorators. So, lets fix these cases and get rid of a
few incorrect constructions in tests!
Strings are safe to use with a word operator in many cases, so this
fixes it to support that.
This was causing some parser errors in
`commerce/shopping_recommendations_service_shots.py` due to some whacky
f-strings. We weren't encountering those errors before, because of other
f-string parsing issues that we were hitting first.
Currently, a lot of libcst node tests use data providers that use tuples
instead of dictionaries. This just makes the tests unreadable, so this
is one of several diffs aiming to solve this issue.
Currently, a lot of libcst node tests use data providers that use tuples
instead of dictionaries. This just makes the tests unreadable, so this
is one of several diffs aiming to solve this issue.
Currently, a lot of libcst node tests use data providers that use tuples
instead of dictionaries. This just makes the tests unreadable, so this
is one of several diffs aiming to solve this issue.
Currently, a lot of libcst node tests use data providers that use tuples
instead of dictionaries. This just makes the tests unreadable, so this
is one of several diffs aiming to solve this issue.
Currently, a lot of libcst node tests use data providers that use tuples
instead of dictionaries. This just makes the tests unreadable, so this
is one of several diffs aiming to solve this issue.
Refactors various metadata related classes to better reflect intended
usage of the API.
The metadata runner now deep clones the module and returns a copy
containing the metadata information. Metadata providers also return the
tree to enforce the idea that the tree is immutable (even though no
copying is actually done and providers write directly to the original
tree).
Currently, a lot of libcst node tests use data providers that use tuples
instead of dictionaries. This just makes the tests unreadable, so this
is one of several diffs aiming to solve this issue.
Previously, while `f"abc" f"def"` and `"abc" "def"` would parse as a
ConcatenatedString, `f"" ""` wouldn't because it mixes f-strings and
simple strings.
This joins `atom_string` and `atom_fstring` into a single `atom_string`
rule, which has an added benefit of simplifying the code.
Update codegen and tests to cover list, list comprehension and generator
expressions. Also cleans up some extraneous position tracking code in
codegen.
Instead of generating Pass nodes on removal, allow empty IndentedBlock,
SimpleStatementLine and SimpleStatementSuite nodes. Have them insert a
'pass' token manually when they encounter having nothing inside of
them. This allows us to more easily figure out when we should keep a
node around, and means that we no longer construct new nodes with
surprising subnodes based on somewhat-related user input. It also
matches how we do MaybeSentinel support for the rest of Python's
syntactic sugar. As a bonus (well, the point of this diff actually), we
no longer add a pass statement every time we remove another statement,
only when we actually need to.
Changes `CSTNode.visit()` to `CSTNode._visit_impl()` and uses `visit()`
as a public entry point to prevent users from running visitors with
metadata dependencies on nodes other than Modules.
Converts metadata providers to extend from either CSTVisitor or from
BatchableCSTVisitor and updates the runner to batch providers when
possible. There is a known issue in this diff where metadata may be
computed multiple times on the same pass which is addressed in
a later diff. There should be no correctness issues in this diff as
metadata computation should have no side effects.
Defines two new visitor base classes for libcst:
- `CSTTransformer` provides an visitor that can be used to create a new
tree. This is a drop in replacement for the old `CSTVisitor` in
`_base_visitor.py`.
- `CSTVisitor` provides a visitor that does not allow the user to mutate
the tree by restricting access to the updated_node in `on_leave`.
