* [RemoveUnusedImports] Support string type annotations
This PR adds support for detecting imports being used by string type
annotations, as well as imports suppressed by comments.
It breaks up the existing visitor into multiple smaller, single-purpose
visitors, and composes them together.
Basically move the apply_annotations code from Pyre.
Make it a ContextAwareTransformer named ApplyTypeAnnotationsVisitor. Use GatherImportsVisitor to collect imports. Add a static method `add_stub_to_context` so that users can schedule a stub whose types are to be applied later. Use `assertCodemod` and `data_provider` in the tests. Add documentation. Remove fixmes.
This is especially helpful for checking qualified names of nodes against one item
or a list of items that you wish to match against. I chose to create a new matcher
instead of widening the type of `MatchMetadata` to take in either a value or
a callable. I was originally going to do the former, but having a MatchIfTrue
and a MatchMetadataIfTrue felt more orthogonal and became easier to explain than
a single MatchIfTrue that could take two types of values.
Findall does what you expect given a matcher and a tree: It returns all nodes
that exist in that tree which match the given matcher. For convenience, findall
works with regular LibCST trees as well as MetadataWrappers. It is also provided
as a helper method on the matcher transforms.
In certain cases (e.g. inside Instagram's lint framework) we know that
our tree originates from the parser, so we know that there shouldn't be
any duplicate nodes in our tree.
MetadataWrapper exists to copy the tree ensuring that there's no
duplicate nodes.
This diff provides an escape hatch on MetadataWrapper that allows us to
save a little time and avoid a copy when we know that it's safe to skip
the copy.
As part of this, I ran into some issues with `InitVar` and pyre, so I
removed `@dataclass` from the class. This means that this is techincally
a breaking change if someone depended on the MetadataWrapper being an
actual dataclass, but I think this is unlikely. I implemented `__repr__`
and added tests for hashing/equality behavior.
**Context:** This is an experimental performance optimization that we're
hoping to use for our internal linter at Instagram. I added some
documentation, but it's unsupported, and isn't very user-friendly.
This adds `ExperimentalReentrantCodegenProvider`, which tracks the
codegen's internal state (indentation level, character offsets,
encoding, etc.) and for each statement, it stores a `CodegenPartial`
object.
The `CodegenPartial` object has enough information about the previous
codegen pass to run the codegen on part of a tree and patch the result
back into the original module's string.
In cases where we need to generate a bunch of small independent patches
for the same file (and we can't just generate a new tree with each patch
applied), this *should* be a faster alternative.
I don't have any performance numbers because I still need to test this
end-to-end with our internal codebase, but I'd be shocked if it was
slower than what we're doing.
This could theoretically live outside of LibCST, but it depends on a
whole bunch of LibCST internals, so there's some value in making sure
that this is in sync with the rest of LibCST.
There are going to be many more where this comes from, but these are the three cases that have come up most frequently, so I started with these. Hopefully this helps give additional direction to people using LibCST.
This adds the ability to match on metadata for a node as if it was an attribute on the node. It also opens up the ability to match on node attributes via metadata only. It allows for metadata match specifications with similar flexibility to standard matchers.
I missed this in #114 when renaming `SyntacticPositionProvider` to
`PositionProvider` because it's a different file extension and I was
only grepping for rst and py files.
While these classes are used by the codegen implementation, conceptually
they're part of `libcst.metadata`, so we should export them from
`libcst.metadata` instead of the top-level `libcst` package.
I discussed the high-level idea here with @DragonMinded a few months
ago, but this isn't set in stone. If people have better ideas for names,
I'd love to hear it.
Publicly-Visible Changes
------------------------
- SyntacticPositionProvider is deprecated. The new name is
PositionProvider.
- BasicPositionProvider is deprecated. The new name is
WhitespaceInclusivePositionProvider.
- Documentation is updated to better explain these renamed providers and
how to use them.
The prefixes "Syntactic" and "Basic" were pretty bad because they're
just concepts that we made up for LibCST.
The idea for the new names is that most users will want the
SyntacticPositionProvider, and so we should name things so that the user
will naturally gravitate towards the correct choice.
There's some argument that we shouldn't even bother exposing
WhitespaceInclusivePositionProvider, but we already need to implement it
as a fallback for PositionProvider, and it might be useful for some
niche use-cases.
Once we have another major version bump, we can remove the old class
names. The old class names have already be removed from the
documentation so that new users aren't tempted to use them.
Internal-Only Changes
---------------------
- `PositionProvider` is now `_PositionProviderUnion`. This type alias
was never a public API (and probably never will be).
- `BasicCodegenState` is now
`WhitespaceInclusivePositionProvidingCodegenState`.
- `SyntacticCodegenState` is now `PositionProvidingCodegenState`.
Explaining the implementation details of scopes to someone unfamiliar
with compilers can be tricky. Hopefully this helps.
- Rephrased the definition of a scope to be more applicable to Python
(remove references to "blocks"), and made it use an example for
(hopefully) better clarity.
- New scopes are also created for comprehensions.
- Set a fixed width (400px) for the scope diagram, since it was too
large before.
- Tweaked some tenses.
- Add a final call to action: "LibCST allows you to inspect these
scopes"
Add a RemoveFromParent() function as a convenience to returning RemovalSentinel.REMOVE.
Introduce a `deep_remove()` on CSTNode analogous to `deep_replace()` but for removing.
