Commit graph

739 commits

Author SHA1 Message Date
Frank Liu
b75343e74e
Create CodemodCommand Remove/Add Import helper functions (#1432)
* Create helper functions to abstract away usage of RemoveImportsVisitor's remove unused import functions in CodemodCommand

* Create helper functions to abstract away usage of AddImportsVisitor's add needed import functions in CodemodCommand

* Add tests for CodemodCommand helper functions

Add comprehensive tests for the new helper methods:
- remove_unused_import
- remove_unused_import_by_node
- add_needed_import

Tests cover simple cases, from imports, aliased imports,
relative imports, and combined add/remove operations.
2025-12-17 09:28:24 -08:00
Frank Liu
b66c0e2822
[CodemodCommand] Make transform_module supported_transforms order deterministic by using List over Dict (#1424)
Some checks are pending
CI / test (macos-latest, 3.13) (push) Waiting to run
CI / test (macos-latest, 3.13t) (push) Waiting to run
CI / test (macos-latest, 3.14) (push) Waiting to run
CI / test (macos-latest, 3.14t) (push) Waiting to run
CI / test (macos-latest, 3.9) (push) Waiting to run
CI / test (ubuntu-latest, 3.10) (push) Waiting to run
CI / test (ubuntu-latest, 3.11) (push) Waiting to run
CI / test (ubuntu-latest, 3.12) (push) Waiting to run
CI / test (ubuntu-latest, 3.13) (push) Waiting to run
CI / test (ubuntu-latest, 3.13t) (push) Waiting to run
CI / test (ubuntu-latest, 3.14) (push) Waiting to run
CI / test (ubuntu-latest, 3.14t) (push) Waiting to run
CI / test (ubuntu-latest, 3.9) (push) Waiting to run
CI / test (windows-latest, 3.10) (push) Waiting to run
CI / test (windows-latest, 3.11) (push) Waiting to run
CI / test (windows-latest, 3.12) (push) Waiting to run
CI / test (windows-latest, 3.13) (push) Waiting to run
CI / test (windows-latest, 3.13t) (push) Waiting to run
CI / test (windows-latest, 3.14) (push) Waiting to run
CI / test (windows-latest, 3.14t) (push) Waiting to run
CI / test (windows-latest, 3.9) (push) Waiting to run
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / docs (push) Waiting to run
CI / Rust unit tests (push) Waiting to run
CI / Rustfmt (push) Waiting to run
CI / build (push) Waiting to run
pypi_upload / build (push) Waiting to run
pypi_upload / Upload wheels to pypi (push) Blocked by required conditions
GitHub Actions Security Analysis with zizmor 🌈 / zizmor latest via PyPI (push) Waiting to run
2025-11-02 20:27:32 -05:00
martin
de5635394b
fix: circular import error (#1406)
* fix: circular import error
2025-09-25 23:44:58 -04:00
martin
3b5329aa20
feat: add support for PEP758 (#1401)
Some checks are pending
CI / test (macos-latest, 3.13) (push) Waiting to run
CI / test (macos-latest, 3.13t) (push) Waiting to run
CI / test (macos-latest, 3.14) (push) Waiting to run
CI / test (macos-latest, 3.14t) (push) Waiting to run
CI / test (macos-latest, 3.9) (push) Waiting to run
CI / test (ubuntu-latest, 3.10) (push) Waiting to run
CI / test (ubuntu-latest, 3.11) (push) Waiting to run
CI / test (ubuntu-latest, 3.12) (push) Waiting to run
CI / test (ubuntu-latest, 3.13) (push) Waiting to run
CI / test (ubuntu-latest, 3.13t) (push) Waiting to run
CI / test (ubuntu-latest, 3.14) (push) Waiting to run
CI / test (ubuntu-latest, 3.14t) (push) Waiting to run
CI / test (ubuntu-latest, 3.9) (push) Waiting to run
CI / test (windows-latest, 3.10) (push) Waiting to run
CI / test (windows-latest, 3.11) (push) Waiting to run
CI / test (windows-latest, 3.12) (push) Waiting to run
CI / test (windows-latest, 3.13) (push) Waiting to run
CI / test (windows-latest, 3.13t) (push) Waiting to run
CI / test (windows-latest, 3.14) (push) Waiting to run
CI / test (windows-latest, 3.14t) (push) Waiting to run
CI / test (windows-latest, 3.9) (push) Waiting to run
CI / lint (push) Waiting to run
CI / typecheck (push) Waiting to run
CI / docs (push) Waiting to run
CI / Rust unit tests (push) Waiting to run
CI / Rustfmt (push) Waiting to run
CI / build (push) Waiting to run
pypi_upload / build (push) Waiting to run
pypi_upload / Upload wheels to pypi (push) Blocked by required conditions
GitHub Actions Security Analysis with zizmor 🌈 / zizmor latest via PyPI (push) Waiting to run
PEP758 removes the requirement for parentheses to surround exceptions
in except and except* expressions when 'as' is not present.

This pr implements support for parsing these types of statements
2025-09-09 11:16:49 -04:00
martin
48668dfabb
Support parsing of t-strings #1374 (#1398)
#1343
Adds support to parse t-strings

Couple things of note:

TemplatedString* is largely a copy of FormattedString*
Since clients operate of libcst objects I consider this this part of a public API - following the python grammar (where TStrings are distinct from FStrings) seems like a good way to avoid changes to the API in the future.
Within the tokenizer we reuse the fstring machinery
I consider this an implementation detail, fstrings and tstrings are (for now) identical, we can change this later without changes to the public api.
Since 2 -> we have a new FTStringType enum
We need to discriminate between f and t strings to know which token to return, a bit clumsy to use in my opinion - so looking for feedback here on how to improve this.
2025-09-09 11:16:20 -04:00
Stephen Morton
d721a06c3f
generate Attribute nodes when applying type annotations (#1396)
Some checks failed
CI / docs (push) Has been cancelled
CI / Rust unit tests (push) Has been cancelled
CI / test (macos-latest, 3.10) (push) Has been cancelled
CI / test (macos-latest, 3.11) (push) Has been cancelled
CI / test (macos-latest, 3.12) (push) Has been cancelled
CI / test (macos-latest, 3.13) (push) Has been cancelled
CI / test (macos-latest, 3.13t) (push) Has been cancelled
CI / test (macos-latest, 3.14) (push) Has been cancelled
CI / test (ubuntu-latest, 3.11) (push) Has been cancelled
CI / test (ubuntu-latest, 3.13) (push) Has been cancelled
CI / test (ubuntu-latest, 3.13t) (push) Has been cancelled
CI / test (ubuntu-latest, 3.14) (push) Has been cancelled
CI / test (windows-latest, 3.10) (push) Has been cancelled
CI / test (macos-latest, 3.14t) (push) Has been cancelled
CI / test (macos-latest, 3.9) (push) Has been cancelled
CI / test (ubuntu-latest, 3.10) (push) Has been cancelled
CI / test (ubuntu-latest, 3.12) (push) Has been cancelled
CI / test (ubuntu-latest, 3.14t) (push) Has been cancelled
CI / test (ubuntu-latest, 3.9) (push) Has been cancelled
CI / test (windows-latest, 3.11) (push) Has been cancelled
CI / test (windows-latest, 3.12) (push) Has been cancelled
CI / test (windows-latest, 3.13) (push) Has been cancelled
CI / test (windows-latest, 3.13t) (push) Has been cancelled
CI / test (windows-latest, 3.14) (push) Has been cancelled
CI / test (windows-latest, 3.14t) (push) Has been cancelled
CI / Rustfmt (push) Has been cancelled
CI / build (push) Has been cancelled
pypi_upload / build (push) Has been cancelled
GitHub Actions Security Analysis with zizmor 🌈 / zizmor latest via PyPI (push) Has been cancelled
pypi_upload / Upload wheels to pypi (push) Has been cancelled
* generate Attribute nodes when applying type annotations

The old version generated an incorrect CST which
happened to work as long as you didn't do further processing.

* add a test
2025-09-03 16:54:44 -04:00
Ken Kawamoto
7090a0db2b
fixes match statements to work with PositionProvider (#1389)
* add failing test

* fix issue

* fixes an issue with PositionProvider not working with case statement

* remove comments

---------

Co-authored-by: steve <steve@patreon.com>
2025-08-04 17:27:13 -04:00
Thomas Serre
b395d7ccf7
Fix noqa comments (#1379) 2025-08-04 17:03:20 -04:00
martin
9542fc3882
remove entry points to pure parser (#1375)
* rm: ci

* rm: entry point

* fix: tests

* fix: remove combine step from ci

* linter fixes

* omit the _parser

* fix newlines

* fix: remove optional

* fix: linter

---------

Co-authored-by: thereversiblewheel <martin.li@uwaterloo.ca>
2025-07-30 16:27:20 +00:00
Lysandros Nikolaou
db38266f1d
Upgrade PyYAML-ft version and use new module name (#1353)
* Upgrade PyYAML-ft version and use new module name

* add pyre ignore

---------

Co-authored-by: Zsolt Dollenstein <zsol@meta.com>
2025-06-10 17:21:21 +01:00
zaicruvoir1rominet
ca1f81f049
Avoid raising bare Exception (#1168)
* Keep old exception messages (avoid breaking-changes for users relying on exception messages)

* Move ``get_expected_str`` out of _exceptions.py, where it does not belong, to its own file in _parser/_parsing_check.py
2025-06-07 01:53:44 -07:00
Zsolt Dollenstein
e12eef5810
add helper to convert nodes to matchers (#1351)
* add helper to convert nodes to matchers

* suppress type error
2025-06-04 14:02:21 -07:00
Zsolt Dollenstein
3dc2289bf6
codegen: Support pipe syntax for Union types (#1336)
From 3.14 onwards, we'll get `foo | bar` instead of `typing.Union[foo, bar]` as the annotation for union types (including optional). This PR prepares the codegen script for this.
2025-05-26 08:40:54 +01:00
Zsolt Dollenstein
b560ae815c
Threadpool should be used if GIL is disabled. (#1335) 2025-05-25 20:13:12 +01:00
Zsolt Dollenstein
16ed48d74b
Enable support for free-threading (#1295)
This PR:
1. marks the `libcst.native` module as free-threading-compatible
2. replaces the use of ProcessPoolExecutor with ThreadPoolExecutor if free-threaded CPython is detected at runtime
2025-05-25 11:43:18 +01:00
Zsolt Dollenstein
52acdf4163
cli: Instantiate Codemods per file (#1334)
Instead of sharing instances of a Codemod across many files, this PR allows passing in a Codemod class to `parallel_exec_transform_with_prettyprint` which will then instantiate the Codemod for each file.  `tool._codemod_impl` now starts using this API.

The old behavior is deprecated, because sharing codemod instances across files is a surprising behavior, and causes hard-to-diagnose bugs when a Codemod keeps track of its state via instance variables.
2025-05-25 09:23:10 +01:00
Zsolt Dollenstein
d002c14d6b
Replace multiprocessing with ProcessPoolExecutor (#1294)
Instead of relying on `multiprocessing.Pool`, this PR replaces the implementation of `parallel_exec_transform_with_prettyprint` with `concurrent.futures.ProcessPoolExecutor`
2025-05-22 08:18:20 +01:00
dependabot[bot]
be0b668d08
Bump black from 24.8.0 to 25.1.0 (#1290)
* Bump black from 24.8.0 to 25.1.0

Bumps [black](https://github.com/psf/black) from 24.8.0 to 25.1.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/24.8.0...25.1.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix formatting and tests

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amethyst Reese <amethyst@n7.gg>
2025-05-19 20:53:44 -04:00
Nathan Goldbaum
a4804cf07e
allow configuring empty formatter lists in codemod CLI (#1319)
* allow configuring empty formatter lists

* appease linter
2025-04-02 10:19:27 +01:00
Danny Yang
6b483c6113
Add codemod to rename typing aliases of builtins (#1267)
* add codemod to rename typing aliases of builtins

* format
2025-01-09 18:59:00 +00:00
Danny Yang
776452f351
Add codemod to fix variadic callable annotations (#1269)
* add fix variadic callable codemod

* format
2025-01-02 19:49:03 -05:00
Danny Yang
d26987202b
Add codemod to convert typing.Union to | (#1270)
* add union to or codemod

* lint

* early return
2025-01-02 19:48:55 -05:00
Zsolt Dollenstein
a3b5529bb3
rename: fix renaming toplevel names (#1260)
For toplevel module names imported via `import foo`, the rename codemod would fail to change these. This PR fixes that.
2024-12-11 20:30:33 +00:00
Zsolt Dollenstein
d24192a40f
rename: don't eat commas unnecessarily (#1256)
#1254 was a bit too aggressive in removing commas. They shouldn't be removed if there are parenthesis around the imported names.
2024-12-02 16:13:12 +00:00
Zsolt Dollenstein
8c30fcef30
rename: don't leave trailing commas (#1254)
When renaming the last element of a `from a import b,c` import, don't leave the trailing comma after `b`
2024-12-02 10:00:59 +00:00
Zsolt Dollenstein
c05ac74b9a
refactor: allow scheduled_removals to accept a tuple (#1253)
This fixes a TODO
2024-12-02 10:00:35 +00:00
Zsolt Dollenstein
a36432c958
rename: Fix imports with aliases (#1252)
When renaming `a.b` -> `c.d`, in imports like `import a.b as x` the as_name wasn't correctly removed even though references to `x` were renamed to `c.d`.

This PR makes the codemod remove the `x` asname in these cases.
2024-11-29 11:23:59 +00:00
Zsolt Dollenstein
28e0f397b2
rename: handle imports via a parent module (#1251)
When requesting a rename for `a.b.c`, we want to act on `import a` when it's used to access `a.b.c`
2024-11-28 20:02:23 +00:00
Zsolt Dollenstein
6fdca74c90
rename: store state in scratch (#1250)
This PR changes RenameCodemod to store its per-module state in `self.context.scratch` which gets properly reset between files.
2024-11-28 14:59:43 +00:00
Kirill Ignatev
9fd67bca49
fix certain matchers breaking under multiprocessing by initializing them late (#1204)
* Add is_property check

Skip properties to prevent exceptions

* Delayed initialization of matchers

To support multiprocessing on Windows/macOS
Issue #1181

* Add a test for matcher decorators with multiprocessing
2024-09-25 11:29:54 +01:00
dependabot[bot]
bf5fb4132e
Bump black from 23.12.1 to 24.8.0 (#1186)
* Bump black from 23.12.1 to 24.8.0

Bumps [black](https://github.com/psf/black) from 23.12.1 to 24.8.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/23.12.1...24.8.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update formatting

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amethyst Reese <amethyst@n7.gg>
2024-08-22 16:46:01 -07:00
Kirill Ignatev
45234f198c
Clear warnings for each file in comemod cli (#1184)
* Clean warnings for each file in comemod cli

* Fix ZeroDivisionError: float division by zero

When codemodding too fast

* Recreate CodemodContext for each file

Keep only context.metadata_manager
Remove wrapper from context defaults on each file
2024-08-05 22:41:51 +01:00
Kirill Ignatev
b0d145dddd
Add validation for If node (#1177)
* Add validation for If node

Don't allow no space no parentheses.
2024-07-30 09:01:07 +01:00
Jia Chen
e20e757159
Remove uses of # pyre-placeholder-stub (#1174) 2024-07-20 09:04:25 +01:00
Zsolt Dollenstein
8b97600fb3
fix various Match statement visitation errors (#1161)
Fixes #1160.

This PR also

- fixes `whitespace_before_colon` being swallowed during visitation on `MatchCase`s
- adds a new type of roundtrip test that catches issues of this class: the test applies a noop transformer to exercise the visitation API and compares the result with the original source.
- adds a few more cases to the match fixture
2024-06-12 17:29:25 +01:00
Camillo
9f6e27600f
FullyQualifiedNameProvider: Optionally consider pyproject.toml files when determining a file's module name and package (#1148) 2024-06-12 10:36:50 +01:00
Zsolt Dollenstein
db696e6348
fix: don't reset context.scratch between files (#1151)
#453 fixed scratch leaking between files by setting it to empty, but that drops all the scratch space that was set up before the codemod runs (e.g. in the transformer's constructor)

This PR improves the fix by preserving the initial scratch.
2024-05-21 15:52:49 -04:00
martin
71b0a1288b
Implement Type Defaults for Type Parameters (PEP 696) (#1141)
Co-authored-by: thereversiblewheel <martin.li@uwaterloo.ca>
2024-05-20 11:26:38 -04:00
zaicruvoir1rominet
6bbc69316b
Add the ability to dump CST to .dot (graphviz) files (#1147)
* Make the nodes fields filtering process - from libcst.tool - public, so that other libraries may provide their own custom representation of LibCST graphs.

* Create functions to access & filter CST-node fields (with appropriate docstrings & tests), in libcst.helpers

* Add new CST-node fields functions to helpers documentation.
2024-05-20 11:25:13 -04:00
zaicruvoir1rominet
efc53af608
Add helper functions for common ways of filtering nodes (#1137)
* Make the nodes fields filtering process - from libcst.tool - public, so that other libraries may provide their own custom representation of LibCST graphs.

* Create functions to access & filter CST-node fields (with appropriate docstrings & tests), in libcst.helpers

* Add new CST-node fields functions to helpers documentation.
2024-05-13 10:20:47 +01:00
Camillo
82f804a66a
Fix Literal parse error in RemoveImportsVisitor (#1130) 2024-05-03 22:36:20 +01:00
Zsolt Dollenstein
4fb66a33e6
remove mypy_extensions import (#1128) 2024-04-03 21:10:44 +01:00
Zsolt Dollenstein
2ffca10845
remove typing dependencies (#1126)
Summary:
This PR removes the `typing_extensions` and `typing_inspect` dependencies as we can now rely on the built-in `typing` module since Python 3.9.

Test Plan:
existing tests
2024-04-03 19:50:14 +01:00
Zsolt Dollenstein
724026aa65
Remove reference to distutil (#1099)
Distutil has been removed in Python 3.12.

Tested by:
```
py -m libcst.tool codemod noop.NOOPCommand .\libcst\tool.py
Calculating full-repo metadata...
Executing codemod...
Finished codemodding 1 files!
 - Transformed 1 files successfully.
 - Skipped 0 files.
 - Failed to codemod 0 files.
 - 0 warnings were generated.
```
2024-02-02 20:58:56 +00:00
Zsolt Dollenstein
55f3e34dfc
Add roundtrip tests from Python (#1098)
Our current roundtrip tests only excerise the Rust codepaths. This PR runs the same roundtrip scenarios but from Python.
2024-02-02 20:50:07 +00:00
Zsolt Dollenstein
c854c986b6
Fix parsing list matchers without explicit brackets (#1097)
```
match a:
  case 1, 2: pass
```

This is parsed correctly by the grammar, but the default values of `MatchList.lbracket` and `MatchList.rbracket` are inconsistent between Python and Rust, causing the above snippet to round-trip (from Python) to:
```
match a:
  case [1, 2]: pass
```

Fixes #1096.
2024-02-02 20:49:25 +00:00
anonymousdouble
dbbfe1e0b8
Update test_fix_pyre_directives.py (#1082)
* Update test_fix_pyre_directives.py

refactor with fstring to format string to make code more Pythonic.

* Update test_fix_pyre_directives.py

refactor with fstring to format string to make code more Pythonic.

* Update test_fix_pyre_directives.py

refactor with fstring to format string to make code more Pythonic.

* Update test_fix_pyre_directives.py

refactor with fstring to format string to make code more Pythonic.

* Update test_fix_pyre_directives.py

refactor with chain constant value assignment to make code more Pythonic

* Update test_fix_pyre_directives.py

refactor with chain constant value assignment to make code more Pythonic
2024-01-03 13:06:33 -08:00
Kyle Into
83f0daed42
fix filepathprovider generic type (#1036)
* fix filepathprovider type

* remove extra import
2023-10-05 20:18:39 -07:00
Sergii Dymchenko
7ca5d7f173
Fix link in type_inference_provider.py (#1035)
Same change as https://github.com/Instagram/LibCST/pull/913, but in the docstring.
2023-10-05 15:46:25 -07:00
Zsolt Dollenstein
738dc2f893
Upgrade pyre (#1032)
* Upgrade pyre

* regen fixtures
2023-10-02 09:43:17 -07:00