Commit graph

683 commits

Author SHA1 Message Date
Zsolt Dollenstein
ea4ea347f1
Allow no whitespace after lambda body in certain cases
Like in `[lambda:()for _ in _]`

Reported in #930.
2023-05-28 11:05:33 +01:00
Zsolt Dollenstein
862b702637
Allow no spaces around walrus in certain cases
Like in `[_:=''for _ in _]`

Raised in #930.
2023-05-28 10:56:59 +01:00
Zsolt Dollenstein
4ce3660127
Allow no spaces after as in a contextmanager in certain cases
Like in `with foo()as():pass`

Raised in #930.
2023-05-28 10:47:51 +01:00
Zsolt Dollenstein
aa5167189c
Allow no space around an ifexp in certain cases
For example in `_ if _ else""if _ else _`.

Raised in #930. Also fixes #854.
2023-05-28 10:29:30 +01:00
Zsolt Dollenstein
f45ba63ca6
Allow any expression in comprehensions' evaluated expression
This PR relaxes the accepted types for the `elt` field in `ListComp`, `SetComp`, and `GenExp`, as well as the `key` and `value` fields in `DictComp`.

Fixes #500.
2023-05-28 10:28:26 +01:00
Zsolt Dollenstein
744d6fd3cd
Allow no whitespace between lambda keyword and params in certain cases
Python accepts code where `lambda` follows a `*`, so this PR relaxes validation rules for Lambdas.

Raised in #930.
2023-05-27 20:21:24 +01:00
Sigurd Ljødal
a594fe1dd2
Fix type of evaluated_value on string to allow bytes (#721)
* Fix type of evaluated_value on string

This can return bytes if the string is a bytestring, e.g.:

    In [1]: import libcst as cst

    In [2]: cst.parse_expression('b"foo"').evaluated_value
    Out[2]: b'foo'

* Fix type errors from changed signature
2023-05-26 13:43:05 +01:00
Zsolt Dollenstein
193fab4357
Switch default parser implementation to native (#929)
The old parser is now only available using LIBCST_PARSER_TYPE=pure
2023-05-25 18:24:59 +01:00
John Litborn
38b708b5ed
relax validation rules on decorators (#926)
* relax validation on decorators

* allow any expression

---------

Co-authored-by: Zsolt Dollenstein <zsol.zsol@gmail.com>
2023-05-23 09:56:49 +01:00
Shantanu
2055342fd6
Support PEP 604 in ApplyTypeAnnotationsVisitor (#868) 2023-04-21 13:38:19 +01:00
Steven Troxler
f0a4d62c3b
Fix spelling and grammar in some comments (#908)
I'm going to go ahead and land this, I don't think it needs review
2023-04-13 14:42:57 -07:00
Rebecca Chen
f936db240f
Fix ApplyTypeAnnotationsVisitor behavior on attribute assignments. (#903)
* Fixes an issue where ApplyTypeAnnotationsVisitor would crash on code
  like `SomeClass.some_attribute = 42` with a "Name is not a valid
  identifier" error message.
* Changes the above-mentioned error message to include the bad name in
  the message, for easier debugging.
* Adds tests for all valid assignment targets, as described here:
  https://libcst.readthedocs.io/en/latest/nodes.html#libcst.BaseAssignTargetExpression.
2023-04-05 14:23:53 -07:00
Aarni Koskela
4f810dbc13
Allow running codemods without configuring in YAML (#879)
* Simplify command specifier parsing

* Allow running codemods without configuring in YAML

This enables codemodding things by just plonking a CodemodCommand class
into any old importable module and running
`python -m libcst.tool codemod -x some_module.SomeClass ...`
2023-03-27 10:59:48 +01:00
Aarni Koskela
ae42deed9b
Ensure current Python interpreter is used for subprocesses (#898) 2023-03-27 10:56:25 +01:00
Aarni Koskela
9381fee9ab
Use subprocess.DEVNULL instead of opening os.devnull by hand (#897) 2023-03-24 10:03:11 +00:00
dependabot[bot]
46509dd5e1
Bump black from 22.12.0 to 23.1.0 (#860) 2023-03-15 11:53:50 +00:00
Amethyst Reese
6a7b82e2b6 PEP 621 + hatch to run tests/lint/etc
Moves PEP 621 metadata from `setup.py` and `requirements*.txt` into the
`[project]` table of `pyproject.toml`. This enables using hatch as a
task runner for the project, where previously one would need to remember
a bunch of different commands, or repeatedly consult the readme's
developer guide to find all of the relevant commands.

This creates the following hatch commands:

- docs
- fixtures
- format
- lint
- test
- typecheck

It also updates all of the github actions workflows to use the
appropriate hatch commands, and the readme's developer guide, so that
there is only one source of truth for what constitutes running tests.

The "test" workflows now drop the matrix distinction between "pure" or
"native", and run tests in both modes from a single build.

ghstack-source-id: 8834da7825
Pull Request resolved: https://github.com/Instagram/LibCST/pull/893
2023-03-14 19:37:41 -07:00
Mikhail Podtserkovskiy
497f7784c5
Fix: relative imports from '' package are not allowed (#894) 2023-03-14 15:07:40 -07:00
Amethyst Reese
c876db6d2d Add new FilePathProvider
Caches file path information on the root `Module` node.
Resolves paths when caching, so they are always absolute paths.

Adds a new `chdir` helper to change working directory and automatically
revert to previous directory, which makes testing file paths with the
`"."` repo root easier.

ghstack-source-id: 3413905fc1
Pull Request resolved: https://github.com/Instagram/LibCST/pull/892
2023-03-13 18:33:46 -07:00
Amethyst Reese
f9536b522f
Pass root path to FullyQualifiedNameProvider (#867)
This allows FullyQualifiedNameProvider to work with absolute paths,
rather than assuming all paths given will be relative to the current
directory. This enables tools like Fixit to provide a root path, and
have the FullyQualifiedNameProvider correctly scope the final results
relative to that root path.

This does require that both the root path and the given file paths
match the other as relative or absolute, due to the
`calculate_module_and_package` helper comparing file paths relative
to the root path, but this seems like a reasonable tradeoff, and
unlikely to cause a problem in normal use cases.
2023-02-22 14:36:10 -08:00
Amethyst Reese
d94687e378
Script to regenerate test fixtures (#872)
Upgrading Pyre requires updating test fixtures with any upstream changes
to Pyre's query results for the `simple_class.py` fixture.

This adds a new `scripts/` directory to the repo, with a script to
regenerate test fixtures. The script regenerates the cache data fixture,
and updates the `TypeInferenceProvider` tests to use `assertDictEqual`
and helpful error messages for better behavior in future mismatches.

This also includes a slight bump to Pyre 0.9.10 to fix install issues on
Apple Silicon M1 Macs, and regenerated fixtures using the script above.
2023-02-22 12:35:23 -08:00
Sagar Badiyani
de28541fa3
[AddImportsVisitor] Docstring Check Only for the Top Element of the Body (#841)
* Initial Commit

* lint fix
2023-01-24 13:51:56 +00:00
Carl Meyer
f668e88dd2
fix PEP 604 union annotations in decorators (#828) 2022-11-29 15:24:24 -07:00
Matthew Shaer
c44b182e88
Adding a provider which can tell what accessor to use to go from the parent to that child node (#807) 2022-11-10 09:38:32 +00:00
MapleCCC
973895a6c0
Several trivial refactors (#770)
* Enumeration members are singletons. Copying on them would be no-op

* Avoid generating unnecessary `pass` statement

* Several trivial refactor

* Avoid building unnecessary intermediate lists, which are mere slight waste of time and space

* Remove unused import, an overlook from commit 8e6bf9e9

* `collections.abc.Mapping.get()` defaults to return `None` when key doesn't exist

* Just use unittest's `assertRaises` to specify expected exception types, instead of catching every possible `Exception`s, which could suppress legitimate errors and hide bugs

* We know for sure that the body of `CSTTypedTransformerFunctions` won't be empty, so don't bother with complex formal completeness
2022-09-14 14:33:45 +01:00
MapleCCC
667c713b38
Fix the bug that the use of formatter in codemods has undetermined target Python version, resulting in hard-to-reason-with behavior (#771)
* When codemod, specify the black formatter to use the same target Python version we use

* Fix the `test_codemod_formatter_error_input` unit test

* Remove an unused import in `test_codemod_cli` module
2022-09-14 14:31:36 +01:00
MapleCCC
27aa23f056
Raise informative exception when metadata is unresolved in a metadata-based match (#757)
* Raise informative exception when metadata is unresolved in a metadata-based match, instead of silently hide potential errors

* Fix unit test of `findall`

* Add unit test to cover the case of a resolved metadata provider doesn't provide metadata for all nodes

* Document the behavior of metadata-based match when the metadata provider is unresolved
2022-08-29 16:47:02 +01:00
MapleCCC
64811b7795
Tighten the metadata type of ExpressionContextProvider (#760) 2022-08-29 16:46:08 +01:00
MapleCCC
fc622ce790
Fix bug when TypeOf is one of options in OneOf / AllOf (#756)
* Fix a bug when one of the option of `OneOf` is a `TypeOf`

* Disallow `TypeOf` in `AllOf`, analogous to how `OneOf` is disallowed in `AllOf`
2022-08-26 18:54:07 +01:00
MapleCCC
73cfc7f7fa
Fix docstring of FullRepoManager (#750)
The render error originates from how we violate the syntax rules of the `field list` markup element of reStructuredText. The `specification of field list states](https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#field-lists) that a multi-line `field body` must be indented relative to the `field marker`.
2022-08-17 13:42:31 +01:00
zzl
47e5ea15e1
Fix parse error message for number parsing (#724)
Co-authored-by: zzl0 <zhuzhaolong0@mail.com>
2022-08-04 11:33:26 +01:00
Chenguang Zhu
7cb229d175
Implement lazy loading mechanism for QualifiedNameProvider (#720)
* Implement lazy loading mechanism for expensive metadata providers
* Add support for lazy values in metadata matchers
* Fix type issues and implement lazy value support in base metadata provider too
* Add unit tests for BaseMetadataProvider

Co-authored-by: Zsolt Dollenstein <zsol.zsol@gmail.com>
2022-07-09 08:34:29 +01:00
Luke Petre
b3eda508d4
Fixing prefix matching bug from 0.4.6 (#719) 2022-07-05 11:25:39 +01:00
Zsolt Dollenstein
343f56f607
[parser] bail on deeply nested expressions (#718) 2022-07-04 14:45:42 +01:00
Zsolt Dollenstein
9925117391
Support whitespace after ParamSlash (#713)
* add whitespace_after field to ParamSlash
* codegen
2022-06-26 09:42:37 +01:00
Luke Petre
aa4a2790db
Remove unnecessary qname work (#709) 2022-06-21 21:35:05 +01:00
Luke Petre
779163701c
Faster qualified name format (#710) 2022-06-21 14:26:45 +01:00
Luke Petre
42164f8672
Cache the scope name prefix to prevent scope traversal in a tight loop (#708)
* Cache the scope name prefix to prevent scope traversal in a tight loop

* Adding pyre-fixme. this attribute iclearly has a type in the base class.

* Clarify why we do join(filter(None,...
2022-06-21 10:11:02 +01:00
Sergei Lebedev
306a5f8175
convert_type_comments now preserves comments following type comments (#702)
For example,

    y = 5  # type: int  # foo

is converted to

    y: int = 5  # foo
2022-06-20 07:39:12 -06:00
zzl
7ca1bd1cd5
expression: fix SimpleString's quote method (#704)
* expression: fix SimpleString's quote method

* Add missing copyright header

Co-authored-by: zzl0 <zhuzhaolong0@mail.com>
Co-authored-by: Luke Petre <lpetre@fb.com>
2022-06-17 13:05:05 +01:00
Luke Petre
6f28c799bb
Fix slow perf in 0.4.2+ (#698) 2022-06-16 12:45:02 +01:00
Zsolt Dollenstein
ebe1851c2b
Add support for PEP-646 (#696) 2022-06-13 09:52:31 -06:00
Luke Petre
149599ee88
Restore the 0.4.1 behavior for libcst.helpers.get_absolute_module (#684) 2022-05-11 07:42:37 -04:00
Luke Petre
c6559671aa
Consider access information when computing qualified names for nodes (#682)
* Write a test case showing qualified names for shadowed assignments

* Consider accesses when looking up names of nodes

* Fix format

* Fix typecheck
2022-05-04 09:50:26 -04:00
Zsolt Dollenstein
c30bbcfa48
make sure ParserError's raw_line is zero-indexed (#681) 2022-05-04 05:51:49 -06:00
toofar
6e0c63ae9c
rename codemod: Correct last renamed import from (#675)
Correct the renamed import from structure when renaming last imported name from a module.

Given

    from a.b import qux

    print(qux)

And providing old_name="a.b.qux" and new_name="a:b.qux" I expect the
following output (as described int the command description):

    from a import b

    print(b.qux)

But what I get is:

    from a import b.qux

    print(b.qux)

It pulls the old name up into the new one.

The provided test is the important part but I've attempted a fix too. I
suspect there is a better one and that the special casing of the "this
is that last name" situation shouldn't be needed. For instance there is
import removing code in leave_Module and renaming the first of many
names (as opposed to the last) happily adds a correct import line.
I didn't manage to grok the code and all the concepts it requires to
provide a better fix though.

This leaves the alias adjustments to the existing code and just does the
module renaming the int he special casing block.
I don't know why scheduling removal of the updated node is required, it makes
the tests pass though.
2022-04-14 17:13:40 -07:00
Martin DeMello
abc566cd4a
add an always_qualify_annotations argument to the type annotation visitor (#676) 2022-04-13 14:23:21 -07:00
Martin DeMello
e6f208c7db
Qualify imported symbols when the dequalified form would cause a conflict (#674)
* Qualify imported symbols when the dequalified form would cause a conflict.

Adds a preliminary pass that scans the stub file for all imported
symbols, and collects the ones that cannot be safely dequalified.

Fixes #673

* review fixes

* handle symbol conflicts between the stub and the main file

* fix type errors
2022-04-12 12:12:55 -07:00
wiyr
0a8ae91d39
fix qualified name get bug (#669)
* fix qualified name get bug

* added unittest
2022-04-08 08:25:15 -07:00
Martin DeMello
869af036e0
Preserve as-imports when merging type annotations. (#664)
* Preserve as-imports when merging type annotations.

Fixes #661

* fix test and bad rebase

* ufmt
2022-04-07 15:12:30 -07:00