Commit graph

5385 commits

Author SHA1 Message Date
Alex Waygood
d6b24b690a
[pycodestyle] Fix whitespace-related false positives and false negatives inside type-parameter lists (#13704) 2024-10-10 17:24:17 +01:00
Alex Waygood
5b4afd30ca
Harmonise methods for distinguishing different Python source types (#13682) 2024-10-09 13:18:52 +00:00
Micha Reiser
b9827a4122
Remove layout values from AnyStringPart (#13681) 2024-10-09 07:25:40 +01:00
Carl Meyer
93eff7f174
[red-knot] type inference/checking test framework (#13636)
## Summary

Adds a markdown-based test framework for writing tests of type inference
and type checking. Fixes #11664.

Implements the basic required features. A markdown test file is a suite
of tests, each test can contain one or more Python files, with
optionally specified path/name. The test writes all files to an
in-memory file system, runs red-knot, and matches the resulting
diagnostics against `Type: ` and `Error: ` assertions embedded in the
Python source as comments.

We will want to add features like incremental tests, setting custom
configuration for tests, writing non-Python files, testing syntax
errors, capturing full diagnostic output, etc. There's also plenty of
room for improved UX (colored output?).

## Test Plan

Lots of tests!

Sample of the current output when a test fails:

```
     Running tests/inference.rs (target/debug/deps/inference-7c96590aa84de2a4)

running 1 test
test inference::path_1_resources_inference_numbers_md ... FAILED

failures:

---- inference::path_1_resources_inference_numbers_md stdout ----
inference/numbers.md - Numbers - Floats
  /src/test.py
    line 2: unexpected error: [invalid-assignment] "Object of type `Literal["str"]` is not assignable to `int`"

thread 'inference::path_1_resources_inference_numbers_md' panicked at crates/red_knot_test/src/lib.rs:60:5:
Some tests failed.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    inference::path_1_resources_inference_numbers_md

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.19s

error: test failed, to rerun pass `-p red_knot_test --test inference`
```

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-08 12:33:19 -07:00
Micha Reiser
fc661e193a
Normalize implicit concatenated f-string quotes per part (#13539) 2024-10-08 09:59:17 +00:00
Zanie Blue
42fcbef876
Fix typo in allow-unused-imports documentation (#13669) 2024-10-07 14:08:36 -05:00
Alex Waygood
71b52b83e4
[red-knot] Allow type[] to be subscripted (#13667)
Fixed a TODO by adding another TODO. It's the red-knot way!

## Summary

`builtins.type` can be subscripted at runtime on Python 3.9+, even
though it has no `__class_getitem__` method and its metaclass (which
is... itself) has no `__getitem__` method. The special case is
[hardcoded directly into `PyObject_GetItem` in
CPython](744caa8ef4/Objects/abstract.c (L181-L184)).
We just have to replicate the special case in our semantic model.

This will fail at runtime on Python <3.9. However, there's a bunch of
outstanding questions (detailed in the TODO comment I added) regarding
how we deal with subscriptions of other generic types on lower Python
versions. Since we want to avoid too many false positives for now, I
haven't tried to address this; I've just made `type` subscriptable on
all Python versions.

## Test Plan

`cargo test -p red_knot_python_semantic --lib`
2024-10-07 19:43:47 +01:00
Zanie Blue
fb90f5a13d
Add known limitation to C416 with dictionaries (#13627)
Part of https://github.com/astral-sh/ruff/issues/13625

See also #13629
2024-10-07 16:20:45 +00:00
Alex Waygood
d7484e6942
[red-knot] Improve type inference for except handlers where a tuple of exception classes is caught (#13646) 2024-10-07 16:13:06 +01:00
Dylan
14ee5dbfde
[refurb] Count codepoints not bytes for slice-to-remove-prefix-or-suffix (FURB188) (#13631) 2024-10-07 16:13:28 +02:00
Alex Waygood
27ac34d683
Rework S606 (start-process-with-no-shell) docs to make clear the security motivations (#13658)
Helps with #13614. This docs rewrite draws on the [documentation for the
original bandit
rule](https://bandit.readthedocs.io/en/latest/plugins/b606_start_process_with_no_shell.html).
2024-10-07 13:31:01 +01:00
Sid
31ca1c3064
[flake8-async] allow async generators (ASYNC100) (#13639)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Treat async generators as "await" in ASYNC100.

Fixes #13637

## Test Plan

Updated snapshot
2024-10-07 07:25:54 -05:00
qdegraaf
646e4136d7
[flake8-bugbear] Tweak B905 message to not suggest setting parameter strict= to False (#13656)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-07 11:56:17 +00:00
Alex Waygood
58a11b33da
Fixup docs markup for RUF027 (#13659) 2024-10-07 11:49:45 +00:00
Aleksei Latyshev
73aa6ea417
[refurb] implement hardcoded-string-charset (FURB156) (#13530)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-10-07 07:35:14 +00:00
renovate[bot]
38d872ea4c
Update Rust crate hashbrown to 0.15.0 (#13652)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-10-07 08:50:59 +02:00
Simon
8108f83810
[red-knot] feat: add StringLiteral and LiteralString comparison (#13634)
## Summary

Implements string literal comparisons and fallbacks to `str` instance
for `LiteralString`.
Completes an item in #13618

## Test Plan

- Adds a dedicated test with non exhaustive cases

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-05 12:22:30 -07:00
Simon
f1205177fd
[red-knot] fix: when simplifying union, True & False -> instance(bool) (#13644) 2024-10-05 19:01:10 +01:00
Simon
1c2cafc101
[red-knot] more ergonomic and efficient handling of known builtin classes (#13615) 2024-10-05 18:03:46 +01:00
Alex Waygood
7c5a7d909c
[red-knot] Improve tests relating to type inference for exception handlers (#13643) 2024-10-05 16:59:36 +00:00
Zanie Blue
2a365bb278
Mark PLE1141 fix as unsafe (#13629)
Closes https://github.com/astral-sh/ruff/issues/13343

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-04 14:22:26 -05:00
Zanie Blue
020f4d4a54
Add test cases for RUF006 with lambdas (#13628)
As discussed in https://github.com/astral-sh/ruff/issues/13619
2024-10-04 14:09:43 -05:00
Simon
888930b7d3
[red-knot] feat: implement integer comparison (#13571)
## Summary

Implements the comparison operator for `[Type::IntLiteral]` and
`[Type::BooleanLiteral]` (as an artifact of special handling of `True` and
`False` in python).
Sets the framework to implement more comparison for types known at
static time (e.g. `BooleanLiteral`, `StringLiteral`), allowing us to only
implement cases of the triplet `<left> Type`, `<right> Type`, `CmpOp`.
Contributes to #12701 (without checking off an item yet).

## Test Plan

- Added a test for the comparison of literals that should include most
cases of note.
- Added a test for the comparison of int instances

Please note that the cases do not cover 100% of the branches as there
are many and the current testing strategy with variables make this
fairly confusing once we have too many in one test.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-04 10:40:59 -07:00
Zanie Blue
d726f09cf0
Fix PTH123 false positive when open is passed a file descriptor (#13616)
Closes https://github.com/astral-sh/ruff/issues/12871

Includes some minor semantic type inference extensions changes to help
with reliably detecting integers
2024-10-04 08:48:47 -05:00
Dhruv Manilawala
975be9c1c6
Bump version to 0.6.9 (#13624) 2024-10-04 18:51:13 +05:30
Zanie Blue
99e4566fce
Mark FURB118 fix as unsafe (#13613)
Closes https://github.com/astral-sh/ruff/issues/13421
2024-10-03 21:39:22 +00:00
Simon Høxbro Hansen
7ad07c2c5d
Add allow-unused-imports setting for unused-import rule (F401) (#13601)
## Summary
Resolves https://github.com/astral-sh/ruff/issues/9962 by allowing a
configuration setting `allowed-unused-imports`

TODO:
- [x] Figure out the correct name and place for the setting; currently,
I have added it top level.
- [x] The comparison is pretty naive. I tried using `glob::Pattern` but
couldn't get it to work in the configuration.
- [x] Add tests
- [x] Update documentations

## Test Plan

`cargo test`
2024-10-03 19:44:44 +00:00
Zanie Blue
cc1f766622
Preserve trivia (i.e. comments) in PLR5501 (#13573)
Closes https://github.com/astral-sh/ruff/issues/13545

As described in the issue, we move comments before the inner `if`
statement to before the newly constructed `elif` statement (previously
`else`).
2024-10-03 10:22:20 -05:00
cake-monotone
3728d5b3a2
[pyupgrade] Fix UP043 to apply to collections.abc.Generator and collections.abc.AsyncGenerator (#13611)
## Summary

fix #13602 

Currently, `UP043` only applies to typing.Generator, but it should also
support collections.abc.Generator.

This update ensures `UP043` correctly handles both
`collections.abc.Generator` and `collections.abc.AsyncGenerator`

### UP043
> `UP043`
> Python 3.13 introduced the ability for type parameters to specify
default values. As such, the default type arguments for some types in
the standard library (e.g., Generator, AsyncGenerator) are now optional.
> Omitting type parameters that match the default values can make the
code more concise and easier to read.

```py
Generator[int, None, None] -> Generator[int]
```
2024-10-03 13:06:15 +01:00
Dhruv Manilawala
7e3894f5b3
Avoid short circuiting B017 for multiple context managers (#13609)
## Summary

fixes: #13603
2024-10-03 15:35:05 +05:30
Charlie Marsh
c3b40da0d2
Use backticks for code in red-knot messages (#13599)
## Summary

...and remove periods from messages that don't span more than a single
sentence.

This is more consistent with how we present user-facing messages in uv
(which has a defined style guide).
2024-10-02 03:14:28 +00:00
Charlie Marsh
ef45185dbc
Allow users to provide custom diagnostic messages when unwrapping calls (#13597)
## Summary

You can now call `return_ty_result` to operate on a `Result` directly
thereby using your own diagnostics, as in:

```rust
return dunder_getitem_method
    .call(self.db, &[slice_ty])
    .return_ty_result(self.db, value.as_ref().into(), self)
    .unwrap_or_else(|err| {
        self.add_diagnostic(
            (&**value).into(),
            "call-non-callable",
            format_args!(
                "Method `__getitem__` is not callable on object of type '{}'.",
                value_ty.display(self.db),
            ),
        );
        err.return_ty()
    });
```
2024-10-01 21:22:13 +00:00
Charlie Marsh
961fc98344
Use __class_getitem__ for more specific non-subscript errors (#13596) 2024-10-01 18:16:00 +00:00
Charlie Marsh
0a6dc8e1b8
Support __getitem__ type inference for subscripts (#13579)
## Summary

Follow-up to https://github.com/astral-sh/ruff/pull/13562, to add
support for "arbitrary" subscript operations.
2024-10-01 18:04:16 +00:00
Charlie Marsh
8d54996ffb
Avoid indirection in class.__call__ lookup (#13595) 2024-10-01 18:01:36 +00:00
Alex Waygood
73e884b232
[red-knot] [minor] Improve helper methods for builtin types (#13594) 2024-10-01 18:38:33 +01:00
Charlie Marsh
edba60106b
Support classes that implement __call__ (#13580)
## Summary

This looked straightforward and removes some TODOs.
2024-10-01 17:15:46 +00:00
Alex Waygood
043fba7a57
[red-knot] Fix a few details around Type::call (#13593) 2024-10-01 16:49:09 +00:00
Alex Waygood
20d997784d
ruff_benchmark: open all tomllib files in the red-knot benchmark (#13589) 2024-10-01 17:47:36 +01:00
Alex Waygood
82324678cf
Rename the ruff_vendored crate to red_knot_vendored (#13586) 2024-10-01 16:16:59 +01:00
Zanie Blue
cfd5d63917
Use operator specific messaging in division by zero diagnostics (#13588)
Requested at
https://github.com/astral-sh/ruff/pull/13576#discussion_r1782530971
2024-10-01 08:58:38 -05:00
Alex Waygood
2a36b47f13
[red-knot] Remove Type::RevealType (#13567) 2024-10-01 10:01:03 +00:00
github-actions[bot]
360af1bc32
Sync vendored typeshed stubs (#13578)
Close and reopen this PR to trigger CI

Co-authored-by: typeshedbot <>
2024-10-01 08:05:19 +01:00
Zanie Blue
3af3f74c66
Update dedent_to to support blocks that are composed of comments (#13572)
While looking into https://github.com/astral-sh/ruff/issues/13545 I
noticed that we return `None` here if you pass a block of comments. This
is annoying because it causes `adjust_indentation` to fall back to
LibCST which panics when it cannot find a statement.
2024-10-01 04:38:03 +00:00
Zanie Blue
45f01e7872
Add diagnostic for integer division by zero (#13576)
Adds a diagnostic for division by the integer zero in `//`, `/`, and
`%`.

Doesn't handle `<int> / 0.0` because we don't track the values of float
literals.
2024-09-30 22:38:52 +00:00
Simon
6cdf996af6
[red-knot] feat: introduce a new [Type::Todo] variant (#13548)
This variant shows inference that is not yet implemented..

## Summary

PR #13500 reopened the idea of adding a new type variant to keep track
of not-implemented features in Red Knot.

It was based off of #12986 with a more generic approach of keeping track
of different kind of unknowns. Discussion in #13500 agreed that keeping
track of different `Unknown` is complicated for now, and this feature is
better achieved through a new variant of `Type`.

### Requirements

Requirements for this implementation can be summed up with some extracts
of comment from @carljm on the previous PR

> So at the moment we are leaning towards simplifying this PR to just
use a new top-level variant, which behaves like Any and Unknown but
represents inference that is not yet implemented in red-knot.

> I think the general rule should be that Todo should propagate only
when the presence of the input Todo caused the output to be unknown.
>
> To take a specific example, the inferred result of addition must be
Unknown if either operand is Unknown. That is, Unknown + X will always
be Unknown regardless of what X is. (Same for X + Unknown.) In this
case, I believe that Unknown + Todo (or Todo + Unknown) should result in
Unknown, not result in Todo. If we fix the upstream source of the Todo,
the result would still be Unknown, so it's not useful to propagate the
Todo in this case: it wrongly suggests that the output is unknown
because of a todo item.

## Test Plan

This PR does not introduce new tests, but it did required to edit some
tests with the display of `[Type::Todo]` (currently `@Todo`), which
suggests that those test are placeholders requirements for features we
don't support yet.
2024-09-30 14:28:06 -07:00
Zanie Blue
9d8a4c0057
Improve display of assert_public_ty assertion failures (#13577)
While working on https://github.com/astral-sh/ruff/pull/13576 I noticed
that it was really hard to tell which assertion failed in some of these
test cases. This could be expanded to elsewhere, but I've heard this
test suite format won't be around for long?
2024-09-30 16:12:26 -05:00
Charlie Marsh
c9c748a79e
Add some basic subscript type inference (#13562)
## Summary

Just for tuples and strings -- the easiest cases. I think most of the
rest require generic support?
2024-09-30 16:50:46 -04:00
Zanie Blue
32c746bd82
Fix inference when integers are divided (#13575)
Fixes the `Operator::Div` case and adds `Operator::FloorDiv` support

Closes https://github.com/astral-sh/ruff/issues/13570
2024-09-30 15:50:37 -05:00
Charlie Marsh
d86b73eb3d
Add unary inference for integer and boolean literals (#13559)
## Summary

Just trying to familiarize myself with the general patterns, testing,
etc.

Part of https://github.com/astral-sh/ruff/issues/12701.
2024-09-30 16:29:06 +00:00