Commit graph

10205 commits

Author SHA1 Message Date
Eric Mark Martin
e50fc049ab
[red-knot] visibility_constraint analysis for match cases (#17077)
## Summary

Add visibility constraint analysis for pattern predicate kinds
`Singleton`, `Or`, and `Class`.

## Test Plan

update conditional/match.md
2025-04-03 11:15:33 +00:00
Micha Reiser
66355a6185
[red-knot] Fix playground crashes when diagnostics are stale (#17165)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

Fixes a crash in the playground where it crashed with an "index out of
bounds" error in the `Diagnostic::to_range` call
after deleting content at the end of the file. 

The root cause was that the playground uses `useDeferred` to avoid too
frequent `checkFile` calls (to get a smoother UX).
However, this has the problem that the rendered `diagnostics` can be
stable (from before the last change).
Rendering the diagnostics can then fail because the `toRange` call
queries the latest content and not the content
from when the diagnostics were created.

The fix is "easy" in the sense that we now eagerly perform the `toRange`
calls. This way, it doesn't matter
when the diagnostics are stale for a few ms. 

This problem can only be observed on examples where Red Knot is "slow"
(takes more than ~16ms to check) because
only then does `useDeferred` "debounce" the `check` calls.
2025-04-03 10:18:36 +02:00
Dhruv Manilawala
177afabe18
[red-knot] Callable types are disjoint from literals (#17160)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

A callable type is disjoint from other literal types. For example,
`Type::StringLiteral` must be an instance of exactly `str`, not a
subclass of `str`, and `str` is not callable. The same applies to other
literal types.

This should hopefully fix #17144, I couldn't produce any failures after
running property tests multiple times.

## Test Plan

Add test cases for disjointness check between callable and other literal
types.

Run property tests multiple times.
2025-04-03 03:38:13 +05:30
Alex Waygood
28c68934a4
[red-knot] Fix inference for pow between two literal integers (#17161)
## Summary

Python `**` works differently to Rust `**`!

## Test Plan

Added an mdtest for various edge cases, and checked in the Python REPL
that we infer the correct type in all the new cases tested.
2025-04-02 21:25:57 +00:00
Alex Waygood
195bb433db
[red-knot] Add GitHub PR annotations when mdtests fail in CI (#17150)
## Summary

This PR adds a CI job that causes GitHub to add annotations to a PR diff
when mdtest assertions fail. For example:

<details>
<summary>Screenshot</summary>


![image](https://github.com/user-attachments/assets/bb2a649b-46ab-429d-a576-b36545940eaf)

</details>

## Motivation

Debugging mdtest failures locally is currently a really nice experience:
- Errors are displayed with pretty colours, which makes them much more
readable
- If you run the test from inside an IDE, you can CTRL-click on a path
and jump directly to the line that had the failing assertion
- If you use
[`mdtest.py`](https://github.com/astral-sh/ruff/blob/main/crates/red_knot_python_semantic/mdtest.py),
you don't even need to recompile anything after changing an assertion in
an mdtest, amd the test results instantly live-update with each change
to the MarkDown file

Debugging mdtest failures in CI is much more unpleasant, however.
Sometimes an error message is just

> [static-assert-error] Argument evaluates to `False`

...which doesn't tell you very much unless you navigate to the line in
question that has the failing mdtest assertion. The line in question
might not even be touched by the PR, and even if it is, it can be hard
to find the line if the PR touches many files. Unlike locally, you can't
click on the error and jump straight to the line that contains the
failing assertion. You also don't get colourised output in CI
(https://github.com/astral-sh/ruff/issues/13939).

GitHub PR annotations should make it really easy to debug why mdtests
are failing on PRs, making PR review much easier.

## Test Plan

I opened a PR to my fork
[here](https://github.com/AlexWaygood/ruff/pull/11/files) with some
bogus changes to an mdtest to show what it looks like when there are
failures in CI and this job has been added. Scroll down to
`crates/red_knot_python_semantic/resources/mdtest/type_properties/is_equivalent_to.md`
on the "files changed" tab for that PR to see the annotations.
2025-04-02 21:51:52 +01:00
Alex Waygood
c2bb5d5250
[red-knot] Fix equivalence of differently ordered unions that contain Callable types (#17145)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/17058.

Equivalent callable types were not understood as equivalent when they
appeared nested inside unions and intersections. This PR fixes that by
ensuring that `Callable` elements nested inside unions, intersections
and tuples have their representations normalized before one union type
is compared with another for equivalence, or before one intersection
type is compared with another for equivalence.

The normalizations applied to a `Callable` type are:
- the type of the default value is stripped from all parameters (only
whether the parameter _has_ a default value is relevant to whether one
`Callable` type is equivalent to another)
- The names of the parameters are stripped from positional-only
parameters, variadic parameters and keyword-variadic parameters
- Unions and intersections that are present (top-level or nested) inside
parameter annotations or return annotations are normalized.

Adding a `CallableType::normalized()` method also allows us to simplify
the implementation of `CallableType::is_equivalent_to()`.

### Should these normalizations be done eagerly as part of a
`CallableType` constructor?

I considered this. It's something that we could still consider doing in
the future; this PR doesn't rule it out as a possibility. However, I
didn't pursue it for now, for several reasons:
1. Our current `Display` implementation doesn't handle well the
possibility that a parameter might not have a name or an annotated type.
Callable types with parameters like this would be displayed as follows:
   ```py
   (, ,) -> None: ...
   ```

That's fixable! It could easily become something like `(Unknown,
Unknown) -> None: ...`. But it also illustrates that we probably want to
retain the parameter names when displaying the signature of a `lambda`
function if you're hovering over a reference to the lambda in an IDE.
Currently we don't have a `LambdaType` struct for representing `lambda`
functions; if we wanted to eagerly normalize signatures when creating
`CallableType`s, we'd probably have to add a `LambdaType` struct so that
we would retain the full signature of a `lambda` function, rather than
representing it as an eagerly simplified `CallableType`.
2. In order to ensure that it's impossible to create `CallableType`s
without the parameters being normalized, I'd either have to create an
alternative `SimplifiedSignature` struct (which would duplicate a lot of
code), or move `CallableType` to a new module so that the only way of
constructing a `CallableType` instance would be via a constructor method
that performs the normalizations eagerly on the callable's signature.
Again, this isn't a dealbreaker, and I think it's still an option, but
it would be a lot of churn, and it didn't seem necessary for now. Doing
it this way, at least to start with, felt like it would create a diff
that's easier to review and felt like it would create fewer merge
conflicts for others.

## Test Plan

- Added a regression mdtest for
https://github.com/astral-sh/ruff/issues/17058
- Ran `QUICKCHECK_TESTS=1000000 cargo test --release -p
red_knot_python_semantic -- --ignored types::property_tests::stable`
2025-04-02 17:43:34 +00:00
David Peter
cb7dae1e96
[red-knot] Add initial set of tests for unreachable code (#17159)
## Summary

Add an initial set of tests that will eventually document our behavior
around unreachable code. In the last section of this suite, I argue why
we should never type check unreachable sections and never emit any
diagnostics in these sections.
2025-04-02 19:39:44 +02:00
Wei Lee
8833484b10
[airflow] Move AIR302 to AIR301 and AIR303 to AIR302 (#17151)
## Summary

Following up the discussion in
https://github.com/astral-sh/ruff/issues/14626#issuecomment-2766548545,
we're to reorganize airflow rules. Before this discussion happens, we
combine required changes and suggested changes in to one single error
code.

This PR first rename the original error code to the new error code as we
discussed. We will gradually extract suggested changes out of AIR301 and
AIR302 to AIR311 and AIR312 in the following PRs

## Test Plan

Except for file, error code rename, the test case should work as it used
to be.
2025-04-02 23:01:31 +05:30
Andrew Gallant
adeba3dca7 ruff_db: simplify lifetimes on DiagnosticDisplay
I initially split the lifetime out into three distinct lifetimes on
near-instinct because I moved the struct into the public API. But
because they are all shared borrows, and because there are no other APIs
on `DisplayDiagnostic` to access individual fields (and probably never
will be), it's probably fine to just specify one lifetime. Because of
subtyping, the one lifetime will be the shorter of the three.

There's also the point that `ruff_db` isn't _really_ a public API, since
it isn't a library that others depend on. So my instinct is probably a
bit off there.
2025-04-02 12:47:02 -04:00
David Peter
af988bf866
[red-knot] Detect division-by-zero in unions and intersections (#17157)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

With this PR, we emit a diagnostic for this case where
previously didn't:
```py
from typing import Literal

def f(m: int, n: Literal[-1, 0, 1]):
    # error: [division-by-zero] "Cannot divide object of type `int` by zero"
    return m / n
```

## Test Plan

New Markdown test
2025-04-02 18:21:27 +02:00
Wei Lee
f989c2c3af
[airflow] Add autofix infrastructure to AIR302 name checks (#16965)
## Summary

Add autofix infrastructure to `AIR302` name checks and use this logic to 
fix`"airflow", "api_connexion", "security", "requires_access_dataset"`, `"airflow", "Dataset"` and `"airflow",
"datasets", "Dataset"`

## Test Plan

The existing test fixture reflects the update
2025-04-02 15:27:51 +00:00
trag1c
c2512b4c50
[flake8-bandit] Mark str and list[str] literals as trusted input (S603) (#17136)
## Summary

Closes #17112. Allows passing in string and list-of-strings literals
into `subprocess.run` (and related) calls without marking them as
untrusted input:
```py
import subprocess

subprocess.run("true")

# "instant" named expressions are also allowed
subprocess.run(c := "ls")
```

## Test Plan

Added test cases covering new behavior, passed with `cargo nextest run`.
2025-04-02 11:22:37 -04:00
Wei Lee
6bc2b04c49
[airflow] Add autofix for AIR302 attribute checks (#16977)
## Summary

Add autofix logic to AIR302 check_method

## Test Plan

test fixtures have been updated accordingly
2025-04-02 15:18:24 +00:00
Wei Lee
fc2a0950eb
[airflow] Extend AIR302 with additional symbols (#17085)
## Summary

* ``airflow.auth.managers.base_auth_manager.is_authorized_dataset`` has
been moved to
``airflow.api_fastapi.auth.managers.base_auth_manager.is_authorized_asset``
in Airflow 3.0
* ``airflow.auth.managers.models.resource_details.DatasetDetails`` has
been moved to
``airflow.api_fastapi.auth.managers.models.resource_details.AssetDetails``
in Airflow 3.0
* Dag arguments `default_view` and `orientation` has been removed in
Airflow 3.0
* `airflow.models.baseoperatorlink.BaseOperatorLink` has been moved to
`airflow.sdk.definitions.baseoperatorlink.BaseOperatorLink` in Airflow
3.0
* ``airflow.notifications.basenotifier.BaseNotifier`` has been moved to
``airflow.sdk.BaseNotifier`` in Airflow 3.0
* ``airflow.utils.log.secrets_masker`` has been moved to
``airflow.sdk.execution_time.secrets_masker`` in Airflow 3.0
* ``airflow...DAG.allow_future_exec_dates`` has been removed in Airflow
3.0
* `airflow.utils.db.create_session` has een removed in Airflow 3.0
* `airflow.sensors.base_sensor_operator.BaseSensorOperator` has been
moved to `airflow.sdk.bases.sensor.BaseSensorOperator` removed Airflow
3.0
* `airflow.utils.file.TemporaryDirectory` has been removed in Airflow
3.0 and can be replaced by `tempfile.TemporaryDirectory`
* `airflow.utils.file.mkdirs` has been removed in Airflow 3.0 and can be
replaced by `pathlib.Path({path}).mkdir`

## Test Plan

Test fixture has been added for these changes
2025-04-02 20:38:52 +05:30
Wei Lee
33bd08f49b
[airflow] Move AIR301 to AIR002 (#16978)
## Summary

Unlike other AIR3XX rules, this best practice can be applied to Airflow
1 and Airflow 2 as well. Thus, we think it might make sense for use to
move it to AIR002 so that the first number of the error align to Airflow
version as possible to reduce confusion

## Test Plan

the test fixture has been updated
2025-04-02 20:37:35 +05:30
Wei Lee
5d57788328
[airflow] Add autofix for AIR302 method checks (#16976)
## Summary

Add autofix logic to `AIR302` method checks

## Test Plan

Test fixtures have been updated accordingly
2025-04-02 20:35:49 +05:30
Andrew Gallant
718b0cadf4 ruff_db: switch diagnostic rendering over to std::fmt::Display
It was already using this approach internally, so this is "just" a
matter of rejiggering the public API of `Diagnostic`.

We were previously writing directly to a `std::io::Write` since it was
thought that this worked better with the linear typing fakery. Namely,
it increased confidence that the diagnostic rendering was actually
written somewhere useful, instead of just being converted to a string
that could potentially get lost.

For reasons discussed in #17130, the linear type fakery was removed.
And so there is less of a reason to require a `std::io::Write`
implementation for diagnostic rendering. Indeed, this would sometimes
result in `unwrap()` calls when one wants to convert to a `String`.
2025-04-02 11:01:16 -04:00
Micha Reiser
24498e383d
[red-knot] Add 'Goto type definition' to the playground (#17055)
## Summary

This PR adds Goto type definition to the playground, using the same
infrastructure as the LSP.


The main *challenge* with implementing this feature was that the editor
can now participate in which tab is open.

## Known limitations

The same as for the LSP. Most notably, navigating to types defined in
typeshed isn't supported.

## Test Plan


https://github.com/user-attachments/assets/22dad7c8-7ac7-463f-b066-5d5b2c45d1fe
2025-04-02 16:35:31 +02:00
Andrew Gallant
28c7e724e3 red_knot_ide: update snapshots
This just adds an extra blank line. I think these tests were written
against the new renderer before it was used by Red Knot's `main`
function. Once I did that, I saw that it was missing a blank line, and
so I added it to match the status quo. But that means these snapshots
have become stale. So this commit updates them.
2025-04-02 10:10:01 -04:00
Andrew Gallant
185bcfef1a red_knot_python_semantic: remove comment about TypeCheckDiagnostic
I put this in its own commit in case all of the information removed here
was controversial. But it *looks* stale to me. At the very least,
`TypeCheckDiagnostic` no longer exists, so that would need to be fixed.
And it doesn't really make sense to me (at this point) to make
`Diagnostic` a Salsa struct, particularly since we are keen on using it
in Ruff (at some point).
2025-04-02 10:10:01 -04:00
Andrew Gallant
c30e80a3f4 ruff_db: delete most of the old diagnostic code
We do keep around `OldSecondaryDiagnosticMessage`, since that's part of
the Red Knot `InferContext` API. But it's a rather simple type, and
we'll be able to delete it entirely once `InferContext` exposes the new
`Diagnostic` type directly.

Since we aren't consuming `OldSecondaryDiagnosticMessage` any more, we
can now accept a slice instead of a vec. (Thanks Clippy.)
2025-04-02 10:10:01 -04:00
Andrew Gallant
4e169e5f6c red_knot: use Diagnostic inside of red knot
This replaces things like `TypeCheckDiagnostic` with the new Diagnostic`
type.

This is a "surgical" replacement where we retain the existing API of
of diagnostic reporting such that _most_ of Red Knot doesn't need to be
changed to support this update. But it will enable us to start using the
new diagnostic renderer and to delete the old renderer. It also paves
the path for exposing the new `Diagnostic` data model to the broader Red
Knot codebase.
2025-04-02 10:10:01 -04:00
Andrew Gallant
883b8e3870 ruff_db: port concise diagnostic rendering to new renderer
Previously, this was only available in the old renderer.
To avoid regressions, we just copy it to the new renderer.
We don't bother with DRY because the old renderer will be
deleted very soon.
2025-04-02 10:10:01 -04:00
Andrew Gallant
2ca2f73ba8 ruff_db: tweak line terminators emitted by diagnostic rendering
This change just brings diagnostic rendering into parity
with the status quo.
2025-04-02 10:10:01 -04:00
Andrew Gallant
90f0766210 ruff_db: make Diagnostic::print use a non-mutable borrow
Now that we don't need to update the `printed` flag, this can just be an
immutable borrow.

(Arguably this should have been an immutable borrow even initially, but
I didn't want to introduce interior mutability without a more compelling
justification.)
2025-04-02 10:10:01 -04:00
Andrew Gallant
a9527edbbe ruff_db: switch Diagnostic to use Arc, drop linear type fakery
The switch to `Arc` was done because Salsa sometimes requires cloning a
`Diagnostic` (or something that contains a `Diagnostic`). And so it
probably makes sense to make this cheap.

Since `Diagnostic` exposes a mutable API, we adopt "clone on write"
semantics. Although, it's more like, "clone on write when the `Arc` has
more than one reference." In the common case of creating a `Diagnostic`
and then immediately mutating it, no additional copies should be made
over the status quo.

We also drop the linear type fakery. Its interaction with Salsa is
somewhat awkward, and it has been suggested that there will be points
where diagnostics will be dropped unceremoniously without an opportunity
to tag them as having been ignored. Moreover, this machinery was added
out of "good sense" and isn't actually motivated by real world problems
with accidentally ignoring diagnostics. So that makes it easier, I
think, to just kick this out entirely instead of trying to find a way to
make it work.
2025-04-02 10:10:01 -04:00
Andrew Gallant
57be814acb ruff_db: add method to create sub-diagnostics from old secondary messages
This is temporary to scaffold the refactor.

The main idea is that we want to take the `InferContext` API,
*as it is*, and migrate that to the new diagnostic data model
*internally*. Then we can rip out the old stuff and iterate
on the API.
2025-04-02 10:10:01 -04:00
Andrew Gallant
a8a18e7171 red_knot_python_semantic: remove WithDiagnostic trait
I did this mostly because it wasn't buying us much, and I'm
trying to simplify the public API of the types I'd like to
refactor in order to make the refactor simpler.

If we really want something like this, we can re-add it
later.
2025-04-02 10:10:01 -04:00
Andrew Gallant
a953373892 red_knot_python_semantic: remove Deref impl on TypeCheckDiagnostics
I removed this to see how much code was depending internally on the
`&[Arc<TypeCheckDiagnostic>]` representation. Thankfully, it was just
one place. So I just removed the `Deref` impl in favor of adding an
explicit `iter` method.

In general, I think using `Deref` for things like this is _somewhat_ of
an abuse. The tip-off is if there are `&self` or `&mut self` methods on
the type, then it's probably not a good candidate for `Deref`.
2025-04-02 10:10:01 -04:00
Brent Westbrook
d382065f8a
[syntax-errors] Reimplement PLE0118 (#17135)
Summary
--

This PR reimplements
[load-before-global-declaration
(PLE0118)](https://docs.astral.sh/ruff/rules/load-before-global-declaration/)
as a semantic syntax error.

I added a `global` method to the `SemanticSyntaxContext` trait to make
this very easy, at least in ruff. Does red-knot have something similar?

If this approach will also work in red-knot, I think some of the other
PLE rules are also compile-time errors in CPython, PLE0117 in
particular. 0115 and 0116 also mention `SyntaxError`s in their docs, but
I haven't confirmed them in the REPL yet.

Test Plan
--

Existing linter tests for PLE0118. I think this actually can't be tested
very easily in an inline test because the `TestContext` doesn't have a
real way to track globals.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-04-02 13:03:44 +00:00
Brent Westbrook
d45593288f
[syntax-errors] Starred expressions in return, yield, and for (#17134)
Summary
--

Fixes https://github.com/astral-sh/ruff/issues/16520 by flagging single,
starred expressions in `return`, `yield`, and
`for` statements.

I thought `yield from` would also be included here, but that error is
emitted by
the CPython parser:

```pycon
>>> ast.parse("def f(): yield from *x")
Traceback (most recent call last):
  File "<python-input-214>", line 1, in <module>
    ast.parse("def f(): yield from *x")
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/ast.py", line 54, in parse
    return compile(source, filename, mode, flags,
                   _feature_version=feature_version, optimize=optimize)
  File "<unknown>", line 1
    def f(): yield from *x
                        ^
SyntaxError: invalid syntax
```

And we also already catch it in our parser.

Test Plan
--

New inline tests and updates to existing tests.
2025-04-02 08:38:25 -04:00
Micha Reiser
2ae39edccf
[red-knot] Goto type definition (#16901)
## Summary

Implement basic *Goto type definition* support for Red Knot's LSP.

This PR also builds the foundation for other LSP operations. E.g., Goto
definition, hover, etc., should be able to reuse some, if not most,
logic introduced in this PR.

The basic steps of resolving the type definitions are:

1. Find the closest token for the cursor offset. This is a bit more
subtle than I first anticipated because the cursor could be positioned
right between the callee and the `(` in `call(test)`, in which case we
want to resolve the type for `call`.
2. Find the node with the minimal range that fully encloses the token
found in 1. I somewhat suspect that 1 and 2 could be done at the same
time but it complicated things because we also need to compute the spine
(ancestor chain) for the node and there's no guarantee that the found
nodes have the same ancestors
3. Reduce the node found in 2. to a node that is a valid goto target.
This may require traversing upwards to e.g. find the closest expression.
4. Resolve the type for the goto target
5. Resolve the location for the type, return it to the LSP

## Design decisions

The current implementation navigates to the inferred type. I think this
is what we want because it means that it correctly accounts for
narrowing (in which case we want to go to the narrowed type because
that's the value's type at the given position). However, it does have
the downside that Goto type definition doesn't work whenever we infer `T
& Unknown` because intersection types aren't supported. I'm not sure
what to do about this specific case, other than maybe ignoring `Unkown`
in Goto type definition if the type is an intersection?

## Known limitations

* Types defined in the vendored typeshed aren't supported because the
client can't open files from the red knot binary (we can either
implement our own file protocol and handler OR extract the typeshed
files and point there). See
https://github.com/astral-sh/ruff/issues/17041
* Red Knot only exposes an API to get types for expressions and
definitions. However, there are many other nodes with identifiers that
can have a type (e.g. go to type of a globals statement, match patterns,
...). We can add support for those in separate PRs (after we figure out
how to query the types from the semantic model). See
https://github.com/astral-sh/ruff/issues/17113
* We should have a higher-level API for the LSP that doesn't directly
call semantic queries. I intentionally decided not to design that API
just yet.


## Test plan


https://github.com/user-attachments/assets/fa077297-a42d-4ec8-b71f-90c0802b4edb

Goto type definition on a union

<img width="1215" alt="Screenshot 2025-04-01 at 13 02 55"
src="https://github.com/user-attachments/assets/689cabcc-4a86-4a18-b14a-c56f56868085"
/>



Note: I recorded this using a custom typeshed path so that navigating to
builtins works.
2025-04-02 12:12:48 +00:00
cake-monotone
7e97910704
[red-knot] Fix _NotImplementedType check for Python >=3.10 (#17143)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

from https://github.com/astral-sh/ruff/pull/17034#discussion_r2024222525

This is a simple PR to fix the invalid behavior of `NotImplemented` on
Python >=3.10.


## Test Plan

I think it would be better if we could run mdtest across multiple Python
versions in GitHub Actions.

<!-- How was it tested? -->

---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
2025-04-02 10:02:59 +00:00
David Peter
ae2cf91a36
[red-knot] Decorators and properties (#17017)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

Add support for decorators on function as well as support
for properties by adding special handling for `@property` and `@<name of
property>.setter`/`.getter` decorators.

closes https://github.com/astral-sh/ruff/issues/16987

## Ecosystem results

- ✔️ A lot of false positives are fixed by our new
understanding of properties
- 🔴 A bunch of new false positives (typically
`possibly-unbound-attribute` or `invalid-argument-type`) occur because
we currently do not perform type narrowing on attributes. And with the
new understanding of properties, this becomes even more relevant. In
many cases, the narrowing occurs through an assertion, so this is also
something that we need to implement to get rid of these false positives.
- 🔴 A few new false positives occur because we do not
understand generics, and therefore some calls to custom setters fail.
- 🔴 Similarly, some false positives occur because we do not
understand protocols yet.
- ✔️ Seems like a true positive to me. [The
setter](e624d8edfa/src/packaging/specifiers.py (L752-L754))
only accepts `bools`, but `None` is assigned in [this
line](e624d8edfa/tests/test_specifiers.py (L688)).
  ```
+ error[lint:invalid-assignment]
/tmp/mypy_primer/projects/packaging/tests/test_specifiers.py:688:9:
Invalid assignment to data descriptor attribute `prereleases` on type
`SpecifierSet` with custom `__set__` method
  ```
- ✔️ This is arguable also a true positive. The setter
[here](0c6c75644f/rich/table.py (L359-L363))
returns `Table`, but typeshed wants [setters to return
`None`](bf8d2a9912/stdlib/builtins.pyi (L1298)).
  ```
+ error[lint:invalid-argument-type]
/tmp/mypy_primer/projects/rich/rich/table.py:359:5: Object of type
`Literal[padding]` cannot be assigned to parameter 2 (`fset`) of bound
method `setter`; expected type `(Any, Any, /) -> None`
  ```  

## Follow ups

- Fix the `@no_type_check` regression
- Implement class decorators

## Test Plan

New Markdown test suites for decorators and properties.
2025-04-02 09:27:46 +02:00
Mohammad Amin Ghasemi
e1b5b0de71
[flake8-import-conventions] Add import numpy.typing as npt to default flake8-import-conventions.aliases (#17133)
## Summary
Adds import `numpy.typing as npt` to `default in
flake8-import-conventions.aliases`
Resolves #17028

## Test Plan
Manually ran local ruff on the altered fixture and also ran `cargo test`
2025-04-02 09:25:46 +02:00
Dhruv Manilawala
f63024843c
[red-knot] Move tuple containing Never tests (#17137)
Refer
https://github.com/astral-sh/ruff/pull/17094#discussion_r2023682840
2025-04-02 02:46:18 +00:00
Matthew Mckee
eb3e176309
[red-knot] Add callable subtyping for callable instances and bound methods (#17105)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

Trying to improve #17005
Partially fixes #16953

## Test Plan

Update is_subtype_of.md

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-04-01 23:40:44 +00:00
Dhruv Manilawala
d38f6fcc55
[red-knot] Add property tests for callable types (#17006)
## Summary

Part of #15382, this PR adds property tests for callable types.

Specifically, this PR updates the property tests to generate an
arbitrary signature for a general callable type which includes:
* Arbitrary combination of parameter kinds in the correct order
* Arbitrary number of parameters
* Arbitrary optional types for annotation and return type
* Arbitrary parameter names (no duplicate names), optional for
positional-only parameters

## Test Plan

```
QUICKCHECK_TESTS=100000 cargo test -p red_knot_python_semantic -- --ignored types::property_tests::stable
```

Also, the commands in CI:


d72b4100a3/.github/workflows/daily_property_tests.yaml (L47-L52)
2025-04-02 01:07:42 +05:30
Dhruv Manilawala
6be0a5057d
[red-knot] Disjointness for callable types (#17094)
## Summary

Part of #15382, this PR adds support for disjointness between two
callable types. They are never disjoint because there exists a callable
type that's a subtype of all other callable types:
```py
(*args: object, **kwargs: object) -> Never
```

The `Never` is a subtype of every fully static type thus a callable type
that has the return type of `Never` means that it is a subtype of every
return type.

## Test Plan

Add test cases related to mixed parameter kinds, gradual form (`...`)
and `Never` type.
2025-04-01 19:00:27 +00:00
Alex Waygood
d6dcc377f7
[red-knot] Flatten Type::Callable into four Type variants (#17126)
## Summary

Currently our `Type::Callable` wraps a four-variant `CallableType` enum.
But as time has gone on, I think we've found that the four variants in
`CallableType` are really more different to each other than they are
similar to each other:
- `GeneralCallableType` is a structural type describing all callable
types with a certain signature, but the other three types are "literal
types", more similar to the `FunctionLiteral` variant
- `GeneralCallableType` is not a singleton or a single-valued type, but
the other three are all single-valued types
(`WrapperDescriptorDunderGet` is even a singleton type)
- `GeneralCallableType` has (or should have) ambiguous truthiness, but
all possible inhabitants of the other three types are always truthy.
- As a structural type, `GeneralCallableType` can contain inner unions
and intersections that must be sorted in some contexts in our internal
model, but this is not true for the other three variants.

This PR flattens `Type::Callable` into four distinct `Type::` variants.
In the process, it fixes a number of latent bugs that were concealed by
the current architecture but are laid bare by the refactor. Unit tests
for these bugs are included in the PR.
2025-04-01 19:30:06 +01:00
Alex Waygood
a43b683d08
mdtest.py: do a full mdtest run immediately when the script is executed (#17128)
## Summary

Currently if I run `uv run crates/red_knot_python_semantic/mdtest.py`
from the Ruff repo root, I get this output:

```
~/dev/ruff (main) % uv run crates/red_knot_python_semantic/mdtest.py
Ready to watch for changes...
```

...And I then have to make some spurious whitespace changes or something
to a test file in order to get the script to actually run mdtest. This
PR changes mdtest.py so that it does an initial run of all mdtests when
you invoke the script, and _then_ starts watching for changes in test
files/Rust code.
2025-04-01 19:27:55 +01:00
Dhruv Manilawala
d29d4956de
[red-knot] Fix callable subtyping for standard parameters (#17125)
## Summary

This PR fixes a bug in callable subtyping to consider both the
positional and keyword form of the standard parameter in the supertype
when matching against variadic, keyword-only and keyword-variadic
parameter in the subtype.

This is done by collecting the unmatched standard parameters and then
checking them against the keyword-only / keyword-variadic parameters
after the positional loop.

## Test Plan

Add test cases.
2025-04-01 23:37:35 +05:30
Alex Waygood
c74ba00219
[red-knot] Fix more redundant-cast false positives (#17119)
## Summary

There are quite a few places we infer `Todo` types currently, and some
of them are nested somewhat deeply in type expressions. These can cause
spurious issues for the new `redundant-cast` diagnostics. We fixed all
the false positives we saw in the mypy_primer report before merging
https://github.com/astral-sh/ruff/pull/17100, but I think there are
still lots of places where we'd emit false positives due to this check
-- we currently don't run on that many projects at all in our
mypy_primer check:


d0c8eaa092/.github/workflows/mypy_primer.yaml (L71)

This PR fixes some more false positives from this diagnostic by making
the `Type::contains_todo()` method more expansive.

## Test Plan

I added a regression test which causes us to emit a spurious diagnostic
on `main`, but does not with this PR.
2025-04-01 19:03:42 +01:00
github-actions[bot]
a15404a5c1
Sync vendored typeshed stubs (#17106)
Close and reopen this PR to trigger CI

Co-authored-by: typeshedbot <>
2025-04-01 17:44:27 +01:00
Carl Meyer
3f63c08728
[red-knot] support Any as a class in typeshed (#17107)
## Summary

In https://github.com/python/typeshed/pull/13520 the typeshed definition
of `typing.Any` was changed from `Any = object()` to `class Any: ...`.
Our automated typeshed updater pulled down this change in
https://github.com/astral-sh/ruff/pull/17106, with the consequence that
we no longer understand `Any`, which is... not good.

This PR gives us the ability to understand `Any` defined as a class
instead of `object()`. It doesn't remove our ability to understand the
old form. Perhaps at some point we'll want to remove it, but for now we
may as well support both old and new typeshed?

This also directly patches typeshed to use the new form of `Any`; this
is purely to work around our tests that no known class is inferred as
`Unknown`, which otherwise fail with the old typeshed and the changes in
this PR. (All other tests pass.) This patch to typeshed will shortly be
subsumed by https://github.com/astral-sh/ruff/pull/17106 anyway.

## Test Plan

Without the typeshed change in this PR, all tests pass except for the
two `known_class_doesnt_fallback_to_unknown_unexpectedly_*` tests (so we
still support the old form of defining `Any`). With the typeshed change
in this PR, all tests pass, so we now support the new form in a way that
is indistinguishable to our test suite from the old form. And
indistinguishable to the ecosystem check: after rebasing
https://github.com/astral-sh/ruff/pull/17106 on this PR, there's zero
ecosystem impact.
2025-04-01 16:38:23 +00:00
Micha Reiser
5a876ed25e
Visit Identifier node as part of the SourceOrderVisitor (#17110)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

I don't remember exactly when we made `Identifier` a node but it is now
considered a node (it implements `AnyNodeRef`, it has a range). However,
we never updated
the `SourceOrderVisitor` to visit identifiers because we never had a use
case for it and visiting new nodes can change how the formatter
associates comments (breaking change!).
This PR updates the `SourceOrderVisitor` to visit identifiers and
changes the formatter comment visitor to skip identifiers (updating the
visitor might be desired because it could help simplifying some comment
placement logic but this is out of scope for this PR).

## Test Plan

Tests, updated snapshot tests
2025-04-01 16:58:09 +02:00
Alex Waygood
49c25993eb
[red-knot] Don't infer Todo for quite so many tuple type expressions (#17116)
## Summary

I noticed we were inferring `Todo` as the declared type for annotations
such as `x: tuple[list[int], list[int]]`. This PR reworks our annotation
parsing so that we instead infer `tuple[Todo, Todo]` for this
annotation, which is quite a bit more precise.

## Test Plan

Existing mdtest updated.
2025-04-01 15:44:02 +01:00
Aarni Koskela
8e6a83b33e
CI: Run pre-commit on depot machine (#17120)
## Summary

Sibling/alternate of #17108 (see
https://github.com/astral-sh/ruff/pull/17108#issuecomment-2769200896).

See if running the pre-commit CI step on Depot machines makes
WASM-compiled shellcheck faster.

## Test Plan

> How was it tested?

We're doing it live!
2025-04-01 14:22:02 +01:00
Max Mynter
d0c8eaa092
Error instead of panic! when running Ruff from a deleted directory (#16903) (#17054)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
Closes #16903

## Summary
Check if the current working directory exist. If not, provide an error
instead of panicking.

Fixed a stale comment in `resolve_default_files`.

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan
I added a test to the `resolve_files.rs`. 

Manual testing follows steps of #16903 :
- Terminal 1
```bash
mkdir tmp
cd tmp
```
- Terminal 2
```bash
rm -rf tmp
```
- Terminal 1
```bash
ruff check
```

## Open Issues / Questions to Reviewer
All tests pass when executed with `cargo nextest run`.

However, with `cargo test` the parallelization makes the other tests
fail as we change the `pwd`.

Serial execution with `cargo test` seems to require [another dependency
or some
workarounds](https://stackoverflow.com/questions/51694017/how-can-i-avoid-running-some-tests-in-parallel).

Do you think an additional dependency or test complexity is worth
testing this small edge case, do you have another implementation idea,
or should i rather remove the test?
  
---
P.S.: I'm currently participating in a batch at the [Recurse
Center](https://www.recurse.com/) and would love to contribute more for
the next six weeks to improve my Rust. Let me know if you're open to
mentoring/reviewing and/or if you have specific areas where help would
be most valued.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-04-01 14:17:07 +02:00
Dylan
aa93005d8d
Control flow graph: setup (#17064)
This PR contains the scaffolding for a new control flow graph
implementation, along with its application to the `unreachable` rule. At
the moment, the implementation is a maximal over-approximation: no
control flow is modeled and all statements are counted as reachable.
With each additional statement type we support, this approximation will
improve.

So this PR just contains:
- A `ControlFlowGraph` struct and builder
- Support for printing the flow graph as a Mermaid graph
- Snapshot tests for the actual graphs
- (a very bad!) reimplementation of `unreachable` using the new structs
- Snapshot tests for `unreachable`

# Instructions for Viewing Mermaid snapshots
Unfortunately I don't know how to convince GitHub to render the Mermaid
graphs in the snapshots. However, you can view these locally in VSCode
if you install an extension that supports Mermaid graphs in Markdown,
and then add this to your `settings.json`:

```json
  "files.associations": {
"*.md.snap": "markdown",
  }
  ```
2025-04-01 05:53:42 -05:00