Commit graph

6371 commits

Author SHA1 Message Date
Douglas Creager
ff376fc262
[red-knot] Allow explicit specialization of generic classes (#17023)
This PR lets you explicitly specialize a generic class using a subscript
expression. It introduces three new Rust types for representing classes:

- `NonGenericClass`
- `GenericClass` (not specialized)
- `GenericAlias` (specialized)

and two enum wrappers:

- `ClassType` (a non-generic class or generic alias, represents a class
_type_ at runtime)
- `ClassLiteralType` (a non-generic class or generic class, represents a
class body in the AST)

We also add internal support for specializing callables, in particular
function literals. (That is, the internal `Type` representation now
attaches an optional specialization to a function literal.) This is used
in this PR for the methods of a generic class, but should also give us
most of what we need for specializing generic _functions_ (which this PR
does not yet tackle).

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-04-09 11:18:46 -04:00
Wei Lee
7c81408c54
[airflow] Refactor AIR301 logic and fix typos (AIR301) (#17293)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

* Simplify match conditions in AIR301
* Fix
* `airflow.datasets.manager.DatasetManager` →
`airflow.assets.manager.AssetManager`
* `airflow.www.auth.has_access_dataset` →
`airflow.www.auth.has_access_dataset`

## Test Plan

<!-- How was it tested? -->
The test fixture has been updated accordingly
2025-04-09 10:46:17 -04:00
Wei Lee
7207c86971
[airflow] Extract AIR312 from AIR302 rules (AIR302, AIR312) (#17152)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

As discussed in
https://github.com/astral-sh/ruff/issues/14626#issuecomment-2766146129,
we're to separate suggested changes from required changes.

The following symbols has been moved to AIR312 from AIR302. They still
work in Airflow 3.0, but they're suggested to be changed as they're
expected to be removed in future version

```python
from airflow.hooks.filesystem import FSHook
from airflow.hooks.package_index import PackageIndexHook
from airflow.hooks.subprocess import (SubprocessHook, SubprocessResult, working_directory)
from airflow.operators.bash import BashOperator
from airflow.operators.datetime import BranchDateTimeOperator, target_times_as_dates
from airflow.operators.trigger_dagrun import TriggerDagRunLink, TriggerDagRunOperator
from airflow.operators.empty import EmptyOperator
from airflow.operators.latest_only import LatestOnlyOperator
from airflow.operators.python import (BranchPythonOperator, PythonOperator, PythonVirtualenvOperator, ShortCircuitOperator)
from airflow.operators.weekday import BranchDayOfWeekOperator
from airflow.sensors.date_time import DateTimeSensor, DateTimeSensorAsync
from airflow.sensors.external_task import ExternalTaskMarker, ExternalTaskSensor, ExternalTaskSensorLink
from airflow.sensors.filesystem import FileSensor
from airflow.sensors.time_sensor import TimeSensor, TimeSensorAsync
from airflow.sensors.time_delta import TimeDeltaSensor, TimeDeltaSensorAsync, WaitSensor
from airflow.sensors.weekday import DayOfWeekSensor
from airflow.triggers.external_task import DagStateTrigger, WorkflowTrigger
from airflow.triggers.file import FileTrigger
from airflow.triggers.temporal import DateTimeTrigger, TimeDeltaTrigger
```

## Test Plan

<!-- How was it tested? -->

The test fixture has been updated acccordingly

---------

Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
2025-04-09 10:43:07 -04:00
Alex Waygood
6ec4c6a97e
[red-knot] Improve handling of visibility constraints in external modules when resolving * imports (#17286) 2025-04-09 14:36:52 +00:00
Alex Waygood
f1ba596f22
[red-knot] Add more tests for * imports (#17315) 2025-04-09 15:10:30 +01:00
Micha Reiser
8249a72412
[red-knot] Default python-platform to current platform (#17183)
Some checks are pending
CI / cargo clippy (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

As discussed in https://github.com/astral-sh/ruff/issues/16983 and
"mitigate" said issue for the alpha.

This PR changes the default for `PythonPlatform` to be the current
platform rather than `all`.

I'm not sure if we should be as sophisticated as supporting `ios` and
`android` as defaults but it was easy...

## Test Plan

Updated Markdown tests.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2025-04-09 12:05:18 +02:00
David Peter
00e00b9ad6
[red-knot] Add new 'unreachable code' test case (#17306)
## Summary

This is a new test case that I don't know how to handle yet. It leads to
many false positives in `rich/tests/test_win32_console.py`, which does
something like:

```py
if sys.platform == "win32":
    from windows_only_module import some_symbol

    some_other_symbol = 1

    def some_test_case():
        use(some_symbol)  # Red Knot: unresolved-reference
        use(some_other_symbol)  # Red Knot: unresolved-reference
```

Also adds a test for using unreachable symbols in type annotations or as
class bases.
2025-04-09 11:45:42 +02:00
David Peter
2cee86d807
[red-knot] Add custom __setattr__ support (#16748)
## Summary

Add support for classes with a custom `__setattr__` method.

## Test Plan

New Markdown tests, ecosystem checks.
2025-04-09 08:04:11 +02:00
Mike Perlov
fab7d820bd
[red-knot] Add __init__ arguments check when doing try_call on a class literal (#16512)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / mkdocs (push) Waiting to run
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

* Addresses #16511 for simple cases where only `__init__` method is
bound on class or doesn't exist at all.
* fixes a bug with argument counting in bound method diagnostics

Caveats:
* No handling of `__new__` or modified `__call__` on metaclass.
* This leads to a couple of false positive errors in tests

## Test Plan

- A couple new cases in mdtests
- cargo nextest run -p red_knot_python_semantic --no-fail-fast

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
2025-04-08 17:26:20 -04:00
Denys Kyslytsyn
ed14dbb1a2
[flake8-pie] Avoid false positive for multiple assignment with auto() (PIE796) (#17274)
This fix closes #16868 

I noticed the issue is assigned, but the assignee appears to be actively
working on another pull request. I hope that’s okay!

<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

As of Python 3.11.1, `enum.auto()` can be used in multiple assignments.
This pattern should not trigger non-unique-enums check.
Reference: [Python docs on
enum.auto()](https://docs.python.org/3/library/enum.html#enum.auto)

This fix updates the check logic to skip enum variant statements where
the right-hand side is a tuple containing a call to `enum.auto()`.

## Test Plan

<!-- How was it tested? -->

The added test case uses the example from the original issue. It
previously triggered a false positive, but now passes successfully.
2025-04-08 15:53:27 -04:00
Brent Westbrook
058439d5d3
[syntax-errors] Async comprehension in sync comprehension (#17177)
Some checks are pending
CI / cargo build (msrv) (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
Summary
--

Detect async comprehensions nested in sync comprehensions in async
functions before Python 3.11, when this was [changed].

The actual logic of this rule is very straightforward, but properly
tracking the async scopes took a bit of work. An alternative to the
current approach is to offload the `in_async_context` check into the
`SemanticSyntaxContext` trait, but that actually required much more
extensive changes to the `TestContext` and also to ruff's semantic
model, as you can see in the changes up to
31554b473507034735bd410760fde6341d54a050. This version has the benefit
of mostly centralizing the state tracking in `SemanticSyntaxChecker`,
although there was some subtlety around deferred function body traversal
that made the changes to `Checker` more intrusive too (hence the new
linter test).

The `Checkpoint` struct/system is obviously overkill for now since it's
only tracking a single `bool`, but I thought it might be more useful
later.

[changed]: https://github.com/python/cpython/issues/77527

Test Plan
--

New inline tests and a new linter integration test.
2025-04-08 12:50:52 -04:00
Wei Lee
dc02732d4d
[airflow] Expand module path check to individual symbols (AIR302) (#17278)
## Summary

### Improvement
Expand the following moved module into individual symbols.

* airflow.triggers.temporal
* airflow.triggers.file
* airflow.triggers.external_task
* airflow.hooks.subprocess
* airflow.hooks.package_index
* airflow.hooks.filesystem
* airflow.sensors.weekday
* airflow.sensors.time_delta
* airflow.sensors.time_sensor
* airflow.sensors.date_time
* airflow.operators.weekday
* airflow.operators.datetime
* airflow.operators.bash 

This removes `Replacement::ImportPathMoved`.

## Fix
During the expansion, the following paths were also fixed

* airflow.sensors.s3_key_sensor.S3KeySensor →
airflow.providers.amazon.aws.sensors.S3KeySensor
* airflow.operators.sql.SQLThresholdCheckOperator →
airflow.providers.common.sql.operators.sql.SQLThresholdCheckOperator
* airflow.hooks.druid_hook.DruidDbApiHook →
airflow.providers.apache.druid.hooks.druid.DruidDbApiHook
* airflow.hooks.druid_hook.DruidHook →
airflow.providers.apache.druid.hooks.druid.DruidHook
* airflow.kubernetes.pod_generator.extend_object_field →
airflow.providers.cncf.kubernetes.pod_generator.extend_object_field
* airflow.kubernetes.pod_launcher.PodLauncher →
airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodLauncher
* airflow.kubernetes.pod_launcher.PodStatus →
airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodStatus
* airflow.kubernetes.pod_generator.PodDefaults →
airflow.providers.cncf.kubernetes.pod_generator.PodDefaults
* airflow.kubernetes.pod_launcher_deprecated.PodDefaults →
airflow.providers.cncf.kubernetes.pod_launcher_deprecated.PodDefaults

### Refactor
As many symbols are moved into the same module,
`SourceModuleMovedToProvider` is introduced for grouping similar logic

## Test Plan
2025-04-08 09:03:27 -04:00
Brent Westbrook
0891689d2f
[syntax-errors] Check annotations in annotated assignments (#17283)
Summary
--

This PR extends the checks in #17101 and #17282 to annotated assignments
after Python 3.13.

Currently stacked on #17282 to include `await`.

Test Plan
--

New inline tests. These are simpler than the other cases because there's
no place to put generics.
2025-04-08 08:56:25 -04:00
Brent Westbrook
127a45622f
[syntax-errors] Extend annotation checks to await (#17282)
Summary
--

This PR extends the changes in #17101 to include `await` in the same
positions.

I also renamed the `valid_annotation_function` test to include `_py313`
and explicitly passed a Python version to contrast it with the `_py314`
version.

Test Plan
--

New test cases added to existing files.
2025-04-08 08:55:43 -04:00
David Peter
b662c3ff7e
[red-knot] Add support for assert_never (#17287)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

We already have partial "support" for `assert_never`, because it is
annotated as
```pyi
def assert_never(arg: Never, /) -> Never: ...
```
in typeshed. So we already emit a `invalid-argument-type` diagnostic if
the argument type to `assert_never` is not assignable to `Never`.

That is not enough, however. Gradual types like `Any`, `Unknown`,
`@Todo(…)` or `Any & int` can be assignable to `Never`. Which means that
we didn't issue any diagnostic in those cases.

Also, it seems like `assert_never` deserves a dedicated diagnostic
message, not just a generic "invalid argument type" error.

## Test Plan

New Markdown tests.
2025-04-08 09:31:49 +02:00
Denys Kyslytsyn
97dd6d120c
[flake8-pytest-style] Avoid false positive for legacy form of pytest.raises (PT011) (#17231)
This fix closes #17026 

## Summary

The check for the `PytestRaisesTooBroad` rule is now skipped if there is
a second positional argument present, which means `pytest.raises` is
used as a function.

## Test Plan

Tested on the example from the issue, which now passes the check.
```Python3
pytest.raises(Exception, func, *func_args, **func_kwargs).match("error message")
```

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-04-08 09:24:47 +02:00
InSync
34e06f2d17
[red-knot] Do not show types for literal expressions on hover (#17290)
## Summary

Resolves #17289.

After this change, Red Knot will no longer show types on hover for
`None`, `...`, `True`, `False`, numbers, strings (but not f-strings),
and bytes literals.

## Test Plan

Unit tests.
2025-04-08 09:05:51 +02:00
David Peter
a388c73752
[red-knot] Fix dead-code clippy warning (#17291)
## Summary

Failed run on main:
1432681231
2025-04-08 08:56:59 +02:00
David Peter
60f2e67454
[red-knot] Reachability analysis (#17199)
## Summary

This implements a new approach to silencing `unresolved-reference`
diagnostics by keeping track of the reachability of each use of a
symbol. The changes merged in
https://github.com/astral-sh/ruff/pull/17169 are still needed for the
"Use of variable in nested function" test case, but that could also be
solved in another way eventually (see
https://github.com/astral-sh/ruff/issues/15777). We can use the same
technique to silence `unresolved-import` and `unresolved-attribute`
false-positives, but I think this could be merged in isolation.

## Test Plan

New Markdown tests, ecosystem tests
2025-04-08 08:37:20 +02:00
Micha Reiser
cb7f56fb20
[red-knot] Don't use latency-sensitive for handlers (#17227)
## Summary

The priority latency-sensitive is reserved for actions that need to run
immediately because they would otherwise block the user's action. An
example of this is a format request. VS code blocks the editor until the
save action is complete. That's why formatting a document is very
sensitive to delays and it's important that we always have a worker
thread available to run a format request *immediately*. Another example
are code completions, where it's important that they appear immediately
when the user types.

On the other hand, showing diagnostics, hover, or inlay hints has high
priority but users are used that the editor takes a few ms to compute
the overlay.
Computing this information can also be expensive (e.g. find all
references), blocking the worker for quiet some time (a few 100ms).
That's why it's important
that those requests don't clog the sensitive worker threads.
2025-04-08 08:33:30 +02:00
David Peter
3657f798c9
[red-knot] Add --python-platform CLI option (#17284)
## Summary

Add a new `--python-platform` command-line option, in analogy to
`--python-version`.

## Test Plan

Added new integration test.
2025-04-07 21:04:44 +02:00
Matthew Mckee
4a4a376f02
[red-knot] Allow ellipsis default params in stub functions (#17243)
Some checks are pending
CI / cargo clippy (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

Fixes #17234

## Test Plan

Add tests to functions/paremeters.md

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-04-07 17:34:59 +00:00
Max Mynter
f1d4ba32cc
Fix RUF100 to detect unused file-level noqa directives with specific codes (#17042) (#17061)
Closes #17042

## Summary
This PR fixes the issue outlined in #17042 where RUF100 (unused-noqa)
fails to detect unused file-level noqa directives (`# ruff: noqa` or `#
ruff: noqa: {code}`).

The issue stems from two underlying causes:

1. For blanket file-level directives (`# ruff: noqa`), there's a
circular dependency: the directive exempts all rules including RUF100
itself, which prevents checking for usage. This isn't changed by this
PR. I would argue it is intendend behavior - a blanket `# ruff: noqa`
directive should exempt all rules including RUF100 itself.

2. For code-specific file-level directives (e.g. `# ruff: noqa: F841`),
the handling was missing in the `check_noqa` function. This is added in
this PR.

## Notes
- For file-level directives, the `matches` array is pre-populated with
the specified codes during parsing, unlike line-level directives which
only populate their `matches` array when actually suppressing
diagnostics. This difference requires the somewhat clunky handling of
both cases. I would appreciate guidance on a cleaner design :)

- A more fundamental solution would be to change how file-level
directives initialize the `matches` array in
`FileNoqaDirectives::extract()`, but that requires more substantial
changes as it breaks existing functionality. I suspect discussions in
#16483 are relevant for this.

## Test Plan
- Local verification
- Added a test case and fixture
2025-04-07 09:21:52 -05:00
Wei Lee
1e9e423362
[airflow] Update oudated AIR301, AIR302 rules (#17123)
## Summary

Some of the migration rules has been changed during Airflow 3
development. The following are new AIR302 rules. Corresponding AIR301
has also been removed.

* airflow.sensors.external_task_sensor.ExternalTaskMarker →
airflow.providers.standard.sensors.external_task.ExternalTaskMarker
* airflow.sensors.external_task_sensor.ExternalTaskSensor →
airflow.providers.standard.sensors.external_task.ExternalTaskSensor
* airflow.sensors.external_task_sensor.ExternalTaskSensorLink →
airflow.providers.standard.sensors.external_task.ExternalTaskSensorLink
* airflow.sensors.time_delta_sensor.TimeDeltaSensor →
airflow.providers.standard.sensors.time_delta.TimeDeltaSensor
* airflow.operators.dagrun_operator.TriggerDagRunLink →
airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunLink
* airflow.operators.dagrun_operator.TriggerDagRunOperator →
airflow.providers.standard.operators.trigger_dagrun.TriggerDagRunOperator
* airflow.operators.python_operator.BranchPythonOperator →
airflow.providers.standard.operators.python.BranchPythonOperator
* airflow.operators.python_operator.PythonOperator →
airflow.providers.standard.operators.python.PythonOperator
* airflow.operators.python_operator.PythonVirtualenvOperator →
airflow.providers.standard.operators.python.PythonVirtualenvOperator
* airflow.operators.python_operator.ShortCircuitOperator →
airflow.providers.standard.operators.python.ShortCircuitOperator
* airflow.operators.latest_only_operator.LatestOnlyOperator →
airflow.providers.standard.operators.latest_only.LatestOnlyOperator
* airflow.sensors.date_time_sensor.DateTimeSensor →
airflow.providers.standard.sensors.DateTimeSensor
* airflow.operators.email_operator.EmailOperator →
airflow.providers.smtp.operators.smtp.EmailOperator
* airflow.operators.email.EmailOperator →
airflow.providers.smtp.operators.smtp.EmailOperator
* airflow.operators.bash.BashOperator →
airflow.providers.standard.operators.bash.BashOperator
* airflow.operators.EmptyOperator →
airflow.providers.standard.operators.empty.EmptyOperator

closes: https://github.com/astral-sh/ruff/issues/17103

## Test Plan

The test fixture has been updated and checked after each change and
later reorganized in the latest commit
2025-04-07 09:45:56 -04:00
Micha Reiser
6cc2d02dfa
[red-knot] Support stub packages (#17204)
## Summary

This PR adds support for stub packages, except for partial stub packages
(a stub package is always considered non-partial).

I read the specification at
[typing.python.org/en/latest/spec/distributing.html#stub-only-packages](https://typing.python.org/en/latest/spec/distributing.html#stub-only-packages)
but I found it lacking some details, especially on how to handle
namespace packages or when the regular and stub packages disagree on
whether they're namespace packages. I tried to document my decisions in
the mdtests where the specification isn't clear and compared the
behavior to Pyright.

Mypy seems to only support stub packages in the venv folder. At least,
it never picked up my stub packages otherwise. I decided not to spend
too much time fighting mypyp, which is why I focused the comparison
around Pyright

Closes https://github.com/astral-sh/ruff/issues/16612

## Test plan

Added mdtests
2025-04-07 14:40:50 +02:00
Andrew Gallant
c12c76e9c8 ruff_annotate_snippets: address unused code warnings
Fixes #17230
2025-04-07 08:24:08 -04:00
Alex Waygood
81cf860dc8
[red-knot] Add a couple more tests for * imports (#17270)
## Summary

Some more edge cases that I thought of while working on integrating
knowledge of statically known branches into the `*`-import machinery

## Test Plan

`cargo test -p red_knot_python_semantic`
2025-04-07 11:12:28 +00:00
Micha Reiser
3150812ac4
[red-knot] Add 'Format document' to playground (#17217)
Some checks are pending
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary
This is more "because we can" than something we need. 

But since we're already building an "almost IDE" 

## Test Plan



https://github.com/user-attachments/assets/3a4bdad1-ba32-455a-9909-cfeb8caa1b28
2025-04-07 09:26:03 +02:00
Micha Reiser
67f8c30b53
[red-knot] Remove global dead_code from red-knot-server (#17229)
## Summary

Use more local `expect(dead_code)` suppressions instead of a global
`allow(dead_code)` in `lib.rs`.

Remove some methods that are either easy to add later, are less likely
to be needed for red knot, or it's unclear if we'd add it the same way
as in ruff.
2025-04-06 22:09:24 +01:00
Alex Waygood
ac5d220d75
[red-knot] Fix python setting in mdtests, and rewrite a site-packages test as an mdtest (#17222)
## Summary

This PR does the following things:
- Fixes the `python` configuration setting for mdtest (added in
https://github.com/astral-sh/ruff/pull/17221) so that it expects a path
pointing to a venv's `sys.prefix` variable rather than the a path
pointing to the venv's `site-packages` subdirectory. This brings the
`python` setting in mdtest in sync with our CLI `--python` flag.
- Tweaks mdtest so that it automatically creates a valid `pyvenv.cfg`
file for you if you don't specify one. This makes it much more ergonomic
to write an mdtest with a custom `python` setting: red-knot will reject
a `python` setting that points to a directory that doesn't have a
`pyvenv.cfg` file in it
- Tweaks mdtest so that it doesn't check a custom `pyvenv.cfg` as Python
source code if you _do_ add a custom `pyvenv.cfg` file for your mock
virtual environment in an mdtest. (You get a lot of diagnostics about
Python syntax errors in the `pyvenv.cfg` file, otherwise!)
- Rewrites the test added in
https://github.com/astral-sh/ruff/pull/17178 as an mdtest, and deletes
the original test that was added in that PR

## Test Plan

I verified that the new mdtest fails if I revert the changes to
`resolver.rs` that were added in
https://github.com/astral-sh/ruff/pull/17178
2025-04-06 18:24:32 +01:00
Matthew Mckee
73a9974d8a
Fix CallableTypeOf display signature (#17235) 2025-04-06 18:12:52 +01:00
Micha Reiser
1c652e6b98
[red-knot] Allow setting python in mdtests (#17221)
## Summary

This PR extends the mdtest options to allow setting the
`environment.python` option.

## Test Plan

I let @AlexWaygood write a test and he'll tell me if it works 😆
2025-04-05 16:43:31 +02:00
cake-monotone
7e6d3838bd
[red-knot] Add cycle handling to narrow constraints queries (#17209)
## Summary

This PR fixes the cycle issue that was causing problems in the `support
super` PR.

### Affected queries
- `all_narrowing_constraints_for_expression`
- `all_negative_narrowing_constraints_for_expression`


--

Additionally, `bidict` and `werkzeug` have been added to the
project-selection list in `mypy_primer`.
This PR also addresses the panics that occurred while analyzing those
packages:

- `bidict`: panic triggered by
`all_narrowing_constraints_for_expression`
- `werkzeug`: panic triggered by
`all_negative_narrowing_constraints_for_expression`

I think the mypy-primer results for this PR can serve as sufficient test
:)
2025-04-04 22:26:20 -07:00
David Peter
1a6a10b30f
[red-knot] Empty tuple is always-falsy (#17213)
Some checks are pending
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

Fix assignability of `tuple[()]` to `AlwaysFalsy`.

closes #17202 

## Test Plan

Ran the property tests for a while
2025-04-04 22:00:28 +02:00
Brent Westbrook
95d6ed40cc
Bump 0.11.4 (#17212) 2025-04-04 14:09:10 -04:00
Brent Westbrook
acc5662e8b
[syntax-errors] Allow yield in base classes and annotations (#17206)
Summary
--

This PR fixes the issue pointed out by @JelleZijlstra in
https://github.com/astral-sh/ruff/pull/17101#issuecomment-2777480204.
Namely, I conflated two very different errors from CPython:

```pycon
>>> def m[T](x: (yield from 1)): ...
  File "<python-input-310>", line 1
    def m[T](x: (yield from 1)): ...
                 ^^^^^^^^^^^^
SyntaxError: yield expression cannot be used within the definition of a generic
>>> def m(x: (yield from 1)): ...
  File "<python-input-311>", line 1
    def m(x: (yield from 1)): ...
              ^^^^^^^^^^^^
SyntaxError: 'yield from' outside function
>>> def outer():
...     def m(x: (yield from 1)): ...
...
>>>
```

I thought the second error was the same as the first, but `yield` (and
`yield from`) is actually valid in this position when inside a function
scope. The same is true for base classes, as pointed out in the original
comment.

We don't currently raise an error for `yield` outside of a function, but
that should be handled separately.

On the upside, this had the benefit of removing the
`InvalidExpressionPosition::BaseClass` variant and the
`allow_named_expr` field from the visitor because they were both no
longer used.

Test Plan
--

Updated inline tests.
2025-04-04 13:48:28 -04:00
David Salvisberg
33a56f198b
Don't skip visiting non-tuple slice in typing.Annotated subscripts (#17201)
Some checks are pending
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
Fixes: #17196

## Summary

Skipping these nodes for malformed type expressions would lead to
incorrect semantic state, which can in turn mean we emit false positives
for rules like `unused-variable`(`F841`)

## Test Plan

`cargo nextest run`
2025-04-04 18:47:40 +05:30
Micha Reiser
ffa824e037
[red-knot] Add Type.definition method (#17153)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
## Summary

This is a follow up to the goto type definition PR. Specifically, that
we want to avoid exposing too many semantic model internals publicly.

I want to get some feedback on the approach taken. I think it goes into
the right direction but I'm not super happy with it.
The basic idea is that we add a `Type::definition` method which does the
"goto type definition". The parts that I think make it awkward:

* We can't directly return `Definition` because we don't create a
`Definition` for modules (but we could?). Although I think it makes
sense to possibly have a more public wrapper type anyway?
* It doesn't handle unions and intersections. Mainly because not all
elements in an intersection may have a definition and we only want to
show a navigation target for intersections if there's only a single
positive element (besides maybe `Unknown`).


An alternative design or an addition to this design is to introduce a
`SemanticAnalysis(Db)` struct that has methods like
`type_definition(&self, type)` which explicitly exposes the methods we
want. I don't feel comfortable design this API yet because it's unclear
how fine granular it has to be (and if it is very fine granular,
directly using `Type` might be better after all)


## Test Plan

`cargo test`
2025-04-04 10:56:20 +02:00
Max Mynter
98b95c9c38
Implement Invalid rule provided as rule RUF102 with --fix (#17138)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

Closes #17084

## Summary
This PR adds a new rule (RUF102) to detect and fix invalid rule codes in
`noqa` comments.
Invalid rule codes in `noqa` directives serve no purpose and may
indicate outdated code suppressions.

This extends the previous behaviour originating from
`crates/ruff_linter/src/noqa.rs` which would only emit a warnigs.
With this rule a `--fix` is available.

The rule:
1. Analyzes all `noqa` directives to identify invalid rule codes
2. Provides autofix functionality to:
   - Remove the entire comment if all codes are invalid
   - Remove only the invalid codes when mixed with valid codes
3. Preserves original comment formatting and whitespace where possible

Example cases:
- `# noqa: XYZ111` → Remove entire comment (keep empty line)
- `# noqa: XYZ222, XYZ333` → Remove entire comment (keep empty line)
-  `# noqa: F401, INVALID123` → Keep only valid codes (`# noqa: F401`)

## Test Plan
- Added tests in
`crates/ruff_linter/resources/test/fixtures/ruff/RUF102.py` covering
different example cases.

<!-- How was it tested? -->

## Notes 
- This does not handle cases where parsing fails. E.g. `# noqa:
NON_EXISTENT, ANOTHER_INVALID` causes a `LexicalError` and the
diagnostic is not propagated and we cannot handle the diagnostic. I am
also unsure what proper `fix` handling would be and making the user
aware we don't understand the codes is probably the best bet.
- The rule is added to the Preview rule group as it's a new addition

## Questions
- Should we remove the warnings, now that we have a rule?
- Is the current fix behavior appropriate for all cases, particularly
the handling of whitespace and line deletions?
- I'm new to the codebase; let me know if there are rule utilities which
could have used but didn't.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2025-04-04 08:05:59 +00:00
Micha Reiser
a4ba10ff0a
[red-knot] Add basic on-hover to playground and LSP (#17057)
## Summary

Implement a very basic hover in the playground and LSP.

It's basic, because it only shows the type on-hover. Most other LSPs
also show:

* The signature of the symbol beneath the cursor. E.g. `class
Test(a:int, b:int)` (we want something like
54f7da25f9/packages/pyright-internal/src/analyzer/typeEvaluator.ts (L21929-L22129))
* The symbols' documentation
* Do more fancy markdown rendering

I decided to defer these features for now because it requires new
semantic APIs (similar to *goto definition*), and investing in fancy
rendering only makes sense once we have the relevant data.

Closes [#16826](https://github.com/astral-sh/ruff/issues/16826)

## Test Plan



https://github.com/user-attachments/assets/044aeee4-58ad-4d4e-9e26-ac2a712026be


https://github.com/user-attachments/assets/4a1f4004-2982-4cf2-9dfd-cb8b84ff2ecb
2025-04-04 08:13:43 +02:00
Carl Meyer
bf0306887a
[red-knot] don't remove negations when simplifying constrained typevars (#17189)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[Knot Playground] Release / publish (push) Waiting to run
For two non-disjoint types `P` and `Q`, the simplification of `(P | Q) &
~Q` is not `P`, but `P & ~Q`. In other words, the non-empty set `P & Q`
is also excluded from the type.

The same applies for a constrained typevar `[T: (P, Q)]`: `T & ~Q`
should simplify to `P & ~Q`, not just `P`.

Implementing this is actually purely a matter of removing code from the
constrained typevar simplification logic; we just need to not bother
removing the negations. If the negations are actually redundant (because
the constraint types are disjoint), normal intersection simplification
will already eliminate them (as shown in the added test.)
2025-04-03 16:30:57 -07:00
Brent Westbrook
4f924bb975
[minor] Fix extra semicolon for clippy (#17188) 2025-04-03 18:17:00 -04:00
Brent Westbrook
c2b2e42ad3
[syntax-errors] Invalid syntax in annotations (#17101)
Summary
--

This PR detects the use of invalid syntax in annotation scopes,
including
`yield` and `yield from` expressions and named expressions. I combined a
few
different types of CPython errors here, but I think the resulting error
messages
still make sense and are even preferable to what CPython gives. For
example, we
report `yield expression cannot be used in a type annotation` for both
of these:

```pycon
>>> def f[T](x: (yield 1)): ...
  File "<python-input-26>", line 1
    def f[T](x: (yield 1)): ...
                 ^^^^^^^
SyntaxError: yield expression cannot be used within the definition of a generic
>>> def foo() -> (yield x): ...
  File "<python-input-28>", line 1
    def foo() -> (yield x): ...
                  ^^^^^^^
SyntaxError: 'yield' outside function
```

Fixes https://github.com/astral-sh/ruff/issues/11118.

Test Plan
--

New inline tests, along with some updates to existing tests.
2025-04-03 17:56:55 -04:00
Brent Westbrook
24b1b1d52c
[syntax-errors] Duplicate attributes in match class pattern (#17186)
Summary
--

Detects duplicate attributes in a `match` class pattern:

```python
match x:
    case Class(x=1, x=2): ...
```

which are more analogous to the similar check for mapping patterns than
to the
multiple assignments rule.

I also realized that both this and the mapping check would only work on
top-level patterns, despite the possibility that they can be nested
inside other
patterns:

```python
match x:
    case [{"x": 1, "x": 2}]: ...  # false negative in the old version
```

and moved these checks into the recursive pattern visitor instead.

I also tidied up some of the names like the `multiple_case_assignment`
function
and the `MultipleCaseAssignmentVisitor`, which are now doing more than
checking
for multiple assignments.

Test Plan
--

New inline tests for both classes and mappings.
2025-04-03 17:55:37 -04:00
Brent Westbrook
6a07dd227d
[syntax-errors] Fix multiple assignment for class keyword argument (#17184)
Summary
--

Fixes #17181. The cases being tested with multiple *keys* being equal
are actually a slightly different error, more like the error for
`MatchMapping` than like the other multiple assignment errors:

```pycon
>>> match x:
...     case Class(x=x, x=x): ...
...
  File "<python-input-249>", line 2
    case Class(x=x, x=x): ...
                      ^
SyntaxError: attribute name repeated in class pattern: x
>>> match x:
...     case {"x": 1, "x": 2}: ...
...
  File "<python-input-251>", line 2
    case {"x": 1, "x": 2}: ...
         ^^^^^^^^^^^^^^^^
SyntaxError: mapping pattern checks duplicate key ('x')
>>> match x:
...     case [x, x]: ...
...
  File "<python-input-252>", line 2
    case [x, x]: ...
             ^
SyntaxError: multiple assignments to name 'x' in pattern
```

This PR just stops the false positive reported in the issue, but I will
quickly follow it up with a new rule (or possibly combined with the
mapping rule) catching the repeated attributes separately.

Test Plan
--

New inline `test_ok` and updating the `test_err` cases to have duplicate
values instead of keys.
2025-04-03 17:32:39 -04:00
Micha Reiser
e7684d3493
Enable overindented docs lint (#17182)
## Summary

It turns out that `a.` isn't a list format supported by rustdoc. I
changed the documentation to use `1.`, `2.` instead.

## Test Plan

`cargo clippy`
2025-04-03 20:27:11 +01:00
Douglas Creager
64e7e1aa64
[red-knot] Add Type::TypeVar variant (#17102)
This adds a new `Type` variant for holding an instance of a typevar
inside of a generic function or class. We don't handle specializing the
typevars yet, but this should implement most of the typing rules for
inside the generic function/class, where we don't know yet which
specific type the typevar will be specialized to.

This PR does _not_ yet handle the constraint that multiple occurrences
of the typevar must be specialized to the _same_ time. (There is an
existing test case for this in `generics/functions.md` which is still
marked as TODO.)

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-04-03 14:36:29 -04:00
Micha Reiser
8a4158c5f8
Upgrade to Rust 1.86 and bump MSRV to 1.84 (#17171)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

I decided to disable the new
[`needless_continue`](https://rust-lang.github.io/rust-clippy/master/index.html#needless_continue)
rule because I often found the explicit `continue` more readable over an
empty block or having to invert the condition of an other branch.


## Test Plan

`cargo test`

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-04-03 15:59:44 +00:00
David Peter
fedd982fd5
[red-knot] Avoid unresolved-reference in unreachable code (#17169)
## Summary

This PR changes the inferred type for symbols in unreachable sections of
code to `Never` (instead of reporting them as unbound), in order to
silence false positive diagnostics. See the lengthy comment in the code
for further details.

## Test Plan

- Updated Markdown tests.
- Manually verified a couple of ecosystem diagnostic changes.
2025-04-03 16:52:11 +02:00
Alex Waygood
a1eb834a5f
Fix relative import resolution in site-packages packages when the site-packages search path is a subdirectory of the first-party search path (#17178)
## Summary

If a package in `site-packages` had this directory structure:

```py
# bar/__init__.py
from .a import A

# bar/a.py
class A: ...
```

then we would fail to resolve the `from .a import A` import _if_ (as is
usually the case!) the `site-packages` search path was located inside a
`.venv` directory that was a subdirectory of the project's first-party
search path. The reason for this is a bug in `file_to_module` in the
module resolver. In this loop, we would identify that
`/project_root/.venv/lib/python3.13/site-packages/foo/__init__.py` can
be turned into a path relative to the first-party search path
(`/project_root`):


6e2b8f9696/crates/red_knot_python_semantic/src/module_resolver/resolver.rs (L101-L110)

but we'd then try to turn the relative path
(.venv/lib/python3.13/site-packages/foo/__init__.py`) into a module
path, realise that it wasn't a valid module path... and therefore
immediately `break` out of the loop before trying any other search paths
(such as the `site-packages` search path).

This bug was originally reported on Discord by @MatthewMckee4.

## Test Plan

I added a unit test for `file_to_module` in `resolver.rs`, and an
integration test that shows we can now resolve the import correctly in
`infer.rs`.
2025-04-03 15:48:05 +01:00