ruff/crates
Alex Waygood c2bb5d5250
[red-knot] Fix equivalence of differently ordered unions that contain Callable types (#17145)
## Summary

Fixes https://github.com/astral-sh/ruff/issues/17058.

Equivalent callable types were not understood as equivalent when they
appeared nested inside unions and intersections. This PR fixes that by
ensuring that `Callable` elements nested inside unions, intersections
and tuples have their representations normalized before one union type
is compared with another for equivalence, or before one intersection
type is compared with another for equivalence.

The normalizations applied to a `Callable` type are:
- the type of the default value is stripped from all parameters (only
whether the parameter _has_ a default value is relevant to whether one
`Callable` type is equivalent to another)
- The names of the parameters are stripped from positional-only
parameters, variadic parameters and keyword-variadic parameters
- Unions and intersections that are present (top-level or nested) inside
parameter annotations or return annotations are normalized.

Adding a `CallableType::normalized()` method also allows us to simplify
the implementation of `CallableType::is_equivalent_to()`.

### Should these normalizations be done eagerly as part of a
`CallableType` constructor?

I considered this. It's something that we could still consider doing in
the future; this PR doesn't rule it out as a possibility. However, I
didn't pursue it for now, for several reasons:
1. Our current `Display` implementation doesn't handle well the
possibility that a parameter might not have a name or an annotated type.
Callable types with parameters like this would be displayed as follows:
   ```py
   (, ,) -> None: ...
   ```

That's fixable! It could easily become something like `(Unknown,
Unknown) -> None: ...`. But it also illustrates that we probably want to
retain the parameter names when displaying the signature of a `lambda`
function if you're hovering over a reference to the lambda in an IDE.
Currently we don't have a `LambdaType` struct for representing `lambda`
functions; if we wanted to eagerly normalize signatures when creating
`CallableType`s, we'd probably have to add a `LambdaType` struct so that
we would retain the full signature of a `lambda` function, rather than
representing it as an eagerly simplified `CallableType`.
2. In order to ensure that it's impossible to create `CallableType`s
without the parameters being normalized, I'd either have to create an
alternative `SimplifiedSignature` struct (which would duplicate a lot of
code), or move `CallableType` to a new module so that the only way of
constructing a `CallableType` instance would be via a constructor method
that performs the normalizations eagerly on the callable's signature.
Again, this isn't a dealbreaker, and I think it's still an option, but
it would be a lot of churn, and it didn't seem necessary for now. Doing
it this way, at least to start with, felt like it would create a diff
that's easier to review and felt like it would create fewer merge
conflicts for others.

## Test Plan

- Added a regression mdtest for
https://github.com/astral-sh/ruff/issues/17058
- Ran `QUICKCHECK_TESTS=1000000 cargo test --release -p
red_knot_python_semantic -- --ignored types::property_tests::stable`
2025-04-02 17:43:34 +00:00
..
red_knot ruff_db: switch diagnostic rendering over to std::fmt::Display 2025-04-02 11:01:16 -04:00
red_knot_ide ruff_db: switch diagnostic rendering over to std::fmt::Display 2025-04-02 11:01:16 -04:00
red_knot_project red_knot: use Diagnostic inside of red knot 2025-04-02 10:10:01 -04:00
red_knot_python_semantic [red-knot] Fix equivalence of differently ordered unions that contain Callable types (#17145) 2025-04-02 17:43:34 +00:00
red_knot_server red_knot: use Diagnostic inside of red knot 2025-04-02 10:10:01 -04:00
red_knot_test ruff_db: switch diagnostic rendering over to std::fmt::Display 2025-04-02 11:01:16 -04:00
red_knot_vendored Sync vendored typeshed stubs (#17106) 2025-04-01 17:44:27 +01:00
red_knot_wasm ruff_db: switch diagnostic rendering over to std::fmt::Display 2025-04-02 11:01:16 -04:00
ruff [flake8-import-conventions] Add import numpy.typing as npt to default flake8-import-conventions.aliases (#17133) 2025-04-02 09:25:46 +02:00
ruff_annotate_snippets Update pre-commit dependencies (#17073) 2025-03-31 07:42:15 +00:00
ruff_benchmark red_knot: use Diagnostic inside of red knot 2025-04-02 10:10:01 -04:00
ruff_cache Fix cache key collisions for paths with separators (#12159) 2024-07-03 07:36:46 -05:00
ruff_db ruff_db: simplify lifetimes on DiagnosticDisplay 2025-04-02 12:47:02 -04:00
ruff_dev Pass ParserOptions to the parser (#16220) 2025-02-19 10:50:50 -05:00
ruff_diagnostics Show errors for attempted fixes only when passed --verbose (#15237) 2025-01-03 08:50:13 -06:00
ruff_formatter Fixing more spelling errors (#16926) 2025-03-23 10:55:14 -07:00
ruff_graph Pass ParserOptions to the parser (#16220) 2025-02-19 10:50:50 -05:00
ruff_index [red-knot] Don't use separate ID types for each alist (#16415) 2025-02-28 14:55:55 -05:00
ruff_linter [airflow] Move AIR302 to AIR301 and AIR303 to AIR302 (#17151) 2025-04-02 23:01:31 +05:30
ruff_macros Add knot.toml schema (#15735) 2025-02-07 10:59:40 +01:00
ruff_notebook bump MSRV to 1.83 (#16294) 2025-02-26 06:12:43 -08:00
ruff_python_ast Visit Identifier node as part of the SourceOrderVisitor (#17110) 2025-04-01 16:58:09 +02:00
ruff_python_ast_integration_tests Visit Identifier node as part of the SourceOrderVisitor (#17110) 2025-04-01 16:58:09 +02:00
ruff_python_codegen Pass ParserOptions to the parser (#16220) 2025-02-19 10:50:50 -05:00
ruff_python_formatter Visit Identifier node as part of the SourceOrderVisitor (#17110) 2025-04-01 16:58:09 +02:00
ruff_python_index Extract LineIndex independent methods from Locator (#13938) 2024-10-28 07:53:41 +00:00
ruff_python_literal Preserve triple quotes and prefixes for strings (#15818) 2025-02-04 08:41:06 -05:00
ruff_python_parser [syntax-errors] Reimplement PLE0118 (#17135) 2025-04-02 13:03:44 +00:00
ruff_python_resolver Update pre-commit dependencies (#17073) 2025-03-31 07:42:15 +00:00
ruff_python_semantic Control flow graph: setup (#17064) 2025-04-01 05:53:42 -05:00
ruff_python_stdlib Revert "Add all PEP-585 names to UP006 rule" (#15250) 2025-01-04 12:23:53 +01:00
ruff_python_trivia [red-knot] Ignore surrounding whitespace when looking for <!-- snapshot-diagnostics --> directives in mdtests (#16380) 2025-02-27 13:25:31 +00:00
ruff_python_trivia_integration_tests Pass ParserOptions to the parser (#16220) 2025-02-19 10:50:50 -05:00
ruff_server Update pre-commit dependencies (#17073) 2025-03-31 07:42:15 +00:00
ruff_source_file [pyupgrade] Do not report when a UTF-8 comment is followed by a non-UTF-8 one (UP009) (#14728) 2024-12-11 10:30:41 +00:00
ruff_text_size [ruff] itertools.starmap(..., zip(...)) (RUF058) (#15483) 2025-01-16 15:18:12 +01:00
ruff_wasm Bump 0.11.2 (#16896) 2025-03-21 09:17:07 -04:00
ruff_workspace [flake8-import-conventions] Add import numpy.typing as npt to default flake8-import-conventions.aliases (#17133) 2025-04-02 09:25:46 +02:00