Commit graph

657 commits

Author SHA1 Message Date
Jack O'Connor
604ca32178
Merge b2d6f35366 into 0d2cd84df4 2025-11-16 18:29:33 +00:00
David Peter
f5fb5c388a
[ty] Dataclasses: __hash__ semantics and unsafe_hash (#21470)
Some checks are pending
CI / test scripts (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / pre-commit (push) Waiting to run
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
## Summary

Implement the semantics of `__hash__` for dataclasses and add support
for `unsafe_hash`

## Test Plan

New Markdown tests.
2025-11-16 09:52:30 +00:00
David Peter
dbd72480a9
[ty] Dataclass transform: complete set of parameters (#21474)
## Summary

We previously only allowed models to overwrite the
`{eq,order,kw_only,frozen}_defaults` of the dataclass-transformer, but
all other standard-dataclass parameters should be equally supported with
the same behavior.

## Test Plan

Added regression tests.
2025-11-16 09:46:45 +00:00
Alex Waygood
3065f8dbbc
[ty] Improve diagnostics for invalid exceptions (#21475)
Some checks are pending
CI / cargo fmt (push) Waiting to run
CI / Determine changes (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
## Summary

Not a high-priority task... but it _is_ a weekend :P

This PR improves our diagnostics for invalid exceptions. Specifically:
- We now give a special-cased ``help: Did you mean
`NotImplementedError`` subdiagnostic for `except NotImplemented`, `raise
NotImplemented` and `raise <EXCEPTION> from NotImplemented`
- If the user catches a tuple of exceptions (`except (foo, bar, baz):`)
and multiple elements in the tuple are invalid, we now collect these
into a single diagnostic rather than emitting a separate diagnostic for
each tuple element
- The explanation of why the `except`/`raise` was invalid ("must be a
`BaseException` instance or `BaseException` subclass", etc.) is
relegated to a subdiagnostic. This makes the top-level diagnostic
summary much more concise.

## Test Plan

Lots of snapshots. And here's some screenshots:

<details>
<summary>Screenshots</summary>

<img width="1770" height="1520" alt="image"
src="https://github.com/user-attachments/assets/7f27fd61-c74d-4ddf-ad97-ea4fd24d06fd"
/>

<img width="1916" height="1392" alt="image"
src="https://github.com/user-attachments/assets/83e5027c-8798-48a6-a0ec-1babfc134000"
/>

<img width="1696" height="588" alt="image"
src="https://github.com/user-attachments/assets/1bc16048-6eb4-4dfa-9ace-dd271074530f"
/>

</details>
2025-11-15 22:12:00 +00:00
github-actions[bot]
efa2b5167f
[ty] Sync vendored typeshed stubs (#21466)
Co-authored-by: typeshedbot <>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-11-15 17:12:32 +00:00
David Peter
29acc1e860
[ty] Support class-arguments for dataclass transformers (#21457)
## Summary

Allow metaclass-based and baseclass-based dataclass-transformers to
overwrite the default behavior using class arguments:

```py
class Person(Model, order=True):
    # ...
```

## Conformance tests

Four new tests passing!

## Test Plan

New Markdown tests
2025-11-15 17:47:48 +01:00
Jack O'Connor
b2d6f35366 test that structural assignment works for recursive TypedDicts too 2025-11-14 17:16:51 -08:00
Jack O'Connor
8b0a217911 [ty] implement TypedDict structural assignment 2025-11-14 17:04:27 -08:00
Douglas Creager
698231a47a
[ty] Implement constraint implication for compound types (#21366)
Some checks are pending
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
This PR updates the constraint implication type relationship to work on
compound types as well. (A compound type is a non-atomic type, like
`list[T]`.)

The goal of constraint implication is to check whether the requirements
of a constraint imply that a particular subtyping relationship holds.
Before, we were only checking atomic typevars. That would let us verify
that the constraint set `T ≤ bool` implies that `T` is always a subtype
of `int`. (In this case, the lhs of the subtyping check, `T`, is an
atomic typevar.)

But we weren't recursing into compound types, to look for nested
occurrences of typevars. That means that we weren't able to see that `T
≤ bool` implies that `Covariant[T]` is always a subtype of
`Covariant[int]`.

Doing this recursion means that we have to carry the constraint set
along with us as we recurse into types as part of `has_relation_to`, by
adding constraint implication as a new `TypeRelation` variant. (Before
it was just a method on `ConstraintSet`.)

---------

Co-authored-by: David Peter <sharkdp@users.noreply.github.com>
2025-11-14 18:43:00 -05:00
Alex Waygood
d63b4b0383
[ty] Improve diagnostic range for non-subscriptable diagnostics (#21461)
## Summary

Currently our diagnostic only covers the range of the thing being
subscripted:

<img width="1702" height="312" alt="image"
src="https://github.com/user-attachments/assets/7e630431-e846-46ca-93c1-139f11aaba11"
/>

But it should probably cover the _whole_ subscript expression (arguably
the more "incorrect" bit is the `["foo"]` part of this expression, not
the `x` part of this expression!)

## Test Plan

Added a snapshot

Co-authored-by: Brent Westbrook
<36778786+ntBre@users.noreply.github.com>
2025-11-14 21:15:14 +00:00
Ibraheem Ahmed
c5d654bce8
[ty] Improve literal promotion heuristics (#21439)
## Summary

Extends literal promotion to apply to any generic method, as opposed to
only generic class constructors. This PR also improves our literal
promotion heuristics to only promote literals in non-covariant position
in the return type, and avoid promotion if the literal is present in
non-covariant position in any argument type.

Resolves https://github.com/astral-sh/ty/issues/1357.
2025-11-14 16:13:56 -05:00
Alex Waygood
3e7e91724c
[ty] Further improve details around which expressions should be deferred in stub files (#21456)
## Summary

- Always restore the previous `deferred_state` after parsing a type
expression: we don't want that state leaking out into other contexts
where we shouldn't be deferring expression inference
- Always defer the right-hand-side of a PEP-613 type alias in a stub
file, allowing for forward references on the right-hand side of `T:
TypeAlias = X | Y` in a stub file

Addresses @carljm's review in
https://github.com/astral-sh/ruff/pull/21401#discussion_r2524260153

## Test Plan

I added a regression test for a regression that the first version of
this PR introduced (we need to make sure the r.h.s. of a PEP-613
`TypeAlias`es is always deferred in a stub file)
2025-11-14 21:07:02 +00:00
Ibraheem Ahmed
2a2b719f00
[ty] Improve generic class constructor inference (#21442)
## Summary

We currently fail to account for the type context when inferring generic
classes constructed with `__new__`, or synthesized `__init__` for
dataclasses.
2025-11-14 20:25:47 +00:00
Ibraheem Ahmed
ffb7bdd595
[ty] Propagate type context through conditional expressions (#21443)
## Summary

Resolves https://github.com/astral-sh/ty/issues/1543.
2025-11-14 15:19:08 -05:00
Bhuminjay Soni
8529d79a70
[ty] name is parameter and global is a syntax error (#21312)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
2025-11-14 18:15:34 +00:00
Alex Waygood
8599c7e5b3
[ty] Fixup a few details around version-specific dataclass features (#21453)
Some checks are pending
CI / mkdocs (push) Waiting to run
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
2025-11-14 15:04:55 +00:00
Alex Waygood
5f501374c4
[ty] Support attribute-expression TYPE_CHECKING conditionals (#21449) 2025-11-14 13:11:49 +00:00
David Peter
e9a5337136
[ty] Support stringified annotations in value-position Annotated instances (#21447)
## Summary

Infer the first argument `type` inside `Annotated[type, …]` as a type
expression. This allows us to support stringified annotations inside
`Annotated`.

## Ecosystem

* The removed diagnostic on `prefect` shows that we now understand the
`State.data` type annotation in
`src/prefect/client/schemas/objects.py:230`, which uses a stringified
annotation in `Annoated`. The other diagnostics are downstream changes
that result from this, it seems to be a commonly used data type.
* `artigraph` does something like `Annotated[cast(Any,
field_info.annotation), *field_info.metadata]` which I'm not sure we
need to allow? It's unfortunate since this is probably supported at
runtime, but it seems reasonable that they need to add a `# type:
ignore` for that.
* `pydantic` uses something like `Annotated[(self.annotation,
*self.metadata)]` but adds a `# type: ignore`

## Test Plan

New Markdown test
2025-11-14 13:09:09 +00:00
David Peter
05cf53aae8
[ty] Type inference for genererator expressions (#21437)
## Summary

Add type inference for (async) generator expressions.

closes https://github.com/astral-sh/ty/issues/1510

## Test Plan

New Markdown tests.
2025-11-14 13:04:11 +00:00
David Peter
6a26f86778
[ty] Make __getattr__ available for ModuleType instances (#21450)
## Summary

Typeshed has a (fake) `__getattr__` method on `types.ModuleType` with a
return type of `Any`. We ignore this method when accessing attributes on
module *literals*, but with this PR, we respect this method when dealing
with `ModuleType` itself. That is, we allow arbitrary attribute accesses
on instances of `types.ModuleType`. This is useful because dynamic
import mechanisms such as `importlib.import_module` use `ModuleType` as
a return type.

closes https://github.com/astral-sh/ty/issues/1346

## Ecosystem

Massive reduction in diagnostics. The few new diagnostics are true
positives.

## Test Plan

Added regression test.
2025-11-14 13:59:14 +01:00
David Peter
696d7a5d68
[ty] Add synthetic members to completions on dataclasses (#21446)
## Summary

Add synthetic members to completions on dataclasses and dataclass
instances.

Also, while we're at it, add support for `__weakref__` and
`__match_args__`.

closes https://github.com/astral-sh/ty/issues/1542

## Test Plan

New Markdown tests
2025-11-14 11:31:20 +01:00
David Peter
66e9d57797
[ty] Support legacy typing special forms in implicit type aliases (#21433)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
## Summary

Support various legacy `typing` special forms (`List`, `Dict`, …) in
implicit type aliases.

## Ecosystem impact

A lot of true positives (e.g. on `alerta`)!

## Test Plan

New Markdown tests
2025-11-14 09:08:58 +01:00
David Peter
9e80e5a3a6
[ty] Support type[…] and Type[…] in implicit type aliases (#21421)
## Summary

Support `type[…]` in implicit type aliases, for example:
```py
SubclassOfInt = type[int]

reveal_type(SubclassOfInt)  # GenericAlias

def _(subclass_of_int: SubclassOfInt):
    reveal_type(subclass_of_int)  # type[int]
```

part of https://github.com/astral-sh/ty/issues/221

## Typing conformance

```diff
-specialtypes_type.py:138:5: error[type-assertion-failure] Argument does not have asserted type `type[Any]`
-specialtypes_type.py:140:5: error[type-assertion-failure] Argument does not have asserted type `type[Any]`
```

Two new tests passing ✔️ 

```diff
-specialtypes_type.py:146:1: error[unresolved-attribute] Object of type `GenericAlias` has no attribute `unknown`
```

An `TA4.unknown` attribute on a PEP 613 alias (`TA4: TypeAlias =
type[Any]`) is being accessed, and the conformance suite expects this to
be an error. Since we currently use the inferred type for these type
aliases (and possibly in the future as well), we treat this as a direct
access of the attribute on `type[Any]`, which falls back to an access on
`Any` itself, which succeeds. 🔴

```
+specialtypes_type.py:152:16: error[invalid-type-form] `typing.TypeVar` is not a generic class
+specialtypes_type.py:156:16: error[invalid-type-form] `typing.TypeVar` is not a generic class
```

New errors because we don't handle `T = TypeVar("T"); MyType = type[T];
MyType[T]` yet. Support for this is being tracked in
https://github.com/astral-sh/ty/issues/221 🔴

## Ecosystem impact

Looks mostly good, a few known problems. 

## Test Plan

New Markdown tests
2025-11-13 19:02:24 +01:00
Alex Waygood
90b32f3b3b
[ty] Ensure annotation/type expressions in stub files are always deferred (#21401)
Some checks are pending
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
2025-11-13 17:14:54 +00:00
David Peter
04ab9170d6
[ty] Further improve subscript assignment diagnostics (#21411)
## Summary

Further improve subscript assignment diagnostics, especially for
`dict`s:

```py
config: dict[str, int] = {}

config["retries"] = "three"
```

<img width="1276" height="274" alt="image"
src="https://github.com/user-attachments/assets/9762c733-8d1c-4a57-8c8a-99825071dc7d"
/>

I have many more ideas, but this looks like a reasonable first step.
Thank you @AlexWaygood for some of the suggestions here.

## Test Plan

Update tests
2025-11-13 13:31:14 +01:00
Alex Waygood
cd183c5e1f
[ty] Use the return type of __get__ for descriptor lookups even when __get__ is called with incorrect arguments (#21424) 2025-11-13 12:05:10 +00:00
David Peter
2f6f3e1042
[ty] Faster subscript assignment checks for (unions of) TypedDicts (#21378)
## Summary

We synthesize a (potentially large) set of `__setitem__` overloads for
every item in a `TypedDict`. Previously, validation of subscript
assignments on `TypedDict`s relied on actually calling `__setitem__`
with the provided key and value types, which implied that we needed to
do the full overload call evaluation for this large set of overloads.
This PR improves the performance of subscript assignment checks on
`TypedDict`s by validating the assignment directly instead of calling
`__setitem__`.

This PR also adds better handling for assignments to subscripts on union
and intersection types (but does not attempt to make it perfect). It
achieves this by distributing the check over unions and intersections,
instead of calling `__setitem__` on the union/intersection directly. We
already do something similar when validating *attribute* assignments.

## Ecosystem impact

* A lot of diagnostics change their rule type, and/or split into
multiple diagnostics. The new version is more verbose, but easier to
understand, in my opinion
* Almost all of the invalid-key diagnostics come from pydantic, and they
should all go away (including many more) when we implement
https://github.com/astral-sh/ty/issues/1479
* Everything else looks correct to me. There may be some new diagnostics
due to the fact that we now check intersections.

## Test Plan

New Markdown tests.
2025-11-12 20:16:38 +01:00
Shunsuke Shibayama
9dd666d677
[ty] fix global symbol lookup from eager scopes (#21317)
## Summary

cf. https://github.com/astral-sh/ruff/pull/20962

In the following code, `foo` in the comprehension was not reported as
unresolved:

```python
# error: [unresolved-reference] "Name `foo` used when not defined"
foo
foo = [
    # no error!
    # revealed: Divergent
    reveal_type(x) for _ in () for x in [foo]
]

baz = [
    # error: [unresolved-reference] "Name `baz` used when not defined"
    # revealed: Unknown
    reveal_type(x) for _ in () for x in [baz]
]
```

In fact, this is a more serious bug than it looks: for `foo`,
[`explicit_global_symbol` is
called](6cc3393ccd/crates/ty_python_semantic/src/types/infer/builder.rs (L8052)),
causing a symbol that should actually be `Undefined` to be reported as
being of type `Divergent`.

This PR fixes this bug. As a result, the code in
`mdtest/regression/pr_20962_comprehension_panics.md` no longer panics.

## Test Plan

`corpus\cyclic_symbol_in_comprehension.py` is added.
New tests are added in `mdtest/comprehensions/basic.md`.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Carl Meyer <carl@astral.sh>
2025-11-12 10:15:51 -08:00
David Peter
84c3cecad6
[ty] Baseline for subscript assignment diagnostics (#21404)
## Summary

Add (snapshot) tests for subscript assignment diagnostics. This is
mainly intended to establish a baseline before I hope to improve some of
these messages.
2025-11-12 15:29:26 +01:00
David Peter
e8e8180888
[ty] Implicit type aliases: Add support for typing.Union (#21363)
## Summary

Add support for `typing.Union` in implicit type aliases / in value
position.

## Typing conformance tests

Two new tests are passing

## Ecosystem impact

* The 2k new `invalid-key` diagnostics on pydantic are caused by
https://github.com/astral-sh/ty/issues/1479#issuecomment-3513854645.
* Everything else I've checked is either a known limitation (often
related to type narrowing, because union types are often narrowed down
to a subset of options), or a true positive.

## Test Plan

New Markdown tests
2025-11-12 12:59:14 +01:00
Shaygan Hooshyari
988c38c013
[ty] Skip eagerly evaluated scopes for attribute storing (#20856)
Some checks are pending
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
## Summary

Fix https://github.com/astral-sh/ty/issues/664

This PR adds support for storing attributes in comprehension scopes (any
eager scope.)

For example in the following code we infer type of `z` correctly:

```py
class C:
    def __init__(self):
        [None for self.z in range(1)]
reveal_type(C().z) # previously [unresolved-attribute] but now shows Unknown | int
```

The fix works by adjusting the following logics:

To identify if an attriute is an assignment to self or cls we need to
check the scope is a method. To allow comprehension scopes here we skip
any eager scope in the check.
Also at this stage the code checks if self or the first method argument
is shadowed by another binding that eager scope to prevent this:

```py
class D:
    g: int

class C:
    def __init__(self):
        [[None for self.g in range(1)] for self in [D()]]
reveal_type(C().g) # [unresolved-attribute]
```

When determining scopes that attributes might be defined after
collecting all the methods of the class the code also returns any
decendant scope that is eager and only has eager parents until the
method scope.

When checking reachability of a attribute definition if the attribute is
defined in an eager scope we use the reachability of the first non eager
scope which must be a method. This allows attributes to be marked as
reachable and be seen.


There are also which I didn't add support for:

```py
class C:
    def __init__(self):
        def f():
            [None for self.z in range(1)]
        f()

reveal_type(C().z) # [unresolved-attribute]
```

In the above example we will not even return the comprehension scope as
an attribute scope because there is a non eager scope (`f` function)
between the comprehension and the `__init__` method

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-11-11 14:45:34 -08:00
Alex Waygood
43297d3455
[ty] Support isinstance() and issubclass() narrowing when the second argument is a typing.py stdlib alias (#21391)
## Summary

A followup to https://github.com/astral-sh/ruff/pull/21386

## Test Plan

New mdtests added
2025-11-11 21:09:24 +00:00
Mahmoud Saada
4373974dd9
[ty] Fix false positive for Final attribute assignment in __init__ (#21158)
## Summary

Fixes https://github.com/astral-sh/ty/issues/1409

This PR allows `Final` instance attributes to be initialized in
`__init__` methods, as mandated by the Python typing specification (PEP
591). Previously, ty incorrectly prevented this initialization, causing
false positive errors.

The fix checks if we're inside an `__init__` method before rejecting
Final attribute assignments, allowing assignments during
instance initialization while still preventing reassignment elsewhere.

## Test Plan

- Added new test coverage in `final.md` for the reported issue with
`Self` annotations
- Updated existing tests that were incorrectly expecting errors 
- All 278 mdtest tests pass
- Manually tested with real-world code examples

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-11-11 12:54:05 -08:00
Aria Desires
e4374f14ed
[ty] Consider from thispackage import y to re-export y in __init__.pyi (#21387)
Fixes https://github.com/astral-sh/ty/issues/1487

This one is a true extension of non-standard semantics, and is therefore
a certified Hot Take we might conclude is simply a Bad Take (let's see
what ecosystem tests say...).
2025-11-11 14:41:14 -05:00
Alex Waygood
03bd0619e9
[ty] Silence false-positive diagnostics when using typing.Dict or typing.Callable as the second argument to isinstance() (#21386) 2025-11-11 19:30:01 +00:00
Aria Desires
bd8812127d
[ty] support absolute from imports introducing local submodules in __init__.py files (#21372)
By resolving `.` and the LHS of the from import during semantic
indexing, we can check if the LHS is a submodule of `.`, and handle
`from whatever.thispackage.x.y import z` exactly like we do `from .x.y
import z`.

Fixes https://github.com/astral-sh/ty/issues/1484
2025-11-11 13:04:42 -05:00
Alex Waygood
44b0c9ebac
[ty] Allow PEP-604 unions in stubs and TYPE_CHECKING blocks prior to 3.10 (#21379)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
2025-11-11 14:33:43 +00:00
Douglas Creager
33b942c7ad
[ty] Handle annotated self parameter in constructor of non-invariant generic classes (#21325)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
This manifested as an error when inferring the type of a PEP-695 generic
class via its constructor parameters:

```py
class D[T, U]:
    @overload
    def __init__(self: "D[str, U]", u: U) -> None: ...
    @overload
    def __init__(self, t: T, u: U) -> None: ...
    def __init__(self, *args) -> None: ...

# revealed: D[Unknown, str]
# SHOULD BE: D[str, str]
reveal_type(D("string"))
```

This manifested because `D` is inferred to be bivariant in both `T` and
`U`. We weren't seeing this in the equivalent example for legacy
typevars, since those default to invariant. (This issue also showed up
for _covariant_ typevars, so this issue was not limited to bivariance.)

The underlying cause was because of a heuristic that we have in our
current constraint solver, which attempts to handle situations like
this:

```py
def f[T](t: T | None): ...
f(None)
```

Here, the `None` argument matches the non-typevar union element, so this
argument should not add any constraints on what `T` can specialize to.
Our previous heuristic would check for this by seeing if the argument
type is a subtype of the parameter annotation as a whole — even if it
isn't a union! That would cause us to erroneously ignore the `self`
parameter in our constructor call, since bivariant classes are
equivalent to each other, regardless of their specializations.

The quick fix is to move this heuristic "down a level", so that we only
apply it when the parameter annotation is a union. This heuristic should
go away completely 🤞 with the new constraint solver.
2025-11-10 19:46:49 -05:00
Aria Desires
9ce3230add
[ty] Make implicit submodule imports only occur in global scope (#21370)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Blocked by required conditions
CI / cargo test (macos-latest) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
This loses any ability to have "per-function" implicit submodule
imports, to avoid the "ok but now we need per-scope imports" and "ok but
this should actually introduce a global that only exists during this
function" problems. A simple and clean implementation with no weird
corners.

Fixes https://github.com/astral-sh/ty/issues/1482
2025-11-10 18:59:48 -05:00
Aria Desires
2bc6c78e26
[ty] introduce local variables for from imports of submodules in __init__.py(i) (#21173)
This rips out the previous implementation in favour of a new
implementation with 3 rules:

- **froms are locals**: a `from..import` can only define locals, it does
not have global
side-effects. Specifically any submodule attribute `a` that's implicitly
introduced by either
`from .a import b` or `from . import a as b` (in an `__init__.py(i)`) is
a local and not a
global. If you do such an import at the top of a file you won't notice
this. However if you do
such an import in a function, that means it will only be function-scoped
(so you'll need to do
it in every function that wants to access it, making your code less
sensitive to execution
    order).

- **first from first serve**: only the *first* `from..import` in an
`__init__.py(i)` that imports a
particular direct submodule of the current package introduces that
submodule as a local.
Subsequent imports of the submodule will not introduce that local. This
reflects the fact that
in actual python only the first import of a submodule (in the entire
execution of the program)
introduces it as an attribute of the package. By "first" we mean "the
first time in this scope
(or any parent scope)". This pairs well with the fact that we are
specifically introducing a
local (as long as you don't accidentally shadow or overwrite the local).

- **dot re-exports**: `from . import a` in an `__init__.pyi` is
considered a re-export of `a`
(equivalent to `from . import a as a`). This is required to properly
handle many stubs in the
    wild. Currently it must be *exactly* `from . import ...`.
    
This implementation is intentionally limited/conservative (notably,
often requiring a from import to be relative). I'm going to file a ton
of followups for improvements so that their impact can be evaluated
separately.


Fixes https://github.com/astral-sh/ty/issues/133
2025-11-10 23:04:56 +00:00
Jack O'Connor
5f3e086ee4 [ty] implement typing.NewType by adding Type::NewTypeInstance 2025-11-10 14:55:47 -08:00
Ibraheem Ahmed
3656b44877
[ty] Use type context for inference of generic constructors (#20933)
## Summary

Resolves https://github.com/astral-sh/ty/issues/1228.

This PR is stacked on https://github.com/astral-sh/ruff/pull/21210.
2025-11-10 21:49:48 +00:00
Ibraheem Ahmed
98869f0307
[ty] Improve generic call expression inference (#21210)
## Summary

Implements https://github.com/astral-sh/ty/issues/1356 and https://github.com/astral-sh/ty/issues/136#issuecomment-3413669994.
2025-11-10 21:29:05 +00:00
justin
f63a9f2334
[ty] Fix incorrect inference of enum.auto() for enums with non-int mixins, and imprecise inference of enum.auto() for single-member enums (#20541)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2025-11-10 17:53:08 +00:00
David Peter
ab46c8de0f
[ty] Add support for properties that return Self (#21335)
## Summary

Detect usages of implicit `self` in property getters, which allows us to
treat their signature as being generic.

closes https://github.com/astral-sh/ty/issues/1502

## Typing conformance

Two new type assertions that are succeeding.

## Ecosystem results

Mostly look good. There are a few new false positives related to a bug
with constrained typevars that is unrelated to the work here. I reported
this as https://github.com/astral-sh/ty/issues/1503.

## Test Plan

Added regression tests.
2025-11-10 11:13:36 +01:00
David Peter
238f151371
[ty] Add support for Optional and Annotated in implicit type aliases (#21321)
## Summary

Add support for `Optional` and `Annotated` in implicit type aliases

part of https://github.com/astral-sh/ty/issues/221

## Typing conformance changes

New expected diagnostics.

## Ecosystem

A lot of true positives, some known limitations unrelated to this PR.

## Test Plan

New Markdown tests
2025-11-10 10:24:38 +01:00
Alex Waygood
73b1fce74a
[ty] Add diagnostics for isinstance() and issubclass() calls that use invalid PEP-604 unions for their second argument (#21343)
## Summary

This PR adds extra validation for `isinstance()` and `issubclass()`
calls that use `UnionType` instances for their second argument.
According to typeshed's annotations, any `UnionType` is accepted for the
second argument, but this isn't true at runtime: at runtime, all
elements in the `UnionType` must either be class objects or be `None` in
order for the `isinstance()` or `issubclass()` call to reliably succeed:

```pycon
% uvx python3.14                            
Python 3.14.0 (main, Oct 10 2025, 12:54:13) [Clang 20.1.4 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import LiteralString
>>> import types
>>> type(LiteralString | int) is types.UnionType
True
>>> isinstance(42, LiteralString | int)
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    isinstance(42, LiteralString | int)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/alexw/Library/Application Support/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/typing.py", line 559, in __instancecheck__
    raise TypeError(f"{self} cannot be used with isinstance()")
TypeError: typing.LiteralString cannot be used with isinstance()
```

## Test Plan

Added mdtests/snapshots
2025-11-10 08:46:31 +00:00
Alex Waygood
020ff1723b
[ty] Add narrowing for isinstance() and issubclass() checks that use PEP-604 unions (#21334)
Some checks failed
CI / Determine changes (push) Has been cancelled
CI / cargo fmt (push) Has been cancelled
CI / python package (push) Has been cancelled
CI / pre-commit (push) Has been cancelled
CI / mkdocs (push) Has been cancelled
[ty Playground] Release / publish (push) Has been cancelled
CI / cargo clippy (push) Has been cancelled
CI / cargo test (linux) (push) Has been cancelled
CI / cargo test (linux, release) (push) Has been cancelled
CI / cargo test (${{ github.repository == 'astral-sh/ruff' && 'depot-windows-2022-16' || 'windows-latest' }}) (push) Has been cancelled
CI / cargo test (macos-latest) (push) Has been cancelled
CI / test ruff-lsp (push) Has been cancelled
CI / cargo test (wasm) (push) Has been cancelled
CI / cargo build (msrv) (push) Has been cancelled
CI / cargo fuzz build (push) Has been cancelled
CI / fuzz parser (push) Has been cancelled
CI / test scripts (push) Has been cancelled
CI / ecosystem (push) Has been cancelled
CI / Fuzz for new ty panics (push) Has been cancelled
CI / cargo shear (push) Has been cancelled
CI / ty completion evaluation (push) Has been cancelled
CI / formatter instabilities and black similarity (push) Has been cancelled
CI / check playground (push) Has been cancelled
CI / benchmarks instrumented (ruff) (push) Has been cancelled
CI / benchmarks instrumented (ty) (push) Has been cancelled
CI / benchmarks walltime (medium|multithreaded) (push) Has been cancelled
CI / benchmarks walltime (small|large) (push) Has been cancelled
2025-11-08 18:20:46 +00:00
Douglas Creager
faae72b836
[ty] Clarify behavior of constraint sets for gradual upper bounds and constraints (#21287)
When checking whether a constraint set is satisfied, if a typevar has a
non-fully-static upper bound or constraint, we are free to choose any
materialization that makes the check succeed.

In non-inferable positions, we have to show that the constraint set is
satisfied for all valid specializations, so it's best to choose the most
restrictive materialization, since that minimizes the set of valid
specializations that have to pass.

In inferable positions, we only have to show that the constraint set is
satisfied for _some_ valid specializations, so it's best to choose the
most permissive materialization, since that maximizes our chances of
finding a specialization that passes.
2025-11-07 14:01:39 -05:00
David Peter
ed18112cfa
[ty] Add support for Literals in implicit type aliases (#21296)
## Summary

Add support for `Literal` types in implicit type aliases.

part of https://github.com/astral-sh/ty/issues/221

## Ecosystem analysis

This looks good to me, true positives and known problems.

## Test Plan

New Markdown tests.
2025-11-07 17:46:55 +01:00