ruff/crates/ty_python_semantic/resources/mdtest/generics/legacy/variables.md
Carl Meyer 8248193ed9
[ty] defer inference of legacy TypeVar bound/constraints/defaults (#20598)
## Summary

This allows us to handle self-referential bounds/constraints/defaults
without panicking.

Handles more cases from https://github.com/astral-sh/ty/issues/256

This also changes the way we infer the types of legacy TypeVars. Rather
than understanding a constructor call to `typing[_extension].TypeVar`
inside of any (arbitrarily nested) expression, and having to use a
special `assigned_to` field of the semantic index to try to best-effort
figure out what name the typevar was assigned to, we instead understand
the creation of a legacy `TypeVar` only in the supported syntactic
position (RHS of a simple un-annotated assignment with one target). In
any other position, we just infer it as creating an opaque instance of
`typing.TypeVar`. (This behavior matches all other type checkers.)

So we now special-case TypeVar creation in `TypeInferenceBuilder`, as a
special case of an assignment definition, rather than deeper inside call
binding. This does mean we re-implement slightly more of
argument-parsing, but in practice this is minimal and easy to handle
correctly.

This is easier to implement if we also make the RHS of a simple (no
unpacking) one-target assignment statement no longer a standalone
expression. Which is fine to do, because simple one-target assignments
don't need to infer the RHS more than once. This is a bonus performance
(0-3% across various projects) and significant memory-usage win, since
most assignment statements are simple one-target assignment statements,
meaning we now create many fewer standalone-expression salsa
ingredients.

This change does mean that inference of manually-constructed
`TypeAliasType` instances can no longer find its Definition in
`assigned_to`, which regresses go-to-definition for these aliases. In a
future PR, `TypeAliasType` will receive the same treatment that
`TypeVar` did in this PR (moving its special-case inference into
`TypeInferenceBuilder` and supporting it only in the correct syntactic
position, and lazily inferring its value type to support recursion),
which will also fix the go-to-definition regression. (I decided a
temporary edge-case regression is better in this case than doubling the
size of this PR.)

This PR also tightens up and fixes various aspects of the validation of
`TypeVar` creation, as seen in the tests.

We still (for now) treat all typevars as instances of `typing.TypeVar`,
even if they were created using `typing_extensions.TypeVar`. This means
we'll wrongly error on e.g. `T.__default__` on Python 3.11, even if `T`
is a `typing_extensions.TypeVar` instance at runtime. We share this
wrong behavior with both mypy and pyrefly. It will be easier to fix
after we pull in https://github.com/python/typeshed/pull/14840.

There are some issues that showed up here with typevar identity and
`MarkTypeVarsInferable`; the fix here (using the new `original` field
and `is_identical_to` methods on `BoundTypeVarInstance` and
`TypeVarInstance`) is a bit kludgy, but it can go away when we eliminate
`MarkTypeVarsInferable`.

## Test Plan

Added and updated mdtests.

### Conformance suite impact

The impact here is all positive:

* We now correctly error on a legacy TypeVar with exactly one constraint
type given.
* We now correctly error on a legacy TypeVar with both an upper bound
and constraints specified.

### Ecosystem impact

Basically none; in the setuptools case we just issue slightly different
errors on an invalid TypeVar definition, due to the modified validation
code.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2025-10-09 21:08:37 +00:00

10 KiB

Legacy type variables

The tests in this file focus on how type variables are defined using the legacy notation. Most uses of type variables are tested in other files in this directory; we do not duplicate every test for both type variable syntaxes.

Unless otherwise specified, all quotations come from the Generics section of the typing spec.

Diagnostics for invalid type variables are snapshotted in diagnostics/legacy_typevars.md.

Type variables

Defining legacy type variables

Generics can be parameterized by using a factory available in typing called TypeVar.

This was the only way to create type variables prior to PEP 695/Python 3.12. It is still available in newer Python releases.

from typing import TypeVar

T = TypeVar("T")
reveal_type(type(T))  # revealed: <class 'TypeVar'>
reveal_type(T)  # revealed: typing.TypeVar
reveal_type(T.__name__)  # revealed: Literal["T"]

The typevar name can also be provided as a keyword argument:

from typing import TypeVar

T = TypeVar(name="T")
reveal_type(T.__name__)  # revealed: Literal["T"]

Must be directly assigned to a variable

A TypeVar() expression must always directly be assigned to a variable (it should not be used as part of a larger expression).

from typing import TypeVar

T = TypeVar("T")
# error: [invalid-legacy-type-variable]
U: TypeVar = TypeVar("U")

# error: [invalid-legacy-type-variable]
tuple_with_typevar = ("foo", TypeVar("W"))
reveal_type(tuple_with_typevar[1])  # revealed: TypeVar
from typing_extensions import TypeVar

T = TypeVar("T")
# error: [invalid-legacy-type-variable]
U: TypeVar = TypeVar("U")

# error: [invalid-legacy-type-variable]
tuple_with_typevar = ("foo", TypeVar("W"))
reveal_type(tuple_with_typevar[1])  # revealed: TypeVar

TypeVar parameter must match variable name

The argument to TypeVar() must be a string equal to the variable name to which it is assigned.

from typing import TypeVar

# error: [invalid-legacy-type-variable]
T = TypeVar("Q")

No redefinition

Type variables must not be redefined.

from typing import TypeVar

T = TypeVar("T")

# TODO: error
T = TypeVar("T")

No variadic arguments

from typing import TypeVar

types = (int, str)

# error: [invalid-legacy-type-variable]
T = TypeVar("T", *types)
reveal_type(T)  # revealed: TypeVar

# error: [invalid-legacy-type-variable]
S = TypeVar("S", **{"bound": int})
reveal_type(S)  # revealed: TypeVar

Type variables with a default

Note that the __default__ property is only available in Python ≥3.13.

[environment]
python-version = "3.13"
from typing import TypeVar

T = TypeVar("T", default=int)
reveal_type(type(T))  # revealed: <class 'TypeVar'>
reveal_type(T)  # revealed: typing.TypeVar
reveal_type(T.__default__)  # revealed: int
reveal_type(T.__bound__)  # revealed: None
reveal_type(T.__constraints__)  # revealed: tuple[()]

S = TypeVar("S")
reveal_type(S.__default__)  # revealed: NoDefault

Using other typevars as a default

[environment]
python-version = "3.13"
from typing import Generic, TypeVar, Union

T = TypeVar("T")
U = TypeVar("U", default=T)
V = TypeVar("V", default=Union[T, U])

class Valid(Generic[T, U, V]): ...

reveal_type(Valid())  # revealed: Valid[Unknown, Unknown, Unknown]
reveal_type(Valid[int]())  # revealed: Valid[int, int, int]
reveal_type(Valid[int, str]())  # revealed: Valid[int, str, int | str]
reveal_type(Valid[int, str, None]())  # revealed: Valid[int, str, None]

# TODO: error, default value for U isn't available in the generic context
class Invalid(Generic[U]): ...

Type variables with an upper bound

from typing import TypeVar

T = TypeVar("T", bound=int)
reveal_type(type(T))  # revealed: <class 'TypeVar'>
reveal_type(T)  # revealed: typing.TypeVar
reveal_type(T.__bound__)  # revealed: int
reveal_type(T.__constraints__)  # revealed: tuple[()]

S = TypeVar("S")
reveal_type(S.__bound__)  # revealed: None

The upper bound must be a valid type expression:

from typing import TypedDict

# error: [invalid-type-form]
T = TypeVar("T", bound=TypedDict)

Type variables with constraints

from typing import TypeVar

T = TypeVar("T", int, str)
reveal_type(type(T))  # revealed: <class 'TypeVar'>
reveal_type(T)  # revealed: typing.TypeVar
reveal_type(T.__constraints__)  # revealed: tuple[int, str]

S = TypeVar("S")
reveal_type(S.__constraints__)  # revealed: tuple[()]

Constraints are not simplified relative to each other, even if one is a subtype of the other:

T = TypeVar("T", int, bool)
reveal_type(T.__constraints__)  # revealed: tuple[int, bool]

S = TypeVar("S", float, str)
reveal_type(S.__constraints__)  # revealed: tuple[int | float, str]

Cannot have only one constraint

TypeVar supports constraining parametric types to a fixed set of possible types...There should be at least two constraints, if any; specifying a single constraint is disallowed.

from typing import TypeVar

# error: [invalid-legacy-type-variable]
T = TypeVar("T", int)

Cannot have both bound and constraint

from typing import TypeVar

# error: [invalid-legacy-type-variable]
T = TypeVar("T", int, str, bound=bytes)

Cannot be both covariant and contravariant

To facilitate the declaration of container types where covariant or contravariant type checking is acceptable, type variables accept keyword arguments covariant=True or contravariant=True. At most one of these may be passed.

from typing import TypeVar

# error: [invalid-legacy-type-variable]
T = TypeVar("T", covariant=True, contravariant=True)

Boolean parameters must be unambiguous

from typing_extensions import TypeVar

def cond() -> bool:
    return True

# error: [invalid-legacy-type-variable]
T = TypeVar("T", covariant=cond())

# error: [invalid-legacy-type-variable]
U = TypeVar("U", contravariant=cond())

# error: [invalid-legacy-type-variable]
V = TypeVar("V", infer_variance=cond())

Invalid keyword arguments

from typing import TypeVar

# error: [invalid-legacy-type-variable]
T = TypeVar("T", invalid_keyword=True)
from typing import TypeVar

# error: [invalid-legacy-type-variable]
T = TypeVar("T", invalid_keyword=True)

Constructor signature versioning

For typing.TypeVar

[environment]
python-version = "3.10"

In a stub file, features from the latest supported Python version can be used on any version. There's no need to require use of typing_extensions.TypeVar in a stub file, when the type checker can understand the typevar definition perfectly well either way, and there can be no runtime error. (Perhaps it's arguable whether this special case is worth it, but other type checkers do it, so we maintain compatibility.)

from typing import TypeVar
T = TypeVar("T", default=int)

But this raises an error in a non-stub file:

from typing import TypeVar

# error: [invalid-legacy-type-variable]
T = TypeVar("T", default=int)

For typing_extensions.TypeVar

typing_extensions.TypeVar always supports the latest features, on any Python version.

[environment]
python-version = "3.10"
from typing_extensions import TypeVar

T = TypeVar("T", default=int)
# TODO: should not error, should reveal `int`
# error: [unresolved-attribute]
reveal_type(T.__default__)  # revealed: Unknown

Callability

A typevar bound to a Callable type is callable:

from typing import Callable, TypeVar

T = TypeVar("T", bound=Callable[[], int])

def bound(f: T):
    reveal_type(f)  # revealed: T@bound
    reveal_type(f())  # revealed: int

Same with a constrained typevar, as long as all constraints are callable:

T = TypeVar("T", Callable[[], int], Callable[[], str])

def constrained(f: T):
    reveal_type(f)  # revealed: T@constrained
    reveal_type(f())  # revealed: int | str

Meta-type

The meta-type of a typevar is the same as the meta-type of the upper bound, or the union of the meta-types of the constraints:

from typing import TypeVar

T_normal = TypeVar("T_normal")

def normal(x: T_normal):
    reveal_type(type(x))  # revealed: type

T_bound_object = TypeVar("T_bound_object", bound=object)

def bound_object(x: T_bound_object):
    reveal_type(type(x))  # revealed: type

T_bound_int = TypeVar("T_bound_int", bound=int)

def bound_int(x: T_bound_int):
    reveal_type(type(x))  # revealed: type[int]

T_constrained = TypeVar("T_constrained", int, str)

def constrained(x: T_constrained):
    reveal_type(type(x))  # revealed: type[int] | type[str]

Cycles

Bounds and constraints

A typevar's bounds and constraints cannot be generic, cyclic or otherwise:

from typing import Any, TypeVar

S = TypeVar("S")

# TODO: error
T = TypeVar("T", bound=list[S])

# TODO: error
U = TypeVar("U", list["T"], str)

# TODO: error
V = TypeVar("V", list["V"], str)

However, they are lazily evaluated and can cyclically refer to their own type:

from typing import TypeVar, Generic

T = TypeVar("T", bound=list["G"])

class G(Generic[T]):
    x: T

reveal_type(G[list[G]]().x)  # revealed: list[G[Unknown]]

Defaults

[environment]
python-version = "3.13"

Defaults can be generic, but can only refer to typevars from the same scope if they were defined earlier in that scope:

from typing import Generic, TypeVar

T = TypeVar("T")
U = TypeVar("U", default=T)

class C(Generic[T, U]):
    x: T
    y: U

reveal_type(C[int, str]().x)  # revealed: int
reveal_type(C[int, str]().y)  # revealed: str
reveal_type(C[int]().x)  # revealed: int
reveal_type(C[int]().y)  # revealed: int

# TODO: error
V = TypeVar("V", default="V")

class D(Generic[V]):
    x: V

# TODO: we shouldn't leak a typevar like this in type inference
reveal_type(D().x)  # revealed: V@D

Regression

Use of typevar with default inside a function body that binds it

[environment]
python-version = "3.13"
from typing import Generic, TypeVar

_DataT = TypeVar("_DataT", bound=int, default=int)

class Event(Generic[_DataT]):
    def __init__(self, data: _DataT) -> None:
        self.data = data

def async_fire_internal(event_data: _DataT):
    event: Event[_DataT] | None = None
    event = Event(event_data)