mirror of
https://github.com/astral-sh/ruff.git
synced 2025-10-23 00:31:55 +00:00
5a570c8e6d
## Summary
For PEP 695 generic functions and classes, there is an extra "type
params scope" (a child of the outer scope, and wrapping the body scope)
in which the type parameters are defined; class bases and function
parameter/return annotations are resolved in that type-params scope.
This PR fixes some longstanding bugs in how we resolve name loads from
inside these PEP 695 type parameter scopes, and also defers type
inference of PEP 695 typevar bounds/constraints/default, so we can
handle cycles without panicking.
We were previously treating these type-param scopes as lazy nested
scopes, which is wrong. In fact they are eager nested scopes; the class
`C` here inherits `int`, not `str`, and previously we got that wrong:
```py
Base = int
class C[T](Base): ...
Base = str
```
But certain syntactic positions within type param scopes (typevar
bounds/constraints/defaults) are lazy at runtime, and we should use
deferred name resolution for them. This also means they can have cycles;
in order to handle that without panicking in type inference, we need to
actually defer their type inference until after we have constructed the
`TypeVarInstance`.
PEP 695 does specify that typevar bounds and constraints cannot be
generic, and that typevar defaults can only reference prior typevars,
not later ones. This reduces the scope of (valid from the type-system
perspective) cycles somewhat, although cycles are still possible (e.g.
`class C[T: list[C]]`). And this is a type-system-only restriction; from
the runtime perspective an "invalid" case like `class C[T: T]` actually
works fine.
I debated whether to implement the PEP 695 restrictions as a way to
avoid some cycles up-front, but I ended up deciding against that; I'd
rather model the runtime name-resolution semantics accurately, and
implement the PEP 695 restrictions as a separate diagnostic on top.
(This PR doesn't yet implement those diagnostics, thus some `# TODO:
error` in the added tests.)
Introducing the possibility of cyclic typevars made typevar display
potentially stack overflow. For now I've handled this by simply removing
typevar details (bounds/constraints/default) from typevar display. This
impacts display of two kinds of types. If you `reveal_type(T)` on an
unbound `T` you now get just `typing.TypeVar` instead of
`typing.TypeVar("T", ...)` where `...` is the bound/constraints/default.
This matches pyright and mypy; pyrefly uses `type[TypeVar[T]]` which
seems a bit confusing, but does include the name. (We could easily
include the name without cycle issues, if there's a syntax we like for
that.)
It also means that displaying a generic function type like `def f[T:
int](x: T) -> T: ...` now displays as `f[T](x: T) -> T` instead of `f[T:
int](x: T) -> T`. This matches pyright and pyrefly; mypy does include
bound/constraints/defaults of typevars in function/callable type
display. If we wanted to add this, we would either need to thread a
visitor through all the type display code, or add a `decycle` type
transformation that replaced recursive reoccurrence of a type with a
marker.
## Test Plan
Added mdtests and modified existing tests to improve their correctness.
After this PR, there's only a single remaining py-fuzzer seed in the
0-500 range that panics! (Before this PR, there were 10; the fuzzer
likes to generate cyclic PEP 695 syntax.)
## Ecosystem report
It's all just the changes to `TypeVar` display.
5.8 KiB
5.8 KiB
Legacy type variables
The tests in this file focus on how type variables are defined using the legacy notation. Most uses of type variables are tested in other files in this directory; we do not duplicate every test for both type variable syntaxes.
Unless otherwise specified, all quotations come from the Generics section of the typing spec.
Type variables
Defining legacy type variables
Generics can be parameterized by using a factory available in
typingcalledTypeVar.
This was the only way to create type variables prior to PEP 695/Python 3.12. It is still available in newer Python releases.
from typing import TypeVar
T = TypeVar("T")
reveal_type(type(T)) # revealed: <class 'TypeVar'>
reveal_type(T) # revealed: typing.TypeVar
reveal_type(T.__name__) # revealed: Literal["T"]
Directly assigned to a variable
A
TypeVar()expression must always directly be assigned to a variable (it should not be used as part of a larger expression).
from typing import TypeVar
T = TypeVar("T")
# TODO: no error
# error: [invalid-legacy-type-variable]
U: TypeVar = TypeVar("U")
# error: [invalid-legacy-type-variable] "A legacy `typing.TypeVar` must be immediately assigned to a variable"
# error: [invalid-type-form] "Function calls are not allowed in type expressions"
TestList = list[TypeVar("W")]
TypeVar parameter must match variable name
The argument to
TypeVar()must be a string equal to the variable name to which it is assigned.
from typing import TypeVar
# error: [invalid-legacy-type-variable] "The name of a legacy `typing.TypeVar` (`Q`) must match the name of the variable it is assigned to (`T`)"
T = TypeVar("Q")
No redefinition
Type variables must not be redefined.
from typing import TypeVar
T = TypeVar("T")
# TODO: error
T = TypeVar("T")
Type variables with a default
Note that the __default__ property is only available in Python ≥3.13.
[environment]
python-version = "3.13"
from typing import TypeVar
T = TypeVar("T", default=int)
reveal_type(type(T)) # revealed: <class 'TypeVar'>
reveal_type(T) # revealed: typing.TypeVar
reveal_type(T.__default__) # revealed: int
reveal_type(T.__bound__) # revealed: None
reveal_type(T.__constraints__) # revealed: tuple[()]
S = TypeVar("S")
reveal_type(S.__default__) # revealed: NoDefault
Using other typevars as a default
from typing import Generic, TypeVar, Union
T = TypeVar("T")
U = TypeVar("U", default=T)
V = TypeVar("V", default=Union[T, U])
class Valid(Generic[T, U, V]): ...
reveal_type(Valid()) # revealed: Valid[Unknown, Unknown, Unknown]
reveal_type(Valid[int]()) # revealed: Valid[int, int, int]
reveal_type(Valid[int, str]()) # revealed: Valid[int, str, int | str]
reveal_type(Valid[int, str, None]()) # revealed: Valid[int, str, None]
# TODO: error, default value for U isn't available in the generic context
class Invalid(Generic[U]): ...
Type variables with an upper bound
from typing import TypeVar
T = TypeVar("T", bound=int)
reveal_type(type(T)) # revealed: <class 'TypeVar'>
reveal_type(T) # revealed: typing.TypeVar
reveal_type(T.__bound__) # revealed: int
reveal_type(T.__constraints__) # revealed: tuple[()]
S = TypeVar("S")
reveal_type(S.__bound__) # revealed: None
Type variables with constraints
from typing import TypeVar
T = TypeVar("T", int, str)
reveal_type(type(T)) # revealed: <class 'TypeVar'>
reveal_type(T) # revealed: typing.TypeVar
reveal_type(T.__constraints__) # revealed: tuple[int, str]
S = TypeVar("S")
reveal_type(S.__constraints__) # revealed: tuple[()]
Cannot have only one constraint
TypeVarsupports constraining parametric types to a fixed set of possible types...There should be at least two constraints, if any; specifying a single constraint is disallowed.
from typing import TypeVar
# TODO: error: [invalid-type-variable-constraints]
T = TypeVar("T", int)
Cannot be both covariant and contravariant
To facilitate the declaration of container types where covariant or contravariant type checking is acceptable, type variables accept keyword arguments
covariant=Trueorcontravariant=True. At most one of these may be passed.
from typing import TypeVar
# error: [invalid-legacy-type-variable]
T = TypeVar("T", covariant=True, contravariant=True)
Variance parameters must be unambiguous
from typing import TypeVar
def cond() -> bool:
return True
# error: [invalid-legacy-type-variable]
T = TypeVar("T", covariant=cond())
# error: [invalid-legacy-type-variable]
U = TypeVar("U", contravariant=cond())
Callability
A typevar bound to a Callable type is callable:
from typing import Callable, TypeVar
T = TypeVar("T", bound=Callable[[], int])
def bound(f: T):
reveal_type(f) # revealed: T@bound
reveal_type(f()) # revealed: int
Same with a constrained typevar, as long as all constraints are callable:
T = TypeVar("T", Callable[[], int], Callable[[], str])
def constrained(f: T):
reveal_type(f) # revealed: T@constrained
reveal_type(f()) # revealed: int | str
Meta-type
The meta-type of a typevar is the same as the meta-type of the upper bound, or the union of the meta-types of the constraints:
from typing import TypeVar
T_normal = TypeVar("T_normal")
def normal(x: T_normal):
reveal_type(type(x)) # revealed: type
T_bound_object = TypeVar("T_bound_object", bound=object)
def bound_object(x: T_bound_object):
reveal_type(type(x)) # revealed: type
T_bound_int = TypeVar("T_bound_int", bound=int)
def bound_int(x: T_bound_int):
reveal_type(type(x)) # revealed: type[int]
T_constrained = TypeVar("T_constrained", int, str)
def constrained(x: T_constrained):
reveal_type(type(x)) # revealed: type[int] | type[str]