## Summary This allows us to handle self-referential bounds/constraints/defaults without panicking. Handles more cases from https://github.com/astral-sh/ty/issues/256 This also changes the way we infer the types of legacy TypeVars. Rather than understanding a constructor call to `typing[_extension].TypeVar` inside of any (arbitrarily nested) expression, and having to use a special `assigned_to` field of the semantic index to try to best-effort figure out what name the typevar was assigned to, we instead understand the creation of a legacy `TypeVar` only in the supported syntactic position (RHS of a simple un-annotated assignment with one target). In any other position, we just infer it as creating an opaque instance of `typing.TypeVar`. (This behavior matches all other type checkers.) So we now special-case TypeVar creation in `TypeInferenceBuilder`, as a special case of an assignment definition, rather than deeper inside call binding. This does mean we re-implement slightly more of argument-parsing, but in practice this is minimal and easy to handle correctly. This is easier to implement if we also make the RHS of a simple (no unpacking) one-target assignment statement no longer a standalone expression. Which is fine to do, because simple one-target assignments don't need to infer the RHS more than once. This is a bonus performance (0-3% across various projects) and significant memory-usage win, since most assignment statements are simple one-target assignment statements, meaning we now create many fewer standalone-expression salsa ingredients. This change does mean that inference of manually-constructed `TypeAliasType` instances can no longer find its Definition in `assigned_to`, which regresses go-to-definition for these aliases. In a future PR, `TypeAliasType` will receive the same treatment that `TypeVar` did in this PR (moving its special-case inference into `TypeInferenceBuilder` and supporting it only in the correct syntactic position, and lazily inferring its value type to support recursion), which will also fix the go-to-definition regression. (I decided a temporary edge-case regression is better in this case than doubling the size of this PR.) This PR also tightens up and fixes various aspects of the validation of `TypeVar` creation, as seen in the tests. We still (for now) treat all typevars as instances of `typing.TypeVar`, even if they were created using `typing_extensions.TypeVar`. This means we'll wrongly error on e.g. `T.__default__` on Python 3.11, even if `T` is a `typing_extensions.TypeVar` instance at runtime. We share this wrong behavior with both mypy and pyrefly. It will be easier to fix after we pull in https://github.com/python/typeshed/pull/14840. There are some issues that showed up here with typevar identity and `MarkTypeVarsInferable`; the fix here (using the new `original` field and `is_identical_to` methods on `BoundTypeVarInstance` and `TypeVarInstance`) is a bit kludgy, but it can go away when we eliminate `MarkTypeVarsInferable`. ## Test Plan Added and updated mdtests. ### Conformance suite impact The impact here is all positive: * We now correctly error on a legacy TypeVar with exactly one constraint type given. * We now correctly error on a legacy TypeVar with both an upper bound and constraints specified. ### Ecosystem impact Basically none; in the setuptools case we just issue slightly different errors on an invalid TypeVar definition, due to the modified validation code. --------- Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
10 KiB
Legacy type variables
The tests in this file focus on how type variables are defined using the legacy notation. Most uses of type variables are tested in other files in this directory; we do not duplicate every test for both type variable syntaxes.
Unless otherwise specified, all quotations come from the Generics section of the typing spec.
Diagnostics for invalid type variables are snapshotted in diagnostics/legacy_typevars.md.
Type variables
Defining legacy type variables
Generics can be parameterized by using a factory available in
typingcalledTypeVar.
This was the only way to create type variables prior to PEP 695/Python 3.12. It is still available in newer Python releases.
from typing import TypeVar
T = TypeVar("T")
reveal_type(type(T)) # revealed: <class 'TypeVar'>
reveal_type(T) # revealed: typing.TypeVar
reveal_type(T.__name__) # revealed: Literal["T"]
The typevar name can also be provided as a keyword argument:
from typing import TypeVar
T = TypeVar(name="T")
reveal_type(T.__name__) # revealed: Literal["T"]
Must be directly assigned to a variable
A
TypeVar()expression must always directly be assigned to a variable (it should not be used as part of a larger expression).
from typing import TypeVar
T = TypeVar("T")
# error: [invalid-legacy-type-variable]
U: TypeVar = TypeVar("U")
# error: [invalid-legacy-type-variable]
tuple_with_typevar = ("foo", TypeVar("W"))
reveal_type(tuple_with_typevar[1]) # revealed: TypeVar
from typing_extensions import TypeVar
T = TypeVar("T")
# error: [invalid-legacy-type-variable]
U: TypeVar = TypeVar("U")
# error: [invalid-legacy-type-variable]
tuple_with_typevar = ("foo", TypeVar("W"))
reveal_type(tuple_with_typevar[1]) # revealed: TypeVar
TypeVar parameter must match variable name
The argument to
TypeVar()must be a string equal to the variable name to which it is assigned.
from typing import TypeVar
# error: [invalid-legacy-type-variable]
T = TypeVar("Q")
No redefinition
Type variables must not be redefined.
from typing import TypeVar
T = TypeVar("T")
# TODO: error
T = TypeVar("T")
No variadic arguments
from typing import TypeVar
types = (int, str)
# error: [invalid-legacy-type-variable]
T = TypeVar("T", *types)
reveal_type(T) # revealed: TypeVar
# error: [invalid-legacy-type-variable]
S = TypeVar("S", **{"bound": int})
reveal_type(S) # revealed: TypeVar
Type variables with a default
Note that the __default__ property is only available in Python ≥3.13.
[environment]
python-version = "3.13"
from typing import TypeVar
T = TypeVar("T", default=int)
reveal_type(type(T)) # revealed: <class 'TypeVar'>
reveal_type(T) # revealed: typing.TypeVar
reveal_type(T.__default__) # revealed: int
reveal_type(T.__bound__) # revealed: None
reveal_type(T.__constraints__) # revealed: tuple[()]
S = TypeVar("S")
reveal_type(S.__default__) # revealed: NoDefault
Using other typevars as a default
[environment]
python-version = "3.13"
from typing import Generic, TypeVar, Union
T = TypeVar("T")
U = TypeVar("U", default=T)
V = TypeVar("V", default=Union[T, U])
class Valid(Generic[T, U, V]): ...
reveal_type(Valid()) # revealed: Valid[Unknown, Unknown, Unknown]
reveal_type(Valid[int]()) # revealed: Valid[int, int, int]
reveal_type(Valid[int, str]()) # revealed: Valid[int, str, int | str]
reveal_type(Valid[int, str, None]()) # revealed: Valid[int, str, None]
# TODO: error, default value for U isn't available in the generic context
class Invalid(Generic[U]): ...
Type variables with an upper bound
from typing import TypeVar
T = TypeVar("T", bound=int)
reveal_type(type(T)) # revealed: <class 'TypeVar'>
reveal_type(T) # revealed: typing.TypeVar
reveal_type(T.__bound__) # revealed: int
reveal_type(T.__constraints__) # revealed: tuple[()]
S = TypeVar("S")
reveal_type(S.__bound__) # revealed: None
The upper bound must be a valid type expression:
from typing import TypedDict
# error: [invalid-type-form]
T = TypeVar("T", bound=TypedDict)
Type variables with constraints
from typing import TypeVar
T = TypeVar("T", int, str)
reveal_type(type(T)) # revealed: <class 'TypeVar'>
reveal_type(T) # revealed: typing.TypeVar
reveal_type(T.__constraints__) # revealed: tuple[int, str]
S = TypeVar("S")
reveal_type(S.__constraints__) # revealed: tuple[()]
Constraints are not simplified relative to each other, even if one is a subtype of the other:
T = TypeVar("T", int, bool)
reveal_type(T.__constraints__) # revealed: tuple[int, bool]
S = TypeVar("S", float, str)
reveal_type(S.__constraints__) # revealed: tuple[int | float, str]
Cannot have only one constraint
TypeVarsupports constraining parametric types to a fixed set of possible types...There should be at least two constraints, if any; specifying a single constraint is disallowed.
from typing import TypeVar
# error: [invalid-legacy-type-variable]
T = TypeVar("T", int)
Cannot have both bound and constraint
from typing import TypeVar
# error: [invalid-legacy-type-variable]
T = TypeVar("T", int, str, bound=bytes)
Cannot be both covariant and contravariant
To facilitate the declaration of container types where covariant or contravariant type checking is acceptable, type variables accept keyword arguments
covariant=Trueorcontravariant=True. At most one of these may be passed.
from typing import TypeVar
# error: [invalid-legacy-type-variable]
T = TypeVar("T", covariant=True, contravariant=True)
Boolean parameters must be unambiguous
from typing_extensions import TypeVar
def cond() -> bool:
return True
# error: [invalid-legacy-type-variable]
T = TypeVar("T", covariant=cond())
# error: [invalid-legacy-type-variable]
U = TypeVar("U", contravariant=cond())
# error: [invalid-legacy-type-variable]
V = TypeVar("V", infer_variance=cond())
Invalid keyword arguments
from typing import TypeVar
# error: [invalid-legacy-type-variable]
T = TypeVar("T", invalid_keyword=True)
from typing import TypeVar
# error: [invalid-legacy-type-variable]
T = TypeVar("T", invalid_keyword=True)
Constructor signature versioning
For typing.TypeVar
[environment]
python-version = "3.10"
In a stub file, features from the latest supported Python version can be used on any version.
There's no need to require use of typing_extensions.TypeVar in a stub file, when the type checker
can understand the typevar definition perfectly well either way, and there can be no runtime error.
(Perhaps it's arguable whether this special case is worth it, but other type checkers do it, so we
maintain compatibility.)
from typing import TypeVar
T = TypeVar("T", default=int)
But this raises an error in a non-stub file:
from typing import TypeVar
# error: [invalid-legacy-type-variable]
T = TypeVar("T", default=int)
For typing_extensions.TypeVar
typing_extensions.TypeVar always supports the latest features, on any Python version.
[environment]
python-version = "3.10"
from typing_extensions import TypeVar
T = TypeVar("T", default=int)
# TODO: should not error, should reveal `int`
# error: [unresolved-attribute]
reveal_type(T.__default__) # revealed: Unknown
Callability
A typevar bound to a Callable type is callable:
from typing import Callable, TypeVar
T = TypeVar("T", bound=Callable[[], int])
def bound(f: T):
reveal_type(f) # revealed: T@bound
reveal_type(f()) # revealed: int
Same with a constrained typevar, as long as all constraints are callable:
T = TypeVar("T", Callable[[], int], Callable[[], str])
def constrained(f: T):
reveal_type(f) # revealed: T@constrained
reveal_type(f()) # revealed: int | str
Meta-type
The meta-type of a typevar is the same as the meta-type of the upper bound, or the union of the meta-types of the constraints:
from typing import TypeVar
T_normal = TypeVar("T_normal")
def normal(x: T_normal):
reveal_type(type(x)) # revealed: type
T_bound_object = TypeVar("T_bound_object", bound=object)
def bound_object(x: T_bound_object):
reveal_type(type(x)) # revealed: type
T_bound_int = TypeVar("T_bound_int", bound=int)
def bound_int(x: T_bound_int):
reveal_type(type(x)) # revealed: type[int]
T_constrained = TypeVar("T_constrained", int, str)
def constrained(x: T_constrained):
reveal_type(type(x)) # revealed: type[int] | type[str]
Cycles
Bounds and constraints
A typevar's bounds and constraints cannot be generic, cyclic or otherwise:
from typing import Any, TypeVar
S = TypeVar("S")
# TODO: error
T = TypeVar("T", bound=list[S])
# TODO: error
U = TypeVar("U", list["T"], str)
# TODO: error
V = TypeVar("V", list["V"], str)
However, they are lazily evaluated and can cyclically refer to their own type:
from typing import TypeVar, Generic
T = TypeVar("T", bound=list["G"])
class G(Generic[T]):
x: T
reveal_type(G[list[G]]().x) # revealed: list[G[Unknown]]
Defaults
[environment]
python-version = "3.13"
Defaults can be generic, but can only refer to typevars from the same scope if they were defined earlier in that scope:
from typing import Generic, TypeVar
T = TypeVar("T")
U = TypeVar("U", default=T)
class C(Generic[T, U]):
x: T
y: U
reveal_type(C[int, str]().x) # revealed: int
reveal_type(C[int, str]().y) # revealed: str
reveal_type(C[int]().x) # revealed: int
reveal_type(C[int]().y) # revealed: int
# TODO: error
V = TypeVar("V", default="V")
class D(Generic[V]):
x: V
# TODO: we shouldn't leak a typevar like this in type inference
reveal_type(D().x) # revealed: V@D
Regression
Use of typevar with default inside a function body that binds it
[environment]
python-version = "3.13"
from typing import Generic, TypeVar
_DataT = TypeVar("_DataT", bound=int, default=int)
class Event(Generic[_DataT]):
def __init__(self, data: _DataT) -> None:
self.data = data
def async_fire_internal(event_data: _DataT):
event: Event[_DataT] | None = None
event = Event(event_data)