[ty] Patch Self for fallback-methods on NamedTuples and TypedDicts (#20328)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks-instrumented (push) Blocked by required conditions
CI / benchmarks-walltime (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run

## Summary

We use classes like
[`_typeshed._type_checker_internals.NamedTupleFallback`](d9c76e1d9f/stdlib/_typeshed/_type_checker_internals.pyi (L54-L75))
to tack on additional attributes/methods to instances of user-defined
`NamedTuple`s (or `TypedDict`s), even though these classes are not
present in the MRO of those types.

The problem is that those classes use implicit and explicit `Self`
annotations which refer to `NamedTupleFallback` itself, instead of to
the actual type that we're adding those methods to:
```py
class NamedTupleFallback(tuple[Any, ...]):
    # […]
    def _replace(self, **kwargs: Any) -> typing_extensions.Self: ...
```

In effect, when we access `_replace` on an instance of a custom
`NamedTuple` instance, its `self` parameter and return type refer to the
wrong `Self`. This leads to incorrect *"Argument to bound method
`_replace` is incorrect: Argument type `Person` does not satisfy upper
bound `NamedTupleFallback` of type variable `Self`"* errors on #18007.
It would also lead to similar errors on `TypedDict`s, if they would
already implement assignability properly.


## Test Plan

I applied the following patch to typeshed and verified that no errors
appear anymore.

<details>

```diff
diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi
index feb22aae00..8e41034f19 100644
--- a/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi
+++ b/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi
@@ -29,27 +29,27 @@ class TypedDictFallback(Mapping[str, object], metaclass=ABCMeta):
         __readonly_keys__: ClassVar[frozenset[str]]
         __mutable_keys__: ClassVar[frozenset[str]]
 
-    def copy(self) -> typing_extensions.Self: ...
+    def copy(self: typing_extensions.Self) -> typing_extensions.Self: ...
     # Using Never so that only calls using mypy plugin hook that specialize the signature
     # can go through.
-    def setdefault(self, k: Never, default: object) -> object: ...
+    def setdefault(self: typing_extensions.Self, k: Never, default: object) -> object: ...
     # Mypy plugin hook for 'pop' expects that 'default' has a type variable type.
-    def pop(self, k: Never, default: _T = ...) -> object: ...  # pyright: ignore[reportInvalidTypeVarUse]
-    def update(self, m: typing_extensions.Self, /) -> None: ...
-    def __delitem__(self, k: Never) -> None: ...
-    def items(self) -> dict_items[str, object]: ...
-    def keys(self) -> dict_keys[str, object]: ...
-    def values(self) -> dict_values[str, object]: ...
+    def pop(self: typing_extensions.Self, k: Never, default: _T = ...) -> object: ...  # pyright: ignore[reportInvalidTypeVarUse]
+    def update(self: typing_extensions.Self, m: typing_extensions.Self, /) -> None: ...
+    def __delitem__(self: typing_extensions.Self, k: Never) -> None: ...
+    def items(self: typing_extensions.Self) -> dict_items[str, object]: ...
+    def keys(self: typing_extensions.Self) -> dict_keys[str, object]: ...
+    def values(self: typing_extensions.Self) -> dict_values[str, object]: ...
     @overload
-    def __or__(self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
+    def __or__(self: typing_extensions.Self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
     @overload
-    def __or__(self, value: dict[str, Any], /) -> dict[str, object]: ...
+    def __or__(self: typing_extensions.Self, value: dict[str, Any], /) -> dict[str, object]: ...
     @overload
-    def __ror__(self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
+    def __ror__(self: typing_extensions.Self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
     @overload
-    def __ror__(self, value: dict[str, Any], /) -> dict[str, object]: ...
+    def __ror__(self: typing_extensions.Self, value: dict[str, Any], /) -> dict[str, object]: ...
     # supposedly incompatible definitions of __or__ and __ior__
-    def __ior__(self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...  # type: ignore[misc]
+    def __ior__(self: typing_extensions.Self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...  # type: ignore[misc]
 
 # Fallback type providing methods and attributes that appear on all `NamedTuple` types.
 class NamedTupleFallback(tuple[Any, ...]):
@@ -61,18 +61,18 @@ class NamedTupleFallback(tuple[Any, ...]):
         __orig_bases__: ClassVar[tuple[Any, ...]]
 
     @overload
-    def __init__(self, typename: str, fields: Iterable[tuple[str, Any]], /) -> None: ...
+    def __init__(self: typing_extensions.Self, typename: str, fields: Iterable[tuple[str, Any]], /) -> None: ...
     @overload
     @typing_extensions.deprecated(
         "Creating a typing.NamedTuple using keyword arguments is deprecated and support will be removed in Python 3.15"
     )
-    def __init__(self, typename: str, fields: None = None, /, **kwargs: Any) -> None: ...
+    def __init__(self: typing_extensions.Self, typename: str, fields: None = None, /, **kwargs: Any) -> None: ...
     @classmethod
     def _make(cls, iterable: Iterable[Any]) -> typing_extensions.Self: ...
-    def _asdict(self) -> dict[str, Any]: ...
-    def _replace(self, **kwargs: Any) -> typing_extensions.Self: ...
+    def _asdict(self: typing_extensions.Self) -> dict[str, Any]: ...
+    def _replace(self: typing_extensions.Self, **kwargs: Any) -> typing_extensions.Self: ...
     if sys.version_info >= (3, 13):
-        def __replace__(self, **kwargs: Any) -> typing_extensions.Self: ...
+        def __replace__(self: typing_extensions.Self, **kwargs: Any) -> typing_extensions.Self: ...
 
 # Non-default variations to accommodate couroutines, and `AwaitableGenerator` having a 4th type parameter.
 _S = TypeVar("_S")
```

</details>
This commit is contained in:
David Peter 2025-09-15 16:21:53 +02:00 committed by GitHub
parent 9a9ebc316c
commit 25cbf38a47
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 188 additions and 7 deletions

View file

@ -272,16 +272,34 @@ reveal_type(Person._make) # revealed: bound method <class 'Person'>._make(itera
reveal_type(Person._asdict) # revealed: def _asdict(self) -> dict[str, Any]
reveal_type(Person._replace) # revealed: def _replace(self, **kwargs: Any) -> Self@_replace
# TODO: should be `Person` once we support `Self`
# TODO: should be `Person` once we support implicit type of `self`
reveal_type(Person._make(("Alice", 42))) # revealed: Unknown
person = Person("Alice", 42)
reveal_type(person._asdict()) # revealed: dict[str, Any]
# TODO: should be `Person` once we support `Self`
# TODO: should be `Person` once we support implicit type of `self`
reveal_type(person._replace(name="Bob")) # revealed: Unknown
```
When accessing them on child classes of generic `NamedTuple`s, the return type is specialized
accordingly:
```py
from typing import NamedTuple, Generic, TypeVar
T = TypeVar("T")
class Box(NamedTuple, Generic[T]):
content: T
class IntBox(Box[int]):
pass
# TODO: should be `IntBox` once we support the implicit type of `self`
reveal_type(IntBox(1)._replace(content=42)) # revealed: Unknown
```
## `collections.namedtuple`
```py

View file

@ -627,6 +627,18 @@ alice: Employee = {"name": "Alice", "employee_id": 1}
# error: [missing-typed-dict-key] "Missing required key 'employee_id' in TypedDict `Employee` constructor"
eve: Employee = {"name": "Eve"}
def combine(p: Person, e: Employee):
# TODO: Should be `Person` once we support the implicit type of self
reveal_type(p.copy()) # revealed: Unknown
# TODO: Should be `Employee` once we support the implicit type of self
reveal_type(e.copy()) # revealed: Unknown
reveal_type(p | p) # revealed: Person
reveal_type(e | e) # revealed: Employee
# TODO: Should be `Person` once we support the implicit type of self and subtyping for TypedDicts
reveal_type(p | e) # revealed: Employee
```
When inheriting from a `TypedDict` with a different `total` setting, inherited fields maintain their