Commit graph

17 commits

Author SHA1 Message Date
David Peter
25cbf38a47
[ty] Patch Self for fallback-methods on NamedTuples and TypedDicts (#20328)
Some checks are pending
CI / cargo build (release) (push) Waiting to run
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / mkdocs (push) Waiting to run
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks-instrumented (push) Blocked by required conditions
CI / benchmarks-walltime (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
## Summary

We use classes like
[`_typeshed._type_checker_internals.NamedTupleFallback`](d9c76e1d9f/stdlib/_typeshed/_type_checker_internals.pyi (L54-L75))
to tack on additional attributes/methods to instances of user-defined
`NamedTuple`s (or `TypedDict`s), even though these classes are not
present in the MRO of those types.

The problem is that those classes use implicit and explicit `Self`
annotations which refer to `NamedTupleFallback` itself, instead of to
the actual type that we're adding those methods to:
```py
class NamedTupleFallback(tuple[Any, ...]):
    # […]
    def _replace(self, **kwargs: Any) -> typing_extensions.Self: ...
```

In effect, when we access `_replace` on an instance of a custom
`NamedTuple` instance, its `self` parameter and return type refer to the
wrong `Self`. This leads to incorrect *"Argument to bound method
`_replace` is incorrect: Argument type `Person` does not satisfy upper
bound `NamedTupleFallback` of type variable `Self`"* errors on #18007.
It would also lead to similar errors on `TypedDict`s, if they would
already implement assignability properly.


## Test Plan

I applied the following patch to typeshed and verified that no errors
appear anymore.

<details>

```diff
diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi
index feb22aae00..8e41034f19 100644
--- a/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi
+++ b/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi
@@ -29,27 +29,27 @@ class TypedDictFallback(Mapping[str, object], metaclass=ABCMeta):
         __readonly_keys__: ClassVar[frozenset[str]]
         __mutable_keys__: ClassVar[frozenset[str]]
 
-    def copy(self) -> typing_extensions.Self: ...
+    def copy(self: typing_extensions.Self) -> typing_extensions.Self: ...
     # Using Never so that only calls using mypy plugin hook that specialize the signature
     # can go through.
-    def setdefault(self, k: Never, default: object) -> object: ...
+    def setdefault(self: typing_extensions.Self, k: Never, default: object) -> object: ...
     # Mypy plugin hook for 'pop' expects that 'default' has a type variable type.
-    def pop(self, k: Never, default: _T = ...) -> object: ...  # pyright: ignore[reportInvalidTypeVarUse]
-    def update(self, m: typing_extensions.Self, /) -> None: ...
-    def __delitem__(self, k: Never) -> None: ...
-    def items(self) -> dict_items[str, object]: ...
-    def keys(self) -> dict_keys[str, object]: ...
-    def values(self) -> dict_values[str, object]: ...
+    def pop(self: typing_extensions.Self, k: Never, default: _T = ...) -> object: ...  # pyright: ignore[reportInvalidTypeVarUse]
+    def update(self: typing_extensions.Self, m: typing_extensions.Self, /) -> None: ...
+    def __delitem__(self: typing_extensions.Self, k: Never) -> None: ...
+    def items(self: typing_extensions.Self) -> dict_items[str, object]: ...
+    def keys(self: typing_extensions.Self) -> dict_keys[str, object]: ...
+    def values(self: typing_extensions.Self) -> dict_values[str, object]: ...
     @overload
-    def __or__(self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
+    def __or__(self: typing_extensions.Self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
     @overload
-    def __or__(self, value: dict[str, Any], /) -> dict[str, object]: ...
+    def __or__(self: typing_extensions.Self, value: dict[str, Any], /) -> dict[str, object]: ...
     @overload
-    def __ror__(self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
+    def __ror__(self: typing_extensions.Self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
     @overload
-    def __ror__(self, value: dict[str, Any], /) -> dict[str, object]: ...
+    def __ror__(self: typing_extensions.Self, value: dict[str, Any], /) -> dict[str, object]: ...
     # supposedly incompatible definitions of __or__ and __ior__
-    def __ior__(self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...  # type: ignore[misc]
+    def __ior__(self: typing_extensions.Self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...  # type: ignore[misc]
 
 # Fallback type providing methods and attributes that appear on all `NamedTuple` types.
 class NamedTupleFallback(tuple[Any, ...]):
@@ -61,18 +61,18 @@ class NamedTupleFallback(tuple[Any, ...]):
         __orig_bases__: ClassVar[tuple[Any, ...]]
 
     @overload
-    def __init__(self, typename: str, fields: Iterable[tuple[str, Any]], /) -> None: ...
+    def __init__(self: typing_extensions.Self, typename: str, fields: Iterable[tuple[str, Any]], /) -> None: ...
     @overload
     @typing_extensions.deprecated(
         "Creating a typing.NamedTuple using keyword arguments is deprecated and support will be removed in Python 3.15"
     )
-    def __init__(self, typename: str, fields: None = None, /, **kwargs: Any) -> None: ...
+    def __init__(self: typing_extensions.Self, typename: str, fields: None = None, /, **kwargs: Any) -> None: ...
     @classmethod
     def _make(cls, iterable: Iterable[Any]) -> typing_extensions.Self: ...
-    def _asdict(self) -> dict[str, Any]: ...
-    def _replace(self, **kwargs: Any) -> typing_extensions.Self: ...
+    def _asdict(self: typing_extensions.Self) -> dict[str, Any]: ...
+    def _replace(self: typing_extensions.Self, **kwargs: Any) -> typing_extensions.Self: ...
     if sys.version_info >= (3, 13):
-        def __replace__(self, **kwargs: Any) -> typing_extensions.Self: ...
+        def __replace__(self: typing_extensions.Self, **kwargs: Any) -> typing_extensions.Self: ...
 
 # Non-default variations to accommodate couroutines, and `AwaitableGenerator` having a 4th type parameter.
 _S = TypeVar("_S")
```

</details>
2025-09-15 16:21:53 +02:00
Alex Waygood
f77315776c
[ty] Better error message for attempting to assign to a read-only property (#20150) 2025-08-29 13:22:23 +00:00
Alex Waygood
4242905b36
[ty] Detect NamedTuple classes where fields without default values follow fields with default values (#19945) 2025-08-19 08:56:08 +00:00
Alex Waygood
fbf24be8ae
[ty] Detect illegal multiple inheritance with NamedTuple (#19943) 2025-08-18 12:03:01 +00:00
Douglas Creager
b892e4548e
[ty] Track when type variables are inferable or not (#19786)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks-instrumented (push) Blocked by required conditions
CI / benchmarks-walltime (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
`Type::TypeVar` now distinguishes whether the typevar in question is
inferable or not.

A typevar is _not inferable_ inside the body of the generic class or
function that binds it:

```py
def f[T](t: T) -> T:
    return t
```

The infered type of `t` in the function body is `TypeVar(T,
NotInferable)`. This represents how e.g. assignability checks need to be
valid for all possible specializations of the typevar. Most of the
existing assignability/etc logic only applies to non-inferable typevars.

Outside of the function body, the typevar is _inferable_:

```py
f(4)
```

Here, the parameter type of `f` is `TypeVar(T, Inferable)`. This
represents how e.g. assignability doesn't need to hold for _all_
specializations; instead, we need to find the constraints under which
this specific assignability check holds.

This is in support of starting to perform specialization inference _as
part of_ performing the assignability check at the call site.

In the [[POPL2015][]] paper, this concept is called _monomorphic_ /
_polymorphic_, but I thought _non-inferable_ / _inferable_ would be
clearer for us.

Depends on #19784 

[POPL2015]: https://doi.org/10.1145/2676726.2676991

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-08-16 18:25:03 -04:00
Alex Waygood
f4d8826428
[ty] Fix error message for invalidly providing type arguments to NamedTuple when it occurs in a type expression (#19940) 2025-08-16 17:45:15 +00:00
Alex Waygood
26d6c3831f
[ty] Represent NamedTuple as an opaque special form, not a class (#19915) 2025-08-15 18:20:14 +01:00
Alex Waygood
f6093452ed
[ty] Synthesize read-only properties for all declared members on NamedTuple classes (#19899) 2025-08-14 21:25:45 +00:00
Alex Waygood
9f6146a13d
[ty] Add precise inference for indexing, slicing and unpacking NamedTuple instances (#19560)
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
2025-08-13 15:19:44 +00:00
Douglas Creager
585ce12ace
[ty] typing.Self is bound by the method, not the class (#19784)
This fixes our logic for binding a legacy typevar with its binding
context. (To recap, a legacy typevar starts out "unbound" when it is
first created, and each time it's used in a generic class or function,
we "bind" it with the corresponding `Definition`.)

We treat `typing.Self` the same as a legacy typevar, and so we apply
this binding logic to it too. Before, we were using the enclosing class
as its binding context. But that's not correct — it's the method where
`typing.Self` is used that binds the typevar. (Each invocation of the
method will find a new specialization of `Self` based on the specific
instance type containing the invoked method.)

This required plumbing through some additional state to the
`in_type_expression` method.

This also revealed that we weren't handling `Self`-typed instance
attributes correctly (but were coincidentally not getting the expected
false positive diagnostics).
2025-08-06 17:26:17 -04:00
Douglas Creager
06cd249a9b
[ty] Track different uses of legacy typevars, including context when rendering typevars (#19604)
Some checks are pending
CI / mkdocs (push) Waiting to run
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / check playground (push) Blocked by required conditions
CI / benchmarks-instrumented (push) Blocked by required conditions
CI / benchmarks-walltime (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run
This PR introduces a few related changes:

- We now keep track of each time a legacy typevar is bound in a
different generic context (e.g. class, function), and internally create
a new `TypeVarInstance` for each usage. This means the rest of the code
can now assume that salsa-equivalent `TypeVarInstance`s refer to the
same typevar, even taking into account that legacy typevars can be used
more than once.

- We also go ahead and track the binding context of PEP 695 typevars.
That's _much_ easier to track since we have the binding context right
there during type inference.

- With that in place, we can now include the name of the binding context
when rendering typevars (e.g. `T@f` instead of `T`)
2025-08-01 12:20:32 -04:00
Matthew Mckee
e8ea40012a
[ty] Add generic inference for dataclasses (#18443)
## Summary

An issue seen here https://github.com/astral-sh/ty/issues/500

The `__init__` method of dataclasses had no inherited generic context,
so we could not infer the type of an instance from a constructor call
with generics

## Test Plan

Add tests to classes.md` in generics folder
2025-06-03 09:59:43 -07:00
David Peter
e67b35743a
[ty] NamedTuple 'fallback' attributes (#18127)
## Summary

Add various attributes to `NamedTuple` classes/instances that are
available at runtime.

closes https://github.com/astral-sh/ty/issues/417

## Test Plan

New Markdown tests
2025-05-16 12:56:43 +02:00
David Peter
97d7b46936
[ty] Do not look up __init__ on instances (#18092)
## Summary

Dunder methods are never looked up on instances. We do this implicitly
in `try_call_dunder`, but the corresponding flag was missing in the
instance-construction code where we use `member_lookup_with_policy`
directly.

fixes https://github.com/astral-sh/ty/issues/322

## Test Plan

Added regression test.
2025-05-14 15:33:42 +02:00
Abhijeet Prasad Bodas
235b74a310
[ty] Add more tests for NamedTuples (#17975)
## Summary

Add more tests and TODOs for `NamedTuple` support, based on the typing
spec: https://typing.python.org/en/latest/spec/namedtuples.html

## Test Plan

This PR adds new tests.
2025-05-10 10:46:08 +02:00
Alex Waygood
d1bb10a66b
[ty] Understand classes that inherit from subscripted Protocol[] as generic (#17832) 2025-05-09 17:39:15 +01:00
Micha Reiser
b51c4f82ea
Rename Red Knot (#17820) 2025-05-03 19:49:15 +02:00
Renamed from crates/red_knot_python_semantic/resources/mdtest/named_tuple.md (Browse further)