This PR updates the mdtests that test how our generics solver interacts
with our new constraint set implementation. Because the rendering of a
constraint set can get long, this standardizes on putting the `revealed`
assertion on a separate line. We also add a `static_assert` test for
each constraint set to verify that they are all coerced into simple
`bool`s correctly.
This is a pure reformatting (not even a refactoring!) that changes no
behavior. I've pulled it out of #20093 to reduce the amount of effort
that will be required to review that PR.
We have several functions in `ty_extensions` for testing our constraint
set implementation. This PR refactors those functions so that they are
all methods of the `ConstraintSet` class, rather than being standalone
top-level functions. 🎩 to @sharkdp for pointing out that
`KnownBoundMethod` gives us what we need to implement that!
This PR adds the new **_constraint implication_** relationship between
types, aka `is_subtype_of_given`, which tests whether one type is a
subtype of another _assuming that the constraints in a particular
constraint set hold_.
For concrete types, constraint implication is exactly the same as
subtyping. (A concrete type is any fully static type that is not a
typevar. It can _contain_ a typevar, though — `list[T]` is considered
concrete.)
The interesting case is typevars. The other typing relationships (TODO:
will) all "punt" on the question when considering a typevar, by
translating the desired relationship into a constraint set. At some
point, though, we need to resolve a constraint set; at that point, we
can no longer punt on the question. Unlike with concrete types, the
answer will depend on the constraint set that we are considering.
That PR title might be a bit inscrutable.
Consider the two constraints `T ≤ bool` and `T ≤ int`. Since `bool ≤
int`, by transitivity `T ≤ bool` implies `T ≤ int`. (Every type that is
a subtype of `bool` is necessarily also a subtype of `int`.) That means
that `T ≤ bool ∧ T ≰ int` is an impossible combination of constraints,
and is therefore not a valid input to any BDD. We say that that
assignment is not in the _domain_ of the BDD.
The implication `T ≤ bool → T ≤ int` can be rewritten as `T ≰ bool ∨ T ≤
int`. (That's the definition of implication.) If we construct that
constraint set in an mdtest, we should get a constraint set that is
always satisfiable. Previously, that constraint set would correctly
_display_ as `always`, but a `static_assert` on it would fail.
The underlying cause is that our `is_always_satisfied` method would only
test if the BDD was the `AlwaysTrue` terminal node. `T ≰ bool ∨ T ≤ int`
does not simplify that far, because we purposefully keep around those
constraints in the BDD structure so that it's easier to compare against
other BDDs that reference those constraints.
To fix this, we need a more nuanced definition of "always satisfied".
Instead of evaluating to `true` for _every_ input, we only need it to
evaluate to `true` for every _valid_ input — that is, every input in its
domain.
## Summary
Infer a type of `Self` for unannotated `self` parameters in methods of
classes.
part of https://github.com/astral-sh/ty/issues/159
closes https://github.com/astral-sh/ty/issues/1081
## Conformance tests changes
```diff
+enums_member_values.py:85:9: error[invalid-assignment] Object of type `int` is not assignable to attribute `_value_` of type `str`
```
A true positive ✔️
```diff
-generics_self_advanced.py:35:9: error[type-assertion-failure] Argument does not have asserted type `Self@method2`
-generics_self_basic.py:14:9: error[type-assertion-failure] Argument does not have asserted type `Self@set_scale
```
Two false positives going away ✔️
```diff
+generics_syntax_infer_variance.py:82:9: error[invalid-assignment] Cannot assign to final attribute `x` on type `Self@__init__`
```
This looks like a true positive to me, even if it's not marked with `#
E` ✔️
```diff
+protocols_explicit.py:56:9: error[invalid-assignment] Object of type `tuple[int, int, str]` is not assignable to attribute `rgb` of type `tuple[int, int, int]`
```
True positive ✔️
```
+protocols_explicit.py:85:9: error[invalid-attribute-access] Cannot assign to ClassVar `cm1` from an instance of type `Self@__init__`
```
This looks like a true positive to me, even if it's not marked with `#
E`. But this is consistent with our understanding of `ClassVar`, I
think. ✔️
```py
+qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__`
+qualifiers_final_annotation.py:65:9: error[invalid-assignment] Cannot assign to final attribute `ID7` on type `Self@method1`
```
New true positives ✔️
```py
+qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__`
+qualifiers_final_annotation.py:57:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__`
+qualifiers_final_annotation.py:59:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__`
```
This is a new false positive, but that's a pre-existing issue on main
(if you annotate with `Self`):
https://play.ty.dev/3ee1c56d-7e13-43bb-811a-7a81e236e6ab❌ => reported
as https://github.com/astral-sh/ty/issues/1409
## Ecosystem
* There are 5931 new `unresolved-attribute` and 3292 new
`possibly-missing-attribute` attribute errors, way too many to look at
all of them. I randomly sampled 15 of these errors and found:
* 13 instances where there was simply no such attribute that we could
plausibly see. Sometimes [I didn't find it
anywhere](8644d886c6/openlibrary/plugins/openlibrary/tests/test_listapi.py (L33)).
Sometimes it was set externally on the object. Sometimes there was some
[`setattr` dynamicness going
on](a49f6b927d/setuptools/wheel.py (L88-L94)).
I would consider all of them to be true positives.
* 1 instance where [attribute was set on `obj` in
`__new__`](9e87b44fd4/sympy/tensor/array/array_comprehension.py (L45C1-L45C36)),
which we don't support yet
* 1 instance [where the attribute was defined via `__slots__`
](e250ec0fc8/lib/spack/spack/vendor/pyrsistent/_pdeque.py (L48C5-L48C14))
* I see 44 instances [of the false positive
above](https://github.com/astral-sh/ty/issues/1409) with `Final`
instance attributes being set in `__init__`. I don't think this should
block this PR.
## Test Plan
New Markdown tests.
---------
Co-authored-by: Shaygan Hooshyari <sh.hooshyari@gmail.com>
This PR adds another useful simplification when rendering constraint
sets: `T = int` instead of `T = int ∧ T ≠ str`. (The "smaller"
constraint `T = int` implies the "larger" constraint `T ≠ str`.
Constraint set clauses are intersections, and if one constraint in a
clause implies another, we can throw away the "larger" constraint.)
While we're here, we also normalize the bounds of a constraint, so that
we equate e.g. `T ≤ int | str` with `T ≤ str | int`, and change the
ordering of BDD variables so that all constraints with the same typevar
are ordered adjacent to each other.
Lastly, we also add a new `display_graph` helper method that prints out
the full graph structure of a BDD.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Fall back to `C[Divergent]` if we are trying to specialize `C[T]` with a
type that itself already contains deeply nested specialized generic
classes. This is a way to prevent infinite recursion for cases like
`self.x = [self.x]` where type inference for the implicit instance
attribute would not converge.
closes https://github.com/astral-sh/ty/issues/1383
closes https://github.com/astral-sh/ty/issues/837
## Test Plan
Regression tests.
This is an alternative to #21012 that more narrowly handles this logic
in the stub-mapping machinery rather than pervasively allowing us to
identify cached files as typeshed stubs. Much of the logic is the same
(pulling the logic out of ty_server so it can be reused).
I don't have a good sense for if one approach is "better" or "worse" in
terms of like, semantics and Weird Bugs that this can cause. This one is
just "less spooky in its broad consequences" and "less muddying of
separation of concerns" and puts the extra logic on a much colder path.
I won't be surprised if one day the previous implementation needs to be
revisited for its more sweeping effects but for now this is good.
Fixes https://github.com/astral-sh/ty/issues/1054
## Summary
We currently panic in the seemingly rare case where the type of a
default value of a parameter depends on the callable itself:
```py
class C:
def f(self: C):
self.x = lambda a=self.x: a
```
Types of default values are only used for display reasons, and it's
unclear if we even want to track them (or if we should rather track the
actual value). So it didn't seem to me that we should spend a lot of
effort (and runtime) trying to achieve a theoretically correct type here
(which would be infinite).
Instead, we simply replace *nested* default types with `Unknown`, i.e.
only if the type of the default value is a callable itself.
closes https://github.com/astral-sh/ty/issues/1402
## Test Plan
Regression tests
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This PR implements a new semantic syntax error where name is parameter &
global.
## Test Plan
<!-- How was it tested? -->
I have written inline test as directed in #17412
---------
Signed-off-by: 11happy <soni5happy@gmail.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
## Summary
Only run the "pull types" test after performing the "actual" mdtest. We
observed that the order matters. There is currently one mdtest which
panics when checked in the CLI or the playground. With this change, it
also panics in the mdtest suite.
reopens https://github.com/astral-sh/ty/issues/837?
## Summary
- Type checkers (and type-checker authors) think in terms of types, but
I think most Python users think in terms of values. Rather than saying
that a _type_ `X` "has no attribute `foo`" (which I think sounds strange
to many users), say that "an object of type `X` has no attribute `foo`"
- Special-case certain types so that the diagnostic messages read more
like normal English: rather than saying "Type `<class 'Foo'>` has no
attribute `bar`" or "Object of type `<class 'Foo'>` has no attribute
`bar`", just say "Class `Foo` has no attribute `bar`"
## Test Plan
Mdtests and snapshots updated
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This PR implements semantic syntax error where alternative patterns bind
different names
## Test Plan
<!-- How was it tested? -->
I have written inline tests as directed in #17412
---------
Signed-off-by: 11happy <soni5happy@gmail.com>
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
## Summary
Derived from #20900
Implement `VarianceInferable` for `KnownInstanceType` (especially for
`KnownInstanceType::TypeAliasType`).
The variance of a type alias matches its value type. In normal usage,
type aliases are expanded to value types, so the variance of a type
alias can be obtained without implementing this. However, for example,
if we want to display the variance when hovering over a type alias, we
need to be able to obtain the variance of the type alias itself (cf.
#20900).
## Test Plan
I couldn't come up with a way to test this in mdtest, so I'm testing it
in a test submodule at the end of `types.rs`.
I also added a test to `mdtest/generics/pep695/variance.md`, but it
passes without the changes in this PR.
## Summary
Support `dataclass_transform` when used on a (base) class.
## Typing conformance
* The changes in `dataclasses_transform_class.py` look good, just a few
mistakes due to missing `alias` support.
* I didn't look closely at the changes in
`dataclasses_transform_converter.py` since we don't support `converter`
yet.
## Ecosystem impact
The impact looks huge, but it's concentrated on a single project (ibis).
Their setup looks more or less like this:
* the real `Annotatable`:
d7083c2c96/ibis/common/grounds.py (L100-L101)
* the real `DataType`:
d7083c2c96/ibis/expr/datatypes/core.py (L161-L179)
* the real `Array`:
d7083c2c96/ibis/expr/datatypes/core.py (L1003-L1006)
```py
from typing import dataclass_transform
@dataclass_transform()
class Annotatable:
pass
class DataType(Annotatable):
nullable: bool = True
class Array[T](DataType):
value_type: T
```
They expect something like `Array([1, 2])` to work, but ty, pyright,
mypy, and pyrefly would all expect there to be a first argument for the
`nullable` field on `DataType`. I don't really understand on what
grounds they expect the `nullable` field to be excluded from the
signature, but this seems to be the main reason for the new diagnostics
here. Not sure if related, but it looks like their typing setup is not
really complete
(https://github.com/ibis-project/ibis/issues/6844#issuecomment-1868274770,
this thread also mentions `dataclass_transform`).
## Test Plan
Update pre-existing tests.
Detect legacy namespace packages and treat them like namespace packages
when looking them up as the *parent* of the module we're interested in.
In all other cases treat them like a regular package.
(This PR is coauthored by @MichaReiser in a shared coding session)
Fixes https://github.com/astral-sh/ty/issues/838
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Prefer the declared type for collection literals, e.g.,
```py
x: list[Any] = [1, "2", (3,)]
reveal_type(x) # list[Any]
```
This solves a large part of https://github.com/astral-sh/ty/issues/136
for invariant generics, where respecting the declared type is a lot more
important. It also means that annotated dict literals with `dict[_,
Any]` is a way out of https://github.com/astral-sh/ty/issues/1248.
We have to track whether a typevar appears in a position where it's
inferable or not. In a non-inferable position (in the body of the
generic class or function that binds it), assignability must hold for
every possible specialization of the typevar. In an inferable position,
it only needs to hold for _some_ specialization.
https://github.com/astral-sh/ruff/pull/20093 is working on using
constraint sets to model assignability of typevars, and the constraint
sets that we produce will be the same for inferable vs non-inferable
typevars; what changes is what we _compare_ that constraint set to. (For
a non-inferable typevar, the constraint set must equal the set of valid
specializations; for an inferable typevar, it must not be `never`.)
When I first added support for tracking inferable vs non-inferable
typevars, it seemed like it would be easiest to have separate `Type`
variants for each. The alternative (which lines up with the Δ set in
[POPL15](https://doi.org/10.1145/2676726.2676991)) would be to
explicitly plumb through a list of inferable typevars through our type
property methods. That seemed cumbersome.
In retrospect, that was the wrong decision. We've had to jump through
hoops to translate types between the inferable and non-inferable
variants, which has been quite brittle. Combined with the original point
above, that much of the assignability logic will become more identical
between inferable and non-inferable, there is less justification for the
two `Type` variants. And plumbing an extra `inferable` parameter through
all of these methods turns out to not be as bad as I anticipated.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Use the declared type of variables as type context for the RHS of assignment expressions, e.g.,
```py
x: list[int | str]
x = [1]
reveal_type(x) # revealed: list[int | str]
```
