<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
This PR implements F406
https://docs.astral.sh/ruff/rules/undefined-local-with-nested-import-star-usage/
as a semantic syntax error
## Test Plan
I have written inline tests as directed in #17412
---------
Signed-off-by: 11happy <soni5happy@gmail.com>
Previously, we used a very fine-grained representation for individual
constraints: each constraint was _either_ a range constraint, a
not-equivalent constraint, or an incomparable constraint. These three
pieces are enough to represent all of the "real" constraints we need to
create — range constraints and their negation.
However, it meant that we weren't picking up as many chances to simplify
constraint sets as we could. Our simplification logic depends on being
able to look at _pairs_ of constraints or clauses to see if they
simplify relative to each other. With our fine-grained representation,
we could easily encounter situations that we should have been able to
simplify, but that would require looking at three or more individual
constraints.
For instance, negating a range constraint would produce:
```
¬(Base ≤ T ≤ Super) = ((T ≤ Base) ∧ (T ≠ Base)) ∨ (T ≁ Base) ∨
((Super ≤ T) ∧ (T ≠ Super)) ∨ (T ≁ Super)
```
That is, `T` must be (strictly) less than `Base`, (strictly) greater
than `Super`, or incomparable to either.
If we tried to union those back together, we should get `always`, since
`x ∨ ¬x` should always be true, no matter what `x` is. But instead we
would get:
```
(Base ≤ T ≤ Super) ∨ ((T ≤ Base) ∧ (T ≠ Base)) ∨ (T ≁ Base) ∨ ((Super ≤ T) ∧ (T ≠
Super)) ∨ (T ≁ Super)
```
Nothing would simplify relative to each other, because we'd have to look
at all five union elements to see that together they do in fact combine
to `always`.
The fine-grained representation was nice, because it made it easier to
[work out the math](https://dcreager.net/theory/constraints/) for
intersections and unions of each kind of constraint. But being able to
simplify is more important, since the example above comes up immediately
in #20093 when trying to handle constrained typevars.
The fix in this PR is to go back to a more coarse-grained
representation, where each individual constraint consists of a positive
range (which might be `always` / `Never ≤ T ≤ object`), and zero or more
negative ranges. The intuition is to think of a constraint as a region
of the type space (representable as a range) with zero or more "holes"
removed from it.
With this representation, negating a range constraint produces:
```
¬(Base ≤ T ≤ Super) = (always ∧ ¬(Base ≤ T ≤ Super))
```
(That looks trivial, because it is! We just move the positive range to
the negative side.)
The math is not that much harder than before, because there are only
three combinations to consider (each for intersection and union) —
though the fact that there can be multiple holes in a constraint does
require some nested loops. But the mdtest suite gives me confidence that
this is not introducing any new issues, and it definitely removes a
troublesome TODO.
(As an aside, this change also means that we are back to having each
clause contain no more than one individual constraint for any typevar.
This turned out to be important, because part of our simplification
logic was also depending on that!)
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
This mainly removes an internal inconsistency, where we didn't remove
the `Self` type variable when eagerly binding `Self` to an instance
type. It has no observable effect, apparently.
builds on top of https://github.com/astral-sh/ruff/pull/20328
## Test Plan
None
## Summary
Fixes https://github.com/astral-sh/ty/issues/1161
Include `NamedTupleFallback` members in `NamedTuple` instance
completions.
- Augment instance attribute completions when completing on NamedTuple
instances by merging members from
`_typeshed._type_checker_internals.NamedTupleFallback`
## Test Plan
Adds a minimal completion test `namedtuple_fallback_instance_methods`
---------
Co-authored-by: David Peter <mail@david-peter.de>
## Summary
This project was [recently removed from
mypy_primer](https://github.com/astral-sh/ruff/pull/20378), so we need
to remove it from `good.txt` in order for ecosystem-analyzer to work
correctly.
## Test Plan
Run mypy_primer and ecosystem-analyzer on this branch.
## Summary
https://github.com/astral-sh/ruff/pull/20165 added a lot of false
positives around calls to `builtins.open()`, because our missing support
for PEP-613 type aliases means that we don't understand typeshed's
overloads for `builtins.open()` at all yet, and therefore always select
the first overload. This didn't use to matter very much, but now that we
have a much stricter implementation of protocol assignability/subtyping
it matters a lot, because most of the stdlib functions dealing with I/O
(`pickle`, `marshal`, `io`, `json`, etc.) are annotated in typeshed as
taking in protocols of some kind.
In lieu of full PEP-613 support, which is blocked on various things and
might not land in time for our next alpha release, this PR adds some
temporary special-casing for `builtins.open()` to avoid the false
positives. We just infer `Todo` for anything that isn't meant to match
typeshed's first `open()` overload. This should be easy to rip out again
once we have proper support for PEP-613 type aliases, which hopefully
should be pretty soon!
## Test Plan
Added an mdtest
## Summary
Fixes https://github.com/astral-sh/ty/issues/377.
We were treating any function as being assignable to any callback
protocol, because we were trying to figure out a type's `Callable`
supertype by looking up the `__call__` attribute on the type's
meta-type. But a function-literal's meta-type is `types.FunctionType`,
and `types.FunctionType.__call__` is `(...) -> Any`, which is not very
helpful!
While working on this PR, I also realised that assignability between
class-literals and callback protocols was somewhat broken too, so I
fixed that at the same time.
## Test Plan
Added mdtests
## Summary
This looks like it should fix the errors that we've been seeing in sympy
in recent mypy-primer runs.
## Test Plan
I wasn't able to reproduce the sympy failures locally; it looks like
there is probably a dependency on the order in which files are checked.
So I don't have a minimal reproducible example, and wasn't able to add a
test :/ Obviously I would be happier if we could commit a regression
test here, but since the change is straightforward and clearly
desirable, I'm not sure how many hours it's worth trying to track it
down.
Mypy-primer is still failing in CI on this PR, because it fails on the
"old" ty commit already (i.e. on main). But it passes [on a no-op PR
stacked on top of this](https://github.com/astral-sh/ruff/pull/20370),
which strongly suggests this PR fixes the problem.
## Summary
This PR addresses an issue for a variadic argument when involved in
argument type expansion of overload call evaluation.
The issue is that the expansion of the variadic argument could result in
argument list of different arity. For example, in `*args: tuple[int] |
tuple[int, str]`, the expansion would lead to the variadic argument
being unpacked into 1 and 2 element respectively. This means that the
parameter matching that was performed initially isn't sufficient and
each expanded argument list would need to redo the parameter matching
again.
This is currently done by redoing the parameter matching directly,
maintaining the state of argument forms (and the conflicting forms), and
updating the `Bindings` values if it changes.
Closes: astral-sh/ty#735
## Test Plan
Update existing mdtest.
This PR removes the `Constraints` trait. We removed the `bool`
implementation several weeks back, and are using `ConstraintSet`
everywhere. There have been discussions about trying to include the
reason for an assignability failure as part of the result, but that
there are no concrete plans to do so soon, and it's not clear that we'll
need the `Constraints` trait to do that. (We can ideally just update the
`ConstraintSet` type directly.)
In the meantime, this just complicates the code for no good reason.
This PR is a pure refactoring, and contains no behavioral changes.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Previously, `Type::object` would find the definition of the `object`
class in typeshed, load that in (to produce a `ClassLiteral` and
`ClassType`), and then create a `NominalInstance` of that class.
It's possible that we are using a typeshed that doesn't define `object`.
We will not be able to do much useful work with that kind of typeshed,
but it's still a possibility that we have to support at least without
panicking. Previously, we would handle this situation by falling back on
`Unknown`.
In most cases, that's a perfectly fine fallback! But `object` is also
our top type — the type of all values. `Unknown` is _not_ an acceptable
stand-in for the top type.
This PR adds a new `NominalInstance` variant for "instances of
`object`". Unlike other nominal instances, we do not need to load in
`object`'s `ClassType` to instantiate this variant. We will use this new
variant even when the current typeshed does not define an `object`
class, ensuring that we have a fully static representation of our top
type at all times.
There are several operations that need access to a nominal instance's
class, and for this new `object` variant we load it lazily only when
it's needed. That means this operation is now fallible, since this is
where the "typeshed doesn't define `object`" failure shows up.
This new approach also has the benefit of avoiding some salsa cycles
that were cropping up while I was debugging #20093, since the new
constraint set representation was trying to instantiate `Type::object`
while in the middle of processing its definition in typeshed. Cycle
handling was kicking in correctly and returning the `Unknown` fallback
mentioned above. But the constraint set implementation depends on
`Type::object` being a distinct and fully static type, highlighting that
this is a correctness fix, not just an optimization fix.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Use `Type::Divergent` to avoid "too many iterations" panic on an
infinitely-nested tuple in an implicit instance attribute.
The regression here is from checking all tuple elements to see if they
contain a Divergent type. It's 5% on one project, 1% on another, and
zero on the rest. I spent some time looking into eliminating this
regression by tracking a flag on inference results to note if they could
possibly contain any Divergent type, but this doesn't really work --
there are too many different ways a type containing a Divergent type
could enter an inference result. Still thinking about whether there are
other ways to reduce this. One option is if we see certain kinds of
non-atomic types that are commonly expensive to check for Divergent, we
could make `has_divergent_type` a Salsa query on those types.
## Test Plan
Added mdtest.
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
The debug representation isn't as useful as calling `.display(db)`, but
it's still kind-of annoying when `dbg!()` calls don't compile locally
due to the compiler not being able to guarantee that an object of type
`impl Constraints` implements `Debug`
## Summary
`CallableTypeOf[bound_method]` would previously bind `self` to the
bound method type itself, instead of binding it to the instance type
stored inside the bound method type.
## Test Plan
Added regression test
This PR adds a new `ty_extensions.ConstraintSet` class, which is used to
expose constraint sets to our mdtest framework. This lets us write a
large collection of unit tests that exercise the invariants and rewrite
rules of our constraint set implementation.
As part of this, `is_assignable_to` and friends are updated to return a
`ConstraintSet` instead of a `bool`, and we implement
`ConstraintSet.__bool__` to return when a constraint set is always
satisfied. That lets us still use
`static_assert(is_assignable_to(...))`, since the assertion will coerce
the constraint set to a bool, and also lets us
`reveal_type(is_assignable_to(...))` to see more detail about
whether/when the two types are assignable. That lets us get rid of
`reveal_when_assignable_to` and friends, since they are now redundant
with the expanded capabilities of `is_assignable_to`.
## Summary
When adding an enum literal `E = Literal[Color.RED]` to a union which
already contained a subtype of that enum literal(!), we were previously
not simplifying the union correctly. My assumption is that our property
tests didn't catch that earlier, because the only possible non-trivial
subytpe of an enum literal that I can think of is `Any & E`. And in
order for that to be detected by the property tests, it would have to
randomly generate `Any & E | E` and then also compare that with `E` on
the other side (in an equivalence test, or the subtyping-antisymmetry
test).
closes https://github.com/astral-sh/ty/issues/1155
## Test Plan
* Added a regression test.
* I also ran the property tests for a while, but probably not for two
months worth of daily CI runs.
The constraint representation that we added in #19997 was subtly wrong,
in that it didn't correctly model that type assignability is a _partial_
order — it's possible for two types to be incomparable, with neither a
subtype of the other. That means the negation of a constraint like `T ≤
t` (typevar `T` must be a subtype of `t`) is **_not_** `t < T`, but
rather `t < T ∨ T ≁ t` (using ≁ to mean "not comparable to").
That means we need to update our constraint representation to be an
enum, so that we can track both _range_ constraints (upper/lower bound
on the typevar), and these new _incomparable_ constraints.
Since we need an enum now, that also lets us simplify how we were
modeling range constraints. Before, we let the lower/upper bounds be
either open (<) or closed (≤). Now, range constraints are always closed,
and we add a third kind of constraint for _not equivalent_ (≠). We can
translate an open upper bound `T < t` into `T ≤ t ∧ T ≠ t`.
We already had the logic for doing adding _clauses_ to a _set_ by doing
a pairwise simplification. We copy that over to where we add
_constraints_ to a _clause_. To calculate the intersection or union of
two constraints, the new enum representation makes it easy to break down
all of the possibilities into a small number of cases: intersect range
with range, intersect range with not-equivalent, etc. I've done the math
[here](https://dcreager.net/theory/constraints/) to show that the
simplifications for each of these cases is correct.
## Summary
This is a follow-up to https://github.com/astral-sh/ruff/pull/19321.
Now lazy snapshots are updated to take into account new bindings on
every symbol reassignment.
```python
def outer(x: A | None):
if x is None:
x = A()
reveal_type(x) # revealed: A
def inner() -> None:
# lazy snapshot: {x: A}
reveal_type(x) # revealed: A
inner()
def outer() -> None:
x = None
x = 1
def inner() -> None:
# lazy snapshot: {x: Literal[1]} -> {x: Literal[1, 2]}
reveal_type(x) # revealed: Literal[1, 2]
inner()
x = 2
```
Closesastral-sh/ty#559.
## Test Plan
Some TODOs in `public_types.md` now work properly.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Adds support for generic PEP695 type aliases, e.g.,
```python
type A[T] = T
reveal_type(A[int]) # A[int]
```
Resolves https://github.com/astral-sh/ty/issues/677.
## Summary
Support cases like the following, where we need the generic context to
include both `Self` and `T` (not just `T`):
```py
from typing import Self
class C:
def method[T](self: Self, arg: T): ...
C().method(1)
```
closes https://github.com/astral-sh/ty/issues/1131
## Test Plan
Added regression test
## Summary
The sub-checks for assignability and subtyping of materializations
performed in `has_relation_in_invariant_position` and
`is_subtype_in_invariant_position` need to propagate the
`HasRelationToVisitor`, or we can stack overflow.
A side effect of this change is that we also propagate the
`ConstraintSet` through, rather than using `C::from_bool`, which I think
may also become important for correctness in cases involving type
variables (though it isn't testable yet, since we aren't yet actually
creating constraints other than always-true and always-false.)
## Test Plan
Added mdtest (derived from code found in pydantic) which
stack-overflowed before this PR.
With this change incorporated, pydantic now checks successfully on my
draft PR for PEP 613 TypeAlias support.
Now that https://github.com/astral-sh/ruff/pull/20263 is merged, we can
update mypy_primer and add the new `egglog-python` project to
`good.txt`. The ecosystem-analyzer run shows that we now add 1,356
diagnostics (where we had over 5,000 previously, due to the unsupported
project layout).
## Summary
Add backreferences to the original item declaration in TypedDict
diagnostics.
Thanks to @AlexWaygood for the suggestion.
## Test Plan
Updated snapshots
## Summary
An annotated assignment `name: annotation` without a right-hand side was
previously not covered by the range returned from
`DefinitionKind::full_range`, because we did expand the range to include
the right-hand side (if there was one), but failed to include the
annotation.
## Test Plan
Updated snapshot tests
## Summary
Add support for `typing.ReadOnly` as a type qualifier to mark
`TypedDict` fields as being read-only. If you try to mutate them, you
get a new diagnostic:
<img width="787" height="234" alt="image"
src="https://github.com/user-attachments/assets/f62fddf9-4961-4bcd-ad1c-747043ebe5ff"
/>
## Test Plan
* New Markdown tests
* The typing conformance changes are all correct. There are some false
negatives, but those are related to the missing support for the
functional form of `TypedDict`, or to overriding of fields via
inheritance. Both of these topics are tracked in
https://github.com/astral-sh/ty/issues/154
