<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
This PR aims to close#16605.
## Summary
This PR introduces a new rule (`RUF061`) that detects non-contextmanager
usage of `pytest.raises`, `pytest.warns`, and `pytest.deprecated_call`.
This pattern is discouraged and [was proposed in
flake8-pytest-style](https://github.com/m-burst/flake8-pytest-style/pull/332),
but the corresponding PR has been open for over a month without
activity.
Additionally, this PR provides an unsafe fix for simple cases where the
non-contextmanager form can be transformed into the context manager
form. Examples of supported patterns are listed in `RUF061_raises.py`,
`RUF061_warns.py`, and `RUF061_deprecated_call.py` test files.
The more complex case from the original issue (involving two separate
statements):
```python
excinfo = pytest.raises(ValueError, int, "hello")
assert excinfo.match("^invalid literal")
```
is getting fixed like this:
```python
with pytest.raises(ValueError) as excinfo:
int("hello")
assert excinfo.match("^invalid literal")
```
Putting match in the raises call requires multi-statement
transformation, which I am not sure how to implement.
## Test Plan
<!-- How was it tested? -->
New test files were added to cover various usages of the
non-contextmanager form of pytest.raises, warns, and deprecated_call.
Closes#18671
Note that while this has, I believe, always been invalid syntax, it was
reported as a different syntax error until Python 3.12:
Python 3.11:
```pycon
>>> x = 1
>>> f"{x! s}"
File "<stdin>", line 1
f"{x! s}"
^
SyntaxError: f-string: invalid conversion character: expected 's', 'r', or 'a'
```
Python 3.12:
```pycon
>>> x = 1
>>> f"{x! s}"
File "<stdin>", line 1
f"{x! s}"
^^^
SyntaxError: f-string: conversion type must come right after the exclamanation mark
```
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Solves #18257
## Test Plan
<!-- How was it tested? -->
Snapshots updated with some cases (negative, positive, mixed
annotations).
Summary
--
As @AlexWaygood noted on the 0.12 release blog post draft, the existing
example is a bit confusing. Either `**/*.py` or just `*.py`, as I went
with here, makes more sense, although the old version (`scripts/**.py`)
also worked when I tested it. However, this probably shouldn't be relied
upon since the [globset](https://docs.rs/globset/latest/globset/#syntax)
docs say:
> Using ** anywhere else is illegal
where "anywhere else" comes after the listing of the three valid
positions:
1. At the start of a pattern (`**/`)
2. At the end of a pattern (`/**`)
3. Or directly between two slashes (`/**/`)
I think the current version is luckily treated the same as a single `*`,
and the default globbing settings allow it to match subdirectories such
that the new example pattern will apply to the whole `scripts` tree in a
project like this:
```
.
├── README.md
├── pyproject.toml
├── scripts
│ ├── matching.py
│ └── sub
│ └── nested.py
└── src
└── main.py
```
Test Plan
--
Local testing of the new pattern, but the specifics of the pattern
aren't as important as having a more intuitive-looking/correct example.
Specifically, this PR reverts "Make completions an opt-in LSP feature
(#17921)",
corresponding to commit 51e2effd2d.
In practice, this means you don't need to opt into completions working
by enabling experimental features. i.e., I was able to remove this from
my LSP configuration:
```
"experimental": {
"completions": {
"enable": true
}
},
```
There's still a lot of work left to do to make completions awesome, but
I think it's in a state where it would be useful to get real user
feedback. It's also meaningfully using ty to provide completions that
use type information.
Ref astral-sh/ty#86
## Summary
Fixes https://github.com/astral-sh/ruff/issues/18628 by avoiding a fix
if there are "unknown" arguments, including any keyword arguments and
more than the expected 2 positional arguments.
I'm a bit on the fence here because it also seems reasonable to avoid a
diagnostic at all. Especially in the final test case I added (`not
my_dict.get(default=False)`), the hint suggesting to remove
`default=False` seems pretty misleading. At the same time, I guess the
diagnostic at least calls attention to the call site, which could help
to fix the missing argument bug too.
As I commented on the issue, I double-checked that keyword arguments are
invalid as far back as Python 3.8, even though the positional-only
marker was only added to the
[docs](https://docs.python.org/3.11/library/stdtypes.html#dict.get) in
3.12 (link is to 3.11, showing its absence).
## Test Plan
New tests derived from the bug report
## Stabilization
This was planned to be stabilized in 0.12, and the bug is less severe
than some others, but if there's nobody opposed, I will plan **not to
stabilize** this one for now.
## Summary
Part of [#117](https://github.com/astral-sh/ty/issues/117).
`TypeIs[]` is a special form that allows users to define their own
narrowing functions. Despite the syntax, `TypeIs` is not a generic and,
on its own, it is meaningless as a type.
[Officially](https://typing.python.org/en/latest/spec/narrowing.html#typeis),
a function annotated as returning a `TypeIs[T]` is a <i>type narrowing
function</i>, where `T` is called the <i>`TypeIs` return type</i>.
A `TypeIs[T]` may or may not be bound to a symbol. Only bound types have
narrowing effect:
```python
def f(v: object = object()) -> TypeIs[int]: ...
a: str = returns_str()
if reveal_type(f()): # Unbound: TypeIs[int]
reveal_type(a) # str
if reveal_type(f(a)): # Bound: TypeIs[a, int]
reveal_type(a) # str & int
```
Delayed usages of a bound type has no effect, however:
```python
b = f(a)
if b:
reveal_type(a) # str
```
A `TypeIs[T]` type:
* Is fully static when `T` is fully static.
* Is a singleton/single-valued when it is bound.
* Has exactly two runtime inhabitants when it is unbound: `True` and
`False`.
In other words, an unbound type have ambiguous truthiness.
It is possible to infer more precise truthiness for bound types;
however, that is not part of this change.
`TypeIs[T]` is a subtype of or otherwise assignable to `bool`. `TypeIs`
is invariant with respect to the `TypeIs` return type: `TypeIs[int]` is
neither a subtype nor a supertype of `TypeIs[bool]`. When ty sees a
function marked as returning `TypeIs[T]`, its `return`s will be checked
against `bool` instead. ty will also report such functions if they don't
accept a positional argument. Addtionally, a type narrowing function
call with no positional arguments (e.g., `f()` in the example above)
will be considered invalid.
## Test Plan
Markdown tests.
---------
Co-authored-by: Carl Meyer <carl@astral.sh>
## Summary
Consider the following example, which leads to a excessively large
runtime on `main`. The reason for this is the following. When inferring
types for `self.a`, we look up the `a` attribute on `C`. While looking
for implicit instance attributes, we go through every method and check
for `self.a = …` assignments. There are no such assignments here, but we
always have an implicit `self.a = <unbound>` binding at the beginning
over every method. This binding accumulates a complex visibility
constraint in `C.f`, due to the `isinstance` checks. While evaluating
that constraint, we need to infer the type of `self.b`. There's no
binding for `self.b` either, but there's also an implicit `self.b =
<unbound>` binding with the same complex visibility constraint
(involving `self.b` recursively). This leads to a combinatorial
explosion:
```py
class C:
def f(self: "C"):
if isinstance(self.a, str):
return
if isinstance(self.b, str):
return
if isinstance(self.b, str):
return
if isinstance(self.b, str):
return
# repeat 20 times
```
(note that the `self` parameter here is annotated explicitly because we
currently still infer `Unknown` for `self` otherwise)
The fix proposed here is rather simple: when there are no `self.name =
…` attribute assignments in a given method, we skip evaluating the
visibility constraint of the implicit `self.name = <unbound>` binding.
This should also generally help with performance, because that's a very
common case.
This is *not* a fix for cases where there *are* actual bindings in the
method. When we add `self.a = 1; self.b = 1` to that example above, we
still see that combinatorial explosion of runtime. I still think it's
worth to make this optimization, as it fixes the problems with `pandas`
and `sqlalchemy` reported by users. I will open a ticket to track that
separately.
closes https://github.com/astral-sh/ty/issues/627
closes https://github.com/astral-sh/ty/issues/641
## Test Plan
* Made sure that `ty` finishes quickly on the MREs in
https://github.com/astral-sh/ty/issues/627
* Made sure that `ty` finishes quickly on `pandas`
* Made sure that `ty` finishes quickly on `sqlalchemy`
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Fixes false positive in B909 (`loop-iterator-mutation`) where mutations
inside return/break statements were incorrectly flagged as violations.
The fix adds tracking for when mutations occur within return/break
statements and excludes them from violation detection, as they don't
cause the iteration issues B909 is designed to prevent.
## Test Plan
- Added test cases covering the reported false positive scenarios to
`B909.py`
- Verified existing B909 tests continue to pass (no regressions)
- Ran `cargo test -p ruff_linter --lib flake8_bugbear` successfully
Fixes#18399
## Summary
Garbage collect ASTs once we are done checking a given file. Queries
with a cross-file dependency on the AST will reparse the file on demand.
This reduces ty's peak memory usage by ~20-30%.
The primary change of this PR is adding a `node_index` field to every
AST node, that is assigned by the parser. `ParsedModule` can use this to
create a flat index of AST nodes any time the file is parsed (or
reparsed). This allows `AstNodeRef` to simply index into the current
instance of the `ParsedModule`, instead of storing a pointer directly.
The indices are somewhat hackily (using an atomic integer) assigned by
the `parsed_module` query instead of by the parser directly. Assigning
the indices in source-order in the (recursive) parser turns out to be
difficult, and collecting the nodes during semantic indexing is
impossible as `SemanticIndex` does not hold onto a specific
`ParsedModuleRef`, which the pointers in the flat AST are tied to. This
means that we have to do an extra AST traversal to assign and collect
the nodes into a flat index, but the small performance impact (~3% on
cold runs) seems worth it for the memory savings.
Part of https://github.com/astral-sh/ty/issues/214.
## Summary
This PR closes https://github.com/astral-sh/ty/issues/238.
Since `DefinitionState::Deleted` was introduced in #18041, support for
the `del` statement (and deletion of except handler names) is
straightforward.
However, it is difficult to determine whether references to attributes
or subscripts are unresolved after they are deleted. This PR only
invalidates narrowing by assignment if the attribute or subscript is
deleted.
## Test Plan
`mdtest/del.md` is added.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Fixes https://github.com/astral-sh/ruff/issues/18612 by:
- Bailing out without a fix in the case of `*args`, which I don't think
we can fix reliably
- Using an `Edit::deletion` from `remove_argument` instead of an
`Edit::range_replacement` in the presence of unrecognized keyword
arguments
I thought we could always switch to the `Edit::deletion` approach
initially, but it caused problems when `maxlen` was passed positionally,
which we didn't have any existing tests for.
The replacement fix can easily delete comments, so I also marked the fix
unsafe in these cases and updated the docs accordingly.
## Test Plan
New test cases derived from the issue.
## Stabilization
These are pretty significant changes, much like those to PYI059 in
https://github.com/astral-sh/ruff/pull/18611 (and based a bit on the
implementation there!), so I think it probably makes sense to
un-stabilize this for the 0.12 release, but I'm open to other thoughts
there.
## Summary
This is to support https://github.com/astral-sh/ruff/pull/18607.
This PR adds support for generating the top materialization (or upper
bound materialization) and the bottom materialization (or lower bound
materialization) of a type. This is the most general and the most
specific form of the type which is fully static, respectively.
More concretely, `T'`, the top materialization of `T`, is the type `T`
with all occurrences
of dynamic type (`Any`, `Unknown`, `@Todo`) replaced as follows:
- In covariant position, it's replaced with `object`
- In contravariant position, it's replaced with `Never`
- In invariant position, it's replaced with an unresolved type variable
(For an invariant position, it should actually be replaced with an
existential type, but this is not currently representable in our type
system, so we use an unresolved type variable for now instead.)
The bottom materialization is implemented in the same way, except we
start out in "contravariant" position.
## Test Plan
Add test cases for various types.
## Summary
Minor documentation update to make `mypy_primer` instructions a bit more
verbose/helpful for running against a local branch
## Test Plan
N/A
This makes it work for a number of additional cases, like nested
attribute access and things like `[].<CURSOR>`.
The basic idea is that instead of selecting a covering node closest to a
leaf that contains the cursor, we walk up the tree as much as we can.
This lets us access the correct `ExprAttribute` node when performing
nested access.
This routine lets us climb up the AST tree when we find
a contiguous sequence of nodes that satisfy our predicate.
This will be useful for making things like `a.b.<CURSOR>`
work. That is, we don't want the `ExprAttribute` closest
to a leaf. We also don't always want the `ExprAttribute`
closest to the root. Rather, (I think) we want the
`ExprAttribute` closest to the root that has an unbroken
chain to the `ExprAttribute` closest to the leaf.
This commit doesn't change any functionality, but instead changes the
representation of `CoveringNode` to make the implementation simpler (as
well as planned future additions). By putting the found node last in the
list of ancestors (now just generically called `nodes`), we reduce the
amount of special case handling we need.
The downside is that the representation now allows invalid states (a
`CoveringNode` with no elements). But I think this is well mitigated by
encapsulation.
## Summary
Fixes https://github.com/astral-sh/ty/issues/557
## Test Plan
Stable property tests succeed with a million iterations. Added mdtests.
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Summary
--
Updates the rule docs to explicitly state how cases like
`Decimal("0.1")` are handled (not affected) because the discussion of
"float casts" referring to values like `nan` and `inf` is otherwise a
bit confusing.
These changes are based on suggestions from @AlexWaygood on Notion, with
a slight adjustment to use 0.1 instead of 0.5 since it causes a more
immediate issue in the REPL:
```pycon
>>> from decimal import Decimal
>>> Decimal(0.5) == Decimal("0.5")
True
>>> Decimal(0.1) == Decimal("0.1")
False
```
Test plan
--
N/a
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Summary
--
This PR updates the docs for PLW1641 to place less emphasis on the
example of inheriting a parent class's `__hash__` implementation by both
reducing the length of the example and warning that it may be unsound in
general, as @AlexWaygood pointed out on Notion.
Test plan
--
Existing tests
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Fixes https://github.com/astral-sh/ruff/issues/18602 by:
1. Avoiding a fix when `*args` are present
2. Inserting the `Generic` base class right before the first keyword
argument, if one is present
In an intermediate commit, I also had special handling to avoid a fix in
the `**kwargs` case, but this is treated (roughly) as a normal keyword,
and I believe handling it properly falls out of the other keyword fix.
I also updated the `add_argument` utility function to insert new
arguments right before the keyword argument list instead of at the very
end of the argument list. This changed a couple of snapshots unrelated
to `PYI059`, but there shouldn't be any functional changes to other
rules because all other calls to `add_argument` were adding a keyword
argument anyway.
## Test Plan
Existing PYI059 cases, plus new tests based on the issue
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
## Summary
Closes https://github.com/astral-sh/ty/issues/577. Make global
`__debug__` a `bool` constant.
## Test Plan
Mdtest `global-constants.md` was created to check if resolved type was
`bool`.
---------
Co-authored-by: David Peter <mail@david-peter.de>
Summary
--
Fixes#18590 by adding parentheses around lambdas and if expressions in
`for` loop iterators for FURB122 and FURB142. I also updated the docs on
the helper function to reflect the part actually being parenthesized and
the new checks.
The `lambda` case actually causes a `TypeError` at runtime, but I think
it's still worth handling to avoid causing a syntax error.
```pycon
>>> s = set()
... for x in (1,) if True else (2,):
... s.add(-x)
... for x in lambda: 0:
... s.discard(-x)
...
Traceback (most recent call last):
File "<python-input-0>", line 4, in <module>
for x in lambda: 0:
^^^^^^^^^
TypeError: 'function' object is not iterable
```
Test Plan
--
New test cases based on the bug report
---------
Co-authored-by: Dylan <dylwil3@gmail.com>
