Commit graph

250 commits

Author SHA1 Message Date
Charlie Marsh
70bdde4085
Handle unions in augmented assignments (#14045)
## Summary

Removing more TODOs from the augmented assignment test suite. Now, if
the _target_ is a union, we correctly infer the union of results:

```python
if flag:
    f = Foo()
else:
    f = 42.0
f += 12
```
2024-11-01 19:49:18 +00:00
TomerBin
34a5d7cb7f
[red-knot] Infer type of if-expression if test has statically known truthiness (#14048)
## Summary

Detecting statically known truthy or falsy test in if expressions
(ternary).

## Test Plan

new mdtest
2024-11-01 12:23:18 -07:00
Charlie Marsh
487941ea66
Handle maybe-unbound __iadd__-like operators in augmented assignments (#14044)
## Summary

One of the follow-ups from augmented assignment inference, now that
`Type::Unbound` has been removed.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-11-01 13:15:35 -04:00
github-actions[bot]
7c2da4f06e
Sync vendored typeshed stubs (#14030)
Co-authored-by: typeshedbot <>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-11-01 10:51:56 +00:00
Carl Meyer
b8acadd6a2
[red-knot] have mdformat wrap mdtest files to 100 columns (#14020)
This makes it easier to read and edit (and review changes to) these
files as source, even though it doesn't affect the rendering.
2024-10-31 21:00:51 +00:00
David Peter
53fa32a389
[red-knot] Remove Type::Unbound (#13980)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

- Remove `Type::Unbound`
- Handle (potential) unboundness as a concept orthogonal to the type
system (see new `Symbol` type)
- Improve existing and add new diagnostics related to (potential)
unboundness

closes #13671 

## Test Plan

- Update existing markdown-based tests
- Add new tests for added/modified functionality
2024-10-31 20:05:53 +01:00
Alex Waygood
d1189c20df
[red-knot] Add failing tests for iterating over maybe-iterable unions (#14016) 2024-10-31 18:20:21 +00:00
Micha Reiser
76e4277696
[red-knot] Handle context managers in (sync) with statements (#13998) 2024-10-31 08:18:18 +00:00
Dhruv Manilawala
2629527559
Fix panic when filling up types vector during unpacking (#14006)
## Summary

This PR fixes a panic which can occur in an unpack assignment when:
* (number of target expressions) - (number of tuple types) > 2
* There's a starred expression

The reason being that the `insert` panics because the index is greater
than the length.

This is an error case and so practically it should occur very rarely.
The solution is to resize the types vector to match the number of
expressions and then insert the starred expression type.

## Test Plan

Add a new test case.
2024-10-30 19:13:57 +00:00
Dhruv Manilawala
bf20061268
Separate type check diagnostics builder (#13978)
## Summary

This PR creates a new `TypeCheckDiagnosticsBuilder` for the
`TypeCheckDiagnostics` struct. The main motivation behind this is to
separate the helpers required to build the diagnostics from the type
inference builder itself. This allows us to use such helpers outside of
the inference builder like for example in the unpacking logic in
https://github.com/astral-sh/ruff/pull/13979.

## Test Plan

`cargo insta test`
2024-10-30 18:50:31 +00:00
Charlie Marsh
eddc8d7644
Add failing tests for augmented assignments with partial binding (#14002)
## Summary

These cases aren't handled correctly yet -- some of them are waiting on
refactors to `Unbound` before fixing. Part of #12699.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-30 14:22:34 -04:00
Charlie Marsh
b1ce8a3949
Use Never instead of None for stores (#13984)
## Summary

See:
https://github.com/astral-sh/ruff/pull/13981#issuecomment-2445472433
2024-10-30 12:03:50 -04:00
Charlie Marsh
262c04f297
Use binary semantics when __iadd__ et al are unbound (#13987)
## Summary

I noticed that augmented assignments on floats were yielding "not
supported" diagnostics. If the dunder isn't bound at all, we should use
binary operator semantics, rather than treating it as not-callable.
2024-10-30 13:09:22 +00:00
Charlie Marsh
71536a43db
Add remaining augmented assignment dunders (#13985)
## Summary

See: https://github.com/astral-sh/ruff/issues/12699
2024-10-30 13:02:29 +00:00
Alex Waygood
e6dcdf3e49
Switch off the single_match_else Clippy lint (#13994) 2024-10-30 12:24:16 +00:00
Alex Waygood
42c70697d8
[red-knot] Fix bug where union of two iterable types was not recognised as iterable (#13992) 2024-10-30 11:54:16 +00:00
Charlie Marsh
1607d88c22
Use consistent diagnostic messages in augmented assignment inference (#13986) 2024-10-29 22:57:53 -04:00
Charlie Marsh
c6b82151dd
Add augmented assignment inference for -= operator (#13981)
## Summary

See: https://github.com/astral-sh/ruff/issues/12699
2024-10-29 22:14:27 -04:00
Alex Waygood
39cf46ecd6
[red-knot] Improve ergonomics for the PySlice trait (#13983) 2024-10-29 20:40:59 +00:00
David Peter
96b3c400fe
[red-knot] Minor follow-up on slice expression inference (#13982)
## Summary

Minor follow-up to #13917 — thanks @AlexWaygood for the post-merge
review.

- Add
SliceLiteralType::as_tuple
- Use .expect() instead of SAFETY
comment
- Match on ::try_from
result
- Add TODO comment regarding raising a diagnostic for `"foo"["bar":"baz"]`
2024-10-29 19:40:57 +00:00
Alex Waygood
8d98aea6c4
[red-knot] Infer attribute expressions in type annotations (#13967) 2024-10-29 11:06:44 +00:00
Alex Waygood
d2c9f5e43c
[red-knot] Fallback to attributes on types.ModuleType if a symbol can't be found in locals or globals (#13904) 2024-10-29 10:59:03 +00:00
Alex Waygood
7dd0c7f4bd
[red-knot] Infer tuple types from annotations (#13943)
## Summary

This PR adds support for heterogenous `tuple` annotations to red-knot.

It does the following:
- Extends `infer_type_expression` so that it understands tuple
annotations
- Changes `infer_type_expression` so that `ExprStarred` nodes in type
annotations are inferred as `Todo` rather than `Unknown` (they're valid
in PEP-646 tuple annotations)
- Extends `Type::is_subtype_of` to understand when one heterogenous
tuple type can be understood to be a subtype of another (without this
change, the PR would have introduced new false-positive errors to some
existing mdtests).
2024-10-29 10:30:03 +00:00
David Peter
56c796acee
[red-knot] Slice expression types & subscript expressions with slices (#13917)
## Summary

- Add a new `Type::SliceLiteral` variant
- Infer `SliceLiteral` types for slice expressions, such as
`<int-literal>:<int-literal>:<int-literal>`.
- Infer "sliced" literal types for subscript expressions using slices,
such as `<string-literal>[<slice-literal>]`.
- Infer types for expressions involving slices of tuples:
`<tuple>[<slice-literal>]`.

closes #13853

## Test Plan

- Unit tests for indexing/slicing utility functions
- Markdown-based tests for
  - Subscript expressions `tuple[slice]`
  - Subscript expressions `string_literal[slice]`
  - Subscript expressions `bytes_literal[slice]`
2024-10-29 10:17:31 +01:00
Raphael Gaschignard
2fe203292a
[red-knot] Distribute intersections on negation (#13962)
## Summary

This does two things:
- distribute negated intersections when building up intersections (i.e.
going from `A & ~(B & C)` to `(A & ~B) | (A & ~C)`) (fixing #13931)

## Test Plan

`cargo test`
2024-10-29 02:56:04 +00:00
Charlie Marsh
b19862c64a
Rename operator-unsupported to unsupported-operator (#13973)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13959.
2024-10-28 21:34:12 -04:00
TomerBin
9a0dade925
[red-knot] Type narrowing inside boolean expressions (#13970)
## Summary

This PR adds type narrowing in `and` and `or` expressions, for example:

```py
class A: ...

x: A | None = A() if bool_instance() else None

isinstance(x, A) or reveal_type(x)  # revealed: None
``` 

## Test Plan
New mdtests 😍
2024-10-28 18:17:48 -07:00
TomerBin
74cf66e4c2
[red-knot] Narrowing - Not operator (#13942)
## Summary

After #13918 has landed, narrowing constraint negation became easy, so
adding support for `not` operator.

## Test Plan

Added a new mdtest file for `not` expression.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-28 20:27:26 +00:00
Charlie Marsh
6f52d573ef
Support inference for PEP 604 union annotations (#13964)
## Summary

Supports return type inference for, e.g., `def f() -> int | None:`.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-28 10:13:01 -04:00
TomerBin
66c3aaa307
[red-knot] - Flow-control for boolean operations (#13940)
## Summary

As python uses short-circuiting boolean operations in runtime, we should
mimic that logic in redknot as well.
For example, we should detect that in the following code `x` might be
undefined inside the block:

```py
if flag or (x := 1):
    print(x) 
```

## Test Plan

Added mdtest suit for boolean expressions.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-27 03:33:01 +00:00
cake-monotone
b6ffa51c16
[red-knot] Type inference for comparisons between arbitrary instances (#13903)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-10-26 18:19:56 +00:00
TomerBin
35f007f17f
[red-knot] Type narrow in else clause (#13918)
## Summary

Add support for type narrowing in elif and else scopes as part of
#13694.

## Test Plan

- mdtest
- builder unit test for union negation.

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-10-26 16:22:57 +00:00
Micha Reiser
6aaf1d9446
[red-knot] Remove lint-phase (#13922)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-25 18:40:52 +00:00
Alex Waygood
5eb87aa56e
[red-knot] Infer Todo, not Unknown, for PEP-604 unions in annotations (#13908) 2024-10-25 18:21:31 +00:00
Micha Reiser
32b57b2ee4
Enable nursery rules: 'redundant_clone', 'debug_assert_with_mut_call', and 'unused_peekable' (#13920) 2024-10-25 09:46:30 +02:00
Alex Waygood
3eb454699a
[red-knot] Format mdtest Python snippets more concisely (#13905) 2024-10-24 11:09:31 +00:00
David Peter
77ae0ccf0f
[red-knot] Infer subscript expression types for bytes literals (#13901)
## Summary

Infer subscript expression types for bytes literals:
```py
b = b"\x00abc\xff"

reveal_type(b[0])  # revealed: Literal[b"\x00"]
reveal_type(b[1])  # revealed: Literal[b"a"]
reveal_type(b[-1])  # revealed: Literal[b"\xff"]
reveal_type(b[-2])  # revealed: Literal[b"c"]

reveal_type(b[False])  # revealed: Literal[b"\x00"]
reveal_type(b[True])  # revealed: Literal[b"a"]
```


part of #13689
(https://github.com/astral-sh/ruff/issues/13689#issuecomment-2404285064)

## Test Plan

- New Markdown-based tests (see `mdtest/subscript/bytes.md`)
- Added missing test for `string_literal[bool_literal]`
2024-10-24 12:07:41 +02:00
David Peter
2c57c2dc8a
[red-knot] Type narrowing for isinstance checks (#13894)
## Summary

Add type narrowing for `isinstance(object, classinfo)` [1] checks:
```py
x = 1 if flag else "a"

if isinstance(x, int):
    reveal_type(x)  # revealed: Literal[1]
```

closes #13893

[1] https://docs.python.org/3/library/functions.html#isinstance

## Test Plan

New Markdown-based tests in `narrow/isinstance.md`.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-23 20:51:33 +02:00
David Peter
387076d212
[red-knot] Use track_caller for expect_ methods (#13884)
## Summary

A minor quality-of-life improvement: add
[`#[track_caller]`](https://doc.rust-lang.org/reference/attributes/codegen.html#the-track_caller-attribute)
attribute to `Type::expect_xyz()` methods and some `TypeInference` methods such that the panic-location
is reported one level higher up in the stack trace.

before: reports location inside the `Type::expect_class_literal()`
method. Not very useful.
```
thread 'types::infer::tests::deferred_annotation_builtin' panicked at crates/red_knot_python_semantic/src/types.rs:304:14:
Expected a Type::ClassLiteral variant
```

after: reports location at the `Type::expect_class_literal()` call site,
where the error was made.
```
thread 'types::infer::tests::deferred_annotation_builtin' panicked at crates/red_knot_python_semantic/src/types/infer.rs:4302:14:
Expected a Type::ClassLiteral variant
```

## Test Plan

Called `expect_class_literal()` on something that's not a
`Type::ClassLiteral` and saw that the error was reported at the call
site.
2024-10-23 12:48:19 +02:00
David Peter
f335fe4d4a
[red-knot] rename {Class,Module,Function} => {Class,Module,Function}Literal (#13873)
## Summary

* Rename `Type::Class` => `Type::ClassLiteral`
* Rename `Type::Function` => `Type::FunctionLiteral`
* Do not rename `Type::Module`
* Remove `*Literal` suffixes in `display::LiteralTypeKind` variants, as
per clippy suggestion
* Get rid of `Type::is_class()` in favor of `is_subtype_of(…, 'type')`;
modifiy `is_subtype_of` to support this.
* Add new `Type::is_xyz()` methods and use them instead of matching on
`Type` variants.

closes #13863 

## Test Plan

New `is_subtype_of_class_literals` unit test.

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-22 22:10:53 +02:00
David Peter
c6ce52c29e
[red-knot] Treat empty intersection as 'object', fix intersection simplification (#13880)
## Summary

- Properly treat the empty intersection as being of type `object`.
- Consequently, change the simplification method to explicitly add
`Never` to the positive side of the intersection when collapsing a type
such as `int & str` to `Never`, as opposed to just clearing both the
positive and the negative side.
- Minor code improvement in `bindings_ty`: use `peekable()` to check
whether the iterator over constraints is empty, instead of handling
first and subsequent elements separately.

fixes #13870

## Test Plan

- New unit tests for `IntersectionBuilder` to make sure the empty
intersection represents `object`.
- Markdown-based regression test for the original issue in #13870
2024-10-22 21:02:46 +02:00
David Peter
46c0961b0b
[red-knot] is_subtype_of: treat literals as subtype of 'object' (#13876)
Add the following subtype relations:
- `BooleanLiteral <: object`
- `IntLiteral <: object`
- `StringLiteral <: object`
- `LiteralString <: object`
- `BytesLiteral <: object`

Added a test case for `bool <: int`.

## Test Plan

New unit tests.
2024-10-22 13:32:51 +02:00
Alex
9d102799f9
[red-knot] Support for not-equal narrowing (#13749)
Add type narrowing for `!=` expression as stated in
#13694.

###  Test Plan

Add tests in new md format.

---------

Co-authored-by: David Peter <mail@david-peter.de>
2024-10-21 23:08:33 +02:00
TomerBin
a77512df68
[red-knot] Improve chained comparisons handling (#13825)
## Summary

A small fix for comparisons of multiple comparators.
Instead of comparing each comparator to the leftmost item, we should
compare it to the closest item on the left.

While implementing this, I noticed that we don’t yet narrow Yoda
comparisons (e.g., `True is x`), so I didn’t change that behavior in
this PR.

## Test Plan

Added some mdtests 🎉
2024-10-21 12:38:08 -07:00
David Peter
fa7626160b
[red-knot] handle unions on the LHS of is_subtype_of (#13857)
## Summary

Just a drive-by change that occurred to me while I was looking at
`Type::is_subtype_of`: the existing pattern for unions on the *right
hand side*:
```rs
            (ty, Type::Union(union)) => union
                .elements(db)
                .iter()
                .any(|&elem_ty| ty.is_subtype_of(db, elem_ty)),
```
is not (generally) correct if the *left hand side* is a union.

## Test Plan

Added new test cases for `is_subtype_of` and `!is_subtype_of`
2024-10-21 20:12:03 +02:00
David Peter
d9ef83bfef
[red-knot] Consistently rename BoolLiteral => BooleanLiteral (#13856)
## Summary

- Consistent naming: `BoolLiteral` => `BooleanLiteral` (it's mainly the
`Ty::BoolLiteral` variant that was renamed)

  I tripped over this a few times now, so I thought I'll smooth it out.
- Add a new test case for `Literal[True] <: bool`, as suggested here:
https://github.com/astral-sh/ruff/pull/13781#discussion_r1804922827
2024-10-21 13:55:50 +02:00
Alex Waygood
02bca9a8d0
Modernize build scripts (#13837)
Use the modern `cargo::KEY=VALUE` syntax that was stabilised in MSRV 1.77, rather than the deprecated `cargo:KEY=VALUE` syntax.
2024-10-20 22:35:35 +01:00
Alex Waygood
72adb09bf3
Simplify iteration idioms (#13834)
Remove unnecessary uses of `.as_ref()`, `.iter()`, `&**` and similar, mostly in situations when iterating over variables. Many of these changes are only possible following #13826, when we bumped our MSRV to 1.80: several useful implementations on `&Box<[T]>` were only stabilised in Rust 1.80. Some of these changes we could have done earlier, however.
2024-10-20 22:25:27 +01:00
Aditya Pratap Singh
7fd8e30eed
[red-knot] Cleanup generated names of mdtest tests (#13831)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-10-20 15:11:53 +00:00
Shaygan Hooshyari
0f0fff4d5a
[red-knot] Implement more types in binary and unary expressions (#13803)
Implemented some points from
https://github.com/astral-sh/ruff/issues/12701

- Handle Unknown and Any in Unary operation
- Handle Boolean in binary operations
- Handle instances in unary operation
- Consider division by False to be division by zero

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-10-20 01:57:21 +00:00