Commit graph

4890 commits

Author SHA1 Message Date
Simon
6cdf996af6
[red-knot] feat: introduce a new [Type::Todo] variant (#13548)
This variant shows inference that is not yet implemented..

## Summary

PR #13500 reopened the idea of adding a new type variant to keep track
of not-implemented features in Red Knot.

It was based off of #12986 with a more generic approach of keeping track
of different kind of unknowns. Discussion in #13500 agreed that keeping
track of different `Unknown` is complicated for now, and this feature is
better achieved through a new variant of `Type`.

### Requirements

Requirements for this implementation can be summed up with some extracts
of comment from @carljm on the previous PR

> So at the moment we are leaning towards simplifying this PR to just
use a new top-level variant, which behaves like Any and Unknown but
represents inference that is not yet implemented in red-knot.

> I think the general rule should be that Todo should propagate only
when the presence of the input Todo caused the output to be unknown.
>
> To take a specific example, the inferred result of addition must be
Unknown if either operand is Unknown. That is, Unknown + X will always
be Unknown regardless of what X is. (Same for X + Unknown.) In this
case, I believe that Unknown + Todo (or Todo + Unknown) should result in
Unknown, not result in Todo. If we fix the upstream source of the Todo,
the result would still be Unknown, so it's not useful to propagate the
Todo in this case: it wrongly suggests that the output is unknown
because of a todo item.

## Test Plan

This PR does not introduce new tests, but it did required to edit some
tests with the display of `[Type::Todo]` (currently `@Todo`), which
suggests that those test are placeholders requirements for features we
don't support yet.
2024-09-30 14:28:06 -07:00
Zanie Blue
9d8a4c0057
Improve display of assert_public_ty assertion failures (#13577)
While working on https://github.com/astral-sh/ruff/pull/13576 I noticed
that it was really hard to tell which assertion failed in some of these
test cases. This could be expanded to elsewhere, but I've heard this
test suite format won't be around for long?
2024-09-30 16:12:26 -05:00
Charlie Marsh
c9c748a79e
Add some basic subscript type inference (#13562)
## Summary

Just for tuples and strings -- the easiest cases. I think most of the
rest require generic support?
2024-09-30 16:50:46 -04:00
Zanie Blue
32c746bd82
Fix inference when integers are divided (#13575)
Fixes the `Operator::Div` case and adds `Operator::FloorDiv` support

Closes https://github.com/astral-sh/ruff/issues/13570
2024-09-30 15:50:37 -05:00
Charlie Marsh
d86b73eb3d
Add unary inference for integer and boolean literals (#13559)
## Summary

Just trying to familiarize myself with the general patterns, testing,
etc.

Part of https://github.com/astral-sh/ruff/issues/12701.
2024-09-30 16:29:06 +00:00
Alex Waygood
5f4b282327
[red-knot] Allow calling bool() with no arguments (#13568) 2024-09-30 13:18:01 +00:00
aditya pillai
d9267132d6
Fix leftover references to red_knot_python_semantic/vendor/ (#13561)
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
2024-09-30 11:32:02 +00:00
Edouard Choinière
bee498d635
[flake8-use-pathlib] Fix typo in link to Path.stat (PTH116) (#13546)
## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

There was a typo in the links of the docs of PTH116, where Path.stat
used to link to Path.group.
Another rule, PTH202, does it correctly: 

ec72e675d9/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/os_path_getsize.rs (L33)

This PR only fixes a one word typo.

## Test Plan

<!-- How was it tested? -->
I did not test that the doc generation framework picked up these
changes, I assume it will do it successfully.
2024-09-28 12:01:41 -04:00
TomerBin
ec72e675d9
Red Knot - Infer the return value of bool() (#13538)
## Summary
Following #13449, this PR adds custom handling for the bool constructor,
so when the input type has statically known truthiness value, it will be
used as the return value of the bool function.
For example, in the following snippet x will now be resolved to
`Literal[True]` instead of `bool`.
```python
x = bool(1)
```

## Test Plan
Some cargo tests were added.
2024-09-27 12:11:55 -07:00
Simon
1639488082
[red-knot] support fstring expressions (#13511)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Implement inference for `f-string`, contributes to #12701.

### First Implementation

When looking at the way `mypy` handles things, I noticed the following:
- No variables (e.g. `f"hello"`) ⇒ `LiteralString`
- Any variable (e.g. `f"number {1}"`) ⇒ `str`

My first commit (1ba5d0f13fdf70ed8b2b1a41433b32fc9085add2) implements
exactly this logic, except that we deal with string literals just like
`infer_string_literal_expression` (if below `MAX_STRING_LITERAL_SIZE`,
show `Literal["exact string"]`)

### Second Implementation

My second commit (90326ce9af5549af7b4efae89cd074ddf68ada14) pushes
things a bit further to handle cases where the expression within the
`f-string` are all literal values (string representation known at static
time).

Here's an example of when this could happen in code:
```python
BASE_URL = "https://httpbin.org"
VERSION = "v1"
endpoint = f"{BASE_URL}/{VERSION}/post"  # Literal["https://httpbin.org/v1/post"]
```
As this can be sightly more costly (additional allocations), I don't
know if we want this feature.

## Test Plan

- Added a test `fstring_expression` covering all cases I can think of

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2024-09-27 10:29:21 -07:00
Micha Reiser
f3e464ea4c
refactor: Simplify quote selection logic (#13536) 2024-09-27 14:40:28 +02:00
Micha Reiser
253f5f269a
refactor: Rename FormatStringContinuation to FormatImplicitConcatenatedString (#13531) 2024-09-27 08:24:50 +00:00
Micha Reiser
c046101b79
Fix codeblock dynamic line length calculation for indented examples (#13523) 2024-09-27 09:09:07 +02:00
Zanie Blue
7706f561a9
Do not offer an invalid fix for PLR1716 when the comparisons contain parenthesis (#13527)
Related to https://github.com/astral-sh/ruff/issues/13524

Doesn't offer a valid fix, opting to instead just not offer a fix at
all. If someone points me to a good way to handle parenthesis here I'm
down to try to fix the fix separately, but it looks quite hard.
2024-09-26 19:01:06 +00:00
Henry Jiang
f5e3662446
Remove jemalloc crate when building on AIX (#13529)
## Summary
Building ruff on AIX breaks on `tiki-jemalloc-sys` due to OS header
incompatibility

## Test Plan
`cargo test`

Co-authored-by: Henry Jiang <henry.jiang1@ibm.com>
2024-09-26 13:20:54 -04:00
Junzhuo ZHOU
a354d9ead6
Expose internal types as public access (#13509) 2024-09-26 17:34:30 +02:00
Zanie Blue
58a8e9c511
Fix handling of slices in tuples for FURB118, e.g., x[:, 1] (#13518)
There was already handling for the singleton `x[:]` case but not the
tuple case.

Closes https://github.com/astral-sh/ruff/issues/13508
2024-09-26 14:20:03 +00:00
ukyen
e83388dcea
Don't raise D208 when last line is non-empty (#13372)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-09-26 14:53:21 +02:00
Micha Reiser
ae39ce56c0
Bump version to 0.6.8 (#13522) 2024-09-26 14:09:03 +02:00
Micha Reiser
ff2d214e11
Don't skip over imports and other nodes containing nested statements in import collector (#13521) 2024-09-26 11:57:05 +00:00
Micha Reiser
9442cd8fae
Parenthesize match..case if guards (#13513) 2024-09-26 06:44:33 +00:00
Micha Reiser
8012707348
Align formatting of patterns in match-cases with expression formatting in clause headers (#13510) 2024-09-26 08:35:22 +02:00
haarisr
7c83af419c
red-knot: Implement the not operator for all Type variants (#13432)
Signed-off-by: haaris <haarisrahman@gmail.com>
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-09-25 13:44:19 -07:00
Zanie Blue
bbb044ebda
Detect tuples bound to variadic positional arguments i.e. *args (#13512)
In https://github.com/astral-sh/ruff/pull/13503, we added supported for
detecting variadic keyword arguments as dictionaries, here we use the
same strategy for detecting variadic positional arguments as tuples.
2024-09-25 10:03:25 -05:00
Zanie Blue
481065238b
Avoid UP028 false negatives with non-reference shadowed bindings of loop variables (#13504)
Closes https://github.com/astral-sh/ruff/issues/13266

Avoids false negatives for shadowed bindings that aren't actually
references to the loop variable. There are some shadowed bindings we
need to support still, e.g., `del` requires the loop variable to exist.
2024-09-25 10:03:09 -05:00
Zanie Blue
11f06e0d55
Detect SIM910 when using variadic keyword arguments, i.e., **kwargs (#13503)
Closes https://github.com/astral-sh/ruff/issues/13493
2024-09-25 10:02:59 -05:00
Dylan
f27a8b8c7a
[internal] ComparableExpr (f)strings and bytes made invariant under concatenation (#13301) 2024-09-25 16:58:57 +02:00
Vince van Noort
ca0ae0a484
[pylint] Implement boolean-chained-comparison (R1716) (#13435)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-09-25 09:14:12 +00:00
TomerBin
be1d5e3368
[red-knot] Add Type::bool and boolean expression inference (#13449) 2024-09-25 00:02:26 +00:00
Simon Brugman
03503f7f56
C401 message missing closing parenthesis (#13498) 2024-09-24 14:55:32 +02:00
Charlie Marsh
ff4b6d11fa
Detect basic wildcard imports in ruff analyze graph (#13486)
## Summary

I guess we can just ignore the `*` entirely for now? This will add the
`__init__.py` for anything that's importing a package.
2024-09-23 18:09:00 -04:00
Charlie Marsh
96e7f3f96f
Exit gracefully on broken pipe errors (#13485)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13483.

Closes https://github.com/astral-sh/ruff/issues/13442.

## Test Plan

```
❯ cargo run analyze graph ../django | head -n 10
   Compiling ruff v0.6.7 (/Users/crmarsh/workspace/ruff/crates/ruff)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.63s
     Running `target/debug/ruff analyze graph ../django`
warning: `ruff analyze graph` is experimental and may change without warning
{
  "/Users/crmarsh/workspace/django/django/__init__.py": [
    "/Users/crmarsh/workspace/django/django/apps/__init__.py",
    "/Users/crmarsh/workspace/django/django/conf/__init__.py",
    "/Users/crmarsh/workspace/django/django/urls/__init__.py",
    "/Users/crmarsh/workspace/django/django/utils/log.py",
    "/Users/crmarsh/workspace/django/django/utils/version.py"
  ],
  "/Users/crmarsh/workspace/django/django/__main__.py": [
    "/Users/crmarsh/workspace/django/django/core/management/__init__.py"
```
2024-09-23 13:48:43 +00:00
Charlie Marsh
90dc7438ee
Avoid panic when analyze graph hits broken pipe (#13484)
## Summary

I think we should also make the change that @BurntSushi recommended in
the linked issue, but this gets rid of the panic.

See: https://github.com/astral-sh/ruff/issues/13483

See: https://github.com/astral-sh/ruff/issues/13442

## Test Plan

```
warning: `ruff analyze graph` is experimental and may change without warning
{
  "/Users/crmarsh/workspace/django/django/__init__.py": [
    "/Users/crmarsh/workspace/django/django/apps/__init__.py",
    "/Users/crmarsh/workspace/django/django/conf/__init__.py",
    "/Users/crmarsh/workspace/django/django/urls/__init__.py",
    "/Users/crmarsh/workspace/django/django/utils/log.py",
    "/Users/crmarsh/workspace/django/django/utils/version.py"
  ],
  "/Users/crmarsh/workspace/django/django/__main__.py": [
    "/Users/crmarsh/workspace/django/django/core/management/__init__.py"
ruff failed
  Cause: Broken pipe (os error 32)
```
2024-09-23 09:43:09 -04:00
Steve C
7c55330534
Fix formatting for analyze direction values (#13476) 2024-09-23 09:18:28 +02:00
Charlie Marsh
7441da287f
Skip traversal for non-compound statements (#13441)
## Summary

None of these can contain imports.
2024-09-21 20:47:30 +00:00
Charlie Marsh
c2a5179d75
Reuse BTreeSets in module resolver (#13440)
## Summary

For dependencies, there's no reason to re-allocate here, since we know
the paths are unique.
2024-09-21 20:14:32 +00:00
Charlie Marsh
17c4690b5e
Bump version to v0.6.7 (#13439) 2024-09-21 13:16:36 -04:00
Charlie Marsh
f06d44e6e5
Use forget for module resolver database (#13438)
## Summary

A tiny bit faster and the `red-knot` CLI does the same thing.
2024-09-21 17:00:02 +00:00
Micha Reiser
653c09001a
Use an empty vendored file system in Ruff (#13436)
## Summary

This PR changes removes the typeshed stubs from the vendored file system
shipped with ruff
and instead ships an empty "typeshed".

Making the typeshed files optional required extracting the typshed files
into a new `ruff_vendored` crate. I do like this even if all our builds
always include typeshed because it means `red_knot_python_semantic`
contains less code that needs compiling.

This also allows us to use deflate because the compression algorithm
doesn't matter for an archive containing a single, empty file.

## Test Plan

`cargo test`

I verified with ` cargo tree -f "{p} {f}" -p <package> ` that:

* red_knot_wasm: enables `deflate` compression
* red_knot: enables `zstd` compression
* `ruff`: uses stored


I'm not quiet sure how to build the binary that maturin builds but
comparing the release artifact size with `strip = true` shows a `1.5MB`
size reduction

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-09-21 16:31:42 +00:00
Micha Reiser
8921fbb54c
vendored_typeshed_versions should use db.vendored (#13434) 2024-09-21 16:35:06 +02:00
Charlie Marsh
3018303c87
Avoid parsing with Salsa (#13437)
## Summary

For reasons I haven't investigated, this speeds up the resolver about 2x
(from 6.404s to 3.612s on an extremely large codebase).

## Test Plan

\cc @BurntSushi 

```
[andrew@duff rippling]$ time ruff analyze graph --preview > /dev/null

real    3.274
user    16.039
sys     7.609
maxmem  11631 MB
faults  0
[andrew@duff rippling]$ time ruff-patch analyze graph --preview > /dev/null

real    1.841
user    14.625
sys     3.639
maxmem  7173 MB
faults  0
[andrew@duff rippling]$ time ruff-patch2 analyze graph --preview > /dev/null

real    2.087
user    15.333
sys     4.869
maxmem  8642 MB
faults  0
```

Where that's `main`, then (`ruff-patch`) using the version with no
`File`, no `SemanticModel`, then (`ruff-patch2`) using `File`.
2024-09-21 13:52:16 +00:00
haarisr
6c303b2445
red-knot: Add not unary operator for boolean literals (#13422)
## Summary

Contributes to #12701

## Test Plan

Added test for boolean literals

Signed-off-by: haaris <haarisrahman@gmail.com>
2024-09-20 15:24:38 -07:00
Charlie Marsh
7579a792c7
Add test coverage for non-Python globs (#13430) 2024-09-20 20:46:00 +00:00
Charlie Marsh
ff11db61b4
Add Python version support to ruff analyze CLI (#13426) 2024-09-20 15:40:47 -04:00
Charlie Marsh
2823487bf8
Respect lint.exclude in ruff check --add-noqa (#13427)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13423.
2024-09-20 19:39:36 +00:00
Charlie Marsh
910fac781d
Add exclude support to ruff analyze (#13425)
## Summary

Closes https://github.com/astral-sh/ruff/issues/13424.
2024-09-20 15:34:35 -04:00
Carl Meyer
149fb2090e
[red-knot] more efficient UnionBuilder::add (#13411)
Avoid quadratic time in subsumed elements when adding a super-type of
existing union elements.

Reserve space in advance when adding multiple elements (from another
union) to a union.

Make union elements a `Box<[Type]>` instead of an `FxOrderSet`; the set
doesn't buy much since the rules of union uniqueness are defined in
terms of supertype/subtype, not in terms of simple type identity.

Move sealed-boolean handling out of a separate `UnionBuilder::simplify`
method and into `UnionBuilder::add`; now that `add` is iterating
existing elements anyway, this is more efficient.

Remove `UnionType::contains`, since it's now `O(n)` and we shouldn't
really need it, generally we care about subtype/supertype, not type
identity. (Right now it's used for `Type::Unbound`, which shouldn't even
be a type.)

Add support for `is_subtype_of` for the `object` type.

Addresses comments on https://github.com/astral-sh/ruff/pull/13401
2024-09-20 10:49:45 -07:00
Carl Meyer
40c65dcfa7
[red-knot] dedicated error message for all-union-elements not callable (#13412)
This was mentioned in an earlier review, and seemed easy enough to just
do it. No need to repeat all the types twice when it gives no additional
information.
2024-09-20 08:08:43 -07:00
yahayaohinoyi
03f3a4e855
[pycodestyle] Fix: Don't autofix if the first line ends in a question mark? (D400) (#13399)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-09-20 11:05:26 +00:00
Micha Reiser
531ebf6dff
Fix parentheses around return type annotations (#13381) 2024-09-20 09:23:53 +02:00