Commit graph

1145 commits

Author SHA1 Message Date
Simon Høxbro Hansen
2ba5677700
Improvements to RUF015 (#7848)
## Summary

Resolves https://github.com/astral-sh/ruff/issues/7618. 

The list of builtin iterator is not exhaustive.

## Test Plan

`cargo test`

``` python
a = [1, 2]

examples = [
    enumerate(a),
    filter(lambda x: x, a),
    map(int, a),
    reversed(a),
    zip(a),
    iter(a),
]

for example in examples:
    print(next(example))
```
2023-10-08 14:49:45 +00:00
Tom Kuson
62f1ee08e7
[refurb] Implement single-item-membership-test (FURB171) (#7815)
## Summary

Implement
[`no-single-item-in`](https://github.com/dosisod/refurb/blob/master/refurb/checks/iterable/no_single_item_in.py)
as `single-item-membership-test` (`FURB171`).

Uses the helper function `generate_comparison` from the `pycodestyle`
implementations; this function should probably be moved, but I am not
sure where at the moment.

Update: moved it to `ruff_python_ast::helpers`.

Related to #1348.

## Test Plan

`cargo test`
2023-10-08 14:08:47 +00:00
Zanie Blue
4b537d1297
Update non-pep695-type-alias to require --unsafe-fixes outside of stub files (#7836)
Closes https://github.com/astral-sh/ruff/issues/6434
2023-10-06 14:56:40 -05:00
Charlie Marsh
ad265fa6bc
Allow f-string modifications in line-shrinking cases (#7818)
## Summary

This PR resolves an issue raised in
https://github.com/astral-sh/ruff/discussions/7810, whereby we don't fix
an f-string that exceeds the line length _even if_ the resultant code is
_shorter_ than the current code.

As part of this change, I've also refactored and extracted some common
logic we use around "ensuring a fix isn't breaking the line length
rules".

## Test Plan

`cargo test`
2023-10-04 15:24:07 -04:00
Jelle Zijlstra
600471e45f
Fix SIM110 with a yield in the condition (#7801)
And allow "await" in the loop iterable.

Fixes #7800
2023-10-04 08:59:19 -04:00
Jelle Zijlstra
7b4fb4fb5d
Fix issues with SIM101 (adjacent isinstance() calls) (#7798)
- Only trigger for immediately adjacent isinstance() calls with the same
target
- Preserve order of or conditions

Two existing tests changed:
- One was incorrectly reordering the or conditions, and is now correct.
- Another was combining two non-adjacent isinstance() calls. It's safe
enough in that example,
but this isn't safe to do in general, and it feels low-value to come up
with a heuristic for
when it is safe, so it seems better to not combine the calls in that
case.

Fixes https://github.com/astral-sh/ruff/issues/7797
2023-10-04 04:42:13 +00:00
Charlie Marsh
90c259beb9
Respect msgspec.Struct default-copy semantics (#7786)
## Summary

The carve-out we have in `RUF012` for Pydantic classes also applies to
`msgspec.Struct`.

Closes https://github.com/astral-sh/ruff/issues/7785.
2023-10-03 16:51:25 +00:00
Tom Kuson
37d21c0d54
Check sequence type before triggering unnecessary-enumerate (FURB148) len suggestion (#7781)
## Summary

Check that the sequence type is a list, set, dict, or tuple before
recommending replacing the `enumerate(...)` call with `range(len(...))`.
Document behaviour so users are aware of the type inference limitation
leading to false negatives.

Closes #7656.
2023-10-03 14:39:14 +00:00
Charlie Marsh
c040fac12f
Preserve trailing comments in C414 fixes (#7775)
Closes https://github.com/astral-sh/ruff/issues/7772.
2023-10-03 04:36:51 +00:00
Tom Kuson
e129f77bcf
Extend reimplemented-starmap (FURB140) to catch calls with a single and starred argument (#7768) 2023-10-02 21:38:05 -04:00
Charlie Marsh
75f759ed55
Upgrade LibCST to support Python 3.12 (#7764)
## Summary

We'll revert back to the crates.io release once it's up-to-date, but
better to get this out now that Python 3.12 is released.

## Test Plan

`cargo test`
2023-10-02 12:14:35 -04:00
konsti
f70e8a7524
Fix PLE251 rules with f-string escaping (#7741)
**Summary** The `value` of the `FStringMiddle` for `f"""}}ab"""` is
`}ab`, i.e. the curly brace escaping is decoded. If we iterate over
string this gives us false indices causing exploding fixes for PLE251
rules (PLE2510, PLE2512, PLE2513, PLE2514, PLE2515). Instead, we now use
the source range.

Handles
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1741998106
Handles
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1741998256

**Test Plan** Minimized fuzzing cases as fixtures.
2023-10-02 08:43:39 +00:00
Charlie Marsh
2838f7af98
Skip all bracketed expressions when locating comparison ops (#7740)
Closes https://github.com/astral-sh/ruff/issues/7737.
2023-10-01 14:57:40 +00:00
Charlie Marsh
1cf3b5676f
Perform insertions before replacements (#7739)
## Summary

If we have, e.g.:

```python
sum((
            factor.dims for factor in bases
        ), [])
```

We generate three edits: two insertions (for the `operator` and
`functools` imports), and then one replacement (for the `sum` call
itself). We need to ensure that the insertions come before the
replacement; otherwise, the edits will appear overlapping and
out-of-order.

Closes https://github.com/astral-sh/ruff/issues/7718.
2023-10-01 14:53:54 +00:00
Charlie Marsh
1646939383
Ignore overlong pragma comments when enforcing linter line length (#7692)
## Summary

This PR modifies the `line-too-long` and `doc-line-too-long` rules to
ignore lines that are too long due to the presence of a pragma comment
(e.g., `# type: ignore` or `# noqa`). That is, if a line only exceeds
the limit due to the pragma comment, it will no longer be flagged as
"too long". This behavior mirrors that of the formatter, thus ensuring
that we don't flag lines under E501 that the formatter would otherwise
avoid wrapping.

As a concrete example, given a line length of 88, the following would
_no longer_ be considered an E501 violation:

```python
# The string literal is 88 characters, including quotes.
"shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:sh"  # type: ignore
```

This, however, would:

```python
# The string literal is 89 characters, including quotes.
"shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:shape:sha"  # type: ignore
```

In addition to mirroring the formatter, this also means that adding a
pragma comment (like `# noqa`) won't _cause_ additional violations to
appear (namely, E501). It's very common for users to add a `# type:
ignore` or similar to a line, only to find that they then have to add a
suppression comment _after_ it that was required before, as in `# type:
ignore # noqa: E501`.

Closes https://github.com/astral-sh/ruff/issues/7471.

## Test Plan

`cargo test`
2023-09-29 23:26:52 +00:00
Charlie Marsh
e9f8b91eb5
Preserve parentheses in quadratic-list-summation (#7719)
Closes https://github.com/astral-sh/ruff/issues/7718.
2023-09-29 20:04:56 +00:00
Charlie Marsh
b5280061f8
Use fixed source code for parser context (#7717)
## Summary

The parser now uses the raw source code as global context and slices
into it to parse debug text. It turns out we were always passing in the
_old_ source code, so when code was fixed, we were making invalid
accesses. This PR modifies the call to use the _fixed_ source code,
which will always be consistent with the tokens.

Closes https://github.com/astral-sh/ruff/issues/7711.

## Test Plan

`cargo test`
2023-09-29 14:10:32 -04:00
Charlie Marsh
253fbb665f
Track fix isolation in unnecessary-pass (#7715)
## Summary

This wasn't necessary in the past, since we _only_ applied this rule to
bodies that contained two statements, one of which was a `pass`. Now
that it applies to any `pass` in a block with multiple statements, we
can run into situations in which we remove both passes, and so need to
apply the fixes in isolation.

See:
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1741107573.
2023-09-29 17:23:04 +00:00
Dhruv Manilawala
e62e245c61
Add support for PEP 701 (#7376)
## Summary

This PR adds support for PEP 701 in Ruff. This is a rollup PR of all the
other individual PRs. The separate PRs were created for logic separation
and code reviews. Refer to each pull request for a detail description on
the change.

Refer to the PR description for the list of pull requests within this PR.

## Test Plan

### Formatter ecosystem checks

Explanation for the change in ecosystem check:
https://github.com/astral-sh/ruff/pull/7597#issue-1908878183

#### `main`

```
| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76083 |              1789 |              1631 |
| django       |           0.99983 |              2760 |                36 |
| transformers |           0.99963 |              2587 |               319 |
| twine        |           1.00000 |                33 |                 0 |
| typeshed     |           0.99983 |              3496 |                18 |
| warehouse    |           0.99967 |               648 |                15 |
| zulip        |           0.99972 |              1437 |                21 |
```

#### `dhruv/pep-701`

```
| project      | similarity index  | total files       | changed files     |
|--------------|------------------:|------------------:|------------------:|
| cpython      |           0.76051 |              1789 |              1632 |
| django       |           0.99983 |              2760 |                36 |
| transformers |           0.99963 |              2587 |               319 |
| twine        |           1.00000 |                33 |                 0 |
| typeshed     |           0.99983 |              3496 |                18 |
| warehouse    |           0.99967 |               648 |                15 |
| zulip        |           0.99972 |              1437 |                21 |
```
2023-09-29 02:55:39 +00:00
Daniel Parizher
78b8741352
[refurb] Implement implicit-cwd (FURB177) (#7704)
## Summary

Implement
[`no-implicit-cwd`](https://github.com/dosisod/refurb/blob/master/docs/checks.md#furb177-no-implicit-cwd)
as `implicit-cwd`

Related to #1348.

## Test Plan

`cargo test`
2023-09-29 02:18:59 +00:00
Tom Kuson
3347524164
Extend unnecessary-pass (PIE790) to trigger on all unnecessary pass statements (#7697)
## Summary

Extend `unnecessary-pass` (`PIE790`) to trigger on all unnecessary
`pass` statements by checking for `pass` statements in any class or
function body with more than one statement.

Closes #7600.

## Test Plan

`cargo test`
2023-09-29 01:39:11 +00:00
Mathieu Kniewallner
598974545b
feat(rules): implement flake8-bandit S505 (#7703)
Part of #1646.

## Summary

Implement `S505`
([`weak_cryptographic_key`](https://bandit.readthedocs.io/en/latest/plugins/b505_weak_cryptographic_key.html))
rule from `bandit`.

For this rule, `bandit` [reports the issue
with](https://github.com/PyCQA/bandit/blob/1.7.5/bandit/plugins/weak_cryptographic_key.py#L47-L56):
- medium severity for DSA/RSA < 2048 bits and EC < 224 bits
- high severity for DSA/RSA < 1024 bits and EC < 160 bits

Since Ruff does not handle severities for `bandit`-related rules, we
could either report the issue if we have lower values than medium
severity, or lower values than high one. Two reasons led me to choose
the first option:
- a medium severity issue is still a security issue we would want to
report to the user, who can then decide to either handle the issue or
ignore it
- `bandit` [maps the EC key algorithms to their respective key lengths
in
bits](https://github.com/PyCQA/bandit/blob/1.7.5/bandit/plugins/weak_cryptographic_key.py#L112-L133),
but there is no value below 160 bits, so technically `bandit` would
never report medium severity issues for EC keys, only high ones

Another consideration is that as shared just above, for EC key
algorithms, `bandit` has a mapping to map the algorithms to their
respective key lengths. In the implementation in Ruff, I rather went
with an explicit list of EC algorithms known to be vulnerable (which
would thus be reported) rather than implementing a mapping to retrieve
the associated key length and comparing it with the minimum value.

## Test Plan

Snapshot tests from
https://github.com/PyCQA/bandit/blob/1.7.5/examples/weak_cryptographic_key_sizes.py.
2023-09-28 21:27:37 -04:00
Mathieu Kniewallner
cfbebcf354
fix(rules): improve S507 detection (#7661)
## Summary

Follow-up on https://github.com/astral-sh/ruff/pull/7528 that improves
detections of mis-usages of policy in `paramiko`.

First commit applies the same fix as in `bandit`
(https://github.com/PyCQA/bandit/pull/1064), as `paramiko` supports
passing both a class and a class instance for the policy in
`set_missing_host_key_policy`
(8e389c7766/paramiko/client.py (L171-L191)).

Second commit improve the detection of `paramiko` import paths that
trigger a violation, as `AutoAddPolicy`, `WarningPolicy` and `SSHClient`
are not only exposed in `paramiko.client`, but also in `paramiko`
(66117732de/paramiko/__init__.py (L121-L164)).

## Test Plan

Snapshot tests.
2023-09-28 21:35:59 +00:00
Charlie Marsh
5e75467757
Insert necessary padding in B014 fixes (#7699)
See:
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1739801758.
2023-09-28 21:11:09 +00:00
Charlie Marsh
9611f8134f
Parenthesize multi-line attributes in B009 (#7701)
Closes
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1739800901.
2023-09-28 16:59:00 -04:00
konsti
1e173f7909
Rename Autofix to Fix (#7657)
**Summary** Mostly mechanical symbol rename and search-and-replace, with
small changes to the markdown docs to read better
2023-09-28 10:53:05 +00:00
qdegraaf
c8360a1333
Expand DeprecatedLogWarn to check for Expr::Atrribute calls (#7677)
## Summary

`PGH002`, which checks for use of deprecated `logging.warn` calls, did
not check for calls made on the attribute `warn` yet. Since
https://github.com/astral-sh/ruff/pull/7521 we check both cases for
similar rules wherever possible. To be consistent this PR expands PGH002
to do the same.

## Test Plan

Expanded existing fixtures with `logger.warn()` calls

## Issue links

Fixes final inconsistency mentioned in
https://github.com/astral-sh/ruff/issues/7502
2023-09-27 11:38:52 -04:00
Charlie Marsh
0a8cad2550
Allow named expressions in __all__ assignments (#7673)
## Summary

This PR adds support for named expressions when analyzing `__all__`
assignments, as per https://github.com/astral-sh/ruff/issues/7672. It
also loosens the enforcement around assignments like: `__all__ =
list(some_other_expression)`. We shouldn't flag these as invalid, even
though we can't analyze the members, since we _know_ they evaluate to a
`list`.

Closes https://github.com/astral-sh/ruff/issues/7672.

## Test Plan

`cargo test`
2023-09-27 00:36:55 -04:00
Simon Høxbro Hansen
fbbc982c29
Ensure that B006 autofixes are inserted after imports (#7629)
## Summary

Fixes #7616 by ensuring that
[B006](https://docs.astral.sh/ruff/rules/mutable-argument-default/#mutable-argument-default-b006)
fixes are inserted after module imports.

I have created a new test file, `B006_5.py`. This is mainly because I
have been working on this on and off, and the merge conflicts were
easier to handle in a separate file. If needed, I can move it into
another file.

## Test Plan

`cargo test`
2023-09-27 01:26:29 +00:00
qdegraaf
2aef46cb6f
Add Expr::Name checks to rules which use is_logger_candidate (#7521)
## Summary

Expands several rules to also check for `Expr::Name` values. As they
would previously not consider:
```python
from logging import error

error("foo")
```
as potential violations
```python
import logging

logging.error("foo")
```
as potential violations leading to inconsistent behaviour. 

The rules impacted are:

- `BLE001`
- `TRY400`
- `TRY401`
- `PLE1205`
- `PLE1206`
- `LOG007`
- `G001`-`G004`
- `G101`
- `G201`
- `G202`

## Test Plan

Fixtures for all impacted rules expanded. 

## Issue Link

Refers: https://github.com/astral-sh/ruff/issues/7502
2023-09-27 00:21:22 +00:00
konsti
26f9b4a8e6
Don't suggest replacing builtin.open() with Path.open() if the latter doesn't support all options (#7637)
**Summary** Check that `closefd` and `opener` aren't being used with
`builtin.open()` before suggesting `Path.open()` because pathlib doesn't
support these arguments.

Closes #7620

**Test Plan** New cases in the fixture.
2023-09-26 09:07:35 +00:00
Charlie Marsh
93b5d8a0fb
Implement our own small-integer optimization (#7584)
## Summary

This is a follow-up to #7469 that attempts to achieve similar gains, but
without introducing malachite. Instead, this PR removes the `BigInt`
type altogether, instead opting for a simple enum that allows us to
store small integers directly and only allocate for values greater than
`i64`:

```rust
/// A Python integer literal. Represents both small (fits in an `i64`) and large integers.
#[derive(Clone, PartialEq, Eq, Hash)]
pub struct Int(Number);

#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum Number {
    /// A "small" number that can be represented as an `i64`.
    Small(i64),
    /// A "large" number that cannot be represented as an `i64`.
    Big(Box<str>),
}

impl std::fmt::Display for Number {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            Number::Small(value) => write!(f, "{value}"),
            Number::Big(value) => write!(f, "{value}"),
        }
    }
}
```

We typically don't care about numbers greater than `isize` -- our only
uses are comparisons against small constants (like `1`, `2`, `3`, etc.),
so there's no real loss of information, except in one or two rules where
we're now a little more conservative (with the worst-case being that we
don't flag, e.g., an `itertools.pairwise` that uses an extremely large
value for the slice start constant). For simplicity, a few diagnostics
now show a dedicated message when they see integers that are out of the
supported range (e.g., `outdated-version-block`).

An additional benefit here is that we get to remove a few dependencies,
especially `num-bigint`.

## Test Plan

`cargo test`
2023-09-25 15:13:21 +00:00
Charlie Marsh
39ddad7454
Refactor FURB105 into explicit cases (#7634)
## Summary

I was having trouble keeping track of the various cases here, so opted
to refactor into a more explicit `match`.
2023-09-24 18:46:09 +00:00
Charlie Marsh
f32b0eef9c
Flag FURB105 with starred kwargs (#7630) 2023-09-24 14:28:20 +00:00
Dhruv Manilawala
15813a65f3
Update return type for PT022 autofix (#7613)
## Summary

This PR fixes the autofix behavior for `PT022` to create an additional
edit for the return type if it's present. The edit will update the
return type from `Generator[T, ...]` to `T`. As per the [official
documentation](https://docs.python.org/3/library/typing.html?highlight=typing%20generator#typing.Generator),
the first position is the yield type, so we can ignore other positions.

```python
typing.Generator[YieldType, SendType, ReturnType]
```

## Test Plan

Add new test cases, `cargo test` and review the snapshots.

fixes: #7610
2023-09-24 06:39:47 +00:00
Tom Kuson
604cf521b5
[refurb] Implement print-empty-string (FURB105) (#7617)
## Summary

Implement
[`simplify-print`](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/print.py)
as `print-empty-string` (`FURB105`).

Extends the original rule in that it also checks for multiple empty
string positional arguments with an empty string separator.

Related to #1348.

## Test Plan

`cargo test`
2023-09-24 04:10:36 +00:00
Charlie Marsh
1a22eae98c
Use deletion for D215 full-line removals (#7625)
Closes https://github.com/astral-sh/ruff/issues/7619.
2023-09-23 22:44:55 +00:00
Charlie Marsh
8ba8896a7f
Skip BOM when inserting start-of-file imports (#7622)
See:
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1732387485.
2023-09-23 19:36:50 +00:00
Charlie Marsh
b194f59aab
Avoid flagging B009 and B010 on starred expressions (#7621)
See:
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1732387247.
2023-09-23 19:08:19 +00:00
Charlie Marsh
a51b0b02f0
Treat os.error as an OSError alias (#7582)
Closes https://github.com/astral-sh/ruff/issues/7580.
2023-09-21 21:18:14 +00:00
Charlie Marsh
87a0cd219f
Detect asyncio.get_running_loop calls in RUF006 (#7562)
## Summary

We can do a good enough job detecting this with our existing semantic
model.

Closes https://github.com/astral-sh/ruff/issues/3237.
2023-09-21 04:37:38 +00:00
Charlie Marsh
ad893f8295
Avoid invalid fix for parenthesized values in F601 (#7559)
Closes https://github.com/astral-sh/ruff/issues/4897.
2023-09-21 01:28:11 +00:00
Charlie Marsh
621bed55c0
Add padding in PERF102 fixes (#7554)
Closes https://github.com/astral-sh/ruff/issues/7097.
2023-09-20 19:33:54 -04:00
Charlie Marsh
a0917ec658
Avoid inserting imports directly after continuation (#7553)
## Summary

This is extremely rare in practice, but common in the fuzzer issues so
worth fixing quickly.

Closes https://github.com/astral-sh/ruff/issues/7199.
2023-09-20 21:26:48 +00:00
Charlie Marsh
5849a75223
Rename ruff crate to ruff_linter (#7529) 2023-09-20 08:38:27 +02:00