Commit graph

2519 commits

Author SHA1 Message Date
dependabot[bot]
288c07d911
Bump rayon from 1.7.0 to 1.8.0 (#7564)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-21 11:52:41 +02:00
Micha Reiser
f8f1cd5016
Introduce FormatterSettings (#7545) 2023-09-21 08:01:24 +02:00
Charlie Marsh
87a0cd219f
Detect asyncio.get_running_loop calls in RUF006 (#7562)
## Summary

We can do a good enough job detecting this with our existing semantic
model.

Closes https://github.com/astral-sh/ruff/issues/3237.
2023-09-21 04:37:38 +00:00
Charlie Marsh
b685ee4749
Enable tab completion for ruff rule (#7560)
## Summary

Writing `ruff rule E1`, then tab, shows:

<img width="724" alt="Screen Shot 2023-09-20 at 9 29 09 PM"
src="00297f24-8828-4485-a00e-6af1ab4e7875">

Closes https://github.com/astral-sh/ruff/issues/2812.
2023-09-20 22:05:39 -04:00
Charlie Marsh
ad893f8295
Avoid invalid fix for parenthesized values in F601 (#7559)
Closes https://github.com/astral-sh/ruff/issues/4897.
2023-09-21 01:28:11 +00:00
Charlie Marsh
621bed55c0
Add padding in PERF102 fixes (#7554)
Closes https://github.com/astral-sh/ruff/issues/7097.
2023-09-20 19:33:54 -04:00
Charlie Marsh
a0917ec658
Avoid inserting imports directly after continuation (#7553)
## Summary

This is extremely rare in practice, but common in the fuzzer issues so
worth fixing quickly.

Closes https://github.com/astral-sh/ruff/issues/7199.
2023-09-20 21:26:48 +00:00
Charlie Marsh
5df0326bc8
Treat parameters-with-newline as empty in function formatting (#7550)
## Summary

If a function has no parameters (and no comments within the parameters'
`()`), we're supposed to wrap the return annotation _whenever_ it
breaks. However, our `empty_parameters` test didn't properly account for
the case in which the parameters include a newline (but no other
content), like:

```python
def get_dashboards_hierarchy(
) -> Dict[Type['BaseDashboard'], List[Type['BaseDashboard']]]:
    """Get hierarchy of dashboards classes.

    Returns:
        Dict of dashboards classes.
    """
    dashboards_hierarchy = {}
```

This PR fixes that detection. Instead of lexing, it now checks if the
parameters itself is empty (or if it contains comments).

Closes https://github.com/astral-sh/ruff/issues/7457.
2023-09-20 16:20:22 -04:00
Micha Reiser
6540321966
Move Settings and ResolverSettings to ruff_workspace
## Summary

## Stack Summary

This stack splits `Settings` into `FormatterSettings` and `LinterSettings` and moves it into `ruff_workspace`. This change is necessary to add the `FormatterSettings` to `Settings` without adding `ruff_python_formatter` as a dependency to `ruff_linter` (and the linter should not contain the formatter settings). 

A quick overview of our settings struct at play:

* `Options`: 1:1 representation of the options in the `pyproject.toml` or `ruff.toml`.  Used for deserialization.
* `Configuration`: Resolved `Options`, potentially merged from multiple configurations (when using `extend`). The representation is very close if not identical to the `Options`.
* `Settings`: The resolved configuration that uses a data format optimized for reading. Optional fields are initialized with their default values. Initialized by `Configuration::into_settings` .

The goal of this stack is to split `Settings` into tool-specific resolved `Settings` that are independent of each other. This comes at the advantage that the individual crates don't need to know anything about the other tools. The downside is that information gets duplicated between `Settings`. Right now the duplication is minimal (`line-length`, `tab-width`) but we may need to come up with a solution if more expensive data needs sharing.

This stack focuses on `Settings`. Splitting `Configuration` into some smaller structs is something I'll follow up on later. 

## PR Summary

This PR moves the `ResolverSettings` and `Settings` struct to `ruff_workspace`. `LinterSettings` remains in `ruff_linter` because it gets passed to lint rules, the `Checker` etc.

## Test Plan

`cargo test`
2023-09-20 17:24:28 +02:00
Micha Reiser
b34278e0cd
Introduce LinterSettings
## Stack Summary

This stack splits `Settings` into `FormatterSettings` and `LinterSettings` and moves it into `ruff_workspace`. This change is necessary to add the `FormatterSettings` to `Settings` without adding `ruff_python_formatter` as a dependency to `ruff_linter` (and the linter should not contain the formatter settings). 

A quick overview of our settings struct at play:

* `Options`: 1:1 representation of the options in the `pyproject.toml` or `ruff.toml`.  Used for deserialization.
* `Configuration`: Resolved `Options`, potentially merged from multiple configurations (when using `extend`). The representation is very close if not identical to the `Options`.
* `Settings`: The resolved configuration that uses a data format optimized for reading. Optional fields are initialized with their default values. Initialized by `Configuration::into_settings` .

The goal of this stack is to split `Settings` into tool-specific resolved `Settings` that are independent of each other. This comes at the advantage that the individual crates don't need to know anything about the other tools. The downside is that information gets duplicated between `Settings`. Right now the duplication is minimal (`line-length`, `tab-width`) but we may need to come up with a solution if more expensive data needs sharing.

This stack focuses on `Settings`. Splitting `Configuration` into some smaller structs is something I'll follow up on later. 

## PR Summary

This PR extracts the linter-specific settings into a new `LinterSettings` struct and adds it as a `linter` field to the `Settings` struct. This is in preparation for moving `Settings` from `ruff_linter` to `ruff_workspace`

## Test Plan

`cargo test`
2023-09-20 17:02:34 +02:00
Micha Reiser
222f1c37b8
Add empty lines between options and move documents (#7547) 2023-09-20 16:40:25 +02:00
Micha Reiser
8f41eab0c7
Extract ResolverSettings
## Stack Summary

This stack splits `Settings` into `FormatterSettings` and `LinterSettings` and moves it into `ruff_workspace`. This change is necessary to add the `FormatterSettings` to `Settings` without adding `ruff_python_formatter` as a dependency to `ruff_linter` (and the linter should not contain the formatter settings). 

A quick overview of our settings struct at play:

* `Options`: 1:1 representation of the options in the `pyproject.toml` or `ruff.toml`.  Used for deserialization.
* `Configuration`: Resolved `Options`, potentially merged from multiple configurations (when using `extend`). The representation is very close if not identical to the `Options`.
* `Settings`: The resolved configuration that uses a data format optimized for reading. Optional fields are initialized with their default values. Initialized by `Configuration::into_settings` .

The goal of this stack is to split `Settings` into tool-specific resolved `Settings` that are independent of each other. This comes at the advantage that the individual crates don't need to know anything about the other tools. The downside is that information gets duplicated between `Settings`. Right now the duplication is minimal (`line-length`, `tab-width`) but we may need to come up with a solution if more expensive data needs sharing.

This stack focuses on `Settings`. Splitting `Configuration` into some smaller structs is something I'll follow up on later. 

## PR Summary

This PR extracts a `ResolverSettings` struct that holds all the resolver-relevant fields (uninteresting for the `Formatter` or `Linter`). This will allow us to move the `ResolverSettings` out of `ruff_linter` further up in the stack.


## Test Plan

`cargo test`

(I'll to more extensive testing at the top of this stack)
2023-09-20 16:37:49 +02:00
Micha Reiser
83daddbeb7
Rename ConfigProcessor to ConfigurationTransformer (#7536) 2023-09-20 14:17:06 +00:00
Micha Reiser
b19eec9b2a
Unify Settings and AllSettings (#7532) 2023-09-20 13:56:07 +00:00
Micha Reiser
ca3c15858d
Use portable hash function for CacheKey (#7533) 2023-09-20 15:19:59 +02:00
Micha Reiser
bb4f7c681a
Rename format option to output-format (#7514) 2023-09-20 15:18:58 +02:00
Dhruv Manilawala
0a167dd20b
Use strict sorted and union for NoQA mapping insertion (#7531)
## Summary

This PR fixes the way NoQA range is inserted to the `NoqaMapping`.

Previously, the way the mapping insertion logic worked was as follows:
1. If the range which is to be inserted _touched_ the previous range, meaning
that the end of the previous range was the same as the start of the new
range, then the new range was added in addition to the previous range.
2. Else, if the new range intersected the previous range, then the previous
   range was replaced with the new _intersection_ of the two ranges.

The problem with this logic is that it does not work for the following case:
```python
assert foo, \
    """multi-line
    string"""
```

Now, the comments cannot be added to the same line which ends with a continuation
character. So, the `NoQA` directive has to be added to the next line. But, the
next line is also a triple-quoted string, so the `NoQA` directive for that line
needs to be added to the next line. This creates a **union** pattern instead of an
**intersection** pattern.

But, only union doesn't suffice because (1) means that for the edge case where
the range touch only at the end, the union won't take place.

### Solution

1. Replace '<=' with '<' to have a _strict_ insertion case
2. Use union instead of intersection

## Test Plan

Add a new test case. Run the test suite to ensure that nothing is broken.

### Integration

1. Make a `test.py` file with the following contents:

    ```python
    assert foo, \
        """multi-line
        string"""
    ```

2. Run the following command:

    ```console
	$ cargo run --bin ruff -- check --isolated --no-cache --select=F821 test.py
	/Users/dhruv/playground/ruff/fstring.py:1:8: F821 Undefined name `foo`
    Found 1 error.
    ```

3. Use `--add-noqa`:

    ```console
	$ cargo run --bin ruff -- check --isolated --no-cache --select=F821 --add-noqa test.py
    Added 1 noqa directive.
    ```

4. Check that the NoQA directive was added in the correct position:

    ```python
    assert foo, \
        """multi-line
        string"""  # noqa: F821
    ```

5. Run the `check` command to ensure that the NoQA directive is respected:

    ```console
	$ cargo run --bin ruff -- check --isolated --no-cache --select=F821 test.py
    ```

fixes: #7530
2023-09-20 11:07:19 +00:00
Micha Reiser
192463c2fb
Allow parenthesized content exceed configured line width (#7490) 2023-09-20 08:39:25 +02:00
Charlie Marsh
5849a75223
Rename ruff crate to ruff_linter (#7529) 2023-09-20 08:38:27 +02:00
Mathieu Kniewallner
dcbd8eacd8
feat(rules): implement flake8-bandit S507 (ssh_no_host_key_verification) (#7528)
Part of #1646.

## Summary

Implement `S507`
([`ssh_no_host_key_verification`](https://bandit.readthedocs.io/en/latest/plugins/b507_ssh_no_host_key_verification.html))
rule from `bandit`.

## Test Plan

Snapshot test from
https://github.com/PyCQA/bandit/blob/1.7.5/examples/no_host_key_verification.py,
with several additions to test for more cases (most notably passing the
parameter as a named argument).
2023-09-19 20:31:45 -04:00
Micha Reiser
297ec2c2d2
Use Settings where AllSettings isn't required (#7518) 2023-09-19 23:16:20 +02:00
Tom Kuson
511cc25fc4
[refurb] Implement unnecessary-enumerate (FURB148) (#7454)
## Summary

Implement
[`no-ignored-enumerate-items`](https://github.com/dosisod/refurb/blob/master/refurb/checks/builtin/no_ignored_enumerate.py)
as `unnecessary-enumerate` (`FURB148`).

The auto-fix considers if a `start` argument is passed to the
`enumerate()` function. If only the index is used, then the suggested
fix is to pass the `start` value to the `range()` function. So,

```python
for i, _ in enumerate(xs, 1):
    ...
```

becomes

```python
for i in range(1, len(xs)):
    ...
```

If the index is ignored and only the value is ignored, and if a start
value greater than zero is passed to `enumerate()`, the rule doesn't
produce a suggestion. I couldn't find a unanimously accepted best way to
iterate over a collection whilst skipping the first n elements. The rule
still triggers, however.

Related to #1348.

## Test Plan

`cargo test`

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-09-19 20:19:28 +00:00
Charlie Marsh
4c4eceee36
Add dangling comment handling for lambda expressions (#7493)
## Summary

This PR adds dangling comment handling for `lambda` expressions. In
short, comments around the `lambda` and the `:` are all considered
dangling. Comments that come between the `lambda` and the `:` may be
moved after the colon for simplicity (this is an odd position for a
comment anyway), unless they also precede the lambda parameters, in
which case they're formatted before the parameters.

Closes https://github.com/astral-sh/ruff/issues/7470.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-19 15:23:51 -04:00
Charlie Marsh
e07670ad97
Add dangling comment handling to dictionary key-value pairs (#7495)
## Summary

This PR fixes a formatting instability by changing the comment handling
around the `:` in a dictionary to mirror that of the `:` in a lambda: we
treat comments around the `:` as dangling, then format them after the
`:`.

Closes https://github.com/astral-sh/ruff/issues/7458.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1631 |
| django | 0.99983 | 2760 | 36 |
| transformers | 0.99956 | 2587 | 404 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99969 | 1437 | 21 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1631 |
| django | 0.99983 | 2760 | 36 |
| transformers | 0.99956 | 2587 | 404 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99969 | 1437 | 21 |
2023-09-19 19:17:21 +00:00
Charlie Marsh
b792140579
Ensure that LOG007 only triggers on .exception() calls (#7524)
## Summary

Previously, this rule triggered for `logging.info("...",
exc_info=False)`.
2023-09-19 17:29:54 +00:00
Tom Kuson
36a60bd50e
Add documentation to non-empty-stub-body (#7519)
## Summary

Add documentation to `non-empty-stub-body` (`PYI010`) rule. Related to
#2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-19 13:21:11 -04:00
konsti
4ae463d04b
Add a test for stmt assign breaking in preview mode (#7516)
In preview mode, black will consistently break the right side first.
This doesn't work yet, but we'll need the test later.
2023-09-19 16:16:40 +02:00
konsti
6dade5b9ab
Tokenizer: Emit only a single bogus token (#7425)
**Summary** Instead of emitting a bogus token per char, we now only emit
on single last bogus token. This leads to much more concise output.

**Test Plan** Updated fixtures
2023-09-19 16:06:03 +02:00
Charlie Marsh
97510c888b
Show --no-X variants in CLI help (#7504)
I'd really like this to render as `--preview / --no-preview`, but I
looked for a while in the Clap internals and issue tracker
(https://github.com/clap-rs/clap/issues/815) and I really can't figure
out a way to do it -- this seems like the best we can do? It's also what
they do in Orogene.

Closes https://github.com/astral-sh/ruff/issues/7486.
2023-09-19 12:27:30 +00:00
konsti
94b68f201b
Fix stylist indentation with a formfeed (#7489)
**Summary** In python, a formfeed is technically undefined behaviour
(https://docs.python.org/3/reference/lexical_analysis.html#indentation):
> A formfeed character may be present at the start of the line; it will
be ignored for
> the indentation calculations above. Formfeed characters occurring
elsewhere in the
> leading whitespace have an undefined effect (for instance, they may
reset the space
> count to zero).

In practice, they just reset the indentation:


df8b3a46a7/Parser/tokenizer.c (L1819-L1821)
a41bb2733f/crates/ruff_python_parser/src/lexer.rs (L664-L667)

The stylist didn't handle formfeeds previously and would produce invalid
indents. The remedy is to cut everything before a form feed.

Checks box for
https://github.com/astral-sh/ruff/issues/7455#issuecomment-1722458825

**Test Plan** Unit test for the stylist and a regression test for the
rule
2023-09-19 12:01:16 +02:00
Micha Reiser
37b7d0f921
fix: Compiler warning about unused map_or (#7508) 2023-09-19 08:10:01 +00:00
Micha Reiser
6a4dbd622b
Add optimized best_fit_parenthesize IR (#7475) 2023-09-19 06:29:05 +00:00
Charlie Marsh
28b48ab902
Avoid flagging starred expressions in UP007 (#7505)
## Summary

These can't be fixed, because fixing them would lead to invalid syntax.
So flagging them also feels misleading.

Closes https://github.com/astral-sh/ruff/issues/7452.
2023-09-19 03:37:38 +00:00
Valeriy Savchenko
4123d074bd
[refurb] Implement reimplemented-starmap rule (FURB140) (#7253)
## Summary

This PR is part of a bigger effort of re-implementing `refurb` rules
#1348. It adds support for
[FURB140](https://github.com/dosisod/refurb/blob/master/refurb/checks/itertools/use_starmap.py)

## Test Plan

I included a new test + checked that all other tests pass.
2023-09-19 02:18:54 +00:00
Mathieu Kniewallner
c6ba7dfbc6
feat(rules): implement flake8-bandit S201 (flask_debug_true) (#7503)
Part of #1646.

## Summary

Implement `S201`
([`flask_debug_true`](https://bandit.readthedocs.io/en/latest/plugins/b201_flask_debug_true.html))
rule from `bandit`.

I am fairly new to Rust and Ruff's codebase, so there might be better
ways to implement the rule or write the code.

## Test Plan

Snapshot test from
https://github.com/PyCQA/bandit/blob/1.7.5/examples/flask_debug.py, with
a few additions in the "unrelated" part to test a bit more cases.
2023-09-19 00:43:28 +00:00
Micael Jarniac
40f6456add
Use MkDocs' not_in_nav (#5498)
Closes #5497
Needs MkDocs 1.5 to be released.
- [x] https://github.com/mkdocs/mkdocs/milestone/15

## Summary
Uses MkDocs' `not_in_nav` config to hide spam about files in
`docs/rules/` not being in nav.
2023-09-19 00:01:43 +00:00
Micha Reiser
3e1dffab20
refactor: Use OnceCell in Memoized (#7500) 2023-09-18 19:58:55 +00:00
Micha Reiser
3336d23f48
perf: Reduce best fitting allocations (#7411) 2023-09-18 19:49:44 +00:00
Jonathan Plasse
2421805033
Avoid N802 violations for @overload methods (#7498)
Close #7479

The `@override` was already implemented

## Test Plan

Tested the code in the issue. After removing all the noqa's, only one
occurrence of `BadName()` raised a violation.
Added a fixture
2023-09-18 14:32:40 -04:00
qdegraaf
bccba5d73f
[flake8-logging] Implement LOG007: ExceptionWithoutExcInfo (#7410)
## Summary

This PR implements a new rule for `flake8-logging` plugin that checks
for uses of `logging.exception()` with `exc_info` set to `False` or a
falsy value. It suggests using `logging.error` in these cases instead.

I am unsure about the name. Open to suggestions there, went with the
most explicit name I could think of in the meantime.

Refer https://github.com/astral-sh/ruff/issues/7248

## Test Plan

Added a new fixture cases and ran `cargo test`
2023-09-18 17:46:12 +00:00
Tom Kuson
0bfdb15ecf
Add documentation to pass-statement-stub-body (#7496)
## Summary

Add documentation to `pass-statement-stub-body` (`PYI009`) rule. Related
to #2646.

## Test Plan

`python scripts/check_docs_formatted.py`
2023-09-18 17:33:15 +00:00
Charlie Marsh
728539291f
Move FormatExprDict to top of expr_dict.rs (#7494)
Put the node itself up top, and internal structs down below.
2023-09-18 11:55:18 -04:00
Dhruv Manilawala
c2bd8af59a
Remove triple-quoted string ranges computation (#7476)
## Summary

This is a follow-up PR for #7435 to remove the now unused triple-quoted
string ranges from the indexer.
2023-09-18 20:57:49 +05:30
Jaap Roes
c946bf157e
Extend bad-dunder-method-name to permit __html__ (#7492)
## Summary

Fixes #7478

## Test Plan

`cargo test`
2023-09-18 15:16:22 +00:00
Charlie Marsh
8ab2519717
Respect parentheses for precedence in await (#7468)
## Summary

We were using `Parenthesize::IfBreaks` universally for `await`, but
dropping parentheses can change the AST due to precedence. It turns out
that Black's rules aren't _exactly_ the same as operator precedence
(e.g., they leave parentheses around `await ([1, 2, 3])`, although they
aren't strictly required).

Closes https://github.com/astral-sh/ruff/issues/7467.

## Test Plan

`cargo test`

No change in similarity.

Before:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |

After:

| project | similarity index | total files | changed files |

|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-18 09:56:41 -04:00
konsti
c4d85d6fb6
Fix ''' ""''' formatting (#7485)
## Summary

`''' ""'''` is an edge case that was previously incorrectly formatted as
`""" """""`.

Fixes #7460

## Test Plan

Added regression test
2023-09-18 10:28:15 +00:00
dependabot[bot]
8243db74fe
Bump indoc from 2.0.3 to 2.0.4 (#7483)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 11:29:50 +02:00
Micha Reiser
0346e781d4
Fix handling of newlines in empty files (#7473) 2023-09-18 06:08:10 +00:00
Tom Kuson
b66bfa6570
Extend bad-dunder-method-name to permit attrs dunders (#7472)
## Summary

Closes #7451.

## Test Plan

`cargo test`
2023-09-17 16:13:33 -04:00
Charlie Marsh
28273eb00b
Avoid flagging starred elements in C402 (#7466)
## Summary

Rewriting these is not valid syntax.

Part of https://github.com/astral-sh/ruff/issues/7455.
2023-09-17 15:23:27 +00:00