Commit graph

3455 commits

Author SHA1 Message Date
Micha Reiser
341c2698a7
Run doctests as part of CI pipeline (#9939) 2024-02-12 10:18:58 +01:00
Alex Waygood
8ec56277e9
Allow arbitrary configuration options to be overridden via the CLI (#9599)
Fixes #8368
Fixes https://github.com/astral-sh/ruff/issues/9186

## Summary

Arbitrary TOML strings can be provided via the command-line to override
configuration options in `pyproject.toml` or `ruff.toml`. As an example:
to run over typeshed and respect typeshed's `pyproject.toml`, but
override a specific isort setting and enable an additional pep8-naming
setting:

```
cargo run -- check ../typeshed --no-cache --config ../typeshed/pyproject.toml --config "lint.isort.combine-as-imports=false" --config "lint.extend-select=['N801']"
```

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Zanie Blue <contact@zanie.dev>
2024-02-09 21:56:37 +00:00
Alex Waygood
d387d0ba82
RUF022, RUF023: Ensure closing parentheses for multiline sequences are always on their own line (#9793)
## Summary

Currently these rules apply the heuristic that if the original sequence
doesn't have a newline in between the final sequence item and the
closing parenthesis, the autofix won't add one for you. The feedback
from @ThiefMaster, however, was that this was producing slightly unusual
formatting -- things like this:

```py
__all__ = [
    "b", "c",
    "a", "d"]
```

were being autofixed to this:

```py
__all__ = [
    "a",
    "b",
    "c",
    "d"]
```

When, if it was _going_ to be exploded anyway, they'd prefer something
like this (with the closing parenthesis on its own line, and a trailing comma added):

```py
__all__ = [
    "a",
    "b",
    "c",
    "d",
]
```

I'm still pretty skeptical that we'll be able to please everybody here
with the formatting choices we make; _but_, on the other hand, this
_specific_ change is pretty easy to make.

## Test Plan

`cargo test`. I also ran the autofixes for RUF022 and RUF023 on CPython
to check how they looked; they looked fine to me.
2024-02-09 21:27:44 +00:00
Charlie Marsh
6f0e4ad332
Remove unnecessary string cloning from the parser (#9884)
Closes https://github.com/astral-sh/ruff/issues/9869.
2024-02-09 16:03:27 -05:00
trag1c
7ca515c0aa
Corrected PTH203–PTH205 rule descriptions (#9914)
## Summary
Closes #9898.

## Test Plan
```sh
python scripts/generate_mkdocs.py && mkdocs serve -f mkdocs.public.yml
```
2024-02-09 15:47:07 -05:00
Micha Reiser
1ce07d65bd
Use usize instead of TextSize for indent_len (#9903) 2024-02-09 20:41:36 +00:00
Charlie Marsh
52ebfc9718
Respect duplicates when rewriting type aliases (#9905)
## Summary

If a generic appears multiple times on the right-hand side, we should
only include it once on the left-hand side when rewriting.

Closes https://github.com/astral-sh/ruff/issues/9904.
2024-02-09 14:02:41 +00:00
Hoël Bagard
12a91f4e90
Fix E30X panics on blank lines with trailing white spaces (#9907) 2024-02-09 14:00:26 +00:00
Mikko Leppänen
b4f2882b72
[pydocstyle-D405] Allow using parameters as a sub-section header (#9894)
## Summary

This review contains a fix for
[D405](https://docs.astral.sh/ruff/rules/capitalize-section-name/)
(capitalize-section-name)
The problem is that Ruff considers the sub-section header as a normal
section if it has the same name as some section name. For instance, a
function/method has an argument named "parameters". This only applies if
you use Numpy style docstring.

See: [ISSUE](https://github.com/astral-sh/ruff/issues/9806)

The following will not raise D405 after the fix:
```python  
def some_function(parameters: list[str]):
    """A function with a parameters parameter

    Parameters
    ----------

    parameters:
        A list of string parameters
    """
    ...
```


## Test Plan

```bash
cargo test
```

---------

Co-authored-by: Mikko Leppänen <mikko.leppanen@vaisala.com>
Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2024-02-08 21:54:32 -05:00
Charlie Marsh
49fe1b85f2
Reduce size of Expr from 80 to 64 bytes (#9900)
## Summary

This PR reduces the size of `Expr` from 80 to 64 bytes, by reducing the
sizes of...

- `ExprCall` from 72 to 56 bytes, by using boxed slices for `Arguments`.
- `ExprCompare` from 64 to 48 bytes, by using boxed slices for its
various vectors.

In testing, the parser gets a bit faster, and the linter benchmarks
improve quite a bit.
2024-02-09 02:53:13 +00:00
Micha Reiser
bd8123c0d8
Fix clippy unused variable warning (#9902) 2024-02-08 22:13:31 +00:00
Micha Reiser
49c5e715f9
Filter out test rules in RuleSelector JSON schema (#9901) 2024-02-08 21:06:51 +00:00
Micha Reiser
fe7d965334
Reduce Result<Tok, LexicalError> size by using Box<str> instead of String (#9885) 2024-02-08 20:36:22 +00:00
Hoël Bagard
9027169125
[pycodestyle] Add blank line(s) rules (E301, E302, E303, E304, E305, E306) (#9266)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-02-08 18:35:08 +00:00
Micha Reiser
688177ff6a
Use Rust 1.76 (#9897) 2024-02-08 18:20:08 +00:00
trag1c
eb2784c495
Corrected Path symlink method name (PTH114) (#9896)
## Summary
Corrects mentions of `Path.is_link` to `Path.is_symlink` (the former
doesn't exist).

## Test Plan
```sh
python scripts/generate_mkdocs.py && mkdocs serve -f mkdocs.public.yml
```
2024-02-08 13:09:28 -05:00
Charlie Marsh
6fffde72e7
Use memchr for string lexing (#9888)
## Summary

On `main`, string lexing consists of walking through the string
character-by-character to search for the closing quote (with some
nuance: we also need to skip escaped characters, and error if we see
newlines in non-triple-quoted strings). This PR rewrites `lex_string` to
instead use `memchr` to search for the closing quote, which is
significantly faster. On my machine, at least, the `globals.py`
benchmark (which contains a lot of docstrings) gets 40% faster...

```text
lexer/numpy/globals.py  time:   [3.6410 µs 3.6496 µs 3.6585 µs]
                        thrpt:  [806.53 MiB/s 808.49 MiB/s 810.41 MiB/s]
                 change:
                        time:   [-40.413% -40.185% -39.984%] (p = 0.00 < 0.05)
                        thrpt:  [+66.623% +67.181% +67.822%]
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
lexer/unicode/pypinyin.py
                        time:   [12.422 µs 12.445 µs 12.467 µs]
                        thrpt:  [337.03 MiB/s 337.65 MiB/s 338.27 MiB/s]
                 change:
                        time:   [-9.4213% -9.1930% -8.9586%] (p = 0.00 < 0.05)
                        thrpt:  [+9.8401% +10.124% +10.401%]
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
lexer/pydantic/types.py time:   [107.45 µs 107.50 µs 107.56 µs]
                        thrpt:  [237.11 MiB/s 237.24 MiB/s 237.35 MiB/s]
                 change:
                        time:   [-4.0108% -3.7005% -3.3787%] (p = 0.00 < 0.05)
                        thrpt:  [+3.4968% +3.8427% +4.1784%]
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe
lexer/numpy/ctypeslib.py
                        time:   [46.123 µs 46.165 µs 46.208 µs]
                        thrpt:  [360.36 MiB/s 360.69 MiB/s 361.01 MiB/s]
                 change:
                        time:   [-19.313% -18.996% -18.710%] (p = 0.00 < 0.05)
                        thrpt:  [+23.016% +23.451% +23.935%]
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) low mild
  1 (1.00%) high mild
  4 (4.00%) high severe
lexer/large/dataset.py  time:   [231.07 µs 231.19 µs 231.33 µs]
                        thrpt:  [175.87 MiB/s 175.97 MiB/s 176.06 MiB/s]
                 change:
                        time:   [-2.0437% -1.7663% -1.4922%] (p = 0.00 < 0.05)
                        thrpt:  [+1.5148% +1.7981% +2.0864%]
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  5 (5.00%) high mild
  5 (5.00%) high severe
```
2024-02-08 17:23:06 +00:00
Jane Lewis
ad313b9089
RUF027 no longer has false negatives with string literals inside of method calls (#9865)
Fixes #9857.

## Summary

Statements like `logging.info("Today it is: {day}")` will no longer be
ignored by RUF027. As before, statements like `"Today it is:
{day}".format(day="Tuesday")` will continue to be ignored.

## Test Plan

The snapshot tests were expanded to include new cases. Additionally, the
snapshot tests have been split in two to separate positive cases from
negative cases.
2024-02-08 10:00:20 -05:00
Charlie Marsh
f76a3e8502
Detect mark_safe usages in decorators (#9887)
## Summary

Django's `mark_safe` can also be used as a decorator, so we should
detect usages of `@mark_safe` for the purpose of the relevant Bandit
rule.

Closes https://github.com/astral-sh/ruff/issues/9780.
2024-02-07 23:10:46 -05:00
Tom Kuson
ed07fa08bd
Fix list formatting in documention (#9886)
## Summary

Adds a blank line to render the list correctly.

## Test Plan

Ocular inspection
2024-02-07 20:01:21 -05:00
Charlie Marsh
45937426c7
Fix blank-line docstring rules for module-level docstrings (#9878)
## Summary

Given:

```python
"""Make a summary line.

Note:
----
  Per the code comment the next two lines are blank. "// The first blank line is the line containing the closing
      triple quotes, so we need at least two."

"""
```

It turns out we excluded the line ending in `"""`, because it's empty
(unlike for functions, where it consists of the indent). This PR changes
the `following_lines` iterator to always include the trailing newline,
which gives us correct and consistent handling between function and
module-level docstrings.

Closes https://github.com/astral-sh/ruff/issues/9877.
2024-02-07 16:48:28 -05:00
Charlie Marsh
533dcfb114
Add a note regarding ignore-without-code (#9879)
Closes https://github.com/astral-sh/ruff/issues/9863.
2024-02-07 21:20:18 +00:00
Hugo van Kemenade
bc023f47a1
Fix typo in option name: output_format -> output-format (#9874) 2024-02-07 16:17:58 +00:00
Jack McIvor
aa38307415
Add more NPY002 violations (#9862) 2024-02-07 09:54:11 -05:00
Charlie Marsh
e9ddd4819a
Make show-settings filters directory-agnostic (#9866)
Closes https://github.com/astral-sh/ruff/issues/9864.
2024-02-07 03:20:27 +00:00
Micha Reiser
fdb5eefb33
Improve trailing comma rule performance (#9867) 2024-02-06 23:04:36 +00:00
Charlie Marsh
daae28efc7
Respect async with in timeout-without-await (#9859)
Closes https://github.com/astral-sh/ruff/issues/9855.
2024-02-06 12:04:24 -05:00
Charlie Marsh
c34908f5ad
Use memchr for tab-indentation detection (#9853)
## Summary

The benchmarks show a pretty consistent 1% speedup here for all-rules,
though not enough to trigger our threshold of course:

![Screenshot 2024-02-05 at 11 55
59 PM](317dca3f-f25f-46f5-8ea8-894a1747d006)
2024-02-06 09:44:56 -05:00
Charlie Marsh
a662c2447c
Ignore builtins when detecting missing f-strings (#9849)
## Summary

Reported on Discord: if the name maps to a builtin, it's not bound
locally, so is very unlikely to be intended as an f-string expression.
2024-02-05 23:49:56 -05:00
Seo Sanghyeon
df7fb95cbc
Index multiline f-strings (#9837)
Fix #9777.
2024-02-05 21:25:33 -05:00
Eero Vaher
cd5bcd815d
Mention a related setting in C408 description (#9839)
#2977 added the `allow-dict-calls-with-keyword-arguments` configuration
option for the `unnecessary-collection-call (C408)` rule, but it did not
update the rule description.
2024-02-06 03:57:53 +05:30
Charlie Marsh
0ccca4083a
Bump version to v0.2.1 (#9843) 2024-02-05 15:31:05 -05:00
Charlie Marsh
041ce1e166
Respect generic Protocol in ellipsis removal (#9841)
Closes https://github.com/astral-sh/ruff/issues/9840.
2024-02-05 19:36:16 +00:00
Dhruv Manilawala
36b752876e
Implement AnyNode/AnyNodeRef for FStringFormatSpec (#9836)
## Summary

This PR adds the `AnyNode` and `AnyNodeRef` implementation for
`FStringFormatSpec` node which will be required in the f-string
formatting.

The main usage for this is so that we can pass in the node directly to
`suppressed_node` in case debug expression is used to format is as
verbatim text.
2024-02-05 19:23:43 +00:00
Micha Reiser
b3dc565473
Add --range option to ruff format (#9733)
Co-authored-by: T-256 <132141463+T-256@users.noreply.github.com>
2024-02-05 19:21:45 +00:00
Thomas M Kehrenberg
e708c08b64
Fix default for max-positional-args (#9838)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary
`max-positional-args` defaults to `max-args` if it's not specified and
the default to `max-args` is 5, so saying that the default is 3 is
definitely wrong. Ideally, we wouldn't specify a default at all for this
config option, but I don't think that's possible?

<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

<!-- How was it tested? -->
Not sure.
2024-02-05 16:58:14 +00:00
Charlie Marsh
9781563ef6
Add fast-path for comment detection (#9808)
## Summary

When we fall through to parsing, the comment-detection rule is a
significant portion of lint time. This PR adds an additional fast
heuristic whereby we abort if a comment contains two consecutive name
tokens (via the zero-allocation lexer). For the `ctypeslib.py`, which
has a few cases that are now caught by this, it's a 2.5x speedup for the
rule (and a 20% speedup for token-based rules).
2024-02-05 11:00:18 -05:00
Zanie Blue
84aea7f0c8
Drop __get__ and __set__ from unnecessary-dunder-call (#9791)
These are for descriptors which affects the behavior of the object _as a
property_; I do not think they should be called directly but there is no
alternative when working with the object directly.

Closes https://github.com/astral-sh/ruff/issues/9789
2024-02-05 10:54:29 -05:00
Shaygan Hooshyari
b47f85eb69
Preview Style: Format module level docstring (#9725)
Co-authored-by: Micha Reiser <micha@reiser.io>
2024-02-05 15:03:34 +00:00
Micha Reiser
80fc02e7d5
Don't trim last empty line in docstrings (#9813) 2024-02-05 13:29:24 +00:00
Charlie Marsh
602f8b8250
Remove CST-based fixer for C408 (#9822)
## Summary

We have to keep the fixer for a specific case: `dict` calls that include
keyword-argument members.
2024-02-04 22:26:51 -05:00
Charlie Marsh
a6bc4b2e48
Remove CST-based fixers for C405 and C409 (#9821) 2024-02-05 02:17:34 +00:00
Charlie Marsh
c5fa0ccffb
Remove CST-based fixers for C400, C401, C410, and C418 (#9819) 2024-02-04 21:00:11 -05:00
Charlie Marsh
dd77d29d0e
Remove LibCST-based fixer for C403 (#9818)
## Summary

Experimenting with rewriting one of the comprehension fixes _without_
LibCST.
2024-02-04 20:08:19 -05:00
Charlie Marsh
ad0121660e
Run dunder method rule on methods directly (#9815)
This stood out in the flamegraph and I realized it requires us to
traverse over all statements in the class (unnecessarily).
2024-02-04 14:24:57 -05:00
Charlie Marsh
5c99967c4d
Short-circuit typing matches based on imports (#9800) 2024-02-04 14:06:44 -05:00
Charlie Marsh
c53aae0b6f
Add our own ignored-names abstractions (#9802)
## Summary

These run over nearly every identifier. It's rare to override them, so
when not provided, we can just use a match against the hardcoded default
set.
2024-02-03 09:48:07 -05:00
Charlie Marsh
2352de2277
Slight speed-up for lowercase and uppercase identifier checks (#9798)
It turns out that for ASCII identifiers, this is nearly 2x faster:

```
Parser/before     time:   [15.388 ns 15.395 ns 15.406 ns]
Parser/after      time:   [8.3786 ns 8.5821 ns 8.7715 ns]
```
2024-02-03 14:40:41 +00:00
Jane Lewis
e0a6034cbb
Implement RUF027: Missing F-String Syntax lint (#9728)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Fixes #8151

This PR implements a new rule, `RUF027`.

## What it does
Checks for strings that contain f-string syntax but are not f-strings.

### Why is this bad?
An f-string missing an `f` at the beginning won't format anything, and
instead treat the interpolation syntax as literal.

### Example

```python
name = "Sarah"
dayofweek = "Tuesday"
msg = "Hello {name}! It is {dayofweek} today!"
```

It should instead be:
```python
name = "Sarah"
dayofweek = "Tuesday"
msg = f"Hello {name}! It is {dayofweek} today!"
```

## Heuristics
Since there are many possible string literals which contain syntax
similar to f-strings yet are not intended to be,
this lint will disqualify any literal that satisfies any of the
following conditions:
1. The string literal is a standalone expression. For example, a
docstring.
2. The literal is part of a function call with keyword arguments that
match at least one variable (for example: `format("Message: {value}",
value = "Hello World")`)
3. The literal (or a parent expression of the literal) has a direct
method call on it (for example: `"{value}".format(...)`)
4. The string has no `{...}` expression sections, or uses invalid
f-string syntax.
5. The string references variables that are not in scope, or it doesn't
capture variables at all.
6. Any format specifiers in the potential f-string are invalid.

## Test Plan

I created a new test file, `RUF027.py`, which is both an example of what
the lint should catch and a way to test edge cases that may trigger
false positives.
2024-02-03 00:21:03 +00:00
Emil Telstad
25d93053da
Update max-pos-args example to max-positional-args. (#9797) 2024-02-02 20:29:13 +00:00