Commit graph

2218 commits

Author SHA1 Message Date
Charlie Marsh
1044d66c1c
Add support for PatternMatchMapping formatting (#6836)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

Adds support for `PatternMatchMapping` -- i.e., cases like:

```python
match foo:
    case {"a": 1, "b": 2, **rest}:
        pass
```

Unfortunately, this node has _three_ kinds of dangling comments:

```python
{  # "open parenthesis comment"
   key: pattern,
   **  # end-of-line "double star comment"
   # own-line "double star comment"
   rest  # end-of-line "after rest comment"
   # own-line "after rest comment"
}
```

Some of the complexity comes from the fact that in `**rest`, `rest` is
an _identifier_, not a node, so we have to handle comments _after_ it as
dangling on the enclosing node, rather than trailing on `**rest`. (We
could change the AST to use `PatternMatchAs` there, which would be more
permissive than the grammar but not totally crazy -- `PatternMatchAs` is
used elsewhere to mean "a single identifier".)

Closes https://github.com/astral-sh/ruff/issues/6644.

## Test Plan

`cargo test`
2023-08-25 04:33:34 +00:00
Charlie Marsh
813d7da7ec
Respect own-line leading comments before parenthesized nodes (#6820)
## Summary

This PR ensures that if an expression has an own-line leading comment
_before_ its open parentheses, we render it as such.

For example, given:

```python
[ # foo
    # bar
    ( # baz
        1
    )
]
```

On `main`, we format as:

```python
[  # foo
    (
        # bar
        # baz
        1
    )
]
```

As of this PR, we format as:

```python
[  # foo
    # bar
    (  # baz
        1
    )
]
```

## Test Plan

`cargo test`
2023-08-25 00:18:05 -04:00
Charlie Marsh
59e70896c0
Fix formatting of comments between function and arguments (#6826)
## Summary

We now format comments between a function and its arguments as dangling.
Like with other strange placements, I've biased towards preserving the
existing formatting, rather than attempting to reorder the comments.

Closes https://github.com/astral-sh/ruff/issues/6818.

## Test Plan

`cargo test`

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820          |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |

After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820          |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |
2023-08-25 04:06:56 +00:00
Charlie Marsh
f754ad5898
Handle bracketed comments on sequence patterns (#6801)
## Summary

This PR ensures that we handle bracketed comments on sequences, like `#
comment` here:

```python
match x:
    case [ # comment
        1, 2
    ]:
        pass
```

The handling is very similar to other, similar nodes, except that we do
need some special logic to determine whether the sequence is
parenthesized, similar to our logic for tuples.

## Test Plan

`cargo test`
2023-08-25 04:03:27 +00:00
Charlie Marsh
474e8fbcd4
Format all attribute dot comments manually (#6825)
## Summary

This PR modifies our formatting of comments around the `.` in an
attribute. Specifically, the goal here is to avoid _reordering_
comments, and the net effect is that we generally leave comments
where-they-are when dealing with comments between around the dot (which
you can also think of as comments between attributes).

All comments around the dot are now treated as dangling and formatted
manually, with the exception of end-of-line or parenthesized comments on
the value, like those marked as trailing here, which remain trailing:

```python
(
    (
        a # trailing end-of-line
        # trailing own-line
    ) # dangling before dot end-of-line
    .b # trailing end-of-line
)
```

Closes https://github.com/astral-sh/ruff/issues/6823.

## Test Plan

`cargo test`

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820          |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |

After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76050          |
| django       | 0.99820   |
| transformers | 0.99800          |
| twine        | 0.99876          |
| typeshed     | 0.99953          |
| warehouse    | 0.99615          |
| zulip        | 0.99729          |
2023-08-25 03:50:56 +00:00
Charlie Marsh
6f23469e00
Handle pattern parentheses in FormatPattern (#6800)
## Summary

This PR fixes the duplicate-parenthesis problem that's visible in the
tests from https://github.com/astral-sh/ruff/pull/6799. The issue is
that we might have parentheses around the entire match-case pattern,
like in `(1)` here:

```python
match foo:
    case (1):
        y = 0
```

In this case, the inner expression (`1`) will _think_ it's
parenthesized, but we'll _also_ detect the parentheses at the case level
-- so they get rendered by the case, then again by the expression.
Instead, if we detect parentheses at the case level, we can force-off
the parentheses for the pattern using a design similar to the way we
handle parentheses on expressions.

Closes https://github.com/astral-sh/ruff/issues/6753.

## Test Plan

`cargo test`
2023-08-25 03:45:49 +00:00
Charlie Marsh
281ce56dc1
Make isort's detect-same-package behavior configurable (#6833)
## Summary

Our first-party import detection uses a heuristic that doesn't exist in
isort: if an import appears to be from within the same package as the
containing file, we mark it as first-party. For example, if you have a
directory `./foo/__init__.py`, and you import `from foo import bar` in
`./foo/baz.py`, we'll mark that as first-party. (See:
https://github.com/astral-sh/ruff/pull/1266.)

This is often unnecessary, and arguably should be removed (though it
does have some important use-cases that are otherwise unserved -- I
believe Dagster uses it to ensure that all packages mark imports from
within the same package as first-party, but not imports _across_
different first-party packages)... but it does exist, and it does help
in cases in which the `src` field is not properly configured.

This PR adds an option to turn off this behavior:

```toml
[tool.ruff.isort]
detect-same-package = false
```

This is being introduced to help codebases migrating over from isort
that may want more consistent behavior with their current sorting.

## Test Plan

`cargo test`
2023-08-24 14:09:26 -04:00
Zanie Blue
3bd199cdf6
Update function-call-in-argument-default (B008) to ignore arguments with immutable annotations (#6784)
Extends #6781 
Part of https://github.com/astral-sh/ruff/issues/3762
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->
Allows calls in argument defaults if the argument is annotated as an
immutable type to avoid false positives.

## Test Plan

<!-- How was it tested? -->

Snapshots
2023-08-24 11:12:59 -05:00
Harutaka Kawamura
948cd29b23
Skip serializing cell ID if it's None (#6851)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

<!-- What's the purpose of the change? What does it do, and why? -->

Fix #6834 

## Test Plan

<!-- How was it tested? -->

Need tests?

---------

Co-authored-by: Dhruv Manilawala <dhruvmanila@gmail.com>
2023-08-24 13:20:25 +00:00
Micha Reiser
1e7d1968b1
Printer: Slice based queue and stack (#6819) 2023-08-24 14:49:27 +02:00
Micha Reiser
8b46b71038
Fix parenthesizing of implicit strings (#6852) 2023-08-24 12:31:02 +00:00
Micha Reiser
1cd7790a8a
Use BestFits for non-fluent attribute chains (#6817) 2023-08-24 14:09:25 +02:00
konsti
d376cb4c2a
Improve formatter contributor docs (#6776)
The docs were out of date, and the new version incorporates some
feedback.

I tried to keep the language concise and the information ordered by how
early you need it, so people can get the relevant information quickly
before jumping into the code.

I did some minor format_dev changes for consistency in the docs.

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-24 10:45:08 +00:00
Micha Reiser
04a9a8dd03
Maybe parenthesize long constants and names (#6816) 2023-08-24 09:47:57 +00:00
Micha Reiser
e4c13846e3
Fix Printer group modes (#6815) 2023-08-24 08:42:59 +02:00
Harutaka Kawamura
205d234856
Format PatternMatchStar (#6653) 2023-08-24 01:58:05 +00:00
Charlie Marsh
4889b84338
Document logger-objects setting in flake8-logging-format rules (#6832)
Closes https://github.com/astral-sh/ruff/issues/6764.
2023-08-24 00:18:51 +00:00
Charlie Marsh
847432cacf
Avoid attempting to fix PT018 in multi-statement lines (#6829)
## Summary

These fixes will _always_ fail, so we should avoid trying to construct
them in the first place.

Closes https://github.com/astral-sh/ruff/issues/6812.
2023-08-23 19:09:34 -04:00
Charlie Marsh
9b6e008cf1
Remove remaining usages of try_set_fix (#6828)
There are just a few remaining usages of this deprecated method.
2023-08-23 22:36:09 +00:00
Charlie Marsh
39c6665ff9
Remove remaining usage of set_fix_from_edit (#6827)
This method is deprecated; we have one last usage, so removing it.
2023-08-23 18:25:39 -04:00
Zanie Blue
0688883404
Fix uncessary-coding-comment fix when there's leading content (#6775)
Closes https://github.com/astral-sh/ruff/issues/6756

Including whitespace, code, and continuations.
2023-08-23 12:00:06 -05:00
Zanie Blue
3bb638875f
Fix native-literals handling of int literal with attribute access (#6792)
Closes https://github.com/astral-sh/ruff/issues/6788 by special casing
integer literals with attribute access — either retaining parenthesis
for literals with values (e.g. `int(7).denominator` to
`(7).denominator)` or leaving calls without values (e.g.
`int().denominator`) unchanged.
2023-08-23 11:22:05 -05:00
Charlie Marsh
26e63ab137
Remove lexing from flake8-pytest-style (#6795)
## Summary

Another drive-by change to remove unnecessary custom lexing. We just
need to know the parenthesized range, so we can use...
`parenthesized_range`. I've also updated `parenthesized_range` to
support nested parentheses.

## Test Plan

`cargo test`
2023-08-23 15:54:11 +00:00
Zanie Blue
417a1d0717
Update mutable-argument-default (B006) to use extend-immutable-calls when determining if annotations are immutable (#6781)
Part of https://github.com/astral-sh/ruff/issues/3762
2023-08-23 15:44:35 +00:00
Micha Reiser
34b2ae73b4
Extend BestFitting with mode (#6814) 2023-08-23 17:23:45 +02:00
Charlie Marsh
71c25e4f9d
Implement FormatPatternMatchValue (#6799)
## Summary

This is effectively #6608, but with additional tests.

We aren't properly handling parenthesized patterns, but that needs to be
dealt with separately as it's somewhat involved.

Closes #6555
2023-08-23 14:01:14 +00:00
Micha Reiser
4bdd99f882
Fix: Re-add missing node start positions (#6780) 2023-08-23 09:59:36 +02:00
Charlie Marsh
1e6d1182bf
Improve comment handling around PatternMatchAs (#6797)
## Summary

Follows up on
https://github.com/astral-sh/ruff/pull/6652#discussion_r1300871033 with
some modifications to the `PatternMatchAs` comment handling.
Specifically, any comments between the `as` and the end are now
formatted as dangling, and we now insert some newlines in the
appropriate places.

## Test Plan

`cargo test`
2023-08-23 04:48:20 +00:00
Charlie Marsh
d08f697a04
Remove lexing for colon-matching use cases (#6803)
It's much simpler to just search ahead for the first colon.
2023-08-23 04:44:51 +00:00
Charlie Marsh
4bc5eddf91
Handle open-parenthesis comments on match case (#6798)
## Summary

Ensures that we retain the open-parenthesis comment in cases like:
```python
match pattern_comments:
    case (  # leading
        only_leading
    ):
        ...
```

Previously, this was treated as a leading comment on `only_leading`.

## Test Plan

`cargo test`
2023-08-23 00:40:18 -04:00
Charlie Marsh
5f5de52aba
Confine repeated-equality-comparison-target to names and attributes (#6802)
Empirically, Pylint does this, so seems reasonable to follow.
2023-08-23 03:56:37 +00:00
Tom Kuson
1cb1bd731c
Extend repeated-equality-comparison-target to check for mixed orderings and Yoda conditions. (#6691) 2023-08-23 03:45:30 +00:00
Dhruv Manilawala
db2e548f4f
Simplify ANN204 autofix to use Parameters range (#6793)
## Summary

This PR fixes the bug where the decorator parentheses weren't being considered
when computing the autofix for `ANN204`. The existing logic would only look
for balanced parentheses and not multiple pairs of parentheses.

The solution is to remove the logic to generate the autofix and use the
`Parameters` end range directly which includes the parentheses as well.

## Test Plan

Add test case for `ANN204` with decorator being called

fixes: #6790
2023-08-23 09:05:46 +05:30
Harutaka Kawamura
94f5f18ddb
Format PatternMatchSequence (#6676) 2023-08-23 00:44:33 +00:00
Luc Khai Hai
c34a342ab4
Format PatternMatchAs (#6652)
## Summary

Add formatting for `PatternMatchAs`.

This closes #6641.

## Test Plan

Add tests for comments.
2023-08-22 23:58:15 +00:00
Charlie Marsh
42ff833d00
Remove comment lexing from isort (#6794)
## Summary

No need to lex to find comments -- we already know their locations via
`Indexer`.
2023-08-22 21:26:38 +00:00
Konrad Listwan-Ciesielski
e1f4438498
Add docs for DTZ007 (#6757) 2023-08-22 21:12:50 +00:00
Charlie Marsh
1acdec3e29
Add a note on __future__ imports and keep-runtime-typing to pyupgrade rules (#6746)
Closes https://github.com/astral-sh/ruff/issues/6740.
2023-08-22 17:04:00 -04:00
Charlie Marsh
ca2bb20063
Fallback to end-of-file if ends in trailing continuation (#6789)
## Summary

Given:

```python
def end_of_file():
    if False:
        return 1
    x = 2 \

```

Then when searching for the end of the `x = 2` statement, we'd reach a
panic as we'd hit the last line (`\\`) and abort, since the universal
iterator doesn't return trailing newlines. Instead, we should just use
the end of the file as the fallback.

Closes https://github.com/astral-sh/ruff/issues/6787.

## Test Plan

`cargo test`
2023-08-22 15:12:26 -04:00
Dhruv Manilawala
2e00983762
Avoid C417 for lambda with default and variadic parameters (#6752)
## Summary

Avoid `C417` for `lambda` with default and variadic parameters.

## Test Plan

`cargo test` and checking if it generates any autofix errors as test
cases
for `lambda` with default parameters already exists.

fixes: #6715
2023-08-23 00:38:08 +05:30
Dhruv Manilawala
fb7caf43c8
Update lexer tests to use snapshots (#6658)
## Summary

This PR updates the lexer tests to use the snapshot testing framework.
It also
makes the following changes:
* Remove the use of macros in the lexer tests
* Use `test_case` for EOL tests

## Test Plan

```
cargo test --package ruff_python_parser --lib --all-features -- lexer::tests --no-capture
```
2023-08-22 18:23:19 +00:00
Charlie Marsh
214eb707a6
Parenthesize expressions prior to LibCST parsing (#6742)
<!--
Thank you for contributing to Ruff! To help us out with reviewing,
please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR adds a utility for transforming expressions via LibCST that
automatically wraps the expression in parentheses, applies a
user-provided transformation, then strips the parentheses from the
generated code. LibCST can't parse arbitrary expression ranges, since
some expressions may require parenthesization in order to be parsed
properly. For example:

```python
option = (
    '{name}={value}'
    .format(nam=name, value=value)
)
```

In this case, the expression range is:

```python
'{name}={value}'
    .format(nam=name, value=value)
```

Which isn't valid on its own. So, instead, we add "fake" parentheses
around the expression.

We were already doing this in a few places, so this is mostly
formalizing and DRYing up that pattern.

Closes https://github.com/astral-sh/ruff/issues/6720.
2023-08-22 17:45:05 +00:00
Zanie Blue
5c1f7fd5dd
Add networkx to conventional aliases (#6778)
Closes https://github.com/astral-sh/ruff/issues/6763
2023-08-22 11:49:04 -05:00
Charlie Marsh
cc278c24e2
Allow up to two empty lines after top-level imports (#6777)
## Summary

For imports, we enforce that there's _at least_ one empty line after an
import (assuming the next statement is _not_ an import), but allow up to
two at the module level.

Closes https://github.com/astral-sh/ruff/issues/6760.

## Test Plan

`cargo test`
2023-08-22 12:27:40 -04:00
Charlie Marsh
558b56f8a8
Avoid fixing D200 for docstrings that end in escapes (#6779)
Appease the fuzzers! Closes
https://github.com/astral-sh/ruff/issues/6755.
2023-08-22 16:25:37 +00:00
Charlie Marsh
749da6589a
Fix isolation groups for unused imports (#6774)
## Summary

The isolation group for unused imports was relying on
`checker.semantic().current_statement()`, which isn't valid for that
rule, since it runs over the _scope_, not the statement. Instead, we
need to lookup the isolation group based on the `NodeId` of the
statement.

Our tests didn't catch this, because we mostly have cases that look like
this:

```python
if TYPE_CHECKING:
    import shelve
    import importlib
```

In this case, the two fixes to remove the two unused imports are
considered overlapping (since we delete the _full_ line, and the two
_full_ lines touch, and we consider exactly-adjacent fixes to be
overlapping), and so they don't run in a single pass due to the
non-overlapping-fixes requirement. That is: the isolation groups aren't
required for this case. They are, however, required for cases like:

```python
if TYPE_CHECKING:
    import shelve

    import importlib
```

...where the fixes don't overlap.

Closes https://github.com/astral-sh/ruff/issues/6758.

## Test Plan

`cargo test`
2023-08-22 11:55:27 -04:00
Charlie Marsh
d2eace3377
Prefer range_* edit methods (#6751) 2023-08-22 15:46:04 +00:00
Micha Reiser
ccac9681e1
Preserve yield parentheses (#6766) 2023-08-22 10:27:20 +00:00
Micha Reiser
b52cc84df6
Omit tuple parentheses in for statements except when absolutely necessary (#6765) 2023-08-22 12:18:59 +02:00
Micha Reiser
fec6fc2fab
Preserve empty lines between try clause headers (#6759) 2023-08-22 11:50:28 +02:00