Micha Reiser
192463c2fb
Allow parenthesized content exceed configured line width ( #7490 )
2023-09-20 08:39:25 +02:00
Charlie Marsh
4c4eceee36
Add dangling comment handling for lambda expressions ( #7493 )
...
## Summary
This PR adds dangling comment handling for `lambda` expressions. In
short, comments around the `lambda` and the `:` are all considered
dangling. Comments that come between the `lambda` and the `:` may be
moved after the colon for simplicity (this is an odd position for a
comment anyway), unless they also precede the lambda parameters, in
which case they're formatted before the parameters.
Closes https://github.com/astral-sh/ruff/issues/7470 .
## Test Plan
`cargo test`
No change in similarity.
Before:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
After:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-19 15:23:51 -04:00
Charlie Marsh
e07670ad97
Add dangling comment handling to dictionary key-value pairs ( #7495 )
...
## Summary
This PR fixes a formatting instability by changing the comment handling
around the `:` in a dictionary to mirror that of the `:` in a lambda: we
treat comments around the `:` as dangling, then format them after the
`:`.
Closes https://github.com/astral-sh/ruff/issues/7458 .
## Test Plan
`cargo test`
No change in similarity.
Before:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1631 |
| django | 0.99983 | 2760 | 36 |
| transformers | 0.99956 | 2587 | 404 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99969 | 1437 | 21 |
After:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1631 |
| django | 0.99983 | 2760 | 36 |
| transformers | 0.99956 | 2587 | 404 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99969 | 1437 | 21 |
2023-09-19 19:17:21 +00:00
Micha Reiser
6a4dbd622b
Add optimized best_fit_parenthesize IR ( #7475 )
2023-09-19 06:29:05 +00:00
Charlie Marsh
728539291f
Move FormatExprDict to top of expr_dict.rs ( #7494 )
...
Put the node itself up top, and internal structs down below.
2023-09-18 11:55:18 -04:00
Charlie Marsh
8ab2519717
Respect parentheses for precedence in await ( #7468 )
...
## Summary
We were using `Parenthesize::IfBreaks` universally for `await`, but
dropping parentheses can change the AST due to precedence. It turns out
that Black's rules aren't _exactly_ the same as operator precedence
(e.g., they leave parentheses around `await ([1, 2, 3])`, although they
aren't strictly required).
Closes https://github.com/astral-sh/ruff/issues/7467 .
## Test Plan
`cargo test`
No change in similarity.
Before:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
After:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 398 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-18 09:56:41 -04:00
konsti
c4d85d6fb6
Fix ''' ""''' formatting ( #7485 )
...
## Summary
`''' ""'''` is an edge case that was previously incorrectly formatted as
`""" """""`.
Fixes #7460
## Test Plan
Added regression test
2023-09-18 10:28:15 +00:00
Charlie Marsh
422ff82f4a
Avoid extra parentheses in yield expressions ( #7444 )
...
## Summary
This is equivalent to https://github.com/astral-sh/ruff/pull/7424 , but
for `yield` and `yield from` expressions. Specifically, we want to avoid
adding unnecessary extra parentheses for `yield expr` when `expr` itself
does not require parentheses.
## Test Plan
`cargo test`
No change in any of the similarity metrics.
Before:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
After:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99929 | 648 | 16 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 14:46:56 -04:00
Charlie Marsh
aae02cf275
Fix broken is_expression_parenthesized call from rebase ( #7442 )
2023-09-16 17:22:16 +00:00
Charlie Marsh
22770fb4be
Avoid extra parentheses in await expressions ( #7424 )
...
## Summary
This PR aligns the await parenthesizing with the unary case, which is:
if the value is already parenthesized, avoid parenthesizing; otherwise,
only parenthesize if the _value_ needs parenthesizing.
Closes https://github.com/astral-sh/ruff/issues/7420 .
## Test Plan
`cargo test`
No change in similarity.
Before:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
After:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 13:10:35 -04:00
Charlie Marsh
1880cceac1
Avoid extra parentheses in unary expressions ( #7428 )
...
## Summary
This PR applies a similar fix to unary expressions as in
https://github.com/astral-sh/ruff/pull/7424 . Specifically, we only need
to parenthesize the entire operator if the operand itself doesn't have
parentheses, and requires parentheses.
Closes https://github.com/astral-sh/ruff/issues/7423 .
## Test Plan
`cargo test`
No change in similarity.
Before:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
After:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99982 | 2760 | 37 |
| transformers | 0.99957 | 2587 | 399 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99923 | 648 | 18 |
| zulip | 0.99962 | 1437 | 22 |
2023-09-16 13:07:38 -04:00
Micha Reiser
c907317199
Fix build ( #7437 )
2023-09-16 14:50:36 +00:00
Micha Reiser
916dd5b7fa
fix: Use BestFit layout for subscript ( #7409 )
2023-09-16 16:21:45 +02:00
konsti
2cbe1733c8
Use CommentRanges in backwards lexing ( #7360 )
...
## Summary
The tokenizer was split into a forward and a backwards tokenizer. The
backwards tokenizer uses the same names as the forwards ones (e.g.
`next_token`). The backwards tokenizer gets the comment ranges that we
already built to skip comments.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-09-16 03:21:45 +00:00
Charlie Marsh
cc9e84c144
Format trailing operator comments as dangling ( #7427 )
...
## Summary
Given a trailing operator comment in a unary expression, like:
```python
if (
not # comment
a):
...
```
We were attaching these to the operand (`a`), but formatting them in the
unary operator via special handling. Parents shouldn't format the
comments of their children, so this instead attaches them as dangling
comments on the unary expression. (No intended change in formatting.)
2023-09-15 20:34:09 -04:00
Charlie Marsh
34c1cb7d11
Treat parenthesized power operands as non-simple ( #7371 )
...
Closes https://github.com/astral-sh/ruff/issues/7318 .
2023-09-14 15:36:21 +00:00
Micha Reiser
675c86c175
fix: Group fluent subscript ( #7386 )
2023-09-14 13:04:14 +02:00
Charlie Marsh
11287f944f
Avoid re-parenthesizing call chains whose inner values are parenthesized ( #7373 )
...
## Summary
Given a statement like:
```python
result = (
f(111111111111111111111111111111111111111111111111111111111111111111111111111111111)
+ 1
)()
```
When we go to parenthesize the target of the assignment, we use
`maybe_parenthesize_expression` with `Parenthesize::IfBreaks`. This then
checks if the call on the right-hand side needs to be parenthesized, the
implementation of which looks like:
```rust
impl NeedsParentheses for ExprCall {
fn needs_parentheses(
&self,
_parent: AnyNodeRef,
context: &PyFormatContext,
) -> OptionalParentheses {
if CallChainLayout::from_expression(self.into(), context.source())
== CallChainLayout::Fluent
{
OptionalParentheses::Multiline
} else if context.comments().has_dangling(self) {
OptionalParentheses::Always
} else {
self.func.needs_parentheses(self.into(), context)
}
}
}
```
Checking for `self.func.needs_parentheses(self.into(), context)` is
problematic, since, as in the example above, `self.func` may _already_
be parenthesized -- in which case, we _don't_ want to parenthesize the
entire expression. If we do, we end up with this non-ideal formatting:
```python
result = (
(
f(
111111111111111111111111111111111111111111111111111111111111111111111111111111111
)
+ 1
)()
)
```
This PR modifies the `NeedsParentheses` implementations for call chain
expressions to return `Never` if the inner expression has its own
parentheses, in which case, the formatting implementations for those
expressions will preserve them anyway.
Closes https://github.com/astral-sh/ruff/issues/7370 .
## Test Plan
Zulip improves a bit, everything else is unchanged.
Before:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99981 | 2760 | 40 |
| transformers | 0.99944 | 2587 | 413 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99834 | 648 | 20 |
| zulip | 0.99956 | 1437 | 23 |
After:
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99981 | 2760 | 40 |
| transformers | 0.99944 | 2587 | 413 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99983 | 3496 | 18 |
| warehouse | 0.99834 | 648 | 20 |
| **zulip** | **0.99962** | **1437** | **22** |
2023-09-14 05:05:37 -04:00
Micha Reiser
a65efcf459
fix: Don't omit optional parentheses for subscripts ( #7380 )
2023-09-14 08:43:53 +00:00
Micha Reiser
2d9b39871f
Introduce IndentWidth ( #7301 )
2023-09-13 14:52:24 +02:00
Micha Reiser
08f19226b9
Fix panic when formatting binary expression with two implicit concatenated string operands ( #7287 )
2023-09-12 09:49:51 +02:00
Micha Reiser
1e6df19a35
Bool expression comment placement ( #7269 )
2023-09-12 06:39:57 +00:00
Micha Reiser
0a07a2ca62
Extract string part and normalized string ( #7219 )
2023-09-08 12:56:55 +02:00
Micha Reiser
47a253fb62
Add PreviewMode option to formatter
...
## Summary
This PR adds the `--preview` and `--no-preview` options to the `format` command (hidden) and passes it through to the formatte.
## Test Plan
I added the `dbg(f.options().preview())` statement in `FormatNodeRule::fmt` and verified that the option gets correctly passed to the formatter.
2023-09-08 12:04:28 +02:00
Micha Reiser
41f0aad7b3
Add FString support to binary like formatting
...
## Summary
This is the last part of the string - binary like formatting. It adds support for handling fstrings the same as "regular" strings.
## Test Plan
I added a test for both binary and comparison.
Small improvements across several projects
**This PR**
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99966 | 2760 | 58 |
| **transformers** | 0.99929 | 2587 | 454 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99978 | 3496 | 2173 |
| **warehouse** | 0.99825 | 648 | 22 |
| **zulip** | 0.99950 | 1437 | 27 |
**Base**
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99966 | 2760 | 58 |
| transformers | 0.99928 | 2587 | 454 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99978 | 3496 | 2173 |
| warehouse | 0.99824 | 648 | 22 |
| zulip | 0.99948 | 1437 | 28 |
<!-- How was it tested? -->
2023-09-08 11:48:57 +02:00
qdegraaf
05951dd338
Fix inconsistent expr_lambda formatting ( #6318 )
2023-09-08 09:40:58 +00:00
Micha Reiser
c260762900
Formatter: Implicit concatenation in compare expressions
...
## Summary
This PR implements the logic for breaking implicit concatenated strings before compare expressions by building on top of #7145
The main change is a new `BinaryLike` enum that has the `BinaryExpression` and `CompareExpression` variants. Supporting both variants requires some downstream changes but doesn't introduce any new concepts.
## Test Plan
I added a few more tests. The compatibility improvements are minor but we now perfectly match black on twine 🥳
**PR**
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1632 |
| django | 0.99966 | 2760 | 58 |
| transformers | 0.99928 | 2587 | 454 |
| **twine** | 1.00000 | 33 | 0 | <-- improved
| typeshed | 0.99978 | 3496 | 2173 |
| **warehouse** | 0.99824 | 648 | 22 | <-- improved
| zulip | 0.99948 | 1437 | 28 |
**Base**
| project | similarity index | total files | changed files |
|--------------|------------------:|------------------:|------------------:|
| cpython | 0.76083 | 1789 | 1633 |
| django | 0.99966 | 2760 | 58 |
| transformers | 0.99928 | 2587 | 454 |
| twine | 0.99982 | 33 | 1 |
| typeshed | 0.99978 | 3496 | 2173 |
| warehouse | 0.99823 | 648 | 23 |
| zulip | 0.99948 | 1437 | 28 |
2023-09-08 11:32:20 +02:00
Micha Reiser
a352f2f092
Preserve generator parentheses in single argument call expressions ( #7226 )
2023-09-08 10:53:34 +02:00
Micha Reiser
e376c3ff7e
Split implicit concatenated strings before binary expressions ( #7145 )
2023-09-08 06:51:26 +00:00
Dhruv Manilawala
04f2842e4f
Move ExprConstant::kind to StringConstant::unicode ( #7180 )
2023-09-06 07:39:25 +00:00
Micha Reiser
5f59101811
Memoize text width ( #6552 )
2023-09-06 07:10:13 +00:00
Charlie Marsh
ece30e7c69
Preserve parentheses around partial call chains ( #7109 )
2023-09-04 10:57:04 +01:00
Micha Reiser
c05e4628b1
Introduce Token element ( #7048 )
2023-09-02 10:05:47 +02:00
Micha Reiser
92143afeee
Group binary operators with same precedence only ( #7010 )
2023-08-31 09:19:45 +02:00
Micha Reiser
eb552da8a9
Avoid parenthesizing multiline strings in binary expressions ( #6973 )
2023-08-30 16:03:17 +02:00
Charlie Marsh
e2b2b1759f
Handle keyword comments between = and value ( #6883 )
...
## Summary
This PR adds comment handling for comments between the `=` and the
`value` for keywords, as in the following cases:
```python
func(
x # dangling
= # dangling
# dangling
1,
** # dangling
y
)
```
(Comments after the `**` were already handled in some cases, but I've
unified the handling with the `=` handling.)
Note that, previously, comments between the `**` and its value were
rendered as trailing comments on the value (so they'd appear after `y`).
This struck me as odd since it effectively re-ordered the comment with
respect to its closest AST node (the value). I've made them leading
comments, though I don't know that that's a significant improvement. I
could also imagine us leaving them where they are.
2023-08-30 09:52:51 -04:00
Chris Pryer
a3f4d7745a
Use reserved width to include line suffix measurement ( #6901 )
...
Co-authored-by: Micha Reiser <micha@reiser.io>
2023-08-30 08:07:11 +00:00
Victor Hugo Gomes
31947af6a3
Don't "flatten" nested if expressions when formatting ( #6996 )
2023-08-30 04:11:58 +00:00
Charlie Marsh
b404e54f33
Remove unnecessary Comment#slice calls ( #6997 )
2023-08-30 00:44:11 +00:00
Micha Reiser
715d86dae9
Remove Comprehension priority ( #6947 )
2023-08-29 08:30:15 +02:00
Micha Reiser
adb48692d6
Use optional parentheses for tuples in return statements ( #6875 )
2023-08-29 08:30:05 +02:00
Charlie Marsh
aea7500c1e
Allow Locator#slice to take Ranged ( #6922 )
...
## Summary
As a small quality-of-life improvement, the locator can now slice like
`locator.slice(stmt)` instead of requiring
`locator.slice(stmt.range())`.
## Test Plan
`cargo test`
2023-08-28 11:08:39 -04:00
Micha Reiser
60097bebcd
Handle implicit strings in `can_omit_parentheses ( #6940 )
2023-08-28 12:20:29 +00:00
Charlie Marsh
fc89976c24
Move Ranged into ruff_text_size ( #6919 )
...
## Summary
The motivation here is that this enables us to implement `Ranged` in
crates that don't depend on `ruff_python_ast`.
Largely a mechanical refactor with a lot of regex, Clippy help, and
manual fixups.
## Test Plan
`cargo test`
2023-08-27 14:12:51 -04:00
Micha Reiser
9d77552e18
Add tab width option ( #6848 )
2023-08-26 12:29:58 +02:00
Charlie Marsh
edb9b0c62a
Use the formatter prelude in more files ( #6882 )
...
Removes a bunch of imports that are made redundant by the prelude.
2023-08-25 16:51:07 -04:00
Micha Reiser
29a0c1003b
Use BestFit layout even for attributes with a short name ( #6872 )
2023-08-25 17:47:02 +02:00
David Szotten
1c66bb80b7
fix is_raw_string for multiple prefixes ( #6865 )
...
fix `is_raw_string` in the presence of other prefixes (like `rb"foo"`)
fixes #6864
2023-08-25 09:58:26 +02:00
Charlie Marsh
813d7da7ec
Respect own-line leading comments before parenthesized nodes ( #6820 )
...
## Summary
This PR ensures that if an expression has an own-line leading comment
_before_ its open parentheses, we render it as such.
For example, given:
```python
[ # foo
# bar
( # baz
1
)
]
```
On `main`, we format as:
```python
[ # foo
(
# bar
# baz
1
)
]
```
As of this PR, we format as:
```python
[ # foo
# bar
( # baz
1
)
]
```
## Test Plan
`cargo test`
2023-08-25 00:18:05 -04:00
Charlie Marsh
59e70896c0
Fix formatting of comments between function and arguments ( #6826 )
...
## Summary
We now format comments between a function and its arguments as dangling.
Like with other strange placements, I've biased towards preserving the
existing formatting, rather than attempting to reorder the comments.
Closes https://github.com/astral-sh/ruff/issues/6818 .
## Test Plan
`cargo test`
Before:
| project | similarity index |
|--------------|------------------|
| cpython | 0.76050 |
| django | 0.99820 |
| transformers | 0.99800 |
| twine | 0.99876 |
| typeshed | 0.99953 |
| warehouse | 0.99615 |
| zulip | 0.99729 |
After:
| project | similarity index |
|--------------|------------------|
| cpython | 0.76050 |
| django | 0.99820 |
| transformers | 0.99800 |
| twine | 0.99876 |
| typeshed | 0.99953 |
| warehouse | 0.99615 |
| zulip | 0.99729 |
2023-08-25 04:06:56 +00:00
