Treat empty-line separated comments as trailing statement comments (#6999)

## Summary

This PR modifies our between-statement comment handling such that
comments that are not separated by a statement by any newlines continue
to be treated as leading comments on the statement, but comments that
_are_ separated are instead formatted as trailing comments on the
preceding statement.

See, e.g., the originating snippet:

```python
DEFAULT_TEMPLATE = "flatpages/default.html"

# This view is called from FlatpageFallbackMiddleware.process_response
# when a 404 is raised, which often means CsrfViewMiddleware.process_view
# has not been called even if CsrfViewMiddleware is installed. So we need
# to use @csrf_protect, in case the template needs {% csrf_token %}.
# However, we can't just wrap this view; if no matching flatpage exists,
# or a redirect is required for authentication, the 404 needs to be returned
# without any CSRF checks. Therefore, we only
# CSRF protect the internal implementation.


def flatpage(request, url):
    pass
```

Here, we need to ensure that the `def flatpage` is precede by two empty
lines. However, we want those two empty lines to be enforced from the
_end_ of the comment block, _unless_ the comments are directly atop the
`def flatpage`.

I played with this a bit, and I think the simplest conceptual model and
implementation is to instead treat those as trailing comments on the
preceding node. The main difficulty with this approach is that, in order
to be fully compatible with Black, we'd sometimes need to insert
newlines _between_ the preceding node and its trailing comments. See,
e.g.:

```python
def func():
    ...
# comment

x = 1
```

In this case, we'd need to insert two blank lines between `def func():
...` and `# comment`, but `# comment` is trailing comment on `def
func(): ...`. So, we'd need to take this case into account in the
various nodes that _require_ newlines after them: functions, classes,
and imports. After some discussion, we've opted _not_ to support this,
and just treat these as trailing comments -- so we won't insert newlines
there. This means our handling is still identical to Black's on
Black-formatted code, but avoids moving such trailing comments on
unformatted code.

I dislike that the empty handling is so complex, and that it's split
between so many different nodes, but this is really tricky. Continuing
to treat these as leading comments is very difficult too, since we'd
need to do similar tricks for the leading comment handling in those
nodes, and influencing leading comments is even harder, since they're
all formatted _before_ the node itself.

Closes https://github.com/astral-sh/ruff/issues/6761.

## Test Plan

`cargo test`

Surprisingly, it doesn't change the similarity at all (apart from a
0.00001 change in CPython), but I manually confirmed that it did fix the
originating issue in Django.

Before:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76082          |
| django       | 0.99921          |
| transformers | 0.99854          |
| twine        | 0.99982          |
| typeshed     | 0.99953          |
| warehouse    | 0.99648          |
| zulip        | 0.99928          |


After:

| project      | similarity index |
|--------------|------------------|
| cpython      | 0.76081          |
| django       | 0.99921          |
| transformers | 0.99854          |
| twine        | 0.99982          |
| typeshed     | 0.99953          |
| warehouse    | 0.99648          |
| zulip        | 0.99928          |
This commit is contained in:
Charlie Marsh 2023-08-31 21:55:05 +01:00 committed by GitHub
parent 51d69b448c
commit 376d3caf47
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
21 changed files with 1060 additions and 695 deletions

View file

@ -1,55 +0,0 @@
def test():
# fmt: off
a_very_small_indent
(
not_fixed
)
if True:
pass
more
# fmt: on
formatted
def test():
a_small_indent
# fmt: off
# fix under-indented comments
(or_the_inner_expression +
expressions
)
if True:
pass
# fmt: on
# fmt: off
def test():
pass
# It is necessary to indent comments because the following fmt: on comment because it otherwise becomes a trailing comment
# of the `test` function if the "proper" indentation is larger than 2 spaces.
# fmt: on
disabled + formatting;
# fmt: on
formatted;
def test():
pass
# fmt: off
"""A multiline strings
that should not get formatted"""
"A single quoted multiline \
string"
disabled + formatting;
# fmt: on
formatted;

View file

@ -0,0 +1,36 @@
def func():
pass
# fmt: off
x = 1
# fmt: on
# fmt: off
def func():
pass
# fmt: on
x = 1
# fmt: off
def func():
pass
# fmt: on
def func():
pass
# fmt: off
def func():
pass
# fmt: off
def func():
pass
# fmt: on
def func():
pass
# fmt: on
def func():
pass

View file

@ -0,0 +1,161 @@
###
# Blank lines around functions
###
x = 1
# comment
def f():
pass
if True:
x = 1
# comment
def f():
pass
x = 1
# comment
def f():
pass
x = 1
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
# comment
def f():
pass
# comment
def f():
pass
# comment
###
# Blank lines around imports.
###
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x # comment
# comment
import y
def f(): pass # comment
# comment
x = 1
def f():
pass
# comment
x = 1

View file

@ -1,11 +1,11 @@
use std::borrow::Cow;
use unicode_width::UnicodeWidthChar;
use ruff_text_size::{Ranged, TextLen, TextRange};
use unicode_width::UnicodeWidthChar;
use ruff_formatter::{format_args, write, FormatError, FormatOptions, SourceCode};
use ruff_python_ast::node::{AnyNodeRef, AstNode};
use ruff_python_trivia::{lines_after, lines_after_ignoring_trivia, lines_before};
use ruff_text_size::{Ranged, TextLen, TextRange};
use crate::comments::{CommentLinePosition, SourceComment};
use crate::context::NodeLevel;
@ -299,10 +299,10 @@ impl Format<PyFormatContext<'_>> for FormatComment<'_> {
}
}
// Helper that inserts the appropriate number of empty lines before a comment, depending on the node level.
// Top level: Up to two empty lines
// parenthesized: A single empty line
// other: Up to a single empty line
/// Helper that inserts the appropriate number of empty lines before a comment, depending on the node level:
/// - Top-level: Up to two empty lines.
/// - Parenthesized: A single empty line.
/// - Otherwise: Up to a single empty line.
pub(crate) const fn empty_lines(lines: u32) -> FormatEmptyLines {
FormatEmptyLines { lines }
}
@ -475,3 +475,45 @@ fn normalize_comment<'a>(
Ok(Cow::Owned(std::format!("# {}", content.trim_start())))
}
/// Format the empty lines between a node and its trailing comments.
///
/// For example, given:
/// ```python
/// def func():
/// ...
/// # comment
/// ```
///
/// This builder will insert two empty lines before the comment.
/// ```
pub(crate) const fn empty_lines_before_trailing_comments(
comments: &[SourceComment],
expected: u32,
) -> FormatEmptyLinesBeforeTrailingComments {
FormatEmptyLinesBeforeTrailingComments { comments, expected }
}
#[derive(Copy, Clone, Debug)]
pub(crate) struct FormatEmptyLinesBeforeTrailingComments<'a> {
/// The trailing comments of the node.
comments: &'a [SourceComment],
/// The expected number of empty lines before the trailing comments.
expected: u32,
}
impl Format<PyFormatContext<'_>> for FormatEmptyLinesBeforeTrailingComments<'_> {
fn fmt(&self, f: &mut Formatter<PyFormatContext>) -> FormatResult<()> {
if let Some(comment) = self
.comments
.iter()
.find(|comment| comment.line_position().is_own_line())
{
let actual = lines_before(comment.start(), f.context().source()).saturating_sub(1);
for _ in actual..self.expected {
write!(f, [empty_line()])?;
}
}
Ok(())
}
}

View file

@ -425,7 +425,7 @@ fn handle_own_line_comment_around_body<'a>(
return CommentPlacement::Default(comment);
};
// If there's any non-trivia token between the preceding node and the comment, than it means
// If there's any non-trivia token between the preceding node and the comment, then it means
// we're past the case of the alternate branch, defer to the default rules
// ```python
// if a:
@ -446,11 +446,78 @@ fn handle_own_line_comment_around_body<'a>(
}
// Check if we're between bodies and should attach to the following body.
handle_own_line_comment_between_branches(comment, preceding, locator).or_else(|comment| {
// Otherwise, there's no following branch or the indentation is too deep, so attach to the
// recursively last statement in the preceding body with the matching indentation.
handle_own_line_comment_after_branch(comment, preceding, locator)
})
handle_own_line_comment_between_branches(comment, preceding, locator)
.or_else(|comment| {
// Otherwise, there's no following branch or the indentation is too deep, so attach to the
// recursively last statement in the preceding body with the matching indentation.
handle_own_line_comment_after_branch(comment, preceding, locator)
})
.or_else(|comment| handle_own_line_comment_between_statements(comment, locator))
}
/// Handles own-line comments between statements. If an own-line comment is between two statements,
/// it's treated as a leading comment of the following statement _if_ there are no empty lines
/// separating the comment and the statement; otherwise, it's treated as a trailing comment of the
/// preceding statement.
///
/// For example, this comment would be a trailing comment of `x = 1`:
/// ```python
/// x = 1
/// # comment
///
/// y = 2
/// ```
///
/// However, this comment would be a leading comment of `y = 2`:
/// ```python
/// x = 1
///
/// # comment
/// y = 2
/// ```
fn handle_own_line_comment_between_statements<'a>(
comment: DecoratedComment<'a>,
locator: &Locator,
) -> CommentPlacement<'a> {
let Some(preceding) = comment.preceding_node() else {
return CommentPlacement::Default(comment);
};
let Some(following) = comment.following_node() else {
return CommentPlacement::Default(comment);
};
// We're looking for comments between two statements, like:
// ```python
// x = 1
// # comment
// y = 2
// ```
if !preceding.is_statement() || !following.is_statement() {
return CommentPlacement::Default(comment);
}
// If the comment is directly attached to the following statement; make it a leading
// comment:
// ```python
// x = 1
//
// # leading comment
// y = 2
// ```
//
// Otherwise, if there's at least one empty line, make it a trailing comment:
// ```python
// x = 1
// # trailing comment
//
// y = 2
// ```
if max_empty_lines(locator.slice(TextRange::new(comment.end(), following.start()))) == 0 {
CommentPlacement::leading(following, comment)
} else {
CommentPlacement::trailing(preceding, comment)
}
}
/// Handles own line comments between two branches of a node.
@ -1837,6 +1904,7 @@ fn max_empty_lines(contents: &str) -> u32 {
}
}
max_new_lines = newlines.max(max_new_lines);
max_new_lines.saturating_sub(1)
}

View file

@ -16,7 +16,13 @@ expression: comments.debug(test_case.source_code)
},
],
"dangling": [],
"trailing": [],
"trailing": [
SourceComment {
text: "# own line comment",
position: OwnLine,
formatted: false,
},
],
},
Node {
kind: StmtIf,
@ -48,19 +54,4 @@ expression: comments.debug(test_case.source_code)
"dangling": [],
"trailing": [],
},
Node {
kind: StmtExpr,
range: 234..246,
source: `test(10, 20)`,
}: {
"leading": [
SourceComment {
text: "# own line comment",
position: OwnLine,
formatted: false,
},
],
"dangling": [],
"trailing": [],
},
}

View file

@ -3,6 +3,21 @@ source: crates/ruff_python_formatter/src/comments/mod.rs
expression: comments.debug(test_case.source_code)
---
{
Node {
kind: StmtMatch,
range: 27..550,
source: `match pt:⏎`,
}: {
"leading": [],
"dangling": [],
"trailing": [
SourceComment {
text: "# After match comment",
position: OwnLine,
formatted: false,
},
],
},
Node {
kind: MatchCase,
range: 84..132,
@ -108,19 +123,4 @@ expression: comments.debug(test_case.source_code)
},
],
},
Node {
kind: StmtExpr,
range: 656..670,
source: `print("other")`,
}: {
"leading": [
SourceComment {
text: "# After match comment",
position: OwnLine,
formatted: false,
},
],
"dangling": [],
"trailing": [],
},
}

View file

@ -3,7 +3,9 @@ use ruff_python_ast::{Decorator, StmtClassDef};
use ruff_python_trivia::lines_after_ignoring_trivia;
use ruff_text_size::Ranged;
use crate::comments::format::empty_lines_before_trailing_comments;
use crate::comments::{leading_comments, trailing_comments, SourceComment};
use crate::context::NodeLevel;
use crate::prelude::*;
use crate::statement::clause::{clause_body, clause_header, ClauseHeader};
use crate::statement::suite::SuiteKind;
@ -108,7 +110,33 @@ impl FormatNodeRule<StmtClassDef> for FormatStmtClassDef {
),
clause_body(body, trailing_definition_comments).with_kind(SuiteKind::Class),
]
)?;
// If the class contains trailing comments, insert newlines before them.
// For example, given:
// ```python
// class Class:
// ...
// # comment
// ```
//
// At the top-level, reformat as:
// ```python
// class Class:
// ...
//
//
// # comment
// ```
empty_lines_before_trailing_comments(
comments.trailing(item),
if f.context().node_level() == NodeLevel::TopLevel {
2
} else {
1
},
)
.fmt(f)
}
fn fmt_dangling_comments(

View file

@ -1,9 +1,11 @@
use crate::comments::format::empty_lines_before_trailing_comments;
use ruff_formatter::write;
use ruff_python_ast::{Parameters, StmtFunctionDef};
use ruff_python_trivia::{SimpleTokenKind, SimpleTokenizer};
use ruff_text_size::Ranged;
use crate::comments::SourceComment;
use crate::context::NodeLevel;
use crate::expression::maybe_parenthesize_expression;
use crate::expression::parentheses::{Parentheses, Parenthesize};
use crate::prelude::*;
@ -144,7 +146,33 @@ impl FormatNodeRule<StmtFunctionDef> for FormatStmtFunctionDef {
),
clause_body(body, trailing_definition_comments).with_kind(SuiteKind::Function),
]
)?;
// If the function contains trailing comments, insert newlines before them.
// For example, given:
// ```python
// def func():
// ...
// # comment
// ```
//
// At the top-level, reformat as:
// ```python
// def func():
// ...
//
//
// # comment
// ```
empty_lines_before_trailing_comments(
comments.trailing(item),
if f.context().node_level() == NodeLevel::TopLevel {
2
} else {
1
},
)
.fmt(f)
}
fn fmt_dangling_comments(

View file

@ -2,7 +2,7 @@ use ruff_formatter::{write, FormatOwnedWithRule, FormatRefWithRule, FormatRuleWi
use ruff_python_ast::helpers::is_compound_statement;
use ruff_python_ast::node::AnyNodeRef;
use ruff_python_ast::{self as ast, Constant, Expr, ExprConstant, Stmt, Suite};
use ruff_python_trivia::{lines_after_ignoring_trivia, lines_before};
use ruff_python_trivia::{lines_after, lines_after_ignoring_trivia, lines_before};
use ruff_text_size::{Ranged, TextRange};
use crate::comments::{leading_comments, trailing_comments, Comments};
@ -143,7 +143,11 @@ impl FormatRule<Suite, PyFormatContext<'_>> for FormatSuite {
};
while let Some(following) = iter.next() {
if is_class_or_function_definition(preceding)
// Add empty lines before and after a function or class definition. If the preceding
// node is a function or class, and contains trailing comments, then the statement
// itself will add the requisite empty lines when formatting its comments.
if (is_class_or_function_definition(preceding)
&& !comments.has_trailing_own_line(preceding))
|| is_class_or_function_definition(following)
{
match self.kind {
@ -191,9 +195,13 @@ impl FormatRule<Suite, PyFormatContext<'_>> for FormatSuite {
empty_line().fmt(f)?;
}
}
} else if is_import_definition(preceding) && !is_import_definition(following) {
} else if is_import_definition(preceding)
&& (!is_import_definition(following) || comments.has_leading(following))
{
// Enforce _at least_ one empty line after an import statement (but allow up to
// two at the top-level).
// two at the top-level). In this context, "after an import statement" means that
// that the previous node is an import, and the following node is an import _or_ has
// a leading comment.
match self.kind {
SuiteKind::TopLevel => {
match lines_after_ignoring_trivia(preceding.end(), source) {
@ -274,16 +282,21 @@ impl FormatRule<Suite, PyFormatContext<'_>> for FormatSuite {
// it then counts the lines between the statement and the trailing comment, which is
// always 0. This is why it skips any trailing trivia (trivia that's on the same line)
// and counts the lines after.
lines_after_ignoring_trivia(offset, source)
lines_after(offset, source)
};
let end = comments
.trailing(preceding)
.last()
.map_or(preceding.end(), |comment| comment.slice().end());
match node_level {
NodeLevel::TopLevel => match count_lines(preceding.end()) {
NodeLevel::TopLevel => match count_lines(end) {
0 | 1 => hard_line_break().fmt(f)?,
2 => empty_line().fmt(f)?,
_ => write!(f, [empty_line(), empty_line()])?,
},
NodeLevel::CompoundStatement => match count_lines(preceding.end()) {
NodeLevel::CompoundStatement => match count_lines(end) {
0 | 1 => hard_line_break().fmt(f)?,
_ => empty_line().fmt(f)?,
},

View file

@ -162,7 +162,7 @@ def f():
```diff
--- Black
+++ Ruff
@@ -1,29 +1,182 @@
@@ -1,29 +1,205 @@
+# This file doesn't use the standard decomposition.
+# Decorator syntax test cases are separated by double # comments.
+# Those before the 'output' comment are valid under the old syntax.
@ -179,6 +179,7 @@ def f():
+
+##
+
+
+@decorator()
+def f():
+ ...
@ -186,6 +187,7 @@ def f():
+
+##
+
+
+@decorator(arg)
+def f():
+ ...
@ -193,6 +195,7 @@ def f():
+
+##
+
+
+@decorator(kwarg=0)
+def f():
+ ...
@ -200,49 +203,50 @@ def f():
+
+##
+
+
+@decorator(*args)
+def f():
+ ...
+
+
##
-@decorator()()
+##
+
+
+@decorator(**kwargs)
def f():
...
+def f():
+ ...
+
+
+##
+
+
##
-@(decorator)
+@decorator(*args, **kwargs)
def f():
...
+def f():
+ ...
+
+
+##
+
+
##
-@sequence["decorator"]
+@decorator(
+ *args,
+ **kwargs,
+)
def f():
...
+def f():
+ ...
+
+
+##
+
+
##
-@decorator[List[str]]
+@dotted.decorator
def f():
...
+def f():
+ ...
+
+
+##
+
+
##
-@var := decorator
+@dotted.decorator(arg)
+def f():
+ ...
@ -250,43 +254,54 @@ def f():
+
+##
+
+
+@dotted.decorator(kwarg=0)
+def f():
+ ...
+
+
+##
##
-@decorator()()
+
+@dotted.decorator(*args)
+def f():
+ ...
def f():
...
+
+
+##
##
-@(decorator)
+
+@dotted.decorator(**kwargs)
+def f():
+ ...
def f():
...
+
+
+##
##
-@sequence["decorator"]
+
+@dotted.decorator(*args, **kwargs)
+def f():
+ ...
def f():
...
+
+
+##
##
-@decorator[List[str]]
+
+@dotted.decorator(
+ *args,
+ **kwargs,
+)
+def f():
+ ...
def f():
...
+
+
+##
##
-@var := decorator
+
+@double.dotted.decorator
+def f():
@ -295,6 +310,7 @@ def f():
+
+##
+
+
+@double.dotted.decorator(arg)
+def f():
+ ...
@ -302,6 +318,7 @@ def f():
+
+##
+
+
+@double.dotted.decorator(kwarg=0)
+def f():
+ ...
@ -309,6 +326,7 @@ def f():
+
+##
+
+
+@double.dotted.decorator(*args)
+def f():
+ ...
@ -316,6 +334,7 @@ def f():
+
+##
+
+
+@double.dotted.decorator(**kwargs)
+def f():
+ ...
@ -323,6 +342,7 @@ def f():
+
+##
+
+
+@double.dotted.decorator(*args, **kwargs)
+def f():
+ ...
@ -330,6 +350,7 @@ def f():
+
+##
+
+
+@double.dotted.decorator(
+ *args,
+ **kwargs,
@ -340,6 +361,7 @@ def f():
+
+##
+
+
+@_(sequence["decorator"])
+def f():
+ ...
@ -347,6 +369,7 @@ def f():
+
+##
+
+
+@eval("sequence['decorator']")
def f():
...
@ -371,6 +394,7 @@ def f():
##
@decorator()
def f():
...
@ -378,6 +402,7 @@ def f():
##
@decorator(arg)
def f():
...
@ -385,6 +410,7 @@ def f():
##
@decorator(kwarg=0)
def f():
...
@ -392,6 +418,7 @@ def f():
##
@decorator(*args)
def f():
...
@ -399,6 +426,7 @@ def f():
##
@decorator(**kwargs)
def f():
...
@ -406,6 +434,7 @@ def f():
##
@decorator(*args, **kwargs)
def f():
...
@ -413,6 +442,7 @@ def f():
##
@decorator(
*args,
**kwargs,
@ -423,6 +453,7 @@ def f():
##
@dotted.decorator
def f():
...
@ -430,6 +461,7 @@ def f():
##
@dotted.decorator(arg)
def f():
...
@ -437,6 +469,7 @@ def f():
##
@dotted.decorator(kwarg=0)
def f():
...
@ -444,6 +477,7 @@ def f():
##
@dotted.decorator(*args)
def f():
...
@ -451,6 +485,7 @@ def f():
##
@dotted.decorator(**kwargs)
def f():
...
@ -458,6 +493,7 @@ def f():
##
@dotted.decorator(*args, **kwargs)
def f():
...
@ -465,6 +501,7 @@ def f():
##
@dotted.decorator(
*args,
**kwargs,
@ -475,6 +512,7 @@ def f():
##
@double.dotted.decorator
def f():
...
@ -482,6 +520,7 @@ def f():
##
@double.dotted.decorator(arg)
def f():
...
@ -489,6 +528,7 @@ def f():
##
@double.dotted.decorator(kwarg=0)
def f():
...
@ -496,6 +536,7 @@ def f():
##
@double.dotted.decorator(*args)
def f():
...
@ -503,6 +544,7 @@ def f():
##
@double.dotted.decorator(**kwargs)
def f():
...
@ -510,6 +552,7 @@ def f():
##
@double.dotted.decorator(*args, **kwargs)
def f():
...
@ -517,6 +560,7 @@ def f():
##
@double.dotted.decorator(
*args,
**kwargs,
@ -527,6 +571,7 @@ def f():
##
@_(sequence["decorator"])
def f():
...
@ -534,6 +579,7 @@ def f():
##
@eval("sequence['decorator']")
def f():
...

View file

@ -1,304 +0,0 @@
---
source: crates/ruff_python_formatter/tests/fixtures.rs
input_file: crates/ruff_python_formatter/resources/test/fixtures/black/simple_cases/empty_lines.py
---
## Input
```py
"""Docstring."""
# leading comment
def f():
NO = ''
SPACE = ' '
DOUBLESPACE = ' '
t = leaf.type
p = leaf.parent # trailing comment
v = leaf.value
if t in ALWAYS_NO_SPACE:
pass
if t == token.COMMENT: # another trailing comment
return DOUBLESPACE
assert p is not None, f"INTERNAL ERROR: hand-made leaf without parent: {leaf!r}"
prev = leaf.prev_sibling
if not prev:
prevp = preceding_leaf(p)
if not prevp or prevp.type in OPENING_BRACKETS:
return NO
if prevp.type == token.EQUAL:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.argument,
}:
return NO
elif prevp.type == token.DOUBLESTAR:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.dictsetmaker,
}:
return NO
###############################################################################
# SECTION BECAUSE SECTIONS
###############################################################################
def g():
NO = ''
SPACE = ' '
DOUBLESPACE = ' '
t = leaf.type
p = leaf.parent
v = leaf.value
# Comment because comments
if t in ALWAYS_NO_SPACE:
pass
if t == token.COMMENT:
return DOUBLESPACE
# Another comment because more comments
assert p is not None, f'INTERNAL ERROR: hand-made leaf without parent: {leaf!r}'
prev = leaf.prev_sibling
if not prev:
prevp = preceding_leaf(p)
if not prevp or prevp.type in OPENING_BRACKETS:
# Start of the line or a bracketed expression.
# More than one line for the comment.
return NO
if prevp.type == token.EQUAL:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.argument,
}:
return NO
```
## Black Differences
```diff
--- Black
+++ Ruff
@@ -49,7 +49,6 @@
# SECTION BECAUSE SECTIONS
###############################################################################
-
def g():
NO = ""
SPACE = " "
```
## Ruff Output
```py
"""Docstring."""
# leading comment
def f():
NO = ""
SPACE = " "
DOUBLESPACE = " "
t = leaf.type
p = leaf.parent # trailing comment
v = leaf.value
if t in ALWAYS_NO_SPACE:
pass
if t == token.COMMENT: # another trailing comment
return DOUBLESPACE
assert p is not None, f"INTERNAL ERROR: hand-made leaf without parent: {leaf!r}"
prev = leaf.prev_sibling
if not prev:
prevp = preceding_leaf(p)
if not prevp or prevp.type in OPENING_BRACKETS:
return NO
if prevp.type == token.EQUAL:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.argument,
}:
return NO
elif prevp.type == token.DOUBLESTAR:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.dictsetmaker,
}:
return NO
###############################################################################
# SECTION BECAUSE SECTIONS
###############################################################################
def g():
NO = ""
SPACE = " "
DOUBLESPACE = " "
t = leaf.type
p = leaf.parent
v = leaf.value
# Comment because comments
if t in ALWAYS_NO_SPACE:
pass
if t == token.COMMENT:
return DOUBLESPACE
# Another comment because more comments
assert p is not None, f"INTERNAL ERROR: hand-made leaf without parent: {leaf!r}"
prev = leaf.prev_sibling
if not prev:
prevp = preceding_leaf(p)
if not prevp or prevp.type in OPENING_BRACKETS:
# Start of the line or a bracketed expression.
# More than one line for the comment.
return NO
if prevp.type == token.EQUAL:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.argument,
}:
return NO
```
## Black Output
```py
"""Docstring."""
# leading comment
def f():
NO = ""
SPACE = " "
DOUBLESPACE = " "
t = leaf.type
p = leaf.parent # trailing comment
v = leaf.value
if t in ALWAYS_NO_SPACE:
pass
if t == token.COMMENT: # another trailing comment
return DOUBLESPACE
assert p is not None, f"INTERNAL ERROR: hand-made leaf without parent: {leaf!r}"
prev = leaf.prev_sibling
if not prev:
prevp = preceding_leaf(p)
if not prevp or prevp.type in OPENING_BRACKETS:
return NO
if prevp.type == token.EQUAL:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.argument,
}:
return NO
elif prevp.type == token.DOUBLESTAR:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.dictsetmaker,
}:
return NO
###############################################################################
# SECTION BECAUSE SECTIONS
###############################################################################
def g():
NO = ""
SPACE = " "
DOUBLESPACE = " "
t = leaf.type
p = leaf.parent
v = leaf.value
# Comment because comments
if t in ALWAYS_NO_SPACE:
pass
if t == token.COMMENT:
return DOUBLESPACE
# Another comment because more comments
assert p is not None, f"INTERNAL ERROR: hand-made leaf without parent: {leaf!r}"
prev = leaf.prev_sibling
if not prev:
prevp = preceding_leaf(p)
if not prevp or prevp.type in OPENING_BRACKETS:
# Start of the line or a bracketed expression.
# More than one line for the comment.
return NO
if prevp.type == token.EQUAL:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.argument,
}:
return NO
```

View file

@ -198,7 +198,15 @@ d={'a':1,
```diff
--- Black
+++ Ruff
@@ -63,15 +63,15 @@
@@ -5,6 +5,7 @@
from third_party import X, Y, Z
from library import some_connection, some_decorator
+
# fmt: off
from third_party import (X,
Y, Z)
@@ -63,15 +64,15 @@
something = {
# fmt: off
@ -217,7 +225,7 @@ d={'a':1,
# fmt: on
goes + here,
andhere,
@@ -122,8 +122,10 @@
@@ -122,8 +123,10 @@
"""
# fmt: off
@ -229,7 +237,7 @@ d={'a':1,
# fmt: on
pass
@@ -138,7 +140,7 @@
@@ -138,7 +141,7 @@
now . considers . multiple . fmt . directives . within . one . prefix
# fmt: on
# fmt: off
@ -238,7 +246,7 @@ d={'a':1,
# fmt: on
@@ -178,14 +180,18 @@
@@ -178,14 +181,18 @@
$
""",
# fmt: off
@ -271,6 +279,7 @@ import sys
from third_party import X, Y, Z
from library import some_connection, some_decorator
# fmt: off
from third_party import (X,
Y, Z)

View file

@ -110,15 +110,7 @@ elif unformatted:
},
)
@@ -74,7 +73,6 @@
class Factory(t.Protocol):
def this_will_be_formatted(self, **kwargs) -> Named:
...
-
# fmt: on
@@ -82,6 +80,6 @@
@@ -82,6 +81,6 @@
if x:
return x
# fmt: off
@ -206,6 +198,7 @@ class Named(t.Protocol):
class Factory(t.Protocol):
def this_will_be_formatted(self, **kwargs) -> Named:
...
# fmt: on

View file

@ -0,0 +1,93 @@
---
source: crates/ruff_python_formatter/tests/fixtures.rs
input_file: crates/ruff_python_formatter/resources/test/fixtures/black/simple_cases/fmtpass_imports.py
---
## Input
```py
# Regression test for https://github.com/psf/black/issues/3438
import ast
import collections # fmt: skip
import dataclasses
# fmt: off
import os
# fmt: on
import pathlib
import re # fmt: skip
import secrets
# fmt: off
import sys
# fmt: on
import tempfile
import zoneinfo
```
## Black Differences
```diff
--- Black
+++ Ruff
@@ -3,6 +3,7 @@
import ast
import collections # fmt: skip
import dataclasses
+
# fmt: off
import os
# fmt: on
```
## Ruff Output
```py
# Regression test for https://github.com/psf/black/issues/3438
import ast
import collections # fmt: skip
import dataclasses
# fmt: off
import os
# fmt: on
import pathlib
import re # fmt: skip
import secrets
# fmt: off
import sys
# fmt: on
import tempfile
import zoneinfo
```
## Black Output
```py
# Regression test for https://github.com/psf/black/issues/3438
import ast
import collections # fmt: skip
import dataclasses
# fmt: off
import os
# fmt: on
import pathlib
import re # fmt: skip
import secrets
# fmt: off
import sys
# fmt: on
import tempfile
import zoneinfo
```

View file

@ -4,61 +4,6 @@ input_file: crates/ruff_python_formatter/resources/test/fixtures/ruff/fmt_on_off
---
## Input
```py
def test():
# fmt: off
a_very_small_indent
(
not_fixed
)
if True:
pass
more
# fmt: on
formatted
def test():
a_small_indent
# fmt: off
# fix under-indented comments
(or_the_inner_expression +
expressions
)
if True:
pass
# fmt: on
# fmt: off
def test():
pass
# It is necessary to indent comments because the following fmt: on comment because it otherwise becomes a trailing comment
# of the `test` function if the "proper" indentation is larger than 2 spaces.
# fmt: on
disabled + formatting;
# fmt: on
formatted;
def test():
pass
# fmt: off
"""A multiline strings
that should not get formatted"""
"A single quoted multiline \
string"
disabled + formatting;
# fmt: on
formatted;
```
## Outputs
@ -72,63 +17,6 @@ magic-trailing-comma = Respect
```
```py
def test():
# fmt: off
a_very_small_indent
(
not_fixed
)
if True:
pass
more
# fmt: on
formatted
def test():
a_small_indent
# fmt: off
# fix under-indented comments
(or_the_inner_expression +
expressions
)
if True:
pass
# fmt: on
# fmt: off
def test():
pass
# It is necessary to indent comments because the following fmt: on comment because it otherwise becomes a trailing comment
# of the `test` function if the "proper" indentation is larger than 2 spaces.
# fmt: on
disabled + formatting;
# fmt: on
formatted
def test():
pass
# fmt: off
"""A multiline strings
that should not get formatted"""
"A single quoted multiline \
string"
disabled + formatting
# fmt: on
formatted
```
@ -142,63 +30,6 @@ magic-trailing-comma = Respect
```
```py
def test():
# fmt: off
a_very_small_indent
(
not_fixed
)
if True:
pass
more
# fmt: on
formatted
def test():
a_small_indent
# fmt: off
# fix under-indented comments
(or_the_inner_expression +
expressions
)
if True:
pass
# fmt: on
# fmt: off
def test():
pass
# It is necessary to indent comments because the following fmt: on comment because it otherwise becomes a trailing comment
# of the `test` function if the "proper" indentation is larger than 2 spaces.
# fmt: on
disabled + formatting;
# fmt: on
formatted
def test():
pass
# fmt: off
"""A multiline strings
that should not get formatted"""
"A single quoted multiline \
string"
disabled + formatting
# fmt: on
formatted
```
@ -212,63 +43,6 @@ magic-trailing-comma = Respect
```
```py
def test():
# fmt: off
a_very_small_indent
(
not_fixed
)
if True:
pass
more
# fmt: on
formatted
def test():
a_small_indent
# fmt: off
# fix under-indented comments
(or_the_inner_expression +
expressions
)
if True:
pass
# fmt: on
# fmt: off
def test():
pass
# It is necessary to indent comments because the following fmt: on comment because it otherwise becomes a trailing comment
# of the `test` function if the "proper" indentation is larger than 2 spaces.
# fmt: on
disabled + formatting;
# fmt: on
formatted
def test():
pass
# fmt: off
"""A multiline strings
that should not get formatted"""
"A single quoted multiline \
string"
disabled + formatting
# fmt: on
formatted
```

View file

@ -45,6 +45,8 @@ not_fixed
more
else:
other
# fmt: on
```
@ -72,6 +74,8 @@ not_fixed
more
else:
other
# fmt: on
```
@ -99,6 +103,8 @@ not_fixed
more
else:
other
# fmt: on
```

View file

@ -0,0 +1,90 @@
---
source: crates/ruff_python_formatter/tests/fixtures.rs
input_file: crates/ruff_python_formatter/resources/test/fixtures/ruff/fmt_on_off/newlines.py
---
## Input
```py
def func():
pass
# fmt: off
x = 1
# fmt: on
# fmt: off
def func():
pass
# fmt: on
x = 1
# fmt: off
def func():
pass
# fmt: on
def func():
pass
# fmt: off
def func():
pass
# fmt: off
def func():
pass
# fmt: on
def func():
pass
# fmt: on
def func():
pass
```
## Output
```py
def func():
pass
# fmt: off
x = 1
# fmt: on
# fmt: off
def func():
pass
# fmt: on
x = 1
# fmt: off
def func():
pass
# fmt: on
def func():
pass
# fmt: off
def func():
pass
# fmt: off
def func():
pass
# fmt: on
def func():
pass
# fmt: on
def func():
pass
```

View file

@ -0,0 +1,345 @@
---
source: crates/ruff_python_formatter/tests/fixtures.rs
input_file: crates/ruff_python_formatter/resources/test/fixtures/ruff/newlines.py
---
## Input
```py
###
# Blank lines around functions
###
x = 1
# comment
def f():
pass
if True:
x = 1
# comment
def f():
pass
x = 1
# comment
def f():
pass
x = 1
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
# comment
def f():
pass
# comment
def f():
pass
# comment
###
# Blank lines around imports.
###
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x # comment
# comment
import y
def f(): pass # comment
# comment
x = 1
def f():
pass
# comment
x = 1
```
## Output
```py
###
# Blank lines around functions
###
x = 1
# comment
def f():
pass
if True:
x = 1
# comment
def f():
pass
x = 1
# comment
def f():
pass
x = 1
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
x = 1
# comment
# comment
def f():
pass
# comment
def f():
pass
# comment
def f():
pass
# comment
###
# Blank lines around imports.
###
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x
# comment
import y
def f():
import x # comment
# comment
import y
def f():
pass # comment
# comment
x = 1
def f():
pass
# comment
x = 1
```

View file

@ -191,10 +191,9 @@ assert (
# Trailing test value own-line
# Test dangler
), "Some string" # Trailing msg same-line
# Trailing assert
def test():
assert (
{

View file

@ -406,6 +406,7 @@ def test(
### Different function argument wrappings
def single_line(aaaaaaaaaaaaaaaaaaaaaaaaaaaaa, bbbbbbbbbbbbbbb, ccccccccccccccccc):
pass
@ -511,6 +512,7 @@ def type_param_comments[ # trailing bracket comment
# Different type parameter wrappings
def single_line[Aaaaaaaaaaaaaaaaaaaaaaaaaaaaa, Bbbbbbbbbbbbbbb, Ccccccccccccccccc]():
pass