mirrors/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-07-24 13:33:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Prefer expanding parenthesized expressions before operands

Browse source 
<!--
Thank you for contributing to Ruff! To help us out with reviewing, please consider the following:

- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->

## Summary

This PR implements Black's behavior where it first splits off parenthesized expressions before splitting before operands to avoid unnecessary parentheses:

```python
# We want 
if a + [ 
	b,
	c
]: 
	pass

# Rather than
if (
    a
    + [b, c]
): 
	pass
```

This is implemented by using the new IR elements introduced in #5596. 

* We give the group wrapping the optional parentheses an ID (`parentheses_id`)
* We use `conditional_group` for the lower priority groups  (all non-parenthesized expressions) with the condition that the `parentheses_id` group breaks (we want to split before operands only if the parentheses are necessary)
* We use `fits_expanded` to wrap all other parenthesized expressions (lists, dicts, sets), to prevent that expanding e.g. a list expands the `parentheses_id` group. We gate the `fits_expand` to only apply if the `parentheses_id` group fits (because we  prefer `a\n+[b, c]` over expanding `[b, c]` if the whole expression gets parenthesized).

We limit using `fits_expanded` and `conditional_group` only to expressions that themselves are not in parentheses (checking the conditions isn't free)

## Test Plan

It increases the Jaccard index for Django from 0.915 to 0.917

## Incompatibilites

There are two incompatibilities left that I'm aware of (there may be more, I didn't go through all snapshot differences). 

### Long string literals
I  commented on the regression. The issue is that a very long string (or any content without a split point) may not fit when only breaking the right side. The formatter than inserts the optional parentheses. But this is kind of useless because the overlong string will still not fit, because there are no new split points. 

I think we should ignore this incompatibility for now


### Expressions on statement level

I don't fully understand the logic behind this yet, but black doesn't break before the operators for the following example even though the expression exceeds the configured line width

```python
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa < bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb > ccccccccccccccccccccccccccccc == ddddddddddddddddddddd
```

But it would if the expression is used inside of a condition. 

What I understand so far is that Black doesn't insert optional parentheses on the expression statement level (and a few other places) and, therefore, only breaks after opening parentheses. I propose to keep this deviation for now to avoid overlong-lines and use the compatibility report to make a decision if we should implement the same behavior.
This commit is contained in:
Micha Reiser 2023-07-11 14:07:39 +02:00 • committed by GitHub
parent d30e9125eb
commit 715250a179
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
26 changed files with 680 additions and 943 deletions
Download patch file Download diff file Expand all files Collapse all files

21
crates/ruff_python_formatter/src/builders.rs
View file

@ -21,12 +21,21 @@ pub(crate) struct OptionalParentheses<'a, 'ast> {
impl<'ast> Format<PyFormatContext<'ast>> for OptionalParentheses<'_, 'ast> {
fn fmt(&self, f: &mut Formatter<PyFormatContext<'ast>>) -> FormatResult<()> {
group(&format_args![
let saved_level = f.context().node_level();
f.context_mut()
.set_node_level(NodeLevel::ParenthesizedExpression);
let result = group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&Arguments::from(&self.inner)),
if_group_breaks(&text(")"))
if_group_breaks(&text(")")),
])
.fmt(f)
.fmt(f);
f.context_mut().set_node_level(saved_level);
result
}
}
@ -113,7 +122,9 @@ impl<'fmt, 'ast, 'buf> JoinNodesBuilder<'fmt, 'ast, 'buf> {
0 | 1 => hard_line_break().fmt(self.fmt),
_ => empty_line().fmt(self.fmt),
},
NodeLevel::Expression => hard_line_break().fmt(self.fmt),
NodeLevel::Expression(_) | NodeLevel::ParenthesizedExpression => {
hard_line_break().fmt(self.fmt)
}
}?;
}
@ -353,7 +364,7 @@ no_leading_newline = 30"#
// Removes all empty lines
#[test]
fn ranged_builder_parenthesized_level() {
let printed = format_ranged(NodeLevel::Expression);
let printed = format_ranged(NodeLevel::Expression(None));
assert_eq!(
&printed,

4
crates/ruff_python_formatter/src/comments/format.rs
View file

@ -318,7 +318,9 @@ impl Format<PyFormatContext<'_>> for FormatEmptyLines {
},
// Remove all whitespace in parenthesized expressions
NodeLevel::Expression => write!(f, [hard_line_break()]),
NodeLevel::Expression(_) | NodeLevel::ParenthesizedExpression => {
write!(f, [hard_line_break()])
}
}
}
}

9
crates/ruff_python_formatter/src/context.rs
View file

@ -1,6 +1,6 @@
use crate::comments::Comments;
use crate::PyFormatOptions;
use ruff_formatter::{FormatContext, SourceCode};
use ruff_formatter::{FormatContext, GroupId, SourceCode};
use ruff_python_ast::source_code::Locator;
use std::fmt::{Debug, Formatter};
@ -78,6 +78,9 @@ pub(crate) enum NodeLevel {
/// (`if`, `while`, `match`, etc.).
CompoundStatement,
/// Formatting nodes that are enclosed in a parenthesized expression.
Expression,
/// The root or any sub-expression.
Expression(Option<GroupId>),
/// Formatting nodes that are enclosed by a parenthesized (any `[]`, `{}` or `()`) expression.
ParenthesizedExpression,
}

216
crates/ruff_python_formatter/src/expression/binary_like.rs
View file

@ -1,216 +0,0 @@
//! This module provides helper utilities to format an expression that has a left side, an operator,
//! and a right side (binary like).
use rustpython_parser::ast::{self, Expr};
use ruff_formatter::{format_args, write};
use crate::expression::parentheses::{is_expression_parenthesized, Parentheses};
use crate::prelude::*;
/// Trait to implement a binary like syntax that has a left operand, an operator, and a right operand.
pub(super) trait FormatBinaryLike<'ast> {
/// The type implementing the formatting of the operator.
type FormatOperator: Format<PyFormatContext<'ast>>;
/// Formats the binary like expression to `f`.
fn fmt_binary(
&self,
parentheses: Option<Parentheses>,
f: &mut PyFormatter<'ast, '_>,
) -> FormatResult<()> {
let left = self.left()?;
let operator = self.operator();
let right = self.right()?;
let layout = if parentheses == Some(Parentheses::Custom) {
self.binary_layout(f.context().contents())
} else {
BinaryLayout::Default
};
match layout {
BinaryLayout::Default => self.fmt_default(f),
BinaryLayout::ExpandLeft => {
let left = left.format().memoized();
let right = right.format().memoized();
write!(
f,
[best_fitting![
// Everything on a single line
format_args![group(&left), space(), operator, space(), right],
// Break the left over multiple lines, keep the right flat
format_args![
group(&left).should_expand(true),
space(),
operator,
space(),
right
],
// The content doesn't fit, indent the content and break before the operator.
format_args![
text("("),
block_indent(&format_args![
left,
hard_line_break(),
operator,
space(),
right
]),
text(")")
]
]
.with_mode(BestFittingMode::AllLines)]
)
}
BinaryLayout::ExpandRight => {
let left_group = f.group_id("BinaryLeft");
write!(
f,
[
// Wrap the left in a group and gives it an id. The printer first breaks the
// right side if `right` contains any line break because the printer breaks
// sequences of groups from right to left.
// Indents the left side if the group breaks.
group(&format_args![
if_group_breaks(&text("(")),
indent_if_group_breaks(
&format_args![
soft_line_break(),
left.format(),
soft_line_break_or_space(),
operator,
space()
],
left_group
)
])
.with_group_id(Some(left_group)),
// Wrap the right in a group and indents its content but only if the left side breaks
group(&indent_if_group_breaks(&right.format(), left_group)),
// If the left side breaks, insert a hard line break to finish the indent and close the open paren.
if_group_breaks(&format_args![hard_line_break(), text(")")])
.with_group_id(Some(left_group))
]
)
}
BinaryLayout::ExpandRightThenLeft => {
// The formatter expands group-sequences from right to left, and expands both if
// there isn't enough space when expanding only one of them.
write!(
f,
[
group(&left.format()),
space(),
operator,
space(),
group(&right.format())
]
)
}
}
}
/// Determines which binary layout to use.
fn binary_layout(&self, source: &str) -> BinaryLayout {
if let (Ok(left), Ok(right)) = (self.left(), self.right()) {
BinaryLayout::from_left_right(left, right, source)
} else {
BinaryLayout::Default
}
}
/// Formats the node according to the default layout.
fn fmt_default(&self, f: &mut PyFormatter<'ast, '_>) -> FormatResult<()>;
/// Returns the left operator
fn left(&self) -> FormatResult<&Expr>;
/// Returns the right operator.
fn right(&self) -> FormatResult<&Expr>;
/// Returns the object that formats the operator.
fn operator(&self) -> Self::FormatOperator;
}
fn can_break_expr(expr: &Expr, source: &str) -> bool {
let can_break = match expr {
Expr::Tuple(ast::ExprTuple {
elts: expressions, ..
})
| Expr::List(ast::ExprList {
elts: expressions, ..
})
| Expr::Set(ast::ExprSet {
elts: expressions, ..
})
| Expr::Dict(ast::ExprDict {
values: expressions,
..
}) => !expressions.is_empty(),
Expr::Call(ast::ExprCall { args, keywords, .. }) => {
!(args.is_empty() && keywords.is_empty())
}
Expr::ListComp(_) | Expr::SetComp(_) | Expr::DictComp(_) | Expr::GeneratorExp(_) => true,
Expr::UnaryOp(ast::ExprUnaryOp { operand, .. }) => can_break_expr(operand.as_ref(), source),
_ => false,
};
can_break || is_expression_parenthesized(expr.into(), source)
}
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub(super) enum BinaryLayout {
/// Put each operand on their own line if either side expands
Default,
/// Try to expand the left to make it fit. Add parentheses if the left or right don't fit.
///
///```python
/// [
/// a,
/// b
/// ] & c
///```
ExpandLeft,
/// Try to expand the right to make it fix. Add parentheses if the left or right don't fit.
///
/// ```python
/// a & [
/// b,
/// c
/// ]
/// ```
ExpandRight,
/// Both the left and right side can be expanded. Try in the following order:
/// * expand the right side
/// * expand the left side
/// * expand both sides
///
/// to make the expression fit
///
/// ```python
/// [
/// a,
/// b
/// ] & [
/// c,
/// d
/// ]
/// ```
ExpandRightThenLeft,
}
impl BinaryLayout {
pub(super) fn from_left_right(left: &Expr, right: &Expr, source: &str) -> BinaryLayout {
match (can_break_expr(left, source), can_break_expr(right, source)) {
(false, false) => BinaryLayout::Default,
(true, false) => BinaryLayout::ExpandLeft,
(false, true) => BinaryLayout::ExpandRight,
(true, true) => BinaryLayout::ExpandRightThenLeft,
}
}
}

55
crates/ruff_python_formatter/src/expression/expr_bin_op.rs
View file

@ -1,8 +1,7 @@
use crate::comments::{trailing_comments, trailing_node_comments, Comments};
use crate::expression::binary_like::{BinaryLayout, FormatBinaryLike};
use crate::expression::parentheses::{
default_expression_needs_parentheses, is_expression_parenthesized, NeedsParentheses,
Parenthesize,
default_expression_needs_parentheses, in_parentheses_only_group, is_expression_parenthesized,
NeedsParentheses, Parenthesize,
};
use crate::expression::Parentheses;
use crate::prelude::*;
@ -31,24 +30,11 @@ impl FormatRuleWithOptions<ExprBinOp, PyFormatContext<'_>> for FormatExprBinOp {
impl FormatNodeRule<ExprBinOp> for FormatExprBinOp {
fn fmt_fields(&self, item: &ExprBinOp, f: &mut PyFormatter) -> FormatResult<()> {
item.fmt_binary(self.parentheses, f)
}
fn fmt_dangling_comments(&self, _node: &ExprBinOp, _f: &mut PyFormatter) -> FormatResult<()> {
// Handled inside of `fmt_fields`
Ok(())
}
}
impl<'ast> FormatBinaryLike<'ast> for ExprBinOp {
type FormatOperator = FormatOwnedWithRule<Operator, FormatOperator, PyFormatContext<'ast>>;
fn fmt_default(&self, f: &mut PyFormatter<'ast, '_>) -> FormatResult<()> {
let comments = f.context().comments().clone();
let format_inner = format_with(|f: &mut PyFormatter| {
let source = f.context().contents();
let binary_chain: SmallVec<[&ExprBinOp; 4]> = iter::successors(Some(self), |parent| {
let binary_chain: SmallVec<[&ExprBinOp; 4]> = iter::successors(Some(item), |parent| {
parent.left.as_bin_op_expr().and_then(|bin_expression| {
if is_expression_parenthesized(bin_expression.as_any_node_ref(), source) {
None
@ -63,7 +49,7 @@ impl<'ast> FormatBinaryLike<'ast> for ExprBinOp {
let left_most = binary_chain.last().unwrap();
// Format the left most expression
group(&left_most.left.format()).fmt(f)?;
in_parentheses_only_group(&left_most.left.format()).fmt(f)?;
// Iterate upwards in the binary expression tree and, for each level, format the operator
// and the right expression.
@ -100,13 +86,13 @@ impl<'ast> FormatBinaryLike<'ast> for ExprBinOp {
space().fmt(f)?;
}
group(&right.format()).fmt(f)?;
in_parentheses_only_group(&right.format()).fmt(f)?;
// It's necessary to format the trailing comments because the code bypasses
// `FormatNodeRule::fmt` for the nested binary expressions.
// Don't call the formatting function for the most outer binary expression because
// these comments have already been formatted.
if current != self {
if current != item {
trailing_node_comments(current).fmt(f)?;
}
}
@ -114,19 +100,12 @@ impl<'ast> FormatBinaryLike<'ast> for ExprBinOp {
Ok(())
});
group(&format_inner).fmt(f)
in_parentheses_only_group(&format_inner).fmt(f)
}
fn left(&self) -> FormatResult<&Expr> {
Ok(&self.left)
}
fn right(&self) -> FormatResult<&Expr> {
Ok(&self.right)
}
fn operator(&self) -> Self::FormatOperator {
self.op.into_format()
fn fmt_dangling_comments(&self, _node: &ExprBinOp, _f: &mut PyFormatter) -> FormatResult<()> {
// Handled inside of `fmt_fields`
Ok(())
}
}
@ -200,18 +179,6 @@ impl NeedsParentheses for ExprBinOp {
source: &str,
comments: &Comments,
) -> Parentheses {
match default_expression_needs_parentheses(self.into(), parenthesize, source, comments) {
Parentheses::Optional => {
if self.binary_layout(source) == BinaryLayout::Default
|| comments.has_leading_comments(self.right.as_ref())
|| comments.has_dangling_comments(self)
{
Parentheses::Optional
} else {
Parentheses::Custom
}
}
parentheses => parentheses,
}
default_expression_needs_parentheses(self.into(), parenthesize, source, comments)
}
}

101
crates/ruff_python_formatter/src/expression/expr_bool_op.rs
View file

@ -1,13 +1,11 @@
use crate::comments::{leading_comments, Comments};
use crate::expression::binary_like::{BinaryLayout, FormatBinaryLike};
use crate::expression::parentheses::{
default_expression_needs_parentheses, NeedsParentheses, Parentheses, Parenthesize,
default_expression_needs_parentheses, in_parentheses_only_group, NeedsParentheses, Parentheses,
Parenthesize,
};
use crate::prelude::*;
use ruff_formatter::{
write, FormatError, FormatOwnedWithRule, FormatRefWithRule, FormatRuleWithOptions,
};
use rustpython_parser::ast::{BoolOp, Expr, ExprBoolOp};
use ruff_formatter::{write, FormatOwnedWithRule, FormatRefWithRule, FormatRuleWithOptions};
use rustpython_parser::ast::{BoolOp, ExprBoolOp};
#[derive(Default)]
pub struct FormatExprBoolOp {
@ -24,64 +22,48 @@ impl FormatRuleWithOptions<ExprBoolOp, PyFormatContext<'_>> for FormatExprBoolOp
impl FormatNodeRule<ExprBoolOp> for FormatExprBoolOp {
fn fmt_fields(&self, item: &ExprBoolOp, f: &mut PyFormatter) -> FormatResult<()> {
item.fmt_binary(self.parentheses, f)
}
}
impl<'ast> FormatBinaryLike<'ast> for ExprBoolOp {
type FormatOperator = FormatOwnedWithRule<BoolOp, FormatBoolOp, PyFormatContext<'ast>>;
fn binary_layout(&self, source: &str) -> BinaryLayout {
match self.values.as_slice() {
[left, right] => BinaryLayout::from_left_right(left, right, source),
[..] => BinaryLayout::Default,
}
}
fn fmt_default(&self, f: &mut PyFormatter<'ast, '_>) -> FormatResult<()> {
let ExprBoolOp {
range: _,
op,
values,
} = self;
} = item;
let mut values = values.iter();
let comments = f.context().comments().clone();
let inner = format_with(|f: &mut PyFormatter| {
let mut values = values.iter();
let comments = f.context().comments().clone();
let Some(first) = values.next() else {
return Ok(());
};
let Some(first) = values.next() else {
return Ok(());
};
write!(f, [group(&first.format())])?;
write!(f, [in_parentheses_only_group(&first.format())])?;
for value in values {
let leading_value_comments = comments.leading_comments(value);
// Format the expressions leading comments **before** the operator
if leading_value_comments.is_empty() {
write!(f, [soft_line_break_or_space()])?;
} else {
write!(
f,
[hard_line_break(), leading_comments(leading_value_comments)]
)?;
}
for value in values {
let leading_value_comments = comments.leading_comments(value);
// Format the expressions leading comments **before** the operator
if leading_value_comments.is_empty() {
write!(f, [soft_line_break_or_space()])?;
} else {
write!(
f,
[hard_line_break(), leading_comments(leading_value_comments)]
[
op.format(),
space(),
in_parentheses_only_group(&value.format())
]
)?;
}
write!(f, [op.format(), space(), group(&value.format())])?;
}
Ok(())
});
Ok(())
}
fn left(&self) -> FormatResult<&Expr> {
self.values.first().ok_or(FormatError::SyntaxError)
}
fn right(&self) -> FormatResult<&Expr> {
self.values.last().ok_or(FormatError::SyntaxError)
}
fn operator(&self) -> Self::FormatOperator {
self.op.into_format()
in_parentheses_only_group(&inner).fmt(f)
}
}
@ -92,24 +74,7 @@ impl NeedsParentheses for ExprBoolOp {
source: &str,
comments: &Comments,
) -> Parentheses {
match default_expression_needs_parentheses(self.into(), parenthesize, source, comments) {
Parentheses::Optional => match self.binary_layout(source) {
BinaryLayout::Default => Parentheses::Optional,
BinaryLayout::ExpandRight
| BinaryLayout::ExpandLeft
| BinaryLayout::ExpandRightThenLeft
if self
.values
.last()
.map_or(false, |right| comments.has_leading_comments(right)) =>
{
Parentheses::Optional
}
_ => Parentheses::Custom,
},
parentheses => parentheses,
}
default_expression_needs_parentheses(self.into(), parenthesize, source, comments)
}
}

73
crates/ruff_python_formatter/src/expression/expr_compare.rs
View file

@ -1,14 +1,11 @@
use crate::comments::{leading_comments, Comments};
use crate::expression::binary_like::{BinaryLayout, FormatBinaryLike};
use crate::expression::parentheses::{
default_expression_needs_parentheses, NeedsParentheses, Parentheses, Parenthesize,
default_expression_needs_parentheses, in_parentheses_only_group, NeedsParentheses, Parentheses,
Parenthesize,
};
use crate::prelude::*;
use crate::FormatNodeRule;
use ruff_formatter::{
write, FormatError, FormatOwnedWithRule, FormatRefWithRule, FormatRuleWithOptions,
};
use rustpython_parser::ast::Expr;
use ruff_formatter::{write, FormatOwnedWithRule, FormatRefWithRule, FormatRuleWithOptions};
use rustpython_parser::ast::{CmpOp, ExprCompare};
#[derive(Default)]
@ -27,35 +24,16 @@ impl FormatRuleWithOptions<ExprCompare, PyFormatContext<'_>> for FormatExprCompa
impl FormatNodeRule<ExprCompare> for FormatExprCompare {
fn fmt_fields(&self, item: &ExprCompare, f: &mut PyFormatter) -> FormatResult<()> {
item.fmt_binary(self.parentheses, f)
}
}
impl<'ast> FormatBinaryLike<'ast> for ExprCompare {
type FormatOperator = FormatOwnedWithRule<CmpOp, FormatCmpOp, PyFormatContext<'ast>>;
fn binary_layout(&self, source: &str) -> BinaryLayout {
if self.ops.len() == 1 {
match self.comparators.as_slice() {
[right] => BinaryLayout::from_left_right(&self.left, right, source),
[..] => BinaryLayout::Default,
}
} else {
BinaryLayout::Default
}
}
fn fmt_default(&self, f: &mut PyFormatter<'ast, '_>) -> FormatResult<()> {
let ExprCompare {
range: _,
left,
ops,
comparators,
} = self;
} = item;
let comments = f.context().comments().clone();
write!(f, [group(&left.format())])?;
write!(f, [in_parentheses_only_group(&left.format())])?;
assert_eq!(comparators.len(), ops.len());
@ -74,24 +52,18 @@ impl<'ast> FormatBinaryLike<'ast> for ExprCompare {
)?;
}
write!(f, [operator.format(), space(), group(&comparator.format())])?;
write!(
f,
[
operator.format(),
space(),
in_parentheses_only_group(&comparator.format())
]
)?;
}
Ok(())
}
fn left(&self) -> FormatResult<&Expr> {
Ok(self.left.as_ref())
}
fn right(&self) -> FormatResult<&Expr> {
self.comparators.last().ok_or(FormatError::SyntaxError)
}
fn operator(&self) -> Self::FormatOperator {
let op = *self.ops.first().unwrap();
op.into_format()
}
}
impl NeedsParentheses for ExprCompare {
@ -101,24 +73,7 @@ impl NeedsParentheses for ExprCompare {
source: &str,
comments: &Comments,
) -> Parentheses {
match default_expression_needs_parentheses(self.into(), parenthesize, source, comments) {
parentheses @ Parentheses::Optional => match self.binary_layout(source) {
BinaryLayout::Default => parentheses,
BinaryLayout::ExpandRight
| BinaryLayout::ExpandLeft
| BinaryLayout::ExpandRightThenLeft
if self
.comparators
.last()
.map_or(false, |right| comments.has_leading_comments(right)) =>
{
parentheses
}
_ => Parentheses::Custom,
},
parentheses => parentheses,
}
default_expression_needs_parentheses(self.into(), parenthesize, source, comments)
}
}

12
crates/ruff_python_formatter/src/expression/expr_if_exp.rs
View file

@ -1,10 +1,11 @@
use crate::comments::{leading_comments, Comments};
use crate::expression::parentheses::{
default_expression_needs_parentheses, NeedsParentheses, Parentheses, Parenthesize,
default_expression_needs_parentheses, in_parentheses_only_group, NeedsParentheses, Parentheses,
Parenthesize,
};
use crate::{AsFormat, FormatNodeRule, PyFormatter};
use ruff_formatter::prelude::{group, soft_line_break_or_space, space, text};
use ruff_formatter::{format_args, write, Buffer, FormatResult};
use crate::prelude::*;
use crate::FormatNodeRule;
use ruff_formatter::{format_args, write};
use rustpython_parser::ast::ExprIfExp;
#[derive(Default)]
@ -19,12 +20,13 @@ impl FormatNodeRule<ExprIfExp> for FormatExprIfExp {
orelse,
} = item;
let comments = f.context().comments().clone();
// We place `if test` and `else orelse` on a single line, so the `test` and `orelse` leading
// comments go on the line before the `if` or `else` instead of directly ahead `test` or
// `orelse`
write!(
f,
[group(&format_args![
[in_parentheses_only_group(&format_args![
body.format(),
soft_line_break_or_space(),
leading_comments(comments.leading_comments(test.as_ref())),

25
crates/ruff_python_formatter/src/expression/expr_subscript.rs
View file

@ -1,12 +1,15 @@
use rustpython_parser::ast::ExprSubscript;
use ruff_formatter::{format_args, write};
use ruff_python_ast::node::AstNode;
use crate::comments::{trailing_comments, Comments};
use crate::context::NodeLevel;
use crate::expression::parentheses::{
default_expression_needs_parentheses, NeedsParentheses, Parentheses, Parenthesize,
};
use crate::{AsFormat, FormatNodeRule, PyFormatter};
use ruff_formatter::prelude::{group, soft_block_indent, text};
use ruff_formatter::{format_args, write, Buffer, FormatResult};
use ruff_python_ast::node::AstNode;
use rustpython_parser::ast::ExprSubscript;
use crate::prelude::*;
use crate::FormatNodeRule;
#[derive(Default)]
pub struct FormatExprSubscript;
@ -27,10 +30,20 @@ impl FormatNodeRule<ExprSubscript> for FormatExprSubscript {
"The subscript expression must have at most a single comment, the one after the bracket"
);
if let NodeLevel::Expression(Some(group_id)) = f.context().node_level() {
// Enforce the optional parentheses for parenthesized values.
f.context_mut().set_node_level(NodeLevel::Expression(None));
let result = value.format().fmt(f);
f.context_mut()
.set_node_level(NodeLevel::Expression(Some(group_id)));
result?;
} else {
value.format().fmt(f)?;
}
write!(
f,
[group(&format_args![
value.format(),
text("["),
trailing_comments(dangling_comments),
soft_block_indent(&slice.format()),

327
crates/ruff_python_formatter/src/expression/mod.rs
View file

@ -1,16 +1,23 @@
use rustpython_parser::ast;
use rustpython_parser::ast::{Expr, Operator};
use std::cmp::Ordering;
use crate::builders::optional_parentheses;
use ruff_formatter::{
format_args, FormatOwnedWithRule, FormatRefWithRule, FormatRule, FormatRuleWithOptions,
};
use ruff_python_ast::node::AnyNodeRef;
use ruff_python_ast::visitor::preorder::{walk_expr, PreorderVisitor};
use crate::comments::Comments;
use crate::context::NodeLevel;
use crate::expression::expr_tuple::TupleParentheses;
use crate::expression::parentheses::{NeedsParentheses, Parentheses, Parenthesize};
use crate::expression::parentheses::{
is_expression_parenthesized, parenthesized, NeedsParentheses, Parentheses, Parenthesize,
};
use crate::expression::string::StringLayout;
use crate::prelude::*;
use ruff_formatter::{
format_args, write, FormatOwnedWithRule, FormatRefWithRule, FormatRule, FormatRuleWithOptions,
};
use rustpython_parser::ast::Expr;
pub(crate) mod binary_like;
pub(crate) mod expr_attribute;
pub(crate) mod expr_await;
pub(crate) mod expr_bin_op;
@ -99,26 +106,63 @@ impl FormatRule<Expr, PyFormatContext<'_>> for FormatExpr {
Expr::Slice(expr) => expr.format().fmt(f),
});
let saved_level = f.context().node_level();
f.context_mut().set_node_level(NodeLevel::Expression);
let result = match parentheses {
Parentheses::Always => {
write!(
f,
[group(&format_args![
text("("),
soft_block_indent(&format_expr),
text(")")
])]
)
}
Parentheses::Always => parenthesized("(", &format_expr, ")").fmt(f),
// Add optional parentheses. Ignore if the item renders parentheses itself.
Parentheses::Optional => optional_parentheses(&format_expr).fmt(f),
Parentheses::Custom | Parentheses::Never => Format::fmt(&format_expr, f),
};
Parentheses::Optional => {
if can_omit_optional_parentheses(item, f.context()) {
let saved_level = f.context().node_level();
f.context_mut().set_node_level(saved_level);
// The group id is used as a condition in [`in_parentheses_only`] to create a conditional group
// that is only active if the optional parentheses group expands.
let parens_id = f.group_id("optional_parentheses");
f.context_mut()
.set_node_level(NodeLevel::Expression(Some(parens_id)));
// We can't use `soft_block_indent` here because that would always increment the indent,
// even if the group does not break (the indent is not soft). This would result in
// too deep indentations if a `parenthesized` group expands. Using `indent_if_group_breaks`
// gives us the desired *soft* indentation that is only present if the optional parentheses
// are shown.
let result = group(&format_args![
if_group_breaks(&text("(")),
indent_if_group_breaks(
&format_args![soft_line_break(), format_expr],
parens_id
),
soft_line_break(),
if_group_breaks(&text(")"))
])
.with_group_id(Some(parens_id))
.fmt(f);
f.context_mut().set_node_level(saved_level);
result
} else {
optional_parentheses(&format_expr).fmt(f)
}
}
Parentheses::Custom | Parentheses::Never => {
let saved_level = f.context().node_level();
let new_level = match saved_level {
NodeLevel::TopLevel | NodeLevel::CompoundStatement => {
NodeLevel::Expression(None)
}
level @ (NodeLevel::Expression(_) | NodeLevel::ParenthesizedExpression) => {
level
}
};
f.context_mut().set_node_level(new_level);
let result = Format::fmt(&format_expr, f);
f.context_mut().set_node_level(saved_level);
result
}
};
result
}
@ -178,3 +222,240 @@ impl<'ast> IntoFormat<PyFormatContext<'ast>> for Expr {
FormatOwnedWithRule::new(self, FormatExpr::default())
}
}
/// Tests if it is safe to omit the optional parentheses.
///
/// We prefer parentheses at least in the following cases:
/// * The expression contains more than one unparenthesized expression with the same priority. For example,
/// the expression `a * b * c` contains two multiply operations. We prefer parentheses in that case.
/// `(a * b) * c` or `a * b + c` are okay, because the subexpression is parenthesized, or the expression uses operands with a lower priority
/// * The expression contains at least one parenthesized sub expression (optimization to avoid unnecessary work)
///
/// This mimics Black's [`_maybe_split_omitting_optional_parens`](https://github.com/psf/black/blob/d1248ca9beaf0ba526d265f4108836d89cf551b7/src/black/linegen.py#L746-L820)
fn can_omit_optional_parentheses(expr: &Expr, context: &PyFormatContext) -> bool {
let mut visitor = MaxOperatorPriorityVisitor::new(context.contents());
visitor.visit_subexpression(expr);
let (max_operator_priority, operation_count, any_parenthesized_expression) = visitor.finish();
if operation_count > 1 {
false
} else if max_operator_priority == OperatorPriority::Attribute {
true
} else {
// Only use the more complex IR when there is any expression that we can possibly split by
any_parenthesized_expression
}
}
#[derive(Clone, Debug)]
struct MaxOperatorPriorityVisitor<'input> {
max_priority: OperatorPriority,
max_priority_count: u32,
any_parenthesized_expressions: bool,
source: &'input str,
}
impl<'input> MaxOperatorPriorityVisitor<'input> {
fn new(source: &'input str) -> Self {
Self {
source,
max_priority: OperatorPriority::None,
max_priority_count: 0,
any_parenthesized_expressions: false,
}
}
fn update_max_priority(&mut self, current_priority: OperatorPriority) {
self.update_max_priority_with_count(current_priority, 1);
}
fn update_max_priority_with_count(&mut self, current_priority: OperatorPriority, count: u32) {
match self.max_priority.cmp(&current_priority) {
Ordering::Less => {
self.max_priority_count = count;
self.max_priority = current_priority;
}
Ordering::Equal => {
self.max_priority_count += count;
}
Ordering::Greater => {}
}
}
// Visits a subexpression, ignoring whether it is parenthesized or not
fn visit_subexpression(&mut self, expr: &'input Expr) {
match expr {
Expr::Dict(_) | Expr::List(_) | Expr::Tuple(_) | Expr::Set(_) => {
self.any_parenthesized_expressions = true;
// The values are always parenthesized, don't visit.
return;
}
Expr::ListComp(_) | Expr::SetComp(_) | Expr::DictComp(_) => {
self.any_parenthesized_expressions = true;
self.update_max_priority(OperatorPriority::Comprehension);
return;
}
// It's impossible for a file smaller or equal to 4GB to contain more than 2^32 comparisons
// because each comparison requires a left operand, and `n` `operands` and right sides.
#[allow(clippy::cast_possible_truncation)]
Expr::BoolOp(ast::ExprBoolOp {
range: _,
op: _,
values,
}) => self.update_max_priority_with_count(
OperatorPriority::BooleanOperation,
values.len().saturating_sub(1) as u32,
),
Expr::BinOp(ast::ExprBinOp {
op,
left: _,
right: _,
range: _,
}) => self.update_max_priority(OperatorPriority::from(*op)),
Expr::IfExp(_) => {
// + 1 for the if and one for the else
self.update_max_priority_with_count(OperatorPriority::Conditional, 2);
}
// It's impossible for a file smaller or equal to 4GB to contain more than 2^32 comparisons
// because each comparison requires a left operand, and `n` `operands` and right sides.
#[allow(clippy::cast_possible_truncation)]
Expr::Compare(ast::ExprCompare {
range: _,
left: _,
ops,
comparators: _,
}) => {
self.update_max_priority_with_count(OperatorPriority::Comparator, ops.len() as u32);
}
Expr::Call(ast::ExprCall {
range: _,
func,
args: _,
keywords: _,
}) => {
self.any_parenthesized_expressions = true;
// Only walk the function, the arguments are always parenthesized
self.visit_expr(func);
return;
}
Expr::Subscript(_) => {
// Don't walk the value. Splitting before the value looks weird.
// Don't walk the slice, because the slice is always parenthesized.
return;
}
Expr::UnaryOp(ast::ExprUnaryOp {
range: _,
op,
operand: _,
}) => {
if op.is_invert() {
self.update_max_priority(OperatorPriority::BitwiseInversion);
}
}
// `[a, b].test[300].dot`
Expr::Attribute(ast::ExprAttribute {
range: _,
value,
attr: _,
ctx: _,
}) => {
if has_parentheses(value, self.source) {
self.update_max_priority(OperatorPriority::Attribute);
}
}
Expr::NamedExpr(_)
| Expr::GeneratorExp(_)
| Expr::Lambda(_)
| Expr::Await(_)
| Expr::Yield(_)
| Expr::YieldFrom(_)
| Expr::FormattedValue(_)
| Expr::JoinedStr(_)
| Expr::Constant(_)
| Expr::Starred(_)
| Expr::Name(_)
| Expr::Slice(_) => {}
};
walk_expr(self, expr);
}
fn finish(self) -> (OperatorPriority, u32, bool) {
(
self.max_priority,
self.max_priority_count,
self.any_parenthesized_expressions,
)
}
}
impl<'input> PreorderVisitor<'input> for MaxOperatorPriorityVisitor<'input> {
fn visit_expr(&mut self, expr: &'input Expr) {
// Rule only applies for non-parenthesized expressions.
if is_expression_parenthesized(AnyNodeRef::from(expr), self.source) {
self.any_parenthesized_expressions = true;
} else {
self.visit_subexpression(expr);
}
}
}
fn has_parentheses(expr: &Expr, source: &str) -> bool {
matches!(
expr,
Expr::Dict(_)
| Expr::List(_)
| Expr::Tuple(_)
| Expr::Set(_)
| Expr::ListComp(_)
| Expr::SetComp(_)
| Expr::DictComp(_)
| Expr::Call(_)
| Expr::Subscript(_)
) || is_expression_parenthesized(AnyNodeRef::from(expr), source)
}
#[derive(Debug, Copy, Clone, Ord, PartialOrd, Eq, PartialEq)]
enum OperatorPriority {
None,
Attribute,
Comparator,
Exponential,
BitwiseInversion,
Multiplicative,
Additive,
Shift,
BitwiseAnd,
BitwiseOr,
BitwiseXor,
// TODO(micha)
#[allow(unused)]
String,
BooleanOperation,
Conditional,
Comprehension,
}
impl From<ast::Operator> for OperatorPriority {
fn from(value: Operator) -> Self {
match value {
Operator::Add | Operator::Sub => OperatorPriority::Additive,
Operator::Mult
| Operator::MatMult
| Operator::Div
| Operator::Mod
| Operator::FloorDiv => OperatorPriority::Multiplicative,
Operator::Pow => OperatorPriority::Exponential,
Operator::LShift | Operator::RShift => OperatorPriority::Shift,
Operator::BitOr => OperatorPriority::BitwiseOr,
Operator::BitXor => OperatorPriority::BitwiseXor,
Operator::BitAnd => OperatorPriority::BitwiseAnd,
}
}
}

71
crates/ruff_python_formatter/src/expression/parentheses.rs
View file

@ -1,7 +1,9 @@
use crate::comments::Comments;
use crate::context::NodeLevel;
use crate::prelude::*;
use crate::trivia::{first_non_trivia_token, first_non_trivia_token_rev, Token, TokenKind};
use ruff_formatter::{format_args, write, Argument, Arguments};
use ruff_formatter::prelude::tag::Condition;
use ruff_formatter::{format_args, Argument, Arguments};
use ruff_python_ast::node::AnyNodeRef;
use rustpython_parser::ast::Ranged;
@ -122,6 +124,9 @@ pub(crate) fn is_expression_parenthesized(expr: AnyNodeRef, contents: &str) -> b
)
}
/// Formats `content` enclosed by the `left` and `right` parentheses. The implementation also ensures
/// that expanding the parenthesized expression (or any of its children) doesn't enforce the
/// optional parentheses around the outer-most expression to materialize.
pub(crate) fn parenthesized<'content, 'ast, Content>(
left: &'static str,
content: &'content Content,
@ -145,13 +150,67 @@ pub(crate) struct FormatParenthesized<'content, 'ast> {
impl<'ast> Format<PyFormatContext<'ast>> for FormatParenthesized<'_, 'ast> {
fn fmt(&self, f: &mut Formatter<PyFormatContext<'ast>>) -> FormatResult<()> {
write!(
f,
[group(&format_args![
let inner = format_with(|f| {
group(&format_args![
text(self.left),
&soft_block_indent(&Arguments::from(&self.content)),
text(self.right)
])]
)
])
.fmt(f)
});
let current_level = f.context().node_level();
f.context_mut()
.set_node_level(NodeLevel::ParenthesizedExpression);
let result = if let NodeLevel::Expression(Some(group_id)) = current_level {
// Use fits expanded if there's an enclosing group that adds the optional parentheses.
// This ensures that expanding this parenthesized expression does not expand the optional parentheses group.
fits_expanded(&inner)
.with_condition(Some(Condition::if_group_fits_on_line(group_id)))
.fmt(f)
} else {
// It's not necessary to wrap the content if it is not inside of an optional_parentheses group.
inner.fmt(f)
};
f.context_mut().set_node_level(current_level);
result
}
}
/// Makes `content` a group, but only if the outer expression is parenthesized (a list, parenthesized expression, dict, ...)
/// or if the expression gets parenthesized because it expands over multiple lines.
pub(crate) fn in_parentheses_only_group<'content, 'ast, Content>(
content: &'content Content,
) -> FormatInParenthesesOnlyGroup<'content, 'ast>
where
Content: Format<PyFormatContext<'ast>>,
{
FormatInParenthesesOnlyGroup {
content: Argument::new(content),
}
}
pub(crate) struct FormatInParenthesesOnlyGroup<'content, 'ast> {
content: Argument<'content, PyFormatContext<'ast>>,
}
impl<'ast> Format<PyFormatContext<'ast>> for FormatInParenthesesOnlyGroup<'_, 'ast> {
fn fmt(&self, f: &mut Formatter<PyFormatContext<'ast>>) -> FormatResult<()> {
if let NodeLevel::Expression(Some(group_id)) = f.context().node_level() {
// If this content is enclosed by a group that adds the optional parentheses, then *disable*
// this group *except* if the optional parentheses are shown.
conditional_group(
&Arguments::from(&self.content),
Condition::if_group_breaks(group_id),
)
.fmt(f)
} else {
// Unconditionally group the content if it is not enclosed by an optional parentheses group.
group(&Arguments::from(&self.content)).fmt(f)
}
}
}

3
crates/ruff_python_formatter/src/other/arguments.rs
View file

@ -31,7 +31,8 @@ impl FormatNodeRule<Arguments> for FormatArguments {
} = item;
let saved_level = f.context().node_level();
f.context_mut().set_node_level(NodeLevel::Expression);
f.context_mut()
.set_node_level(NodeLevel::ParenthesizedExpression);
let comments = f.context().comments().clone();
let dangling = comments.dangling_comments(item);