Add initial formatter implementation (#2883)

# Summary

This PR contains the code for the autoformatter proof-of-concept.

## Crate structure

The primary formatting hook is the `fmt` function in `crates/ruff_python_formatter/src/lib.rs`.

The current formatter approach is outlined in `crates/ruff_python_formatter/src/lib.rs`, and is structured as follows:

- Tokenize the code using the RustPython lexer.
- In `crates/ruff_python_formatter/src/trivia.rs`, extract a variety of trivia tokens from the token stream. These include comments, trailing commas, and empty lines.
- Generate the AST via the RustPython parser.
- In `crates/ruff_python_formatter/src/cst.rs`, convert the AST to a CST structure. As of now, the CST is nearly identical to the AST, except that every node gets a `trivia` vector. But we might want to modify it further.
- In `crates/ruff_python_formatter/src/attachment.rs`, attach each trivia token to the corresponding CST node. The logic for this is mostly in `decorate_trivia` and is ported almost directly from Prettier (given each token, find its preceding, following, and enclosing nodes, then attach the token to the appropriate node in a second pass).
- In `crates/ruff_python_formatter/src/newlines.rs`, normalize newlines to match Black’s preferences. This involves traversing the CST and inserting or removing `TriviaToken` values as we go.
- Call `format!` on the CST, which delegates to type-specific formatter implementations (e.g., `crates/ruff_python_formatter/src/format/stmt.rs` for `Stmt` nodes, and similar for `Expr` nodes; the others are trivial). Those type-specific implementations delegate to kind-specific functions (e.g., `format_func_def`).

## Testing and iteration

The formatter is being developed against the Black test suite, which was copied over in-full to `crates/ruff_python_formatter/resources/test/fixtures/black`.

The Black fixtures had to be modified to create `[insta](https://github.com/mitsuhiko/insta)`-compatible snapshots, which now exist in the repo.

My approach thus far has been to try and improve coverage by tackling fixtures one-by-one.

## What works, and what doesn’t

- *Most* nodes are supported at a basic level (though there are a few stragglers at time of writing, like `StmtKind::Try`).
- Newlines are properly preserved in most cases.
- Magic trailing commas are properly preserved in some (but not all) cases.
- Trivial leading and trailing standalone comments mostly work (although maybe not at the end of a file).
- Inline comments, and comments within expressions, often don’t work -- they work in a few cases, but it’s one-off right now. (We’re probably associating them with the “right” nodes more often than we are actually rendering them in the right place.)
- We don’t properly normalize string quotes. (At present, we just repeat any constants verbatim.)
- We’re mishandling a bunch of wrapping cases (if we treat Black as the reference implementation). Here are a few examples (demonstrating Black's stable behavior):

```py
# In some cases, if the end expression is "self-closing" (functions,
# lists, dictionaries, sets, subscript accesses, and any length-two
# boolean operations that end in these elments), Black
# will wrap like this...
if some_expression and f(
    b,
    c,
    d,
):
    pass

# ...whereas we do this:
if (
    some_expression
    and f(
        b,
        c,
        d,
    )
):
    pass

# If function arguments can fit on a single line, then Black will
# format them like this, rather than exploding them vertically.
if f(
    a, b, c, d, e, f, g, ...
):
    pass
```

- We don’t properly preserve parentheses in all cases. Black preserves parentheses in some but not all cases.
This commit is contained in:
Charlie Marsh 2023-02-14 23:06:35 -05:00 committed by GitHub
parent f661c90bd7
commit ca49b00e55
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
134 changed files with 12044 additions and 18 deletions

View file

@ -0,0 +1,31 @@
use crate::core::visitor;
use crate::core::visitor::Visitor;
use crate::cst::{Expr, Stmt};
use crate::trivia::{decorate_trivia, TriviaIndex, TriviaToken};
struct AttachmentVisitor {
index: TriviaIndex,
}
impl<'a> Visitor<'a> for AttachmentVisitor {
fn visit_stmt(&mut self, stmt: &'a mut Stmt) {
let trivia = self.index.stmt.remove(&stmt.id());
if let Some(comments) = trivia {
stmt.trivia.extend(comments);
}
visitor::walk_stmt(self, stmt);
}
fn visit_expr(&mut self, expr: &'a mut Expr) {
let trivia = self.index.expr.remove(&expr.id());
if let Some(comments) = trivia {
expr.trivia.extend(comments);
}
visitor::walk_expr(self, expr);
}
}
pub fn attach(python_cst: &mut [Stmt], trivia: Vec<TriviaToken>) {
let index = decorate_trivia(trivia, python_cst);
AttachmentVisitor { index }.visit_body(python_cst);
}

View file

@ -0,0 +1,77 @@
use ruff_formatter::prelude::*;
use ruff_formatter::{write, Format};
use ruff_text_size::TextRange;
use crate::context::ASTFormatContext;
use crate::core::types::Range;
use crate::trivia::{Relationship, Trivia, TriviaKind};
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub struct Literal {
range: Range,
}
impl Format<ASTFormatContext<'_>> for Literal {
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
let (text, start, end) = f.context().locator().slice(self.range);
f.write_element(FormatElement::StaticTextSlice {
text,
range: TextRange::new(start.try_into().unwrap(), end.try_into().unwrap()),
})
}
}
// TODO(charlie): We still can't use this everywhere we'd like. We need the AST
// to include ranges for all `Ident` nodes.
#[inline]
pub const fn literal(range: Range) -> Literal {
Literal { range }
}
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub struct LeadingComments<'a> {
comments: &'a [Trivia],
}
impl Format<ASTFormatContext<'_>> for LeadingComments<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
for comment in self.comments {
if matches!(comment.relationship, Relationship::Leading) {
if let TriviaKind::StandaloneComment(range) = comment.kind {
write!(f, [hard_line_break()])?;
write!(f, [literal(range)])?;
}
}
}
Ok(())
}
}
#[inline]
pub const fn leading_comments(comments: &[Trivia]) -> LeadingComments {
LeadingComments { comments }
}
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub struct TrailingComments<'a> {
comments: &'a [Trivia],
}
impl Format<ASTFormatContext<'_>> for TrailingComments<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
for comment in self.comments {
if matches!(comment.relationship, Relationship::Trailing) {
if let TriviaKind::StandaloneComment(range) = comment.kind {
write!(f, [hard_line_break()])?;
write!(f, [literal(range)])?;
}
}
}
Ok(())
}
}
#[inline]
pub const fn trailing_comments(comments: &[Trivia]) -> TrailingComments {
TrailingComments { comments }
}

View file

@ -0,0 +1,11 @@
use std::path::PathBuf;
use clap::{command, Parser};
#[derive(Parser)]
#[command(author, version, about, long_about = None)]
pub struct Cli {
/// Python file to round-trip.
#[arg(required = true)]
pub file: PathBuf,
}

View file

@ -0,0 +1,28 @@
use ruff_formatter::{FormatContext, SimpleFormatOptions};
use crate::core::locator::Locator;
pub struct ASTFormatContext<'a> {
options: SimpleFormatOptions,
locator: Locator<'a>,
}
impl<'a> ASTFormatContext<'a> {
pub fn new(options: SimpleFormatOptions, locator: Locator<'a>) -> Self {
Self { options, locator }
}
}
impl FormatContext for ASTFormatContext<'_> {
type Options = SimpleFormatOptions;
fn options(&self) -> &Self::Options {
&self.options
}
}
impl<'a> ASTFormatContext<'a> {
pub fn locator(&'a self) -> &'a Locator {
&self.locator
}
}

View file

@ -0,0 +1,153 @@
//! Struct used to efficiently slice source code at (row, column) Locations.
use std::rc::Rc;
use once_cell::unsync::OnceCell;
use rustpython_parser::ast::Location;
use crate::core::types::Range;
pub struct Locator<'a> {
contents: &'a str,
contents_rc: Rc<str>,
index: OnceCell<Index>,
}
pub enum Index {
Ascii(Vec<usize>),
Utf8(Vec<Vec<usize>>),
}
/// Compute the starting byte index of each line in ASCII source code.
fn index_ascii(contents: &str) -> Vec<usize> {
let mut index = Vec::with_capacity(48);
index.push(0);
let bytes = contents.as_bytes();
for (i, byte) in bytes.iter().enumerate() {
if *byte == b'\n' {
index.push(i + 1);
}
}
index
}
/// Compute the starting byte index of each character in UTF-8 source code.
fn index_utf8(contents: &str) -> Vec<Vec<usize>> {
let mut index = Vec::with_capacity(48);
let mut current_row = Vec::with_capacity(48);
let mut current_byte_offset = 0;
let mut previous_char = '\0';
for char in contents.chars() {
current_row.push(current_byte_offset);
if char == '\n' {
if previous_char == '\r' {
current_row.pop();
}
index.push(current_row);
current_row = Vec::with_capacity(48);
}
current_byte_offset += char.len_utf8();
previous_char = char;
}
index.push(current_row);
index
}
/// Compute the starting byte index of each line in source code.
pub fn index(contents: &str) -> Index {
if contents.is_ascii() {
Index::Ascii(index_ascii(contents))
} else {
Index::Utf8(index_utf8(contents))
}
}
/// Truncate a [`Location`] to a byte offset in ASCII source code.
fn truncate_ascii(location: Location, index: &[usize], contents: &str) -> usize {
if location.row() - 1 == index.len() && location.column() == 0
|| (!index.is_empty()
&& location.row() - 1 == index.len() - 1
&& index[location.row() - 1] + location.column() >= contents.len())
{
contents.len()
} else {
index[location.row() - 1] + location.column()
}
}
/// Truncate a [`Location`] to a byte offset in UTF-8 source code.
fn truncate_utf8(location: Location, index: &[Vec<usize>], contents: &str) -> usize {
if (location.row() - 1 == index.len() && location.column() == 0)
|| (location.row() - 1 == index.len() - 1
&& location.column() == index[location.row() - 1].len())
{
contents.len()
} else {
index[location.row() - 1][location.column()]
}
}
/// Truncate a [`Location`] to a byte offset in source code.
fn truncate(location: Location, index: &Index, contents: &str) -> usize {
match index {
Index::Ascii(index) => truncate_ascii(location, index, contents),
Index::Utf8(index) => truncate_utf8(location, index, contents),
}
}
impl<'a> Locator<'a> {
pub fn new(contents: &'a str) -> Self {
Locator {
contents,
contents_rc: Rc::from(contents),
index: OnceCell::new(),
}
}
fn get_or_init_index(&self) -> &Index {
self.index.get_or_init(|| index(self.contents))
}
pub fn slice_source_code_until(&self, location: Location) -> &'a str {
let index = self.get_or_init_index();
let offset = truncate(location, index, self.contents);
&self.contents[..offset]
}
pub fn slice_source_code_at(&self, location: Location) -> &'a str {
let index = self.get_or_init_index();
let offset = truncate(location, index, self.contents);
&self.contents[offset..]
}
pub fn slice_source_code_range(&self, range: &Range) -> &'a str {
let index = self.get_or_init_index();
let start = truncate(range.location, index, self.contents);
let end = truncate(range.end_location, index, self.contents);
&self.contents[start..end]
}
pub fn slice(&self, range: Range) -> (Rc<str>, usize, usize) {
let index = self.get_or_init_index();
let start = truncate(range.location, index, self.contents);
let end = truncate(range.end_location, index, self.contents);
(Rc::clone(&self.contents_rc), start, end)
}
pub fn partition_source_code_at(
&self,
outer: &Range,
inner: &Range,
) -> (&'a str, &'a str, &'a str) {
let index = self.get_or_init_index();
let outer_start = truncate(outer.location, index, self.contents);
let outer_end = truncate(outer.end_location, index, self.contents);
let inner_start = truncate(inner.location, index, self.contents);
let inner_end = truncate(inner.end_location, index, self.contents);
(
&self.contents[outer_start..inner_start],
&self.contents[inner_start..inner_end],
&self.contents[inner_end..outer_end],
)
}
}

View file

@ -0,0 +1,4 @@
pub mod locator;
pub mod rustpython_helpers;
pub mod types;
pub mod visitor;

View file

@ -0,0 +1,29 @@
use rustpython_parser::ast::{Mod, Suite};
use rustpython_parser::error::ParseError;
use rustpython_parser::lexer::LexResult;
use rustpython_parser::mode::Mode;
use rustpython_parser::{lexer, parser};
/// Collect tokens up to and including the first error.
pub fn tokenize(contents: &str) -> Vec<LexResult> {
let mut tokens: Vec<LexResult> = vec![];
for tok in lexer::make_tokenizer(contents) {
let is_err = tok.is_err();
tokens.push(tok);
if is_err {
break;
}
}
tokens
}
/// Parse a full Python program from its tokens.
pub(crate) fn parse_program_tokens(
lxr: Vec<LexResult>,
source_path: &str,
) -> anyhow::Result<Suite, ParseError> {
parser::parse_tokens(lxr, Mode::Module, source_path).map(|top| match top {
Mod::Module { body, .. } => body,
_ => unreachable!(),
})
}

View file

@ -0,0 +1,76 @@
use std::ops::Deref;
use rustpython_parser::ast::Location;
use crate::cst::{Expr, Located, Stmt};
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, PartialOrd, Ord)]
pub struct Range {
pub location: Location,
pub end_location: Location,
}
impl Range {
pub fn new(location: Location, end_location: Location) -> Self {
Self {
location,
end_location,
}
}
pub fn from_located<T>(located: &Located<T>) -> Self {
Range::new(located.location, located.end_location.unwrap())
}
}
#[derive(Debug, Copy, Clone)]
pub struct RefEquality<'a, T>(pub &'a T);
impl<'a, T> std::hash::Hash for RefEquality<'a, T> {
fn hash<H>(&self, state: &mut H)
where
H: std::hash::Hasher,
{
(self.0 as *const T).hash(state);
}
}
impl<'a, 'b, T> PartialEq<RefEquality<'b, T>> for RefEquality<'a, T> {
fn eq(&self, other: &RefEquality<'b, T>) -> bool {
std::ptr::eq(self.0, other.0)
}
}
impl<'a, T> Eq for RefEquality<'a, T> {}
impl<'a, T> Deref for RefEquality<'a, T> {
type Target = T;
fn deref(&self) -> &T {
self.0
}
}
impl<'a> From<&RefEquality<'a, Stmt>> for &'a Stmt {
fn from(r: &RefEquality<'a, Stmt>) -> Self {
r.0
}
}
impl<'a> From<&RefEquality<'a, Expr>> for &'a Expr {
fn from(r: &RefEquality<'a, Expr>) -> Self {
r.0
}
}
impl<'a> From<&RefEquality<'a, rustpython_parser::ast::Stmt>> for &'a rustpython_parser::ast::Stmt {
fn from(r: &RefEquality<'a, rustpython_parser::ast::Stmt>) -> Self {
r.0
}
}
impl<'a> From<&RefEquality<'a, rustpython_parser::ast::Expr>> for &'a rustpython_parser::ast::Expr {
fn from(r: &RefEquality<'a, rustpython_parser::ast::Expr>) -> Self {
r.0
}
}

View file

@ -0,0 +1,574 @@
use rustpython_parser::ast::Constant;
use crate::cst::{
Alias, Arg, Arguments, Boolop, Cmpop, Comprehension, Excepthandler, ExcepthandlerKind, Expr,
ExprContext, ExprKind, Keyword, MatchCase, Operator, Pattern, PatternKind, Stmt, StmtKind,
Unaryop, Withitem,
};
pub trait Visitor<'a> {
fn visit_stmt(&mut self, stmt: &'a mut Stmt) {
walk_stmt(self, stmt);
}
fn visit_annotation(&mut self, expr: &'a mut Expr) {
walk_expr(self, expr);
}
fn visit_expr(&mut self, expr: &'a mut Expr) {
walk_expr(self, expr);
}
fn visit_constant(&mut self, constant: &'a mut Constant) {
walk_constant(self, constant);
}
fn visit_expr_context(&mut self, expr_context: &'a mut ExprContext) {
walk_expr_context(self, expr_context);
}
fn visit_boolop(&mut self, boolop: &'a mut Boolop) {
walk_boolop(self, boolop);
}
fn visit_operator(&mut self, operator: &'a mut Operator) {
walk_operator(self, operator);
}
fn visit_unaryop(&mut self, unaryop: &'a mut Unaryop) {
walk_unaryop(self, unaryop);
}
fn visit_cmpop(&mut self, cmpop: &'a mut Cmpop) {
walk_cmpop(self, cmpop);
}
fn visit_comprehension(&mut self, comprehension: &'a mut Comprehension) {
walk_comprehension(self, comprehension);
}
fn visit_excepthandler(&mut self, excepthandler: &'a mut Excepthandler) {
walk_excepthandler(self, excepthandler);
}
fn visit_format_spec(&mut self, format_spec: &'a mut Expr) {
walk_expr(self, format_spec);
}
fn visit_arguments(&mut self, arguments: &'a mut Arguments) {
walk_arguments(self, arguments);
}
fn visit_arg(&mut self, arg: &'a mut Arg) {
walk_arg(self, arg);
}
fn visit_keyword(&mut self, keyword: &'a mut Keyword) {
walk_keyword(self, keyword);
}
fn visit_alias(&mut self, alias: &'a mut Alias) {
walk_alias(self, alias);
}
fn visit_withitem(&mut self, withitem: &'a mut Withitem) {
walk_withitem(self, withitem);
}
fn visit_match_case(&mut self, match_case: &'a mut MatchCase) {
walk_match_case(self, match_case);
}
fn visit_pattern(&mut self, pattern: &'a mut Pattern) {
walk_pattern(self, pattern);
}
fn visit_body(&mut self, body: &'a mut [Stmt]) {
walk_body(self, body);
}
}
pub fn walk_body<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, body: &'a mut [Stmt]) {
for stmt in body {
visitor.visit_stmt(stmt);
}
}
pub fn walk_stmt<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, stmt: &'a mut Stmt) {
match &mut stmt.node {
StmtKind::FunctionDef {
args,
body,
decorator_list,
returns,
..
} => {
visitor.visit_arguments(args);
for expr in decorator_list {
visitor.visit_expr(expr);
}
for expr in returns {
visitor.visit_annotation(expr);
}
visitor.visit_body(body);
}
StmtKind::AsyncFunctionDef {
args,
body,
decorator_list,
returns,
..
} => {
visitor.visit_arguments(args);
for expr in decorator_list {
visitor.visit_expr(expr);
}
for expr in returns {
visitor.visit_annotation(expr);
}
visitor.visit_body(body);
}
StmtKind::ClassDef {
bases,
keywords,
body,
decorator_list,
..
} => {
for expr in bases {
visitor.visit_expr(expr);
}
for keyword in keywords {
visitor.visit_keyword(keyword);
}
for expr in decorator_list {
visitor.visit_expr(expr);
}
visitor.visit_body(body);
}
StmtKind::Return { value } => {
if let Some(expr) = value {
visitor.visit_expr(expr);
}
}
StmtKind::Delete { targets } => {
for expr in targets {
visitor.visit_expr(expr);
}
}
StmtKind::Assign { targets, value, .. } => {
visitor.visit_expr(value);
for expr in targets {
visitor.visit_expr(expr);
}
}
StmtKind::AugAssign { target, op, value } => {
visitor.visit_expr(target);
visitor.visit_operator(op);
visitor.visit_expr(value);
}
StmtKind::AnnAssign {
target,
annotation,
value,
..
} => {
visitor.visit_annotation(annotation);
if let Some(expr) = value {
visitor.visit_expr(expr);
}
visitor.visit_expr(target);
}
StmtKind::For {
target,
iter,
body,
orelse,
..
} => {
visitor.visit_expr(iter);
visitor.visit_expr(target);
visitor.visit_body(body);
visitor.visit_body(orelse);
}
StmtKind::AsyncFor {
target,
iter,
body,
orelse,
..
} => {
visitor.visit_expr(iter);
visitor.visit_expr(target);
visitor.visit_body(body);
visitor.visit_body(orelse);
}
StmtKind::While { test, body, orelse } => {
visitor.visit_expr(test);
visitor.visit_body(body);
visitor.visit_body(orelse);
}
StmtKind::If { test, body, orelse } => {
visitor.visit_expr(test);
visitor.visit_body(body);
visitor.visit_body(orelse);
}
StmtKind::With { items, body, .. } => {
for withitem in items {
visitor.visit_withitem(withitem);
}
visitor.visit_body(body);
}
StmtKind::AsyncWith { items, body, .. } => {
for withitem in items {
visitor.visit_withitem(withitem);
}
visitor.visit_body(body);
}
StmtKind::Match { subject, cases } => {
// TODO(charlie): Handle `cases`.
visitor.visit_expr(subject);
for match_case in cases {
visitor.visit_match_case(match_case);
}
}
StmtKind::Raise { exc, cause } => {
if let Some(expr) = exc {
visitor.visit_expr(expr);
};
if let Some(expr) = cause {
visitor.visit_expr(expr);
};
}
StmtKind::Try {
body,
handlers,
orelse,
finalbody,
} => {
visitor.visit_body(body);
for excepthandler in handlers {
visitor.visit_excepthandler(excepthandler);
}
visitor.visit_body(orelse);
visitor.visit_body(finalbody);
}
StmtKind::Assert { test, msg } => {
visitor.visit_expr(test);
if let Some(expr) = msg {
visitor.visit_expr(expr);
}
}
StmtKind::Import { names } => {
for alias in names {
visitor.visit_alias(alias);
}
}
StmtKind::ImportFrom { names, .. } => {
for alias in names {
visitor.visit_alias(alias);
}
}
StmtKind::Global { .. } => {}
StmtKind::Nonlocal { .. } => {}
StmtKind::Expr { value } => visitor.visit_expr(value),
StmtKind::Pass => {}
StmtKind::Break => {}
StmtKind::Continue => {}
}
}
pub fn walk_expr<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, expr: &'a mut Expr) {
match &mut expr.node {
ExprKind::BoolOp { op, values } => {
visitor.visit_boolop(op);
for expr in values {
visitor.visit_expr(expr);
}
}
ExprKind::NamedExpr { target, value } => {
visitor.visit_expr(target);
visitor.visit_expr(value);
}
ExprKind::BinOp { left, op, right } => {
visitor.visit_expr(left);
visitor.visit_operator(op);
visitor.visit_expr(right);
}
ExprKind::UnaryOp { op, operand } => {
visitor.visit_unaryop(op);
visitor.visit_expr(operand);
}
ExprKind::Lambda { args, body } => {
visitor.visit_arguments(args);
visitor.visit_expr(body);
}
ExprKind::IfExp { test, body, orelse } => {
visitor.visit_expr(test);
visitor.visit_expr(body);
visitor.visit_expr(orelse);
}
ExprKind::Dict { keys, values } => {
for expr in keys.iter_mut().flatten() {
visitor.visit_expr(expr);
}
for expr in values {
visitor.visit_expr(expr);
}
}
ExprKind::Set { elts } => {
for expr in elts {
visitor.visit_expr(expr);
}
}
ExprKind::ListComp { elt, generators } => {
for comprehension in generators {
visitor.visit_comprehension(comprehension);
}
visitor.visit_expr(elt);
}
ExprKind::SetComp { elt, generators } => {
for comprehension in generators {
visitor.visit_comprehension(comprehension);
}
visitor.visit_expr(elt);
}
ExprKind::DictComp {
key,
value,
generators,
} => {
for comprehension in generators {
visitor.visit_comprehension(comprehension);
}
visitor.visit_expr(key);
visitor.visit_expr(value);
}
ExprKind::GeneratorExp { elt, generators } => {
for comprehension in generators {
visitor.visit_comprehension(comprehension);
}
visitor.visit_expr(elt);
}
ExprKind::Await { value } => visitor.visit_expr(value),
ExprKind::Yield { value } => {
if let Some(expr) = value {
visitor.visit_expr(expr);
}
}
ExprKind::YieldFrom { value } => visitor.visit_expr(value),
ExprKind::Compare {
left,
ops,
comparators,
} => {
visitor.visit_expr(left);
for cmpop in ops {
visitor.visit_cmpop(cmpop);
}
for expr in comparators {
visitor.visit_expr(expr);
}
}
ExprKind::Call {
func,
args,
keywords,
} => {
visitor.visit_expr(func);
for expr in args {
visitor.visit_expr(expr);
}
for keyword in keywords {
visitor.visit_keyword(keyword);
}
}
ExprKind::FormattedValue {
value, format_spec, ..
} => {
visitor.visit_expr(value);
if let Some(expr) = format_spec {
visitor.visit_format_spec(expr);
}
}
ExprKind::JoinedStr { values } => {
for expr in values {
visitor.visit_expr(expr);
}
}
ExprKind::Constant { value, .. } => visitor.visit_constant(value),
ExprKind::Attribute { value, ctx, .. } => {
visitor.visit_expr(value);
visitor.visit_expr_context(ctx);
}
ExprKind::Subscript { value, slice, ctx } => {
visitor.visit_expr(value);
visitor.visit_expr(slice);
visitor.visit_expr_context(ctx);
}
ExprKind::Starred { value, ctx } => {
visitor.visit_expr(value);
visitor.visit_expr_context(ctx);
}
ExprKind::Name { ctx, .. } => {
visitor.visit_expr_context(ctx);
}
ExprKind::List { elts, ctx } => {
for expr in elts {
visitor.visit_expr(expr);
}
visitor.visit_expr_context(ctx);
}
ExprKind::Tuple { elts, ctx } => {
for expr in elts {
visitor.visit_expr(expr);
}
visitor.visit_expr_context(ctx);
}
ExprKind::Slice { lower, upper, step } => {
if let Some(expr) = lower {
visitor.visit_expr(expr);
}
if let Some(expr) = upper {
visitor.visit_expr(expr);
}
if let Some(expr) = step {
visitor.visit_expr(expr);
}
}
}
}
pub fn walk_constant<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, constant: &'a mut Constant) {
if let Constant::Tuple(constants) = constant {
for constant in constants {
visitor.visit_constant(constant);
}
}
}
pub fn walk_comprehension<'a, V: Visitor<'a> + ?Sized>(
visitor: &mut V,
comprehension: &'a mut Comprehension,
) {
visitor.visit_expr(&mut comprehension.iter);
visitor.visit_expr(&mut comprehension.target);
for expr in &mut comprehension.ifs {
visitor.visit_expr(expr);
}
}
pub fn walk_excepthandler<'a, V: Visitor<'a> + ?Sized>(
visitor: &mut V,
excepthandler: &'a mut Excepthandler,
) {
match &mut excepthandler.node {
ExcepthandlerKind::ExceptHandler { type_, body, .. } => {
if let Some(expr) = type_ {
visitor.visit_expr(expr);
}
visitor.visit_body(body);
}
}
}
pub fn walk_arguments<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, arguments: &'a mut Arguments) {
for arg in &mut arguments.posonlyargs {
visitor.visit_arg(arg);
}
for arg in &mut arguments.args {
visitor.visit_arg(arg);
}
if let Some(arg) = &mut arguments.vararg {
visitor.visit_arg(arg);
}
for arg in &mut arguments.kwonlyargs {
visitor.visit_arg(arg);
}
for expr in &mut arguments.kw_defaults {
visitor.visit_expr(expr);
}
if let Some(arg) = &mut arguments.kwarg {
visitor.visit_arg(arg);
}
for expr in &mut arguments.defaults {
visitor.visit_expr(expr);
}
}
pub fn walk_arg<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, arg: &'a mut Arg) {
if let Some(expr) = &mut arg.node.annotation {
visitor.visit_annotation(expr);
}
}
pub fn walk_keyword<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, keyword: &'a mut Keyword) {
visitor.visit_expr(&mut keyword.node.value);
}
pub fn walk_withitem<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, withitem: &'a mut Withitem) {
visitor.visit_expr(&mut withitem.context_expr);
if let Some(expr) = &mut withitem.optional_vars {
visitor.visit_expr(expr);
}
}
pub fn walk_match_case<'a, V: Visitor<'a> + ?Sized>(
visitor: &mut V,
match_case: &'a mut MatchCase,
) {
visitor.visit_pattern(&mut match_case.pattern);
if let Some(expr) = &mut match_case.guard {
visitor.visit_expr(expr);
}
visitor.visit_body(&mut match_case.body);
}
pub fn walk_pattern<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, pattern: &'a mut Pattern) {
match &mut pattern.node {
PatternKind::MatchValue { value } => visitor.visit_expr(value),
PatternKind::MatchSingleton { value } => visitor.visit_constant(value),
PatternKind::MatchSequence { patterns } => {
for pattern in patterns {
visitor.visit_pattern(pattern);
}
}
PatternKind::MatchMapping { keys, patterns, .. } => {
for expr in keys {
visitor.visit_expr(expr);
}
for pattern in patterns {
visitor.visit_pattern(pattern);
}
}
PatternKind::MatchClass {
cls,
patterns,
kwd_patterns,
..
} => {
visitor.visit_expr(cls);
for pattern in patterns {
visitor.visit_pattern(pattern);
}
for pattern in kwd_patterns {
visitor.visit_pattern(pattern);
}
}
PatternKind::MatchStar { .. } => {}
PatternKind::MatchAs { pattern, .. } => {
if let Some(pattern) = pattern {
visitor.visit_pattern(pattern);
}
}
PatternKind::MatchOr { patterns } => {
for pattern in patterns {
visitor.visit_pattern(pattern);
}
}
}
}
#[allow(unused_variables)]
pub fn walk_expr_context<'a, V: Visitor<'a> + ?Sized>(
visitor: &mut V,
expr_context: &'a mut ExprContext,
) {
}
#[allow(unused_variables)]
pub fn walk_boolop<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, boolop: &'a mut Boolop) {}
#[allow(unused_variables)]
pub fn walk_operator<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, operator: &'a mut Operator) {}
#[allow(unused_variables)]
pub fn walk_unaryop<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, unaryop: &'a mut Unaryop) {}
#[allow(unused_variables)]
pub fn walk_cmpop<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, cmpop: &'a mut Cmpop) {}
#[allow(unused_variables)]
pub fn walk_alias<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, alias: &'a mut Alias) {}

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,33 @@
use ruff_formatter::prelude::*;
use ruff_formatter::write;
use ruff_text_size::TextSize;
use crate::context::ASTFormatContext;
use crate::cst::Alias;
use crate::shared_traits::AsFormat;
pub struct FormatAlias<'a> {
item: &'a Alias,
}
impl AsFormat<ASTFormatContext<'_>> for Alias {
type Format<'a> = FormatAlias<'a>;
fn format(&self) -> Self::Format<'_> {
FormatAlias { item: self }
}
}
impl Format<ASTFormatContext<'_>> for FormatAlias<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext>) -> FormatResult<()> {
let alias = self.item;
write!(f, [dynamic_text(&alias.node.name, TextSize::default())])?;
if let Some(asname) = &alias.node.asname {
write!(f, [text(" as ")])?;
write!(f, [dynamic_text(asname, TextSize::default())])?;
}
Ok(())
}
}

View file

@ -0,0 +1,33 @@
use ruff_formatter::prelude::*;
use ruff_formatter::write;
use ruff_text_size::TextSize;
use crate::context::ASTFormatContext;
use crate::cst::Arg;
use crate::shared_traits::AsFormat;
pub struct FormatArg<'a> {
item: &'a Arg,
}
impl AsFormat<ASTFormatContext<'_>> for Arg {
type Format<'a> = FormatArg<'a>;
fn format(&self) -> Self::Format<'_> {
FormatArg { item: self }
}
}
impl Format<ASTFormatContext<'_>> for FormatArg<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext>) -> FormatResult<()> {
let arg = self.item;
write!(f, [dynamic_text(&arg.node.arg, TextSize::default())])?;
if let Some(annotation) = &arg.node.annotation {
write!(f, [text(": ")])?;
write!(f, [annotation.format()])?;
}
Ok(())
}
}

View file

@ -0,0 +1,123 @@
use ruff_formatter::prelude::*;
use ruff_formatter::{format_args, write, Format};
use crate::context::ASTFormatContext;
use crate::cst::Arguments;
use crate::shared_traits::AsFormat;
pub struct FormatArguments<'a> {
item: &'a Arguments,
}
impl AsFormat<ASTFormatContext<'_>> for Arguments {
type Format<'a> = FormatArguments<'a>;
fn format(&self) -> Self::Format<'_> {
FormatArguments { item: self }
}
}
impl Format<ASTFormatContext<'_>> for FormatArguments<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
let args = self.item;
let mut first = true;
let defaults_start = args.posonlyargs.len() + args.args.len() - args.defaults.len();
for (i, arg) in args.posonlyargs.iter().chain(&args.args).enumerate() {
if !std::mem::take(&mut first) {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
}
write!(
f,
[group(&format_args![format_with(|f| {
write!(f, [arg.format()])?;
if let Some(i) = i.checked_sub(defaults_start) {
if arg.node.annotation.is_some() {
write!(f, [space()])?;
write!(f, [text("=")])?;
write!(f, [space()])?;
} else {
write!(f, [text("=")])?;
}
write!(f, [args.defaults[i].format()])?;
}
Ok(())
})])]
)?;
if i + 1 == args.posonlyargs.len() {
if !std::mem::take(&mut first) {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
}
write!(f, [text("/")])?;
}
}
if let Some(vararg) = &args.vararg {
if !std::mem::take(&mut first) {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
}
first = false;
write!(f, [text("*")])?;
write!(f, [vararg.format()])?;
} else if !args.kwonlyargs.is_empty() {
if !std::mem::take(&mut first) {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
}
first = false;
write!(f, [text("*")])?;
}
let defaults_start = args.kwonlyargs.len() - args.kw_defaults.len();
for (i, kwarg) in args.kwonlyargs.iter().enumerate() {
if !std::mem::take(&mut first) {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
}
write!(
f,
[group(&format_args![format_with(|f| {
write!(f, [kwarg.format()])?;
if let Some(default) = i
.checked_sub(defaults_start)
.and_then(|i| args.kw_defaults.get(i))
{
if kwarg.node.annotation.is_some() {
write!(f, [space()])?;
write!(f, [text("=")])?;
write!(f, [space()])?;
} else {
write!(f, [text("=")])?;
}
write!(f, [default.format()])?;
}
Ok(())
})])]
)?;
}
if let Some(kwarg) = &args.kwarg {
if !std::mem::take(&mut first) {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
}
write!(f, [text("**")])?;
write!(f, [kwarg.format()])?;
}
if !first {
write!(f, [if_group_breaks(&text(","))])?;
}
Ok(())
}
}

View file

@ -0,0 +1,32 @@
use ruff_formatter::prelude::*;
use ruff_formatter::write;
use crate::context::ASTFormatContext;
use crate::cst::Boolop;
use crate::shared_traits::AsFormat;
pub struct FormatBoolop<'a> {
item: &'a Boolop,
}
impl AsFormat<ASTFormatContext<'_>> for Boolop {
type Format<'a> = FormatBoolop<'a>;
fn format(&self) -> Self::Format<'_> {
FormatBoolop { item: self }
}
}
impl Format<ASTFormatContext<'_>> for FormatBoolop<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext>) -> FormatResult<()> {
let boolop = self.item;
write!(
f,
[text(match boolop {
Boolop::And => "and",
Boolop::Or => "or",
})]
)?;
Ok(())
}
}

View file

@ -0,0 +1,47 @@
use ruff_formatter::prelude::*;
use ruff_formatter::{write, Format};
use ruff_text_size::TextSize;
use crate::context::ASTFormatContext;
use crate::cst::Stmt;
use crate::shared_traits::AsFormat;
#[derive(Copy, Clone)]
pub struct Block<'a> {
body: &'a [Stmt],
}
impl Format<ASTFormatContext<'_>> for Block<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
for (i, stmt) in self.body.iter().enumerate() {
if i > 0 {
write!(f, [hard_line_break()])?;
}
write!(f, [stmt.format()])?;
}
Ok(())
}
}
#[inline]
pub fn block(body: &[Stmt]) -> Block {
Block { body }
}
pub(crate) const fn join_names(names: &[String]) -> JoinNames {
JoinNames { names }
}
pub(crate) struct JoinNames<'a> {
names: &'a [String],
}
impl<Context> Format<Context> for JoinNames<'_> {
fn fmt(&self, f: &mut Formatter<Context>) -> FormatResult<()> {
let mut join = f.join_with(text(", "));
for name in self.names {
join.entry(&dynamic_text(name, TextSize::default()));
}
join.finish()
}
}

View file

@ -0,0 +1,40 @@
use ruff_formatter::prelude::*;
use ruff_formatter::write;
use crate::context::ASTFormatContext;
use crate::cst::Cmpop;
use crate::shared_traits::AsFormat;
pub struct FormatCmpop<'a> {
item: &'a Cmpop,
}
impl AsFormat<ASTFormatContext<'_>> for Cmpop {
type Format<'a> = FormatCmpop<'a>;
fn format(&self) -> Self::Format<'_> {
FormatCmpop { item: self }
}
}
impl Format<ASTFormatContext<'_>> for FormatCmpop<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext>) -> FormatResult<()> {
let unaryop = self.item;
write!(
f,
[text(match unaryop {
Cmpop::Eq => "==",
Cmpop::NotEq => "!=",
Cmpop::Lt => "<",
Cmpop::LtE => "<=",
Cmpop::Gt => ">",
Cmpop::GtE => ">=",
Cmpop::Is => "is",
Cmpop::IsNot => "is not",
Cmpop::In => "in",
Cmpop::NotIn => "not in",
})]
)?;
Ok(())
}
}

View file

@ -0,0 +1,43 @@
use ruff_formatter::prelude::*;
use ruff_formatter::write;
use crate::context::ASTFormatContext;
use crate::cst::Comprehension;
use crate::shared_traits::AsFormat;
pub struct FormatComprehension<'a> {
item: &'a Comprehension,
}
impl AsFormat<ASTFormatContext<'_>> for Comprehension {
type Format<'a> = FormatComprehension<'a>;
fn format(&self) -> Self::Format<'_> {
FormatComprehension { item: self }
}
}
impl Format<ASTFormatContext<'_>> for FormatComprehension<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
let comprehension = self.item;
write!(f, [soft_line_break_or_space()])?;
write!(f, [text("for")])?;
write!(f, [space()])?;
// TODO(charlie): If this is an unparenthesized tuple, we need to avoid expanding it.
// Should this be set on the context?
write!(f, [group(&comprehension.target.format())])?;
write!(f, [space()])?;
write!(f, [text("in")])?;
write!(f, [space()])?;
write!(f, [group(&comprehension.iter.format())])?;
for if_clause in &comprehension.ifs {
write!(f, [soft_line_break_or_space()])?;
write!(f, [text("if")])?;
write!(f, [space()])?;
write!(f, [if_clause.format()])?;
}
Ok(())
}
}

View file

@ -0,0 +1,923 @@
#![allow(unused_variables, clippy::too_many_arguments)]
use ruff_formatter::prelude::*;
use ruff_formatter::{format_args, write};
use ruff_text_size::TextSize;
use rustpython_parser::ast::Constant;
use crate::builders::literal;
use crate::context::ASTFormatContext;
use crate::core::types::Range;
use crate::cst::{
Arguments, Boolop, Cmpop, Comprehension, Expr, ExprKind, Keyword, Operator, Unaryop,
};
use crate::format::helpers::{is_self_closing, is_simple_power, is_simple_slice};
use crate::shared_traits::AsFormat;
use crate::trivia::{Parenthesize, Relationship, TriviaKind};
pub struct FormatExpr<'a> {
item: &'a Expr,
}
fn format_starred(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
value: &Expr,
) -> FormatResult<()> {
write!(f, [text("*"), value.format()])?;
// Apply any inline comments.
let mut first = true;
for range in expr.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
Ok(())
}
fn format_name(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
_id: &str,
) -> FormatResult<()> {
write!(f, [literal(Range::from_located(expr))])?;
// Apply any inline comments.
let mut first = true;
for range in expr.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
Ok(())
}
fn format_subscript(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
value: &Expr,
slice: &Expr,
) -> FormatResult<()> {
write!(
f,
[
value.format(),
text("["),
group(&format_args![soft_block_indent(&slice.format())]),
text("]")
]
)?;
Ok(())
}
fn format_tuple(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
elts: &[Expr],
) -> FormatResult<()> {
// If we're already parenthesized, avoid adding any "mandatory" parentheses.
// TODO(charlie): We also need to parenthesize tuples on the right-hand side of an
// assignment if the target is exploded. And sometimes the tuple gets exploded, like
// if the LHS is an exploded list? Lots of edge cases here.
if elts.len() == 1 {
write!(
f,
[group(&format_args![soft_block_indent(&format_with(|f| {
write!(f, [elts[0].format()])?;
write!(f, [text(",")])?;
Ok(())
}))])]
)?;
} else if !elts.is_empty() {
write!(
f,
[group(&format_with(|f| {
if matches!(expr.parentheses, Parenthesize::IfExpanded) {
write!(f, [if_group_breaks(&text("("))])?;
}
if matches!(
expr.parentheses,
Parenthesize::IfExpanded | Parenthesize::Always
) {
write!(
f,
[soft_block_indent(&format_with(|f| {
// TODO(charlie): If the magic trailing comma isn't present, and the
// tuple is _already_ expanded, we're not supposed to add this.
let magic_trailing_comma = expr
.trivia
.iter()
.any(|c| matches!(c.kind, TriviaKind::MagicTrailingComma));
if magic_trailing_comma {
write!(f, [expand_parent()])?;
}
for (i, elt) in elts.iter().enumerate() {
write!(f, [elt.format()])?;
if i < elts.len() - 1 {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
} else {
if magic_trailing_comma {
write!(f, [if_group_breaks(&text(","))])?;
}
}
}
Ok(())
}))]
)?;
} else {
let magic_trailing_comma = expr
.trivia
.iter()
.any(|c| matches!(c.kind, TriviaKind::MagicTrailingComma));
if magic_trailing_comma {
write!(f, [expand_parent()])?;
}
for (i, elt) in elts.iter().enumerate() {
write!(f, [elt.format()])?;
if i < elts.len() - 1 {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
} else {
if magic_trailing_comma {
write!(f, [if_group_breaks(&text(","))])?;
}
}
}
}
if matches!(expr.parentheses, Parenthesize::IfExpanded) {
write!(f, [if_group_breaks(&text(")"))])?;
}
Ok(())
}))]
)?;
}
Ok(())
}
fn format_slice(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
lower: Option<&Expr>,
upper: Option<&Expr>,
step: Option<&Expr>,
) -> FormatResult<()> {
// https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#slices
let is_simple = lower.map_or(true, is_simple_slice)
&& upper.map_or(true, is_simple_slice)
&& step.map_or(true, is_simple_slice);
if let Some(lower) = lower {
write!(f, [lower.format()])?;
if !is_simple {
write!(f, [space()])?;
}
}
write!(f, [text(":")])?;
if let Some(upper) = upper {
if !is_simple {
write!(f, [space()])?;
}
write!(f, [upper.format()])?;
if !is_simple && step.is_some() {
write!(f, [space()])?;
}
}
if let Some(step) = step {
if !is_simple && upper.is_some() {
write!(f, [space()])?;
}
write!(f, [text(":")])?;
if !is_simple {
write!(f, [space()])?;
}
write!(f, [step.format()])?;
}
Ok(())
}
fn format_list(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
elts: &[Expr],
) -> FormatResult<()> {
write!(f, [text("[")])?;
if !elts.is_empty() {
let magic_trailing_comma = expr
.trivia
.iter()
.any(|c| matches!(c.kind, TriviaKind::MagicTrailingComma));
write!(
f,
[group(&format_args![soft_block_indent(&format_with(|f| {
if magic_trailing_comma {
write!(f, [expand_parent()])?;
}
for (i, elt) in elts.iter().enumerate() {
write!(f, [elt.format()])?;
if i < elts.len() - 1 {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
} else {
write!(f, [if_group_breaks(&text(","))])?;
}
}
Ok(())
}))])]
)?;
}
write!(f, [text("]")])?;
Ok(())
}
fn format_set(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
elts: &[Expr],
) -> FormatResult<()> {
if elts.is_empty() {
write!(f, [text("set()")])?;
Ok(())
} else {
write!(f, [text("{")])?;
if !elts.is_empty() {
let magic_trailing_comma = expr
.trivia
.iter()
.any(|c| matches!(c.kind, TriviaKind::MagicTrailingComma));
write!(
f,
[group(&format_args![soft_block_indent(&format_with(|f| {
if magic_trailing_comma {
write!(f, [expand_parent()])?;
}
for (i, elt) in elts.iter().enumerate() {
write!(f, [group(&format_args![elt.format()])])?;
if i < elts.len() - 1 {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
} else {
write!(f, [if_group_breaks(&text(","))])?;
}
}
Ok(())
}))])]
)?;
}
write!(f, [text("}")])?;
Ok(())
}
}
fn format_call(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
func: &Expr,
args: &[Expr],
keywords: &[Keyword],
) -> FormatResult<()> {
write!(f, [func.format()])?;
if args.is_empty() && keywords.is_empty() {
write!(f, [text("(")])?;
write!(f, [text(")")])?;
} else {
write!(f, [text("(")])?;
let magic_trailing_comma = expr
.trivia
.iter()
.any(|c| matches!(c.kind, TriviaKind::MagicTrailingComma));
write!(
f,
[group(&format_args![soft_block_indent(&format_with(|f| {
if magic_trailing_comma {
write!(f, [expand_parent()])?;
}
for (i, arg) in args.iter().enumerate() {
write!(f, [group(&format_args![arg.format()])])?;
if i < args.len() - 1 || !keywords.is_empty() {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
} else {
if magic_trailing_comma || (args.len() + keywords.len() > 1) {
write!(f, [if_group_breaks(&text(","))])?;
}
}
}
for (i, keyword) in keywords.iter().enumerate() {
write!(
f,
[group(&format_args![&format_with(|f| {
if let Some(arg) = &keyword.node.arg {
write!(f, [dynamic_text(arg, TextSize::default())])?;
write!(f, [text("=")])?;
write!(f, [keyword.node.value.format()])?;
} else {
write!(f, [text("**")])?;
write!(f, [keyword.node.value.format()])?;
}
Ok(())
})])]
)?;
if i < keywords.len() - 1 {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
} else {
if magic_trailing_comma || (args.len() + keywords.len() > 1) {
write!(f, [if_group_breaks(&text(","))])?;
}
}
}
// Apply any dangling trailing comments.
for trivia in &expr.trivia {
if matches!(trivia.relationship, Relationship::Dangling) {
if let TriviaKind::StandaloneComment(range) = trivia.kind {
write!(f, [expand_parent()])?;
write!(f, [hard_line_break()])?;
write!(f, [literal(range)])?;
}
}
}
Ok(())
}))])]
)?;
write!(f, [text(")")])?;
}
Ok(())
}
fn format_list_comp(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
elt: &Expr,
generators: &[Comprehension],
) -> FormatResult<()> {
write!(f, [text("[")])?;
write!(
f,
[group(&format_args![soft_block_indent(&format_with(|f| {
write!(f, [elt.format()])?;
for generator in generators {
write!(f, [generator.format()])?;
}
Ok(())
}))])]
)?;
write!(f, [text("]")])?;
Ok(())
}
fn format_set_comp(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
elt: &Expr,
generators: &[Comprehension],
) -> FormatResult<()> {
write!(f, [text("{")])?;
write!(
f,
[group(&format_args![soft_block_indent(&format_with(|f| {
write!(f, [elt.format()])?;
for generator in generators {
write!(f, [generator.format()])?;
}
Ok(())
}))])]
)?;
write!(f, [text("}")])?;
Ok(())
}
fn format_dict_comp(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
key: &Expr,
value: &Expr,
generators: &[Comprehension],
) -> FormatResult<()> {
write!(f, [text("{")])?;
write!(
f,
[group(&format_args![soft_block_indent(&format_with(|f| {
write!(f, [key.format()])?;
write!(f, [text(":")])?;
write!(f, [space()])?;
write!(f, [value.format()])?;
for generator in generators {
write!(f, [generator.format()])?;
}
Ok(())
}))])]
)?;
write!(f, [text("}")])?;
Ok(())
}
fn format_generator_exp(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
elt: &Expr,
generators: &[Comprehension],
) -> FormatResult<()> {
write!(
f,
[group(&format_args![soft_block_indent(&format_with(|f| {
write!(f, [elt.format()])?;
for generator in generators {
write!(f, [generator.format()])?;
}
Ok(())
}))])]
)?;
Ok(())
}
fn format_await(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
value: &Expr,
) -> FormatResult<()> {
write!(f, [text("await")])?;
write!(f, [space()])?;
if is_self_closing(value) {
write!(f, [group(&format_args![value.format()])])?;
} else {
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_args![value.format()]),
if_group_breaks(&text(")")),
])]
)?;
}
Ok(())
}
fn format_yield(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
value: Option<&Expr>,
) -> FormatResult<()> {
write!(f, [text("yield")])?;
if let Some(value) = value {
write!(f, [space()])?;
if is_self_closing(value) {
write!(f, [group(&format_args![value.format()])])?;
} else {
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_args![value.format()]),
if_group_breaks(&text(")")),
])]
)?;
}
}
Ok(())
}
fn format_yield_from(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
value: &Expr,
) -> FormatResult<()> {
write!(
f,
[group(&format_args![soft_block_indent(&format_with(|f| {
write!(f, [text("yield")])?;
write!(f, [space()])?;
write!(f, [text("from")])?;
write!(f, [space()])?;
if is_self_closing(value) {
write!(f, [value.format()])?;
} else {
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_args![value.format()]),
if_group_breaks(&text(")")),
])]
)?;
}
Ok(())
})),])]
)?;
Ok(())
}
fn format_compare(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
left: &Expr,
ops: &[Cmpop],
comparators: &[Expr],
) -> FormatResult<()> {
write!(f, [group(&format_args![left.format()])])?;
for (i, op) in ops.iter().enumerate() {
write!(f, [soft_line_break_or_space()])?;
write!(f, [op.format()])?;
write!(f, [space()])?;
write!(f, [group(&format_args![comparators[i].format()])])?;
}
Ok(())
}
fn format_joined_str(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
_values: &[Expr],
) -> FormatResult<()> {
write!(f, [literal(Range::from_located(expr))])?;
Ok(())
}
fn format_constant(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
_constant: &Constant,
_kind: Option<&str>,
) -> FormatResult<()> {
write!(f, [literal(Range::from_located(expr))])?;
Ok(())
}
fn format_dict(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
keys: &[Option<Expr>],
values: &[Expr],
) -> FormatResult<()> {
write!(f, [text("{")])?;
if !keys.is_empty() {
let magic_trailing_comma = expr
.trivia
.iter()
.any(|c| matches!(c.kind, TriviaKind::MagicTrailingComma));
write!(
f,
[soft_block_indent(&format_with(|f| {
if magic_trailing_comma {
write!(f, [expand_parent()])?;
}
for (i, (k, v)) in keys.iter().zip(values).enumerate() {
if let Some(k) = k {
write!(f, [k.format()])?;
write!(f, [text(":")])?;
write!(f, [space()])?;
if is_self_closing(v) {
write!(f, [v.format()])?;
} else {
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_args![v.format()]),
if_group_breaks(&text(")")),
])]
)?;
}
} else {
write!(f, [text("**")])?;
if is_self_closing(v) {
write!(f, [v.format()])?;
} else {
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_args![v.format()]),
if_group_breaks(&text(")")),
])]
)?;
}
}
if i < keys.len() - 1 {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
} else {
write!(f, [if_group_breaks(&text(","))])?;
}
}
Ok(())
}))]
)?;
}
write!(f, [text("}")])?;
Ok(())
}
fn format_attribute(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
value: &Expr,
attr: &str,
) -> FormatResult<()> {
write!(f, [value.format()])?;
write!(f, [text(".")])?;
write!(f, [dynamic_text(attr, TextSize::default())])?;
// Apply any inline comments.
let mut first = true;
for range in expr.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
Ok(())
}
fn format_bool_op(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
op: &Boolop,
values: &[Expr],
) -> FormatResult<()> {
let mut first = true;
for value in values {
if std::mem::take(&mut first) {
write!(f, [group(&format_args![value.format()])])?;
} else {
write!(f, [soft_line_break_or_space()])?;
write!(f, [op.format()])?;
write!(f, [space()])?;
write!(f, [group(&format_args![value.format()])])?;
}
}
// Apply any inline comments.
let mut first = true;
for range in expr.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
Ok(())
}
fn format_bin_op(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
left: &Expr,
op: &Operator,
right: &Expr,
) -> FormatResult<()> {
// https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-breaks-binary-operators
let is_simple =
matches!(op, Operator::Pow) && (is_simple_power(left) && is_simple_power(right));
write!(f, [left.format()])?;
if !is_simple {
write!(f, [soft_line_break_or_space()])?;
}
write!(f, [op.format()])?;
if !is_simple {
write!(f, [space()])?;
}
write!(f, [group(&format_args![right.format()])])?;
// Apply any inline comments.
let mut first = true;
for range in expr.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
Ok(())
}
fn format_unary_op(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
op: &Unaryop,
operand: &Expr,
) -> FormatResult<()> {
write!(f, [op.format()])?;
// TODO(charlie): Do this in the normalization pass.
if !matches!(op, Unaryop::Not)
&& matches!(
operand.node,
ExprKind::BoolOp { .. } | ExprKind::Compare { .. } | ExprKind::BinOp { .. }
)
{
let parenthesized = matches!(operand.parentheses, Parenthesize::Always);
if !parenthesized {
write!(f, [text("(")])?;
}
write!(f, [operand.format()])?;
if !parenthesized {
write!(f, [text(")")])?;
}
} else {
write!(f, [operand.format()])?;
}
Ok(())
}
fn format_lambda(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
args: &Arguments,
body: &Expr,
) -> FormatResult<()> {
write!(f, [text("lambda")])?;
if !args.args.is_empty() {
write!(f, [space()])?;
write!(f, [args.format()])?;
}
write!(f, [text(":")])?;
write!(f, [space()])?;
write!(f, [body.format()])?;
Ok(())
}
fn format_if_exp(
f: &mut Formatter<ASTFormatContext<'_>>,
expr: &Expr,
test: &Expr,
body: &Expr,
orelse: &Expr,
) -> FormatResult<()> {
write!(f, [group(&format_args![body.format()])])?;
write!(f, [soft_line_break_or_space()])?;
write!(f, [text("if")])?;
write!(f, [space()])?;
write!(f, [group(&format_args![test.format()])])?;
write!(f, [soft_line_break_or_space()])?;
write!(f, [text("else")])?;
write!(f, [space()])?;
write!(f, [group(&format_args![orelse.format()])])?;
Ok(())
}
impl Format<ASTFormatContext<'_>> for FormatExpr<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
if matches!(self.item.parentheses, Parenthesize::Always) {
write!(f, [text("(")])?;
}
// Any leading comments come on the line before.
for trivia in &self.item.trivia {
if matches!(trivia.relationship, Relationship::Leading) {
if let TriviaKind::StandaloneComment(range) = trivia.kind {
write!(f, [expand_parent()])?;
write!(f, [literal(range)])?;
write!(f, [hard_line_break()])?;
}
}
}
match &self.item.node {
ExprKind::BoolOp { op, values } => format_bool_op(f, self.item, op, values),
// ExprKind::NamedExpr { .. } => {}
ExprKind::BinOp { left, op, right } => format_bin_op(f, self.item, left, op, right),
ExprKind::UnaryOp { op, operand } => format_unary_op(f, self.item, op, operand),
ExprKind::Lambda { args, body } => format_lambda(f, self.item, args, body),
ExprKind::IfExp { test, body, orelse } => {
format_if_exp(f, self.item, test, body, orelse)
}
ExprKind::Dict { keys, values } => format_dict(f, self.item, keys, values),
ExprKind::Set { elts, .. } => format_set(f, self.item, elts),
ExprKind::ListComp { elt, generators } => {
format_list_comp(f, self.item, elt, generators)
}
ExprKind::SetComp { elt, generators } => format_set_comp(f, self.item, elt, generators),
ExprKind::DictComp {
key,
value,
generators,
} => format_dict_comp(f, self.item, key, value, generators),
ExprKind::GeneratorExp { elt, generators } => {
format_generator_exp(f, self.item, elt, generators)
}
ExprKind::Await { value } => format_await(f, self.item, value),
ExprKind::Yield { value } => format_yield(f, self.item, value.as_deref()),
ExprKind::YieldFrom { value } => format_yield_from(f, self.item, value),
ExprKind::Compare {
left,
ops,
comparators,
} => format_compare(f, self.item, left, ops, comparators),
ExprKind::Call {
func,
args,
keywords,
} => format_call(f, self.item, func, args, keywords),
// ExprKind::FormattedValue { .. } => {}
ExprKind::JoinedStr { values } => format_joined_str(f, self.item, values),
ExprKind::Constant { value, kind } => {
format_constant(f, self.item, value, kind.as_deref())
}
ExprKind::Attribute { value, attr, .. } => format_attribute(f, self.item, value, attr),
ExprKind::Subscript { value, slice, .. } => {
format_subscript(f, self.item, value, slice)
}
ExprKind::Starred { value, .. } => format_starred(f, self.item, value),
ExprKind::Name { id, .. } => format_name(f, self.item, id),
ExprKind::List { elts, .. } => format_list(f, self.item, elts),
ExprKind::Tuple { elts, .. } => format_tuple(f, self.item, elts),
ExprKind::Slice { lower, upper, step } => format_slice(
f,
self.item,
lower.as_deref(),
upper.as_deref(),
step.as_deref(),
),
_ => {
unimplemented!("Implement ExprKind: {:?}", self.item.node)
}
}?;
// Any trailing comments come on the lines after.
for trivia in &self.item.trivia {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::StandaloneComment(range) = trivia.kind {
write!(f, [expand_parent()])?;
write!(f, [literal(range)])?;
write!(f, [hard_line_break()])?;
}
}
}
if matches!(self.item.parentheses, Parenthesize::Always) {
write!(f, [text(")")])?;
}
Ok(())
}
}
impl AsFormat<ASTFormatContext<'_>> for Expr {
type Format<'a> = FormatExpr<'a>;
fn format(&self) -> Self::Format<'_> {
FormatExpr { item: self }
}
}

View file

@ -0,0 +1,87 @@
use crate::cst::{Expr, ExprKind, Unaryop};
pub fn is_self_closing(expr: &Expr) -> bool {
match &expr.node {
ExprKind::Tuple { .. }
| ExprKind::List { .. }
| ExprKind::Set { .. }
| ExprKind::Dict { .. }
| ExprKind::ListComp { .. }
| ExprKind::SetComp { .. }
| ExprKind::DictComp { .. }
| ExprKind::GeneratorExp { .. }
| ExprKind::Call { .. }
| ExprKind::Name { .. }
| ExprKind::Constant { .. }
| ExprKind::Subscript { .. } => true,
ExprKind::Lambda { body, .. } => is_self_closing(body),
ExprKind::BinOp { left, right, .. } => {
matches!(left.node, ExprKind::Constant { .. } | ExprKind::Name { .. })
&& matches!(
right.node,
ExprKind::Tuple { .. }
| ExprKind::List { .. }
| ExprKind::Set { .. }
| ExprKind::Dict { .. }
| ExprKind::ListComp { .. }
| ExprKind::SetComp { .. }
| ExprKind::DictComp { .. }
| ExprKind::GeneratorExp { .. }
| ExprKind::Call { .. }
| ExprKind::Subscript { .. }
)
}
ExprKind::BoolOp { values, .. } => values.last().map_or(false, |expr| {
matches!(
expr.node,
ExprKind::Tuple { .. }
| ExprKind::List { .. }
| ExprKind::Set { .. }
| ExprKind::Dict { .. }
| ExprKind::ListComp { .. }
| ExprKind::SetComp { .. }
| ExprKind::DictComp { .. }
| ExprKind::GeneratorExp { .. }
| ExprKind::Call { .. }
| ExprKind::Subscript { .. }
)
}),
ExprKind::UnaryOp { operand, .. } => is_self_closing(operand),
_ => false,
}
}
/// Return `true` if an [`Expr`] adheres to Black's definition of a non-complex
/// expression, in the context of a slice operation.
pub fn is_simple_slice(expr: &Expr) -> bool {
match &expr.node {
ExprKind::UnaryOp { op, operand } => {
if matches!(op, Unaryop::Not) {
false
} else {
is_simple_slice(operand)
}
}
ExprKind::Constant { .. } => true,
ExprKind::Name { .. } => true,
_ => false,
}
}
/// Return `true` if an [`Expr`] adheres to Black's definition of a non-complex
/// expression, in the context of a power operation.
pub fn is_simple_power(expr: &Expr) -> bool {
match &expr.node {
ExprKind::UnaryOp { op, operand } => {
if matches!(op, Unaryop::Not) {
false
} else {
is_simple_slice(operand)
}
}
ExprKind::Constant { .. } => true,
ExprKind::Name { .. } => true,
ExprKind::Attribute { .. } => true,
_ => false,
}
}

View file

@ -0,0 +1,13 @@
mod alias;
mod arg;
mod arguments;
mod boolop;
pub mod builders;
mod cmpop;
mod comprehension;
mod expr;
mod helpers;
mod operator;
mod stmt;
mod unaryop;
mod withitem;

View file

@ -0,0 +1,45 @@
use ruff_formatter::prelude::*;
use ruff_formatter::write;
use crate::context::ASTFormatContext;
use crate::cst::Operator;
use crate::shared_traits::AsFormat;
pub struct FormatOperator<'a> {
item: &'a Operator,
}
impl AsFormat<ASTFormatContext<'_>> for Operator {
type Format<'a> = FormatOperator<'a>;
fn format(&self) -> Self::Format<'_> {
FormatOperator { item: self }
}
}
impl Format<ASTFormatContext<'_>> for FormatOperator<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
let operator = self.item;
write!(
f,
[text(match operator {
Operator::Add => "+",
Operator::Sub => "-",
Operator::Mult => "*",
Operator::MatMult => "@",
Operator::Div => "/",
Operator::Mod => "%",
Operator::Pow => "**",
Operator::LShift => "<<",
Operator::RShift => ">>",
Operator::BitOr => "|",
Operator::BitXor => "^",
Operator::BitAnd => "&",
Operator::FloorDiv => "//",
})]
)?;
Ok(())
}
}

View file

@ -0,0 +1,829 @@
#![allow(unused_variables, clippy::too_many_arguments)]
use ruff_formatter::prelude::*;
use ruff_formatter::{format_args, write};
use ruff_text_size::TextSize;
use crate::builders::literal;
use crate::context::ASTFormatContext;
use crate::cst::{Alias, Arguments, Expr, ExprKind, Keyword, Stmt, StmtKind, Withitem};
use crate::format::builders::{block, join_names};
use crate::format::helpers::is_self_closing;
use crate::shared_traits::AsFormat;
use crate::trivia::{Parenthesize, Relationship, TriviaKind};
fn format_break(f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
write!(f, [text("break")])
}
fn format_pass(f: &mut Formatter<ASTFormatContext<'_>>, stmt: &Stmt) -> FormatResult<()> {
// Write the statement body.
write!(f, [text("pass")])?;
// Apply any inline comments.
let mut first = true;
for range in stmt.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
Ok(())
}
fn format_continue(f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
write!(f, [text("continue")])
}
fn format_global(f: &mut Formatter<ASTFormatContext<'_>>, names: &[String]) -> FormatResult<()> {
write!(f, [text("global")])?;
if !names.is_empty() {
write!(f, [space(), join_names(names)])?;
}
Ok(())
}
fn format_nonlocal(f: &mut Formatter<ASTFormatContext<'_>>, names: &[String]) -> FormatResult<()> {
write!(f, [text("nonlocal")])?;
if !names.is_empty() {
write!(f, [space(), join_names(names)])?;
}
Ok(())
}
fn format_delete(f: &mut Formatter<ASTFormatContext<'_>>, targets: &[Expr]) -> FormatResult<()> {
write!(f, [text("del")])?;
match targets.len() {
0 => Ok(()),
1 => write!(f, [space(), targets[0].format()]),
_ => {
write!(
f,
[
space(),
group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_with(|f| {
for (i, target) in targets.iter().enumerate() {
write!(f, [target.format()])?;
if i < targets.len() - 1 {
write!(f, [text(","), soft_line_break_or_space()])?;
} else {
write!(f, [if_group_breaks(&text(","))])?;
}
}
Ok(())
})),
if_group_breaks(&text(")")),
])
]
)
}
}
}
fn format_class_def(
f: &mut Formatter<ASTFormatContext<'_>>,
name: &str,
bases: &[Expr],
keywords: &[Keyword],
body: &[Stmt],
decorator_list: &[Expr],
) -> FormatResult<()> {
for decorator in decorator_list {
write!(f, [text("@"), decorator.format(), hard_line_break()])?;
}
write!(
f,
[
text("class"),
space(),
dynamic_text(name, TextSize::default())
]
)?;
if !bases.is_empty() || !keywords.is_empty() {
let format_bases = format_with(|f| {
for (i, expr) in bases.iter().enumerate() {
write!(f, [expr.format()])?;
if i < bases.len() - 1 || !keywords.is_empty() {
write!(f, [text(","), soft_line_break_or_space()])?;
} else {
write!(f, [if_group_breaks(&text(","))])?;
}
for (i, keyword) in keywords.iter().enumerate() {
if let Some(arg) = &keyword.node.arg {
write!(
f,
[
dynamic_text(arg, TextSize::default()),
text("="),
keyword.node.value.format()
]
)?;
} else {
write!(f, [text("**"), keyword.node.value.format()])?;
}
if i < keywords.len() - 1 {
write!(f, [text(","), soft_line_break_or_space()])?;
} else {
write!(f, [if_group_breaks(&text(","))])?;
}
}
}
Ok(())
});
write!(
f,
[
text("("),
group(&soft_block_indent(&format_bases)),
text(")")
]
)?;
}
write!(f, [text(":"), block_indent(&block(body))])
}
fn format_func_def(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
name: &str,
args: &Arguments,
returns: Option<&Expr>,
body: &[Stmt],
decorator_list: &[Expr],
async_: bool,
) -> FormatResult<()> {
for decorator in decorator_list {
write!(f, [text("@"), decorator.format(), hard_line_break()])?;
}
if async_ {
write!(f, [text("async"), space()])?;
}
write!(
f,
[
text("def"),
space(),
dynamic_text(name, TextSize::default()),
text("("),
group(&soft_block_indent(&format_with(|f| {
if stmt
.trivia
.iter()
.any(|c| matches!(c.kind, TriviaKind::MagicTrailingComma))
{
write!(f, [expand_parent()])?;
}
write!(f, [args.format()])
}))),
text(")")
]
)?;
if let Some(returns) = returns {
write!(f, [text(" -> "), returns.format()])?;
}
write!(f, [text(":")])?;
// Apply any inline comments.
let mut first = true;
for range in stmt.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
write!(f, [block_indent(&format_args![block(body)])])
}
fn format_assign(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
targets: &[Expr],
value: &Expr,
) -> FormatResult<()> {
write!(f, [targets[0].format()])?;
for target in &targets[1..] {
// TODO(charlie): This doesn't match Black's behavior. We need to parenthesize
// this expression sometimes.
write!(f, [text(" = "), target.format()])?;
}
write!(f, [text(" = ")])?;
if is_self_closing(value) {
write!(f, [group(&value.format())])?;
} else {
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&value.format()),
if_group_breaks(&text(")")),
])]
)?;
}
// Apply any inline comments.
let mut first = true;
for range in stmt.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
Ok(())
}
fn format_ann_assign(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
target: &Expr,
annotation: &Expr,
value: Option<&Expr>,
simple: usize,
) -> FormatResult<()> {
let need_parens = matches!(target.node, ExprKind::Name { .. }) && simple == 0;
if need_parens {
write!(f, [text("(")])?;
}
write!(f, [target.format()])?;
if need_parens {
write!(f, [text(")")])?;
}
write!(f, [text(": "), annotation.format()])?;
if let Some(value) = value {
write!(
f,
[
space(),
text("="),
space(),
group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&value.format()),
if_group_breaks(&text(")")),
])
]
)?;
}
Ok(())
}
fn format_for(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
target: &Expr,
iter: &Expr,
body: &[Stmt],
_orelse: &[Stmt],
_type_comment: Option<&str>,
) -> FormatResult<()> {
write!(
f,
[
text("for"),
space(),
group(&target.format()),
space(),
text("in"),
space(),
group(&iter.format()),
text(":"),
block_indent(&block(body))
]
)
}
fn format_while(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
test: &Expr,
body: &[Stmt],
orelse: &[Stmt],
) -> FormatResult<()> {
write!(f, [text("while"), space()])?;
if is_self_closing(test) {
write!(f, [test.format()])?;
} else {
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&test.format()),
if_group_breaks(&text(")")),
])]
)?;
}
write!(f, [text(":"), block_indent(&block(body))])?;
if !orelse.is_empty() {
write!(f, [text("else:"), block_indent(&block(orelse))])?;
}
Ok(())
}
fn format_if(
f: &mut Formatter<ASTFormatContext<'_>>,
test: &Expr,
body: &[Stmt],
orelse: &[Stmt],
) -> FormatResult<()> {
write!(f, [text("if"), space()])?;
if is_self_closing(test) {
write!(f, [test.format()])?;
} else {
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&test.format()),
if_group_breaks(&text(")")),
])]
)?;
}
write!(f, [text(":"), block_indent(&block(body))])?;
if !orelse.is_empty() {
if orelse.len() == 1 {
if let StmtKind::If { test, body, orelse } = &orelse[0].node {
write!(f, [text("el")])?;
format_if(f, test, body, orelse)?;
} else {
write!(f, [text("else:"), block_indent(&block(orelse))])?;
}
} else {
write!(f, [text("else:"), block_indent(&block(orelse))])?;
}
}
Ok(())
}
fn format_raise(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
exc: Option<&Expr>,
cause: Option<&Expr>,
) -> FormatResult<()> {
write!(f, [text("raise")])?;
if let Some(exc) = exc {
write!(f, [space(), exc.format()])?;
if let Some(cause) = cause {
write!(f, [space(), text("from"), space(), cause.format()])?;
}
}
Ok(())
}
fn format_return(
f: &mut Formatter<ASTFormatContext<'_>>,
value: Option<&Expr>,
) -> FormatResult<()> {
write!(f, [text("return")])?;
if let Some(value) = value {
write!(f, [space(), value.format()])?;
}
Ok(())
}
fn format_assert(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
test: &Expr,
msg: Option<&Expr>,
) -> FormatResult<()> {
write!(f, [text("assert"), space()])?;
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&test.format()),
if_group_breaks(&text(")")),
])]
)?;
if let Some(msg) = msg {
write!(
f,
[
text(","),
space(),
group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&msg.format()),
if_group_breaks(&text(")")),
])
]
)?;
}
Ok(())
}
fn format_import(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
names: &[Alias],
) -> FormatResult<()> {
write!(
f,
[
text("import"),
space(),
group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_with(|f| {
for (i, name) in names.iter().enumerate() {
write!(f, [name.format()])?;
if i < names.len() - 1 {
write!(f, [text(","), soft_line_break_or_space()])?;
} else {
write!(f, [if_group_breaks(&text(","))])?;
}
}
Ok(())
})),
if_group_breaks(&text(")")),
])
]
)
}
fn format_import_from(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
module: Option<&str>,
names: &[Alias],
level: Option<&usize>,
) -> FormatResult<()> {
write!(f, [text("from")])?;
write!(f, [space()])?;
if let Some(level) = level {
for _ in 0..*level {
write!(f, [text(".")])?;
}
}
if let Some(module) = module {
write!(f, [dynamic_text(module, TextSize::default())])?;
}
write!(f, [space()])?;
write!(f, [text("import")])?;
write!(f, [space()])?;
if names.iter().any(|name| name.node.name == "*") {
write!(f, [text("*")])?;
} else {
let magic_trailing_comma = stmt
.trivia
.iter()
.any(|c| matches!(c.kind, TriviaKind::MagicTrailingComma));
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_with(|f| {
if magic_trailing_comma {
write!(f, [expand_parent()])?;
}
for (i, name) in names.iter().enumerate() {
write!(f, [name.format()])?;
if i < names.len() - 1 {
write!(f, [text(",")])?;
write!(f, [soft_line_break_or_space()])?;
} else {
write!(f, [if_group_breaks(&text(","))])?;
}
}
Ok(())
})),
if_group_breaks(&text(")")),
])]
)?;
}
// Apply any inline comments.
let mut first = true;
for range in stmt.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
Ok(())
}
fn format_expr(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
expr: &Expr,
) -> FormatResult<()> {
if matches!(stmt.parentheses, Parenthesize::Always) {
write!(
f,
[group(&format_args![
text("("),
soft_block_indent(&format_args![expr.format()]),
text(")"),
])]
)?;
} else if is_self_closing(expr) {
write!(f, [group(&format_args![expr.format()])])?;
} else {
write!(
f,
[group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_args![expr.format()]),
if_group_breaks(&text(")")),
])]
)?;
}
// Apply any inline comments.
let mut first = true;
for range in stmt.trivia.iter().filter_map(|trivia| {
if matches!(trivia.relationship, Relationship::Trailing) {
if let TriviaKind::InlineComment(range) = trivia.kind {
Some(range)
} else {
None
}
} else {
None
}
}) {
if std::mem::take(&mut first) {
write!(f, [text(" ")])?;
}
write!(f, [literal(range)])?;
}
Ok(())
}
fn format_with_(
f: &mut Formatter<ASTFormatContext<'_>>,
stmt: &Stmt,
items: &[Withitem],
body: &[Stmt],
type_comment: Option<&str>,
async_: bool,
) -> FormatResult<()> {
if async_ {
write!(f, [text("async"), space()])?;
}
write!(
f,
[
text("with"),
space(),
group(&format_args![
if_group_breaks(&text("(")),
soft_block_indent(&format_with(|f| {
for (i, item) in items.iter().enumerate() {
write!(f, [item.format()])?;
if i < items.len() - 1 {
write!(f, [text(","), soft_line_break_or_space()])?;
} else {
write!(f, [if_group_breaks(&text(","))])?;
}
}
Ok(())
})),
if_group_breaks(&text(")")),
]),
text(":"),
block_indent(&block(body))
]
)
}
pub struct FormatStmt<'a> {
item: &'a Stmt,
}
impl Format<ASTFormatContext<'_>> for FormatStmt<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
// Any leading comments come on the line before.
for trivia in &self.item.trivia {
if matches!(trivia.relationship, Relationship::Leading) {
match trivia.kind {
TriviaKind::EmptyLine => {
write!(f, [empty_line()])?;
}
TriviaKind::StandaloneComment(range) => {
write!(f, [literal(range), hard_line_break()])?;
}
_ => {}
}
}
}
match &self.item.node {
StmtKind::Pass => format_pass(f, self.item),
StmtKind::Break => format_break(f),
StmtKind::Continue => format_continue(f),
StmtKind::Global { names } => format_global(f, names),
StmtKind::Nonlocal { names } => format_nonlocal(f, names),
StmtKind::FunctionDef {
name,
args,
body,
decorator_list,
returns,
..
} => format_func_def(
f,
self.item,
name,
args,
returns.as_deref(),
body,
decorator_list,
false,
),
StmtKind::AsyncFunctionDef {
name,
args,
body,
decorator_list,
returns,
..
} => format_func_def(
f,
self.item,
name,
args,
returns.as_deref(),
body,
decorator_list,
true,
),
StmtKind::ClassDef {
name,
bases,
keywords,
body,
decorator_list,
} => format_class_def(f, name, bases, keywords, body, decorator_list),
StmtKind::Return { value } => format_return(f, value.as_ref()),
StmtKind::Delete { targets } => format_delete(f, targets),
StmtKind::Assign { targets, value, .. } => format_assign(f, self.item, targets, value),
// StmtKind::AugAssign { .. } => {}
StmtKind::AnnAssign {
target,
annotation,
value,
simple,
} => format_ann_assign(f, self.item, target, annotation, value.as_deref(), *simple),
StmtKind::For {
target,
iter,
body,
orelse,
type_comment,
} => format_for(
f,
self.item,
target,
iter,
body,
orelse,
type_comment.as_deref(),
),
// StmtKind::AsyncFor { .. } => {}
StmtKind::While { test, body, orelse } => {
format_while(f, self.item, test, body, orelse)
}
StmtKind::If { test, body, orelse } => format_if(f, test, body, orelse),
StmtKind::With {
items,
body,
type_comment,
} => format_with_(
f,
self.item,
items,
body,
type_comment.as_ref().map(String::as_str),
false,
),
StmtKind::AsyncWith {
items,
body,
type_comment,
} => format_with_(
f,
self.item,
items,
body,
type_comment.as_ref().map(String::as_str),
true,
),
// StmtKind::Match { .. } => {}
StmtKind::Raise { exc, cause } => {
format_raise(f, self.item, exc.as_deref(), cause.as_deref())
}
// StmtKind::Try { .. } => {}
StmtKind::Assert { test, msg } => {
format_assert(f, self.item, test, msg.as_ref().map(|expr| &**expr))
}
StmtKind::Import { names } => format_import(f, self.item, names),
StmtKind::ImportFrom {
module,
names,
level,
} => format_import_from(
f,
self.item,
module.as_ref().map(String::as_str),
names,
level.as_ref(),
),
// StmtKind::Nonlocal { .. } => {}
StmtKind::Expr { value } => format_expr(f, self.item, value),
_ => {
unimplemented!("Implement StmtKind: {:?}", self.item.node)
}
}?;
// Any trailing comments come on the lines after.
for trivia in &self.item.trivia {
if matches!(trivia.relationship, Relationship::Trailing) {
match trivia.kind {
TriviaKind::EmptyLine => {
write!(f, [empty_line()])?;
}
TriviaKind::StandaloneComment(range) => {
write!(f, [literal(range), hard_line_break()])?;
}
_ => {}
}
}
}
Ok(())
}
}
impl AsFormat<ASTFormatContext<'_>> for Stmt {
type Format<'a> = FormatStmt<'a>;
fn format(&self) -> Self::Format<'_> {
FormatStmt { item: self }
}
}

View file

@ -0,0 +1,37 @@
use ruff_formatter::prelude::*;
use ruff_formatter::write;
use crate::context::ASTFormatContext;
use crate::cst::Unaryop;
use crate::shared_traits::AsFormat;
pub struct FormatUnaryop<'a> {
item: &'a Unaryop,
}
impl AsFormat<ASTFormatContext<'_>> for Unaryop {
type Format<'a> = FormatUnaryop<'a>;
fn format(&self) -> Self::Format<'_> {
FormatUnaryop { item: self }
}
}
impl Format<ASTFormatContext<'_>> for FormatUnaryop<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext>) -> FormatResult<()> {
let unaryop = self.item;
write!(
f,
[
text(match unaryop {
Unaryop::Invert => "~",
Unaryop::Not => "not",
Unaryop::UAdd => "+",
Unaryop::USub => "-",
}),
matches!(unaryop, Unaryop::Not).then_some(space())
]
)?;
Ok(())
}
}

View file

@ -0,0 +1,32 @@
use ruff_formatter::prelude::*;
use ruff_formatter::write;
use crate::context::ASTFormatContext;
use crate::cst::Withitem;
use crate::shared_traits::AsFormat;
pub struct FormatWithitem<'a> {
item: &'a Withitem,
}
impl AsFormat<ASTFormatContext<'_>> for Withitem {
type Format<'a> = FormatWithitem<'a>;
fn format(&self) -> Self::Format<'_> {
FormatWithitem { item: self }
}
}
impl Format<ASTFormatContext<'_>> for FormatWithitem<'_> {
fn fmt(&self, f: &mut Formatter<ASTFormatContext>) -> FormatResult<()> {
let withitem = self.item;
write!(f, [withitem.context_expr.format()])?;
if let Some(optional_vars) = &withitem.optional_vars {
write!(f, [space(), text("as"), space()])?;
write!(f, [optional_vars.format()])?;
}
Ok(())
}
}

View file

@ -0,0 +1,143 @@
use anyhow::Result;
use ruff_formatter::{format, Formatted, IndentStyle, SimpleFormatOptions};
use rustpython_parser::lexer::LexResult;
use crate::attachment::attach;
use crate::context::ASTFormatContext;
use crate::core::locator::Locator;
use crate::core::rustpython_helpers;
use crate::cst::Stmt;
use crate::newlines::normalize_newlines;
use crate::parentheses::normalize_parentheses;
mod attachment;
pub mod builders;
pub mod cli;
pub mod context;
mod core;
mod cst;
mod format;
mod newlines;
mod parentheses;
pub mod shared_traits;
#[cfg(test)]
mod test;
pub mod trivia;
pub fn fmt(contents: &str) -> Result<Formatted<ASTFormatContext>> {
// Tokenize once.
let tokens: Vec<LexResult> = rustpython_helpers::tokenize(contents);
// Extract trivia.
let trivia = trivia::extract_trivia_tokens(&tokens);
// Parse the AST.
let python_ast = rustpython_helpers::parse_program_tokens(tokens, "<filename>")?;
// Convert to a CST.
let mut python_cst: Vec<Stmt> = python_ast.into_iter().map(Into::into).collect();
// Attach trivia.
attach(&mut python_cst, trivia);
normalize_newlines(&mut python_cst);
normalize_parentheses(&mut python_cst);
format!(
ASTFormatContext::new(
SimpleFormatOptions {
indent_style: IndentStyle::Space(4),
line_width: 88.try_into().unwrap(),
},
Locator::new(contents)
),
[format::builders::block(&python_cst)]
)
.map_err(Into::into)
}
#[cfg(test)]
mod tests {
use std::path::Path;
use anyhow::Result;
use test_case::test_case;
use crate::fmt;
use crate::test::test_resource_path;
#[test_case(Path::new("simple_cases/class_blank_parentheses.py"); "class_blank_parentheses")]
#[test_case(Path::new("simple_cases/class_methods_new_line.py"); "class_methods_new_line")]
#[test_case(Path::new("simple_cases/beginning_backslash.py"); "beginning_backslash")]
#[test_case(Path::new("simple_cases/import_spacing.py"); "import_spacing")]
fn passing(path: &Path) -> Result<()> {
let snapshot = format!("{}", path.display());
let content = std::fs::read_to_string(test_resource_path(
Path::new("fixtures/black").join(path).as_path(),
))?;
let formatted = fmt(&content)?;
insta::assert_display_snapshot!(snapshot, formatted.print()?.as_code());
Ok(())
}
#[test_case(Path::new("simple_cases/collections.py"); "collections")]
#[test_case(Path::new("simple_cases/bracketmatch.py"); "bracketmatch")]
fn passing_modulo_string_normalization(path: &Path) -> Result<()> {
fn adjust_quotes(contents: &str) -> String {
// Replace all single quotes with double quotes.
contents.replace('\'', "\"")
}
let snapshot = format!("{}", path.display());
let content = std::fs::read_to_string(test_resource_path(
Path::new("fixtures/black").join(path).as_path(),
))?;
let formatted = fmt(&content)?;
insta::assert_display_snapshot!(snapshot, adjust_quotes(formatted.print()?.as_code()));
Ok(())
}
#[ignore]
// Passing apart from one deviation in RHS tuple assignment.
#[test_case(Path::new("simple_cases/tupleassign.py"); "tupleassign")]
// Lots of deviations, _mostly_ related to string normalization and wrapping.
#[test_case(Path::new("simple_cases/expression.py"); "expression")]
#[test_case(Path::new("simple_cases/function.py"); "function")]
#[test_case(Path::new("simple_cases/function2.py"); "function2")]
#[test_case(Path::new("simple_cases/power_op_spacing.py"); "power_op_spacing")]
fn failing(path: &Path) -> Result<()> {
let snapshot = format!("{}", path.display());
let content = std::fs::read_to_string(test_resource_path(
Path::new("fixtures/black").join(path).as_path(),
))?;
let formatted = fmt(&content)?;
insta::assert_display_snapshot!(snapshot, formatted.print()?.as_code());
Ok(())
}
/// Use this test to debug the formatting of some snipped
#[ignore]
#[test]
fn quick_test() {
let src = r#"
{
k: v for k, v in a_very_long_variable_name_that_exceeds_the_line_length_by_far_keep_going
}
"#;
let formatted = fmt(src).unwrap();
// Uncomment the `dbg` to print the IR.
// Use `dbg_write!(f, []) instead of `write!(f, [])` in your formatting code to print some IR
// inside of a `Format` implementation
// dbg!(formatted.document());
let printed = formatted.print().unwrap();
assert_eq!(
printed.as_code(),
r#"{
k: v
for k, v in a_very_long_variable_name_that_exceeds_the_line_length_by_far_keep_going
}"#
);
}
}

View file

@ -0,0 +1,16 @@
use std::fs;
use anyhow::Result;
use clap::Parser as ClapParser;
use ruff_python_formatter::cli::Cli;
use ruff_python_formatter::fmt;
fn main() -> Result<()> {
let cli = Cli::parse();
let contents = fs::read_to_string(cli.file)?;
#[allow(clippy::print_stdout)]
{
println!("{}", fmt(&contents)?.print()?.as_code());
}
Ok(())
}

View file

@ -0,0 +1,198 @@
use rustpython_parser::ast::Constant;
use crate::core::visitor;
use crate::core::visitor::Visitor;
use crate::cst::{Expr, ExprKind, Stmt, StmtKind};
use crate::trivia::{Relationship, Trivia, TriviaKind};
#[derive(Copy, Clone)]
enum Depth {
TopLevel,
Nested,
}
impl Depth {
fn max_newlines(self) -> usize {
match self {
Self::TopLevel => 2,
Self::Nested => 1,
}
}
}
#[derive(Copy, Clone)]
enum Scope {
Module,
Class,
Function,
}
#[derive(Debug, Copy, Clone)]
enum Trailer {
None,
ClassDef,
FunctionDef,
Import,
Docstring,
Generic,
}
struct NewlineNormalizer {
depth: Depth,
trailer: Trailer,
scope: Scope,
}
impl<'a> Visitor<'a> for NewlineNormalizer {
fn visit_stmt(&mut self, stmt: &'a mut Stmt) {
// Remove any runs of empty lines greater than two in a row.
let mut count = 0;
stmt.trivia.retain(|c| {
if matches!(c.kind, TriviaKind::EmptyLine) {
count += 1;
count <= self.depth.max_newlines()
} else {
count = 0;
true
}
});
if matches!(self.trailer, Trailer::None) {
// If this is the first statement in the block, remove any leading empty lines.
let mut seen_non_empty = false;
stmt.trivia.retain(|c| {
if seen_non_empty {
true
} else {
if matches!(c.kind, TriviaKind::EmptyLine) {
false
} else {
seen_non_empty = true;
true
}
}
});
} else {
// If the previous statement was a function or similar, ensure we have the
// appropriate number of lines to start.
let required_newlines = match self.trailer {
Trailer::FunctionDef | Trailer::ClassDef => self.depth.max_newlines(),
Trailer::Docstring if matches!(self.scope, Scope::Class) => 1,
Trailer::Import => {
if matches!(
stmt.node,
StmtKind::Import { .. } | StmtKind::ImportFrom { .. }
) {
0
} else {
1
}
}
_ => 0,
};
let present_newlines = stmt
.trivia
.iter()
.take_while(|c| matches!(c.kind, TriviaKind::EmptyLine))
.count();
if present_newlines < required_newlines {
for _ in 0..(required_newlines - present_newlines) {
stmt.trivia.insert(
0,
Trivia {
kind: TriviaKind::EmptyLine,
relationship: Relationship::Leading,
},
);
}
}
// If the current statement is a function or similar, Ensure we have an
// appropriate number of lines above.
if matches!(
stmt.node,
StmtKind::FunctionDef { .. }
| StmtKind::AsyncFunctionDef { .. }
| StmtKind::ClassDef { .. }
) {
let num_to_insert = self.depth.max_newlines()
- stmt
.trivia
.iter()
.take_while(|c| matches!(c.kind, TriviaKind::EmptyLine))
.count();
for _ in 0..num_to_insert {
stmt.trivia.insert(
0,
Trivia {
kind: TriviaKind::EmptyLine,
relationship: Relationship::Leading,
},
);
}
}
}
self.trailer = match &stmt.node {
StmtKind::FunctionDef { .. } | StmtKind::AsyncFunctionDef { .. } => {
Trailer::FunctionDef
}
// TODO(charlie): This needs to be the first statement in a class or function.
StmtKind::Expr { value, .. } => {
if let ExprKind::Constant {
value: Constant::Str(..),
..
} = &value.node
{
Trailer::Docstring
} else {
Trailer::Generic
}
}
StmtKind::ClassDef { .. } => Trailer::ClassDef,
StmtKind::Import { .. } | StmtKind::ImportFrom { .. } => Trailer::Import,
_ => Trailer::Generic,
};
let prev_scope = self.scope;
self.scope = match &stmt.node {
StmtKind::FunctionDef { .. } | StmtKind::AsyncFunctionDef { .. } => Scope::Function,
StmtKind::ClassDef { .. } => Scope::Class,
_ => prev_scope,
};
visitor::walk_stmt(self, stmt);
self.scope = prev_scope;
}
fn visit_expr(&mut self, expr: &'a mut Expr) {
expr.trivia
.retain(|c| !matches!(c.kind, TriviaKind::EmptyLine));
visitor::walk_expr(self, expr);
}
fn visit_body(&mut self, body: &'a mut [Stmt]) {
let prev_depth = self.depth;
let prev_trailer = self.trailer;
self.depth = Depth::Nested;
self.trailer = Trailer::None;
visitor::walk_body(self, body);
self.trailer = prev_trailer;
self.depth = prev_depth;
}
}
pub fn normalize_newlines(python_cst: &mut [Stmt]) {
let mut normalizer = NewlineNormalizer {
depth: Depth::TopLevel,
trailer: Trailer::None,
scope: Scope::Module,
};
for stmt in python_cst.iter_mut() {
normalizer.visit_stmt(stmt);
}
}

View file

@ -0,0 +1,169 @@
use crate::core::visitor;
use crate::core::visitor::Visitor;
use crate::cst::{Expr, ExprKind, Stmt, StmtKind};
use crate::trivia::{Parenthesize, TriviaKind};
/// Modify an [`Expr`] to infer parentheses, rather than respecting any user-provided trivia.
fn use_inferred_parens(expr: &mut Expr) {
// Remove parentheses, unless it's a generator expression, in which case, keep them.
if !matches!(expr.node, ExprKind::GeneratorExp { .. }) {
expr.trivia
.retain(|trivia| !matches!(trivia.kind, TriviaKind::Parentheses));
}
// If it's a tuple, add parentheses if it's a singleton; otherwise, we only need parentheses
// if the tuple expands.
if let ExprKind::Tuple { elts, .. } = &expr.node {
expr.parentheses = if elts.len() > 1 {
Parenthesize::IfExpanded
} else {
Parenthesize::Always
};
}
}
struct ParenthesesNormalizer {}
impl<'a> Visitor<'a> for ParenthesesNormalizer {
fn visit_stmt(&mut self, stmt: &'a mut Stmt) {
// Always remove parentheses around statements, unless it's an expression statement,
// in which case, remove parentheses around the expression.
let before = stmt.trivia.len();
stmt.trivia
.retain(|trivia| !matches!(trivia.kind, TriviaKind::Parentheses));
let after = stmt.trivia.len();
if let StmtKind::Expr { value } = &mut stmt.node {
if before != after {
stmt.parentheses = Parenthesize::Always;
value.parentheses = Parenthesize::Never;
}
}
// In a variety of contexts, remove parentheses around sub-expressions. Right now, the
// pattern is consistent (and repeated), but it may not end up that way.
// https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#parentheses
match &mut stmt.node {
StmtKind::FunctionDef { .. } => {}
StmtKind::AsyncFunctionDef { .. } => {}
StmtKind::ClassDef { .. } => {}
StmtKind::Return { value } => {
if let Some(value) = value {
use_inferred_parens(value);
}
}
StmtKind::Delete { .. } => {}
StmtKind::Assign { targets, value, .. } => {
for target in targets {
use_inferred_parens(target);
}
use_inferred_parens(value);
}
StmtKind::AugAssign { value, .. } => {
use_inferred_parens(value);
}
StmtKind::AnnAssign { value, .. } => {
if let Some(value) = value {
use_inferred_parens(value);
}
}
StmtKind::For { target, iter, .. } | StmtKind::AsyncFor { target, iter, .. } => {
use_inferred_parens(target);
use_inferred_parens(iter);
}
StmtKind::While { test, .. } => {
use_inferred_parens(test);
}
StmtKind::If { test, .. } => {
use_inferred_parens(test);
}
StmtKind::With { .. } => {}
StmtKind::AsyncWith { .. } => {}
StmtKind::Match { .. } => {}
StmtKind::Raise { .. } => {}
StmtKind::Try { .. } => {}
StmtKind::Assert { test, msg } => {
use_inferred_parens(test);
if let Some(msg) = msg {
use_inferred_parens(msg);
}
}
StmtKind::Import { .. } => {}
StmtKind::ImportFrom { .. } => {}
StmtKind::Global { .. } => {}
StmtKind::Nonlocal { .. } => {}
StmtKind::Expr { .. } => {}
StmtKind::Pass => {}
StmtKind::Break => {}
StmtKind::Continue => {}
}
visitor::walk_stmt(self, stmt);
}
fn visit_expr(&mut self, expr: &'a mut Expr) {
// Always retain parentheses around expressions.
let before = expr.trivia.len();
expr.trivia
.retain(|trivia| !matches!(trivia.kind, TriviaKind::Parentheses));
let after = expr.trivia.len();
if before != after {
expr.parentheses = Parenthesize::Always;
}
match &mut expr.node {
ExprKind::BoolOp { .. } => {}
ExprKind::NamedExpr { .. } => {}
ExprKind::BinOp { .. } => {}
ExprKind::UnaryOp { .. } => {}
ExprKind::Lambda { .. } => {}
ExprKind::IfExp { .. } => {}
ExprKind::Dict { .. } => {}
ExprKind::Set { .. } => {}
ExprKind::ListComp { .. } => {}
ExprKind::SetComp { .. } => {}
ExprKind::DictComp { .. } => {}
ExprKind::GeneratorExp { .. } => {}
ExprKind::Await { .. } => {}
ExprKind::Yield { .. } => {}
ExprKind::YieldFrom { .. } => {}
ExprKind::Compare { .. } => {}
ExprKind::Call { .. } => {}
ExprKind::FormattedValue { .. } => {}
ExprKind::JoinedStr { .. } => {}
ExprKind::Constant { .. } => {}
ExprKind::Attribute { .. } => {}
ExprKind::Subscript { value, slice, .. } => {
// If the slice isn't manually parenthesized, ensure that we _never_ parenthesize
// the value.
if !slice
.trivia
.iter()
.any(|trivia| matches!(trivia.kind, TriviaKind::Parentheses))
{
value.parentheses = Parenthesize::Never;
}
}
ExprKind::Starred { .. } => {}
ExprKind::Name { .. } => {}
ExprKind::List { .. } => {}
ExprKind::Tuple { .. } => {}
ExprKind::Slice { .. } => {}
}
visitor::walk_expr(self, expr);
}
}
/// Normalize parentheses in a Python CST.
///
/// It's not always possible to determine the correct parentheses to use during formatting
/// from the node (and trivia) alone; sometimes, we need to know the parent node. This
/// visitor normalizes parentheses via a top-down traversal, which simplifies the formatting
/// code later on.
///
/// TODO(charlie): It's weird that we have both `TriviaKind::Parentheses` (which aren't used
/// during formatting) and `Parenthesize` (which are used during formatting).
pub fn normalize_parentheses(python_cst: &mut [Stmt]) {
let mut normalizer = ParenthesesNormalizer {};
normalizer.visit_body(python_cst);
}

View file

@ -0,0 +1,113 @@
#![allow(clippy::all)]
/// Used to get an object that knows how to format this object.
pub trait AsFormat<Context> {
type Format<'a>: ruff_formatter::Format<Context>
where
Self: 'a;
/// Returns an object that is able to format this object.
fn format(&self) -> Self::Format<'_>;
}
/// Implement [`AsFormat`] for references to types that implement [`AsFormat`].
impl<T, C> AsFormat<C> for &T
where
T: AsFormat<C>,
{
type Format<'a> = T::Format<'a> where Self: 'a;
fn format(&self) -> Self::Format<'_> {
AsFormat::format(&**self)
}
}
/// Implement [`AsFormat`] for [`Option`] when `T` implements [`AsFormat`]
///
/// Allows to call format on optional AST fields without having to unwrap the
/// field first.
impl<T, C> AsFormat<C> for Option<T>
where
T: AsFormat<C>,
{
type Format<'a> = Option<T::Format<'a>> where Self: 'a;
fn format(&self) -> Self::Format<'_> {
self.as_ref().map(AsFormat::format)
}
}
/// Used to convert this object into an object that can be formatted.
///
/// The difference to [`AsFormat`] is that this trait takes ownership of `self`.
pub trait IntoFormat<Context> {
type Format: ruff_formatter::Format<Context>;
fn into_format(self) -> Self::Format;
}
/// Implement [`IntoFormat`] for [`Option`] when `T` implements [`IntoFormat`]
///
/// Allows to call format on optional AST fields without having to unwrap the
/// field first.
impl<T, Context> IntoFormat<Context> for Option<T>
where
T: IntoFormat<Context>,
{
type Format = Option<T::Format>;
fn into_format(self) -> Self::Format {
self.map(IntoFormat::into_format)
}
}
/// Formatting specific [`Iterator`] extensions
pub trait FormattedIterExt {
/// Converts every item to an object that knows how to format it.
fn formatted<Context>(self) -> FormattedIter<Self, Self::Item, Context>
where
Self: Iterator + Sized,
Self::Item: IntoFormat<Context>,
{
FormattedIter {
inner: self,
options: std::marker::PhantomData,
}
}
}
impl<I> FormattedIterExt for I where I: std::iter::Iterator {}
pub struct FormattedIter<Iter, Item, Context>
where
Iter: Iterator<Item = Item>,
{
inner: Iter,
options: std::marker::PhantomData<Context>,
}
impl<Iter, Item, Context> std::iter::Iterator for FormattedIter<Iter, Item, Context>
where
Iter: Iterator<Item = Item>,
Item: IntoFormat<Context>,
{
type Item = Item::Format;
fn next(&mut self) -> Option<Self::Item> {
Some(self.inner.next()?.into_format())
}
}
impl<Iter, Item, Context> std::iter::FusedIterator for FormattedIter<Iter, Item, Context>
where
Iter: std::iter::FusedIterator<Item = Item>,
Item: IntoFormat<Context>,
{
}
impl<Iter, Item, Context> std::iter::ExactSizeIterator for FormattedIter<Iter, Item, Context>
where
Iter: Iterator<Item = Item> + std::iter::ExactSizeIterator,
Item: IntoFormat<Context>,
{
}

View file

@ -0,0 +1,28 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
x = (123456789).bit_count()
x = (123456).__abs__()
x = (0.1).is_integer()
x = (1.0).imag
x = (1e1).imag
x = (1e-1).real
x = (123456789.123456789).hex()
x = (123456789.123456789e123456789).real
x = (123456789e123456789).conjugate()
x = 123456789j.real
x = 123456789.123456789j.__add__(0b1011.bit_length())
x = 0xB1ACC.conjugate()
x = 0b1011.conjugate()
x = 0o777.real
x = (0.000000006).hex()
x = -100.0000j
if (10).real:
...
y = 100[no]
y = 100(no)

View file

@ -0,0 +1,15 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
def bob(): \
# pylint: disable=W9016
pass
def bobtwo(): \
\
# some comment here
pass

View file

@ -0,0 +1,102 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
#!/usr/bin/env python3
# fmt: on
# Some license here.
#
# Has many lines. Many, many lines.
# Many, many, many lines.
"""Module docstring.
Possibly also many, many lines.
"""
import os.path
import sys
import a
from b.c import X # some noqa comment
try:
import fast
except ImportError:
import slow as fast
# Some comment before a function.
y = 1
(
# some strings
y # type: ignore
)
def function(default=None):
"""Docstring comes first.
Possibly many lines.
"""
# FIXME: Some comment about why this function is crap but still in production.
import inner_imports
if inner_imports.are_evil():
# Explains why we have this if.
# In great detail indeed.
x = X()
return x.method1() # type: ignore
# This return is also commented for some reason.
return default
# Explains why we use global state.
GLOBAL_STATE = {"a": a(1), "b": a(2), "c": a(3)}
# Another comment!
# This time two lines.
class Foo:
"""Docstring for class Foo. Example from Sphinx docs."""
#: Doc comment for class attribute Foo.bar.
#: It can have multiple lines.
bar = 1
flox = 1.5 #: Doc comment for Foo.flox. One line only.
baz = 2
"""Docstring for class attribute Foo.baz."""
def __init__(self):
#: Doc comment for instance attribute qux.
self.qux = 3
self.spam = 4
"""Docstring for instance attribute spam."""
#' <h1>This is pweave!</h1>
@fast(really=True)
async def wat():
# This comment, for some reason \
# contains a trailing backslash.
async with X.open_async() as x: # Some more comments
result = await x.method1()
# Comment after ending a block.
if result:
print("A OK", file=sys.stdout)
# Comment between things.
print()
# Some closing comments.
# Maybe Vim or Emacs directives for formatting.
# Who knows.

View file

@ -0,0 +1,178 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
from com.my_lovely_company.my_lovely_team.my_lovely_project.my_lovely_component import (
MyLovelyCompanyTeamProjectComponent, # NOT DRY
)
from com.my_lovely_company.my_lovely_team.my_lovely_project.my_lovely_component import (
MyLovelyCompanyTeamProjectComponent as component, # DRY
)
# Please keep __all__ alphabetized within each category.
__all__ = [
# Super-special typing primitives.
"Any",
"Callable",
"ClassVar",
# ABCs (from collections.abc).
"AbstractSet", # collections.abc.Set.
"ByteString",
"Container",
# Concrete collection types.
"Counter",
"Deque",
"Dict",
"DefaultDict",
"List",
"Set",
"FrozenSet",
"NamedTuple", # Not really a type.
"Generator",
]
not_shareables = [
# singletons
True,
False,
NotImplemented,
...,
# builtin types and objects
type,
object,
object(),
Exception(),
42,
100.0,
"spam",
# user-defined types and objects
Cheese,
Cheese("Wensleydale"),
SubBytes(b"spam"),
]
if "PYTHON" in os.environ:
add_compiler(compiler_from_env())
else:
# for compiler in compilers.values():
# add_compiler(compiler)
add_compiler(compilers[(7.0, 32)])
# add_compiler(compilers[(7.1, 64)])
# Comment before function.
def inline_comments_in_brackets_ruin_everything():
if typedargslist:
parameters.children = [children[0], body, children[-1]] # (1 # )1
parameters.children = [
children[0],
body,
children[-1], # type: ignore
]
else:
parameters.children = [
parameters.children[0], # (2 what if this was actually long
body,
parameters.children[-1], # )2
]
parameters.children = [parameters.what_if_this_was_actually_long.children[0], body, parameters.children[-1]] # type: ignore
if (
self._proc is not None
# has the child process finished?
and self._returncode is None
# the child process has finished, but the
# transport hasn't been notified yet?
and self._proc.poll() is None
):
pass
# no newline before or after
short = [
# one
1,
# two
2,
]
# no newline after
call(
arg1,
arg2,
"""
short
""",
arg3=True,
)
############################################################################
call2(
# short
arg1,
# but
arg2,
# multiline
"""
short
""",
# yup
arg3=True,
)
lcomp = [
element for element in collection if element is not None # yup # yup # right
]
lcomp2 = [
# hello
element
# yup
for element in collection
# right
if element is not None
]
lcomp3 = [
# This one is actually too long to fit in a single line.
element.split("\n", 1)[0]
# yup
for element in collection.select_elements()
# right
if element is not None
]
while True:
if False:
continue
# and round and round we go
# and round and round we go
# let's return
return Node(
syms.simple_stmt,
[Node(statement, result), Leaf(token.NEWLINE, "\n")], # FIXME: \r\n?
)
CONFIG_FILES = (
[
CONFIG_FILE,
]
+ SHARED_CONFIG_FILES
+ USER_CONFIG_FILES
) # type: Final
class Test:
def _init_host(self, parsed) -> None:
if parsed.hostname is None or not parsed.hostname.strip(): # type: ignore
pass
#######################
### SECTION COMMENT ###
#######################
instruction() # comment with bad spacing
# END COMMENTS
# MORE END COMMENTS

View file

@ -0,0 +1,53 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
# The percent-percent comments are Spyder IDE cells.
#%%
def func():
x = """
a really long string
"""
lcomp3 = [
# This one is actually too long to fit in a single line.
element.split("\n", 1)[0]
# yup
for element in collection.select_elements()
# right
if element is not None
]
# Capture each of the exceptions in the MultiError along with each of their causes and contexts
if isinstance(exc_value, MultiError):
embedded = []
for exc in exc_value.exceptions:
if exc not in _seen:
embedded.append(
# This should be left alone (before)
traceback.TracebackException.from_exception(
exc,
limit=limit,
lookup_lines=lookup_lines,
capture_locals=capture_locals,
# copy the set of _seen exceptions so that duplicates
# shared between sub-exceptions are not omitted
_seen=set(_seen),
)
# This should be left alone (after)
)
# everything is fine if the expression isn't nested
traceback.TracebackException.from_exception(
exc,
limit=limit,
lookup_lines=lookup_lines,
capture_locals=capture_locals,
# copy the set of _seen exceptions so that duplicates
# shared between sub-exceptions are not omitted
_seen=set(_seen),
)
#%%

View file

@ -0,0 +1,100 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
from com.my_lovely_company.my_lovely_team.my_lovely_project.my_lovely_component import (
MyLovelyCompanyTeamProjectComponent, # NOT DRY
)
from com.my_lovely_company.my_lovely_team.my_lovely_project.my_lovely_component import (
MyLovelyCompanyTeamProjectComponent as component, # DRY
)
class C:
@pytest.mark.parametrize(
("post_data", "message"),
[
# metadata_version errors.
(
{},
"None is an invalid value for Metadata-Version. Error: This field is"
" required. see"
" https://packaging.python.org/specifications/core-metadata",
),
(
{"metadata_version": "-1"},
"'-1' is an invalid value for Metadata-Version. Error: Unknown Metadata"
" Version see"
" https://packaging.python.org/specifications/core-metadata",
),
# name errors.
(
{"metadata_version": "1.2"},
"'' is an invalid value for Name. Error: This field is required. see"
" https://packaging.python.org/specifications/core-metadata",
),
(
{"metadata_version": "1.2", "name": "foo-"},
"'foo-' is an invalid value for Name. Error: Must start and end with a"
" letter or numeral and contain only ascii numeric and '.', '_' and"
" '-'. see https://packaging.python.org/specifications/core-metadata",
),
# version errors.
(
{"metadata_version": "1.2", "name": "example"},
"'' is an invalid value for Version. Error: This field is required. see"
" https://packaging.python.org/specifications/core-metadata",
),
(
{"metadata_version": "1.2", "name": "example", "version": "dog"},
"'dog' is an invalid value for Version. Error: Must start and end with"
" a letter or numeral and contain only ascii numeric and '.', '_' and"
" '-'. see https://packaging.python.org/specifications/core-metadata",
),
],
)
def test_fails_invalid_post_data(
self, pyramid_config, db_request, post_data, message
):
pyramid_config.testing_securitypolicy(userid=1)
db_request.POST = MultiDict(post_data)
def foo(list_a, list_b):
results = (
User.query.filter(User.foo == "bar")
.filter( # Because foo.
db.or_(User.field_a.astext.in_(list_a), User.field_b.astext.in_(list_b))
)
.filter(User.xyz.is_(None))
# Another comment about the filtering on is_quux goes here.
.filter(db.not_(User.is_pending.astext.cast(db.Boolean).is_(True)))
.order_by(User.created_at.desc())
.with_for_update(key_share=True)
.all()
)
return results
def foo2(list_a, list_b):
# Standalone comment reasonably placed.
return (
User.query.filter(User.foo == "bar")
.filter(
db.or_(User.field_a.astext.in_(list_a), User.field_b.astext.in_(list_b))
)
.filter(User.xyz.is_(None))
)
def foo3(list_a, list_b):
return (
# Standlone comment but weirdly placed.
User.query.filter(User.foo == "bar")
.filter(
db.or_(User.field_a.astext.in_(list_a), User.field_b.astext.in_(list_b))
)
.filter(User.xyz.is_(None))
)

View file

@ -0,0 +1,77 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
while True:
if something.changed:
do.stuff() # trailing comment
# Comment belongs to the `if` block.
# This one belongs to the `while` block.
# Should this one, too? I guess so.
# This one is properly standalone now.
for i in range(100):
# first we do this
if i % 33 == 0:
break
# then we do this
print(i)
# and finally we loop around
with open(some_temp_file) as f:
data = f.read()
try:
with open(some_other_file) as w:
w.write(data)
except OSError:
print("problems")
import sys
# leading function comment
def wat():
...
# trailing function comment
# SECTION COMMENT
# leading 1
@deco1
# leading 2
@deco2(with_args=True)
# leading 3
@deco3
def decorated1():
...
# leading 1
@deco1
# leading 2
@deco2(with_args=True)
# leading function comment
def decorated1():
...
# Note: this is fixed in
# Preview.empty_lines_before_class_or_def_with_leading_comments.
# In the current style, the user will have to split those lines by hand.
some_instruction
# This comment should be split from `some_instruction` by two lines but isn't.
def g():
...
if __name__ == "__main__":
main()

View file

@ -0,0 +1,124 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
from typing import Any, Tuple
def f(
a, # type: int
):
pass
# test type comments
def f(a, b, c, d, e, f, g, h, i):
# type: (int, int, int, int, int, int, int, int, int) -> None
pass
def f(
a, # type: int
b, # type: int
c, # type: int
d, # type: int
e, # type: int
f, # type: int
g, # type: int
h, # type: int
i, # type: int
):
# type: (...) -> None
pass
def f(
arg, # type: int
*args, # type: *Any
default=False, # type: bool
**kwargs, # type: **Any
):
# type: (...) -> None
pass
def f(
a, # type: int
b, # type: int
c, # type: int
d, # type: int
):
# type: (...) -> None
element = 0 # type: int
another_element = 1 # type: float
another_element_with_long_name = 2 # type: int
another_really_really_long_element_with_a_unnecessarily_long_name_to_describe_what_it_does_enterprise_style = (
3
) # type: int
an_element_with_a_long_value = calls() or more_calls() and more() # type: bool
tup = (
another_element,
another_really_really_long_element_with_a_unnecessarily_long_name_to_describe_what_it_does_enterprise_style,
) # type: Tuple[int, int]
a = (
element
+ another_element
+ another_element_with_long_name
+ element
+ another_element
+ another_element_with_long_name
) # type: int
def f(
x, # not a type comment
y, # type: int
):
# type: (...) -> None
pass
def f(
x, # not a type comment
): # type: (int) -> None
pass
def func(
a=some_list[0], # type: int
): # type: () -> int
c = call(
0.0123,
0.0456,
0.0789,
0.0123,
0.0456,
0.0789,
0.0123,
0.0456,
0.0789,
a[-1], # type: ignore
)
c = call(
"aaaaaaaa", "aaaaaaaa", "aaaaaaaa", "aaaaaaaa", "aaaaaaaa", "aaaaaaaa", "aaaaaaaa" # type: ignore
)
result = ( # aaa
"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
)
AAAAAAAAAAAAA = [AAAAAAAAAAAAA] + SHARED_AAAAAAAAAAAAA + USER_AAAAAAAAAAAAA + AAAAAAAAAAAAA # type: ignore
call_to_some_function_asdf(
foo,
[AAAAAAAAAAAAAAAAAAAAAAA, AAAAAAAAAAAAAAAAAAAAAAA, AAAAAAAAAAAAAAAAAAAAAAA, BBBBBBBBBBBB], # type: ignore
)
aaaaaaaaaaaaa, bbbbbbbbb = map(list, map(itertools.chain.from_iterable, zip(*items))) # type: ignore[arg-type]

View file

@ -0,0 +1,29 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
from .config import (
ConfigTypeAttributes,
Int,
Path, # String,
# DEFAULT_TYPE_ATTRIBUTES,
)
result = 1 # A simple comment
result = (1,) # Another one
result = 1 #  type: ignore
result = 1 # This comment is talking about type: ignore
square = Square(4) #  type: Optional[Square]
def function(a: int = 42):
"""This docstring is already formatted
a
b
"""
# There's a NBSP + 3 spaces before
# And 4 spaces on the next line
pass

View file

@ -0,0 +1,187 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
class C:
def test(self) -> None:
with patch("black.out", print):
self.assertEqual(
unstyle(str(report)), "1 file reformatted, 1 file failed to reformat."
)
self.assertEqual(
unstyle(str(report)),
"1 file reformatted, 1 file left unchanged, 1 file failed to reformat.",
)
self.assertEqual(
unstyle(str(report)),
"2 files reformatted, 1 file left unchanged, 1 file failed to"
" reformat.",
)
self.assertEqual(
unstyle(str(report)),
"2 files reformatted, 2 files left unchanged, 2 files failed to"
" reformat.",
)
for i in (a,):
if (
# Rule 1
i % 2 == 0
# Rule 2
and i % 3 == 0
):
while (
# Just a comment
call()
# Another
):
print(i)
xxxxxxxxxxxxxxxx = Yyyy2YyyyyYyyyyy(
push_manager=context.request.resource_manager,
max_items_to_push=num_items,
batch_size=Yyyy2YyyyYyyyyYyyy.FULL_SIZE,
).push(
# Only send the first n items.
items=items[:num_items]
)
return (
'Utterly failed doctest test for %s\n File "%s", line %s, in %s\n\n%s'
% (test.name, test.filename, lineno, lname, err)
)
def omitting_trailers(self) -> None:
get_collection(
hey_this_is_a_very_long_call, it_has_funny_attributes, really=True
)[OneLevelIndex]
get_collection(
hey_this_is_a_very_long_call, it_has_funny_attributes, really=True
)[OneLevelIndex][TwoLevelIndex][ThreeLevelIndex][FourLevelIndex]
d[0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][
22
]
assignment = (
some.rather.elaborate.rule() and another.rule.ending_with.index[123]
)
def easy_asserts(self) -> None:
assert {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
} == expected, "Not what we expected"
assert expected == {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}, "Not what we expected"
assert expected == {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}
def tricky_asserts(self) -> None:
assert {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
} == expected(
value, is_going_to_be="too long to fit in a single line", srsly=True
), "Not what we expected"
assert {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
} == expected, (
"Not what we expected and the message is too long to fit in one line"
)
assert expected(
value, is_going_to_be="too long to fit in a single line", srsly=True
) == {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}, "Not what we expected"
assert expected == {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}, (
"Not what we expected and the message is too long to fit in one line"
" because it's too long"
)
dis_c_instance_method = """\
%3d 0 LOAD_FAST 1 (x)
2 LOAD_CONST 1 (1)
4 COMPARE_OP 2 (==)
6 LOAD_FAST 0 (self)
8 STORE_ATTR 0 (x)
10 LOAD_CONST 0 (None)
12 RETURN_VALUE
""" % (
_C.__init__.__code__.co_firstlineno + 1,
)
assert (
expectedexpectedexpectedexpectedexpectedexpectedexpectedexpectedexpect
== {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}
)

View file

@ -0,0 +1,187 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
class C:
def test(self) -> None:
with patch("black.out", print):
self.assertEqual(
unstyle(str(report)), "1 file reformatted, 1 file failed to reformat."
)
self.assertEqual(
unstyle(str(report)),
"1 file reformatted, 1 file left unchanged, 1 file failed to reformat.",
)
self.assertEqual(
unstyle(str(report)),
"2 files reformatted, 1 file left unchanged, 1 file failed to"
" reformat.",
)
self.assertEqual(
unstyle(str(report)),
"2 files reformatted, 2 files left unchanged, 2 files failed to"
" reformat.",
)
for i in (a,):
if (
# Rule 1
i % 2 == 0
# Rule 2
and i % 3 == 0
):
while (
# Just a comment
call()
# Another
):
print(i)
xxxxxxxxxxxxxxxx = Yyyy2YyyyyYyyyyy(
push_manager=context.request.resource_manager,
max_items_to_push=num_items,
batch_size=Yyyy2YyyyYyyyyYyyy.FULL_SIZE,
).push(
# Only send the first n items.
items=items[:num_items]
)
return (
'Utterly failed doctest test for %s\n File "%s", line %s, in %s\n\n%s'
% (test.name, test.filename, lineno, lname, err)
)
def omitting_trailers(self) -> None:
get_collection(
hey_this_is_a_very_long_call, it_has_funny_attributes, really=True
)[OneLevelIndex]
get_collection(
hey_this_is_a_very_long_call, it_has_funny_attributes, really=True
)[OneLevelIndex][TwoLevelIndex][ThreeLevelIndex][FourLevelIndex]
d[0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][
22
]
assignment = (
some.rather.elaborate.rule() and another.rule.ending_with.index[123]
)
def easy_asserts(self) -> None:
assert {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
} == expected, "Not what we expected"
assert expected == {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}, "Not what we expected"
assert expected == {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}
def tricky_asserts(self) -> None:
assert {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
} == expected(
value, is_going_to_be="too long to fit in a single line", srsly=True
), "Not what we expected"
assert {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
} == expected, (
"Not what we expected and the message is too long to fit in one line"
)
assert expected(
value, is_going_to_be="too long to fit in a single line", srsly=True
) == {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}, "Not what we expected"
assert expected == {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}, (
"Not what we expected and the message is too long to fit in one line"
" because it's too long"
)
dis_c_instance_method = """\
%3d 0 LOAD_FAST 1 (x)
2 LOAD_CONST 1 (1)
4 COMPARE_OP 2 (==)
6 LOAD_FAST 0 (self)
8 STORE_ATTR 0 (x)
10 LOAD_CONST 0 (None)
12 RETURN_VALUE
""" % (
_C.__init__.__code__.co_firstlineno + 1,
)
assert (
expectedexpectedexpectedexpectedexpectedexpectedexpectedexpectedexpect
== {
key1: value1,
key2: value2,
key3: value3,
key4: value4,
key5: value5,
key6: value6,
key7: value7,
key8: value8,
key9: value9,
}
)

View file

@ -0,0 +1,225 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
class MyClass:
"""Multiline
class docstring
"""
def method(self):
"""Multiline
method docstring
"""
pass
def foo():
"""This is a docstring with
some lines of text here
"""
return
def bar():
"""This is another docstring
with more lines of text
"""
return
def baz():
'''"This" is a string with some
embedded "quotes"'''
return
def troz():
"""Indentation with tabs
is just as OK
"""
return
def zort():
"""Another
multiline
docstring
"""
pass
def poit():
"""
Lorem ipsum dolor sit amet.
Consectetur adipiscing elit:
- sed do eiusmod tempor incididunt ut labore
- dolore magna aliqua
- enim ad minim veniam
- quis nostrud exercitation ullamco laboris nisi
- aliquip ex ea commodo consequat
"""
pass
def under_indent():
"""
These lines are indented in a way that does not
make sense.
"""
pass
def over_indent():
"""
This has a shallow indent
- But some lines are deeper
- And the closing quote is too deep
"""
pass
def single_line():
"""But with a newline after it!"""
pass
def this():
r"""
'hey ho'
"""
def that():
""" "hey yah" """
def and_that():
"""
"hey yah" """
def and_this():
'''
"hey yah"'''
def multiline_whitespace():
""" """
def oneline_whitespace():
""" """
def empty():
""""""
def single_quotes():
"testing"
def believe_it_or_not_this_is_in_the_py_stdlib():
'''
"hey yah"'''
def ignored_docstring():
"""a => \
b"""
def single_line_docstring_with_whitespace():
"""This should be stripped"""
def docstring_with_inline_tabs_and_space_indentation():
"""hey
tab separated value
tab at start of line and then a tab separated value
multiple tabs at the beginning and inline
mixed tabs and spaces at beginning. next line has mixed tabs and spaces only.
line ends with some tabs
"""
def docstring_with_inline_tabs_and_tab_indentation():
"""hey
tab separated value
tab at start of line and then a tab separated value
multiple tabs at the beginning and inline
mixed tabs and spaces at beginning. next line has mixed tabs and spaces only.
line ends with some tabs
"""
pass
def backslash_space():
"""\ """
def multiline_backslash_1():
"""
hey\there\
\ """
def multiline_backslash_2():
"""
hey there \ """
# Regression test for #3425
def multiline_backslash_really_long_dont_crash():
"""
hey there hello guten tag hi hoow are you ola zdravstvuyte ciao como estas ca va \ """
def multiline_backslash_3():
"""
already escaped \\"""
def my_god_its_full_of_stars_1():
"I'm sorry Dave\u2001"
# the space below is actually a \u2001, removed in output
def my_god_its_full_of_stars_2():
"I'm sorry Dave"
def docstring_almost_at_line_limit():
"""long docstring................................................................."""
def docstring_almost_at_line_limit2():
"""long docstring.................................................................
..................................................................................
"""
def docstring_at_line_limit():
"""long docstring................................................................"""
def multiline_docstring_at_line_limit():
"""first line-----------------------------------------------------------------------
second line----------------------------------------------------------------------"""
def stable_quote_normalization_with_immediate_inner_single_quote(self):
"""'<text here>
<text here, since without another non-empty line black is stable>
"""

View file

@ -0,0 +1,10 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
# Make sure when the file ends with class's docstring,
# It doesn't add extra blank lines.
class ClassWithDocstring:
"""A docstring."""

View file

@ -0,0 +1,98 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
"""Docstring."""
# leading comment
def f():
NO = ''
SPACE = ' '
DOUBLESPACE = ' '
t = leaf.type
p = leaf.parent # trailing comment
v = leaf.value
if t in ALWAYS_NO_SPACE:
pass
if t == token.COMMENT: # another trailing comment
return DOUBLESPACE
assert p is not None, f"INTERNAL ERROR: hand-made leaf without parent: {leaf!r}"
prev = leaf.prev_sibling
if not prev:
prevp = preceding_leaf(p)
if not prevp or prevp.type in OPENING_BRACKETS:
return NO
if prevp.type == token.EQUAL:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.argument,
}:
return NO
elif prevp.type == token.DOUBLESTAR:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.dictsetmaker,
}:
return NO
###############################################################################
# SECTION BECAUSE SECTIONS
###############################################################################
def g():
NO = ''
SPACE = ' '
DOUBLESPACE = ' '
t = leaf.type
p = leaf.parent
v = leaf.value
# Comment because comments
if t in ALWAYS_NO_SPACE:
pass
if t == token.COMMENT:
return DOUBLESPACE
# Another comment because more comments
assert p is not None, f'INTERNAL ERROR: hand-made leaf without parent: {leaf!r}'
prev = leaf.prev_sibling
if not prev:
prevp = preceding_leaf(p)
if not prevp or prevp.type in OPENING_BRACKETS:
# Start of the line or a bracketed expression.
# More than one line for the comment.
return NO
if prevp.type == token.EQUAL:
if prevp.parent and prevp.parent.type in {
syms.typedargslist,
syms.varargslist,
syms.parameters,
syms.arglist,
syms.argument,
}:
return NO

View file

@ -0,0 +1,260 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
...
'some_string'
b'\\xa3'
Name
None
True
False
1
1.0
1j
True or False
True or False or None
True and False
True and False and None
(Name1 and Name2) or Name3
Name1 and Name2 or Name3
Name1 or (Name2 and Name3)
Name1 or Name2 and Name3
(Name1 and Name2) or (Name3 and Name4)
Name1 and Name2 or Name3 and Name4
Name1 or (Name2 and Name3) or Name4
Name1 or Name2 and Name3 or Name4
v1 << 2
1 >> v2
1 % finished
1 + v2 - v3 * 4 ^ 5 ** v6 / 7 // 8
((1 + v2) - (v3 * 4)) ^ (((5 ** v6) / 7) // 8)
not great
~great
+value
-1
~int and not v1 ^ 123 + v2 | True
(~int) and (not ((v1 ^ (123 + v2)) | True))
+really ** -confusing ** ~operator ** -precedence
flags & ~ select.EPOLLIN and waiters.write_task is not None
lambda arg: None
lambda a=True: a
lambda a, b, c=True: a
lambda a, b, c=True, *, d=(1 << v2), e='str': a
lambda a, b, c=True, *vararg, d=(v1 << 2), e='str', **kwargs: a + b
manylambdas = lambda x=lambda y=lambda z=1: z: y(): x()
foo = (lambda port_id, ignore_missing: {"port1": port1_resource, "port2": port2_resource}[port_id])
1 if True else 2
str or None if True else str or bytes or None
(str or None) if True else (str or bytes or None)
str or None if (1 if True else 2) else str or bytes or None
(str or None) if (1 if True else 2) else (str or bytes or None)
((super_long_variable_name or None) if (1 if super_long_test_name else 2) else (str or bytes or None))
{'2.7': dead, '3.7': (long_live or die_hard)}
{'2.7': dead, '3.7': (long_live or die_hard), **{'3.6': verygood}}
{**a, **b, **c}
{'2.7', '3.6', '3.7', '3.8', '3.9', ('4.0' if gilectomy else '3.10')}
({'a': 'b'}, (True or False), (+value), 'string', b'bytes') or None
()
(1,)
(1, 2)
(1, 2, 3)
[]
[1, 2, 3, 4, 5, 6, 7, 8, 9, (10 or A), (11 or B), (12 or C)]
[1, 2, 3,]
[*a]
[*range(10)]
[*a, 4, 5,]
[4, *a, 5,]
[this_is_a_very_long_variable_which_will_force_a_delimiter_split, element, another, *more]
{i for i in (1, 2, 3)}
{(i ** 2) for i in (1, 2, 3)}
{(i ** 2) for i, _ in ((1, 'a'), (2, 'b'), (3, 'c'))}
{((i ** 2) + j) for i in (1, 2, 3) for j in (1, 2, 3)}
[i for i in (1, 2, 3)]
[(i ** 2) for i in (1, 2, 3)]
[(i ** 2) for i, _ in ((1, 'a'), (2, 'b'), (3, 'c'))]
[((i ** 2) + j) for i in (1, 2, 3) for j in (1, 2, 3)]
{i: 0 for i in (1, 2, 3)}
{i: j for i, j in ((1, 'a'), (2, 'b'), (3, 'c'))}
{a: b * 2 for a, b in dictionary.items()}
{a: b * -2 for a, b in dictionary.items()}
{k: v for k, v in this_is_a_very_long_variable_which_will_cause_a_trailing_comma_which_breaks_the_comprehension}
Python3 > Python2 > COBOL
Life is Life
call()
call(arg)
call(kwarg='hey')
call(arg, kwarg='hey')
call(arg, another, kwarg='hey', **kwargs)
call(this_is_a_very_long_variable_which_will_force_a_delimiter_split, arg, another, kwarg='hey', **kwargs) # note: no trailing comma pre-3.6
call(*gidgets[:2])
call(a, *gidgets[:2])
call(**self.screen_kwargs)
call(b, **self.screen_kwargs)
lukasz.langa.pl
call.me(maybe)
1 .real
1.0 .real
....__class__
list[str]
dict[str, int]
tuple[str, ...]
tuple[
str, int, float, dict[str, int]
]
tuple[str, int, float, dict[str, int],]
very_long_variable_name_filters: t.List[
t.Tuple[str, t.Union[str, t.List[t.Optional[str]]]],
]
xxxx_xxxxx_xxxx_xxx: Callable[..., List[SomeClass]] = classmethod( # type: ignore
sync(async_xxxx_xxx_xxxx_xxxxx_xxxx_xxx.__func__)
)
xxxx_xxx_xxxx_xxxxx_xxxx_xxx: Callable[..., List[SomeClass]] = classmethod( # type: ignore
sync(async_xxxx_xxx_xxxx_xxxxx_xxxx_xxx.__func__)
)
xxxx_xxx_xxxx_xxxxx_xxxx_xxx: Callable[
..., List[SomeClass]
] = classmethod(sync(async_xxxx_xxx_xxxx_xxxxx_xxxx_xxx.__func__)) # type: ignore
slice[0]
slice[0:1]
slice[0:1:2]
slice[:]
slice[:-1]
slice[1:]
slice[::-1]
slice[d :: d + 1]
slice[:c, c - 1]
numpy[:, 0:1]
numpy[:, :-1]
numpy[0, :]
numpy[:, i]
numpy[0, :2]
numpy[:N, 0]
numpy[:2, :4]
numpy[2:4, 1:5]
numpy[4:, 2:]
numpy[:, (0, 1, 2, 5)]
numpy[0, [0]]
numpy[:, [i]]
numpy[1 : c + 1, c]
numpy[-(c + 1) :, d]
numpy[:, l[-2]]
numpy[:, ::-1]
numpy[np.newaxis, :]
(str or None) if (sys.version_info[0] > (3,)) else (str or bytes or None)
{'2.7': dead, '3.7': long_live or die_hard}
{'2.7', '3.6', '3.7', '3.8', '3.9', '4.0' if gilectomy else '3.10'}
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or A, 11 or B, 12 or C]
(SomeName)
SomeName
(Good, Bad, Ugly)
(i for i in (1, 2, 3))
((i ** 2) for i in (1, 2, 3))
((i ** 2) for i, _ in ((1, 'a'), (2, 'b'), (3, 'c')))
(((i ** 2) + j) for i in (1, 2, 3) for j in (1, 2, 3))
(*starred,)
{"id": "1","type": "type","started_at": now(),"ended_at": now() + timedelta(days=10),"priority": 1,"import_session_id": 1,**kwargs}
a = (1,)
b = 1,
c = 1
d = (1,) + a + (2,)
e = (1,).count(1)
f = 1, *range(10)
g = 1, *"ten"
what_is_up_with_those_new_coord_names = (coord_names + set(vars_to_create)) + set(vars_to_remove)
what_is_up_with_those_new_coord_names = (coord_names | set(vars_to_create)) - set(vars_to_remove)
result = session.query(models.Customer.id).filter(models.Customer.account_id == account_id, models.Customer.email == email_address).order_by(models.Customer.id.asc()).all()
result = session.query(models.Customer.id).filter(models.Customer.account_id == account_id, models.Customer.email == email_address).order_by(models.Customer.id.asc(),).all()
Ø = set()
authors.łukasz.say_thanks()
mapping = {
A: 0.25 * (10.0 / 12),
B: 0.1 * (10.0 / 12),
C: 0.1 * (10.0 / 12),
D: 0.1 * (10.0 / 12),
}
def gen():
yield from outside_of_generator
a = (yield)
b = ((yield))
c = (((yield)))
async def f():
await some.complicated[0].call(with_args=(True or (1 is not 1)))
print(* [] or [1])
print(**{1: 3} if False else {x: x for x in range(3)})
print(* lambda x: x)
assert(not Test),("Short message")
assert this is ComplexTest and not requirements.fit_in_a_single_line(force=False), "Short message"
assert(((parens is TooMany)))
for x, in (1,), (2,), (3,): ...
for y in (): ...
for z in (i for i in (1, 2, 3)): ...
for i in (call()): ...
for j in (1 + (2 + 3)): ...
while(this and that): ...
for addr_family, addr_type, addr_proto, addr_canonname, addr_sockaddr in socket.getaddrinfo('google.com', 'http'):
pass
a = aaaa.bbbb.cccc.dddd.eeee.ffff.gggg.hhhh.iiii.jjjj.kkkk.llll.mmmm.nnnn.oooo.pppp in qqqq.rrrr.ssss.tttt.uuuu.vvvv.xxxx.yyyy.zzzz
a = aaaa.bbbb.cccc.dddd.eeee.ffff.gggg.hhhh.iiii.jjjj.kkkk.llll.mmmm.nnnn.oooo.pppp not in qqqq.rrrr.ssss.tttt.uuuu.vvvv.xxxx.yyyy.zzzz
a = aaaa.bbbb.cccc.dddd.eeee.ffff.gggg.hhhh.iiii.jjjj.kkkk.llll.mmmm.nnnn.oooo.pppp is qqqq.rrrr.ssss.tttt.uuuu.vvvv.xxxx.yyyy.zzzz
a = aaaa.bbbb.cccc.dddd.eeee.ffff.gggg.hhhh.iiii.jjjj.kkkk.llll.mmmm.nnnn.oooo.pppp is not qqqq.rrrr.ssss.tttt.uuuu.vvvv.xxxx.yyyy.zzzz
if (
threading.current_thread() != threading.main_thread() and
threading.current_thread() != threading.main_thread() or
signal.getsignal(signal.SIGINT) != signal.default_int_handler
):
return True
if (
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa |
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
):
return True
if (
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa &
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
):
return True
if (
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
):
return True
if (
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa -
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
):
return True
if (
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa *
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
):
return True
if (
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa /
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
):
return True
if (
~ aaaa.a + aaaa.b - aaaa.c * aaaa.d / aaaa.e | aaaa.f & aaaa.g % aaaa.h ^ aaaa.i << aaaa.k >> aaaa.l ** aaaa.m // aaaa.n
):
return True
if (
~ aaaaaaaa.a + aaaaaaaa.b - aaaaaaaa.c @ aaaaaaaa.d / aaaaaaaa.e | aaaaaaaa.f & aaaaaaaa.g % aaaaaaaa.h ^ aaaaaaaa.i << aaaaaaaa.k >> aaaaaaaa.l ** aaaaaaaa.m // aaaaaaaa.n
):
return True
if (
~ aaaaaaaaaaaaaaaa.a + aaaaaaaaaaaaaaaa.b - aaaaaaaaaaaaaaaa.c * aaaaaaaaaaaaaaaa.d @ aaaaaaaaaaaaaaaa.e | aaaaaaaaaaaaaaaa.f & aaaaaaaaaaaaaaaa.g % aaaaaaaaaaaaaaaa.h ^ aaaaaaaaaaaaaaaa.i << aaaaaaaaaaaaaaaa.k >> aaaaaaaaaaaaaaaa.l ** aaaaaaaaaaaaaaaa.m // aaaaaaaaaaaaaaaa.n
):
return True
aaaaaaaaaaaaaaaa + aaaaaaaaaaaaaaaa - aaaaaaaaaaaaaaaa * (aaaaaaaaaaaaaaaa + aaaaaaaaaaaaaaaa) / (aaaaaaaaaaaaaaaa + aaaaaaaaaaaaaaaa + aaaaaaaaaaaaaaaa)
aaaaaaaaaaaaaaaa + aaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa >> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa << aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
bbbb >> bbbb * bbbb
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ^bbbb.a & aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa^aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
last_call()
# standalone comment at ENDMARKER

View file

@ -0,0 +1,229 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
#!/usr/bin/env python3
import asyncio
import sys
from third_party import X, Y, Z
from library import some_connection, some_decorator
# fmt: off
from third_party import (X,
Y, Z)
# fmt: on
f"trigger 3.6 mode"
# Comment 1
# Comment 2
# fmt: off
def func_no_args():
a; b; c
if True: raise RuntimeError
if False: ...
for i in range(10):
print(i)
continue
exec('new-style exec', {}, {})
return None
async def coroutine(arg, exec=False):
'Single-line docstring. Multiline is harder to reformat.'
async with some_connection() as conn:
await conn.do_what_i_mean('SELECT bobby, tables FROM xkcd', timeout=2)
await asyncio.sleep(1)
@asyncio.coroutine
@some_decorator(
with_args=True,
many_args=[1,2,3]
)
def function_signature_stress_test(number:int,no_annotation=None,text:str='default',* ,debug:bool=False,**kwargs) -> str:
return text[number:-1]
# fmt: on
def spaces(a=1, b=(), c=[], d={}, e=True, f=-1, g=1 if False else 2, h="", i=r""):
offset = attr.ib(default=attr.Factory(lambda: _r.uniform(1, 2)))
assert task._cancel_stack[: len(old_stack)] == old_stack
def spaces_types(
a: int = 1,
b: tuple = (),
c: list = [],
d: dict = {},
e: bool = True,
f: int = -1,
g: int = 1 if False else 2,
h: str = "",
i: str = r"",
):
...
def spaces2(result=_core.Value(None)):
...
something = {
# fmt: off
key: 'value',
}
def subscriptlist():
atom[
# fmt: off
'some big and',
'complex subscript',
# fmt: on
goes + here,
andhere,
]
def import_as_names():
# fmt: off
from hello import a, b
'unformatted'
# fmt: on
def testlist_star_expr():
# fmt: off
a , b = *hello
'unformatted'
# fmt: on
def yield_expr():
# fmt: off
yield hello
'unformatted'
# fmt: on
"formatted"
# fmt: off
( yield hello )
'unformatted'
# fmt: on
def example(session):
# fmt: off
result = session\
.query(models.Customer.id)\
.filter(models.Customer.account_id == account_id,
models.Customer.email == email_address)\
.order_by(models.Customer.id.asc())\
.all()
# fmt: on
def off_and_on_without_data():
"""All comments here are technically on the same prefix.
The comments between will be formatted. This is a known limitation.
"""
# fmt: off
# hey, that won't work
# fmt: on
pass
def on_and_off_broken():
"""Another known limitation."""
# fmt: on
# fmt: off
this=should.not_be.formatted()
and_=indeed . it is not formatted
because . the . handling . inside . generate_ignored_nodes()
now . considers . multiple . fmt . directives . within . one . prefix
# fmt: on
# fmt: off
# ...but comments still get reformatted even though they should not be
# fmt: on
def long_lines():
if True:
typedargslist.extend(
gen_annotated_params(
ast_args.kwonlyargs,
ast_args.kw_defaults,
parameters,
implicit_default=True,
)
)
# fmt: off
a = (
unnecessary_bracket()
)
# fmt: on
_type_comment_re = re.compile(
r"""
^
[\t ]*
\#[ ]type:[ ]*
(?P<type>
[^#\t\n]+?
)
(?<!ignore) # note: this will force the non-greedy + in <type> to match
# a trailing space which is why we need the silliness below
(?<!ignore[ ]{1})(?<!ignore[ ]{2})(?<!ignore[ ]{3})(?<!ignore[ ]{4})
(?<!ignore[ ]{5})(?<!ignore[ ]{6})(?<!ignore[ ]{7})(?<!ignore[ ]{8})
(?<!ignore[ ]{9})(?<!ignore[ ]{10})
[\t ]*
(?P<nl>
(?:\#[^\n]*)?
\n?
)
$
""",
# fmt: off
re.MULTILINE|re.VERBOSE
# fmt: on
)
def single_literal_yapf_disable():
"""Black does not support this."""
BAZ = {(1, 2, 3, 4), (5, 6, 7, 8), (9, 10, 11, 12)} # yapf: disable
cfg.rule(
"Default",
"address",
xxxx_xxxx=["xxx-xxxxxx-xxxxxxxxxx"],
xxxxxx="xx_xxxxx",
xxxxxxx="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
xxxxxxxxx_xxxx=True,
xxxxxxxx_xxxxxxxxxx=False,
xxxxxx_xxxxxx=2,
xxxxxx_xxxxx_xxxxxxxx=70,
xxxxxx_xxxxxx_xxxxx=True,
# fmt: off
xxxxxxx_xxxxxxxxxxxx={
"xxxxxxxx": {
"xxxxxx": False,
"xxxxxxx": False,
"xxxx_xxxxxx": "xxxxx",
},
"xxxxxxxx-xxxxx": {
"xxxxxx": False,
"xxxxxxx": True,
"xxxx_xxxxxx": "xxxxxx",
},
},
# fmt: on
xxxxxxxxxx_xxxxxxxxxxx_xxxxxxx_xxxxxxxxx=5,
)
# fmt: off
yield 'hello'
# No formatting to the end of the file
l=[1,2,3]
d={'a':1,
'b':2}

View file

@ -0,0 +1,46 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
import pytest
TmSt = 1
TmEx = 2
# fmt: off
# Test data:
# Position, Volume, State, TmSt/TmEx/None, [call, [arg1...]]
@pytest.mark.parametrize('test', [
# Test don't manage the volume
[
('stuff', 'in')
],
])
def test_fader(test):
pass
def check_fader(test):
pass
def verify_fader(test):
# misaligned comment
pass
def verify_fader(test):
"""Hey, ho."""
assert test.passed()
def test_calculate_fades():
calcs = [
# one is zero/none
(0, 4, 0, 0, 10, 0, 0, 6, 10),
(None, 4, 0, 0, 10, 0, 0, 6, 10),
]
# fmt: on

View file

@ -0,0 +1,21 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
# fmt: off
x = [
1, 2,
3, 4,
]
# fmt: on
# fmt: off
x = [
1, 2,
3, 4,
]
# fmt: on
x = [1, 2, 3, 4]

View file

@ -0,0 +1,26 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
# fmt: off
@test([
1, 2,
3, 4,
])
# fmt: on
def f():
pass
@test(
[
1,
2,
3,
4,
]
)
def f():
pass

View file

@ -0,0 +1,93 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
# Regression test for https://github.com/psf/black/issues/3129.
setup(
entry_points={
# fmt: off
"console_scripts": [
"foo-bar"
"=foo.bar.:main",
# fmt: on
] # Includes an formatted indentation.
},
)
# Regression test for https://github.com/psf/black/issues/2015.
run(
# fmt: off
[
"ls",
"-la",
]
# fmt: on
+ path,
check=True,
)
# Regression test for https://github.com/psf/black/issues/3026.
def test_func():
# yapf: disable
if unformatted( args ):
return True
# yapf: enable
elif b:
return True
return False
# Regression test for https://github.com/psf/black/issues/2567.
if True:
# fmt: off
for _ in range( 1 ):
# fmt: on
print ( "This won't be formatted" )
print ( "This won't be formatted either" )
else:
print("This will be formatted")
# Regression test for https://github.com/psf/black/issues/3184.
class A:
async def call(param):
if param:
# fmt: off
if param[0:4] in (
"ABCD", "EFGH"
) :
# fmt: on
print ( "This won't be formatted" )
elif param[0:4] in ("ZZZZ",):
print ( "This won't be formatted either" )
print("This will be formatted")
# Regression test for https://github.com/psf/black/issues/2985.
class Named(t.Protocol):
# fmt: off
@property
def this_wont_be_formatted ( self ) -> str: ...
class Factory(t.Protocol):
def this_will_be_formatted(self, **kwargs) -> Named:
...
# fmt: on
# Regression test for https://github.com/psf/black/issues/3436.
if x:
return x
# fmt: off
elif unformatted:
# fmt: on
will_be_formatted()

View file

@ -0,0 +1,9 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
a, b = 1, 2
c = 6 # fmt: skip
d = 5

View file

@ -0,0 +1,17 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
l1 = [
"This list should be broken up",
"into multiple lines",
"because it is way too long",
]
l2 = ["But this list shouldn't", "even though it also has", "way too many characters in it"] # fmt: skip
l3 = [
"I have",
"trailing comma",
"so I should be braked",
]

View file

@ -0,0 +1,16 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
a = 3
# fmt: off
b, c = 1, 2
d = 6 # fmt: skip
e = 5
# fmt: on
f = [
"This is a very long line that should be formatted into a clearer line ",
"by rearranging.",
]

View file

@ -0,0 +1,13 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
a = 2
# fmt: skip
l = [
1,
2,
3,
]

View file

@ -0,0 +1,15 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
a, b, c = 3, 4, 5
if (
a == 3
and b != 9 # fmt: skip
and c is not None
):
print("I'm good!")
else:
print("I'm bad")

View file

@ -0,0 +1,11 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
class A:
def f(self):
for line in range(10):
if True:
pass # fmt: skip

View file

@ -0,0 +1,10 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
a = "this is some code"
b = 5 # fmt:skip
c = 9 # fmt: skip
d = "thisisasuperlongstringthisisasuperlongstringthisisasuperlongstringthisisasuperlongstring" # fmt:skip

View file

@ -0,0 +1,68 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
# Make sure a leading comment is not removed.
def some_func( unformatted, args ): # fmt: skip
print("I am some_func")
return 0
# Make sure this comment is not removed.
# Make sure a leading comment is not removed.
async def some_async_func( unformatted, args): # fmt: skip
print("I am some_async_func")
await asyncio.sleep(1)
# Make sure a leading comment is not removed.
class SomeClass( Unformatted, SuperClasses ): # fmt: skip
def some_method( self, unformatted, args ): # fmt: skip
print("I am some_method")
return 0
async def some_async_method( self, unformatted, args ): # fmt: skip
print("I am some_async_method")
await asyncio.sleep(1)
# Make sure a leading comment is not removed.
if unformatted_call( args ): # fmt: skip
print("First branch")
# Make sure this is not removed.
elif another_unformatted_call( args ): # fmt: skip
print("Second branch")
else : # fmt: skip
print("Last branch")
while some_condition( unformatted, args ): # fmt: skip
print("Do something")
for i in some_iter( unformatted, args ): # fmt: skip
print("Do something")
async def test_async_for():
async for i in some_async_iter( unformatted, args ): # fmt: skip
print("Do something")
try : # fmt: skip
some_call()
except UnformattedError as ex: # fmt: skip
handle_exception()
finally : # fmt: skip
finally_call()
with give_me_context( unformatted, args ): # fmt: skip
print("Do something")
async def test_async_with():
async with give_me_async_context( unformatted, args ): # fmt: skip
print("Do something")

View file

@ -0,0 +1,15 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
f"f-string without formatted values is just a string"
f"{{NOT a formatted value}}"
f'{{NOT \'a\' "formatted" "value"}}'
f"some f-string with {a} {few():.2f} {formatted.values!r}"
f'some f-string with {a} {few(""):.2f} {formatted.values!r}'
f"{f'''{'nested'} inner'''} outer"
f"\"{f'{nested} inner'}\" outer"
f"space between opening braces: { {a for a in (1, 2, 3)}}"
f'Hello \'{tricky + "example"}\''

View file

@ -0,0 +1,101 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
#!/usr/bin/env python3
import asyncio
import sys
from third_party import X, Y, Z
from library import some_connection, \
some_decorator
f'trigger 3.6 mode'
def func_no_args():
a; b; c
if True: raise RuntimeError
if False: ...
for i in range(10):
print(i)
continue
exec("new-style exec", {}, {})
return None
async def coroutine(arg, exec=False):
"Single-line docstring. Multiline is harder to reformat."
async with some_connection() as conn:
await conn.do_what_i_mean('SELECT bobby, tables FROM xkcd', timeout=2)
await asyncio.sleep(1)
@asyncio.coroutine
@some_decorator(
with_args=True,
many_args=[1,2,3]
)
def function_signature_stress_test(number:int,no_annotation=None,text:str="default",* ,debug:bool=False,**kwargs) -> str:
return text[number:-1]
def spaces(a=1, b=(), c=[], d={}, e=True, f=-1, g=1 if False else 2, h="", i=r''):
offset = attr.ib(default=attr.Factory( lambda: _r.uniform(10000, 200000)))
assert task._cancel_stack[:len(old_stack)] == old_stack
def spaces_types(a: int = 1, b: tuple = (), c: list = [], d: dict = {}, e: bool = True, f: int = -1, g: int = 1 if False else 2, h: str = "", i: str = r''): ...
def spaces2(result= _core.Value(None)):
assert fut is self._read_fut, (fut, self._read_fut)
def example(session):
result = session.query(models.Customer.id).filter(
models.Customer.account_id == account_id,
models.Customer.email == email_address,
).order_by(
models.Customer.id.asc()
).all()
def long_lines():
if True:
typedargslist.extend(
gen_annotated_params(ast_args.kwonlyargs, ast_args.kw_defaults, parameters, implicit_default=True)
)
typedargslist.extend(
gen_annotated_params(
ast_args.kwonlyargs, ast_args.kw_defaults, parameters, implicit_default=True,
# trailing standalone comment
)
)
_type_comment_re = re.compile(
r"""
^
[\t ]*
\#[ ]type:[ ]*
(?P<type>
[^#\t\n]+?
)
(?<!ignore) # note: this will force the non-greedy + in <type> to match
# a trailing space which is why we need the silliness below
(?<!ignore[ ]{1})(?<!ignore[ ]{2})(?<!ignore[ ]{3})(?<!ignore[ ]{4})
(?<!ignore[ ]{5})(?<!ignore[ ]{6})(?<!ignore[ ]{7})(?<!ignore[ ]{8})
(?<!ignore[ ]{9})(?<!ignore[ ]{10})
[\t ]*
(?P<nl>
(?:\#[^\n]*)?
\n?
)
$
""", re.MULTILINE | re.VERBOSE
)
def trailing_comma():
mapping = {
A: 0.25 * (10.0 / 12),
B: 0.1 * (10.0 / 12),
C: 0.1 * (10.0 / 12),
D: 0.1 * (10.0 / 12),
}
def f(
a,
**kwargs,
) -> A:
return (
yield from A(
very_long_argument_name1=very_long_value_for_the_argument,
very_long_argument_name2=very_long_value_for_the_argument,
**kwargs,
)
)
def __await__(): return (yield)

View file

@ -0,0 +1,59 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
def f(
a,
**kwargs,
) -> A:
with cache_dir():
if something:
result = (
CliRunner().invoke(black.main, [str(src1), str(src2), "--diff", "--check"])
)
limited.append(-limited.pop()) # negate top
return A(
very_long_argument_name1=very_long_value_for_the_argument,
very_long_argument_name2=-very.long.value.for_the_argument,
**kwargs,
)
def g():
"Docstring."
def inner():
pass
print("Inner defs should breathe a little.")
def h():
def inner():
pass
print("Inner defs should breathe a little.")
if os.name == "posix":
import termios
def i_should_be_followed_by_only_one_newline():
pass
elif os.name == "nt":
try:
import msvcrt
def i_should_be_followed_by_only_one_newline():
pass
except ImportError:
def i_should_be_followed_by_only_one_newline():
pass
elif False:
class IHopeYouAreHavingALovelyDay:
def __call__(self):
print("i_should_be_followed_by_only_one_newline")
else:
def foo():
pass
with hmm_but_this_should_get_two_preceding_newlines():
pass

View file

@ -0,0 +1,67 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
def f(a,):
d = {'key': 'value',}
tup = (1,)
def f2(a,b,):
d = {'key': 'value', 'key2': 'value2',}
tup = (1,2,)
def f(a:int=1,):
call(arg={'explode': 'this',})
call2(arg=[1,2,3],)
x = {
"a": 1,
"b": 2,
}["a"]
if a == {"a": 1,"b": 2,"c": 3,"d": 4,"e": 5,"f": 6,"g": 7,"h": 8,}["a"]:
pass
def xxxxxxxxxxxxxxxxxxxxxxxxxxxx() -> Set[
"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
]:
json = {"k": {"k2": {"k3": [1,]}}}
# The type annotation shouldn't get a trailing comma since that would change its type.
# Relevant bug report: https://github.com/psf/black/issues/2381.
def some_function_with_a_really_long_name() -> (
returning_a_deeply_nested_import_of_a_type_i_suppose
):
pass
def some_method_with_a_really_long_name(very_long_parameter_so_yeah: str, another_long_parameter: int) -> (
another_case_of_returning_a_deeply_nested_import_of_a_type_i_suppose_cause_why_not
):
pass
def func() -> (
also_super_long_type_annotation_that_may_cause_an_AST_related_crash_in_black(this_shouldn_t_get_a_trailing_comma_too)
):
pass
def func() -> ((also_super_long_type_annotation_that_may_cause_an_AST_related_crash_in_black(
this_shouldn_t_get_a_trailing_comma_too
))
):
pass
# Make sure inner one-element tuple won't explode
some_module.some_function(
argument1, (one_element_tuple,), argument4, argument5, argument6
)
# Inner trailing comma causes outer to explode
some_module.some_function(
argument1, (one, two,), argument4, argument5, argument6
)

View file

@ -0,0 +1,69 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
def function(**kwargs):
t = a**2 + b**3
return t ** 2
def function_replace_spaces(**kwargs):
t = a **2 + b** 3 + c ** 4
def function_dont_replace_spaces():
{**a, **b, **c}
a = 5**~4
b = 5 ** f()
c = -(5**2)
d = 5 ** f["hi"]
e = lazy(lambda **kwargs: 5)
f = f() ** 5
g = a.b**c.d
h = 5 ** funcs.f()
i = funcs.f() ** 5
j = super().name ** 5
k = [(2**idx, value) for idx, value in pairs]
l = mod.weights_[0] == pytest.approx(0.95**100, abs=0.001)
m = [([2**63], [1, 2**63])]
n = count <= 10**5
o = settings(max_examples=10**6)
p = {(k, k**2): v**2 for k, v in pairs}
q = [10**i for i in range(6)]
r = x**y
a = 5.0**~4.0
b = 5.0 ** f()
c = -(5.0**2.0)
d = 5.0 ** f["hi"]
e = lazy(lambda **kwargs: 5)
f = f() ** 5.0
g = a.b**c.d
h = 5.0 ** funcs.f()
i = funcs.f() ** 5.0
j = super().name ** 5.0
k = [(2.0**idx, value) for idx, value in pairs]
l = mod.weights_[0] == pytest.approx(0.95**100, abs=0.001)
m = [([2.0**63.0], [1.0, 2**63.0])]
n = count <= 10**5.0
o = settings(max_examples=10**6.0)
p = {(k, k**2): v**2.0 for k, v in pairs}
q = [10.5**i for i in range(6)]
# WE SHOULD DEFINITELY NOT EAT THESE COMMENTS (https://github.com/psf/black/issues/2873)
if hasattr(view, "sum_of_weights"):
return np.divide( # type: ignore[no-any-return]
view.variance, # type: ignore[union-attr]
view.sum_of_weights, # type: ignore[union-attr]
out=np.full(view.sum_of_weights.shape, np.nan), # type: ignore[union-attr]
where=view.sum_of_weights**2 > view.sum_of_weights_squared, # type: ignore[union-attr]
)
return np.divide(
where=view.sum_of_weights_of_weight_long**2 > view.sum_of_weights_squared, # type: ignore
)

View file

@ -0,0 +1,61 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
x = (1)
x = (1.2)
data = (
"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
).encode()
async def show_status():
while True:
try:
if report_host:
data = (
f"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
).encode()
except Exception as e:
pass
def example():
return (("xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"))
def example1():
return ((1111111111111111111111111111111111111111111111111111111111111111111111111111111111111))
def example1point5():
return ((((((1111111111111111111111111111111111111111111111111111111111111111111111111111111111111))))))
def example2():
return (("xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"))
def example3():
return ((1111111111111111111111111111111111111111111111111111111111111111111111111111111))
def example4():
return ((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((True))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
def example5():
return ((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((()))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
def example6():
return ((((((((({a:a for a in [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]})))))))))
def example7():
return ((((((((({a:a for a in [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20000000000000000000]})))))))))
def example8():
return (((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((None)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))

View file

@ -0,0 +1,37 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
slice[a.b : c.d]
slice[d :: d + 1]
slice[d + 1 :: d]
slice[d::d]
slice[0]
slice[-1]
slice[:-1]
slice[::-1]
slice[:c, c - 1]
slice[c, c + 1, d::]
slice[ham[c::d] :: 1]
slice[ham[cheese**2 : -1] : 1 : 1, ham[1:2]]
slice[:-1:]
slice[lambda: None : lambda: None]
slice[lambda x, y, *args, really=2, **kwargs: None :, None::]
slice[1 or 2 : True and False]
slice[not so_simple : 1 < val <= 10]
slice[(1 for i in range(42)) : x]
slice[:: [i for i in range(42)]]
async def f():
slice[await x : [i async for i in arange(42)] : 42]
# These are from PEP-8:
ham[1:9], ham[1:9:3], ham[:9:3], ham[1::3], ham[1:9:]
ham[lower:upper], ham[lower:upper:], ham[lower::step]
# ham[lower+offset : upper+offset]
ham[: upper_fn(x) : step_fn(x)], ham[:: step_fn(x)]
ham[lower + offset : upper + offset]

View file

@ -0,0 +1,26 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
#!/usr/bin/env python3
name = "Łukasz"
(f"hello {name}", f"hello {name}")
(b"", b"")
("", "")
(r"", R"")
(rf"", rf"", Rf"", Rf"", rf"", rf"", Rf"", Rf"")
(rb"", rb"", Rb"", Rb"", rb"", rb"", Rb"", Rb"")
def docstring_singleline():
R"""2020 was one hell of a year. The good news is that we were able to"""
def docstring_multiline():
R"""
clear out all of the issues opened in that time :p
"""

View file

@ -0,0 +1,64 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
importA
(
()
<< 0
** 101234234242352525425252352352525234890264906820496920680926538059059209922523523525
) #
assert sort_by_dependency(
{
"1": {"2", "3"},
"2": {"2a", "2b"},
"3": {"3a", "3b"},
"2a": set(),
"2b": set(),
"3a": set(),
"3b": set(),
}
) == ["2a", "2b", "2", "3a", "3b", "3", "1"]
importA
0
0 ^ 0 #
class A:
def foo(self):
for _ in range(10):
aaaaaaaaaaaaaaaaaaa = bbbbbbbbbbbbbbb.cccccccccc(
xxxxxxxxxxxx
) # pylint: disable=no-member
def test(self, othr):
return 1 == 2 and (
name,
description,
self.default,
self.selected,
self.auto_generated,
self.parameters,
self.meta_data,
self.schedule,
) == (
name,
description,
othr.default,
othr.selected,
othr.auto_generated,
othr.parameters,
othr.meta_data,
othr.schedule,
)
assert a_function(
very_long_arguments_that_surpass_the_limit,
which_is_eighty_eight_in_this_case_plus_a_bit_more,
) == {"x": "this need to pass the line limit as well", "b": "but only by a little bit"}

View file

@ -0,0 +1,40 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
if e1234123412341234.winerror not in (
_winapi.ERROR_SEM_TIMEOUT,
_winapi.ERROR_PIPE_BUSY,
) or _check_timeout(t):
pass
if x:
if y:
new_id = (
max(
Vegetable.objects.order_by("-id")[0].id,
Mineral.objects.order_by("-id")[0].id,
)
+ 1
)
class X:
def get_help_text(self):
return ngettext(
"Your password must contain at least %(min_length)d character.",
"Your password must contain at least %(min_length)d characters.",
self.min_length,
) % {"min_length": self.min_length}
class A:
def b(self):
if self.connection.mysql_is_mariadb and (
10,
4,
3,
) < self.connection.mysql_version < (10, 5, 2):
pass

View file

@ -0,0 +1,12 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
if e123456.get_tk_patchlevel() >= (8, 6, 0, "final") or (
8,
5,
8,
) <= get_tk_patchlevel() < (8, 6):
pass

View file

@ -0,0 +1,14 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
if True:
if True:
if True:
return _(
"qweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweas "
+ "qweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqwegqweasdzxcqweasdzxc.",
"qweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqweasdzxcqwe",
) % {"reported_username": reported_username, "report_reason": report_reason}

View file

@ -0,0 +1,15 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
ä = 1
µ = 2
蟒 = 3
x󠄀 = 4
មុ = 1
Q̇_per_meter = 4
A᧚ = 3
A፩ = 8

View file

@ -0,0 +1,13 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
# This is a standalone comment.
sdfjklsdfsjldkflkjsf, sdfjsdfjlksdljkfsdlkf, sdfsdjfklsdfjlksdljkf, sdsfsdfjskdflsfsdf = 1, 2, 3
# This is as well.
this_will_be_wrapped_in_parens, = struct.unpack(b"12345678901234567890")
(a,) = call()

View file

@ -0,0 +1,7 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
print("hello, world")

View file

@ -0,0 +1,9 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
for ((x in {}) or {})["a"] in x:
pass
pem_spam = lambda l, spam={"x": 3}: not spam.get(l.strip())
lambda x=lambda y={1: 3}: y["x" : lambda y: {1: 2}]: x

View file

@ -0,0 +1,36 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
class SimpleClassWithBlankParentheses:
pass
class ClassWithSpaceParentheses:
first_test_data = 90
second_test_data = 100
def test_func(self):
return None
class ClassWithEmptyFunc(object):
def func_with_blank_parentheses():
return 5
def public_func_with_blank_parentheses():
return None
def class_under_the_func_with_blank_parentheses():
class InsideFunc:
pass
class NormalClass:
def func_for_testing(self, first, second):
sum = first + second
return sum

View file

@ -0,0 +1,171 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
class ClassSimplest:
pass
class ClassWithSingleField:
a = 1
class ClassWithJustTheDocstring:
"""Just a docstring."""
class ClassWithInit:
def __init__(self):
pass
class ClassWithTheDocstringAndInit:
"""Just a docstring."""
def __init__(self):
pass
class ClassWithInitAndVars:
cls_var = 100
def __init__(self):
pass
class ClassWithInitAndVarsAndDocstring:
"""Test class"""
cls_var = 100
def __init__(self):
pass
class ClassWithDecoInit:
@deco
def __init__(self):
pass
class ClassWithDecoInitAndVars:
cls_var = 100
@deco
def __init__(self):
pass
class ClassWithDecoInitAndVarsAndDocstring:
"""Test class"""
cls_var = 100
@deco
def __init__(self):
pass
class ClassSimplestWithInner:
class Inner:
pass
class ClassSimplestWithInnerWithDocstring:
class Inner:
"""Just a docstring."""
def __init__(self):
pass
class ClassWithSingleFieldWithInner:
a = 1
class Inner:
pass
class ClassWithJustTheDocstringWithInner:
"""Just a docstring."""
class Inner:
pass
class ClassWithInitWithInner:
class Inner:
pass
def __init__(self):
pass
class ClassWithInitAndVarsWithInner:
cls_var = 100
class Inner:
pass
def __init__(self):
pass
class ClassWithInitAndVarsAndDocstringWithInner:
"""Test class"""
cls_var = 100
class Inner:
pass
def __init__(self):
pass
class ClassWithDecoInitWithInner:
class Inner:
pass
@deco
def __init__(self):
pass
class ClassWithDecoInitAndVarsWithInner:
cls_var = 100
class Inner:
pass
@deco
def __init__(self):
pass
class ClassWithDecoInitAndVarsAndDocstringWithInner:
"""Test class"""
cls_var = 100
class Inner:
pass
@deco
def __init__(self):
pass
class ClassWithDecoInitAndVarsAndDocstringWithInner2:
"""Test class"""
class Inner:
pass
cls_var = 100
@deco
def __init__(self):
pass

View file

@ -0,0 +1,105 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
import core, time, a
from . import A, B, C
# keeps existing trailing comma
from foo import (
bar,
)
# also keeps existing structure
from foo import (
baz,
qux,
)
# `as` works as well
from foo import (
xyzzy as magic,
)
a = {
1,
2,
3,
}
b = {1, 2, 3}
c = {
1,
2,
3,
}
x = (1,)
y = (narf(),)
nested = {
(1, 2, 3),
(4, 5, 6),
}
nested_no_trailing_comma = {(1, 2, 3), (4, 5, 6)}
nested_long_lines = [
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"cccccccccccccccccccccccccccccccccccccccc",
(1, 2, 3),
"dddddddddddddddddddddddddddddddddddddddd",
]
{
"oneple": (1,),
}
{"oneple": (1,)}
["ls", "lsoneple/%s" % (foo,)]
x = {"oneple": (1,)}
y = {
"oneple": (1,),
}
assert False, (
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa wraps %s"
% bar
)
# looping over a 1-tuple should also not get wrapped
for x in (1,):
pass
for (x,) in (1,), (2,), (3,):
pass
[
1,
2,
3,
]
division_result_tuple = (6 / 2,)
print("foo %r", (foo.bar,))
if True:
IGNORED_TYPES_FOR_ATTRIBUTE_CHECKING = (
Config.IGNORED_TYPES_FOR_ATTRIBUTE_CHECKING
| {pylons.controllers.WSGIController}
)
if True:
ec2client.get_waiter("instance_stopped").wait(
InstanceIds=[instance.id],
WaiterConfig={
"Delay": 5,
},
)
ec2client.get_waiter("instance_stopped").wait(
InstanceIds=[instance.id],
WaiterConfig={
"Delay": 5,
},
)
ec2client.get_waiter("instance_stopped").wait(
InstanceIds=[instance.id],
WaiterConfig={
"Delay": 5,
},
)

View file

@ -0,0 +1,70 @@
---
source: src/source_code/mod.rs
assertion_line: 0
expression: formatted
---
"""The asyncio package, tracking PEP 3156."""
# flake8: noqa
from logging import WARNING
from logging import (
ERROR,
)
import sys
# This relies on each of the submodules having an __all__ variable.
from .base_events import *
from .coroutines import *
from .events import * # comment here
from .futures import *
from .locks import * # comment here
from .protocols import *
from ..runners import * # comment here
from ..queues import *
from ..streams import *
from some_library import (
Just,
Enough,
Libraries,
To,
Fit,
In,
This,
Nice,
Split,
Which,
We,
No,
Longer,
Use,
)
from name_of_a_company.extremely_long_project_name.component.ttypes import (
CuteLittleServiceHandlerFactoryyy,
)
from name_of_a_company.extremely_long_project_name.extremely_long_component_name.ttypes import *
from .a.b.c.subprocess import *
from . import tasks
from . import A, B, C
from . import (
SomeVeryLongNameAndAllOfItsAdditionalLetters1,
SomeVeryLongNameAndAllOfItsAdditionalLetters2,
)
__all__ = (
base_events.__all__
+ coroutines.__all__
+ events.__all__
+ futures.__all__
+ locks.__all__
+ protocols.__all__
+ runners.__all__
+ queues.__all__
+ streams.__all__
+ tasks.__all__
)

View file

@ -0,0 +1,7 @@
#![cfg(test)]
use std::path::Path;
pub fn test_resource_path(path: impl AsRef<Path>) -> std::path::PathBuf {
Path::new("./resources/test/").join(path)
}

View file

@ -0,0 +1,855 @@
use rustc_hash::FxHashMap;
use rustpython_parser::ast::Location;
use rustpython_parser::lexer::{LexResult, Tok};
use crate::core::types::Range;
use crate::cst::{Alias, Excepthandler, ExcepthandlerKind, Expr, ExprKind, Stmt, StmtKind};
#[derive(Clone, Debug)]
pub enum Node<'a> {
Mod(&'a [Stmt]),
Stmt(&'a Stmt),
Expr(&'a Expr),
Alias(&'a Alias),
Excepthandler(&'a Excepthandler),
}
impl Node<'_> {
pub fn id(&self) -> usize {
match self {
Node::Mod(nodes) => nodes as *const _ as usize,
Node::Stmt(node) => node.id(),
Node::Expr(node) => node.id(),
Node::Alias(node) => node.id(),
Node::Excepthandler(node) => node.id(),
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum TriviaTokenKind {
StandaloneComment,
InlineComment,
MagicTrailingComma,
EmptyLine,
Parentheses,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct TriviaToken {
pub start: Location,
pub end: Location,
pub kind: TriviaTokenKind,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum TriviaKind {
StandaloneComment(Range),
InlineComment(Range),
MagicTrailingComma,
EmptyLine,
Parentheses,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Relationship {
Leading,
Trailing,
Dangling,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Parenthesize {
/// Always parenthesize the statement or expression.
Always,
/// Never parenthesize the statement or expression.
Never,
/// Parenthesize the statement or expression if it expands.
IfExpanded,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Trivia {
pub kind: TriviaKind,
pub relationship: Relationship,
}
impl Trivia {
pub fn from_token(token: &TriviaToken, relationship: Relationship) -> Self {
match token.kind {
TriviaTokenKind::MagicTrailingComma => Self {
kind: TriviaKind::MagicTrailingComma,
relationship,
},
TriviaTokenKind::EmptyLine => Self {
kind: TriviaKind::EmptyLine,
relationship,
},
TriviaTokenKind::StandaloneComment => Self {
kind: TriviaKind::StandaloneComment(Range::new(token.start, token.end)),
relationship,
},
TriviaTokenKind::InlineComment => Self {
kind: TriviaKind::InlineComment(Range::new(token.start, token.end)),
relationship,
},
TriviaTokenKind::Parentheses => Self {
kind: TriviaKind::Parentheses,
relationship,
},
}
}
}
pub fn extract_trivia_tokens(lxr: &[LexResult]) -> Vec<TriviaToken> {
let mut tokens = vec![];
let mut prev_tok: Option<(&Location, &Tok, &Location)> = None;
let mut prev_non_newline_tok: Option<(&Location, &Tok, &Location)> = None;
let mut prev_semantic_tok: Option<(&Location, &Tok, &Location)> = None;
let mut parens = vec![];
for (start, tok, end) in lxr.iter().flatten() {
// Add empty lines.
if let Some((prev, ..)) = prev_non_newline_tok {
for row in prev.row() + 1..start.row() {
tokens.push(TriviaToken {
start: Location::new(row, 0),
end: Location::new(row + 1, 0),
kind: TriviaTokenKind::EmptyLine,
});
}
}
// Add comments.
if let Tok::Comment(_) = tok {
tokens.push(TriviaToken {
start: *start,
end: *end,
kind: if prev_non_newline_tok.map_or(true, |(prev, ..)| prev.row() < start.row()) {
TriviaTokenKind::StandaloneComment
} else {
TriviaTokenKind::InlineComment
},
});
}
// Add magic trailing commas.
if matches!(
tok,
Tok::Rpar | Tok::Rsqb | Tok::Rbrace | Tok::Equal | Tok::Newline
) {
if let Some((prev_start, prev_tok, prev_end)) = prev_semantic_tok {
if prev_tok == &Tok::Comma {
tokens.push(TriviaToken {
start: *prev_start,
end: *prev_end,
kind: TriviaTokenKind::MagicTrailingComma,
});
}
}
}
if matches!(tok, Tok::Lpar) {
if prev_tok.map_or(true, |(_, prev_tok, _)| {
!matches!(prev_tok, Tok::Name { .. })
}) {
parens.push((start, true));
} else {
parens.push((start, false));
}
} else if matches!(tok, Tok::Rpar) {
let (start, explicit) = parens.pop().unwrap();
if explicit {
tokens.push(TriviaToken {
start: *start,
end: *end,
kind: TriviaTokenKind::Parentheses,
});
}
}
prev_tok = Some((start, tok, end));
// Track the most recent non-whitespace token.
if !matches!(tok, Tok::Newline | Tok::NonLogicalNewline,) {
prev_non_newline_tok = Some((start, tok, end));
}
// Track the most recent semantic token.
if !matches!(
tok,
Tok::Newline | Tok::NonLogicalNewline | Tok::Comment(..)
) {
prev_semantic_tok = Some((start, tok, end));
}
}
tokens
}
fn sorted_child_nodes_inner<'a>(node: &Node<'a>, result: &mut Vec<Node<'a>>) {
match node {
Node::Mod(nodes) => {
for stmt in nodes.iter() {
result.push(Node::Stmt(stmt));
}
}
Node::Stmt(stmt) => match &stmt.node {
StmtKind::Return { value } => {
if let Some(value) = value {
result.push(Node::Expr(value));
}
}
StmtKind::Expr { value } => {
result.push(Node::Expr(value));
}
StmtKind::Pass => {}
StmtKind::Assign { targets, value, .. } => {
for target in targets {
result.push(Node::Expr(target));
}
result.push(Node::Expr(value));
}
StmtKind::FunctionDef {
args,
body,
decorator_list,
returns,
..
}
| StmtKind::AsyncFunctionDef {
args,
body,
decorator_list,
returns,
..
} => {
for decorator in decorator_list {
result.push(Node::Expr(decorator));
}
// TODO(charlie): Retain order.
for arg in &args.posonlyargs {
if let Some(expr) = &arg.node.annotation {
result.push(Node::Expr(expr));
}
}
for arg in &args.args {
if let Some(expr) = &arg.node.annotation {
result.push(Node::Expr(expr));
}
}
if let Some(arg) = &args.vararg {
if let Some(expr) = &arg.node.annotation {
result.push(Node::Expr(expr));
}
}
for arg in &args.kwonlyargs {
if let Some(expr) = &arg.node.annotation {
result.push(Node::Expr(expr));
}
}
if let Some(arg) = &args.kwarg {
if let Some(expr) = &arg.node.annotation {
result.push(Node::Expr(expr));
}
}
for expr in &args.defaults {
result.push(Node::Expr(expr));
}
for expr in &args.kw_defaults {
result.push(Node::Expr(expr));
}
if let Some(returns) = returns {
result.push(Node::Expr(returns));
}
for stmt in body {
result.push(Node::Stmt(stmt));
}
}
StmtKind::ClassDef {
bases,
keywords,
body,
decorator_list,
..
} => {
for decorator in decorator_list {
result.push(Node::Expr(decorator));
}
for base in bases {
result.push(Node::Expr(base));
}
for keyword in keywords {
result.push(Node::Expr(&keyword.node.value));
}
for stmt in body {
result.push(Node::Stmt(stmt));
}
}
StmtKind::Delete { targets } => {
for target in targets {
result.push(Node::Expr(target));
}
}
StmtKind::AugAssign { target, value, .. } => {
result.push(Node::Expr(target));
result.push(Node::Expr(value));
}
StmtKind::AnnAssign {
target,
annotation,
value,
..
} => {
result.push(Node::Expr(target));
result.push(Node::Expr(annotation));
if let Some(value) = value {
result.push(Node::Expr(value));
}
}
StmtKind::For {
target,
iter,
body,
orelse,
..
}
| StmtKind::AsyncFor {
target,
iter,
body,
orelse,
..
} => {
result.push(Node::Expr(target));
result.push(Node::Expr(iter));
for stmt in body {
result.push(Node::Stmt(stmt));
}
for stmt in orelse {
result.push(Node::Stmt(stmt));
}
}
StmtKind::While { test, body, orelse } => {
result.push(Node::Expr(test));
for stmt in body {
result.push(Node::Stmt(stmt));
}
for stmt in orelse {
result.push(Node::Stmt(stmt));
}
}
StmtKind::If { test, body, orelse } => {
result.push(Node::Expr(test));
for stmt in body {
result.push(Node::Stmt(stmt));
}
for stmt in orelse {
result.push(Node::Stmt(stmt));
}
}
StmtKind::With { items, body, .. } | StmtKind::AsyncWith { items, body, .. } => {
for item in items {
result.push(Node::Expr(&item.context_expr));
if let Some(expr) = &item.optional_vars {
result.push(Node::Expr(expr));
}
}
for stmt in body {
result.push(Node::Stmt(stmt));
}
}
StmtKind::Match { .. } => {
todo!("Support match statements");
}
StmtKind::Raise { exc, cause } => {
if let Some(exc) = exc {
result.push(Node::Expr(exc));
}
if let Some(cause) = cause {
result.push(Node::Expr(cause));
}
}
StmtKind::Assert { test, msg } => {
result.push(Node::Expr(test));
if let Some(msg) = msg {
result.push(Node::Expr(msg));
}
}
StmtKind::Break => {}
StmtKind::Continue => {}
StmtKind::Try {
body,
handlers,
orelse,
finalbody,
} => {
for stmt in body {
result.push(Node::Stmt(stmt));
}
for handler in handlers {
result.push(Node::Excepthandler(handler));
}
for stmt in orelse {
result.push(Node::Stmt(stmt));
}
for stmt in finalbody {
result.push(Node::Stmt(stmt));
}
}
StmtKind::Import { names } => {
for name in names {
result.push(Node::Alias(name));
}
}
StmtKind::ImportFrom { names, .. } => {
for name in names {
result.push(Node::Alias(name));
}
}
StmtKind::Global { .. } => {
// TODO(charlie): Ident, not sure how to handle?
}
StmtKind::Nonlocal { .. } => {
// TODO(charlie): Ident, not sure how to handle?
}
},
// TODO(charlie): Actual logic, this doesn't do anything.
Node::Expr(expr) => match &expr.node {
ExprKind::BoolOp { values, .. } => {
for value in values {
result.push(Node::Expr(value));
}
}
ExprKind::NamedExpr { target, value } => {
result.push(Node::Expr(target));
result.push(Node::Expr(value));
}
ExprKind::BinOp { left, right, .. } => {
result.push(Node::Expr(left));
result.push(Node::Expr(right));
}
ExprKind::UnaryOp { operand, .. } => {
result.push(Node::Expr(operand));
}
ExprKind::Lambda { body, args, .. } => {
// TODO(charlie): Arguments.
for expr in &args.defaults {
result.push(Node::Expr(expr));
}
for expr in &args.kw_defaults {
result.push(Node::Expr(expr));
}
result.push(Node::Expr(body));
}
ExprKind::IfExp { test, body, orelse } => {
result.push(Node::Expr(body));
result.push(Node::Expr(test));
result.push(Node::Expr(orelse));
}
ExprKind::Dict { keys, values } => {
for key in keys.iter().flatten() {
result.push(Node::Expr(key));
}
for value in values {
result.push(Node::Expr(value));
}
}
ExprKind::Set { elts } => {
for elt in elts {
result.push(Node::Expr(elt));
}
}
ExprKind::ListComp { elt, generators } => {
result.push(Node::Expr(elt));
for generator in generators {
result.push(Node::Expr(&generator.target));
result.push(Node::Expr(&generator.iter));
for expr in &generator.ifs {
result.push(Node::Expr(expr));
}
}
}
ExprKind::SetComp { elt, generators } => {
result.push(Node::Expr(elt));
for generator in generators {
result.push(Node::Expr(&generator.target));
result.push(Node::Expr(&generator.iter));
for expr in &generator.ifs {
result.push(Node::Expr(expr));
}
}
}
ExprKind::DictComp {
key,
value,
generators,
} => {
result.push(Node::Expr(key));
result.push(Node::Expr(value));
for generator in generators {
result.push(Node::Expr(&generator.target));
result.push(Node::Expr(&generator.iter));
for expr in &generator.ifs {
result.push(Node::Expr(expr));
}
}
}
ExprKind::GeneratorExp { elt, generators } => {
result.push(Node::Expr(elt));
for generator in generators {
result.push(Node::Expr(&generator.target));
result.push(Node::Expr(&generator.iter));
for expr in &generator.ifs {
result.push(Node::Expr(expr));
}
}
}
ExprKind::Await { value } => {
result.push(Node::Expr(value));
}
ExprKind::Yield { value } => {
if let Some(value) = value {
result.push(Node::Expr(value));
}
}
ExprKind::YieldFrom { value } => {
result.push(Node::Expr(value));
}
ExprKind::Compare {
left, comparators, ..
} => {
result.push(Node::Expr(left));
for comparator in comparators {
result.push(Node::Expr(comparator));
}
}
ExprKind::Call {
func,
args,
keywords,
} => {
result.push(Node::Expr(func));
for arg in args {
result.push(Node::Expr(arg));
}
for keyword in keywords {
result.push(Node::Expr(&keyword.node.value));
}
}
ExprKind::FormattedValue {
value, format_spec, ..
} => {
result.push(Node::Expr(value));
if let Some(format_spec) = format_spec {
result.push(Node::Expr(format_spec));
}
}
ExprKind::JoinedStr { values } => {
for value in values {
result.push(Node::Expr(value));
}
}
ExprKind::Constant { .. } => {}
ExprKind::Attribute { value, .. } => {
result.push(Node::Expr(value));
}
ExprKind::Subscript { value, slice, .. } => {
result.push(Node::Expr(value));
result.push(Node::Expr(slice));
}
ExprKind::Starred { value, .. } => {
result.push(Node::Expr(value));
}
ExprKind::Name { .. } => {}
ExprKind::List { elts, .. } => {
for elt in elts {
result.push(Node::Expr(elt));
}
}
ExprKind::Tuple { elts, .. } => {
for elt in elts {
result.push(Node::Expr(elt));
}
}
ExprKind::Slice { lower, upper, step } => {
if let Some(lower) = lower {
result.push(Node::Expr(lower));
}
if let Some(upper) = upper {
result.push(Node::Expr(upper));
}
if let Some(step) = step {
result.push(Node::Expr(step));
}
}
},
Node::Alias(..) => {}
Node::Excepthandler(excepthandler) => {
// TODO(charlie): Ident.
let ExcepthandlerKind::ExceptHandler { type_, body, .. } = &excepthandler.node;
if let Some(type_) = type_ {
result.push(Node::Expr(type_));
}
for stmt in body {
result.push(Node::Stmt(stmt));
}
}
}
}
pub fn sorted_child_nodes<'a>(node: &Node<'a>) -> Vec<Node<'a>> {
let mut result = Vec::new();
sorted_child_nodes_inner(node, &mut result);
result
}
pub fn decorate_token<'a>(
token: &TriviaToken,
node: &Node<'a>,
enclosing_node: Option<Node<'a>>,
enclosed_node: Option<Node<'a>>,
cache: &mut FxHashMap<usize, Vec<Node<'a>>>,
) -> (
Option<Node<'a>>,
Option<Node<'a>>,
Option<Node<'a>>,
Option<Node<'a>>,
) {
let child_nodes = cache
.entry(node.id())
.or_insert_with(|| sorted_child_nodes(node));
let mut preceding_node = None;
let mut following_node = None;
let mut enclosed_node = enclosed_node;
let mut left = 0;
let mut right = child_nodes.len();
while left < right {
let middle = (left + right) / 2;
let child = &child_nodes[middle];
let start = match &child {
Node::Stmt(node) => node.location,
Node::Expr(node) => node.location,
Node::Alias(node) => node.location,
Node::Excepthandler(node) => node.location,
Node::Mod(..) => unreachable!("Node::Mod cannot be a child node"),
};
let end = match &child {
Node::Stmt(node) => node.end_location.unwrap(),
Node::Expr(node) => node.end_location.unwrap(),
Node::Alias(node) => node.end_location.unwrap(),
Node::Excepthandler(node) => node.end_location.unwrap(),
Node::Mod(..) => unreachable!("Node::Mod cannot be a child node"),
};
if let Some(existing) = &enclosed_node {
// Special-case: if we're dealing with a statement that's a single expression,
// we want to treat the expression as the enclosed node.
let existing_start = match &existing {
Node::Stmt(node) => node.location,
Node::Expr(node) => node.location,
Node::Alias(node) => node.location,
Node::Excepthandler(node) => node.location,
Node::Mod(..) => unreachable!("Node::Mod cannot be a child node"),
};
let existing_end = match &existing {
Node::Stmt(node) => node.end_location.unwrap(),
Node::Expr(node) => node.end_location.unwrap(),
Node::Alias(node) => node.end_location.unwrap(),
Node::Excepthandler(node) => node.end_location.unwrap(),
Node::Mod(..) => unreachable!("Node::Mod cannot be a child node"),
};
if start == existing_start && end == existing_end {
enclosed_node = Some(child.clone());
}
} else {
if token.start <= start && token.end >= end {
enclosed_node = Some(child.clone());
}
}
// The comment is completely contained by this child node.
if token.start >= start && token.end <= end {
return decorate_token(
token,
&child.clone(),
Some(child.clone()),
enclosed_node,
cache,
);
}
if end <= token.start {
// This child node falls completely before the comment.
// Because we will never consider this node or any nodes
// before it again, this node must be the closest preceding
// node we have encountered so far.
preceding_node = Some(child.clone());
left = middle + 1;
continue;
}
if token.end <= start {
// This child node falls completely after the comment.
// Because we will never consider this node or any nodes after
// it again, this node must be the closest following node we
// have encountered so far.
following_node = Some(child.clone());
right = middle;
continue;
}
return (None, None, None, enclosed_node);
}
(
preceding_node,
following_node,
enclosing_node,
enclosed_node,
)
}
#[derive(Debug, Default)]
pub struct TriviaIndex {
pub stmt: FxHashMap<usize, Vec<Trivia>>,
pub expr: FxHashMap<usize, Vec<Trivia>>,
pub alias: FxHashMap<usize, Vec<Trivia>>,
pub excepthandler: FxHashMap<usize, Vec<Trivia>>,
pub withitem: FxHashMap<usize, Vec<Trivia>>,
}
fn add_comment<'a>(comment: Trivia, node: &Node<'a>, trivia: &mut TriviaIndex) {
match node {
Node::Mod(_) => {}
Node::Stmt(node) => {
trivia
.stmt
.entry(node.id())
.or_insert_with(Vec::new)
.push(comment);
}
Node::Expr(node) => {
trivia
.expr
.entry(node.id())
.or_insert_with(Vec::new)
.push(comment);
}
Node::Alias(node) => {
trivia
.alias
.entry(node.id())
.or_insert_with(Vec::new)
.push(comment);
}
Node::Excepthandler(node) => {
trivia
.excepthandler
.entry(node.id())
.or_insert_with(Vec::new)
.push(comment);
}
}
}
pub fn decorate_trivia(tokens: Vec<TriviaToken>, python_ast: &[Stmt]) -> TriviaIndex {
let mut stack = vec![];
let mut cache = FxHashMap::default();
for token in &tokens {
let (preceding_node, following_node, enclosing_node, enclosed_node) =
decorate_token(token, &Node::Mod(python_ast), None, None, &mut cache);
stack.push((
preceding_node,
following_node,
enclosing_node,
enclosed_node,
));
}
let mut trivia_index = TriviaIndex::default();
for (index, token) in tokens.into_iter().enumerate() {
let (preceding_node, following_node, enclosing_node, enclosed_node) = &stack[index];
match token.kind {
TriviaTokenKind::EmptyLine | TriviaTokenKind::StandaloneComment => {
if let Some(following_node) = following_node {
// Always a leading comment.
add_comment(
Trivia::from_token(&token, Relationship::Leading),
following_node,
&mut trivia_index,
);
} else if let Some(enclosing_node) = enclosing_node {
// TODO(charlie): Prettier puts this `else if` after `preceding_note`.
add_comment(
Trivia::from_token(&token, Relationship::Dangling),
enclosing_node,
&mut trivia_index,
);
} else if let Some(preceding_node) = preceding_node {
add_comment(
Trivia::from_token(&token, Relationship::Trailing),
preceding_node,
&mut trivia_index,
);
} else {
unreachable!("Attach token to the ast: {:?}", token);
}
}
TriviaTokenKind::InlineComment => {
if let Some(preceding_node) = preceding_node {
// There is content before this comment on the same line, but
// none after it, so prefer a trailing comment of the previous node.
add_comment(
Trivia::from_token(&token, Relationship::Trailing),
preceding_node,
&mut trivia_index,
);
} else if let Some(enclosing_node) = enclosing_node {
// TODO(charlie): Prettier puts this later, and uses `Relationship::Dangling`.
add_comment(
Trivia::from_token(&token, Relationship::Trailing),
enclosing_node,
&mut trivia_index,
);
} else if let Some(following_node) = following_node {
add_comment(
Trivia::from_token(&token, Relationship::Leading),
following_node,
&mut trivia_index,
);
} else {
unreachable!("Attach token to the ast: {:?}", token);
}
}
TriviaTokenKind::MagicTrailingComma => {
if let Some(enclosing_node) = enclosing_node {
add_comment(
Trivia::from_token(&token, Relationship::Trailing),
enclosing_node,
&mut trivia_index,
);
} else {
unreachable!("Attach token to the ast: {:?}", token);
}
}
TriviaTokenKind::Parentheses => {
if let Some(enclosed_node) = enclosed_node {
add_comment(
Trivia::from_token(&token, Relationship::Leading),
enclosed_node,
&mut trivia_index,
);
} else {
unreachable!("Attach token to the ast: {:?}", token);
}
}
}
}
trivia_index
}