mirror of
https://github.com/astral-sh/ruff.git
synced 2025-09-19 16:50:45 +00:00
Add initial formatter implementation (#2883)
# Summary This PR contains the code for the autoformatter proof-of-concept. ## Crate structure The primary formatting hook is the `fmt` function in `crates/ruff_python_formatter/src/lib.rs`. The current formatter approach is outlined in `crates/ruff_python_formatter/src/lib.rs`, and is structured as follows: - Tokenize the code using the RustPython lexer. - In `crates/ruff_python_formatter/src/trivia.rs`, extract a variety of trivia tokens from the token stream. These include comments, trailing commas, and empty lines. - Generate the AST via the RustPython parser. - In `crates/ruff_python_formatter/src/cst.rs`, convert the AST to a CST structure. As of now, the CST is nearly identical to the AST, except that every node gets a `trivia` vector. But we might want to modify it further. - In `crates/ruff_python_formatter/src/attachment.rs`, attach each trivia token to the corresponding CST node. The logic for this is mostly in `decorate_trivia` and is ported almost directly from Prettier (given each token, find its preceding, following, and enclosing nodes, then attach the token to the appropriate node in a second pass). - In `crates/ruff_python_formatter/src/newlines.rs`, normalize newlines to match Black’s preferences. This involves traversing the CST and inserting or removing `TriviaToken` values as we go. - Call `format!` on the CST, which delegates to type-specific formatter implementations (e.g., `crates/ruff_python_formatter/src/format/stmt.rs` for `Stmt` nodes, and similar for `Expr` nodes; the others are trivial). Those type-specific implementations delegate to kind-specific functions (e.g., `format_func_def`). ## Testing and iteration The formatter is being developed against the Black test suite, which was copied over in-full to `crates/ruff_python_formatter/resources/test/fixtures/black`. The Black fixtures had to be modified to create `[insta](https://github.com/mitsuhiko/insta)`-compatible snapshots, which now exist in the repo. My approach thus far has been to try and improve coverage by tackling fixtures one-by-one. ## What works, and what doesn’t - *Most* nodes are supported at a basic level (though there are a few stragglers at time of writing, like `StmtKind::Try`). - Newlines are properly preserved in most cases. - Magic trailing commas are properly preserved in some (but not all) cases. - Trivial leading and trailing standalone comments mostly work (although maybe not at the end of a file). - Inline comments, and comments within expressions, often don’t work -- they work in a few cases, but it’s one-off right now. (We’re probably associating them with the “right” nodes more often than we are actually rendering them in the right place.) - We don’t properly normalize string quotes. (At present, we just repeat any constants verbatim.) - We’re mishandling a bunch of wrapping cases (if we treat Black as the reference implementation). Here are a few examples (demonstrating Black's stable behavior): ```py # In some cases, if the end expression is "self-closing" (functions, # lists, dictionaries, sets, subscript accesses, and any length-two # boolean operations that end in these elments), Black # will wrap like this... if some_expression and f( b, c, d, ): pass # ...whereas we do this: if ( some_expression and f( b, c, d, ) ): pass # If function arguments can fit on a single line, then Black will # format them like this, rather than exploding them vertically. if f( a, b, c, d, e, f, g, ... ): pass ``` - We don’t properly preserve parentheses in all cases. Black preserves parentheses in some but not all cases.
This commit is contained in:
parent
f661c90bd7
commit
ca49b00e55
134 changed files with 12044 additions and 18 deletions
77
crates/ruff_python_formatter/src/builders.rs
Normal file
77
crates/ruff_python_formatter/src/builders.rs
Normal file
|
@ -0,0 +1,77 @@
|
|||
use ruff_formatter::prelude::*;
|
||||
use ruff_formatter::{write, Format};
|
||||
use ruff_text_size::TextRange;
|
||||
|
||||
use crate::context::ASTFormatContext;
|
||||
use crate::core::types::Range;
|
||||
use crate::trivia::{Relationship, Trivia, TriviaKind};
|
||||
|
||||
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
|
||||
pub struct Literal {
|
||||
range: Range,
|
||||
}
|
||||
|
||||
impl Format<ASTFormatContext<'_>> for Literal {
|
||||
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
|
||||
let (text, start, end) = f.context().locator().slice(self.range);
|
||||
f.write_element(FormatElement::StaticTextSlice {
|
||||
text,
|
||||
range: TextRange::new(start.try_into().unwrap(), end.try_into().unwrap()),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TODO(charlie): We still can't use this everywhere we'd like. We need the AST
|
||||
// to include ranges for all `Ident` nodes.
|
||||
#[inline]
|
||||
pub const fn literal(range: Range) -> Literal {
|
||||
Literal { range }
|
||||
}
|
||||
|
||||
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
|
||||
pub struct LeadingComments<'a> {
|
||||
comments: &'a [Trivia],
|
||||
}
|
||||
|
||||
impl Format<ASTFormatContext<'_>> for LeadingComments<'_> {
|
||||
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
|
||||
for comment in self.comments {
|
||||
if matches!(comment.relationship, Relationship::Leading) {
|
||||
if let TriviaKind::StandaloneComment(range) = comment.kind {
|
||||
write!(f, [hard_line_break()])?;
|
||||
write!(f, [literal(range)])?;
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub const fn leading_comments(comments: &[Trivia]) -> LeadingComments {
|
||||
LeadingComments { comments }
|
||||
}
|
||||
|
||||
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
|
||||
pub struct TrailingComments<'a> {
|
||||
comments: &'a [Trivia],
|
||||
}
|
||||
|
||||
impl Format<ASTFormatContext<'_>> for TrailingComments<'_> {
|
||||
fn fmt(&self, f: &mut Formatter<ASTFormatContext<'_>>) -> FormatResult<()> {
|
||||
for comment in self.comments {
|
||||
if matches!(comment.relationship, Relationship::Trailing) {
|
||||
if let TriviaKind::StandaloneComment(range) = comment.kind {
|
||||
write!(f, [hard_line_break()])?;
|
||||
write!(f, [literal(range)])?;
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
#[inline]
|
||||
pub const fn trailing_comments(comments: &[Trivia]) -> TrailingComments {
|
||||
TrailingComments { comments }
|
||||
}
|
Loading…
Add table
Add a link
Reference in a new issue