mirror of
https://github.com/astral-sh/ruff.git
synced 2025-09-29 05:15:12 +00:00
ca49b00e55
# Summary This PR contains the code for the autoformatter proof-of-concept. ## Crate structure The primary formatting hook is the `fmt` function in `crates/ruff_python_formatter/src/lib.rs`. The current formatter approach is outlined in `crates/ruff_python_formatter/src/lib.rs`, and is structured as follows: - Tokenize the code using the RustPython lexer. - In `crates/ruff_python_formatter/src/trivia.rs`, extract a variety of trivia tokens from the token stream. These include comments, trailing commas, and empty lines. - Generate the AST via the RustPython parser. - In `crates/ruff_python_formatter/src/cst.rs`, convert the AST to a CST structure. As of now, the CST is nearly identical to the AST, except that every node gets a `trivia` vector. But we might want to modify it further. - In `crates/ruff_python_formatter/src/attachment.rs`, attach each trivia token to the corresponding CST node. The logic for this is mostly in `decorate_trivia` and is ported almost directly from Prettier (given each token, find its preceding, following, and enclosing nodes, then attach the token to the appropriate node in a second pass). - In `crates/ruff_python_formatter/src/newlines.rs`, normalize newlines to match Black’s preferences. This involves traversing the CST and inserting or removing `TriviaToken` values as we go. - Call `format!` on the CST, which delegates to type-specific formatter implementations (e.g., `crates/ruff_python_formatter/src/format/stmt.rs` for `Stmt` nodes, and similar for `Expr` nodes; the others are trivial). Those type-specific implementations delegate to kind-specific functions (e.g., `format_func_def`). ## Testing and iteration The formatter is being developed against the Black test suite, which was copied over in-full to `crates/ruff_python_formatter/resources/test/fixtures/black`. The Black fixtures had to be modified to create `[insta](https://github.com/mitsuhiko/insta)`-compatible snapshots, which now exist in the repo. My approach thus far has been to try and improve coverage by tackling fixtures one-by-one. ## What works, and what doesn’t - *Most* nodes are supported at a basic level (though there are a few stragglers at time of writing, like `StmtKind::Try`). - Newlines are properly preserved in most cases. - Magic trailing commas are properly preserved in some (but not all) cases. - Trivial leading and trailing standalone comments mostly work (although maybe not at the end of a file). - Inline comments, and comments within expressions, often don’t work -- they work in a few cases, but it’s one-off right now. (We’re probably associating them with the “right” nodes more often than we are actually rendering them in the right place.) - We don’t properly normalize string quotes. (At present, we just repeat any constants verbatim.) - We’re mishandling a bunch of wrapping cases (if we treat Black as the reference implementation). Here are a few examples (demonstrating Black's stable behavior): ```py # In some cases, if the end expression is "self-closing" (functions, # lists, dictionaries, sets, subscript accesses, and any length-two # boolean operations that end in these elments), Black # will wrap like this... if some_expression and f( b, c, d, ): pass # ...whereas we do this: if ( some_expression and f( b, c, d, ) ): pass # If function arguments can fit on a single line, then Black will # format them like this, rather than exploding them vertically. if f( a, b, c, d, e, f, g, ... ): pass ``` - We don’t properly preserve parentheses in all cases. Black preserves parentheses in some but not all cases.
113 lines
2.9 KiB
Rust
113 lines
2.9 KiB
Rust
#![allow(clippy::all)]
|
|
|
|
/// Used to get an object that knows how to format this object.
|
|
pub trait AsFormat<Context> {
|
|
type Format<'a>: ruff_formatter::Format<Context>
|
|
where
|
|
Self: 'a;
|
|
|
|
/// Returns an object that is able to format this object.
|
|
fn format(&self) -> Self::Format<'_>;
|
|
}
|
|
|
|
/// Implement [`AsFormat`] for references to types that implement [`AsFormat`].
|
|
impl<T, C> AsFormat<C> for &T
|
|
where
|
|
T: AsFormat<C>,
|
|
{
|
|
type Format<'a> = T::Format<'a> where Self: 'a;
|
|
|
|
fn format(&self) -> Self::Format<'_> {
|
|
AsFormat::format(&**self)
|
|
}
|
|
}
|
|
|
|
/// Implement [`AsFormat`] for [`Option`] when `T` implements [`AsFormat`]
|
|
///
|
|
/// Allows to call format on optional AST fields without having to unwrap the
|
|
/// field first.
|
|
impl<T, C> AsFormat<C> for Option<T>
|
|
where
|
|
T: AsFormat<C>,
|
|
{
|
|
type Format<'a> = Option<T::Format<'a>> where Self: 'a;
|
|
|
|
fn format(&self) -> Self::Format<'_> {
|
|
self.as_ref().map(AsFormat::format)
|
|
}
|
|
}
|
|
|
|
/// Used to convert this object into an object that can be formatted.
|
|
///
|
|
/// The difference to [`AsFormat`] is that this trait takes ownership of `self`.
|
|
pub trait IntoFormat<Context> {
|
|
type Format: ruff_formatter::Format<Context>;
|
|
|
|
fn into_format(self) -> Self::Format;
|
|
}
|
|
|
|
/// Implement [`IntoFormat`] for [`Option`] when `T` implements [`IntoFormat`]
|
|
///
|
|
/// Allows to call format on optional AST fields without having to unwrap the
|
|
/// field first.
|
|
impl<T, Context> IntoFormat<Context> for Option<T>
|
|
where
|
|
T: IntoFormat<Context>,
|
|
{
|
|
type Format = Option<T::Format>;
|
|
|
|
fn into_format(self) -> Self::Format {
|
|
self.map(IntoFormat::into_format)
|
|
}
|
|
}
|
|
|
|
/// Formatting specific [`Iterator`] extensions
|
|
pub trait FormattedIterExt {
|
|
/// Converts every item to an object that knows how to format it.
|
|
fn formatted<Context>(self) -> FormattedIter<Self, Self::Item, Context>
|
|
where
|
|
Self: Iterator + Sized,
|
|
Self::Item: IntoFormat<Context>,
|
|
{
|
|
FormattedIter {
|
|
inner: self,
|
|
options: std::marker::PhantomData,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl<I> FormattedIterExt for I where I: std::iter::Iterator {}
|
|
|
|
pub struct FormattedIter<Iter, Item, Context>
|
|
where
|
|
Iter: Iterator<Item = Item>,
|
|
{
|
|
inner: Iter,
|
|
options: std::marker::PhantomData<Context>,
|
|
}
|
|
|
|
impl<Iter, Item, Context> std::iter::Iterator for FormattedIter<Iter, Item, Context>
|
|
where
|
|
Iter: Iterator<Item = Item>,
|
|
Item: IntoFormat<Context>,
|
|
{
|
|
type Item = Item::Format;
|
|
|
|
fn next(&mut self) -> Option<Self::Item> {
|
|
Some(self.inner.next()?.into_format())
|
|
}
|
|
}
|
|
|
|
impl<Iter, Item, Context> std::iter::FusedIterator for FormattedIter<Iter, Item, Context>
|
|
where
|
|
Iter: std::iter::FusedIterator<Item = Item>,
|
|
Item: IntoFormat<Context>,
|
|
{
|
|
}
|
|
|
|
impl<Iter, Item, Context> std::iter::ExactSizeIterator for FormattedIter<Iter, Item, Context>
|
|
where
|
|
Iter: Iterator<Item = Item> + std::iter::ExactSizeIterator,
|
|
Item: IntoFormat<Context>,
|
|
{
|
|
}
|