Add initial formatter implementation (#2883)

# Summary

This PR contains the code for the autoformatter proof-of-concept.

## Crate structure

The primary formatting hook is the `fmt` function in `crates/ruff_python_formatter/src/lib.rs`.

The current formatter approach is outlined in `crates/ruff_python_formatter/src/lib.rs`, and is structured as follows:

- Tokenize the code using the RustPython lexer.
- In `crates/ruff_python_formatter/src/trivia.rs`, extract a variety of trivia tokens from the token stream. These include comments, trailing commas, and empty lines.
- Generate the AST via the RustPython parser.
- In `crates/ruff_python_formatter/src/cst.rs`, convert the AST to a CST structure. As of now, the CST is nearly identical to the AST, except that every node gets a `trivia` vector. But we might want to modify it further.
- In `crates/ruff_python_formatter/src/attachment.rs`, attach each trivia token to the corresponding CST node. The logic for this is mostly in `decorate_trivia` and is ported almost directly from Prettier (given each token, find its preceding, following, and enclosing nodes, then attach the token to the appropriate node in a second pass).
- In `crates/ruff_python_formatter/src/newlines.rs`, normalize newlines to match Black’s preferences. This involves traversing the CST and inserting or removing `TriviaToken` values as we go.
- Call `format!` on the CST, which delegates to type-specific formatter implementations (e.g., `crates/ruff_python_formatter/src/format/stmt.rs` for `Stmt` nodes, and similar for `Expr` nodes; the others are trivial). Those type-specific implementations delegate to kind-specific functions (e.g., `format_func_def`).

## Testing and iteration

The formatter is being developed against the Black test suite, which was copied over in-full to `crates/ruff_python_formatter/resources/test/fixtures/black`.

The Black fixtures had to be modified to create `[insta](https://github.com/mitsuhiko/insta)`-compatible snapshots, which now exist in the repo.

My approach thus far has been to try and improve coverage by tackling fixtures one-by-one.

## What works, and what doesn’t

- *Most* nodes are supported at a basic level (though there are a few stragglers at time of writing, like `StmtKind::Try`).
- Newlines are properly preserved in most cases.
- Magic trailing commas are properly preserved in some (but not all) cases.
- Trivial leading and trailing standalone comments mostly work (although maybe not at the end of a file).
- Inline comments, and comments within expressions, often don’t work -- they work in a few cases, but it’s one-off right now. (We’re probably associating them with the “right” nodes more often than we are actually rendering them in the right place.)
- We don’t properly normalize string quotes. (At present, we just repeat any constants verbatim.)
- We’re mishandling a bunch of wrapping cases (if we treat Black as the reference implementation). Here are a few examples (demonstrating Black's stable behavior):

```py
# In some cases, if the end expression is "self-closing" (functions,
# lists, dictionaries, sets, subscript accesses, and any length-two
# boolean operations that end in these elments), Black
# will wrap like this...
if some_expression and f(
    b,
    c,
    d,
):
    pass

# ...whereas we do this:
if (
    some_expression
    and f(
        b,
        c,
        d,
    )
):
    pass

# If function arguments can fit on a single line, then Black will
# format them like this, rather than exploding them vertically.
if f(
    a, b, c, d, e, f, g, ...
):
    pass
```

- We don’t properly preserve parentheses in all cases. Black preserves parentheses in some but not all cases.

crates/ruff_python_formatter/src/shared_traits.rs

@@ -0,0 +1,113 @@
+#![allow(clippy::all)]
+
+/// Used to get an object that knows how to format this object.
+pub trait AsFormat<Context> {
+    type Format<'a>: ruff_formatter::Format<Context>
+    where
+        Self: 'a;
+
+    /// Returns an object that is able to format this object.
+    fn format(&self) -> Self::Format<'_>;
+}
+
+/// Implement [`AsFormat`] for references to types that implement [`AsFormat`].
+impl<T, C> AsFormat<C> for &T
+where
+    T: AsFormat<C>,
+{
+    type Format<'a> = T::Format<'a> where Self: 'a;
+
+    fn format(&self) -> Self::Format<'_> {
+        AsFormat::format(&**self)
+    }
+}
+
+/// Implement [`AsFormat`] for [`Option`] when `T` implements [`AsFormat`]
+///
+/// Allows to call format on optional AST fields without having to unwrap the
+/// field first.
+impl<T, C> AsFormat<C> for Option<T>
+where
+    T: AsFormat<C>,
+{
+    type Format<'a> = Option<T::Format<'a>> where Self: 'a;
+
+    fn format(&self) -> Self::Format<'_> {
+        self.as_ref().map(AsFormat::format)
+    }
+}
+
+/// Used to convert this object into an object that can be formatted.
+///
+/// The difference to [`AsFormat`] is that this trait takes ownership of `self`.
+pub trait IntoFormat<Context> {
+    type Format: ruff_formatter::Format<Context>;
+
+    fn into_format(self) -> Self::Format;
+}
+
+/// Implement [`IntoFormat`] for [`Option`] when `T` implements [`IntoFormat`]
+///
+/// Allows to call format on optional AST fields without having to unwrap the
+/// field first.
+impl<T, Context> IntoFormat<Context> for Option<T>
+where
+    T: IntoFormat<Context>,
+{
+    type Format = Option<T::Format>;
+
+    fn into_format(self) -> Self::Format {
+        self.map(IntoFormat::into_format)
+    }
+}
+
+/// Formatting specific [`Iterator`] extensions
+pub trait FormattedIterExt {
+    /// Converts every item to an object that knows how to format it.
+    fn formatted<Context>(self) -> FormattedIter<Self, Self::Item, Context>
+    where
+        Self: Iterator + Sized,
+        Self::Item: IntoFormat<Context>,
+    {
+        FormattedIter {
+            inner: self,
+            options: std::marker::PhantomData,
+        }
+    }
+}
+
+impl<I> FormattedIterExt for I where I: std::iter::Iterator {}
+
+pub struct FormattedIter<Iter, Item, Context>
+where
+    Iter: Iterator<Item = Item>,
+{
+    inner: Iter,
+    options: std::marker::PhantomData<Context>,
+}
+
+impl<Iter, Item, Context> std::iter::Iterator for FormattedIter<Iter, Item, Context>
+where
+    Iter: Iterator<Item = Item>,
+    Item: IntoFormat<Context>,
+{
+    type Item = Item::Format;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        Some(self.inner.next()?.into_format())
+    }
+}
+
+impl<Iter, Item, Context> std::iter::FusedIterator for FormattedIter<Iter, Item, Context>
+where
+    Iter: std::iter::FusedIterator<Item = Item>,
+    Item: IntoFormat<Context>,
+{
+}
+
+impl<Iter, Item, Context> std::iter::ExactSizeIterator for FormattedIter<Iter, Item, Context>
+where
+    Iter: Iterator<Item = Item> + std::iter::ExactSizeIterator,
+    Item: IntoFormat<Context>,
+{
+}