New AST nodes for f-string elements (#8835)

Rebase of #6365 authored by @davidszotten.

## Summary

This PR updates the AST structure for an f-string elements.

The main **motivation** behind this change is to have a dedicated node
for the string part of an f-string. Previously, the existing
`ExprStringLiteral` node was used for this purpose which isn't exactly
correct. The `ExprStringLiteral` node should include the quotes as well
in the range but the f-string literal element doesn't include the quote
as it's a specific part within an f-string. For example,

```python
f"foo {x}"
# ^^^^
# This is the literal part of an f-string
```

The introduction of `FStringElement` enum is helpful which represent
either the literal part or the expression part of an f-string.

### Rule Updates

This means that there'll be two nodes representing a string depending on
the context. One for a normal string literal while the other is a string
literal within an f-string. The AST checker is updated to accommodate
this change. The rules which work on string literal are updated to check
on the literal part of f-string as well.

#### Notes

1. The `Expr::is_literal_expr` method would check for
`ExprStringLiteral` and return true if so. But now that we don't
represent the literal part of an f-string using that node, this improves
the method's behavior and confines to the actual expression. We do have
the `FStringElement::is_literal` method.
2. We avoid checking if we're in a f-string context before adding to
`string_type_definitions` because the f-string literal is now a
dedicated node and not part of `Expr`.
3. Annotations cannot use f-string so we avoid changing any rules which
work on annotation and checks for `ExprStringLiteral`.

## Test Plan

- All references of `Expr::StringLiteral` were checked to see if any of
the rules require updating to account for the f-string literal element
node.
- New test cases are added for rules which check against the literal
part of an f-string.
- Check the ecosystem results and ensure it remains unchanged.

## Performance

There's a performance penalty in the parser. The reason for this remains
unknown as it seems that the generated assembly code is now different
for the `__reduce154` function. The reduce function body is just popping
the `ParenthesizedExpr` on top of the stack and pushing it with the new
location.

- The size of `FStringElement` enum is the same as `Expr` which is what
it replaces in `FString::format_spec`
- The size of `FStringExpressionElement` is the same as
`ExprFormattedValue` which is what it replaces

I tried reducing the `Expr` enum from 80 bytes to 72 bytes but it hardly
resulted in any performance gain. The difference can be seen here:
- Original profile: https://share.firefox.dev/3Taa7ES
- Profile after boxing some node fields:
https://share.firefox.dev/3GsNXpD

### Backtracking

I tried backtracking the changes to see if any of the isolated change
produced this regression. The problem here is that the overall change is
so small that there's only a single checkpoint where I can backtrack and
that checkpoint results in the same regression. This checkpoint is to
revert using `Expr` to the `FString::format_spec` field. After this
point, the change would revert back to the original implementation.

## Review process

The review process is similar to #7927. The first set of commits update
the node structure, parser, and related AST files. Then, further commits
update the linter and formatter part to account for the AST change.

---------

Co-authored-by: David Szotten <davidszotten@gmail.com>

crates/ruff_python_ast/src/nodes.rs

@@ -590,8 +590,6 @@ pub enum Expr {
     Compare(ExprCompare),
     #[is(name = "call_expr")]
     Call(ExprCall),
-    #[is(name = "formatted_value_expr")]
-    FormattedValue(ExprFormattedValue),
     #[is(name = "f_string_expr")]
     FString(ExprFString),
     #[is(name = "string_literal_expr")]
@@ -919,19 +917,51 @@ impl From<ExprCall> for Expr {
     }
 }
 
-/// See also [FormattedValue](https://docs.python.org/3/library/ast.html#ast.FormattedValue)
 #[derive(Clone, Debug, PartialEq)]
-pub struct ExprFormattedValue {
+pub struct FStringFormatSpec {
     pub range: TextRange,
-    pub value: Box<Expr>,
-    pub debug_text: Option<DebugText>,
-    pub conversion: ConversionFlag,
-    pub format_spec: Option<Box<Expr>>,
+    pub elements: Vec<FStringElement>,
 }
 
-impl From<ExprFormattedValue> for Expr {
-    fn from(payload: ExprFormattedValue) -> Self {
-        Expr::FormattedValue(payload)
+impl Ranged for FStringFormatSpec {
+    fn range(&self) -> TextRange {
+        self.range
+    }
+}
+
+/// See also [FormattedValue](https://docs.python.org/3/library/ast.html#ast.FormattedValue)
+#[derive(Clone, Debug, PartialEq)]
+pub struct FStringExpressionElement {
+    pub range: TextRange,
+    pub expression: Box<Expr>,
+    pub debug_text: Option<DebugText>,
+    pub conversion: ConversionFlag,
+    pub format_spec: Option<Box<FStringFormatSpec>>,
+}
+
+impl Ranged for FStringExpressionElement {
+    fn range(&self) -> TextRange {
+        self.range
+    }
+}
+
+#[derive(Clone, Debug, PartialEq)]
+pub struct FStringLiteralElement {
+    pub range: TextRange,
+    pub value: String,
+}
+
+impl Ranged for FStringLiteralElement {
+    fn range(&self) -> TextRange {
+        self.range
+    }
+}
+
+impl Deref for FStringLiteralElement {
+    type Target = str;
+
+    fn deref(&self) -> &Self::Target {
+        self.value.as_str()
     }
 }
 
@@ -1064,7 +1094,7 @@ impl FStringValue {
         self.parts().filter_map(|part| part.as_f_string())
     }
 
-    /// Returns an iterator over all the f-string elements contained in this value.
+    /// Returns an iterator over all the [`FStringElement`] contained in this value.
     ///
     /// An f-string element is what makes up an [`FString`] i.e., it is either a
     /// string literal or an expression. In the following example,
@@ -1075,8 +1105,8 @@ impl FStringValue {
     ///
     /// The f-string elements returned would be string literal (`"bar "`),
     /// expression (`x`) and string literal (`"qux"`).
-    pub fn elements(&self) -> impl Iterator<Item = &Expr> {
-        self.f_strings().flat_map(|fstring| fstring.values.iter())
+    pub fn elements(&self) -> impl Iterator<Item = &FStringElement> {
+        self.f_strings().flat_map(|fstring| fstring.elements.iter())
     }
 }
 
@@ -1113,7 +1143,7 @@ impl Ranged for FStringPart {
 #[derive(Clone, Debug, PartialEq)]
 pub struct FString {
     pub range: TextRange,
-    pub values: Vec<Expr>,
+    pub elements: Vec<FStringElement>,
 }
 
 impl Ranged for FString {
@@ -1132,6 +1162,21 @@ impl From<FString> for Expr {
     }
 }
 
+#[derive(Clone, Debug, PartialEq, is_macro::Is)]
+pub enum FStringElement {
+    Literal(FStringLiteralElement),
+    Expression(FStringExpressionElement),
+}
+
+impl Ranged for FStringElement {
+    fn range(&self) -> TextRange {
+        match self {
+            FStringElement::Literal(node) => node.range(),
+            FStringElement::Expression(node) => node.range(),
+        }
+    }
+}
+
 /// An AST node that represents either a single string literal or an implicitly
 /// concatenated string literals.
 #[derive(Clone, Debug, Default, PartialEq)]
@@ -3483,11 +3528,6 @@ impl Ranged for crate::nodes::ExprCall {
         self.range
     }
 }
-impl Ranged for crate::nodes::ExprFormattedValue {
-    fn range(&self) -> TextRange {
-        self.range
-    }
-}
 impl Ranged for crate::nodes::ExprFString {
     fn range(&self) -> TextRange {
         self.range
@@ -3553,7 +3593,6 @@ impl Ranged for crate::Expr {
             Self::YieldFrom(node) => node.range(),
             Self::Compare(node) => node.range(),
             Self::Call(node) => node.range(),
-            Self::FormattedValue(node) => node.range(),
             Self::FString(node) => node.range(),
             Self::StringLiteral(node) => node.range(),
             Self::BytesLiteral(node) => node.range(),