mirror of
https://github.com/astral-sh/ruff.git
synced 2025-08-21 19:05:09 +00:00
New AST nodes for f-string elements (#8835)
Rebase of #6365 authored by @davidszotten. ## Summary This PR updates the AST structure for an f-string elements. The main **motivation** behind this change is to have a dedicated node for the string part of an f-string. Previously, the existing `ExprStringLiteral` node was used for this purpose which isn't exactly correct. The `ExprStringLiteral` node should include the quotes as well in the range but the f-string literal element doesn't include the quote as it's a specific part within an f-string. For example, ```python f"foo {x}" # ^^^^ # This is the literal part of an f-string ``` The introduction of `FStringElement` enum is helpful which represent either the literal part or the expression part of an f-string. ### Rule Updates This means that there'll be two nodes representing a string depending on the context. One for a normal string literal while the other is a string literal within an f-string. The AST checker is updated to accommodate this change. The rules which work on string literal are updated to check on the literal part of f-string as well. #### Notes 1. The `Expr::is_literal_expr` method would check for `ExprStringLiteral` and return true if so. But now that we don't represent the literal part of an f-string using that node, this improves the method's behavior and confines to the actual expression. We do have the `FStringElement::is_literal` method. 2. We avoid checking if we're in a f-string context before adding to `string_type_definitions` because the f-string literal is now a dedicated node and not part of `Expr`. 3. Annotations cannot use f-string so we avoid changing any rules which work on annotation and checks for `ExprStringLiteral`. ## Test Plan - All references of `Expr::StringLiteral` were checked to see if any of the rules require updating to account for the f-string literal element node. - New test cases are added for rules which check against the literal part of an f-string. - Check the ecosystem results and ensure it remains unchanged. ## Performance There's a performance penalty in the parser. The reason for this remains unknown as it seems that the generated assembly code is now different for the `__reduce154` function. The reduce function body is just popping the `ParenthesizedExpr` on top of the stack and pushing it with the new location. - The size of `FStringElement` enum is the same as `Expr` which is what it replaces in `FString::format_spec` - The size of `FStringExpressionElement` is the same as `ExprFormattedValue` which is what it replaces I tried reducing the `Expr` enum from 80 bytes to 72 bytes but it hardly resulted in any performance gain. The difference can be seen here: - Original profile: https://share.firefox.dev/3Taa7ES - Profile after boxing some node fields: https://share.firefox.dev/3GsNXpD ### Backtracking I tried backtracking the changes to see if any of the isolated change produced this regression. The problem here is that the overall change is so small that there's only a single checkpoint where I can backtrack and that checkpoint results in the same regression. This checkpoint is to revert using `Expr` to the `FString::format_spec` field. After this point, the change would revert back to the original implementation. ## Review process The review process is similar to #7927. The first set of commits update the node structure, parser, and related AST files. Then, further commits update the linter and formatter part to account for the AST change. --------- Co-authored-by: David Szotten <davidszotten@gmail.com>
This commit is contained in:
parent
fcc08894cf
commit
cdac90ef68
77 changed files with 1714 additions and 1925 deletions
|
@ -1534,42 +1534,6 @@ impl<'ast> IntoFormat<PyFormatContext<'ast>> for ast::ExprCall {
|
|||
}
|
||||
}
|
||||
|
||||
impl FormatRule<ast::ExprFormattedValue, PyFormatContext<'_>>
|
||||
for crate::expression::expr_formatted_value::FormatExprFormattedValue
|
||||
{
|
||||
#[inline]
|
||||
fn fmt(&self, node: &ast::ExprFormattedValue, f: &mut PyFormatter) -> FormatResult<()> {
|
||||
FormatNodeRule::<ast::ExprFormattedValue>::fmt(self, node, f)
|
||||
}
|
||||
}
|
||||
impl<'ast> AsFormat<PyFormatContext<'ast>> for ast::ExprFormattedValue {
|
||||
type Format<'a> = FormatRefWithRule<
|
||||
'a,
|
||||
ast::ExprFormattedValue,
|
||||
crate::expression::expr_formatted_value::FormatExprFormattedValue,
|
||||
PyFormatContext<'ast>,
|
||||
>;
|
||||
fn format(&self) -> Self::Format<'_> {
|
||||
FormatRefWithRule::new(
|
||||
self,
|
||||
crate::expression::expr_formatted_value::FormatExprFormattedValue::default(),
|
||||
)
|
||||
}
|
||||
}
|
||||
impl<'ast> IntoFormat<PyFormatContext<'ast>> for ast::ExprFormattedValue {
|
||||
type Format = FormatOwnedWithRule<
|
||||
ast::ExprFormattedValue,
|
||||
crate::expression::expr_formatted_value::FormatExprFormattedValue,
|
||||
PyFormatContext<'ast>,
|
||||
>;
|
||||
fn into_format(self) -> Self::Format {
|
||||
FormatOwnedWithRule::new(
|
||||
self,
|
||||
crate::expression::expr_formatted_value::FormatExprFormattedValue::default(),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
impl FormatRule<ast::ExprFString, PyFormatContext<'_>>
|
||||
for crate::expression::expr_f_string::FormatExprFString
|
||||
{
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue