mirror of
https://github.com/astral-sh/ruff.git
synced 2025-08-23 03:44:31 +00:00
New AST nodes for f-string elements (#8835)
Rebase of #6365 authored by @davidszotten. ## Summary This PR updates the AST structure for an f-string elements. The main **motivation** behind this change is to have a dedicated node for the string part of an f-string. Previously, the existing `ExprStringLiteral` node was used for this purpose which isn't exactly correct. The `ExprStringLiteral` node should include the quotes as well in the range but the f-string literal element doesn't include the quote as it's a specific part within an f-string. For example, ```python f"foo {x}" # ^^^^ # This is the literal part of an f-string ``` The introduction of `FStringElement` enum is helpful which represent either the literal part or the expression part of an f-string. ### Rule Updates This means that there'll be two nodes representing a string depending on the context. One for a normal string literal while the other is a string literal within an f-string. The AST checker is updated to accommodate this change. The rules which work on string literal are updated to check on the literal part of f-string as well. #### Notes 1. The `Expr::is_literal_expr` method would check for `ExprStringLiteral` and return true if so. But now that we don't represent the literal part of an f-string using that node, this improves the method's behavior and confines to the actual expression. We do have the `FStringElement::is_literal` method. 2. We avoid checking if we're in a f-string context before adding to `string_type_definitions` because the f-string literal is now a dedicated node and not part of `Expr`. 3. Annotations cannot use f-string so we avoid changing any rules which work on annotation and checks for `ExprStringLiteral`. ## Test Plan - All references of `Expr::StringLiteral` were checked to see if any of the rules require updating to account for the f-string literal element node. - New test cases are added for rules which check against the literal part of an f-string. - Check the ecosystem results and ensure it remains unchanged. ## Performance There's a performance penalty in the parser. The reason for this remains unknown as it seems that the generated assembly code is now different for the `__reduce154` function. The reduce function body is just popping the `ParenthesizedExpr` on top of the stack and pushing it with the new location. - The size of `FStringElement` enum is the same as `Expr` which is what it replaces in `FString::format_spec` - The size of `FStringExpressionElement` is the same as `ExprFormattedValue` which is what it replaces I tried reducing the `Expr` enum from 80 bytes to 72 bytes but it hardly resulted in any performance gain. The difference can be seen here: - Original profile: https://share.firefox.dev/3Taa7ES - Profile after boxing some node fields: https://share.firefox.dev/3GsNXpD ### Backtracking I tried backtracking the changes to see if any of the isolated change produced this regression. The problem here is that the overall change is so small that there's only a single checkpoint where I can backtrack and that checkpoint results in the same regression. This checkpoint is to revert using `Expr` to the `FString::format_spec` field. After this point, the change would revert back to the original implementation. ## Review process The review process is similar to #7927. The first set of commits update the node structure, parser, and related AST files. Then, further commits update the linter and formatter part to account for the AST change. --------- Co-authored-by: David Szotten <davidszotten@gmail.com>
This commit is contained in:
parent
fcc08894cf
commit
cdac90ef68
77 changed files with 1714 additions and 1925 deletions
|
@ -23,7 +23,6 @@ pub enum ExpressionRef<'a> {
|
|||
YieldFrom(&'a ast::ExprYieldFrom),
|
||||
Compare(&'a ast::ExprCompare),
|
||||
Call(&'a ast::ExprCall),
|
||||
FormattedValue(&'a ast::ExprFormattedValue),
|
||||
FString(&'a ast::ExprFString),
|
||||
StringLiteral(&'a ast::ExprStringLiteral),
|
||||
BytesLiteral(&'a ast::ExprBytesLiteral),
|
||||
|
@ -67,7 +66,6 @@ impl<'a> From<&'a Expr> for ExpressionRef<'a> {
|
|||
Expr::YieldFrom(value) => ExpressionRef::YieldFrom(value),
|
||||
Expr::Compare(value) => ExpressionRef::Compare(value),
|
||||
Expr::Call(value) => ExpressionRef::Call(value),
|
||||
Expr::FormattedValue(value) => ExpressionRef::FormattedValue(value),
|
||||
Expr::FString(value) => ExpressionRef::FString(value),
|
||||
Expr::StringLiteral(value) => ExpressionRef::StringLiteral(value),
|
||||
Expr::BytesLiteral(value) => ExpressionRef::BytesLiteral(value),
|
||||
|
@ -172,11 +170,6 @@ impl<'a> From<&'a ast::ExprCall> for ExpressionRef<'a> {
|
|||
Self::Call(value)
|
||||
}
|
||||
}
|
||||
impl<'a> From<&'a ast::ExprFormattedValue> for ExpressionRef<'a> {
|
||||
fn from(value: &'a ast::ExprFormattedValue) -> Self {
|
||||
Self::FormattedValue(value)
|
||||
}
|
||||
}
|
||||
impl<'a> From<&'a ast::ExprFString> for ExpressionRef<'a> {
|
||||
fn from(value: &'a ast::ExprFString) -> Self {
|
||||
Self::FString(value)
|
||||
|
@ -273,7 +266,6 @@ impl<'a> From<ExpressionRef<'a>> for AnyNodeRef<'a> {
|
|||
ExpressionRef::YieldFrom(expression) => AnyNodeRef::ExprYieldFrom(expression),
|
||||
ExpressionRef::Compare(expression) => AnyNodeRef::ExprCompare(expression),
|
||||
ExpressionRef::Call(expression) => AnyNodeRef::ExprCall(expression),
|
||||
ExpressionRef::FormattedValue(expression) => AnyNodeRef::ExprFormattedValue(expression),
|
||||
ExpressionRef::FString(expression) => AnyNodeRef::ExprFString(expression),
|
||||
ExpressionRef::StringLiteral(expression) => AnyNodeRef::ExprStringLiteral(expression),
|
||||
ExpressionRef::BytesLiteral(expression) => AnyNodeRef::ExprBytesLiteral(expression),
|
||||
|
@ -317,7 +309,6 @@ impl Ranged for ExpressionRef<'_> {
|
|||
ExpressionRef::YieldFrom(expression) => expression.range(),
|
||||
ExpressionRef::Compare(expression) => expression.range(),
|
||||
ExpressionRef::Call(expression) => expression.range(),
|
||||
ExpressionRef::FormattedValue(expression) => expression.range(),
|
||||
ExpressionRef::FString(expression) => expression.range(),
|
||||
ExpressionRef::StringLiteral(expression) => expression.range(),
|
||||
ExpressionRef::BytesLiteral(expression) => expression.range(),
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue