Avoid reusing nested, interpolated quotes before Python 3.12 (#20930)
Some checks are pending
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (macos) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / ty completion evaluation (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
CI / benchmarks instrumented (ty) (push) Blocked by required conditions
CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
CI / benchmarks walltime (small|large) (push) Blocked by required conditions
[ty Playground] Release / publish (push) Waiting to run

## Summary

Fixes #20774 by tracking whether an `InterpolatedStringState` element is
nested inside of another interpolated element. This feels like kind of a
naive fix, so I'm welcome to other ideas. But it resolves the problem in
the issue and clears up the syntax error in the black compatibility
test, without affecting many other cases.

The other affected case is actually interesting too because the
[input](96b156303b/crates/ruff_python_formatter/resources/test/fixtures/ruff/expression/fstring.py (L707))
is invalid, but the previous quote selection fixed the invalid syntax:

```pycon
Python 3.11.13 (main, Sep  2 2025, 14:20:25) [Clang 20.1.4 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f'{1: abcd "{'aa'}" }'  # input
  File "<stdin>", line 1
    f'{1: abcd "{'aa'}" }'
                  ^^
SyntaxError: f-string: expecting '}'
>>> f'{1: abcd "{"aa"}" }'  # old output
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier ' abcd "aa" ' for object of type 'int'
>>> f'{1: abcd "{'aa'}" }'  # new output
  File "<stdin>", line 1
    f'{1: abcd "{'aa'}" }'
                  ^^
SyntaxError: f-string: expecting '}'
```

We now preserve the invalid syntax in the input.

Unfortunately, this also seems to be another edge case I didn't consider
in https://github.com/astral-sh/ruff/pull/20867 because we don't flag
this as a syntax error after 0.14.1:

<details><summary>Shell output</summary>
<p>

```
> uvx ruff@0.14.0 check --ignore ALL --target-version py311 - <<EOF
f'{1: abcd "{'aa'}" }'
EOF
invalid-syntax: Cannot reuse outer quote character in f-strings on Python 3.11 (syntax was added in Python 3.12)
 --> -:1:14
  |
1 | f'{1: abcd "{'aa'}" }'
  |              ^
  |

Found 1 error.
> uvx ruff@0.14.1 check --ignore ALL --target-version py311 - <<EOF
f'{1: abcd "{'aa'}" }'
EOF
All checks passed!
> uvx python@3.11 -m ast <<EOF
f'{1: abcd "{'aa'}" }'
EOF
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/brent/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/ast.py", line 1752, in <module>
    main()
  File "/home/brent/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/ast.py", line 1748, in main
    tree = parse(source, args.infile.name, args.mode, type_comments=args.no_type_comments)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/brent/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<stdin>", line 1
    f'{1: abcd "{'aa'}" }'
                  ^^
SyntaxError: f-string: expecting '}'
```

</p>
</details> 


I assumed that was the same `ParseError` as the one caused by
`f"{1:""}"`, but this is a nested interpolation inside of the format
spec.

## Test Plan

New test copied from the black compatibility test. I guess this is a
duplicate now, I started working on this branch before the new black
tests were imported, so I could delete the separate test in our fixtures
if that's preferable.
This commit is contained in:
Brent Westbrook 2025-10-17 08:49:16 -04:00 committed by GitHub
parent cfbd42c22a
commit 0115fd3757
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 52 additions and 26 deletions

View file

@ -748,3 +748,7 @@ print(f"{ # Tuple with multiple elements that doesn't fit on a single line gets
# Regression tests for https://github.com/astral-sh/ruff/issues/15536
print(f"{ {}, 1, }")
# The inner quotes should not be changed to double quotes before Python 3.12
f"{f'''{'nested'} inner'''} outer"

View file

@ -144,6 +144,12 @@ pub(crate) enum InterpolatedStringState {
///
/// The containing `FStringContext` is the surrounding f-string context.
InsideInterpolatedElement(InterpolatedStringContext),
/// The formatter is inside more than one nested f-string, such as in `nested` in:
///
/// ```py
/// f"{f'''{'nested'} inner'''} outer"
/// ```
NestedInterpolatedElement(InterpolatedStringContext),
/// The formatter is outside an f-string.
#[default]
Outside,
@ -152,12 +158,18 @@ pub(crate) enum InterpolatedStringState {
impl InterpolatedStringState {
pub(crate) fn can_contain_line_breaks(self) -> Option<bool> {
match self {
InterpolatedStringState::InsideInterpolatedElement(context) => {
InterpolatedStringState::InsideInterpolatedElement(context)
| InterpolatedStringState::NestedInterpolatedElement(context) => {
Some(context.is_multiline())
}
InterpolatedStringState::Outside => None,
}
}
/// Returns `true` if the interpolated string state is [`NestedInterpolatedElement`].
pub(crate) fn is_nested(self) -> bool {
matches!(self, Self::NestedInterpolatedElement(..))
}
}
/// The position of a top-level statement in the module.

View file

@ -181,10 +181,16 @@ impl Format<PyFormatContext<'_>> for FormatInterpolatedElement<'_> {
let item = format_with(|f: &mut PyFormatter| {
// Update the context to be inside the f-string expression element.
let f = &mut WithInterpolatedStringState::new(
InterpolatedStringState::InsideInterpolatedElement(self.context),
f,
);
let state = match f.context().interpolated_string_state() {
InterpolatedStringState::InsideInterpolatedElement(_)
| InterpolatedStringState::NestedInterpolatedElement(_) => {
InterpolatedStringState::NestedInterpolatedElement(self.context)
}
InterpolatedStringState::Outside => {
InterpolatedStringState::InsideInterpolatedElement(self.context)
}
};
let f = &mut WithInterpolatedStringState::new(state, f);
write!(f, [bracket_spacing, expression.format()])?;

View file

@ -46,8 +46,15 @@ impl<'a, 'src> StringNormalizer<'a, 'src> {
.unwrap_or(self.context.options().quote_style());
let supports_pep_701 = self.context.options().target_version().supports_pep_701();
// Preserve the existing quote style for nested interpolations more than one layer deep, if
// PEP 701 isn't supported.
if !supports_pep_701 && self.context.interpolated_string_state().is_nested() {
return QuoteStyle::Preserve;
}
// For f-strings and t-strings prefer alternating the quotes unless The outer string is triple quoted and the inner isn't.
if let InterpolatedStringState::InsideInterpolatedElement(parent_context) =
if let InterpolatedStringState::InsideInterpolatedElement(parent_context)
| InterpolatedStringState::NestedInterpolatedElement(parent_context) =
self.context.interpolated_string_state()
{
let parent_flags = parent_context.flags();

View file

@ -28,12 +28,11 @@ but none started with prefix {parentdir_prefix}"
f'{{NOT \'a\' "formatted" "value"}}'
f"some f-string with {a} {few():.2f} {formatted.values!r}"
-f'some f-string with {a} {few(""):.2f} {formatted.values!r}'
-f"{f'''{'nested'} inner'''} outer"
+f"some f-string with {a} {few(''):.2f} {formatted.values!r}"
f"{f'''{'nested'} inner'''} outer"
-f"\"{f'{nested} inner'}\" outer"
-f"space between opening braces: { {a for a in (1, 2, 3)}}"
-f'Hello \'{tricky + "example"}\''
+f"some f-string with {a} {few(''):.2f} {formatted.values!r}"
+f"{f'''{"nested"} inner'''} outer"
+f'"{f"{nested} inner"}" outer'
+f"space between opening braces: { {a for a in (1, 2, 3)} }"
+f"Hello '{tricky + 'example'}'"
@ -49,7 +48,7 @@ f"{{NOT a formatted value}}"
f'{{NOT \'a\' "formatted" "value"}}'
f"some f-string with {a} {few():.2f} {formatted.values!r}"
f"some f-string with {a} {few(''):.2f} {formatted.values!r}"
f"{f'''{"nested"} inner'''} outer"
f"{f'''{'nested'} inner'''} outer"
f'"{f"{nested} inner"}" outer'
f"space between opening braces: { {a for a in (1, 2, 3)} }"
f"Hello '{tricky + 'example'}'"
@ -72,17 +71,3 @@ f'Hello \'{tricky + "example"}\''
f"Tried directories {str(rootdirs)} \
but none started with prefix {parentdir_prefix}"
```
## New Unsupported Syntax Errors
error[invalid-syntax]: Cannot reuse outer quote character in f-strings on Python 3.10 (syntax was added in Python 3.12)
--> fstring.py:6:9
|
4 | f"some f-string with {a} {few():.2f} {formatted.values!r}"
5 | f"some f-string with {a} {few(''):.2f} {formatted.values!r}"
6 | f"{f'''{"nested"} inner'''} outer"
| ^
7 | f'"{f"{nested} inner"}" outer'
8 | f"space between opening braces: { {a for a in (1, 2, 3)} }"
|
warning: Only accept new syntax errors if they are also present in the input. The formatter should not introduce syntax errors.

View file

@ -754,6 +754,10 @@ print(f"{ # Tuple with multiple elements that doesn't fit on a single line gets
# Regression tests for https://github.com/astral-sh/ruff/issues/15536
print(f"{ {}, 1, }")
# The inner quotes should not be changed to double quotes before Python 3.12
f"{f'''{'nested'} inner'''} outer"
```
## Outputs
@ -1532,7 +1536,7 @@ f'{f"""other " """}'
f'{1: hy "user"}'
f'{1:hy "user"}'
f'{1: abcd "{1}" }'
f'{1: abcd "{"aa"}" }'
f'{1: abcd "{'aa'}" }'
f'{1=: "abcd {'aa'}}'
f"{x:a{z:hy \"user\"}} '''"
@ -1581,6 +1585,10 @@ print(
# Regression tests for https://github.com/astral-sh/ruff/issues/15536
print(f"{ {}, 1 }")
# The inner quotes should not be changed to double quotes before Python 3.12
f"{f'''{'nested'} inner'''} outer"
```
@ -2359,7 +2367,7 @@ f'{f"""other " """}'
f'{1: hy "user"}'
f'{1:hy "user"}'
f'{1: abcd "{1}" }'
f'{1: abcd "{"aa"}" }'
f'{1: abcd "{'aa'}" }'
f'{1=: "abcd {'aa'}}'
f"{x:a{z:hy \"user\"}} '''"
@ -2408,6 +2416,10 @@ print(
# Regression tests for https://github.com/astral-sh/ruff/issues/15536
print(f"{ {}, 1 }")
# The inner quotes should not be changed to double quotes before Python 3.12
f"{f'''{'nested'} inner'''} outer"
```