mirror of
				https://github.com/astral-sh/ruff.git
				synced 2025-10-31 12:05:57 +00:00 
			
		
		
		
	Avoid reusing nested, interpolated quotes before Python 3.12 (#20930)
	
		
			
	
		
	
	
		
	
		
			Some checks are pending
		
		
	
	
		
			
				
	
				CI / cargo test (linux, release) (push) Blocked by required conditions
				
			
		
			
				
	
				CI / Determine changes (push) Waiting to run
				
			
		
			
				
	
				CI / cargo fmt (push) Waiting to run
				
			
		
			
				
	
				CI / cargo clippy (push) Blocked by required conditions
				
			
		
			
				
	
				CI / cargo test (linux) (push) Blocked by required conditions
				
			
		
			
				
	
				CI / cargo test (windows) (push) Blocked by required conditions
				
			
		
			
				
	
				CI / cargo test (macos) (push) Blocked by required conditions
				
			
		
			
				
	
				CI / cargo test (wasm) (push) Blocked by required conditions
				
			
		
			
				
	
				CI / cargo build (msrv) (push) Blocked by required conditions
				
			
		
			
				
	
				CI / cargo fuzz build (push) Blocked by required conditions
				
			
		
			
				
	
				CI / fuzz parser (push) Blocked by required conditions
				
			
		
			
				
	
				CI / test scripts (push) Blocked by required conditions
				
			
		
			
				
	
				CI / ecosystem (push) Blocked by required conditions
				
			
		
			
				
	
				CI / Fuzz for new ty panics (push) Blocked by required conditions
				
			
		
			
				
	
				CI / cargo shear (push) Blocked by required conditions
				
			
		
			
				
	
				CI / ty completion evaluation (push) Blocked by required conditions
				
			
		
			
				
	
				CI / python package (push) Waiting to run
				
			
		
			
				
	
				CI / pre-commit (push) Waiting to run
				
			
		
			
				
	
				CI / mkdocs (push) Waiting to run
				
			
		
			
				
	
				CI / formatter instabilities and black similarity (push) Blocked by required conditions
				
			
		
			
				
	
				CI / test ruff-lsp (push) Blocked by required conditions
				
			
		
			
				
	
				CI / check playground (push) Blocked by required conditions
				
			
		
			
				
	
				CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
				
			
		
			
				
	
				CI / benchmarks instrumented (ty) (push) Blocked by required conditions
				
			
		
			
				
	
				CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
				
			
		
			
				
	
				CI / benchmarks walltime (small|large) (push) Blocked by required conditions
				
			
		
			
				
	
				[ty Playground] Release / publish (push) Waiting to run
				
			
		
		
	
	
				
					
				
			
		
			Some checks are pending
		
		
	
	CI / cargo test (linux, release) (push) Blocked by required conditions
				
			CI / Determine changes (push) Waiting to run
				
			CI / cargo fmt (push) Waiting to run
				
			CI / cargo clippy (push) Blocked by required conditions
				
			CI / cargo test (linux) (push) Blocked by required conditions
				
			CI / cargo test (windows) (push) Blocked by required conditions
				
			CI / cargo test (macos) (push) Blocked by required conditions
				
			CI / cargo test (wasm) (push) Blocked by required conditions
				
			CI / cargo build (msrv) (push) Blocked by required conditions
				
			CI / cargo fuzz build (push) Blocked by required conditions
				
			CI / fuzz parser (push) Blocked by required conditions
				
			CI / test scripts (push) Blocked by required conditions
				
			CI / ecosystem (push) Blocked by required conditions
				
			CI / Fuzz for new ty panics (push) Blocked by required conditions
				
			CI / cargo shear (push) Blocked by required conditions
				
			CI / ty completion evaluation (push) Blocked by required conditions
				
			CI / python package (push) Waiting to run
				
			CI / pre-commit (push) Waiting to run
				
			CI / mkdocs (push) Waiting to run
				
			CI / formatter instabilities and black similarity (push) Blocked by required conditions
				
			CI / test ruff-lsp (push) Blocked by required conditions
				
			CI / check playground (push) Blocked by required conditions
				
			CI / benchmarks instrumented (ruff) (push) Blocked by required conditions
				
			CI / benchmarks instrumented (ty) (push) Blocked by required conditions
				
			CI / benchmarks walltime (medium|multithreaded) (push) Blocked by required conditions
				
			CI / benchmarks walltime (small|large) (push) Blocked by required conditions
				
			[ty Playground] Release / publish (push) Waiting to run
				
			## Summary
Fixes #20774 by tracking whether an `InterpolatedStringState` element is
nested inside of another interpolated element. This feels like kind of a
naive fix, so I'm welcome to other ideas. But it resolves the problem in
the issue and clears up the syntax error in the black compatibility
test, without affecting many other cases.
The other affected case is actually interesting too because the
[input](96b156303b/crates/ruff_python_formatter/resources/test/fixtures/ruff/expression/fstring.py (L707))
is invalid, but the previous quote selection fixed the invalid syntax:
```pycon
Python 3.11.13 (main, Sep  2 2025, 14:20:25) [Clang 20.1.4 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> f'{1: abcd "{'aa'}" }'  # input
  File "<stdin>", line 1
    f'{1: abcd "{'aa'}" }'
                  ^^
SyntaxError: f-string: expecting '}'
>>> f'{1: abcd "{"aa"}" }'  # old output
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier ' abcd "aa" ' for object of type 'int'
>>> f'{1: abcd "{'aa'}" }'  # new output
  File "<stdin>", line 1
    f'{1: abcd "{'aa'}" }'
                  ^^
SyntaxError: f-string: expecting '}'
```
We now preserve the invalid syntax in the input.
Unfortunately, this also seems to be another edge case I didn't consider
in https://github.com/astral-sh/ruff/pull/20867 because we don't flag
this as a syntax error after 0.14.1:
<details><summary>Shell output</summary>
<p>
```
> uvx ruff@0.14.0 check --ignore ALL --target-version py311 - <<EOF
f'{1: abcd "{'aa'}" }'
EOF
invalid-syntax: Cannot reuse outer quote character in f-strings on Python 3.11 (syntax was added in Python 3.12)
 --> -:1:14
  |
1 | f'{1: abcd "{'aa'}" }'
  |              ^
  |
Found 1 error.
> uvx ruff@0.14.1 check --ignore ALL --target-version py311 - <<EOF
f'{1: abcd "{'aa'}" }'
EOF
All checks passed!
> uvx python@3.11 -m ast <<EOF
f'{1: abcd "{'aa'}" }'
EOF
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/brent/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/ast.py", line 1752, in <module>
    main()
  File "/home/brent/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/ast.py", line 1748, in main
    tree = parse(source, args.infile.name, args.mode, type_comments=args.no_type_comments)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/brent/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<stdin>", line 1
    f'{1: abcd "{'aa'}" }'
                  ^^
SyntaxError: f-string: expecting '}'
```
</p>
</details> 
I assumed that was the same `ParseError` as the one caused by
`f"{1:""}"`, but this is a nested interpolation inside of the format
spec.
## Test Plan
New test copied from the black compatibility test. I guess this is a
duplicate now, I started working on this branch before the new black
tests were imported, so I could delete the separate test in our fixtures
if that's preferable.
			
			
This commit is contained in:
		
							parent
							
								
									cfbd42c22a
								
							
						
					
					
						commit
						0115fd3757
					
				
					 6 changed files with 52 additions and 26 deletions
				
			
		|  | @ -748,3 +748,7 @@ print(f"{  # Tuple with multiple elements that doesn't fit on a single line gets | |||
| 
 | ||||
| # Regression tests for https://github.com/astral-sh/ruff/issues/15536 | ||||
| print(f"{ {}, 1, }") | ||||
| 
 | ||||
| 
 | ||||
| # The inner quotes should not be changed to double quotes before Python 3.12 | ||||
| f"{f'''{'nested'} inner'''} outer" | ||||
|  |  | |||
|  | @ -144,6 +144,12 @@ pub(crate) enum InterpolatedStringState { | |||
|     ///
 | ||||
|     /// The containing `FStringContext` is the surrounding f-string context.
 | ||||
|     InsideInterpolatedElement(InterpolatedStringContext), | ||||
|     /// The formatter is inside more than one nested f-string, such as in `nested` in:
 | ||||
|     ///
 | ||||
|     /// ```py
 | ||||
|     /// f"{f'''{'nested'} inner'''} outer"
 | ||||
|     /// ```
 | ||||
|     NestedInterpolatedElement(InterpolatedStringContext), | ||||
|     /// The formatter is outside an f-string.
 | ||||
|     #[default] | ||||
|     Outside, | ||||
|  | @ -152,12 +158,18 @@ pub(crate) enum InterpolatedStringState { | |||
| impl InterpolatedStringState { | ||||
|     pub(crate) fn can_contain_line_breaks(self) -> Option<bool> { | ||||
|         match self { | ||||
|             InterpolatedStringState::InsideInterpolatedElement(context) => { | ||||
|             InterpolatedStringState::InsideInterpolatedElement(context) | ||||
|             | InterpolatedStringState::NestedInterpolatedElement(context) => { | ||||
|                 Some(context.is_multiline()) | ||||
|             } | ||||
|             InterpolatedStringState::Outside => None, | ||||
|         } | ||||
|     } | ||||
| 
 | ||||
|     /// Returns `true` if the interpolated string state is [`NestedInterpolatedElement`].
 | ||||
|     pub(crate) fn is_nested(self) -> bool { | ||||
|         matches!(self, Self::NestedInterpolatedElement(..)) | ||||
|     } | ||||
| } | ||||
| 
 | ||||
| /// The position of a top-level statement in the module.
 | ||||
|  |  | |||
|  | @ -181,10 +181,16 @@ impl Format<PyFormatContext<'_>> for FormatInterpolatedElement<'_> { | |||
| 
 | ||||
|             let item = format_with(|f: &mut PyFormatter| { | ||||
|                 // Update the context to be inside the f-string expression element.
 | ||||
|                 let f = &mut WithInterpolatedStringState::new( | ||||
|                     InterpolatedStringState::InsideInterpolatedElement(self.context), | ||||
|                     f, | ||||
|                 ); | ||||
|                 let state = match f.context().interpolated_string_state() { | ||||
|                     InterpolatedStringState::InsideInterpolatedElement(_) | ||||
|                     | InterpolatedStringState::NestedInterpolatedElement(_) => { | ||||
|                         InterpolatedStringState::NestedInterpolatedElement(self.context) | ||||
|                     } | ||||
|                     InterpolatedStringState::Outside => { | ||||
|                         InterpolatedStringState::InsideInterpolatedElement(self.context) | ||||
|                     } | ||||
|                 }; | ||||
|                 let f = &mut WithInterpolatedStringState::new(state, f); | ||||
| 
 | ||||
|                 write!(f, [bracket_spacing, expression.format()])?; | ||||
| 
 | ||||
|  |  | |||
|  | @ -46,8 +46,15 @@ impl<'a, 'src> StringNormalizer<'a, 'src> { | |||
|             .unwrap_or(self.context.options().quote_style()); | ||||
|         let supports_pep_701 = self.context.options().target_version().supports_pep_701(); | ||||
| 
 | ||||
|         // Preserve the existing quote style for nested interpolations more than one layer deep, if
 | ||||
|         // PEP 701 isn't supported.
 | ||||
|         if !supports_pep_701 && self.context.interpolated_string_state().is_nested() { | ||||
|             return QuoteStyle::Preserve; | ||||
|         } | ||||
| 
 | ||||
|         // For f-strings and t-strings prefer alternating the quotes unless The outer string is triple quoted and the inner isn't.
 | ||||
|         if let InterpolatedStringState::InsideInterpolatedElement(parent_context) = | ||||
|         if let InterpolatedStringState::InsideInterpolatedElement(parent_context) | ||||
|         | InterpolatedStringState::NestedInterpolatedElement(parent_context) = | ||||
|             self.context.interpolated_string_state() | ||||
|         { | ||||
|             let parent_flags = parent_context.flags(); | ||||
|  |  | |||
|  | @ -28,12 +28,11 @@ but none started with prefix {parentdir_prefix}" | |||
|  f'{{NOT \'a\' "formatted" "value"}}' | ||||
|  f"some f-string with {a} {few():.2f} {formatted.values!r}" | ||||
| -f'some f-string with {a} {few(""):.2f} {formatted.values!r}' | ||||
| -f"{f'''{'nested'} inner'''} outer" | ||||
| +f"some f-string with {a} {few(''):.2f} {formatted.values!r}" | ||||
|  f"{f'''{'nested'} inner'''} outer" | ||||
| -f"\"{f'{nested} inner'}\" outer" | ||||
| -f"space between opening braces: { {a for a in (1, 2, 3)}}" | ||||
| -f'Hello \'{tricky + "example"}\'' | ||||
| +f"some f-string with {a} {few(''):.2f} {formatted.values!r}" | ||||
| +f"{f'''{"nested"} inner'''} outer" | ||||
| +f'"{f"{nested} inner"}" outer' | ||||
| +f"space between opening braces: { {a for a in (1, 2, 3)} }" | ||||
| +f"Hello '{tricky + 'example'}'" | ||||
|  | @ -49,7 +48,7 @@ f"{{NOT a formatted value}}" | |||
| f'{{NOT \'a\' "formatted" "value"}}' | ||||
| f"some f-string with {a} {few():.2f} {formatted.values!r}" | ||||
| f"some f-string with {a} {few(''):.2f} {formatted.values!r}" | ||||
| f"{f'''{"nested"} inner'''} outer" | ||||
| f"{f'''{'nested'} inner'''} outer" | ||||
| f'"{f"{nested} inner"}" outer' | ||||
| f"space between opening braces: { {a for a in (1, 2, 3)} }" | ||||
| f"Hello '{tricky + 'example'}'" | ||||
|  | @ -72,17 +71,3 @@ f'Hello \'{tricky + "example"}\'' | |||
| f"Tried directories {str(rootdirs)} \ | ||||
| but none started with prefix {parentdir_prefix}" | ||||
| ``` | ||||
| 
 | ||||
| ## New Unsupported Syntax Errors | ||||
| 
 | ||||
| error[invalid-syntax]: Cannot reuse outer quote character in f-strings on Python 3.10 (syntax was added in Python 3.12) | ||||
|  --> fstring.py:6:9 | ||||
|   | | ||||
| 4 | f"some f-string with {a} {few():.2f} {formatted.values!r}" | ||||
| 5 | f"some f-string with {a} {few(''):.2f} {formatted.values!r}" | ||||
| 6 | f"{f'''{"nested"} inner'''} outer" | ||||
|   |         ^ | ||||
| 7 | f'"{f"{nested} inner"}" outer' | ||||
| 8 | f"space between opening braces: { {a for a in (1, 2, 3)} }" | ||||
|   | | ||||
| warning: Only accept new syntax errors if they are also present in the input. The formatter should not introduce syntax errors. | ||||
|  |  | |||
|  | @ -754,6 +754,10 @@ print(f"{  # Tuple with multiple elements that doesn't fit on a single line gets | |||
| 
 | ||||
| # Regression tests for https://github.com/astral-sh/ruff/issues/15536 | ||||
| print(f"{ {}, 1, }") | ||||
| 
 | ||||
| 
 | ||||
| # The inner quotes should not be changed to double quotes before Python 3.12 | ||||
| f"{f'''{'nested'} inner'''} outer" | ||||
| ``` | ||||
| 
 | ||||
| ## Outputs | ||||
|  | @ -1532,7 +1536,7 @@ f'{f"""other " """}' | |||
| f'{1: hy "user"}' | ||||
| f'{1:hy "user"}' | ||||
| f'{1: abcd "{1}" }' | ||||
| f'{1: abcd "{"aa"}" }' | ||||
| f'{1: abcd "{'aa'}" }' | ||||
| f'{1=: "abcd {'aa'}}' | ||||
| f"{x:a{z:hy \"user\"}} '''" | ||||
| 
 | ||||
|  | @ -1581,6 +1585,10 @@ print( | |||
| 
 | ||||
| # Regression tests for https://github.com/astral-sh/ruff/issues/15536 | ||||
| print(f"{ {}, 1 }") | ||||
| 
 | ||||
| 
 | ||||
| # The inner quotes should not be changed to double quotes before Python 3.12 | ||||
| f"{f'''{'nested'} inner'''} outer" | ||||
| ``` | ||||
| 
 | ||||
| 
 | ||||
|  | @ -2359,7 +2367,7 @@ f'{f"""other " """}' | |||
| f'{1: hy "user"}' | ||||
| f'{1:hy "user"}' | ||||
| f'{1: abcd "{1}" }' | ||||
| f'{1: abcd "{"aa"}" }' | ||||
| f'{1: abcd "{'aa'}" }' | ||||
| f'{1=: "abcd {'aa'}}' | ||||
| f"{x:a{z:hy \"user\"}} '''" | ||||
| 
 | ||||
|  | @ -2408,6 +2416,10 @@ print( | |||
| 
 | ||||
| # Regression tests for https://github.com/astral-sh/ruff/issues/15536 | ||||
| print(f"{ {}, 1 }") | ||||
| 
 | ||||
| 
 | ||||
| # The inner quotes should not be changed to double quotes before Python 3.12 | ||||
| f"{f'''{'nested'} inner'''} outer" | ||||
| ``` | ||||
| 
 | ||||
| 
 | ||||
|  |  | |||
		Loading…
	
	Add table
		Add a link
		
	
		Reference in a new issue
	
	 Brent Westbrook
						Brent Westbrook