[syntax-errors] Parenthesized context managers before Python 3.9 (#16523)

Summary
--

I thought this was very complicated based on the comment here:
https://github.com/astral-sh/ruff/pull/16106#issuecomment-2653505671 and
on some of the discussion in the CPython issue here:
https://github.com/python/cpython/issues/56991. However, after a little
bit of experimentation, I think it boils down to this example:

```python
with (x as y): ...
```

The issue is parentheses around a `with` item with an `optional_var`, as
we (and
[Python](https://docs.python.org/3/library/ast.html#ast.withitem)) call
the trailing variable name (`y` in this case). It's not actually about
line breaks after all, except that line breaks are allowed in
parenthesized expressions, which explains the validity of cases like


```pycon
>>> with (
...     x,
...     y
... ) as foo:
...     pass
... 
```

even on Python 3.8.

I followed [pyright]'s example again here on the diagnostic range (just
the opening paren) and the wording of the error.


Test Plan
--
Inline tests

[pyright]:
https://pyright-play.net/?pythonVersion=3.7&strict=true&code=FAdwlgLgFgBAFAewA4FMB2cBEAzBCB0EAHhJgJQwCGAzjLgmQFwz6tA
This commit is contained in:
Brent Westbrook 2025-03-17 08:54:55 -04:00 committed by GitHub
parent 8d3643f409
commit 75a562d313
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 800 additions and 0 deletions

View file

@ -661,6 +661,46 @@ pub enum UnsupportedSyntaxErrorKind {
TypeAliasStatement,
TypeParamDefault,
/// Represents the use of a parenthesized `with` item before Python 3.9.
///
/// ## Examples
///
/// As described in [BPO 12782], `with` uses like this were not allowed on Python 3.8:
///
/// ```python
/// with (open("a_really_long_foo") as foo,
/// open("a_really_long_bar") as bar):
/// pass
/// ```
///
/// because parentheses were not allowed within the `with` statement itself (see [this comment]
/// in particular). However, parenthesized expressions were still allowed, including the cases
/// below, so the issue can be pretty subtle and relates specifically to parenthesized items
/// with `as` bindings.
///
/// ```python
/// with (foo, bar): ... # okay
/// with (
/// open('foo.txt')) as foo: ... # also okay
/// with (
/// foo,
/// bar,
/// baz,
/// ): ... # also okay, just a tuple
/// with (
/// foo,
/// bar,
/// baz,
/// ) as tup: ... # also okay, binding the tuple
/// ```
///
/// This restriction was lifted in 3.9 but formally included in the [release notes] for 3.10.
///
/// [BPO 12782]: https://github.com/python/cpython/issues/56991
/// [this comment]: https://github.com/python/cpython/issues/56991#issuecomment-1093555141
/// [release notes]: https://docs.python.org/3/whatsnew/3.10.html#summary-release-highlights
ParenthesizedContextManager,
/// Represents the use of a [PEP 646] star expression in an index.
///
/// ## Examples
@ -798,6 +838,9 @@ impl Display for UnsupportedSyntaxError {
UnsupportedSyntaxErrorKind::TypeParamDefault => {
"Cannot set default type for a type parameter"
}
UnsupportedSyntaxErrorKind::ParenthesizedContextManager => {
"Cannot use parentheses within a `with` statement"
}
UnsupportedSyntaxErrorKind::StarExpressionInIndex => {
"Cannot use star expression in index"
}
@ -861,6 +904,9 @@ impl UnsupportedSyntaxErrorKind {
UnsupportedSyntaxErrorKind::TypeParameterList => Change::Added(PythonVersion::PY312),
UnsupportedSyntaxErrorKind::TypeAliasStatement => Change::Added(PythonVersion::PY312),
UnsupportedSyntaxErrorKind::TypeParamDefault => Change::Added(PythonVersion::PY313),
UnsupportedSyntaxErrorKind::ParenthesizedContextManager => {
Change::Added(PythonVersion::PY39)
}
UnsupportedSyntaxErrorKind::StarExpressionInIndex => {
Change::Added(PythonVersion::PY311)
}

View file

@ -2066,8 +2066,49 @@ impl<'src> Parser<'src> {
return vec![];
}
let open_paren_range = self.current_token_range();
if self.at(TokenKind::Lpar) {
if let Some(items) = self.try_parse_parenthesized_with_items() {
// test_ok tuple_context_manager_py38
// # parse_options: {"target-version": "3.8"}
// with (
// foo,
// bar,
// baz,
// ) as tup: ...
// test_err tuple_context_manager_py38
// # parse_options: {"target-version": "3.8"}
// # these cases are _syntactically_ valid before Python 3.9 because the `with` item
// # is parsed as a tuple, but this will always cause a runtime error, so we flag it
// # anyway
// with (foo, bar): ...
// with (
// open('foo.txt')) as foo: ...
// with (
// foo,
// bar,
// baz,
// ): ...
// with (foo,): ...
// test_ok parenthesized_context_manager_py39
// # parse_options: {"target-version": "3.9"}
// with (foo as x, bar as y): ...
// with (foo, bar as y): ...
// with (foo as x, bar): ...
// test_err parenthesized_context_manager_py38
// # parse_options: {"target-version": "3.8"}
// with (foo as x, bar as y): ...
// with (foo, bar as y): ...
// with (foo as x, bar): ...
self.add_unsupported_syntax_error(
UnsupportedSyntaxErrorKind::ParenthesizedContextManager,
open_paren_range,
);
self.expect(TokenKind::Rpar);
items
} else {