language-servers/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-11-01 20:31:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

Raise syntax error for unparenthesized generator expr in multi-argument call (#12445)

Browse source 
## Summary

This PR fixes a bug to raise a syntax error when an unparenthesized
generator expression is used as an argument to a call when there are
more than one argument.

For reference, the grammar is:
```
primary:
    | ...
    | primary genexp 
    | primary '(' [arguments] ')' 
    | ...

genexp:
    | '(' ( assignment_expression | expression !':=') for_if_clauses ')' 
```

The `genexp` requires the parenthesis as mentioned in the grammar. So,
the grammar for a call expression is either a name followed by a
generator expression or a name followed by a list of argument. In the
former case, the parenthesis are excluded because the generator
expression provides them while in the later case, the parenthesis are
explicitly provided for a list of arguments which means that the
generator expression requires it's own parenthesis.

This was discovered in https://github.com/astral-sh/ruff/issues/12420.

## Test Plan

Add test cases for valid and invalid syntax.

Make sure that the parser from CPython also raises this at the parsing
step:
```console
$ python3.13 -m ast parser/_.py
  File "parser/_.py", line 1
    total(1, 2, x for x in range(5), 6)
                ^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized

$ python3.13 -m ast parser/_.py
  File "parser/_.py", line 1
    sum(x for x in range(10), 10)
        ^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
```
This commit is contained in:
Dhruv Manilawala 2024-07-22 14:44:20 +05:30 • committed by GitHub
parent f8735e1ee8
commit 978909fcf4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 336 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

213
crates/ruff_python_parser/tests/snapshots/invalid_syntax@args_unparenthesized_generator.py.snap Normal file
View file

@ -0,0 +1,213 @@
---
source: crates/ruff_python_parser/tests/fixtures.rs
input_file: crates/ruff_python_parser/resources/inline/err/args_unparenthesized_generator.py
---
## AST
```
Module(
ModModule {
range: 0..65,
body: [
Expr(
StmtExpr {
range: 0..28,
value: Call(
ExprCall {
range: 0..28,
func: Name(
ExprName {
range: 0..3,
id: Name("sum"),
ctx: Load,
},
),
arguments: Arguments {
range: 3..28,
args: [
Generator(
ExprGenerator {
range: 4..24,
elt: Name(
ExprName {
range: 4..5,
id: Name("x"),
ctx: Load,
},
),
generators: [
Comprehension {
range: 6..24,
target: Name(
ExprName {
range: 10..11,
id: Name("x"),
ctx: Store,
},
),
iter: Call(
ExprCall {
range: 15..24,
func: Name(
ExprName {
range: 15..20,
id: Name("range"),
ctx: Load,
},
),
arguments: Arguments {
range: 20..24,
args: [
NumberLiteral(
ExprNumberLiteral {
range: 21..23,
value: Int(
10,
),
},
),
],
keywords: [],
},
},
),
ifs: [],
is_async: false,
},
],
parenthesized: false,
},
),
NumberLiteral(
ExprNumberLiteral {
range: 26..27,
value: Int(
5,
),
},
),
],
keywords: [],
},
},
),
},
),
Expr(
StmtExpr {
range: 29..64,
value: Call(
ExprCall {
range: 29..64,
func: Name(
ExprName {
range: 29..34,
id: Name("total"),
ctx: Load,
},
),
arguments: Arguments {
range: 34..64,
args: [
NumberLiteral(
ExprNumberLiteral {
range: 35..36,
value: Int(
1,
),
},
),
NumberLiteral(
ExprNumberLiteral {
range: 38..39,
value: Int(
2,
),
},
),
Generator(
ExprGenerator {
range: 41..60,
elt: Name(
ExprName {
range: 41..42,
id: Name("x"),
ctx: Load,
},
),
generators: [
Comprehension {
range: 43..60,
target: Name(
ExprName {
range: 47..48,
id: Name("x"),
ctx: Store,
},
),
iter: Call(
ExprCall {
range: 52..60,
func: Name(
ExprName {
range: 52..57,
id: Name("range"),
ctx: Load,
},
),
arguments: Arguments {
range: 57..60,
args: [
NumberLiteral(
ExprNumberLiteral {
range: 58..59,
value: Int(
5,
),
},
),
],
keywords: [],
},
},
),
ifs: [],
is_async: false,
},
],
parenthesized: false,
},
),
NumberLiteral(
ExprNumberLiteral {
range: 62..63,
value: Int(
6,
),
},
),
],
keywords: [],
},
},
),
},
),
],
},
)
```
## Errors
|
1 | sum(x for x in range(10), 5)
| ^^^^^^^^^^^^^^^^^^^^ Syntax Error: Unparenthesized generator expression cannot be used here
2 | total(1, 2, x for x in range(5), 6)
|
|
1 | sum(x for x in range(10), 5)
2 | total(1, 2, x for x in range(5), 6)
| ^^^^^^^^^^^^^^^^^^^ Syntax Error: Unparenthesized generator expression cannot be used here
|

91
crates/ruff_python_parser/tests/snapshots/valid_syntax@args_unparenthesized_generator.py.snap Normal file
View file

@ -0,0 +1,91 @@
---
source: crates/ruff_python_parser/tests/fixtures.rs
input_file: crates/ruff_python_parser/resources/inline/ok/args_unparenthesized_generator.py
---
## AST
```
Module(
ModModule {
range: 0..26,
body: [
Expr(
StmtExpr {
range: 0..25,
value: Call(
ExprCall {
range: 0..25,
func: Name(
ExprName {
range: 0..3,
id: Name("sum"),
ctx: Load,
},
),
arguments: Arguments {
range: 3..25,
args: [
Generator(
ExprGenerator {
range: 4..24,
elt: Name(
ExprName {
range: 4..5,
id: Name("x"),
ctx: Load,
},
),
generators: [
Comprehension {
range: 6..24,
target: Name(
ExprName {
range: 10..11,
id: Name("x"),
ctx: Store,
},
),
iter: Call(
ExprCall {
range: 15..24,
func: Name(
ExprName {
range: 15..20,
id: Name("range"),
ctx: Load,
},
),
arguments: Arguments {
range: 20..24,
args: [
NumberLiteral(
ExprNumberLiteral {
range: 21..23,
value: Int(
10,
),
},
),
],
keywords: [],
},
},
),
ifs: [],
is_async: false,
},
],
parenthesized: false,
},
),
],
keywords: [],
},
},
),
},
),
],
},
)
```