language-servers/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-09-29 13:25:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3
ruff/crates/ruff_python_parser/src/snapshots
History
Charlie Marsh 68f605e80a
Fix WithItem ranges for parenthesized, non-as items (#6782) 
## Summary

This PR attempts to address a problem in the parser related to the
range's of `WithItem` nodes in certain contexts -- specifically,
`WithItem` nodes in parentheses that do not have an `as` token after
them.

For example,
[here](https://play.ruff.rs/71be2d0b-2a04-4c7e-9082-e72bff152679):

```python
with (a, b):
    pass
```

The range of the `WithItem` `a` is set to the range of `(a, b)`, as is
the range of the `WithItem` `b`. In other words, when we have this kind
of sequence, we use the range of the entire parenthesized context,
rather than the ranges of the items themselves.

Note that this also applies to cases
[like](https://play.ruff.rs/c551e8e9-c3db-4b74-8cc6-7c4e3bf3713a):

```python
with (a, b, c as d):
    pass
```

You can see the issue in the parser here:

```rust
#[inline]
WithItemsNoAs: Vec<ast::WithItem> = {
    <location:@L> <all:OneOrMore<Test<"all">>> <end_location:@R> => {
        all.into_iter().map(|context_expr| ast::WithItem { context_expr, optional_vars: None, range: (location..end_location).into() }).collect()
    },
}
```

Fixing this issue is... very tricky. The naive approach is to use the
range of the `context_expr` as the range for the `WithItem`, but that
range will be incorrect when the `context_expr` is itself parenthesized.
For example, _that_ solution would fail here, since the range of the
first `WithItem` would be that of `a`, rather than `(a)`:

```python
with ((a), b):
    pass
```

The `with` parsing in general is highly precarious due to ambiguities in
the grammar. Changing it in _any_ way seems to lead to an ambiguous
grammar that LALRPOP fails to translate. Consensus seems to be that we
don't really understand _why_ the current grammar works (i.e., _how_ it
avoids these ambiguities as-is).

The solution implemented here is to avoid changing the grammar itself,
and instead change the shape of the nodes returned by various rules in
the grammar. Specifically, everywhere that we return `Expr`, we instead
return `ParenthesizedExpr`, which includes a parenthesized range and the
underlying `Expr` itself. (If an `Expr` isn't parenthesized, the ranges
will be equivalent.) In `WithItemsNoAs`, we can then use the
parenthesized range as the range for the `WithItem`.
2023-08-31 16:21:29 +01:00
..
ruff_python_parser__context__tests__ann_assign_name.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__context__tests__assign_attribute.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__context__tests__assign_for.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__context__tests__assign_list.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__context__tests__assign_list_comp.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__context__tests__assign_name.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__context__tests__assign_named_expr.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__context__tests__assign_set_comp.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__context__tests__assign_starred.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__context__tests__assign_subscript.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__context__tests__assign_tuple.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__context__tests__assign_with.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__context__tests__aug_assign_attribute.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__context__tests__aug_assign_name.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__context__tests__aug_assign_subscript.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__context__tests__del_attribute.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__context__tests__del_name.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__context__tests__del_subscript.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__function__tests__function_kw_only_args.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_kw_only_args_with_defaults.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_no_args.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_no_args_with_ranges.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_pos_and_kw_only_args.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_pos_and_kw_only_args_with_defaults.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_pos_and_kw_only_args_with_defaults_and_varargs.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_pos_and_kw_only_args_with_defaults_and_varargs_and_kwargs.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_pos_args.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_pos_args_with_defaults.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__function_pos_args_with_ranges.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__function__tests__lambda_kw_only_args.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__function__tests__lambda_kw_only_args_with_defaults.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__function__tests__lambda_no_args.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__function__tests__lambda_pos_and_kw_only_args.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__function__tests__lambda_pos_args.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__function__tests__lambda_pos_args_with_defaults.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__lexer__tests__assignment.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__empty_ipython_escape_command.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__ipython_escape_command.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__ipython_escape_command_assignment.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__ipython_escape_command_indentation.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__ipython_help_end_escape_command.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__logical_newline_line_comment.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__non_logical_newline_in_string_continuation.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__numbers.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__operators.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__lexer__tests__string.snap Update lexer tests to use snapshots (#6658) 2023-08-22 18:23:19 +00:00
ruff_python_parser__parser__tests__decorator_ranges.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__parser__tests__dict_unpacking.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__parser__tests__generator_expression_argument.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__parser__tests__ipython_escape_commands.snap Rename Magic* to IpyEscape* (#6395) 2023-08-09 13:28:18 +00:00
ruff_python_parser__parser__tests__match.snap Introduce AST nodes for PatternMatchClass arguments (#6881) 2023-08-26 14:45:44 +00:00
ruff_python_parser__parser__tests__match_as_identifier.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__parser__tests__named_expression.snap Expand NamedExpr range to include full range of parenthesized value (#6632) 2023-08-17 14:34:05 +00:00
ruff_python_parser__parser__tests__numeric_literals.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__parser__tests__numeric_literals_attribute_access.snap Introduce an Arguments AST node for function calls and class definitions (#6259) 2023-08-02 10:01:13 -04:00
ruff_python_parser__parser__tests__parenthesized_with_statement.snap Fix WithItem ranges for parenthesized, non-as items (#6782) 2023-08-31 16:21:29 +01:00
ruff_python_parser__parser__tests__parse_bool_op_and.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_bool_op_or.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_class.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__parser__tests__parse_class_generic_types.snap Add a TypeParams node to the AST (#6261) 2023-08-02 14:12:45 +00:00
ruff_python_parser__parser__tests__parse_dict_comprehension.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_double_list_comprehension.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_empty.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_f_string.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__parser__tests__parse_function_definition.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__parser__tests__parse_generator_comprehension.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_if_elif_else.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_if_else_generator_comprehension.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_kwargs.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__parser__tests__parse_lambda.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__parser__tests__parse_lambda_no_args.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__parser__tests__parse_list_comprehension.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_named_expression_generator_comprehension.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__parse_print_2.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__parser__tests__parse_print_hello.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__parser__tests__parse_string.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__parser__tests__parse_tuples.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__parser__tests__parse_type_declaration.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__parser__tests__patma.snap Introduce AST nodes for PatternMatchClass arguments (#6881) 2023-08-26 14:45:44 +00:00
ruff_python_parser__parser__tests__slice.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__parser__tests__star_index.snap Remove unsupported type_comment field 2023-08-01 12:53:13 +02:00
ruff_python_parser__parser__tests__try.snap Remove Stmt::TryStar (#6566) 2023-08-14 13:39:44 -04:00
ruff_python_parser__parser__tests__try_star.snap Remove Stmt::TryStar (#6566) 2023-08-14 13:39:44 -04:00
ruff_python_parser__parser__tests__type_as_identifier.snap Make Parameters an optional field on ExprLambda (#6669) 2023-08-18 15:34:54 +00:00
ruff_python_parser__parser__tests__variadic_generics.snap Remove async AST node variants for with, for, and def (#6369) 2023-08-07 16:36:02 +00:00
ruff_python_parser__parser__tests__with_statement.snap Fix WithItem ranges for parenthesized, non-as items (#6782) 2023-08-31 16:21:29 +01:00
ruff_python_parser__string__tests__backspace_alias.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__bell_alias.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__carriage_return_alias.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__character_tabulation_with_justification_alias.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__delete_alias.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__double_quoted_byte.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__escape_alias.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__escape_char_in_byte_literal.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__escape_octet.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__form_feed_alias.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__fstring_constant_range.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__fstring_escaped_character.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__fstring_escaped_newline.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__fstring_line_continuation.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__fstring_parse_self_documenting_base.snap add DebugText for self-documenting f-strings (#6167) 2023-08-01 07:55:03 +02:00
ruff_python_parser__string__tests__fstring_parse_self_documenting_base_more.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__fstring_parse_self_documenting_format.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__fstring_unescaped_newline.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__hts_alias.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_empty_fstring.snap Pull in RustPython parser (#6099) 2023-07-27 09:29:11 +00:00
ruff_python_parser__string__tests__parse_f_string_concat_1.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_f_string_concat_2.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_f_string_concat_3.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_f_string_concat_4.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_fstring.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_fstring_equals.snap add DebugText for self-documenting f-strings (#6167) 2023-08-01 07:55:03 +02:00
ruff_python_parser__string__tests__parse_fstring_nested_concatenation_string_spec.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_fstring_nested_spec.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_fstring_nested_string_spec.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_fstring_not_equals.snap add DebugText for self-documenting f-strings (#6167) 2023-08-01 07:55:03 +02:00
ruff_python_parser__string__tests__parse_fstring_not_nested_spec.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_fstring_self_doc_prec_space.snap add DebugText for self-documenting f-strings (#6167) 2023-08-01 07:55:03 +02:00
ruff_python_parser__string__tests__parse_fstring_self_doc_trailing_space.snap add DebugText for self-documenting f-strings (#6167) 2023-08-01 07:55:03 +02:00
ruff_python_parser__string__tests__parse_fstring_yield_expr.snap add DebugText for self-documenting f-strings (#6167) 2023-08-01 07:55:03 +02:00
ruff_python_parser__string__tests__parse_string_concat.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_string_triple_quotes_with_kind.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_u_f_string_concat_1.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_u_f_string_concat_2.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_u_string_concat_1.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__parse_u_string_concat_2.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__raw_byte_literal_1.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__raw_byte_literal_2.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__raw_fstring.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__single_quoted_byte.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00
ruff_python_parser__string__tests__triple_quoted_raw_fstring.snap Add an implicit concatenation flag to string and bytes constants (#6512) 2023-08-14 13:46:54 +00:00