mirrors/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-07-19 02:55:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Update string nodes for implicit concatenation (#7927)

Browse source 
## Summary

This PR updates the string nodes (`ExprStringLiteral`,
`ExprBytesLiteral`, and `ExprFString`) to account for implicit string
concatenation.

### Motivation

In Python, implicit string concatenation are joined while parsing
because the interpreter doesn't require the information for each part.
While that's feasible for an interpreter, it falls short for a static
analysis tool where having such information is more useful. Currently,
various parts of the code uses the lexer to get the individual string
parts.

One of the main challenge this solves is that of string formatting.
Currently, the formatter relies on the lexer to get the individual
string parts, and formats them including the comments accordingly. But,
with PEP 701, f-string can also contain comments. Without this change,
it becomes very difficult to add support for f-string formatting.

### Implementation

The initial proposal was made in this discussion:
https://github.com/astral-sh/ruff/discussions/6183#discussioncomment-6591993.
There were various AST designs which were explored for this task which
are available in the linked internal document[^1].

The selected variant was the one where the nodes were kept as it is
except that the `implicit_concatenated` field was removed and instead a
new struct was added to the `Expr*` struct. This would be a private
struct would contain the actual implementation of how the AST is
designed for both single and implicitly concatenated strings.

This implementation is achieved through an enum with two variants:
`Single` and `Concatenated` to avoid allocating a vector even for single
strings. There are various public methods available on the value struct
to query certain information regarding the node.

The nodes are structured in the following way:

```
ExprStringLiteral - "foo" "bar"
|- StringLiteral - "foo"
|- StringLiteral - "bar"

ExprBytesLiteral - b"foo" b"bar"
|- BytesLiteral - b"foo"
|- BytesLiteral - b"bar"

ExprFString - "foo" f"bar {x}"
|- FStringPart::Literal - "foo"
|- FStringPart::FString - f"bar {x}"
  |- StringLiteral - "bar "
  |- FormattedValue - "x"
```

[^1]: Internal document:
https://www.notion.so/astral-sh/Implicit-String-Concatenation-e036345dc48943f89e416c087bf6f6d9?pvs=4

#### Visitor

The way the nodes are structured is that the entire string, including
all the parts that are implicitly concatenation, is a single node
containing individual nodes for the parts. The previous section has a
representation of that tree for all the string nodes. This means that
new visitor methods are added to visit the individual parts of string,
bytes, and f-strings for `Visitor`, `PreorderVisitor`, and
`Transformer`.

## Test Plan

- `cargo insta test --workspace --all-features --unreferenced reject`
- Verify that the ecosystem results are unchanged
This commit is contained in:
Dhruv Manilawala 2023-11-24 17:55:41 -06:00 • committed by GitHub
parent 2590aa30ae
commit 017e829115
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
121 changed files with 27666 additions and 25501 deletions
Download patch file Download diff file Expand all files Collapse all files

56
crates/ruff_python_parser/src/python.lalrpop
View file

@ -11,7 +11,7 @@ use crate::{
lexer::{LexicalError, LexicalErrorType},
function::{ArgumentList, parse_arguments, validate_pos_params, validate_arguments},
context::set_context,
string::{StringType, concatenate_strings, parse_fstring_middle, parse_string_literal},
string::{StringType, concatenated_strings, parse_fstring_middle, parse_string_literal},
token::{self, StringKind},
invalid,
};
@ -491,7 +491,7 @@ MatchStatement: ast::Stmt = {
}
)
},
<location:@L> "match" <tuple_location:@L> <elts:TwoOrMore<TestOrStarNamedExpr, ",">> ","? <tuple_end_location:@R> ":" "\n" Indent <cases:MatchCase+> Dedent => {
<location:@L> "match" <tuple_location:@L> <elts:TwoOrMoreSep<TestOrStarNamedExpr, ",">> ","? <tuple_end_location:@R> ":" "\n" Indent <cases:MatchCase+> Dedent => {
let end_location = cases
.last()
.unwrap()
@ -542,7 +542,7 @@ Patterns: ast::Pattern = {
range: (location..end_location).into()
},
),
<location:@L> <patterns:TwoOrMore<Pattern, ",">> ","? <end_location:@R> => {
<location:@L> <patterns:TwoOrMoreSep<Pattern, ",">> ","? <end_location:@R> => {
ast::Pattern::MatchSequence(
ast::PatternMatchSequence {
patterns,
@ -579,7 +579,7 @@ AsPattern: ast::Pattern = {
OrPattern: ast::Pattern = {
<pattern:ClosedPattern> => pattern,
<location:@L> <patterns:TwoOrMore<ClosedPattern, "|">> <end_location:@R> => {
<location:@L> <patterns:TwoOrMoreSep<ClosedPattern, "|">> <end_location:@R> => {
ast::Pattern::MatchOr(
ast::PatternMatchOr { patterns, range: (location..end_location).into() }
)
@ -677,8 +677,12 @@ LiteralPattern: ast::Pattern = {
value: Box::new(value.into()),
range: (location..end_location).into()
}.into(),
<location:@L> <strings:StringLiteral+> <end_location:@R> =>? Ok(ast::PatternMatchValue {
value: Box::new(concatenate_strings(strings, (location..end_location).into())?),
<location:@L> <string:StringLiteral> <end_location:@R> => ast::PatternMatchValue {
value: Box::new(string.into()),
range: (location..end_location).into()
}.into(),
<location:@L> <strings:TwoOrMore<StringLiteral>> <end_location:@R> =>? Ok(ast::PatternMatchValue {
value: Box::new(concatenated_strings(strings, (location..end_location).into())?),
range: (location..end_location).into()
}.into()),
}
@ -721,6 +725,7 @@ ValuePattern: ast::Pattern = {
MappingKey: ast::Expr = {
MatchNameOrAttr,
String,
<e:NumberExpr> => e.into(),
<e:AddOpExpr> => e.into(),
<location:@L> "None" <end_location:@R> => ast::ExprNoneLiteral {
@ -734,7 +739,6 @@ MappingKey: ast::Expr = {
value: false,
range: (location..end_location).into()
}.into(),
<location:@L> <strings:StringLiteralOrFString+> <end_location:@R> =>? Ok(concatenate_strings(strings, (location..end_location).into())?),
}
MatchMappingEntry: (ast::Expr, ast::Pattern) = {
@ -1561,7 +1565,7 @@ SubscriptList: ast::ParenthesizedExpr = {
range: (location..end_location).into(),
}.into()
},
<location:@L> <elts:TwoOrMore<Subscript, ",">> ","? <end_location:@R> => {
<location:@L> <elts:TwoOrMoreSep<Subscript, ",">> ","? <end_location:@R> => {
let elts = elts.into_iter().map(ast::Expr::from).collect();
ast::ExprTuple {
elts,
@ -1587,23 +1591,29 @@ SliceOp: Option<ast::ParenthesizedExpr> = {
<location:@L> ":" <e:Test<"all">?> => e,
}
String: ast::Expr = {
<location:@L> <string:StringLiteralOrFString> => string.into(),
<location:@L> <strings:TwoOrMore<StringLiteralOrFString>> <end_location:@R> =>? {
Ok(concatenated_strings(strings, (location..end_location).into())?)
}
};
StringLiteralOrFString: StringType = {
StringLiteral,
FStringExpr,
};
StringLiteral: StringType = {
<start_location:@L> <string:string> =>? {
<location:@L> <string:string> <end_location:@R> =>? {
let (source, kind, triple_quoted) = string;
Ok(parse_string_literal(&source, kind, triple_quoted, start_location)?)
Ok(parse_string_literal(&source, kind, triple_quoted, (location..end_location).into())?)
}
};
FStringExpr: StringType = {
<location:@L> FStringStart <values:FStringMiddlePattern*> FStringEnd <end_location:@R> => {
StringType::FString(ast::ExprFString {
StringType::FString(ast::FString {
values,
implicit_concatenated: false,
range: (location..end_location).into()
})
}
@ -1611,9 +1621,9 @@ FStringExpr: StringType = {
FStringMiddlePattern: ast::Expr = {
FStringReplacementField,
<start_location:@L> <fstring_middle:fstring_middle> =>? {
<location:@L> <fstring_middle:fstring_middle> <end_location:@R> =>? {
let (source, is_raw) = fstring_middle;
Ok(parse_fstring_middle(&source, is_raw, start_location)?)
Ok(parse_fstring_middle(&source, is_raw, (location..end_location).into())?)
}
};
@ -1661,9 +1671,8 @@ FStringFormatSpecSuffix: ast::Expr = {
FStringFormatSpec: ast::Expr = {
<location:@L> <values:FStringMiddlePattern*> <end_location:@R> => {
ast::ExprFString {
ast::FString {
values,
implicit_concatenated: false,
range: (location..end_location).into()
}.into()
},
@ -1685,7 +1694,7 @@ FStringConversion: (TextSize, ast::ConversionFlag) = {
};
Atom<Goal>: ast::ParenthesizedExpr = {
<location:@L> <strings:StringLiteralOrFString+> <end_location:@R> =>? Ok(concatenate_strings(strings, (location..end_location).into())?.into()),
<expr:String> => expr.into(),
<location:@L> <value:Number> <end_location:@R> => ast::ExprNumberLiteral {
value,
range: (location..end_location).into(),
@ -1926,9 +1935,18 @@ OneOrMore<T>: Vec<T> = {
};
/// Two or more items that are separated by `Sep`
TwoOrMore<T, Sep>: Vec<T> = {
TwoOrMoreSep<T, Sep>: Vec<T> = {
<e1:T> Sep <e2:T> => vec![e1, e2],
<mut v: TwoOrMore<T, Sep>> Sep <e:T> => {
<mut v: TwoOrMoreSep<T, Sep>> Sep <e:T> => {
v.push(e);
v
}
};
/// Two or more items that are contiguous.
TwoOrMore<T>: Vec<T> = {
<e1:T> <e2:T> => vec![e1, e2],
<mut v: TwoOrMore<T>> <e:T> => {
v.push(e);
v
}

44905
crates/ruff_python_parser/src/python.rs
View file

File diff suppressed because it is too large Load diff

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__invalid__tests__ok_attribute_weird.snap
View file

@ -14,9 +14,15 @@ Ok(
value: StringLiteral(
ExprStringLiteral {
range: 0..5,
value: "foo",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..5,
value: "foo",
unicode: false,
},
),
},
},
),
attr: Identifier {

48
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__dict_unpacking.snap
View file

@ -10,9 +10,15 @@ Dict(
StringLiteral(
ExprStringLiteral {
range: 1..4,
value: "a",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 1..4,
value: "a",
unicode: false,
},
),
},
},
),
),
@ -21,9 +27,15 @@ Dict(
StringLiteral(
ExprStringLiteral {
range: 16..19,
value: "d",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 16..19,
value: "d",
unicode: false,
},
),
},
},
),
),
@ -32,9 +44,15 @@ Dict(
StringLiteral(
ExprStringLiteral {
range: 6..9,
value: "b",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 6..9,
value: "b",
unicode: false,
},
),
},
},
),
Name(
@ -47,9 +65,15 @@ Dict(
StringLiteral(
ExprStringLiteral {
range: 21..24,
value: "e",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 21..24,
value: "e",
unicode: false,
},
),
},
},
),
],

1400
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__fstrings.snap
View file

File diff suppressed because it is too large Load diff

336
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__fstrings_with_unicode.snap
View file

@ -9,40 +9,55 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..29,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..5,
value: "foo",
unicode: true,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 9..14,
value: Name(
ExprName {
range: 10..13,
id: "bar",
ctx: Load,
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..6,
value: "foo",
unicode: true,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 7..15,
values: [
FormattedValue(
ExprFormattedValue {
range: 9..14,
value: Name(
ExprName {
range: 10..13,
id: "bar",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
Literal(
StringLiteral {
range: 16..21,
value: "baz",
unicode: false,
},
),
Literal(
StringLiteral {
range: 22..29,
value: " some",
unicode: false,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 17..28,
value: "baz some",
unicode: false,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},
@ -53,40 +68,55 @@ expression: parse_ast
value: FString(
ExprFString {
range: 30..59,
values: [
StringLiteral(
ExprStringLiteral {
range: 31..34,
value: "foo",
unicode: false,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 38..43,
value: Name(
ExprName {
range: 39..42,
id: "bar",
ctx: Load,
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 30..35,
value: "foo",
unicode: false,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 36..44,
values: [
FormattedValue(
ExprFormattedValue {
range: 38..43,
value: Name(
ExprName {
range: 39..42,
id: "bar",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
Literal(
StringLiteral {
range: 45..51,
value: "baz",
unicode: true,
},
),
Literal(
StringLiteral {
range: 52..59,
value: " some",
unicode: false,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 47..58,
value: "baz some",
unicode: true,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},
@ -97,40 +127,55 @@ expression: parse_ast
value: FString(
ExprFString {
range: 60..89,
values: [
StringLiteral(
ExprStringLiteral {
range: 61..64,
value: "foo",
unicode: false,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 68..73,
value: Name(
ExprName {
range: 69..72,
id: "bar",
ctx: Load,
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 60..65,
value: "foo",
unicode: false,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 66..74,
values: [
FormattedValue(
ExprFormattedValue {
range: 68..73,
value: Name(
ExprName {
range: 69..72,
id: "bar",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
Literal(
StringLiteral {
range: 75..80,
value: "baz",
unicode: false,
},
),
Literal(
StringLiteral {
range: 81..89,
value: " some",
unicode: true,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 76..88,
value: "baz some",
unicode: false,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},
@ -141,40 +186,83 @@ expression: parse_ast
value: FString(
ExprFString {
range: 90..128,
values: [
StringLiteral(
ExprStringLiteral {
range: 92..103,
value: "foobar ",
unicode: true,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 103..108,
value: Name(
ExprName {
range: 104..107,
id: "baz",
ctx: Load,
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 90..96,
value: "foo",
unicode: true,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 97..116,
values: [
StringLiteral(
ExprStringLiteral {
range: 99..103,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 99..103,
value: "bar ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 103..108,
value: Name(
ExprName {
range: 104..107,
id: "baz",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 108..115,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 108..115,
value: " really",
unicode: false,
},
),
},
},
),
],
},
),
Literal(
StringLiteral {
range: 117..123,
value: "bar",
unicode: true,
},
),
Literal(
StringLiteral {
range: 124..128,
value: "no",
unicode: false,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 108..127,
value: " reallybarno",
unicode: false,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},

36
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__generator_expression_argument.snap
View file

@ -11,9 +11,15 @@ Call(
value: StringLiteral(
ExprStringLiteral {
range: 0..3,
value: " ",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..3,
value: " ",
unicode: false,
},
),
},
},
),
attr: Identifier {
@ -66,9 +72,15 @@ Call(
left: StringLiteral(
ExprStringLiteral {
range: 43..53,
value: "LIMIT %d",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 43..53,
value: "LIMIT %d",
unicode: false,
},
),
},
},
),
op: Mod,
@ -104,9 +116,15 @@ Call(
left: StringLiteral(
ExprStringLiteral {
range: 91..102,
value: "OFFSET %d",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 91..102,
value: "OFFSET %d",
unicode: false,
},
),
},
},
),
op: Mod,

48
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__match.snap
View file

@ -14,9 +14,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 8..14,
value: "test",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 8..14,
value: "test",
unicode: false,
},
),
},
},
),
),
@ -97,9 +103,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 81..88,
value: "label",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 81..88,
value: "label",
unicode: false,
},
),
},
},
),
),
@ -108,9 +120,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 90..96,
value: "test",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 90..96,
value: "test",
unicode: false,
},
),
},
},
),
],
@ -126,9 +144,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 118..125,
value: "label",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 118..125,
value: "label",
unicode: false,
},
),
},
},
),
],

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_class.snap
View file

@ -116,9 +116,15 @@ expression: "parse_suite(source, \"<test>\").unwrap()"
StringLiteral(
ExprStringLiteral {
range: 80..89,
value: "default",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 80..89,
value: "default",
unicode: false,
},
),
},
},
),
),

34
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_f_string.snap
View file

@ -9,17 +9,31 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..14,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..13,
value: "Hello world",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..14,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..13,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..13,
value: "Hello world",
unicode: false,
},
),
},
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_kwargs.snap
View file

@ -22,9 +22,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 8..20,
value: "positional",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 8..20,
value: "positional",
unicode: false,
},
),
},
},
),
],

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_print_2.snap
View file

@ -22,9 +22,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 6..19,
value: "Hello world",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 6..19,
value: "Hello world",
unicode: false,
},
),
},
},
),
NumberLiteral(

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_print_hello.snap
View file

@ -22,9 +22,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 6..19,
value: "Hello world",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 6..19,
value: "Hello world",
unicode: false,
},
),
},
},
),
],

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_string.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..13,
value: "Hello world",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..13,
value: "Hello world",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_type_declaration.snap
View file

@ -81,9 +81,15 @@ expression: "parse_suite(source, \"<test>\").unwrap()"
right: StringLiteral(
ExprStringLiteral {
range: 48..61,
value: "ForwardRefY",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 48..61,
value: "ForwardRefY",
unicode: false,
},
),
},
},
),
},

84
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__patma.snap
View file

@ -501,9 +501,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 484..489,
value: "seq",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 484..489,
value: "seq",
unicode: false,
},
),
},
},
),
),
@ -530,9 +536,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 518..523,
value: "map",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 518..523,
value: "map",
unicode: false,
},
),
},
},
),
),
@ -821,9 +833,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 664..667,
value: "X",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 664..667,
value: "X",
unicode: false,
},
),
},
},
),
},
@ -1551,9 +1569,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 1287..1292,
value: "foo",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 1287..1292,
value: "foo",
unicode: false,
},
),
},
},
),
],
@ -2469,9 +2493,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 2036..2038,
value: "",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2036..2038,
value: "",
unicode: false,
},
),
},
},
),
},
@ -2513,9 +2543,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 2064..2066,
value: "",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2064..2066,
value: "",
unicode: false,
},
),
},
},
),
},
@ -3131,9 +3167,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 2449..2452,
value: "X",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2449..2452,
value: "X",
unicode: false,
},
),
},
},
),
},

184
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__try.snap
View file

@ -81,50 +81,64 @@ expression: parse_ast
FString(
ExprFString {
range: 62..81,
values: [
StringLiteral(
ExprStringLiteral {
range: 64..71,
value: "caught ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 71..80,
value: Call(
ExprCall {
range: 72..79,
func: Name(
ExprName {
range: 72..76,
id: "type",
ctx: Load,
value: FStringValue {
inner: Single(
FString(
FString {
range: 62..81,
values: [
StringLiteral(
ExprStringLiteral {
range: 64..71,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 64..71,
value: "caught ",
unicode: false,
},
),
},
},
),
arguments: Arguments {
range: 76..79,
args: [
Name(
ExprName {
range: 77..78,
id: "e",
ctx: Load,
FormattedValue(
ExprFormattedValue {
range: 71..80,
value: Call(
ExprCall {
range: 72..79,
func: Name(
ExprName {
range: 72..76,
id: "type",
ctx: Load,
},
),
arguments: Arguments {
range: 76..79,
args: [
Name(
ExprName {
range: 77..78,
id: "e",
ctx: Load,
},
),
],
keywords: [],
},
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
],
@ -175,50 +189,64 @@ expression: parse_ast
FString(
ExprFString {
range: 114..133,
values: [
StringLiteral(
ExprStringLiteral {
range: 116..123,
value: "caught ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 123..132,
value: Call(
ExprCall {
range: 124..131,
func: Name(
ExprName {
range: 124..128,
id: "type",
ctx: Load,
value: FStringValue {
inner: Single(
FString(
FString {
range: 114..133,
values: [
StringLiteral(
ExprStringLiteral {
range: 116..123,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 116..123,
value: "caught ",
unicode: false,
},
),
},
},
),
arguments: Arguments {
range: 128..131,
args: [
Name(
ExprName {
range: 129..130,
id: "e",
ctx: Load,
FormattedValue(
ExprFormattedValue {
range: 123..132,
value: Call(
ExprCall {
range: 124..131,
func: Name(
ExprName {
range: 124..128,
id: "type",
ctx: Load,
},
),
arguments: Arguments {
range: 128..131,
args: [
Name(
ExprName {
range: 129..130,
id: "e",
ctx: Load,
},
),
],
keywords: [],
},
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
],

328
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__try_star.snap
View file

@ -27,9 +27,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 30..34,
value: "eg",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 30..34,
value: "eg",
unicode: false,
},
),
},
},
),
List(
@ -193,83 +199,103 @@ expression: parse_ast
FString(
ExprFString {
range: 133..179,
values: [
StringLiteral(
ExprStringLiteral {
range: 135..142,
value: "caught ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 142..151,
value: Call(
ExprCall {
range: 143..150,
func: Name(
ExprName {
range: 143..147,
id: "type",
ctx: Load,
value: FStringValue {
inner: Single(
FString(
FString {
range: 133..179,
values: [
StringLiteral(
ExprStringLiteral {
range: 135..142,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 135..142,
value: "caught ",
unicode: false,
},
),
},
},
),
arguments: Arguments {
range: 147..150,
args: [
Name(
ExprName {
range: 148..149,
id: "e",
FormattedValue(
ExprFormattedValue {
range: 142..151,
value: Call(
ExprCall {
range: 143..150,
func: Name(
ExprName {
range: 143..147,
id: "type",
ctx: Load,
},
),
arguments: Arguments {
range: 147..150,
args: [
Name(
ExprName {
range: 148..149,
id: "e",
ctx: Load,
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 151..164,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 151..164,
value: " with nested ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 164..178,
value: Attribute(
ExprAttribute {
range: 165..177,
value: Name(
ExprName {
range: 165..166,
id: "e",
ctx: Load,
},
),
attr: Identifier {
id: "exceptions",
range: 167..177,
},
ctx: Load,
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 151..164,
value: " with nested ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 164..178,
value: Attribute(
ExprAttribute {
range: 165..177,
value: Name(
ExprName {
range: 165..166,
id: "e",
ctx: Load,
debug_text: None,
conversion: None,
format_spec: None,
},
),
attr: Identifier {
id: "exceptions",
range: 167..177,
},
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
],
},
),
),
],
implicit_concatenated: false,
},
},
),
],
@ -320,83 +346,103 @@ expression: parse_ast
FString(
ExprFString {
range: 213..259,
values: [
StringLiteral(
ExprStringLiteral {
range: 215..222,
value: "caught ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 222..231,
value: Call(
ExprCall {
range: 223..230,
func: Name(
ExprName {
range: 223..227,
id: "type",
ctx: Load,
value: FStringValue {
inner: Single(
FString(
FString {
range: 213..259,
values: [
StringLiteral(
ExprStringLiteral {
range: 215..222,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 215..222,
value: "caught ",
unicode: false,
},
),
},
},
),
arguments: Arguments {
range: 227..230,
args: [
Name(
ExprName {
range: 228..229,
id: "e",
FormattedValue(
ExprFormattedValue {
range: 222..231,
value: Call(
ExprCall {
range: 223..230,
func: Name(
ExprName {
range: 223..227,
id: "type",
ctx: Load,
},
),
arguments: Arguments {
range: 227..230,
args: [
Name(
ExprName {
range: 228..229,
id: "e",
ctx: Load,
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 231..244,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 231..244,
value: " with nested ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 244..258,
value: Attribute(
ExprAttribute {
range: 245..257,
value: Name(
ExprName {
range: 245..246,
id: "e",
ctx: Load,
},
),
attr: Identifier {
id: "exceptions",
range: 247..257,
},
ctx: Load,
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 231..244,
value: " with nested ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 244..258,
value: Attribute(
ExprAttribute {
range: 245..257,
value: Name(
ExprName {
range: 245..246,
id: "e",
ctx: Load,
debug_text: None,
conversion: None,
format_spec: None,
},
),
attr: Identifier {
id: "exceptions",
range: 247..257,
},
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
],
},
),
),
],
implicit_concatenated: false,
},
},
),
],

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__unicode_aliases.snap
View file

@ -18,9 +18,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 4..37,
value: "\u{8}another cool trick",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 4..37,
value: "\u{8}another cool trick",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__backspace_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..15,
value: "\u{8}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..15,
value: "\u{8}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__bell_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..9,
value: "\u{7}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..9,
value: "\u{7}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__carriage_return_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..21,
value: "\r",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..21,
value: "\r",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__character_tabulation_with_justification_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..45,
value: "\u{89}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..45,
value: "\u{89}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__delete_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..12,
value: "\u{7f}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..12,
value: "\u{7f}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__dont_panic_on_8_in_octal_escape.snap
View file

@ -18,9 +18,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 7..16,
value: "\u{3}8[1m",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 7..16,
value: "\u{3}8[1m",
unicode: false,
},
),
},
},
),
},

524
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__double_quoted_byte.snap
View file

@ -9,265 +9,271 @@ expression: parse_ast
value: BytesLiteral(
ExprBytesLiteral {
range: 0..738,
value: [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63,
64,
65,
66,
67,
68,
69,
70,
71,
72,
73,
74,
75,
76,
77,
78,
79,
80,
81,
82,
83,
84,
85,
86,
87,
88,
89,
90,
91,
92,
93,
94,
95,
96,
97,
98,
99,
100,
101,
102,
103,
104,
105,
106,
107,
108,
109,
110,
111,
112,
113,
114,
115,
116,
117,
118,
119,
120,
121,
122,
123,
124,
125,
126,
127,
128,
129,
130,
131,
132,
133,
134,
135,
136,
137,
138,
139,
140,
141,
142,
143,
144,
145,
146,
147,
148,
149,
150,
151,
152,
153,
154,
155,
156,
157,
158,
159,
160,
161,
162,
163,
164,
165,
166,
167,
168,
169,
170,
171,
172,
173,
174,
175,
176,
177,
178,
179,
180,
181,
182,
183,
184,
185,
186,
187,
188,
189,
190,
191,
192,
193,
194,
195,
196,
197,
198,
199,
200,
201,
202,
203,
204,
205,
206,
207,
208,
209,
210,
211,
212,
213,
214,
215,
216,
217,
218,
219,
220,
221,
222,
223,
224,
225,
226,
227,
228,
229,
230,
231,
232,
233,
234,
235,
236,
237,
238,
239,
240,
241,
242,
243,
244,
245,
246,
247,
248,
249,
250,
251,
252,
253,
254,
255,
],
implicit_concatenated: false,
value: BytesLiteralValue {
inner: Single(
BytesLiteral {
range: 0..738,
value: [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63,
64,
65,
66,
67,
68,
69,
70,
71,
72,
73,
74,
75,
76,
77,
78,
79,
80,
81,
82,
83,
84,
85,
86,
87,
88,
89,
90,
91,
92,
93,
94,
95,
96,
97,
98,
99,
100,
101,
102,
103,
104,
105,
106,
107,
108,
109,
110,
111,
112,
113,
114,
115,
116,
117,
118,
119,
120,
121,
122,
123,
124,
125,
126,
127,
128,
129,
130,
131,
132,
133,
134,
135,
136,
137,
138,
139,
140,
141,
142,
143,
144,
145,
146,
147,
148,
149,
150,
151,
152,
153,
154,
155,
156,
157,
158,
159,
160,
161,
162,
163,
164,
165,
166,
167,
168,
169,
170,
171,
172,
173,
174,
175,
176,
177,
178,
179,
180,
181,
182,
183,
184,
185,
186,
187,
188,
189,
190,
191,
192,
193,
194,
195,
196,
197,
198,
199,
200,
201,
202,
203,
204,
205,
206,
207,
208,
209,
210,
211,
212,
213,
214,
215,
216,
217,
218,
219,
220,
221,
222,
223,
224,
225,
226,
227,
228,
229,
230,
231,
232,
233,
234,
235,
236,
237,
238,
239,
240,
241,
242,
243,
244,
245,
246,
247,
248,
249,
250,
251,
252,
253,
254,
255,
],
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__escape_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..12,
value: "\u{1b}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..12,
value: "\u{1b}",
unicode: false,
},
),
},
},
),
},

32
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__escape_char_in_byte_literal.snap
View file

@ -9,19 +9,25 @@ expression: parse_ast
value: BytesLiteral(
ExprBytesLiteral {
range: 0..13,
value: [
111,
109,
107,
109,
111,
107,
92,
88,
97,
97,
],
implicit_concatenated: false,
value: BytesLiteralValue {
inner: Single(
BytesLiteral {
range: 0..13,
value: [
111,
109,
107,
109,
111,
107,
92,
88,
97,
97,
],
},
),
},
},
),
},

22
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__escape_octet.snap
View file

@ -9,14 +9,20 @@ expression: parse_ast
value: BytesLiteral(
ExprBytesLiteral {
range: 0..14,
value: [
35,
97,
4,
83,
52,
],
implicit_concatenated: false,
value: BytesLiteralValue {
inner: Single(
BytesLiteral {
range: 0..14,
value: [
35,
97,
4,
83,
52,
],
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__form_feed_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..15,
value: "\u{c}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..15,
value: "\u{c}",
unicode: false,
},
),
},
},
),
},

138
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_constant_range.snap
View file

@ -9,63 +9,89 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..5,
value: "aaa",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..5,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..5,
value: "aaa",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 5..10,
value: Name(
ExprName {
range: 6..9,
id: "bbb",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 10..13,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 10..13,
value: "ccc",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 13..18,
value: Name(
ExprName {
range: 14..17,
id: "ddd",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 18..21,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 18..21,
value: "eee",
unicode: false,
},
),
},
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 5..10,
value: Name(
ExprName {
range: 6..9,
id: "bbb",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 10..13,
value: "ccc",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 13..18,
value: Name(
ExprName {
range: 14..17,
id: "ddd",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 18..21,
value: "eee",
unicode: false,
implicit_concatenated: false,
},
),
],
implicit_concatenated: false,
},
},
),
},

64
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_escaped_character.snap
View file

@ -9,32 +9,46 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..8,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..4,
value: "\\",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..8,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..4,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..4,
value: "\\",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 4..7,
value: Name(
ExprName {
range: 5..6,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 4..7,
value: Name(
ExprName {
range: 5..6,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

64
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_escaped_newline.snap
View file

@ -9,32 +9,46 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..8,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..4,
value: "\n",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..8,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..4,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..4,
value: "\n",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 4..7,
value: Name(
ExprName {
range: 5..6,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 4..7,
value: Name(
ExprName {
range: 5..6,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

64
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_line_continuation.snap
View file

@ -9,32 +9,46 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..9,
values: [
StringLiteral(
ExprStringLiteral {
range: 3..5,
value: "\\\n",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..9,
values: [
StringLiteral(
ExprStringLiteral {
range: 3..5,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 3..5,
value: "\\\n",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

52
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_parse_self_documenting_base.snap
View file

@ -9,29 +9,37 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Name(
ExprName {
range: 3..7,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Name(
ExprName {
range: 3..7,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

136
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_parse_self_documenting_base_more.snap
View file

@ -9,65 +9,85 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..38,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..6,
value: "mix ",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..38,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..6,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..6,
value: "mix ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 6..13,
value: Name(
ExprName {
range: 7..11,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 13..28,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 13..28,
value: " with text and ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 28..37,
value: Name(
ExprName {
range: 29..35,
id: "second",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 6..13,
value: Name(
ExprName {
range: 7..11,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 13..28,
value: " with text and ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 28..37,
value: Name(
ExprName {
range: 29..35,
id: "second",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

96
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_parse_self_documenting_format.snap
View file

@ -9,46 +9,68 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..14,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..13,
value: Name(
ExprName {
range: 3..7,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 9..12,
values: [
StringLiteral(
ExprStringLiteral {
range: 9..12,
value: ">10",
unicode: false,
implicit_concatenated: false,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..14,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..13,
value: Name(
ExprName {
range: 3..7,
id: "user",
ctx: Load,
},
),
],
implicit_concatenated: false,
},
),
),
},
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 9..12,
value: FStringValue {
inner: Single(
FString(
FString {
range: 9..12,
values: [
StringLiteral(
ExprStringLiteral {
range: 9..12,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 9..12,
value: ">10",
unicode: false,
},
),
},
},
),
],
},
),
),
},
},
),
),
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

64
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_unescaped_newline.snap
View file

@ -9,32 +9,46 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..11,
values: [
StringLiteral(
ExprStringLiteral {
range: 4..5,
value: "\n",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..11,
values: [
StringLiteral(
ExprStringLiteral {
range: 4..5,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 4..5,
value: "\n",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__hts_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..9,
value: "\u{88}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..9,
value: "\u{88}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_empty_fstring.snap
View file

@ -9,8 +9,16 @@ expression: "parse_suite(r#\"f\"\"\"#, \"<test>\").unwrap()"
value: FString(
ExprFString {
range: 0..3,
values: [],
implicit_concatenated: false,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..3,
values: [],
},
),
),
},
},
),
},

43
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_f_string_concat_1.snap
View file

@ -9,17 +9,40 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..17,
values: [
StringLiteral(
ExprStringLiteral {
range: 1..16,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
},
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
},
),
FString(
FString {
range: 9..17,
values: [
StringLiteral(
ExprStringLiteral {
range: 11..16,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 11..16,
value: "world",
unicode: false,
},
),
},
},
),
],
},
),
],
),
],
implicit_concatenated: true,
},
},
),
},

43
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_f_string_concat_2.snap
View file

@ -9,17 +9,40 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..17,
values: [
StringLiteral(
ExprStringLiteral {
range: 1..16,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
},
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
},
),
FString(
FString {
range: 9..17,
values: [
StringLiteral(
ExprStringLiteral {
range: 11..16,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 11..16,
value: "world",
unicode: false,
},
),
},
},
),
],
},
),
],
),
],
implicit_concatenated: true,
},
},
),
},

75
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_f_string_concat_3.snap
View file

@ -9,33 +9,62 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 1..16,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 16..21,
value: StringLiteral(
ExprStringLiteral {
range: 17..20,
value: "!",
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
implicit_concatenated: false,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 9..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 11..16,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 11..16,
value: "world",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 16..21,
value: StringLiteral(
ExprStringLiteral {
range: 17..20,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 17..20,
value: "!",
unicode: false,
},
),
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
],
),
],
implicit_concatenated: true,
},
},
),
},

90
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_f_string_concat_4.snap
View file

@ -9,41 +9,69 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..31,
values: [
StringLiteral(
ExprStringLiteral {
range: 1..16,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 16..21,
value: StringLiteral(
ExprStringLiteral {
range: 17..20,
value: "!",
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
implicit_concatenated: false,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 9..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 11..16,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 11..16,
value: "world",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 16..21,
value: StringLiteral(
ExprStringLiteral {
range: 17..20,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 17..20,
value: "!",
unicode: false,
},
),
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
Literal(
StringLiteral {
range: 23..31,
value: "again!",
unicode: false,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 24..30,
value: "again!",
unicode: false,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},

94
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring.snap
View file

@ -9,47 +9,61 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..18,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..5,
value: Name(
ExprName {
range: 3..4,
id: "a",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..18,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..5,
value: Name(
ExprName {
range: 3..4,
id: "a",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
FormattedValue(
ExprFormattedValue {
range: 5..10,
value: Name(
ExprName {
range: 7..8,
id: "b",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 10..17,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 10..17,
value: "{foo}",
unicode: false,
},
),
},
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 5..10,
value: Name(
ExprName {
range: 7..8,
id: "b",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 10..17,
value: "{foo}",
unicode: false,
implicit_concatenated: false,
},
),
],
implicit_concatenated: false,
},
},
),
},

74
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_equals.snap
View file

@ -9,43 +9,51 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..12,
value: Compare(
ExprCompare {
range: 3..11,
left: NumberLiteral(
ExprNumberLiteral {
range: 3..5,
value: Int(
42,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..12,
value: Compare(
ExprCompare {
range: 3..11,
left: NumberLiteral(
ExprNumberLiteral {
range: 3..5,
value: Int(
42,
),
},
),
ops: [
Eq,
],
comparators: [
NumberLiteral(
ExprNumberLiteral {
range: 9..11,
value: Int(
42,
),
},
),
],
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
ops: [
Eq,
],
comparators: [
NumberLiteral(
ExprNumberLiteral {
range: 9..11,
value: Int(
42,
),
},
),
],
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

112
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_nested_concatenation_string_spec.snap
View file

@ -9,49 +9,81 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..16,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..15,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..14,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..14,
value: StringLiteral(
ExprStringLiteral {
range: 8..13,
value: "",
unicode: false,
implicit_concatenated: true,
},
),
debug_text: None,
conversion: None,
format_spec: None,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..16,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..15,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
],
implicit_concatenated: false,
},
),
),
},
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..14,
value: FStringValue {
inner: Single(
FString(
FString {
range: 7..14,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..14,
value: StringLiteral(
ExprStringLiteral {
range: 8..13,
value: StringLiteralValue {
inner: Concatenated(
ConcatenatedStringLiteral {
strings: [
StringLiteral {
range: 8..10,
value: "",
unicode: false,
},
StringLiteral {
range: 11..13,
value: "",
unicode: false,
},
],
value: "",
},
),
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
},
},
),
),
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

94
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_nested_spec.snap
View file

@ -9,48 +9,64 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..15,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..14,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..13,
value: Name(
ExprName {
range: 8..12,
id: "spec",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..15,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..14,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
],
implicit_concatenated: false,
},
),
),
},
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..13,
value: FStringValue {
inner: Single(
FString(
FString {
range: 7..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..13,
value: Name(
ExprName {
range: 8..12,
id: "spec",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
},
},
),
),
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

102
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_nested_string_spec.snap
View file

@ -9,49 +9,71 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..12,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..11,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..11,
value: StringLiteral(
ExprStringLiteral {
range: 8..10,
value: "",
unicode: false,
implicit_concatenated: false,
},
),
debug_text: None,
conversion: None,
format_spec: None,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..12,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
],
implicit_concatenated: false,
},
),
),
},
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..11,
value: FStringValue {
inner: Single(
FString(
FString {
range: 7..11,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..11,
value: StringLiteral(
ExprStringLiteral {
range: 8..10,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 8..10,
value: "",
unicode: false,
},
),
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
},
},
),
),
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

74
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_not_equals.snap
View file

@ -9,43 +9,51 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..11,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..10,
value: Compare(
ExprCompare {
range: 3..9,
left: NumberLiteral(
ExprNumberLiteral {
range: 3..4,
value: Int(
1,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..11,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..10,
value: Compare(
ExprCompare {
range: 3..9,
left: NumberLiteral(
ExprNumberLiteral {
range: 3..4,
value: Int(
1,
),
},
),
ops: [
NotEq,
],
comparators: [
NumberLiteral(
ExprNumberLiteral {
range: 8..9,
value: Int(
2,
),
},
),
],
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
ops: [
NotEq,
],
comparators: [
NumberLiteral(
ExprNumberLiteral {
range: 8..9,
value: Int(
2,
),
},
),
],
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

86
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_not_nested_spec.snap
View file

@ -9,41 +9,63 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..12,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..11,
values: [
StringLiteral(
ExprStringLiteral {
range: 7..11,
value: "spec",
unicode: false,
implicit_concatenated: false,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..12,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
],
implicit_concatenated: false,
},
),
),
},
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..11,
value: FStringValue {
inner: Single(
FString(
FString {
range: 7..11,
values: [
StringLiteral(
ExprStringLiteral {
range: 7..11,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 7..11,
value: "spec",
unicode: false,
},
),
},
},
),
],
},
),
),
},
},
),
),
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

52
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_self_doc_prec_space.snap
View file

@ -9,29 +9,37 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Name(
ExprName {
range: 3..4,
id: "x",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: " =",
},
),
conversion: None,
format_spec: None,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Name(
ExprName {
range: 3..4,
id: "x",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: " =",
},
),
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

52
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_self_doc_trailing_space.snap
View file

@ -9,29 +9,37 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Name(
ExprName {
range: 3..4,
id: "x",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "= ",
},
),
conversion: None,
format_spec: None,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Name(
ExprName {
range: 3..4,
id: "x",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "= ",
},
),
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

40
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_yield_expr.snap
View file

@ -9,23 +9,31 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Yield(
ExprYield {
range: 3..8,
value: None,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Yield(
ExprYield {
range: 3..8,
value: None,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

22
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_string_concat.snap
View file

@ -9,9 +9,25 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..16,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
value: StringLiteralValue {
inner: Concatenated(
ConcatenatedStringLiteral {
strings: [
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
},
StringLiteral {
range: 9..16,
value: "world",
unicode: false,
},
],
value: "Hello world",
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_string_triple_quotes_with_kind.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..20,
value: "Hello, world!",
unicode: true,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..20,
value: "Hello, world!",
unicode: true,
},
),
},
},
),
},

43
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_u_f_string_concat_1.snap
View file

@ -9,17 +9,40 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..18,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..17,
value: "Hello world",
unicode: true,
implicit_concatenated: true,
},
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..9,
value: "Hello ",
unicode: true,
},
),
FString(
FString {
range: 10..18,
values: [
StringLiteral(
ExprStringLiteral {
range: 12..17,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 12..17,
value: "world",
unicode: false,
},
),
},
},
),
],
},
),
],
),
],
implicit_concatenated: true,
},
},
),
},

50
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_u_f_string_concat_2.snap
View file

@ -9,17 +9,47 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..21,
value: "Hello world!",
unicode: true,
implicit_concatenated: true,
},
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..9,
value: "Hello ",
unicode: true,
},
),
FString(
FString {
range: 10..18,
values: [
StringLiteral(
ExprStringLiteral {
range: 12..17,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 12..17,
value: "world",
unicode: false,
},
),
},
},
),
],
},
),
Literal(
StringLiteral {
range: 19..22,
value: "!",
unicode: false,
},
),
],
),
],
implicit_concatenated: true,
},
},
),
},

22
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_u_string_concat_1.snap
View file

@ -9,9 +9,25 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..17,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
value: StringLiteralValue {
inner: Concatenated(
ConcatenatedStringLiteral {
strings: [
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
},
StringLiteral {
range: 9..17,
value: "world",
unicode: true,
},
],
value: "Hello world",
},
),
},
},
),
},

22
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_u_string_concat_2.snap
View file

@ -9,9 +9,25 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..17,
value: "Hello world",
unicode: true,
implicit_concatenated: true,
value: StringLiteralValue {
inner: Concatenated(
ConcatenatedStringLiteral {
strings: [
StringLiteral {
range: 0..9,
value: "Hello ",
unicode: true,
},
StringLiteral {
range: 10..17,
value: "world",
unicode: false,
},
],
value: "Hello world",
},
),
},
},
),
},

20
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__raw_byte_literal_1.snap
View file

@ -9,13 +9,19 @@ expression: parse_ast
value: BytesLiteral(
ExprBytesLiteral {
range: 0..8,
value: [
92,
120,
49,
122,
],
implicit_concatenated: false,
value: BytesLiteralValue {
inner: Single(
BytesLiteral {
range: 0..8,
value: [
92,
120,
49,
122,
],
},
),
},
},
),
},

16
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__raw_byte_literal_2.snap
View file

@ -9,11 +9,17 @@ expression: parse_ast
value: BytesLiteral(
ExprBytesLiteral {
range: 0..6,
value: [
92,
92,
],
implicit_concatenated: false,
value: BytesLiteralValue {
inner: Single(
BytesLiteral {
range: 0..6,
value: [
92,
92,
],
},
),
},
},
),
},

42
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__raw_fstring.snap
View file

@ -9,24 +9,32 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..7,
values: [
FormattedValue(
ExprFormattedValue {
range: 3..6,
value: Name(
ExprName {
range: 4..5,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..7,
values: [
FormattedValue(
ExprFormattedValue {
range: 3..6,
value: Name(
ExprName {
range: 4..5,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

524
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__single_quoted_byte.snap
View file

@ -9,265 +9,271 @@ expression: parse_ast
value: BytesLiteral(
ExprBytesLiteral {
range: 0..738,
value: [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63,
64,
65,
66,
67,
68,
69,
70,
71,
72,
73,
74,
75,
76,
77,
78,
79,
80,
81,
82,
83,
84,
85,
86,
87,
88,
89,
90,
91,
92,
93,
94,
95,
96,
97,
98,
99,
100,
101,
102,
103,
104,
105,
106,
107,
108,
109,
110,
111,
112,
113,
114,
115,
116,
117,
118,
119,
120,
121,
122,
123,
124,
125,
126,
127,
128,
129,
130,
131,
132,
133,
134,
135,
136,
137,
138,
139,
140,
141,
142,
143,
144,
145,
146,
147,
148,
149,
150,
151,
152,
153,
154,
155,
156,
157,
158,
159,
160,
161,
162,
163,
164,
165,
166,
167,
168,
169,
170,
171,
172,
173,
174,
175,
176,
177,
178,
179,
180,
181,
182,
183,
184,
185,
186,
187,
188,
189,
190,
191,
192,
193,
194,
195,
196,
197,
198,
199,
200,
201,
202,
203,
204,
205,
206,
207,
208,
209,
210,
211,
212,
213,
214,
215,
216,
217,
218,
219,
220,
221,
222,
223,
224,
225,
226,
227,
228,
229,
230,
231,
232,
233,
234,
235,
236,
237,
238,
239,
240,
241,
242,
243,
244,
245,
246,
247,
248,
249,
250,
251,
252,
253,
254,
255,
],
implicit_concatenated: false,
value: BytesLiteralValue {
inner: Single(
BytesLiteral {
range: 0..738,
value: [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63,
64,
65,
66,
67,
68,
69,
70,
71,
72,
73,
74,
75,
76,
77,
78,
79,
80,
81,
82,
83,
84,
85,
86,
87,
88,
89,
90,
91,
92,
93,
94,
95,
96,
97,
98,
99,
100,
101,
102,
103,
104,
105,
106,
107,
108,
109,
110,
111,
112,
113,
114,
115,
116,
117,
118,
119,
120,
121,
122,
123,
124,
125,
126,
127,
128,
129,
130,
131,
132,
133,
134,
135,
136,
137,
138,
139,
140,
141,
142,
143,
144,
145,
146,
147,
148,
149,
150,
151,
152,
153,
154,
155,
156,
157,
158,
159,
160,
161,
162,
163,
164,
165,
166,
167,
168,
169,
170,
171,
172,
173,
174,
175,
176,
177,
178,
179,
180,
181,
182,
183,
184,
185,
186,
187,
188,
189,
190,
191,
192,
193,
194,
195,
196,
197,
198,
199,
200,
201,
202,
203,
204,
205,
206,
207,
208,
209,
210,
211,
212,
213,
214,
215,
216,
217,
218,
219,
220,
221,
222,
223,
224,
225,
226,
227,
228,
229,
230,
231,
232,
233,
234,
235,
236,
237,
238,
239,
240,
241,
242,
243,
244,
245,
246,
247,
248,
249,
250,
251,
252,
253,
254,
255,
],
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__string_parser_escaped_mac_eol.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..18,
value: "text more text",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..18,
value: "text more text",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__string_parser_escaped_unix_eol.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..18,
value: "text more text",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..18,
value: "text more text",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__string_parser_escaped_windows_eol.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..19,
value: "text more text",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..19,
value: "text more text",
unicode: false,
},
),
},
},
),
},

42
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__triple_quoted_raw_fstring.snap
View file

@ -9,24 +9,32 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..11,
values: [
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..11,
values: [
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

155
crates/ruff_python_parser/src/string.rs
View file

@ -7,9 +7,9 @@ use crate::lexer::{LexicalError, LexicalErrorType};
use crate::token::{StringKind, Tok};
pub(crate) enum StringType {
Str(ast::ExprStringLiteral),
Bytes(ast::ExprBytesLiteral),
FString(ast::ExprFString),
Str(ast::StringLiteral),
Bytes(ast::BytesLiteral),
FString(ast::FString),
}
impl Ranged for StringType {
@ -22,11 +22,12 @@ impl Ranged for StringType {
}
}
impl StringType {
fn is_unicode(&self) -> bool {
match self {
Self::Str(ast::ExprStringLiteral { unicode, .. }) => *unicode,
_ => false,
impl From<StringType> for Expr {
fn from(string: StringType) -> Self {
match string {
StringType::Str(node) => Expr::from(node),
StringType::Bytes(node) => Expr::from(node),
StringType::FString(node) => Expr::from(node),
}
}
}
@ -35,14 +36,16 @@ struct StringParser<'a> {
rest: &'a str,
kind: StringKind,
location: TextSize,
range: TextRange,
}
impl<'a> StringParser<'a> {
fn new(source: &'a str, kind: StringKind, start: TextSize) -> Self {
fn new(source: &'a str, kind: StringKind, start: TextSize, range: TextRange) -> Self {
Self {
rest: source,
kind,
location: start,
range,
}
}
@ -59,11 +62,6 @@ impl<'a> StringParser<'a> {
self.location
}
#[inline]
fn range(&self, start_location: TextSize) -> TextRange {
TextRange::new(start_location, self.location)
}
/// Returns the next byte in the string, if there is one.
///
/// # Panics
@ -208,7 +206,6 @@ impl<'a> StringParser<'a> {
fn parse_fstring_middle(&mut self) -> Result<Expr, LexicalError> {
let mut value = String::new();
let start_location = self.get_pos();
while let Some(ch) = self.next_char() {
match ch {
// We can encounter a `\` as the last character in a `FStringMiddle`
@ -244,17 +241,15 @@ impl<'a> StringParser<'a> {
ch => value.push(ch),
}
}
Ok(Expr::from(ast::ExprStringLiteral {
Ok(Expr::from(ast::StringLiteral {
value,
unicode: false,
implicit_concatenated: false,
range: self.range(start_location),
range: self.range,
}))
}
fn parse_bytes(&mut self) -> Result<StringType, LexicalError> {
let mut content = String::new();
let start_location = self.get_pos();
while let Some(ch) = self.next_char() {
match ch {
'\\' if !self.kind.is_raw() => {
@ -274,15 +269,13 @@ impl<'a> StringParser<'a> {
}
}
Ok(StringType::Bytes(ast::ExprBytesLiteral {
Ok(StringType::Bytes(ast::BytesLiteral {
value: content.chars().map(|c| c as u8).collect::<Vec<u8>>(),
implicit_concatenated: false,
range: self.range(start_location),
range: self.range,
}))
}
fn parse_string(&mut self) -> Result<StringType, LexicalError> {
let start_location = self.get_pos();
let mut value = String::new();
if self.kind.is_raw() {
@ -301,11 +294,10 @@ impl<'a> StringParser<'a> {
self.parse_escaped_char(&mut value)?;
}
}
Ok(StringType::Str(ast::ExprStringLiteral {
Ok(StringType::Str(ast::StringLiteral {
value,
unicode: self.kind.is_unicode(),
implicit_concatenated: false,
range: self.range(start_location),
range: self.range,
}))
}
@ -322,38 +314,37 @@ pub(crate) fn parse_string_literal(
source: &str,
kind: StringKind,
triple_quoted: bool,
start_location: TextSize,
range: TextRange,
) -> Result<StringType, LexicalError> {
let start_location = start_location
let start_location = range.start()
+ kind.prefix_len()
+ if triple_quoted {
TextSize::from(3)
} else {
TextSize::from(1)
};
StringParser::new(source, kind, start_location).parse()
StringParser::new(source, kind, start_location, range).parse()
}
pub(crate) fn parse_fstring_middle(
source: &str,
is_raw: bool,
start_location: TextSize,
range: TextRange,
) -> Result<Expr, LexicalError> {
let kind = if is_raw {
StringKind::RawString
} else {
StringKind::String
};
StringParser::new(source, kind, start_location).parse_fstring_middle()
StringParser::new(source, kind, range.start(), range).parse_fstring_middle()
}
/// Concatenate a list of string literals into a single string expression.
pub(crate) fn concatenate_strings(
pub(crate) fn concatenated_strings(
strings: Vec<StringType>,
range: TextRange,
) -> Result<Expr, LexicalError> {
#[cfg(debug_assertions)]
debug_assert!(!strings.is_empty());
debug_assert!(strings.len() > 1);
let mut has_fstring = false;
let mut byte_literal_count = 0;
@ -365,7 +356,6 @@ pub(crate) fn concatenate_strings(
}
}
let has_bytes = byte_literal_count > 0;
let implicit_concatenated = strings.len() > 1;
if has_bytes && byte_literal_count < strings.len() {
return Err(LexicalError {
@ -377,111 +367,44 @@ pub(crate) fn concatenate_strings(
}
if has_bytes {
let mut content: Vec<u8> = vec![];
let mut values = Vec::with_capacity(strings.len());
for string in strings {
match string {
StringType::Bytes(ast::ExprBytesLiteral { value, .. }) => content.extend(value),
StringType::Bytes(value) => values.push(value),
_ => unreachable!("Unexpected non-bytes literal."),
}
}
return Ok(ast::ExprBytesLiteral {
value: content,
implicit_concatenated,
return Ok(Expr::from(ast::ExprBytesLiteral {
value: ast::BytesLiteralValue::concatenated(values),
range,
}
.into());
}));
}
if !has_fstring {
let mut content = String::new();
let is_unicode = strings.first().map_or(false, StringType::is_unicode);
let mut values = Vec::with_capacity(strings.len());
for string in strings {
match string {
StringType::Str(ast::ExprStringLiteral { value, .. }) => content.push_str(&value),
StringType::Str(value) => values.push(value),
_ => unreachable!("Unexpected non-string literal."),
}
}
return Ok(ast::ExprStringLiteral {
value: content,
unicode: is_unicode,
implicit_concatenated,
return Ok(Expr::from(ast::ExprStringLiteral {
value: ast::StringLiteralValue::concatenated(values),
range,
}
.into());
}));
}
// De-duplicate adjacent constants.
let mut deduped: Vec<Expr> = vec![];
let mut current = String::new();
let mut current_start = range.start();
let mut current_end = range.end();
let mut is_unicode = false;
let take_current = |current: &mut String, start, end, unicode| -> Expr {
Expr::StringLiteral(ast::ExprStringLiteral {
value: std::mem::take(current),
unicode,
implicit_concatenated,
range: TextRange::new(start, end),
})
};
let mut parts = Vec::with_capacity(strings.len());
for string in strings {
let string_range = string.range();
match string {
StringType::FString(ast::ExprFString { values, .. }) => {
for value in values {
let value_range = value.range();
match value {
Expr::FormattedValue { .. } => {
if !current.is_empty() {
deduped.push(take_current(
&mut current,
current_start,
current_end,
is_unicode,
));
}
deduped.push(value);
is_unicode = false;
}
Expr::StringLiteral(ast::ExprStringLiteral { value, unicode, .. }) => {
if current.is_empty() {
is_unicode |= unicode;
current_start = value_range.start();
}
current_end = value_range.end();
current.push_str(&value);
}
_ => {
unreachable!("Expected `Expr::FormattedValue` or `Expr::StringLiteral`")
}
}
}
}
StringType::Str(ast::ExprStringLiteral { value, unicode, .. }) => {
if current.is_empty() {
is_unicode |= unicode;
current_start = string_range.start();
}
current_end = string_range.end();
current.push_str(&value);
}
StringType::FString(fstring) => parts.push(ast::FStringPart::FString(fstring)),
StringType::Str(string) => parts.push(ast::FStringPart::Literal(string)),
StringType::Bytes(_) => unreachable!("Unexpected bytes literal."),
}
}
if !current.is_empty() {
deduped.push(take_current(
&mut current,
current_start,
current_end,
is_unicode,
));
}
Ok(ast::ExprFString {
values: deduped,
implicit_concatenated,
value: ast::FStringValue::concatenated(parts),
range,
}
.into())