mirrors/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-08-04 02:38:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Update string nodes for implicit concatenation (#7927)

Browse source 
## Summary

This PR updates the string nodes (`ExprStringLiteral`,
`ExprBytesLiteral`, and `ExprFString`) to account for implicit string
concatenation.

### Motivation

In Python, implicit string concatenation are joined while parsing
because the interpreter doesn't require the information for each part.
While that's feasible for an interpreter, it falls short for a static
analysis tool where having such information is more useful. Currently,
various parts of the code uses the lexer to get the individual string
parts.

One of the main challenge this solves is that of string formatting.
Currently, the formatter relies on the lexer to get the individual
string parts, and formats them including the comments accordingly. But,
with PEP 701, f-string can also contain comments. Without this change,
it becomes very difficult to add support for f-string formatting.

### Implementation

The initial proposal was made in this discussion:
https://github.com/astral-sh/ruff/discussions/6183#discussioncomment-6591993.
There were various AST designs which were explored for this task which
are available in the linked internal document[^1].

The selected variant was the one where the nodes were kept as it is
except that the `implicit_concatenated` field was removed and instead a
new struct was added to the `Expr*` struct. This would be a private
struct would contain the actual implementation of how the AST is
designed for both single and implicitly concatenated strings.

This implementation is achieved through an enum with two variants:
`Single` and `Concatenated` to avoid allocating a vector even for single
strings. There are various public methods available on the value struct
to query certain information regarding the node.

The nodes are structured in the following way:

```
ExprStringLiteral - "foo" "bar"
|- StringLiteral - "foo"
|- StringLiteral - "bar"

ExprBytesLiteral - b"foo" b"bar"
|- BytesLiteral - b"foo"
|- BytesLiteral - b"bar"

ExprFString - "foo" f"bar {x}"
|- FStringPart::Literal - "foo"
|- FStringPart::FString - f"bar {x}"
  |- StringLiteral - "bar "
  |- FormattedValue - "x"
```

[^1]: Internal document:
https://www.notion.so/astral-sh/Implicit-String-Concatenation-e036345dc48943f89e416c087bf6f6d9?pvs=4

#### Visitor

The way the nodes are structured is that the entire string, including
all the parts that are implicitly concatenation, is a single node
containing individual nodes for the parts. The previous section has a
representation of that tree for all the string nodes. This means that
new visitor methods are added to visit the individual parts of string,
bytes, and f-strings for `Visitor`, `PreorderVisitor`, and
`Transformer`.

## Test Plan

- `cargo insta test --workspace --all-features --unreferenced reject`
- Verify that the ecosystem results are unchanged
This commit is contained in:
Dhruv Manilawala 2023-11-24 17:55:41 -06:00 • committed by GitHub
parent 2590aa30ae
commit 017e829115
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
121 changed files with 27666 additions and 25501 deletions
Download patch file Download diff file Expand all files Collapse all files

16
crates/ruff_linter/src/checkers/ast/analyze/definitions.rs
View file

@ -158,21 +158,23 @@ pub(crate) fn definitions(checker: &mut Checker) {
}
// Extract a `Docstring` from a `Definition`.
let Some(expr) = docstring else {
let Some(string_literal) = docstring else {
pydocstyle::rules::not_missing(checker, definition, *visibility);
continue;
};
let contents = checker.locator().slice(expr);
let contents = checker.locator().slice(string_literal);
let indentation = checker.locator().slice(TextRange::new(
checker.locator.line_start(expr.start()),
expr.start(),
checker.locator.line_start(string_literal.start()),
string_literal.start(),
));
if expr.implicit_concatenated {
if string_literal.value.is_implicit_concatenated() {
#[allow(deprecated)]
let location = checker.locator.compute_source_location(expr.start());
let location = checker
.locator
.compute_source_location(string_literal.start());
warn_user!(
"Docstring at {}:{}:{} contains implicit string concatenation; ignoring...",
relativize_path(checker.path),
@ -186,7 +188,7 @@ pub(crate) fn definitions(checker: &mut Checker) {
let body_range = raw_contents_range(contents).unwrap();
let docstring = Docstring {
definition,
expr,
expr: string_literal,
contents,
body_range,
indentation,

20
crates/ruff_linter/src/checkers/ast/analyze/expression.rs
View file

@ -988,15 +988,22 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
pylint::rules::await_outside_async(checker, expr);
}
}
Expr::FString(fstring @ ast::ExprFString { values, .. }) => {
Expr::FString(f_string_expr @ ast::ExprFString { value, .. }) => {
if checker.enabled(Rule::FStringMissingPlaceholders) {
pyflakes::rules::f_string_missing_placeholders(fstring, checker);
pyflakes::rules::f_string_missing_placeholders(checker, f_string_expr);
}
if checker.enabled(Rule::ExplicitFStringTypeConversion) {
for f_string in value.f_strings() {
ruff::rules::explicit_f_string_type_conversion(checker, f_string);
}
}
if checker.enabled(Rule::HardcodedSQLExpression) {
flake8_bandit::rules::hardcoded_sql_expression(checker, expr);
}
if checker.enabled(Rule::ExplicitFStringTypeConversion) {
ruff::rules::explicit_f_string_type_conversion(checker, expr, values);
if checker.enabled(Rule::UnicodeKindPrefix) {
for string_literal in value.literals() {
pyupgrade::rules::unicode_kind_prefix(checker, string_literal);
}
}
}
Expr::BinOp(ast::ExprBinOp {
@ -1278,6 +1285,11 @@ pub(crate) fn expression(expr: &Expr, checker: &mut Checker) {
if checker.enabled(Rule::HardcodedTempFile) {
flake8_bandit::rules::hardcoded_tmp_directory(checker, string);
}
if checker.enabled(Rule::UnicodeKindPrefix) {
for string_part in string.value.parts() {
pyupgrade::rules::unicode_kind_prefix(checker, string_part);
}
}
if checker.source_type.is_stub() {
if checker.enabled(Rule::StringOrBytesTooLong) {
flake8_pyi::rules::string_or_bytes_too_long(checker, expr);

6
crates/ruff_linter/src/checkers/ast/mod.rs
View file

@ -1303,9 +1303,9 @@ where
fn visit_format_spec(&mut self, format_spec: &'b Expr) {
match format_spec {
Expr::FString(ast::ExprFString { values, .. }) => {
for value in values {
self.visit_expr(value);
Expr::FString(ast::ExprFString { value, .. }) => {
for expr in value.elements() {
self.visit_expr(expr);
}
}
_ => unreachable!("Unexpected expression for format_spec"),

4
crates/ruff_linter/src/checkers/tokens.rs
View file

@ -94,10 +94,6 @@ pub(crate) fn check_tokens(
pycodestyle::rules::tab_indentation(&mut diagnostics, tokens, locator, indexer);
}
if settings.rules.enabled(Rule::UnicodeKindPrefix) {
pyupgrade::rules::unicode_kind_prefix(&mut diagnostics, tokens);
}
if settings.rules.any_enabled(&[
Rule::InvalidCharacterBackspace,
Rule::InvalidCharacterSub,

29
crates/ruff_linter/src/cst/matchers.rs
View file

@ -1,9 +1,10 @@
use crate::fix::codemods::CodegenStylist;
use anyhow::{bail, Result};
use libcst_native::{
Arg, Attribute, Call, Comparison, CompoundStatement, Dict, Expression, FunctionDef,
GeneratorExp, If, Import, ImportAlias, ImportFrom, ImportNames, IndentedBlock, Lambda,
ListComp, Module, Name, SmallStatement, Statement, Suite, Tuple, With,
Arg, Attribute, Call, Comparison, CompoundStatement, Dict, Expression, FormattedString,
FormattedStringContent, FormattedStringExpression, FunctionDef, GeneratorExp, If, Import,
ImportAlias, ImportFrom, ImportNames, IndentedBlock, Lambda, ListComp, Module, Name,
SmallStatement, Statement, Suite, Tuple, With,
};
use ruff_python_codegen::Stylist;
@ -153,6 +154,28 @@ pub(crate) fn match_lambda<'a, 'b>(expression: &'a Expression<'b>) -> Result<&'a
}
}
pub(crate) fn match_formatted_string<'a, 'b>(
expression: &'a mut Expression<'b>,
) -> Result<&'a mut FormattedString<'b>> {
if let Expression::FormattedString(formatted_string) = expression {
Ok(formatted_string)
} else {
bail!("Expected Expression::FormattedString");
}
}
pub(crate) fn match_formatted_string_expression<'a, 'b>(
formatted_string_content: &'a mut FormattedStringContent<'b>,
) -> Result<&'a mut FormattedStringExpression<'b>> {
if let FormattedStringContent::Expression(formatted_string_expression) =
formatted_string_content
{
Ok(formatted_string_expression)
} else {
bail!("Expected FormattedStringContent::Expression")
}
}
pub(crate) fn match_function_def<'a, 'b>(
statement: &'a mut Statement<'b>,
) -> Result<&'a mut FunctionDef<'b>> {

1
crates/ruff_linter/src/registry.rs
View file

@ -297,7 +297,6 @@ impl Rule {
| Rule::TabIndentation
| Rule::TrailingCommaOnBareTuple
| Rule::TypeCommentInStub
| Rule::UnicodeKindPrefix
| Rule::UselessSemicolon
| Rule::UTF8EncodingDeclaration => LintSource::Tokens,
Rule::IOError => LintSource::Io,

2
crates/ruff_linter/src/rules/airflow/rules/task_variable_name.rs
View file

@ -81,7 +81,7 @@ pub(crate) fn variable_name_task_id(
let ast::ExprStringLiteral { value: task_id, .. } = keyword.value.as_string_literal_expr()?;
// If the target name is the same as the task_id, no violation.
if id == task_id {
if task_id == id {
return None;
}

1
crates/ruff_linter/src/rules/flake8_annotations/rules/definition.rs
View file

@ -508,7 +508,6 @@ fn check_dynamically_typed<F>(
if let Expr::StringLiteral(ast::ExprStringLiteral {
range,
value: string,
..
}) = annotation
{
// Quoted annotations

2
crates/ruff_linter/src/rules/flake8_bandit/helpers.rs
View file

@ -10,7 +10,7 @@ static PASSWORD_CANDIDATE_REGEX: Lazy<Regex> = Lazy::new(|| {
pub(super) fn string_literal(expr: &Expr) -> Option<&str> {
match expr {
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => Some(value),
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => Some(value.as_str()),
_ => None,
}
}

2
crates/ruff_linter/src/rules/flake8_bandit/rules/hardcoded_bind_all_interfaces.rs
View file

@ -35,7 +35,7 @@ impl Violation for HardcodedBindAllInterfaces {
/// S104
pub(crate) fn hardcoded_bind_all_interfaces(string: &ExprStringLiteral) -> Option<Diagnostic> {
if string.value == "0.0.0.0" {
if string.value.as_str() == "0.0.0.0" {
Some(Diagnostic::new(HardcodedBindAllInterfaces, string.range))
} else {
None

115
crates/ruff_linter/src/rules/flake8_bandit/rules/hardcoded_sql_expression.rs
View file

@ -5,13 +5,12 @@ use ruff_python_ast::{self as ast, Expr, Operator};
use ruff_diagnostics::{Diagnostic, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast::helpers::any_over_expr;
use ruff_python_semantic::SemanticModel;
use ruff_python_ast::str::raw_contents;
use ruff_source_file::Locator;
use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
use super::super::helpers::string_literal;
static SQL_REGEX: Lazy<Regex> = Lazy::new(|| {
Regex::new(r"(?i)\b(select\s.+\sfrom\s|delete\s+from\s|(insert|replace)\s.+\svalues\s|update\s.+\sset\s)")
.unwrap()
@ -46,53 +45,77 @@ impl Violation for HardcodedSQLExpression {
}
}
fn has_string_literal(expr: &Expr) -> bool {
string_literal(expr).is_some()
}
fn matches_sql_statement(string: &str) -> bool {
SQL_REGEX.is_match(string)
}
fn matches_string_format_expression(expr: &Expr, semantic: &SemanticModel) -> bool {
match expr {
// "select * from table where val = " + "str" + ...
// "select * from table where val = %s" % ...
Expr::BinOp(ast::ExprBinOp {
op: Operator::Add | Operator::Mod,
..
}) => {
// Only evaluate the full BinOp, not the nested components.
if semantic
.current_expression_parent()
.map_or(true, |parent| !parent.is_bin_op_expr())
{
if any_over_expr(expr, &has_string_literal) {
return true;
}
}
false
}
Expr::Call(ast::ExprCall { func, .. }) => {
let Expr::Attribute(ast::ExprAttribute { attr, value, .. }) = func.as_ref() else {
return false;
};
// "select * from table where val = {}".format(...)
attr == "format" && string_literal(value).is_some()
}
// f"select * from table where val = {val}"
Expr::FString(_) => true,
_ => false,
}
/// Concatenates the contents of an f-string, without the prefix and quotes,
/// and escapes any special characters.
///
/// ## Example
///
/// ```python
/// "foo" f"bar {x}" "baz"
/// ```
///
/// becomes `foobar {x}baz`.
fn concatenated_f_string(expr: &ast::ExprFString, locator: &Locator) -> String {
expr.value
.parts()
.filter_map(|part| {
raw_contents(locator.slice(part)).map(|s| s.escape_default().to_string())
})
.collect()
}
/// S608
pub(crate) fn hardcoded_sql_expression(checker: &mut Checker, expr: &Expr) {
if matches_string_format_expression(expr, checker.semantic()) {
if matches_sql_statement(&checker.generator().expr(expr)) {
checker
.diagnostics
.push(Diagnostic::new(HardcodedSQLExpression, expr.range()));
let content = match expr {
// "select * from table where val = " + "str" + ...
Expr::BinOp(ast::ExprBinOp {
op: Operator::Add, ..
}) => {
// Only evaluate the full BinOp, not the nested components.
if !checker
.semantic()
.current_expression_parent()
.map_or(true, |parent| !parent.is_bin_op_expr())
{
return;
}
if !any_over_expr(expr, &Expr::is_string_literal_expr) {
return;
}
checker.generator().expr(expr)
}
// "select * from table where val = %s" % ...
Expr::BinOp(ast::ExprBinOp {
left,
op: Operator::Mod,
..
}) => {
let Some(string) = left.as_string_literal_expr() else {
return;
};
string.value.escape_default().to_string()
}
Expr::Call(ast::ExprCall { func, .. }) => {
let Expr::Attribute(ast::ExprAttribute { attr, value, .. }) = func.as_ref() else {
return;
};
// "select * from table where val = {}".format(...)
if attr != "format" {
return;
}
let Some(string) = value.as_string_literal_expr() else {
return;
};
string.value.escape_default().to_string()
}
// f"select * from table where val = {val}"
Expr::FString(f_string) => concatenated_f_string(f_string, checker.locator()),
_ => return,
};
if SQL_REGEX.is_match(&content) {
checker
.diagnostics
.push(Diagnostic::new(HardcodedSQLExpression, expr.range()));
}
}

2
crates/ruff_linter/src/rules/flake8_bandit/rules/hardcoded_tmp_directory.rs
View file

@ -76,7 +76,7 @@ pub(crate) fn hardcoded_tmp_directory(checker: &mut Checker, string: &ast::ExprS
checker.diagnostics.push(Diagnostic::new(
HardcodedTempFile {
string: string.value.clone(),
string: string.value.to_string(),
},
string.range,
));

10
crates/ruff_linter/src/rules/flake8_bandit/rules/suspicious_function_call.rs
View file

@ -854,11 +854,11 @@ pub(crate) fn suspicious_function_call(checker: &mut Checker, call: &ExprCall) {
["six", "moves", "urllib", "request", "urlopen" | "urlretrieve" | "Request"] => {
// If the `url` argument is a string literal, allow `http` and `https` schemes.
if call.arguments.args.iter().all(|arg| !arg.is_starred_expr()) && call.arguments.keywords.iter().all(|keyword| keyword.arg.is_some()) {
if let Some(Expr::StringLiteral(ast::ExprStringLiteral { value: url, .. })) = &call.arguments.find_argument("url", 0) {
let url = url.trim_start();
if url.starts_with("http://") || url.starts_with("https://") {
return None;
}
if let Some(Expr::StringLiteral(ast::ExprStringLiteral { value, .. })) = &call.arguments.find_argument("url", 0) {
let url = value.trim_start();
if url.starts_with("http://") || url.starts_with("https://") {
return None;
}
}
}
Some(SuspiciousURLOpenUsage.into())

2
crates/ruff_linter/src/rules/flake8_pie/rules/unnecessary_dict_kwargs.rs
View file

@ -111,7 +111,7 @@ pub(crate) fn unnecessary_dict_kwargs(checker: &mut Checker, expr: &Expr, kwargs
fn as_kwarg(key: &Expr) -> Option<&str> {
if let Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) = key {
if is_identifier(value) {
return Some(value);
return Some(value.as_str());
}
}
None

2
crates/ruff_linter/src/rules/flake8_pyi/rules/unrecognized_platform.rs
View file

@ -130,7 +130,7 @@ pub(crate) fn unrecognized_platform(checker: &mut Checker, test: &Expr) {
if !matches!(value.as_str(), "linux" | "win32" | "cygwin" | "darwin") {
checker.diagnostics.push(Diagnostic::new(
UnrecognizedPlatformName {
platform: value.clone(),
platform: value.to_string(),
},
right.range(),
));

9
crates/ruff_linter/src/rules/flake8_pytest_style/rules/helpers.rs
View file

@ -55,8 +55,13 @@ pub(super) fn is_empty_or_null_string(expr: &Expr) -> bool {
match expr {
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => value.is_empty(),
Expr::NoneLiteral(_) => true,
Expr::FString(ast::ExprFString { values, .. }) => {
values.iter().all(is_empty_or_null_string)
Expr::FString(ast::ExprFString { value, .. }) => {
value.parts().all(|f_string_part| match f_string_part {
ast::FStringPart::Literal(literal) => literal.is_empty(),
ast::FStringPart::FString(f_string) => {
f_string.values.iter().all(is_empty_or_null_string)
}
})
}
_ => false,
}

14
crates/ruff_linter/src/rules/flake8_pytest_style/rules/parametrize.rs
View file

@ -252,7 +252,7 @@ fn elts_to_csv(elts: &[Expr], generator: Generator) -> Option<String> {
return None;
}
let node = Expr::StringLiteral(ast::ExprStringLiteral {
let node = Expr::from(ast::StringLiteral {
value: elts.iter().fold(String::new(), |mut acc, elt| {
if let Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) = elt {
if !acc.is_empty() {
@ -262,9 +262,7 @@ fn elts_to_csv(elts: &[Expr], generator: Generator) -> Option<String> {
}
acc
}),
unicode: false,
implicit_concatenated: false,
range: TextRange::default(),
..ast::StringLiteral::default()
});
Some(generator.expr(&node))
}
@ -324,9 +322,9 @@ fn check_names(checker: &mut Checker, decorator: &Decorator, expr: &Expr) {
elts: names
.iter()
.map(|name| {
Expr::StringLiteral(ast::ExprStringLiteral {
Expr::from(ast::StringLiteral {
value: (*name).to_string(),
..ast::ExprStringLiteral::default()
..ast::StringLiteral::default()
})
})
.collect(),
@ -357,9 +355,9 @@ fn check_names(checker: &mut Checker, decorator: &Decorator, expr: &Expr) {
elts: names
.iter()
.map(|name| {
Expr::StringLiteral(ast::ExprStringLiteral {
Expr::from(ast::StringLiteral {
value: (*name).to_string(),
..ast::ExprStringLiteral::default()
..ast::StringLiteral::default()
})
})
.collect(),

21
crates/ruff_linter/src/rules/flake8_simplify/rules/ast_expr.rs
View file

@ -166,14 +166,14 @@ pub(crate) fn use_capital_environment_variables(checker: &mut Checker, expr: &Ex
}
let capital_env_var = env_var.to_ascii_uppercase();
if &capital_env_var == env_var {
if capital_env_var == env_var.as_str() {
return;
}
checker.diagnostics.push(Diagnostic::new(
UncapitalizedEnvironmentVariables {
expected: SourceCodeSnippet::new(capital_env_var),
actual: SourceCodeSnippet::new(env_var.clone()),
actual: SourceCodeSnippet::new(env_var.to_string()),
},
arg.range(),
));
@ -197,12 +197,7 @@ fn check_os_environ_subscript(checker: &mut Checker, expr: &Expr) {
if id != "os" || attr != "environ" {
return;
}
let Expr::StringLiteral(ast::ExprStringLiteral {
value: env_var,
unicode,
..
}) = slice.as_ref()
else {
let Expr::StringLiteral(ast::ExprStringLiteral { value: env_var, .. }) = slice.as_ref() else {
return;
};
@ -211,21 +206,21 @@ fn check_os_environ_subscript(checker: &mut Checker, expr: &Expr) {
}
let capital_env_var = env_var.to_ascii_uppercase();
if &capital_env_var == env_var {
if capital_env_var == env_var.as_str() {
return;
}
let mut diagnostic = Diagnostic::new(
UncapitalizedEnvironmentVariables {
expected: SourceCodeSnippet::new(capital_env_var.clone()),
actual: SourceCodeSnippet::new(env_var.clone()),
actual: SourceCodeSnippet::new(env_var.to_string()),
},
slice.range(),
);
let node = ast::ExprStringLiteral {
let node = ast::StringLiteral {
value: capital_env_var,
unicode: *unicode,
..ast::ExprStringLiteral::default()
unicode: env_var.is_unicode(),
..ast::StringLiteral::default()
};
let new_env_var = node.into();
diagnostic.set_fix(Fix::unsafe_edit(Edit::range_replacement(

2
crates/ruff_linter/src/rules/flake8_use_pathlib/rules/path_constructor_current_directory.rs
View file

@ -65,7 +65,7 @@ pub(crate) fn path_constructor_current_directory(checker: &mut Checker, expr: &E
return;
}
let [Expr::StringLiteral(ast::ExprStringLiteral { value, range, .. })] = args.as_slice() else {
let [Expr::StringLiteral(ast::ExprStringLiteral { value, range })] = args.as_slice() else {
return;
};

6
crates/ruff_linter/src/rules/flynt/helpers.rs
View file

@ -15,9 +15,9 @@ fn to_formatted_value_expr(inner: &Expr) -> Expr {
/// Convert a string to a constant string expression.
pub(super) fn to_constant_string(s: &str) -> Expr {
let node = ast::ExprStringLiteral {
value: s.to_owned(),
..ast::ExprStringLiteral::default()
let node = ast::StringLiteral {
value: s.to_string(),
..ast::StringLiteral::default()
};
node.into()
}

9
crates/ruff_linter/src/rules/flynt/rules/static_join_to_fstring.rs
View file

@ -62,7 +62,7 @@ fn is_static_length(elts: &[Expr]) -> bool {
fn build_fstring(joiner: &str, joinees: &[Expr]) -> Option<Expr> {
// If all elements are string constants, join them into a single string.
if joinees.iter().all(Expr::is_string_literal_expr) {
let node = ast::ExprStringLiteral {
let node = ast::StringLiteral {
value: joinees
.iter()
.filter_map(|expr| {
@ -73,9 +73,7 @@ fn build_fstring(joiner: &str, joinees: &[Expr]) -> Option<Expr> {
}
})
.join(joiner),
unicode: false,
implicit_concatenated: false,
range: TextRange::default(),
..ast::StringLiteral::default()
};
return Some(node.into());
}
@ -95,9 +93,8 @@ fn build_fstring(joiner: &str, joinees: &[Expr]) -> Option<Expr> {
fstring_elems.push(helpers::to_f_string_element(expr)?);
}
let node = ast::ExprFString {
let node = ast::FString {
values: fstring_elems,
implicit_concatenated: false,
range: TextRange::default(),
};
Some(node.into())

105
crates/ruff_linter/src/rules/pyflakes/rules/f_string_missing_placeholders.rs
View file

@ -1,7 +1,6 @@
use ruff_diagnostics::{AlwaysFixableViolation, Diagnostic, Edit, Fix};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast::{self as ast, Expr, PySourceType};
use ruff_python_parser::{lexer, AsMode, Tok};
use ruff_python_ast::{self as ast, Expr};
use ruff_source_file::Locator;
use ruff_text_size::{Ranged, TextRange, TextSize};
@ -46,74 +45,36 @@ impl AlwaysFixableViolation for FStringMissingPlaceholders {
}
}
/// Return an iterator containing a two-element tuple for each f-string part
/// in the given [`ExprFString`] expression.
///
/// The first element of the tuple is the f-string prefix range, and the second
/// element is the entire f-string range. It returns an iterator because of the
/// possibility of multiple f-strings implicitly concatenated together.
///
/// For example,
///
/// ```python
/// f"first" rf"second"
/// # ^ ^ (prefix range)
/// # ^^^^^^^^ ^^^^^^^^^^ (token range)
/// ```
///
/// would return `[(0..1, 0..8), (10..11, 9..19)]`.
///
/// This function assumes that the given f-string expression is without any
/// placeholder expressions.
///
/// [`ExprFString`]: `ruff_python_ast::ExprFString`
fn fstring_prefix_and_tok_range<'a>(
fstring: &'a ast::ExprFString,
locator: &'a Locator,
source_type: PySourceType,
) -> impl Iterator<Item = (TextRange, TextRange)> + 'a {
let contents = locator.slice(fstring);
let mut current_f_string_start = fstring.start();
lexer::lex_starts_at(contents, source_type.as_mode(), fstring.start())
.flatten()
.filter_map(move |(tok, range)| match tok {
Tok::FStringStart => {
current_f_string_start = range.start();
None
}
Tok::FStringEnd => {
let first_char =
locator.slice(TextRange::at(current_f_string_start, TextSize::from(1)));
// f"..." => f_position = 0
// fr"..." => f_position = 0
// rf"..." => f_position = 1
let f_position = u32::from(!(first_char == "f" || first_char == "F"));
Some((
TextRange::at(
current_f_string_start + TextSize::from(f_position),
TextSize::from(1),
),
TextRange::new(current_f_string_start, range.end()),
))
}
_ => None,
})
}
/// F541
pub(crate) fn f_string_missing_placeholders(fstring: &ast::ExprFString, checker: &mut Checker) {
if !fstring.values.iter().any(Expr::is_formatted_value_expr) {
for (prefix_range, tok_range) in
fstring_prefix_and_tok_range(fstring, checker.locator(), checker.source_type)
{
let mut diagnostic = Diagnostic::new(FStringMissingPlaceholders, tok_range);
diagnostic.set_fix(convert_f_string_to_regular_string(
prefix_range,
tok_range,
checker.locator(),
));
checker.diagnostics.push(diagnostic);
}
pub(crate) fn f_string_missing_placeholders(checker: &mut Checker, expr: &ast::ExprFString) {
if expr
.value
.f_strings()
.any(|f_string| f_string.values.iter().any(Expr::is_formatted_value_expr))
{
return;
}
for f_string in expr.value.f_strings() {
let first_char = checker
.locator()
.slice(TextRange::at(f_string.start(), TextSize::new(1)));
// f"..." => f_position = 0
// fr"..." => f_position = 0
// rf"..." => f_position = 1
let f_position = u32::from(!(first_char == "f" || first_char == "F"));
let prefix_range = TextRange::at(
f_string.start() + TextSize::new(f_position),
TextSize::new(1),
);
let mut diagnostic = Diagnostic::new(FStringMissingPlaceholders, f_string.range());
diagnostic.set_fix(convert_f_string_to_regular_string(
prefix_range,
f_string.range(),
checker.locator(),
));
checker.diagnostics.push(diagnostic);
}
}
@ -129,12 +90,12 @@ fn unescape_f_string(content: &str) -> String {
/// Generate a [`Fix`] to rewrite an f-string as a regular string.
fn convert_f_string_to_regular_string(
prefix_range: TextRange,
tok_range: TextRange,
node_range: TextRange,
locator: &Locator,
) -> Fix {
// Extract the f-string body.
let mut content =
unescape_f_string(locator.slice(TextRange::new(prefix_range.end(), tok_range.end())));
unescape_f_string(locator.slice(TextRange::new(prefix_range.end(), node_range.end())));
// If the preceding character is equivalent to the quote character, insert a space to avoid a
// syntax error. For example, when removing the `f` prefix in `""f""`, rewrite to `"" ""`
@ -151,6 +112,6 @@ fn convert_f_string_to_regular_string(
Fix::safe_edit(Edit::replacement(
content,
prefix_range.start(),
tok_range.end(),
node_range.end(),
))
}

4
crates/ruff_linter/src/rules/pyflakes/rules/strings.rs
View file

@ -641,7 +641,7 @@ pub(crate) fn percent_format_missing_arguments(
for key in keys.iter().flatten() {
match key {
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => {
keywords.insert(value);
keywords.insert(value.as_str());
}
_ => {
return; // Dynamic keys present
@ -652,7 +652,7 @@ pub(crate) fn percent_format_missing_arguments(
let missing: Vec<&String> = summary
.keywords
.iter()
.filter(|k| !keywords.contains(k))
.filter(|k| !keywords.contains(k.as_str()))
.collect();
if !missing.is_empty() {

4
crates/ruff_linter/src/rules/pylint/helpers.rs
View file

@ -11,8 +11,8 @@ use crate::settings::LinterSettings;
pub(super) fn type_param_name(arguments: &Arguments) -> Option<&str> {
// Handle both `TypeVar("T")` and `TypeVar(name="T")`.
let name_param = arguments.find_argument("name", 0)?;
if let Expr::StringLiteral(ast::ExprStringLiteral { value: name, .. }) = &name_param {
Some(name)
if let Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) = &name_param {
Some(value)
} else {
None
}

47
crates/ruff_linter/src/rules/pylint/rules/assert_on_string_literal.rs
View file

@ -69,27 +69,34 @@ pub(crate) fn assert_on_string_literal(checker: &mut Checker, test: &Expr) {
test.range(),
));
}
Expr::FString(ast::ExprFString { values, .. }) => {
checker.diagnostics.push(Diagnostic::new(
AssertOnStringLiteral {
kind: if values.iter().all(|value| match value {
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => {
value.is_empty()
}
_ => false,
}) {
Kind::Empty
} else if values.iter().any(|value| match value {
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => {
!value.is_empty()
}
_ => false,
}) {
Kind::NonEmpty
Expr::FString(ast::ExprFString { value, .. }) => {
let kind = if value.parts().all(|f_string_part| match f_string_part {
ast::FStringPart::Literal(literal) => literal.is_empty(),
ast::FStringPart::FString(f_string) => f_string.values.iter().all(|value| {
if let Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) = value {
value.is_empty()
} else {
Kind::Unknown
},
},
false
}
}),
}) {
Kind::Empty
} else if value.parts().any(|f_string_part| match f_string_part {
ast::FStringPart::Literal(literal) => !literal.is_empty(),
ast::FStringPart::FString(f_string) => f_string.values.iter().any(|value| {
if let Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) = value {
!value.is_empty()
} else {
false
}
}),
}) {
Kind::NonEmpty
} else {
Kind::Unknown
};
checker.diagnostics.push(Diagnostic::new(
AssertOnStringLiteral { kind },
test.range(),
));
}

4
crates/ruff_linter/src/rules/pylint/rules/bad_string_format_type.rs
View file

@ -124,8 +124,8 @@ fn equivalent(format: &CFormatSpec, value: &Expr) -> bool {
ResolvedPythonType::Atom(atom) => {
// Special case where `%c` allows single character strings to be formatted
if format.format_char == 'c' {
if let Expr::StringLiteral(string) = value {
let mut chars = string.chars();
if let Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) = value {
let mut chars = value.chars();
if chars.next().is_some() && chars.next().is_none() {
return true;
}

38
crates/ruff_linter/src/rules/pyupgrade/rules/native_literals.rs
View file

@ -3,7 +3,7 @@ use std::str::FromStr;
use ruff_diagnostics::{AlwaysFixableViolation, Diagnostic, Edit, Fix};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_ast::{self as ast, Expr};
use ruff_python_ast::{self as ast, Expr, LiteralExpressionRef};
use ruff_text_size::{Ranged, TextRange};
use crate::checkers::ast::Checker;
@ -52,20 +52,24 @@ impl LiteralType {
}
}
impl TryFrom<&Expr> for LiteralType {
impl TryFrom<LiteralExpressionRef<'_>> for LiteralType {
type Error = ();
fn try_from(expr: &Expr) -> Result<Self, Self::Error> {
match expr {
Expr::StringLiteral(_) => Ok(LiteralType::Str),
Expr::BytesLiteral(_) => Ok(LiteralType::Bytes),
Expr::NumberLiteral(ast::ExprNumberLiteral { value, .. }) => match value {
ast::Number::Int(_) => Ok(LiteralType::Int),
ast::Number::Float(_) => Ok(LiteralType::Float),
ast::Number::Complex { .. } => Err(()),
},
Expr::BooleanLiteral(_) => Ok(LiteralType::Bool),
_ => Err(()),
fn try_from(literal_expr: LiteralExpressionRef<'_>) -> Result<Self, Self::Error> {
match literal_expr {
LiteralExpressionRef::StringLiteral(_) => Ok(LiteralType::Str),
LiteralExpressionRef::BytesLiteral(_) => Ok(LiteralType::Bytes),
LiteralExpressionRef::NumberLiteral(ast::ExprNumberLiteral { value, .. }) => {
match value {
ast::Number::Int(_) => Ok(LiteralType::Int),
ast::Number::Float(_) => Ok(LiteralType::Float),
ast::Number::Complex { .. } => Err(()),
}
}
LiteralExpressionRef::BooleanLiteral(_) => Ok(LiteralType::Bool),
LiteralExpressionRef::NoneLiteral(_) | LiteralExpressionRef::EllipsisLiteral(_) => {
Err(())
}
}
}
}
@ -194,12 +198,16 @@ pub(crate) fn native_literals(
checker.diagnostics.push(diagnostic);
}
Some(arg) => {
let Some(literal_expr) = arg.as_literal_expr() else {
return;
};
// Skip implicit string concatenations.
if arg.is_implicit_concatenated_string() {
if literal_expr.is_implicit_concatenated() {
return;
}
let Ok(arg_literal_type) = LiteralType::try_from(arg) else {
let Ok(arg_literal_type) = LiteralType::try_from(literal_expr) else {
return;
};

29
crates/ruff_linter/src/rules/pyupgrade/rules/unicode_kind_prefix.rs
View file

@ -1,11 +1,10 @@
use ruff_diagnostics::{AlwaysFixableViolation, Diagnostic, Edit, Fix};
use ruff_macros::{derive_message_formats, violation};
use ruff_python_parser::lexer::LexResult;
use ruff_python_parser::{StringKind, Tok};
use ruff_python_ast::StringLiteral;
use ruff_text_size::{Ranged, TextRange, TextSize};
use crate::checkers::ast::Checker;
/// ## What it does
/// Checks for uses of the Unicode kind prefix (`u`) in strings.
///
@ -40,19 +39,13 @@ impl AlwaysFixableViolation for UnicodeKindPrefix {
}
/// UP025
pub(crate) fn unicode_kind_prefix(diagnostics: &mut Vec<Diagnostic>, tokens: &[LexResult]) {
for (token, range) in tokens.iter().flatten() {
if let Tok::String {
kind: StringKind::Unicode,
..
} = token
{
let mut diagnostic = Diagnostic::new(UnicodeKindPrefix, *range);
diagnostic.set_fix(Fix::safe_edit(Edit::range_deletion(TextRange::at(
range.start(),
TextSize::from(1),
))));
diagnostics.push(diagnostic);
}
pub(crate) fn unicode_kind_prefix(checker: &mut Checker, string: &StringLiteral) {
if string.unicode {
let mut diagnostic = Diagnostic::new(UnicodeKindPrefix, string.range);
diagnostic.set_fix(Fix::safe_edit(Edit::range_deletion(TextRange::at(
string.start(),
TextSize::from(1),
))));
checker.diagnostics.push(diagnostic);
}
}

10
crates/ruff_linter/src/rules/refurb/rules/read_whole_file.rs
View file

@ -244,14 +244,10 @@ fn match_open_keywords(
/// Match open mode to see if it is supported.
fn match_open_mode(mode: &Expr) -> Option<ReadMode> {
let ast::ExprStringLiteral {
value,
implicit_concatenated: false,
..
} = mode.as_string_literal_expr()?
else {
let ast::ExprStringLiteral { value, .. } = mode.as_string_literal_expr()?;
if value.is_implicit_concatenated() {
return None;
};
}
match value.as_str() {
"r" => Some(ReadMode::Text),
"rb" => Some(ReadMode::Bytes),

103
crates/ruff_linter/src/rules/ruff/rules/explicit_f_string_type_conversion.rs
View file

@ -1,7 +1,4 @@
use anyhow::{bail, Result};
use libcst_native::{
ConcatenatedString, Expression, FormattedStringContent, FormattedStringExpression,
};
use ruff_diagnostics::{AlwaysFixableViolation, Diagnostic, Edit, Fix};
use ruff_macros::{derive_message_formats, violation};
@ -11,7 +8,10 @@ use ruff_source_file::Locator;
use ruff_text_size::Ranged;
use crate::checkers::ast::Checker;
use crate::cst::matchers::{match_call_mut, match_name, transform_expression};
use crate::cst::matchers::{
match_call_mut, match_formatted_string, match_formatted_string_expression, match_name,
transform_expression,
};
/// ## What it does
/// Checks for uses of `str()`, `repr()`, and `ascii()` as explicit type
@ -52,25 +52,14 @@ impl AlwaysFixableViolation for ExplicitFStringTypeConversion {
}
/// RUF010
pub(crate) fn explicit_f_string_type_conversion(
checker: &mut Checker,
expr: &Expr,
values: &[Expr],
) {
for (index, formatted_value) in values
.iter()
.filter_map(|expr| {
if let Expr::FormattedValue(expr) = &expr {
Some(expr)
} else {
None
}
})
.enumerate()
{
let ast::ExprFormattedValue {
pub(crate) fn explicit_f_string_type_conversion(checker: &mut Checker, f_string: &ast::FString) {
for (index, expr) in f_string.values.iter().enumerate() {
let Some(ast::ExprFormattedValue {
value, conversion, ..
} = formatted_value;
}) = expr.as_formatted_value_expr()
else {
continue;
};
// Skip if there's already a conversion flag.
if !conversion.is_none() {
@ -123,7 +112,7 @@ pub(crate) fn explicit_f_string_type_conversion(
let mut diagnostic = Diagnostic::new(ExplicitFStringTypeConversion, value.range());
diagnostic.try_set_fix(|| {
convert_call_to_conversion_flag(expr, index, checker.locator(), checker.stylist())
convert_call_to_conversion_flag(f_string, index, checker.locator(), checker.stylist())
});
checker.diagnostics.push(diagnostic);
}
@ -131,15 +120,17 @@ pub(crate) fn explicit_f_string_type_conversion(
/// Generate a [`Fix`] to replace an explicit type conversion with a conversion flag.
fn convert_call_to_conversion_flag(
expr: &Expr,
f_string: &ast::FString,
index: usize,
locator: &Locator,
stylist: &Stylist,
) -> Result<Fix> {
let source_code = locator.slice(expr);
let source_code = locator.slice(f_string);
transform_expression(source_code, stylist, |mut expression| {
let formatted_string = match_formatted_string(&mut expression)?;
// Replace the formatted call expression at `index` with a conversion flag.
let formatted_string_expression = match_part(index, &mut expression)?;
let formatted_string_expression =
match_formatted_string_expression(&mut formatted_string.parts[index])?;
let call = match_call_mut(&mut formatted_string_expression.expression)?;
let name = match_name(&call.func)?;
match name.value {
@ -157,63 +148,5 @@ fn convert_call_to_conversion_flag(
formatted_string_expression.expression = call.args[0].value.clone();
Ok(expression)
})
.map(|output| Fix::safe_edit(Edit::range_replacement(output, expr.range())))
}
/// Return the [`FormattedStringContent`] at the given index in an f-string or implicit
/// string concatenation.
fn match_part<'a, 'b>(
index: usize,
expr: &'a mut Expression<'b>,
) -> Result<&'a mut FormattedStringExpression<'b>> {
match expr {
Expression::ConcatenatedString(expr) => Ok(collect_parts(expr).remove(index)),
Expression::FormattedString(expr) => {
// Find the formatted expression at the given index. The `parts` field contains a mix
// of string literals and expressions, but our `index` only counts expressions. All
// the boxing and mutability makes this difficult to write in a functional style.
let mut format_index = 0;
for part in &mut expr.parts {
if let FormattedStringContent::Expression(expr) = part {
if format_index == index {
return Ok(expr);
}
format_index += 1;
}
}
bail!("Index out of bounds: `{index}`")
}
_ => bail!("Unexpected expression: `{:?}`", expr),
}
}
/// Given an implicit string concatenation, return a list of all the formatted expressions.
fn collect_parts<'a, 'b>(
expr: &'a mut ConcatenatedString<'b>,
) -> Vec<&'a mut FormattedStringExpression<'b>> {
fn inner<'a, 'b>(
string: &'a mut libcst_native::String<'b>,
formatted_expressions: &mut Vec<&'a mut FormattedStringExpression<'b>>,
) {
match string {
libcst_native::String::Formatted(expr) => {
for part in &mut expr.parts {
if let FormattedStringContent::Expression(expr) = part {
formatted_expressions.push(expr);
}
}
}
libcst_native::String::Concatenated(expr) => {
inner(&mut expr.left, formatted_expressions);
inner(&mut expr.right, formatted_expressions);
}
libcst_native::String::Simple(_) => {}
}
}
let mut formatted_expressions = vec![];
inner(&mut expr.left, &mut formatted_expressions);
inner(&mut expr.right, &mut formatted_expressions);
formatted_expressions
.map(|output| Fix::safe_edit(Edit::range_replacement(output, f_string.range())))
}

1
crates/ruff_linter/src/rules/ruff/rules/implicit_optional.rs
View file

@ -184,7 +184,6 @@ pub(crate) fn implicit_optional(checker: &mut Checker, parameters: &Parameters)
if let Expr::StringLiteral(ast::ExprStringLiteral {
range,
value: string,
..
}) = annotation.as_ref()
{
// Quoted annotation.

1
crates/ruff_linter/src/rules/ruff/typing.rs
View file

@ -111,7 +111,6 @@ impl<'a> TypingTarget<'a> {
Expr::StringLiteral(ast::ExprStringLiteral {
value: string,
range,
..
}) => parse_type_annotation(string, *range, locator.contents())
.map_or(None, |(expr, _)| Some(TypingTarget::ForwardReference(expr))),
_ => semantic.resolve_call_path(expr).map_or(

19
crates/ruff_linter/src/rules/tryceratops/rules/raise_vanilla_args.rs
View file

@ -89,10 +89,21 @@ pub(crate) fn raise_vanilla_args(checker: &mut Checker, expr: &Expr) {
/// some whitespace).
fn contains_message(expr: &Expr) -> bool {
match expr {
Expr::FString(ast::ExprFString { values, .. }) => {
for value in values {
if contains_message(value) {
return true;
Expr::FString(ast::ExprFString { value, .. }) => {
for f_string_part in value.parts() {
match f_string_part {
ast::FStringPart::Literal(literal) => {
if literal.chars().any(char::is_whitespace) {
return true;
}
}
ast::FStringPart::FString(f_string) => {
for value in &f_string.values {
if contains_message(value) {
return true;
}
}
}
}
}
}

159
crates/ruff_python_ast/src/comparable.rs
View file

@ -529,6 +529,91 @@ impl<'a> From<&'a ast::ElifElseClause> for ComparableElifElseClause<'a> {
}
}
#[derive(Debug, PartialEq, Eq, Hash)]
pub enum ComparableLiteral<'a> {
None,
Ellipsis,
Bool(&'a bool),
Str(Vec<ComparableStringLiteral<'a>>),
Bytes(Vec<ComparableBytesLiteral<'a>>),
Number(ComparableNumber<'a>),
}
impl<'a> From<ast::LiteralExpressionRef<'a>> for ComparableLiteral<'a> {
fn from(literal: ast::LiteralExpressionRef<'a>) -> Self {
match literal {
ast::LiteralExpressionRef::NoneLiteral(_) => Self::None,
ast::LiteralExpressionRef::EllipsisLiteral(_) => Self::Ellipsis,
ast::LiteralExpressionRef::BooleanLiteral(ast::ExprBooleanLiteral {
value, ..
}) => Self::Bool(value),
ast::LiteralExpressionRef::StringLiteral(ast::ExprStringLiteral { value, .. }) => {
Self::Str(value.parts().map(Into::into).collect())
}
ast::LiteralExpressionRef::BytesLiteral(ast::ExprBytesLiteral { value, .. }) => {
Self::Bytes(value.parts().map(Into::into).collect())
}
ast::LiteralExpressionRef::NumberLiteral(ast::ExprNumberLiteral { value, .. }) => {
Self::Number(value.into())
}
}
}
}
#[derive(Debug, PartialEq, Eq, Hash)]
pub struct ComparableFString<'a> {
values: Vec<ComparableExpr<'a>>,
}
impl<'a> From<&'a ast::FString> for ComparableFString<'a> {
fn from(fstring: &'a ast::FString) -> Self {
Self {
values: fstring.values.iter().map(Into::into).collect(),
}
}
}
#[derive(Debug, PartialEq, Eq, Hash)]
pub enum ComparableFStringPart<'a> {
Literal(ComparableStringLiteral<'a>),
FString(ComparableFString<'a>),
}
impl<'a> From<&'a ast::FStringPart> for ComparableFStringPart<'a> {
fn from(f_string_part: &'a ast::FStringPart) -> Self {
match f_string_part {
ast::FStringPart::Literal(string_literal) => Self::Literal(string_literal.into()),
ast::FStringPart::FString(f_string) => Self::FString(f_string.into()),
}
}
}
#[derive(Debug, PartialEq, Eq, Hash)]
pub struct ComparableStringLiteral<'a> {
value: &'a str,
}
impl<'a> From<&'a ast::StringLiteral> for ComparableStringLiteral<'a> {
fn from(string_literal: &'a ast::StringLiteral) -> Self {
Self {
value: string_literal.value.as_str(),
}
}
}
#[derive(Debug, PartialEq, Eq, Hash)]
pub struct ComparableBytesLiteral<'a> {
value: &'a [u8],
}
impl<'a> From<&'a ast::BytesLiteral> for ComparableBytesLiteral<'a> {
fn from(bytes_literal: &'a ast::BytesLiteral) -> Self {
Self {
value: bytes_literal.value.as_slice(),
}
}
}
#[derive(Debug, PartialEq, Eq, Hash)]
pub struct ExprBoolOp<'a> {
op: ComparableBoolOp,
@ -641,48 +726,17 @@ pub struct ExprFormattedValue<'a> {
#[derive(Debug, PartialEq, Eq, Hash)]
pub struct ExprFString<'a> {
values: Vec<ComparableExpr<'a>>,
}
#[derive(Debug, PartialEq, Eq, Hash)]
pub enum ComparableLiteral<'a> {
None,
Ellipsis,
Bool(&'a bool),
Str(&'a str),
Bytes(&'a [u8]),
Number(ComparableNumber<'a>),
}
impl<'a> From<ast::LiteralExpressionRef<'a>> for ComparableLiteral<'a> {
fn from(literal: ast::LiteralExpressionRef<'a>) -> Self {
match literal {
ast::LiteralExpressionRef::NoneLiteral(_) => Self::None,
ast::LiteralExpressionRef::EllipsisLiteral(_) => Self::Ellipsis,
ast::LiteralExpressionRef::BooleanLiteral(ast::ExprBooleanLiteral {
value, ..
}) => Self::Bool(value),
ast::LiteralExpressionRef::StringLiteral(ast::ExprStringLiteral { value, .. }) => {
Self::Str(value)
}
ast::LiteralExpressionRef::BytesLiteral(ast::ExprBytesLiteral { value, .. }) => {
Self::Bytes(value)
}
ast::LiteralExpressionRef::NumberLiteral(ast::ExprNumberLiteral { value, .. }) => {
Self::Number(value.into())
}
}
}
parts: Vec<ComparableFStringPart<'a>>,
}
#[derive(Debug, PartialEq, Eq, Hash)]
pub struct ExprStringLiteral<'a> {
value: &'a str,
parts: Vec<ComparableStringLiteral<'a>>,
}
#[derive(Debug, PartialEq, Eq, Hash)]
pub struct ExprBytesLiteral<'a> {
value: &'a [u8],
parts: Vec<ComparableBytesLiteral<'a>>,
}
#[derive(Debug, PartialEq, Eq, Hash)]
@ -933,28 +987,21 @@ impl<'a> From<&'a ast::Expr> for ComparableExpr<'a> {
debug_text: debug_text.as_ref(),
format_spec: format_spec.as_ref().map(Into::into),
}),
ast::Expr::FString(ast::ExprFString {
values,
implicit_concatenated: _,
range: _,
}) => Self::FString(ExprFString {
values: values.iter().map(Into::into).collect(),
}),
ast::Expr::StringLiteral(ast::ExprStringLiteral {
value,
// Compare strings based on resolved value, not representation (i.e., ignore whether
// the string was implicitly concatenated).
implicit_concatenated: _,
unicode: _,
range: _,
}) => Self::StringLiteral(ExprStringLiteral { value }),
ast::Expr::BytesLiteral(ast::ExprBytesLiteral {
value,
// Compare bytes based on resolved value, not representation (i.e., ignore whether
// the bytes was implicitly concatenated).
implicit_concatenated: _,
range: _,
}) => Self::BytesLiteral(ExprBytesLiteral { value }),
ast::Expr::FString(ast::ExprFString { value, range: _ }) => {
Self::FString(ExprFString {
parts: value.parts().map(Into::into).collect(),
})
}
ast::Expr::StringLiteral(ast::ExprStringLiteral { value, range: _ }) => {
Self::StringLiteral(ExprStringLiteral {
parts: value.parts().map(Into::into).collect(),
})
}
ast::Expr::BytesLiteral(ast::ExprBytesLiteral { value, range: _ }) => {
Self::BytesLiteral(ExprBytesLiteral {
parts: value.parts().map(Into::into).collect(),
})
}
ast::Expr::NumberLiteral(ast::ExprNumberLiteral { value, range: _ }) => {
Self::NumberLiteral(ExprNumberLiteral {
value: value.into(),

41
crates/ruff_python_ast/src/expression.rs
View file

@ -361,3 +361,44 @@ impl Ranged for LiteralExpressionRef<'_> {
}
}
}
impl<'a> From<LiteralExpressionRef<'a>> for AnyNodeRef<'a> {
fn from(value: LiteralExpressionRef<'a>) -> Self {
match value {
LiteralExpressionRef::StringLiteral(expression) => {
AnyNodeRef::ExprStringLiteral(expression)
}
LiteralExpressionRef::BytesLiteral(expression) => {
AnyNodeRef::ExprBytesLiteral(expression)
}
LiteralExpressionRef::NumberLiteral(expression) => {
AnyNodeRef::ExprNumberLiteral(expression)
}
LiteralExpressionRef::BooleanLiteral(expression) => {
AnyNodeRef::ExprBooleanLiteral(expression)
}
LiteralExpressionRef::NoneLiteral(expression) => {
AnyNodeRef::ExprNoneLiteral(expression)
}
LiteralExpressionRef::EllipsisLiteral(expression) => {
AnyNodeRef::ExprEllipsisLiteral(expression)
}
}
}
}
impl LiteralExpressionRef<'_> {
/// Returns `true` if the literal is either a string or bytes literal that
/// is implicitly concatenated.
pub fn is_implicit_concatenated(&self) -> bool {
match self {
LiteralExpressionRef::StringLiteral(expression) => {
expression.value.is_implicit_concatenated()
}
LiteralExpressionRef::BytesLiteral(expression) => {
expression.value.is_implicit_concatenated()
}
_ => false,
}
}
}

17
crates/ruff_python_ast/src/helpers.rs
View file

@ -133,10 +133,12 @@ pub fn any_over_expr(expr: &Expr, func: &dyn Fn(&Expr) -> bool) -> bool {
return true;
}
match expr {
Expr::BoolOp(ast::ExprBoolOp { values, .. })
| Expr::FString(ast::ExprFString { values, .. }) => {
Expr::BoolOp(ast::ExprBoolOp { values, .. }) => {
values.iter().any(|expr| any_over_expr(expr, func))
}
Expr::FString(ast::ExprFString { value, .. }) => {
value.elements().any(|expr| any_over_expr(expr, func))
}
Expr::NamedExpr(ast::ExprNamedExpr {
target,
value,
@ -1139,11 +1141,14 @@ impl Truthiness {
}
Expr::NoneLiteral(_) => Self::Falsey,
Expr::EllipsisLiteral(_) => Self::Truthy,
Expr::FString(ast::ExprFString { values, .. }) => {
if values.is_empty() {
Expr::FString(ast::ExprFString { value, .. }) => {
if value.parts().all(|part| match part {
ast::FStringPart::Literal(string_literal) => string_literal.is_empty(),
ast::FStringPart::FString(f_string) => f_string.values.is_empty(),
}) {
Self::Falsey
} else if values.iter().any(|value| {
if let Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) = &value {
} else if value.elements().any(|expr| {
if let Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) = &expr {
!value.is_empty()
} else {
false

232
crates/ruff_python_ast/src/node.rs
View file

@ -113,6 +113,9 @@ pub enum AnyNode {
TypeParamTypeVar(TypeParamTypeVar),
TypeParamTypeVarTuple(TypeParamTypeVarTuple),
TypeParamParamSpec(TypeParamParamSpec),
FString(ast::FString),
StringLiteral(ast::StringLiteral),
BytesLiteral(ast::BytesLiteral),
}
impl AnyNode {
@ -204,6 +207,9 @@ impl AnyNode {
| AnyNode::TypeParamTypeVar(_)
| AnyNode::TypeParamTypeVarTuple(_)
| AnyNode::TypeParamParamSpec(_)
| AnyNode::FString(_)
| AnyNode::StringLiteral(_)
| AnyNode::BytesLiteral(_)
| AnyNode::ElifElseClause(_) => None,
}
}
@ -296,6 +302,9 @@ impl AnyNode {
| AnyNode::TypeParamTypeVar(_)
| AnyNode::TypeParamTypeVarTuple(_)
| AnyNode::TypeParamParamSpec(_)
| AnyNode::FString(_)
| AnyNode::StringLiteral(_)
| AnyNode::BytesLiteral(_)
| AnyNode::ElifElseClause(_) => None,
}
}
@ -388,6 +397,9 @@ impl AnyNode {
| AnyNode::TypeParamTypeVar(_)
| AnyNode::TypeParamTypeVarTuple(_)
| AnyNode::TypeParamParamSpec(_)
| AnyNode::FString(_)
| AnyNode::StringLiteral(_)
| AnyNode::BytesLiteral(_)
| AnyNode::ElifElseClause(_) => None,
}
}
@ -480,6 +492,9 @@ impl AnyNode {
| AnyNode::TypeParamTypeVar(_)
| AnyNode::TypeParamTypeVarTuple(_)
| AnyNode::TypeParamParamSpec(_)
| AnyNode::FString(_)
| AnyNode::StringLiteral(_)
| AnyNode::BytesLiteral(_)
| AnyNode::ElifElseClause(_) => None,
}
}
@ -572,6 +587,9 @@ impl AnyNode {
| AnyNode::TypeParamTypeVar(_)
| AnyNode::TypeParamTypeVarTuple(_)
| AnyNode::TypeParamParamSpec(_)
| AnyNode::FString(_)
| AnyNode::StringLiteral(_)
| AnyNode::BytesLiteral(_)
| AnyNode::ElifElseClause(_) => None,
}
}
@ -683,6 +701,9 @@ impl AnyNode {
Self::TypeParamTypeVar(node) => AnyNodeRef::TypeParamTypeVar(node),
Self::TypeParamTypeVarTuple(node) => AnyNodeRef::TypeParamTypeVarTuple(node),
Self::TypeParamParamSpec(node) => AnyNodeRef::TypeParamParamSpec(node),
Self::FString(node) => AnyNodeRef::FString(node),
Self::StringLiteral(node) => AnyNodeRef::StringLiteral(node),
Self::BytesLiteral(node) => AnyNodeRef::BytesLiteral(node),
Self::ElifElseClause(node) => AnyNodeRef::ElifElseClause(node),
}
}
@ -2674,14 +2695,17 @@ impl AstNode for ast::ExprFString {
where
V: PreorderVisitor<'a> + ?Sized,
{
let ast::ExprFString {
values,
implicit_concatenated: _,
range: _,
} = self;
let ast::ExprFString { value, range: _ } = self;
for expr in values {
visitor.visit_expr(expr);
for f_string_part in value.parts() {
match f_string_part {
ast::FStringPart::Literal(string_literal) => {
visitor.visit_string_literal(string_literal);
}
ast::FStringPart::FString(f_string) => {
visitor.visit_f_string(f_string);
}
}
}
}
}
@ -2713,10 +2737,15 @@ impl AstNode for ast::ExprStringLiteral {
AnyNode::from(self)
}
fn visit_preorder<'a, V>(&'a self, _visitor: &mut V)
fn visit_preorder<'a, V>(&'a self, visitor: &mut V)
where
V: PreorderVisitor<'a> + ?Sized,
{
let ast::ExprStringLiteral { value, range: _ } = self;
for string_literal in value.parts() {
visitor.visit_string_literal(string_literal);
}
}
}
impl AstNode for ast::ExprBytesLiteral {
@ -2747,10 +2776,15 @@ impl AstNode for ast::ExprBytesLiteral {
AnyNode::from(self)
}
fn visit_preorder<'a, V>(&'a self, _visitor: &mut V)
fn visit_preorder<'a, V>(&'a self, visitor: &mut V)
where
V: PreorderVisitor<'a> + ?Sized,
{
let ast::ExprBytesLiteral { value, range: _ } = self;
for bytes_literal in value.parts() {
visitor.visit_bytes_literal(bytes_literal);
}
}
}
impl AstNode for ast::ExprNumberLiteral {
@ -4273,6 +4307,114 @@ impl AstNode for ast::TypeParamParamSpec {
let ast::TypeParamParamSpec { range: _, name: _ } = self;
}
}
impl AstNode for ast::FString {
fn cast(kind: AnyNode) -> Option<Self>
where
Self: Sized,
{
if let AnyNode::FString(node) = kind {
Some(node)
} else {
None
}
}
fn cast_ref(kind: AnyNodeRef) -> Option<&Self> {
if let AnyNodeRef::FString(node) = kind {
Some(node)
} else {
None
}
}
fn as_any_node_ref(&self) -> AnyNodeRef {
AnyNodeRef::from(self)
}
fn into_any_node(self) -> AnyNode {
AnyNode::from(self)
}
fn visit_preorder<'a, V>(&'a self, visitor: &mut V)
where
V: PreorderVisitor<'a> + ?Sized,
{
let ast::FString { values, range: _ } = self;
for expr in values {
visitor.visit_expr(expr);
}
}
}
impl AstNode for ast::StringLiteral {
fn cast(kind: AnyNode) -> Option<Self>
where
Self: Sized,
{
if let AnyNode::StringLiteral(node) = kind {
Some(node)
} else {
None
}
}
fn cast_ref(kind: AnyNodeRef) -> Option<&Self> {
if let AnyNodeRef::StringLiteral(node) = kind {
Some(node)
} else {
None
}
}
fn as_any_node_ref(&self) -> AnyNodeRef {
AnyNodeRef::from(self)
}
fn into_any_node(self) -> AnyNode {
AnyNode::from(self)
}
fn visit_preorder<'a, V>(&'a self, _visitor: &mut V)
where
V: PreorderVisitor<'a> + ?Sized,
{
}
}
impl AstNode for ast::BytesLiteral {
fn cast(kind: AnyNode) -> Option<Self>
where
Self: Sized,
{
if let AnyNode::BytesLiteral(node) = kind {
Some(node)
} else {
None
}
}
fn cast_ref(kind: AnyNodeRef) -> Option<&Self> {
if let AnyNodeRef::BytesLiteral(node) = kind {
Some(node)
} else {
None
}
}
fn as_any_node_ref(&self) -> AnyNodeRef {
AnyNodeRef::from(self)
}
fn into_any_node(self) -> AnyNode {
AnyNode::from(self)
}
fn visit_preorder<'a, V>(&'a self, _visitor: &mut V)
where
V: PreorderVisitor<'a> + ?Sized,
{
}
}
impl From<Stmt> for AnyNode {
fn from(stmt: Stmt) -> Self {
match stmt {
@ -4882,6 +5024,24 @@ impl From<TypeParamParamSpec> for AnyNode {
}
}
impl From<ast::FString> for AnyNode {
fn from(node: ast::FString) -> Self {
AnyNode::FString(node)
}
}
impl From<ast::StringLiteral> for AnyNode {
fn from(node: ast::StringLiteral) -> Self {
AnyNode::StringLiteral(node)
}
}
impl From<ast::BytesLiteral> for AnyNode {
fn from(node: ast::BytesLiteral) -> Self {
AnyNode::BytesLiteral(node)
}
}
impl Ranged for AnyNode {
fn range(&self) -> TextRange {
match self {
@ -4970,6 +5130,9 @@ impl Ranged for AnyNode {
AnyNode::TypeParamTypeVar(node) => node.range(),
AnyNode::TypeParamTypeVarTuple(node) => node.range(),
AnyNode::TypeParamParamSpec(node) => node.range(),
AnyNode::FString(node) => node.range(),
AnyNode::StringLiteral(node) => node.range(),
AnyNode::BytesLiteral(node) => node.range(),
AnyNode::ElifElseClause(node) => node.range(),
}
}
@ -5062,6 +5225,9 @@ pub enum AnyNodeRef<'a> {
TypeParamTypeVar(&'a TypeParamTypeVar),
TypeParamTypeVarTuple(&'a TypeParamTypeVarTuple),
TypeParamParamSpec(&'a TypeParamParamSpec),
FString(&'a ast::FString),
StringLiteral(&'a ast::StringLiteral),
BytesLiteral(&'a ast::BytesLiteral),
ElifElseClause(&'a ast::ElifElseClause),
}
@ -5153,6 +5319,9 @@ impl<'a> AnyNodeRef<'a> {
AnyNodeRef::TypeParamTypeVar(node) => NonNull::from(*node).cast(),
AnyNodeRef::TypeParamTypeVarTuple(node) => NonNull::from(*node).cast(),
AnyNodeRef::TypeParamParamSpec(node) => NonNull::from(*node).cast(),
AnyNodeRef::FString(node) => NonNull::from(*node).cast(),
AnyNodeRef::StringLiteral(node) => NonNull::from(*node).cast(),
AnyNodeRef::BytesLiteral(node) => NonNull::from(*node).cast(),
AnyNodeRef::ElifElseClause(node) => NonNull::from(*node).cast(),
}
}
@ -5250,6 +5419,9 @@ impl<'a> AnyNodeRef<'a> {
AnyNodeRef::TypeParamTypeVar(_) => NodeKind::TypeParamTypeVar,
AnyNodeRef::TypeParamTypeVarTuple(_) => NodeKind::TypeParamTypeVarTuple,
AnyNodeRef::TypeParamParamSpec(_) => NodeKind::TypeParamParamSpec,
AnyNodeRef::FString(_) => NodeKind::FString,
AnyNodeRef::StringLiteral(_) => NodeKind::StringLiteral,
AnyNodeRef::BytesLiteral(_) => NodeKind::BytesLiteral,
AnyNodeRef::ElifElseClause(_) => NodeKind::ElifElseClause,
}
}
@ -5342,6 +5514,9 @@ impl<'a> AnyNodeRef<'a> {
| AnyNodeRef::TypeParamTypeVar(_)
| AnyNodeRef::TypeParamTypeVarTuple(_)
| AnyNodeRef::TypeParamParamSpec(_)
| AnyNodeRef::FString(_)
| AnyNodeRef::StringLiteral(_)
| AnyNodeRef::BytesLiteral(_)
| AnyNodeRef::ElifElseClause(_) => false,
}
}
@ -5434,6 +5609,9 @@ impl<'a> AnyNodeRef<'a> {
| AnyNodeRef::TypeParamTypeVar(_)
| AnyNodeRef::TypeParamTypeVarTuple(_)
| AnyNodeRef::TypeParamParamSpec(_)
| AnyNodeRef::FString(_)
| AnyNodeRef::StringLiteral(_)
| AnyNodeRef::BytesLiteral(_)
| AnyNodeRef::ElifElseClause(_) => false,
}
}
@ -5525,6 +5703,9 @@ impl<'a> AnyNodeRef<'a> {
| AnyNodeRef::TypeParamTypeVar(_)
| AnyNodeRef::TypeParamTypeVarTuple(_)
| AnyNodeRef::TypeParamParamSpec(_)
| AnyNodeRef::FString(_)
| AnyNodeRef::StringLiteral(_)
| AnyNodeRef::BytesLiteral(_)
| AnyNodeRef::ElifElseClause(_) => false,
}
}
@ -5617,6 +5798,9 @@ impl<'a> AnyNodeRef<'a> {
| AnyNodeRef::TypeParamTypeVar(_)
| AnyNodeRef::TypeParamTypeVarTuple(_)
| AnyNodeRef::TypeParamParamSpec(_)
| AnyNodeRef::FString(_)
| AnyNodeRef::StringLiteral(_)
| AnyNodeRef::BytesLiteral(_)
| AnyNodeRef::ElifElseClause(_) => false,
}
}
@ -5709,6 +5893,9 @@ impl<'a> AnyNodeRef<'a> {
| AnyNodeRef::TypeParamTypeVar(_)
| AnyNodeRef::TypeParamTypeVarTuple(_)
| AnyNodeRef::TypeParamParamSpec(_)
| AnyNodeRef::FString(_)
| AnyNodeRef::StringLiteral(_)
| AnyNodeRef::BytesLiteral(_)
| AnyNodeRef::ElifElseClause(_) => false,
}
}
@ -5829,6 +6016,9 @@ impl<'a> AnyNodeRef<'a> {
AnyNodeRef::TypeParamTypeVar(node) => node.visit_preorder(visitor),
AnyNodeRef::TypeParamTypeVarTuple(node) => node.visit_preorder(visitor),
AnyNodeRef::TypeParamParamSpec(node) => node.visit_preorder(visitor),
AnyNodeRef::FString(node) => node.visit_preorder(visitor),
AnyNodeRef::StringLiteral(node) => node.visit_preorder(visitor),
AnyNodeRef::BytesLiteral(node) => node.visit_preorder(visitor),
AnyNodeRef::ElifElseClause(node) => node.visit_preorder(visitor),
}
}
@ -6355,6 +6545,24 @@ impl<'a> From<&'a TypeParamParamSpec> for AnyNodeRef<'a> {
}
}
impl<'a> From<&'a ast::FString> for AnyNodeRef<'a> {
fn from(node: &'a ast::FString) -> Self {
AnyNodeRef::FString(node)
}
}
impl<'a> From<&'a ast::StringLiteral> for AnyNodeRef<'a> {
fn from(node: &'a ast::StringLiteral) -> Self {
AnyNodeRef::StringLiteral(node)
}
}
impl<'a> From<&'a ast::BytesLiteral> for AnyNodeRef<'a> {
fn from(node: &'a ast::BytesLiteral) -> Self {
AnyNodeRef::BytesLiteral(node)
}
}
impl<'a> From<&'a Stmt> for AnyNodeRef<'a> {
fn from(stmt: &'a Stmt) -> Self {
match stmt {
@ -6606,6 +6814,9 @@ impl Ranged for AnyNodeRef<'_> {
AnyNodeRef::TypeParamTypeVar(node) => node.range(),
AnyNodeRef::TypeParamTypeVarTuple(node) => node.range(),
AnyNodeRef::TypeParamParamSpec(node) => node.range(),
AnyNodeRef::FString(node) => node.range(),
AnyNodeRef::StringLiteral(node) => node.range(),
AnyNodeRef::BytesLiteral(node) => node.range(),
}
}
}
@ -6701,4 +6912,7 @@ pub enum NodeKind {
TypeParamTypeVar,
TypeParamTypeVarTuple,
TypeParamParamSpec,
FString,
StringLiteral,
BytesLiteral,
}

528
crates/ruff_python_ast/src/nodes.rs
View file

@ -1,14 +1,17 @@
#![allow(clippy::derive_partial_eq_without_eq)]
use itertools::Itertools;
use std::cell::OnceCell;
use std::fmt;
use std::fmt::Debug;
use std::ops::Deref;
use crate::{int, LiteralExpressionRef};
use itertools::Either::{Left, Right};
use itertools::Itertools;
use ruff_text_size::{Ranged, TextRange, TextSize};
use crate::{int, LiteralExpressionRef};
/// See also [mod](https://docs.python.org/3/library/ast.html#ast.mod)
#[derive(Clone, Debug, PartialEq, is_macro::Is)]
pub enum Mod {
@ -640,6 +643,7 @@ impl Expr {
)
}
/// Returns [`LiteralExpressionRef`] if the expression is a literal expression.
pub fn as_literal_expr(&self) -> Option<LiteralExpressionRef<'_>> {
match self {
Expr::StringLiteral(expr) => Some(LiteralExpressionRef::StringLiteral(expr)),
@ -651,24 +655,6 @@ impl Expr {
_ => None,
}
}
pub fn is_implicit_concatenated_string(&self) -> bool {
match self {
Expr::StringLiteral(ExprStringLiteral {
implicit_concatenated,
..
})
| Expr::BytesLiteral(ExprBytesLiteral {
implicit_concatenated,
..
})
| Expr::FString(ExprFString {
implicit_concatenated,
..
}) => *implicit_concatenated,
_ => false,
}
}
}
/// An AST node used to represent a IPython escape command at the expression level.
@ -984,13 +970,17 @@ pub struct DebugText {
pub trailing: String,
}
/// See also [JoinedStr](https://docs.python.org/3/library/ast.html#ast.JoinedStr)
/// An AST node used to represent an f-string.
///
/// This type differs from the original Python AST ([JoinedStr]) in that it
/// doesn't join the implicitly concatenated parts into a single string. Instead,
/// it keeps them separate and provide various methods to access the parts.
///
/// [JoinedStr]: https://docs.python.org/3/library/ast.html#ast.JoinedStr
#[derive(Clone, Debug, PartialEq)]
pub struct ExprFString {
pub range: TextRange,
pub values: Vec<Expr>,
/// Whether the f-string contains multiple string tokens that were implicitly concatenated.
pub implicit_concatenated: bool,
pub value: FStringValue,
}
impl From<ExprFString> for Expr {
@ -999,12 +989,155 @@ impl From<ExprFString> for Expr {
}
}
/// The value representing an [`ExprFString`].
#[derive(Clone, Debug, PartialEq)]
pub struct FStringValue {
inner: FStringValueInner,
}
impl FStringValue {
/// Creates a new f-string with the given value.
pub fn single(value: FString) -> Self {
Self {
inner: FStringValueInner::Single(FStringPart::FString(value)),
}
}
/// Creates a new f-string with the given values that represents an implicitly
/// concatenated f-string.
///
/// # Panics
///
/// Panics if `values` is less than 2. Use [`FStringValue::single`] instead.
pub fn concatenated(values: Vec<FStringPart>) -> Self {
assert!(values.len() > 1);
Self {
inner: FStringValueInner::Concatenated(values),
}
}
/// Returns `true` if the f-string is implicitly concatenated, `false` otherwise.
pub fn is_implicit_concatenated(&self) -> bool {
matches!(self.inner, FStringValueInner::Concatenated(_))
}
/// Returns an iterator over all the [`FStringPart`]s contained in this value.
pub fn parts(&self) -> impl Iterator<Item = &FStringPart> {
match &self.inner {
FStringValueInner::Single(part) => Left(std::iter::once(part)),
FStringValueInner::Concatenated(parts) => Right(parts.iter()),
}
}
/// Returns an iterator over all the [`FStringPart`]s contained in this value
/// that allows modification.
pub(crate) fn parts_mut(&mut self) -> impl Iterator<Item = &mut FStringPart> {
match &mut self.inner {
FStringValueInner::Single(part) => Left(std::iter::once(part)),
FStringValueInner::Concatenated(parts) => Right(parts.iter_mut()),
}
}
/// Returns an iterator over the [`StringLiteral`] parts contained in this value.
///
/// Note that this doesn't nest into the f-string parts. For example,
///
/// ```python
/// "foo" f"bar {x}" "baz" f"qux"
/// ```
///
/// Here, the string literal parts returned would be `"foo"` and `"baz"`.
pub fn literals(&self) -> impl Iterator<Item = &StringLiteral> {
self.parts().filter_map(|part| part.as_literal())
}
/// Returns an iterator over the [`FString`] parts contained in this value.
///
/// Note that this doesn't nest into the f-string parts. For example,
///
/// ```python
/// "foo" f"bar {x}" "baz" f"qux"
/// ```
///
/// Here, the f-string parts returned would be `f"bar {x}"` and `f"qux"`.
pub fn f_strings(&self) -> impl Iterator<Item = &FString> {
self.parts().filter_map(|part| part.as_f_string())
}
/// Returns an iterator over all the f-string elements contained in this value.
///
/// An f-string element is what makes up an [`FString`] i.e., it is either a
/// string literal or an expression. In the following example,
///
/// ```python
/// "foo" f"bar {x}" "baz" f"qux"
/// ```
///
/// The f-string elements returned would be string literal (`"bar "`),
/// expression (`x`) and string literal (`"qux"`).
pub fn elements(&self) -> impl Iterator<Item = &Expr> {
self.f_strings().flat_map(|fstring| fstring.values.iter())
}
}
/// An internal representation of [`FStringValue`].
#[derive(Clone, Debug, PartialEq)]
enum FStringValueInner {
/// A single f-string i.e., `f"foo"`.
///
/// This is always going to be `FStringPart::FString` variant which is
/// maintained by the `FStringValue::single` constructor.
Single(FStringPart),
/// An implicitly concatenated f-string i.e., `"foo" f"bar {x}"`.
Concatenated(Vec<FStringPart>),
}
/// An f-string part which is either a string literal or an f-string.
#[derive(Clone, Debug, PartialEq, is_macro::Is)]
pub enum FStringPart {
Literal(StringLiteral),
FString(FString),
}
impl Ranged for FStringPart {
fn range(&self) -> TextRange {
match self {
FStringPart::Literal(string_literal) => string_literal.range(),
FStringPart::FString(f_string) => f_string.range(),
}
}
}
/// An AST node that represents a single f-string which is part of an [`ExprFString`].
#[derive(Clone, Debug, PartialEq)]
pub struct FString {
pub range: TextRange,
pub values: Vec<Expr>,
}
impl Ranged for FString {
fn range(&self) -> TextRange {
self.range
}
}
impl From<FString> for Expr {
fn from(payload: FString) -> Self {
ExprFString {
range: payload.range,
value: FStringValue::single(payload),
}
.into()
}
}
/// An AST node that represents either a single string literal or an implicitly
/// concatenated string literals.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct ExprStringLiteral {
pub range: TextRange,
pub value: String,
pub unicode: bool,
pub implicit_concatenated: bool,
pub value: StringLiteralValue,
}
impl From<ExprStringLiteral> for Expr {
@ -1019,7 +1152,134 @@ impl Ranged for ExprStringLiteral {
}
}
impl Deref for ExprStringLiteral {
/// The value representing a [`ExprStringLiteral`].
#[derive(Clone, Debug, Default, PartialEq)]
pub struct StringLiteralValue {
inner: StringLiteralValueInner,
}
impl StringLiteralValue {
/// Creates a new single string literal with the given value.
pub fn single(string: StringLiteral) -> Self {
Self {
inner: StringLiteralValueInner::Single(string),
}
}
/// Creates a new string literal with the given values that represents an
/// implicitly concatenated strings.
///
/// # Panics
///
/// Panics if `strings` is less than 2. Use [`StringLiteralValue::single`]
/// instead.
pub fn concatenated(strings: Vec<StringLiteral>) -> Self {
assert!(strings.len() > 1);
Self {
inner: StringLiteralValueInner::Concatenated(ConcatenatedStringLiteral {
strings,
value: OnceCell::new(),
}),
}
}
/// Returns `true` if the string literal is implicitly concatenated.
pub const fn is_implicit_concatenated(&self) -> bool {
matches!(self.inner, StringLiteralValueInner::Concatenated(_))
}
/// Returns `true` if the string literal is a unicode string.
///
/// For an implicitly concatenated string, it returns `true` only if the first
/// string literal is a unicode string.
pub fn is_unicode(&self) -> bool {
self.parts().next().map_or(false, |part| part.unicode)
}
/// Returns an iterator over all the [`StringLiteral`] parts contained in this value.
pub fn parts(&self) -> impl Iterator<Item = &StringLiteral> {
match &self.inner {
StringLiteralValueInner::Single(value) => Left(std::iter::once(value)),
StringLiteralValueInner::Concatenated(value) => Right(value.strings.iter()),
}
}
/// Returns an iterator over all the [`StringLiteral`] parts contained in this value
/// that allows modification.
pub(crate) fn parts_mut(&mut self) -> impl Iterator<Item = &mut StringLiteral> {
match &mut self.inner {
StringLiteralValueInner::Single(value) => Left(std::iter::once(value)),
StringLiteralValueInner::Concatenated(value) => Right(value.strings.iter_mut()),
}
}
/// Returns the concatenated string value as a [`str`].
pub fn as_str(&self) -> &str {
match &self.inner {
StringLiteralValueInner::Single(value) => value.as_str(),
StringLiteralValueInner::Concatenated(value) => value.as_str(),
}
}
}
impl PartialEq<str> for StringLiteralValue {
fn eq(&self, other: &str) -> bool {
self.as_str() == other
}
}
impl PartialEq<String> for StringLiteralValue {
fn eq(&self, other: &String) -> bool {
self.as_str() == other
}
}
impl Deref for StringLiteralValue {
type Target = str;
fn deref(&self) -> &Self::Target {
self.as_str()
}
}
impl fmt::Display for StringLiteralValue {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(self.as_str())
}
}
/// An internal representation of [`StringLiteralValue`].
#[derive(Clone, Debug, PartialEq)]
enum StringLiteralValueInner {
/// A single string literal i.e., `"foo"`.
Single(StringLiteral),
/// An implicitly concatenated string literals i.e., `"foo" "bar"`.
Concatenated(ConcatenatedStringLiteral),
}
impl Default for StringLiteralValueInner {
fn default() -> Self {
Self::Single(StringLiteral::default())
}
}
/// An AST node that represents a single string literal which is part of an
/// [`ExprStringLiteral`].
#[derive(Clone, Debug, Default, PartialEq)]
pub struct StringLiteral {
pub range: TextRange,
pub value: String,
pub unicode: bool,
}
impl Ranged for StringLiteral {
fn range(&self) -> TextRange {
self.range
}
}
impl Deref for StringLiteral {
type Target = str;
fn deref(&self) -> &Self::Target {
@ -1027,11 +1287,69 @@ impl Deref for ExprStringLiteral {
}
}
impl StringLiteral {
/// Extracts a string slice containing the entire `String`.
pub fn as_str(&self) -> &str {
self
}
}
impl From<StringLiteral> for Expr {
fn from(payload: StringLiteral) -> Self {
ExprStringLiteral {
range: payload.range,
value: StringLiteralValue::single(payload),
}
.into()
}
}
/// An internal representation of [`StringLiteral`] that represents an
/// implicitly concatenated string.
#[derive(Clone)]
struct ConcatenatedStringLiteral {
/// Each string literal that makes up the concatenated string.
strings: Vec<StringLiteral>,
/// The concatenated string value.
value: OnceCell<String>,
}
impl ConcatenatedStringLiteral {
/// Extracts a string slice containing the entire concatenated string.
fn as_str(&self) -> &str {
self.value
.get_or_init(|| self.strings.iter().map(StringLiteral::as_str).collect())
}
}
impl PartialEq for ConcatenatedStringLiteral {
fn eq(&self, other: &Self) -> bool {
if self.strings.len() != other.strings.len() {
return false;
}
// The `zip` here is safe because we have checked the length of both parts.
self.strings
.iter()
.zip(other.strings.iter())
.all(|(s1, s2)| s1 == s2)
}
}
impl Debug for ConcatenatedStringLiteral {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("ConcatenatedStringLiteral")
.field("strings", &self.strings)
.field("value", &self.as_str())
.finish()
}
}
/// An AST node that represents either a single bytes literal or an implicitly
/// concatenated bytes literals.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct ExprBytesLiteral {
pub range: TextRange,
pub value: Vec<u8>,
pub implicit_concatenated: bool,
pub value: BytesLiteralValue,
}
impl From<ExprBytesLiteral> for Expr {
@ -1046,6 +1364,140 @@ impl Ranged for ExprBytesLiteral {
}
}
/// The value representing a [`ExprBytesLiteral`].
#[derive(Clone, Debug, Default, PartialEq)]
pub struct BytesLiteralValue {
inner: BytesLiteralValueInner,
}
impl BytesLiteralValue {
/// Creates a new single bytes literal with the given value.
pub fn single(value: BytesLiteral) -> Self {
Self {
inner: BytesLiteralValueInner::Single(value),
}
}
/// Creates a new bytes literal with the given values that represents an
/// implicitly concatenated bytes.
///
/// # Panics
///
/// Panics if `values` is less than 2. Use [`BytesLiteralValue::single`]
/// instead.
pub fn concatenated(values: Vec<BytesLiteral>) -> Self {
assert!(values.len() > 1);
Self {
inner: BytesLiteralValueInner::Concatenated(values),
}
}
/// Returns `true` if the bytes literal is implicitly concatenated.
pub const fn is_implicit_concatenated(&self) -> bool {
matches!(self.inner, BytesLiteralValueInner::Concatenated(_))
}
/// Returns an iterator over all the [`BytesLiteral`] parts contained in this value.
pub fn parts(&self) -> impl Iterator<Item = &BytesLiteral> {
match &self.inner {
BytesLiteralValueInner::Single(value) => Left(std::iter::once(value)),
BytesLiteralValueInner::Concatenated(values) => Right(values.iter()),
}
}
/// Returns an iterator over all the [`BytesLiteral`] parts contained in this value
/// that allows modification.
pub(crate) fn parts_mut(&mut self) -> impl Iterator<Item = &mut BytesLiteral> {
match &mut self.inner {
BytesLiteralValueInner::Single(value) => Left(std::iter::once(value)),
BytesLiteralValueInner::Concatenated(values) => Right(values.iter_mut()),
}
}
/// Returns `true` if the concatenated bytes has a length of zero.
pub fn is_empty(&self) -> bool {
self.parts().all(|part| part.is_empty())
}
/// Returns the length of the concatenated bytes.
pub fn len(&self) -> usize {
self.parts().map(|part| part.len()).sum()
}
/// Returns an iterator over the bytes of the concatenated bytes.
fn bytes(&self) -> impl Iterator<Item = u8> + '_ {
self.parts()
.flat_map(|part| part.as_slice().iter().copied())
}
}
impl PartialEq<[u8]> for BytesLiteralValue {
fn eq(&self, other: &[u8]) -> bool {
if self.len() != other.len() {
return false;
}
// The `zip` here is safe because we have checked the length of both parts.
self.bytes()
.zip(other.iter().copied())
.all(|(b1, b2)| b1 == b2)
}
}
/// An internal representation of [`BytesLiteralValue`].
#[derive(Clone, Debug, PartialEq)]
enum BytesLiteralValueInner {
/// A single bytes literal i.e., `b"foo"`.
Single(BytesLiteral),
/// An implicitly concatenated bytes literals i.e., `b"foo" b"bar"`.
Concatenated(Vec<BytesLiteral>),
}
impl Default for BytesLiteralValueInner {
fn default() -> Self {
Self::Single(BytesLiteral::default())
}
}
/// An AST node that represents a single bytes literal which is part of an
/// [`ExprBytesLiteral`].
#[derive(Clone, Debug, Default, PartialEq)]
pub struct BytesLiteral {
pub range: TextRange,
pub value: Vec<u8>,
}
impl Ranged for BytesLiteral {
fn range(&self) -> TextRange {
self.range
}
}
impl Deref for BytesLiteral {
type Target = [u8];
fn deref(&self) -> &Self::Target {
self.value.as_slice()
}
}
impl BytesLiteral {
/// Extracts a byte slice containing the entire [`BytesLiteral`].
pub fn as_slice(&self) -> &[u8] {
self
}
}
impl From<BytesLiteral> for Expr {
fn from(payload: BytesLiteral) -> Self {
ExprBytesLiteral {
range: payload.range,
value: BytesLiteralValue::single(payload),
}
.into()
}
}
#[derive(Clone, Debug, PartialEq)]
pub struct ExprNumberLiteral {
pub range: TextRange,
@ -3088,12 +3540,12 @@ impl Ranged for crate::Expr {
Self::Call(node) => node.range(),
Self::FormattedValue(node) => node.range(),
Self::FString(node) => node.range(),
Expr::StringLiteral(node) => node.range(),
Expr::BytesLiteral(node) => node.range(),
Expr::NumberLiteral(node) => node.range(),
Expr::BooleanLiteral(node) => node.range(),
Expr::NoneLiteral(node) => node.range(),
Expr::EllipsisLiteral(node) => node.range(),
Self::StringLiteral(node) => node.range(),
Self::BytesLiteral(node) => node.range(),
Self::NumberLiteral(node) => node.range(),
Self::BooleanLiteral(node) => node.range(),
Self::NoneLiteral(node) => node.range(),
Self::EllipsisLiteral(node) => node.range(),
Self::Attribute(node) => node.range(),
Self::Subscript(node) => node.range(),
Self::Starred(node) => node.range(),
@ -3101,7 +3553,7 @@ impl Ranged for crate::Expr {
Self::List(node) => node.range(),
Self::Tuple(node) => node.range(),
Self::Slice(node) => node.range(),
Expr::IpyEscapeCommand(node) => node.range(),
Self::IpyEscapeCommand(node) => node.range(),
}
}
}

62
crates/ruff_python_ast/src/visitor.rs
View file

@ -4,10 +4,10 @@ pub mod preorder;
pub mod transformer;
use crate::{
self as ast, Alias, Arguments, BoolOp, CmpOp, Comprehension, Decorator, ElifElseClause,
ExceptHandler, Expr, ExprContext, Keyword, MatchCase, Operator, Parameter, Parameters, Pattern,
PatternArguments, PatternKeyword, Stmt, TypeParam, TypeParamTypeVar, TypeParams, UnaryOp,
WithItem,
self as ast, Alias, Arguments, BoolOp, BytesLiteral, CmpOp, Comprehension, Decorator,
ElifElseClause, ExceptHandler, Expr, ExprContext, FString, FStringPart, Keyword, MatchCase,
Operator, Parameter, Parameters, Pattern, PatternArguments, PatternKeyword, Stmt,
StringLiteral, TypeParam, TypeParamTypeVar, TypeParams, UnaryOp, WithItem,
};
/// A trait for AST visitors. Visits all nodes in the AST recursively in evaluation-order.
@ -98,6 +98,15 @@ pub trait Visitor<'a> {
fn visit_elif_else_clause(&mut self, elif_else_clause: &'a ElifElseClause) {
walk_elif_else_clause(self, elif_else_clause);
}
fn visit_f_string(&mut self, f_string: &'a FString) {
walk_f_string(self, f_string);
}
fn visit_string_literal(&mut self, string_literal: &'a StringLiteral) {
walk_string_literal(self, string_literal);
}
fn visit_bytes_literal(&mut self, bytes_literal: &'a BytesLiteral) {
walk_bytes_literal(self, bytes_literal);
}
}
pub fn walk_body<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, body: &'a [Stmt]) {
@ -475,14 +484,27 @@ pub fn walk_expr<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, expr: &'a Expr) {
visitor.visit_format_spec(expr);
}
}
Expr::FString(ast::ExprFString { values, .. }) => {
for expr in values {
visitor.visit_expr(expr);
Expr::FString(ast::ExprFString { value, .. }) => {
for part in value.parts() {
match part {
FStringPart::Literal(string_literal) => {
visitor.visit_string_literal(string_literal);
}
FStringPart::FString(f_string) => visitor.visit_f_string(f_string),
}
}
}
Expr::StringLiteral(_)
| Expr::BytesLiteral(_)
| Expr::NumberLiteral(_)
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => {
for string_literal in value.parts() {
visitor.visit_string_literal(string_literal);
}
}
Expr::BytesLiteral(ast::ExprBytesLiteral { value, .. }) => {
for bytes_literal in value.parts() {
visitor.visit_bytes_literal(bytes_literal);
}
}
Expr::NumberLiteral(_)
| Expr::BooleanLiteral(_)
| Expr::NoneLiteral(_)
| Expr::EllipsisLiteral(_) => {}
@ -576,6 +598,12 @@ pub fn walk_except_handler<'a, V: Visitor<'a> + ?Sized>(
}
}
pub fn walk_f_string<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, f_string: &'a FString) {
for expr in &f_string.values {
visitor.visit_expr(expr);
}
}
pub fn walk_format_spec<'a, V: Visitor<'a> + ?Sized>(visitor: &mut V, format_spec: &'a Expr) {
visitor.visit_expr(format_spec);
}
@ -746,3 +774,17 @@ pub fn walk_cmp_op<'a, V: Visitor<'a> + ?Sized>(visitor: &V, cmp_op: &'a CmpOp)
#[allow(unused_variables)]
pub fn walk_alias<'a, V: Visitor<'a> + ?Sized>(visitor: &V, alias: &'a Alias) {}
#[allow(unused_variables)]
pub fn walk_string_literal<'a, V: Visitor<'a> + ?Sized>(
visitor: &V,
string_literal: &'a StringLiteral,
) {
}
#[allow(unused_variables)]
pub fn walk_bytes_literal<'a, V: Visitor<'a> + ?Sized>(
visitor: &V,
bytes_literal: &'a BytesLiteral,
) {
}

58
crates/ruff_python_ast/src/visitor/preorder.rs
View file

@ -1,7 +1,8 @@
use crate::{
Alias, Arguments, BoolOp, CmpOp, Comprehension, Decorator, ElifElseClause, ExceptHandler, Expr,
Keyword, MatchCase, Mod, Operator, Parameter, ParameterWithDefault, Parameters, Pattern,
PatternArguments, PatternKeyword, Singleton, Stmt, TypeParam, TypeParams, UnaryOp, WithItem,
Alias, Arguments, BoolOp, BytesLiteral, CmpOp, Comprehension, Decorator, ElifElseClause,
ExceptHandler, Expr, FString, Keyword, MatchCase, Mod, Operator, Parameter,
ParameterWithDefault, Parameters, Pattern, PatternArguments, PatternKeyword, Singleton, Stmt,
StringLiteral, TypeParam, TypeParams, UnaryOp, WithItem,
};
use crate::{AnyNodeRef, AstNode};
@ -152,6 +153,21 @@ pub trait PreorderVisitor<'a> {
fn visit_elif_else_clause(&mut self, elif_else_clause: &'a ElifElseClause) {
walk_elif_else_clause(self, elif_else_clause);
}
#[inline]
fn visit_f_string(&mut self, f_string: &'a FString) {
walk_f_string(self, f_string);
}
#[inline]
fn visit_string_literal(&mut self, string_literal: &'a StringLiteral) {
walk_string_literal(self, string_literal);
}
#[inline]
fn visit_bytes_literal(&mut self, bytes_literal: &'a BytesLiteral) {
walk_bytes_literal(self, bytes_literal);
}
}
pub fn walk_module<'a, V>(visitor: &mut V, module: &'a Mod)
@ -530,6 +546,42 @@ where
{
}
#[inline]
pub fn walk_f_string<'a, V>(visitor: &mut V, f_string: &'a FString)
where
V: PreorderVisitor<'a> + ?Sized,
{
let node = AnyNodeRef::from(f_string);
if visitor.enter_node(node).is_traverse() {
f_string.visit_preorder(visitor);
}
visitor.leave_node(node);
}
#[inline]
pub fn walk_string_literal<'a, V>(visitor: &mut V, string_literal: &'a StringLiteral)
where
V: PreorderVisitor<'a> + ?Sized,
{
let node = AnyNodeRef::from(string_literal);
if visitor.enter_node(node).is_traverse() {
string_literal.visit_preorder(visitor);
}
visitor.leave_node(node);
}
#[inline]
pub fn walk_bytes_literal<'a, V>(visitor: &mut V, bytes_literal: &'a BytesLiteral)
where
V: PreorderVisitor<'a> + ?Sized,
{
let node = AnyNodeRef::from(bytes_literal);
if visitor.enter_node(node).is_traverse() {
bytes_literal.visit_preorder(visitor);
}
visitor.leave_node(node);
}
#[inline]
pub fn walk_alias<'a, V>(visitor: &mut V, alias: &'a Alias)
where

60
crates/ruff_python_ast/src/visitor/transformer.rs
View file

@ -1,8 +1,8 @@
use crate::{
self as ast, Alias, Arguments, BoolOp, CmpOp, Comprehension, Decorator, ElifElseClause,
ExceptHandler, Expr, ExprContext, Keyword, MatchCase, Operator, Parameter, Parameters, Pattern,
PatternArguments, PatternKeyword, Stmt, TypeParam, TypeParamTypeVar, TypeParams, UnaryOp,
WithItem,
self as ast, Alias, Arguments, BoolOp, BytesLiteral, CmpOp, Comprehension, Decorator,
ElifElseClause, ExceptHandler, Expr, ExprContext, FString, Keyword, MatchCase, Operator,
Parameter, Parameters, Pattern, PatternArguments, PatternKeyword, Stmt, StringLiteral,
TypeParam, TypeParamTypeVar, TypeParams, UnaryOp, WithItem,
};
/// A trait for transforming ASTs. Visits all nodes in the AST recursively in evaluation-order.
@ -85,6 +85,15 @@ pub trait Transformer {
fn visit_elif_else_clause(&self, elif_else_clause: &mut ElifElseClause) {
walk_elif_else_clause(self, elif_else_clause);
}
fn visit_f_string(&self, f_string: &mut FString) {
walk_f_string(self, f_string);
}
fn visit_string_literal(&self, string_literal: &mut StringLiteral) {
walk_string_literal(self, string_literal);
}
fn visit_bytes_literal(&self, bytes_literal: &mut BytesLiteral) {
walk_bytes_literal(self, bytes_literal);
}
}
pub fn walk_body<V: Transformer + ?Sized>(visitor: &V, body: &mut [Stmt]) {
@ -462,14 +471,29 @@ pub fn walk_expr<V: Transformer + ?Sized>(visitor: &V, expr: &mut Expr) {
visitor.visit_format_spec(expr);
}
}
Expr::FString(ast::ExprFString { values, .. }) => {
for expr in values {
visitor.visit_expr(expr);
Expr::FString(ast::ExprFString { value, .. }) => {
for f_string_part in value.parts_mut() {
match f_string_part {
ast::FStringPart::Literal(string_literal) => {
visitor.visit_string_literal(string_literal);
}
ast::FStringPart::FString(f_string) => {
visitor.visit_f_string(f_string);
}
}
}
}
Expr::StringLiteral(_)
| Expr::BytesLiteral(_)
| Expr::NumberLiteral(_)
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => {
for string_literal in value.parts_mut() {
visitor.visit_string_literal(string_literal);
}
}
Expr::BytesLiteral(ast::ExprBytesLiteral { value, .. }) => {
for bytes_literal in value.parts_mut() {
visitor.visit_bytes_literal(bytes_literal);
}
}
Expr::NumberLiteral(_)
| Expr::BooleanLiteral(_)
| Expr::NoneLiteral(_)
| Expr::EllipsisLiteral(_) => {}
@ -560,6 +584,12 @@ pub fn walk_except_handler<V: Transformer + ?Sized>(
}
}
pub fn walk_f_string<V: Transformer + ?Sized>(visitor: &V, f_string: &mut FString) {
for expr in &mut f_string.values {
visitor.visit_expr(expr);
}
}
pub fn walk_format_spec<V: Transformer + ?Sized>(visitor: &V, format_spec: &mut Expr) {
visitor.visit_expr(format_spec);
}
@ -730,3 +760,13 @@ pub fn walk_cmp_op<V: Transformer + ?Sized>(visitor: &V, cmp_op: &mut CmpOp) {}
#[allow(unused_variables)]
pub fn walk_alias<V: Transformer + ?Sized>(visitor: &V, alias: &mut Alias) {}
#[allow(unused_variables)]
pub fn walk_string_literal<V: Transformer + ?Sized>(
visitor: &V,
string_literal: &mut StringLiteral,
) {
}
#[allow(unused_variables)]
pub fn walk_bytes_literal<V: Transformer + ?Sized>(visitor: &V, bytes_literal: &mut BytesLiteral) {}

62
crates/ruff_python_codegen/src/generator.rs
View file

@ -1081,17 +1081,18 @@ impl<'a> Generator<'a> {
*conversion,
format_spec.as_deref(),
),
Expr::FString(ast::ExprFString { values, .. }) => {
self.unparse_f_string(values, false);
Expr::FString(ast::ExprFString { value, .. }) => {
self.unparse_f_string_value(value, false);
}
Expr::StringLiteral(ast::ExprStringLiteral { value, unicode, .. }) => {
if *unicode {
self.p("u");
}
self.p_str_repr(value);
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => {
self.unparse_string_literal_value(value);
}
Expr::BytesLiteral(ast::ExprBytesLiteral { value, .. }) => {
self.p_bytes_repr(value);
let mut first = true;
for bytes_literal in value.parts() {
self.p_delim(&mut first, " ");
self.p_bytes_repr(&bytes_literal.value);
}
}
Expr::NumberLiteral(ast::ExprNumberLiteral { value, .. }) => {
static INF_STR: &str = "1e309";
@ -1275,6 +1276,36 @@ impl<'a> Generator<'a> {
}
}
fn unparse_string_literal(&mut self, string_literal: &ast::StringLiteral) {
if string_literal.unicode {
self.p("u");
}
self.p_str_repr(&string_literal.value);
}
fn unparse_string_literal_value(&mut self, value: &ast::StringLiteralValue) {
let mut first = true;
for string_literal in value.parts() {
self.p_delim(&mut first, " ");
self.unparse_string_literal(string_literal);
}
}
fn unparse_f_string_value(&mut self, value: &ast::FStringValue, is_spec: bool) {
let mut first = true;
for f_string_part in value.parts() {
self.p_delim(&mut first, " ");
match f_string_part {
ast::FStringPart::Literal(string_literal) => {
self.unparse_string_literal(string_literal);
}
ast::FStringPart::FString(f_string) => {
self.unparse_f_string(&f_string.values, is_spec);
}
}
}
}
fn unparse_f_string_body(&mut self, values: &[Expr], is_spec: bool) {
for value in values {
self.unparse_f_string_elem(value, is_spec);
@ -1325,10 +1356,10 @@ impl<'a> Generator<'a> {
fn unparse_f_string_elem(&mut self, expr: &Expr, is_spec: bool) {
match expr {
Expr::StringLiteral(ast::ExprStringLiteral { value, .. }) => {
self.unparse_f_string_literal(value);
self.unparse_f_string_literal(value.as_str());
}
Expr::FString(ast::ExprFString { values, .. }) => {
self.unparse_f_string(values, is_spec);
Expr::FString(ast::ExprFString { value, .. }) => {
self.unparse_f_string_value(value, is_spec);
}
Expr::FormattedValue(ast::ExprFormattedValue {
value,
@ -1678,7 +1709,7 @@ class Foo:
assert_eq!(round_trip(r"u'hello'"), r#"u"hello""#);
assert_eq!(round_trip(r"r'hello'"), r#""hello""#);
assert_eq!(round_trip(r"b'hello'"), r#"b"hello""#);
assert_eq!(round_trip(r#"("abc" "def" "ghi")"#), r#""abcdefghi""#);
assert_eq!(round_trip(r#"("abc" "def" "ghi")"#), r#""abc" "def" "ghi""#);
assert_eq!(round_trip(r#""he\"llo""#), r#"'he"llo'"#);
assert_eq!(round_trip(r#"f"abc{'def'}{1}""#), r#"f"abc{'def'}{1}""#);
assert_eq!(round_trip(r#"f'abc{"def"}{1}'"#), r#"f"abc{'def'}{1}""#);
@ -1693,6 +1724,13 @@ class Foo:
assert_round_trip!(r#"f"{a=!r:0.05f}""#);
}
#[test]
fn implicit_string_concatenation() {
assert_round_trip!(r#""first" "second" "third""#);
assert_round_trip!(r#"b"first" b"second" b"third""#);
assert_round_trip!(r#""first" "second" f"third {var}""#);
}
#[test]
fn indent() {
assert_eq!(

4
crates/ruff_python_formatter/src/comments/placement.rs
View file

@ -283,13 +283,13 @@ fn handle_enclosed_comment<'a>(
AnyNodeRef::StmtWith(with_) => handle_with_comment(comment, with_),
AnyNodeRef::ExprCall(_) => handle_call_comment(comment),
AnyNodeRef::ExprStringLiteral(_) => {
if let Some(AnyNodeRef::ExprFString(fstring)) = comment.enclosing_parent() {
if let Some(AnyNodeRef::FString(fstring)) = comment.enclosing_parent() {
CommentPlacement::dangling(fstring, comment)
} else {
CommentPlacement::Default(comment)
}
}
AnyNodeRef::ExprFString(fstring) => CommentPlacement::dangling(fstring, comment),
AnyNodeRef::FString(fstring) => CommentPlacement::dangling(fstring, comment),
AnyNodeRef::ExprList(_)
| AnyNodeRef::ExprSet(_)
| AnyNodeRef::ExprListComp(_)

8
crates/ruff_python_formatter/src/expression/expr_bin_op.rs
View file

@ -35,13 +35,13 @@ impl NeedsParentheses for ExprBinOp {
) -> OptionalParentheses {
if parent.is_expr_await() {
OptionalParentheses::Always
} else if self.left.is_literal_expr() {
} else if let Some(literal_expr) = self.left.as_literal_expr() {
// Multiline strings are guaranteed to never fit, avoid adding unnecessary parentheses
if !self.left.is_implicit_concatenated_string()
&& is_multiline_string(self.left.as_ref().into(), context.source())
if !literal_expr.is_implicit_concatenated()
&& is_multiline_string(literal_expr.into(), context.source())
&& has_parentheses(&self.right, context).is_some()
&& !context.comments().has_dangling(self)
&& !context.comments().has(self.left.as_ref())
&& !context.comments().has(literal_expr)
&& !context.comments().has(self.right.as_ref())
{
OptionalParentheses::Never

2
crates/ruff_python_formatter/src/expression/expr_bytes_literal.rs
View file

@ -31,7 +31,7 @@ impl NeedsParentheses for ExprBytesLiteral {
_parent: AnyNodeRef,
context: &PyFormatContext,
) -> OptionalParentheses {
if self.implicit_concatenated {
if self.value.is_implicit_concatenated() {
OptionalParentheses::Multiline
} else if is_multiline_string(self.into(), context.source()) {
OptionalParentheses::Never

8
crates/ruff_python_formatter/src/expression/expr_compare.rs
View file

@ -37,11 +37,11 @@ impl NeedsParentheses for ExprCompare {
) -> OptionalParentheses {
if parent.is_expr_await() {
OptionalParentheses::Always
} else if self.left.is_literal_expr() {
} else if let Some(literal_expr) = self.left.as_literal_expr() {
// Multiline strings are guaranteed to never fit, avoid adding unnecessary parentheses
if !self.left.is_implicit_concatenated_string()
&& is_multiline_string(self.left.as_ref().into(), context.source())
&& !context.comments().has(self.left.as_ref())
if !literal_expr.is_implicit_concatenated()
&& is_multiline_string(literal_expr.into(), context.source())
&& !context.comments().has(literal_expr)
&& self.comparators.first().is_some_and(|right| {
has_parentheses(right, context).is_some() && !context.comments().has(right)
})

2
crates/ruff_python_formatter/src/expression/expr_f_string.rs
View file

@ -34,7 +34,7 @@ impl NeedsParentheses for ExprFString {
_parent: AnyNodeRef,
context: &PyFormatContext,
) -> OptionalParentheses {
if self.implicit_concatenated {
if self.value.is_implicit_concatenated() {
OptionalParentheses::Multiline
} else if memchr2(b'\n', b'\r', context.source()[self.range].as_bytes()).is_none() {
OptionalParentheses::BestFit

2
crates/ruff_python_formatter/src/expression/expr_string_literal.rs
View file

@ -46,7 +46,7 @@ impl NeedsParentheses for ExprStringLiteral {
_parent: AnyNodeRef,
context: &PyFormatContext,
) -> OptionalParentheses {
if self.implicit_concatenated {
if self.value.is_implicit_concatenated() {
OptionalParentheses::Multiline
} else if is_multiline_string(self.into(), context.source()) {
OptionalParentheses::Never

23
crates/ruff_python_formatter/src/expression/mod.rs
View file

@ -804,18 +804,17 @@ impl<'input> CanOmitOptionalParenthesesVisitor<'input> {
return;
}
Expr::StringLiteral(ast::ExprStringLiteral {
implicit_concatenated: true,
..
})
| Expr::BytesLiteral(ast::ExprBytesLiteral {
implicit_concatenated: true,
..
})
| Expr::FString(ast::ExprFString {
implicit_concatenated: true,
..
}) => {
Expr::StringLiteral(ast::ExprStringLiteral { value, .. })
if value.is_implicit_concatenated() =>
{
self.update_max_precedence(OperatorPrecedence::String);
}
Expr::BytesLiteral(ast::ExprBytesLiteral { value, .. })
if value.is_implicit_concatenated() =>
{
self.update_max_precedence(OperatorPrecedence::String);
}
Expr::FString(ast::ExprFString { value, .. }) if value.is_implicit_concatenated() => {
self.update_max_precedence(OperatorPrecedence::String);
}

253
crates/ruff_python_formatter/src/expression/string.rs
View file

@ -2,13 +2,11 @@ use std::borrow::Cow;
use bitflags::bitflags;
use ruff_formatter::{format_args, write, FormatError};
use ruff_formatter::{format_args, write};
use ruff_python_ast::AnyNodeRef;
use ruff_python_ast::{
self as ast, ExprBytesLiteral, ExprFString, ExprStringLiteral, ExpressionRef,
};
use ruff_python_parser::lexer::{lex_starts_at, LexicalError, LexicalErrorType};
use ruff_python_parser::{Mode, Tok};
use ruff_source_file::Locator;
use ruff_text_size::{Ranged, TextLen, TextRange, TextSize};
@ -52,7 +50,7 @@ impl<'a> AnyString<'a> {
.trim_start_matches(|c| c != '"' && c != '\'');
let triple_quoted =
unprefixed.starts_with(r#"""""#) || unprefixed.starts_with(r"'''");
if f_string.values.iter().any(|value| match value {
if f_string.value.elements().any(|value| match value {
Expr::FormattedValue(ast::ExprFormattedValue { range, .. }) => {
let string_content = locator.slice(*range);
if triple_quoted {
@ -74,18 +72,29 @@ impl<'a> AnyString<'a> {
/// Returns `true` if the string is implicitly concatenated.
pub(super) fn is_implicit_concatenated(&self) -> bool {
match self {
Self::String(ExprStringLiteral {
implicit_concatenated,
..
}) => *implicit_concatenated,
Self::Bytes(ExprBytesLiteral {
implicit_concatenated,
..
}) => *implicit_concatenated,
Self::FString(ExprFString {
implicit_concatenated,
..
}) => *implicit_concatenated,
Self::String(ExprStringLiteral { value, .. }) => value.is_implicit_concatenated(),
Self::Bytes(ExprBytesLiteral { value, .. }) => value.is_implicit_concatenated(),
Self::FString(ExprFString { value, .. }) => value.is_implicit_concatenated(),
}
}
fn parts(&self) -> Vec<AnyStringPart<'a>> {
match self {
Self::String(ExprStringLiteral { value, .. }) => {
value.parts().map(AnyStringPart::String).collect()
}
Self::Bytes(ExprBytesLiteral { value, .. }) => {
value.parts().map(AnyStringPart::Bytes).collect()
}
Self::FString(ExprFString { value, .. }) => value
.parts()
.map(|f_string_part| match f_string_part {
ast::FStringPart::Literal(string_literal) => {
AnyStringPart::String(string_literal)
}
ast::FStringPart::FString(f_string) => AnyStringPart::FString(f_string),
})
.collect(),
}
}
}
@ -120,6 +129,33 @@ impl<'a> From<&AnyString<'a>> for ExpressionRef<'a> {
}
}
#[derive(Clone, Debug)]
enum AnyStringPart<'a> {
String(&'a ast::StringLiteral),
Bytes(&'a ast::BytesLiteral),
FString(&'a ast::FString),
}
impl<'a> From<&AnyStringPart<'a>> for AnyNodeRef<'a> {
fn from(value: &AnyStringPart<'a>) -> Self {
match value {
AnyStringPart::String(part) => AnyNodeRef::StringLiteral(part),
AnyStringPart::Bytes(part) => AnyNodeRef::BytesLiteral(part),
AnyStringPart::FString(part) => AnyNodeRef::FString(part),
}
}
}
impl Ranged for AnyStringPart<'_> {
fn range(&self) -> TextRange {
match self {
Self::String(part) => part.range(),
Self::Bytes(part) => part.range(),
Self::FString(part) => part.range(),
}
}
}
pub(super) struct FormatString<'a> {
string: &'a AnyString<'a>,
layout: StringLayout,
@ -185,7 +221,7 @@ impl<'a> Format<PyFormatContext<'_>> for FormatString<'a> {
// comments.
if let AnyString::FString(fstring) = self.string {
let comments = f.context().comments();
fstring.values.iter().for_each(|value| {
fstring.value.elements().for_each(|value| {
comments.mark_verbatim_node_comments_formatted(value.into());
});
}
@ -193,60 +229,6 @@ impl<'a> Format<PyFormatContext<'_>> for FormatString<'a> {
}
}
/// A builder for the f-string range.
///
/// For now, this is limited to the outermost f-string and doesn't support
/// nested f-strings.
#[derive(Debug, Default)]
struct FStringRangeBuilder {
start_location: TextSize,
end_location: TextSize,
nesting: u32,
}
impl FStringRangeBuilder {
fn visit_token(&mut self, token: &Tok, range: TextRange) {
match token {
Tok::FStringStart => {
if self.nesting == 0 {
self.start_location = range.start();
}
self.nesting += 1;
}
Tok::FStringEnd => {
// We can assume that this will never overflow because we know
// that the program once parsed to a valid AST which means that
// the start and end tokens for f-strings are balanced.
self.nesting -= 1;
if self.nesting == 0 {
self.end_location = range.end();
}
}
_ => {}
}
}
/// Returns `true` if the lexer is currently inside of a f-string.
///
/// It'll return `false` once the `FStringEnd` token for the outermost
/// f-string is visited.
const fn in_fstring(&self) -> bool {
self.nesting > 0
}
/// Returns the complete range of the previously visited f-string.
///
/// This method should only be called once the lexer is outside of any
/// f-string otherwise it might return an invalid range.
///
/// It doesn't consume the builder because there can be multiple f-strings
/// throughout the source code.
fn finish(&self) -> TextRange {
debug_assert!(!self.in_fstring());
TextRange::new(self.start_location, self.end_location)
}
}
struct FormatStringContinuation<'a> {
string: &'a AnyString<'a>,
}
@ -262,129 +244,24 @@ impl Format<PyFormatContext<'_>> for FormatStringContinuation<'_> {
let comments = f.context().comments().clone();
let locator = f.context().locator();
let quote_style = f.options().quote_style();
let mut dangling_comments = comments.dangling(self.string);
let string_range = self.string.range();
let string_content = locator.slice(string_range);
// The AST parses implicit concatenation as a single string.
// Call into the lexer to extract the individual chunks and format each string on its own.
// This code does not yet implement the automatic joining of strings that fit on the same line
// because this is a black preview style.
let lexer = lex_starts_at(string_content, Mode::Expression, string_range.start());
// The lexer emits multiple tokens for a single f-string literal. Each token
// will have it's own range but we require the complete range of the f-string.
let mut fstring_range_builder = FStringRangeBuilder::default();
let mut joiner = f.join_with(in_parentheses_only_soft_line_break_or_space());
for token in lexer {
let (token, token_range) = match token {
Ok(spanned) => spanned,
Err(LexicalError {
error: LexicalErrorType::IndentationError,
..
}) => {
// This can happen if the string continuation appears anywhere inside of a parenthesized expression
// because the lexer doesn't know about the parentheses. For example, the following snipped triggers an Indentation error
// ```python
// {
// "key": (
// [],
// 'a'
// 'b'
// 'c'
// )
// }
// ```
// Ignoring the error here is *safe* because we know that the program once parsed to a valid AST.
continue;
}
Err(_) => {
return Err(FormatError::syntax_error(
"Unexpected lexer error in string formatting",
));
}
};
for part in self.string.parts() {
let normalized = StringPart::from_source(part.range(), &locator).normalize(
self.string.quoting(&locator),
&locator,
quote_style,
);
fstring_range_builder.visit_token(&token, token_range);
// We need to ignore all the tokens within the f-string as there can
// be `String` tokens inside it as well. For example,
//
// ```python
// f"foo {'bar'} foo"
// # ^^^^^
// # Ignore any logic for this `String` token
// ```
//
// Here, we're interested in the complete f-string, not the individual
// tokens inside it.
if fstring_range_builder.in_fstring() {
continue;
}
match token {
Tok::String { .. } | Tok::FStringEnd => {
let token_range = if token.is_f_string_end() {
fstring_range_builder.finish()
} else {
token_range
};
// ```python
// (
// "a"
// # leading
// "the comment above"
// )
// ```
let leading_comments_end = dangling_comments
.partition_point(|comment| comment.start() <= token_range.start());
let (leading_part_comments, rest) =
dangling_comments.split_at(leading_comments_end);
// ```python
// (
// "a" # trailing comment
// "the comment above"
// )
// ```
let trailing_comments_end = rest.partition_point(|comment| {
comment.line_position().is_end_of_line()
&& !locator.contains_line_break(TextRange::new(
token_range.end(),
comment.start(),
))
});
let (trailing_part_comments, rest) = rest.split_at(trailing_comments_end);
let part = StringPart::from_source(token_range, &locator);
let normalized =
part.normalize(self.string.quoting(&locator), &locator, quote_style);
joiner.entry(&format_args![
line_suffix_boundary(),
leading_comments(leading_part_comments),
normalized,
trailing_comments(trailing_part_comments)
]);
dangling_comments = rest;
}
Tok::Comment(_)
| Tok::NonLogicalNewline
| Tok::Newline
| Tok::Indent
| Tok::Dedent => continue,
token => unreachable!("Unexpected token {token:?}"),
}
joiner.entry(&format_args![
line_suffix_boundary(),
leading_comments(comments.leading(&part)),
normalized,
trailing_comments(comments.trailing(&part))
]);
}
debug_assert!(dangling_comments.is_empty());
joiner.finish()
}
}

12
crates/ruff_python_formatter/src/statement/suite.rs
View file

@ -569,10 +569,14 @@ impl<'a> DocstringStmt<'a> {
};
match value.as_ref() {
Expr::StringLiteral(value) if !value.implicit_concatenated => Some(DocstringStmt {
docstring: stmt,
suite_kind,
}),
Expr::StringLiteral(ast::ExprStringLiteral { value, .. })
if !value.is_implicit_concatenated() =>
{
Some(DocstringStmt {
docstring: stmt,
suite_kind,
})
}
_ => None,
}
}

26
crates/ruff_python_formatter/tests/normalizer.rs
View file

@ -51,20 +51,16 @@ impl Transformer for Normalizer {
transformer::walk_stmt(self, stmt);
}
fn visit_expr(&self, expr: &mut Expr) {
if let Expr::StringLiteral(string_literal) = expr {
// Normalize a string by (1) stripping any leading and trailing space from each
// line, and (2) removing any blank lines from the start and end of the string.
string_literal.value = string_literal
.value
.lines()
.map(str::trim)
.collect::<Vec<_>>()
.join("\n")
.trim()
.to_owned();
}
transformer::walk_expr(self, expr);
fn visit_string_literal(&self, string_literal: &mut ast::StringLiteral) {
// Normalize a string by (1) stripping any leading and trailing space from each
// line, and (2) removing any blank lines from the start and end of the string.
string_literal.value = string_literal
.value
.lines()
.map(str::trim)
.collect::<Vec<_>>()
.join("\n")
.trim()
.to_owned();
}
}

56
crates/ruff_python_parser/src/python.lalrpop
View file

@ -11,7 +11,7 @@ use crate::{
lexer::{LexicalError, LexicalErrorType},
function::{ArgumentList, parse_arguments, validate_pos_params, validate_arguments},
context::set_context,
string::{StringType, concatenate_strings, parse_fstring_middle, parse_string_literal},
string::{StringType, concatenated_strings, parse_fstring_middle, parse_string_literal},
token::{self, StringKind},
invalid,
};
@ -491,7 +491,7 @@ MatchStatement: ast::Stmt = {
}
)
},
<location:@L> "match" <tuple_location:@L> <elts:TwoOrMore<TestOrStarNamedExpr, ",">> ","? <tuple_end_location:@R> ":" "\n" Indent <cases:MatchCase+> Dedent => {
<location:@L> "match" <tuple_location:@L> <elts:TwoOrMoreSep<TestOrStarNamedExpr, ",">> ","? <tuple_end_location:@R> ":" "\n" Indent <cases:MatchCase+> Dedent => {
let end_location = cases
.last()
.unwrap()
@ -542,7 +542,7 @@ Patterns: ast::Pattern = {
range: (location..end_location).into()
},
),
<location:@L> <patterns:TwoOrMore<Pattern, ",">> ","? <end_location:@R> => {
<location:@L> <patterns:TwoOrMoreSep<Pattern, ",">> ","? <end_location:@R> => {
ast::Pattern::MatchSequence(
ast::PatternMatchSequence {
patterns,
@ -579,7 +579,7 @@ AsPattern: ast::Pattern = {
OrPattern: ast::Pattern = {
<pattern:ClosedPattern> => pattern,
<location:@L> <patterns:TwoOrMore<ClosedPattern, "|">> <end_location:@R> => {
<location:@L> <patterns:TwoOrMoreSep<ClosedPattern, "|">> <end_location:@R> => {
ast::Pattern::MatchOr(
ast::PatternMatchOr { patterns, range: (location..end_location).into() }
)
@ -677,8 +677,12 @@ LiteralPattern: ast::Pattern = {
value: Box::new(value.into()),
range: (location..end_location).into()
}.into(),
<location:@L> <strings:StringLiteral+> <end_location:@R> =>? Ok(ast::PatternMatchValue {
value: Box::new(concatenate_strings(strings, (location..end_location).into())?),
<location:@L> <string:StringLiteral> <end_location:@R> => ast::PatternMatchValue {
value: Box::new(string.into()),
range: (location..end_location).into()
}.into(),
<location:@L> <strings:TwoOrMore<StringLiteral>> <end_location:@R> =>? Ok(ast::PatternMatchValue {
value: Box::new(concatenated_strings(strings, (location..end_location).into())?),
range: (location..end_location).into()
}.into()),
}
@ -721,6 +725,7 @@ ValuePattern: ast::Pattern = {
MappingKey: ast::Expr = {
MatchNameOrAttr,
String,
<e:NumberExpr> => e.into(),
<e:AddOpExpr> => e.into(),
<location:@L> "None" <end_location:@R> => ast::ExprNoneLiteral {
@ -734,7 +739,6 @@ MappingKey: ast::Expr = {
value: false,
range: (location..end_location).into()
}.into(),
<location:@L> <strings:StringLiteralOrFString+> <end_location:@R> =>? Ok(concatenate_strings(strings, (location..end_location).into())?),
}
MatchMappingEntry: (ast::Expr, ast::Pattern) = {
@ -1561,7 +1565,7 @@ SubscriptList: ast::ParenthesizedExpr = {
range: (location..end_location).into(),
}.into()
},
<location:@L> <elts:TwoOrMore<Subscript, ",">> ","? <end_location:@R> => {
<location:@L> <elts:TwoOrMoreSep<Subscript, ",">> ","? <end_location:@R> => {
let elts = elts.into_iter().map(ast::Expr::from).collect();
ast::ExprTuple {
elts,
@ -1587,23 +1591,29 @@ SliceOp: Option<ast::ParenthesizedExpr> = {
<location:@L> ":" <e:Test<"all">?> => e,
}
String: ast::Expr = {
<location:@L> <string:StringLiteralOrFString> => string.into(),
<location:@L> <strings:TwoOrMore<StringLiteralOrFString>> <end_location:@R> =>? {
Ok(concatenated_strings(strings, (location..end_location).into())?)
}
};
StringLiteralOrFString: StringType = {
StringLiteral,
FStringExpr,
};
StringLiteral: StringType = {
<start_location:@L> <string:string> =>? {
<location:@L> <string:string> <end_location:@R> =>? {
let (source, kind, triple_quoted) = string;
Ok(parse_string_literal(&source, kind, triple_quoted, start_location)?)
Ok(parse_string_literal(&source, kind, triple_quoted, (location..end_location).into())?)
}
};
FStringExpr: StringType = {
<location:@L> FStringStart <values:FStringMiddlePattern*> FStringEnd <end_location:@R> => {
StringType::FString(ast::ExprFString {
StringType::FString(ast::FString {
values,
implicit_concatenated: false,
range: (location..end_location).into()
})
}
@ -1611,9 +1621,9 @@ FStringExpr: StringType = {
FStringMiddlePattern: ast::Expr = {
FStringReplacementField,
<start_location:@L> <fstring_middle:fstring_middle> =>? {
<location:@L> <fstring_middle:fstring_middle> <end_location:@R> =>? {
let (source, is_raw) = fstring_middle;
Ok(parse_fstring_middle(&source, is_raw, start_location)?)
Ok(parse_fstring_middle(&source, is_raw, (location..end_location).into())?)
}
};
@ -1661,9 +1671,8 @@ FStringFormatSpecSuffix: ast::Expr = {
FStringFormatSpec: ast::Expr = {
<location:@L> <values:FStringMiddlePattern*> <end_location:@R> => {
ast::ExprFString {
ast::FString {
values,
implicit_concatenated: false,
range: (location..end_location).into()
}.into()
},
@ -1685,7 +1694,7 @@ FStringConversion: (TextSize, ast::ConversionFlag) = {
};
Atom<Goal>: ast::ParenthesizedExpr = {
<location:@L> <strings:StringLiteralOrFString+> <end_location:@R> =>? Ok(concatenate_strings(strings, (location..end_location).into())?.into()),
<expr:String> => expr.into(),
<location:@L> <value:Number> <end_location:@R> => ast::ExprNumberLiteral {
value,
range: (location..end_location).into(),
@ -1926,9 +1935,18 @@ OneOrMore<T>: Vec<T> = {
};
/// Two or more items that are separated by `Sep`
TwoOrMore<T, Sep>: Vec<T> = {
TwoOrMoreSep<T, Sep>: Vec<T> = {
<e1:T> Sep <e2:T> => vec![e1, e2],
<mut v: TwoOrMore<T, Sep>> Sep <e:T> => {
<mut v: TwoOrMoreSep<T, Sep>> Sep <e:T> => {
v.push(e);
v
}
};
/// Two or more items that are contiguous.
TwoOrMore<T>: Vec<T> = {
<e1:T> <e2:T> => vec![e1, e2],
<mut v: TwoOrMore<T>> <e:T> => {
v.push(e);
v
}

44905
crates/ruff_python_parser/src/python.rs
View file

File diff suppressed because it is too large Load diff

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__invalid__tests__ok_attribute_weird.snap
View file

@ -14,9 +14,15 @@ Ok(
value: StringLiteral(
ExprStringLiteral {
range: 0..5,
value: "foo",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..5,
value: "foo",
unicode: false,
},
),
},
},
),
attr: Identifier {

48
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__dict_unpacking.snap
View file

@ -10,9 +10,15 @@ Dict(
StringLiteral(
ExprStringLiteral {
range: 1..4,
value: "a",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 1..4,
value: "a",
unicode: false,
},
),
},
},
),
),
@ -21,9 +27,15 @@ Dict(
StringLiteral(
ExprStringLiteral {
range: 16..19,
value: "d",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 16..19,
value: "d",
unicode: false,
},
),
},
},
),
),
@ -32,9 +44,15 @@ Dict(
StringLiteral(
ExprStringLiteral {
range: 6..9,
value: "b",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 6..9,
value: "b",
unicode: false,
},
),
},
},
),
Name(
@ -47,9 +65,15 @@ Dict(
StringLiteral(
ExprStringLiteral {
range: 21..24,
value: "e",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 21..24,
value: "e",
unicode: false,
},
),
},
},
),
],

1400
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__fstrings.snap
View file

File diff suppressed because it is too large Load diff

336
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__fstrings_with_unicode.snap
View file

@ -9,40 +9,55 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..29,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..5,
value: "foo",
unicode: true,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 9..14,
value: Name(
ExprName {
range: 10..13,
id: "bar",
ctx: Load,
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..6,
value: "foo",
unicode: true,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 7..15,
values: [
FormattedValue(
ExprFormattedValue {
range: 9..14,
value: Name(
ExprName {
range: 10..13,
id: "bar",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
Literal(
StringLiteral {
range: 16..21,
value: "baz",
unicode: false,
},
),
Literal(
StringLiteral {
range: 22..29,
value: " some",
unicode: false,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 17..28,
value: "baz some",
unicode: false,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},
@ -53,40 +68,55 @@ expression: parse_ast
value: FString(
ExprFString {
range: 30..59,
values: [
StringLiteral(
ExprStringLiteral {
range: 31..34,
value: "foo",
unicode: false,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 38..43,
value: Name(
ExprName {
range: 39..42,
id: "bar",
ctx: Load,
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 30..35,
value: "foo",
unicode: false,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 36..44,
values: [
FormattedValue(
ExprFormattedValue {
range: 38..43,
value: Name(
ExprName {
range: 39..42,
id: "bar",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
Literal(
StringLiteral {
range: 45..51,
value: "baz",
unicode: true,
},
),
Literal(
StringLiteral {
range: 52..59,
value: " some",
unicode: false,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 47..58,
value: "baz some",
unicode: true,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},
@ -97,40 +127,55 @@ expression: parse_ast
value: FString(
ExprFString {
range: 60..89,
values: [
StringLiteral(
ExprStringLiteral {
range: 61..64,
value: "foo",
unicode: false,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 68..73,
value: Name(
ExprName {
range: 69..72,
id: "bar",
ctx: Load,
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 60..65,
value: "foo",
unicode: false,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 66..74,
values: [
FormattedValue(
ExprFormattedValue {
range: 68..73,
value: Name(
ExprName {
range: 69..72,
id: "bar",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
Literal(
StringLiteral {
range: 75..80,
value: "baz",
unicode: false,
},
),
Literal(
StringLiteral {
range: 81..89,
value: " some",
unicode: true,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 76..88,
value: "baz some",
unicode: false,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},
@ -141,40 +186,83 @@ expression: parse_ast
value: FString(
ExprFString {
range: 90..128,
values: [
StringLiteral(
ExprStringLiteral {
range: 92..103,
value: "foobar ",
unicode: true,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 103..108,
value: Name(
ExprName {
range: 104..107,
id: "baz",
ctx: Load,
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 90..96,
value: "foo",
unicode: true,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 97..116,
values: [
StringLiteral(
ExprStringLiteral {
range: 99..103,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 99..103,
value: "bar ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 103..108,
value: Name(
ExprName {
range: 104..107,
id: "baz",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 108..115,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 108..115,
value: " really",
unicode: false,
},
),
},
},
),
],
},
),
Literal(
StringLiteral {
range: 117..123,
value: "bar",
unicode: true,
},
),
Literal(
StringLiteral {
range: 124..128,
value: "no",
unicode: false,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 108..127,
value: " reallybarno",
unicode: false,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},

36
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__generator_expression_argument.snap
View file

@ -11,9 +11,15 @@ Call(
value: StringLiteral(
ExprStringLiteral {
range: 0..3,
value: " ",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..3,
value: " ",
unicode: false,
},
),
},
},
),
attr: Identifier {
@ -66,9 +72,15 @@ Call(
left: StringLiteral(
ExprStringLiteral {
range: 43..53,
value: "LIMIT %d",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 43..53,
value: "LIMIT %d",
unicode: false,
},
),
},
},
),
op: Mod,
@ -104,9 +116,15 @@ Call(
left: StringLiteral(
ExprStringLiteral {
range: 91..102,
value: "OFFSET %d",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 91..102,
value: "OFFSET %d",
unicode: false,
},
),
},
},
),
op: Mod,

48
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__match.snap
View file

@ -14,9 +14,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 8..14,
value: "test",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 8..14,
value: "test",
unicode: false,
},
),
},
},
),
),
@ -97,9 +103,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 81..88,
value: "label",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 81..88,
value: "label",
unicode: false,
},
),
},
},
),
),
@ -108,9 +120,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 90..96,
value: "test",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 90..96,
value: "test",
unicode: false,
},
),
},
},
),
],
@ -126,9 +144,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 118..125,
value: "label",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 118..125,
value: "label",
unicode: false,
},
),
},
},
),
],

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_class.snap
View file

@ -116,9 +116,15 @@ expression: "parse_suite(source, \"<test>\").unwrap()"
StringLiteral(
ExprStringLiteral {
range: 80..89,
value: "default",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 80..89,
value: "default",
unicode: false,
},
),
},
},
),
),

34
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_f_string.snap
View file

@ -9,17 +9,31 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..14,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..13,
value: "Hello world",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..14,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..13,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..13,
value: "Hello world",
unicode: false,
},
),
},
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_kwargs.snap
View file

@ -22,9 +22,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 8..20,
value: "positional",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 8..20,
value: "positional",
unicode: false,
},
),
},
},
),
],

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_print_2.snap
View file

@ -22,9 +22,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 6..19,
value: "Hello world",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 6..19,
value: "Hello world",
unicode: false,
},
),
},
},
),
NumberLiteral(

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_print_hello.snap
View file

@ -22,9 +22,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 6..19,
value: "Hello world",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 6..19,
value: "Hello world",
unicode: false,
},
),
},
},
),
],

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_string.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..13,
value: "Hello world",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..13,
value: "Hello world",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__parse_type_declaration.snap
View file

@ -81,9 +81,15 @@ expression: "parse_suite(source, \"<test>\").unwrap()"
right: StringLiteral(
ExprStringLiteral {
range: 48..61,
value: "ForwardRefY",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 48..61,
value: "ForwardRefY",
unicode: false,
},
),
},
},
),
},

84
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__patma.snap
View file

@ -501,9 +501,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 484..489,
value: "seq",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 484..489,
value: "seq",
unicode: false,
},
),
},
},
),
),
@ -530,9 +536,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 518..523,
value: "map",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 518..523,
value: "map",
unicode: false,
},
),
},
},
),
),
@ -821,9 +833,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 664..667,
value: "X",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 664..667,
value: "X",
unicode: false,
},
),
},
},
),
},
@ -1551,9 +1569,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 1287..1292,
value: "foo",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 1287..1292,
value: "foo",
unicode: false,
},
),
},
},
),
],
@ -2469,9 +2493,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 2036..2038,
value: "",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2036..2038,
value: "",
unicode: false,
},
),
},
},
),
},
@ -2513,9 +2543,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 2064..2066,
value: "",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2064..2066,
value: "",
unicode: false,
},
),
},
},
),
},
@ -3131,9 +3167,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 2449..2452,
value: "X",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2449..2452,
value: "X",
unicode: false,
},
),
},
},
),
},

184
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__try.snap
View file

@ -81,50 +81,64 @@ expression: parse_ast
FString(
ExprFString {
range: 62..81,
values: [
StringLiteral(
ExprStringLiteral {
range: 64..71,
value: "caught ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 71..80,
value: Call(
ExprCall {
range: 72..79,
func: Name(
ExprName {
range: 72..76,
id: "type",
ctx: Load,
value: FStringValue {
inner: Single(
FString(
FString {
range: 62..81,
values: [
StringLiteral(
ExprStringLiteral {
range: 64..71,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 64..71,
value: "caught ",
unicode: false,
},
),
},
},
),
arguments: Arguments {
range: 76..79,
args: [
Name(
ExprName {
range: 77..78,
id: "e",
ctx: Load,
FormattedValue(
ExprFormattedValue {
range: 71..80,
value: Call(
ExprCall {
range: 72..79,
func: Name(
ExprName {
range: 72..76,
id: "type",
ctx: Load,
},
),
arguments: Arguments {
range: 76..79,
args: [
Name(
ExprName {
range: 77..78,
id: "e",
ctx: Load,
},
),
],
keywords: [],
},
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
],
@ -175,50 +189,64 @@ expression: parse_ast
FString(
ExprFString {
range: 114..133,
values: [
StringLiteral(
ExprStringLiteral {
range: 116..123,
value: "caught ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 123..132,
value: Call(
ExprCall {
range: 124..131,
func: Name(
ExprName {
range: 124..128,
id: "type",
ctx: Load,
value: FStringValue {
inner: Single(
FString(
FString {
range: 114..133,
values: [
StringLiteral(
ExprStringLiteral {
range: 116..123,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 116..123,
value: "caught ",
unicode: false,
},
),
},
},
),
arguments: Arguments {
range: 128..131,
args: [
Name(
ExprName {
range: 129..130,
id: "e",
ctx: Load,
FormattedValue(
ExprFormattedValue {
range: 123..132,
value: Call(
ExprCall {
range: 124..131,
func: Name(
ExprName {
range: 124..128,
id: "type",
ctx: Load,
},
),
arguments: Arguments {
range: 128..131,
args: [
Name(
ExprName {
range: 129..130,
id: "e",
ctx: Load,
},
),
],
keywords: [],
},
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
],

328
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__try_star.snap
View file

@ -27,9 +27,15 @@ expression: parse_ast
StringLiteral(
ExprStringLiteral {
range: 30..34,
value: "eg",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 30..34,
value: "eg",
unicode: false,
},
),
},
},
),
List(
@ -193,83 +199,103 @@ expression: parse_ast
FString(
ExprFString {
range: 133..179,
values: [
StringLiteral(
ExprStringLiteral {
range: 135..142,
value: "caught ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 142..151,
value: Call(
ExprCall {
range: 143..150,
func: Name(
ExprName {
range: 143..147,
id: "type",
ctx: Load,
value: FStringValue {
inner: Single(
FString(
FString {
range: 133..179,
values: [
StringLiteral(
ExprStringLiteral {
range: 135..142,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 135..142,
value: "caught ",
unicode: false,
},
),
},
},
),
arguments: Arguments {
range: 147..150,
args: [
Name(
ExprName {
range: 148..149,
id: "e",
FormattedValue(
ExprFormattedValue {
range: 142..151,
value: Call(
ExprCall {
range: 143..150,
func: Name(
ExprName {
range: 143..147,
id: "type",
ctx: Load,
},
),
arguments: Arguments {
range: 147..150,
args: [
Name(
ExprName {
range: 148..149,
id: "e",
ctx: Load,
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 151..164,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 151..164,
value: " with nested ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 164..178,
value: Attribute(
ExprAttribute {
range: 165..177,
value: Name(
ExprName {
range: 165..166,
id: "e",
ctx: Load,
},
),
attr: Identifier {
id: "exceptions",
range: 167..177,
},
ctx: Load,
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 151..164,
value: " with nested ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 164..178,
value: Attribute(
ExprAttribute {
range: 165..177,
value: Name(
ExprName {
range: 165..166,
id: "e",
ctx: Load,
debug_text: None,
conversion: None,
format_spec: None,
},
),
attr: Identifier {
id: "exceptions",
range: 167..177,
},
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
],
},
),
),
],
implicit_concatenated: false,
},
},
),
],
@ -320,83 +346,103 @@ expression: parse_ast
FString(
ExprFString {
range: 213..259,
values: [
StringLiteral(
ExprStringLiteral {
range: 215..222,
value: "caught ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 222..231,
value: Call(
ExprCall {
range: 223..230,
func: Name(
ExprName {
range: 223..227,
id: "type",
ctx: Load,
value: FStringValue {
inner: Single(
FString(
FString {
range: 213..259,
values: [
StringLiteral(
ExprStringLiteral {
range: 215..222,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 215..222,
value: "caught ",
unicode: false,
},
),
},
},
),
arguments: Arguments {
range: 227..230,
args: [
Name(
ExprName {
range: 228..229,
id: "e",
FormattedValue(
ExprFormattedValue {
range: 222..231,
value: Call(
ExprCall {
range: 223..230,
func: Name(
ExprName {
range: 223..227,
id: "type",
ctx: Load,
},
),
arguments: Arguments {
range: 227..230,
args: [
Name(
ExprName {
range: 228..229,
id: "e",
ctx: Load,
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 231..244,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 231..244,
value: " with nested ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 244..258,
value: Attribute(
ExprAttribute {
range: 245..257,
value: Name(
ExprName {
range: 245..246,
id: "e",
ctx: Load,
},
),
attr: Identifier {
id: "exceptions",
range: 247..257,
},
ctx: Load,
},
),
],
keywords: [],
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 231..244,
value: " with nested ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 244..258,
value: Attribute(
ExprAttribute {
range: 245..257,
value: Name(
ExprName {
range: 245..246,
id: "e",
ctx: Load,
debug_text: None,
conversion: None,
format_spec: None,
},
),
attr: Identifier {
id: "exceptions",
range: 247..257,
},
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
],
},
),
),
],
implicit_concatenated: false,
},
},
),
],

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__parser__tests__unicode_aliases.snap
View file

@ -18,9 +18,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 4..37,
value: "\u{8}another cool trick",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 4..37,
value: "\u{8}another cool trick",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__backspace_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..15,
value: "\u{8}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..15,
value: "\u{8}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__bell_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..9,
value: "\u{7}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..9,
value: "\u{7}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__carriage_return_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..21,
value: "\r",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..21,
value: "\r",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__character_tabulation_with_justification_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..45,
value: "\u{89}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..45,
value: "\u{89}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__delete_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..12,
value: "\u{7f}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..12,
value: "\u{7f}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__dont_panic_on_8_in_octal_escape.snap
View file

@ -18,9 +18,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 7..16,
value: "\u{3}8[1m",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 7..16,
value: "\u{3}8[1m",
unicode: false,
},
),
},
},
),
},

524
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__double_quoted_byte.snap
View file

@ -9,265 +9,271 @@ expression: parse_ast
value: BytesLiteral(
ExprBytesLiteral {
range: 0..738,
value: [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63,
64,
65,
66,
67,
68,
69,
70,
71,
72,
73,
74,
75,
76,
77,
78,
79,
80,
81,
82,
83,
84,
85,
86,
87,
88,
89,
90,
91,
92,
93,
94,
95,
96,
97,
98,
99,
100,
101,
102,
103,
104,
105,
106,
107,
108,
109,
110,
111,
112,
113,
114,
115,
116,
117,
118,
119,
120,
121,
122,
123,
124,
125,
126,
127,
128,
129,
130,
131,
132,
133,
134,
135,
136,
137,
138,
139,
140,
141,
142,
143,
144,
145,
146,
147,
148,
149,
150,
151,
152,
153,
154,
155,
156,
157,
158,
159,
160,
161,
162,
163,
164,
165,
166,
167,
168,
169,
170,
171,
172,
173,
174,
175,
176,
177,
178,
179,
180,
181,
182,
183,
184,
185,
186,
187,
188,
189,
190,
191,
192,
193,
194,
195,
196,
197,
198,
199,
200,
201,
202,
203,
204,
205,
206,
207,
208,
209,
210,
211,
212,
213,
214,
215,
216,
217,
218,
219,
220,
221,
222,
223,
224,
225,
226,
227,
228,
229,
230,
231,
232,
233,
234,
235,
236,
237,
238,
239,
240,
241,
242,
243,
244,
245,
246,
247,
248,
249,
250,
251,
252,
253,
254,
255,
],
implicit_concatenated: false,
value: BytesLiteralValue {
inner: Single(
BytesLiteral {
range: 0..738,
value: [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63,
64,
65,
66,
67,
68,
69,
70,
71,
72,
73,
74,
75,
76,
77,
78,
79,
80,
81,
82,
83,
84,
85,
86,
87,
88,
89,
90,
91,
92,
93,
94,
95,
96,
97,
98,
99,
100,
101,
102,
103,
104,
105,
106,
107,
108,
109,
110,
111,
112,
113,
114,
115,
116,
117,
118,
119,
120,
121,
122,
123,
124,
125,
126,
127,
128,
129,
130,
131,
132,
133,
134,
135,
136,
137,
138,
139,
140,
141,
142,
143,
144,
145,
146,
147,
148,
149,
150,
151,
152,
153,
154,
155,
156,
157,
158,
159,
160,
161,
162,
163,
164,
165,
166,
167,
168,
169,
170,
171,
172,
173,
174,
175,
176,
177,
178,
179,
180,
181,
182,
183,
184,
185,
186,
187,
188,
189,
190,
191,
192,
193,
194,
195,
196,
197,
198,
199,
200,
201,
202,
203,
204,
205,
206,
207,
208,
209,
210,
211,
212,
213,
214,
215,
216,
217,
218,
219,
220,
221,
222,
223,
224,
225,
226,
227,
228,
229,
230,
231,
232,
233,
234,
235,
236,
237,
238,
239,
240,
241,
242,
243,
244,
245,
246,
247,
248,
249,
250,
251,
252,
253,
254,
255,
],
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__escape_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..12,
value: "\u{1b}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..12,
value: "\u{1b}",
unicode: false,
},
),
},
},
),
},

32
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__escape_char_in_byte_literal.snap
View file

@ -9,19 +9,25 @@ expression: parse_ast
value: BytesLiteral(
ExprBytesLiteral {
range: 0..13,
value: [
111,
109,
107,
109,
111,
107,
92,
88,
97,
97,
],
implicit_concatenated: false,
value: BytesLiteralValue {
inner: Single(
BytesLiteral {
range: 0..13,
value: [
111,
109,
107,
109,
111,
107,
92,
88,
97,
97,
],
},
),
},
},
),
},

22
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__escape_octet.snap
View file

@ -9,14 +9,20 @@ expression: parse_ast
value: BytesLiteral(
ExprBytesLiteral {
range: 0..14,
value: [
35,
97,
4,
83,
52,
],
implicit_concatenated: false,
value: BytesLiteralValue {
inner: Single(
BytesLiteral {
range: 0..14,
value: [
35,
97,
4,
83,
52,
],
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__form_feed_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..15,
value: "\u{c}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..15,
value: "\u{c}",
unicode: false,
},
),
},
},
),
},

138
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_constant_range.snap
View file

@ -9,63 +9,89 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..5,
value: "aaa",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..5,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..5,
value: "aaa",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 5..10,
value: Name(
ExprName {
range: 6..9,
id: "bbb",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 10..13,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 10..13,
value: "ccc",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 13..18,
value: Name(
ExprName {
range: 14..17,
id: "ddd",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 18..21,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 18..21,
value: "eee",
unicode: false,
},
),
},
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 5..10,
value: Name(
ExprName {
range: 6..9,
id: "bbb",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 10..13,
value: "ccc",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 13..18,
value: Name(
ExprName {
range: 14..17,
id: "ddd",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 18..21,
value: "eee",
unicode: false,
implicit_concatenated: false,
},
),
],
implicit_concatenated: false,
},
},
),
},

64
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_escaped_character.snap
View file

@ -9,32 +9,46 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..8,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..4,
value: "\\",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..8,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..4,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..4,
value: "\\",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 4..7,
value: Name(
ExprName {
range: 5..6,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 4..7,
value: Name(
ExprName {
range: 5..6,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

64
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_escaped_newline.snap
View file

@ -9,32 +9,46 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..8,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..4,
value: "\n",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..8,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..4,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..4,
value: "\n",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 4..7,
value: Name(
ExprName {
range: 5..6,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 4..7,
value: Name(
ExprName {
range: 5..6,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

64
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_line_continuation.snap
View file

@ -9,32 +9,46 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..9,
values: [
StringLiteral(
ExprStringLiteral {
range: 3..5,
value: "\\\n",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..9,
values: [
StringLiteral(
ExprStringLiteral {
range: 3..5,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 3..5,
value: "\\\n",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

52
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_parse_self_documenting_base.snap
View file

@ -9,29 +9,37 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Name(
ExprName {
range: 3..7,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..10,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..9,
value: Name(
ExprName {
range: 3..7,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

136
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_parse_self_documenting_base_more.snap
View file

@ -9,65 +9,85 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..38,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..6,
value: "mix ",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..38,
values: [
StringLiteral(
ExprStringLiteral {
range: 2..6,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 2..6,
value: "mix ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 6..13,
value: Name(
ExprName {
range: 7..11,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 13..28,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 13..28,
value: " with text and ",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 28..37,
value: Name(
ExprName {
range: 29..35,
id: "second",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 6..13,
value: Name(
ExprName {
range: 7..11,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 13..28,
value: " with text and ",
unicode: false,
implicit_concatenated: false,
},
),
FormattedValue(
ExprFormattedValue {
range: 28..37,
value: Name(
ExprName {
range: 29..35,
id: "second",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

96
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_parse_self_documenting_format.snap
View file

@ -9,46 +9,68 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..14,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..13,
value: Name(
ExprName {
range: 3..7,
id: "user",
ctx: Load,
},
),
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 9..12,
values: [
StringLiteral(
ExprStringLiteral {
range: 9..12,
value: ">10",
unicode: false,
implicit_concatenated: false,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..14,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..13,
value: Name(
ExprName {
range: 3..7,
id: "user",
ctx: Load,
},
),
],
implicit_concatenated: false,
},
),
),
},
debug_text: Some(
DebugText {
leading: "",
trailing: "=",
},
),
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 9..12,
value: FStringValue {
inner: Single(
FString(
FString {
range: 9..12,
values: [
StringLiteral(
ExprStringLiteral {
range: 9..12,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 9..12,
value: ">10",
unicode: false,
},
),
},
},
),
],
},
),
),
},
},
),
),
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

64
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__fstring_unescaped_newline.snap
View file

@ -9,32 +9,46 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..11,
values: [
StringLiteral(
ExprStringLiteral {
range: 4..5,
value: "\n",
unicode: false,
implicit_concatenated: false,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..11,
values: [
StringLiteral(
ExprStringLiteral {
range: 4..5,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 4..5,
value: "\n",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 5..8,
value: Name(
ExprName {
range: 6..7,
id: "x",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
implicit_concatenated: false,
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__hts_alias.snap
View file

@ -9,9 +9,15 @@ expression: parse_ast
value: StringLiteral(
ExprStringLiteral {
range: 0..9,
value: "\u{88}",
unicode: false,
implicit_concatenated: false,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 0..9,
value: "\u{88}",
unicode: false,
},
),
},
},
),
},

12
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_empty_fstring.snap
View file

@ -9,8 +9,16 @@ expression: "parse_suite(r#\"f\"\"\"#, \"<test>\").unwrap()"
value: FString(
ExprFString {
range: 0..3,
values: [],
implicit_concatenated: false,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..3,
values: [],
},
),
),
},
},
),
},

43
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_f_string_concat_1.snap
View file

@ -9,17 +9,40 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..17,
values: [
StringLiteral(
ExprStringLiteral {
range: 1..16,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
},
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
},
),
FString(
FString {
range: 9..17,
values: [
StringLiteral(
ExprStringLiteral {
range: 11..16,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 11..16,
value: "world",
unicode: false,
},
),
},
},
),
],
},
),
],
),
],
implicit_concatenated: true,
},
},
),
},

43
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_f_string_concat_2.snap
View file

@ -9,17 +9,40 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..17,
values: [
StringLiteral(
ExprStringLiteral {
range: 1..16,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
},
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
},
),
FString(
FString {
range: 9..17,
values: [
StringLiteral(
ExprStringLiteral {
range: 11..16,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 11..16,
value: "world",
unicode: false,
},
),
},
},
),
],
},
),
],
),
],
implicit_concatenated: true,
},
},
),
},

75
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_f_string_concat_3.snap
View file

@ -9,33 +9,62 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 1..16,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 16..21,
value: StringLiteral(
ExprStringLiteral {
range: 17..20,
value: "!",
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
implicit_concatenated: false,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 9..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 11..16,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 11..16,
value: "world",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 16..21,
value: StringLiteral(
ExprStringLiteral {
range: 17..20,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 17..20,
value: "!",
unicode: false,
},
),
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
],
),
],
implicit_concatenated: true,
},
},
),
},

90
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_f_string_concat_4.snap
View file

@ -9,41 +9,69 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..31,
values: [
StringLiteral(
ExprStringLiteral {
range: 1..16,
value: "Hello world",
unicode: false,
implicit_concatenated: true,
},
),
FormattedValue(
ExprFormattedValue {
range: 16..21,
value: StringLiteral(
ExprStringLiteral {
range: 17..20,
value: "!",
value: FStringValue {
inner: Concatenated(
[
Literal(
StringLiteral {
range: 0..8,
value: "Hello ",
unicode: false,
implicit_concatenated: false,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
FString(
FString {
range: 9..22,
values: [
StringLiteral(
ExprStringLiteral {
range: 11..16,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 11..16,
value: "world",
unicode: false,
},
),
},
},
),
FormattedValue(
ExprFormattedValue {
range: 16..21,
value: StringLiteral(
ExprStringLiteral {
range: 17..20,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 17..20,
value: "!",
unicode: false,
},
),
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
Literal(
StringLiteral {
range: 23..31,
value: "again!",
unicode: false,
},
),
],
),
StringLiteral(
ExprStringLiteral {
range: 24..30,
value: "again!",
unicode: false,
implicit_concatenated: true,
},
),
],
implicit_concatenated: true,
},
},
),
},

94
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring.snap
View file

@ -9,47 +9,61 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..18,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..5,
value: Name(
ExprName {
range: 3..4,
id: "a",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..18,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..5,
value: Name(
ExprName {
range: 3..4,
id: "a",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
FormattedValue(
ExprFormattedValue {
range: 5..10,
value: Name(
ExprName {
range: 7..8,
id: "b",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 10..17,
value: StringLiteralValue {
inner: Single(
StringLiteral {
range: 10..17,
value: "{foo}",
unicode: false,
},
),
},
},
),
],
},
),
),
FormattedValue(
ExprFormattedValue {
range: 5..10,
value: Name(
ExprName {
range: 7..8,
id: "b",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
StringLiteral(
ExprStringLiteral {
range: 10..17,
value: "{foo}",
unicode: false,
implicit_concatenated: false,
},
),
],
implicit_concatenated: false,
},
},
),
},

74
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_equals.snap
View file

@ -9,43 +9,51 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..12,
value: Compare(
ExprCompare {
range: 3..11,
left: NumberLiteral(
ExprNumberLiteral {
range: 3..5,
value: Int(
42,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..12,
value: Compare(
ExprCompare {
range: 3..11,
left: NumberLiteral(
ExprNumberLiteral {
range: 3..5,
value: Int(
42,
),
},
),
ops: [
Eq,
],
comparators: [
NumberLiteral(
ExprNumberLiteral {
range: 9..11,
value: Int(
42,
),
},
),
],
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
ops: [
Eq,
],
comparators: [
NumberLiteral(
ExprNumberLiteral {
range: 9..11,
value: Int(
42,
),
},
),
],
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

112
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_nested_concatenation_string_spec.snap
View file

@ -9,49 +9,81 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..16,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..15,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..14,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..14,
value: StringLiteral(
ExprStringLiteral {
range: 8..13,
value: "",
unicode: false,
implicit_concatenated: true,
},
),
debug_text: None,
conversion: None,
format_spec: None,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..16,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..15,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
],
implicit_concatenated: false,
},
),
),
},
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..14,
value: FStringValue {
inner: Single(
FString(
FString {
range: 7..14,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..14,
value: StringLiteral(
ExprStringLiteral {
range: 8..13,
value: StringLiteralValue {
inner: Concatenated(
ConcatenatedStringLiteral {
strings: [
StringLiteral {
range: 8..10,
value: "",
unicode: false,
},
StringLiteral {
range: 11..13,
value: "",
unicode: false,
},
],
value: "",
},
),
},
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
},
},
),
),
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

94
crates/ruff_python_parser/src/snapshots/ruff_python_parser__string__tests__parse_fstring_nested_spec.snap
View file

@ -9,48 +9,64 @@ expression: parse_ast
value: FString(
ExprFString {
range: 0..15,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..14,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..13,
value: Name(
ExprName {
range: 8..12,
id: "spec",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
value: FStringValue {
inner: Single(
FString(
FString {
range: 0..15,
values: [
FormattedValue(
ExprFormattedValue {
range: 2..14,
value: Name(
ExprName {
range: 3..6,
id: "foo",
ctx: Load,
},
),
],
implicit_concatenated: false,
},
),
),
},
debug_text: None,
conversion: None,
format_spec: Some(
FString(
ExprFString {
range: 7..13,
value: FStringValue {
inner: Single(
FString(
FString {
range: 7..13,
values: [
FormattedValue(
ExprFormattedValue {
range: 7..13,
value: Name(
ExprName {
range: 8..12,
id: "spec",
ctx: Load,
},
),
debug_text: None,
conversion: None,
format_spec: None,
},
),
],
},
),
),
},
},
),
),
},
),
],
},
),
),
],
implicit_concatenated: false,
},
},
),
},

Some files were not shown because too many files have changed in this diff Show more