As of [this cpython PR](https://github.com/python/cpython/pull/135996),
it is not allowed to concatenate t-strings with non-t-strings,
implicitly or explicitly. Expressions such as `"foo" t"{bar}"` are now
syntax errors.
This PR updates some AST nodes and parsing to reflect this change.
The structural change is that `TStringPart` is no longer needed, since,
as in the case of `BytesStringLiteral`, the only possibilities are that
we have a single `TString` or a vector of such (representing an implicit
concatenation of t-strings). This removes a level of nesting from many
AST expressions (which is what all the snapshot changes reflect), and
simplifies some logic in the implementation of visitors, for example.
The other change of note is in the parser. When we meet an implicit
concatenation of string-like literals, we now count the number of
t-string literals. If these do not exhaust the total number of
implicitly concatenated pieces, then we emit a syntax error. To recover
from this syntax error, we encode any t-string pieces as _invalid_
string literals (which means we flag them as invalid, record their
range, and record the value as `""`). Note that if at least one of the
pieces is an f-string we prefer to parse the entire string as an
f-string; otherwise we parse it as a string.
This logic is exactly the same as how we currently treat
`BytesStringLiteral` parsing and error recovery - and carries with it
the same pros and cons.
Finally, note that I have not implemented any changes in the
implementation of the formatter. As far as I can tell, none are needed.
I did change a few of the fixtures so that we are always concatenating
t-strings with t-strings.
## Summary
Changing `BLE001` (blind-except) so that it does not flag `except`
clauses which include `logging.critical(..., exc_info=True)`.
## Test Plan
It passes the following (whereas the `main` branch does not):
```sh
$ cargo run -p ruff -- check somefile.py --no-cache --select=BLE001
```
```python
# somefile.py
import logging
try:
print("Hello world!")
except Exception:
logging.critical("Did not run.", exc_info=True)
```
Related: https://github.com/astral-sh/ruff/issues/19519
Small rewording to indicate that core development is done but that we
may add breaking changes.
Feel free to bikeshed!
Test:
```console
❯ echo "t''" | cargo run -p ruff -- check --no-cache --isolated --target-version py314 -
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.13s
Running `target/debug/ruff check --no-cache --isolated --target-version py314 -`
warning: Support for Python 3.14 is in preview and may undergo breaking changes. Enable `preview` to remove this warning.
All checks passed!
```
Summary
--
I looked at other uses of `TextEmitter`, and I think this should be the
only one affected by this. The other integration tests must work
properly since they're run with `assert_cmd_snapshot!`, which I assume
triggers the `SHOULD_COLORIZE` case, and the `cfg!(test)` check will
work for uses in `ruff_linter`.
4a4dc38b5b/crates/ruff_linter/src/message/text.rs (L36-L44)
Alternatively, we could probably move this to a CLI test instead.
Test Plan
--
`cargo test -p ruff`, which was failing on `main` with color codes in
the output before this
## Summary
Expand cases in which ruff can offer a fix for `RUF039` (some of which
are unsafe).
While turning `"\n"` (== `\n`) into `r"\n"` (== `\\n`) is not equivalent
at run-time, it's still functionally equivalent to do so in the context
of [regex
patterns](https://docs.python.org/3/library/re.html#regular-expression-syntax)
as they themselves interpret the escape sequence. Therefore, an unsafe
fix can be offered.
Further, this PR also makes ruff offer fixes for byte string literals,
not only strings literals as before.
## Test Plan
Tests for all escape sequences have been added.
## Related
Closes: https://github.com/astral-sh/ruff/issues/16713
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
## Summary
The generated fix for `RUF033` would cause a syntax error for named
expressions as parameter defaults.
```python
from dataclasses import InitVar, dataclass
@dataclass
class Foo:
def __post_init__(self, bar: int = (x := 1)) -> None:
pass
```
would be turned into
```python
from dataclasses import InitVar, dataclass
@dataclass
class Foo:
x: InitVar[int] = x := 1
def __post_init__(self, bar: int = (x := 1)) -> None:
pass
```
instead of the syntactically correct
```python
# ...
x: InitVar[int] = (x := 1)
# ...
```
## Test Plan
Test reproducer (plus some extra tests) have been added to the test
suite.
## Related
Fixes: https://github.com/astral-sh/ruff/issues/18950
## Summary
closes#19204
## Test Plan
1. test case is added in dedicated file
2. locally tested the code manually
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Co-authored-by: CodeMan62 <sharmahimanshu150082007@gmail.com>
## Summary
This PR moves most of the work of rendering concise diagnostics in Ruff
into `ruff_db`, where the code is shared with ty. To accomplish this
without breaking backwards compatibility in Ruff, there are two main
changes on the `ruff_db`/ty side:
- Added the logic from Ruff for remapping notebook line numbers to cells
- Reordered the fields in the diagnostic to match Ruff and rustc
```text
# old
error[invalid-assignment] try.py:3:1: Object of type `Literal[1]` is not
assignable to `str`
# new
try.py:3:1: error[invalid-assignment]: Object of type `Literal[1]` is
not assignable to `str`
```
I don't think the notebook change failed any tests on its own, and only
a handful of snaphots changed in ty after reordering the fields, but
this will obviously affect any other uses of the concise format, outside
of tests, too.
The other big change should only affect Ruff:
- Added three new `DisplayDiagnosticConfig` options
Micha and I hoped that we could get by with one option
(`hide_severity`), but Ruff also toggles `show_fix_status` itself,
independently (there are cases where we want neither severity nor the
fix status), and during the implementation I realized we also needed
access to an `Applicability`. The main goal here is to suppress the
severity (`error` above) because ruff only uses the `error` severity and
to use the secondary/noqa code instead of the line name
(`invalid-assignment` above).
```text
# ty - same as "new" above
try.py:3:1: error[invalid-assignment]: Object of type `Literal[1]` is
not assignable to `str`
# ruff
try.py:3:1: RUF123 [*] Object of type `Literal[1]` is not assignable to
`str`
```
This part of the concise diagnostic is actually shared with the `full`
output format in Ruff, but with the settings above, there are no
snapshot changes to either format.
## Test Plan
Existing tests with the handful of updates mentioned above, as well as
some new tests in the `concise` module.
Also this PR. Swapping the fields might have broken mypy_primer, unless
it occasionally times out on its own.
I also ran this script in the root of my Ruff checkout, which also has
CPython in it:
```shell
flags=(--isolated --no-cache --no-respect-gitignore --output-format concise .)
diff <(target/release/ruff check ${flags[@]} 2> /dev/null) \
<(ruff check ${flags[@]} 2> /dev/null)
```
This yielded an expected diff due to some t-string error changes on main
since 0.12.4:
```diff
33622c33622
< crates/ruff_python_parser/resources/inline/err/f_string_lambda_without_parentheses.py:1:15: SyntaxError: Expected an element of or the end of the f-string
---
> crates/ruff_python_parser/resources/inline/err/f_string_lambda_without_parentheses.py:1:15: SyntaxError: Expected an f-string or t-string element or the end of the f-string or t-string
33742c33742
< crates/ruff_python_parser/resources/inline/err/implicitly_concatenated_unterminated_string_multiline.py:4:1: SyntaxError: Expected an element of or the end of the f-string
---
> crates/ruff_python_parser/resources/inline/err/implicitly_concatenated_unterminated_string_multiline.py:4:1: SyntaxError: Expected an f-string or t-string element or the end of the f-string or t-string
34131c34131
< crates/ruff_python_parser/resources/inline/err/t_string_lambda_without_parentheses.py:2:15: SyntaxError: Expected an element of or the end of the t-string
---
> crates/ruff_python_parser/resources/inline/err/t_string_lambda_without_parentheses.py:2:15: SyntaxError: Expected an f-string or t-string element or the end of the f-string or t-string
```
So modulo color, the results are identical on 38,186 errors in our test
suite and CPython 3.10.
---------
Co-authored-by: David Peter <mail@david-peter.de>
Summary
--
This PR tweaks Ruff's internal usage of the new diagnostic model to more
closely
match the intended use, as I understand it. Specifically, it moves the
fix/help
suggestion from the primary annotation's message to a subdiagnostic. In
turn, it
adds the secondary/noqa code as the new primary annotation message. As
shown in
the new `ruff_db` tests, this more closely mirrors Ruff's current
diagnostic
output.
I also added `Severity::Help` to render the fix suggestion with a
`help:` prefix
instead of `info:`.
These changes don't have any external impact now but should help a bit
with #19415.
Test Plan
--
New full output format tests in `ruff_db`
Rendered Diagnostics
--
Full diagnostic output from `annotate-snippets` in this PR:
```
error[unused-import]: `os` imported but unused
--> fib.py:1:8
|
1 | import os
| ^^
|
help: Remove unused import: `os`
```
Current Ruff output for the same code:
```
fib.py:1:8: F401 [*] `os` imported but unused
|
1 | import os
| ^^ F401
|
= help: Remove unused import: `os`
```
Proposed final output after #19415:
```
F401 [*] `os` imported but unused
--> fib.py:1:8
|
1 | import os
| ^^
|
help: Remove unused import: `os`
```
These are slightly updated from
https://github.com/astral-sh/ruff/pull/19464#issuecomment-3097377634
below to remove the extra noqa codes in the primary annotation messages
for the first and third cases.
Parsing the (invalid) expression `f"{\t"i}"` caused a panic because the
`TStringMiddle` character was "unreachable" due the way the parser
recovered from the line continuation (it ate the t-string start).
The cause of the issue is as follows:
The parser begins parsing the f-string and expects to see a list of
objects, essentially alternating between _interpolated elements_ and
ordinary strings. It is happy to see the first left brace, but then
there is a lexical error caused by the line-continuation character. So
instead of the parser seeing a list of elements with just one member, it
sees a list that starts like this:
- Interpolated element with an invalid token, stored as a `Name`
- Something else built from tokens beginning with `TStringStart` and
`TStringMiddle`
When it sees the `TStringStart` error recovery says "that's a list
element I don't know what to do with, let's skip it". When it sees
`TStringMiddle` it says "oh, that looks like the middle of _some
interpolated string_ so let's try to parse it as one of the literal
elements of my `FString`". Unfortunately, the function being used to
parse individual list elements thinks (arguably correctly) that it's not
possible to have a `TStringMiddle` sitting in your `FString`, and hits
`unreachable`.
Two potential ways (among many) to solve this issue are:
1. Allow a `TStringMiddle` as a valid "literal" part of an f-string
during parsing (with the hope/understanding that this would only occur
in an invalid context)
2. Skip the `TStringMiddle` as an "unexpected/invalid list item" in the
same way that we skipped `TStringStart`.
I have opted for the second approach since it seems somehow more morally
correct, even though it loses more information. To implement this, the
recovery context needs to know whether we are in an f-string or t-string
- hence the changes to that enum. As a bonus we get slightly more
specific error messages in some cases.
Closes#18860
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Closes#18739
## Test Plan
<!-- How was it tested? -->
---------
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
Summary
--
This PR moves the JUnit output format to the new rendering
infrastructure. As I
mention in a TODO in the code, there's some code that will be shared
with the
`grouped` output format. Hopefully I'll have that PR up too by the time
this one
is reviewed.
Test Plan
--
Existing tests moved to `ruff_db`
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Fixes#19076
An attempt at fixing #19076 where the rule could change program behavior
by incorrectly converting from_float/from_decimal method calls to
constructor calls.
The fix implements argument validation using Ruff's existing type
inference system (`ResolvedPythonType`, `typing::is_int`,
`typing::is_float`) to determine when conversions are actually safe,
adds logic to detect invalid method calls (wrong argument counts,
incorrect keyword names) and suppress fixes for them, and changes the
default fix applicability from `Safe` to `Unsafe` with safe fixes only
offered when the argument type is known to be compatible and no
problematic keywords are used.
One uncertainty is whether the type inference catches all possible edge
cases in complex codebases, but the new approach is significantly more
conservative and safer than the previous implementation.
## Test Plan
I updated the existing test fixtures with edge cases from the issue and
manually verified behavior with temporary test files for
valid/unsafe/invalid scenarios.
---------
Co-authored-by: Brent Westbrook <brentrwestbrook@gmail.com>
Summary
--
This is a very simple output format, the only decision is what to do if
the file
is missing from the diagnostic. For now, I opted to `unwrap_or_default`
both the
path and the `OneIndexed` row number, giving `:1: main diagnostic
message` in
the test without a file.
Another quirk here is that the path is relativized. I just pasted in the
`relativize_path` and `get_cwd` implementations from `ruff_linter::fs`
for now,
but maybe there's a better place for them.
I didn't see any details about why this needs to be relativized in the
original
[issue](https://github.com/astral-sh/ruff/issues/1953),
[PR](https://github.com/astral-sh/ruff/pull/1995), or in the pylint
[docs](https://flake8.pycqa.org/en/latest/internal/formatters.html#pylint-formatter),
but it did change the results of the CLI integration test when I tried
deleting
it. I haven't been able to reproduce that in the CLI, though, so it may
only
happen with `Command::current_dir`.
Test Plan
--
Tests ported from `ruff_linter` and a new test for the case with no file
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Another output format like #19133. This is the
[reviewdog](https://github.com/reviewdog/reviewdog) output format, which
is somewhat similar to regular JSON. Like #19270, in the first commit I
converted from using `json!` to `Serialize` structs, then in the second
commit I moved the module to `ruff_db`.
The reviewdog
[schema](320a8e73a9/proto/rdf/jsonschema/DiagnosticResult.json)
seems a bit more flexible than our JSON schema, so I'm not sure if we
need any preview checks here. I'll flag the places I wasn't sure about
as review comments.
## Test Plan
New tests in `rdjson.rs`, ported from the old `rjdson.rs` module, as
well as the new CLI output tests.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972Fixes#14346
This PR makes [bidirectional-unicode
(PLE2502)](https://docs.astral.sh/ruff/rules/bidirectional-unicode/#bidirectional-unicode-ple2502)'s
example error out-of-the-box, by converting it to use one of the test
cases. The documentation in general is also updated to replace
"bidirectional unicode character" with "bidirectional formatting
character", as those are the only ones checked for, and the "unicode"
suffix is redundant. The new example section looks like this:
<img width="1074" height="264" alt="image"
src="https://github.com/user-attachments/assets/cc1d2cb4-b590-4f20-a4d2-15b744872cdd"
/>
The "References" section link is also updated to reflect the rule's
actual behavior.
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
This PR fixes#7172 by suppressing the fixes for
[docstring-missing-returns
(DOC201)](https://docs.astral.sh/ruff/rules/docstring-missing-returns/#docstring-missing-returns-doc201)
/ [docstring-extraneous-returns
(DOC202)](https://docs.astral.sh/ruff/rules/docstring-extraneous-returns/#docstring-extraneous-returns-doc202)
if there is a surrounding line continuation character `\` that would
make the fix cause a syntax error.
To do this, the lints are changed from `AlwaysFixableViolation` to
`Violation` with `FixAvailability::Sometimes`.
In the case of `DOC201`, the fix is not given if the non-break line ends
in a line continuation character `\`. Note that lines are iterated in
reverse from the docstring to the function definition.
In the case of `DOC202`, the fix is not given if the docstring ends with
a line continuation character `\`.
## Test Plan
<!-- How was it tested? -->
Added a test case.
## Summary
Part of #18972
This PR makes [for-loop-writes
(FURB122)](https://docs.astral.sh/ruff/rules/for-loop-writes/#for-loop-writes-furb122)'s
example error out-of-the-box. I also had to re-name the second case's
variables to get both to raise at the same time, I suspect because of
limitations in ruff's current semantic model. New names subject to
bikeshedding, I just went with the least effort `_b` for binary suffix.
[Old example](https://play.ruff.rs/19e8e47a-8058-4013-aef5-e9b5eab65962)
```py
with Path("file").open("w") as f:
for line in lines:
f.write(line)
with Path("file").open("wb") as f:
for line in lines:
f.write(line.encode())
```
[New example](https://play.ruff.rs/e96b00e5-3c63-47c3-996d-dace420dd711)
```py
from pathlib import Path
with Path("file").open("w") as f:
for line in lines:
f.write(line)
with Path("file").open("wb") as f_b:
for line_b in lines_b:
f_b.write(line_b.encode())
```
The "Use instead" section was also modified similarly.
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
Part of #18972
This PR makes
[implicit-cwd(FURB177)](https://docs.astral.sh/ruff/rules/implicit-cwd/)'s
example error out-of-the-box.
[Old example](https://play.ruff.rs/a0bef229-9626-426f-867f-55cb95ee64d8)
```python
cwd = Path().resolve()
```
[New example](https://play.ruff.rs/bdbea4af-e276-4603-a1b6-88757dfaa399)
```python
from pathlib import Path
cwd = Path().resolve()
```
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [non-pep695-type-alias
(UP040)](https://docs.astral.sh/ruff/rules/non-pep695-type-alias/#non-pep695-type-alias-up040)'s
example error out-of-the-box.
[Old example](https://play.ruff.rs/6beca1be-45cd-4e5a-aafa-6a0584c10d64)
```py
ListOfInt: TypeAlias = list[int]
PositiveInt = TypeAliasType("PositiveInt", Annotated[int, Gt(0)])
```
[New example](https://play.ruff.rs/bbad34da-bf07-44e6-9f34-53337e8f57d4)
```py
from typing import Annotated, TypeAlias, TypeAliasType
from annotated_types import Gt
ListOfInt: TypeAlias = list[int]
PositiveInt = TypeAliasType("PositiveInt", Annotated[int, Gt(0)])
```
Imports were also added to the "Use instead" section.
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
Part of #18972
This PR makes [timeout-error-alias
(UP041)](https://docs.astral.sh/ruff/rules/timeout-error-alias/#timeout-error-alias-up041)'s
example error out-of-the-box.
[Old example](https://play.ruff.rs/87e20352-d80a-46ec-98a2-6f6ea700438b)
```py
raise asyncio.TimeoutError
```
[New example](https://play.ruff.rs/d3b95557-46a2-4856-bd71-30d5f3f5ca44)
```py
import asyncio
raise asyncio.TimeoutError
```
## Test Plan
<!-- How was it tested? -->
N/A, no functionality/tests affected
## Summary
This was originally stacked on #19129, but some of the changes I made
for JSON also impacted the Azure format, so I went ahead and combined
them. The main changes here are:
- Implementing `FileResolver` for Ruff's `EmitterContext`
- Adding `FileResolver::notebook_index` and `FileResolver::is_notebook`
methods
- Adding a `DisplayDiagnostics` (with an "s") type for rendering a group
of diagnostics at once
- Adding `Azure`, `Json`, and `JsonLines` as new `DiagnosticFormat`s
I tried a couple of alternatives to the `FileResolver::notebook` methods
like passing down the `NotebookIndex` separately and trying to reparse a
`Notebook` from Ruff's `SourceFile`. The latter seemed promising, but
the `SourceFile` only stores the concatenated plain text of the
notebook, not the re-parsable JSON. I guess the current version is just
a variation on passing the `NotebookIndex`, but at least we can reuse
the existing `resolver` argument. I think a lot of this can be cleaned
up once Ruff has its own actual file resolver.
As suggested, I also tried deleting the corresponding `Emitter` files in
`ruff_linter`, but it doesn't look like git was able to follow this as a
rename. It did, however, track that the tests were moved, so the
snapshots should be easy to review.
## Test Plan
Existing Ruff tests ported to tests in `ruff_db`. I think some other
existing ruff tests also cover parts of this refactor.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
See https://github.com/astral-sh/ruff/pull/19133#discussion_r2198413586
for recent discussion. This PR moves to using structs for the types in
our JSON output format instead of the `json!` macro.
I didn't rename any of the `message` references because that should be
handled when rebasing #19133 onto this.
My plan for handling the `preview` behavior with the new diagnostics is
to use a wrapper enum. Something like:
```rust
#[derive(Serialize)]
#[serde(untagged)]
pub(crate) enum JsonDiagnostic<'a> {
Old(OldJsonDiagnostic<'a>),
}
#[derive(Serialize)]
pub(crate) struct OldJsonDiagnostic<'a> {
// ...
}
```
Initially I thought I could use a `&dyn Serialize` for the affected
fields, but I see that `Serialize` isn't dyn-compatible in testing this
now.
## Test Plan
Existing tests. One quirk of the new types is that their fields are in
alphabetical order. I guess `json!` sorts the fields alphabetically? The
tests were failing before I sorted the struct fields.
## Other formats
It looks like the `rdjson`, `sarif`, and `gitlab` formats also use
`json!`, so if we decide to merge this, I can do something similar for
those before moving them to the new diagnostic format.
