mirror of
https://github.com/astral-sh/ruff.git
synced 2025-08-16 16:40:36 +00:00
47c9ed07f2
## Summary This PR fixes a bug introduced in https://github.com/astral-sh/ruff/pull/12008 which didn't consider the two character newline after the line continuation character. For example, consider the following code highlighted with whitespaces: ```py call(foo # comment \\r\n \r\n def bar():\r\n ....pass\r\n ``` The lexer is at `def` when it's running the re-lexing logic and trying to move back to a newline character. It encounters `\n` and it's being escaped (incorrect) but `\r` is being escaped, so it moves the lexer to `\n` character. This creates an overlap in token ranges which causes the panic. ``` Name 0..4 Lpar 4..5 Name 5..8 Comment 9..20 NonLogicalNewline 20..22 <-- overlap between Newline 21..22 <-- these two tokens NonLogicalNewline 22..23 Def 23..26 ... ``` fixes: #12028 ## Test Plan Add a test case with line continuation and windows style newline character. |
||
|---|---|---|
| .. | ||
| lexer | ||
| parser | ||
| snapshots | ||
| error.rs | ||
| lexer.rs | ||
| lib.rs | ||
| string.rs | ||
| token.rs | ||
| token_set.rs | ||
| token_source.rs | ||
| typing.rs | ||