ruff/crates/ruff_python_parser/tests
Dhruv Manilawala 47c9ed07f2
Consider 2-character EOL before line continuation (#12035)
## Summary

This PR fixes a bug introduced in
https://github.com/astral-sh/ruff/pull/12008 which didn't consider the
two character newline after the line continuation character.

For example, consider the following code highlighted with whitespaces:
```py
call(foo # comment \\r\n
\r\n
def bar():\r\n
....pass\r\n
```
The lexer is at `def` when it's running the re-lexing logic and trying
to move back to a newline character. It encounters `\n` and it's being
escaped (incorrect) but `\r` is being escaped, so it moves the lexer to
`\n` character. This creates an overlap in token ranges which causes the
panic.

```
Name 0..4
Lpar 4..5
Name 5..8
Comment 9..20
NonLogicalNewline 20..22 <-- overlap between
Newline 21..22           <-- these two tokens
NonLogicalNewline 22..23
Def 23..26
...
```

fixes: #12028 

## Test Plan

Add a test case with line continuation and windows style newline
character.
2024-06-26 14:00:48 +05:30
..
snapshots Consider 2-character EOL before line continuation (#12035) 2024-06-26 14:00:48 +05:30
fixtures.rs Update parser tests to validate token ranges (#12019) 2024-06-25 08:14:28 +00:00
generate_inline_tests.rs Replace LALRPOP parser with hand-written parser (#10036) 2024-04-18 17:57:39 +05:30