ruff/crates/ruff_python_parser
Dhruv Manilawala d3cd61f804
Use empty range when there's "gap" in token source (#11032)
## Summary

This fixes a bug where the parser would panic when there is a "gap" in
the token source.

What's a gap?

The reason it's `<=` instead of just `==` is because there could be
whitespaces between
the two tokens. For example:

```python
#     last token end
#     | current token (newline) start
#     v v
def foo \n
#      ^
#      assume there's trailing whitespace here
```

Or, there could tokens that are considered "trivia" and thus aren't
emitted by the token
source. These are comments and non-logical newlines. For example:

```python
#     last token end
#     v
def foo # comment\n
#                ^ current token (newline) start
```

In either of the above cases, there's a "gap" between the end of the
last token and start
of the current token.

## Test Plan

Add test cases and update the snapshots.
2024-04-19 11:36:26 +00:00
..
resources Use empty range when there's "gap" in token source (#11032) 2024-04-19 11:36:26 +00:00
src Use empty range when there's "gap" in token source (#11032) 2024-04-19 11:36:26 +00:00
tests Use empty range when there's "gap" in token source (#11032) 2024-04-19 11:36:26 +00:00
Cargo.toml Replace LALRPOP parser with hand-written parser (#10036) 2024-04-18 17:57:39 +05:30
CONTRIBUTING.md Replace LALRPOP parser with hand-written parser (#10036) 2024-04-18 17:57:39 +05:30