ruff/crates/ruff_python_parser
Dhruv Manilawala 60ea72a6bc
Add list terminator kind for error recovery (#11843)
## Summary

This PR adds a new enum to determine the kind of terminator token i.e.,
is it actually terminates the list or is it used for error recovery.

This is important because the parser should take the error recovery
route in case the terminator token is used for better error recovery.
This will then try to re-lex the token if it's the case.

I haven't updated any reference to use this new enum as otherwise it'll
update the snapshots. I plan to do that in a follow-up PR so that it's
easier to reason about.

## Test plan

`cargo insta test`
2024-06-12 08:33:26 +00:00
..
resources Use speculative parsing for with-items (#11770) 2024-06-06 08:59:56 +00:00
src Add list terminator kind for error recovery (#11843) 2024-06-12 08:33:26 +00:00
tests Separate terminator token for f-string elements kind (#11842) 2024-06-12 13:57:35 +05:30
Cargo.toml Remove less used parser dependencies (#11718) 2024-06-03 13:08:24 +00:00
CONTRIBUTING.md Add basic docs for the parser crate (#11199) 2024-04-29 17:08:07 +00:00
README.md Add basic docs for the parser crate (#11199) 2024-04-29 17:08:07 +00:00

Ruff Python Parser

Ruff's Python parser is a hand-written recursive descent parser which can parse Python source code into an Abstract Syntax Tree (AST). It also utilizes the Pratt parsing technique to parse expressions with different precedence.

Try out the parser in the playground.

Python version support

The parser supports the latest Python syntax, which is currently Python 3.12. It does not throw syntax errors if it encounters a syntax feature that is not supported by the target-version. This will be fixed in a future release (see https://github.com/astral-sh/ruff/issues/6591).

Contributing

Refer to the contributing guidelines to get started and GitHub issues with the parser label for issues that need help.