Add basic docs for the parser crate (#11199)

## Summary

This PR adds a basic README for the `ruff_python_parser` crate and
updates the CONTRIBUTING docs with the fuzzer and benchmark section.

Additionally, it also updates some inline documentation within the
parser crate and splits the `parse_program` function into
`parse_single_expression` and `parse_module` which will be called by
matching against the `Mode`.

This PR doesn't go into too much internal detail around the parser logic
due to the following reasons:
1. Where should the docs go? Should it be as a module docs in `lib.rs`
or in README?
2. The parser is still evolving and could include a lot of refactors
with the future work (feedback loop and improved error recovery and
resilience)

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
This commit is contained in:
Dhruv Manilawala 2024-04-29 22:38:07 +05:30 committed by GitHub
parent 0ed7af35ec
commit 04a922866a
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 199 additions and 67 deletions

View file

@ -0,0 +1,22 @@
# Ruff Python Parser
Ruff's Python parser is a hand-written [recursive descent parser] which can parse
Python source code into an Abstract Syntax Tree (AST). It also utilizes the [Pratt
parsing](https://matklad.github.io/2020/04/13/simple-but-powerful-pratt-parsing.html)
technique to parse expressions with different [precedence](https://docs.python.org/3/reference/expressions.html#operator-precedence).
Try out the parser in the [playground](https://play.ruff.rs/?secondary=AST).
## Python version support
The parser supports the latest Python syntax, which is currently Python 3.12.
It does not throw syntax errors if it encounters a syntax feature that is not
supported by the [`target-version`](https://docs.astral.sh/ruff/settings/#target-version).
This will be fixed in a future release (see <https://github.com/astral-sh/ruff/issues/6591>).
## Contributing
Refer to the [contributing guidelines](./CONTRIBUTING.md) to get started and GitHub issues with the
[parser label](https://github.com/astral-sh/ruff/issues?q=is:open+is:issue+label:parser) for issues that need help.
[recursive descent parser]: https://en.wikipedia.org/wiki/Recursive_descent_parser