Pass ParserOptions to the parser (#16220)

## Summary

This is part of the preparation for detecting syntax errors in the
parser from https://github.com/astral-sh/ruff/pull/16090/. As suggested
in [this
comment](https://github.com/astral-sh/ruff/pull/16090/#discussion_r1953084509),
I started working on a `ParseOptions` struct that could be stored in the
parser. For this initial refactor, I only made it hold the existing
`Mode` option, but for syntax errors, we will also need it to have a
`PythonVersion`. For that use case, I'm picturing something like a
`ParseOptions::with_python_version` method, so you can extend the
current calls to something like

```rust
ParseOptions::from(mode).with_python_version(settings.target_version)
```

But I thought it was worth adding `ParseOptions` alone without changing
any other behavior first.

Most of the diff is just updating call sites taking `Mode` to take
`ParseOptions::from(Mode)` or those taking `PySourceType`s to take
`ParseOptions::from(PySourceType)`. The interesting changes are in the
new `parser/options.rs` file and smaller parts of `parser/mod.rs` and
`ruff_python_parser/src/lib.rs`.

## Test Plan

Existing tests, this should not change any behavior.
This commit is contained in:
Brent Westbrook 2025-02-19 10:50:50 -05:00 committed by GitHub
parent cfc6941d5c
commit 97d0659ce3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 148 additions and 93 deletions

View file

@ -5,7 +5,7 @@ pub use range::format_range;
use ruff_formatter::prelude::*;
use ruff_formatter::{format, write, FormatError, Formatted, PrintError, Printed, SourceCode};
use ruff_python_ast::{AnyNodeRef, Mod};
use ruff_python_parser::{parse, AsMode, ParseError, Parsed};
use ruff_python_parser::{parse, ParseError, ParseOptions, Parsed};
use ruff_python_trivia::CommentRanges;
use ruff_text_size::Ranged;
@ -112,7 +112,7 @@ pub fn format_module_source(
options: PyFormatOptions,
) -> Result<Printed, FormatModuleError> {
let source_type = options.source_type();
let parsed = parse(source, source_type.as_mode())?;
let parsed = parse(source, ParseOptions::from(source_type))?;
let comment_ranges = CommentRanges::from(parsed.tokens());
let formatted = format_module_ast(&parsed, &comment_ranges, source, options)?;
Ok(formatted.print()?)
@ -154,7 +154,7 @@ mod tests {
use insta::assert_snapshot;
use ruff_python_ast::PySourceType;
use ruff_python_parser::{parse, AsMode};
use ruff_python_parser::{parse, ParseOptions};
use ruff_python_trivia::CommentRanges;
use ruff_text_size::{TextRange, TextSize};
@ -199,7 +199,7 @@ def main() -> None:
// Parse the AST.
let source_path = "code_inline.py";
let parsed = parse(source, source_type.as_mode()).unwrap();
let parsed = parse(source, ParseOptions::from(source_type)).unwrap();
let comment_ranges = CommentRanges::from(parsed.tokens());
let options = PyFormatOptions::from_extension(Path::new(source_path));
let formatted = format_module_ast(&parsed, &comment_ranges, source, options).unwrap();