Document ruff_dev and format_dev (#5648)

## Summary

Document all `ruff_dev` subcommands and document the `format_dev` flags
in the formatter readme.

CC @zanieb please flag everything that isn't clear or missing

## Test Plan

n/a
This commit is contained in:
konsti 2023-07-12 16:18:22 +02:00 committed by GitHub
parent 5665968b42
commit f0aa6bd4d3
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 145 additions and 2 deletions

View file

@ -550,3 +550,134 @@ cargo instruments -t time --bench linter --profile release-debug -p ruff_benchma
- You may want to pass an additional filter to run a single test file
Otherwise, follow the instructions from the linux section.
## `cargo dev`
`cargo dev` is a shortcut for `cargo run --package ruff_dev --bin ruff_dev`. You can run some useful
utils with it:
- `cargo dev print-ast <file>`: Print the AST of a python file using the
[RustPython parser](https://github.com/astral-sh/RustPython-Parser/tree/main/parser) that is
mainly used in Ruff. For `if True: pass # comment`, you can see the syntax tree, the byte offsets
for start and stop of each node and also how the `:` token, the comment and whitespace are not
represented anymore:
```text
[
If(
StmtIf {
range: 0..13,
test: Constant(
ExprConstant {
range: 3..7,
value: Bool(
true,
),
kind: None,
},
),
body: [
Pass(
StmtPass {
range: 9..13,
},
),
],
orelse: [],
},
),
]
```
- `cargo dev print-tokens <file>`: Print the tokens that the AST is built upon. Again for
`if True: pass # comment`:
```text
0 If 2
3 True 7
7 Colon 8
9 Pass 13
14 Comment(
"# comment",
) 23
23 Newline 24
```
- `cargo dev print-cst <file>`: Print the CST of a python file using
[LibCST](https://github.com/Instagram/LibCST), which is used in addition to the RustPython parser
in Ruff. E.g. for `if True: pass # comment` everything including the whitespace is represented:
```text
Module {
body: [
Compound(
If(
If {
test: Name(
Name {
value: "True",
lpar: [],
rpar: [],
},
),
body: SimpleStatementSuite(
SimpleStatementSuite {
body: [
Pass(
Pass {
semicolon: None,
},
),
],
leading_whitespace: SimpleWhitespace(
" ",
),
trailing_whitespace: TrailingWhitespace {
whitespace: SimpleWhitespace(
" ",
),
comment: Some(
Comment(
"# comment",
),
),
newline: Newline(
None,
Real,
),
},
},
),
orelse: None,
leading_lines: [],
whitespace_before_test: SimpleWhitespace(
" ",
),
whitespace_after_test: SimpleWhitespace(
"",
),
is_elif: false,
},
),
),
],
header: [],
footer: [],
default_indent: " ",
default_newline: "\n",
has_trailing_newline: true,
encoding: "utf-8",
}
```
- `cargo dev generate-all`: Update `ruff.schema.json`, `docs/configuration.md` and `docs/rules`.
You can also set `RUFF_UPDATE_SCHEMA=1` to update `ruff.schema.json` during `cargo test`.
- `cargo dev generate-cli-help`, `cargo dev generate-docs` and `cargo dev generate-json-schema`:
Update just `docs/configuration.md`, `docs/rules` and `ruff.schema.json` respectively.
- `cargo dev generate-options`: Generate a markdown-compatible table of all `pyproject.toml`
options. Used for <https://beta.ruff.rs/docs/settings/>
- `cargo dev generate-rules-table`: Generate a markdown-compatible table of all rules. Used for <https://beta.ruff.rs/docs/rules/>
- `cargo dev round-trip <python file or jupyter notebook>`: Read a Python file or Jupyter Notebook,
parse it, serialize the parsed representation and write it back. Used to check how good our
representation is so that fixes don't rewrite irrelevant parts of a file.
- `cargo dev format_dev`: See ruff_python_formatter README.md

View file

@ -21,7 +21,7 @@ pub(crate) enum Mode {
/// Don't write to the file, check if the file is up-to-date and error if not.
Check,
/// Write the generated help to stdout (rather than to `docs/configuration.md`).
/// Write the generated help to stdout.
DryRun,
}

View file

@ -1,4 +1,6 @@
//! Generate a Markdown-compatible listing of configuration options.
//! Generate a Markdown-compatible listing of configuration options for `pyproject.toml`.
//!
//! Used for <https://beta.ruff.rs/docs/settings/>.
use itertools::Itertools;
use ruff::settings::options::Options;

View file

@ -1,4 +1,6 @@
//! Generate a Markdown-compatible table of supported lint rules.
//!
//! Used for <https://beta.ruff.rs/docs/rules/>.
use itertools::Itertools;
use strum::IntoEnumIterator;

View file

@ -276,6 +276,14 @@ python scripts/check_ecosystem.py --checkouts target/checkouts --projects github
cargo run --bin ruff_dev -- format-dev --stability-check --multi-project target/checkouts
```
Compared to `ruff check`, `cargo run --bin ruff_dev -- format-dev` has 4 additional options:
- `--write`: Format the files and write them back to disk
- `--stability-check`: Format twice (but don't write to disk) and check for differences and crashes
- `--multi-project`: Treat every subdirectory as a separate project. Useful for ecosystem checks.
- `--error-file`: Use together with `--multi-project`, this writes all errors (but not status
messages) to a file.
## The orphan rules and trait structure
For the formatter, we would like to implement `Format` from the rust_formatter crate for all AST