Changes the title and adds some notes re the old formatter ecosystem
checks in light of #8223
Does not remove it as I'm not sure where else we test for instabilities.
Closes#7239
- Refactors `scripts/check_ecosystem.py` into a new Python project at
`python/ruff-ecosystem`
- Includes
[documentation](https://github.com/astral-sh/ruff/blob/zanie/ecosystem-format/python/ruff-ecosystem/README.md)
now
- Provides a `ruff-ecosystem` CLI
- Fixes bug where `ruff check` report included "fixable" summary line
- Adds truncation to `ruff check` reports
- Otherwise we often won't see the `ruff format` reports
- The truncation uses some very simple heuristics and could be improved
in the future
- Identifies diagnostic changes that occur just because a violation's
fix available changes
- We still show the diff for the line because it's could matter _where_
this changes, but we could improve this
- Similarly, we could improve detection of diagnostic changes where just
the message changes
- Adds support for JSON ecosystem check output
- I added this primarily for development purposes
- If there are no changes, only errors while processing projects, we
display a different summary message
- When caching repositories, we now checkout the requested ref
- Adds `ruff format` reports, which format with the baseline then the
use `format --diff` to generate a report
- Runs all CI jobs when the CI workflow is changed
## Known problems
- Since we must format the project to get a baseline, the permalink line
numbers do not exactly correspond to the correct range
- This looks... hard. I tried using `git diff` and some wonky hunk
matching to recover the original line numbers but it doesn't seem worth
it. I think we should probably commit the formatted changes to a fork or
something if we want great results here. Consequently, I've just used
the start line instead of a range for now.
- I don't love the comment structure — it'd be nice, perhaps, to have
separate headings for the linter and formatter.
- However, the `pr-comment` workflow is an absolute pain to change
because it runs _separately_ from this pull request so I if I want to
make edits to it I can only test it via manual workflow dispatch.
- Lines are not printed "as we go" which means they're all held in
memory, presumably this would be a problem for large-scale ecosystem
checks
- We are encountering a hard limit with the maximum comment length
supported by GitHub. We will need to move the bulk of the report
elsewhere.
## Future work
- Update `ruff-ecosystem` to support non-default projects and
`check_ecosystem_all.py` behavior
- Remove existing ecosystem check scripts
- Add preview mode toggle (#8076)
- Add a toggle for truncation
- Add hints for quick reproduction of runs locally
- Consider parsing JSON output of Ruff instead of using regex to parse
the text output
- Links to project repositories should use the commit hash we checked
against
- When caching repositories, we should pull the latest changes for the
ref
- Sort check diffs by path and rule code only (changes in messages
should not change order)
- Update check diffs to distinguish between new violations and changes
in messages
- Add "fix" diffs
- Remove existing formatter similarity reports
- On release pull request, compare to the previous tag instead
---------
Co-authored-by: konsti <konstin@mailbox.org>
**Summary** Prepare for the black preview style becoming the black
stable style at the end of the year.
This adds a new test file to compare stable and preview on some relevant
preview options in black, and makes `format_dev` understand the black
preview flag. I've added poetry as a project that uses preview.
I've implemented one specific deviation (collapsing of stub
implementation in non-stub files) which showed up in poetry for testing.
This also improves poetry compatibility from 0.99891 to 0.99919.
Fixes#7440
New compatibility stats:
| project | similarity index | total files | changed files |
|----------------|------------------:|------------------:|------------------:|
| cpython | 0.75803 | 1799 | 1647 |
| django | 0.99983 | 2772 | 35 |
| home-assistant | 0.99953 | 10596 | 189 |
| poetry | 0.99919 | 317 | 12 |
| transformers | 0.99963 | 2657 | 332 |
| twine | 1.00000 | 33 | 0 |
| typeshed | 0.99978 | 3669 | 20 |
| warehouse | 0.99969 | 654 | 15 |
| zulip | 0.99970 | 1459 | 22 |
## Summary
This PR updates our documentation for the upcoming formatter release.
Broadly, the documentation is now structured as follows:
- Overview
- Tutorial
- Installing Ruff
- The Ruff Linter
- Overview
- `ruff check`
- Rule selection
- Error suppression
- Exit codes
- The Ruff Formatter
- Overview
- `ruff format`
- Philosophy
- Configuration
- Format suppression
- Exit codes
- Black compatibility
- Known deviations
- Configuring Ruff
- pyproject.toml
- File discovery
- Configuration discovery
- CLI
- Shell autocompletion
- Preview
- Rules
- Settings
- Integrations
- `pre-commit`
- VS Code
- LSP
- PyCharm
- GitHub Actions
- FAQ
- Contributing
The major changes include:
- Removing the "Usage" section from the docs, and instead folding that
information into "Integrations" and the new Linter and Formatter
sections.
- Breaking up "Configuration" into "Configuring Ruff" (for generic
configuration), and new Linter- and Formatter-specific sections.
- Updating all example configurations to use `[tool.ruff.lint]` and
`[tool.ruff.format]`.
My suggestion is to pull and build the docs locally, and review by
reading them in the browser rather than trying to parse all the code
changes.
Closes https://github.com/astral-sh/ruff/issues/7235.
Closes https://github.com/astral-sh/ruff/issues/7647.
- Add changelog entry for 0.1.1
- Bump version to 0.1.1
- Require preview for fix added in #7967
- Allow duplicate headings in changelog (markdownlint setting)
## Summary
We don't enable E501 by default, but `line-length` is a useful example
for configuration, so we now set `--extend-select` in the tutorial with
a note to that effect.
I've also updated all the outputs to match the latest CLI behavior, and
changed the example from `List` to `Sequence` because `List` now spits
out two diagnostics (one for the import, one for the usage), which IMO
is confusing for beginners.
## Summary
The markdown documentation was present, but in the wrong place, so was
not displaying on the website. I moved it and added some references.
Related to #2646.
## Test Plan
`python scripts/check_docs_formatted.py`
## Summary
Fix CI (broken in #7496).
The code snippet was formatted as Black would format a stub file, but
the CI script doesn't know that (it assumes all code snippets are
non-stub files). Easier to ignore.
Sorry for breaking CI!
## Test Plan
`python scripts/check_docs_formatted.py`
## Summary
We're planning to move the documentation from
[https://beta.ruff.rs/docs](https://beta.ruff.rs/docs) to
[https://docs.astral.sh/ruff](https://docs.astral.sh/ruff), for a few
reasons:
1. We want to remove the `beta` from the domain, as Ruff is no longer
considered beta software.
2. We want to migrate to a structure that could accommodate multiple
future tools living under one domain.
The docs are actually already live at
[https://docs.astral.sh/ruff](https://docs.astral.sh/ruff), but later
today, I'll add a permanent redirect from the previous to the new
domain. **All existing links will continue to work, now and in
perpetuity.**
This PR contains the code changes necessary for the updated
documentation. As part of this effort, I moved the playground and
documentation from my personal Cloudflare account to our team Cloudflare
account (hence the new `--project-name` references). After merging, I'll
also update the secrets on this repo.
## Summary
This PR adds a benchmarking script for the formatter, which benchmarks
the Ruff formatter against Black, yapf, and autopep8.
Three benchmarks are included:
1. Format everything.
2. Format everything, but use a single thread.
3. Format everything, but `--check` (don't write to disk).
There's some nuance in figuring out the right combination of arguments
to each command, but the _main_ nuance is to ensure that we always run
the given formatter (and modify the target repo in-place) prior to
benchmarking it, so that the formatters aren't disadvantaged by the
existing formatting of the target repo. (E.g.: prior to benchmarking
Black's preview style, we need to make sure we format the target repo
with Black's preview style; otherwise, preview style appears much
slower.)
Part of https://github.com/astral-sh/ruff/issues/7309.
With https://github.com/django/django/pull/17181 merged, this removes an
odd edge case (tuple expression statements aka bogus trailing commas
after statements that turn them into a tuple without you noticing) that
we don't want to care about because the input code is ~wrong from the
similarity index. I've took this opportunity to update the revisions of
all projects we test.
main
| project | similarity index |
|--------------|------------------|
| cpython | 0.75477 |
| django | 0.99814 |
| transformers | 0.99621 |
| twine | 0.99876 |
| typeshed | 0.99953 |
| warehouse | 0.99601 |
| zulip | 0.99727 |
this PR
| project | similarity index |
|--------------|------------------|
| cpython | 0.75996 |
| django | 0.99819 |
| transformers | 0.99622 |
| twine | 0.99876 |
| typeshed | 0.99953 |
| warehouse | 0.99607 |
| zulip | 0.99729 |
## Summary
In #6387, we accidentally added `git -C "$dir/django" checkout
95e4d6b81312fdd9f8ebf3385be1c1331168b5cf` as the transformers checkout
(duplicated line from the Django case). This PR fixes the SHA, and
spaces out the cases to make it more visible. I _think_ the net effect
here is that we've been formatting `main` on transformers, rather than
the SHA?
Adding five new projects. Some of these have seen issues filed, the
others, I just tabbed through our dependency pain and looked for some
reasonably-large projects that enabled rules beyond the default rule
set.
From the formatter progress CI logs:
```
2023-08-07T03:49:02.5178602Z + mkdir -p /home/runner/work/ruff/ruff/target/progress_projects
2023-08-07T03:49:02.5193474Z + '[' '!' -d /home/runner/work/ruff/ruff/target/progress_projects/build ']'
2023-08-07T03:49:02.5194228Z + '[' '!' -d /home/runner/work/ruff/ruff/target/progress_projects/django ']'
2023-08-07T03:49:02.5194966Z + git clone --filter=tree:0 https://github.com/django/django /home/runner/work/ruff/ruff/target/progress_projects/django
2023-08-07T03:49:02.5209260Z Cloning into '/home/runner/work/ruff/ruff/target/progress_projects/django'...
```
```
2023-08-07T03:51:17.4726088Z [2m2023-08-07T03:51:17.472404Z[0m [31mERROR[0m Failed /home/runner/work/ruff/ruff/target/progress_projects/build: no python files in ["/home/runner/work/ruff/ruff/target/progress_projects/build"]
```
Seems that build exists but is an empty cached folder. These changes
should fix this by a) checking for `.git` instead of just the folder
existing b) running the commit checkout unconditionally. The latter is
also important if we ever want to update the SHAs.
**Summary** Prompted by
https://github.com/astral-sh/ruff/pull/6257#issuecomment-1661308410, it
tried to make the ecosystem script output on failure better
understandable. All log messages are now written to a file, which is
printed on error. Running locally progress is still shown.
Looking through the log output i saw that we currently log syntax errors
in input, which is confusing because they aren't actual errors, but we
don't check that these files don't change due to parser regressions or
improvements. I added `--files-with-errors` to catch that.
**Test Plan** CI
Adds rule to convert type aliases defined with annotations i.e. `x:
TypeAlias = int` to the new PEP-695 syntax e.g. `type x = int`.
Does not support using new generic syntax for type variables, will be
addressed in a follow-up.
Added as part of pyupgrade — ~the code 100 as chosen to avoid collision
with real pyupgrade codes~.
Part of #4617
Builds on #5062
**Summary** Print the errors when the formatter ecosystem checks failed.
Im not happy that we current collect the log in the first place, but
this is the less invasive change and we need it to unblock reviewing
#6152.
**Test Plan**
1547787940
## Summary
This PR implements pycodestyle's E241 (tab after comma) and E242
(multiple whitespace after comma) lints.
These are marked as nursery rules like many other pycodestyle rules.
Refs #2402
## Test Plan
E24.py copied from pycodestyle.
**Summary**
Updated doc comments for `missing_whitespace_around_operator.rs`. Online
docs also benefit from this update.
**Test Plan**
Checked docs via
[mkdocs](389fe13c93/CONTRIBUTING.md (L267-L296))
## Summary
This is an error message only change to lead an implementor of a new
rule that has an unformatted or invalid bad example to the
right code.
## Test Plan
n/a
## Summary
Updated doc comment for `tab_indentation.rs`. Online docs also benefit
from this update.
## Test Plan
Checked docs via
[mkdocs](389fe13c93/CONTRIBUTING.md (L267-L296))
**Summary** Add a formatter progress testing script to CI. This script
will 1) print the black compability on each run 2) catch regressions wrt
to formatter stability, emitting invalid syntax and other kinds of bugs
(e.g. #5917) before they land on main 3) have an additional layer of
real world tests when implementing new nodes or other new formatter
code.
This is currently a bash script, i'm not sure if we want to keep it that
way, or switch to e.g. the regular ecosystem scripts. The output
separation of `format_dev` could also use some polishing. We should also
consider pinning commits so we don't get spurious regression when they
change their code.
**Test Plan** The script extends CI.
