Commit graph

104 commits

Author SHA1 Message Date
konstin
550b643e33
Add script for ecosystem wide checks of all rules and fixes (#4326)
* Add script for ecosystem wide checks of all rules and fixes

This adds my personal script for checking an entire checkout of ~2.1k packages for
panics, autofix errors and similar problems. It's not really meant to be used by anybody else but i thought it's better if it lives in the repo than if it doesn't.

For reference, this is the current output of failing autofixes: https://gist.github.com/konstin/c3fada0135af6cacec74f166adf87a00. Trimmed down to the useful information: https://gist.github.com/konstin/c864f4c300c7903a24fdda49635c5da9

* Keep github template intact

* Remove the need for ripgrep

* sort output
2023-05-22 15:23:25 +02:00
konstin
625849b846
Ecosystem CI: Optionally diff fixes (#4193)
* Generate fixes when using --show-fixes

Example command: `cargo run --bin ruff -- --no-cache --select F401
--show-source --show-fixes
crates/ruff/resources/test/fixtures/pyflakes/F401_9.py`

Before, `--show-fixes` was ignored:

```
crates/ruff/resources/test/fixtures/pyflakes/F401_9.py:4:22: F401 [*] `foo.baz` imported but unused
  |
4 | __all__ = ("bar",)
5 | from foo import bar, baz
  |                      ^^^ F401
  |
  = help: Remove unused import: `foo.baz`

Found 1 error.
[*] 1 potentially fixable with the --fix option.
```

After:

```
crates/ruff/resources/test/fixtures/pyflakes/F401_9.py:4:22: F401 [*] `foo.baz` imported but unused
  |
4 | __all__ = ("bar",)
5 | from foo import bar, baz
  |                      ^^^ F401
  |
  = help: Remove unused import: `foo.baz`

ℹ Suggested fix
1 1 | """Test: late-binding of `__all__`."""
2 2 |
3 3 | __all__ = ("bar",)
4   |-from foo import bar, baz
  4 |+from foo import bar

Found 1 error.
[*] 1 potentially fixable with the --fix option.
```

Also fixes git clone
2023-05-19 09:49:57 +00:00
Charlie Marsh
85f67b2ee3
Make the AST Checker pub(crate) (#4498) 2023-05-18 15:17:26 +00:00
Charlie Marsh
6b1062ccc3
Enable pycodestyle rules under new "nursery" category (#4407) 2023-05-16 21:21:58 +00:00
Charlie Marsh
39fa38cb35
Enable pycodestyle rules (#3689) 2023-05-16 20:39:43 +00:00
Tyler Yep
01b372a75c
Implement flake8-future-annotations FA100 (#3979) 2023-05-14 03:00:06 +00:00
konstin
6a52577630
Ecosystem CI: Allow storing checkouts locally (#4192)
* Ecosystem CI: Allow storing checkouts locally

This adds a --checkouts options to (re)use a local directory instead of checkouts into a tempdir

* Fix missing path conversion
2023-05-11 17:36:44 +02:00
Calum Young
b76b4b6016
List rule changes in ecosystem (#4371)
* Count changes for each rule

* Handle case where rule matches were found in a line

* List and sort by changes

* Remove detail from rule changes

* Add comment about leading :

* Only print rule changes if rule changes are present

* Use re.search and match group

* Remove dict().items()

* Use match group to extract rule code
2023-05-11 16:33:15 +02:00
konstin
0096938789
Optionally show fixes when using --features ecosystem_ci with cargo and --show-fixes at runtime (#4191)
* Generate fixes when using --show-fixes

Example command: `cargo run --bin ruff -- --no-cache --select F401
--show-source --show-fixes
crates/ruff/resources/test/fixtures/pyflakes/F401_9.py`

Before, `--show-fixes` was ignored:

```
crates/ruff/resources/test/fixtures/pyflakes/F401_9.py:4:22: F401 [*] `foo.baz` imported but unused
  |
4 | __all__ = ("bar",)
5 | from foo import bar, baz
  |                      ^^^ F401
  |
  = help: Remove unused import: `foo.baz`

Found 1 error.
[*] 1 potentially fixable with the --fix option.
```

After:

```
crates/ruff/resources/test/fixtures/pyflakes/F401_9.py:4:22: F401 [*] `foo.baz` imported but unused
  |
4 | __all__ = ("bar",)
5 | from foo import bar, baz
  |                      ^^^ F401
  |
  = help: Remove unused import: `foo.baz`

ℹ Suggested fix
1 1 | """Test: late-binding of `__all__`."""
2 2 |
3 3 | __all__ = ("bar",)
4   |-from foo import bar, baz
  4 |+from foo import bar

Found 1 error.
[*] 1 potentially fixable with the --fix option.
```

* Add `--format ecosystem-ci`

* cargo dev generate-all

* Put behind cargo feature

* Regenerate docs

* Don't test ecosystem_ci feature on CI

* Use top level flag instead

* Fix

* Simplify code based on #4191

* Remove old TODO comment
2023-05-10 17:45:57 +02:00
Calum Young
03f141f53d
Check that all rules have descriptions (#4315) 2023-05-09 16:53:23 +00:00
Calum Young
8dea47afc1
Update mkdocs unformatted example error message (#4312) 2023-05-09 12:36:13 -04:00
Calum Young
cd41de2588
Check docs formatting check (#4270) 2023-05-08 19:03:22 +00:00
Aarni Koskela
d7a369e7dc
Update confusable character mapping (#4274) 2023-05-08 14:20:44 -04:00
Jonathan Plasse
a32617911a
Use --filter=blob:none to clone CPython faster (#4156) 2023-04-30 13:39:22 +02:00
Micha Reiser
cab65b25da
Replace row/column based Location with byte-offsets. (#3931) 2023-04-26 18:11:02 +00:00
konstin
19d8913e32
Use musl in ecosystem docker (#3998)
This prevents errors when the host glibc is newer than the one in the docker container
2023-04-26 05:54:53 +02:00
Charlie Marsh
e160a52bfd
Raise percent-format upgrade rule (UP031) for hanging modulos (#3953) 2023-04-12 23:59:20 -04:00
Micha Reiser
e8aebee3f6
Pretty print Diagnostics in snapshot tests (#3906) 2023-04-11 09:03:00 +00:00
konstin
454c6d9c2f
Extended ecosystem check with scraped data (#3858) 2023-04-06 22:39:48 +00:00
Jonathan Plasse
f3f9a9f297
Fix pre-commit CI job exit code (#3833) 2023-03-31 14:47:04 -04:00
Jonathan Plasse
fec4fa39a7
Improve add_rule.py and add_plugin.py scripts (#3725) 2023-03-25 16:05:39 +00:00
Jonathan Plasse
7f3b748401
Fix Ruff pre-commit hook errors (#3706) 2023-03-23 22:52:24 -04:00
Zhengbo Wang
b08326162b
Doc/CLN: pass pre-commit (#3604) 2023-03-19 19:20:11 +00:00
Henry Schreiner
4bdb2dd362
ci(check_ecosystem): add PyPa/build (#3569) 2023-03-18 19:09:22 -04:00
Henry Schreiner
53a4743631
ci: fix check_ecosystem (#3602) 2023-03-18 19:03:08 -04:00
Charlie Marsh
16a350c731
Reduce usage of ALL in ecosystem CI (#3590) 2023-03-18 13:13:09 -04:00
Henry Schreiner
d9ed0aae69
ci(check_ecosystem): add cibuildwheel (#3567) 2023-03-16 22:34:56 -04:00
Henry Schreiner
bbc87b7177
ci(check_ecosystem): add scikit-build-core (#3563) 2023-03-16 19:46:42 -04:00
Xuehai Pan
e99e1fae2b
ci: add python/typeshed to ecosystem check (#3559) 2023-03-16 14:19:48 -04:00
Calum Young
6c576872d4
List changes for all ecosystem repos (#3461) 2023-03-12 14:30:38 -04:00
Charlie Marsh
7062d1db16
Run ecosystem CI checks without --isolated (#3445) 2023-03-10 19:03:51 -05:00
Samuel Cormier-Iijima
cfa2924664
Setup ecosystem CI (#3390)
This PR sets up an "ecosystem" check as an optional part of the CI step for pull requests. The primary piece of this is a new script in `scripts/check_ecosystem.py` which takes two ruff binaries as input and compares their outputs against a corpus of open-source code in parallel. I used ruff's `text` reporting format and stdlib's `difflib` (rather than JSON output and jsondiffs) to avoid adding another dependency. There is a new ecosystem-comment workflow to add a comment to the PR (see [this link](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/) which explains why it needs to be done as a new workflow for security reasons).
2023-03-10 17:39:07 -05:00
Charlie Marsh
024caca233
Introduce a ruff_diagnostics crate (#3409)
## Summary

This PR moves `Diagnostic`, `DiagnosticKind`, and `Fix` into their own crate, which will enable us to further split up Ruff, since sub-linter crates (which need to implement functions that return `Diagnostic`) can now depend on `ruff_diagnostics` rather than Ruff.
2023-03-09 20:48:57 +00:00
konstin
348a38d261
Deprecate define violation (#3358)
* Add `#[violation]` proc macro as a replacement for `define_violation!`

* Switch all rules to #[violation]
2023-03-06 10:59:06 +00:00
konstin
2168404fc2
flake8-pyi PYI006 bad version info comparison (#3291)
Implement PYI006 "bad version info comparison"

## What it does

Ensures that you only `<` and `>=` for version info comparisons with
`sys.version_info` in `.pyi` files. All other comparisons such as
`<`, `<=` and `==` are banned.

## Why is this bad?

```python
>>> import sys
>>> print(sys.version_info)
sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0)
>>> print(sys.version_info > (3, 8))
True
>>> print(sys.version_info == (3, 8))
False
>>> print(sys.version_info <= (3, 8))
False
>>> print(sys.version_info in (3, 8))
False
```

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
2023-03-01 18:58:57 +01:00
Charlie Marsh
8066607ea3
Add a preliminary tutorial (#3281) 2023-02-28 20:31:27 +00:00
Jonathan Plasse
d285f5c90a
Run automatically format code blocks with Black (#3191) 2023-02-27 10:14:05 -05:00
Carlos Gonçalves
1c01b3c934
fix(docs): broken links inside Configuration.md (#3205) 2023-02-24 18:55:33 +00:00
Jonathan Plasse
39b9a1637f
Fix Markdown errors in docs (#3187) 2023-02-24 13:06:48 -05:00
Jonathan Plasse
6e54cd8233
Normalize relative markdown links (#3190) 2023-02-23 16:24:31 -05:00
Jonathan Plasse
a688a237d7
Add black to pre-commit (#3192) 2023-02-23 16:24:23 -05:00
Charlie Marsh
74e18b6cff
Split up some docs sections (#3154) 2023-02-22 20:18:10 -05:00
Charlie Marsh
b9bfb81e36
Move configuration out of README and into permanent docs (#3150) 2023-02-22 19:25:53 -05:00
Micha Reiser
f72ed255e5
chore: Use LF on all platforms (#3005)
I worked on #2993 and ran into issues that the formatter tests are failing on Windows because `writeln!` emits `\n` as line terminator on all platforms, but `git` on Windows converted the line endings in the snapshots to `\r\n`.

I then tried to replicate the issue on my Windows machine and was surprised that all linter snapshot tests are failing on my machine. I figured out after some time that it is due to my global git config keeping the input line endings rather than converting to `\r\n`. 

Luckily, I've been made aware of #2033 which introduced an "override" for the `assert_yaml_snapshot` macro that normalizes new lines, by splitting the formatted string using the platform-specific newline character. This is a clever approach and gives nice diffs for multiline fixes but makes assumptions about the setup contributors use and requires special care whenever we use line endings inside of tests. 

I recommend that we remove the special new line handling and use `.gitattributes` to enforce the use of `LF` on all platforms [guide](https://docs.github.com/en/get-started/getting-started-with-git/configuring-git-to-handle-line-endings). This gives us platform agnostic tests without having to worry about line endings in our tests or different git configurations.

## Note

It may be necessary for Windows contributors to run the following command to update the line endings of their files

```bash
git rm --cached -r .
git reset --hard
```
2023-02-20 20:13:37 +00:00
Charlie Marsh
c25be31eb1
Fix documentation-link detection in generate_mkdocs.py (#3030) 2023-02-19 12:20:44 +00:00
Charlie Marsh
cd1f57b713
Move FAQ into MkDocs (#2984) 2023-02-17 13:15:53 +00:00
Charlie Marsh
a0912deb2b
Move editor integrations into MkDocs (#2983) 2023-02-17 13:12:20 +00:00
Martin Fischer
d658bfc024 Remove options from README 2023-02-17 07:55:50 -05:00
Martin Fischer
8195873cdf Remove rule tables from README 2023-02-17 07:55:50 -05:00
Charlie Marsh
298498e934
Add an Acknowledgements section to the README (#2907) 2023-02-15 00:25:07 +00:00