An extremely fast Python linter and code formatter, written in Rust.
Find a file
Charlie Marsh 5ea3e42513
Always use identifier ranges to store bindings (#5110)
## Summary

At present, when we store a binding, we include a `TextRange` alongside
it. The `TextRange` _sometimes_ matches the exact range of the
identifier to which the `Binding` is linked, but... not always.

For example, given:

```python
x = 1
```

The binding we create _will_ use the range of `x`, because the left-hand
side is an `Expr::Name`, which has a valid range on it.

However, given:

```python
try:
  pass
except ValueError as e:
  pass
```

When we create a binding for `e`, we don't have a `TextRange`... The AST
doesn't give us one. So we end up extracting it via lexing.

This PR extends that pattern to the rest of the binding kinds, to ensure
that whenever we create a binding, we always use the range of the bound
name. This leads to better diagnostics in cases like pattern matching,
whereby the diagnostic for "unused variable `x`" here used to include
`*x`, instead of just `x`:

```python
def f(provided: int) -> int:
    match provided:
        case [_, *x]:
            pass
```

This is _also_ required for symbol renames, since we track writes as
bindings -- so we need to know the ranges of the bound symbols.

By storing these bindings precisely, we can also remove the
`binding.trimmed_range` abstraction -- since bindings already use the
"trimmed range".

To implement this behavior, I took some of our existing utilities (like
the code we had for `except ValueError as e` above), migrated them from
a full lexer to a zero-allocation lexer that _only_ identifies
"identifiers", and moved the behavior into a trait, so we can now do
`stmt.identifier(locator)` to get the range for the identifier.

Honestly, we might end up discarding much of this if we decide to put
ranges on all identifiers
(https://github.com/astral-sh/RustPython-Parser/pull/8). But even if we
do, this will _still_ be a good change, because the lexer introduced
here is useful beyond names (e.g., we use it find the `except` keyword
in an exception handler, to find the `else` after a `for` loop, and so
on). So, I'm fine committing this even if we end up changing our minds
about the right approach.

Closes #5090.

## Benchmarks

No significant change, with one statistically significant improvement
(-2.1654% on `linter/all-rules/large/dataset.py`):

```
linter/default-rules/numpy/globals.py
                        time:   [73.922 µs 73.955 µs 73.986 µs]
                        thrpt:  [39.882 MiB/s 39.898 MiB/s 39.916 MiB/s]
                 change:
                        time:   [-0.5579% -0.4732% -0.3980%] (p = 0.00 < 0.05)
                        thrpt:  [+0.3996% +0.4755% +0.5611%]
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) low severe
  1 (1.00%) low mild
  1 (1.00%) high mild
linter/default-rules/pydantic/types.py
                        time:   [1.4909 ms 1.4917 ms 1.4926 ms]
                        thrpt:  [17.087 MiB/s 17.096 MiB/s 17.106 MiB/s]
                 change:
                        time:   [+0.2140% +0.2741% +0.3392%] (p = 0.00 < 0.05)
                        thrpt:  [-0.3380% -0.2734% -0.2136%]
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
linter/default-rules/numpy/ctypeslib.py
                        time:   [688.97 µs 691.34 µs 694.15 µs]
                        thrpt:  [23.988 MiB/s 24.085 MiB/s 24.168 MiB/s]
                 change:
                        time:   [-1.3282% -0.7298% -0.1466%] (p = 0.02 < 0.05)
                        thrpt:  [+0.1468% +0.7351% +1.3461%]
                        Change within noise threshold.
Found 15 outliers among 100 measurements (15.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  12 (12.00%) high severe
linter/default-rules/large/dataset.py
                        time:   [3.3872 ms 3.4032 ms 3.4191 ms]
                        thrpt:  [11.899 MiB/s 11.954 MiB/s 12.011 MiB/s]
                 change:
                        time:   [-0.6427% -0.2635% +0.0906%] (p = 0.17 > 0.05)
                        thrpt:  [-0.0905% +0.2642% +0.6469%]
                        No change in performance detected.
Found 20 outliers among 100 measurements (20.00%)
  1 (1.00%) low severe
  2 (2.00%) low mild
  4 (4.00%) high mild
  13 (13.00%) high severe

linter/all-rules/numpy/globals.py
                        time:   [148.99 µs 149.21 µs 149.42 µs]
                        thrpt:  [19.748 MiB/s 19.776 MiB/s 19.805 MiB/s]
                 change:
                        time:   [-0.7340% -0.5068% -0.2778%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2785% +0.5094% +0.7395%]
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high severe
linter/all-rules/pydantic/types.py
                        time:   [3.0362 ms 3.0396 ms 3.0441 ms]
                        thrpt:  [8.3779 MiB/s 8.3903 MiB/s 8.3997 MiB/s]
                 change:
                        time:   [-0.0957% +0.0618% +0.2125%] (p = 0.45 > 0.05)
                        thrpt:  [-0.2121% -0.0618% +0.0958%]
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low severe
  3 (3.00%) low mild
  5 (5.00%) high mild
  2 (2.00%) high severe
linter/all-rules/numpy/ctypeslib.py
                        time:   [1.6879 ms 1.6894 ms 1.6909 ms]
                        thrpt:  [9.8478 MiB/s 9.8562 MiB/s 9.8652 MiB/s]
                 change:
                        time:   [-0.2279% -0.0888% +0.0436%] (p = 0.18 > 0.05)
                        thrpt:  [-0.0435% +0.0889% +0.2284%]
                        No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
  4 (4.00%) low mild
  1 (1.00%) high severe
linter/all-rules/large/dataset.py
                        time:   [7.1520 ms 7.1586 ms 7.1654 ms]
                        thrpt:  [5.6777 MiB/s 5.6831 MiB/s 5.6883 MiB/s]
                 change:
                        time:   [-2.5626% -2.1654% -1.7780%] (p = 0.00 < 0.05)
                        thrpt:  [+1.8102% +2.2133% +2.6300%]
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
```
2023-06-15 18:43:19 +00:00
.cargo Add Formatter benchmark (#4860) 2023-06-05 21:05:42 +02:00
.devcontainer Add devcontainer support (#4676) (#4678) 2023-05-30 14:49:51 +02:00
.github Add GitHub CODEOWNERS file (#5054) 2023-06-14 05:21:59 +00:00
assets Add Astral badge to the repo (#4401) 2023-05-12 19:27:38 +00:00
crates Always use identifier ranges to store bindings (#5110) 2023-06-15 18:43:19 +00:00
docs Update tutorial doc typo (#5088) 2023-06-14 11:17:35 -04:00
fuzz Improve ruff_parse_simple to find UTF-8 violations (#5008) 2023-06-12 12:10:23 -04:00
playground Update links to point to Astral org (#4949) 2023-06-08 11:43:40 -04:00
python/ruff Portably find ruff binary path from Python (#2574) 2023-02-04 17:19:27 -05:00
scripts Complete pyupgrade documentation (#5096) 2023-06-14 23:43:12 +00:00
.editorconfig markdownlint: enforce 100 char max length (#4698) 2023-05-28 22:45:56 -04:00
.gitattributes chore: Use LF on all platforms (#3005) 2023-02-20 20:13:37 +00:00
.gitignore Implement basic module formatting (#4784) 2023-06-01 15:25:50 +02:00
.markdownlint.yaml markdownlint: enforce 100 char max length (#4698) 2023-05-28 22:45:56 -04:00
.pre-commit-config.yaml Consider ignore-names in all pep8 naming rules (#5079) 2023-06-14 16:57:09 +02:00
_typos.toml Consider ignore-names in all pep8 naming rules (#5079) 2023-06-14 16:57:09 +02:00
BREAKING_CHANGES.md Update links to point to Astral org (#4949) 2023-06-08 11:43:40 -04:00
Cargo.lock Support glob patterns in pep8_naming ignore-names (#5024) 2023-06-13 17:37:13 +02:00
Cargo.toml Use consistent Cargo.toml metadata in all crates (#5015) 2023-06-12 00:02:40 +00:00
clippy.toml [numpy] deprecated type aliases (#2810) 2023-02-14 23:45:12 +00:00
CODE_OF_CONDUCT.md Run automatically format code blocks with Black (#3191) 2023-02-27 10:14:05 -05:00
CONTRIBUTING.md Update CONTRIBUTING.md guide (#5017) 2023-06-12 00:20:59 +00:00
LICENSE [perflint] Add perflint plugin, add first rule PERF102 (#4821) 2023-06-13 01:54:44 +00:00
mkdocs.template.yml Update links to point to Astral org (#4949) 2023-06-08 11:43:40 -04:00
pyproject.toml Update links to point to Astral org (#4949) 2023-06-08 11:43:40 -04:00
README.md Implement copyright notice detection (#4701) 2023-06-11 02:17:58 +00:00
ruff.schema.json Add pyproject.toml to include option doc (#5080) 2023-06-14 15:55:12 +00:00
rust-toolchain Upgrade to Rust 1.70 (#4848) 2023-06-04 17:51:47 +00:00

Ruff

Ruff image image image Actions status

Discord | Docs | Playground

An extremely fast Python linter, written in Rust.

Shows a bar chart with benchmark results.

Linting the CPython codebase from scratch.

  • 10-100x faster than existing linters
  • 🐍 Installable via pip
  • 🛠️ pyproject.toml support
  • 🤝 Python 3.11 compatibility
  • 📦 Built-in caching, to avoid re-analyzing unchanged files
  • 🔧 Autofix support, for automatic error correction (e.g., automatically remove unused imports)
  • 📏 Over 500 built-in rules
  • ⚖️ Near-parity with the built-in Flake8 rule set
  • 🔌 Native re-implementations of dozens of Flake8 plugins, like flake8-bugbear
  • ⌨️ First-party editor integrations for VS Code and more
  • 🌎 Monorepo-friendly, with hierarchical and cascading configuration

Ruff aims to be orders of magnitude faster than alternative tools while integrating more functionality behind a single, common interface.

Ruff can be used to replace Flake8 (plus dozens of plugins), isort, pydocstyle, yesqa, eradicate, pyupgrade, and autoflake, all while executing tens or hundreds of times faster than any individual tool.

Ruff is extremely actively developed and used in major open-source projects like:

...and many more.

Ruff is backed by Astral. Read the launch post, or the original project announcement.

Testimonials

Sebastián Ramírez, creator of FastAPI:

Ruff is so fast that sometimes I add an intentional bug in the code just to confirm it's actually running and checking the code.

Nick Schrock, founder of Elementl, co-creator of GraphQL:

Why is Ruff a gamechanger? Primarily because it is nearly 1000x faster. Literally. Not a typo. On our largest module (dagster itself, 250k LOC) pylint takes about 2.5 minutes, parallelized across 4 cores on my M1. Running ruff against our entire codebase takes .4 seconds.

Bryan Van de Ven, co-creator of Bokeh, original author of Conda:

Ruff is ~150-200x faster than flake8 on my machine, scanning the whole repo takes ~0.2s instead of ~20s. This is an enormous quality of life improvement for local dev. It's fast enough that I added it as an actual commit hook, which is terrific.

Timothy Crosley, creator of isort:

Just switched my first project to Ruff. Only one downside so far: it's so fast I couldn't believe it was working till I intentionally introduced some errors.

Tim Abbott, lead developer of Zulip:

This is just ridiculously fast... ruff is amazing.

Table of Contents

For more, see the documentation.

  1. Getting Started
  2. Configuration
  3. Rules
  4. Contributing
  5. Support
  6. Acknowledgements
  7. Who's Using Ruff?
  8. License

Getting Started

For more, see the documentation.

Installation

Ruff is available as ruff on PyPI:

pip install ruff

You can also install Ruff via Homebrew, Conda, and with a variety of other package managers.

Usage

To run Ruff, try any of the following:

ruff check .                        # Lint all files in the current directory (and any subdirectories)
ruff check path/to/code/            # Lint all files in `/path/to/code` (and any subdirectories)
ruff check path/to/code/*.py        # Lint all `.py` files in `/path/to/code`
ruff check path/to/code/to/file.py  # Lint `file.py`

Ruff can also be used as a pre-commit hook:

- repo: https://github.com/astral-sh/ruff-pre-commit
  # Ruff version.
  rev: v0.0.272
  hooks:
    - id: ruff

Ruff can also be used as a VS Code extension or alongside any other editor through the Ruff LSP.

Ruff can also be used as a GitHub Action via ruff-action:

name: Ruff
on: [ push, pull_request ]
jobs:
  ruff:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: chartboost/ruff-action@v1

Configuration

Ruff can be configured through a pyproject.toml, ruff.toml, or .ruff.toml file (see: Configuration, or Settings for a complete list of all configuration options).

If left unspecified, the default configuration is equivalent to:

[tool.ruff]
# Enable pycodestyle (`E`) and Pyflakes (`F`) codes by default.
select = ["E", "F"]
ignore = []

# Allow autofix for all enabled rules (when `--fix`) is provided.
fixable = ["A", "B", "C", "D", "E", "F", "G", "I", "N", "Q", "S", "T", "W", "ANN", "ARG", "BLE", "COM", "DJ", "DTZ", "EM", "ERA", "EXE", "FBT", "ICN", "INP", "ISC", "NPY", "PD", "PGH", "PIE", "PL", "PT", "PTH", "PYI", "RET", "RSE", "RUF", "SIM", "SLF", "TCH", "TID", "TRY", "UP", "YTT"]
unfixable = []

# Exclude a variety of commonly ignored directories.
exclude = [
    ".bzr",
    ".direnv",
    ".eggs",
    ".git",
    ".git-rewrite",
    ".hg",
    ".mypy_cache",
    ".nox",
    ".pants.d",
    ".pytype",
    ".ruff_cache",
    ".svn",
    ".tox",
    ".venv",
    "__pypackages__",
    "_build",
    "buck-out",
    "build",
    "dist",
    "node_modules",
    "venv",
]

# Same as Black.
line-length = 88

# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

# Assume Python 3.10.
target-version = "py310"

[tool.ruff.mccabe]
# Unlike Flake8, default to a complexity level of 10.
max-complexity = 10

Some configuration options can be provided via the command-line, such as those related to rule enablement and disablement, file discovery, logging level, and more:

ruff check path/to/code/ --select F401 --select F403 --quiet

See ruff help for more on Ruff's top-level commands, or ruff help check for more on the linting command.

Rules

Ruff supports over 500 lint rules, many of which are inspired by popular tools like Flake8, isort, pyupgrade, and others. Regardless of the rule's origin, Ruff re-implements every rule in Rust as a first-party feature.

By default, Ruff enables Flake8's E and F rules. Ruff supports all rules from the F category, and a subset of the E category, omitting those stylistic rules made obsolete by the use of an autoformatter, like Black.

If you're just getting started with Ruff, the default rule set is a great place to start: it catches a wide variety of common errors (like unused imports) with zero configuration.

Beyond the defaults, Ruff re-implements some of the most popular Flake8 plugins and related code quality tools, including:

For a complete enumeration of the supported rules, see Rules.

Contributing

Contributions are welcome and highly appreciated. To get started, check out the contributing guidelines.

You can also join us on Discord.

Support

Having trouble? Check out the existing issues on GitHub, or feel free to open a new one.

You can also ask for help on Discord.

Acknowledgements

Ruff's linter draws on both the APIs and implementation details of many other tools in the Python ecosystem, especially Flake8, Pyflakes, pycodestyle, pydocstyle, pyupgrade, and isort.

In some cases, Ruff includes a "direct" Rust port of the corresponding tool. We're grateful to the maintainers of these tools for their work, and for all the value they've provided to the Python community.

Ruff's autoformatter is built on a fork of Rome's rome_formatter, and again draws on both the APIs and implementation details of Rome, Prettier, and Black.

Ruff is also influenced by a number of tools outside the Python ecosystem, like Clippy and ESLint.

Ruff is the beneficiary of a large number of contributors.

Ruff is released under the MIT license.

Who's Using Ruff?

Ruff is used by a number of major open-source projects and companies, including:

Show Your Support

If you're using Ruff, consider adding the Ruff badge to project's README.md:

[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

...or README.rst:

.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json
    :target: https://github.com/astral-sh/ruff
    :alt: Ruff

...or, as HTML:

<a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v2.json" alt="Ruff" style="max-width:100%;"></a>

License

MIT