ruff/scripts/ty_benchmark
Ibraheem Ahmed aa1fad61e0
Some checks failed
CI / cargo fmt (push) Has been cancelled
[ty Playground] Release / publish (push) Has been cancelled
CI / Determine changes (push) Has been cancelled
CI / cargo build (release) (push) Has been cancelled
CI / python package (push) Has been cancelled
CI / pre-commit (push) Has been cancelled
CI / mkdocs (push) Has been cancelled
CI / cargo clippy (push) Has been cancelled
CI / cargo test (linux) (push) Has been cancelled
CI / cargo test (linux, release) (push) Has been cancelled
CI / cargo test (windows) (push) Has been cancelled
CI / cargo test (wasm) (push) Has been cancelled
CI / cargo build (msrv) (push) Has been cancelled
CI / cargo fuzz build (push) Has been cancelled
CI / fuzz parser (push) Has been cancelled
CI / test scripts (push) Has been cancelled
CI / ecosystem (push) Has been cancelled
CI / Fuzz for new ty panics (push) Has been cancelled
CI / cargo shear (push) Has been cancelled
CI / formatter instabilities and black similarity (push) Has been cancelled
CI / test ruff-lsp (push) Has been cancelled
CI / check playground (push) Has been cancelled
CI / benchmarks (push) Has been cancelled
Support relative --ty-path in ty-benchmark (#18385)
## Summary

This currently doesn't work because the benchmark changes the working
directory. Also updates the process name to make it easier to compare
two local ty binaries.
2025-05-30 18:19:20 -04:00
..
src/benchmark Support relative --ty-path in ty-benchmark (#18385) 2025-05-30 18:19:20 -04:00
pyproject.toml Rename Red Knot (#17820) 2025-05-03 19:49:15 +02:00
README.md Rename Red Knot (#17820) 2025-05-03 19:49:15 +02:00
uv.lock Rename Red Knot (#17820) 2025-05-03 19:49:15 +02:00

Getting started

  1. Install uv
  • Unix: curl -LsSf https://astral.sh/uv/install.sh | sh
  • Windows: powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
  1. Build ty: cargo build --bin ty --release
  2. cd into the benchmark directory: cd scripts/ty_benchmark
  3. Run benchmarks: uv run benchmark

Known limitations

ty only implements a tiny fraction of Mypy's and Pyright's functionality, so the benchmarks aren't in any way a fair comparison today. However, they'll become more meaningful as we build out more type checking features in ty.

Windows support

The script should work on Windows, but we haven't tested it yet. We do make use of shlex which has known limitations when using non-POSIX shells.