[ty] Add a DateType benchmark (#19148)
Some checks are pending
CI / cargo fuzz build (push) Blocked by required conditions
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / Fuzz for new ty panics (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / check playground (push) Blocked by required conditions
CI / benchmarks-instrumented (push) Blocked by required conditions
CI / benchmarks-walltime (push) Blocked by required conditions

## Summary

The [`DateType`](https://github.com/glyph/DateType) library has some
very large protocols in it. Currently we type-check it quite quickly,
but the current version of https://github.com/astral-sh/ruff/pull/18659
makes our execution time on this library pathologically slow. That PR
doesn't seem to have a big impact on any of our current benchmarks,
however, so it seems we have some missing coverage in this area; I
therefore propose that we add `DateType` as a benchmark.

Currently the benchmark runs pretty quickly (about half the runtime of
attrs, which is our fastest real-world benchmark currently), and the
library has 0 third-party dependencies, so the benchmark is quick to
setup.

## Test Plan

`cargo bench -p ruff_benchmark --bench=ty`
This commit is contained in:
Alex Waygood 2025-07-04 21:11:47 +01:00 committed by GitHub
parent 1c710c2840
commit 44f2f77748
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -498,11 +498,8 @@ fn bench_project(benchmark: &ProjectBenchmark, criterion: &mut Criterion) {
let diagnostics = result.len();
assert!(
diagnostics > 1 && diagnostics <= max_diagnostics,
"Expected between {} and {} diagnostics but got {}",
1,
max_diagnostics,
diagnostics
diagnostics <= max_diagnostics,
"Expected <={max_diagnostics} diagnostics but got {diagnostics}"
);
}
@ -570,6 +567,23 @@ fn anyio(criterion: &mut Criterion) {
bench_project(&benchmark, criterion);
}
fn datetype(criterion: &mut Criterion) {
let benchmark = ProjectBenchmark::new(
RealWorldProject {
name: "DateType",
repository: "https://github.com/glyph/DateType",
commit: "57c9c93cf2468069f72945fc04bf27b64100dad8",
paths: vec![SystemPath::new("src")],
dependencies: vec![],
max_dep_date: "2025-07-04",
python_version: PythonVersion::PY313,
},
0,
);
bench_project(&benchmark, criterion);
}
criterion_group!(check_file, benchmark_cold, benchmark_incremental);
criterion_group!(
micro,
@ -578,5 +592,5 @@ criterion_group!(
benchmark_complex_constrained_attributes_1,
benchmark_complex_constrained_attributes_2,
);
criterion_group!(project, anyio, attrs, hydra);
criterion_group!(project, anyio, attrs, hydra, datetype);
criterion_main!(check_file, micro, project);