mirror of
https://github.com/astral-sh/ruff.git
synced 2025-08-01 17:32:25 +00:00
93eff7f174
## Summary Adds a markdown-based test framework for writing tests of type inference and type checking. Fixes #11664. Implements the basic required features. A markdown test file is a suite of tests, each test can contain one or more Python files, with optionally specified path/name. The test writes all files to an in-memory file system, runs red-knot, and matches the resulting diagnostics against `Type: ` and `Error: ` assertions embedded in the Python source as comments. We will want to add features like incremental tests, setting custom configuration for tests, writing non-Python files, testing syntax errors, capturing full diagnostic output, etc. There's also plenty of room for improved UX (colored output?). ## Test Plan Lots of tests! Sample of the current output when a test fails: ``` Running tests/inference.rs (target/debug/deps/inference-7c96590aa84de2a4) running 1 test test inference::path_1_resources_inference_numbers_md ... FAILED failures: ---- inference::path_1_resources_inference_numbers_md stdout ---- inference/numbers.md - Numbers - Floats /src/test.py line 2: unexpected error: [invalid-assignment] "Object of type `Literal["str"]` is not assignable to `int`" thread 'inference::path_1_resources_inference_numbers_md' panicked at crates/red_knot_test/src/lib.rs:60:5: Some tests failed. note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace failures: inference::path_1_resources_inference_numbers_md test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.19s error: test failed, to rerun pass `-p red_knot_test --test inference` ``` --------- Co-authored-by: Micha Reiser <micha@reiser.io> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> |
||
|---|---|---|
| .. | ||
| src | ||
| tests | ||
| Cargo.toml | ||