language-servers/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-09-30 05:45:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

[red-knot] type inference/checking test framework (#13636)

Browse source 
## Summary

Adds a markdown-based test framework for writing tests of type inference
and type checking. Fixes #11664.

Implements the basic required features. A markdown test file is a suite
of tests, each test can contain one or more Python files, with
optionally specified path/name. The test writes all files to an
in-memory file system, runs red-knot, and matches the resulting
diagnostics against `Type: ` and `Error: ` assertions embedded in the
Python source as comments.

We will want to add features like incremental tests, setting custom
configuration for tests, writing non-Python files, testing syntax
errors, capturing full diagnostic output, etc. There's also plenty of
room for improved UX (colored output?).

## Test Plan

Lots of tests!

Sample of the current output when a test fails:

```
     Running tests/inference.rs (target/debug/deps/inference-7c96590aa84de2a4)

running 1 test
test inference::path_1_resources_inference_numbers_md ... FAILED

failures:

---- inference::path_1_resources_inference_numbers_md stdout ----
inference/numbers.md - Numbers - Floats
  /src/test.py
    line 2: unexpected error: [invalid-assignment] "Object of type `Literal["str"]` is not assignable to `int`"

thread 'inference::path_1_resources_inference_numbers_md' panicked at crates/red_knot_test/src/lib.rs:60:5:
Some tests failed.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    inference::path_1_resources_inference_numbers_md

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.19s

error: test failed, to rerun pass `-p red_knot_test --test inference`
```

---------

Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
This commit is contained in:
Carl Meyer 2024-10-08 12:33:19 -07:00 • committed by GitHub
parent fc661e193a
commit 93eff7f174
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 2504 additions and 25 deletions
Download patch file Download diff file Expand all files Collapse all files

2
crates/red_knot_python_semantic/src/types.rs
View file

@ -17,7 +17,7 @@ use crate::types::narrow::narrowing_constraint;
use crate::{Db, FxOrderSet, Module};
pub(crate) use self::builder::{IntersectionBuilder, UnionBuilder};
pub(crate) use self::diagnostic::TypeCheckDiagnostics;
pub use self::diagnostic::{TypeCheckDiagnostic, TypeCheckDiagnostics};
pub(crate) use self::display::TypeArrayDisplay;
pub(crate) use self::infer::{
infer_deferred_types, infer_definition_types, infer_expression_types, infer_scope_types,

22
crates/red_knot_python_semantic/src/types/infer.rs
View file

@ -3683,28 +3683,6 @@ mod tests {
Ok(())
}
#[test]
fn number_literal() -> anyhow::Result<()> {
let mut db = setup_db();
db.write_dedented(
"src/a.py",
"
a = 1
b = 9223372036854775808
c = 1.45
d = 2j
",
)?;
assert_public_ty(&db, "src/a.py", "a", "Literal[1]");
assert_public_ty(&db, "src/a.py", "b", "int");
assert_public_ty(&db, "src/a.py", "c", "float");
assert_public_ty(&db, "src/a.py", "d", "complex");
Ok(())
}
#[test]
fn negated_int_literal() -> anyhow::Result<()> {
let mut db = setup_db();