[red-knot] Detect semantic syntax errors (#17463)

Summary
--

This PR extends semantic syntax error detection to red-knot. The main
changes here are:

1. Adding `SemanticSyntaxChecker` and `Vec<SemanticSyntaxError>` fields
to the `SemanticIndexBuilder`
2. Calling `SemanticSyntaxChecker::visit_stmt` and `visit_expr` in the
`SemanticIndexBuilder`'s `visit_stmt` and `visit_expr` methods
3. Implementing `SemanticSyntaxContext` for `SemanticIndexBuilder`
4. Adding new mdtests to test the context implementation and show
diagnostics

(3) is definitely the trickiest and required (I think) a minor addition
to the `SemanticIndexBuilder`. I tried to look around for existing code
performing the necessary checks, but I definitely could have missed
something or misused the existing code even when I found it.

There's still one TODO around `global` statement handling. I don't think
there's an existing way to look this up, but I'm happy to work on that
here or in a separate PR. This currently only affects detection of one
error (`LoadBeforeGlobalDeclaration` or
[PLE0118](https://docs.astral.sh/ruff/rules/load-before-global-declaration/)
in ruff), so it's not too big of a problem even if we leave the TODO.

Test Plan
--

New mdtests, as well as new errors for existing mdtests

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
This commit is contained in:
Brent Westbrook 2025-04-23 09:52:58 -04:00 committed by GitHub
parent 624f5c6c22
commit e7f38fe74b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 431 additions and 66 deletions

View file

@ -32,6 +32,10 @@ pub struct SemanticSyntaxChecker {
/// Python considers it a syntax error to import from `__future__` after any other
/// non-`__future__`-importing statements.
seen_futures_boundary: bool,
/// The checker has traversed past the module docstring boundary (i.e. seen any statement in the
/// module).
seen_module_docstring_boundary: bool,
}
impl SemanticSyntaxChecker {
@ -506,7 +510,7 @@ impl SemanticSyntaxChecker {
// update internal state
match stmt {
Stmt::Expr(StmtExpr { value, .. })
if !ctx.seen_docstring_boundary() && value.is_string_literal_expr() => {}
if !self.seen_module_docstring_boundary && value.is_string_literal_expr() => {}
Stmt::ImportFrom(StmtImportFrom { module, .. }) => {
// Allow __future__ imports until we see a non-__future__ import.
if !matches!(module.as_deref(), Some("__future__")) {
@ -520,6 +524,8 @@ impl SemanticSyntaxChecker {
self.seen_futures_boundary = true;
}
}
self.seen_module_docstring_boundary = true;
}
/// Check `expr` for semantic syntax errors and update the checker's internal state.
@ -881,7 +887,7 @@ impl Display for SemanticSyntaxError {
f.write_str("`return` statement outside of a function")
}
SemanticSyntaxErrorKind::AwaitOutsideAsyncFunction(kind) => {
write!(f, "`{kind}` outside of an asynchronous function")
write!(f, "{kind} outside of an asynchronous function")
}
}
}
@ -1207,9 +1213,9 @@ pub enum AwaitOutsideAsyncFunctionKind {
impl Display for AwaitOutsideAsyncFunctionKind {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(match self {
AwaitOutsideAsyncFunctionKind::Await => "await",
AwaitOutsideAsyncFunctionKind::AsyncFor => "async for",
AwaitOutsideAsyncFunctionKind::AsyncWith => "async with",
AwaitOutsideAsyncFunctionKind::Await => "`await`",
AwaitOutsideAsyncFunctionKind::AsyncFor => "`async for`",
AwaitOutsideAsyncFunctionKind::AsyncWith => "`async with`",
AwaitOutsideAsyncFunctionKind::AsyncComprehension => "asynchronous comprehension",
})
}
@ -1584,9 +1590,6 @@ where
/// x # here, classes break function scopes
/// ```
pub trait SemanticSyntaxContext {
/// Returns `true` if a module's docstring boundary has been passed.
fn seen_docstring_boundary(&self) -> bool;
/// Returns `true` if `__future__`-style type annotations are enabled.
fn future_annotations_or_stub(&self) -> bool;