ruff/crates/red_knot_python_semantic
Dhruv Manilawala 7027344dfc
Add scope and definitions for comprehensions (#12748)
## Summary

This PR adds scope and definition for comprehension nodes. This includes
the following nodes:
* List comprehension
* Dictionary comprehension
* Set comprehension 
* Generator expression

### Scope

Each expression here adds it's own scope with one caveat - the `iter`
expression of the first generator is part of the parent scope. For
example, in the following code snippet the `iter1` variable is evaluated
in the outer scope.

```py
[x for x in iter1]
```

> The iterable expression in the leftmost for clause is evaluated
directly in the enclosing scope and then passed as an argument to the
implicitly nested scope.
>
> Reference:
https://docs.python.org/3/reference/expressions.html#displays-for-lists-sets-and-dictionaries

There's another special case for assignment expressions:

> There is one special case: an assignment expression occurring in a
list, set or dict comprehension or in a generator expression (below
collectively referred to as “comprehensions”) binds the target in the
containing scope, honoring a nonlocal or global declaration for the
target in that scope, if one exists.
>
> Reference: https://peps.python.org/pep-0572/#scope-of-the-target

For example, in the following code snippet, the variables `a` and `b`
are available after the comprehension while `x` isn't:
```py
[a := 1 for x in range(2) if (b := 2)]
```

### Definition

Each comprehension node adds a single definition, the "target" variable
(`[_ for target in iter]`). This has been accounted for and a new
variant has been added to `DefinitionKind`.

### Type Inference

Currently, type inference is limited to a single scope. It doesn't
_enter_ in another scope to infer the types of the remaining expressions
of a node. To accommodate this, the type inference for a **scope**
requires new methods which _doesn't_ infer the type of the `iter`
expression of the leftmost outer generator (that's defined in the
enclosing scope).

The type inference for the scope region is split into two parts:
* `infer_generator_expression` (similarly for comprehensions) infers the
type of the `iter` expression of the leftmost outer generator
* `infer_generator_expression_scope` (similarly for comprehension)
infers the type of the remaining expressions except for the one
mentioned in the previous point

The type inference for the **definition** also needs to account for this
special case of leftmost generator. This is done by defining a `first`
boolean parameter which indicates whether this comprehension definition
occurs first in the enclosing expression.

## Test Plan

New test cases were added to validate multiple scenarios. Refer to the
documentation for each test case which explains what is being tested.
2024-08-13 07:00:33 +05:30
..
src Add scope and definitions for comprehensions (#12748) 2024-08-13 07:00:33 +05:30
vendor/typeshed [red-knot] Merge the semantic and module-resolver crates (#12751) 2024-08-08 15:34:11 +01:00
build.rs [red-knot] Merge the semantic and module-resolver crates (#12751) 2024-08-08 15:34:11 +01:00
Cargo.toml [red-knot] Add filename and source location for diagnostics (#12842) 2024-08-12 15:56:30 +00:00
README.md [red-knot] Merge the semantic and module-resolver crates (#12751) 2024-08-08 15:34:11 +01:00

Red Knot

Semantic analysis for the red-knot project.

Vendored types for the stdlib

This crate vendors typeshed's stubs for the standard library. The vendored stubs can be found in crates/red_knot_python_semantic/vendor/typeshed. The file crates/red_knot_python_semantic/vendor/typeshed/source_commit.txt tells you the typeshed commit that our vendored stdlib stubs currently correspond to.

The typeshed stubs are updated every two weeks via an automated PR using the sync_typeshed.yaml workflow in the .github/workflows directory. This workflow can also be triggered at any time via workflow dispatch.