ruff/crates/red_knot_python_semantic/src
Dhruv Manilawala 7027344dfc
Add scope and definitions for comprehensions (#12748)
## Summary

This PR adds scope and definition for comprehension nodes. This includes
the following nodes:
* List comprehension
* Dictionary comprehension
* Set comprehension 
* Generator expression

### Scope

Each expression here adds it's own scope with one caveat - the `iter`
expression of the first generator is part of the parent scope. For
example, in the following code snippet the `iter1` variable is evaluated
in the outer scope.

```py
[x for x in iter1]
```

> The iterable expression in the leftmost for clause is evaluated
directly in the enclosing scope and then passed as an argument to the
implicitly nested scope.
>
> Reference:
https://docs.python.org/3/reference/expressions.html#displays-for-lists-sets-and-dictionaries

There's another special case for assignment expressions:

> There is one special case: an assignment expression occurring in a
list, set or dict comprehension or in a generator expression (below
collectively referred to as “comprehensions”) binds the target in the
containing scope, honoring a nonlocal or global declaration for the
target in that scope, if one exists.
>
> Reference: https://peps.python.org/pep-0572/#scope-of-the-target

For example, in the following code snippet, the variables `a` and `b`
are available after the comprehension while `x` isn't:
```py
[a := 1 for x in range(2) if (b := 2)]
```

### Definition

Each comprehension node adds a single definition, the "target" variable
(`[_ for target in iter]`). This has been accounted for and a new
variant has been added to `DefinitionKind`.

### Type Inference

Currently, type inference is limited to a single scope. It doesn't
_enter_ in another scope to infer the types of the remaining expressions
of a node. To accommodate this, the type inference for a **scope**
requires new methods which _doesn't_ infer the type of the `iter`
expression of the leftmost outer generator (that's defined in the
enclosing scope).

The type inference for the scope region is split into two parts:
* `infer_generator_expression` (similarly for comprehensions) infers the
type of the `iter` expression of the leftmost outer generator
* `infer_generator_expression_scope` (similarly for comprehension)
infers the type of the remaining expressions except for the one
mentioned in the previous point

The type inference for the **definition** also needs to account for this
special case of leftmost generator. This is done by defining a `first`
boolean parameter which indicates whether this comprehension definition
occurs first in the enclosing expression.

## Test Plan

New test cases were added to validate multiple scenarios. Refer to the
documentation for each test case which explains what is being tested.
2024-08-13 07:00:33 +05:30
..
module_resolver Eagerly validate search paths (#12783) 2024-08-12 07:46:59 +00:00
semantic_index Add scope and definitions for comprehensions (#12748) 2024-08-13 07:00:33 +05:30
types Add scope and definitions for comprehensions (#12748) 2024-08-13 07:00:33 +05:30
ast_node_ref.rs [red-knot] per-definition inference, use-def maps (#12269) 2024-07-16 11:02:30 -07:00
builtins.rs [red-knot] Merge the semantic and module-resolver crates (#12751) 2024-08-08 15:34:11 +01:00
db.rs [red-knot] Merge the semantic and module-resolver crates (#12751) 2024-08-08 15:34:11 +01:00
lib.rs Remove red_knot_python_semantic::python_version::TargetVersion (#12790) 2024-08-10 14:28:31 +01:00
module_name.rs [red-knot] Merge the semantic and module-resolver crates (#12751) 2024-08-08 15:34:11 +01:00
node_key.rs [red-knot] Extract red_knot_python_semantic crate (#11926) 2024-06-20 13:24:24 +02:00
program.rs Eagerly validate search paths (#12783) 2024-08-12 07:46:59 +00:00
python_version.rs Remove red_knot_python_semantic::python_version::TargetVersion (#12790) 2024-08-10 14:28:31 +01:00
semantic_index.rs Add scope and definitions for comprehensions (#12748) 2024-08-13 07:00:33 +05:30
semantic_model.rs [red-knot] Add filename and source location for diagnostics (#12842) 2024-08-12 15:56:30 +00:00
types.rs [red-knot] add IntersectionBuilder (#12791) 2024-08-12 11:56:04 -07:00