Rename Red Knot (#17820)

crates/ty_python_semantic/resources/mdtest/scopes/builtin.md

@@ -0,0 +1,32 @@
+# Builtin scope
+
+## Conditionally global or builtin
+
+If a builtin name is conditionally defined as a global, a name lookup should union the builtin type
+with the conditionally-defined type:
+
+```py
+def returns_bool() -> bool:
+    return True
+
+if returns_bool():
+    chr: int = 1
+
+def f():
+    reveal_type(chr)  # revealed: int | (def chr(i: SupportsIndex, /) -> str)
+```
+
+## Conditionally global or builtin, with annotation
+
+Same is true if the name is annotated:
+
+```py
+def returns_bool() -> bool:
+    return True
+
+if returns_bool():
+    chr: int = 1
+
+def f():
+    reveal_type(chr)  # revealed: int | (def chr(i: SupportsIndex, /) -> str)
+```

crates/ty_python_semantic/resources/mdtest/scopes/eager.md

@@ -0,0 +1,412 @@
+# Eager scopes
+
+Some scopes are executed eagerly: references to variables defined in enclosing scopes are resolved
+_immediately_. This is in contrast to (for instance) function scopes, where those references are
+resolved when the function is called.
+
+## Function definitions
+
+Function definitions are evaluated lazily.
+
+```py
+x = 1
+
+def f():
+    reveal_type(x)  # revealed: Unknown | Literal[2]
+
+x = 2
+```
+
+## Class definitions
+
+Class definitions are evaluated eagerly.
+
+```py
+def _():
+    x = 1
+
+    class A:
+        reveal_type(x)  # revealed: Literal[1]
+
+        y = x
+
+    x = 2
+
+    reveal_type(A.y)  # revealed: Unknown | Literal[1]
+```
+
+## List comprehensions
+
+List comprehensions are evaluated eagerly.
+
+```py
+def _():
+    x = 1
+
+    # revealed: Literal[1]
+    [reveal_type(x) for a in range(1)]
+
+    x = 2
+```
+
+## Set comprehensions
+
+Set comprehensions are evaluated eagerly.
+
+```py
+def _():
+    x = 1
+
+    # revealed: Literal[1]
+    {reveal_type(x) for a in range(1)}
+
+    x = 2
+```
+
+## Dict comprehensions
+
+Dict comprehensions are evaluated eagerly.
+
+```py
+def _():
+    x = 1
+
+    # revealed: Literal[1]
+    {a: reveal_type(x) for a in range(1)}
+
+    x = 2
+```
+
+## Generator expressions
+
+Generator expressions don't necessarily run eagerly, but in practice usually they do, so assuming
+they do is the better default.
+
+```py
+def _():
+    x = 1
+
+    # revealed: Literal[1]
+    list(reveal_type(x) for a in range(1))
+
+    x = 2
+```
+
+But that does lead to incorrect results when the generator expression isn't run immediately:
+
+```py
+def evaluated_later():
+    x = 1
+
+    # revealed: Literal[1]
+    y = (reveal_type(x) for a in range(1))
+
+    x = 2
+
+    # The generator isn't evaluated until here, so at runtime, `x` will evaluate to 2, contradicting
+    # our inferred type.
+    print(next(y))
+```
+
+Though note that “the iterable expression in the leftmost `for` clause is immediately evaluated”
+\[[spec][generators]\]:
+
+```py
+def iterable_evaluated_eagerly():
+    x = 1
+
+    # revealed: Literal[1]
+    y = (a for a in [reveal_type(x)])
+
+    x = 2
+
+    # Even though the generator isn't evaluated until here, the first iterable was evaluated
+    # immediately, so our inferred type is correct.
+    print(next(y))
+```
+
+## Top-level eager scopes
+
+All of the above examples behave identically when the eager scopes are directly nested in the global
+scope.
+
+### Class definitions
+
+```py
+x = 1
+
+class A:
+    reveal_type(x)  # revealed: Literal[1]
+
+    y = x
+
+x = 2
+
+reveal_type(A.y)  # revealed: Unknown | Literal[1]
+```
+
+### List comprehensions
+
+```py
+x = 1
+
+# revealed: Literal[1]
+[reveal_type(x) for a in range(1)]
+
+x = 2
+
+# error: [unresolved-reference]
+[y for a in range(1)]
+y = 1
+```
+
+### Set comprehensions
+
+```py
+x = 1
+
+# revealed: Literal[1]
+{reveal_type(x) for a in range(1)}
+
+x = 2
+
+# error: [unresolved-reference]
+{y for a in range(1)}
+y = 1
+```
+
+### Dict comprehensions
+
+```py
+x = 1
+
+# revealed: Literal[1]
+{a: reveal_type(x) for a in range(1)}
+
+x = 2
+
+# error: [unresolved-reference]
+{a: y for a in range(1)}
+y = 1
+```
+
+### Generator expressions
+
+```py
+x = 1
+
+# revealed: Literal[1]
+list(reveal_type(x) for a in range(1))
+
+x = 2
+
+# error: [unresolved-reference]
+list(y for a in range(1))
+y = 1
+```
+
+`evaluated_later.py`:
+
+```py
+x = 1
+
+# revealed: Literal[1]
+y = (reveal_type(x) for a in range(1))
+
+x = 2
+
+# The generator isn't evaluated until here, so at runtime, `x` will evaluate to 2, contradicting
+# our inferred type.
+print(next(y))
+```
+
+`iterable_evaluated_eagerly.py`:
+
+```py
+x = 1
+
+# revealed: Literal[1]
+y = (a for a in [reveal_type(x)])
+
+x = 2
+
+# Even though the generator isn't evaluated until here, the first iterable was evaluated
+# immediately, so our inferred type is correct.
+print(next(y))
+```
+
+## Lazy scopes are "sticky"
+
+As we look through each enclosing scope when resolving a reference, lookups become lazy as soon as
+we encounter any lazy scope, even if there are other eager scopes that enclose it.
+
+### Eager scope within eager scope
+
+If we don't encounter a lazy scope, lookup remains eager. The resolved binding is not necessarily in
+the immediately enclosing scope. Here, the list comprehension and class definition are both eager
+scopes, and we immediately resolve the use of `x` to (only) the `x = 1` binding.
+
+```py
+def _():
+    x = 1
+
+    class A:
+        # revealed: Literal[1]
+        [reveal_type(x) for a in range(1)]
+
+    x = 2
+```
+
+### Class definition bindings are not visible in nested scopes
+
+Class definitions are eager scopes, but any bindings in them are explicitly not visible to any
+nested scopes. (Those nested scopes are typically (lazy) function definitions, but the rule also
+applies to nested eager scopes like comprehensions and other class definitions.)
+
+```py
+def _():
+    x = 1
+
+    class A:
+        x = 4
+
+        # revealed: Literal[1]
+        [reveal_type(x) for a in range(1)]
+
+        class B:
+            # revealed: Literal[1]
+            [reveal_type(x) for a in range(1)]
+
+    x = 2
+
+x = 1
+
+def _():
+    class C:
+        # revealed: Unknown | Literal[1]
+        [reveal_type(x) for _ in [1]]
+        x = 2
+```
+
+### Eager scope within a lazy scope
+
+The list comprehension is an eager scope, and it is enclosed within a function definition, which is
+a lazy scope. Because we pass through this lazy scope before encountering any bindings or
+definitions, the lookup is lazy.
+
+```py
+def _():
+    x = 1
+
+    def f():
+        # revealed: Unknown | Literal[2]
+        [reveal_type(x) for a in range(1)]
+    x = 2
+```
+
+### Lazy scope within an eager scope
+
+The function definition is a lazy scope, and it is enclosed within a class definition, which is an
+eager scope. Even though we pass through an eager scope before encountering any bindings or
+definitions, the lookup remains lazy.
+
+```py
+def _():
+    x = 1
+
+    class A:
+        def f():
+            # revealed: Unknown | Literal[2]
+            reveal_type(x)
+
+    x = 2
+```
+
+### Lazy scope within a lazy scope
+
+No matter how many lazy scopes we pass through before encountering a binding or definition, the
+lookup remains lazy.
+
+```py
+def _():
+    x = 1
+
+    def f():
+        def g():
+            # revealed: Unknown | Literal[2]
+            reveal_type(x)
+    x = 2
+```
+
+### Eager scope within a lazy scope within another eager scope
+
+We have a list comprehension (eager scope), enclosed within a function definition (lazy scope),
+enclosed within a class definition (eager scope), all of which we must pass through before
+encountering any binding of `x`. Even though the last scope we pass through is eager, the lookup is
+lazy, since we encountered a lazy scope on the way.
+
+```py
+def _():
+    x = 1
+
+    class A:
+        def f():
+            # revealed: Unknown | Literal[2]
+            [reveal_type(x) for a in range(1)]
+
+    x = 2
+```
+
+## Annotations
+
+Type annotations are sometimes deferred. When they are, the types that are referenced in an
+annotation are looked up lazily, even if they occur in an eager scope.
+
+### Eager annotations in a Python file
+
+```py
+from typing import ClassVar
+
+x = int
+
+class C:
+    var: ClassVar[x]
+
+reveal_type(C.var)  # revealed: int
+
+x = str
+```
+
+### Deferred annotations in a Python file
+
+```py
+from __future__ import annotations
+
+from typing import ClassVar
+
+x = int
+
+class C:
+    var: ClassVar[x]
+
+reveal_type(C.var)  # revealed: Unknown | str
+
+x = str
+```
+
+### Deferred annotations in a stub file
+
+```pyi
+from typing import ClassVar
+
+x = int
+
+class C:
+    var: ClassVar[x]
+
+reveal_type(C.var)  # revealed: str
+
+x = str
+```
+
+[generators]: https://docs.python.org/3/reference/expressions.html#generator-expressions

crates/ty_python_semantic/resources/mdtest/scopes/global.md

@@ -0,0 +1,177 @@
+# `global` references
+
+## Implicit global in function
+
+A name reference to a never-defined symbol in a function is implicitly a global lookup.
+
+```py
+x = 1
+
+def f():
+    reveal_type(x)  # revealed: Unknown | Literal[1]
+```
+
+## Explicit global in function
+
+```py
+x = 1
+
+def f():
+    global x
+    reveal_type(x)  # revealed: Unknown | Literal[1]
+```
+
+## Unassignable type in function
+
+```py
+x: int = 1
+
+def f():
+    y: int = 1
+    # error: [invalid-assignment] "Object of type `Literal[""]` is not assignable to `int`"
+    y = ""
+
+    global x
+    # TODO: error: [invalid-assignment] "Object of type `Literal[""]` is not assignable to `int`"
+    x = ""
+```
+
+## Nested intervening scope
+
+A `global` statement causes lookup to skip any bindings in intervening scopes:
+
+```py
+x: int = 1
+
+def outer():
+    x: str = ""
+
+    def inner():
+        global x
+        # TODO: revealed: int
+        reveal_type(x)  # revealed: str
+```
+
+## Narrowing
+
+An assignment following a `global` statement should narrow the type in the local scope after the
+assignment.
+
+```py
+x: int | None
+
+def f():
+    global x
+    x = 1
+    reveal_type(x)  # revealed: Literal[1]
+```
+
+## `nonlocal` and `global`
+
+A binding cannot be both `nonlocal` and `global`. This should emit a semantic syntax error. CPython
+marks the `nonlocal` line, while `mypy`, `pyright`, and `ruff` (`PLE0115`) mark the `global` line.
+
+```py
+x = 1
+
+def f():
+    x = 1
+    def g() -> None:
+        nonlocal x
+        global x  # TODO: error: [invalid-syntax] "name 'x' is nonlocal and global"
+        x = None
+```
+
+## Global declaration after `global` statement
+
+```py
+def f():
+    global x
+    # TODO this should also not be an error
+    y = x  # error: [unresolved-reference] "Name `x` used when not defined"
+    x = 1  # No error.
+
+x = 2
+```
+
+## Semantic syntax errors
+
+Using a name prior to its `global` declaration in the same scope is a syntax error.
+
+```py
+x = 1
+
+def f():
+    print(x)  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x
+    print(x)
+
+def f():
+    global x
+    print(x)  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x
+    print(x)
+
+def f():
+    print(x)  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x, y
+    print(x)
+
+def f():
+    global x, y
+    print(x)  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x, y
+    print(x)
+
+def f():
+    x = 1  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x
+    x = 1
+
+def f():
+    global x
+    x = 1  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x
+    x = 1
+
+def f():
+    del x  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x, y
+    del x
+
+def f():
+    global x, y
+    del x  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x, y
+    del x
+
+def f():
+    del x  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x
+    del x
+
+def f():
+    global x
+    del x  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x
+    del x
+
+def f():
+    del x  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x, y
+    del x
+
+def f():
+    global x, y
+    del x  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x, y
+    del x
+
+def f():
+    print(f"{x=}")  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+    global x
+
+# still an error in module scope
+x = None  # TODO: error: [invalid-syntax] name `x` is used prior to global declaration
+global x
+```

crates/ty_python_semantic/resources/mdtest/scopes/moduletype_attrs.md

@@ -0,0 +1,171 @@
+# Implicit globals from `types.ModuleType`
+
+## Implicit `ModuleType` globals
+
+All modules are instances of `types.ModuleType`. If a name can't be found in any local or global
+scope, we look it up as an attribute on `types.ModuleType` in typeshed before deciding that the name
+is unbound.
+
+```py
+reveal_type(__name__)  # revealed: str
+reveal_type(__file__)  # revealed: str | None
+reveal_type(__loader__)  # revealed: LoaderProtocol | None
+reveal_type(__package__)  # revealed: str | None
+reveal_type(__doc__)  # revealed: str | None
+reveal_type(__spec__)  # revealed: ModuleSpec | None
+
+reveal_type(__path__)  # revealed: @Todo(specialized non-generic class)
+
+class X:
+    reveal_type(__name__)  # revealed: str
+
+def foo():
+    reveal_type(__name__)  # revealed: str
+```
+
+However, three attributes on `types.ModuleType` are not present as implicit module globals; these
+are excluded:
+
+```py
+# error: [unresolved-reference]
+# revealed: Unknown
+reveal_type(__getattr__)
+
+# error: [unresolved-reference]
+# revealed: Unknown
+reveal_type(__dict__)
+
+# error: [unresolved-reference]
+# revealed: Unknown
+reveal_type(__init__)
+```
+
+## Accessed as attributes
+
+`ModuleType` attributes can also be accessed as attributes on module-literal types. The special
+attributes `__dict__` and `__init__`, and all attributes on `builtins.object`, can also be accessed
+as attributes on module-literal types, despite the fact that these are inaccessible as globals from
+inside the module:
+
+```py
+import typing
+
+reveal_type(typing.__name__)  # revealed: str
+reveal_type(typing.__init__)  # revealed: bound method ModuleType.__init__(name: str, doc: str | None = ellipsis) -> None
+
+# These come from `builtins.object`, not `types.ModuleType`:
+reveal_type(typing.__eq__)  # revealed: bound method ModuleType.__eq__(value: object, /) -> bool
+
+reveal_type(typing.__class__)  # revealed: Literal[ModuleType]
+
+reveal_type(typing.__dict__)  # revealed: dict[str, Any]
+```
+
+Typeshed includes a fake `__getattr__` method in the stub for `types.ModuleType` to help out with
+dynamic imports; but we ignore that for module-literal types where we know exactly which module
+we're dealing with:
+
+```py
+# error: [unresolved-attribute]
+reveal_type(typing.__getattr__)  # revealed: Unknown
+```
+
+## `types.ModuleType.__dict__` takes precedence over global variable `__dict__`
+
+It's impossible to override the `__dict__` attribute of `types.ModuleType` instances from inside the
+module; we should prioritise the attribute in the `types.ModuleType` stub over a variable named
+`__dict__` in the module's global namespace:
+
+`foo.py`:
+
+```py
+__dict__ = "foo"
+
+reveal_type(__dict__)  # revealed: Literal["foo"]
+```
+
+`bar.py`:
+
+```py
+import foo
+from foo import __dict__ as foo_dict
+
+reveal_type(foo.__dict__)  # revealed: dict[str, Any]
+reveal_type(foo_dict)  # revealed: dict[str, Any]
+```
+
+## Conditionally global or `ModuleType` attribute
+
+Attributes overridden in the module namespace take priority. If a builtin name is conditionally
+defined as a global, however, a name lookup should union the `ModuleType` type with the
+conditionally defined type:
+
+```py
+__file__ = 42
+
+def returns_bool() -> bool:
+    return True
+
+if returns_bool():
+    __name__ = 1
+
+reveal_type(__file__)  # revealed: Literal[42]
+reveal_type(__name__)  # revealed: Literal[1] | str
+```
+
+## Conditionally global or `ModuleType` attribute, with annotation
+
+The same is true if the name is annotated:
+
+```py
+__file__: int = 42
+
+def returns_bool() -> bool:
+    return True
+
+if returns_bool():
+    __name__: int = 1
+
+reveal_type(__file__)  # revealed: Literal[42]
+reveal_type(__name__)  # revealed: Literal[1] | str
+```
+
+## Implicit global attributes in the current module override implicit globals from builtins
+
+Here, we take the type of the implicit global symbol `__name__` from the `types.ModuleType` stub
+(which in this custom typeshed specifies the type as `bytes`). This is because the `main` module has
+an implicit `__name__` global that shadows the builtin `__name__` symbol.
+
+```toml
+[environment]
+typeshed = "/typeshed"
+```
+
+`/typeshed/stdlib/builtins.pyi`:
+
+```pyi
+class object: ...
+class int: ...
+class bytes: ...
+
+__name__: int = 42
+```
+
+`/typeshed/stdlib/types.pyi`:
+
+```pyi
+class ModuleType:
+    __name__: bytes
+```
+
+`/typeshed/stdlib/typing_extensions.pyi`:
+
+```pyi
+def reveal_type(obj, /): ...
+```
+
+`main.py`:
+
+```py
+reveal_type(__name__)  # revealed: bytes
+```

crates/ty_python_semantic/resources/mdtest/scopes/nonlocal.md

@@ -0,0 +1,45 @@
+# Nonlocal references
+
+## One level up
+
+```py
+def f():
+    x = 1
+    def g():
+        reveal_type(x)  # revealed: Unknown | Literal[1]
+```
+
+## Two levels up
+
+```py
+def f():
+    x = 1
+    def g():
+        def h():
+            reveal_type(x)  # revealed: Unknown | Literal[1]
+```
+
+## Skips class scope
+
+```py
+def f():
+    x = 1
+
+    class C:
+        x = 2
+        def g():
+            reveal_type(x)  # revealed: Unknown | Literal[1]
+```
+
+## Skips annotation-only assignment
+
+```py
+def f():
+    x = 1
+    def g():
+        # it's pretty weird to have an annotated assignment in a function where the
+        # name is otherwise not defined; maybe should be an error?
+        x: int
+        def h():
+            reveal_type(x)  # revealed: Unknown | Literal[1]
+```

crates/ty_python_semantic/resources/mdtest/scopes/unbound.md

@@ -0,0 +1,57 @@
+# Unbound
+
+## Unbound class variable
+
+Name lookups within a class scope fall back to globals, but lookups of class attributes don't.
+
+```py
+def coinflip() -> bool:
+    return True
+
+flag = coinflip()
+x = 1
+
+class C:
+    y = x
+    if flag:
+        x = 2
+
+# error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C]` is possibly unbound"
+reveal_type(C.x)  # revealed: Unknown | Literal[2]
+reveal_type(C.y)  # revealed: Unknown | Literal[1]
+```
+
+## Possibly unbound in class and global scope
+
+```py
+def coinflip() -> bool:
+    return True
+
+if coinflip():
+    x = "abc"
+
+class C:
+    if coinflip():
+        x = 1
+
+    # error: [possibly-unresolved-reference]
+    y = x
+
+reveal_type(C.y)  # revealed: Unknown | Literal[1, "abc"]
+```
+
+## Unbound function local
+
+An unbound function local that has definitions in the scope does not fall back to globals.
+
+```py
+x = 1
+
+def f():
+    # error: [unresolved-reference]
+    # revealed: Unknown
+    reveal_type(x)
+    x = 2
+    # revealed: Literal[2]
+    reveal_type(x)
+```