ruff/crates/red_knot_python_semantic/resources/mdtest/attributes.md
David Peter 1feb3cf41a
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / cargo fmt (push) Waiting to run
CI / cargo clippy (push) Blocked by required conditions
CI / cargo test (linux) (push) Blocked by required conditions
CI / cargo test (linux, release) (push) Blocked by required conditions
CI / cargo test (windows) (push) Blocked by required conditions
CI / cargo test (wasm) (push) Blocked by required conditions
CI / cargo build (release) (push) Waiting to run
CI / cargo build (msrv) (push) Blocked by required conditions
CI / cargo fuzz build (push) Blocked by required conditions
CI / fuzz parser (push) Blocked by required conditions
CI / test scripts (push) Blocked by required conditions
CI / ecosystem (push) Blocked by required conditions
CI / cargo shear (push) Blocked by required conditions
CI / python package (push) Waiting to run
CI / pre-commit (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / formatter instabilities and black similarity (push) Blocked by required conditions
CI / test ruff-lsp (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions
[red-knot] Use Unknown | T_inferred for undeclared public symbols (#15674)
## Summary

Use `Unknown | T_inferred` as the type for *undeclared* public symbols.

## Test Plan

- Updated existing tests
- New test for external `__slots__` modifications.
- New tests for external modifications of public symbols.
2025-01-24 12:47:48 +01:00

17 KiB

Attributes

Tests for attribute access on various kinds of types.

Class and instance variables

Pure instance variables

Variable only declared/bound in __init__

Variables only declared and/or bound in __init__ are pure instance variables. They cannot be accessed on the class itself.

class C:
    def __init__(self, value2: int, flag: bool = False) -> None:
        # bound but not declared
        self.pure_instance_variable1 = "value set in __init__"

        # bound but not declared - with type inferred from parameter
        self.pure_instance_variable2 = value2

        # declared but not bound
        self.pure_instance_variable3: bytes

        # declared and bound
        self.pure_instance_variable4: bool = True

        # possibly undeclared/unbound
        if flag:
            self.pure_instance_variable5: str = "possibly set in __init__"

c_instance = C(1)

# TODO: should be `Literal["value set in __init__"]`, or `Unknown | Literal[…]` to allow
# assignments to this unannotated attribute from other scopes.
reveal_type(c_instance.pure_instance_variable1)  # revealed: @Todo(implicit instance attribute)

# TODO: should be `int`
reveal_type(c_instance.pure_instance_variable2)  # revealed: @Todo(implicit instance attribute)

# TODO: should be `bytes`
reveal_type(c_instance.pure_instance_variable3)  # revealed: @Todo(implicit instance attribute)

# TODO: should be `bool`
reveal_type(c_instance.pure_instance_variable4)  # revealed: @Todo(implicit instance attribute)

# TODO: should be `str`
# We probably don't want to emit a diagnostic for this being possibly undeclared/unbound.
# mypy and pyright do not show an error here.
reveal_type(c_instance.pure_instance_variable5)  # revealed: @Todo(implicit instance attribute)

# TODO: If we choose to infer a precise `Literal[…]` type for the instance attribute (see
# above), this should be an error: incompatible types in assignment. If we choose to infer
# a gradual `Unknown | Literal[…]` type, this assignment is fine.
c_instance.pure_instance_variable1 = "value set on instance"

# TODO: this should be an error (incompatible types in assignment)
c_instance.pure_instance_variable2 = "incompatible"

# TODO: we already show an error here but the message might be improved?
# mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
# error: [unresolved-attribute] "Type `Literal[C]` has no attribute `pure_instance_variable1`"
reveal_type(C.pure_instance_variable1)  # revealed: Unknown

# TODO: this should be an error (pure instance variables cannot be accessed on the class)
# mypy shows no error here, but pyright raises "reportAttributeAccessIssue"
C.pure_instance_variable1 = "overwritten on class"

c_instance.pure_instance_variable4 = False

# TODO: After this assignment to the attribute within this scope, we may eventually want to narrow
# the `bool` type (see above) for this instance variable to `Literal[False]` here. This is unsound
# in general (we don't know what else happened to `c_instance` between the assignment and the use
# here), but mypy and pyright support this. In conclusion, this could be `bool` but should probably
# be `Literal[False]`.
reveal_type(c_instance.pure_instance_variable4)  # revealed: @Todo(implicit instance attribute)

Variable declared in class body and declared/bound in __init__

The same rule applies even if the variable is declared (not bound!) in the class body: it is still a pure instance variable.

class C:
    pure_instance_variable: str

    def __init__(self) -> None:
        self.pure_instance_variable = "value set in __init__"

c_instance = C()

reveal_type(c_instance.pure_instance_variable)  # revealed: str

# TODO: we currently plan to emit a diagnostic here. Note that both mypy
# and pyright show no error in this case! So we may reconsider this in
# the future, if it turns out to produce too many false positives.
reveal_type(C.pure_instance_variable)  # revealed: str

# TODO: same as above. We plan to emit a diagnostic here, even if both mypy
# and pyright allow this.
C.pure_instance_variable = "overwritten on class"

# error: [invalid-assignment] "Object of type `Literal[1]` is not assignable to attribute `pure_instance_variable` of type `str`"
c_instance.pure_instance_variable = 1

Variable only defined in unrelated method

We also recognize pure instance variables if they are defined in a method that is not __init__.

class C:
    def set_instance_variable(self) -> None:
        self.pure_instance_variable = "value set in method"

c_instance = C()

# Not that we would use this in static analysis, but for a more realistic example, let's actually
# call the method, so that the attribute is bound if this example is actually run.
c_instance.set_instance_variable()

# TODO: should be `Literal["value set in method"]` or `Unknown | Literal[…]` (see above).
reveal_type(c_instance.pure_instance_variable)  # revealed: @Todo(implicit instance attribute)

# TODO: We already show an error here, but the message might be improved?
# error: [unresolved-attribute]
reveal_type(C.pure_instance_variable)  # revealed: Unknown

# TODO: this should be an error
C.pure_instance_variable = "overwritten on class"

Variable declared in class body and not bound anywhere

If a variable is declared in the class body but not bound anywhere, we still consider it a pure instance variable and allow access to it via instances.

class C:
    pure_instance_variable: str

c_instance = C()

reveal_type(c_instance.pure_instance_variable)  # revealed: str

# TODO: mypy and pyright do not show an error here, but we plan to emit a diagnostic.
# The type could be changed to 'Unknown' if we decide to emit an error?
reveal_type(C.pure_instance_variable)  # revealed: str

# TODO: mypy and pyright do not show an error here, but we plan to emit one.
C.pure_instance_variable = "overwritten on class"

Pure class variables (ClassVar)

Annotated with ClassVar type qualifier

Class variables annotated with the typing.ClassVar type qualifier are pure class variables. They cannot be overwritten on instances, but they can be accessed on instances.

For more details, see the typing spec on ClassVar.

from typing import ClassVar

class C:
    pure_class_variable1: ClassVar[str] = "value in class body"
    pure_class_variable2: ClassVar = 1

    def method(self):
        # TODO: this should be an error
        self.pure_class_variable1 = "value set through instance"

reveal_type(C.pure_class_variable1)  # revealed: str

# TODO: Should be `Unknown | Literal[1]`.
reveal_type(C.pure_class_variable2)  # revealed: Unknown

c_instance = C()

# It is okay to access a pure class variable on an instance.
reveal_type(c_instance.pure_class_variable1)  # revealed: str

# TODO: Should be `Unknown | Literal[1]`.
reveal_type(c_instance.pure_class_variable2)  # revealed: Unknown

# error: [invalid-attribute-access] "Cannot assign to ClassVar `pure_class_variable1` from an instance of type `C`"
c_instance.pure_class_variable1 = "value set on instance"

C.pure_class_variable1 = "overwritten on class"

# error: [invalid-assignment] "Object of type `Literal[1]` is not assignable to attribute `pure_class_variable1` of type `str`"
C.pure_class_variable1 = 1

class Subclass(C):
    pure_class_variable1: ClassVar[str] = "overwritten on subclass"

reveal_type(Subclass.pure_class_variable1)  # revealed: str

Variable only mentioned in a class method

We also consider a class variable to be a pure class variable if it is only mentioned in a class method.

class C:
    @classmethod
    def class_method(cls):
        cls.pure_class_variable = "value set in class method"

# for a more realistic example, let's actually call the method
C.class_method()

# TODO: We currently plan to support this and show no error here.
# mypy shows an error here, pyright does not.
# error: [unresolved-attribute]
reveal_type(C.pure_class_variable)  # revealed: Unknown

C.pure_class_variable = "overwritten on class"

# TODO: should be `Literal["overwritten on class"]`
# error: [unresolved-attribute]
reveal_type(C.pure_class_variable)  # revealed: Unknown

c_instance = C()
# TODO: should be `Literal["overwritten on class"]`
reveal_type(c_instance.pure_class_variable)  # revealed: @Todo(implicit instance attribute)

# TODO: should raise an error.
c_instance.pure_class_variable = "value set on instance"

Instance variables with class-level default values

These are instance attributes, but the fact that we can see that they have a binding (not a declaration) in the class body means that reading the value from the class directly is also permitted. This is the only difference for these attributes as opposed to "pure" instance attributes.

Basic

class C:
    variable_with_class_default1: str = "value in class body"
    variable_with_class_default2 = 1

    def instance_method(self):
        self.variable_with_class_default1 = "value set in instance method"

reveal_type(C.variable_with_class_default1)  # revealed: str

reveal_type(C.variable_with_class_default2)  # revealed: Unknown | Literal[1]

c_instance = C()

reveal_type(c_instance.variable_with_class_default1)  # revealed: str
reveal_type(c_instance.variable_with_class_default2)  # revealed: Unknown | Literal[1]

c_instance.variable_with_class_default1 = "value set on instance"

reveal_type(C.variable_with_class_default1)  # revealed: str

# TODO: Could be Literal["value set on instance"], or still `str` if we choose not to
# narrow the type.
reveal_type(c_instance.variable_with_class_default1)  # revealed: str

C.variable_with_class_default1 = "overwritten on class"

# TODO: Could be `Literal["overwritten on class"]`, or still `str` if we choose not to
# narrow the type.
reveal_type(C.variable_with_class_default1)  # revealed: str

# TODO: should still be `Literal["value set on instance"]`, or `str`.
reveal_type(c_instance.variable_with_class_default1)  # revealed: str

Union of attributes

def _(flag: bool):
    if flag:
        class C1:
            x = 1

    else:
        class C1:
            x = 2

    class C2:
        if flag:
            x = 3
        else:
            x = 4

    reveal_type(C1.x)  # revealed: Unknown | Literal[1, 2]
    reveal_type(C2.x)  # revealed: Unknown | Literal[3, 4]

Inherited class attributes

Basic

class A:
    X = "foo"

class B(A): ...
class C(B): ...

reveal_type(C.X)  # revealed: Unknown | Literal["foo"]

Multiple inheritance

class O: ...

class F(O):
    X = 56

class E(O):
    X = 42

class D(O): ...
class C(D, F): ...
class B(E, D): ...
class A(B, C): ...

# revealed: tuple[Literal[A], Literal[B], Literal[E], Literal[C], Literal[D], Literal[F], Literal[O], Literal[object]]
reveal_type(A.__mro__)

# `E` is earlier in the MRO than `F`, so we should use the type of `E.X`
reveal_type(A.X)  # revealed: Unknown | Literal[42]

Unions with possibly unbound paths

Definite boundness within a class

In this example, the x attribute is not defined in the C2 element of the union:

def _(flag1: bool, flag2: bool):
    class C1:
        x = 1

    class C2: ...

    class C3:
        x = 3

    C = C1 if flag1 else C2 if flag2 else C3

    # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
    reveal_type(C.x)  # revealed: Unknown | Literal[1, 3]

Possibly-unbound within a class

We raise the same diagnostic if the attribute is possibly-unbound in at least one element of the union:

def _(flag: bool, flag1: bool, flag2: bool):
    class C1:
        x = 1

    class C2:
        if flag:
            x = 2

    class C3:
        x = 3

    C = C1 if flag1 else C2 if flag2 else C3

    # error: [possibly-unbound-attribute] "Attribute `x` on type `Literal[C1, C2, C3]` is possibly unbound"
    reveal_type(C.x)  # revealed: Unknown | Literal[1, 2, 3]

Unions with all paths unbound

If the symbol is unbound in all elements of the union, we detect that:

def _(flag: bool):
    class C1: ...
    class C2: ...
    C = C1 if flag else C2

    # error: [unresolved-attribute] "Type `Literal[C1, C2]` has no attribute `x`"
    reveal_type(C.x)  # revealed: Unknown

Objects of all types have a __class__ method

import typing_extensions

reveal_type(typing_extensions.__class__)  # revealed: Literal[ModuleType]

a = 42
reveal_type(a.__class__)  # revealed: Literal[int]

b = "42"
reveal_type(b.__class__)  # revealed: Literal[str]

c = b"42"
reveal_type(c.__class__)  # revealed: Literal[bytes]

d = True
reveal_type(d.__class__)  # revealed: Literal[bool]

e = (42, 42)
reveal_type(e.__class__)  # revealed: Literal[tuple]

def f(a: int, b: typing_extensions.LiteralString, c: int | str, d: type[str]):
    reveal_type(a.__class__)  # revealed: type[int]
    reveal_type(b.__class__)  # revealed: Literal[str]
    reveal_type(c.__class__)  # revealed: type[int] | type[str]

    # `type[type]`, a.k.a., either the class `type` or some subclass of `type`.
    # It would be incorrect to infer `Literal[type]` here,
    # as `c` could be some subclass of `str` with a custom metaclass.
    # All we know is that the metaclass must be a (non-strict) subclass of `type`.
    reveal_type(d.__class__)  # revealed: type[type]

reveal_type(f.__class__)  # revealed: Literal[FunctionType]

class Foo: ...

reveal_type(Foo.__class__)  # revealed: Literal[type]

Module attributes

global_symbol: str = "a"
import mod

reveal_type(mod.global_symbol)  # revealed: str
mod.global_symbol = "b"

# error: [invalid-assignment] "Object of type `Literal[1]` is not assignable to attribute `global_symbol` of type `str`"
mod.global_symbol = 1

# error: [invalid-assignment] "Object of type `Literal[1]` is not assignable to attribute `global_symbol` of type `str`"
(_, mod.global_symbol) = (..., 1)

# TODO: this should be an error, but we do not understand list unpackings yet.
[_, mod.global_symbol] = [1, 2]

class IntIterator:
    def __next__(self) -> int:
        return 42

class IntIterable:
    def __iter__(self) -> IntIterator:
        return IntIterator()

# error: [invalid-assignment] "Object of type `int` is not assignable to attribute `global_symbol` of type `str`"
for mod.global_symbol in IntIterable():
    pass

Nested attributes

class Outer:
    class Nested:
        class Inner:
            attr: int = 1
import outer.nested.inner

reveal_type(outer.nested.inner.Outer.Nested.Inner.attr)  # revealed: int

# error: [invalid-assignment]
outer.nested.inner.Outer.Nested.Inner.attr = "a"

Literal types

Function-literal attributes

Most attribute accesses on function-literal types are delegated to types.FunctionType, since all functions are instances of that class:

def f(): ...

reveal_type(f.__defaults__)  # revealed: @Todo(full tuple[...] support) | None
reveal_type(f.__kwdefaults__)  # revealed: @Todo(generics) | None

Some attributes are special-cased, however:

def f(): ...

reveal_type(f.__get__)  # revealed: @Todo(`__get__` method on functions)
reveal_type(f.__call__)  # revealed: @Todo(`__call__` method on functions)

Int-literal attributes

Most attribute accesses on int-literal types are delegated to builtins.int, since all literal integers are instances of that class:

reveal_type((2).bit_length)  # revealed: @Todo(bound method)
reveal_type((2).denominator)  # revealed: @Todo(@property)

Some attributes are special-cased, however:

reveal_type((2).numerator)  # revealed: Literal[2]
reveal_type((2).real)  # revealed: Literal[2]

Bool-literal attributes

Most attribute accesses on bool-literal types are delegated to builtins.bool, since all literal bols are instances of that class:

reveal_type(True.__and__)  # revealed: @Todo(bound method)
reveal_type(False.__or__)  # revealed: @Todo(bound method)

Some attributes are special-cased, however:

reveal_type(True.numerator)  # revealed: Literal[1]
reveal_type(False.real)  # revealed: Literal[0]

Bytes-literal attributes

All attribute access on literal bytes types is currently delegated to buitins.bytes:

reveal_type(b"foo".join)  # revealed: @Todo(bound method)
reveal_type(b"foo".endswith)  # revealed: @Todo(bound method)

References

Some of the tests in the Class and instance variables section draw inspiration from pyright's documentation on this topic.