[red-knot] Method calls and the descriptor protocol (#16121)

## Summary

This PR achieves the following:

* Add support for checking method calls, and inferring return types from
method calls. For example:
  ```py
  reveal_type("abcde".find("abc"))  # revealed: int
  reveal_type("foo".encode(encoding="utf-8"))  # revealed: bytes
  
  "abcde".find(123)  # error: [invalid-argument-type]
  
  class C:
      def f(self) -> int:
          pass
  
  reveal_type(C.f)  # revealed: <function `f`>
  reveal_type(C().f)  # revealed: <bound method: `f` of `C`>
  
  C.f()  # error: [missing-argument]
  reveal_type(C().f())  # revealed: int
  ```
* Implement the descriptor protocol, i.e. properly call the `__get__`
method when a descriptor object is accessed through a class object or an
instance of a class. For example:
  ```py
  from typing import Literal
  
  class Ten:
def __get__(self, instance: object, owner: type | None = None) ->
Literal[10]:
          return 10
  
  class C:
      ten: Ten = Ten()
  
  reveal_type(C.ten)  # revealed: Literal[10]
  reveal_type(C().ten)  # revealed: Literal[10]
  ```
* Add support for member lookup on intersection types.
* Support type inference for `inspect.getattr_static(obj, attr)` calls.
This was mostly used as a debugging tool during development, but seems
more generally useful. It can be used to bypass the descriptor protocol.
For the example above:
  ```py
  from inspect import getattr_static
  
  reveal_type(getattr_static(C, "ten"))  # revealed: Ten
  ```
* Add a new `Type::Callable(…)` variant with the following sub-variants:
* `Type::Callable(CallableType::BoundMethod(…))` — represents bound
method objects, e.g. `C().f` above
* `Type::Callable(CallableType::MethodWrapperDunderGet(…))` — represents
`f.__get__` where `f` is a function
* `Type::Callable(WrapperDescriptorDunderGet)` — represents
`FunctionType.__get__`
* Add new known classes:
  * `types.MethodType`
  * `types.MethodWrapperType`
  * `types.WrapperDescriptorType`
  * `builtins.range`

## Performance analysis

On this branch, we do more work. We need to do more call checking, since
we now check all method calls. We also need to do ~twice as many member
lookups, because we need to check if a `__get__` attribute exists on
accessed members.

A brief analysis on `tomllib` shows that we now call `Type::call` 1780
times, compared to 612 calls before.

## Limitations

* Data descriptors are not yet supported, i.e. we do not infer correct
types for descriptor attribute accesses in `Store` context and do not
check writes to descriptor attributes. I felt like this was something
that could be split out as a follow-up without risking a major
architectural change.
* We currently distinguish between `Type::member` (with descriptor
protocol) and `Type::static_member` (without descriptor protocol). The
former corresponds to `obj.attr`, the latter corresponds to
`getattr_static(obj, "attr")`. However, to model some details correctly,
we would also need to distinguish between a static member lookup *with*
and *without* instance variables. The lookup without instance variables
corresponds to `find_name_in_mro`
[here](https://docs.python.org/3/howto/descriptor.html#invocation-from-an-instance).
We currently approximate both using `member_static`, which leads to two
open TODOs. Changing this would be a larger refactoring of
`Type::own_instance_member`, so I chose to leave it out of this PR.

## Test Plan

* New `call/methods.md` test suite for method calls
* New tests in `descriptor_protocol.md`
* New `call/getattr_static.md` test suite for `inspect.getattr_static`
* Various updated tests
This commit is contained in:
David Peter 2025-02-20 23:22:26 +01:00 committed by GitHub
parent f62e5406f2
commit d2e034adcd
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
21 changed files with 1577 additions and 174 deletions

View file

@ -73,12 +73,12 @@ qux = (foo, bar)
reveal_type(qux) # revealed: tuple[Literal["foo"], Literal["bar"]]
# TODO: Infer "LiteralString"
reveal_type(foo.join(qux)) # revealed: @Todo(Attribute access on `StringLiteral` types)
reveal_type(foo.join(qux)) # revealed: @Todo(decorated method)
template: LiteralString = "{}, {}"
reveal_type(template) # revealed: Literal["{}, {}"]
# TODO: Infer `LiteralString`
reveal_type(template.format(foo, bar)) # revealed: @Todo(Attribute access on `StringLiteral` types)
reveal_type(template.format(foo, bar)) # revealed: @Todo(decorated method)
```
### Assignability

View file

@ -1005,8 +1005,8 @@ reveal_type(f.__kwdefaults__) # revealed: @Todo(generics) | None
Some attributes are special-cased, however:
```py
reveal_type(f.__get__) # revealed: @Todo(`__get__` method on functions)
reveal_type(f.__call__) # revealed: @Todo(`__call__` method on functions)
reveal_type(f.__get__) # revealed: <method-wrapper `__get__` of `f`>
reveal_type(f.__call__) # revealed: <bound method `__call__` of `Literal[f]`>
```
### Int-literal attributes
@ -1015,7 +1015,7 @@ Most attribute accesses on int-literal types are delegated to `builtins.int`, si
integers are instances of that class:
```py
reveal_type((2).bit_length) # revealed: @Todo(bound method)
reveal_type((2).bit_length) # revealed: <bound method `bit_length` of `Literal[2]`>
reveal_type((2).denominator) # revealed: @Todo(@property)
```
@ -1029,11 +1029,11 @@ reveal_type((2).real) # revealed: Literal[2]
### Bool-literal attributes
Most attribute accesses on bool-literal types are delegated to `builtins.bool`, since all literal
bols are instances of that class:
bools are instances of that class:
```py
reveal_type(True.__and__) # revealed: @Todo(bound method)
reveal_type(False.__or__) # revealed: @Todo(bound method)
reveal_type(True.__and__) # revealed: @Todo(decorated method)
reveal_type(False.__or__) # revealed: @Todo(decorated method)
```
Some attributes are special-cased, however:
@ -1045,11 +1045,11 @@ reveal_type(False.real) # revealed: Literal[0]
### Bytes-literal attributes
All attribute access on literal `bytes` types is currently delegated to `buitins.bytes`:
All attribute access on literal `bytes` types is currently delegated to `builtins.bytes`:
```py
reveal_type(b"foo".join) # revealed: @Todo(bound method)
reveal_type(b"foo".endswith) # revealed: @Todo(bound method)
reveal_type(b"foo".join) # revealed: <bound method `join` of `Literal[b"foo"]`>
reveal_type(b"foo".endswith) # revealed: <bound method `endswith` of `Literal[b"foo"]`>
```
## Instance attribute edge cases
@ -1136,6 +1136,40 @@ class C:
reveal_type(C().x) # revealed: Unknown
```
### Builtin types attributes
This test can probably be removed eventually, but we currently include it because we do not yet
understand generic bases and protocols, and we want to make sure that we can still use builtin types
in our tests in the meantime. See the corresponding TODO in `Type::static_member` for more
information.
```py
class C:
a_int: int = 1
a_str: str = "a"
a_bytes: bytes = b"a"
a_bool: bool = True
a_float: float = 1.0
a_complex: complex = 1 + 1j
a_tuple: tuple[int] = (1,)
a_range: range = range(1)
a_slice: slice = slice(1)
a_type: type = int
a_none: None = None
reveal_type(C.a_int) # revealed: int
reveal_type(C.a_str) # revealed: str
reveal_type(C.a_bytes) # revealed: bytes
reveal_type(C.a_bool) # revealed: bool
reveal_type(C.a_float) # revealed: int | float
reveal_type(C.a_complex) # revealed: int | float | complex
reveal_type(C.a_tuple) # revealed: tuple[int]
reveal_type(C.a_range) # revealed: range
reveal_type(C.a_slice) # revealed: slice
reveal_type(C.a_type) # revealed: type
reveal_type(C.a_none) # revealed: None
```
## References
Some of the tests in the *Class and instance variables* section draw inspiration from

View file

@ -0,0 +1,133 @@
# `inspect.getattr_static`
## Basic usage
`inspect.getattr_static` is a function that returns attributes of an object without invoking the
descriptor protocol (for caveats, see the [official documentation]).
Consider the following example:
```py
import inspect
class Descriptor:
def __get__(self, instance, owner) -> str:
return 1
class C:
normal: int = 1
descriptor: Descriptor = Descriptor()
```
If we access attributes on an instance of `C` as usual, the descriptor protocol is invoked, and we
get a type of `str` for the `descriptor` attribute:
```py
c = C()
reveal_type(c.normal) # revealed: int
reveal_type(c.descriptor) # revealed: str
```
However, if we use `inspect.getattr_static`, we can see the underlying `Descriptor` type:
```py
reveal_type(inspect.getattr_static(c, "normal")) # revealed: int
reveal_type(inspect.getattr_static(c, "descriptor")) # revealed: Descriptor
```
For non-existent attributes, a default value can be provided:
```py
reveal_type(inspect.getattr_static(C, "normal", "default-arg")) # revealed: int
reveal_type(inspect.getattr_static(C, "non_existent", "default-arg")) # revealed: Literal["default-arg"]
```
When a non-existent attribute is accessed without a default value, the runtime raises an
`AttributeError`. We could emit a diagnostic for this case, but that is currently not supported:
```py
# TODO: we could emit a diagnostic here
reveal_type(inspect.getattr_static(C, "non_existent")) # revealed: Never
```
We can access attributes on objects of all kinds:
```py
import sys
reveal_type(inspect.getattr_static(sys, "platform")) # revealed: LiteralString
reveal_type(inspect.getattr_static(inspect, "getattr_static")) # revealed: Literal[getattr_static]
reveal_type(inspect.getattr_static(1, "real")) # revealed: Literal[1]
```
(Implicit) instance attributes can also be accessed through `inspect.getattr_static`:
```py
class D:
def __init__(self) -> None:
self.instance_attr: int = 1
reveal_type(inspect.getattr_static(D(), "instance_attr")) # revealed: int
```
## Error cases
We can only infer precise types if the attribute is a literal string. In all other cases, we fall
back to `Any`:
```py
import inspect
class C:
x: int = 1
def _(attr_name: str):
reveal_type(inspect.getattr_static(C(), attr_name)) # revealed: Any
reveal_type(inspect.getattr_static(C(), attr_name, 1)) # revealed: Any
```
But we still detect errors in the number or type of arguments:
```py
# error: [missing-argument] "No arguments provided for required parameters `obj`, `attr` of function `getattr_static`"
inspect.getattr_static()
# error: [missing-argument] "No argument provided for required parameter `attr`"
inspect.getattr_static(C())
# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`attr`) of function `getattr_static`; expected type `str`"
inspect.getattr_static(C(), 1)
# error: [too-many-positional-arguments] "Too many positional arguments to function `getattr_static`: expected 3, got 4"
inspect.getattr_static(C(), "x", "default-arg", "one too many")
```
## Possibly unbound attributes
```py
import inspect
def _(flag: bool):
class C:
if flag:
x: int = 1
reveal_type(inspect.getattr_static(C, "x", "default")) # revealed: int | Literal["default"]
```
## Gradual types
```py
import inspect
from typing import Any
def _(a: Any, tuple_of_any: tuple[Any]):
reveal_type(inspect.getattr_static(a, "x", "default")) # revealed: Any | Literal["default"]
# TODO: Ideally, this would just be `Literal[index]`
reveal_type(inspect.getattr_static(tuple_of_any, "index", "default")) # revealed: Literal[index] | Literal["default"]
```
[official documentation]: https://docs.python.org/3/library/inspect.html#inspect.getattr_static

View file

@ -0,0 +1,258 @@
# Methods
## Background: Functions as descriptors
> Note: See also this related section in the descriptor guide: [Functions and methods].
Say we have a simple class `C` with a function definition `f` inside its body:
```py
class C:
def f(self, x: int) -> str:
return "a"
```
Whenever we access the `f` attribute through the class object itself (`C.f`) or through an instance
(`C().f`), this access happens via the descriptor protocol. Functions are (non-data) descriptors
because they implement a `__get__` method. This is crucial in making sure that method calls work as
expected. In general, the signature of the `__get__` method in the descriptor protocol is
`__get__(self, instance, owner)`. The `self` argument is the descriptor object itself (`f`). The
passed value for the `instance` argument depends on whether the attribute is accessed from the class
object (in which case it is `None`), or from an instance (in which case it is the instance of type
`C`). The `owner` argument is the class itself (`C` of type `Literal[C]`). To summarize:
- `C.f` is equivalent to `getattr_static(C, "f").__get__(None, C)`
- `C().f` is equivalent to `getattr_static(C, "f").__get__(C(), C)`
Here, `inspect.getattr_static` is used to bypass the descriptor protocol and directly access the
function attribute. The way the special `__get__` method *on functions* works is as follows. In the
former case, if the `instance` argument is `None`, `__get__` simply returns the function itself. In
the latter case, it returns a *bound method* object:
```py
from inspect import getattr_static
reveal_type(getattr_static(C, "f")) # revealed: Literal[f]
reveal_type(getattr_static(C, "f").__get__) # revealed: <method-wrapper `__get__` of `f`>
reveal_type(getattr_static(C, "f").__get__(None, C)) # revealed: Literal[f]
reveal_type(getattr_static(C, "f").__get__(C(), C)) # revealed: <bound method `f` of `C`>
```
In conclusion, this is why we see the following two types when accessing the `f` attribute on the
class object `C` and on an instance `C()`:
```py
reveal_type(C.f) # revealed: Literal[f]
reveal_type(C().f) # revealed: <bound method `f` of `C`>
```
A bound method is a callable object that contains a reference to the `instance` that it was called
on (can be inspected via `__self__`), and the function object that it refers to (can be inspected
via `__func__`):
```py
bound_method = C().f
reveal_type(bound_method.__self__) # revealed: C
reveal_type(bound_method.__func__) # revealed: Literal[f]
```
When we call the bound method, the `instance` is implicitly passed as the first argument (`self`):
```py
reveal_type(C().f(1)) # revealed: str
reveal_type(bound_method(1)) # revealed: str
```
When we call the function object itself, we need to pass the `instance` explicitly:
```py
C.f(1) # error: [missing-argument]
reveal_type(C.f(C(), 1)) # revealed: str
```
When we access methods from derived classes, they will be bound to instances of the derived class:
```py
class D(C):
pass
reveal_type(D().f) # revealed: <bound method `f` of `D`>
```
If we access an attribute on a bound method object itself, it will defer to `types.MethodType`:
```py
reveal_type(bound_method.__hash__) # revealed: <bound method `__hash__` of `MethodType`>
```
If an attribute is not available on the bound method object, it will be looked up on the underlying
function object. We model this explicitly, which means that we can access `__kwdefaults__` on bound
methods, even though it is not available on `types.MethodType`:
```py
reveal_type(bound_method.__kwdefaults__) # revealed: @Todo(generics) | None
```
## Basic method calls on class objects and instances
```py
class Base:
def method_on_base(self, x: int | None) -> str:
return "a"
class Derived(Base):
def method_on_derived(self, x: bytes) -> tuple[int, str]:
return (1, "a")
reveal_type(Base().method_on_base(1)) # revealed: str
reveal_type(Base.method_on_base(Base(), 1)) # revealed: str
Base().method_on_base("incorrect") # error: [invalid-argument-type]
Base().method_on_base() # error: [missing-argument]
Base().method_on_base(1, 2) # error: [too-many-positional-arguments]
reveal_type(Derived().method_on_base(1)) # revealed: str
reveal_type(Derived().method_on_derived(b"abc")) # revealed: tuple[int, str]
reveal_type(Derived.method_on_base(Derived(), 1)) # revealed: str
reveal_type(Derived.method_on_derived(Derived(), b"abc")) # revealed: tuple[int, str]
```
## Method calls on literals
### Boolean literals
```py
reveal_type(True.bit_length()) # revealed: int
reveal_type(True.as_integer_ratio()) # revealed: tuple[int, Literal[1]]
```
### Integer literals
```py
reveal_type((42).bit_length()) # revealed: int
```
### String literals
```py
reveal_type("abcde".find("abc")) # revealed: int
reveal_type("foo".encode(encoding="utf-8")) # revealed: bytes
"abcde".find(123) # error: [invalid-argument-type]
```
### Bytes literals
```py
reveal_type(b"abcde".startswith(b"abc")) # revealed: bool
```
## Method calls on `LiteralString`
```py
from typing_extensions import LiteralString
def f(s: LiteralString) -> None:
reveal_type(s.find("a")) # revealed: int
```
## Method calls on `tuple`
```py
def f(t: tuple[int, str]) -> None:
reveal_type(t.index("a")) # revealed: int
```
## Method calls on unions
```py
from typing import Any
class A:
def f(self) -> int:
return 1
class B:
def f(self) -> str:
return "a"
def f(a_or_b: A | B, any_or_a: Any | A):
reveal_type(a_or_b.f) # revealed: <bound method `f` of `A`> | <bound method `f` of `B`>
reveal_type(a_or_b.f()) # revealed: int | str
reveal_type(any_or_a.f) # revealed: Any | <bound method `f` of `A`>
reveal_type(any_or_a.f()) # revealed: Any | int
```
## Method calls on `KnownInstance` types
```toml
[environment]
python-version = "3.12"
```
```py
type IntOrStr = int | str
reveal_type(IntOrStr.__or__) # revealed: <bound method `__or__` of `typing.TypeAliasType`>
```
## Error cases: Calling `__get__` for methods
The `__get__` method on `types.FunctionType` has the following overloaded signature in typeshed:
```py
from types import FunctionType, MethodType
from typing import overload
@overload
def __get__(self, instance: None, owner: type, /) -> FunctionType: ...
@overload
def __get__(self, instance: object, owner: type | None = None, /) -> MethodType: ...
```
Here, we test that this signature is enforced correctly:
```py
from inspect import getattr_static
class C:
def f(self, x: int) -> str:
return "a"
method_wrapper = getattr_static(C, "f").__get__
reveal_type(method_wrapper) # revealed: <method-wrapper `__get__` of `f`>
# All of these are fine:
method_wrapper(C(), C)
method_wrapper(C())
method_wrapper(C(), None)
method_wrapper(None, C)
# Passing `None` without an `owner` argument is an
# error: [missing-argument] "No argument provided for required parameter `owner`"
method_wrapper(None)
# Passing something that is not assignable to `type` as the `owner` argument is an
# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 2 (`owner`); expected type `type`"
method_wrapper(None, 1)
# Passing `None` as the `owner` argument when `instance` is `None` is an
# error: [invalid-argument-type] "Object of type `None` cannot be assigned to parameter 2 (`owner`); expected type `type`"
method_wrapper(None, None)
# Calling `__get__` without any arguments is an
# error: [missing-argument] "No argument provided for required parameter `instance`"
method_wrapper()
# Calling `__get__` with too many positional arguments is an
# error: [too-many-positional-arguments] "Too many positional arguments: expected 2, got 3"
method_wrapper(C(), C, "one too many")
```
[functions and methods]: https://docs.python.org/3/howto/descriptor.html#functions-and-methods

View file

@ -22,22 +22,26 @@ class Ten:
pass
class C:
ten = Ten()
ten: Ten = Ten()
c = C()
# TODO: this should be `Literal[10]`
reveal_type(c.ten) # revealed: Unknown | Ten
reveal_type(c.ten) # revealed: Literal[10]
# TODO: This should `Literal[10]`
reveal_type(C.ten) # revealed: Unknown | Ten
reveal_type(C.ten) # revealed: Literal[10]
# These are fine:
c.ten = 10
# TODO: This should not be an error
c.ten = 10 # error: [invalid-assignment]
C.ten = 10
# TODO: Both of these should be errors
# TODO: This should be an error (as the wrong type is being implicitly passed to `Ten.__set__`),
# but the error message is misleading.
# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Ten`"
c.ten = 11
# TODO: same as above
# error: [invalid-assignment] "Object of type `Literal[11]` is not assignable to attribute `ten` of type `Literal[10]`"
C.ten = 11
```
@ -57,24 +61,86 @@ class FlexibleInt:
self._value = int(value)
class C:
flexible_int = FlexibleInt()
flexible_int: FlexibleInt = FlexibleInt()
c = C()
# TODO: should be `int | None`
reveal_type(c.flexible_int) # revealed: Unknown | FlexibleInt
reveal_type(c.flexible_int) # revealed: int | None
# TODO: These should not be errors
# error: [invalid-assignment]
c.flexible_int = 42 # okay
# error: [invalid-assignment]
c.flexible_int = "42" # also okay!
# TODO: should be `int | None`
reveal_type(c.flexible_int) # revealed: Unknown | FlexibleInt
reveal_type(c.flexible_int) # revealed: int | None
# TODO: should be an error
# TODO: This should be an error, but the message needs to be improved.
# error: [invalid-assignment] "Object of type `None` is not assignable to attribute `flexible_int` of type `FlexibleInt`"
c.flexible_int = None # not okay
# TODO: should be `int | None`
reveal_type(c.flexible_int) # revealed: Unknown | FlexibleInt
reveal_type(c.flexible_int) # revealed: int | None
```
## Data and non-data descriptors
Descriptors that define `__set__` or `__delete__` are called *data descriptors*. An example\
of a data descriptor is a `property` with a setter and/or a deleter.\
Descriptors that only define `__get__`, meanwhile, are called *non-data descriptors*. Examples
include\
functions, `classmethod` or `staticmethod`).
The precedence chain for attribute access is (1) data descriptors, (2) instance attributes, and (3)
non-data descriptors.
```py
from typing import Literal
class DataDescriptor:
def __get__(self, instance: object, owner: type | None = None) -> Literal["data"]:
return "data"
def __set__(self, instance: int, value) -> None:
pass
class NonDataDescriptor:
def __get__(self, instance: object, owner: type | None = None) -> Literal["non-data"]:
return "non-data"
class C:
data_descriptor = DataDescriptor()
non_data_descriptor = NonDataDescriptor()
def f(self):
# This explains why data descriptors come first in the precedence chain. If
# instance attributes would take priority, we would override the descriptor
# here. Instead, this calls `DataDescriptor.__set__`, i.e. it does not affect
# the type of the `data_descriptor` attribute.
self.data_descriptor = 1
# However, for non-data descriptors, instance attributes do take precedence.
# So it is possible to override them.
self.non_data_descriptor = 1
c = C()
# TODO: This should ideally be `Unknown | Literal["data"]`.
#
# - Pyright also wrongly shows `int | Literal['data']` here
# - Mypy shows Literal["data"] here, but also shows Literal["non-data"] below.
#
reveal_type(c.data_descriptor) # revealed: Unknown | Literal["data", 1]
reveal_type(c.non_data_descriptor) # revealed: Unknown | Literal["non-data", 1]
reveal_type(C.data_descriptor) # revealed: Unknown | Literal["data"]
reveal_type(C.non_data_descriptor) # revealed: Unknown | Literal["non-data"]
# It is possible to override data descriptors via class objects. The following
# assignment does not call `DataDescriptor.__set__`. For this reason, we infer
# `Unknown | …` for all (descriptor) attributes.
C.data_descriptor = "something else" # This is okay
```
## Built-in `property` descriptor
@ -101,7 +167,7 @@ c = C()
reveal_type(c._name) # revealed: str | None
# Should be `str`
reveal_type(c.name) # revealed: @Todo(bound method)
reveal_type(c.name) # revealed: @Todo(decorated method)
# Should be `builtins.property`
reveal_type(C.name) # revealed: Literal[name]
@ -142,7 +208,7 @@ reveal_type(c1) # revealed: @Todo(return type)
reveal_type(C.get_name()) # revealed: @Todo(return type)
# TODO: should be `str`
reveal_type(C("42").get_name()) # revealed: @Todo(bound method)
reveal_type(C("42").get_name()) # revealed: @Todo(decorated method)
```
## Descriptors only work when used as class variables
@ -160,9 +226,10 @@ class Ten:
class C:
def __init__(self):
self.ten = Ten()
self.ten: Ten = Ten()
reveal_type(C().ten) # revealed: Unknown | Ten
# TODO: Should be Ten
reveal_type(C().ten) # revealed: Literal[10]
```
## Descriptors distinguishing between class and instance access
@ -186,13 +253,166 @@ class Descriptor:
return "called on class object"
class C:
d = Descriptor()
d: Descriptor = Descriptor()
# TODO: should be `Literal["called on class object"]
reveal_type(C.d) # revealed: Unknown | Descriptor
reveal_type(C.d) # revealed: LiteralString
# TODO: should be `Literal["called on instance"]
reveal_type(C().d) # revealed: Unknown | Descriptor
reveal_type(C().d) # revealed: LiteralString
```
## Undeclared descriptor arguments
If a descriptor attribute is not declared, we union with `Unknown`, just like for regular
attributes, since that attribute could be overwritten externally. Even a data descriptor with a
`__set__` method can be overwritten when accessed through a class object.
```py
class Descriptor:
def __get__(self, instance: object, owner: type | None = None) -> int:
return 1
def __set__(self, instance: object, value: int) -> None:
pass
class C:
descriptor = Descriptor()
C.descriptor = "something else"
# This could also be `Literal["something else"]` if we support narrowing of attribute types based on assignments
reveal_type(C.descriptor) # revealed: Unknown | int
```
## Descriptors with incorrect `__get__` signature
```py
class Descriptor:
# `__get__` method with missing parameters:
def __get__(self) -> int:
return 1
class C:
descriptor: Descriptor = Descriptor()
# TODO: This should be an error
reveal_type(C.descriptor) # revealed: Descriptor
```
## Possibly-unbound `__get__` method
```py
def _(flag: bool):
class MaybeDescriptor:
if flag:
def __get__(self, instance: object, owner: type | None = None) -> int:
return 1
class C:
descriptor: MaybeDescriptor = MaybeDescriptor()
# TODO: This should be `MaybeDescriptor | int`
reveal_type(C.descriptor) # revealed: int
```
## Dunder methods
Dunder methods are looked up on the meta type, but we still need to invoke the descriptor protocol:
```py
class SomeCallable:
def __call__(self, x: int) -> str:
return "a"
class Descriptor:
def __get__(self, instance: object, owner: type | None = None) -> SomeCallable:
return SomeCallable()
class B:
__call__: Descriptor = Descriptor()
b_instance = B()
reveal_type(b_instance(1)) # revealed: str
b_instance("bla") # error: [invalid-argument-type]
```
## Functions as descriptors
Functions are descriptors because they implement a `__get__` method. This is crucial in making sure
that method calls work as expected. See [this test suite](./call/methods.md) for more information.
Here, we only demonstrate how `__get__` works on functions:
```py
from inspect import getattr_static
def f(x: object) -> str:
return "a"
reveal_type(f) # revealed: Literal[f]
reveal_type(f.__get__) # revealed: <method-wrapper `__get__` of `f`>
reveal_type(f.__get__(None, type(f))) # revealed: Literal[f]
reveal_type(f.__get__(None, type(f))(1)) # revealed: str
wrapper_descriptor = getattr_static(f, "__get__")
reveal_type(wrapper_descriptor) # revealed: <wrapper-descriptor `__get__` of `function` objects>
reveal_type(wrapper_descriptor(f, None, type(f))) # revealed: Literal[f]
# Attribute access on the method-wrapper `f.__get__` falls back to `MethodWrapperType`:
reveal_type(f.__get__.__hash__) # revealed: <bound method `__hash__` of `MethodWrapperType`>
# Attribute access on the wrapper-descriptor falls back to `WrapperDescriptorType`:
reveal_type(wrapper_descriptor.__qualname__) # revealed: @Todo(@property)
```
We can also bind the free function `f` to an instance of a class `C`:
```py
class C: ...
bound_method = wrapper_descriptor(f, C(), C)
reveal_type(bound_method) # revealed: <bound method `f` of `C`>
```
We can then call it, and the instance of `C` is implicitly passed to the first parameter of `f`
(`x`):
```py
reveal_type(bound_method()) # revealed: str
```
Finally, we test some error cases for the call to the wrapper descriptor:
```py
# Calling the wrapper descriptor without any arguments is an
# error: [missing-argument] "No arguments provided for required parameters `self`, `instance`"
wrapper_descriptor()
# Calling it without the `instance` argument is an also an
# error: [missing-argument] "No argument provided for required parameter `instance`"
wrapper_descriptor(f)
# Calling it without the `owner` argument if `instance` is not `None` is an
# error: [missing-argument] "No argument provided for required parameter `owner`"
wrapper_descriptor(f, None)
# But calling it with an instance is fine (in this case, the `owner` argument is optional):
wrapper_descriptor(f, C())
# Calling it with something that is not a `FunctionType` as the first argument is an
# error: [invalid-argument-type] "Object of type `Literal[1]` cannot be assigned to parameter 1 (`self`); expected type `FunctionType`"
wrapper_descriptor(1, None, type(f))
# Calling it with something that is not a `type` as the `owner` argument is an
# error: [invalid-argument-type] "Object of type `Literal[f]` cannot be assigned to parameter 3 (`owner`); expected type `type`"
wrapper_descriptor(f, None, f)
# Calling it with too many positional arguments is an
# error: [too-many-positional-arguments] "Too many positional arguments: expected 3, got 4"
wrapper_descriptor(f, None, type(f), "one too many")
```
[descriptors]: https://docs.python.org/3/howto/descriptor.html

View file

@ -0,0 +1,15 @@
# Protocols
We do not support protocols yet, but to avoid false positives, we *partially* support some known
protocols.
## `typing.SupportsIndex`
```py
from typing import SupportsIndex, Literal
def _(some_int: int, some_literal_int: Literal[1], some_indexable: SupportsIndex):
a: SupportsIndex = some_int
b: SupportsIndex = some_literal_int
c: SupportsIndex = some_indexable
```

View file

@ -9,7 +9,7 @@ is unbound.
```py
reveal_type(__name__) # revealed: str
reveal_type(__file__) # revealed: str | None
reveal_type(__loader__) # revealed: LoaderProtocol | None
reveal_type(__loader__) # revealed: @Todo(instance attribute on class with dynamic base) | None
reveal_type(__package__) # revealed: str | None
reveal_type(__doc__) # revealed: str | None
@ -54,10 +54,10 @@ inside the module:
import typing
reveal_type(typing.__name__) # revealed: str
reveal_type(typing.__init__) # revealed: @Todo(bound method)
reveal_type(typing.__init__) # revealed: <bound method `__init__` of `ModuleType`>
# These come from `builtins.object`, not `types.ModuleType`:
reveal_type(typing.__eq__) # revealed: @Todo(bound method)
reveal_type(typing.__eq__) # revealed: <bound method `__eq__` of `ModuleType`>
reveal_type(typing.__class__) # revealed: Literal[ModuleType]

View file

@ -66,6 +66,6 @@ It is [recommended](https://docs.python.org/3/library/sys.html#sys.platform) to
```py
import sys
reveal_type(sys.platform.startswith("freebsd")) # revealed: @Todo(Attribute access on `LiteralString` types)
reveal_type(sys.platform.startswith("linux")) # revealed: @Todo(Attribute access on `LiteralString` types)
reveal_type(sys.platform.startswith("freebsd")) # revealed: bool
reveal_type(sys.platform.startswith("linux")) # revealed: bool
```

View file

@ -35,7 +35,7 @@ in strict mode.
```py
def f(x: type):
reveal_type(x) # revealed: type
reveal_type(x.__repr__) # revealed: @Todo(bound method)
reveal_type(x.__repr__) # revealed: <bound method `__repr__` of `type`>
class A: ...
@ -50,7 +50,7 @@ x: type = A() # error: [invalid-assignment]
```py
def f(x: type[object]):
reveal_type(x) # revealed: type
reveal_type(x.__repr__) # revealed: @Todo(bound method)
reveal_type(x.__repr__) # revealed: <bound method `__repr__` of `type`>
class A: ...

View file

@ -75,3 +75,19 @@ class Boom:
reveal_type(bool(Boom())) # revealed: bool
```
### Possibly unbound __bool__ method
```py
from typing import Literal
def flag() -> bool:
return True
class PossiblyUnboundTrue:
if flag():
def __bool__(self) -> Literal[True]:
return True
reveal_type(bool(PossiblyUnboundTrue())) # revealed: bool
```

View file

@ -109,6 +109,7 @@ pub enum KnownModule {
#[allow(dead_code)]
Abc, // currently only used in tests
Collections,
Inspect,
KnotExtensions,
}
@ -123,6 +124,7 @@ impl KnownModule {
Self::Sys => "sys",
Self::Abc => "abc",
Self::Collections => "collections",
Self::Inspect => "inspect",
Self::KnotExtensions => "knot_extensions",
}
}
@ -149,6 +151,7 @@ impl KnownModule {
"sys" => Some(Self::Sys),
"abc" => Some(Self::Abc),
"collections" => Some(Self::Collections),
"inspect" => Some(Self::Inspect),
"knot_extensions" => Some(Self::KnotExtensions),
_ => None,
}

File diff suppressed because it is too large Load diff

View file

@ -5,6 +5,11 @@ use super::Type;
pub(crate) struct CallArguments<'a, 'db>(Vec<Argument<'a, 'db>>);
impl<'a, 'db> CallArguments<'a, 'db> {
/// Create a [`CallArguments`] with no arguments.
pub(crate) fn none() -> Self {
Self(Vec::new())
}
/// Create a [`CallArguments`] from an iterator over non-variadic positional argument types.
pub(crate) fn positional(positional_tys: impl IntoIterator<Item = Type<'db>>) -> Self {
positional_tys
@ -29,6 +34,11 @@ impl<'a, 'db> CallArguments<'a, 'db> {
pub(crate) fn first_argument(&self) -> Option<Type<'db>> {
self.0.first().map(Argument::ty)
}
// TODO this should be eliminated in favor of [`bind_call`]
pub(crate) fn second_argument(&self) -> Option<Type<'db>> {
self.0.get(1).map(Argument::ty)
}
}
impl<'db, 'a, 'b> IntoIterator for &'b CallArguments<'a, 'db> {

View file

@ -193,6 +193,13 @@ impl<'db> CallBinding<'db> {
}
}
pub(crate) fn three_parameter_types(&self) -> Option<(Type<'db>, Type<'db>, Type<'db>)> {
match self.parameter_types() {
[first, second, third] => Some((*first, *second, *third)),
_ => None,
}
}
fn callable_name(&self, db: &'db dyn Db) -> Option<&str> {
match self.callable_ty {
Type::FunctionLiteral(function) => Some(function.name(db)),

View file

@ -72,6 +72,7 @@ impl<'db> ClassBase<'db> {
Type::Never
| Type::BooleanLiteral(_)
| Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::BytesLiteral(_)
| Type::IntLiteral(_)
| Type::StringLiteral(_)

View file

@ -8,8 +8,8 @@ use ruff_python_literal::escape::AsciiEscape;
use crate::types::class_base::ClassBase;
use crate::types::{
ClassLiteralType, InstanceType, IntersectionType, KnownClass, StringLiteralType, Type,
UnionType,
CallableType, ClassLiteralType, InstanceType, IntersectionType, KnownClass, StringLiteralType,
Type, UnionType,
};
use crate::Db;
use rustc_hash::FxHashMap;
@ -88,6 +88,24 @@ impl Display for DisplayRepresentation<'_> {
},
Type::KnownInstance(known_instance) => f.write_str(known_instance.repr(self.db)),
Type::FunctionLiteral(function) => f.write_str(function.name(self.db)),
Type::Callable(CallableType::BoundMethod(bound_method)) => {
write!(
f,
"<bound method `{method}` of `{instance}`>",
method = bound_method.function(self.db).name(self.db),
instance = bound_method.self_instance(self.db).display(self.db)
)
}
Type::Callable(CallableType::MethodWrapperDunderGet(function)) => {
write!(
f,
"<method-wrapper `__get__` of `{function}`>",
function = function.name(self.db)
)
}
Type::Callable(CallableType::WrapperDescriptorDunderGet) => {
f.write_str("<wrapper-descriptor `__get__` of `function` objects>")
}
Type::Union(union) => union.display(self.db).fmt(f),
Type::Intersection(intersection) => intersection.display(self.db).fmt(f),
Type::IntLiteral(n) => n.fmt(f),

View file

@ -3753,6 +3753,7 @@ impl<'db> TypeInferenceBuilder<'db> {
(
op @ (ast::UnaryOp::UAdd | ast::UnaryOp::USub | ast::UnaryOp::Invert),
Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::ModuleLiteral(_)
| Type::ClassLiteral(_)
| Type::SubclassOf(_)
@ -3780,7 +3781,7 @@ impl<'db> TypeInferenceBuilder<'db> {
match operand_type.try_call_dunder(
self.db(),
unary_dunder_method,
&CallArguments::positional([operand_type]),
&CallArguments::none(),
) {
Ok(outcome) => outcome.return_type(self.db()),
Err(e) => {
@ -3967,6 +3968,7 @@ impl<'db> TypeInferenceBuilder<'db> {
// fall back on looking for dunder methods on one of the operand types.
(
Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::ModuleLiteral(_)
| Type::ClassLiteral(_)
| Type::SubclassOf(_)
@ -3983,6 +3985,7 @@ impl<'db> TypeInferenceBuilder<'db> {
| Type::SliceLiteral(_)
| Type::Tuple(_),
Type::FunctionLiteral(_)
| Type::Callable(..)
| Type::ModuleLiteral(_)
| Type::ClassLiteral(_)
| Type::SubclassOf(_)
@ -4029,7 +4032,7 @@ impl<'db> TypeInferenceBuilder<'db> {
.try_call_dunder(
self.db(),
reflected_dunder,
&CallArguments::positional([right_ty, left_ty]),
&CallArguments::positional([left_ty]),
)
.map(|outcome| outcome.return_type(self.db()))
.or_else(|_| {
@ -4037,7 +4040,7 @@ impl<'db> TypeInferenceBuilder<'db> {
.try_call_dunder(
self.db(),
op.dunder(),
&CallArguments::positional([left_ty, right_ty]),
&CallArguments::positional([right_ty]),
)
.map(|outcome| outcome.return_type(self.db()))
})
@ -4953,7 +4956,7 @@ impl<'db> TypeInferenceBuilder<'db> {
match value_ty.try_call_dunder(
self.db(),
"__getitem__",
&CallArguments::positional([value_ty, slice_ty]),
&CallArguments::positional([slice_ty]),
) {
Ok(outcome) => return outcome.return_type(self.db()),
Err(err @ CallDunderError::PossiblyUnbound { .. }) => {

View file

@ -29,9 +29,10 @@ use std::sync::{Arc, Mutex, MutexGuard, OnceLock};
use crate::db::tests::{setup_db, TestDb};
use crate::symbol::{builtins_symbol, known_module_symbol};
use crate::types::{
IntersectionBuilder, KnownClass, KnownInstanceType, SubclassOfType, TupleType, Type, UnionType,
BoundMethodType, CallableType, IntersectionBuilder, KnownClass, KnownInstanceType,
SubclassOfType, TupleType, Type, UnionType,
};
use crate::KnownModule;
use crate::{Db, KnownModule};
use quickcheck::{Arbitrary, Gen};
/// A test representation of a type that can be transformed unambiguously into a real Type,
@ -67,6 +68,24 @@ pub(crate) enum Ty {
SubclassOfAbcClass(&'static str),
AlwaysTruthy,
AlwaysFalsy,
BuiltinsFunction(&'static str),
BuiltinsBoundMethod {
class: &'static str,
method: &'static str,
},
}
#[salsa::tracked]
fn create_bound_method<'db>(
db: &'db dyn Db,
function: Type<'db>,
builtins_class: Type<'db>,
) -> Type<'db> {
Type::Callable(CallableType::BoundMethod(BoundMethodType::new(
db,
function.expect_function_literal(),
builtins_class.to_instance(db),
)))
}
impl Ty {
@ -123,6 +142,13 @@ impl Ty {
),
Ty::AlwaysTruthy => Type::AlwaysTruthy,
Ty::AlwaysFalsy => Type::AlwaysFalsy,
Ty::BuiltinsFunction(name) => builtins_symbol(db, name).expect_type(),
Ty::BuiltinsBoundMethod { class, method } => {
let builtins_class = builtins_symbol(db, class).expect_type();
let function = builtins_class.static_member(db, method).expect_type();
create_bound_method(db, function, builtins_class)
}
}
}
}
@ -173,6 +199,16 @@ fn arbitrary_core_type(g: &mut Gen) -> Ty {
Ty::SubclassOfAbcClass("ABCMeta"),
Ty::AlwaysTruthy,
Ty::AlwaysFalsy,
Ty::BuiltinsFunction("chr"),
Ty::BuiltinsFunction("ascii"),
Ty::BuiltinsBoundMethod {
class: "str",
method: "isascii",
},
Ty::BuiltinsBoundMethod {
class: "int",
method: "bit_length",
},
])
.unwrap()
.clone()

View file

@ -21,6 +21,13 @@ pub(crate) struct Signature<'db> {
}
impl<'db> Signature<'db> {
pub(crate) fn new(parameters: Parameters<'db>, return_ty: Option<Type<'db>>) -> Self {
Self {
parameters,
return_ty,
}
}
/// Return a todo signature: (*args: Todo, **kwargs: Todo) -> Todo
pub(crate) fn todo() -> Self {
Self {
@ -64,6 +71,10 @@ impl<'db> Signature<'db> {
pub(crate) struct Parameters<'db>(Vec<Parameter<'db>>);
impl<'db> Parameters<'db> {
pub(crate) fn new(parameters: impl IntoIterator<Item = Parameter<'db>>) -> Self {
Self(parameters.into_iter().collect())
}
/// Return todo parameters: (*args: Todo, **kwargs: Todo)
fn todo() -> Self {
Self(vec![
@ -233,6 +244,18 @@ pub(crate) struct Parameter<'db> {
}
impl<'db> Parameter<'db> {
pub(crate) fn new(
name: Option<Name>,
annotated_ty: Option<Type<'db>>,
kind: ParameterKind<'db>,
) -> Self {
Self {
name,
annotated_ty,
kind,
}
}
fn from_node_and_kind(
db: &'db dyn Db,
definition: Definition<'db>,

View file

@ -64,8 +64,8 @@ impl<'db> SubclassOfType<'db> {
!self.is_dynamic()
}
pub(crate) fn member(self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
Type::from(self.subclass_of).member(db, name)
pub(crate) fn static_member(self, db: &'db dyn Db, name: &str) -> Symbol<'db> {
Type::from(self.subclass_of).static_member(db, name)
}
/// Return `true` if `self` is a subtype of `other`.

View file

@ -1,5 +1,7 @@
use std::cmp::Ordering;
use crate::types::CallableType;
use super::{
class_base::ClassBase, ClassLiteralType, DynamicType, InstanceType, KnownInstanceType,
TodoType, Type,
@ -54,6 +56,27 @@ pub(super) fn union_elements_ordering<'db>(left: &Type<'db>, right: &Type<'db>)
(Type::FunctionLiteral(_), _) => Ordering::Less,
(_, Type::FunctionLiteral(_)) => Ordering::Greater,
(
Type::Callable(CallableType::BoundMethod(left)),
Type::Callable(CallableType::BoundMethod(right)),
) => left.cmp(right),
(Type::Callable(CallableType::BoundMethod(_)), _) => Ordering::Less,
(_, Type::Callable(CallableType::BoundMethod(_))) => Ordering::Greater,
(
Type::Callable(CallableType::MethodWrapperDunderGet(left)),
Type::Callable(CallableType::MethodWrapperDunderGet(right)),
) => left.cmp(right),
(Type::Callable(CallableType::MethodWrapperDunderGet(_)), _) => Ordering::Less,
(_, Type::Callable(CallableType::MethodWrapperDunderGet(_))) => Ordering::Greater,
(
Type::Callable(CallableType::WrapperDescriptorDunderGet),
Type::Callable(CallableType::WrapperDescriptorDunderGet),
) => Ordering::Equal,
(Type::Callable(CallableType::WrapperDescriptorDunderGet), _) => Ordering::Less,
(_, Type::Callable(CallableType::WrapperDescriptorDunderGet)) => Ordering::Greater,
(Type::Tuple(left), Type::Tuple(right)) => left.cmp(right),
(Type::Tuple(_), _) => Ordering::Less,
(_, Type::Tuple(_)) => Ordering::Greater,