7ea773daf2
## Summary
Part of astral-sh/ty#104, closes: astral-sh/ty#468
This PR implements the argument type expansion which is step 3 of the
overload call evaluation algorithm.
Specifically, this step needs to be taken if type checking resolves to
no matching overload and there are argument types that can be expanded.
## Test Plan
Add new test cases.
## Ecosystem analysis
This PR removes 174 `no-matching-overload` false positives -- I looked
at a lot of them and they all are false positives.
One thing that I'm not able to understand is that in
2b7e3adf27/sphinx/ext/autodoc/preserve_defaults.py (L179)
the inferred type of `value` is `str | None` by ty and Pyright, which is
correct, but it's only ty that raises `invalid-argument-type` error
while Pyright doesn't. The constructor method of `DefaultValue` has
declared type of `str` which is invalid.
There are few cases of false positives resulting due to the fact that ty
doesn't implement narrowing on attribute expressions.
7.8 KiB
Overloads
When ty evaluates the call of an overloaded function, it attempts to "match" the supplied arguments with one or more overloads. This document describes the algorithm that it uses for overload matching, which is the same as the one mentioned in the spec.
Arity check
The first step is to perform arity check. The non-overloaded cases are described in the function document.
overloaded.pyi:
from typing import overload
@overload
def f() -> None: ...
@overload
def f(x: int) -> int: ...
from overloaded import f
# These match a single overload
reveal_type(f()) # revealed: None
reveal_type(f(1)) # revealed: int
# error: [no-matching-overload] "No overload of function `f` matches arguments"
reveal_type(f("a", "b")) # revealed: Unknown
Type checking
The second step is to perform type checking. This is done for all the overloads that passed the arity check.
Single match
overloaded.pyi:
from typing import overload
@overload
def f(x: int) -> int: ...
@overload
def f(x: str) -> str: ...
@overload
def f(x: bytes) -> bytes: ...
Here, all of the calls below pass the arity check for all overloads, so we proceed to type checking which filters out all but the matching overload:
from overloaded import f
reveal_type(f(1)) # revealed: int
reveal_type(f("a")) # revealed: str
reveal_type(f(b"b")) # revealed: bytes
Single match error
overloaded.pyi:
from typing import overload
@overload
def f() -> None: ...
@overload
def f(x: int) -> int: ...
If the arity check only matches a single overload, it should be evaluated as a regular
(non-overloaded) function call. This means that any diagnostics resulted during type checking that
call should be reported directly and not as a no-matching-overload error.
from overloaded import f
reveal_type(f()) # revealed: None
# TODO: This should be `invalid-argument-type` instead
# error: [no-matching-overload]
reveal_type(f("a")) # revealed: Unknown
Multiple matches
overloaded.pyi:
from typing import overload
class A: ...
class B(A): ...
@overload
def f(x: A) -> A: ...
@overload
def f(x: B, y: int = 0) -> B: ...
from overloaded import A, B, f
# These calls pass the arity check, and type checking matches both overloads:
reveal_type(f(A())) # revealed: A
reveal_type(f(B())) # revealed: A
# But, in this case, the arity check filters out the first overload, so we only have one match:
reveal_type(f(B(), 1)) # revealed: B
Argument type expansion
This step is performed only if the previous steps resulted in no matches.
In this case, the algorithm would perform argument type expansion and loops over from the type checking step, evaluating the argument lists.
Expanding the only argument
overloaded.pyi:
from typing import overload
class A: ...
class B: ...
class C: ...
@overload
def f(x: A) -> A: ...
@overload
def f(x: B) -> B: ...
@overload
def f(x: C) -> C: ...
from overloaded import A, B, C, f
def _(ab: A | B, ac: A | C, bc: B | C):
reveal_type(f(ab)) # revealed: A | B
reveal_type(f(bc)) # revealed: B | C
reveal_type(f(ac)) # revealed: A | C
Expanding first argument
If the set of argument lists created by expanding the first argument evaluates successfully, the algorithm shouldn't expand the second argument.
overloaded.pyi:
from typing import Literal, overload
class A: ...
class B: ...
class C: ...
class D: ...
@overload
def f(x: A, y: C) -> A: ...
@overload
def f(x: A, y: D) -> B: ...
@overload
def f(x: B, y: C) -> C: ...
@overload
def f(x: B, y: D) -> D: ...
from overloaded import A, B, C, D, f
def _(a_b: A | B):
reveal_type(f(a_b, C())) # revealed: A | C
reveal_type(f(a_b, D())) # revealed: B | D
# But, if it doesn't, it should expand the second argument and try again:
def _(a_b: A | B, c_d: C | D):
reveal_type(f(a_b, c_d)) # revealed: A | B | C | D
Expanding second argument
If the first argument cannot be expanded, the algorithm should move on to the second argument, keeping the first argument as is.
overloaded.pyi:
from typing import overload
class A: ...
class B: ...
class C: ...
class D: ...
@overload
def f(x: A, y: B) -> B: ...
@overload
def f(x: A, y: C) -> C: ...
@overload
def f(x: B, y: D) -> D: ...
from overloaded import A, B, C, D, f
def _(a: A, bc: B | C, cd: C | D):
# This also tests that partial matching works correctly as the argument type expansion results
# in matching the first and second overloads, but not the third one.
reveal_type(f(a, bc)) # revealed: B | C
# error: [no-matching-overload] "No overload of function `f` matches arguments"
reveal_type(f(a, cd)) # revealed: Unknown
Generics (legacy)
overloaded.pyi:
from typing import TypeVar, overload
_T = TypeVar("_T")
class A: ...
class B: ...
@overload
def f(x: A) -> A: ...
@overload
def f(x: _T) -> _T: ...
from overloaded import A, f
def _(x: int, y: A | int):
reveal_type(f(x)) # revealed: int
reveal_type(f(y)) # revealed: A | int
Generics (PEP 695)
[environment]
python-version = "3.12"
overloaded.pyi:
from typing import overload
class A: ...
class B: ...
@overload
def f(x: B) -> B: ...
@overload
def f[T](x: T) -> T: ...
from overloaded import B, f
def _(x: int, y: B | int):
reveal_type(f(x)) # revealed: int
reveal_type(f(y)) # revealed: B | int
Expanding bool
overloaded.pyi:
from typing import Literal, overload
class T: ...
class F: ...
@overload
def f(x: Literal[True]) -> T: ...
@overload
def f(x: Literal[False]) -> F: ...
from overloaded import f
def _(flag: bool):
reveal_type(f(True)) # revealed: T
reveal_type(f(False)) # revealed: F
reveal_type(f(flag)) # revealed: T | F
Expanding tuple
overloaded.pyi:
from typing import Literal, overload
class A: ...
class B: ...
class C: ...
class D: ...
@overload
def f(x: tuple[A, int], y: tuple[int, Literal[True]]) -> A: ...
@overload
def f(x: tuple[A, int], y: tuple[int, Literal[False]]) -> B: ...
@overload
def f(x: tuple[B, int], y: tuple[int, Literal[True]]) -> C: ...
@overload
def f(x: tuple[B, int], y: tuple[int, Literal[False]]) -> D: ...
from overloaded import A, B, f
def _(x: tuple[A | B, int], y: tuple[int, bool]):
reveal_type(f(x, y)) # revealed: A | B | C | D
Expanding type
There's no special handling for expanding type[A | B] type because ty stores this type in it's
distributed form, which is type[A] | type[B].
overloaded.pyi:
from typing import overload
class A: ...
class B: ...
@overload
def f(x: type[A]) -> A: ...
@overload
def f(x: type[B]) -> B: ...
from overloaded import A, B, f
def _(x: type[A | B]):
reveal_type(x) # revealed: type[A] | type[B]
reveal_type(f(x)) # revealed: A | B
Expanding enums
overloaded.pyi:
from enum import Enum
from typing import Literal, overload
class SomeEnum(Enum):
A = 1
B = 2
C = 3
class A: ...
class B: ...
class C: ...
@overload
def f(x: Literal[SomeEnum.A]) -> A: ...
@overload
def f(x: Literal[SomeEnum.B]) -> B: ...
@overload
def f(x: Literal[SomeEnum.C]) -> C: ...
from overloaded import SomeEnum, A, B, C, f
def _(x: SomeEnum):
reveal_type(f(SomeEnum.A)) # revealed: A
# TODO: This should be `B` once enums are supported and are expanded
reveal_type(f(SomeEnum.B)) # revealed: A
# TODO: This should be `C` once enums are supported and are expanded
reveal_type(f(SomeEnum.C)) # revealed: A
# TODO: This should be `A | B | C` once enums are supported and are expanded
reveal_type(f(x)) # revealed: A