mirror of https://github.com/astral-sh/ruff.git synced 2025-08-08 20:58:05 +00:00

Douglas Creager 7673d46b71

[ty] Splat variadic arguments into parameter list (#18996 )

This PR updates our call binding logic to handle splatted arguments.

Complicating matters is that we have separated call bind analysis into
two phases: parameter matching and type checking. Parameter matching
looks at the arity of the function signature and call site, and assigns
arguments to parameters. Importantly, we don't yet know the type of each
argument! This is needed so that we can decide whether to infer the type
of each argument as a type form or value form, depending on the
requirements of the parameter that the argument was matched to.

This is an issue when splatting an argument, since we need to know how
many elements the splatted argument contains to know how many positional
parameters to match it against. And to know how many elements the
splatted argument has, we need to know its type.

To get around this, we now make the assumption that splatted arguments
can only be used with value-form parameters. (If you end up splatting an
argument into a type-form parameter, we will silently pass in its
value-form type instead.) That allows us to preemptively infer the
(value-form) type of any splatted argument, so that we have its arity
available during parameter matching. We defer inference of non-splatted
arguments until after parameter matching has finished, as before.

We reuse a lot of the new tuple machinery to make this happen — in
particular resizing the tuple spec representing the number of arguments
passed in with the tuple length representing the number of parameters
the splat was matched with.

This work also shows that we might need to change how we are performing
argument expansion during overload resolution. At the moment, when we
expand parameters, we assume that each argument will still be matched to
the same parameters as before, and only retry the type-checking phase.
With splatted arguments, this is no longer the case, since the inferred
arity of each union element might be different than the arity of the
union as a whole, which can affect how many parameters the splatted
argument is matched to. See the regression test case in
`mdtest/call/function.md` for more details.

2025-07-22 14:33:08 -04:00

16 KiB

Raw Blame History

Call expression

Simple

def get_int() -> int:
    return 42

reveal_type(get_int())  # revealed: int

Async

async def get_int_async() -> int:
    return 42

# TODO: we don't yet support `types.CoroutineType`, should be generic `Coroutine[Any, Any, int]`
reveal_type(get_int_async())  # revealed: @Todo(generic types.CoroutineType)

Generic

[environment]
python-version = "3.12"

def get_int[T]() -> int:
    return 42

reveal_type(get_int())  # revealed: int

Decorated

from typing import Callable

def foo() -> int:
    return 42

def decorator(func) -> Callable[[], int]:
    return foo

@decorator
def bar() -> str:
    return "bar"

reveal_type(bar())  # revealed: int

Invalid callable

nonsense = 123
x = nonsense()  # error: "Object of type `Literal[123]` is not callable"

Potentially unbound function

def _(flag: bool):
    if flag:
        def foo() -> int:
            return 42
    # error: [possibly-unresolved-reference]
    reveal_type(foo())  # revealed: int

Splatted arguments

Unknown argument length

def takes_zero() -> None: ...
def takes_one(x: int) -> None: ...
def takes_two(x: int, y: int) -> None: ...
def takes_two_positional_only(x: int, y: int, /) -> None: ...
def takes_two_different(x: int, y: str) -> None: ...
def takes_two_different_positional_only(x: int, y: str, /) -> None: ...
def takes_at_least_zero(*args) -> None: ...
def takes_at_least_one(x: int, *args) -> None: ...
def takes_at_least_two(x: int, y: int, *args) -> None: ...
def takes_at_least_two_positional_only(x: int, y: int, /, *args) -> None: ...

# Test all of the above with a number of different splatted argument types

def _(args: list[int]) -> None:
    takes_zero(*args)
    takes_one(*args)
    takes_two(*args)
    takes_two_positional_only(*args)
    takes_two_different(*args)  # error: [invalid-argument-type]
    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)
    takes_at_least_two_positional_only(*args)

def _(args: tuple[int, ...]) -> None:
    takes_zero(*args)
    takes_one(*args)
    takes_two(*args)
    takes_two_positional_only(*args)
    takes_two_different(*args)  # error: [invalid-argument-type]
    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)
    takes_at_least_two_positional_only(*args)

Fixed-length tuple argument

def takes_zero() -> None: ...
def takes_one(x: int) -> None: ...
def takes_two(x: int, y: int) -> None: ...
def takes_two_positional_only(x: int, y: int, /) -> None: ...
def takes_two_different(x: int, y: str) -> None: ...
def takes_two_different_positional_only(x: int, y: str, /) -> None: ...
def takes_at_least_zero(*args) -> None: ...
def takes_at_least_one(x: int, *args) -> None: ...
def takes_at_least_two(x: int, y: int, *args) -> None: ...
def takes_at_least_two_positional_only(x: int, y: int, /, *args) -> None: ...

# Test all of the above with a number of different splatted argument types

def _(args: tuple[int]) -> None:
    takes_zero(*args)  # error: [too-many-positional-arguments]
    takes_one(*args)
    takes_two(*args)  # error: [missing-argument]
    takes_two_positional_only(*args)  # error: [missing-argument]
    takes_two_different(*args)  # error: [missing-argument]
    takes_two_different_positional_only(*args)  # error: [missing-argument]
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)  # error: [missing-argument]
    takes_at_least_two_positional_only(*args)  # error: [missing-argument]

def _(args: tuple[int, int]) -> None:
    takes_zero(*args)  # error: [too-many-positional-arguments]
    takes_one(*args)  # error: [too-many-positional-arguments]
    takes_two(*args)
    takes_two_positional_only(*args)
    takes_two_different(*args)  # error: [invalid-argument-type]
    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)
    takes_at_least_two_positional_only(*args)

def _(args: tuple[int, str]) -> None:
    takes_zero(*args)  # error: [too-many-positional-arguments]
    takes_one(*args)  # error: [too-many-positional-arguments]
    takes_two(*args)  # error: [invalid-argument-type]
    takes_two_positional_only(*args)  # error: [invalid-argument-type]
    takes_two_different(*args)
    takes_two_different_positional_only(*args)
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)  # error: [invalid-argument-type]
    takes_at_least_two_positional_only(*args)  # error: [invalid-argument-type]

Mixed tuple argument

[environment]
python-version = "3.11"

def takes_zero() -> None: ...
def takes_one(x: int) -> None: ...
def takes_two(x: int, y: int) -> None: ...
def takes_two_positional_only(x: int, y: int, /) -> None: ...
def takes_two_different(x: int, y: str) -> None: ...
def takes_two_different_positional_only(x: int, y: str, /) -> None: ...
def takes_at_least_zero(*args) -> None: ...
def takes_at_least_one(x: int, *args) -> None: ...
def takes_at_least_two(x: int, y: int, *args) -> None: ...
def takes_at_least_two_positional_only(x: int, y: int, /, *args) -> None: ...

# Test all of the above with a number of different splatted argument types

def _(args: tuple[int, *tuple[int, ...]]) -> None:
    takes_zero(*args)  # error: [too-many-positional-arguments]
    takes_one(*args)
    takes_two(*args)
    takes_two_positional_only(*args)
    takes_two_different(*args)  # error: [invalid-argument-type]
    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)
    takes_at_least_two_positional_only(*args)

def _(args: tuple[int, *tuple[str, ...]]) -> None:
    takes_zero(*args)  # error: [too-many-positional-arguments]
    takes_one(*args)
    takes_two(*args)  # error: [invalid-argument-type]
    takes_two_positional_only(*args)  # error: [invalid-argument-type]
    takes_two_different(*args)
    takes_two_different_positional_only(*args)
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)  # error: [invalid-argument-type]
    takes_at_least_two_positional_only(*args)  # error: [invalid-argument-type]

def _(args: tuple[int, int, *tuple[int, ...]]) -> None:
    takes_zero(*args)  # error: [too-many-positional-arguments]
    takes_one(*args)  # error: [too-many-positional-arguments]
    takes_two(*args)
    takes_two_positional_only(*args)
    takes_two_different(*args)  # error: [invalid-argument-type]
    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)
    takes_at_least_two_positional_only(*args)

def _(args: tuple[int, int, *tuple[str, ...]]) -> None:
    takes_zero(*args)  # error: [too-many-positional-arguments]
    takes_one(*args)  # error: [too-many-positional-arguments]
    takes_two(*args)
    takes_two_positional_only(*args)
    takes_two_different(*args)  # error: [invalid-argument-type]
    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)
    takes_at_least_two_positional_only(*args)

def _(args: tuple[int, *tuple[int, ...], int]) -> None:
    takes_zero(*args)  # error: [too-many-positional-arguments]
    takes_one(*args)  # error: [too-many-positional-arguments]
    takes_two(*args)
    takes_two_positional_only(*args)
    takes_two_different(*args)  # error: [invalid-argument-type]
    takes_two_different_positional_only(*args)  # error: [invalid-argument-type]
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)
    takes_at_least_two_positional_only(*args)

def _(args: tuple[int, *tuple[str, ...], int]) -> None:
    takes_zero(*args)  # error: [too-many-positional-arguments]
    takes_one(*args)  # error: [too-many-positional-arguments]
    takes_two(*args)  # error: [invalid-argument-type]
    takes_two_positional_only(*args)  # error: [invalid-argument-type]
    takes_two_different(*args)
    takes_two_different_positional_only(*args)
    takes_at_least_zero(*args)
    takes_at_least_one(*args)
    takes_at_least_two(*args)  # error: [invalid-argument-type]
    takes_at_least_two_positional_only(*args)  # error: [invalid-argument-type]

Argument expansion regression

This is a regression that was highlighted by the ecosystem check, which shows that we might need to rethink how we perform argument expansion during overload resolution. In particular, we might need to retry both match_parameters and check_types for each expansion. Currently we only retry check_types.

The issue is that argument expansion might produce a splatted value with a different arity than what we originally inferred for the unexpanded value, and that in turn can affect which parameters the splatted value is matched with.

The first example correctly produces an error. The tuple[int, str] union element has a precise arity of two, and so parameter matching chooses the first overload. The second element of the tuple does not match the second parameter type, which yielding an invalid-argument-type error.

The third example should produce the same error. However, because we have a union, we do not see the precise arity of each union element during parameter matching. Instead, we infer an arity of "zero or more" for the union as a whole, and use that less precise arity when matching parameters. We therefore consider the second overload to still be a potential candidate for the tuple[int, str] union element. During type checking, we have to force the arity of each union element to match the inferred arity of the union as a whole (turning tuple[int, str] into tuple[int | str, ...]). That less precise tuple type-checks successfully against the second overload, making us incorrectly think that tuple[int, str] is a valid splatted call.

If we update argument expansion to retry parameter matching with the precise arity of each union element, we will correctly rule out the second overload for tuple[int, str], just like we do when splatting that tuple directly (instead of as part of a union).

from typing import overload

@overload
def f(x: int, y: int) -> None: ...
@overload
def f(x: int, y: str, z: int) -> None: ...
def f(*args): ...

# Test all of the above with a number of different splatted argument types

def _(t: tuple[int, str]) -> None:
    f(*t)  # error: [invalid-argument-type]

def _(t: tuple[int, str, int]) -> None:
    f(*t)

def _(t: tuple[int, str] | tuple[int, str, int]) -> None:
    # TODO: error: [invalid-argument-type]
    f(*t)

Wrong argument type

Positional argument, positional-or-keyword parameter

def f(x: int) -> int:
    return 1

# error: 15 [invalid-argument-type] "Argument to function `f` is incorrect: Expected `int`, found `Literal["foo"]`"
reveal_type(f("foo"))  # revealed: int

Positional argument, positional-only parameter

def f(x: int, /) -> int:
    return 1

# error: 15 [invalid-argument-type] "Argument to function `f` is incorrect: Expected `int`, found `Literal["foo"]`"
reveal_type(f("foo"))  # revealed: int

Positional argument, variadic parameter

def f(*args: int) -> int:
    return 1

# error: 15 [invalid-argument-type] "Argument to function `f` is incorrect: Expected `int`, found `Literal["foo"]`"
reveal_type(f("foo"))  # revealed: int

Keyword argument, positional-or-keyword parameter

def f(x: int) -> int:
    return 1

# error: 15 [invalid-argument-type] "Argument to function `f` is incorrect: Expected `int`, found `Literal["foo"]`"
reveal_type(f(x="foo"))  # revealed: int

Keyword argument, keyword-only parameter

def f(*, x: int) -> int:
    return 1

# error: 15 [invalid-argument-type] "Argument to function `f` is incorrect: Expected `int`, found `Literal["foo"]`"
reveal_type(f(x="foo"))  # revealed: int

Keyword argument, keywords parameter

def f(**kwargs: int) -> int:
    return 1

# error: 15 [invalid-argument-type] "Argument to function `f` is incorrect: Expected `int`, found `Literal["foo"]`"
reveal_type(f(x="foo"))  # revealed: int

Correctly match keyword out-of-order

def f(x: int = 1, y: str = "foo") -> int:
    return 1

# error: 15 [invalid-argument-type] "Argument to function `f` is incorrect: Expected `str`, found `Literal[2]`"
# error: 20 [invalid-argument-type] "Argument to function `f` is incorrect: Expected `int`, found `Literal["bar"]`"
reveal_type(f(y=2, x="bar"))  # revealed: int

Too many positional arguments

One too many

def f() -> int:
    return 1

# error: 15 [too-many-positional-arguments] "Too many positional arguments to function `f`: expected 0, got 1"
reveal_type(f("foo"))  # revealed: int

Two too many

def f() -> int:
    return 1

# error: 15 [too-many-positional-arguments] "Too many positional arguments to function `f`: expected 0, got 2"
reveal_type(f("foo", "bar"))  # revealed: int

No too-many-positional if variadic is taken

def f(*args: int) -> int:
    return 1

reveal_type(f(1, 2, 3))  # revealed: int

Multiple keyword arguments map to keyword variadic parameter

def f(**kwargs: int) -> int:
    return 1

reveal_type(f(foo=1, bar=2))  # revealed: int

Missing arguments

No defaults or variadic

def f(x: int) -> int:
    return 1

# error: 13 [missing-argument] "No argument provided for required parameter `x` of function `f`"
reveal_type(f())  # revealed: int

With default

def f(x: int, y: str = "foo") -> int:
    return 1

# error: 13 [missing-argument] "No argument provided for required parameter `x` of function `f`"
reveal_type(f())  # revealed: int

Defaulted argument is not required

def f(x: int = 1) -> int:
    return 1

reveal_type(f())  # revealed: int

With variadic

def f(x: int, *y: str) -> int:
    return 1

# error: 13 [missing-argument] "No argument provided for required parameter `x` of function `f`"
reveal_type(f())  # revealed: int

Variadic argument is not required

def f(*args: int) -> int:
    return 1

reveal_type(f())  # revealed: int

Keywords argument is not required

def f(**kwargs: int) -> int:
    return 1

reveal_type(f())  # revealed: int

Multiple

def f(x: int, y: int) -> int:
    return 1

# error: 13 [missing-argument] "No arguments provided for required parameters `x`, `y` of function `f`"
reveal_type(f())  # revealed: int

Unknown argument

def f(x: int) -> int:
    return 1

# error: 20 [unknown-argument] "Argument `y` does not match any known parameter of function `f`"
reveal_type(f(x=1, y=2))  # revealed: int

Parameter already assigned

def f(x: int) -> int:
    return 1

# error: 18 [parameter-already-assigned] "Multiple values provided for parameter `x` of function `f`"
reveal_type(f(1, x=2))  # revealed: int

Special functions

Some functions require special handling in type inference. Here, we make sure that we still emit proper diagnostics in case of missing or superfluous arguments.

`reveal_type`

from typing_extensions import reveal_type

# error: [missing-argument] "No argument provided for required parameter `obj` of function `reveal_type`"
reveal_type()

# error: [too-many-positional-arguments] "Too many positional arguments to function `reveal_type`: expected 1, got 2"
reveal_type(1, 2)

`static_assert`

from ty_extensions import static_assert

# error: [missing-argument] "No argument provided for required parameter `condition` of function `static_assert`"
static_assert()

# error: [too-many-positional-arguments] "Too many positional arguments to function `static_assert`: expected 2, got 3"
static_assert(True, 2, 3)

`len`

# error: [missing-argument] "No argument provided for required parameter `obj` of function `len`"
len()

# error: [too-many-positional-arguments] "Too many positional arguments to function `len`: expected 1, got 2"
len([], 1)

Type API predicates

from ty_extensions import is_subtype_of

# error: [missing-argument]
is_subtype_of()

# error: [missing-argument]
is_subtype_of(int)

# error: [too-many-positional-arguments]
is_subtype_of(int, int, int)

# error: [too-many-positional-arguments]
is_subtype_of(int, int, int, int)

16 KiB Raw Blame History