mirror of https://github.com/astral-sh/ruff.git synced 2025-10-09 18:02:19 +00:00

[ty] simplify return type of place_from_declarations (#19884 )

## Summary

A [passing
comment](https://github.com/astral-sh/ruff/pull/19711#issuecomment-3169312014)
led me to explore why we didn't report a class attribute as possibly
unbound if it was a method and defined in two different conditional
branches.

I found that the reason was because of our handling of "conflicting
declarations" in `place_from_declarations`. It returned a `Result` which
would be `Err` in case of conflicting declarations.

But we only actually care about conflicting declarations when we are
actually doing type inference on that scope and might emit a diagnostic
about it. And in all cases (including that one), we want to otherwise
proceed with the union of the declared types, as if there was no
conflict.

In several cases we were failing to handle the union of declared types
in the same way as a normal declared type if there was a declared-types
conflict. The `Result` return type made this mistake really easy to
make, as we'd match on e.g. `Ok(Place::Type(...))` and do one thing,
then match on `Err(...)` and do another, even though really both of
those cases should be handled the same.

This PR refactors `place_from_declarations` to instead return a struct
which always represents the declared type we should use in the same way,
as well as carrying the conflicting declared types, if any. This struct
has a method to allow us to explicitly ignore the declared-types
conflict (which is what we want in most cases), as well as a method to
get the declared type and the conflict information, in the case where we
want to emit a diagnostic on the conflict.

## Test Plan

Existing CI; added a test showing that we now understand a
multiply-conditionally-defined method as possibly-unbound.

This does trigger issues on a couple new fuzzer seeds, but the issues
are just new instances of an already-known (and rarely occurring)
problem which I already plan to address in a future PR, so I think it's
OK to land as-is.

I happened to build this initially on top of
https://github.com/astral-sh/ruff/pull/19711, which adds invalid-await
diagnostics, so I also updated some invalid-syntax tests to not await on
an invalid type, since the purpose of those tests is to check the
syntactic location of the `await`, not the validity of the awaited type.

2025-08-13 14:17:08 +00:00

9.3 KiB

Raw Blame History

Tests for invalid types in type expressions

Invalid types are rejected

Many types are illegal in the context of a type expression:

import typing
from ty_extensions import AlwaysTruthy, AlwaysFalsy
from typing_extensions import Literal, Never

class A: ...

def _(
    a: type[int],
    b: AlwaysTruthy,
    c: AlwaysFalsy,
    d: Literal[True],
    e: Literal["bar"],
    f: Literal[b"foo"],
    g: tuple[int, str],
    h: Never,
    i: int,
    j: A,
):
    def foo(): ...
    def invalid(
        a_: a,  # error: [invalid-type-form] "Variable of type `type[int]` is not allowed in a type expression"
        b_: b,  # error: [invalid-type-form]
        c_: c,  # error: [invalid-type-form]
        d_: d,  # error: [invalid-type-form]
        e_: e,  # error: [invalid-type-form]
        f_: f,  # error: [invalid-type-form]
        g_: g,  # error: [invalid-type-form]
        h_: h,  # error: [invalid-type-form]
        i_: typing,  # error: [invalid-type-form]
        j_: foo,  # error: [invalid-type-form]
        k_: i,  # error: [invalid-type-form] "Variable of type `int` is not allowed in a type expression"
        l_: j,  # error: [invalid-type-form] "Variable of type `A` is not allowed in a type expression"
    ):
        reveal_type(a_)  # revealed: Unknown
        reveal_type(b_)  # revealed: Unknown
        reveal_type(c_)  # revealed: Unknown
        reveal_type(d_)  # revealed: Unknown
        reveal_type(e_)  # revealed: Unknown
        reveal_type(f_)  # revealed: Unknown
        reveal_type(g_)  # revealed: Unknown
        reveal_type(h_)  # revealed: Unknown
        reveal_type(i_)  # revealed: Unknown
        reveal_type(j_)  # revealed: Unknown

Invalid AST nodes

def bar() -> None:
    return None

async def baz(): ...
async def outer():  # avoid unrelated syntax errors on yield, yield from, and await
    def _(
        a: 1,  # error: [invalid-type-form] "Int literals are not allowed in this context in a type expression"
        b: 2.3,  # error: [invalid-type-form] "Float literals are not allowed in type expressions"
        c: 4j,  # error: [invalid-type-form] "Complex literals are not allowed in type expressions"
        d: True,  # error: [invalid-type-form] "Boolean literals are not allowed in this context in a type expression"
        e: int | b"foo",  # error: [invalid-type-form] "Bytes literals are not allowed in this context in a type expression"
        f: 1 and 2,  # error: [invalid-type-form] "Boolean operations are not allowed in type expressions"
        g: 1 or 2,  # error: [invalid-type-form] "Boolean operations are not allowed in type expressions"
        h: (foo := 1),  # error: [invalid-type-form] "Named expressions are not allowed in type expressions"
        i: not 1,  # error: [invalid-type-form] "Unary operations are not allowed in type expressions"
        j: lambda: 1,  # error: [invalid-type-form] "`lambda` expressions are not allowed in type expressions"
        k: 1 if True else 2,  # error: [invalid-type-form] "`if` expressions are not allowed in type expressions"
        l: await baz(),  # error: [invalid-type-form] "`await` expressions are not allowed in type expressions"
        m: (yield 1),  # error: [invalid-type-form] "`yield` expressions are not allowed in type expressions"
        n: (yield from [1]),  # error: [invalid-type-form] "`yield from` expressions are not allowed in type expressions"
        o: 1 < 2,  # error: [invalid-type-form] "Comparison expressions are not allowed in type expressions"
        p: bar(),  # error: [invalid-type-form] "Function calls are not allowed in type expressions"
        q: int | f"foo",  # error: [invalid-type-form] "F-strings are not allowed in type expressions"
        r: [1, 2, 3][1:2],  # error: [invalid-type-form] "Slices are not allowed in type expressions"
    ):
        reveal_type(a)  # revealed: Unknown
        reveal_type(b)  # revealed: Unknown
        reveal_type(c)  # revealed: Unknown
        reveal_type(d)  # revealed: Unknown
        reveal_type(e)  # revealed: int | Unknown
        reveal_type(f)  # revealed: Unknown
        reveal_type(g)  # revealed: Unknown
        reveal_type(h)  # revealed: Unknown
        reveal_type(i)  # revealed: Unknown
        reveal_type(j)  # revealed: Unknown
        reveal_type(k)  # revealed: Unknown
        reveal_type(p)  # revealed: Unknown
        reveal_type(q)  # revealed: int | Unknown
        reveal_type(r)  # revealed: @Todo(unknown type subscript)

class Mat:
    def __init__(self, value: int):
        self.value = value

    def __matmul__(self, other) -> int:
        return 42

def invalid_binary_operators(
    a: "1" + "2",  # error: [invalid-type-form] "Invalid binary operator `+` in type annotation"
    b: 3 - 5.0,  # error: [invalid-type-form] "Invalid binary operator `-` in type annotation"
    c: 4 * -2,  # error: [invalid-type-form] "Invalid binary operator `*` in type annotation"
    d: Mat(4) @ Mat(2),  # error: [invalid-type-form] "Invalid binary operator `@` in type annotation"
    e: 10 / 2,  # error: [invalid-type-form] "Invalid binary operator `/` in type annotation"
    f: 10 % 3,  # error: [invalid-type-form] "Invalid binary operator `%` in type annotation"
    g: 2**-0.5,  # error: [invalid-type-form] "Invalid binary operator `**` in type annotation"
    h: 10 // 3,  # error: [invalid-type-form] "Invalid binary operator `//` in type annotation"
    i: 1 << 2,  # error: [invalid-type-form] "Invalid binary operator `<<` in type annotation"
    j: 4 >> 42,  # error: [invalid-type-form] "Invalid binary operator `>>` in type annotation"
    k: 5 ^ 3,  # error: [invalid-type-form] "Invalid binary operator `^` in type annotation"
    l: 5 & 3,  # error: [invalid-type-form] "Invalid binary operator `&` in type annotation"
):
    reveal_type(a)  # revealed: Unknown
    reveal_type(b)  # revealed: Unknown
    reveal_type(c)  # revealed: Unknown
    reveal_type(d)  # revealed: Unknown
    reveal_type(e)  # revealed: Unknown
    reveal_type(f)  # revealed: Unknown
    reveal_type(g)  # revealed: Unknown
    reveal_type(h)  # revealed: Unknown
    reveal_type(i)  # revealed: Unknown
    reveal_type(j)  # revealed: Unknown
    reveal_type(k)  # revealed: Unknown
    reveal_type(l)  # revealed: Unknown

Invalid Collection based AST nodes

[environment]
python-version = "3.12"

def _(
    a: {1: 2},  # error: [invalid-type-form] "Dict literals are not allowed in type expressions"
    b: {1, 2},  # error: [invalid-type-form] "Set literals are not allowed in type expressions"
    c: {k: v for k, v in [(1, 2)]},  # error: [invalid-type-form] "Dict comprehensions are not allowed in type expressions"
    d: [k for k in [1, 2]],  # error: [invalid-type-form] "List comprehensions are not allowed in type expressions"
    e: {k for k in [1, 2]},  # error: [invalid-type-form] "Set comprehensions are not allowed in type expressions"
    f: (k for k in [1, 2]),  # error: [invalid-type-form] "Generator expressions are not allowed in type expressions"
    # error: [invalid-type-form] "List literals are not allowed in this context in a type expression: Did you mean `tuple[int, str]`?"
    g: [int, str],
    # error: [invalid-type-form] "Tuple literals are not allowed in this context in a type expression: Did you mean `tuple[int, str]`?"
    h: (int, str),
    i: (),  # error: [invalid-type-form] "Tuple literals are not allowed in this context in a type expression: Did you mean `tuple[()]`?"
):
    reveal_type(a)  # revealed: Unknown
    reveal_type(b)  # revealed: Unknown
    reveal_type(c)  # revealed: Unknown
    reveal_type(d)  # revealed: Unknown
    reveal_type(e)  # revealed: Unknown
    reveal_type(f)  # revealed: Unknown
    reveal_type(g)  # revealed: Unknown
    reveal_type(h)  # revealed: Unknown
    reveal_type(i)  # revealed: Unknown

# error: [invalid-type-form] "List literals are not allowed in this context in a type expression: Did you mean `list[int]`?"
class name_0[name_2: [int]]:
    pass

# error: [invalid-type-form] "List literals are not allowed in this context in a type expression"
# error: [invalid-type-form] "Dict literals are not allowed in type expressions"
class name_4[name_1: [{}]]:
    pass

Diagnostics for common errors

Module-literal used when you meant to use a class from that module

It's pretty common in Python to accidentally use a module-literal type in a type expression when you meant to use a class by the same name that comes from that module. We emit a nice subdiagnostic for this case:

foo.py:

import datetime

def f(x: datetime): ...  # error: [invalid-type-form]

PIL/Image.py:

class Image: ...

bar.py:

from PIL import Image

def g(x: Image): ...  # error: [invalid-type-form]

List-literal used when you meant to use a list or tuple

def _(
    x: [int],  # error: [invalid-type-form]
) -> [int]:  # error: [invalid-type-form]
    return x

def _(
    x: [int, str],  # error: [invalid-type-form]
) -> [int, str]:  # error: [invalid-type-form]
    return x

Tuple-literal used when you meant to use a tuple

def _(
    x: (),  # error: [invalid-type-form]
) -> ():  # error: [invalid-type-form]
    return x

def _(
    x: (int,),  # error: [invalid-type-form]
) -> (int,):  # error: [invalid-type-form]
    return x

def _(
    x: (int, str),  # error: [invalid-type-form]
) -> (int, str):  # error: [invalid-type-form]
    return x

9.3 KiB Raw Blame History