mirror of
https://github.com/astral-sh/ruff.git
synced 2025-10-28 10:50:26 +00:00
71d711257a
## Summary Quoting from the newly added comment: Module-level globals can be mutated externally. A `MY_CONSTANT = 1` global might be changed to `"some string"` from code outside of the module that we're looking at, and so from a gradual-guarantee perspective, it makes sense to infer a type of `Literal[1] | Unknown` for global symbols. This allows the code that does the mutation to type check correctly, and for code that uses the global, it accurately reflects the lack of knowledge about the type. External modifications (or modifications through `global` statements) that would require a wider type are relatively rare. From a practical perspective, we can therefore achieve a better user experience by trusting the inferred type. Users who need the external mutation to work can always annotate the global with the wider type. And everyone else benefits from more precise type inference. I initially implemented this by applying literal promotion to the type of the unannotated module globals (as suggested in https://github.com/astral-sh/ty/issues/1069), but the ecosystem impact showed a lot of problems (https://github.com/astral-sh/ruff/pull/20643). I fixed/patched some of these problems, but this PR seems like a good first step, and it seems sensible to apply the literal promotion change in a second step that can be evaluated separately. closes https://github.com/astral-sh/ty/issues/1069 ## Ecosystem impact This seems like an (unexpectedly large) net positive with 650 fewer diagnostics overall.. even though this change will certainly catch more true positives. * There are 666 removed `type-assertion-failure` diagnostics, where we were previously used the correct type already, but removing the `Unknown` now leads to an "exact" match. * 1464 of the 1805 total new diagnostics are `unresolved-attribute` errors, most (1365) of which were previously `possibly-missing-attribute` errors. So they could also be counted as "changed" diagnostics. * For code that uses constants like ```py IS_PYTHON_AT_LEAST_3_10 = sys.version_info >= (3, 10) ``` where we would have previously inferred a type of `Literal[True/False] | Unknown`, removing the `Unknown` now allows us to do reachability analysis on branches that use these constants, and so we get a lot of favorable ecosystem changes because of that. * There is code like the following, where we previously emitted `conflicting-argument-forms` diagnostics on calls to the aliased `assert_type`, because its type was `Unknown | def …` (and the call to `Unknown` "used" the type form argument in a non type-form way): ```py if sys.version_info >= (3, 11): import typing assert_type = typing.assert_type else: import typing_extensions assert_type = typing_extensions.assert_type ``` * ~100 new `invalid-argument-type` false positives, due to missing `**kwargs` support (https://github.com/astral-sh/ty/issues/247) ## Typing conformance ```diff +protocols_modules.py:25:1: error[invalid-assignment] Object of type `<module '_protocols_modules1'>` is not assignable to `Options1` ``` This diagnostic should apparently not be there, but it looks like we also fail other tests in that file, so it seems to be a limitation that was previously hidden by `Unknown` somehow. ## Test Plan Updated tests and relatively thorough ecosystem analysis.
4.8 KiB
4.8 KiB
Narrowing for complex targets (attribute expressions, subscripts)
We support type narrowing for attributes and subscripts.
Attribute narrowing
Basic
from ty_extensions import Unknown
class C:
x: int | None = None
c = C()
reveal_type(c.x) # revealed: int | None
if c.x is not None:
reveal_type(c.x) # revealed: int
else:
reveal_type(c.x) # revealed: None
if c.x is not None:
c.x = None
reveal_type(c.x) # revealed: None
c = C()
if c.x is None:
c.x = 1
reveal_type(c.x) # revealed: int
class _:
reveal_type(c.x) # revealed: int
c = C()
class _:
if c.x is None:
c.x = 1
reveal_type(c.x) # revealed: int
# TODO: should be `int`
reveal_type(c.x) # revealed: int | None
class D:
x = None
def unknown() -> Unknown:
return 1
d = D()
reveal_type(d.x) # revealed: Unknown | None
d.x = 1
reveal_type(d.x) # revealed: Literal[1]
d.x = unknown()
reveal_type(d.x) # revealed: Unknown
Narrowing can be "reset" by assigning to the attribute:
c = C()
if c.x is None:
reveal_type(c.x) # revealed: None
c.x = 1
reveal_type(c.x) # revealed: Literal[1]
c.x = None
reveal_type(c.x) # revealed: None
reveal_type(c.x) # revealed: int | None
Narrowing can also be "reset" by assigning to the object:
c = C()
if c.x is None:
reveal_type(c.x) # revealed: None
c = C()
reveal_type(c.x) # revealed: int | None
reveal_type(c.x) # revealed: int | None
Multiple predicates
class C:
value: str | None
def foo(c: C):
if c.value and len(c.value):
reveal_type(c.value) # revealed: str & ~AlwaysFalsy
# error: [invalid-argument-type] "Argument to function `len` is incorrect: Expected `Sized`, found `str | None`"
if len(c.value) and c.value:
reveal_type(c.value) # revealed: str & ~AlwaysFalsy
if c.value is None or not len(c.value):
reveal_type(c.value) # revealed: str | None
else: # c.value is not None and len(c.value)
# TODO: should be # `str & ~AlwaysFalsy`
reveal_type(c.value) # revealed: str
Generic class
[environment]
python-version = "3.12"
class C[T]:
x: T
y: T
def __init__(self, x: T):
self.x = x
self.y = x
def f(a: int | None):
c = C(a)
reveal_type(c.x) # revealed: int | None
reveal_type(c.y) # revealed: int | None
if c.x is not None:
reveal_type(c.x) # revealed: int
# In this case, it may seem like we can narrow it down to `int`,
# but different values may be reassigned to `x` and `y` in another place.
reveal_type(c.y) # revealed: int | None
def g[T](c: C[T]):
reveal_type(c.x) # revealed: T@g
reveal_type(c.y) # revealed: T@g
reveal_type(c) # revealed: C[T@g]
if isinstance(c.x, int):
reveal_type(c.x) # revealed: T@g & int
reveal_type(c.y) # revealed: T@g
reveal_type(c) # revealed: C[T@g]
if isinstance(c.x, int) and isinstance(c.y, int):
reveal_type(c.x) # revealed: T@g & int
reveal_type(c.y) # revealed: T@g & int
# TODO: Probably better if inferred as `C[T & int]` (mypy and pyright don't support this)
reveal_type(c) # revealed: C[T@g]
With intermediate scopes
class C:
def __init__(self):
self.x: int | None = None
self.y: int | None = None
c = C()
reveal_type(c.x) # revealed: int | None
if c.x is not None:
reveal_type(c.x) # revealed: int
reveal_type(c.y) # revealed: int | None
if c.x is not None:
def _():
reveal_type(c.x) # revealed: int | None
def _():
if c.x is not None:
reveal_type(c.x) # revealed: int
Subscript narrowing
Number subscript
def _(t1: tuple[int | None, int | None], t2: tuple[int, int] | tuple[None, None]):
if t1[0] is not None:
reveal_type(t1[0]) # revealed: int
reveal_type(t1[1]) # revealed: int | None
n = 0
if t1[n] is not None:
# Non-literal subscript narrowing are currently not supported, as well as mypy, pyright
reveal_type(t1[0]) # revealed: int | None
reveal_type(t1[n]) # revealed: int | None
reveal_type(t1[1]) # revealed: int | None
if t2[0] is not None:
reveal_type(t2[0]) # revealed: int
# TODO: should be int
reveal_type(t2[1]) # revealed: int | None
String subscript
def _(d: dict[str, str | None]):
if d["a"] is not None:
reveal_type(d["a"]) # revealed: str
reveal_type(d["b"]) # revealed: str | None
Combined attribute and subscript narrowing
class C:
def __init__(self):
self.x: tuple[int | None, int | None] = (None, None)
class D:
def __init__(self):
self.c: tuple[C] | None = None
d = D()
if d.c is not None and d.c[0].x[0] is not None:
reveal_type(d.c[0].x[0]) # revealed: int