mirror of https://github.com/astral-sh/ruff.git synced 2025-10-06 08:30:42 +00:00

[ty] No union with Unknown for module-global symbols (#20664 )

## Summary

Quoting from the newly added comment:

Module-level globals can be mutated externally. A `MY_CONSTANT = 1`
global might be changed to `"some string"` from code outside of the
module that we're looking at, and so from a gradual-guarantee
perspective, it makes sense to infer a type of `Literal[1] | Unknown`
for global symbols. This allows the code that does the mutation to type
check correctly, and for code that uses the global, it accurately
reflects the lack of knowledge about the type.

External modifications (or modifications through `global` statements)
that would require a wider type are relatively rare. From a practical
perspective, we can therefore achieve a better user experience by
trusting the inferred type. Users who need the external mutation to work
can always annotate the global with the wider type. And everyone else
benefits from more precise type inference.

I initially implemented this by applying literal promotion to the type
of the unannotated module globals (as suggested in
https://github.com/astral-sh/ty/issues/1069), but the ecosystem impact
showed a lot of problems (https://github.com/astral-sh/ruff/pull/20643).
I fixed/patched some of these problems, but this PR seems like a good
first step, and it seems sensible to apply the literal promotion change
in a second step that can be evaluated separately.

closes https://github.com/astral-sh/ty/issues/1069

## Ecosystem impact

This seems like an (unexpectedly large) net positive with 650 fewer
diagnostics overall.. even though this change will certainly catch more
true positives.

* There are 666 removed `type-assertion-failure` diagnostics, where we
were previously used the correct type already, but removing the
`Unknown` now leads to an "exact" match.
* 1464 of the 1805 total new diagnostics are `unresolved-attribute`
errors, most (1365) of which were previously
`possibly-missing-attribute` errors. So they could also be counted as
"changed" diagnostics.
* For code that uses constants like
  ```py
  IS_PYTHON_AT_LEAST_3_10 = sys.version_info >= (3, 10)
  ```
where we would have previously inferred a type of `Literal[True/False] |
Unknown`, removing the `Unknown` now allows us to do reachability
analysis on branches that use these constants, and so we get a lot of
favorable ecosystem changes because of that.
* There is code like the following, where we previously emitted
`conflicting-argument-forms` diagnostics on calls to the aliased
`assert_type`, because its type was `Unknown | def …` (and the call to
`Unknown` "used" the type form argument in a non type-form way):
  ```py
  if sys.version_info >= (3, 11):
      import typing
  
      assert_type = typing.assert_type
  else:
      import typing_extensions
  
      assert_type = typing_extensions.assert_type
  ```
* ~100 new `invalid-argument-type` false positives, due to missing
`**kwargs` support (https://github.com/astral-sh/ty/issues/247)

## Typing conformance

```diff
+protocols_modules.py:25:1: error[invalid-assignment] Object of type `<module '_protocols_modules1'>` is not assignable to `Options1`
```

This diagnostic should apparently not be there, but it looks like we
also fail other tests in that file, so it seems to be a limitation that
was previously hidden by `Unknown` somehow.

## Test Plan

Updated tests and relatively thorough ecosystem analysis.

2025-10-01 16:40:30 +02:00

6.6 KiB

Raw Permalink Blame History

`global` references

Implicit global in function

A name reference to a never-defined symbol in a function is implicitly a global lookup.

x = 1

def f():
    reveal_type(x)  # revealed: Literal[1]

Explicit global in function

x = 1

def f():
    global x
    reveal_type(x)  # revealed: Literal[1]

Unassignable type in function

x: int = 1

def f():
    y: int = 1
    # error: [invalid-assignment] "Object of type `Literal[""]` is not assignable to `int`"
    y = ""

    global x
    # error: [invalid-assignment] "Object of type `Literal[""]` is not assignable to `int`"
    x = ""

    global z
    # error: [invalid-assignment] "Object of type `Literal[""]` is not assignable to `int`"
    z = ""

z: int

Nested intervening scope

A global statement causes lookup to skip any bindings in intervening scopes:

x: int = 1

def outer():
    x: str = ""

    def inner():
        global x
        reveal_type(x)  # revealed: int

Narrowing

An assignment following a global statement should narrow the type in the local scope after the assignment.

x: int | None

def f():
    global x
    x = 1
    reveal_type(x)  # revealed: Literal[1]

Same for an if statement:

x: int | None

def f():
    # The `global` keyword isn't necessary here, but this is testing that it doesn't get in the way
    # of narrowing.
    global x
    if x == 1:
        y: int = x  # allowed, because x cannot be None in this branch

`nonlocal` and `global`

A binding cannot be both nonlocal and global. This should emit a semantic syntax error. CPython marks the nonlocal line, while mypy, pyright, and ruff (PLE0115) mark the global line.

x = 1

def f():
    x = 1
    def g() -> None:
        nonlocal x
        global x  # error: [invalid-syntax] "name `x` is nonlocal and global"
        x = None

Global declaration after `global` statement

def f():
    global x
    y = x
    x = 1  # No error.

x = 2

Semantic syntax errors

Using a name prior to its global declaration in the same scope is a syntax error.

x = 1
y = 2

def f():
    print(x)
    global x  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    print(x)

def f():
    global x
    print(x)
    global x  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    print(x)

def f():
    print(x)
    global x, y  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    print(x)

def f():
    global x, y
    print(x)
    global x, y  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    print(x)

def f():
    x = 1
    global x  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    x = 1

def f():
    global x
    x = 1
    global x  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    x = 1

def f():
    del x
    global x, y  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    del x

def f():
    global x, y
    del x
    global x, y  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    del x

def f():
    del x
    global x  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    del x

def f():
    global x
    del x
    global x  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    del x

def f():
    del x
    global x, y  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    del x

def f():
    global x, y
    del x
    global x, y  # error: [invalid-syntax] "name `x` is used prior to global declaration"
    del x

def f():
    print(f"{x=}")
    global x  # error: [invalid-syntax] "name `x` is used prior to global declaration"

# still an error in module scope
x = None
global x  # error: [invalid-syntax] "name `x` is used prior to global declaration"

Local bindings override preceding `global` bindings

x = 42

def f():
    global x
    reveal_type(x)  # revealed: Literal[42]
    x = "56"
    reveal_type(x)  # revealed: Literal["56"]

Local assignment prevents falling back to the outer scope

x = 42

def f():
    # error: [unresolved-reference] "Name `x` used when not defined"
    reveal_type(x)  # revealed: Unknown
    x = "56"
    reveal_type(x)  # revealed: Literal["56"]

Annotating a `global` binding is a syntax error

x: int = 1

def f():
    global x
    x: str = "foo"  # error: [invalid-syntax] "annotated name `x` can't be global"

Global declarations affect the inferred type of the binding

Even if the global declaration isn't used in an assignment, we conservatively assume it could be:

x = 1

def f():
    global x

# TODO: reveal_type(x)  # revealed: Unknown | Literal["1"]

Global variables need an explicit definition in the global scope

You're allowed to use the global keyword to define new global variables that don't have any explicit definition in the global scope, but we consider that fishy and prefer to lint on it:

x = 1
y: int
# z is neither bound nor declared in the global scope

def f():
    global x, y, z  # error: [unresolved-global] "Invalid global declaration of `z`: `z` has no declarations or bindings in the global scope"

You don't need a definition for implicit globals, but you do for built-ins:

def f():
    global __file__  # allowed, implicit global
    global int  # error: [unresolved-global] "Invalid global declaration of `int`: `int` has no declarations or bindings in the global scope"

References to variables before they are defined within a class scope are considered global

If we try to access a variable in a class before it has been defined, the lookup will fall back to global.

import secrets

x: str = "a"

def f(x: int, y: int):
    class C:
        reveal_type(x)  # revealed: int

    class D:
        x = None
        reveal_type(x)  # revealed: None

    class E:
        reveal_type(x)  # revealed: str
        x = None

        # error: [unresolved-reference]
        reveal_type(y)  # revealed: Unknown
        y = None

    # Declarations count as definitions, even if there's no binding.
    class F:
        reveal_type(x)  # revealed: str
        x: int
        reveal_type(x)  # revealed: str

    # Explicitly `nonlocal` variables don't count, even if they're bound.
    class G:
        nonlocal x
        reveal_type(x)  # revealed: int
        x = 42
        reveal_type(x)  # revealed: Literal[42]

    # Possibly-unbound variables get unioned with the fallback lookup.
    class H:
        if secrets.randbelow(2):
            x = None
        reveal_type(x)  # revealed: None | str

6.6 KiB Raw Permalink Blame History

global references