ruff/crates
David Peter e91e2f49db
[red-knot] Trust module-level undeclared symbols in stubs (#17577)
## Summary

Many symbols in typeshed are defined without being declared. For
example:
```pyi
# builtins:
IOError = OSError

# types
LambdaType = FunctionType
NotImplementedType = _NotImplementedType

# typing
Text = str

# random
uniform = _inst.uniform

# optparse
make_option = Option

# all over the place:
_T = TypeVar("_T")
```

Here, we introduce a change that skips widening the public type of these
symbols (by unioning with `Unknown`).

fixes #17032

## Ecosystem analysis

This is difficult to analyze in detail, but I went over most changes and
it looks very favorable to me overall. The diff on the overall numbers
is:
```
errors: 1287 -> 859 (reduction by 428)
warnings: 45 -> 59 (increase by 14)
```

### Removed false positives

`invalid-base` examples:

```diff
- error[lint:invalid-base] /tmp/mypy_primer/projects/pip/src/pip/_vendor/rich/console.py:548:27: Invalid class base with type `Unknown | Literal[_local]` (all bases must be a class, `Any`, `Unknown` or `Todo`)
- error[lint:invalid-base] /tmp/mypy_primer/projects/tornado/tornado/iostream.py:84:25: Invalid class base with type `Unknown | Literal[OSError]` (all bases must be a class, `Any`, `Unknown` or `Todo`)
- error[lint:invalid-base] /tmp/mypy_primer/projects/mitmproxy/test/conftest.py:35:40: Invalid class base with type `Unknown | Literal[_UnixDefaultEventLoopPolicy]` (all bases must be a class, `Any`, `Unknown` or `Todo`)
```

`invalid-exception-caught` examples:

```diff
- error[lint:invalid-exception-caught] /tmp/mypy_primer/projects/cloud-init/cloudinit/cmd/status.py:334:16: Cannot catch object of type `Literal[ProcessExecutionError]` in an exception handler (must be a `BaseException` subclass or a tuple of `BaseException` subclasses)
- error[lint:invalid-exception-caught] /tmp/mypy_primer/projects/jinja/src/jinja2/loaders.py:537:16: Cannot catch object of type `Literal[TemplateNotFound]` in an exception handler (must be a `BaseException` subclass or a tuple of `BaseException` subclasses)
```

`unresolved-reference` examples


7a0265d36e/cloudinit/handlers/jinja_template.py (L120-L123)
(we now understand the `isinstance` narrowing)

```diff
- error[lint:unresolved-attribute] /tmp/mypy_primer/projects/cloud-init/cloudinit/handlers/jinja_template.py:123:16: Type `Exception` has no attribute `errno`
```

`unknown-argument` examples


https://github.com/hauntsaninja/boostedblob/blob/master/boostedblob/request.py#L53

```diff
- error[lint:unknown-argument] /tmp/mypy_primer/projects/boostedblob/boostedblob/request.py:53:17: Argument `connect` does not match any known parameter of bound method `__init__`
```

`unknown-argument`

There are a lot of `__init__`-related changes because we now understand
[`@attr.s`](3d42a6978a/src/attr/__init__.pyi (L387))
as a `@dataclass_transform` annotated symbol. For example:

```diff
- error[lint:unknown-argument] /tmp/mypy_primer/projects/attrs/tests/test_hooks.py:72:18: Argument `x` does not match any known parameter of bound method `__init__`
```

### New false positives

This can happen if a symbol that previously was inferred as `X |
Unknown` was assigned-to, but we don't yet understand the assignability
to `X`:


https://github.com/strawberry-graphql/strawberry/blob/main/strawberry/exceptions/handler.py#L90

```diff
+ error[lint:invalid-assignment] /tmp/mypy_primer/projects/strawberry/strawberry/exceptions/handler.py:90:9: Object of type `def strawberry_threading_exception_handler(args: tuple[type[BaseException], BaseException | None, TracebackType | None, Thread | None]) -> None` is not assignable to attribute `excepthook` of type `(_ExceptHookArgs, /) -> Any`
```

### New true positives


6bbb5519fe/tests/tracer/test_span.py (L714)

```diff
+ error[lint:invalid-argument-type] /tmp/mypy_primer/projects/dd-trace-py/tests/tracer/test_span.py:714:33: Argument to this function is incorrect: Expected `str`, found `Literal[b"\xf0\x9f\xa4\x94"]`
```

### Changed diagnostics

A lot of changed diagnostics because we now show `@Todo(Support for
`typing.TypeVar` instances in type expressions)` instead of `Unknown`
for all kinds of symbols that used a `_T = TypeVar("_T")` as a type. One
prominent example is the `list.__getitem__` method:

`builtins.pyi`:
```pyi
_T = TypeVar("_T")  # previously `TypeVar | Unknown`, now just `TypeVar`

# …

class list(MutableSequence[_T]):
    # …
    @overload
    def __getitem__(self, i: SupportsIndex, /) -> _T: ...
    # …
```

which causes this change in diagnostics:
```py
xs = [1, 2]
reveal_type(xs[0])  # previously `Unknown`, now `@Todo(Support for `typing.TypeVar` instances in type expressions)`
```

## Test Plan

Updated Markdown tests
2025-04-23 19:31:14 +02:00
..
red_knot red_knot_python_semantic: migrate types/diagnostic to new diagnostics 2025-04-22 12:08:03 -04:00
red_knot_ide [red-knot] Detect version-related syntax errors (#16379) 2025-04-17 14:00:30 -04:00
red_knot_project [red-knot] Early return from project.is_file_open for vendored files (#17580) 2025-04-23 15:32:41 +02:00
red_knot_python_semantic [red-knot] Trust module-level undeclared symbols in stubs (#17577) 2025-04-23 19:31:14 +02:00
red_knot_server Server: Use min instead of max to limit the number of threads (#17421) 2025-04-18 01:32:12 +05:30
red_knot_test [red-knot] Detect version-related syntax errors (#16379) 2025-04-17 14:00:30 -04:00
red_knot_vendored Sync vendored typeshed stubs (#17402) 2025-04-15 09:16:42 +02:00
red_knot_wasm Use concise message to show diagnostics in playground (#17357) 2025-04-11 22:44:24 +05:30
ruff [minor] Delete outdated TODO comment (#17565) 2025-04-22 20:23:08 +00:00
ruff_annotate_snippets ruff_annotate_snippets: address unused code warnings 2025-04-07 08:24:08 -04:00
ruff_benchmark [red-knot] make large-union benchmark slow again (#17418) 2025-04-16 14:05:42 +00:00
ruff_cache
ruff_db [red-knot] Detect semantic syntax errors (#17463) 2025-04-23 09:52:58 -04:00
ruff_dev Upgrade to Rust 1.86 and bump MSRV to 1.84 (#17171) 2025-04-03 15:59:44 +00:00
ruff_diagnostics
ruff_formatter Upgrade to Rust 1.86 and bump MSRV to 1.84 (#17171) 2025-04-03 15:59:44 +00:00
ruff_graph [red-knot] Detect version-related syntax errors (#16379) 2025-04-17 14:00:30 -04:00
ruff_index [red-knot] Don't use separate ID types for each alist (#16415) 2025-02-28 14:55:55 -05:00
ruff_linter [airflow] Apply auto fixes to cases where the names have changed in Airflow 3 (AIR301) (#17355) 2025-04-23 12:43:41 -04:00
ruff_macros Upgrade to Rust 1.86 and bump MSRV to 1.84 (#17171) 2025-04-03 15:59:44 +00:00
ruff_notebook Upgrade to Rust 1.86 and bump MSRV to 1.84 (#17171) 2025-04-03 15:59:44 +00:00
ruff_python_ast Remove redundant type_to_visitor_function entries (#17564) 2025-04-23 09:27:00 +02:00
ruff_python_ast_integration_tests Visit Identifier node as part of the SourceOrderVisitor (#17110) 2025-04-01 16:58:09 +02:00
ruff_python_codegen Upgrade to Rust 1.86 and bump MSRV to 1.84 (#17171) 2025-04-03 15:59:44 +00:00
ruff_python_formatter Don't add chaperone space after escaped quote in triple quote (#17216) 2025-04-11 10:21:47 +02:00
ruff_python_index
ruff_python_literal Preserve triple quotes and prefixes for strings (#15818) 2025-02-04 08:41:06 -05:00
ruff_python_parser [red-knot] Detect semantic syntax errors (#17463) 2025-04-23 09:52:58 -04:00
ruff_python_resolver Upgrade to Rust 1.86 and bump MSRV to 1.84 (#17171) 2025-04-03 15:59:44 +00:00
ruff_python_semantic Refactor semantic syntax error scope handling (#17314) 2025-04-09 14:23:29 -04:00
ruff_python_stdlib
ruff_python_trivia [red-knot] Ignore surrounding whitespace when looking for <!-- snapshot-diagnostics --> directives in mdtests (#16380) 2025-02-27 13:25:31 +00:00
ruff_python_trivia_integration_tests Pass ParserOptions to the parser (#16220) 2025-02-19 10:50:50 -05:00
ruff_server [red-knot] Don't use latency-sensitive for handlers (#17227) 2025-04-08 08:33:30 +02:00
ruff_source_file
ruff_text_size
ruff_wasm Bump 0.11.6 (#17449) 2025-04-17 09:20:29 -04:00
ruff_workspace Use python.typing.org for typing documentation links (#17323) 2025-04-09 20:38:20 +02:00