ruff/crates/ruff_python_ast/src
David Peter 0092794302
[ty] Use typing.Self for the first parameter of instance methods (#20517)
## Summary

Modify the (external) signature of instance methods such that the first
parameter uses `Self` unless it is explicitly annotated. This allows us
to correctly type-check more code, and allows us to infer correct return
types for many functions that return `Self`. For example:

```py
from pathlib import Path
from datetime import datetime, timedelta

reveal_type(Path(".config") / ".ty")  # now Path, previously Unknown

def _(dt: datetime, delta: timedelta):
    reveal_type(dt - delta)  # now datetime, previously Unknown
```

part of https://github.com/astral-sh/ty/issues/159

## Performance

I ran benchmarks locally on `attrs`, `freqtrade` and `colour`, the
projects with the largest regressions on CodSpeed. I see much smaller
effects locally, but can definitely reproduce the regression on `attrs`.
From looking at the profiling results (on Codspeed), it seems that we
simply do more type inference work, which seems plausible, given that we
now understand much more return types (of many stdlib functions). In
particular, whenever a function uses an implicit `self` and returns
`Self` (without mentioning `Self` anywhere else in its signature), we
will now infer the correct type, whereas we would previously return
`Unknown`. This also means that we need to invoke the generics solver in
more cases. Comparing half a million lines of log output on attrs, I can
see that we do 5% more "work" (number of lines in the log), and have a
lot more `apply_specialization` events (7108 vs 4304). On freqtrade, I
see similar numbers for `apply_specialization` (11360 vs 5138 calls).
Given these results, I'm not sure if it's generally worth doing more
performance work, especially since none of the code modifications
themselves seem to be likely candidates for regressions.

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./ty_main check /home/shark/ecosystem/attrs` | 92.6 ± 3.6 | 85.9 |
102.6 | 1.00 |
| `./ty_self check /home/shark/ecosystem/attrs` | 101.7 ± 3.5 | 96.9 |
113.8 | 1.10 ± 0.06 |

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./ty_main check /home/shark/ecosystem/freqtrade` | 599.0 ± 20.2 |
568.2 | 627.5 | 1.00 |
| `./ty_self check /home/shark/ecosystem/freqtrade` | 607.9 ± 11.5 |
594.9 | 626.4 | 1.01 ± 0.04 |

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `./ty_main check /home/shark/ecosystem/colour` | 423.9 ± 17.9 | 394.6
| 447.4 | 1.00 |
| `./ty_self check /home/shark/ecosystem/colour` | 426.9 ± 24.9 | 373.8
| 456.6 | 1.01 ± 0.07 |

## Test Plan

New Markdown tests

## Ecosystem report

* apprise: ~300 new diagnostics related to problematic stubs in apprise
😩
* attrs: a new true positive, since [this
function](4e2c89c823/tests/test_make.py (L2135))
is missing a `@staticmethod`?
* Some legitimate true positives
* sympy: lots of new `invalid-operator` false positives in [matrix
multiplication](cf9f4b6805/sympy/matrices/matrixbase.py (L3267-L3269))
due to our limited understanding of [generic `Callable[[Callable[[T1,
T2], T3]], Callable[[T1, T2], T3]]` "identity"
types](cf9f4b6805/sympy/core/decorators.py (L83-L84))
of decorators. This is not related to type-of-self.

## Typing conformance results

The changes are all correct, except for
```diff
+generics_self_usage.py:50:5: error[invalid-assignment] Object of type `def foo(self) -> int` is not assignable to `(typing.Self, /) -> int`
```
which is related to an assignability problem involving type variables on
both sides:
```py
class CallableAttribute:
    def foo(self) -> int:
        return 0

    bar: Callable[[Self], int] = foo  # <- we currently error on this assignment
```

---------

Co-authored-by: Shaygan Hooshyari <sh.hooshyari@gmail.com>
2025-09-29 21:08:08 +02:00
..
visitor Disallow implicit concatenation of t-strings and other string types (#19485) 2025-07-27 12:41:03 +00:00
comparable.rs Update Rust toolchain to 1.89 (#19807) 2025-08-07 18:21:50 +02:00
docstrings.rs Move Python whitespace utilities into new ruff_python_whitespace crate (#4993) 2023-06-10 00:59:57 +00:00
expression.rs Disallow implicit concatenation of t-strings and other string types (#19485) 2025-07-27 12:41:03 +00:00
generated.rs [ty] Add environment variable to dump Salsa memory usage stats (#18928) 2025-06-26 21:27:51 +00:00
helpers.rs [flake8-bandit] Fix truthiness: dict-only ** displays not truthy for shell (S602, S604, S609) (#20177) 2025-09-10 17:06:33 -04:00
identifier.rs Switch to Rust 2024 edition (#18129) 2025-05-16 13:25:28 +02:00
int.rs [ty] Add environment variable to dump Salsa memory usage stats (#18928) 2025-06-26 21:27:51 +00:00
lib.rs include .pyw files by default when linting and formatting (#20458) 2025-09-24 08:39:30 -07:00
name.rs [ty] Reduce size of member table (#19572) 2025-08-07 11:16:04 +02:00
node.rs Disallow implicit concatenation of t-strings and other string types (#19485) 2025-07-27 12:41:03 +00:00
node_index.rs [ty] Shrink size of AstNodeRef (#20028) 2025-08-22 17:03:22 -04:00
nodes.rs [ty] Use typing.Self for the first parameter of instance methods (#20517) 2025-09-29 21:08:08 +02:00
operator_precedence.rs Implement template strings (#17851) 2025-05-30 15:00:56 -05:00
parenthesize.rs Remove customizable reference enum names (#15647) 2025-01-21 13:46:31 -05:00
python_version.rs [ty] Add environment variable to dump Salsa memory usage stats (#18928) 2025-06-26 21:27:51 +00:00
relocate.rs [ty] AST garbage collection (#18482) 2025-06-13 08:40:11 -04:00
script.rs bump MSRV to 1.83 (#16294) 2025-02-26 06:12:43 -08:00
statement_visitor.rs Remove Stmt::TryStar (#6566) 2023-08-14 13:39:44 -04:00
stmt_if.rs Update Rust toolchain to 1.89 (#19807) 2025-08-07 18:21:50 +02:00
str.rs Track t-strings and f-strings for token-based rules and suppression comments (#20357) 2025-09-12 13:00:12 -05:00
str_prefix.rs Implement template strings (#17851) 2025-05-30 15:00:56 -05:00
traversal.rs Use referencial equality in traversal helper methods (#13895) 2024-10-24 11:30:22 +02:00
types.rs Remove RefEquality (#6393) 2023-08-07 16:04:50 +00:00
visitor.rs Disallow implicit concatenation of t-strings and other string types (#19485) 2025-07-27 12:41:03 +00:00
whitespace.rs Switch to Rust 2024 edition (#18129) 2025-05-16 13:25:28 +02:00