[ty] Async for loops and async iterables (#19634)

## Summary

Add support for `async for` loops and async iterables.

part of https://github.com/astral-sh/ty/issues/151

## Ecosystem impact

```diff
- boostedblob/listing.py:445:54: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
```

This is correct. We now find a true positive in the `# type: ignore`'d
code.

All of the other ecosystem hits are of the type

```diff
trio (https://github.com/python-trio/trio)
+ src/trio/_core/_tests/test_guest_mode.py:532:24: error[not-iterable] Object of type `MemorySendChannel[int] | MemoryReceiveChannel[int]` may not be iterable
```

The message is correct, because only `MemoryReceiveChannel` has an
`__aiter__` method, but `MemorySendChannel` does not. What's not correct
is our inferred type here. It should be `MemoryReceiveChannel[int]`, not
the union of the two. This is due to missing unpacking support for tuple
subclasses, which @AlexWaygood is working on. I don't think this should
block merging this PR, because those wrong types are already there,
without this PR.

## Test Plan

New Markdown tests and snapshot tests for diagnostics.
This commit is contained in:
David Peter 2025-07-30 17:40:24 +02:00 committed by GitHub
parent e593761232
commit eb02aa5676
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
17 changed files with 908 additions and 197 deletions

View file

@ -128,7 +128,7 @@ class AsyncIterable:
return AsyncIterator()
async def _():
# revealed: @Todo(async iterables/iterators)
# revealed: int
[reveal_type(x) async for x in AsyncIterable()]
```
@ -147,6 +147,7 @@ class Iterable:
return Iterator()
async def _():
# revealed: @Todo(async iterables/iterators)
# error: [not-iterable] "Object of type `Iterable` is not async-iterable"
# revealed: Unknown
[reveal_type(x) async for x in Iterable()]
```

View file

@ -27,6 +27,7 @@ If all of the comprehensions are `async`, on the other hand, the code was still
```py
async def test():
# error: [not-iterable] "Object of type `range` is not async-iterable"
return [[x async for x in elements(n)] async for n in range(3)]
```

View file

@ -2,27 +2,6 @@
Async `for` loops do not work according to the synchronous iteration protocol.
## Invalid async for loop
```py
async def foo():
class Iterator:
def __next__(self) -> int:
return 42
class Iterable:
def __iter__(self) -> Iterator:
return Iterator()
async for x in Iterator():
pass
# TODO: should reveal `Unknown` because `__aiter__` is not defined
# revealed: @Todo(async iterables/iterators)
# error: [possibly-unresolved-reference]
reveal_type(x)
```
## Basic async for loop
```py
@ -35,11 +14,154 @@ async def foo():
def __aiter__(self) -> IntAsyncIterator:
return IntAsyncIterator()
# TODO(Alex): async iterables/iterators!
async for x in IntAsyncIterable():
pass
# error: [possibly-unresolved-reference]
# revealed: @Todo(async iterables/iterators)
reveal_type(x)
reveal_type(x) # revealed: int
```
## Async for loop with unpacking
```py
async def foo():
class AsyncIterator:
async def __anext__(self) -> tuple[int, str]:
return 42, "hello"
class AsyncIterable:
def __aiter__(self) -> AsyncIterator:
return AsyncIterator()
async for x, y in AsyncIterable():
reveal_type(x) # revealed: int
reveal_type(y) # revealed: str
```
## Error cases
<!-- snapshot-diagnostics -->
### No `__aiter__` method
```py
from typing_extensions import reveal_type
class NotAsyncIterable: ...
async def foo():
# error: [not-iterable] "Object of type `NotAsyncIterable` is not async-iterable"
async for x in NotAsyncIterable():
reveal_type(x) # revealed: Unknown
```
### Synchronously iterable, but not asynchronously iterable
```py
from typing_extensions import reveal_type
async def foo():
class Iterator:
def __next__(self) -> int:
return 42
class Iterable:
def __iter__(self) -> Iterator:
return Iterator()
# error: [not-iterable] "Object of type `Iterator` is not async-iterable"
async for x in Iterator():
reveal_type(x) # revealed: Unknown
```
### No `__anext__` method
```py
from typing_extensions import reveal_type
class NoAnext: ...
class AsyncIterable:
def __aiter__(self) -> NoAnext:
return NoAnext()
async def foo():
# error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
async for x in AsyncIterable():
reveal_type(x) # revealed: Unknown
```
### Possibly unbound `__anext__` method
```py
from typing_extensions import reveal_type
async def foo(flag: bool):
class PossiblyUnboundAnext:
if flag:
async def __anext__(self) -> int:
return 42
class AsyncIterable:
def __aiter__(self) -> PossiblyUnboundAnext:
return PossiblyUnboundAnext()
# error: [not-iterable] "Object of type `AsyncIterable` may not be async-iterable"
async for x in AsyncIterable():
reveal_type(x) # revealed: int
```
### Possibly unbound `__aiter__` method
```py
from typing_extensions import reveal_type
async def foo(flag: bool):
class AsyncIterable:
async def __anext__(self) -> int:
return 42
class PossiblyUnboundAiter:
if flag:
def __aiter__(self) -> AsyncIterable:
return AsyncIterable()
# error: "Object of type `PossiblyUnboundAiter` may not be async-iterable"
async for x in PossiblyUnboundAiter():
reveal_type(x) # revealed: int
```
### Wrong signature for `__aiter__`
```py
from typing_extensions import reveal_type
class AsyncIterator:
async def __anext__(self) -> int:
return 42
class AsyncIterable:
def __aiter__(self, arg: int) -> AsyncIterator: # wrong
return AsyncIterator()
async def foo():
# error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
async for x in AsyncIterable():
reveal_type(x) # revealed: int
```
### Wrong signature for `__anext__`
```py
from typing_extensions import reveal_type
class AsyncIterator:
async def __anext__(self, arg: int) -> int: # wrong
return 42
class AsyncIterable:
def __aiter__(self) -> AsyncIterator:
return AsyncIterator()
async def foo():
# error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
async for x in AsyncIterable():
reveal_type(x) # revealed: int
```

View file

@ -0,0 +1,52 @@
---
source: crates/ty_test/src/lib.rs
expression: snapshot
---
---
mdtest name: async_for.md - Async - Error cases - No `__aiter__` method
mdtest path: crates/ty_python_semantic/resources/mdtest/loops/async_for.md
---
# Python source files
## mdtest_snippet.py
```
1 | from typing_extensions import reveal_type
2 |
3 | class NotAsyncIterable: ...
4 |
5 | async def foo():
6 | # error: [not-iterable] "Object of type `NotAsyncIterable` is not async-iterable"
7 | async for x in NotAsyncIterable():
8 | reveal_type(x) # revealed: Unknown
```
# Diagnostics
```
error[not-iterable]: Object of type `NotAsyncIterable` is not async-iterable
--> src/mdtest_snippet.py:7:20
|
5 | async def foo():
6 | # error: [not-iterable] "Object of type `NotAsyncIterable` is not async-iterable"
7 | async for x in NotAsyncIterable():
| ^^^^^^^^^^^^^^^^^^
8 | reveal_type(x) # revealed: Unknown
|
info: It has no `__aiter__` method
info: rule `not-iterable` is enabled by default
```
```
info[revealed-type]: Revealed type
--> src/mdtest_snippet.py:8:21
|
6 | # error: [not-iterable] "Object of type `NotAsyncIterable` is not async-iterable"
7 | async for x in NotAsyncIterable():
8 | reveal_type(x) # revealed: Unknown
| ^ `Unknown`
|
```

View file

@ -0,0 +1,56 @@
---
source: crates/ty_test/src/lib.rs
expression: snapshot
---
---
mdtest name: async_for.md - Async - Error cases - No `__anext__` method
mdtest path: crates/ty_python_semantic/resources/mdtest/loops/async_for.md
---
# Python source files
## mdtest_snippet.py
```
1 | from typing_extensions import reveal_type
2 |
3 | class NoAnext: ...
4 |
5 | class AsyncIterable:
6 | def __aiter__(self) -> NoAnext:
7 | return NoAnext()
8 |
9 | async def foo():
10 | # error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
11 | async for x in AsyncIterable():
12 | reveal_type(x) # revealed: Unknown
```
# Diagnostics
```
error[not-iterable]: Object of type `AsyncIterable` is not async-iterable
--> src/mdtest_snippet.py:11:20
|
9 | async def foo():
10 | # error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
11 | async for x in AsyncIterable():
| ^^^^^^^^^^^^^^^
12 | reveal_type(x) # revealed: Unknown
|
info: Its `__aiter__` method returns an object of type `NoAnext`, which has no `__anext__` method
info: rule `not-iterable` is enabled by default
```
```
info[revealed-type]: Revealed type
--> src/mdtest_snippet.py:12:21
|
10 | # error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
11 | async for x in AsyncIterable():
12 | reveal_type(x) # revealed: Unknown
| ^ `Unknown`
|
```

View file

@ -0,0 +1,58 @@
---
source: crates/ty_test/src/lib.rs
expression: snapshot
---
---
mdtest name: async_for.md - Async - Error cases - Possibly unbound `__aiter__` method
mdtest path: crates/ty_python_semantic/resources/mdtest/loops/async_for.md
---
# Python source files
## mdtest_snippet.py
```
1 | from typing_extensions import reveal_type
2 |
3 | async def foo(flag: bool):
4 | class AsyncIterable:
5 | async def __anext__(self) -> int:
6 | return 42
7 |
8 | class PossiblyUnboundAiter:
9 | if flag:
10 | def __aiter__(self) -> AsyncIterable:
11 | return AsyncIterable()
12 |
13 | # error: "Object of type `PossiblyUnboundAiter` may not be async-iterable"
14 | async for x in PossiblyUnboundAiter():
15 | reveal_type(x) # revealed: int
```
# Diagnostics
```
error[not-iterable]: Object of type `PossiblyUnboundAiter` may not be async-iterable
--> src/mdtest_snippet.py:14:20
|
13 | # error: "Object of type `PossiblyUnboundAiter` may not be async-iterable"
14 | async for x in PossiblyUnboundAiter():
| ^^^^^^^^^^^^^^^^^^^^^^
15 | reveal_type(x) # revealed: int
|
info: Its `__aiter__` attribute (with type `bound method PossiblyUnboundAiter.__aiter__() -> AsyncIterable`) may not be callable
info: rule `not-iterable` is enabled by default
```
```
info[revealed-type]: Revealed type
--> src/mdtest_snippet.py:15:21
|
13 | # error: "Object of type `PossiblyUnboundAiter` may not be async-iterable"
14 | async for x in PossiblyUnboundAiter():
15 | reveal_type(x) # revealed: int
| ^ `int`
|
```

View file

@ -0,0 +1,58 @@
---
source: crates/ty_test/src/lib.rs
expression: snapshot
---
---
mdtest name: async_for.md - Async - Error cases - Possibly unbound `__anext__` method
mdtest path: crates/ty_python_semantic/resources/mdtest/loops/async_for.md
---
# Python source files
## mdtest_snippet.py
```
1 | from typing_extensions import reveal_type
2 |
3 | async def foo(flag: bool):
4 | class PossiblyUnboundAnext:
5 | if flag:
6 | async def __anext__(self) -> int:
7 | return 42
8 |
9 | class AsyncIterable:
10 | def __aiter__(self) -> PossiblyUnboundAnext:
11 | return PossiblyUnboundAnext()
12 |
13 | # error: [not-iterable] "Object of type `AsyncIterable` may not be async-iterable"
14 | async for x in AsyncIterable():
15 | reveal_type(x) # revealed: int
```
# Diagnostics
```
error[not-iterable]: Object of type `AsyncIterable` may not be async-iterable
--> src/mdtest_snippet.py:14:20
|
13 | # error: [not-iterable] "Object of type `AsyncIterable` may not be async-iterable"
14 | async for x in AsyncIterable():
| ^^^^^^^^^^^^^^^
15 | reveal_type(x) # revealed: int
|
info: Its `__aiter__` method returns an object of type `PossiblyUnboundAnext`, which may not have a `__anext__` method
info: rule `not-iterable` is enabled by default
```
```
info[revealed-type]: Revealed type
--> src/mdtest_snippet.py:15:21
|
13 | # error: [not-iterable] "Object of type `AsyncIterable` may not be async-iterable"
14 | async for x in AsyncIterable():
15 | reveal_type(x) # revealed: int
| ^ `int`
|
```

View file

@ -0,0 +1,57 @@
---
source: crates/ty_test/src/lib.rs
expression: snapshot
---
---
mdtest name: async_for.md - Async - Error cases - Synchronously iterable, but not asynchronously iterable
mdtest path: crates/ty_python_semantic/resources/mdtest/loops/async_for.md
---
# Python source files
## mdtest_snippet.py
```
1 | from typing_extensions import reveal_type
2 |
3 | async def foo():
4 | class Iterator:
5 | def __next__(self) -> int:
6 | return 42
7 |
8 | class Iterable:
9 | def __iter__(self) -> Iterator:
10 | return Iterator()
11 |
12 | # error: [not-iterable] "Object of type `Iterator` is not async-iterable"
13 | async for x in Iterator():
14 | reveal_type(x) # revealed: Unknown
```
# Diagnostics
```
error[not-iterable]: Object of type `Iterator` is not async-iterable
--> src/mdtest_snippet.py:13:20
|
12 | # error: [not-iterable] "Object of type `Iterator` is not async-iterable"
13 | async for x in Iterator():
| ^^^^^^^^^^
14 | reveal_type(x) # revealed: Unknown
|
info: It has no `__aiter__` method
info: rule `not-iterable` is enabled by default
```
```
info[revealed-type]: Revealed type
--> src/mdtest_snippet.py:14:21
|
12 | # error: [not-iterable] "Object of type `Iterator` is not async-iterable"
13 | async for x in Iterator():
14 | reveal_type(x) # revealed: Unknown
| ^ `Unknown`
|
```

View file

@ -0,0 +1,59 @@
---
source: crates/ty_test/src/lib.rs
expression: snapshot
---
---
mdtest name: async_for.md - Async - Error cases - Wrong signature for `__anext__`
mdtest path: crates/ty_python_semantic/resources/mdtest/loops/async_for.md
---
# Python source files
## mdtest_snippet.py
```
1 | from typing_extensions import reveal_type
2 |
3 | class AsyncIterator:
4 | async def __anext__(self, arg: int) -> int: # wrong
5 | return 42
6 |
7 | class AsyncIterable:
8 | def __aiter__(self) -> AsyncIterator:
9 | return AsyncIterator()
10 |
11 | async def foo():
12 | # error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
13 | async for x in AsyncIterable():
14 | reveal_type(x) # revealed: int
```
# Diagnostics
```
error[not-iterable]: Object of type `AsyncIterable` is not async-iterable
--> src/mdtest_snippet.py:13:20
|
11 | async def foo():
12 | # error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
13 | async for x in AsyncIterable():
| ^^^^^^^^^^^^^^^
14 | reveal_type(x) # revealed: int
|
info: Its `__aiter__` method returns an object of type `AsyncIterator`, which has an invalid `__anext__` method
info: Expected signature for `__anext__` is `def __anext__(self): ...`
info: rule `not-iterable` is enabled by default
```
```
info[revealed-type]: Revealed type
--> src/mdtest_snippet.py:14:21
|
12 | # error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
13 | async for x in AsyncIterable():
14 | reveal_type(x) # revealed: int
| ^ `int`
|
```

View file

@ -0,0 +1,59 @@
---
source: crates/ty_test/src/lib.rs
expression: snapshot
---
---
mdtest name: async_for.md - Async - Error cases - Wrong signature for `__aiter__`
mdtest path: crates/ty_python_semantic/resources/mdtest/loops/async_for.md
---
# Python source files
## mdtest_snippet.py
```
1 | from typing_extensions import reveal_type
2 |
3 | class AsyncIterator:
4 | async def __anext__(self) -> int:
5 | return 42
6 |
7 | class AsyncIterable:
8 | def __aiter__(self, arg: int) -> AsyncIterator: # wrong
9 | return AsyncIterator()
10 |
11 | async def foo():
12 | # error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
13 | async for x in AsyncIterable():
14 | reveal_type(x) # revealed: int
```
# Diagnostics
```
error[not-iterable]: Object of type `AsyncIterable` is not async-iterable
--> src/mdtest_snippet.py:13:20
|
11 | async def foo():
12 | # error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
13 | async for x in AsyncIterable():
| ^^^^^^^^^^^^^^^
14 | reveal_type(x) # revealed: int
|
info: Its `__aiter__` method has an invalid signature
info: Expected signature `def __aiter__(self): ...`
info: rule `not-iterable` is enabled by default
```
```
info[revealed-type]: Revealed type
--> src/mdtest_snippet.py:14:21
|
12 | # error: [not-iterable] "Object of type `AsyncIterable` is not async-iterable"
13 | async for x in AsyncIterable():
14 | reveal_type(x) # revealed: int
| ^ `int`
|
```

View file

@ -60,7 +60,7 @@ error[not-iterable]: Object of type `Iterable1` may not be iterable
29 | reveal_type(x) # revealed: int | str
|
info: Its `__iter__` method returns an object of type `Iterator1`, which may have an invalid `__next__` method
info: Expected signature for `__next__` is `def __next__(self): ...`)
info: Expected signature for `__next__` is `def __next__(self): ...`
info: rule `not-iterable` is enabled by default
```

View file

@ -19,14 +19,15 @@ mdtest path: crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syn
5 | # error: 19 [invalid-syntax] "cannot use an asynchronous comprehension inside of a synchronous comprehension on Python 3.10 (syntax was added in 3.11)"
6 | return {n: [x async for x in elements(n)] for n in range(3)}
7 | async def test():
8 | return [[x async for x in elements(n)] async for n in range(3)]
9 | async def f():
10 | [x for x in [1]] and [x async for x in elements(1)]
11 |
12 | async def f():
13 | def g():
14 | pass
15 | [x async for x in elements(1)]
8 | # error: [not-iterable] "Object of type `range` is not async-iterable"
9 | return [[x async for x in elements(n)] async for n in range(3)]
10 | async def f():
11 | [x for x in [1]] and [x async for x in elements(1)]
12 |
13 | async def f():
14 | def g():
15 | pass
16 | [x async for x in elements(1)]
```
# Diagnostics
@ -40,7 +41,23 @@ error[invalid-syntax]
6 | return {n: [x async for x in elements(n)] for n in range(3)}
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot use an asynchronous comprehension inside of a synchronous comprehension on Python 3.10 (syntax was added in 3.11)
7 | async def test():
8 | return [[x async for x in elements(n)] async for n in range(3)]
8 | # error: [not-iterable] "Object of type `range` is not async-iterable"
|
```
```
error[not-iterable]: Object of type `range` is not async-iterable
--> src/mdtest_snippet.py:9:59
|
7 | async def test():
8 | # error: [not-iterable] "Object of type `range` is not async-iterable"
9 | return [[x async for x in elements(n)] async for n in range(3)]
| ^^^^^^^^
10 | async def f():
11 | [x for x in [1]] and [x async for x in elements(1)]
|
info: It has no `__aiter__` method
info: rule `not-iterable` is enabled by default
```