Async iterators, generators, and comprehensions

Stream values incrementally with async for, async generators, and async comprehensions

An async generator (PEP 525, Python 3.6+) produces values lazily while keeping the event loop responsive between items. Defined with `async def` and `yield`, it implements `__aiter__` and `__anext__` for use with `async for`. Async comprehensions (PEP 530, Python 3.6+) use `async for` inside list and set displays. Internally, CPython suspends the generator at each `yield` and resumes on the next `__anext__` call, allowing other tasks to run between yields. This is cooperative, not parallel: values arrive one at a time, but the loop does not block the event loop while waiting for each one. <a href="/async-foundations-awaitables">Learn the awaitable model that drives async generators</a>. <a href="/memory-iterables">Compare async generators with regular generators and iterables</a>.

Understand.
Visualize.
Master.

Python in Depth

An interactive engineering reference for Python internals

Quick note

Streaming shifts memory cost to coordination cost.

:)
Python version

Targets Python 3.10–3.14. Python 3.9 and below are End-of-Life.

TABLE OF CONTENTS
6.5Async iterators, generators, and comprehensions

Stream values incrementally with async for, async generators, and async comprehensions

Async iteration is for values that arrive over awaitable boundaries one at a time. It is streaming concurrency, not list construction with a different keyword.

Core answer

Use async generators when producing each next item may await I/O and consumers should process incrementally.

# [CURRENT - 3.10-3.14] Works on Python 3.10+ [PEP 525]
import asyncio
from dataclasses import dataclass
from collections.abc import AsyncIterator
@dataclass(frozen=True, slots=True)
class Page:
number: int
rows: tuple[str, ...]
async def pages() -> AsyncIterator[Page]:
for number in range(3):
await asyncio.sleep(0.01)
yield Page(number, (f"row-{number}",))
async def main() -> None:
async for page in pages():
print(page)
asyncio.run(main())
See Async Iteration Pull Values

Watch an async consumer pull values one by one from an async iterable while the producer suspends between items.

Why this design exists

PEP 525 brought generator ergonomics to asynchronous production. PEP 530 extended comprehensions where async iteration genuinely feeds a materialized result.

Mechanics and CPython internals

Async iterables expose __aiter__; async iterators expose awaitable __anext__. Async generator frames suspend at yield and await boundaries, preserving local state between pulls. Each async for asks for one next item and handles StopAsyncIteration when the stream ends.

# [CURRENT - 3.10-3.14] Works on Python 3.10+
import asyncio
from dataclasses import dataclass
from collections.abc import AsyncIterator
@dataclass(frozen=True, slots=True)
class Event:
sequence: int
class EventFeed:
def __init__(self, stop: int) -> None:
self.current = 0
self.stop = stop
def __aiter__(self) -> AsyncIterator[Event]:
return self
async def __anext__(self) -> Event:
if self.current >= self.stop:
raise StopAsyncIteration
await asyncio.sleep(0)
self.current += 1
return Event(self.current)
async def main() -> None:
print([event async for event in EventFeed(3)])
asyncio.run(main())

Complexity and tradeoffs

Streaming can keep retained results small, but materializing an async comprehension still consumes O(n) output space. Per-item awaits improve responsiveness and backpressure; they can also add scheduler overhead if the work is tiny and already in memory.

Idiomatic patterns and refactoring

Refactor "fetch all then process" into async iteration when downstream can consume pages incrementally.

# [CURRENT - 3.10-3.14] Works on Python 3.10+
import asyncio
from dataclasses import dataclass
from collections.abc import AsyncIterator
@dataclass(frozen=True, slots=True)
class Batch:
number: int
async def collect_all() -> list[Batch]:
return [Batch(number) for number in range(3)]
async def stream() -> AsyncIterator[Batch]:
for number in range(3):
await asyncio.sleep(0)
yield Batch(number)
async def main() -> None:
print(await collect_all())
print([batch async for batch in stream()])
asyncio.run(main())

Common mistakes and edge cases

Do not expect an async generator to run until it is iterated. Do not hide fan-out inside a generator without bounding it. Close generators or use scoped consumers when cleanup matters.

When to use / When NOT to use

Use async iterators for paginated APIs, streaming reads, and event feeds with awaitable next-item work.

Do not use them for an in-memory list just to make code look asynchronous.

Further reading

  • Official docs: asynchronous iterators
  • Official docs: asynchronous generator functions
  • PEP 525: asynchronous generators
  • PEP 530: asynchronous comprehensions
  • CPython source: generator objects
BOARD NOTESContext
WHY NO BENCHMARK?

This topic is better taught with structure, semantics, and cross-references than with a synthetic chart.

Streaming shifts memory cost to coordination cost.

RELATED GUIDES
NEXT CHECKS
Contribute