Async iterators, generators, and comprehensions

Stream values incrementally with async for, async generators, and async comprehensions

An async generator (PEP 525, Python 3.6+) produces values lazily while keeping the event loop responsive between items. Defined with `async def` and `yield`, it implements `__aiter__` and `__anext__` for use with `async for`. Async comprehensions (PEP 530, Python 3.6+) use `async for` inside list and set displays. Internally, CPython suspends the generator at each `yield` and resumes on the next `__anext__` call, allowing other tasks to run between yields. This is cooperative, not parallel: values arrive one at a time, but the loop does not block the event loop while waiting for each one. <a href="/async-foundations-awaitables">Learn the awaitable model that drives async generators</a>. <a href="/memory-iterables">Compare async generators with regular generators and iterables</a>.

Understand.
Visualize.
Master.

Python in Depth

An interactive engineering reference for Python internals

Quick note

Streaming shifts memory cost to coordination cost.

:)
TABLE OF CONTENTS
6.5Async iterators, generators, and comprehensions

Stream values incrementally with async for, async generators, and async comprehensions

You know async for exists, and maybe you have used it once or twice. The real question is what happens under the hood: each iteration may suspend the event loop, yield control to other tasks, and resume when the next value arrives.

Think of an async iterator like a newspaper delivery route. On a regular route (sync iterator), every house has the paper waiting — you drop and go. On an async route, some houses require buzzing the intercom and waiting for someone to answer. Each stop may pause your progress. async for is the route manager that handles that waiting automatically.

Core answer

Async iteration is for streams whose next value may require waiting.

  • async for consumes an async iterable
  • async generator functions use async def plus yield
  • async comprehensions and async generator expressions make async data pipelines concise

The key point is still cooperative suspension: each next item may require await.

# [OLDER / 3.6-3.9, CURRENT - 3.10-3.14] Requires Python 3.6+ [PEP 525]
import asyncio
async def ticker():
for value in (1, 2, 3):
await asyncio.sleep(0.1)
yield value
See Async Iteration Pull Values

Watch an async consumer pull values one by one from an async iterable while the producer suspends between items.

Asynchronous iteration and async iterables

An async iterable is an object that supports asynchronous iteration. The consumer uses async for, and each step may suspend.

# [OLDER / 3.6-3.9, CURRENT - 3.10-3.14] Requires Python 3.6+ [PEP 492]
import asyncio
async def ticker():
for value in (1, 2, 3):
await asyncio.sleep(0.1)
yield value
async def main():
async for item in ticker():
print(item)
asyncio.run(main())

The important semantic difference from ordinary iteration is that "give me the next item" is not assumed to be immediate. It may involve:

  • waiting on network input
  • waiting on a timer
  • waiting on upstream producer readiness
Async generator functions

An async generator function mixes:

  • async def
  • yield

That makes it the async counterpart of a regular generator function.

# [OLDER / 3.6-3.9, CURRENT - 3.10-3.14] Requires Python 3.6+ [PEP 525]
import asyncio
async def ticker():
for value in (1, 2, 3):
await asyncio.sleep(0.1)
yield value

This is not just syntax sugar. The function expresses a stream whose production steps may suspend between items.

That is why async generators are natural for:

  • paginated network streams
  • incremental protocol frames
  • event feeds
  • backpressure-aware producers
Async comprehensions and generator expressions

Async comprehensions and async generator expressions let you write compact transformations over async sources.

# [OLDER / 3.6-3.9, CURRENT - 3.10-3.14] Requires Python 3.6+ [PEP 530]
import asyncio
async def ticker():
for value in (1, 2, 3):
await asyncio.sleep(0.1)
yield value
async def collect_even():
return [value async for value in ticker() if value % 2 == 0]
print(asyncio.run(collect_even()))

Like ordinary comprehensions, they are expressive. Unlike ordinary comprehensions, the source side can suspend between values.

This makes them good for:

  • small async filtering steps
  • local async transformations
  • concise stream adaptation near the consumer

They are not automatically the best choice for deeply stateful pipelines or complicated error handling. At that point, an explicit loop often reads better.

Mechanism and protocol view

The relevant language constructs are:

  • async for
  • async def with yield
  • async comprehensions

Under the hood, Python is working with asynchronous iteration protocols instead of ordinary __iter__ / __next__.

That distinction matters because the chapter is not "iteration, but with new keywords." The chapter is about protocols that allow waiting between iteration steps.

This also connects to the synchronous iteration guide:

  • regular iterables:
  • async iterables: this guide

The conceptual bridge is useful: both are pull-driven from the consumer side, but only one side can suspend between pulls.

Version context

Current project guidance targets Python 3.10-3.14. Python 3.9 and below are End-of-Life.

Version-sensitive points:

  • async/await syntax: Python 3.5+ PEP 492
  • async generators: Python 3.6+ PEP 525
  • async comprehensions and async generator expressions: Python 3.6+ PEP 530

Because these features are older than the supported project range, they are stable baseline material for this repo.

Edge cases and gotchas

async for does not imply parallel consumption. It means the loop may await each next item.

Async generators are still one stream of values, not an implicit task farm.

Async comprehensions are concise, but they still execute under the same event-loop and cancellation rules as any other async code.

If the source blocks the event-loop thread internally, wrapping it in an async iterator does not rescue it. Async iteration is only as cooperative as the producer behind it.

Production usage

Use async generators when:

  • values arrive over time
  • waiting between items is normal
  • the consumer should process incrementally rather than after full materialization

Use async comprehensions when:

  • the transformation is local and clear
  • you want to keep stream handling compact

Use explicit loops when:

  • cancellation, retries, and cleanup need to be obvious
  • error handling is nontrivial
  • pipeline state is more important than concision

For the broader async scheduling model beneath these constructs, see .

Further depth
  • Language reference: async for statement
  • Language reference: yield expressions
  • PEP 492: Coroutines with async and await syntax
  • PEP 525: Asynchronous Generators
  • PEP 530: Asynchronous Comprehensions
BOARD NOTESContext
WHY NO BENCHMARK?

This topic is better taught with structure, semantics, and cross-references than with a synthetic chart.

Streaming shifts memory cost to coordination cost.

RELATED GUIDES
NEXT CHECKS
Contribute