Container vs flat iterables

Understand references, raw values, arrays, memoryview, and streaming

A list holds everything in memory. A generator produces each value on demand. Python distinguishes three categories: container sequences like `list` and `tuple` that store references to objects, flat sequences like `array.array` and `bytes` that store raw C values inline, and streaming iterables like generators that retain suspension frames and yield values lazily. Container sequences cost memory proportional to the number of elements plus their objects. Flat sequences pack values densely without per-element `PyObject` headers. Streaming iterables use constant memory per step regardless of sequence length. The generator attribute `gi_yieldfrom` tracks sub-iterators when using `yield from`. `f_lasti` on frames tracks the last executed instruction offset for suspension and resume. <a href="/memory-list-alternatives">Compare generators with array and deque for memory efficiency</a>. <a href="/async-iterators-generators">Learn about async generators for lazy streaming</a>.

Understand.
Visualize.
Master.

Python in Depth

An interactive engineering reference for Python internals

Quick note

One-shot iterables need caller discipline.

:)
Python version

Targets Python 3.10–3.14. Python 3.9 and below are End-of-Life.

TABLE OF CONTENTS
4.4Container vs flat iterables

Understand references, raw values, arrays, memoryview, and streaming

Iterable does not mean stored. A list, an array, a file object, and a generator can all feed a for loop while retaining very different memory.

Core answer

Choose a container when you need reuse, indexing, or materialized ownership. Choose streaming iteration when one pass is the correct lifetime for values.

# [CURRENT - 3.10-3.14] Works on Python 3.10+
from dataclasses import dataclass
from collections.abc import Iterator
@dataclass(frozen=True, slots=True)
class Reading:
sensor: str
value: int
def parse(lines: list[str]) -> Iterator[Reading]:
for line in lines:
sensor, value = line.split(":")
yield Reading(sensor, int(value))
stream = parse(["cpu:7", "cpu:9"])
print(next(stream))
print(list(stream))

Why this design exists

Iteration is a protocol, not a storage class. That lets algorithms accept lists, tuples, files, generators, views, and custom iterators without hard-coding how values are retained.

Mechanics and CPython internals

Generators retain a suspended frame and resume at yield. They keep current state, not all future results. Containers store their elements or references now. Flat buffers such as arrays and bytes store raw values more densely than pointer-based containers, but still materialize a value set.

# [CURRENT - 3.10-3.14] Works on Python 3.10+
from dataclasses import dataclass
from itertools import islice
from collections.abc import Iterable
@dataclass(frozen=True, slots=True)
class Window:
values: tuple[int, ...]
def first_window(values: Iterable[int], width: int) -> Window:
return Window(tuple(islice(values, width)))
def counters() -> Iterable[int]:
number = 0
while True:
yield number
number += 1
print(first_window(counters(), 5))

Complexity and tradeoffs

Materializing n values costs O(n) space. Streaming can reduce retained space toward the live iterator state, but one-shot iteration changes retry, inspection, and error-recovery options. Generator code also pays resume and Python-level iteration costs; memory savings do not imply higher throughput.

Idiomatic patterns and refactoring

Refactor eager pipelines into staged iteration when the consumer only needs one pass.

# [CURRENT - 3.10-3.14] Works on Python 3.10+
from dataclasses import dataclass
from collections.abc import Iterable, Iterator
@dataclass(frozen=True, slots=True)
class Accepted:
order_id: str
def accept_eager(ids: list[str]) -> list[Accepted]:
return [Accepted(order_id) for order_id in ids if order_id.startswith("ORD-")]
def accept_stream(ids: Iterable[str]) -> Iterator[Accepted]:
for order_id in ids:
if order_id.startswith("ORD-"):
yield Accepted(order_id)
raw = ["ORD-1", "skip", "ORD-2"]
print(accept_eager(raw))
print(list(accept_stream(raw)))

Common mistakes and edge cases

Do not iterate a generator twice and expect the first values again. Do not materialize a list only to pass it immediately into a single streaming consumer. Do not hide required buffering behind an annotation as broad as Iterable.

When to use / When NOT to use

Use iterators and generators for one-pass transforms, large inputs, and backpressure-friendly production.

Do not stream when callers require random access, multiple passes, stable snapshots, or easy post-failure inspection.

Further reading

  • Official docs: iterator types
  • Official docs: generator expressions
  • Official docs: itertools.islice
  • PEP 255: simple generators
  • CPython source: generator objects
BOARD NOTESContext
WHY NO BENCHMARK?

This topic is better taught with structure, semantics, and cross-references than with a synthetic chart.

One-shot iterables need caller discipline.

RELATED GUIDES
NEXT CHECKS
Contribute