When list is not the right container

array, deque, memoryview, and generators for tighter memory and I/O

Lists are flexible, and that flexibility has a cost. For numeric data, `array.array` stores raw C doubles directly, skipping per-element `PyObject` overhead entirely. For FIFO queues, `collections.deque` provides O(1) appends and pops from both ends, while `list.pop(0)` is O(n) because every remaining reference must shift in the contiguous array. For binary data and buffer protocols, `memoryview` provides zero-copy slices over underlying buffers. Generators produce values on demand instead of holding everything in memory at once. Each alternative trades generality for performance in a specific dimension. This page covers `array`, `deque`, `memoryview`, and generators with memory measurements and use-case guidance. <a href="/memory-container-comparison">Compare all container types side by side</a>. <a href="/memory-iterables">Understand container vs streaming iterables</a>.

Understand.
Visualize.
Master.

Python in Depth

An interactive engineering reference for Python internals

Quick note

The storage model is the optimization.

:)
TABLE OF CONTENTS
4.2When list is not the right container

array, deque, memoryview, and generators for tighter memory and I/O

You reach for list by reflex every time you need an ordered collection. And that is usually fine — until you store a million floats and wonder why memory is exploding, or you pop from the left and wonder why it is so slow. list is a generalist, not a specialist.

Think of list like a universal toolbox drawer: great for mixed contents, easy to reorganize. But when you need to store 10,000 screws, a dedicated organizer (array.array) uses a fraction of the space. When you need fast access from both ends, a bucket on a rope (collections.deque) beats pulling every tool out from the front.

Core answer

Choose the container by representation and mutation pattern:

  • list: heterogeneous Python objects, random access, right-end append
  • array.array: homogeneous C values packed contiguously
  • collections.deque: fast append/pop at both ends
  • memoryview: zero-copy window over binary buffers
# [CURRENT - 3.10-3.14] Works on Python 3.x
from collections import deque
events = deque(maxlen=3)
for event in ["a", "b", "c", "d"]:
events.append(event)
print(list(events)) # ['b', 'c', 'd']
Mechanism and memory layout

A list stores references to Python objects. For numeric data, that means one pointer per element plus one full Python number object per element. An array.array("d") stores raw C double values directly, with no separate Python float object per element. On mainstream 64-bit CPython builds that is typically 8 bytes per element, but the exact width comes from the platform C type.

Measured locally on CPython 3.12.3, 64-bit Linux:

  • sys.getsizeof([1.0] * 1000) -> 8056 bytes for the list container only
  • sys.getsizeof((1.0,) * 1000) -> 8040 bytes for the tuple container only
  • sys.getsizeof(array("d", [1.0] * 1000)) -> 8080 bytes total for the packed doubles

Those list and tuple measurements do not include the float objects referenced by the elements. The array measurement does include the raw numeric payload. That is why the real memory gap becomes dramatic for numeric workloads.

# [CURRENT - 3.10-3.14] Works on Python 3.x
# Example byte counts below were measured on CPython 3.12.3, 64-bit Linux.
import sys
from array import array
n = 10
values = [float(i) for i in range(n)]
list_total = sys.getsizeof(values) + sum(sys.getsizeof(v) for v in values)
values_t = tuple(float(i) for i in range(n))
tuple_total = sys.getsizeof(values_t) + sum(sys.getsizeof(v) for v in values_t)
array_total = sys.getsizeof(array("d", [float(i) for i in range(n)]))
print(list_total) # 376
print(tuple_total) # 360
print(array_total) # 160

This is the same container-vs-flat distinction discussed in : lists and tuples are object containers, arrays are packed value storage.

Deque vs list

list.append() and list.pop() at the right edge are excellent. list.pop(0) and list.insert(0, value) are not: CPython must shift all remaining references because the underlying representation is a contiguous array of pointers.

deque is designed for that queue shape. The official docs describe appends and pops from either side as approximately O(1), while list left-edge mutation incurs O(n) memory movement.

# [CURRENT - 3.10-3.14] Works on Python 3.x
queue = [1, 2, 3]
first = queue.pop(0)
print(first, queue)
# [CURRENT - 3.10-3.14] Works on Python 3.x
from collections import deque
queue = deque([1, 2, 3])
first = queue.popleft()
print(first, queue)

Local timeit measurements on this machine for 1000 operations over a 10000-element container showed roughly:

  • list.pop(0): about 1.0e-3 seconds
  • deque.popleft(): about 2.0e-5 seconds

Do not treat those exact numbers as portable. The stable lesson is the algorithmic one: queue-like left-edge mutation is a deque workload, not a list workload.

Binary buffers and zero-copy access

memoryview exposes the buffer protocol. It lets you slice and mutate bytes-like storage without allocating a copied sub-buffer.

# [CURRENT - 3.10-3.14] Works on Python 3.x
buffer = bytearray(b"abcdefgh")
view = memoryview(buffer)
view[2:5] = b"XYZ"
print(buffer) # bytearray(b'abXYZfgh')

The view object itself is small and does not own the bytes. It holds metadata describing how to interpret another object's memory.

Measured locally on CPython 3.12.3:

  • sys.getsizeof(bytes(1000)) -> 1033
  • sys.getsizeof(bytearray(1000)) -> 1057
  • sys.getsizeof(memoryview(bytearray(1000))) -> 184

That 184 bytes is the view wrapper, not a second 1000-byte copy.

Version context

array.array, collections.deque, and memoryview are stable Python 3 facilities. Current project guidance targets Python 3.10-3.14. Python 3.9 and below are End-of-Life.

The exact byte counts in this guide are CPython 3.12.3, 64-bit measurements taken locally. They are useful for understanding cost shape, not for writing byte-exact production invariants.

Edge cases and gotchas

deque is not a universally faster list. Middle indexing is slower, slicing is not supported, and many APIs still expect ordinary sequences.

array.array only works when every element fits the chosen type code. It is ideal for dense numeric buffers but not for mixed Python objects.

memoryview is for bytes-like storage, not arbitrary object containers. You cannot create a memory view over a list of Python integers.

If the data is one-pass, avoid materializing any container at all. A generator can beat every structure in memory usage because it stores control state instead of the full result.

Production usage

Use ──────────────────────────────────────────────

  • list for general-purpose mutable sequences
  • deque for queues, recent-history buffers, and producer/consumer edges
  • array.array for compact numeric buffers and binary file I/O
  • memoryview for packet parsing, protocol frames, and binary slicing without copies
# [CURRENT - 3.10-3.14] Works on Python 3.x
from array import array
def save_temperatures(readings, path):
payload = array("d", readings)
with open(path, "wb") as fp:
payload.tofile(fp)

When the goal is laziness rather than compact storage, use iterator pipelines from .

Further depth
  • array module
  • collections.deque
  • memoryview
  • sys.getsizeof
BOARD NOTESContext
WHY NO BENCHMARK?

This topic is better taught with structure, semantics, and cross-references than with a synthetic chart.

The storage model is the optimization.

RELATED GUIDES
NEXT CHECKS
Contribute