liveBook · Manning

Generator

As explained in Python Workout: 50 ten-minute exercises

A generator function is a Python function that produces a generator by yielding successive values while preserving its execution state between yields.

When a task must process values gradually rather than build a complete collection, a generator function provides each value only when requested. A function containing yield produces a generator object; each yield pauses execution, preserves local state, and resumes when the next value is requested. section 10.0 section 10.17

A three-value generator

def foo(): yield 1 yield 2 yield 3 g = foo() for one_item in g: print(one_item) Calling foo() creates the generator; the loop repeatedly requests its next value. The function ends automatically after the final yield, so it does not explicitly raise StopIteration. section 10.0 section 10.15

Generators support the iterator protocol while keeping iteration logic in ordinary-looking function code:

  • yield returns the current value and suspends the function until the next request. section 10.12
  • Local variables and parameters retain their values across resumptions, allowing stateful sequences and incremental calculations. section 10.17
  • The surrounding for loop uses the iterator protocol; generator completion signals the end automatically. section 10.0 section 10.15
  • Generators can filter, transform, or augment input and can handle large quantities without producing every result immediately. section 10.15 section 10.26

Streaming lines from files

A generator can list files in a directory, open each usable file, and yield one line at a time. It can combine the directory path with each filename, ignore selected OSError failures, and resume after every yielded line. This keeps file traversal incremental and makes the function useful for data sets larger than convenient in-memory collections. section 10.11 section 10.12

Generator expressions

A generator expression uses parentheses instead of the square brackets of a list comprehension. For example, sum(x*x for x in range(100000)) supplies one square at a time, whereas sum([x*x for x in range(100000)]) first constructs the whole list. section 7.2

def foo():
   yield 1
   yield 2
   yield 3

Use the compact definition to inspect how yield creates successive values, then compare it with the loop in b7. section 10.0

How Python Workout: 50 ten-minute exercises covers it

How deep it goes
Explains yield, automatic completion, suspended execution, and retained locals, but defers generator expressions to a separate treatment.
Examples
Demonstrates small yielded sequences, directory-wide line streaming, elapsed-time reporting, chaining iterables, and memory-saving sums.
What it assumes
Assumes familiarity with Python functions, loops, and iterable data while developing the iterator protocol through examples.
What you can do after
Build it from scratch as a stateful iterator that yields, filters, transforms, or augments values incrementally.
How it connects
Connects generators to the iterator protocol, iterables, generator expressions, list comprehensions, and itertools.

Also explained in

Compare how 12 books explain generator →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Python Workout: 50 ten-minute exercises. The section links point to the parts of the book it draws on.