liveBook · Manning

Generator expression

As explained in The Quick Python Book, Fourth Edition

A generator expression is a Python expression that lazily produces values from an iteration specification without constructing the complete result collection in memory.

A generator expression produces values from an iteration specification without first building the complete result collection. It resembles a comprehension, but uses parentheses rather than the square brackets of a list comprehension. The expression returns a generator object, not a list; a `for` loop can consume its values. section 8.9.1 section 8.4

x = [1, 2, 3, 4]
x_squared = (item * item for item in x)
x_squared
 
<generator object <genexpr> at 0x102176708>
 
for square in x_squared:
    print(square,)
 
1 4 9 16

Squaring four items produces a generator object. Iterating over that object prints `1 4 9 16`; merely displaying the object does not display those values. section 8.9.1

Count without building a list of counts

Given a list of lines, `sum(len(line.split()) for line in lines)` adds the word count for each line. The analogous `sum(len(line) for line in lines)` counts characters. The generator expressions avoid constructing separate lists of per-line counts, but `readlines()` has already loaded the lines into memory. section 8.13.2

Check dictionary entries

`any(not isinstance(_, str) for _ in self.keys())` tests whether any key is not a string; the same pattern can check values. Here `_` is a conventional name for a throw-away iteration variable. section 17.11.2

Choose a generator expression when values can be consumed as they are produced, especially when retaining a complete result list would waste memory. It does not make the rest of a program memory-efficient: reading an entire file or creating another large list can still dominate memory use. section 8.9.1 section 11.5.2

How The Quick Python Book, Fourth Edition covers it

How deep it goes
Explains parentheses, the returned generator object, iteration, and memory trade-offs through examples; does not develop generator internals.
Examples
Squares a list, totals file word and character counts, and checks dictionary keys and values for nonstrings.
What it assumes
Assumes familiarity with Python loops and comprehensions; introduces the generator object and its memory advantage.
What you can do after
Build it from scratch to feed computed values to a loop or aggregation while recognizing when other data remains in memory.
How it connects
Connects generator expressions to comprehensions, iterables, `for` loops, `sum()`, and `any()`.

Also explained in

Compare how 3 books explain generator expression →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of The Quick Python Book, Fourth Edition. The section links point to the parts of the book it draws on.