When a transformation may produce many values, a list comprehension builds the whole result immediately. A generator expression uses the same expression, loop, and optional filter structure but returns values one at a time, delaying computation and reducing memory use. section 7.2 section 7.17
Use generator expressions when:
- A consuming function such as sum or str.join can request values from an iterator. section 7.2
- The input is large enough that materializing a list would consume substantial memory. section 7.17
- You want to transform or filter values without first storing every result. section 7.7
- You need an iterator to return from an iterable object's __iter__ method. section 10.7
Summing transformed values
These forms produce the same sum, but the first constructs a list while the latter two produce values lazily: sum([x*x for x in range(100000)]) sum((x*x for x in range(100000))) sum(x*x for x in range(100000)) When a generator expression is the only argument to a function call, its extra parentheses can be omitted. section 7.2
Processing a file
A file can be translated word by word and joined without building a list of all translated words: def plfile(filename): return ' '.join( plword(one_word) for one_line in open(filename) for one_word in one_line.split() ) The nested loops supply words to the generator, while str.join consumes the resulting strings. The approach may trade a little speed for substantially lower memory use on large input. section 7.17
class Circle():
def __init__(self, data, max_times):
self.data = data
self.max_times = max_times
def __iter__(self):
n = len(self.data)
return (self.data[x % n] for x in range(self.max_times))
Inspect a generator expression returned by __iter__; it yields indexed values from data for a fixed number of iterations. section 10.7
