Eager evaluation

Q: How deeply does Fast Python: High performance techniques for large datasets cover eager evaluation?

Explains execution timing, memory, wasted work, repeated traversal, and measured performance, but does not detail runtime internals.

As explained in Fast Python: High performance techniques for large datasets

Eager evaluation is an execution strategy that computes expressions or performs operations immediately rather than deferring them until their results are needed.

Eager evaluation computes an expression when it is specified, producing results immediately rather than deferring work until those results are needed. Pandas and NumPy use this model, while Dask constructs a task graph and computes later. section 10.1.2

Eager execution trades simplicity and repeated access against memory use and unnecessary work:

Materializes all results at once, so a list can consume substantially more memory than a generator processing values one at a time. section 2.5.1
Computes results that may never be consumed; a lazy computation can stop after finding a single qualifying value. section 2.5.1
Makes repeated traversal useful when computation is inexpensive and the materialized representation fits in memory. section 2.5.1
Requires measurement rather than assumption: constructing list(range(1000000)) took far longer than creating range(1000000) in the demonstrated benchmark. section 2.2.3

Eagerly materializing a generator

A generator can stream file temperatures with yield, retaining only the current value during iteration. Converting it with temperatures = list(temperatures) eagerly computes and stores the sequence, which helps repeated visits when its memory cost is acceptable. section 2.5.1

import numpy as np
import dask.dataframe as dd                   #1
 
taxes = dd.read_csv("FY2016-STC-Category-Table.csv", sep="\t")
taxes["Amount"] = taxes["Amount"].str.replace(",",
  "").replace("X", np.nan).astype(float)
taxes["Tax_Type"] = taxes["Tax_Type"].astype(
  "category").cat.as_known()                  #2
pivot = taxes.pivot_table(index="Geo_Name",
  columns="Tax_Type", values="Amount")
has_property_info = pivot[
  ~pivot["Property Taxes"].isna()].index      #3
 
pivot_clean = pivot.loc[has_property_info]
frac_property = pivot_clean["Property Taxes"] / pivot_clean["Total Taxes"]

A Dask data-processing pipeline makes the execution boundary visible: transformations build a task graph, and compute() triggers the result. section 10.1.2

How Fast Python: High performance techniques for large datasets covers it

Cover of Fast Python: High performance techniques for large datasets

liveBook manning.com

Read Fast Python: High performance techniques for large datasets on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Deep Learning with R, Third EditionEager execution is an execution mode in which operations run immediately and produce results without first constructing a deferred computation graph.
liveBook manning.com

Functional Programming in Scala, Second EditionStrict code evaluates an argument or expression before the surrounding computation proceeds, rather than delaying evaluation until its value is needed.
liveBook manning.com

Deep Learning with PyTorch, Second EditionAn execution model in which operations run immediately rather than being first built into a static graph.
liveBook manning.com

Functional Programming in JavaScript

Compare how 5 books explain eager evaluation →

Related concepts

lazy evaluationcontrasts with · Lazy evaluation is an evaluation strategy that defers computing an expression or value until its result is needed, potentially avoiding the computation entirely.
PyTorchused by · PyTorch is an open-source machine-learning framework for Python that provides tensor computation, automatic differentiation, and tools for building and training neural networks.
applicative applyused by · Applicative apply is an operation that applies a function within an applicative context to a value within a compatible context while combining their effects.
schedulingused by · Scheduling is the planning and coordination of when and in what order activities or computational work occur, potentially assigning them to available resources subject to constraints.
code compilationcontrasts with · Code compilation is the translation of source code or another formal representation into an executable, intermediate, or lower-level representation.

This explainer was generated by Manning's knowledge graph from the text of Fast Python: High performance techniques for large datasets. The section links point to the parts of the book it draws on.