How deeply does Deep Learning with Python, Third Edition cover XLA?

Explains activation, first-call overhead, and later speedups, but does not descend into compiler internals.

XLA

As explained in Deep Learning with Python, Third Edition

Accelerated Linear Algebra, TensorFlow's optional compiler used to further optimize and speed up function execution.

XLA, short for Accelerated Linear Algebra, is TensorFlow's high-performance compiler for optimizing function execution. It provides a faster compilation path than ordinary graph mode, with the trade-off of additional work during the first call. section 3.1 section 3.3.1

Compile a TensorFlow function with XLA

Add jit_compile=True to a TensorFlow function: @tf.function(jit_compile=True) def dense(inputs, W, b): return tf.nn.relu(tf.matmul(inputs, W) + b). TensorFlow compiles dense into an optimized program; the first call takes longer, while later calls can run faster. section 3.3.1

Use XLA with a simple workflow:

Debug the function in the ordinary Python runtime first. section 3.3.1
Apply @tf.function to enable TensorFlow graph mode. section 3.3.1
Set jit_compile=True when the extra compilation work is worthwhile. section 3.3.1

@tf.function(jit_compile=True)
def dense(inputs, W, b):
    return tf.nn.relu(tf.matmul(inputs, W) + b)

Inspect the compact TensorFlow example that enables XLA for a dense operation. section 3.3.1

How Deep Learning with Python, Third Edition covers it

Cover of Deep Learning with Python, Third Edition

liveBook manning.com

Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Deep Learning with JAXA compiler and runtime for linear algebra and array programs that optimizes computations such as those produced by JAX.
liveBook manning.com

Deep Learning with R, Third EditionXLA is a compiler for accelerating linear algebra and machine-learning computations across supported hardware.

Compare how 3 books explain XLA →

Related concepts

JAXused by · JAX is a Python library for high-performance array-based numerical computing with composable transformations including automatic differentiation, compilation, vectorization, and parallelization.
just-in-time compilationused by · Just-in-time compilation translates or optimizes program code during execution, often using runtime information to generate specialized executable instructions.
parallel mapused by · A parallel map is an operation that applies a computation concurrently across inputs or processing devices and returns the corresponding results.
Jaxprused by · JAX's simple intermediate representation of a traced function, used as the input to later transformations and compilation.
kernel fusionaddresses · Kernel fusion is an optimization that combines multiple computational operations into fewer executable units to reduce launch overhead and intermediate data movement.
TPUuses · A Tensor Processing Unit (TPU) is a specialized processor designed to accelerate tensor computations, particularly for machine-learning workloads.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.