How deeply does Deep Learning with R, Third Edition cover device mesh?

Defines device grids, named axes, and a 2 × 4 construction before assigning variable layouts with LayoutMap.

Device mesh

As explained in Deep Learning with R, Third Edition

A device mesh is a multidimensional logical arrangement of computational devices organized along named axes.

A device mesh is a grid of computational devices organized along named axes. One axis can support data parallelism while another supports model parallelism. section 18.2.2

A 2 × 4 GPU mesh

An eight-GPU arrangement can use two devices along the data axis and four along the model axis. Keras then makes two copies of a model split across four GPUs, processing separate sub-batches in parallel. section 18.2.2

device_mesh <- keras$distribution$DeviceMesh(
  shape = shape(2, 4),  #1
  axis_names = c("data", "model")  #2
)

Create a named two-dimensional mesh in Keras by specifying its shape and axis names. section 18.2.2

How Deep Learning with R, Third Edition covers it

Cover of Deep Learning with R, Third Edition

liveBook manning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Deep Learning with JAXAn arrangement of multiple devices in a grid used to assign different tensor shards to different devices for parallel computation.
liveBook manning.com

Deep Learning with Python, Third EditionA grid-like arrangement of devices used to assign different parallelism roles along named axes.

Compare how 3 books explain device mesh →

Related concepts

dataset splitused by · A dataset split is the division of a dataset into distinct subsets assigned different purposes, such as training, validation, or testing.
model parallelismused by · Model parallelism is a distributed-computing technique that partitions a machine-learning model’s parameters, layers, or computations across multiple devices.
data paralleluses · Data parallelism is a parallel-computing technique that applies the same computation concurrently to separate portions of data.
distributed traininguses · Distributed training is the coordinated training of a machine-learning model across multiple computing devices, processes, or machines by distributing computation or data among them.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.