liveBook · Manning

Batching

As explained in Timeless Algorithms, The Foundational Ideas

A mini-batch is a small subset of training examples processed together during an iteration of model training.

A mini-batch is a small group of training observations processed together for one parameter update. Instead of calculating an update from the entire training set, the network makes predictions, measures loss, computes gradients, and updates its parameters for each mini-batch. section 11.2.5 section 11.2.6

A batch of one observation gives stochastic gradient descent; with several observations, the gradients are averaged before the update. An epoch is one pass through the training data, so dividing the data into mini-batches produces multiple updates per epoch. Repeated small updates accumulate across batches and epochs. section 11.3.5 section 11.2.6

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains batch size, gradient averaging, and updates within an epoch, without detailing batch-selection strategies.
Examples
Contrasts an update from one training observation with an update averaging gradients across several observations.
What you can do after
Apply mini-batches to structure repeated neural-network updates across training examples and epochs.
How it connects
Connects mini-batches to stochastic gradient descent, backpropagation, loss, and epochs.

Also explained in

Compare how 23 books explain batching →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.