A mini-batch is a small group of training observations processed together for one parameter update. Instead of calculating an update from the entire training set, the network makes predictions, measures loss, computes gradients, and updates its parameters for each mini-batch. section 11.2.5 section 11.2.6
A batch of one observation gives stochastic gradient descent; with several observations, the gradients are averaged before the update. An epoch is one pass through the training data, so dividing the data into mini-batches produces multiple updates per epoch. Repeated small updates accumulate across batches and epochs. section 11.3.5 section 11.2.6






















