Batching groups training examples so a model processes them together during one update or evaluation step. It improves computational efficiency by enabling parallel work on GPUs and CPUs. section 5.3.2 section 6.3.3
A practical batching workflow usually follows these stages:
- Transform raw examples into tensors or numerical token representations, then choose a batch size and optionally shuffle training data. section 3.2.2 section 5.3.2
- Group examples into batches; for sequence data, sort examples by length when possible so each batch contains similarly sized inputs. section 2.2.2
- Pad shorter sequences to the longest sequence in the batch, then create masks so padding does not affect model attention or loss counts. section 2.2.2 section 8.2.2
- Iterate through each batch, compute the loss, update model parameters, and repeat across epochs. section 6.3.3
Two common batch layouts
A vision pipeline loads Fashion-MNIST images with DataLoader(batch_size=128, shuffle=True). A translation pipeline groups German and English token sequences, pads them, and wraps them in a Batch object containing source masks, shifted targets, and token counts. section 5.3.2 section 2.2.2
def seq_padding(X, padding=PAD):
L = [len(x) for x in X]
ML = max(L)
padded_seq = np.array([np.concatenate([x,
[padding] * (ML - len(x))])
if len(x) < ML else x for x in X])
return padded_seq
Shows a sequence-padding function that finds the batch maximum length and appends padding values to shorter arrays. section 2.2.2
