Batch size is the number of examples processed together. In a training loop, a model can make predictions for a batch, calculate a loss, and use that loss for an optimizer update. Batching also shapes the input tensor: its first dimension counts examples, not features. section 6.2.2
Ten examples through one linear model
An `nn.Linear(1, 1)` model accepts an input shaped `10 × 1`: ten examples, each with one input feature. A single model call returns ten outputs. The model still has one weight and one bias; adding examples does not add features or parameters. section 6.2.2
The same first-dimension convention applies to images. A batch of RGB images has shape `B × C × H × W`, where `B` is batch size, `C` is the channel count, and `H` and `W` are image dimensions. For just one image, `unsqueeze(0)` adds a batch dimension of size one before passing it to `nn.Conv2d`. section 4.1.3 section 8.2
Batch size can also be an argument to a data-selection function. A `get_batch(batch_size=64)` helper selects that many input–target pairs; calling `get_batch(3)` returns three pairs. The variable-length sequences are padded to a common length so they can be processed together. section 9.3
# In[]:
def get_batch(batch_size=64):
random_idx = torch.randint(0, X.size(0), (batch_size,))
inputs = X[random_idx]
labels = Y[random_idx]
return inputs, labels
inputs, labels = get_batch(3)
print(inputs)
print(labels)
# Out[]:
tensor([[ 0, 2, 18, 9, 26, 5, 25, 4, 1, 0, 0],
[ 0, 21, 26, 9, 1, 19, 0, 0, 0, 0, 0],
[ 0, 14, 15, 15, 18, 21, 12, 1, 9, 14, 0]])
tensor([[ 2, 18, 9, 26, 5, 25, 4, 1, 0, -1, -1],
[21, 26, 9, 1, 19, 0, -1, -1, -1, -1, -1],
[14, 15, 15, 18, 21, 12, 1, 9, 14, 0, -1]])
Compare the requested size of three with the three rows in each returned tensor. Each input row stays paired with its label row. section 9.3
Batch size is a practical resource setting, not just a shape label. When iterating over a dataset with a `DataLoader`, test `batch_size` alongside `num_workers` and watch for out-of-memory failures; the highest workable combination depends on the machine. section 13.12
Serving uses a related idea for a different job: combine incoming requests for one GPU inference pass. A queued worker starts when it has a full batch or the oldest request reaches a maximum wait time. That request batch is not a training update. section 17.1.3 section 17.1.4
