A loss is a scalar measure of the discrepancy between a network’s output and the true labels. Training seeks to minimize it. section 4.2.2
For a linear layer, the loss gradients are computed as:
- Weight gradient: dL/dW = Xᵀ @ dL/dY. section 4.2.2
- Input gradient: dL/dX = dL/dY @ Wᵀ. section 4.2.2
- Bias gradient: dL/db = sum(dL/dY, axis=0). section 4.2.2









































