A scalar is a single numeric value. In PyTorch, a scalar is represented as a zero-dimensional tensor, making it suitable for model parameters such as a weight and bias. section 5.3.1
Scalars work naturally with tensors through broadcasting: multiplying a scalar scales every element, while adding one offsets every element. Broadcasting also aligns compatible dimensions, including dimensions of size 1. section 5.3.1 section 3.4
Scalar parameters in a PyTorch model
The zero-dimensional tensors w and b broadcast across the input vector t_u: w * t_u + b. A mean operation can then reduce element-wise squared differences to a scalar loss. section 5.3.1
# In[5]:
w = torch.ones(())
b = torch.zeros(())
t_p = model(t_u, w, b)
t_p
# Out[5]:
tensor([35.7000, 55.9000, 58.2000, 81.9000, 56.3000, 48.9000, 33.9000,
21.8000, 48.4000, 60.4000, 68.4000])
Inspect scalar tensor initialization and its broadcasted use as model weight and bias. section 5.3.1
