A parameter is a trainable value that learning adjusts during model training. In an LLM, weight parameters are stored in linear layers and shape how modules such as multi-head attention and the GPTModel transform data. section 5.0
PyTorch exposes parameters at two useful levels:
- Access a linear layer's weights through its .weight attribute after creating it with torch.nn.Linear(...). section 5.0
- Collect all trainable parameters, including weights and biases, through model.parameters() for training. section 5.0
Inspecting parameters
Create a layer with new_layer = torch.nn.Linear(...), inspect its learned weight with new_layer.weight, or retrieve the model's complete trainable parameter collection with model.parameters(). section 5.0
